The Latest and Greatest AI Chips. How Fast and How Much?

The Latest and Greatest AI Chips

How Fast and How Much?

We’ve heard of Artificial Intelligence (AI) and the computers required to carry out the job required. They consume massive power but can accomplish near-miraculous data processing tasks. But how fast and how much are values mostly unknown to most of us. Let’s look under the hood to get some idea of what’s inside.

The latest and greatest NVIDIA chip, the Blackwell GB200, is a key component of the NVIDIA GB200 NVL72, a multi-node, liquid-cooled, rack-scale system, which comprises 600,000 parts, including 72 GB200s. It's designed to be capable of delivering 720 petaflops* for training and 1.4 exaflops* for inferencing. (And it reportedly weighs 3,000 pounds – about the same as a mid-sized car.) Nvidia's Blackwell AI chip is expected to cost between $30,000 and $40,000 per unit, according to CEO Jensen Huang. This price is competitive with current Nvidia AI chips.

Nvidia's Blackwell generation boosts power consumption even further, with the B200 consuming up to 1,200W, and the GB200 (which combines two B200 GPUs and one Grace CPU) expected to consume 2,700W (Watts per hour). To better understand 2700W, note that the home A/C unit normally consumes about 3000W/hr.

Conclusion. AI chips are big, heavy, fast, and consume vast amounts of electrical power. It is no wonder that modern data centers require their own power-generating plants (also huge).

NOTES.

1. For reference, a typical PC CPU chip weighs about 1.0 ounces. So, the Blackwell chip weights 3000/(1/16) = 48000 PC CPU chips. At $500 per CPU chip, this comes to about $24 million.

2. For reference, the speed of the Blackwell is roughly 10,000 times faster than that of a typical desktop PC. Typical supercomputers run at speeds of 100 petaflops, which is faster than the Blackwell, but not by orders of magnitude. These speeds are almost beyond imagination – certainly mine.

3. Intel claims its Gaudi 3 chip is 1.5 times faster than the Blackwell. It should be launched later in 2024. If you know anything about Gaudi architecture in Spain, you will be amused.

4. The new Tesla Data Center in Austin, TX will employ up to 85,000 Blackwell chips by the end of 2024.

5. It is easy to understand why China wants to take over Taiwan, where these chips are manufactured. And not to bomb it to smithereens (easy), but intact. It’s easy to see why the USA (and the rest of the world) should try to prevent this. Whoever thought chips would be more important than bombs in warfare?

6. All the information, except for a few comparisons, in this report is available on the Internet.

*Petaflop: a unit of computing speed equal to one thousand million million (10¹⁵) floating-point operations per second. An exaflop (10¹⁸) is 1000 times faster than a petaflop. A FLOP is the time it takes to multiply floating point numbers, not just integers, but all kinds of numbers, where an exponent is needed to indicate magnitude) in the range from the tiniest to the largest.

P.S. When I did supercomputing some years back, chip speed was measured in GIGAFLOPS. These new chips are millions of times faster.

Human Simulation is Possible - by Humans

Human Simulation: Agent-Based Modeling and the Computational Case for Our Reality 1. Introduction In the essay, we introduce the notion that humanity could be a deliberate simulation based on an extremely simple single-celled organism, hardly anything more complex than that. This will allow an (apparently) new interpretation of the notion that humanity is simulated, but that there is no master engineer at a massive number of dials and switches, responsible for every twist and turn of our universe and lives. We begin with the notion that the universe exists and operates under its own laws. So, given the Earth [2] , when and where life first began, it was infused with a living agent, self-actualized, reproductive, and with other basic rules for its operations, interactions, and much of what single-celled organisms do today. The idea from this occurred some years back when writing code in ActionScript (formerly Flash), I was able to create...

Where is AI (Artificial Intelligence) Going?

How to view Artificial Intelligence (AI). Imagine you go to the store to buy a TV, but all they have are 1950s models, black and white, circular screens, picture rolls, and picture imperfect, no remote. You’d say no thanks. Back in the day, they sold wildly. The TV was a must-have for everyone with $250 to spend* (about $3000 today). Compared to where AI is today, this is more or less where TVs were 70 years ago. In only a few decades AI will be advanced beyond comprehension, just like TVs today are from the 50s viewpoint. Just like we could not imagine where the video concept was going back then, we cannot really imagine where AI is going. Buckle up. But it will be spectacular. *Back then minimum wage was $0.75/hr. Thus, a TV cost more than eight weeks' wages. -------------------------

Infinity of Theories

I do love the Stock Markets, all of them. Why? Because of the abundance of theories. Every day, no matter how the market moves there are dozens of pundits offering forth their theory as to why it went down – or up, where it’s going tomorrow or next week, and why it must crash soon or maybe elevate another couple thousand more points. Ok, this is fine. But the next day these same folks may expound theories postulating the opposite. This proves market pundits have no essential understanding of economics as related to the market. This proves pundits prefer expedience to understanding. I am constantly besieged by brokerage firms telling me if I would only invest a million, they would help me reach my goals. If I had this information, how to make a lot of money, I would borrow every cent I could and apply it to making a bunch of money. I’d not tell a soul. So, why are they trying to help me? Well, ...

First Blog

Search This Blog