The Latest and Greatest AI Chips. How Fast and How Much?

The Latest and Greatest AI Chips

How Fast and How Much?

We’ve heard of Artificial Intelligence (AI) and the computers required to carry out the job required. They consume massive power but can accomplish near-miraculous data processing tasks. But how fast and how much are values mostly unknown to most of us. Let’s look under the hood to get some idea of what’s inside.

The latest and greatest NVIDIA chip, the Blackwell GB200, is a key component of the NVIDIA GB200 NVL72, a multi-node, liquid-cooled, rack-scale system, which comprises 600,000 parts, including 72 GB200s. It's designed to be capable of delivering 720 petaflops* for training and 1.4 exaflops* for inferencing. (And it reportedly weighs 3,000 pounds – about the same as a mid-sized car.) Nvidia's Blackwell AI chip is expected to cost between $30,000 and $40,000 per unit, according to CEO Jensen Huang. This price is competitive with current Nvidia AI chips.

Nvidia's Blackwell generation boosts power consumption even further, with the B200 consuming up to 1,200W, and the GB200 (which combines two B200 GPUs and one Grace CPU) expected to consume 2,700W (Watts per hour). To better understand 2700W, note that the home A/C unit normally consumes about 3000W/hr.

Conclusion. AI chips are big, heavy, fast, and consume vast amounts of electrical power. It is no wonder that modern data centers require their own power-generating plants (also huge).

NOTES.

1. For reference, a typical PC CPU chip weighs about 1.0 ounces. So, the Blackwell chip weights 3000/(1/16) = 48000 PC CPU chips. At $500 per CPU chip, this comes to about $24 million.

2. For reference, the speed of the Blackwell is roughly 10,000 times faster than that of a typical desktop PC. Typical supercomputers run at speeds of 100 petaflops, which is faster than the Blackwell, but not by orders of magnitude. These speeds are almost beyond imagination – certainly mine.

3. Intel claims its Gaudi 3 chip is 1.5 times faster than the Blackwell. It should be launched later in 2024. If you know anything about Gaudi architecture in Spain, you will be amused.

4. The new Tesla Data Center in Austin, TX will employ up to 85,000 Blackwell chips by the end of 2024.

5. It is easy to understand why China wants to take over Taiwan, where these chips are manufactured. And not to bomb it to smithereens (easy), but intact. It’s easy to see why the USA (and the rest of the world) should try to prevent this. Whoever thought chips would be more important than bombs in warfare?

6. All the information, except for a few comparisons, in this report is available on the Internet.

*Petaflop: a unit of computing speed equal to one thousand million million (10¹⁵) floating-point operations per second. An exaflop (10¹⁸) is 1000 times faster than a petaflop. A FLOP is the time it takes to multiply floating point numbers, not just integers, but all kinds of numbers, where an exponent is needed to indicate magnitude) in the range from the tiniest to the largest.

P.S. When I did supercomputing some years back, chip speed was measured in GIGAFLOPS. These new chips are millions of times faster.

UNCERTAINTY IS CERTAIN

Uncertainty is Certain G. Donald Allen 12/12/2024 1. Introduction . This short essay is about uncertainty in people from both secular and nonsecular viewpoints. One point that will emerge is that randomly based uncertainty can be a driver for religious structure. Many groups facing uncertainty about their future are deeply religious or rely on faith as a source of comfort, resilience, and guidance. The intersection of uncertainty and religiosity often stems from the human need to find meaning, hope, and stability in the face of unpredictable or challenging circumstances. We first take up the connections of uncertainty to religion for the first real profession, farming, noting that hunting has many similar uncertainties. Below are groups that commonly lean on religious beliefs amidst uncertainty. This short essay is a follow-up to a previous piece on certainty (https://used-ideas.blogspot.com/2024/12/certainty-is-also-emotion.html). U...

CERTAINTY IS ALSO AN EMOTION

Certainty is also a Feeling Certainty is often viewed as a mental state tied to knowledge and confidence, but it also functions as a feeling with distinct emotional and physiological components. While it arises from cognitive processes, certainty also has a subjective and emotional quality that makes it more than just a rational judgment. It provides a sense of assurance and security that shapes human experience in profound ways. Emotional Dimension . At its core, certainty evokes emotions that influence how we perceive and interact with the world. When someone feels certain, they often experience relief, comfort, or empowerment. These emotions are particularly strong when uncertainty or doubt is resolved, offering a sense of closure. For example, solving a complex problem or having a belief validated by evidence brings not just intellectual satisfaction but also emotional reassurance. Subjectivity. Certainty is inherently personal and subjective. It depends on individual...

Lies, Deceit, and the National Agenda

The world you grew up in is no more. The world of reasonable honesty and reasonable lies has been replaced by abject dishonesty and blatant lies. Lies. Yes. People have always told them. You have told them; so have I. We need lies; they are a foundational structure of social living. They both deceive and protect. Children tell them to their parents to avoid consequences, like punishment. Adults tell them to their bosses, to enhance their position and/or avoid consequences of poor performance. Our bosses tell them to their boards to suggest business is good, the project is on target, or the detractors are wrong. The boards tell them to shareholders to protect their own credibility and most importantly, stock values. Our politicians tell lies to their constituents, though sometimes innocently with them not actually knowing much more than they've been told. They enhance their positio...

First Blog

Search This Blog