The Latest and Greatest AI Chips. How Fast and How Much?

The Latest and Greatest AI Chips

How Fast and How Much?

We’ve heard of Artificial Intelligence (AI) and the computers required to carry out the job required. They consume massive power but can accomplish near-miraculous data processing tasks. But how fast and how much are values mostly unknown to most of us. Let’s look under the hood to get some idea of what’s inside.

The latest and greatest NVIDIA chip, the Blackwell GB200, is a key component of the NVIDIA GB200 NVL72, a multi-node, liquid-cooled, rack-scale system, which comprises 600,000 parts, including 72 GB200s. It's designed to be capable of delivering 720 petaflops* for training and 1.4 exaflops* for inferencing. (And it reportedly weighs 3,000 pounds – about the same as a mid-sized car.) Nvidia's Blackwell AI chip is expected to cost between $30,000 and $40,000 per unit, according to CEO Jensen Huang. This price is competitive with current Nvidia AI chips.

Nvidia's Blackwell generation boosts power consumption even further, with the B200 consuming up to 1,200W, and the GB200 (which combines two B200 GPUs and one Grace CPU) expected to consume 2,700W (Watts per hour). To better understand 2700W, note that the home A/C unit normally consumes about 3000W/hr.

Conclusion. AI chips are big, heavy, fast, and consume vast amounts of electrical power. It is no wonder that modern data centers require their own power-generating plants (also huge).

NOTES.

1. For reference, a typical PC CPU chip weighs about 1.0 ounces. So, the Blackwell chip weights 3000/(1/16) = 48000 PC CPU chips. At $500 per CPU chip, this comes to about $24 million.

2. For reference, the speed of the Blackwell is roughly 10,000 times faster than that of a typical desktop PC. Typical supercomputers run at speeds of 100 petaflops, which is faster than the Blackwell, but not by orders of magnitude. These speeds are almost beyond imagination – certainly mine.

3. Intel claims its Gaudi 3 chip is 1.5 times faster than the Blackwell. It should be launched later in 2024. If you know anything about Gaudi architecture in Spain, you will be amused.

4. The new Tesla Data Center in Austin, TX will employ up to 85,000 Blackwell chips by the end of 2024.

5. It is easy to understand why China wants to take over Taiwan, where these chips are manufactured. And not to bomb it to smithereens (easy), but intact. It’s easy to see why the USA (and the rest of the world) should try to prevent this. Whoever thought chips would be more important than bombs in warfare?

6. All the information, except for a few comparisons, in this report is available on the Internet.

*Petaflop: a unit of computing speed equal to one thousand million million (10¹⁵) floating-point operations per second. An exaflop (10¹⁸) is 1000 times faster than a petaflop. A FLOP is the time it takes to multiply floating point numbers, not just integers, but all kinds of numbers, where an exponent is needed to indicate magnitude) in the range from the tiniest to the largest.

P.S. When I did supercomputing some years back, chip speed was measured in GIGAFLOPS. These new chips are millions of times faster.

Accepting Fake Information

Every day, we are all bombarded with information, especially on news channels. One group claims it's false; another calls it the truth. How can we know when to accept it or alternatively how can we know it's false? There are several factors which influence acceptance of fake or false information. Here are the big four. Some just don’t have the knowledge to discern fact/truth from fiction/fact/false*. Some fake information is cleverly disguised and simply appears to be correct. Some fake information is accepted because the person wants to believe it. Some fake information is accepted because there is no other information to the contrary. However, the acceptance of information of any kind become a kind of truth , and this is a well studied topic. In the link below is an essay on “The Truth About Truth.” This shows simply that what is your point of view, different types of information are generally accepted, fake or not. https://www.linkedin.com/posts/g-donald-allen-420b03

Your Brain Within Your Brain

Your Bicameral Brain by Don Allen Have you ever gone to another room to get something, but when you got there you forgot what you were after? Have you ever experienced a flash of insight, but when you went to look it up online, you couldn’t even remember the keyword? You think you forgot it completely. How can it happen so fast? You worry your memory is failing. Are you merely absent-minded? You try to be amused. But maybe you didn’t forget. Just maybe that flash of insight, clear and present for an instant, was never given in the verbal form, but another type of intelligence you possess, that you use, and that communicates only to you. We are trained to live in a verbal world, where words matter most. Aside from emotions, we are unable to conjure up other, nonverbal, forms of intelligence we primitively, pre-verbally, possess but don’t know how to use. Alas, we live in a world of words, stewing in the alphabet, sleeping under pages of paragraphs, almost ignoring one of

Is Artificial Intelligence Conscious?

Is Artificial Intelligence Conscious? I truly like the study of consciousness, though it is safe to say no one really knows what it is. Some philosophers has avoided the problem by claiming consciousness simply doesn’t exist. It's the ultimate escape clause. However, the "therefore, it does not exist" argument also applies to "truth", "God", and even "reality" all quite beyond a consensus description for at least three millennia. For each issue or problem defying description or understanding, simply escape the problem by claiming it doesn’t exist. Problem solved or problem avoided? Alternately, as Daniel Dennett explains consciousness as an account of the various calculations occurring in the brain at close to the same time. However, he goes on to say that consciousness is so insignificant, especially compared to our exalted notions of it, that it might as well not exist [1] . Oh, well. Getting back to consciousness, most of us have view

First Blog

Search This Blog