Everything about the new beast H100

[raahul_rahl]

https://musingsonai.substack.com/p/edition-13-everything-about-the-new

[Hendrikto]

Wow, these prices are insane. 250k for a single GPU, at 80% profit margin…

[raahul_rahl → Hendrikto]

Thats why they are becoming VC.

"Last week, AI news was dominated by Databricks' massive $500 million round. Who was at the center of that fundraising? NVIDIA. This news is only surprising if you haven't been paying attention to NVIDIA's recent investment spree. The recent additions to the portfolio feature some of the hottest companies in the AI space. Take a look and judge for yourself:

Hugging Face: NVIDIA was part of the recent $200 million round that pushed Hugging Face's valuation to $4 billion.

AI21 Labs: NVIDIA invested in the recent $155 million Series C that valued the OpenAI competitor at $1.4 billion.

Imbue: NVIDIA joined the $200 million Series B that valued the AI reasoning platform at $1 billion.

Cohere: NVIDIA was part of the $270 million round that valued Cohere at $2.1 billion.

Adept AI: NVIDIA participated in the $350 million round that valued Adept at $1 billion."

Source https://musingsonai.substack.com/p/edition-14-last-15-days-i...

[tmikaeld → Hendrikto]

That's not for a single GPU, but a cluster of them in a packaged solution.

Per-GPU price is $25,000 to $30,000 estimated cost to make them is $3,320.

[raahul_rahl → tmikaeld]

I took the cost from https://www.semianalysis.com/

[Smerity → Hendrikto]

That section of the text is ambiguous but it's $250k for 8 x H100s (PCI-e) and $350k for 8 x H100 (SXM).

The prices above are plus or minus from Lambda Labs[1].

[1]: https://lambdalabs.com/products/scalar

[fragmede]

Fascinatingly in-depth, but my question is, why not cool the cards with liquid nitrogen and run them at higher clock speed?

[raahul_rahl → fragmede]

Thats a nice one!!!

[ai_biden]

Twitter is coming back with H100 - that's exciting for the space.