First off, it’s a HW/SW solution and things like CUDA/NCCL/etc make a HUGE difference.
Second, the token/watt ratio of every other option is nearly an order of magnitude difference in real world tests. When you add in custom silicon like moronic Grok/Dojo and you see that there aren’t really any close competitors when using custom spins. That is money down the drain IMO. Best bet for most enterprises is to buy 25% AMD and 75% H100 if they can get it.
I think Blackwell is potentially a long term generational problem due to power limitations in most data centers for now.
First off, it’s a HW/SW solution and things like CUDA/NCCL/etc make a HUGE difference.
Second, the token/watt ratio of every other option is nearly an order of magnitude difference in real world tests. When you add in custom silicon like moronic Grok/Dojo and you see that there aren’t really any close competitors when using custom spins. That is money down the drain IMO. Best bet for most enterprises is to buy 25% AMD and 75% H100 if they can get it.
I think Blackwell is potentially a long term generational problem due to power limitations in most data centers for now.