On: Chiplet ASIC supercomputers for LLMs like GPT-4

[bhouston]

The $0.87/hour price you gave is theoretical and also we know any price in a paper for compute is wrong by the time of publication.

Pragmatically the prices are closer to $2/hr according to this recent post here on Hacker News: https://llm-utils.org/Nvidia+H100+and+A100+GPUs+-+comparing+...

Although again prices change on a daily basis on spot providers.

[jacquesm → bhouston]

> The $0.87/hour price you gave is theoretical

https://cloud.google.com/blog/products/compute/a2-vms-with-n...

That's as close as I got to verifying that price.

[bhouston → jacquesm]

Another list that shows pricing both constant and spot. The best GCP spot price is $1.1, but Jarvis seems to say its spot for the 40GB A100 is $0.69:

https://fullstackdeeplearning.com/cloud-gpus/

I feel there are more fair criticisms of that paper than its inclusion of the snapshot price of variable priced compute resource.

[jacquesm → bhouston]

Sure, to me it more of an extra item than the main one but it is one that you can readily verify because most of the other claims are far more vague. If they're willing to fudge on that one then I have much less confidence in the rest of their claims.

[kraken12 → jacquesm]

Seems like HN comments have determined that the cost number is not fudged..