On: Sharing AI progress in mathematics

[orlp]

I'd really like some clarity on what that means. E.g. DeepMind has 'cheated' with this in the past, claiming that AlphaZero only took 4 hours to reach super-human chess levels while conveniently leaving out the fact that it was 4 hours x 5000+ TPUs. Sure it's impressive that it only took 4 hours wall-clock but it's very misleading as to cost.

Can we get a number in Blackwell GPU-hours, kWh, or some other compute-scaled metric?

[timjver → orlp]

>OpenAI has 'cheated' with this in the past, claiming that AlphaZero [...]

That doesn't sound right

[orlp → timjver]

Oops, edited.

[machomaster → orlp]

They did say that. "3 hours of ChatGPT Pro thinking compute"

[orlp → machomaster]

Yes, what does that mean?

[pixl97 → orlp]

Depends what your metrics are. If you suddenly found a way to have 9 women make a baby in one month that is huge.

[orlp → pixl97]

I'm not denying that, but I'd still like to know what that cost.