On: How Meta trains large language models at scale

[refulgentis]

This breaks my brain, because I know Google trains it models on TPUs and they're seen as faster, and if they're better at inference, and can train, then why is Nvidia in a unique position? My understanding was always it's as simple as it required esoteric tooling

[smueller1234 → refulgentis]

Multiple types of TPUs.

(I work for Google, but the above is public information.)

[vineyardmike → refulgentis]

Because people generally don’t use TPUs outside of Google. The tooling is different, the access is metered through GCP, etc.

Nvidia is in a vaguely unique position in that their products have great tooling support and few companies sell silicon at their scale.

[refulgentis → vineyardmike]

Correct, I'm pointing out politely that's in conflict with the person I'm replying to.

[koe123 → refulgentis]

Possibly naive but I very much view CUDA and its integration into ML frameworks being nvidias moat