On: “Math 2.0” will need to value mathematical progress more holistically

[bonoboTP]

LLMs also output a distribution over tokens at the output. I don't see the point you're making.

[catlifeonmars → bonoboTP]

That’s interesting right? The distribution by itself doesn’t provide the structure required. You need an activation function (idk what the term of architecture for this is in ML) to impose some sort of semantic structure on top of the distribution (eg softmax)