I'm not really trying to 'refute' the original statement. Certainly, an LLM is just doing some calculations that can be done deterministically in principle. However, I think it's worth pointing out that there are practical barriers to doing those particular calculations both deterministically and efficiently. People who worry about LLM output not being reproducible aren't necessarily misunderstanding what an LLM is doing; they are responding to a real feature of most practical LLM implementations.
If you wanted to and had enough engineering effort to spare, you could run an LLM deterministically at relatively small impacts to performance.
One approach is to make sure you run things in the same order. Another is to change your operations so that more of them become associative or even commutative.
See eg the paper 'A Lattice-Based Approach to Deterministic Parallelism' for some interesting ideas on the latter.
