it annoys me how eager people are to hurl the word stochastic as pejorative. Statistics are a great tool for gleaning information from stochastic processes; statistics don't contribute randomness. Random sampling is necessary in order not to bias a sample, it's not used to contribute randomness to the sample but to preserve/measure the underlying distribution. (not meant to imply that training is random sampling)
It's a pejorative only because determinism is what makes computers useful in the first place. You get a consistent result, every single time, unlike if you have a human in the loop. Because LLMs are stochastic, they have removed the thing that makes computers useful to us, thus it's a pejorative.
What do you mean by determinism here? That you ask the computer 2+2 and it gives 4 as you expected or that if you ask the computer 2+2 and it hallucinates 5, you want it to always hallucinate 5?
Which one it doesn't do for you? Does it sometimes answer 4, sometimes 5?
There are definitely models that will always give 100% of the time the exact same answer, bit-for-bit, given the same input and seed. There are generative image models you can run locally doing just that. But you can also run some the SOTA chinese LLMs at "temperature 0" and, given the same input, they'll always give you the exact same output.
Because it's just a machine doing computation.
In the beginning of LLMs some "engineers" have tried to hand-wave non-sensical explanation as to why LLMs couldn't possibly be deterministic but: the open-weights models that can be run in a 100% deterministic way are way more powerful than the SOTA models of back then, so those explanation were pure rubbish bollocks.
Now of course if you run a complex chain of events, with LLMs doing calls to other LLMs, where some of them go fetch infos on unreliable networks, with infos that may have changed, then, logically, you won't always get the same answer.
