Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

what if you set top_p=1, temperature=0, and always run it on the same local hardware


Horace He at Thinking Machines just dropped an awesome article describing exactly this: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

TL;DR: assuming you've squashed all regular non-determinism (itself a tall ask), you either need to ensure you always batch requests deterministically, or ensure all kernels are "batch invariant" (which is absolutely not common practice to do).


Maybe if you run it on CPU. (Maybe on GPU if all batching is disabled, but I wouldn't bet on it.)


cosmic wave will get you




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: