On: Post Apocalyptic Computing

[sdenton4]

Lots of models are trained with dropout, which is kinda like bitrot at very high rates...

[clayhacks → sdenton4]

But that’s during training, not during inference. Also it’s more structured in where the dropout is happening. I do think that points to them being somewhat resilient but we haven’t had LLMs exist long enough for a good test