On: How attention sinks keep language models stable

[derbOac]

> This seems to go beyond just transformers

And beyond that. Sometimes I feel like AI research is reinventing wheels that exist elsewhere. Maybe just the wheels, but still.