On: How LLMs Work, Explained Without Math

[eggdaft]

The context comes from the attention mechanism, not from word embeddings.

[mjburgess → eggdaft]

Run attention on an ordinal word embedding and see what happens

[eggdaft → mjburgess]

Well yes, necessary but not sufficient, obviously.