>to think of this as only an incremental improvement over markov chains is underselling the advance.
Erm, citations needed. It's a giant, inefficient and shitty KNN model, which is capable of mimicking markov chains. Wonderful marketing achievement and not much else.
https://www.gwern.net/GPT-3 (Edit: in case it's not clear, I suspect if you give an honest perusal of that page, and the pages it links, and the pages they link, you'll come away with a different opinion.)
Erm, citations needed. It's a giant, inefficient and shitty KNN model, which is capable of mimicking markov chains. Wonderful marketing achievement and not much else.