AI #9: The Merge and the Million Tokens

[paulpauper]

https://thezvi.substack.com/p/ai-9-the-merge-and-the-million-tokens

[sharemywin]

I wonder if the size of the context window dilutes the attention. Already prompts that are too big cause the thing to lose track of what it needs to do.