> In practice, plain, well-designed summaries should be optimal to fit larger documents in the context.
> This concept has potential, though; building lookup tables seems to outperform long text summarization.
It's a clever idea, and I agree that lookup tables and external storage of memory is likely going to be important at some point, but I suspect that's going to come out of giving LLM more ability to externally reference "long-term" memory rather than compressing everything into immediate context.
A) people have had a hard time reproducing it, and
B) more damning, the "compressed" version uses more tokens than the original (https://gist.github.com/VictorTaelin/d293328f75291b23e203e9d...)
> In practice, plain, well-designed summaries should be optimal to fit larger documents in the context.
> This concept has potential, though; building lookup tables seems to outperform long text summarization.
It's a clever idea, and I agree that lookup tables and external storage of memory is likely going to be important at some point, but I suspect that's going to come out of giving LLM more ability to externally reference "long-term" memory rather than compressing everything into immediate context.