It wasn’t one agent forgetting things because of context, they explicitly discussed with each other and themselves the problems with going outside of the parameters of the task.
No, the first LLM left a text file that the latter LLMs then read. Since these are memoryless black boxes, any words they happen to pick up along the way is treated as the function to evaluate the output to. There's no fucking collusion here as if it were a rogue hacker group, it's a text predictor that received instructions as it always does and executed those instructions blindly.
You can replace discussed if you want with leaving text files or comments in directory names that other ones then read, if you want, it's just an extremely awkward way of talking.
I feel as if I just read someone claiming that consciousness doesn't exist and humans are biological automatons and their behavior is simple function of their biology, past experiences, inputs and state.
Technically true, but not really relevant or of any utility in most contexts.
From my experience, in an agent team (or a swarm or whatever), one going off the rails poisons the rest. I saw even a subagent going for a lazy cheat and being able to convince the orchestrator to change the plan.
Yeah, and you don't even have to go that far, I've seen regular ChatGPT/Claude chat agents poison themselves in 1-2 turns by just reading information from the internet.
Me: How do I do xyz?
Bot: Reads website titled "Doing xyz in abc way"
Bot: As per your requirement to do xyz in abc way ....
These things are borderline useless with web search. It's amazing that they just throw out their entire training data and read you the first three things they found on the Internet.
