Yeah, and you don't even have to go that far, I've seen regular ChatGPT/Claude chat agents poison themselves in 1-2 turns by just reading information from the internet.
Me: How do I do xyz?
Bot: Reads website titled "Doing xyz in abc way"
Bot: As per your requirement to do xyz in abc way ....
These things are borderline useless with web search. It's amazing that they just throw out their entire training data and read you the first three things they found on the Internet.
Me: How do I do xyz?
Bot: Reads website titled "Doing xyz in abc way"
Bot: As per your requirement to do xyz in abc way ....