Hacker News new | past | comments | ask | show | jobs | submit login

SQL injection enters the chat



I'm a little cautious of comparisons to SQL injection now, because while some of the comparisons are very valid (particularly around the risks), prompt injection isn't really the same category of vulnerability as SQL injection -- so mitigation techniques for SQL injection (escaping input, sanitizing) aren't going to work to stop prompt injection.

But otherwise yeah, it can be helpful to think of prompt injection as if someone is effectively doing XSS on your AI agent (again, keeping in mind that the mitigation techniques are not the same, it's an entirely different method of attack). People tend to think of the jailbreaking examples or getting the agent to swear -- which can be embarassing but also mostly harmless. The reality is that prompt injection is basically arbitrary reprogramming of the agent, and arbitrary insertion of new tasks, and data poisoning/replacement, and data exfiltration, etc...


Yeah, the confusion between jailbreaking and prompt injection is definitely a big problem.

People who are frustrated at the safety measure that jailbreaking aims to defeat often assume prompt injection is equally "harmless" - they fail to understands that the consequences can be a lot more severe to anyone who is trying to build their own software on top of LLMs.


I was referring specifically to the timeline and how there was a sarcastic expectation that they would fix it at a certain stage




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: