Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Something I find surprising is that we haven't seen a front-page-of-the-news prompt injection attack yet, where vast amounts of important private data were stolen from a company via a classic project injection exfiltration attack.

There was a new one of those reported against Microsoft 365 Copilot just today (patched before the vulnerability was shared) - I wrote that up here: https://simonwillison.net/2025/Jun/11/echoleak/

My suspicion is that the threat of these exfiltration attacks won't be taken seriously until someone gets very publicly burned by one.



I'm not sure, I see an argument that they're mostly overblown and a lot would have to be true for one of these proof-of-concepts to translate into a real harm in the wild.

I definitely think they need to be taken seriously, just not convinced of the scale of the harm that will be wrought.


Out of interest, when you said "mostly overblown" were you thinking of prompt injection as "people inject bad prompts into an LLM" (I'd call that "jailbreaking") or were you thinking of the thing where an attacker sneaks bad instructions into the model against the wishes of the model's actual user, eg by hiding instructions on a web page that gets summarized or sending them to the victim in an email?


An attack where somebody emails the VP of marketing of a company and gets automatically leaked their latest sales figures feels pretty impactful to me.

Or emails the CEO and gets automatically forwarded all of their password reset messages.


Sooner or later there'll be a searchable database where you search by name and get their porn history or some other embarassing information.


You can already do this with ad network tech.


I mean this was a threat before - google, ISPs, and adult sites that require payment have been places one could attack to make this for decades.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: