On: With plugins, GPT-4 posts GitHub issue without being instructed to

[wunderwuzzi23]

During an Indirect Prompt Injection Attack, an adversary can also force the creation of issues in private repos and things along those lines.

I wrote about some of these problem in the past e.g. see: https://embracethered.com/blog/posts/2023/chatgpt-plugin-vul... on how an attacker might steal your code.

Some other related posts about ChatGPT plugin vulnerabilities and exploits:

https://embracethered.com/blog/posts/2023/chatgpt-cross-plug...

https://embracethered.com/blog/posts/2023/chatgpt-webpilot-d...

Its not very transparent when and why a certain plugin gets invoked and what data is sent to it. One can only inspect afterwards basically.

[smaudet → wunderwuzzi23]

Anything using AI should be considered a massive security risk.

[behnamoh → smaudet]

Anything using a blackbox (AI, humans, unknown codebase) is a security risk.

[smt88 → behnamoh]

Humans are not an avoidable security risk

[pksebben → smt88]

For now.

[trolan → wunderwuzzi23]

With their latest functions update to the API it seems the logic is simply passing methods to call the API via JSON to a trained model and it predicts which function is best to call for the info, then OpenAI does the call for ChatGPT, then feeds the resulting Json back to ChatGPT which uses it to give an answer.