The idea with these isn't that you can trick chat gpt into giving you bad descriptions of how to do crimes, but that when someone decides to wire it up to something with real world impact, you'll be able to push it towards something you want.
For example, we are told that customer support is about to be fully automated soon. These attacks could be used to eg get refunds for bogus reasons. There is already one real life example I know of that didn't even need tricks,
The space of ChatGPT tricked for fun or fraud is probably just a subset of the error and confusion in which those AIs operate - generally to the confusion of the customers that they "support". I can imagine a future where humans have to deal with an uncanny valley of unreliable and confusing AIs of increasing significance that are wired up to more and more real world systems. And at the same time companies reserving the right to legally back out of any adverse outcomes through a fine-print EULA and unintelligible documentation in hard to find places.
For example, we are told that customer support is about to be fully automated soon. These attacks could be used to eg get refunds for bogus reasons. There is already one real life example I know of that didn't even need tricks,
https://www.forbes.com/sites/marisagarcia/2024/02/19/what-ai...