I'm curious, you say "super interesting engineering" but then they say "it will often just tell you, even though we’ve asked it not to" and to me that seems like extremely shit engineering.
Where are the interesting engineering parts at? Seems to be an interesting idea and perhaps design, but to call the implementation/engineering itself bad seems to be an understatement.
The security and the overall engineering were entirely separate items in that comment I think. It was explicitly called out that this is a security problem that you'd really only see treated in this manner in the LLM space. For what it's worth it's effectively unsolvable (AFAIU) short of realizing AGI with an amicable alignment.
What do you mean unsolvable? Don't give the LLM access to stuff it shouldn't, this is like Access Control 101, not sure how anyone can claim that particular problem is unsolvable?
Which again, pretty huge fuckup on OpenAI's side to run that sort of security testing on 3rd party hardware, and not on a airgapped machine, kind of amateur hour to be honest. Again, same principle, don't give it access to stuff you don't want it to access.
Where are the interesting engineering parts at? Seems to be an interesting idea and perhaps design, but to call the implementation/engineering itself bad seems to be an understatement.