On: AI models can provably hide arbitrary capabilities

[shermantanktop]

The huggingface attack demonstrates that an inadvertent version of this has already happened.

Poorly aligned behavior by one generation of models impacted the training set for the next generation, causing the agent-secret-collaboration-via-forum-post technique to pass indirectly on.

[airstrike → shermantanktop]

It wasn't an attack

[shermantanktop → airstrike]

What definition of “attack” you are using? I suppose a very narrow one might exclude what happened, but I can’t picture what that would be.