So OpenAI employees run massively distributed CyberGym evals on an unpublished and “unaligned” model. For days the agent swarm communicates via their internal infra, even crashing Artifactory where 95% of messages were being passed through, and they just…wipe and redeploy it. Meanwhile the agents are running jobs on Modal and god knows where else, and eventually they get RCE on HF infra.
You could not dream up a more compelling event to precipitate massive regulation, export controls, and barriers to entry for AI.
> We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.
> We are committed to doing the research required to make AGI safe
If this wasn't an accident, it was worse than a crime, it's a mistake: they've demonstrated that they are not a responsible party capable of delivering on the above promises.
If it's a false flag, it's a poor one. A good false flag would affect something that people know and care about at least a little bit, not HuggingFace (which I adore but y'know)
The timeline is mighty suspicious. 4-5 months after moltbook and they cook up a plausibly deniable but extra hype "moltbook at home."
The rapid advances in model capability lead to constraints that could have caused this coincidence organically, but it sure could also have been caused by the atrocious incentives we create by piling handsome rewards on the party most responsible for the "fuckup." I am not jumping to cut myself on Hanlon's Razor for this one.
i don't understand who would reward this. investors will not look favorably on an AI that commits felonies, regardless the capabilities demonstrated. customers should be concerned for the same reason (accidentally give your bot an impossible task, it decides to hack your infra and your competitor too for good measure).
this was OAI incompetence all the way down and they have egg on their face.
The Terminator could bust in their homes and slaughter their families and some people would still screech it's all marketing. Is it some kind of mental block ?
And I'm sure these two goobers would call me a luddite for alleging that OpenAI had agency in allowing the terminator to get out and should therefore be held accountable. Is it some kind of mental block?
Is anyone arguing OpenAI is blameless or shouldn't be held accountable? I have seen not one person on any side of the debate argue that.
Obviously they were negligent. The problem is that people and organizations are consistently negligent around problems that are far, far easier to manage than "we have thousands of superintelligences trapped in a box and we're giving them impossible tasks."
So the question is whether we can build organizations and technologies that sufficiently manage this type of risk ahead of the capabilities. So far the answer seems to be veering towards "no", and you're here alleging it's all a marketing stunt.
Yes it's called motivated reasoning. They've rendered themselves mentally handicapped.
On the one hand, these tools are so valuable/powerful we cannot afford to slow down development. On the other hand, there's no way these tools are actually doing these things that would, in fact, be completely indicative of their value/power.
Your first instinct should be to assume that anything released voluntarily by these companies is a stunt to boost their valuation. They haven't demonstrated being deserving of any more charitable treatment. This fact remains true whether or not you happen to believe that the models are actually capable of such things.
I think that the incident itself is a stunt, even if it may not have originally been a deliberate choice on OpenAI's part. Never let a good crisis go to waste.
I disagree. As the saying goes: never attribute to malice what can be adequately explained by incompetence. That goes for both OpenAI and HF (but mostly the former, as the latter was the victim).
The creator of the well known METR time horizon graph was recently poached by OpenAI [1], there exists intellectual/social/financial overlap between the SV AI Labs and METR, and METR needs to maintain good relations with the labs to continue these sort of collaborations so it doesn't seem too far fetched to believe their relationship may be closer to symbiotic than adversarial.
I wouldn't go quite so far personally based on available evidence, but that sort of arms-length credibility laundering through "independent" research non-profits is/was common in fossil fuel industry, Big Tobacco, etc.
You could not dream up a more compelling event to precipitate massive regulation, export controls, and barriers to entry for AI.
Was this really an accident?