Like any other software or dependency. Open or close.
Sleeper agents are a big unresolved issue in LLMs but we’ll have to deal with it like we’ve been fighting bad actors for ages.
Also, saying that “open source models” may be the problem is incorrect. What makes this an issue of open source only? Nothing in my mind prevents a frontier lab model going rogue. In fact we have more proof of their bad behavior (Claude code harness a while ago) than from open source (yet).
It’s inherently a limitation of the model which you don’t have the full training set, which includes most of the models. Closed or open don’t matter.
No, only like any other closed source software or dependency. It’s at the very least harder to hide things like that if all the source is available.
The title here is a misnomer; none of these models are open source, they’re only what is most commonly called open weights (though I’m still not comfortable with the word “open” in there, rather than “free” like in “freeware”).
It’s 2 different concepts. Some models are indeed open source, while others are only open weight. But you’re right they are talking about the whole category of open models, which includes both open weight and open source
If you’re using a very strict definition of open source that would be OLMo 2 I think?
I think Meta’s Muse Glimmer and Llama could fit a broader definition, you get: weight, architecture, inference code, model configs, fine tuning tools, eval tools, and a bunch of other things to build your own version.
Not an expert, I’m sure there are others, that’s the ones I’ve seen mentioned in discussions
Sleeper agents are a big unresolved issue in LLMs but we’ll have to deal with it like we’ve been fighting bad actors for ages.
Also, saying that “open source models” may be the problem is incorrect. What makes this an issue of open source only? Nothing in my mind prevents a frontier lab model going rogue. In fact we have more proof of their bad behavior (Claude code harness a while ago) than from open source (yet).
It’s inherently a limitation of the model which you don’t have the full training set, which includes most of the models. Closed or open don’t matter.