> Just like code was the battleground back then, open model weights are the new frontier. Think about it—code is just a bunch of instructions, right? Well, model weights are pretty much the same; they're the brains behind AI, telling it how to think and learn. Saying "nah, you can't share those" is like trying to put a genie back in its bottle after it's shown you it can grant wishes.
I think telling a genie "I wish for no more wishes" is a common enough trope.
I'd agree that making weights available is basically irreversible; however making it illegal to make new sets of weights available is probably fairly achievable… at present.
-
Some of the issues with "winning" the battle for encryption include:
1) We also need it to defend normal people from attackers
2) It's simple enough to print onto a T-shirt
3) The developers recognised the value and wanted to share this
The differences with AI include:
1) The most capable models don't fit on most personal devices at present, let alone T-shirts
2) 95% of the advantages can be had from centralised systems without needing to distribute the models directly to everyone
3) A huge number of developers have signed an open letter which is basically screaming "please regulate us! We don't want to be in an arms race with each other to make this more capable! We don't know what we're doing or what risks this has!"
As long as everyone admits that number 3 “please regulate us” is really just asking for regulatory capture and is in no way a good faith move, but rather protectionism. Then I’m good to proceed with these conversations.
These people have not just suddenly developed consciouses. This is a game move.
1) You can run a very large model on existing cheap hardware, with poor performance, but it will run. Also, "most personal devices at present" is doing all the work here. Obviously if there is now demand for devices with hundreds of GB of VRAM then sellers will soon make them and buyers will buy them. That amount of memory is not actually that expensive. And it's presenting a false dichotomy where the only alternatives are companies the size of Microsoft and the ability to run GPT-4 locally on your existing iPhone. There are thousands of medium-sized companies that can afford five to six figures in hardware, and any individual could rent big hardware on any cloud provider -- requiring them to compete with each other -- instead of locking each model behind a single monopoly service.
2) Claiming that the benefit of people being able to run the models locally is only 5% is absurd. The privacy benefit of not sending your sensitive data to a third party is worth more than that by itself, much less setting up some huge institutions to have an oligopoly over the technology and subject everyone to all the abuses inherent in uncompetitive markets. But from the perspective of those institutions they want to classify those benefits as costs, so they try to come up with some malarkey about how they need to gatekeep for "safety" because so they can control you is unsympathetic.
Ah, now I see my mistake; I gave a list of what I see as things of political importance without making it clear this is what I was doing.
That the source code for unbreakable encryption fits onto a T-shirt made it clear to the US government that it was a "speech" issue, and also made it clear that it was in any practical sense unstoppable.
That big models can't fit onto most person devices at present is indeed likely to be a temporary state of affairs, however it does also mean that the political aspect is very different — the senators and members of parliament can't look at the thing and notice that, even if the symbols are arcane and mysterious beyond their education, it's fundamentally quite short and simple.
And, of course, if they're planning legislation, they can easily say "no phones with ${feature} > ${threshold}", which they've already done on various compute hardware over the years, even if the thresholds seem quaint today: https://www.youtube.com/watch?v=l2ThMmgQdpE
Politicians don't care if all of us here think they look silly.
> 2) Claiming that the benefit of people being able to run the models locally is only 5% is absurd. The privacy benefit of not sending your sensitive data to a third party is worth more than that by itself, much less setting up some huge institutions to have an oligopoly over the technology and subject everyone to all the abuses inherent in uncompetitive markets.
This seems a surprising claim, given the popularity of cloud compute, Cloudfare, third-party tracking cookies, DropBox, Slack/MS Teams, Gmail, and web apps run by third parties such as Google Docs, JIRA, Miro, etc.
There are certainly cases where people benefit from local models. The observation of all these non-local examples is part of why I think that benefit is about 5% — though perhaps this is merely sample bias on my part?
> But from the perspective of those institutions they want to classify those benefits as costs, so they try to come up with some malarkey about how they need to gatekeep for "safety" because so they can control you is unsympathetic.
They've been saying this "malarkey" since before they had products to be marketed.
> That big models can't fit onto most person devices at present is indeed likely to be a temporary state of affairs, however it does also mean that the political aspect is very different — the senators and members of parliament can't look at the thing and notice that, even if the symbols are arcane and mysterious beyond their education, it's fundamentally quite short and simple.
You can write the RSA algorithm on a t-shirt but in practice you need a computer to run it.
Likewise, you can fit llama or grok on a thumb drive and carry it around in your pocket, but in practice you need a computer to run it.
> And, of course, if they're planning legislation, they can easily say "no phones with ${feature} > ${threshold}"
But what good is that?
If they're actually trying to accomplish an outcome, those kinds of rules are completely useless. If all they're trying to do is check the "we did something about it" box, there are a hundred other useless things they could do that would be equally ineffective but still check the box and cause less collateral damage.
> This seems a surprising claim, given the popularity of cloud compute, Cloudfare, third-party tracking cookies, DropBox, Slack/MS Teams, Gmail, and web apps run by third parties such as Google Docs, JIRA, Miro, etc.
Notice how these are the kinds of things that major companies with trade secrets to protect have explicitly banned, and individual consumers with no bargaining power suffer while complaining about it?
> They've been saying this "malarkey" since before they had products to be marketed.
Eccentrics have been saying this "malarkey" for a long time. It becomes the official position of the market leader when it is convenient.
> You can write the RSA algorithm on a t-shirt but in practice you need a computer to run it.
Pretend you're a politician: the thing you've been told is a state secret is written onto a T-shirt by a protestor telling you it isn't secret, everyone already knows what it is, you're only holding back the value this can unlock. You, the politician, might feel a bit silly insisting this needs to remain secret even if you think all the 'potential value' talk is sci-fi because you can't imagine someone doing their banking on the computer when there's perfectly friendly teller in the bank's local office.
If the same discussion happens with a magic blob on the magic glass hand rectangle which connects you to the world's information and which is still called a "telephone", you might well incorrectly characterise the file that's actually on the device of the protestor talking to you as "somewhere else" and "we need to stop those naughty people providing you access to this" and never feel silly about your mistake.
> But what good is that?
"Then the people can't run the 'dangerous' models on their phones. Job done, let's get crumpets and tea!" — or insert non-UK metaphor of your choice here; again, I'm inviting you to role-play as a politician rather than to simulate the entire game-theoretic space of the whole planet.
> cause less collateral damage
I made that argument directly to my local MP about the Investigatory Powers Act 2016 when it was still being debated; my argument fell on deaf ears even with actual crypto, it's definitely going to fall on deaf ears when it's potential collateral damage for a tech that's not yet even widely distributed (just widely available).
> Notice how these are the kinds of things that major companies with trade secrets to protect have explicitly banned, and individual consumers with no bargaining power suffer while complaining about it?
No.
Rather the opposite, in fact: each is used by major companies.
> Eccentrics have been saying this "malarkey" for a long time. It becomes the official position of the market leader when it is convenient.
It was the position of OpenAI with GPT-2, which predates their public API by 16 months, and them being a "market leader" by 3 years 9 months:
February 14, 2019: "Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper." - https://openai.com/research/better-language-models
June 11, 2020:
"""What specifically will OpenAI do about misuse of the API, given what you’ve previously said about GPT-2?
With GPT-2, one of our key concerns was malicious use of the model (e.g., for disinformation), which is difficult to prevent once a model is open sourced. […]
We terminate API access for use cases that are found to cause (or are intended to cause) physical, emotional, or psychological harm to people, including but not limited to harassment, intentional deception, radicalization, astroturfing, or spam, as well as applications that have insufficient guardrails to limit misuse by end users. As we gain more experience operating the API in practice, we will continually refine the categories of use we are able to support, both to broaden the range of applications we can support, and to create finer-grained categories for those we have misuse concerns about.""" - https://openai.com/blog/openai-api
People called them names for this at the time, too, calling their fears ridiculous and unfounded. But then their eccentricities turned out to lead to successfully tripping over a money printer and now loads of people interpret everything they do in the worst, the most conspiratorial, way possible.
I'm reminded of the last Narnia book, specifically the Dwarfs who could not see their surrounds.
"""They have chosen cunning instead of belief. Their prison is only in their minds, yet they are in that prison; and so afraid of being taken in that they cannot be taken out."""
Is it really so hard to believe they might be sincere? We've had tales of scientists — and before them alchemists and wizards — undone by their creations for a very long time now. It's a very convenient meme to latch on to; surely even if you do not share this caution, or find it silly for whatever reason, you can at least be aware of it existing in others?
The term "conservative" may be now affiliated with a specific political team more than the concept of being risk-adverse, but the idea is found even in the history of the ancient Romans and the tales from ancient Greece.
And I say this as one who personally found the idea of conservatism strange; who is an "all improvements are change" person; a person who took a long time to learn about the metaphor of Chesterton's Fence; and is a person who even now finds it hard to fully empathise with those who default to "treat all change as a potential regression until proven otherwise" even though I have to force myself into that attitude while writing unit tests.
I think telling a genie "I wish for no more wishes" is a common enough trope.
I'd agree that making weights available is basically irreversible; however making it illegal to make new sets of weights available is probably fairly achievable… at present.
-
Some of the issues with "winning" the battle for encryption include:
1) We also need it to defend normal people from attackers
2) It's simple enough to print onto a T-shirt
3) The developers recognised the value and wanted to share this
The differences with AI include:
1) The most capable models don't fit on most personal devices at present, let alone T-shirts
2) 95% of the advantages can be had from centralised systems without needing to distribute the models directly to everyone
3) A huge number of developers have signed an open letter which is basically screaming "please regulate us! We don't want to be in an arms race with each other to make this more capable! We don't know what we're doing or what risks this has!"