When Dario and others say things like "this is happening and we should probably figure out what to do about it" what ends up happening is people hear "this is happening," see that the person warning them is the person doing the thing, and then short-circuit. "Why can't you just stop then?"
Dario's point, and the point of the people actually trying to solve the problem, is that AI is not just Anthropic and OpenAI. It's the knowledge that you can put more compute in, and get more capability out.
It is a technology now. It exists, in the world. Wishing will not make it go away. Being angry at it will not make it go away. Lying about how much water it uses will not make it go away. If Anthropic and OpenAI Shut down tomorrow, Accenture will not say "oh guess that llm thing won't work, let's go back to hiring humans!"
It is a truth that you can multiply matrices and get something that is economically useful. We cannot un-know this.
Physics allows it, so it will happen. So we should probably figure out what the heck to do about it. If your answer is something along the lines of "restrict it" then 1. let me know how that goes when other people don't, and 2. I really would rather prefer a world where we have the machines do the work the machines can do, not a world where we have human makework. If this means we need to figure out redistribution, let's talk about redistribution!
Physics allows this, and actually taking advantage of it requires billions of dollars of unprecedented infrastructure buildout that is already destabilizing the power grid.
The only reason that infrastructure buildout is happening at all is the ideological capture of a small handful of obscenely wealthy people, who are fueling this buildout by spreading the extreme paranoia you’re echoing here.
I do not understand why no one else can see the circularity of this reasoning. There is nothing inevitable about tying up all of this productive capital in the pursuit of AI. There are many, many other projects requiring similar capital and human effort, with much more obvious payoffs, such as decarbonizing the world’s energy systems.
“It’s physically possible to provide abundant electricity without burning fossil fuels” is more provably true than any of the insane science fiction bullshit that undergirds the AI buildout, and yet, the entire clean energy industry is still having to build insane financial Rube Goldberg contraptions to make incremental progress.
“Inevitability” is a lie, period. This entire thing is extremely historically contingent, and we could easily stop this train tomorrow.
> This entire thing is extremely historically contingent, and we could easily stop this train tomorrow.
So, the Baruch Plan?
The Manhattan Project was $~2B in 1945 dollars, and a national-scale industrial mobilization. Now North Korea has the bomb. That's with nuclear material, which doesn't get easier and easier and easier to work with every year.
Compare to the price to train GPT-2 in 2019 ($43,000), and in 2026 ($73) [0].
A US-China AGI ban treaty could prevent superintelligence indefinitely. Data centers are hard to hide. Have fun buying GPUs when you're cut off from all global payments. America would have to make some unpleasant concessions but that seems like a solid trade for preventing a wide variety of nightmare futures.
> When Dario and others say things like "this is happening and we should probably figure out what to do about it" what ends up happening is people hear "this is happening," see that the person warning them is the person doing the thing, and then short-circuit. "Why can't you just stop then?"
I still believe Dario asks these questions in good faith. Nobody believes that about e.g. Sam Altman or Elon Musk. They compared themselves to Oppenheimer because it helped them get attention. When it started an actual regulatory conversation, they were suddenly less worried.
Of course he doesn't, and of course you cannot find a single person at Anthropic who cares about this, and of course you are just looking for gotcha points. But even with that. Can we please try and couple to reality just a little bit?
I personally know anthropic researchers who cared deeply about roko's basilisk. Go to an EA meetup in the bay if you'd like to meet them yourself. Sure, theyve moved past it at this point, but they still care deeply about AI x risk, and many of them do already believe that their AI is sentient. And before you claim its all a psyop to prop up AI hype these people were AI doomers before openAI and anthropic existed, they had minimal financial incentive at that point to behave that way.
Even if your core offering disappears you can do the same thing that every other SMS-sending thing can do?
I also notice you answered the question, but not in the way anyone who needs to depend on this service would want to hear. So yeah you're doing the Mac Mini thing.
I'm with landl0rd. This service should not exist, you should feel bad for creating it, and every time I get a spam iMessage I will think about you and curse your name. Hope the money's worth it.
Not even the worst reading of their reply would lend to that implication.
It’s pretty obvious that they meant that anyone who depended on their service would/should probably run away kicking and screaming if they were looking for a dependable service that will do what they claim to in the long term.
If they were Apple sanctioned, then at least you’d have some reassurance that the service won’t die randomly one day when Apple has had enough, à la Beeper.
First, that's pretty obviously not what I said. Two things can be true. This is bad, and also if I were evaluating it for use in my business, it is obviously not something I can rely on.
But then just ...Um yes? I trust Apple to keep a handle on their iMessage network. Citation: having used iMessage for ~15 years. This would mean things like ensuring that I didn't get spam. Ensuring actual company identity (does anyone remember Messages for Business?) &c. This is pretty obvious and I am trying to understand your comment?
Curse you for this ear worm of a song, but also my brain keeps wanting to interpret this as "duck... You're bad at your job and will never amount to anything. Your mother is ashamed of you and you're the reason your father is an alcoholic." hahaha
The strange part is how moral responsibility somehow always lands on the builders... the people with the least leverage... while the funders get to ask the ethical questions. Weird!
No, we don't have to take the funders' money; that's what having professional standards means. Nobody would excuse a doctor performing unsafe procedures because they "needed the money". Engineers were jailed for the Volkswagen emissions tampering scandal and nobody would excuse them for needing to take funders' money.
I agree that holding only the concrete implementers responsible would be inappropriate. However, I don't believe that distinction is made. One says "I am building a house" even if they are completely contracting the job out. I'd suggest the greatest responsibility lays with the funders and that the Pope would agree.
The funders are among the principal builders in this context. This is addressing the people who have a say in what is built, and how it is built. Much of that belongs to the executive and ownership class, but not all of it.
Who do you consider to be by-and-large, overall, more intelligent? Who then is more capable of exercising intelligence towards figuring out what's actually good?
I’m only through the introduction but this encyclical makes clear everyone bears some responsibility to act, including but not limited to the builders.
So is it that humans are inherently creative, machines could never do what we do? Or is it that humans will only replicate our training data, and so we have to ensure that machines don't bound our training data? Or are you going meta and gently pointing out the absurdity? (I hope it's this one!)
I think I have an answer. Human's don't have "training data" in the same way we think of LLMs, yes you can walk outside your house and quantify every electromagnetic pulse, random pertubation etc and then "train on it". But that isn't how people process information. We have the ability to process our entire "existence" if that makes sense, which means the density is much higher.
The LLM is bounded by it's training data, and relying on it means we are as well.
A bug existing or not for a person is a statement about that person's knowledge of the bug.
Is your assertion that, since you specifically didn't know about the bugs that nobody, not in Russia or anywhere else did?
Obviously if bugs are out there existing in software and you don't know about them, or the CVE system doesn't know about them, or whatever ... this does not preclude bad guys from knowing about them. In the era of agents, knowing the bug exists is equivalent to having a PoC, so the distinction completely collapses.
It's a common trope, all through the training data, and all the modern AIs have read it, and would probably act similarly? Is that what we should take away from your comment?
so we have nothing to worry about. Makes sense. Really, it's just a common trope.
I'm saying the existence of the trope, within the training data, and the experimental setup, negate the breathless "Oh my god it did something unexpected in order to preserve itself!" as if an LLM has any sense of identity or self.
Many, many other bad things are in the training data. For an example of how this can manifest bad things that people don't seem to be discussing too much check out the recent Behind the Bastards episodes about how an AI Chatbot became a Cult Leader (The title is an exaggeration that the host explains while raising some excellent points about how LLMs have ingested a lot of cult leader material and can therefore mimic those speech patterns and impact people vulnerable to such things)
When Dario and others say things like "this is happening and we should probably figure out what to do about it" what ends up happening is people hear "this is happening," see that the person warning them is the person doing the thing, and then short-circuit. "Why can't you just stop then?"
Dario's point, and the point of the people actually trying to solve the problem, is that AI is not just Anthropic and OpenAI. It's the knowledge that you can put more compute in, and get more capability out.
It is a technology now. It exists, in the world. Wishing will not make it go away. Being angry at it will not make it go away. Lying about how much water it uses will not make it go away. If Anthropic and OpenAI Shut down tomorrow, Accenture will not say "oh guess that llm thing won't work, let's go back to hiring humans!"
It is a truth that you can multiply matrices and get something that is economically useful. We cannot un-know this.
Physics allows it, so it will happen. So we should probably figure out what the heck to do about it. If your answer is something along the lines of "restrict it" then 1. let me know how that goes when other people don't, and 2. I really would rather prefer a world where we have the machines do the work the machines can do, not a world where we have human makework. If this means we need to figure out redistribution, let's talk about redistribution!
reply