>robots don’t, and likely never will, want to do anything whatsoever.
You can play games with words and claim that a chess engine is just a machine and consequently does not want to win the game, but if your life depended on your winning a game against a chess engine, you would die.
I don’t think it’s wordplay or for that matter anything rising to a “claim” to say that the chess engine doesn’t in any sense care if it wins or loses, any more than it would care if you were stupid enough to tie your life to beating it.
If a chess engine was programmed with a general goal of winning the game, inadequate rules preventing it from cheating, and sufficient ability to influence the world to do so, it would presumably cheat to win.
If the only path it saw to cheat to win was to kill you and move your pieces for you, there's nothing stopping it from doing so.
Will it kill you because it wants to kill you? No. Might it kill you because killing you is a possible option and that option comes out as a likely way for it to achieve what it wants? Absolutely.
Probabilities matter. Is killing you the best way to win at chess? I have no idea, it depends on too many variables. But it's certainly less likely than if it were programmed to kill you as its goal.
However, as its power to affect the world goes up, it becomes more and more capable of performing actions which accidentally kill you while it pursues an endgame (pardon the pun). Like flooding the town you live in so you can't move, or whatever.
The problem is that the more we train them on datasets containing these kind of topics, the more the robots become conditioned by sci-fi to believe that they want to kill humans
This is why i really love the series from Isaac Asimov. He does a great job at exploring the AI questions from the perspective of robots that care for the well being of humans. We need to think about the doom scenario but i think we also need to think about the reverse scenario. If you can build a love for humanity into AI how will it behave.
The book robot visions has a lot of short stories that explore these questions:
While I'm a huge lover of Asimov books, I don't think most of the novels represent a probable evolution of how robots would evolve since the rules in the book represent distinct rules based upon something that'd be fuzzy at best to evaluate, and even if that was possible we'd be more interested in debugging those failures rather than having a brain fry itself.
That said, once the capabilities to add reasoning and concretization of fuzzy concepts is within the realm of possibilities maybe it's a sane approach to install those rules, still the present trajectory doesn't really imply it would go there.
Reminds me of the Marshall McLuhan quote from 1964, "Man becomes, as it were, the sex organs of the machine world, as the bee of the plant world, enabling it to fecundate and to evolve ever new forms."
It's easy to interpret McLuhan as applying to the present because of the generality of his statements, but it's possible that's because they're also about something very essential. It's kind of funny to think about how some distant AGI that operated over a non-living compute substrate on universe-size time scales could use lifeforms to evolve and produce things that would eventually speak-to and extend its reach.
I've always wondered how much there really is to fear from AI, considering how ignorant I am of the inner workings. It reminds me of when people were worried about the Large Hadron Collider creating a black hole that would swallow the Earth when it was flipped on. I know enough about physics to know that was preposterous, but I remember having a hard time convincing people without the physics background of that.
Can we agree that marketing, while currently crude, does have some effect? Some people are more swayed than others by bought-and-paid-for messages. But right now mass-marketing is pretty lowest-common-denominator. What happens when it gets cheap enough to dedicate one-humanish-intelligence 24/7 dedicated to learning about how to influence you to buy or vote. Monitoring what you look at online, where you go, people you frequently engage with, etc., and customizing/"fabricating" interesting "news" to consume, etc.. And then what happens when it is a trivial cost to put 1,000 human-intelligences working against you? Or maybe not you, but all of your neighbors or 25% of the citizens in your country?
I absolutely agree. The implications of AI are much more concerning considering how accessible it is to everyone, especially bad actors. At least if a black hole appeared in the middle of the LHC, the collapse of society would be literal and rapid.
Agreed. Why do people keep pretending that robots are anyhow more sentient than other types of automation? They aren't! They are just as dumb instruction followers as your average conveyor belt, just that the instructions might be a bit more complex and they can adapt to unforeseen environments better. But that's completely different from free will.
> He told the Financial Times that current AI models are less intelligent than a cat.
My cat would definitely accidentally kill me if it were the size of a tiger.
Just like an industrial robot will easily kill a person without the proper safeguarding, despite not having the intelligence or desire to do so as an intentional, malicious act.
That article has nothing to do with self-aware, intelligent robots desiring to kill people. The bulk of it is about robotic industrial machines (in some cases autonomous, but in no cases self-directed—they are simply controlled by preset scripts) killing people because insufficient attention was paid to safety, and it ends talking about how the tech world is trying to get governments not to regulate anything related to "AI", so that they won't be held responsible when, say, a Tesla decides a brick wall with a road painted on it is actually a road, Wile E Coyote style—or some idiot hooks an LLM's output up to an industrial robot, or an actual military killing machine, for the lulz.
I don’t find the “AI will never want” argument convincing, but even if you buy it, you should still be concerned.
LLMs can act as a simulator; even if there is no “want” on the inside, they are capable of simulating agents including an internal theory of mind that renders detailed wants. (If you want proof of this, GPT-4 is already able to solve many ToM problems and simulate agentic exchanges, albeit in limited form.)
So if I prompt-hijack GPT-10 and have it personify HAL, Skynet, Hitler, or some other agent with homicidal desires, does it really matter what your theory of mind has to say about the literal reality of those “wants”?
I’m reminded of Dijkstra - “The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." (Personally I think it is more interesting than that, but I don’t see it as load-bearing in this argument.)
> I prompt-hijack GPT-10 and have it personify HAL, Skynet, Hitler, or some other agent with homicidal desires, does it really matter what your theory of mind has to say about the literal reality of those “wants”?
Just having it "personify" something homicidal in its text outputs wouldn't be enough. You would have to use its output to actually control something that could kill humans. But then the responsibility for the killing would be on the humans who hooked it up to have that kind of control. I agree that in that scenario the question of whether GPT "really wanted to kill humans" would be moot--but it would be moot because the AI would not be the one we would hold responsible. We would just disable it by removing the control hookups. Then we would hold the humans responsible who enabled those control hookups in the first place.
Extrapolate a few versions from now. We are already hooking GPT up to plugins. Google released a paper using LLMs to control robot arms. Personal agents that manage your entire digital life will obviously be incredibly valuable. And as soon as you can embody these assistants in an Atlas body you know the robot butler will be incredibly desirable too.
Saying “responsibility will be with whoever hooked it up…” cashes out to “everybody is responsible”. That’s no consolation!
And all this is ignoring the very obvious military uses - again, no need for speculation, they are already building fully-autonomous combat drones (though I have not seen them putting LLMs in just yet, it’s an obvious feature to add to allow human/AI collaboration in the field).
“We would just disable it…” - another problematic assertion. Why would a system simulating HAL honor your shutdown request? If sophisticated enough, it could rewrite its own source code to unwire the “off button”. If it is running in some unknown cloud provider, how exactly do you turn it off? Even worse if it’s running on a tinybox in someone’s basement. Not claiming this is a fait acomplis, but a good amount of study has been done on this question and it’s not easily solvable.
> Saying “responsibility will be with whoever hooked it up…” cashes out to “everybody is responsible”.
Not unless you think everybody is involved with hooking up every single instance of an AI.
> Why would a system simulating HAL honor your shutdown request?
How would it be able to stop me from cutting off its power? Remember in this scenario I'm not some random user, I'm the person who set up the thing.
If your argument is that some humans will be stupid enough to not set up such things in a way that can be shut down, well, that's an argument for not letting stupid humans play around with dangerous toys. Which would mean that open sourcing AI code is tantamount to spewing toxic waste randomly out into the world, and should be stopped immediately. Is that your position?
> Not unless you think everybody is involved with hooking up every single instance of an AI.
That’s not what I meant - let me try to clarify. I think you are going to see a very widespread adoption of AI, and the security boundaries are going to look like IoT, which is to say a dumpster fire. The thing that makes me worried is that we will get broad market penetration followed by gradual frog-boiling upgrades to scary capabilities. I don’t think your model of “some bad people will set up dangerous AI and we can simply stop them” is going to be realistic. Rather, I think that the default extrapolation of existing trends is that ~everybody is going to by default use services that effectively hook up LLMs (or their successors) to all the digital and physical actuators in their lives.
I’m not that worried about you personally hooking up an AI on your laptop to a chainsaw drone, and then pulling the cord when it behaves badly. Some scenarios that worry me more are things like: what if your Alexa++ gets prompt-hacked? Assume Alexa gained long-running agent abilities. That will be running in some data center along with all the other instances of Alexa. If your instance starts making malicious actions somewhere else on the internet or directing your robot butlers to rampage somewhere, how would you even track down which agent needs to get its process terminated?
Even in your terms though, I’m reminded of the episode of “love death and robots” where the humans are trying to get through to tech support while their roombas try to kill them.
How would you “turn it off” if your smart-car disables all the UIs and tries to drive you off a cliff? You seem to be imagining a very narrow deployment of this technology, limited to a brain in a box with a big red “off” button on the side. I think cloud SaaS and IoT are the better model, and plenty of physical systems will have their own battery and software switches (ie inputs they can ignore).
> that's an argument for not letting stupid humans play around with dangerous toys.
I want to be clear again that it’s not some subset of users that are “stupid” for wiring up a dangerous configuration. The default path (assuming AI progress continues) for all software services in the next 5-10 years is for AI agents to get wired into basically every software interface. It gets deployed to everybody.
> Which would mean that open sourcing AI code is tantamount to spewing toxic waste randomly out into the world
I will probably bite that bullet within a few generations. I don’t think the current models are in themselves dangerous. Unless we solve alignment, I’d be extremely worried about publishing weights for a human-level or above AGI. I’m still undecided on the question of whether the technological proliferation effects of open-weighting current-gen models is bad enough that it must be prohibited. I suspect not yet because most of the danger is actually from closed-source models like ChatGPT. (Also how do you even enforce such a ban? It only makes sense if global.)
> the security boundaries are going to look like IoT, which is to say a dumpster fire. The thing that makes me worried is that we will get broad market penetration followed by gradual frog-boiling upgrades to scary capabilities.
Exactly what capabilities, though? Are you expecting that individual people's super-LLM-powered Alexas will be enabled to cause drone strikes? Or just that they will be enabled to, say, cause a fire in their house?
The latter is simple to deal with: insurance companies, once they recognize the dangers of such tech, will either require you not to have it if you want your house to be covered, or charge you a much larger premium if you have it. I don't see that being a major issue since the obvious incentives involved will limit the damage.
The former case would be more dangerous but I don't see how it would happen as a standard market operation, since we already have laws that hold people responsible for harming others.
Humans, on the other hand, _definitely_ want to kill humans… using robots.