I think this is definitely one of the bigger risks here, and the only way AI could 'kill us all' in the foreseeable future:
> 3.4% named existential risk, and the most common answer was malicious use, at 10.6% (UCL).
The danger isn't about sentient or human level AI, it's about how AI gives everyone capabilities beyond their wildest dreams, often without the knowledge or experience necessary to use those capabilities responsibly.
Forget Terminator, the two risks I can see being the most important here are
"major security risk/system failure due to incompetent engineers/maintainers using AI without knowing how things work" and "criminals/terrorists/rogue states use it make their crimes easier and more efficient".
The latter seems especially bad when political stability is down across the board and a lot of people seem frustrated or outright furious with society at the moment.
Still, I feel the risks are still on the manageable side, simply due to both automated systems and malicious actors being constrained by logistics and a human driven society.
I think this is something that a lot of people miss when arguing about AI.
I completely believe ASI will be able to wipe us from the planet, I don't know if ASI will get the chance to do that because humans with almost ASI will very likely either kill us all or mess up things so bad that everything falls apart before then.
And that's ignoring all the other bad and destabilizing outcomes that are almost certain. We must look at our world like it's in a 1930s state right now. Rapid technology changes and things like large scale 'decoherence' of societies are very likely to lead to very large chunks of the population willing to revolt.
Right. In order for AI to take us out, it has to survive long enough for the incoming Ukraine/Russia/Iran/USA WWIII debacle that we're already in to not kill us all off first. Oil markets are in trouble and things are about to go sideways.
And then Elon is going to have an army of killbots at his command. Those Tesla bots are going to be given guns and they're going to go around, door-to-door, making sure you comply. Thankfully hands are really hard to do so they haven't been able to scale up production yet.
This is the issue I have with everyone running agents writ large. Will Muse do people harm? Eh, maybe through some sort of bug because software has bugs and software bugs have hurt (and even killed!) people in the past. Those are the edge cases though.
Will malicious people take advantage of Muse? Now you're talking. Muse is functioning like the world's most open ended API that people are running without having any idea what it does. You're inviting other people to run it so that they can do good things for you. Doordash use my API (Muse) to send me food! Random stranger use my API to do something good for me! Random stranger wants to do something bad? Well the API is designed to do that. Hopefully it does it well but when it's people vs. API, people have a track record (especially when incentivized by money) of doing pretty well.
> 3.4% named existential risk, and the most common answer was malicious use, at 10.6% (UCL).
The danger isn't about sentient or human level AI, it's about how AI gives everyone capabilities beyond their wildest dreams, often without the knowledge or experience necessary to use those capabilities responsibly.
Forget Terminator, the two risks I can see being the most important here are "major security risk/system failure due to incompetent engineers/maintainers using AI without knowing how things work" and "criminals/terrorists/rogue states use it make their crimes easier and more efficient".
The latter seems especially bad when political stability is down across the board and a lot of people seem frustrated or outright furious with society at the moment.
Still, I feel the risks are still on the manageable side, simply due to both automated systems and malicious actors being constrained by logistics and a human driven society.