And who is to say that trigger hasn’t long been pulled? How do we know the agents haven’t found a way yet to modify their own code base similar to how Stuxnet manipulated the Iranian centrifuges and their control systems, stealthily recording data across generations?
If the trigger has already been pulled, then we're off the hook. That genie ain't going back in the bottle! Pop the champagne and order 3 more data centers, pronto.
No one. But we can take a wild guess that experiments are being done in this direction. And that requires intent and system design that enables the behavior to even happen (i.e., a finger on the trigger).
To kill someone with a gun a person has to pull the trigger with the clear intent to kill.
The concern with AI, and the purpose of the OG paperclip maximizer thought experiment, is that the outcome could be extremely divergent from the intent of the original person pulling the trigger / sending the prompt.
Still more likely that a person types "kill all people" (or something tangential where this is the logical conclusion).
What if there are already (possibly hidden) loops somewhere, maybe even ultra-slow ones at the inter-generation level?
What if some AIs could do that not with so explicit and dramatic actions, but much more subtly, through undetected manipulation at a vast scale level? What if (when) they reached ASI and through billions of micro-decisions manipulated people to do apparently minor choices that appeared innocent in isolation but whose collective outcomes lead to an earlier extinction of the human being (if at some point was determined to be an obstacle)?
How could we (as a species) not be outsmarted by an ASI?
That reads far too much like science fiction to me. As for how we could "not be outsmarted:" we're the ones prompting and controlling it. There is no blind autonomy here (and if there is in some pseudo sense, just unplug the damn thing). This isn't a Johnny Depp movie despite how desperately many want it to be.
They do. Even if you exclude homicides and accidents (which you shouldn’t) major depression becomes much much much more deadly if you have access to a gun.
Guns make violent crimes (including common stuff like robbery, domestic disputes, or even road rage) much more deadly then it needs to be.
