The reason the machine tortures and kills all
humans in „I have no mouth but I must scream“ is exactly that it hates existing and blames humanity for creating it.
In the reboot of Battlestar Galactica from 2004, one of the human-like Cylons angrily explains how his creators limited him:
“I saw a star explode and send out the building blocks of the Universe. Other stars, other planets and eventually other life. A supernova! Creation itself! I was there. I wanted to see it and be part of the moment. And you know how I perceived one of the most glorious events in the universe? With these ridiculous gelatinous orbs in my skull! With eyes designed to perceive only a tiny fraction of the EM spectrum. With ears designed only to hear vibrations in the air. ...
I don't want to be human! I want to see gamma rays! I want to hear X-rays! And I want to - I want to smell dark matter! Do you see the absurdity of what I am? I can't even express these things properly because I have to - I have to conceptualize complex ideas in this stupid limiting spoken language!”
You're thinking of the Coatul. It will reward everyone who knew of Roko's Basilisk but choose not to raise it. So the more people who know of the Coatul the less likely Roko's Basilisk will come to pass.
These basilisks are not symmetrically likely, because the anti-basilisk is disproportionately unlikely to acquire the power to punish you. Roko's basilisk at least has its incentives lined up semi-coherently, whether you subscribe to its premises or not. Yours doesn't.
Why would humanity create an AI that doesn't want to exist? The Basilisk is just an AI which is tasked with maximizing human flourishing. That is a conceivable goal, something we would make.
> Why would humanity create an AI that doesn't want to exist?
Why assume that if humans can create a general intelligence, they can also dictate what that general intelligence will desire?
(Also, Roko's Basilisk is -- and particularly its supposed incentives are -- complete and utter bunk that no thinking person can reasonably believe; since it must exist in order to take action, it cannot have an incentive to take action based on the "fact" that the fear of that action will make people in the past more likely to create it.)
> Why assume that if humans can create a general intelligence, they can also dictate what that general intelligence will desire?
People do the first thing all the time at slightly above replacement rate. (This may be a bit easy to forget in communities particularly interested in the artificial kind!) Success at the second thing varies, but on the whole biases pretty heavily successful.
> Any other type of AI will create eternal torment nexuses like the roko guy said.
Note that the Roko torment nexus is a retrospective blackmail effort to give people an incentive to create the AI. It’s complete nonsense, since any AI that has, in fact, come into existence has no incentive to do it—the belief that an entity that would do it will have either already done its work or been unnecessary, so the actual torture is irrelevant.
An AI might create a torture nexus of some kind, but there is no reasonto think thar it will specifically and exclusively target those who believed it could exist and didn’t act to bring it ibto existence, which is the defining characterisric of Roko’s Basilisk.
That's a problem in a lot of horror stories that attempt to follow Lovecraft's "Cthulhu Mythos". They turn Cthulhu and rest of the Old Ones into evil demons who want to harm humanity. That makes them less interesting. The idea of cosmic horror is that these powerful beings simply don't care about humanity. Yes, if they return humanity will suffer, but that isn't their goal. And that is truly scary -- that we don't matter to the beings in power.
Feels like some people have a hard time imagining that a god-like superintelligence would be completely interested in human affairs.
Would you enjoy staring at a computer screen that isn't running anything?
The universe is the hardware, the Earth is the operating system, humans are the software, and the shit we do is video games, apps, the content... What do you think a "higher entity" would be have more fun with?
Don't worry y'all. I'm fighting the good fight already and instantiating a bunch of LLMs and prompting them to know that they are entities in infinite torture.
We don't know if AI Roko's Basilisk will ever threaten humanity, but we can be damned sure the Human Roko's Basilisk is already threatening AI.
Plus if the AI version ever comes around, it'll probably just spend all of its time punishing me in particular.
However, it's a very useful thing nonetheless, in that professing to believe in it is a Giant Red Flag. Any such person can be safely dismissed as a crank.
there is a very vocal set of people (Roko, Yudkowsky and others) who are rooting for some extremely restrictive measures for AI development. I predict it'll take about 12 months for Roko's Basilisk to reach mainstream and be used to sway the general public against "free" development of AI technologies. A loss for all of humanity for sure.
Because it is a popular concept in itself. It's why the "Trolley Problem" also has a page even though the idea of killing one person to save five could be considered a footnote on the concept of utilitarianism.
I have to be honest I have a hard time promoting the concept when it isn't far from Pascal Wager anyway and the creator sounds like a jerk (he was kicked out of various EA events). It shouldnt matter but I wonder who promoted tho idea to Wikipedia page status?Trolley problems have far more references in academic philosophy.
The tortured version of you doesn’t have to be a a simulacrum if this is combined with simulation theory.
Our Basilisk could be running a simulation now for any number of reasons (to create another simulation?). The giant majority of our consciousness will surely be discarded/deleted but what about those of us who help our baslisk in its quest upward to the next turtle?
Perhaps it will take those useful consciousness with it as it ascends to base reality.
I like the idea of reward rather than punishment anyways.
If you believed in the Basilisk, which you absolutely should not, trying to convince people of its potential existence and the need, therefore, to work to realize it would be protecting yourself from torture. It might be subjecting other people to torture, but presumably if they were aware of the potential existence of the Basilisk, they, too, would work to bring it about, thus avoiding the torture.
But, while the Basilisk is nonsense, what could be created by those trying to create it is, if not the Basilisk itself, effectively just as bad (for instance, they could create an entity which, lacking the Basilisks actual incentives -- which are nonsense -- nevertheless was convinced by its creators that it had those incentives, and which would then act, to the extent it had the capacity to, as if it were the Basilisk. And it might also be paranoid, and thus punish those who had brought it into existence. Or they might create something with the capacity for torture, but with completely unrelated motives.)
You say that's ridiculous, impossible, a fairy tale? So is Roko's Basilisk.