That's Eliezer. Less Wrong is his site. He also posts here sometimes. Friendly AI is his big idea.
No, there is not a large movement of people opposed to AI research on skynet grounds, Eliezer is the main advocate. I'm certainly willing to admit that Friendly AI is good and important but I think it's perfectly fine to reimplement and (recursively self-) improve Eurisko. There's always quantum immortality.
I'm glad Eliezer is out there thinking the thoughts that he is thinking. Maybe by the time we're closer to AI people will start really listening to him. For now, AI research continues to crawl along.
Thanks for the link. I read it, and it's very well written. What a great metaphor for conceptualising the speed of thought of a super-AI.
That said, the essay still absolutely reeks of unfounded near-paranoia. The hypothetical alien civilisation with the power to manipulate the output of stars - or even to build a simulation of such astonishing power - must surely know the very basics of risk management. And the superintelligent "earth" seemed completely friendly, all they wanted to do was talk. They must have reasoned that "hacking" the other civilisation with self-replicating nano-machines would destroy them. Why on earth would they do that? You'd expect that the one thing an AI would not want to do is destroy unique information.
Start with a great setup and an engaging story, then right at the end take a totally unjustified turn into pure worst-possible-case disaster fantasy. It really just proves nothing, except that Mr. EY does not believe risks can be managed. And really, if you're going to write the "precautionary principle" this large, we really just couldn't do anything, ever. It reminds me of those nuts arguing against the LHC.
"I'm glad Eliezer is out there thinking the thoughts that he is thinking."
The idea of hostile AI probably predates the notion of friendly AI! It's not a new thing and I'd argue it's always been on the table. It just has to be kept in perspective, like any risk scenario from new technology. EY takes it to the most ridiculous all-or-nothing paranoid extreme when we're not even close to the time when we need to seriously weigh up these things.
There would be fantastic, concrete benefits to having even slightly better AI. Trying to retard research because of some ill-justified nightmare "what if" scenario set in the far-off speculative future is ludicrous and, arguably, morally wrong.
I was trying to be funny. At its best my sense of humor involves statements where you are unclear if I am being sarcastic or serious, because the statements rest somewhere in between. Kind of like my username.
No, there is not a large movement of people opposed to AI research on skynet grounds, Eliezer is the main advocate. I'm certainly willing to admit that Friendly AI is good and important but I think it's perfectly fine to reimplement and (recursively self-) improve Eurisko. There's always quantum immortality.
Some selections from his writing:
http://lesswrong.com/lw/y4/three_worlds_collide_08/ (That Alien Message, my favorite short piece by Eliezer)
http://lesswrong.com/lw/y4/three_worlds_collide_08/ (Three Worlds Collide, novella length fiction)
http://lesswrong.com/lw/r5/the_quantum_physics_sequence/ (Quantum Physics Series)
I'm glad Eliezer is out there thinking the thoughts that he is thinking. Maybe by the time we're closer to AI people will start really listening to him. For now, AI research continues to crawl along.