The thing that scares me about the Singularity is that it's such potent kool-aid for tech geeks. I talk to Singularitarians and there's this fire in their eyes like they've been reborn. They become so certain. They remind me of fundamentalists. If there's one thing more dangerous than a true believer it's a crowd of really smart true believers. I'm afraid, perhaps unnecessarily, of what might be done in the name of "ushering in the singularity".
I don't think we can know whether a singularity is possible until it's upon us, which could be never. I'm singularity agnostic. That doesn't mean we don't think hard about it. That doesn't mean we don't guard ourselves against its perceived dangers. It does mean that we keep our heads and stay skeptical and put out the fires that are already burning.
It's not for nothing that the Singularity is referred to as "The Rapture of the Geeks".
In all seriousness, though, the parallels between some forms of Rapture philosophy and some forms of Singularity philosophy are pretty strong. Uploading your mind into a machine is very similar to being taken up to heaven.
A good line, but still doesn't justify the level of certainty which many of these people have in future events turning out the way they think they will.
Back in the 90s, as a callow teenager, I happened to be lurking on a mailing list for transhumanists. One day someone on the list, well known to many of the others, died. "Fortunately", as the announcement to the list said, "his body was cryogenically frozen, so we'll all see him again after the singularity!" This was followed by a bunch of posts expressing confidence that this would indeed happen.
It was roughly at this point that I decided that these people may be somewhat disconnected from reality, and living in a fantasyland of what they'd like to be true -- while I admit it's possible that this guy will one day be revived, there's no goddamn reason to have any confidence in it happening.
I don't disagree. In fact, there's a whole branch of transhumanists (SIAI and fellow travelers) who may be even more pessimistic than you are, but don't view that as a reason to give up trying for a beneficial Singularity. Me, I'm concentrating on enjoying the ride, and if it turns out that humanity (including existing people) isn't wiped out by mid-century, that'll be a most pleasant surprise.
I'm afraid too--that a lot of smart people are flushing their time down the toilet. It seems like all things that actually bring us closer to singularity (if the word means anything at all) are being done by people who've never heard of it, or might as well have never heard of it.
I doubt you will find anyone on the tech/scientist community who is making major innovations and discoveries never heard of the idea of technological singularities. The idea has been around or a while.
I'm not sure what could be done that would be dangerous in the name of "ushering in the singularity." The worst plausible scenario is that we put money into research avenues that don't pan out. But that happens all the time anyways. The real worst case scenario is that someone gets too afraid of AI and goes all Sarah Connor. But that's not a likely scenario especially because most of the people who take the Singularity idea seriously are people who are generally optimistic about the long-term trends.
I'm not sure what could be done that would be dangerous in the name of "ushering in the singularity."
How about firing up a potentially runaway AI without a stable goal system? Releasing a virus to "improve" people in the releasers' preferred direction? The likelihood of these seems minor (and for the first, might not even been possible), but the consequences are so grave that they bear some thought and worry anyway.
Lots of confusion in the article, given a very complicated subject with widely divergent opinions and the attempt for journalistic non-bias.
One thing that clears up some confusion is the important point that the word "Singularity" means different things to different people. http://yudkowsky.net/singularity/schools
Here's one thing I've never gotten about singularity folks: why the assumption that human ego / personality will persist into post-human evolution?
It's easy enough to buy into a reduction of the basic singularity tenants: a) that we'll understand and be able to model the brain at some point, and b) that at that point evolution can proceed on, well, non-evolutionary time scales.
But it's always seemed to me like that step will have a tragic quality to it as well: that's the day humans become just animals. I don't know why a post-human intelligence would feel a need to preserve human individuals or the human concept of self any more than we feel the need to preserve gerbil personalities.
> It's easy enough to buy into a reduction of the basic singularity tenants ...
I don't even get that far. It may be the case that it's impossible to model the brain to a sufficient level of detail to manufacture conciousness. It may even be the case that our brains are doing something non-computable to produce consciousness, in which case we have to give up on Turing-equivalent machines entirely and move to something that more closely resembles our physical brains. At that point Singularity-scale intelligence depends on the creation of some big-brained biotech Frankenstein, which doesn't scale nearly as well, and seems different enough to require a new name. We just don't know.
I think that's partially the reason why some singularity folks focus on "friendliness" - they see it as quite possible that a superhuman intelligence would be a threat to us, for the reasons you state.
On one hand it's good to see that the subject get's into mainstream media.
On the other hand. The subject is dealt with quite simplistically. And the focus on Kurzweil is giving the story a very one sided and kind of geeky don't take too serious kind of angle.
I realize that even a newspaper like the NYT can't get too nerdy. But there are far more important stories to talk about when it comes to transhumanism even if the singularity doesn't happen. (I am personally skeptic)
I would though make one book advice that isn't following any specific school of thought. Hans Movarec: "Robot - Mere machine to transcend mind"
Its good to see Singularity getting more mainstream coverage. I think Singularity is inevitable, but I don't agree with Kurzweil's timeline or the fact that he is the face of Singularity. I understand that he wants to live forever but that doesn't mean he has to sell over-optimistic theories.
Maybe people who takes his over-optimistic predictions seriously will work harder to drive innovation?
That's probably why Kurzweil's son mentioned he is "less-weird" these days. Just imagine being mocked for evangelizing futurists ideas that were thought impossible are now surfacing into mainstream science. Of course, acquiring a vast amount of knowledge from this Singularity training is absolutely rich. It is only up to you whether or not you want to take the next step of tapping and developing untapped technologies - even if society likes it or not.
I have wondered, in a consciousness transfer scenario, if we can really ever ascertain whether the consciousness is actually transferred or, as I fear is the case, a new consciousness is created and the old one lost.
I assume someone has written a long treatise on the subject, but I have not considered the subject pressing enough to actually research it.
>If you can see the moments of now braided into time, the causal dependencies of future states on past states, the high-level pattern of synapses and the internal narrative as a computation within it - if you can viscerally dispel the classical hallucination of a little billiard ball that is you, and see your nows strung out in the river that never flows - then you can see that signing up for cryonics, being vitrified in liquid nitrogen when you die, and having your brain nanotechnologically reconstructed fifty years later, is actually less of a change than going to sleep, dreaming, and forgetting your dreams when you wake up.
It's one of those weird philosophical questions. The sad part is, there's no possible empirical answer--transferring my memories to a robot, the robot would believe it was me and any test you could possibly devise would confirm that the robot was me, but regarding my subjective experience inside this soggy bag of meat, who knows whether that continues inside the robot or just ends?
EDIT: Useful thought experiment from philosophy, devised during the time Ronald Reagan was still alive. Suppose someone told you that they would take you, erase your memories from your brain, upload Ronald Reagan's memories, and then torture the inhabitant of your body. How willing are you to say, "go ahead! You'll just be torturing Ronald Reagan, not me!"? Personally, I'm not so willing.
For minor surgeries these days, like when I had my wisdom teeth removed, a common combination is a local anesthetic plus a drug, midazolam, that temporarily prevents forming new memories. Patients are conscious and responsive during the operation, but don't remember being conscious; in retrospect they remember it as if they had been knocked out with general anesthetic. An odd thought experiment: if the local anesthesia hadn't worked well and I was actually feeling pain, do I care in retrospect? I sort of don't, I think, oddly enough. I don't remember it, so it doesn't really matter to me either way. I mean, I wouldn't want to see a video of myself experiencing pain, but so long as I neither remember it nor ever have to see it, somehow it doesn't matter to me a whole lot.
Well, yeah--after the fact, it doesn't really matter if you experienced terrible pain that you didn't remember. But by that argument it's fine to torture babies, since they won't remember it.
The question is whether, before the fact, you would go into minor surgery without anesthetic at all knowing perfectly well that you would forget all of the pain afterwards. Maybe you would if it was the best option, but I would wince in anticipation.
Yeah, I think I would wince at it, but I don't think I'd be that opposed. Somehow to me the idea of pain that I know up front I'll never remember isn't all that frightening a prospect, though I suppose I'd still avoid it given the option. It feels almost as if it's really just my body that'll experience pain, not "me", in the sense that I won't remember anything about it, it'll have no lingering traumatic effects, won't be an experience that'll do anything to shape my personality, etc. I'd definitely be much less apprehensive about it than if I knew I would remember it.
Hence the eternal philosophical problem with thought experiments (or "intuition pumps" as some people call them)--different people have different intuitions so maybe your thought experiment won't even make sense to them :)
Hans Moravec wrote a book, _Mind Children_, in which he provided a thought experiment where consciousness could be transfered even by the standards of a process-partisan (as opposed to pattern partisans, for whom every sufficiently like mind is really them). The idea is to do the transfer in real time, building a copy of a tiny portion of the brain, then running the copy-portion in parallel with the original-portion until it's producing the same output given the same input. At that point, you can reroute control through the copy (running at the same speed as the rest of your brain), and remove the brain matter that was copied. If you do this to pieces small enough that the loss of each piece would still leave you essentially you, the whole procedure can smoothly transfer your consciousness into a brain simulation while you're awake and feeling like yourself.
I assume by "consciousness" you mean self-awareness. A similar paradox is viewed within the context of quantum replication, i.e. an aboslute precise material copy of your body is created, maybe even more than one, so the question is, what happens to your "I", your self-awareness in that case.
One approach is to say "it doesn't matter" because self-awareness is illusory. It doesn't exist in the same sense as matter or fields. What does exist in your mind is memory of all state transitions ever happened in your brain. It's like a linked list going back, where each element is a state of the brain. That chain is mistaken for something that exists, whereas it's just a logically linked list of some data. You are nothing but information.
Allow me to expand upon your fear. Our brains are constantly changing in location and constituent atoms, only the pattern of their order and perhaps a hyper-dimensional line of its travel remain constant. So if you are worried about consciousness transferring from meat body to artificial body, it is more pressing to worry about your consciousness transferring from the body you had a moment ago to this one.
I enjoy the idea of Singularity very much, but if this article is representative of what this movement is all about (although it appears not to be), it seems flawed to me. I had the feeling Singularity is oriented toward biology, while I believe it should rather be approached by an anthropologic or mathematical point of view. Besides, Intelligence is often mentioned, yet, has it ever be defined correctly? My first guess would be that Intelligence is only another word for the complexity of a given system. Thus, human intelligence is only a representation of the complexity of our biological system. Considering this, it is indeed inevitable that universe will be saturated by intelligence, as universe is mathematically meant to keep complexifying as «time» passes on.
That said, I have not read much about the subject, and I'd really enjoy some serious text about this theory.
I found this article massively disappointing just because it drowns out the central, important question (is technological progress really accelerating and to what end?) in all this human-interest nonsense about the quaint fervency of futurists like Kurzweil. Fervent belief doesn't always mean false belief, and its not clear to me how the author has earned the right to this apparent condescension. The article gives people who don't care to think about the future consequences of present developments an easy out, which, when you think about it, is a good way of helping to ensure that the very people we're laughing at, and are slightly worried about, are left to follow their own agenda.
its not clear to me how the author has earned the right to this apparent condescension
Do we have to earn the right to be condescending nowadays?
If you ask me, condescension should be the default posture to adopt towards anyone making wild, overconfident assertions based on flimsy evidence. Now, since evidence about the future is awfully hard to come by...
Do we have to earn the right to be condescending nowadays?
To me that's kind of like asking do you have to climb the stairs in order to look down from the balcony, so, yeah, I think it needs to be earned, and I think you have earned it if you've already done the work necessary to determine that a conjecture is wild and based on flimsy evidence. The author of the article didn't do that work. I won't be seduced into conjecture about the ellipsis, but evidence about the future is not so much hard to find (it's everywhere in fact; it's the present state of things) as hard to interpret. Still, there are lots of smart people working on predicting one or another aspect of the future. Often they're wrong, so I share your skepticism, but skepticism is different from denial or dismissal.
I don't think we can know whether a singularity is possible until it's upon us, which could be never. I'm singularity agnostic. That doesn't mean we don't think hard about it. That doesn't mean we don't guard ourselves against its perceived dangers. It does mean that we keep our heads and stay skeptical and put out the fires that are already burning.