I have always thought that SF accounts of intelligent robots were far too hand-wavey.
I predict we'll never have machines with minds like humans. They will always exceed our abilities in certain tasks for which they've been programmed, but will never pass a long-term Turing test; for example, you'll never mistake a bot for a human remote programming colleague, with whom you chat and collaborate on features every day.
I take it as an axiom that the human mind cannot understand the human mind fully, for the same reason that a box cannot contain another equal sized box. The "singularity" requires a series of boxes, each of which contains a bigger one.
Out of curiosity, why would you take that as an axiom? Do human minds possess some qualities above physical reality? Are they unknowable in principle? If you answer yes, how did you arrive to that answer? Isn't there a physical experiment or demonstration that will change your mind?
I think the box analogy is flawed. We have certainly come a long way in understanding our own brains and bodies, even in the past couple of decades, and you could've said the same thing about boxes when people were first venturing into e.g. neuroscience.
The model doesn't have to be entirely contained in a human mind either, that's why we have a series of tools to aid us in modeling new tools, layer after layer. A human mind cannot contain a complete specification / working model of a modern computer either (if you think of all the layers involved), but modern computers do exist.
>> A human mind cannot contain a complete specification / working model of a modern computer either (if you think of all the layers involved), but modern computers do exist
This is an excellent counterargument. I still think minds are many of orders of magnitude more complex, but your example shows that my box analogy is flawed.
"human minds may possess some qualities above our current understanding of physical reality."
While you are free to believe that, you should be aware that you've chosen to leave science and logic behind, and expect to be treated as such. I do not mean that in the sense that you are therefore wrong, or that that is necessarily a bad thing; we must all make choices that leap beyond the light that science can cast, as the light is quite limited in some ways. But nevertheless, you have explicitly tossed anything that can be discussed in concrete terms overboard.
Usually I'm saying this in the context of whether we'll ever have FTL. Yes, you are free to believe that the next supercollider will produce some bizarre result that turns out to be something we can harness for FTL... but meanwhile, in the real world, the evidence gets ever tighter that it is impossible. If you wish to take a flight of fancy, fine, but please be aware that is what you are doing. And do not mistake it choosing to toss rationality overboard for rational argument; you can't have it both ways.
Yes, but it took a couple billion years for natural selection to do it, and it's a Texas sharpshooter fallacy to draw a target around a currently observed humanity that could have turned out much differently. Us hitting that same target in decades rather than millennia is a staggeringly different exercise.
