How to prevent Future ISIS from getting Future AI, or do we just shift from us trying to out-think them to our AI trying to out-think their AI?
If the answer to the latter is "resources" then we're back where we started. Whoever has the biggest AI wins.
The picture seems to be of many AI's all keeping each other in check, but that outcome seems less likely to result in the AI-UN and more like a primordial soup of competing AI's out of which a one-eyed AI will eventually emerge.
No matter how human-friendly an AI we build is, competition will be the final arbiter of whichever AI gains the most leverage. If a bad AI (more aggressive, more selfish, more willing to take shortcuts) beats a good AI (limits its actions to consider humanity), we're poked. If any level of AI can invent a more-competitive AI, we're poked. Once the cat's out of the bag, we have zero influence and our starting point and current intent become irrelevant.
If the answer to the latter is "resources" then we're back where we started. Whoever has the biggest AI wins.
The picture seems to be of many AI's all keeping each other in check, but that outcome seems less likely to result in the AI-UN and more like a primordial soup of competing AI's out of which a one-eyed AI will eventually emerge.
No matter how human-friendly an AI we build is, competition will be the final arbiter of whichever AI gains the most leverage. If a bad AI (more aggressive, more selfish, more willing to take shortcuts) beats a good AI (limits its actions to consider humanity), we're poked. If any level of AI can invent a more-competitive AI, we're poked. Once the cat's out of the bag, we have zero influence and our starting point and current intent become irrelevant.