

Ask HN: Gates, Musk, Hawking, and others have voiced concerns over AI. Why? - Billesper

What evidence is there that developing even a rudimentary general&#x2F;strong AI is remotely likely in the near future (within the next 10, 20, 50, 100 years)?<p>1) What is the state of general AI research? Are there any promising (i.e. having some measure of real progress) approaches currently?<p>2) Are the major problems&#x2F;roadblocks to creating a strong AI even known in any sort of nontrivial sense?<p>(To be clear, please leave discussion about &quot;if we have a strong A.I., it will&#x2F;could be a threat because....&quot; for a different thread. The question is about why the prospect of a general A.I. is being seriously considered in the first place)<p>From doing a bit of research, it seems like most contemporary AI methods are statistical&#x2F;optimization techniques in Machine Learning, etc. These can be extremely powerful tools, but as far as I can tell, they are applied to very specific problem instances. Is there any hope of more powerful&#x2F;general techniques emerging from this area?<p>It seems like a real solution is not simply a matter of time (within the next 100 years or so), but instead will probably take several major breakthroughs and insights coming from unknown places to achieve.
======
justintocci
...because they were asked. put yourself in their position. somebody who may
very well have the power to make you look like an idiot asks you a question.
do you insult them? no! do you give them the boring, "oh, i won't see it in my
lifetime", not if you're smart. if a reporter honors you with their time and
attention and you want them to do right by you you give them something they
can use.

and its completely harmless to warn people about something 100 years off.

------
mswen
When I interviewed John D. Cook for a Profile in Computational Imagination I
asked him a similar questions here is his answer:

John: One danger that I see is algorithms without a human fail-safe. So you
could have false positives, for example, in anti-terrorist algorithms. And
then there’s some twelve-year-old girl that’s arrested for being a terrorist
because of some set of coincidences that set off an algorithm, which is
ridiculous. Something more plausible would be more dangerous, right? I think
the danger could increase as the algorithms get better.

Mike: Because we start to trust them so much, because they’ve been right so
often?

John: Right. If an algorithm is right half the time, it’s easy to say well,
that was a false positive. If an algorithm is usually right--if it’s right
ninety-nine percent of the time--that makes it harder when you’re in the one
percent. And the cost of these false positives is not zero. If you’re falsely
accused of being a terrorist, it’s not as simple as just saying oh no, that’s
not me. Move along nothing to see here. It might take you months or years to
get your life back.

If you want to read more of our conversation it is available at
[http://computationalimagination.com/interview_johndcook.php](http://computationalimagination.com/interview_johndcook.php)

------
hdra
I am not a visionary like the people you mentioned, but one thing that I can
see go wrong with large-scale AI automation is not that the scenario of robot
uprising or something, but more of software bugs that we all are familiar
with.

There have been lots of cases where a bug in an automated trading system
causes millions of losses[0][1], if we have a larger system that manages
everything from the power grid, the gas lines, self-driving cars, your smart
home, to your own personal calendar, it's not hard to imagine the potential
damage if there is even a single bug.

Can be things like edge cases that nobody thought of, or simply unexpected
sensor reading caused by natural disaster, etc.

[0]([http://dougseven.com/2014/04/17/knightmare-a-devops-
cautiona...](http://dougseven.com/2014/04/17/knightmare-a-devops-cautionary-
tale/)) [1]([http://www.bloomberg.com/news/articles/2013-08-20/goldman-
sa...](http://www.bloomberg.com/news/articles/2013-08-20/goldman-says-
exchanges-working-to-resolve-options-order-mishap))

~~~
tixocloud
Yes, it's something we're thinking about in the insurance industry. Lots of
headaches and edge cases that come about with self-driving cars, smart homes,
etc. I also recall the Patriot Missile Software Problem [0]

[0]
[http://sydney.edu.au/engineering/it/~alum/patriot_bug.html](http://sydney.edu.au/engineering/it/~alum/patriot_bug.html)

------
shoo
To answer the title if not the content of your question:

* AI is something that potentially poses an existential risk to humanity, ie, it might wipe out our species.

* AI is a trendier topic to write about than other things: there's lots of possible pop culture references, the general public doesn't need to feel guilty about it

In comparison, something like climate change probably doesn't pose an
existential threat to our species: it may just merely wipe out a fraction
(20%? 80%?) of the human population over the next hundred years.

Edit: to be a bit more on topic, do you think there are very dangerous
technologies around these days that would have been very difficult to
anticipate / seemingly ludicrous to consider back 100 years ago? Would the
development of nuclear weapons in the cold war have seemed like a serious
concern or viable development in the 1850s?

------
ryan_j_naughton
This book by Nick Bostrom will help you find answers: Superintelligence:
Paths, Dangers, Strategies[1]

The author is the director of the Future of Humanity Institute at Oxford.

[1] [http://www.amazon.co.uk/Superintelligence-Dangers-
Strategies...](http://www.amazon.co.uk/Superintelligence-Dangers-Strategies-
Nick-Bostrom/dp/0199678111)

------
lgas
[https://en.wikipedia.org/wiki/Accelerating_change](https://en.wikipedia.org/wiki/Accelerating_change)

------
tanvirzafar
BUMP... following for getting answers....

