
AMA: the OpenAI research team - sushirain
https://www.reddit.com/r/MachineLearning/comments/404r9m/ama_the_openai_research_team/
======
AndrewKemendo
I missed the AMA, but glad to see it here and it looks like there are some
team members monitoring this thread so I'll throw some questions out there.

I did my MS in AGI at the National Intelligence University with Ben Goertzel
as my outside research advisor. My thesis was to determine what were defense
implications of an AGI and who was making the best progress toward actually
building one.

Since then the DoD has started to take an interest in AGI, and in fact today,
during my one weekend a month drill at the Pentagon, I had a great long
conversation with Maj. General Seng [1] who is heading up efforts around
implementing ISR systems with more autonomous capabilities and exploring how
an AGI would be utilized in defense.

One of our big open questions is what's the "stack" for AGI, conceptually? I
didn't come to any conclusion on this and had to make a lot of assumptions to
close out the research. I would be curious to hear the OpenAI team's thoughts
on it.

Will you all be coming to AGI 16 this year in New York?

[1]
[http://www.af.mil/AboutUs/Biographies/Display/tabid/225/Arti...](http://www.af.mil/AboutUs/Biographies/Display/tabid/225/Article/108644/major-
general-jocelyn-m-seng.aspx)

~~~
eli_gottlieb
>Since then the DoD has started to take an interest in AGI, and in fact today,
during my one weekend a month drill at the Pentagon, I had a great long
conversation with Maj. General Seng [1] who is heading up efforts around
implementing ISR systems with more autonomous capabilities and exploring how
an AGI would be utilized in defense.

Isn't that basically the most obviously bad idea ever, so terribly stupid that
there have been several movies chronicling how _massively bad an idea_ it
actually is to build "AGI" for military goals?

~~~
TeMPOraL
This idea is cliché precisely because it's the most obvious course of action
for the military. Don't forget that it's DoD who sponsored the previous AI
summer.

Besides, if they don't do on it, the _enemy_ surely will!

~~~
eli_gottlieb
Because arms races and cold wars with we-all-die-grade weaponry are an
obviously excellent idea!

/s

------
TeMPOraL
Scott Alexander expressed some important concerns about the project[0].

Edited:

<del>So far they've managed to label them as "coming from the LessWrong
background" and subsequently dismiss via appeal to a strawman Paperclip
Maximizer. It doesn't give me much confidence in them.</del>

<ins>Nevermind. I didn't realize this comment was not made by an OpenAI
representative. Also, we could use a strikethrough formatting tag on HN.
'dang?</ins>

I hope they eventually address those points though.

[0] - [http://slatestarcodex.com/2015/12/17/should-ai-be-
open/](http://slatestarcodex.com/2015/12/17/should-ai-be-open/)

~~~
gdb
To be clear, the following comment is what we wrote on the subject:
[https://www.reddit.com/r/MachineLearning/comments/404r9m/ama...](https://www.reddit.com/r/MachineLearning/comments/404r9m/ama_the_openai_research_team/cysejas).
The subsequent replies are not affiliated with OpenAI.

~~~
TeMPOraL
Thanks, I missed that. I didn't notice the OpenAI flair.

I apologize for mistakenly assigning this comment to your group.

------
chegra
1\. Do you believe solving AI will come from a big company where the employees
solving it would have little ownership of the company?

2\. Currently, there is no solid test for Artificial General Intelligence. How
much of a priority is it to create one?

\-------------------------------------------------------

Answering my own questions:

1\. No, the incentive is just not there.

2\. Currently, this is my main goal related to Artificial General
Intelligence. One should know how far or near they are from creating
Artificial General Intelligence.

~~~
AndrewKemendo
_Currently, there is no solid test for Artificial General Intelligence. How
much of a priority is it to create one?_

There is of course no commonly agreed upon one, but in my research but the
best one I found was the Universal Anytime Intelligence Test [1]

[1]
[http://users.dsic.upv.es/proy/anynt/measuring.pdf](http://users.dsic.upv.es/proy/anynt/measuring.pdf)

------
meeper16
How does one monetize their own individual breakthroughs if they develop
something groundbreaking in AI and involved with OpenAI?

------
meeper16
Any chance at asking questions here? Whoops, guess I just did...

~~~
gdb
_waves_

~~~
jk4930
Do you have info materials about OpenAI I could use in AI events?

Will you work with 3rd parties to rise awareness and support, recruit talent
and external collaborators (e.g. research groups)?

~~~
gdb
No particular materials right now besides the website.

We expect to collaborate with many third parties over time. We're just getting
started, so not sure on the specifics yet!

~~~
jk4930
Okay. May I write you an email to establish the connection? I intended to
contact OpenAI later this month, but now that we're talking...

~~~
gdb
Feel free to get in touch: gdb@openai.com!

~~~
jk4930
I'll send you a mail tomorrow. Thanks.

------
zump
Is anyone else irked by the questions about AGI? It's a bunch of fluff put
forward by hacks. We're a long way off.

~~~
chriswarbo
I think there are legitimate research questions and concerns about AGI;
unfortunately there's also a _lot_ of fluff surrounding the area (e.g.
singularity stuff, doomsday scenarios, etc.).

The way I see it, there's only one conceptual barrier to cross between current
AI and "AGI-like" technology, which could be summed up as 'models which take
themselves into account'.

Whilst it's trivial to have software modify itself, we don't have good models
for predicting what those modifications will do (in a way which is more
computationally efficient than just running them). An analogy is how encoding
programs as ANNs lets us perform gradient descent, which we couldn't do if we
encoded them as e.g. string of Java code.

If we find a powerful software model which allows efficient prediction with
white-box self-references, then I think lots of progress will be made quite
quickly.

