
John Carmack: I’m going to work on artificial general intelligence - jbredeche
https://www.facebook.com/100006735798590/posts/2547632585471243/
======
Animats
This is encouraging. If you're going to work on artificial general
intelligence, a reasonable context in which to work on it is game NPCs. They
have to operate in a world, interact with others, survive, and accomplish
goals. Simulator technology is now good enough that you can do quite realistic
worlds. Imagine The Sims, with a lot more internal smarts and real physics, as
a base for work.

Robotics has the same issues, but you spend all your time fussing with the
mechanical machinery. Carmack is a game developer; he can easily connect
whatever he's doing to some kind of game engine.

(Back in the 1990s, I was headed in that direction, got stuck because physics
engines were no good, made some progress on physics engines, and sold off that
technology. Never got back to the AI part. I'd been headed in a direction we
now think is a dead end, anyway. I was trying to use adaptive model-based
control as a form of machine learning. You observe a black box's inputs and
outputs and try to predict the black box. The internal model has delays,
multipliers, integrators, and such. All of these have tuning parameters. You
try to guess at the internal model, tune it, see what it gets wrong, try some
permutations of the model, keep the winners, dump the losers, repeat. It turns
out that the road to machine learning is a huge number of dumb nodes, not a
small number of complicated ones. Oh well.)

~~~
the_af
> _This is encouraging. If you 're going to work on artificial general
> intelligence, a reasonable context in which to work on it is game NPCs._

I don't think so. Game NPCs don't need AI, which would be way overkill; they
just need to provide the illusion of agency. I think for general AI you need a
field where any other option else would be suboptimal or inadequate, but in
videogames general AI is the suboptimal option... more cost effective is to
just fake it!

~~~
seanwilson
> Game NPCs don't need AI

> ... more cost effective is to just fake it!

Many players complain in story heavy games that their choices have no
consequences to the story - this is largely because building stories with
meaningful branches isn't economically feasible.

A game that could make NPCs react to the what the player does dynamically
while also creating a cohesive story for the player to experience would be
absolutely groundbreaking in my opinion.

This is more in the realms of AI story generation but I haven't seen any work
on this that generates stories you would ever mistake as coming from a human
(please correct me if I'm wrong) so it would be amazing to see some progress
here.

~~~
nwallin
You're talking about different problems.

Story AI is basically having a writer sit down and writing a branching story
tree with writing the whole way. At best it's a manually coded directed
acyclic graph.

Tactical AI, ie having the bad guy soldiers move about the battlefield and
shooting back at you in a realistic manner is 100% about faking it. It's
better to despawn non-visible and badly placed enemies and spawn well placed
non-visible enemies than have some super smart AI relocate the badly placed
enemies into better locations. It's better to have simple mechanisms that lead
to difficult to understand behavior than complex behavior that leads to
instinctive behavior.

There was an amazing presentation at gdc maybe 3 years ago that perfectly
articulated this. The game was something about rockets chasing each other. I
wish I could find the link.

~~~
the_af
If you can find the link, please post it. I'm interested!

~~~
nwallin
Found it!

[https://youtu.be/1xWg54mdQos](https://youtu.be/1xWg54mdQos)

------
Jaruzel
Carmack's post in full:

 _Starting this week, I’m moving to a "Consulting CTO” position with Oculus.

I will still have a voice in the development work, but it will only be
consuming a modest slice of my time.

As for what I am going to be doing with the rest of my time: When I think back
over everything I have done across games, aerospace, and VR, I have always
felt that I had at least a vague “line of sight” to the solutions, even if
they were unconventional or unproven. I have sometimes wondered how I would
fare with a problem where the solution really isn’t in sight. I decided that I
should give it a try before I get too old.

I’m going to work on artificial general intelligence (AGI).

I think it is possible, enormously valuable, and that I have a non-negligible
chance of making a difference there, so by a Pascal’s Mugging sort of logic, I
should be working on it.

For the time being at least, I am going to be going about it “Victorian
Gentleman Scientist” style, pursuing my inquiries from home, and drafting my
son into the work.

Runner up for next project was cost effective nuclear fission reactors, which
wouldn’t have been as suitable for that style of work. _

\--

We're at 500 comments at the time of posting this, and no-ones pasted his post
in full to save us having to visit Facebook...

~~~
arcturus17
Thanks it didn’t so much save me as it enabled me, since I have it blocked.

~~~
saiya-jin
same here, can't access it at work, at its damn interesting topic for many of
us

------
pyentropy
Progress in AI is due to data and computational power advances. I wonder what
kind of advances are needed for AGI.

1\. Biological brains are non-differentiable spiking networks much more
complicated than backpropagated ANNs.

2\. Ion channels may or may not be affected by quantum effects.

3\. The search space is huge (but organisms aren't optimal and natural
selection is probably local search)

4\. If it took ~3.8b years to get from cells to humans, how do we fast-
forward:

* brain mapping (replicating the biological "architecture")

* gene editing on animal models to build tissues and/or brains that can be interfaced (and if such interface could exist how do we prevent someone from trying to use human slaves as computers? Using which tissues for computation is torture?)

* simulation with computational models outside of ECT (quantum computers or some new physics phenomenon)

Note: those 3.8b years are from a cell to human. We haven't built anything
remotely similar to a cell. And I'm not claiming that an AGI system will need
cells or spiking nets, most likely a lot of those are redundant. But the
entropy and complexity of biological systems is huge and even rodents can
outperform state of the art models at general tasks.

IMHO, the quickest path to AGI would be to focus on climate change and making
academia more appealing.

~~~
catalogia
From what I understand, quantum effects being essential to the process is a
fringe belief. Penrose is probably the most famous 'serious person' (sorry
Deepak Chopra) to espouse the idea, but I'm inclined to believe that might be
a Linus Pauling/Vitamin C sort of scenario. Penrose started from the
perspective of believing there must be quantum effects, then began fishing for
physical evidence of it.

~~~
pvarangot
Yeah, "quantum mechanics and cognition are very complex and therefore
equivalent", sorry I don't know who to attribute the quote to.

~~~
pgcj_poster
I think you're recalling the end of this comic[1], which was on the front page
of HN a couple weeks ago. So the quote is probably attributable to either
Scott Aaronson or Zach Weinersmith.

[1] [https://www.smbc-comics.com/comic/the-talk-3](https://www.smbc-
comics.com/comic/the-talk-3)

~~~
pvarangot
Yes! Thanks

------
carlosdp
This doesn't surprise me at all. He went on a week long cabin-in-the-middle-
of-nowhere trip about a year ago to dive in to AI (that's all this guy needs
to become pretty damn proficient). (edit: I'm not claiming he's a field expert
in a week guys, just that he can probably learn the basics pretty fast,
especially given ML tech shares many base maths with graphics)

As recent as his last Oculus Connect keynote, he extolled his frustration with
having to do the sort of "managing up" of constantly having to convince others
of a technical path he sees as critical. He's clearly the type that is
happiest when he's deep in a technical problem rather than bureaucracy, and he
likes moving fast.

On top of that, he likes sharing with the community with talks and such, and
ever since going under the FB umbrella, he's had to clear everything he says
in public with Facebook PR, which clearly annoyed him.

He's hungry for a new hard challenge. VR isn't really it right now since it's
more hardware-bound by the need for hard-core optical research than software
right now. With the Quest, he (in my opinion) solidified VR's path to mobile
standalones. It's time to try his hand at another magic trick while he's on
his game.

John's the very definition of a world-class, tried and true
engineer/scientist. He's shown time and time again the ability to dive into a
field and become an expert very quickly (he went from making video games to
literally building space rockets for a good bit before inventing the modern VR
field with Palmer).

If there's anyone I'd trust to both be able to dive into AGI quickly and do it
the right(tm) way, it's John Carmack.

~~~
mochomocha
> He went on a week long cabin-in-the-middle-of-nowhere trip about a year ago
> to dive in to AI (that's all this guy needs to become pretty damn
> proficient).

You must be joking, right? I'm as much of a Carmack fan as anyone here, but
overstating the skills of one personal hero does no good to anyone.

~~~
echelon
What a weird future it would be if Carmack turns out to be the one to figure
out the critical path and get it all working. An entire field of brilliant
researchers be damned.

History books (for as long as those continue to exist) would cite AGI as his
major contribution to society, and his name would be more renowned than Edison
or Tesla. An Einstein. None of his other contributions will matter, as the
machines will replace it all.

Just daydreaming, though.

~~~
derefr
I wonder how many historical figures went through the same thing? Who do we
know for their contributions to field X, when 99% of their life was spent
contributing to field Y?

~~~
DavidSJ
Isaac Newton spent most of his life pursuing alchemy and obscure theological
ideas, and found it a real nuisance whenever anyone pestered him about math or
physics.

~~~
keanzu
Isaac Newton is considered by some to be the greatest mathematician of all
time and is regarded as the "Father of Calculus".

"Taking mathematics from the beginning of the world to the time when Newton
lived, what he has done is much the better part." \- Gottfried Leibniz

[http://www.fabpedigree.com/james/mathmen.htm](http://www.fabpedigree.com/james/mathmen.htm)

~~~
knowThySelfx
"Researchers in England may have finally settled the centuries-old debate over
who gets credit for the creation of calculus.

For years, English scientist Isaac Newton and German philosopher Gottfried
Leibniz both claimed credit for inventing the mathematical system sometime
around the end of the seventeenth century.

Now, a team from the universities of Manchester and Exeter says it knows where
the true credit lies — and it's with someone else completely.

The "Kerala school," a little-known group of scholars and mathematicians in
fourteenth century India, identified the "infinite series" — one of the basic
components of calculus — around 1350."

[https://www.cbc.ca/news/technology/calculus-created-in-
india...](https://www.cbc.ca/news/technology/calculus-created-in-
india-250-years-before-newton-study-1.632433)

[https://en.wikipedia.org/wiki/Kerala_School_of_Astronomy_and...](https://en.wikipedia.org/wiki/Kerala_School_of_Astronomy_and_Mathematics)

~~~
mkl
That story's by non-experts and sounds like it's based on a press release.
There were basic components of calculus well before that too:
[https://en.wikipedia.org/wiki/History_of_calculus](https://en.wikipedia.org/wiki/History_of_calculus)

However, calculus proper (derivatives and integrals of general functions, and
the connections between them) did not exist until Newton and Leibniz. Other
mathematicians made important steps towards it earlier in the 1600s, and if
Newton and Leibniz had not existed, others would have figured it out around
the same time.

~~~
knowThySelfx
[https://insa.nic.in/writereaddata/UpLoadedFiles/IJHS/Vol47_4...](https://insa.nic.in/writereaddata/UpLoadedFiles/IJHS/Vol47_4_11_PPDivakaran.pdf)

[https://en.wikipedia.org/wiki/Aryabhata](https://en.wikipedia.org/wiki/Aryabhata)

~~~
mkl
These are interesting articles that seem to agree with what I said. The first
one defines calculus in a much more limited way, and refers to some of the
earlier basic components I mentioned.

I'm not a historian, but a few months ago I spent some time analysing one of
Fibonacci's trigonometric tables (chords, not sine or sine-differences).
Aryabhata's sine-differences were much earlier.

------
xamuel
I've been dabbling in AGI and it seems like the field has a lot of low-hanging
fruit. I'll bet Carmack can offer some significant contributions.

I'll take an opportunity to plug a paper I recently published on comparing
relative intelligence. The punchline will illuminate the low-hangingness of
the fruit in this field.

Suppose X and Y are AGIs and you want to know which is more intelligent. For
any interactive reward-giving environment E, you could place X into E and see
how much reward X gets; likewise for Y. If X gets more reward, you can
consider that as evidence of X being more intelligent. But there are many
environments, and X might do better in some, Y in others. How can you combine
those pieces of evidence into a final judgment?

The epiphany I had (obvious in hindsight) is that the above situation is
actually an _election_ in disguise. The voters are interactive reward-giving
environments, voting (via their rewards) in an intelligence contest between
different AGIs. This allows us to import centuries of research on voting and
elections! In particular, by using theorems about elections published in the
1970s, I was able to provide an elegant notion of relative intelligence.

The notion I provided is elegant enough that some theorems can even be proved
with it, for example, formalizations of the idea that "higher-intelligence
team-members make higher-intelligence teams". Which emphasizes the low-fruit-
hanginess of the field: as obvious as that idea seems, apparently no-one was
able to prove it with previous formal intelligence measures, probably because
those previous intelligence measures were too complicated to reason about!

Here's the paper:
[https://philpapers.org/archive/ALEIVU.pdf](https://philpapers.org/archive/ALEIVU.pdf)

~~~
solipsism
_For any interactive reward-giving environment E, you could place X into E and
see how much reward X gets; likewise for Y. If X gets more reward, you can
consider that as evidence of X being more intelligent._

That's an odd definition of intelligence. By that definition, a bird is more
"intelligent" than a human at the task of opening a nut. Seems like "fitness"
would be a much more appropriate term.

It seems especially strange to consider this work in the field of _general_
intelligence. Nothing about what you just described is _general_. By this
definition, a chess bot is much more intelligent than the average person. I
don't think we'd say a chess bot has general intelligence.

~~~
NateEag
I think the idea is that you would have many environments and each one is a
voter.

If an AGI candidate wins the board game vote but no others (the hunter-
gathering vote, the walking and crawling vote, the "publish or perish" vote,
etc), it will be trounced by something that is not quite so good at board
games but is more flexible and adaptable - i.e., general.

I'm an AGI skeptic myself, but I do think that's the best attempt at a
formalism for ranking AGI attempts that I've seen so far (disclaimer: as a
skeptic I haven't exactly done a deep dive into the field).

------
phillco
This is a very artfully written statement. It avoids the most more devastating
headline of “John Carmack Stepping Down”. (Yes, going part-time isn’t the same
as leaving entirely, but still).

~~~
nothis
The real headline is that VR is finally confirmed to have hit a major
roadblock. If John Carmack gets frustrated with its progress, there's
something up.

I've long thought that the issue with VR is a conceptual one, not a technical
one and maybe that frustration comes from there. "Running forward" is an
unsolved problem in room scale VR. For a seated experience, you're basically
back to a neat display gimmick + accurate hand tracking.

Any real solutions need, on the one side, real-world physical constructions
(think running threadmills) that soon hit holodeck-level limitations and, on
the other, software that actually benefits from the _real_ technology VR
brings to interactive media: super accurate hand- and head-tracking. The first
gets impractical/impossible soon, the second limits development to a few niche
genres: Shooting ranges, cockpit sims, dance/party games and some vague
"experiences" where the actual tech is pretty much ignored and you just say
"but it feels so immersive!" (honestly, it _does_ work for horror games!).
It's basically motion controls 2.0.

The only place I could see the technology shine is, oddly enough, AR. It has
way less mainstream hype to it but it makes much more sense because you
actually benefit from the tracking of your real-world movement: You're still a
part of it! The holo-lens demos that pop up on youtube might seem clumsy, but
I can totally see a use case for replacing physical monitors with arbitrarily
sized and positioned displays you can virtually move in any office space.
There's rumors of Apple working with Valve on AR tech. If there's any
technology that could follow the smart phone, AR is my bet. I'm honestly
surprised Carmack didn't move in that direction rather than deciding to become
a general AI guru.

~~~
nmfisher
> The real headline is that VR is finally confirmed to have hit a major
> roadblock. If John Carmack gets frustrated with its progress, there's
> something up.

This was the (loud) subtext for me, too.

Carmack doesn't strike me as the type of person to walk away from a problem
lightly. Given how many problems remain unsolved when it comes to VR, I wonder
if he's just admitted that it will never be the endgame he wanted it to be.

Cash-wish, he's obviously sitting pretty. Better off spending your remaining
years working on something you can make a meaningful contribution towards.

------
nwsm
What a joke. Carmack is going to sit at home and solve what teams of
scientists can't do in decades.

I'm complaining less about Carmack wanting to spend his time doing this and
more about the comments here acting like he is some 10000x research scientist.

~~~
drcode
There are few AI researchers (maybe around 5 or so) that could credibly claim
technical accomplishments of any sort in the same ballpark as Carmack's.

People with this level of track record should not be underestimated, there
aren't many of them out there... They matter.

~~~
Barrin92
Carmack has had a lot of commercial success, which is great and I value his
creativity in the space but working on a research question that is about ten
paradigm shifts away is a different task than putting in hard labour to build
games.

~~~
taurath
And maybe the output is he helps bring about one of those ten paradigm shifts,
which would be a wonderful success. He didn’t say “solve” it, he said work on
it.

------
yaseer
John Carmack seems like a guy that would've made fundamental contributions to
science, had he been born a century before.

Computer science now occupies the place physics once did, in its impact on
moving the world forward.

Best of luck to him, I look forward to seeing what he produces!

~~~
glofish
why would being born in the current century preclude one from making
fundamental contributions?

You are making a common mistake of assuming that just because someone is good
at something like programing computers the same skill would translate
identically to a completely different domain.

If anything he is lucky to have been born in an era where his skill of
programming computers could be put to use - otherwise his talents may have
gone to waste, he may have ended up toiling fields his talent untapped and
undiscovered, like that of millions before him.

~~~
teawrecks
He's "good at programming" like Galileo was "good with telescopes." Computers,
telescopes, hammers, they're all just tools, and Carmack has proven himself as
far more than just a handyman.

He's the kind of person where, if you show him what you're working on and he
doesn't understand it, you probably need to go back to the drawing board.

~~~
b3kart
Cult of personality much? Don't get me wrong, Carmack is one of a kind. But
seriously, "if Carmack doesn't understand your idea => your idea is hopeless"
\-- this can not be healthy.

EDIT: I'll elaborate a bit. In my experience in both industry and academia
I've witnessed numerous occasions when brilliant people would get things
wrong, ignore a brilliant idea, follow a hopeless research direction, etc.
etc. Authority matters, but _nobody_ is flawless.

------
hans1729
Heh, from the comments:

> _Congratulations on the new project, and may your hubris not doom us all._

~~~
kotrunga
That's the very comment that stood out the most to me as well...

~~~
dcwca
Good Doom pun

------
leesec
Wow, there are a lot of people in this thread arguing whether or not John
Carmack has the right skills to help AGI, or about the specifics of his
knowledge.

Do you all realize you're arguing about nothing?

Good for him for doing something he seems excited about. Maybe we should all
stop gossiping and go do something we're excited about too.

~~~
voxl
What is actually being argued is the cult worship of a particular engineer. I
am against cult worship, so find the fact that Carmack is entering this space
simply not news worthy. Yet here it is.

Do I wish him the best of luck and hope he cracks the problem? Of course, all
the same I would wish that of an upstart PhD student. Yet, the announcements
of a brilliant PhD student attending a university to work on AGI is somehow
not on hacker news.

This is cult worship of the personality Carmack has amassed, perhaps
completely accidentally. When Carmack actually achieves something interesting
let us discuss it then, not the mere announcement that he will try, as if that
means anything. Read: it doesn't.

~~~
yitchelle
The difference between John and the PhD student is their history of Getting
Stuff Done. I imagine that the PhD student list of Stuff Done is not as
accomplished as John's.

Also the reason why guys like John and Elon are much admired as engineers.
They Get Stuff Done. The chance of something remarkable occurring with these
folks are a lot higher than it is with the PhD student.

~~~
josh2600
Having seen academics fail to convert their knowledge into production grade
systems time and again, I can only say that there are light years of
difference between theory and practice.

------
ekianjo
Other way to read this: big corporations are slowing down on VR. The market
has not taken off as rapidly as they expected so we will see more moves away
from heavy investment in VR.

~~~
baddox
Carmack never struck me as the type to chase the money from fad tech to fad
tech. On the contrary, he seems keen to chase his own interests, and by now I
suspect he is financially secure enough to do so.

~~~
ekianjo
I did not mention Carmack in my comment. Corporations. Facebook probably
decided that having Carmack working on a minuscule market that is barely
growing (look at headset sales per year, it's virtually nothing and there is
no "acceleration" in sight either) was a waste of his talent.

~~~
cma
He’s not working on AGI with Facebook.

------
flipgimble
Like many others posting here I’ve followed John’s work frOm the mid 90’s so
here is my take:

* he has an exceptional quality of cutting through the bullshit and shipping practical software, which is arguably what the vague and uncertain field of AGI needs. If you listen to AGI conference talks in recent years they are focused on aspirational single-idea academic frameworks that haven’t produced results in decades.

* he is still connected to Facebook with billions in resources, and a world class ML team with Yann LeCun at the head.

* his personal brand has been strong enough to have world class developers flock to Oculus. When he is ready to expand his “Victorian Gentleman” alchemy lab with a team, I have no doubt it would be a field-changing think tank.

My hope is that he continues to be open and brutally honest with his progres
and learnings as he’s been with game development and rocketry.

------
jacquesm
AGI is not an engineering problem but a research problem. John Carmack is good
at putting stuff together but how good he is at coming up with novel concepts
for an open research problem remains to be seen. Even the rocketry example
that is hailed here as a success mostly wasn't. That doesn't make me happy, it
would have been far nicer if Armadillo had succeeded, more competition in that
space is better. But for all the work done it was more of an advanced hobby
project along the lines of those guys in the Nordics than something that moved
the needle scientifically.

~~~
dkural
Please take this as nothing more than my subjective opinion: I believe that
humans don't have GI but HI - we have a "world-view" that is very
idiosyncratic to being human, which is essentially heuristics all the way down
- in other words, I don't believe there is a magical novel concept that
explains HI, but that it is a collection of party tricks that evolved over
time, i.e. hacky engineered system.

~~~
solipsism
I don't see why that's not GI. That HI enables us to do incredible things, far
beyond what we evolved to do (we evolved to hunt and fuck and exist in small
groups, not to do quantum mechanics).

~~~
Fricken
It begs the question, though: is the development of AGI contingent on some
magical breakthrough, or is it a matter of endless tinkering and cobbling
until we've got something that works?

~~~
tachyonbeam
It's probably both. There's going to need to be a series of breakthroughs, but
there's also going to be a lot of engineering required. I believe that AGI
won't happen suddenly. It will require putting a lot of pieces together. We'll
get systems that have an incrementally better and better model of their
environment.

Personally, I find it kind of offensive how scientifically-minded people
believe they have the monopoly on generating ideas and making the world
progress. That's clearly not true. Deep learning research wouldn't be where it
is without GPUs, and compilers like TensorFlow and PyTorch. Engineers are huge
drivers of change, they make things happen.

Deep learning research already involves a lot of trial and error. Tinkering
and cobbling as you put it. People can't really tell, just writing things on a
whiteboard, whether it's going to work or not. There might be some
mathematical intuition, but a lot of it is throwing things at the wall and
seeing what sticks, empirical testing. Some very high percentage of the
research being done is basically thrown away.

Furthermore, I would personally say, as someone who works in deep learning,
that we're collectively getting a little myopic. We finally got neural
networks to do cool things. People are very excited, but they're forgetting
neural nets are not the only kind of machine learning technique around. It
actually works really poorly for some things. We're just largely disregarding
every other approach because we have this one cool new toy. So, I don't know,
maybe the next huge breakthrough will come from someone who's a deep learning
outsider, and who's not completely locked into this paradigm and unwilling to
look at anything else.

------
cf
I expect John Carmack to follow a trajectory of someone like David Ha who with
little previous background started to write very creative and thought
provoking papers
([https://scholar.google.com/citations?user=J1j92GsxVUMC&hl=en](https://scholar.google.com/citations?user=J1j92GsxVUMC&hl=en))

It won't be AGI by most definitions but I bet it'll be pretty cool and I'm
happy to have that.

~~~
0-_-0
Dammit, I had the hypernetworks idea recently but he already did it 4 years
ago! Nothing new under the sun...

~~~
cf
Well that just means you capable of having good ideas. Just keep at it and I'm
sure you'll do something great.

------
foobiekr
I’ve worked with legends in a specific space that is less consumer-y than
games and graphics and less weirdly-desperate attention-seeking than ML and so
the people therein are every bit Carmack-level but less visible. As they have
aged out and wealthy’d out of working time, almost all of them have chosen to
retire.

These are extremely high performing individuals who have made global impact.
Shutting down for people like this is very hard and 100% of them have sent out
mails much like Carmack’s Facebook post when the end came. Even the style and
verbiage are similar.

None of them made a dent in their tinkering-phase project and moved on to
normal above-average low-engagement hobbies. They are done.

I read his FB post as a pretty standard retirement announcement as a result. I
think he’s telling us he is done.

~~~
mav3rick
You cant project your colleagues on a third person.

~~~
foobiekr
You can observe common properties of the way people at similar skill levels
behaved however.

~~~
mav3rick
How do I know op's standards are the same as Carmack's ? What I consider great
maybe average after all ?.

Same could have been said when Carmack said he wanted to work on VR. "it's a
retirement letter"

------
bane
I'm surprised nobody has made the connection with the announcement of Horizon.
Carmack joined Oculus in order to build the next generation of 3d Virtual
worlds. It's done, other than some polish, which means he's not needed
anymore.

However Horizon turns out (I'm bearish on it tbh), Carmack has had his shot to
build the digital future, it's now turned out how it has, and there's not much
flexibility left for him to maneuver, it's time to move on.

I think AGI is going to turn out like his shot at Rocketry, big and complex
enough that he'll find his niche and contribute, but not make any significant
breakthroughs.

------
vecplane
I don't understand how he could contribute to the field of AGI research from
home, by himself, and maybe with his son. It's the kind of problem that
requires incredible amounts of data, hardware, and theory to make any
progress.

Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind
or OpenAI?

~~~
nickjj
John Carmack is practically a machine.

He's openly talked about his work ethic in a bunch of places. He's the type of
guy who after a life time of coding calculated he's 100% efficient up until 13
hour work days and then he drops off[0]. Although he did mention working those
long hours is often best working on multiple things instead of 1 topic but
maybe with AGI there's a bunch of different avenues to explore.

[0]:
[https://www.youtube.com/watch?v=udlMSe5-zP8&t=4773](https://www.youtube.com/watch?v=udlMSe5-zP8&t=4773)

~~~
Rapzid
He must have some very good advice on getting a good nights sleep. Makes all
the difference IMHO.

~~~
archagon
He did tweet that unlike many engineers, he can't be productive unless he gets
a full 8 hours of sleep (IIRC).

------
nwsm
Yann LeCun's [0] comment on the post:

Welcome to the club, John.

A word of warning though: There is no such thing as AGI. Reaching human-level
AI is a good goal. But human intelligence is very, very specialized.

[0]
[https://en.wikipedia.org/wiki/Yann_LeCun?fbclid=IwAR2e9mzCqS...](https://en.wikipedia.org/wiki/Yann_LeCun?fbclid=IwAR2e9mzCqS48-OGTH65yiVwO27rhjGC065GxudLetpOOAAvuIN6kcGC5EUM)

~~~
drcode
LeCun is more accomplished and smarter than I will ever be, but his thoughts
on the term 'AGI' just seem like dumb pedantry regarding word definitions to
me.

~~~
username90
Sounds like it is more like an wishful thinking from a person who spent his
life mastering deep nets than a nitpick. Like "Sure deep nets can never be an
AGI, but Humans are not AGI's so we can reach human level performance with
deep nets, it is not a dead end! IT IS NOT A DEAD END!".

I think the saying "Science Advances One Funeral at a Time" applies here.

------
drefanzor
If anyone can initiate the singularity, it's John Carmack.

------
tus88
What this title should have been: John Carmack gives up on VR.

~~~
Keyframe
Following up on “John Carmack gives up on aerospace” and hopefully not
followed up by “John Carmack gives up on AI”.

I like the guy as much as most, but so far it seems like he has been wandering
around. He’s had much success in the early days of 3D video games and that’s
about it. A guy of his calibre would probably make a good impact if he joined
one of the expert teams like DeepMind. No matter how smart he is, AI today is
a completely different ballgame then what he was part of so far. I hope I’m
wrong, but I don’t see him making any sort of breakthrough on his own, with
his son. Maybe he wants to spend more time with his family, which is great, or
he drank koolaid about his own legend. Odds are against him, heavily so. Good
luck, in any case.

------
narrator
Humans optimize for activating the opioid receptor. These receptors are
distributed all over the brain and tied into all sorts of subtle neural
networks. That's why opioid addicts don't do much when they're high. As far as
the entire structure of the brain is concerned, an opioid addict's brain is
done optimizing and the fitness function is pegged at 1.

I think an AGI will end up being like an AI that plays the Sims except we're
the Sims and it's optimizing for our happiness probably by remotely monitoring
our opioid receptor activation and some parameters of general health.

~~~
chillacy
I must be doing a poor job at being a human, given that I have passed the
opportunity to activate my opioid receptors several times in my life so far
(leftovers from surgeries).

~~~
homonculus1
Perhaps the thought of future addiction fails to activate your opioid
receptors in the moment of your decision.

------
avl999
Half of you folks here are like teenage girls in highschool gossiping with
each other about Kristen dumping Drew and instead trying to date Paul.

------
sebsito
First of all we should at least have a common definition what intelligence
even is.

Even then I'm not sure we'd know what General Intelligence would be because
all we know is Human Intelligence or maybe lower level Animal Intelligences
where the problem solving mechanism seems to depend on biological body and
it's form.

Humans navigate the world with automatic impulses which we evolved over time
to deal with way too much signals from the environment so we can filter and
react only to those important.

We can then use consciousness to slowly map new impulses as the environment
changes and go back to autopilot for most of the time.

What if our intelligence isn't general but it's just enough to navigate the
world we can perceive with out senses? What if we'll never be able to
understand e.g. the quantum theory (or at least the part of a world experience
which we call this way)? If there's is superset of out intelligence or
different sets of intelligences which we just don't undestand?

We think that our problem solving can take on any problem but maybe we're only
taking on the problems we can take on, limited to our perception of reality
which can be limited?

So I think instead of calling it AGI the name should be more like Artificial
Human-like Intelligence.

------
laxatives
Is he doing this under the Facebook umbrella? Or departing? Or more-or-less
retiring from regular obligations entirely?

~~~
Someone1234
I think semi-retiring. He's going to be "consulting [on Oculus]" while:

> I am going to be going about it “Victorian Gentleman Scientist” style,
> pursuing my inquiries from home, and drafting my son into the work.

Which to me reads like part time work on Oculus, part time work on this AGI
project. If it is with Facebook it isn't at all clear from the post (plus I'd
assume it would be accompanied by marketing copy in that situation).

------
thrower123
I was wondering why his tweeting about graphics and VR went to 0.

We've gone back into the part of the cycle where VR is an odd curiousity
again, haven't we?

~~~
randomidiot666
Unfortunately VR turned out to be an intensely nauseating puke fest for a lot
of people.

~~~
thrower123
Same as the last go round. In ten years, a new generation will think that this
time, they know how to do it right. Maybe the hardware will even. have caught
up.

~~~
K0SM0S
I'm still convinced we'll have "good enough" barebones yet spatially aware AR
first. It's just less intensive on the front end side, which is the limiting
factor now as I understand it.

VR is a whole other thing, and I think the "uncanny valley" stretches quite
far; ie you need quality really close to what we see in movies to pass the
acceptable threshold beyond a few hours of novelty.

~~~
randomidiot666
VR sickness still applies, even if you have perfectly realistic rendering,
infinite frame rate, infinite resolution, perfect tracking, and zero latency.
It's caused by a discrepancy between the visual and vestibular systems. That
problem cannot be solved by higher quality rendering.

~~~
jobigoud
It's only a problem if you are moving in the virtual world using a different
method than what your body is using in the real world. If the VR world is a
simple 1:1 mapping and you can walk around in it as you do in the physical
space, there is no vection.

~~~
randomidiot666
Yes that's right. But that's a limited scenario. Not the true unbounded VR
experience that we really wanted.

~~~
K0SM0S
I was thinking more along the lines of neural connections, and you probably
sitting in some comfy chair physically while roaming in your mind.

You know, however sci-fi "solves" VR. I agree with your sentiment, current
"let's put displays on a headset" approach seems way too old school to me. A
20th century (flawed) solution to a true 21st century problem (much more than
elec. tech it's bio-extension, very rich cognitive interface).

------
phyzome
Ah yes, AGI: The idea that eats smart people.
[https://idlewords.com/talks/superintelligence.htm](https://idlewords.com/talks/superintelligence.htm)

~~~
marvin
Working towards more general AI technology is not what the author of this is
criticizing. They're criticizing focusing on super-human AI rather than
ethical problems of current AI. "The idea that eats smart people" is just a
meme at this point, normally quoted outside its context.

But even so, history will have to judge whether the author's statements were
true regarding the risk of super-human AI. Or whether a lot of _quite_ smart
people weren't smart _enough_ to realize that there was a real likelihood of
this being possible to achieve faster most thought.

Also, using ridicule as a rhetorical technique isn't the most sound type of
reasoning, regarding the author of your link ;)

------
0xdeadbeefbabe
From the guy who brought us Commander Keen.

Applying intelligence to artificial intelligence has happened before
[https://en.wikipedia.org/wiki/Shakey_the_robot](https://en.wikipedia.org/wiki/Shakey_the_robot).
One of the contributors to Shakey was Alfred Brain.

------
dharma1
Why the announcement? Is it to protect Oculus/FB so it doesn't look like he's
bailing out of VR?

Couldn't he just have said "I'm taking some time off" and then make an
announcement when there's something to announce, ie. "So here's some progress
I've made on AGI"

------
eggy
Great, AGI from the guy who brought DOOM to many. Do we ever learn?

~~~
chrisco255
DOOM 4: AGI confirmed.

~~~
jmts
Hell on Earth.

~~~
lgl
An Oculus (by FACEBOOK) exclusive

------
binarymax
Does anyone have a non-facebook version of this story?

~~~
curiousgal
_Starting this week, I’m moving to a "Consulting CTO” position with Oculus.

I will still have a voice in the development work, but it will only be
consuming a modest slice of my time.

As for what I am going to be doing with the rest of my time: When I think back
over everything I have done across games, aerospace, and VR, I have always
felt that I had at least a vague “line of sight” to the solutions, even if
they were unconventional or unproven. I have sometimes wondered how I would
fare with a problem where the solution really isn’t in sight. I decided that I
should give it a try before I get too old.

I’m going to work on artificial general intelligence (AGI).

I think it is possible, enormously valuable, and that I have a non-negligible
chance of making a difference there, so by a Pascal’s Mugging sort of logic, I
should be working on it.

For the time being at least, I am going to be going about it “Victorian
Gentleman Scientist” style, pursuing my inquiries from home, and drafting my
son into the work.

Runner up for next project was cost effective nuclear fission reactors, which
wouldn’t have been as suitable for that style of work._

~~~
hedvig
Fusion John, we need you in fusion research. It's only 10 years away

------
daenz
Roko's basilisk[0] will remember this.

0\.
[https://rationalwiki.org/wiki/Roko's_basilisk](https://rationalwiki.org/wiki/Roko's_basilisk)

~~~
mactyler
Carmack might just be a basilisker after all!

------
reilly3000
Arguably he got started on this years ago. The story goes that Quake 3's AI
was so sophisticated, that when left alone for 4 years on a bot vs bot server,
the units learned pacifism as a self-preservation strategy.
[https://www.forbes.com/sites/erikkain/2013/07/02/quake-
iii-a...](https://www.forbes.com/sites/erikkain/2013/07/02/quake-iii-arena-
bots-reportedly-stop-fighting-after-4-year-match/#6d05c58a6233)

~~~
hombre_fatal
Seems like this would be easy to actually test instead of rely on a single
anecdote. And the simplest explanation is a bug though I don't believe the
story at all. There are probably multi-year-uptime bot servers in the wild
right now.

Reminds me of when I was new to Quake 3 and found an amazing server: it was
always full of players and full of non-stop action. I played with these people
all the time after school. Nobody said anything, they were 100% business which
was cool. I would often congratulate them on nice kills or commentate on my
victories. Everyone was about the same skill level.

Eventually I realized I was playing on a server that simply filled empty slots
with bots. I was the only human player.

~~~
deviantfero
This reads like a horror story prompt of some kind, eerie!

------
ilaksh
After maybe 10-15 years of being an AGI enthusiast who never dared to pursue
it "seriously", something has prompted me in the last few months to decide to
make my next side project into a "real" AGI effort.

I think maybe its just the number of people who are talking publicly about
working on it that is making me want to "work on it" "seriously"?

I mean, when I get my current side project "out the door" to some degree, I
plan to spend at least a few months where the weekend (or sometimes nights)
project that I actually admit to working on is "AGI research". Previously I
have occasionally spent a few hours here or there mainly passively trying to
learn about some deep learning or AGI thing by skimming papers or watching
videos. But the plan now is to actually work on active learning
projects/experiments for several hours every weekend. For at least two or
three months (or longer if I don't give up before then).

Theoretically at least some of it could be of practical use, although I am
thinking that I may avoid trying to become a deep learning expert because it
seems like people have that covered and it might take me five years. Lol. So I
am trying to think of GPU programming approaches that are new. Which most
likely will turn out to be a waste of time but will certainly be interesting
for me.

------
piinbinary
What's a useful definition of what AGI is? If you have a computer that does
some clever AI stuff, what criteria do you look for to decide that it has a
general intelligence?

~~~
lonelappde
AGI means it can solve any problem posed to it and interface with any sensory
and motor peripheral.

~~~
drdeca
not _every_ problem. Not expected to e.g. solve the halting problem.

If it can solve every problem that a human can solve, then it would be AGI. If
it can solve every problem that a somewhat unintelligent human can solve, I
think that would still count as AGI.

------
dreamcompiler
It's nice that he's doing this and I wish him well. But thousands of very
smart people have been working on AGI since 1958 and many of them initially
thought they could crack the problem in a couple of years. They made a huge
amount of progress, but the more progress we make toward AGI, the farther away
the goal seems to recede.

AGI is very much a research problem. It's not going to be solved with a clever
hack.

------
playing_colours
Based on the interviews with Jon Carmack, he is very intelligent, realistic,
reflective. Therefore, he realistically estimates his skills and the
complexity of a problem. He said he would possible make an impact - and he can
- by potentially attracting more attention to the field from researchers and
investors, by building tools, that would help in researching the problem, by,
at least, showing promising paths. He may start to dig into the problem in
“Victorian Gentleman Scientist” style, but then attract investments and scale
up. We can speculate and guess - time will show, fingers crossed.

This “Victorian Gentleman Scientist” style is something I am longing for. I
cannot go back to the academia now with family, or spend large chunks of my
time on any research, but I really want to be able to afford it. Sure, most
probably, I ll become soon disillusioned with the routine of a researcher, or
jump between topics of research, or just did not contribute anything
meaningful, but I'd really wish there was a possibility for me, other people
to afford such lifestyle.

------
rhacker
I think we just got 50% closer to making AGI happen.

~~~
ekianjo
If your odds were very low to begin with 50 % more wont make much of a
difference.

------
phtrivier
Totally off-topic, but:

> Runner up for next project was cost effective nuclear fission reactors,
> which wouldn’t have been as suitable for that style of work.

What would a (high-level) carreer path for that event _look_ like ?

(Disclaimer: I'm not Carmack-level smart. Not sure I'm anyone-s-level smart.
Asking for a friend.)

------
skokage
>Runner up for next project was cost effective nuclear fission reactors, which
wouldn’t have been as suitable for that style of work.

I can't tell if he was serious about that comment or not... Considering he
builds rockets with free time, it could go either way.

~~~
the_watcher
I took it as "not even I would experiment with nuclear fission reactors from
my house".

------
program_whiz
Maybe I am a naysayer, but John Carmack is going against other companies and
teams with not only billions in funding for hardware and data, but also with
big staffs of experts.

Not only are people who have been in the research for a while more likely to
have good ideas, but having the support of engineers to write tests and data
wrangling, Neuroscientists to bounce ideas off of, and a whole bevy of support
staff is just more likely to produce results.

I'm also not a big fan of the kind of hero worship of "well he wrote Doom, so
this should be a cakewalk." I'm not saying he won't, but he probably won't.
What am I missing here that everyone seems all hyped up about?

~~~
coolassdude6941
His “competitors” are irrelevant. It’s obvious that the existing approach
(dl/rl) is a dead end, at least for AGI. So the idea of a possible genius
working on a new approach with no monetary incentive is exciting.

------
prvc
>For the time being at least, I am going to be going about it “Victorian
Gentleman Scientist” style, pursuing my inquiries from home

Who will fund the necessary computing resources? If not FB, then he will
surely be joining or starting a different org

~~~
haihaibye
In Victorian times, a "gentleman" was someone who had so much money they
didn't need to work.

He's been a major shareholder in 2 companies that have been acquired (Id and
Occulus)

------
zhoujianfu
I’ll take this opportunity to just put out some thoughts I have about
intelligence, wisdom, AI, and AGI.

In general, building/switching contexts in your head takes intelligence (and
the more intelligent you are, the better/faster you are at it), whereas
already having a context in your head is wisdom.

I think of the current state of AI as us being able to teach computers a few
very specific contexts, i.e. imparting wisdom to them.

An AGI would be actually creating intelligence. And they are not the same
thing at all. In fact, some might say your conscience/soul is just this brain
context switcher/creator in action. An AGI would have consciousness.

------
kache_
I can see where he's coming from. He's in the unique position where he can
work on whatever interests him, he has the mathematical background to absorb
all this knowledge. I think a big part of this is that he's worried about his
age catching up to him and disabling his ability to make any meaningful
contribution. One thing to note is that he's going to loop his son into his
research.

The creation of synthetic intelligence will be a result of multiple distinct
breakthroughs. The more people with unlimited resources and high creativity
working on this problem, the more likely those breakthroughs will be made.

------
whywhywhywhy
Sad to see him leave VR but honestly I feel shipping the Quest was a huge
achievement and is a product that sums up what his original vision for VR was
and the only VR device so far with any mainstream potential

------
Nimitz14
Hm. I don't feel like additional engineering expertise is what's missing to
achieve AGI, there's still a lot more science to do I think, and I'm not sure
how good Carmack is at that.

~~~
MagnumPIG
Honestly we don't even know enough in the psychology department to possibly
arrive at AGI anytime soon.

BUT if anyone can clear a hurdle or two...

~~~
bobsil1
AGI won't require hand-designing a copy of the human brain, though knowing
some of the principles would help.

------
protomikron
Wow, that post blew up.

Like most engineers I respect JC for his incredible work, but I really think
AGI is far off and at the moment would be very surprised if there is
significant progress in the next years to come.

I also want you to read this extremely well written (old) blog post about the
topic and I don't think much has improved since:
[https://karpathy.github.io/2012/10/22/state-of-computer-
visi...](https://karpathy.github.io/2012/10/22/state-of-computer-vision/)

------
somewhereoutth
Mathematically speaking, current computers run in discrete (integer/countable)
space, and the human brain exists in continuous (real/uncountable) space.
Cantor showed that continuous space is larger than, and thus cannot be
represented in, discrete space (via diagonal method). I suggest that AGI and
consciousness lie in continuous space, and thus are unreachable with our
current discrete computation model, regardless of how sophisticated we make
it. There exists a cardinality barrier.

~~~
hvasilev
I wonder how these people have decided it is possible with a digital computer,
given that the brain is analog. I wouldn't be surprised if the first step
through AGI is to actually crack the analog computer hardware problem.

------
eaenki
It would be cool if he specified what he was going to work on.

Current narrow AI is all about data and computer power but I don’t see AGI
coming out of more data / more power anytime soon.

------
soulofmischief
I'm happy. Carmack has done a lot of good with Oculus, but it hurt seeing my
hero working for Facebook. I understand the need to remain in a consulting
position, but at least now he's no longer under the corporate leash.

If anyone can hack AI, it's Carmack, and so when I read this headline I had a
moment of fright thinking this meant Carmack was working on AI _for Facebook_.

~~~
Voloskaya
> If anyone can hack AI, it's Carmack

Pretty sure you don't get to AGI with some hacks.

~~~
soulofmischief
Don't be trite; You're well aware that _hack_ in this context means to
understand/make progress.

~~~
Voloskaya
No, I actually understand it as solving it with unexpected tricks.

~~~
soulofmischief
Well, now you can understand it differently.

------
chj
Bold undertaking even for someone as distinguished as John Carmack. I can't
help thinking that this is too broad a goal and requires more theoretical work
than engineering. Maybe making some self-replicating robots that can mate and
give offsprings would be less ambitious (perhaps I am underestimating as
well).

~~~
jobigoud
He just gave the general domain of the project, it would be weird to start
this kind of semi-retirement on a very specific domain. I see this as the
broad topic of interest that he will be investigating, but any actual research
will be done into narrow fields, wherever some interesting idea hasn't been
fully explored yet.

------
nafizh
This is so vague as in borderline sarcasm. Okk, it’s AGI but really, what is
it? Reinforcement learning? Combining symbolic AI with modern advances? Deep
learning theory? AGI is a vacuous term. And no one knows which path would lead
to something similar to human intelligence. That itself is a matter of
research.

~~~
carlosdp
The things you all listed are possible pathways to the goal. All he said is he
wants to start working toward the goal. This guy is serious business, he's
never been one to spew BS. He's a real deal, no bullshit computer scientist
with leagues of novel accomplishments under his belt.

I wouldn't bet against him when he sets his mind to something.

------
unityByFreedom
Another one bites the dust. You only need to do a few problems on Kaggle to
know AGI is a long, long way away, if achievable at all.

AGI proponents tend to claim that we know everything about physics and
biology, and that replicating it is feasible. This is science fiction.

There are much more pressing concerns in the AI space. Godspeed Carmack.

------
tim333
This is probably a great time to try to crack AGI. The cost and power of
computing has just about become ok for the task and the published work out
there like AlphaZero is getting kind of close but probably needs one or two
major algorithmic changes to get there.

------
prando
I am reminded of his programming retreat :).
[https://www.facebook.com/permalink.php?story_fbid=2110408722...](https://www.facebook.com/permalink.php?story_fbid=2110408722526967&id=100006735798590)

------
laichzeit0
We should begin by giving a clear definition of AGI. I’m talking about an
unambiguous universally accepted definition akin to defining what a
“continuous function” is. Until that point this is merely an argument about
words, to paraphrase John Locke.

------
MrZongle2
I don't know if Carmack has a good chance at achieving this goal, but I
certainly wish him the best.

There are certainly far worse applications of a sharp mind like his, and if
this is where his passion has taken him then I'm sure he will be productive.

------
tcbawo
Having listened to his appearance on the Joe Rogan Experience, I can't help
but hear this message in my head with his distinct voice. But also, I think
his brutal honesty and relentless work ethic will suit this problem domain.

------
alexashka
It's interesting that no one has mentioned that this is not unlike Jony Ive
changing roles within Apple. It's just John's way of saying 'I'm leaving
Facebook' without affecting the stock price.

------
mantoto
So apparently Mark Suckerburg reacted to it and 3000 other people.

Anyway he should probably go to Google for this. They look way more advanced
than all others.

And not sure how much money he has but there should be ml involved and that
costs a pretty penny

------
29athrowaway
The more people working on this, the better. I don't expect him to crack the
problem but contributions are welcome.

John is smart, has a reputation, has resources, is well connected and has
time. I hope he suceeds.

------
trentnix
Everything John Carmack touches he makes better. AGI will be no different.

------
tibbydudeza
Always liked him ... humble dude .. will he give us the fast inverse square
root for AGI ???.

But I suspect that "intelligence" is not so structured and reducible as we
would like .. it somehow just works.

------
taurath
People familiar with the ethics around general AI - should we be worried?

------
perseusprime11
For somebody like John Cormack, I read this as 'I am retiring'

------
dfischer
His interview on JRE was pretty enlightening. He definitely has a polymath
background to navigate towards the right direction. I believe he’ll be working
closely with Musk and Neuralink.

------
nikkwong
I'm sure he knows his stuff, and is venturing into this knowing that there's
somewhat of a "hard problem" to be solved here; serious proponents of AI like
Kai-Fung Lee have stated that AGI may never be possible.

The fact that he's making a public statement like this leads me to believe he
may already have some novel solutions on how to tackle the problem. We won't
be expecting to be seeing the same old parlor tricks coming out of John
Carmack. He is already on the forefront of this stuff after all.

That's exhilarating but also terrifying. Our still-barbarian level human
systems are still nowhere near ready to deal with the socioeconomic problems
that may arise with AGI.

~~~
Voloskaya
> leads me to believe he may already have some novel solutions on how to
> tackle the problem.

> That's exhilarating but also terrifying. Our still-barbarian level human
> systems are still nowhere near ready to deal with the socioeconomic problems
> that may arise with AGI.

I think you are reading way too much into his statement. It's extremly
unlikely that he just magically figured out a way to tackle the problem (just
knowing where to get started would be massive).

~~~
drcode
Carmack is a pragmatist- I'd doubt he'd be doing this if he didn't feel like
he had some initial promising directions in mind, though I agree he's probably
very early in the process still.

~~~
Voloskaya
Or maybe when he is talking about making a dent on AGI, he is talking about a
very long term objective (say 20-30 years) and he still has to figure out
everything.

------
ralusek
I think any monumental leap forward is going to come from changing the way
networks are motivated. Simple back propagation attempting to reduce the error
from a fixed objective have proven to be useful, but doesn't actually resemble
how more generalizable intelligence works.

I think getting the feedback loop integrated with something that behaves more
like dopamine/serotonin/pain feedback is going to be the likely direction we'd
need to go. Basically, the network needs to be able to form new objectives and
recognize when it's meeting or failing at those objectives, rather than just
optimizing its network to be less and less bad at predicting specific outputs.

------
bigred100
Is there any reason to expect he’ll have any success at this? Leaving aside
the motivation for a fun hobby that you don’t expect anything out of.

------
boomboomsubban
I have no idea what he is capable of adding to the field, but I'm quite happy
to see a long time proponent of free software enter the field.

------
maxpert
Why do I have a feeling that Oculus is gonna be dead?

------
dwheeler
I am skeptical that he will succeed, but I am happy that he will try. Carmack
is smart, and we need smart people to try hard things.

------
sriku
I'm wondering whether this will set off a wave of folks working on AGI given
that today the tools are way more accessible.

------
ganitarashid
I hope he doesn’t trigger a Phobos Anomalie

------
krick
A slightly more on point title would be "John Carmack steps down as Oculuc
CTO". The rest is pure hype.

------
delegate
So the creator of Doom wants to get into artificial general intelligence ..
what could possibly go wrong :)).

------
fizixer
I would love to do the same, except I'm not at a point where I'm financially
independent.

------
yters
He should first work on figuring out if AGI is possible. Why assume the human
mind is computable?

~~~
jobigoud
That's exactly what a scientist would say. Engineers invent things because
nobody told them they were impossible.

~~~
yters
Or they go on wild goose chases. We call those people cranks, and there are
plenty of them, such as all the people inventing perpetual motion machines
(which arguably could be AGI).

Also, if you address the fundamental question first, Carmack may unlock
something even more powerful than AI. As it is, he's just following the crowd.

------
zerr
Seems like a nice excuse for an early retirement and spending more time at
home with a family :)

------
Ono-Sendai
Cool, I hope he makes blog posts or videos or something, so we can follow
along at home.

------
proc0
I think his expertise in video game engines and rendering has a lot to
contribute here.

------
wruza
Q: Why did John Carmack not worry when his AGI decided to leave a note and go
out?

------
joeevans1000
Carmack likes lisp. I wonder if he'll be using it on this project.

------
sabujp
there's no "software" alone that can create an AGI. It's going to take a mix
of really powerful hardware, high speed interconnects, and software coming
together to do something anywhere close to what sentient (human) beings can
do. Life isn't (yet) star trek and carmack isn't noonian soong. Current approx
say the human brain is ~10^15 flops, there are only a few of these
supercomputers in this regime, with only Summit reaching into the exaflop
regime. Carmack would need full access to all of any of the petaflop machines.
What he'll probably come up with instead is probably some new good algorithms
for making things look like AGI.

------
umvi
There's no such thing as AGI in my opinion. There is no way to create a
"conscious" machine. We might be able to come up with some reasonably
impressive imitations, but nothing that is conscious or actually thinking like
a human.

~~~
wahern
If you can explain precisely _why_ we cannot build a conscious machine, you
could become one of the most revered, and potentially wealthiest, researchers
in the world, as I presume such a proof would necessarily introduce unknown
and very useful science.

I personally don't believe modern machine learning is remotely close to AI,
except perhaps the very lowest rung of the ladder of self-serving AI
definitions. I base that belief on what seem to be the unknowns, reinforced by
predictable failures[1]. But I have very little reason to believe it's
impossible. Not even the possible necessity of quantum effects would seem to
preclude it. Heck, we've already begun harnessing quantum effects in materials
science, computing, biology, and other areas.

Unless you mean that whatever we could eventually come up with would be more
biological than machine or that only a human could think like a human, but
that seems more like word play, the kind of game AI believers play. (That
said, that poses an interesting question: which is more likely to be achieved
first--a designed-from-scratch, DNA-based cellular intelligence, or something
_not_ based on DNA or otherwise mimicking existing organic life? If at all, of
course. Also presuming such a distinction isn't in fact hopelessly quaint and
naive.)

[1] I'm not a naysayer. While I never believed self-driving cars were around
the corner (not even 5 or 10 years out; you can Google my HN comments from
years ago), I have no doubt the science has been useful and can and will and
is put to great, largely unseen use, as is typical of most science.

~~~
umvi
I can't prove it, but I will tell you why I believe it.

We can't build a conscious machine because I believe there is a spiritual
aspect to life that we have thus far failed to empirically observe or measure.
Put another way, I believe all humans have a "spirit" inside of them, and that
said spirit is a prerequisite of conscious thought. This same spirit is what
makes life after death possible. This belief, of course, is an extension of my
belief in God.

Thus, it is impossible to build a truly conscious machine without a spirit to
inhabit said machine.

This all implies, of course, that truly conscious AGI effectively proves God
does not exist since it proves there is nothing special about humans or any
intelligent life for that matter.

In summary, since I believe there is something special about life and that
there is a secret sauce (spirits) that we haven't observed or measured, AGI
projects are always doomed to fail (though they may spawn new interesting
fields of mathematics or computer science)

~~~
wahern
I was raised Catholic and still consider myself Catholic. One aspect of
Catholicism, oft ridiculed, is an emphasis on sacred mysteries. In Catholic
theology these aren't simply a way to explain miracles. Rather, the tenet is
that there are aspects of the world that are unknown and even unknowable. Thus
there are limits to what can be theologically positively affirmed or
categorically rejected.[1] So to say that a machine could never have a soul is
to say too much, at least in the context of Catholic theology.

You're entitled to your belief, of course, but simply asserting the existence
of [Christian] souls and an inter-relationship between souls and intelligence
isn't sufficient to make your claim.

Discussing souls is probably not appropriate subject matter on HN, but I
thought it was worthwhile to make the point that even religions can have and
utilize the concept of unknowns to self-limit its theological reach.
Technically speaking, orthodox (small 'o') Christians (Catholics, Orthodox,
mainline Protestants) only need affirm the Nicene Creed and nothing more, just
like, AFAIU, Muslims need only affirm the Shahada. Thus, an orthodox Christian
is supposed to affirm the Trinity, the resurrection of Jesus, etc, but
otherwise free to accept or reject machine souls, notwithstanding additional
doctrinal strictures of particular denominations that might dictate a
particular choice.

[1] This is why you see seemingly radical priests quoted as admitting that
aliens could have souls. (And also why it's so easy for Bill Maher or Michael
Moore to find a priest to make seemingly contradictory and outrageous
statements; more often than not the priest is playing them more than Maher or
Moore understand, they're just too cynical to appreciate it.) Catholicism is
hardly the only religion, or even Christian religion, with such a concept. But
it's perhaps gone to the greatest lengths to develop and integrate the concept
with modern logic and science.

------
Rerarom
Reminds me of Paul Cohen working on the Riemann hypothesis.

------
option
We need more people like him in AI. Super happy!

------
bitwize
Because of course he is. Because John Carmack.

------
tonfreed
Is this going to be like the AI in Daikatana?

------
negamax
I have seen what happens next in Blackmirror!

------
2OEH8eoCRo0
I'm ready for this post to disappear, strange things start happening in
cyberspace, and Carmack abruptly cans the project and doesn't want to talk
about it.

~~~
zelly
Elaborate

------
mark_l_watson
Good for him. For AGI development, I wonder if the behavior of people playing
multi player VR games on the Oculus platform could be used as training data.

------
ses1984
What a boss.

~~~
mc3
And now, without a boss!

------
daodedickinson
My dream would be that Carmack would go toward Douglass Hofstadter and they'd
move us toward forms that flawed human intelligence has been striving toward,
rather than reifying flaws. I wish the people capable of helping with that
could help with that and then I hope that the rest of us could all gather
together to solve the loneliness CATACLYSM.

------
tehjoker
It's nice being rich.

------
tomerbd
why is this so important why do you consider him such a genius?

------
_pmf_
I just noticed that this is the first Facebook post I ever read.

------
lazyjones
Let's hope Boston Dynamics doesn't hire him. ;-)

~~~
red2awn
Boston Dynamics isn't doing AGI.

~~~
lazyjones
yet...

------
objektif
FYI I also just decided that I am going to work on fusion.

------
madacoo
Is there a non-Facebook source for this information?

------
cvaidya1986
Me too.

------
acollins1331
Maybe general intelligence is not a goal to pursue, isn't general intelligence
of machines what would eventually lead to the 'singularity' that seems far
away now but is plausible in theory? Maybe John's real last name is Connor.

------
codesushi42
This would be great if only he was not at Facebook.

------
iamleppert
It must be hard to be someone like him. Always chasing the original high that
made him famous, the enormous pressure to succeed must be nauseating.

Instead of retiring and relaxing and looking back on an impactful and lucky
career, it says something about how powerful the original emotions were that
led him to his current point.

He will do anything to get back to that state, that place in time, even
sacrifice what are supposed to be the good years of his life stuck behind a
screen.

~~~
neonate
How do you know he isn't just doing what interests him? Not everyone wants to
retire, especially highly creative people.

------
ausjke
he is 49 this year, considered as one of the most genius programmer on earth.
There are professors still doing real work at 90+ year old(yes, the UT
professor goodenough for Nobel prize), John has a long way ahead, best luck!

~~~
chasd00
i've heard he was pretty good, i didn't realize he was this widely respected
and admired. I remember him for that fast inverse square root hack but that's
about it.

~~~
galangalalgol
I think he got that from someone at SGI.

~~~
galangalalgol
Or was the second person to independently come up with it.

------
mahesh_rm
This post feels a little bit like something that'll be upvoted to top spot
many years from now, for one reason or another. :-)

------
xwdv
Imagine inventing artificial general intelligences and then there’s some
public outcry for products powered by natural, organic intelligence instead.

~~~
0xdeadbeefbabe
This.

------
The_rationalist
I hope he'll go the hybrid symbolic and neural network way (causal and
statistical), instead of just statistical.

AGI needs a type system...

I hope I'll achieve AGI before him but it's nice to know there's some real
competition! (because, reader, there are almost 0 researchers seriously trying
to achieve AGI in a not totally bullshit way. Only opencog and Cyc comes to
mind).

~~~
ilaksh
Are you really that sure that the approaches to increasing the generality of
AI being taken by LeCun (self-supervised model learning), Hinton (capsule
networks) and Bengio (state representation learning) are all "total bullshit"?

~~~
The_rationalist
From my reading, Hinton capsule networks seems far from being enough, it could
at best be an incremental improvement. And is unrelated to English semantic
parsing, it seems specialized for computer vision.

~~~
ilaksh
English semantic parsing is small part of AGI. And a system that can only do
that or only for one language is never going to be general.

------
mentat
He invented modern graphics as a practical problem by himself as the sole
researcher. Given the tools at the time that may have been a harder problem.

~~~
justin66
> He invented modern graphics as a practical problem by himself as the sole
> researcher.

No, he didn't, and that is not a claim that he would ever make himself.

~~~
pixelpoet
Agreed, and if anyone could make that claim it'd be Eric Veach (who then went
on to develop Google Adwords).

------
unityByFreedom
One could imagine him beginning to work in gaming AI, but he says he wants to
work on AGI, not simply gaming AI.

I'm personally not convinced that it is encouraging when someone bright sets
their sights on AGI, particularly someone who appears to have never competed
on Kaggle. It screams hubris.

~~~
bitexploder
I am not convinced current AI is the approach to an AGI. I think it is at
least feasible someone outside of this sphere has a reasonable shot at it. It
feels like many AI researchers get caught up in refining existing techniques
that amount to fancy statistics algorithms and data crunching, but not AGI.
Current AI techniques may be synthesized or used in part in some AGI but it’s
clear there is a revolutionary step to be made. Kaggle is almost just an
optimization fest, and not really advancing towards AGI.

~~~
unityByFreedom
> I am not convinced current AI is the approach to an AGI

This is a non-sequitor. I didn't argue that current AI theory, or Kaggle, will
lead us to AGI.

> Kaggle is almost just an optimization fest

Public machine learning competitions have produced a lot of innovative
learning techniques. If Kaggle is so easy, and AGI so hard, it would follow
that anyone tackling AGI would have some experience applying machine learning
competitively in some public space. It doesn't necessarily need to be Kaggle.
Kaggle just happens to be good at hosting such public competitions, and in
fact has surfaced several state-of-the-art implementations. The difference
between prize money (~top 5) and no-prize-money (5-10) may be an "optimization
fest", but without the competition, the solutions presented would have entered
the public sphere at a much slower rate.

EDIT: Please be kind and explain your downvotes.

~~~
jacobush
Ok, I'll take a shot: "it would follow that anyone tackling AGI would have
some experience applying machine learning competitively in some public space".

No, that would absolutely _not_ follow. (I'm a pretty good devil's advocate,
but I can't with this one.)

And given AGI would come from some completely new breakthrough not related to
the current practice of "machine learning", competitions may be completely
moot. They may be great for finding the nice increments in the state of the
art of machine learning, but they are unlikely to help much with AGI.

I could even imagine a stumbling AGI being _very stupid_ compared to just
about any machine learning solution thrown at it - yet being undeniably AGI.
Like a dog not being very good at DOTA, Star Craft or Chess, yet it undeniably
possesses some kind of general intelligence.

~~~
tarsinge
> And given AGI would come from some completely new breakthrough not related
> to the current practice of "machine learning"

I’m not so sure of that. Intuitively AGI feels like being able to generalize
and automatize what is already done in specialized problems, like having a
meta program that that orchestrate and apply specialized subsystems, and adapt
existing one. If playing Go, Starcraft, Speech recognition, Computer vision
are already of the same building blocks, it feels like having a meta program
that‘s just trained to recognize the type of problem and route it to the
appropriate subsystem with some parameters tweaks is a path to AGI. In the dog
example you don’t even need to have subsystem that are that better than humans
individually.

Edit: my point is I feel like AGI is the interface and orchestration between
specialized subsystems we already know how to create. Trying to train a big
network like generalizing Alpha Go is a dead-end, but having simpler sub
networks ready to be trained at a specific problem seems feasible. Much like
the brain is at first seen like a big network, but in practice there are
specialized areas. The key is how are these networks interfaced and which
information they exchange to self adapt. Maybe these interfaces themselves are
sub networks specialized in the problem of interfacing and “tuning
hyperparameters”.

In short: I think when we’ll figure out how to automate Kaggle competitions
(recognize the pattern of the problem, then instantiate and train the relevant
subsystem) we’ll be a good step forward AGI. We don’t need better performance
e.g. in image recognition, just how to figure orchestration.

