> As much as I respect Carmack as a computer graphics expert, I really doubt his competence in machine learning. He doesn't have a single notable paper published.
Publishing papers is the way the academic/scientific world measures notability and/or competence. It's not the way the engineering world that Carmack comes from measures it. They measure it by building. But you're right, we kind of have to just trust that he has the expertise he says he does by his statements since he has not built any modern AI programs (that I know of at least).
> If he really thought that implementing gradient descent and basic stuff in a week long retreat gave him the chops to have serious conversations with AI researchers, he is really deluded.
This is not an accurate account of how he said he developed his knowledge base. Just how he got started so he could have conversations. He said that he spent a retreat learning the basics and then later in the interview he said he took the time to understand the 40 most essential papers in the field as related to him by a well known researcher. He has since largely put the last 4 years of his professional life into this. While we have no proof of his knowledge, given his intelligence and high competence in computer programming and math, I have no doubt that if he did put in the work he could achieve an understanding equivalent to that of your average AI researcher.
That said, of course it makes sense to be skeptical.
> This is not an accurate account of how he said he developed his knowledge base.
I quoted him directly, because I was expecting this kind of response. He took a week off and implemented some stuff from the ground up and was ready to have serious conversations with AI researchers. The 40 papers by Ilya came later. I have read a 100 ML papers and reviewed preprints. That's quite a low bar, especially if you are prone to skip the math and simply read the abstract and conclusions.
His whole approach gives me a ML for hackers vibe and his thoughts on AGI, if it had come from anyone else, would have been described as word salad.
> The 40 papers by Ilya came later. I have read a 100 ML papers and reviewed preprints.
I would say it’s more likely John Carmack is capable of learning the state-of-the-art of AI from 40 papers than a random (pun intended based on username) from 100.
Sure, he must be faster than Geoff Hinton too and it took Hinton a life time.
Funnily enough, I am able to publish ML papers - but John Carmack isn't. I wonder why. I would also like to learn more about all the computer graphics algorithms Carmack has invented before I trust him to invent AGI.
Here is one example of a person I am familiar with - Math Olympiad bronze medalist. Princeton PhD in ML Theory. AI researcher in Google.
Your OP and replies kind of just come across as jealous that John Carmack’s opinion on this stuff is taken seriously whereas relatively unknown folks’ (yourself, the researcher you mentioned) opinions, are not despite the fact that he’s not traditionally credentialed. Like I said in my original post, we should be skeptical of him and his claims. But the way you are dragging this thread out feels like sour grapes. Of course one of the most famous programmers in the world is at least heard out when he dedicates years of his life to a programming adjacent topic. That’s just how fame works.
And calling out Princeton, Google, etc. further exemplifies an academic bubble kind of credentialism. I suspect you don’t realize how it sounds because you’re so in that credential filled world.
I don't have any credentials really. If I had to be jealous I would be jealous of Carmacks work in id and oculus.
I am definitely irritated by the fact that he is able to pull 20M in funding. He has been giving interviews left right and center. We are upvoting Altman, sam Harris,Carmack but nobody cares about the actual AI researchers, all academics, who have brought about the Deep learning revolution.
Your comments about academic bubbles and researchers gives you away. All of the revolutions in AI have been brought about by the academics inside bubbles you are jeering at. The biggest example of academic inside a bubble is Geoff Hinton.
We have to sit here and listen to word salads from
Carmack, Altman (Ilya - who doesn't get interviewed is the actual researcher behind chatgpt), Sam Harris etc, who have very little insights making bold AGI predictions.
I asked to learn about all the algorithms Carmack invented in computer graphics, but haven't heard back. But looks like he has a good grasp on solving AGI by 2030 according to himself. After all, he is a really great programmer.
> I don't have any credentials really. If I had to be jealous I would be jealous of Carmacks work in id and oculus.
Fair, you come across as jealous of him for something... could be that.
> I am definitely irritated by the fact that he is able to pull 20M in funding. He has been giving interviews left right and center. We are upvoting Altman, sam Harris,Carmack but nobody cares about the actual AI researchers, all academics, who have brought about the Deep learning revolution.
There it goes again. Fame is fame is fame is fame. Like I said earlier. That's just how it works. No reason to be against someone for being famous for their non-academic accomplishments if they have something valuable to contribute.
> Your comments about academic bubbles and researchers gives you away. All of the revolutions in AI have been brought about by the academics inside bubbles you are jeering at. The biggest example of academic inside a bubble is Geoff Hinton.
I didn't jeer at anybody I just tried to give you some perspective about how your comments came across. I appreciate all of the researchers but I also live in the real world and understand that people flock to personalities and front-people. That's not necessarily good, but that's the way it is. Just like nobody gives credit to under-secretary of state for coming up with a great new foreign policy.
Waving around great university A or great company B does not make someone any more right, just like being John Carmack doesn't make him any more right.
> We have to sit here and listen to word salads from Carmack, Altman (Ilya - who doesn't get interviewed is the actual researcher behind chatgpt), Sam Harris etc, who have very little insights making bold AGI predictions.
I agree with you. I never said they were right about AGI. Maybe you should be more generous with my replies and actually think I was trying to give you another perspective about how your comments came across.
> I asked to learn about all the algorithms Carmack invented in computer graphics, but haven't heard back. But looks like he has a good grasp on solving AGI by 2030 according to himself. After all, he is a really great programmer.
Again, you're confusing who he is (go back to my original comment in the thread where you jeered at him because he didn't publish any notable papers). He is not a researcher and he is not really a computer scientist. Engineers don't come up with novel algorithms for the most part. He builds things. And being a really good builder is a different but still valuable perspective. But that doesn't mean he's right about AGI.
AI researcher perhaps, but almost none of them understand cognition. They're focussed on getting something that vaguely resembles a part of the brain to predict the next most likely token. Their idea of cognition apparently stops at Skinner.
Any AI researcher worth their salt knows that those models aren't representative of how the human brain works... but they're just the kind of models that work best out of what we know how to implement right now. There are models with stronger cognitive inspiration, but their performance is worse.
Publishing papers is the way the academic/scientific world measures notability and/or competence. It's not the way the engineering world that Carmack comes from measures it. They measure it by building. But you're right, we kind of have to just trust that he has the expertise he says he does by his statements since he has not built any modern AI programs (that I know of at least).
> If he really thought that implementing gradient descent and basic stuff in a week long retreat gave him the chops to have serious conversations with AI researchers, he is really deluded.
This is not an accurate account of how he said he developed his knowledge base. Just how he got started so he could have conversations. He said that he spent a retreat learning the basics and then later in the interview he said he took the time to understand the 40 most essential papers in the field as related to him by a well known researcher. He has since largely put the last 4 years of his professional life into this. While we have no proof of his knowledge, given his intelligence and high competence in computer programming and math, I have no doubt that if he did put in the work he could achieve an understanding equivalent to that of your average AI researcher.
That said, of course it makes sense to be skeptical.