This is such a bad analysis, I don’t even know where to start.
Let me start with the strongest point that this article tries to make:
1. Big Tech will catch up to OpenAI, and yet one year after GPT-4 release, no one has caught up to OpenAI, in fact the only model that beat gpt4 on lmsys is gpt4o lol. This is on benchmarks, if you see in User Space, OpenAI is > 80% and now with gpt4o being released for free, (there’s no other model that comes even close that’s available for free) they will cement their lead even more.
2. Some weird argument about compute becoming prohibitively expensive. MSFT has already promised to build a 100 billion data center. If your bar for judging success is AGI then maybe, but with the compute we already have a useful AI that generals billions in annualized revenue for OpenAI. So I’m not sure why you’d think this won’t keep improving for at least the next decade. There are so many jobs that can be automated before we reach AGI
3. OpenAI lost cracked researchers. OpenAI lost people in the super alignment team, none of the top research leads who implemented GPT4 have left. Ilya arguably was just coasting for the past few years (he’s not credited with any specific contribution in gpt 4 or 3.5). The most capable people in OpenAI are still present at OpenAI
Agree that it was not a great article, but I think the main point he made was that the commercial winners will be those that can deliver to customers. OpenAI with its relationship with Microsoft will have exposure to that but even a lessor LLM that is tightly integrated with your gmail and the google suite, or is running privately on your iPhone with your iCloud data, or is 'free' will be the winner. Apple, Google, and Meta can spend 10s or 100s of billions of dollars to build AI tools to support their other profit making products. I'm a happy paying OpenAI customer, but the majority of the world (where the money is) will happily use the AI that comes with their phone, search engine, or word processor.
> The reason for my bearishness is simple: OpenAI, the software company, will ultimately lose to Apple, Google, and Meta. OpenAI, the hardware company, will also ultimately lose to Apple, Google, and Meta. Their only hope is to be the first to AGI.
My point was they can spend billions and fail on projects and continue being profitable, they don't need to make commercially successful, better AI products than OpenAI to succeed. Apple could release an iPhone based LLM agent that is 50% as good as OpenAI, Meta can spend billions developing an LLM to use internally and give away.
> and yet one year after GPT-4 release, no one has caught up to OpenAI
You're wrong, Claude 3 Opus beat OpenAI’s GPT-4 in Elo rating just a month ago. They since lost the top position, but they are at ELO 1246 compared to GPT-4 at 1250 and GPT-4o at 1287. They are bottlenecked at GPT-4 level with small variations. That's the issue. Why have they stopped advancing? The gap between GPT-3.5 and GPT-4 is so much wider than that between GPT-4 and GPT-4o
The explanation is that we have exhausted the sources of high quality human text. All providers train on essentially the same corpus. Architecture variation doesn't matter. Data is king, and AI used up all the human data.
My take on the path forward - it is easier to imitate than to innovate. LLMs up to now are mostly imitative. In order to become innovative they need to learn from things, not just from people. It's a social process, based on evolution of ideas. It's not unlike DNA and language, intelligence is social. It takes a village to raise a child, it takes a whole world to train an AGI. Humans also take 20 years of experience based training before becoming truly innovative at the edge of human knowledge.
>The gap between GPT-3.5 and GPT-4 is so much wider than that between GPT-4 and GPT-4o
Why is this an argument? GPT4o is obviously a much smaller model than GPT4 and is not meant to replace GPT4. It's meant to replace GPT3.5 as the free version. The fact that OpenAI can make a much smaller model and outperform or perform on par with the original GPT4 is a good sign.
OpenAI is still working on the actual GPT4 replacement.
> OpenAI is still working on the actual GPT4 replacement.
You speculate.
...but, what we can evaluate is what we can see, and what we can see is that there’s now only a marginal difference between what openAI actually has available and what other vendors actually have available.
What we havent seen, is a demo of something that is an order of magnitude more capable, aka. GPT5.
Not a demo. Not a post. Not tweet. Not a hint of something concrete.
Just some vague hand waving that it’s busy baking, take it on faith, it’s gonna be great.
…but let’s be really clear here; what they actually delivered is not great, it is less capable. It is smaller. It’s a consumer model for the masses to talk to.
So, in that context, you have to take in faith that openAI is going to deliver a GPT5, before anyone else.
…but that’s a hunch. That’s a guess.
…and they have some very stiff competition; and it’s very very much not clear to me they’re going to the first ones to drop something that is concretely better than gpt4; because gpt-4o was a big disappointment.
> The fact that OpenAI can make a much smaller model
GPT4o didn’t even need to be better than GPT4 in any benchmarks. Being conversational and running in real time, is such a big difference user experience wise, that the fact that they beat GPT4 is just cherries on top. And it’s almost quasi confirmed that GPT-5 has been trained and that it’s undergoing red teaming and safety checks now and it will be out in the summer, further cementing their 1 year lead.
1. Big Tech will catch up to OpenAI, and yet one year after GPT-4 release, no one has caught up to OpenAI, in fact the only model that beat gpt4 on lmsys is gpt4o lol. This is on benchmarks, if you see in User Space, OpenAI is > 80% and now with gpt4o being released for free, (there’s no other model that comes even close that’s available for free) they will cement their lead even more.
2. Some weird argument about compute becoming prohibitively expensive. MSFT has already promised to build a 100 billion data center. If your bar for judging success is AGI then maybe, but with the compute we already have a useful AI that generals billions in annualized revenue for OpenAI. So I’m not sure why you’d think this won’t keep improving for at least the next decade. There are so many jobs that can be automated before we reach AGI
3. OpenAI lost cracked researchers. OpenAI lost people in the super alignment team, none of the top research leads who implemented GPT4 have left. Ilya arguably was just coasting for the past few years (he’s not credited with any specific contribution in gpt 4 or 3.5). The most capable people in OpenAI are still present at OpenAI
so yeah not a great article