Do you think that human civilization has a finite demand for intelligence? That there is a point that we can hit, and stop there, and say "we've thought of all there is to think about, no more thinking is required?"
Do you think that we are anywhere near that point?
We've already seeing things - like what's happening with AI reverse engineering now. I'm a reverse engineer - and I know that the reason why RE was used as "last resort option" was that expert reverse engineers are rare, expensive, and work slow.
But now that a lot of RE workloads can be tackled by AI instead of a human? The demand for reverse engineering is at a never before seen high. AI made reverse engineering much more viable - and thus, a lot more reverse engineering is happening now.
There are myriads of fields like this - things that go overlooked not because no one wants them, but because the economics of human expertise make them nonviable in most cases. As AI gets better, it can begin to enter those fields.
Except you, of course, need to push the frontier to do it.
> Do you think that human civilization has a finite demand for intelligence?
No, I don't. But for any given task there is a ceiling above which for that task there is no value in additional intelligence. E.g. it takes very little intelligence (none, in fact) to sort a list. More intelligence does not result in a more sorted list.
And so for a huge proportion of what we do there is a ceiling of value that can be extracted by selling AI tools to do any given task.
Combine that with there being at any given time a ceiling of available capital to be apportioned to buy intelligence to carry out the remaining tasks, and the floor of tasks that non-frontier models can do well enough rising, and there will at any given time be a finite amount of capital that will be accessible to frontier models.
It is by no means given that that slice of capital will grow rapidly enough for frontier providers to be able to create an attractive financial return. That depends on frontier labs being able to continue to deliver sufficiently more value than cheaper models at any given time that enough people won't simply wait.
I'm extremely bullish on AI in terms of its ability to transform society over time, but that does not automatically translate into assuming that the current frontier labs are a safe investment bet.
"Sort a list" doesn't exist by itself. It isn't a terminal goal - it's a step towards something else.
If "sort a list" becomes a trivial task, everything that was bottlenecked on "sort a list" is unbottlenecked, and other parts of the process become bottlenecks instead. Thus, the demand for non-sorting tasks grows. The saturation of low end feeds the value into the high end.
> "Sort a list" doesn't exist by itself. It isn't a terminal goal - it's a step towards something else.
Sometimes it is. But that is also irrelevant. Replace "sort a list" with any given goal and the argument remains the same.
> The saturation of low end feeds the value into the high end.
But that process is not immediate. There's a reason I was careful to include "at any given time" in my comment.
For the purpose of whether or not a frontier lab is a good investment, it doesn't necessarily help them if there is an ever rising demand for intelligence if their market is eaten from below just as fast by cheaper models from competitors catching up.
It could be they manage to out-pace their competitors sufficiently that they will produce returns high enough to justify their valuations.
But irrespective of whether the demand for more and more intelligence continues to rise perpetually if 2nd tier labs hollows out the bottom of their market fast enough, it could equally be that most of the market ends up being low-margin, highly-competitive inference that won't justify their valuations.
Given the size of their valuations, even if they turn out to be tremendously profitable businesses, it's still not a given they'll give a good return on investment for investors getting in now.
I'm not saying they won't. I'm saying we can't easily judge that based on just assuming growth at the high end will continue without understanding what their moat is likely to look like.
> everything that was bottlenecked on "sort a list" is unbottlenecked, and other parts of the process become bottlenecks instead. Thus, the demand for non-sorting tasks grows.
Your assumption here is that the other parts of the process are intelligence. In the real world they usually are not e.g. raw materials, politics, zoning, manual labor, clinical trials, traffic, energy ... And usually those things are such huge bottlenecks that Amdahl's Law renders the speed of delivery of the intelligence part irrelavant.
This kind of all depends on being able to continuously make better models indefinitely. I'm not convinced that's possible. If there is a limit, then everybody will get there, and anthropic has no moat at all.
It's always disappointing to see comments by arrogant elitists claiming that most people are essentially NPCs. But even if your claim was correct, the demand for inference compute capacity by businesses, governments, and universities is effectively infinite. If there is a limit then there are no signs that we're getting anywhere close to it.
If his claim is correct, and 99% of people are NPCs incapable of any meaningful thinking, that makes the 1% that actually does all the thinking of human civilization very valuable.
And that, in turn, makes AI even more valuable if it can produce thinking on demand.
That is weird claim considering did not reward "highly intelligent" or "knowing stuff" on itself. Super smart people great at whatever jobs they picked have roughly normal salaries.
Researchers specifically were never highly paid. And positions were competitive - there were way more people willing and able to do those positions then positions themselves. Same goes for art btw.
> If his claim is correct, and 99% of people are NPCs incapable of any meaningful thinking, that makes the 1% that actually does all the thinking of human civilization very valuable.
That's like saying if 99% of birds don't have huge colorful tails that makes peacocks very valuable. In each case it may or may not be true but you are just correlating. (i.e. It's possible that intelligence just isn't as valuable as 'intelligent' people think it is)
Do you think that we are anywhere near that point?
We've already seeing things - like what's happening with AI reverse engineering now. I'm a reverse engineer - and I know that the reason why RE was used as "last resort option" was that expert reverse engineers are rare, expensive, and work slow.
But now that a lot of RE workloads can be tackled by AI instead of a human? The demand for reverse engineering is at a never before seen high. AI made reverse engineering much more viable - and thus, a lot more reverse engineering is happening now.
There are myriads of fields like this - things that go overlooked not because no one wants them, but because the economics of human expertise make them nonviable in most cases. As AI gets better, it can begin to enter those fields.
Except you, of course, need to push the frontier to do it.