Don't need regulatory capture to stop the small fries when the cost of entry is so high. Only ginormous entities can even realistically play in that game right now.
Think about it, where are you gonna get 10000 H100s?
Best you or I can hope for is to fine tune the models the big guys make "open source". None of which will be as serviceable as their closed models.
Also model performance follows a power law. There are strongly diminishing results, where after some point throwing twice as much money at the problem only gains you 1% accuracy. OpenAI knows this, so they're all in on regulatory capture.
It’s always going to be like this. Take Apple. Plucky garage started anti-establishment startup now a giant gate keeper of apps and stores using any means necessary to keep that 30% tax/tariff/whatever you call it
It’s odd you’re invoking moore’s law here, which actually isn’t a law, and is well understood and accepted over at least the last decade across business and academia that it is slowing and is not limitless. As per what “other” advances you’re referring to, I can’t speculate - maybe quantum computing? We are quite a long way away from that being a commercial reality.
It's mainly hardware upgrades (new TSMC nodes), training data quantity/quality improvements, and model architecture (and broader software) changes. Here's a fun project I've had my eye on recently: https://github.com/KellerJordan/modded-nanogpt/
Right now it seems more and more like we will all just end up using Chinese AI. All this regulation is doing is forcing Chinese companies to figure out how to do more with less.
Who cares if Chinese AGI is a year behind OpenAI/Anthropic. At some level, progress will stall and then OpenAI/Anthropic will be on the clock before being undercut out of business on price.
They are probably just positioning now for when that happens and they have to become some kind of regulated utility for US government AI use.
Just another example of having our head up our ass as Americans. Seems like what we are doing will almost guarantee most the world outside the US is using Chinese AI in the future.
I really really freaking wish Americans would stop worrying about who has the best $STUFF, and start worrying about whether their own kids are fed/clothed/educated.
China is too big. This was going to happen eventually. Empires come and go, that's just a fact of life. The question was whether you'd fall back gracefully like Western Europe, now ranking at the top of "World's Happiest Countries", or whether you'll become Russia 2.0 with the biggest guns, richest oligarchs, and the worst quality of life.
Eisenhower 1953 said "Every gun that is made, every warship launched, every rocket fired signifies, in the final sense, a theft from those who hunger and are not fed, those who are cold and are not clothed." Nobody listened. Americans think life is a Marvel movie, and they are Iron Man. Turns out conflict is not a long-term winning strategy.
Nah, training efficiency, inference efficiency, and compute availability all have enough headroom that the third-party and grassroots models will become increasingly available.
The current leaders basically need to either (1) break through the capability ceiling before everybody else catches up, or (2) construct law/regulation that keeps everybody else out of the game. #1 is uncertain, but #2 is well-understood business.
Since they all share that same interest (they'll be duking it with each other either way), the result is an "industry wide" effort to artificially close the door behind them.
This is the problem with the software industry generally. At the end of the day it’s the hardware manufacturers that hold all of the power aka “the means of production” and America has largely gutted it’s manufacturing capacity opting to make a quick buck without a plan to incentivize investment into new manufacturing capacity. China is on a rapid path to surpassing the West even in cutting edge technologies because they also have the domestic production capacity for all of the unsexy capital intensive industrial production that feeds the innovation.
