Are people down to have a bunch of specialized models? The expectation set by OpenAI and everyone else has set is that you will have one model that can do everything for you.
It’s like how we’ve seen basically all gadgets meld into the smart phone. People don’t have Garmin’s and beepers and clock radios anymore (or dedicated phones!). It’s all on the screen that fits in your pocket. Any would-be gadget is now just an app
> The expectation set by OpenAI and everyone else has set is that you will have one model that can do everything for you.
I don’t think that’s the expectation set by “everyone else” in the AI space, even if it arguably is for OpenAI (which has always, at least publicly, had something of a focus on eventual omnicapable superintelligence.) I think Google Antigravity is evidence of this: there’s a main, user selected coding model, but regardless of which coding model is used, there are specialized models used for browser interaction and image generation. While more and more capabilities are at least tolerably supported by the big general purpose models, the range of specialized models seems to be increasing rather than decreasing, and seems likely that, for conplex efforts, combining a general purpose model with a set of focussed, task-specific models will be a useful approach for the forseeable future.
Having everything in my phone is a great convenience for me as a consumer. Pockets are small, and you only have a small number of them in any outfit.
But cloud services run in... the cloud. It's as big as you need it to be. My cloud service can have as many backing services as I want. I can switch them whenever I want. Consumers don't care.
"One model that can do everything for you" is a nice story for the hyper scalers because only companies of their size can pull that off. But I don't think the smartphone analogy holds. The convenience in that world is for the the developers of user-facing apps. Maybe some will want to use an everything model. But plenty will try something specialized. I expect the winner to be determined by which performs better. Developers aren't constrained by size or number of pockets.
I think of the foundational model like CPUs. They're the core of powerful, general-purpose computers, and will likely remain popular and common for most computing solutions. But we also have GPUs, microcontrollers, FPGAs, etc. that don't just act as the core of a wide variety of solutions, but are also paired alongside CPUs for specific use cases that need specialization.
Foundational models are not great for many specific tasks. Assuming that one architecture will eventually work for everything is like saying that x86/amd64/ARM will be all we ever need for processors.
Specialized models are cheaper. For a company you're looking for some task that needs to be done millions of times per day, and where general models can do it well enough that people will pay you more than the general model's API cost to do it. Once you've validated that people will pay you for your API wrapper you can train a specialized model to increase your profit and if necessary lower your pricing so people won't pay OpenAI directly.
It's probably the direction it will go, at least in the near term.
It seems right now like there is a tradeoff between creativity and factuality, with creative models being good at writing and chatting, and factuality models being good at engineering and math.
It why we are getting these specific -code models.
It's really an implementation decision. The end user doesn't need to know their request is routed to a certain model. A smaller specialized model might have identical output to a larger general purpose model, but just be cheaper and faster to run.
I still use the Garmin I bought in 2010. I refuse to turn on my phone's location tracking. Also the single-purpose interface is better and safer than switching between apps and contexts on a general purpose device.
It’s like how we’ve seen basically all gadgets meld into the smart phone. People don’t have Garmin’s and beepers and clock radios anymore (or dedicated phones!). It’s all on the screen that fits in your pocket. Any would-be gadget is now just an app