Previously I've always been very skeptical of rosy pictures of a possible future where "everyone has an ai that's there to accomplish tasks for them" - given that I imagined such ai (if it ever came to exist) being run by the usual big tech who have their own incentives not so cleanly aligned with our own.
Right now, with the availability of open weights for cutting-edge models, it feels like this wave of technological advance is pleasantly decentralised however. I can download and run a model and tinker with things which at least feel like the seeds of such a future, where I _might_ be able to build things with my own interests at heart.
But what happens if these models stop being shared, and how likely is that? Reading about the vast quantities of compute deployed to train them, replicating the successes of the main players with a community of volunteers just seems an order of magnitude less achievable than traditional OSS efforts like Linux. This wave feels so tied to massive scale for its success, what do we do if big-tech stop handing out models?
I think we're all fortunate in that the companies behind the best OSS models, i.e. Meta/llama and Alibaba/Qwen are funding their compute and R&D from secondary business models instead of VC capital or an AI company who's primary business model is direct revenue from their models and will be seeking ROI. That's why I don't expect we can rely on Mistral AI to OSS their best models in the long run since that's their primary business model. This is reflected in their hosting costs which charge a healthy premium that's always more expensive than OpenRouter providers hosting their OSS models.
But I don't see why Meta and Alibaba would stop releasing their best models as OSS, since they benefit from the tooling, optimizations and software ecosystems being developed around their OSS models and don't benefit from a future where the best AI models are centralized behind the big tech Corps. As long as their core business remain profitable I don't expect them to stop improving and sharing their OSS models.
I read an article about humanoid robots yesterday and it scared me that it seemed like the expectation is still that the robot will be 24/7 online and "thinking" using some cloud brain. The current models described more in detail all used Open AI as a brain.
Having a personal robot would be great, but they have to invent a fully offline real positronic brain before I will consider allowing one in my house.
Fully open source might be too much to hope for, but that would obviously be the ideal. If it is closed source it definitely should be offline. I can have another, carefully sandboxed, AI in my computer that can help out with tasks that require online access. No need for the two types to be built into the same device.
Prediction 1: The value isn't in the foundation model, it's in fine tuning and in tightly integrated products.
Prediction 2: The ecosystem around open source models will grow to be much larger, richer, and deeper than closed source models.
If these are true, then OpenAI and Anthropic are in a precarious place. They basically burned a lot of capital to show the open source second movers what to build.
I worry about 2 main pitfalls for junior devs, one more tractable than the other.
Firstly there is the double edged sword of AI when learning. The easy path is to use it as a way to shortcut learning, to get the juice without the pressing, skipping the discomfort of not knowing how to do something. But that's obviously skipping the learning too. The discomfort is necessary. On the flip side, if one uses an llm as a mentor who has all the time in the world for you, you can converse with it to get a deeper understanding, to get feedback, to unearth unknown unknowns etc. So there is an opportunity for the wise and motivated to get accelerated learning if they can avoid the temptation of a crutch.
The less tractable problem is hiring. Why does a company hire junior devs? Because there is a certain proportion of work which doesn't take as much experience and would waste more senior developers time. If AI takes away the lower skill tasks previously assigned to juniors, companies will be less inclined to pay for them.
Of course if nobody invests in juniors, where will the mid and senior developers of tomorrow come from? But that's a tragedy of the commons situation, few companies will wish to invest in developers who are likely to move on before they reap the rewards.
I think the tragedy of the commons problem for juniors has already existed for some time. Previously, companies were reluctant to hire juniors because they had a tendency to leave after a year or two, once you finished training them up. AI will just make the situation a lot worse.
Another reason companies hire juniors is because they cannot find/afford seniors. The demand that stems from this reason will increase over time when companies are not hiring "enough" juniors (because if we aren't hiring juniors we aren't making more seniors, so they become increasingly scarce and expensive).
Yes but then as all else this can easily be cyclic. Too few seniors to hire and they ask for ridiculous packages? Well lets train some of them in house, its not like the situation will explode overnight.
Weird times ahead, probably, but we will be fine, mostly.
perhaps the "attacks" relate to the condescending tone with which you relate your superior skills.
I think most people's amazement with lsp relates to the practical benefits of such a project _not_ being thrown away but taken that last 10% (which is 90% of the work) to make it suitable for so many use cases and selling people on the idea of doing so.
What's amazing about lsp isn't the polish, it's that we've hobbled our selves so much that a tool like it is even useful.
Only having exposure to the algol family of languages does for your mental capabilities what a sugar only diet does for your physical capabilities. It used to be the case that all programmers had exposure to assembly/machine code which broke them out of the worst habits algols instill. No longer.
Pointing out that the majority of programmers today have the mental equivalent of scurvy is somehow condescending but the corp selling false teeth along with their sugar buckets is somehow commendable.
> Pointing out that the majority of programmers today have the mental equivalent of scurvy is somehow condescending
You can (and many people do!) say the exact same thing in a different tone and with different word choice and have people nod along in agreement. If you're finding that people consistently react negatively to you when you say it, please consider that it might be because of the way in which you say it.
I'm one of those who would normally nod along in agreement and writing in support, but your comments here make me want to disagree on principle because you come off as unbearably smug.
>I'm one of those who would normally nod along in agreement and writing in support, but your comments here make me want to disagree on principle because you come off as unbearably smug.
Knowing non-algol languages won't make editor actions any less useful for algol-like. If anything, it'll just make you pretend that you don't need such and as such will end up less productive than you could be.
And editor actions can be useful for any language which either allow you to edit things, or has more than one way to do the same thing (among a bunch of other things), which includes basically everything. Of course editor functionality isn't a thing that'd be 100% beneficial 100% of the time, but it's plenty above 0% if you don't purposefully ignore it.
Kinda feels like this might be an instance of simply simply holding your tools wrong. What about ASM prevents incremental parsing and structured editing from being useful?
Some concrete examples for us lesser mortals please.
In the UK we have various types of pedestrian crossing, but they're an optional convenience. You can use them, or you can find a safe place and time to cross yourself.
I guess the jaywalking law is partly to avoid disrupting traffic, but the most efficient cross is the one when no traffic is present, and that's more likely in the middle of the road than at an intersection.
I just make incremental progress on a daily basis and don't consider stopping work for a week or two quitting. I work in spurts nd focus entirely on one project a day.
Thanks for the recommendation. I've enjoyed a lot of Sanderson books (although not all of them clicked with me) and I'm always looking for a new podcast
Right now, with the availability of open weights for cutting-edge models, it feels like this wave of technological advance is pleasantly decentralised however. I can download and run a model and tinker with things which at least feel like the seeds of such a future, where I _might_ be able to build things with my own interests at heart.
But what happens if these models stop being shared, and how likely is that? Reading about the vast quantities of compute deployed to train them, replicating the successes of the main players with a community of volunteers just seems an order of magnitude less achievable than traditional OSS efforts like Linux. This wave feels so tied to massive scale for its success, what do we do if big-tech stop handing out models?