I have an old apk of Firefox pinned on mobile. I do this because I genuinely believe that for my very limited usecases, the browser has become actively worse.
(Don't worry- I use the system browser for any site I don't fully trust.)
It’ll be priced slightly higher than the cost to actually run. But it’s still not clear what the real cost of the big models is. They seem very subsidised, but by how much?
It remains an unproven hypothesis. The revenue of the top 2-3 labs is still growing nearly exponentially, which is the ultimate piece of data that settles the question empirically for now. Benchmark scores aren't really proof. Benchmaxxing is possible, for example. Only revenue numbers (and gross margins) count.
The ultimate piece is not revenue but profit. At some point these enormous investments will have to be earned back. Good luck with that when open weight models are also continuously improving, have cheap providers and for many are already very usable.
The other point to make is that companies are starting to worry about the risks of externally hosted models.
This is at multiple levels if you have a remote API call as a key part of your workflow/software system.
1. Price risk - might be affordable today - but what about tomorrow?
2. Geopolitical risk - your access might be a victim of geopolitics ( seems much more likely that it used to be ).
3. Model stability/change management - you've got something working at the API get's 'upgraded' and your thing no longer works.
If you are running on open weight models - you are potentially fully in control - ( even if you pay somebody to host - you'd expected there to be multiple hosting options - with the ultimate fallback of being able to host yourself ).
This is funny because I was in the same situation, and actually used Claude to make a custom CAD program inspired by OpenSCAD :) https://fncad.github.io
You definitely need to have a strong sense of code design though. The AIs are not up to writing clean code at project scale on their own, yet.
This is a good example of what I mean! fnCAD appears to be a significantly buggier and highly incomplete version of OpenSCAD, where AI essentially grabbed the low-hanging fruit - albeit an impressively large amount of fruit - and left you with the hard parts. I fail to see how this solved any problems. Maybe it was an experiment, which is fine. But it's not even close to a viable CAD product, even by OpenSCAD's scruffy FOSS standards, and there's no feasible way to get it there without a ton of human work.
Not trying to denigrate the work here, as such. But this certainly didn't convince me that using AI to replace OpenSCAD (or any other major open-source project) is a good idea. The LLMs still aren't even close to being able to pull it off.
It solves all my problems! It's buggy and incomplete because it's "1.0 feature complete" for my own use. I've been doing lots of 3D printing with it, so it's definitely being dogfooded. File bug reports? I'm confident that features can be added as required, it's reasonably clean code.
I mean, to be fair, a one-user project is not ever going to be as bugfree as a tens-of-thousands-of-users project. That's just inherent and not an AI issue. If you judge AI projects by that standard, they'll always come up short. It's a sampling issue. An AI project that's gotten to a level where it competes with a traditional project will always be buggier and less feature complete and polished, because AIs speed up development. It will simply have seen far less, well, polish to get there.
I'd get confused if I was a LLM and you put my entire prompt in a text file attachment. I'd be like, "is this the user or is this a prompt injection??"
If you paste a long enough prompt into either GPT or Claude they turn it into an attachment, so it can happen. I think it's invisible to the model, but somehow not to the summarizer.
Errors compounding is a meme. In iterated as well as verifiable domains, errors dilute instead of compounding because the llm has repeated chances to notice its failure.
Well there can't be direct evidence, it's a private corporation and we don't know how big the model is. But you can look on Openrouter for hosters that offer free models with known sizes, where there's no brand and so no incentive to subsidize, and they don't look wildly bigger than OpenAI/Anthropic API prices.
edit: example: GLM 5.1, a 751B model, is offered for 0.6$/m in, 4.43$/m out. Scuttlebutt (ie. I asked Google's AI) seems to think that Opus 4 is a 1T/5T MoE model, so you can treat it (with some effort) as a 1T model for pricing purposes. Its API pricing is $1.55 in, $25 out, ie. 2x to 5x more than GLM. Idk what to say other than this sounds about right, probably with healthy margin.
I always avoided Ollama because it smelled like a project that was trying so desperately to own the entire workflow. I guess I dodged a bigger bullet than I knew.
reply