LLMs need significant optimization or we get significant improvement on computing power while keeping the energy cost the same. It's similar with smartphone, when at the start it's not feasible because of computing power, and now we have one that can rival 2000s notebooks.
LLMs is too trivial to be expensive
EDIT: I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate
Looking up a project on github, downloading it and using it can give you 10000 lines of perfectly working code for free.
Also, when I use Cursor I have to watch it like a hawk or it deletes random bits of code that are needed or adds in extra code to repair imaginary issues. A good example was that I used it to write a function that inverted the axis on some data that I wanted to present differently, and then added that call into one of the functions generating the data I needed.
Of course, somewhere in the pipeline it added the call into every data generating function. Cue a very confused 20 minutes a week later when I was re-running some experiments.
Are you seriously comparing downloading static code from github with bespoke code generated for your specific problem? LLMs don't keep you from coding, they assist it. Sometimes the output works, sometimes it doesn't (on first or multiple tries). Dismissing the entire approach because it's not perfect yet is shortsighted.
Cheaper models might be around $0.01 per request, and it's not subsidized: we see a lot of different providers offering open source models, which offer quality similar to proprietary ones. On-device generation is also an option now.
For $1 I'm talking about Claude Opus 4. I doubt it's subsidized - it's already much more expensive than the open models.
Thousands of lines of perfectly working code? Did you verify that yourself?
Last time I tried it produced slop, and I've been extremely detailed in my prompt.
Well recently cursor got a heat for rising price and having opaque usage, while anthropic's claude reported to be worse due to optimization. IMO the current LLMs are not sustainable, and prices are expected to increase sooner or later.
Personally, until models comparable with sonnet 3.5 can be run locally on mid range setup, people need to wary that the price of LLM can skyrocket
You can already run a large LLM (like sonnet 3.5) locally on CPU with 128GB of ram which is <300 USD, but can be offset by swap space. Obviously, response speed is going to be slower, but I can't imagine people will pay much more than 20 USD for waiting 30-60 seconds longer for a response.
And obviously consumer hardware is already being more optimized for running models locally.
Imagine telling a person from five years ago that the programs that would basically solve NLP, perform better than experts at many tasks and are hard not to anthropomorphize accidentally are actually "trivial". Good luck with that.
There is a load-bearing “basically” in this statement about the chat bots that just told me that the number of dogs granted forklift certification in 2023 is 8,472.
Sure, maybe solving NLP is too great a claim to make. It is still not at all ordinary that beforehand we could not solve referential questions algorithmically, that we could not extract information from plain text into custom schemas of structured data, and context-aware mechanical translation was really unheard of. Nowadays LLMs can do most of these tasks better than most humans in most scenarios. Many NLP questions at least I find interesting reduce to questions of the explanability of LLMs.
"hard not to anthropomorphize accidentally' is a you problem.
I'm unhappy every time I look in my inbox, as it's a constant reminder there are people (increasingly, scripts and LLMs!) prepared to straight-up lie to me if it means they can take my money or get me to click on a link that's a trap.
Are you anthropomorphizing that, too? You're not gonna last a day.
I didn't mean typical chatbot output, these are luckily still fairly recognizable due to stylistic preferences learned during fine-tuning. I mean actual base model output. Take a SOTA base model and give it the first two paragraphs of some longer text you wrote, and I would bet on many people being unable to distinguish your continuation from the model's autoregressive guesses.
It still doesn't pass the Turing test, and is not close. Five years ago me would be impressed but still adamant that this is not AI, nor is it on the path to AI.
Calling LLMs trivial is a new one. Yea just consume all of the information on the internet and encode it into a statistical model, trivial, child could do it /s
LLMs is too trivial to be expensive
EDIT: I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate