This is just the AI grift and uneducated hypebros. There is zero reason to use an LLM as a classifier. But AIbros with zero technical knowledge will push for it anyways even if it is 1000x slower and 1000x more expensive to run.
You can can't build an SVM-classifier for a ticketing system without domain data. You can ask an LLM to do it, and it will do it very well.
With the LLM you don't even need a data scientist to train the model, you just need someone who can connect to APIs. And the LLM is fine with adding and removing categories. No retraining.
It makes the initial faces incredibly fast. Then it is slow and expensive to run, so once the system settles, it becomes the target for model optimization, e.g, a dedicated, or semi-dedicated model.
Yeah but here's the thing: Ease of setting up and how low that bar is, this is not the most important factor for most companies. Also, yes, you need data to do this with an LLM. In fact the LLM was trained on that data for you beforehand. This is not an argument because you can also buy a pretrained classifier.
You empathize the set up cost but ignore that the model is in the magnitude of a million x in size and compute cost of a classifier model. That's like comparing a CPU with an IC
I mean just have a coding agent do the whole process from start to finish. Tell it you want to seed an SVM from LLM runs on ticket data. I can bet that it'll do it all for you without much input from you at all.
I think AI is making people forget that code can be used for automation.
I see coworkers using AI for tasks like "visit an API endpoint, download a file, put it in S3".
This is so stupid. Okay if you want to use AI, but use it to write python code to do that, why pay in tokens every time you want to run that task, it makes no sense
I see this a lot among non-tech folks, who have a pretty thin understanding of the differences between "code" and "an agent". But do you see this among tech folks as well?
Your S3 case tho I think is a bit misplaced. You're right that if all your keys are configured correctly (API key, S3 key, etc) that a script to do that is like 5 lines in Python (once you get your venv going or use uv to setup the project lol), but tell an agent to just do the thing, and it will make sure that your API keys are in place and that you have access to the S3 bucket you're trying to write to and that the whole thing is in a venv, and then just report back that it's done.
Is it wasteful? Sure. For years I used to feel that way when I saw someone using a messaging app to message a friend across the hall in a loud room or when I saw someone taking a phone picture of their computer monitor and sending it. In the former case your packets are literally going from your phone to a central server and then onto someone else's phone in the same room. But, well, convenience wins.
I can spin up a classification solution with an LLM in ca. 5 minutes that is likely to do an okay job. To build a custom classifier, I need lots of training data first.
That is the reason.
Once you then have a system that is used enough, it starts making sense to look to replace that with a leaner classifier, but LLMs have a place in vast number of places where we'd previously never use a classifier because it was too much effort.
All of the replies talk about how easy and fast it is to set up. That's literally the ONLY reason you would do it with LLMs. If you consider any other factor, it makes zero sense.
> but LLMs have a place in vast number of places where we'd previously never use a classifier because it was too much effort.
Absolutely!
But that doesn't rule out that there are AIbros that will tell you to use an LLM, even if the task could be done with a static analyzer with a few rules. Use an LLM because it's AI!
Not really, although when a certain personality type dominates a scene, people tend to conflate the two. Especially when they are louder than the rest.
There are absolutely good reasons to use smarter things for classification. Svm with tdf idf for example is super fast but will make very bad classifications at times in ways entirely unfixable with that system (thinking back to bats being an acronym for a theology degree classification or similar)
This is a bit too radical and misses some nuance. In order to start building classifiers you at least need some ML domain knowledge, like what to google at least. Meanwhile, point anyone at the jev/open-jev-clones API and have them start classifying in <10 minutes. There's some value there. (same used to be valid for LLMs before jev became a thing. You can use llms as zero-shot classifiers with the right API wrapper, w/ no domain data).
