>> Chatbots are far WORSE than traditional UI for everything. If some product has a chatbot functions it's the first thing I disable, if it's not possible to disable it, I avoid the product.
The chatbot we added to our B2B product is by far the most popular addition we've made this year. Our users are not tech-savvy, they use a lot of apps everyday and don't want to have to learn and keep up with just another UI. So they like being able to type their wants and needs in plain language (or speak it into their phone, if they are in the field) and get a plain language response back with embedded images and charts.
I think LLM chatbots are great for discovery. Better than CLIs, better than WIMPs. Where previously you'd go to a search engine to figure out what command to run or where to find that obscure tickbox, now the search is built right into the interface, and you can give it arbitrarily complex queries in natural language. It turns out there are a huge number of products for which discovery is the bottleneck.
The problem with chatbots is that they don't scale very well with you. A proficient user of your application knows where that tickbox is and can get to it in three clicks (or if you have shortcuts, one keyboard chord). But the chatbot still makes you type a whole (ish) sentence. If that's something you want to do 3 times a minute that application is now unusable.
The neat point of this is building up a nicer hierarchy of scripts that use your API that LLMs can call, having a place to build up UIs and shortcuts that use these, and a way of users making them with LLMs.
A flow of
* Conversation, which uses several API calls
* Put the API calls together as a runnable thing
* Give users a place to put this, now it's a fast way of doing their custom task
If you can make those testable, and make them shareable, this seems very powerful.
> So they like being able to type their wants and needs in plain language (or speak it into their phone, if they are in the field) and get a plain language response back with embedded images and charts.
If you think llms are so unreliable that there's no business context in which they pass a bar for being helpful enough to be used, I strongly recommend you go and speak to actual users about their actual problems and review the state of the art at the moment with LLMs.
LLM unreliability is no impediment in the application at which they most excel. Bullsh*t generation. There they sail way over the bar for e.g. most customer support.
You claon accuracy. LLMs results typically have dire repeatability i.e. same input gives different output. Hence anyone relying on tests for accuracy is kidding themselves.
It’s a different skill but one worth learning. Businesses have been building valuable things on top of non-deterministic processes for a very long time. Even much more classical AI can’t be tested in the same way.
You might not know how to do this, which is fine, but it’s not a new problem.
> Businesses have been building valuable things on top of non-deterministic processes for a very long time
Non-deterministic is one thing, unreliable and inaccurate is quite another. E.g. see monte carlo simulation
Regradless, I didn't suggest chatbot-generated bullsh*t wasn't valuable to businesses. On the contrary. The amazing productivity of these chatbots is every day making it more valuable to all types of bullsh*t-based businesses worldwide.
The chatbot we added to our B2B product is by far the most popular addition we've made this year. Our users are not tech-savvy, they use a lot of apps everyday and don't want to have to learn and keep up with just another UI. So they like being able to type their wants and needs in plain language (or speak it into their phone, if they are in the field) and get a plain language response back with embedded images and charts.
YMMV of course.