Part of me is fascinated by this and thinks it's a great idea, then the cynicism kicks in and I start thinking of how frustrating this could be when Comcast finds it.
Right. We're going from having support bots that give nonsensical or "please reformulate your question" answers to support bots that just make up plausible, but completely wrong answers.
I am with you, but There are ways to mitigate that, one is to make a script that check the URL output for this particular case, another way is to instruct the bot to translate user input to queries and then check whether they can run or not. But yeah, using those bots without constant human supervision seem like a terrible practice
I completely agree that the "grounding" problem is solvable. It shouldn't take too long before we have bots with much more reliable answers.
My worry, from working in the chatbot support industry, is that companies are not aligned to help you solve problems faster. If you get in touch with a real human being about a product issue, they may have sympathy with your issue, or get tired of you, and just refund or whatever.
Meanwhile AI support could have you keep trying less and less likely to work solutions until the sun dies. Companies don't always want you to reach the true solution, and smart AI can act as a wall between you and it.
Of course, whenever insentives are aligned, better AI may significantly improve the quality of support.
Not if you fine-tune your bot on your knowledge based and past solved issues. If it is a known issue or fact, the bot could solve it. If it is outside known solutions, the bot should decline. This can be trained as well.
The optimist in me thinks it could be a good thing. I imagine the vast majority of support queries that a big company gets are from non-technical users who need help with trivial things, and ChatGPT has been excellent, in my experience, at explaining things simply with the right caveats. If the implementation makes it easy to override the bot and get to a human, it could unclog the support lines from easily solvable requests sufficiently to be a net positive.
