You're being downvoted, I suspect for being a tad hyperbolic, but I think you are raising a really important point, which is just the ever more gradual of removing a human's ability to disobey the computer system running everything. And the lack of responsibility for following computer instructions.
It's a tad far-fetched in this specific scenario, but an AI summary that says something like "cancel the subscription for user xyz" and then someone else takes action on that, and XYZ is the wrong ID, what happens?
It's a tad far-fetched in this specific scenario, but an AI summary that says something like "cancel the subscription for user xyz" and then someone else takes action on that, and XYZ is the wrong ID, what happens?