https://en.wikipedia.org/wiki/Therac-25 is the classic case study. 6 injuries due to removing hardware interlocks and replacing them with a software interlock implemented with a flag that was set with an increment instruction instead of just storing "1". (This works fine 255 times! The 256th time has unexpected results.)
As for legal implications... there were basically none. Everyone is sure to include the "NO WARRANTY" disclaimer on their software now. People still build machines without hardware interlocks. People still use programming languages with integer overflows.
If your argument is that users of mathematics or computers are responsible for their actions, I agree with you. My comment is about the researchers arguing (in effect) that no one should have a computer because they might do Therac-25, which I don't agree with at all.
I agree with you. People are worried that an AI might say "do a therac-25" but forget that it might also say "don't do a therac-25". I think it's averages out to neutral. Nobody bans Home Depot from selling a hammer because you might hit your thumb with it. We accept thumb injuries because even while people are out their thumb for a few days, society as a whole gets more work done with hammers than without. I think AI will probably find a similar role. Some idiot is going to make a bot that calls people and makes them buy it gift cards. Someone else will cure cancer. So it goes.
As for legal implications... there were basically none. Everyone is sure to include the "NO WARRANTY" disclaimer on their software now. People still build machines without hardware interlocks. People still use programming languages with integer overflows.