
Do computers make us more safe or less safe? - zdw
https://blog.computationalcomplexity.org/2020/07/do-computers-make-us-more-safe-or-less.html
======
Barrin92
>" _The drone on its own was going to bomb a car. But the human noticed that
there were red roses on the car, so it was a wedding couple, not a terrorist.
If a human had not been involved the drone may have killed an innocent just
married couple!_ "

 _> This scene bothered me. It could EASILY be the other way around: the human
wants to bomb and the drone (which has better vision) notices the roses. Or
there may be many other ways that a computer could be BETTER than a human_

The issue with the rose isn't better vision but better contextualisation,
creating a plausible narrative around what is seen that changes how to act.
That's something at which humans vastly outperform machines, or even more
strongly a skill machines don't at all possess. It could not be 'EASILY' the
other way around, at least at the current levels of technology.

A human can always consult a machine for better raw vision or image
classification, but no machine right now can substitute a human for these high
level judgements. Maybe in a few decades when we have AGI or whatever, but
then the distinction between machine and human is moot anyway.

------
gpcz
Most safety standards that address software's contribution to hazards use some
notion of "controllability" to determine risk. For example, ISO 26262
explicitly calls it controllability [1], and it directly contributes to how
rigorously the software functionality has to be tested.

MIL-STD 882E uses "Software control categories" that determine the risk
introduced by software [2, page 15-17]. Since those definitions are hard to
track, they also provide a table near the end that tries to be a little more
specific [2, page 96].

In other words, the safety community already takes into account the need for
human or mechanical intervention when it comes to risks introduced by
software.

[1] [https://neweagle.net/how-iso-26262-2018-update-affects-
you/](https://neweagle.net/how-iso-26262-2018-update-affects-you/) [2]
[https://www.dau.edu/cop/armyesoh/DAU%20Sponsored%20Documents...](https://www.dau.edu/cop/armyesoh/DAU%20Sponsored%20Documents/MIL-
STD-882E.pdf)

------
grawprog
Personally, I don't think computers make things more or less safe than any
other tool or machine does. They fix some safety problems, while adding
entirely different ones. Computers are tools like any other, they can be
uses.safely or unsafely. It's the application of the tool, rather than the
tool itself that is safe or unsafe.

Before anyone brings up guns, I realize they're an example of an inherently
unsafe tool, they can be used safely, but this example of a tool designed for
murder is not what I'm referring to. I meant more general purpose tools, which
computers are. Though, as mentioned in the article, computers can be used for
murder, it's not their primary intended purpose the way guns are.

------
imheretolearn
[https://news.ycombinator.com/item?id=23969257](https://news.ycombinator.com/item?id=23969257)

------
smitty1e
Really depends upon the scope of "us", no?

For small values of "us", some bad error handling or a dodgy sensor can be
fatal.

For larger values of "us", small amounts of "us" debris are so much stochastic
noise.

MORAL: strive to be the one who gets to manage the definition of "us".

