Different kind of ethics:
"It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter." (lost the source of the quote)
To understand how these problems can be addressed, it is instructive to consider an analogy. Imagine, for a moment, that somewhere in the basement of the Pentagon there are two computers. They are identical hardware, and they run identical software; they are as alike as two peas in a pod. On each of them, there is an executable procedure called "DestroyBaghdad". On one of the machines, executing this procedure does nothing. On the other, the effects of executing the procedure are disastrous, at least for the citizens of Iraq. The only difference between the two machines is a single wire coming out of the back of the second machine.[1]
The point, of course, is that nothing inherent to a computational language is dangerous; danger is posed only insofar as the effects of executing statements in a language include "side-effects" in the external world. If all the "wires" connecting a computational language to the outside world are cut, then executing programs in that language is harmless. (An exception to this claim is that such a program can still tie up the CPU, virtual memory, or swap space on a machine by going into an infinite loop; this can be handled via resource limitations.)
[1]It should be noted that no ethically-trained software engineer would ever consent to
write a "DestroyBaghdad" procedure. Basic professional ethics would instead require
him to write a "DestroyCity" procedure, to which "Baghdad" could be given as a
parameter.
