>Separately we might also ask whether we even want to solve the problem of ethics.
The article's author presents a good example: for AI ethics. If we want to prove "an AI following this system won't kill all humans (or has e.g. less than 0.1% chance of killing all humans)", we need to be able to formally reason about that system.
>In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]
By that logic, why bother with any of the higher, more abstract fields of mathematics?
There are two issues here though. Firstly, what is the right thing to specify? (This is an ethical proposition, even for fairly simple agents.) Secondly, how can this specification be reliably encoded in the AI's world model? (This gets technically more more challenging the more complex the agent is.)
The article's author presents a good example: for AI ethics. If we want to prove "an AI following this system won't kill all humans (or has e.g. less than 0.1% chance of killing all humans)", we need to be able to formally reason about that system.
>In other threads you say you aren't interested in proving which system of ethics is most applicable to human endeavours or anything like that. At this point we get into "higher order truths about chmess" territory, as in - why bother? [1]
By that logic, why bother with any of the higher, more abstract fields of mathematics?