
Reinforcement Learning Under Moral Uncertainty - hardmaru
https://arxiv.org/abs/2006.04734
======
taneq
As I see it, morality is inherently underdefined and subjective. All our
current attempts at creating rigorous moral frameworks lead to intuitively
immoral behaviour under some circumstances. Combining them all together with
some sort of blending function to avoid the weak points might avoid the
kookiest of unintuitive behaviour but I don't think it'll solve the inherent
fuzziness of actual morality.

~~~
abtinf
> All our current attempts at creating rigorous moral frameworks lead to
> intuitively immoral behaviour under some circumstances.

Objectivism does not.

~~~
Nasrudith
I'm not certain if that was a joke or not. Calling altruism immoral certainly
qualifies as intuitively immoral behavior.

Even if one excises Ayn Rand's infamous politics and personal hypocrisies, and
even everything which doesn't follow from claimed principles it isn't very
workable. Rationality is a measure of sanity more than morality even if there
may be some overlap in that a perfectly rational actor wouldn't display
gratitutous cruelty.

Granted everything we have is also imperfect. Pure intuition can easily fall
into nonsensical superstition and prejudices like Pythagorean hatred for beans
and irrational number denial and rationality alone. There are plenty of
"serial killer organ thief doctor utilitarianism" arguments but those first
order calculations fail to consider the impact of it inevitably becoming a
known thing in society. Being incentivized to shoot a doctor if you wind up
alone with one is a larger net harm to society in addition to the obvious
deterrant to being a lone traveler. Those silly exercises aside rationality
alone is insufficient it lacks goal definition. One may rationally persue
extinction of all life in the universe or maximizing human lifespan.

If a sub-goal is linked to another goal it can be found to be irrational if
inconsistent, if painting your dog won't make your crush love you dog-painting
isn't a rational goal. But if you want to just paint your dog for the sake of
doing so (please don't) it can't be said to be less rational than another goal
even if it is more or less obtainable.

------
vga805
The Stanford Encyclopedia of Philosophy recently (March 2020) published an
entry on Computational Philosophy, a newly forming field that employs "the use
of mechanized computational techniques to instantiate, extend, and amplify
philosophical research."

[https://plato.stanford.edu/entries/computational-
philosophy](https://plato.stanford.edu/entries/computational-philosophy)

------
hardmaru
This paper is interesting enough, but the authors also released code for their
environments and experiments used in the paper: [https://github.com/uber-
research/normative-uncertainty](https://github.com/uber-research/normative-
uncertainty)

