
Looking beyond accuracy to improve trust in machine learning - DKupfer2
https://blog.codecentric.de/en/2018/01/look-beyond-accuracy-improve-trust-machine-learning/
======
PaulHoule
I think explainability is a red herring.

Commercial AI is not about winning a Kaggle or ImageNet competition, it is
about meeting requirements.

For instance, if a image recognition model says that a black person looks like
a gorilla, it matters very little why the model says that. What does matter is
that you don't want to insult users, and that is an entirely different problem
than visual recognition.

A sign at my local gas station says they will not serve alcohol to "(i)
someone under the age of 21, or (ii) someone who is visibly intoxicated".

(i) should be implemented with ordinary code. Try to learn it and it will
learn that the age is 20.95 or 21.03333333333333. (ii) has to be learned but
it will never be perfect. Over and over again I hear this drumbeat about
interpretability and very little about the software engineering techniques
that it takes to do things reliably.

Similarly if it comes to giving credit fairly to minorities, this is something
built into the "real-world" evaluation process. You cannot know if the model
is "racist" at the beginning, but after you have made some loans then you can
prove the model is or is not biased by seeing if the default rates are the
same for different groups.

