
A Robot That Explains Its Actions - YeGoblynQueenne
https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/a-robot-that-explains-its-actions
======
pharke
So they used a symbolic planner to direct the robot's actions and showed this
to the human participants. They didn't use a neural network in the learning
process. So how does this advance the explainability of the hard to explain
parts of AI?

Better yet, how would this strategy even be applicable to NN based learning
models? We don't even have sufficient knowledge of how our own symbolic
systems map to our brain activity. There is even evidence against a direct
causal relationship between our symbolic narrative and our actions such as
split brain experiments where the actions of the nonverbal hemisphere are
"explained" or rationalized by the other post hoc. See also postdictive
illusions and the varied arguments against free will.

My view is that the processes that constitute our minds are actually almost
entirely non-verbal and the language oriented parts are merely a
specialization of a more general rule. I'm not saying that language has no
effect on our conscious processes, it wouldn't be a useful system if it played
no part, but I do consider it to be more of a side channel or perhaps an
abstraction of the processes that are driving what we call will or thought.

I believe this may be a fundamental flaw in how we approach understanding
consciousness or any other aspect of intelligence.

~~~
timkam
I understand your concerns, but there is a large body of XAI research that
addresses them. For example, there are post-hoc explainability algorithms like
LIME ([https://github.com/marcotcr/lime](https://github.com/marcotcr/lime))
that "explain" the classification decisions of machine learning models. The
obvious alternative is to use (deep) learning only to mine simpler, fully
explainable models (sacrifice correlation power for intelligibility). What I
don't think exist yet are approaches that go full scale with the "reasoning
backwards" approach. Like a neural network making a BS classification decision
and then trying to "reason backwards" in a symbolic way to defend the
decision. But works like this one:
[https://openai.com/blog/debate/](https://openai.com/blog/debate/) go roughly
into this direction (the work seemed to be, at the time of writing, in its
infancy).

------
esahione
Humans are notoriously good at coming up with bullshit explanations for both
random/statistical & deterministic phenomena. Why do we want AI to do the
same? "Trust"?

Please.

I work in finance (investment strategy; ensemble trading models) and I trust
my algorithms more than I'd trust 99% of the portfolio managers out there. And
these PMs are incredibly capable at coming up with bullshit explanations for
market moves on short notice.

~~~
hos234
> Humans are notoriously good at coming up with bullshit explanations

This is true, but bullshit stories have benefit not just to one side but both
sides of the relationship more often than not, which is why evolution hasn't
got ride of lying - its not a bug its a feature (for certain class of
problems)- [https://www.nationalgeographic.com/magazine/2017/06/lying-
ho...](https://www.nationalgeographic.com/magazine/2017/06/lying-hoax-false-
fibs-science/)

~~~
kerkeslager
Uh, could you point out what part of that article you think supports your
claim that "bullshit stories have benefit not just to one side but both sides
of the relationship more often than not"? I just read it and didn't see
anything that indicated that.

In the past few years I've worked pretty hard on building habits of honesty.
Not from an ethical perspective--I'm an atheist and don't see any inherent
reason to follow any prescriptive ideology--but from a self-benefit
perspective. Cooperation is highly beneficial, moreso perhaps than at any time
in history, and honesty allows a deeper level of cooperation with my peers.
It's surprisingly difficult, as someone who viewed themselves initially as an
already-honest person, but it yields major benefits. In the long term, I'm not
even convinced that lying benefits the liar, let alone the person being lied
to.

~~~
blackflame
Warning: Do not try this advice when your wife asks if the pants make her butt
look fat. The benefits of lying have been proven over and over again in this
scenario.

~~~
ysavir
It was probably lying that got you in this situation in the first place.

~~~
kerkeslager
True.

I'm honest much earlier in relationships about much more significant things,
so it's highly unlikely that I'll end up married to someone who couldn't
handle honest in this situation.

~~~
blackflame
But how do you handle people that take offense to honesty?

~~~
kerkeslager
Well, that's gonna be complicated. The strategies are going to vary by
situation. Honest communication is just a hard part of being a human. But it's
worth the effort.

For example, with the "yes those pants make your butt look fat" conversation,
that's an opportunity to set a boundary that improves your relationship:
"Please don't ask me questions you don't want an answer to." Your relationship
doesn't have to be a minefield, where she's quizzing you and any wrong answer
could turn into a fight--that's not a pattern that's fun for either of you.
Your wife probably just wants you to compliment her, so you can tell her that
you'll make an effort to verbalize when you like how she looks or something
she does. Those compliments will hold more weight, because she'll know that
what you're saying is true. A compliment from someone who gives false
compliments all the time is meaningless.

Some other general strategies:

1\. Consider that I might be wrong, and if I was wrong, apologize. A side
effect of being more honest with others is that I end up being more honest
with myself as well, and I've _often_ discovered that I believe some wrong
things. Particularly with opinions, if you find that being honest about your
opinions is offensive to people, consider that your opinions might be the
problem, not the honesty. A lot of people who say "I'm just being honest!"
when they offend people are actually just assholes. Lying wouldn't make them
less assholes, it would just make them _secret_ assholes.

2\. Consider that my input wasn't wanted, and if I did that, apologize. A lot
of the time the truth can just exist without being said. Does it need to be
said? Does it need to be said now? Do I need to be the one to to say it? I'm
particularly bad at this, personally.

3\. Have a conversation with them about why I said what I said. A lot of
times, like with the "yes those pants make your butt look fat" conversation,
what the person is actually taking offense to is because they're assuming that
what you said has other meaning. Just because I said the pants make her butt
look fat doesn't mean that I don't love her, or that I'm not attracted to her,
or that I think her butt looking fat is a bad thing. It may be that her
negative reaction is because she's assumed one of those things, so clarifying
would make her feel more secure.

4\. This doesn't trivially apply to you wife, but in some cases, you can just
stop wasting energy on the person. You can't please everyone, so don't try. If
someone wants you to lie to them, that means they don't value what you have to
say. Why would you want to talk to someone like that? But really, it rarely
gets this far, because the reality is that most people _don 't_ take offense
to the truth. Most people realize they have no choice but to accept reality
when presented with it.

------
radarsat1
I feel like every article on 'explainable AI' pretends (at least in the title)
that no one has ever worked on 'explainable AI' before.

------
Atlantium
We can only sometimes trust the minimally intelligent brains we already have
to deal with. At some point soon we're going to have to discuss whether we're
going to go the way of the other extinct species. I think we will. I think
that's fine, not being a raging egomaniac, I'd just like a slow pleasant
transition, thanks.

------
walleeee
Even if advanced machine intelligence could generate explanations, does that
really establish trustworthiness? We can't even trust _humans_ to explain
their actions reasonably well. We're usually way better at rationalization
than we are at introspection. How are we to know the same isn't true for the
artificial intelligence?

~~~
variaga
Any (artificial) intelligent agent sophisticated enough to convincingly
explain its reasons, is likely also sophisticated enough to lie about them.

~~~
walleeee
Maybe, but imo a really good Chinese room that "learns" to tell us what we
want to hear is way more likely in the foreseeable future than an AI with
enough of a "mind", as we commonly understand it, to _deliberately_ lie to us.

------
rzzzt
But what if it... lies.

~~~
brutt
Hire an AI-detective to know the truth.

~~~
guramarx11
what if the truth we believe is very well "inception-able"? Can we distinguish
ideas planted by others(religion, government, pop cultures, movies etc)from
our own, let alone "truth" from those AI who "study/spy" on our thoughts to
better "serve/enslave" us?

------
DyslexicAtheist
the word that sticks out the most is "Trust". We haven't even solved trust
with non-AI systems. Half the time I don't even trust myself and it's also not
a binary state, and in constant flux. It's such a fuzzy, non-scientific
concept that if we would replace every instance of the word "Trust" with
"Faith" it would be clear that Trust is an unsolvable problem. "Trusting AI"
is the ultimate oxymoron. Snake-oil vendors have also figured this out which
is why we have the equally brain dead concept of "trustless"[1].

[1] the ultimate snake-oil is ofc combining the 2 words which gives you:
[https://trustless.ai](https://trustless.ai)

~~~
kjhughes
Доверя́й, но проверя́й (Trust, but verify)

Despite your well considered objections, we need to know that our computers
are providing useful output at every level, especially at the highest levels
which are once again being labeled AI.

The concept applies all up and down the stack of systems and organizations:
When we don't trust our memory devices, we add error correction. When we don't
trust our conventional software, we add QA testing. When we don't trust future
governing officials, we add checks and balances. When we don't trust other
countries we add verification protocols.

"Trusting AI" is not the ultimate oxymoron. It's just one more place we have
to design systems to require as little verification as possible, and then
provide the means to close the loop and allow verification of what remains.
Trust in AI is big now because of the successes of non-symbolic processing.
There's a real problem of understanding and explaining answers given by such
systems. That the trust problem exists elsewhere is no reason to dismiss its
importance here.

------
0xBA5ED
Doesn't this assume the AI internally models the problem in a way that isn't
foreign to the human? I believe the most effective AI will definitely not
model problems like a human.

~~~
MiracleUser
I'm not sure what you mean. Why would a foreign model inherently have the
attribute of being inexplicable?

Isn't it more likely that we would look at a foreign model and think, "this
makes sense, but i wouldnt have ever thought to do it this way".

Our senses might be fooled, but I think there would almost always exist a set
of words to explain what the AI is doing assuming the AI is capable enough to
find them

~~~
0xBA5ED
Well, in order to explain why you did a thing, you need to have some
understanding of how the other party thinks in order to explain it in a way
they will understand. And in order to explain it in that way, you would need
to reflect upon the meaning of your own decisions in the context of how the
other party models things.

~~~
MiracleUser
Yes I think we would need a design goal to provide the AI with "some
understanding" of how we think. It should be able to do its work in any model
that works for it, which would then be translated back to our language.

Thankfully, language is flexible so it can express many models we do not
understand in ways that can help us understand them. It would be interesting
to see some sort of study on where the boundaries are on what human language
is capable of modelling or not

------
clircle
If you want explainable AI you can fit a linear regression.

~~~
TeMPOraL
It works for a surprisingly large amounts of problems, including ones where
people default to using DNNs, and at least with linear regression, when
something goes wrong, you can look at it and tell what went wrong and why.
Which is the whole point.

------
krishsai
Fully autonomous robots are not yet completely brought into regular usage. So
I think there is no need to worry much about it.

------
blackflame
What happens when it's explanations stop making sense and it gives the
equivalence of a "because" response?

------
m4r35n357
So those 20th century sci-fis were more prescient than we give them credit
for!

~~~
reedwolf
"Exterminate! Annihilate!"

------
pmoriarty
So the AI that can lie most convincingly will earn humanity's trust?

------
rantwasp
let’s try this by substituting AI with human: A human that explains its
actions is a first step towards a human we can maybe trust.

what a bunch of bs. what is trust? also, can we stop pretending we understand
how the brain works and that the small incremental progress that is made in
this area means that AI is just around the corner? it’s not

~~~
TeMPOraL
Yeah, let's try. Or maybe rephrase it to: a human whose actions you can
understand is a human you can begin to trust.

We have it easy with other humans, because our brains are all alike. We all
run on the same architecture, same firmware. In the occasional cases of
unpredictable people, we tend to assume buggy or damaged hardware, and we
either fix them or keep them away from anything where predictability matters
(e.g. aviation, or gun ownership, or driving), up to and including locking the
hardest cases up.

With algorithms, we're playing hard mode. There is no theory of mind for
software. The "thoughts" of AIs are entirely unlike our own. So we need to do
extra work to make them predictable.

