
Epistemic Learned Helplessness (2013) - norswap
https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/
======
thaumaturgy
Happy to see this here again.

I suspect that argument from authority, like "goto considered harmful" and
Moore's Law, have entered the public consciousness in a warped popular form
that barely resembles the original.

In topics in which I am not a subject matter expert, I can _and should_ defer
to the actual subject matter experts, absent very compelling evidence to the
contrary. I'm not perfect at this, but I first read Epistemic Learned
Helplessness at about the right time several years ago and it re-appears
unbidden in my consciousness to hit me in the back of the head any time I
start thinking an expert might be wrong in their area of expertise.

Reluctance to do this because of misguided applications of the argument-from-
authority fallacy is probably one of the root causes of antivaxx, climate
"skepticism", flat-eartherism, and other hot political topics with a vocal
population who have convinced themselves that authorities are not to be
trusted. You can never successfully argue for instance that "AGW is probably
true because 98% of climate scientists agree with its foundations", because
the answer is always, "that's the argument-from-authority fallacy", even
though reliance on authority is the correct approach. What you inevitably get
instead is two or more laypeople clumsily trying to debate the nuances of a
subject that they have no academic background in.

~~~
taneq
I think the (or at least _an_ ) underlying issue is the conflation of
arguments with their respective fallacies. Saying "98% of climate scientists
agree that AGW is happening" IS an argument from authority, but it's not an
argument-from-authority fallacy, because they are in fact authorities and we
should listen to them. It's only argument-from-authority if there isn't
sufficient evidence that this authority is most likely correct on this count.

Just like pointing out a slippery slope may be entirely correct; it's only a
slippery-slope fallacy if there's no evidence that the slope is, in fact,
slippery.

~~~
nordsieck
> Saying "98% of climate scientists agree that AGW is happening" IS an
> argument from authority, but it's not an argument-from-authority fallacy,
> because they are in fact authorities and we should listen to them.

That is the claim that people make when they make an argument from authority.
That doesn't make it not a logical fallacy.

> It's only argument-from-authority if there isn't sufficient evidence that
> this authority is most likely correct on this count.

Sometimes this works (the Earth orbits the Sun) and sometimes it doesn't
(sanitizing hands/scalpels is actually a good idea between dissecting cadavers
and doing c-sections).

~~~
pdpi
> That is the claim that people make when they make an argument from
> authority. That doesn't make it not a logical fallacy.

This is the point where branding things as fallacies breaks down, and is very
much the point of the article.

Calling “argument from authority” on somebody’s point is one of the weakest
rebuttals you can provide, because you’re not really tackling the substance of
the argument.

In this case, “98% of scientists agree that AGW is happening” implicitly says
“and I have all the arguments provided by those 98% on my side”. Presumably,
if AGW _isn’t_ happening, their arguments are wrong somehow, and you should be
able to point out how. It’s not just appeal to authority, it’s saying “here’s
a big body of research, tell me where it went wrong”.

Ultimately, nobody is an expert on everything, and heuristic thinking has to
take over at some point. Overwhelming expert consensus is a damn good
heuristic to go by.

~~~
nordsieck
> Ultimately, nobody is an expert on everything, and heuristic thinking has to
> take over at some point. Overwhelming expert consensus is a damn good
> heuristic to go by.

Sometimes.

I don't know enough about physics to even really understand the standard
model. I still believe that it's pretty much true.

However this approach is a bit limited. There simply aren't experts in
macroeconomics that could convince me of anything on an appeal to authority
basis in that same way.

Ultimately, arguments are much cleaner and more convincing if you can simply
and directly argue the point. For those arguments that are not so amenable,
expert consensus may be a practical alternative, but at best it's a least bad
choice, not something to be lauded.

------
abakker
There is a connection in his last paragraph to entrepreneurship - I can't
place a finger on the exact phasing but I've seen before on HN an essay about
the need for an entrepreneur to be "all in" and genuinely swayed by their own
arguments. I've definitely observed that in successful business people around
me, and observed that I can't seem to do that to myself. While I'm not an
entrepreneur, I've often thought to start businesses, but I can't ever seem to
summon the self-certainty to commit to any ideas that I suspect are good and
would generate value.

Put other way, if you are susceptible to argument, you must be willing to
accept the risks, or unaware that there _are_ risks in committing fully to an
argument and a premise. This seems particularly suitable to startups where
oftentimes evidence of demand, market size, or growth is conspicuously absent
at the moment of inception.

~~~
yingw787
I like the notion of "strong opinions, weakly held":
[https://bobsutton.typepad.com/my_weblog/2006/07/strong_opini...](https://bobsutton.typepad.com/my_weblog/2006/07/strong_opinions.html)

""" Bob explained that weak opinions are problematic because people aren’t
inspired to develop the best arguments possible for them, or to put forth the
energy required to test them. Bob explained that it was just as important,
however, to not be too attached to what you believe because, otherwise, it
undermines your ability to “see” and “hear” evidence that clashes with your
opinions. """

I think these strong opinions change when there is information to do so, and
that's probably where the whole "stay focused and keep shipping" idea comes
from: collect soft feedback, hard metrics, and otherwise information by
providing a product-as-a-hypothesis. The more often you ship and measure, the
better your direction and your alignment with what people will pay for.

~~~
clairity
or to put it another way, the entrepreneur believes the conclusion as if it's
the premise and then tries to find the appropriate rationalization to support
it.

just like, say, a theist or an atheist, in-between which fits the agnostic
(the epistemically helpless person in this case).

------
oconnor663
> Arguments? You can prove anything with arguments.

This reminds me of a very interesting (and accessible!) article by the
mathematician Terence Tao about what major mathematical results tend to look
like, and how to be more skeptical of your own work:
[https://terrytao.wordpress.com/career-advice/be-sceptical-
of...](https://terrytao.wordpress.com/career-advice/be-sceptical-of-your-own-
work/)

> If you unexpectedly find a problem solving itself almost effortlessly, and
> you can’t quite see why, you should try to analyse your solution more
> sceptically. In particular, the method may also be able to prove much
> stronger statements which are known to be false, which would imply that
> there is a flaw in the method.

------
stcredzero
"Epistemic Learned Helplessness," is simply shorthand for, "We are all finite
and have an incomplete understanding for everything, yet we must still
pragmatically make our way in the world."

You don't have to instantly believe arguments. Sit back and gather data to
assess the predictive power of the model.

~~~
stupidcar
Except the problem the author identifies is that in many fields, such as
history, there's no way for the lay person to judge the predictive power of
competing theories, short of expending outsized effort to become an expert
themselves.

Since they cannot judge theories scientifically, and they cannot judge them
based on how well they're presented, they can only fall back on social
consensus and instinctive sense of weirdness. "Trust the academy", would seem
to be the more accurate shorthand.

~~~
shkkmo
> Since they cannot judge theories scientifically, and they cannot judge them
> based on how well they're presented, they can only fall back on social
> consensus and instinctive sense of weirdness.

Why judge at all? Or rather, why develop certainty when you lack the capacity
to judge accurately?

The answer is that certainty, even when it is unwarranted, is sometimes useful
and necessary. Certainty can also lead to major oversights and failures so it
is something we should use with caution and care. When assessing the
appropriate level of certainty we need to address not only our own level of
expertise, but also the utility (and risk) which adopting a particular
certainty provides.

However, I find this article rather misguided in its conclusions:

> And so on for several more iterations, until the labyrinth of doubt seemed
> inescapable. What finally broke me out wasn’t so much the lucidity of the
> consensus view so much as starting to sample different crackpots. Some were
> almost as bright and rhetorically gifted as Velikovsky, all presented
> insurmountable evidence for their theories, and all had mutually exclusive
> ideas. After all, Noah’s Flood couldn’t have been a cultural memory both of
> the fall of Atlantis and of a change in the Earth’s orbit, let alone of a
> lost Ice Age civilization or of megatsunamis from a meteor strike. So given
> that at least some of those arguments are wrong and all seemed practically
> proven, I am obviously just gullible in the field of ancient history. Given
> a total lack of independent intellectual steering power and no desire to
> spend thirty years building an independent knowledge base of Near Eastern
> history, I choose to just accept the ideas of the prestigious people with
> professorships in Archaeology, rather than those of the universally reviled
> crackpots who write books about Venus being a comet.

The author has discovered their lack of expertise in ancient history. Knowing
that, it seems to me that adopting a high level of certainty about ancient
history based on "mainstream" or "the academy", without a clear idea of the
utility of adopting that certainty, is epistemically irresponsible or at least
unnecessary.

------
pwinnski
This resonates with me strongly. I remember reading G.K. Chesterton writing
about how the most certain people in the world are locked up in the asylum,
absolutely certain of completely false things ("I am Napoleon") while the
rational among us grasp even our religious beliefs with a certain amount of
healthy uncertainty.

Political comment threads strikes me as a good example of people on every side
of every issue being 100% convinced of things that could not possibly all be
true.

------
tlb
Indeed, taking one idea at a time seriously leads to crazy conclusions. You
have to take all the ideas you believe seriously.

If you took seriously the one idea that "the purpose of human life is to
produce more humans" [SolaceQuantum below], you should be trying to have as
many children as possible. For a male with upper-middle class resources, 1000
children should be achievable. (You don't have to raise them, just "produce"
them.) A billionaire with strong organizational skills might manage 10^6
children.

This conflicts with the (unstated) idea that children should be raised well,
in a nurturing environment with good parenting. Also, that overpopulation is
bad. And that other people have an equal right to children. And ...

So I don't think you need epistemic helplessness. You just need a broad
perspective.

------
skrebbel
> _But to these I’d add that a sufficiently smart engineer has never been
> burned by arguments above his skill level before, has never had any reason
> to develop epistemic learned helplessness. If Osama comes up to him with a
> really good argument for terrorism, he thinks “Oh, there’s a good argument
> for terrorism. I guess I should become a terrorist,” as opposed to
> “Arguments? You can prove anything with arguments. I’ll just stay right here
> and not blow myself up.”_

Wow! This explains why so many antivaxxers and the likes are highly educated
people, often with engineering-like background. I never realized that until
reading this. They're smart enough to never have developed a thick enough skin
against convincing-but-wrong arguments, which means that once someone _does_
convince them of something (despite their smartness), they're (we're?) all-in.

Makes me wonder how vulnerable I am to all this. I'm sure in the target
demographic here. Maybe Osama would turn me into a suicide bomber too? Or into
an antivaxxer :-)

------
rossdavidh
I'd take his argument even further. I am pretty good at programming. When I
have written a program, I believe it's right.

I always test, before moving to production. Always. Even if it seems trivial
to prove beyond a shadow of a doubt, by argument, that this code is correct.
Always test, before moving to production.

~~~
abecedarius
We programmers get our noses rubbed in our own mistakes many times a day. Once
in a while we get to find out that code we were sure of for years was
embarrassingly wrong. You would expect people with so much experience of
fallibility to become especially aware of it, especially ready to have their
minds changed by argument, when it comes to unrelated even-more-complicated
topics like politics and economics... if you were an alien unexposed to human
nature.

------
ppeetteerr
> I don’t think I’m overselling myself too much to expect that I could argue
> circles around the average uneducated person.

Everyone thinks they are better than average ;)

~~~
Geimfari
Scott Alexander (SlateStarCodex) is most definitely above average when it
comes to writing and argument-making. He might be the most convincing writer I
know of.

[https://slatestarcodex.com/2017/10/02/different-
worlds/](https://slatestarcodex.com/2017/10/02/different-worlds/)

~~~
ppeetteerr
He's certainly convinced you ;)

------
SolaceQuantum
I'm curious to query the HN crowd: How do we mesh this learned helplessness on
subjects for which there are no definitive studies for? (eg. is the purpose of
a human life to produce more humans? How does one formally prove that, and how
does one prove those proofs are valid, and the organizations that prove the
proofs are valid are also valid?)

~~~
acbabis
I feel like people who study subjective topics like the value of human life
(philosophers) have already internalized "learned helplessness". It's
empirical-oriented people like us who have a harder time grappling with the
idea of not knowing everything

~~~
stcredzero
_It 's empirical-oriented people like us_

Translation: People who have spent so much time playing with highly abstracted
and greatly simplified models, they've lost their grasp on how messy and
complicated the real world can be.

~~~
majewsky
More than that, we're used to dealing with maps when the territory may be
inherently unmappable. It may not just be hard to produce an accurate map for
the entire territory, it may be impossible.

------
ggm
I dislike the name, but I can't think of a better one for the mental
condition.

I find the easiest way out of the "we're in a simulation" conundrum is to ask
why you would behave differently, ethically, inside a simulation to outside:
doesn't the fact you display consistency on your behaviour define you as you?

------
empath75
>That is, if you have a valid argument for something, then you should accept
the conclusion. Even if the conclusion is unpopular, or inconvenient, or you
don’t like it. He envisioned an art of rationality that would make people
believe something after it had been proven to them.

Lewis Carroll wrote an essay on this topic:

[https://en.m.wikisource.org/wiki/What_the_Tortoise_Said_to_A...](https://en.m.wikisource.org/wiki/What_the_Tortoise_Said_to_Achilles)

------
dang
A thread from 2015:
[https://news.ycombinator.com/item?id=10279864](https://news.ycombinator.com/item?id=10279864)

------
DigitalTerminal
Pascal’s Wager seems trivially countered by pointing out that there’s an equal
amount of evidence of the opposite hypothesis.

Or, in a more refined way: There is an effectively infinite number of theories
that have equal evidence (namely: none), and consequently you should give each
one only (1/infinity) attention, i.e. epsilon, i.e. basically zero.

------
Tuna-Fish
Note: The reason this is reposted now is that he used it as a leading post in
a sequence on the subject. The next post in sequence is "Book review: The
Secret of Our Success"

[https://slatestarcodex.com/2019/06/04/book-review-the-
secret...](https://slatestarcodex.com/2019/06/04/book-review-the-secret-of-
our-success/)

------
ForHackernews
This piece would be better without all the preemptive self-congratulations.

~~~
norswap
For how good he actually is, he's actually being quite humble.

And more importantly, the goal is not to make himself look good, but to relate
a point as he experienced it.

------
4gotmypw
I would guess that a lot of appearances can be sculpted, by hitting a
vulnerable person with priors.

I have no idea how frequently this happens, or how it resolves.

------
skadamou
This is essentially the argument I make to friends whenever someone brings up
some homeopathic remedy. Most of us are not smart enough or well educated
enough to really sift through this stuff and sniff out fact from fiction.
Lucky for us there is a whole field of trained professionals who (hopefully)
are.

At the same time though, I think it's worth remembering that the placebo
effect is real and measurable so if someone claims X, Y or Z cures their
headaches maybe they're not lying :-)

~~~
Gibbon1
> smart enough or well educated enough

A really good consideration is experience and observability. Those trump smart
and well educated most of the time. It's the difference between seeing
something and inferring it or simply repeating orthodoxy.

The Observability test is good when dealing with putative authority. Instead
of asking if they are right you first ask, is this something they can observe.
If not their opinion means a lot less.

A good modern example is climate change. Almost no one has the training,
experience and familiarity with actual data to have form a reliable opinion.
Without all three a persons opinion isn't worth much.

------
HNLurker2
Happy to see slatestarcodex classic

