
Misunderstanding Dunning-Kruger - imartin2k
https://theness.com/neurologicablog/index.php/misunderstanding-dunning-kruger/
======
ternaryoperator
> if you think it only applies to other people (which itself, ironically, is
> part of the DK effect) then you miss the core lesson

I have observed in conversation that invariably if someone brings up
Sturgeon's Law [1] either by naming it or quoting it ("90% of everything is
crud|shit|garbage"), they invariably place themselves in the 10%. As if
awareness of the law somehow insulates the speaker. That seems to be the case
with D-K, per this article.

[1]
[https://en.wikipedia.org/wiki/Sturgeon%27s_law](https://en.wikipedia.org/wiki/Sturgeon%27s_law)

~~~
taejo
I have never heard Sturgeon's Law invoked to place the speaker _anywhere_ :
I've only heard it as a _rebuttal_ of a dismissal. If someone haughtily says
they don't watch TV, and cites three shows as reasons why TV is terrible,
well, yes 90% of TV is crap, but so is 90% of opera, 90% of stage plays, 90%
of tweets. If we judge _anything_ based on the median, it's crap, so it makes
more sense to judge things based on their best examples.

And if 90% of invocations of Sturgeon's Law are crap, well, we should judge
them by the best examples too :)

~~~
philipov
I hazard to propose that twitter contains closer to 99.999% crap. 90% might be
the central point, but I think there is still variance between different
mediums. Comparing nominal values without knowing the distribution can be
dangerous, but that's why we have units of deviation, such as zscore.

------
Aeolun
> The best way to summarize the data is to say that the difference between
> self-assessment and performance increases with lower performance below about
> the 80th percentile, with slight underestimation of performance for those
> above the 80th percentile (or a test score of about 70%).

It may just be me, but this does not seem like the best way to ‘summarize’ the
data for general consumption.

The original “dumb people always think they’re smarter than they actually are”
that the author takes issue with seems more sensible.

We can’t expect everyone to be in the top 20%.

~~~
dahart
> dumb people always think they're smarter than they actually are

This is a really bad summary of the original D/K paper, both misleading and
incorrect. Maybe this illustrates nicely why the article was needed. They
didn't knowingly test any dumb people; it wasn't a random population sample,
it was all Cornell undergrads volunteering for extra credit. The results are
relative, not absolute, they were being asked to rate themselves against the
population (their classmates), not actually rate their ability. That involves
guessing how good others are. We don't know how good they all were in absolute
terms, only their relative scores. The results did not conclude anything close
to "always", it only showed an average bias. The results show that, on the
tasks they tested, the people with the best skills under-estimated their
scores relative to their peers.

~~~
vectorEQ
the results are indeed relative. the smarter one gets the same thing still
applies. it's a theory not to point at others and say 'he's on mount stupid'..
but rather one to be wary of yourself being on this curve, which in some sense
or aspect one always is, as one is never a master of everything in their
lives. it's a theory about evaluation of self only applicable in that domain.

------
ncmncm
This article was surprisingly OK.

About all it omits is that people at the bottom of the scale generally didn't
even understand the questions they were asked, so their answers were really
all over the place.

But the key takeaway -- the D-K effect is about _you_ , much of the time, even
if you are expert about some things -- is usually neglected. D-K is about
everyone: The more certain you are, the less likely it is you have reason to
be.

~~~
smsm42
But, judging from the graph in the article, the top quartile actually _under_
-estimates their ability? Only lower 3 quartiles overestimate it, so for
experts it doesn't seem to be true? Unless, of course, I misunderstand DK
effect completely and am not aware of it...

~~~
ChrisSD
Yes, the point is that in your area of expertise you are likely to be more
humble relative to your actual ability (but not too far off, on average).

However outside of your specific area(s) of expertise your confidence in your
ability is likely to far exceed your actual ability (on average).

So experts are still susceptible to DK effect for areas that lie outside their
expertise. Even (or perhaps especially) if they think their expertise gives
them special insight into another field.

~~~
jacobmoe
This makes sense to me. If you're an expert you are comparing yourself to
other experts in your field. On the other hand, if you know anything about a
topic but only interact with people who know nothing at all about it, you
would rate yourself highly.

For example, I read a lot about modern physics but am not a physicist. I would
give myself an above average understanding but would likely fail an exam.

------
theothermkn
> [T]hose who endorse anti-vaccine views were far more likely to estimate that
> they knew as much as experts or doctors, when in fact their knowledge level
> was extremely low.

> Dunning believes that ability to self-assess is also key to the effect.

> As you gain knowledge, this confidence grows.

> Specifically, narcissistic personality seems to put DK on steroids.

That collection of quotes feeds my suspicion that extreme examples of the DK
effect and conspiratorial thinking (For example: vaccine-denial, birtherism,
racism as "expertise" on how the outgroup "really is," and so on.) have in
common the fact that they give those inflicted a much-needed sense of control
over an uncertain universe. Baseball players are known for having
superstitious rituals at the mound or the plate, precisely because pitcher and
batter are engaged in high-stakes, high-variability activities with
unpredictable success rates. Baseball players are hardly traumatized, but
imagine if they were playing for their lives, or their senses of self.

I wouldn't be surprised if the fractured way that the world is presented to us
after the decline of television and the rise of the internet (with its
conflicting narratives) hasn't so traumatized a fraction of the population
that the only way they can construct their sense of selves as in control
enough to not feel permanently precarious is to habitually steer themselves
away from even the hint of dissonance by constantly asserting, "I know. I am
in control. This is how it is."

One of the things having one or a few sources of national news possibly did
was to give one the plausible belief that their set of beliefs was tenable,
stable, and unchallenged by rivals. From a scared monkey perspective, I wonder
if we got lucky with broadcast television, if the perception of unity wasn't
necessary. Whereas now we seem to be constantly having to defend our perceived
in-group.

The point of this ramble is that maybe the psychodynamics of DK are as simple
as, 'Feeling confident feels better, and I need that, and I feel rewarded when
I engage in thinking that rewards that sense.'

------
forrestthewoods
I've always thought Dunning-Kruger is kind of bullshit. (I realize the irony
in stating this.)

In the study a bunch of people took four tests and then estimated their score
RELATIVE TO OTHER TEST TAKERS. That's bullshit!

The test takers were 65 Cornell psychology undergraduates. Cornell is a great
university. All 65 test takers are probably super smart! In fact, that might
have all scored quite similarly. Forcing 65 high performers to estimate their
relative score is bullshit. That's impossible! You don't have enough
information about the other test takers.

I think it would be far more interesting if test takers estimated their
absolute score. Whether that score is better or worse than other test takers
is irrelevant.

But what do I know. I'm probably just incompetent and don't know it.

~~~
robocat
The point of this article was too remind us that just because someone is smart
in one area (Cornell psychology undergraduates) doesn't mean they are smart in
another (humour was measured in original paper - have you read the DK paper?).

The article makes the common mistake of ignoring the other side of the DK
effect: the top quartile estimates their skill downwards.

I like to say I am showing DK tendencies because I often underestimate my
skills ;-)

~~~
forrestthewoods
My unpopular opinion is that the majority of people who mention Dunning-Kruger
or imposter syndrome are primarily just trying to humble brag.

Full Duning-Kruger paper is here.
[https://pdfs.semanticscholar.org/e320/9ca64cbed9a441e5556879...](https://pdfs.semanticscholar.org/e320/9ca64cbed9a441e55568797cbd3683cf7f8c.pdf)

There is some commentary on raw test score estimation. Including this funny
line:

"Participants in the bottom quartile also overestimated their raw score on the
test. On average, they thought they had answered 5.5 problems correctly. In
fact, they had answered an average of 0.3 problems correctly"

I wonder if the raw DK data is available? The paper includes a small handful
of figures. There's quite a few more I'd like to render.

------
eterm
My pet peeve is people thinking that D-K is a negative correlation of ability
and perceived ability, when it's actually a weaker correlation than it
'should' be.

~~~
DalekBaldwin
And due to scale-end effects, you should _expect_ to compute a weaker
correlation from a naive calibration analysis. If you are at the zeroth
percentile, you can only overestimate your performance; if you are at the
hundredth, you can only underestimate it. [http://home.cerge-
ei.cz/ortmann/TrentoCourse/Juslin_etal_Nai...](http://home.cerge-
ei.cz/ortmann/TrentoCourse/Juslin_etal_Naive_Empiricism_Dogmatism_inConfidence_Research.pdf)

~~~
dragonwriter
But interestingly, D-K isn't a symmetric-about-median effect; which is what
you'd expect from scale-end effects alone. The D-K finding was that people
tend to estimate themselves closer to roughly the 70th percentile than they
actually are.

~~~
haberman
70% is the bottom of C grade range.
([https://en.m.wikipedia.org/wiki/Academic_grading_in_the_Unit...](https://en.m.wikipedia.org/wiki/Academic_grading_in_the_United_States))
Honestly I think this can explain a large part of D-K. People just aren't
calibrated to imagine getting a 25% let alone a 0% even in something they are
truly terrible at.

~~~
dragonwriter
> 70% is the bottom of C grade range

70th _percentile_ doesn't correspond to a 70% score, and when it does, it's
almost certainly not on a (common, but very far from universal) grading scale
where it is true that 70% is the bottom of the C range, just like 50th
percentile is very rarely failing, though 50% is in (and not even the top of)
the F range on the same scale.

> Honestly I think this can explain a large part of D-K.

It almost certainly doesn't explain any of the high-end part of the effect,
and an explanation where the low-end and high-end effects are unrelated
coincidences is, while possible, the kind of explanation that a preference for
parsimony would prefer avoiding in the absence of evidence demanding separate
effects.

~~~
haberman
I just looked at the original paper again and I see that in some of the tests
the researchers asked participants to estimate the raw number of questions
they got correct, in addition to the percentile relative to their peers. Even
in raw numbers the bottom quartile overestimated their ability significantly.
So I'm less confident in my theory than I was before.

------
ggm
D-K also stands for Dorling-Kindersley, publishers of wonderful photographic
books which encourage the D-K effect, which is wanting to buy more. Also,
create false sense of understanding of the "aha, thats what a triceratops
looks like" form, leading back to this D-K effect

------
tpush
See also: [https://danluu.com/dunning-kruger/](https://danluu.com/dunning-
kruger/)

------
edoloughlin
I highly recommend the author's podcast: The Skeptics' Guide To The Universe
[1].

[1] [https://www.theskepticsguide.org/](https://www.theskepticsguide.org/)

------
chantelles
The women were much more likely to turn down the opportunity: only 49 percent
of them signed up for the competition, compared with 71 percent of the men.
“That was a proxy for whether women might seek out certain opportunities,”
Ehrlinger told us. “Because they are less confident in general in their
abilities, that led them not to want to pursue future opportunities.” -Dunning
and Ehrlinger

[https://www.theatlantic.com/magazine/archive/2014/05/the-
con...](https://www.theatlantic.com/magazine/archive/2014/05/the-confidence-
gap/359815/)

------
kromem
My takeaway from DK is that the roughly 80% mark is where is the generally
ideal balance between self-image and social acceptance, and that's why
everyone on either side leans toward that inflection point.

Having been on the "exceptional" side of the curve on some things (and
verified though both independent testing and testimonials), it was definitely
socially isolating until I broadened my perspective and accepted that while I
may be past that 80% mark in some things, I was well behind it in many others,
and if I could appreciate the skill others have that I do not, I would have a
much more enjoyable experience of community than comparing others to myself in
the things I was the best at.

In general, I think this is a lot of cognitive dissonance & confirmation bias.
For everything from politics & religion to what foods or TV shows we like,
it's more enjoyable to share our lives with people that enforce our sense of
reality. Coming to terms with the fact you suck at something is really
unpleasant if you haven't also found the things you are awesome at to balance
it out in your self-evaluation. And vice versa, being REALLY awesome at
something kind of sucks and makes you feel you can't connect with others (and
sometimes they will hate you for outshining them if you shove it in their
faces) unless you recognize your own clay feet.

It also explains why echo-chambers like the anti-vaxx FB groups are so
popular. They are exceptionally (in the standard deviation sense) out-of-touch
with reality, which is typically something that is socially isolating, but by
coming together, they can have the benefits of community without sacrificing
their identity of being smarter than doctors. It's basically like a reverse-
MENSA (which has its own issues with being on the wrong side of DK -- IQ tests
are one specific measure of a population variance, and doesn't necessarily
correlate with things like empathy, open-mindedness, or creative thinking -
all things I'd care about having in my social circle moreso than IQ).

Interestingly enough, we can see the same effect in purely objectively
evaluated ranges. Many of us might be familiar with the threshold for how much
income correlates with the maximum happiness (on average, having either less
or more correlates with unhappiness). IIRC the latest research puts it at
about $100,000 times the square root of the household size. While I don't have
the tables in front of me, I'd wager that's roughly the 80th percentile in
income.

The thing is - that 80% mark is simply the ideal average - but the spread
around it is up to us as a society. If we provide better education resources
for those struggling in academics, we can get more people closer to the ideal
(and likely improve their happiness) without necessarily moving the mean. If
we have high marginal tax rates, we can redistribute wealth to close the
"spread" around the ideal, which if the research is correct will actually
improve happiness for the people on both sides of it. If we actually manage to
master our own genes, we can likely do the same for health, innate
intelligence, empathy, depression, etc.

------
throwaway3627
Another dimension to consider is that some people perform well at some
specific things, but not all nonspecific things. And also that performance
varies with situation-to-situation and self-perception varies with mood/time.

In reality, DK is an overall impression tendency rather than a strict rule...
such is the nature of squishy, fungible social "sciences;" reality is messier
than a numeric IQ result.

------
jeffdavis
I wonder if the DK effect is really an implicit narrowing or widening of the
population.

I would guess thay I'm an above-average sprinter. But that's because I'm
assuming there are 2 year olds and 80-year-olds in the population. If you ask
a runner, they are likely to compare themselves against other runners and
probably not even consider someone like me in the population.

------
aklemm
I assure you my understanding of D-K is well above average.

------
codingslave
I feel like I can't go one day without reading somewhere on the internet where
someone mentions the DK effect. If anything, it's a prime example of group
think. It sounds clever, and is probably true in a ton of cases, and so people
repeat it. Over and over. The same is true of occams razor. These are just
ideas, not intellectual vistas.

~~~
ghaff
People love to think they’re really smart and post a Wikipedia link to DK,
Occam’s razor, submarine PR, etc. whenever they think it’s vaguely related
because they think it makes them look clever. They’re not.

~~~
mieseratte
Instagram Intellectuals. Some folks want to show off their hot bods, expensive
cars, and exotic trips. Others want to show off just how smart they are with a
superficial recognition of a pattern, a concept, a thing they know, but not an
actual discussion.

I'm as guilty as any. How does one learn to "shut up" other than through sheer
force of will?

~~~
Mirioron
Is it actually preferable to shut up though? They say that one of the easiest
ways to get a better answer is to be wrong on the internet. Thus if you share
what you know, but end up being wrong, then somebody's going to correct you.

~~~
BuckRogers
Very rarely do people correct you in a way that's instructive. It's usually
just mocking after they get you on one tidbit. A large part of that is because
those who argue on the internet are usually children or people who know
nothing about anything.

~~~
humanrebar
It takes wisdom and humility to be a student if your instructor is a fool or
sociopath.

Especially since one has an excuse to get indignant and closed minded when
that happens.

~~~
lanstin
Hmmm. It takes wisdom and humility to gain an advantage from the networked
human culture. And in other cases it makes one dumber or less wise? So
plugging in a few reasonable initial values it appears the median wisdom will
drop to zero unless a coterie of wise folks are able to partition themselves
off in ycombinator or kuro5hin Or Slashdot or reddit.

------
Traster
I think what is driving the distinction between what the author is claiming
and what WashPo is talking about is that whilst it's true the data suggests
people gernerally over-estimate their ability relative to others when they
under perform, that's not important. It's not important that the average guy
thinks he's smarter than 60% of other people. The important bit is that the
idiot thinks he's smarter than average.

So there's the DK effect, and there's the interesting part of the DK effect.

But in the original article, the line is drawn to Trump. But I don't think
taking a statistical pattern and then applying it to the public persona of a
reality TV star is a good idea. If we're going to object to anything, I'll
object to that first.

------
baxtr
I find it noteworthy that the author seems to be extremely confident that he’s
right.

~~~
dahart
Which, according to the the DK paper makes it more likely that he is in fact
right. There is a positive correlation in their results between confidence and
ability.

------
joker3
I think there's a better explanation at
[https://www.talyarkoni.org/blog/2010/07/07/what-the-
dunning-...](https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-
kruger-effect-is-and-isnt/), which also discusses some possible alternative
explanations.

------
ocschwar
Some people have a really inflated view of their understanding of the Dunning
Kruger effect.

------
breakyerself
I see on a regular basis people who have poor levels of understanding on a
subject and yet believe they know better than the experts.

Climate deniers, anti-vaxxers, anti-GMO, etc. I understand this isn't exactly
what Dunning Kruger is talking about, but I don't have a better short hand
explanation for it.

------
godshatter
When someone invokes the Dunning-Kruger effect, I often reply asking if they
are a trained psychologist and are they sure they are using the term
correctly? This never goes over very well.

------
zzo38computer
Due to Dunning-Kruger, I don't know if I understand Dunning-Kruger so well or
not.

------
throwaway3627
Trump is off the left side of the graph where actual ability is epsilon (~0)
while perceived ability is 100.

~~~
dang
Please don't take HN threads into partisan flamewar.

------
povertyworld
How does the Dunning-Kruger effect affect people with imposter syndrome?

~~~
russellbeattie
It just shows that, empirically, most people who think they have imposter
syndrome are, in fact, imposters.

Most of the time, I suspect that the "I'm not qualified to be here" feeling is
simply normal people having a moment of clarity.

~~~
alkibiades
you can’t say stuff like this in public even though it’s true. it’s hurts too
many peoples feels

