

Christopher Chabris Should Calm Down - jere
http://www.slate.com/articles/health_and_science/science/2013/10/malcolm_gladwell_s_david_and_goliath_he_explains_why_christopher_chabris.html

======
abalone
This is a weak, patronizing response.

 _" I was simply saying that all writing about social science need not be
presented with the formality and precision of the academic world. There is a
place for storytelling, in all of its messiness."_

Chabris didn't say that Gladwell was informal or merely imprecise. He said he
cited studies _that have been proven wrong_ in the service of a story.

In other words, Gladwell did not merely present a scientific conclusion in an
informal, lightly supported manner. _He presented false conclusions as if they
were true,_ invoking the authority of "science", because it told a better
story.

That is an abuse of science and very worthy of scientists being a little
"uncalm" about it.

~~~
md224
That's fair, but I found this portion of Gladwell's response interesting:

> Chabris does not like this study. He thinks it involved too few subjects and
> that its findings were not replicated by a subsequent study. Alter and
> Oppenheimer disagree. They say that the version of desirable difficulty that
> they explore has been confirmed on numerous other occasions. This is the
> kind of intramural argument about the nature and value of evidence that
> social scientists have all the time.

So who am I supposed to trust here? Who has the higher epistemic ground,
Chabris or Alter & Oppenheimer? The problem is that I don't know what the
scientific consensus is on the validity of that study. I'm not sure whose side
I should be taking.

Are Alter & Oppenheimer bad scientists trying to pass off a poor study as
robust scholarship? Or is there a valid difference of opinion here?

~~~
abalone
It's not uncommon for researchers to dispute follow-up studies that fail to
reproduce their results. Their reputations are at stake, if it points to
methodological errors (which seems to be the case here).

The problem here is Gladwell didn't even hint that there is _significant_
doubt about the methodology of study he cited. Note his dismissive term for
it: a "intramural argument". Chabris just "doesn't like" the study, as if it's
a shirt in a color that doesn't suit him.

These are not equivalent things. It's not just a difference of opinion or who
tells a better story or who has more letters after their name. What matters is
the science behind it, and Gladwell is intentionally perverting it in the
service of a better story. That's a huge disservice to the public when it
comes to popularizing science.

------
jere
I have to say, I always take the criticisms of Gladwell very seriously. I read
his first three books several years ago and have constantly felt like a sucker
over the years as his detractors have grown louder.

And yet I find his responses very convincing, especially this one and the
response to the _Ask A Korean!_ article. I don't know, maybe I'm just easily
swayed or maybe it speaks more about how persuasive Gladwell is.

~~~
deelowe
I've read his books as well and definitely feel like I've been lied to. I
almost feel like he was playing on stereotypes and I fell for it. Of course
things aren't as simple as he presents them in outliers, but man did I fall
for it at first.

~~~
jere
>Of course things aren't as simple as he presents them in outliers, but man
did I fall for it at first.

Yea, I had just graduated from college and was binging on popsci. I felt
really smart because all the popsci authors seemed to reference each other and
when I saw the references I thought I was very well read. Obviously, I don't
think that anymore.

This is an embarrassingly accurate description of me at that time:
[http://xkcd.com/863/](http://xkcd.com/863/)

~~~
deelowe
haha. That's a good one.

------
robotresearcher
"The kinds of people who read books in America seem to have no problem with my
writing. But I am clearly a bee in the bonnet of some of the kinds of people
who review books in America."

That's a nice example of the sort of false dichotomy that infuriates
Gladwell's critics.

~~~
ScottWhigham
I'm dense today, I think. I'm afraid I don't see the false dichotomy. I see
"kinds of people who read books in America" and "some of the kinds of people
who review books in America". Is there a 3rd/nth group I'm not seeing -
perhaps "people who read books and who also review books"?

Genuine question

~~~
ableal
To review a book it is not strictly necessary to have read it, at least
according to a few targeted authors, including this case ( _" I just wanted
them to read my book all the way to the end"_).

~~~
ScottWhigham
Ahhh - thank you. I just couldn't figure out what I was missing haha

------
Finster
Aren't some of these arguments against Gladwell along the same lines as people
complaining about Mythbusters' lack of rigor? Well, then I defer to zombie
Feynman.

[http://xkcd.com/397/](http://xkcd.com/397/)

I'm not saying Gladwell is right, but Chabris' argument amounts to pulling one
referenced study out of context and saying "SEE HOW TERRIBLE THIS IS." He's
complaining about sample size... with a single anecdotal example?

Also, social scientists are kind of cute when they get all angry.

~~~
sailfast
I completely agree that bringing a helpful idea to a broader audience is quite
valuable. The problem, however, lies in the combination of lack of precision
and lack of attention span that creates artifacts of conventional wisdom that
rattle around for years and years without question.

Some day, someone's grandkid is going to be spouting something about "10,000
hours" and not even know where it comes from.

~~~
Legion
> Some day, someone's grandkid is going to be spouting something about "10,000
> hours" and not even know where it comes from.

Which will replace the even more flawed understanding of "born with it".

We all go through life with flawed mental models, which aren't perfect but are
better than a complete lack of understanding. Over time, we make those models
in the general consciousness better, even if they continue to lag behind true
scientific understanding.

------
devindotcom
Although I appreciate Gladwell's argument that he combines anecdotal narrative
and scientific observation (usually he provides the former, someone else the
latter), I don't think that the average reader of the books is clear on the
limitations of this approach. I'm afraid many people come away that Gladwell
has discovered some brand new, proven thing, and instead of taking the book as
a series of illustrations of an idea Gladwell finds compelling and wants to
share, they take it as settled science performed meticulously by the man.
That's not a criticism that is limited to Gladwell, either - many 'pop
science' books do the same, and are mistakenly understood as hard science. I
try to point out problematic articles and books whenever I can, but it's not
easy to convince someone that really wants to believe, for instance, that
adversity (or borderline insanity) is the driver of creativity. And don't get
me started on the misappropriation of neuroscience research.

~~~
helipad
> I don't think that the average reader of the books is clear on the
> limitations of this approach

The average reader may not, but at least they've been compelled to pick up a
book, read it, synthesize it and draw their own conclusions - either agreeing
or otherwise. That's better than nothing.

An above-average reader (whatever that is) might read it and be annoyed. A
below-average reader might read it and either not understand it and take it as
gospel.

What's the problem? Truly.

~~~
devindotcom
I wrote what I think the problem is: "instead of taking the book as a series
of illustrations of an idea Gladwell finds compelling and wants to share, they
take it as settled science performed meticulously by the man."

I think people often _aren 't_ synthesizing and drawing their own conclusions
(it is generous to apply this IMHO uncommon quality to the average departure-
lounge-pick-up reader). They are reading Gladwell like a textbook.

~~~
helipad
If that's the case then Gladwell isn't at fault – the reader is.

------
debacle
Responding to reasoned concerns about rigor with ad hominems in the Internet
age is not a recipe for success.

> My point was that the people who read my books appreciate this. They are
> perfectly aware of the strengths and weakness of the narrative form. They
> know what a story can and can’t do, and they understand that narratives
> sometimes begin in one place and end in another.

The point is that they don't and they aren't and they can't - many people
believe that Gladwell is more than a storyteller, and take his scientific
gallavantry as truth. It's irresponsible to ignore that fact, but it's
necessary for Gladwell's business model to do so - people read his works
primarily not because he is a good storyteller, but because they believe he is
telling the truth, as backed up by science.

------
grimtrigger
There is something ugly about Gladwell's response. Gladwell insinuates that
Chabris has a rabid vendetta against Gladwell and is trying to make a name for
himself as an iconoclast. It might be true, but I would have liked the
response to have taken a high road.

1) He emphasizes that Chabris has written <bold>three</bold> critiques of him
as of date and will probably write a fourth

2) He calls the request for debate silly, making Chabris look agressive.

3) Notes that "I clearly drive Chabris crazy"

Everybody loses when the conversation turns personal.

------
EpicEng
I think Chabris should "calm down" as soon as Gladwell starts publishing his
works under the "science fiction" genre.

------
the_watcher
Is there anything wrong with reading and enjoying Gladwell for his prowess as
a storyteller? And not reading his books as scientifically rigorous works?
Whatever you feel about Gladwell, he is an extremely effective storyteller.

~~~
snowwrestler
Of course not, the problem is that he presents himself and his books as
authoritative and accurate, and a lot of people believe him.

------
kirklove
One of the points that always seems to be lost in these criticisms and
rebuttals and debates is that Gladwell spawns conversion. And that's a very
good thing. Nothing should be taken as sacrosanct on either side. It's good to
prod and examine these issues and bring more views to the table. Hopefully
that leads to more questions... not dogma.

~~~
jljljl
Spawning conversations on the basis of poor or misleading evidence is _not_ a
worthwhile endeavor. While nothing should be taken as sacrosanct, one needs to
have solid evidence or a reasonable argument before they start prodding and
examining issues.

Just saying "Hey, what if?" is not an incredible benefit to society if the
claims are not properly testable.

~~~
sjg007
Politicians do this all the time. Fox News and the mainstream media make their
careers out of it. I think we as a society have forgotten how to deconstruct
an argument and fact check the specifics.

~~~
jljljl
Deconstructing arguments is difficult, and fact-checking is incredibly time
consuming with often little immediately benefit (besides the smug sensation of
knowing more that it often provides).

This is why popularizers and journalists have an incredible responsibility to
provide this deconstruction and fact checking on behalf of the public, and
it's why there is so much outrage, particularly from the scientific community,
when they fail.

~~~
sjg007
I totally agree..

~~~
jljljl
I know, i was just building on what you said :)

------
DanBC
> Kahneman understood my book. Why couldn’t these guys?

Writers should not be blaming their readers for misconceptions and
misunderstandings.

> The first striking thing about all three of Chabris’ reviews of David and
> Goliath is how much attention he pays to a study that I mention at the
> beginning of my chapter on dyslexia.

Many of the readers of pop-science books do not know how to properly read
research papers. They don't have access to the libraries; they don't have the
stats knowledge; they don't know how to pick out the decent studies from the
poor studies.

It'd be good if pop-sci writers were doing this work for the readers, or at
least helping readers to do the work.

~~~
devindotcom
I agree - I think criticism of a book like this should both acknowledge that
it tells a compelling story and may open readers' eyes to new ideas — while at
the same time pointing out that those ideas don't necessarily have good
scientific grounding because of reasons A, B, and C that the reader would not
otherwise encounter, not being scientifically minded or not having access to
relevant literature.

------
codex
Chabris dismisses the Alter & Oppenheimer study because no "exact"
replications have been done that "he has found," but then proceeds to give
weight to a similar study, which failed to replicate the findings, even though
that study itself was not an exact replica, because the test questions were
not appropriate for the sample:

"In my review, I criticized Gladwell for describing this experiment at length
without also mentioning that a replication attempt with a much larger and more
representative sample of subjects did not find an advantage for difficult
typefaces. One of the original study's authors wrote to me to argue that his
effect is robust when the test questions are at an appropriate level of
difficulty for the participants in the experiment, and that his effect has in
fact been replicated “conceptually” by other researchers. However, I cannot
find any successful direct replications—repetitions of the experiment that use
the same methods and get the same results—and direct replication is the
evidence that I believe is most relevant."

I'm not sure who to believe here. Legitimate criticism exists, but for every
successful author, there is one who seeks to piggy back on their fame by
attacking it, and for every sensational scientific study, there is more to be
gained by disproving it than by replicating it.

------
wpietri
Ugh:

 _" The kinds of people who read books in America seem to have no problem with
my writing. But I am clearly a bee in the bonnet of some of the kinds of
people who review books in America."_

Which I interpret as: Because I have high sales figures, that clearly proves
that experts are easily upset fussbudgets who you shouldn't listen to.

