
When the Revolution Came for Amy Cuddy - DanBC
https://www.nytimes.com/2017/10/18/magazine/when-the-revolution-came-for-amy-cuddy.html
======
geofft
It occurs to me that the replication crisis largely came into being because
scientists are rewarded for publishing papers that demonstrate some effect and
not for publishing papers that fail to demonstrate anything, and so the right
fix would have been to fix the broken incentives. But apparently (if this
article is to be believed) no such change in incentives has happened - the
only thing is that there are now also incentives for demonstrating that some
other study wasn't replicable. Steadily doing _good science_ , where good
science is expected not to produce results much of the time, continues to have
no support. Is it any wonder that the state of science seems a little better
(at least we can admit the replication crisis now) but people like Cuddy feel
like they have no future in the field, that having the misfortune of the one-
in-twenty chance of getting a really interesting false positive at _p_ = 0.05
means you either have to double down on the study being right after all or you
have to quit the field? Are schools actually looking at past _p_ -hacking
while continuing to demand a huge publication record for hiring? This doesn't
seem like we've solved any fundamental problems.

(Also, as a side note, I'd fully expect that CS is at least as vulnerable to
the same thing. McSherry et al.'s COST work is a good example of something
that seems far _worse_ than _p_ -hacking: instead of having a clearly defined
hypothesis and publishing the 1-in-20 papers that falsely state it's true, as
in other fields, there's nothing clearly defined in scalable systems and 20
out of 20 people who build something excessively complicated can get a paper
out of it.)

~~~
srtjstjsj
> having the misfortune of the one-in-twenty chance of getting a really
> interesting false positive at p = 0.05 means you either have to double down
> on the study

Or you could get a large sample size to bring the p down to 0.01 or whatever.
You know, do science.

~~~
geofft
Well, the smaller your _p_ , the higher your chance of false negatives, too;
maybe there was an actual effect there that was worth more study, and you
dismissed it.

The problem is that a study that's significant at _p_ = 0.05 but not _p_ =
0.01 is worth something to science. It indicates something worth doing several
more studies on, preferably by different teams from different institutes with
non-overlapping confounding factors. But the practice of science gives you
only two options once you find such a study, as a young scientist: promote it
immediately and become visible for having discovered something, or
aggressively ignore it and go find something else that you think is likely to
be immediately promising. "Doing science," as you put it, is the worst option
for you because it ensures you spend a lot more time on something for no
reward -- and the rational self-interest here _is_ perfectly rational, because
if you don't get funding or tenure, you're not going to do any science any
more at all. (As seems to have happened with Dr. Cuddy.)

To be clear, I have no well-formed opinion on whether Dr. Cuddy is a good
scientist, having only read this article. I assume she is, or at least that
she's honest in her desires. But I think talented, honest people are
nonetheless pressured by incentives, and fixing incentives is better for
society (assuming we as a larger society wish our scientists to continue to do
science) than assuming that people will be self-sacrificial and altruistic in
pursuit of a tiny chance at accomplishing the greater good.

~~~
killjoywashere
Here's the structural problem I see: there are acres, literally, acres of
computer science types in SF and certain other enclaves, slaving away on ads
when they would rather be curing cancer. But they don't know the first thing
about cancer.

Meanwhile, there are enclaves where biologists are slaving away capturing data
and not even realizing it's data. "Bookkeeping data" they might call it.

There should be mixers between the biologists and the computer scientists.
Especially non-academic physicians and computer scientists. Look at Kaiser-
Permanente, Sutter Health, or Defense Health Agency. Huge orgs, very little
research. Data falling off the shelves and they don't know what to do with it.

~~~
amazingman
You’re not wrong. Especially not about the ads. What we need, unfortunately,
are (at least) 2 very difficult things to achieve legislatively:

1\. A legal framework for health-related products & services that meaningfully
protects individual privacy, and yet also preserves the conditions for the
kind of velocity and rate of change that brings technological progress to the
masses reliably and quickly.

2\. _Massive_ academic funding for more “pure” research in hardware & software
design, engineering, the philosophy of what’s possible, the phenomenon and
ethics of what we choose to build, and so on. TLDR Pay more people more money
to do important research in the public domain, which I can’t help but think
would attract many of the CS types you mentioned.

~~~
killjoywashere
> a legal framework that protects privacy yet brings technological progress to
> the masses quickly

That exists, but there's upfront educational costs that need to be handled
first.

> Massive academic funding

Look at Europe: their acedemics work much more closely with industry. Get a
someone who can code in a room with a physician, and I garuntee they can find
a problem to work on. Show a successful prototype, and we're way below
"minimum viable product" here. Show a graph with real data and money will
start looking for you.

Computer scientists, mathematicians, physicists, and engineers need to go to
biology conferences. And biologists should be going to math, ML, and software
conferences.

------
hprotagonist
A good working definition of a good researcher, to me, is something like
"strong opinions, loosely held".

I don't think Cuddy, or anyone else, deserves to be dragged through the mud
because their idea turns out not to hold up. That happens _all the time_ , and
it's not a bad thing at all.

You start to deserve that dragging more when you become too attached to your
pet theory and cease to be able to drop it in the face of overwhelming
evidence to the contrary.

You really deserve it when you use what political capital you've acquired to
stomp on your critics despite the fact that they're actually right and you're
actually wrong.

What's interesting here is that a lot of the Sayre's Law-type political
machinations are taking place on a much broader stage than normal. If someone
was to go after my work with this kind of exposure, you bet I'd be defensive
too. It's entirely possible to have a fair and neutral conversation about
research practices without shanking someone's reputation because it seems like
the field needs a sacrificial lamb this week. Nobody wins, and certainly no
good research is advanced, when the knives come out like this with no evidence
of actual malfeasance. (And, frankly, even when there _is_ evidence of
malfeasance. like with the STAP controversy, which lead to at least one actual
death.)

------
danso
Andrew Gelman (of NYU), who is featured in the story and is criticized by
Cuddy as being unfair and trollish, wrote a response:

[http://andrewgelman.com/2017/10/18/beyond-power-pose-
using-r...](http://andrewgelman.com/2017/10/18/beyond-power-pose-using-
replication-failures-better-understanding-data-collection-analysis-better-
science/)

I thought that Gelman wasn't presented in the best of light, especially his
awkward quotes about not wanting to engage in "interpersonal conflict". But he
called the piece "excellent and fair to all sides" and was pretty gracious in
response to Cuddy's criticisms and how the NYT portrayed him. I honestly
wasn't a huge fan of the article but re-reading it after a bit of sleep, and
knowing that Gelman thought it was strong, I appreciated the issues it brought
up. Certainly has to be one of the longest stories about p-hacking/p-curves in
mainstream media.

I sympathize with Cuddy and have no doubt that among the legit attention that
Gelman and other critics have directed toward her, she's faced hatemail
sparked by dumb sexism, which makes it easier to tar the legitimate criticism
as being sexist.

I do think Cuddy needs to face legitimate criticism. She may not have been a
malicious fraudster, but ignoring criticism after a long time is not much
different than actively being deceptive. I also can believe that she faces
hostile criticism that isn't equally distributed among all the other
researchers who used weak, but previously acceptable standards. And I don't
think sexism can be dismissed outright, if people find it safer to talk shit
to a woman compared to a man.

What I wish the story had more of was how scientific criticism is done outside
of Gelman's blog. Gelman is the main scientist/research I follow on the topic.
Apparently some see his snarky critique as toxic, and Gelman says he is
intending to reevaluate the way he writes about things so that his point
doesn't get lost in cheap jokes. How do other blogs do it? I'm somewhat
sympathetic to Gelman because I perceive that there is significant
institutional resistance (and few incentives) to re-examine and correct a peer
review study, and being harsh/edgy/rude and beating the drum over time might
be the only way to get scientists to care? But that doesn't mean his critique
can't be refined for the benefit of his mission.

~~~
Bartweiss
> _I also can believe that she faces hostile criticism that isn 't equally
> distributed among all the other researchers who used weak, but previously
> acceptable standards._

I agree that this is true, but at a certain point it seems like a justified
reaction to unequal importance.

Other Gelman targets like the "himmicanes" paper generated a bit of media
attention without any real consequence. Power poses, meanwhile, have become a
major pop-science hit. They're still being widely read and promoted, even by
tech conferences and news orgs like the BBC. And Cuddy in particular has
chosen an aggressive non-scientific promotion campaign while her co-authors
have made more honest attempts to take on the replication question - if Carvey
was getting the same amount of flak I'd be more concerned that something
unfair was happening.

If power poses don't work, it seems like a much worse indictment of academic
social psych than most other failures to replicate - the results were widely
touted and are still being raised to public acceptance even as they fall
apart. The challenges to stereotype threat, priming, and the IAT are even
scarier - those results have become well-established and even enshrined in
law.

Throughout the replication crisis, my greatest frustration has been the
inconsistent narrative coming from researchers. They're simultaneously
demanding formal academic critiques instead of blogs, ignoring those critiques
when they do exist, and then replying with the same sort of informal
aggression they decried in the first place.

I think you're exactly right that Gelman's aggressive approach is motivated by
the field's unwillingness to acknowledge anything milder. And seeing responses
like Fiske, Cuddy, and Wansink have largely convinced me that there's a major
double-standard at work between the level of seriousness applied when
attacking critics and the level applied when presenting results as academic
truth.

~~~
ghostcluster
> the results were widely touted and are still being raised to public
> acceptance even as they fall apart. The challenges to stereotype threat,
> priming, and the IAT are even scarier

They're in the same boat. It is maddening how much stereotype threat gets
brought up to defend certain ideas, or how it has worked its way into
corporate diversity policy, and it appears to be built on a foundation of sand
[1]. The worst part is, if you question it publicly, you stand a change of
being treated like James Damore.

[1] - [https://www.psychologytoday.com/blog/rabble-
rouser/201512/is...](https://www.psychologytoday.com/blog/rabble-
rouser/201512/is-stereotype-threat-overcooked-overstated-and-oversold)

------
q123321
The most important sentence of the article, which appears at the end:

"Although 11 new papers have recently been published that do not show the
downstream effects of power posing on behaviors, Cuddy is still fighting for
power posing."

This is an astonishing number of replications to be published on one topic in
psychology. 2-3x the next highest number I heave heard of.

The placement of that sentence, plus the flattering photos, leads me to
suspect a publicist was able to place a thumb on the scale.

~~~
skybrian
Are studies of "downstream effects" considered to be replications? It sounds
like they are something different.

~~~
asdf2341
Yes they are considered to be replications.

------
ghostcluster
The work of these social psychologists absolutely deserves the scrutiny they
are receiving. The field is rife with "surprising scientific effects" that do
not replicate and turn out to be completely wrong, yet they linger in the
popular culture and are brought up over and over again to defend ideas and
policies that are not based in reality.

Look at the reach of people like Malcolm Gladwell and stuff like his books and
podcast, and NPR's Hidden Brain, which subsist on recounting these "surprising
studies" and speculating about their profound meaning for society, to gigantic
audiences. And they're wrong! Made up ideological BS with the gravitas of
'serious science' behind them.

Power posing, stereotype threat, the bystander effect, implicit bias. So many
popular hypotheses have snuck into the popular culture and they are _wrong_.
They're based on fudging numbers and statistics and throwing out results that
don't fit the hypothesis. They deserve to be combated in the public sphere.
They don't deserve to be shielded to protect the careers of the hucksters.

------
hirundo
Cheers for scientific revolutions being gentler than political ones. Cuddy
lost her reputation rather than her head. But losing a reputation hurts more,
or at least longer. More civility could have helped, but there's no painless
way to lose status.

The scandal is that Cuddy's work met the standard of the bulk of published
social science until lately. She studied the same phlogiston with the same
tools as her colleagues. But when the revolution that comes for them, gently,
is an epistemic advance, their pain is our gain. Join the revolution, comrad
Cuddy!

~~~
throwawayjava
The tragedy of the ordeal is that _she tried_ and was lined up against the
wall regardless, mostly due to lack of civility and poor communication:

 _> But the email that Simmons and Simonsohn had sent was, in fact, ambiguous:
They had explicitly told her to drop the P-curve and yet left the impression
that the paper was otherwise sound. At my request, Simmons looked back at his
original email. I watched as he read it over. “Oh, yeah,” he said quietly. He
had a pained look on his face. “We did say to drop the graph, didn’t we?” He
read it over again, then sat back. “I didn’t remember that. This may be a big
misunderstanding about — that email is too polite.” Cuddy and Carney had taken
their advice literally._

~~~
conistonwater
" _lined up against the wall_ ", really? I think you would be more convincing
if you toned it down a bit on the metaphors.

~~~
throwawayjava
It was parent's metaphor, not mine. The substance holds.

~~~
conistonwater
I don't see it in that comment. Where? I only see "Cuddy lost her reputation
rather than her head.", which is not at all similar. (I don't see it in NYT
either.)

~~~
throwawayjava
Is there really a substantive difference between losing your head to a
guillotine and being lined up and shot?

(The NYT begins the metaphor with "revolution", parent extends it with
"violent revolution", and I switched out one form of violence with another).

------
yawaramin
I'm sure there's already been a discussion on how much of the effect of power
poses is due to confirmation bias or placebo effect, as opposed to some other
physiological or psychosomatic reason. Especially since in the original study
the subjects weren't actually told what they were doing, but in Dr Cuddy's
talks and books they explicitly are.

It's just curious to me that she and Dr Fiske wouldn't catch this crucial
difference immediately. Effectively it turns power poses into little more than
'the power of positive thinking'.

That said, personal attacks and especially those focusing on her brain
function really need to stop. Psychologists treating each other like shit is
not just ironic, it's a disgrace. If they can't even behave properly with each
other, how can we expect them to treat patients properly?

~~~
09bjb
One note; in general, I think we tend to diminish finding when we find out
that the placebo effect is partly behind them. Wrongly, in my opinion. Better
reactions might be, "Wow, we take this action and our bodies/brains take care
of the rest of the effect through some placebo magic? That's amazing!!"

~~~
yawaramin
That's a great point. No one should be able to argue with Dr Cuddy if she
decides to make her living pushing this 'placebo-trigger' technique.

~~~
tunesmith
But doesn't pointing out a placebo effect also have power? I know the placebo
effect still exists even if you're aware of it being placebo, but does it
lessen? And what if it's accompanied by shaming?

Meaning, say the power pose effect doesn't exist. But early on, it's exciting
and new, and there's a placebo effect (which is fine!), but people also think
it's real.

But then studies get replicated, and there's well-known doubt on whether the
effect is real in a non-placebo manner. There might still be a placebo effect,
but does it shrink?

And then there's a lot of controversy and the study is dragged through the
mud, and power poses are mocked. Might the placebo effect be lessened even
further (or obliterated) from this?

Could this be true even if it _was_ originally valid and not just placebo?

~~~
munchbunny
I think you're moving into territory in human psychology that nobody really
understands.

For the time being, the answer is probably "if it works for you, then keep
doing it." If power posing before an important meeting is how you psyche
yourself up to be unafraid and effective in the meeting, then why not keep
doing it, even if it doesn't seem to work for everyone?

Knowing the replication problems, I won't suggest power posing as a general
thing. Maybe I will tell someone to try a bunch of things including power
posing, and tell them to keep the things that they think are working.

------
tomkat0789
I published a couple of papers in grad school, and what the article describes
made me imagine the peer review process busted open for the world to see.
Reading reviews sucks: good reviewers provide smart reasons that you're wrong,
which you have to beat with more research saying that you're right. The editor
makes the final decision, not always in your favor.

In their "open peer review", the psychologists seemed to have all the class of
a bunch of redditors. An uncomfortable process is made worse! Instead of
motivating people to publish lame results for tenure, this new situation seems
to motivate people to act like Internet trolls or tabloid journalists.

A more critical research community should help advance the field, but these
researchers set a bad example.

------
maxxxxx
I used to love TED talks but over the last few years I have developed an
aversion. It seems a lot of popular TED people have traded scientific
integrity for showmanship (and money). Makes me wonder how Feynman would be
doing today. He was a charismatic talker. He could be a superstar.

------
phunge
I think it's pretty easy for this discussion to become about personalities and
not methodologies. Gelman is pretty brash and speaks his mind, so he could be
correct on methodology and have that get lost in the noise.

So I'll give my opinion as a datascientist: null hypothesis significance
testing is broken, the whole thing needs to get chucked. It's not fixable, and
it's not only p-hacking that's the problem. Go read Frank Harrell, I like his
writings on the topic more than Gelman's.

------
dingbat
has anyone gone down the list of most popular TED Talks and run similar
replications of the claims made in other presentations? itd be interesting if
there were some ratio that scored most-popular TED Talk vs. "B.S." content. i
wonder if there'd be a link between popularity and amount of B.S.?

------
RcouF1uZ4gsC
>This moment of fizziness for the discipline was already underway when Cuddy
arrived at Princeton’s graduate program in 2000, transferring there to follow
her adviser, Susan Fiske, with whom she first worked at the University of
Massachusetts at Amherst.

This is the same Susan Fiske who called people who pointed out statistical
errors in psychology papers "methodological terrorists".
[http://andrewgelman.com/2016/09/21/what-has-happened-down-
he...](http://andrewgelman.com/2016/09/21/what-has-happened-down-here-is-the-
winds-have-changed/) The apple does not fall far from the tree.

~~~
throwawayjava
You mischaracterize. Fiske is not criticizing "people who pointed out
statistical errors" and _begins the entire essay_ by explicitly stating as
such: "Our field has always encourages -- required, really -- peer critiques"!

She is criticizing "unfiltered trash-talk [on social media]... [and] online
vigilantes [who] are attacking individuals, their research programs, and their
careers".

It's the same tension the article is referring to, between civility and public
discourse. On one hand, you'd really like an open dialog about the quality of
evidence presented. On the other hand, there's a real price to be paid when
people are too worried about public humiliation to risk doing science on
certain questions or working with certain people.

As the quote from Jay Van Bavel put it, "It's become like politics — we’ve
created two camps of people who shouldn’t be in two camps in the first place"

This point resonated with me even though I'm on your side of those politics.
What effect would it have on the OSS community if any even slightly bad code
you write could end up in a hit piece published by The Atlantic, and if fact
you witnessed people on Facebook saying your colleague, who maybe did write
some kinda bad code that one time but is on the whole competent, is a
charlatan who doesn't deserve the title of "programmer"?

~~~
lovich
Is that not the case anyway? I admit I have no experience contributing but
that's from hearing things like how the linux development team runs and people
like Sarah Sharp quitting over the abuse. It's given me the impression that if
I don't create something that is up to a teams standard from commit 0 then I
will be ridiculed in a public forum

~~~
throwawayjava
Abuse from your own insular community re: a particular project is one thing;
public abuse published in some of the most popular print media on the planet
resulting in lost jobs/lost job opporunities is quite another.

~~~
lliamander
I certainly have a fair degree of sympathy for Cuddy's losses, but should she
have been given the accolades (TED talk, tenure track at Harvard, etc.) in the
first place?

Her work was interesting no doubt, and in our scientific age we have always
celebrated the novelty and originality of the scientific geniuses that have
advanced our understanding.

But the real value of science has always been novelty _plus certainty_. The
scientists who we esteem should be known as much for their industrious
management of the empirical process (recruiting replicators, statistical
auditors, etc.) as they are for they are for their creative insight.

~~~
throwawayjava
As I indicated up-thread, I'm mostly on your side of these politics.

geofft's comment hits the nail on the head when it comes to what effect this
did vs. should've had on her tenure case, I think. A study that fails to
replicate is no indication of malice or even incompetence. Some say she
should've run self-replications, but how many years self-replication studies
are you willing to run on $20k/yr (in fact, how many post-college years have
you made 20k/yr for a full-time job)?

At some point you get the quality of science you can pay for, and throwing the
grad student under the bus by denying them tenure a decade later is as shitty
and spineless as Equifax throwing an IT guy under the bus for what was pretty
clearly an organizational failure. IMO.

Re: TED talks, TED is not a scientific venue, and most of the things I hear in
TED talks are a hell of a lot less scientific than even Cuddy's most flawed
studies. Also, I don't think those sorts of things should weigh much in hiring
and tenure cases either way (aside from e.g. a usually marginal
"service/outreach" category).

If Harvard hired Cuddy because she gave a popular TED talk, then that was an
extremely poor hiring decision by Harvard. But that's a separate issue from
the one we're discussing.

~~~
lliamander
> As I indicated up-thread, I'm mostly on your side of these politics.

I know, and I think we are mostly in agreement. I generally don't like seeing
individuals dragged through the mud. It's also somewhat unfair to judge her
actions pre-replication crisis by our standards post-replication crisis.

I also agree that a single study failing to replicate is not an instance of
malice or incompetence, in fact I think all scientists should expect it to
happen. But that also means that no career should be built on the back of a
single study. It _seems_ that she is still sticking to her guns even after 11
failed replications. That does cause me some concern.

> but how many years self-replication studies are you willing to run on
> $20k/yr? At some point you get the quality of science you can pay for...

Then maybe one should more hesitant about promoting those results?

> If Harvard hired Cuddy because she gave a popular TED talk, then that was an
> extremely poor hiring decision by Harvard. But that's a separate issue from
> the one we're discussing.

I think it makes me more sympathetic to her case. If Harvard had not hired
her, or TED not promoted her, she would not have been subject to nearly the
same scrutiny or ire as she was. Ironically, she might have had a better shot
at a good long-term career. But who, at that point, would have turned down
either of these prizes? I know I wouldn't have.

~~~
throwawayjava
_> Then maybe one should more hesitant about promoting those results?_

We definitely agree on this. I think all the TED science stuff is silliness.
Cuddy's mistake was both being a serious scientist and being a popular TED
speaker. Popular TED speakers make BS science claims _all the time_ , but they
aren't scientists so it doesn't matter.

But I wouldn't advocate for firing someone because they disagree and think TED
is a good place to popularize science (even if they happen to popularize
science that later turns out to be bad science).

~~~
urschrei
Cuddy wasn't fired, or even denied tenure, though it's probably not a
coincidence that she's no longer (or not currently) on the tenure track at
HBS.

------
nickbauman
How easy it is to forget the scientific method begins only _after_ a
hypothesis is proposed.

------
interstitial
Andrew Gelman's take: [http://andrewgelman.com/2017/10/18/beyond-power-pose-
using-r...](http://andrewgelman.com/2017/10/18/beyond-power-pose-using-
replication-failures-better-understanding-data-collection-analysis-better-
science/)

------
fgjjgutjvnu
NYT tries to frame it as if the rules of science were unfairly changed. Not so
- people had just been overly lax for a while. And sorry, NYT, but scientific
knowledge is not a matter of opinions and goodwill. We can not simply decide
her theories are true after all, because we empathize with her so much.

------
cratermoon
Sounds like straight up sexism to me.

~~~
sctb
If you're going to make a claim like this, could you please also give us some
information? Otherwise you're just starting a flamewar and this is not a site
for those.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

------
breitling
Wasn't this study thoroughly debunked already?

The co-author of this study, Dana Carney, distanced herself from this study
[1]. She doesn't believe all the claims the study made were true.

[1] [http://www.npr.org/2016/10/01/496093672/power-poses-co-
autho...](http://www.npr.org/2016/10/01/496093672/power-poses-co-author-i-do-
not-believe-the-effects-are-real)

~~~
maxerickson
The article includes much discussion of Carney.

------
anigbrowl
tl;dr Promising young researcher skated on stats and talked up faulty finding,
career seriously damaged by lack of replicability; reformers who are too
earnest about a good cause (replication, statistical rigor) can be total
assholes to well-meaning people who just make mistakes; conflict ensues.

It's OK to be oppositional or confrontational to people based on what they say
and do, but self-styled reformists sometimes become personally vindictive in
pursuit of their own interests. Picking a fight with a public figure is an
easy way to gain publicity, and if your own reputation rests on winning that
fight then it can be tempting to carry on into harassment. Gamergate was an
extreme example of this.

~~~
ndh2
Not a real tl;dr.

