
Grit Has a Negligible Effect on Success Compared to Intelligence - cofree
https://journals.sagepub.com/doi/abs/10.1177/1948550620920531
======
candybar
From the study:

> The Short Grit Scale (Duckworth & Quinn, 2009) was used as a measure of grit
> and was administered to most of the NLSY participants in 2013 (6,476 of the
> original sample of 8,984 in 1997). It consists of 8 items in total, 4 items
> for each grit facet—perseverance and consistency of interest. Sample items
> are “I am diligent” (perseverance) and “new ideas and projects sometimes
> distract me from previous ones” (consistency of interest). Answers are given
> on a scale from 1 (very much like me) to 5 (not like me at all).

This is like measuring intelligence by asking people whether they identify
with statements like "I am a fast learner." Most people aren't very self-
aware. I often find that people that work less hard (in general across a wide
range of things) often self-identify as having more grit because people work
hardest when they work on something they are motivated to do, but being able
to work superficially in the absence of motivation for long periods of time
makes people feel they have more grit.

My suspicion is that grit (in the sense that matters, not whatever the grit
researchers have been pushing) is mostly indistinguishable from intelligence
as it's typically measured. I'd be shocked if most intelligence tests don't
already do a better job of determining grit (in terms of the level of
perseverance you could expect from someone) than asking someone if they are
diligent.

~~~
naveen99
As Sam Altman says, think more about what you want to do. Moving fast in a
wrong direction isn’t going to help.

------
osazuwa
A few points for context:

1\. Those R squared numbers in table 2 are quite low. The statistical analysis
itself is painfully basic. I'm a statistician, and I've a bookshelf full of
excellent books written by statisticians who publish in psychology, so they
should have been able to find someone to beef up the analysis a bit. That
said, psych publications is also notorious for bad data analysis. I'm an
outsider to this field, but this feels like an omission.

2\. The paper doesn't use any causal inference techniques despite making a
clear causal claim in the title. They use simple linear regression, which is
not a causal model unless causal assumptions are made explicit. In my book
this is a grave sin, one avoids making causal assumptions that might be hard
to defend, fits a model with linear correlation, then makes causal
conclusions. To understand what I mean by causal model, read this (these
techniques are well-known and if the authors' defense is that they don't know
them, it reflects badly on them):
[https://en.wikipedia.org/wiki/Rubin_causal_model](https://en.wikipedia.org/wiki/Rubin_causal_model)

3\. It is worth thinking about what the intelligence assessment, the AFQT
test, actually quantifies. One critique of AFQT is that it is more of a
measure of literacy than intelligence. Read more here:
[https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti...](https://en.wikipedia.org/wiki/Armed_Services_Vocational_Aptitude_Battery#Test_validity).
Maybe the title should be "Having parents that make sure you can read by age
five has a greater effect on success than grit."

4\. The AFQT and other intelligence tests have a pesky problem of having
averages that change in time. This suggests that much of what they measure is
not innate, but learned.

5\. Social psychology is plagued by a reproducibility crisis. I'm sorry to
say, their papers, no matter how provocative the findings, have to be viewed
with great skepticism. [https://www.psychologytoday.com/us/basics/replication-
crisis](https://www.psychologytoday.com/us/basics/replication-crisis)

6\. The paper says social economic background has a significant effect, but
about the same effect on pay as intelligence. That seems to conflict with a
pile of research that says that social economic background is the strongest
predictor of success. To say that IQ is just as important as your parents'
wealth is pretty eyebrow-raising.

~~~
aidenn0
I have a friend who got a PhD in psychology. She had me look over her analysis
and it was bad. The two "highlights" were:

1\. Her advisor told her to use a student's t-test for multivariate data.

2\. When the P value insufficiently low to reject the null hypothesis, he
walked her through p-hacking the data.

~~~
BrandoElFollito
I had the same thing with a friend who was getting her PhD in nutrition.

She was aware that the statistics were not strong, but they were horrendous.

After helping her to get at least a bit back on track I developped a strong
skepticism to anything published without a rewiew of the stat methods. Which
is almost everything one can read in the press.

------
sambroner
Abstract says: "intelligence contributes 48–90 times more than grit to
educational success and 13 times more to job-market success. Conscientiousness
also contributes to success more than grit but only twice as much..."

Very challenging to evaluate without paying $40 to read the paper. They
mention a difference in how they group the subjects (by intelligence). My gut
take (often wrong!) is that they're grouping the subjects in some way that
pushes part of my understanding of grit into intelligence.

~~~
perl4ever
Seems to me like intelligence and drive ("grit") are multiplicative factors.
Saying either is more important sounds wrong because if either is zero, you
have nothing. And if intelligence is fixed, then working harder is the only
way to improve your chances. From that perspective, "grit" is the only thing
that matters.

~~~
nsl73
There’s inclusive evidence that intelligence is fixed, and there’s evidence
that people that believe intelligence is fixed have worse outcomes.

I think it’s very likely environmental factors and personal mindset have
considerable impacts both positive and negative on intelligence. I also
suspect in the next decade or two research will come out on pharmaceuticals
that show improved mental performance. There’s already the beginnings of this.

~~~
enkid
What evidence do you have that intelligence is fixed? I believe there is
evidence that it is mix of fixed and not fixed attributes, but I don't think
there's any evidence that it is strictly fixed.

~~~
nsl73
Right. The parent comment I was replaying to stated “if intelligence is fixed”
and I was attempting to cast doubt on the clause. I wouldn’t want to take
position that intelligence is fixed in an argument.

~~~
perl4ever
The context was more of a fixed ceiling. Presumably everybody knows and agrees
that being tired, drunk, ill, having dementia, etc. can affect apparent
intelligence. By "fixed intelligence" the implication is that there is a
ceiling where a person's brain has developed and they are fully functional
that isn't straightforwardly or easily raised.

~~~
zepto
We don’t have clear evidence for this being true.

Perhaps for performance on a fixed kind of test or for certain kinds of raw
calculation speed, it is true, but handling complexity and problem solving
skills can continue to develop based on experience and exposure to concepts.

The belief that intelligence is fixed is a trap for the mind.

~~~
perl4ever
I wasn't asserting that it was true, and it seems that you want to argue with
someone who _both_ thinks that "it" is true _and_ has a different definition
of _it_ from what I was using, in context.

It's a morass I'm uninterested in getting into.

~~~
zepto
Once again you violate the spirit of the group with ad hominem.

I agree your original comment used the conditional ‘if’ about the truth of
fixed intelligence.

Such a proposition naturally invites examination of whether the conditions are
in supported by evidence, which is all I am doing.

I am curious if you have a definition of intelligence that sheds light on your
original argument.

I have no stake in what truth you believe.

However I think your argument is clearly incorrect.

As you suggest, I could be basing my conclusion on a different definition to
the one you are using.

But if you won’t produce a clarifying definition, you can’t reasonably make
claim to be operating under any principle of cooperation.

------
fwip
The paper is behind a paywall and not open-access, so here are some pieces of
information I thought were interesting:

"Intelligence" was measured by an individual's performance on the "Armed
Forces Qualifying Test" (AFQT).

"Grit" is self-reported on the "Short Grit Scale" which asks participants to
rate themselves from 1 to 5 on statements like "I am diligent."

"Success" was measured by level of education attained and hourly wage rate in
2015.

Control variables include: Gender, household income as a child and level of
education by parents. Study participants were born in the early 1980's.

Top-level findings:

> Although in our representative sample of the American population grit does
> explain educational and job-market success over and beyond other predictor
> variables such as socioeconomic status, intelligence, and the Big Five
> personality characteristics, its absolute effects are rather minimal. In
> particular, its effects are negligible in comparison to the effect of
> intelligence. Finally, the effects of grit were also considerably weaker
> than the effects of conscientiousness.

~~~
puranjay
> Grit" is self-reported on the "Short Grit Scale" which asks participants to
> rate themselves from 1 to 5 on statements like "I am diligent."

This is a pretty awful way to measure grit and makes me doubt the results

~~~
gwern
But the measurement of Conscientiousness was much worse, and it still
outpredicted Grit considerably:

"Given the NLSY 2-item measure of conscientiousness and the 8-item measure of
grit, this comparison is likely to overestimate the effect of grit relative to
conscientiousness (Crede et al., 2012 for the consequences of using short
measures of the Big Five personality traits)."

Plus, going from an 8-item test to a longer one isn't going to close a gap
with IQ that large, and as they point out, the reliability is decent enough
and pro-Grit authors have no trouble making claims based on, so what's sauce
for the goose is sauce for the gander:

"The reliability of the scale in our data was .72, which is comparable to the
reliability reported by Duckworth and Quinn (2009) and within the ranges
reported in the meta-analysis of Crede et al. (2017)."

------
m3kw9
There is grit before and after a certain point on a “trajectory” that matters.
And intelligence may get you far, maybe grit will help you complete the last
mile. So no I don’t think you can definitely say what the study says

------
kortilla
TLDR; grit helps, but only when in the same range of intelligence as your
peers.

This isn’t particularly shocking. There aren’t a lot of high paying jobs for
dumb people and nepotism doesn’t scale enough to make up for that.

~~~
anongraddebt
This. There isn't much more to take away from the paper.

As an aside, there seems to be some sort of snowball effect that might
confound a naive interpretation of the role of intelligence. For example,
getting a high SAT score (and AP scores) is likelier to lead to Ivy admission.
Ivy admission is likely to lead to above average training, as well as more
influential social networks. This can lead to interviews with above average
firms... etc. You get the picture.

Now, intelligence played a role in the high test scores and admission to an
elite undergrad, and still plays a role throughout the journey of a successful
(whatever the definition) individual. However, there are many feeback loops
and amplifying/compounding first and second order effects as well. Quantifying
the effect of grit, intelligence, etc seems very difficult.

------
gridlockd
[https://www.youtube.com/watch?v=n3Mn6V1IzHw](https://www.youtube.com/watch?v=n3Mn6V1IzHw)

------
bsanr2
Comparing two ill-defined high-level cognitive traits to determine which will
more greatly effect success in a unmeritocratic society and extrapolating the
results to general humanity: what could possibly go wrong?

>The measure of intelligence was taken from the 1999 wave, the measure of grit
was taken from the 2013 wave, and the measures of suc- cess from the 2015
wave. Thus, intelligence was measured when the participants were aged 15–19,
grit was measured when the participants were aged 29–34, and success was mea-
sured when they were aged 31–36.

Jesus.

~~~
rowanG077
American society is many things but it is not a low merit society. If you are
significantly better then someone else at something you will, in general, reap
the rewards.

~~~
bsanr2
I'm appalled that this passes muster for you as a rational statement, knowing
all we do about how parental wealth and education, professional network size
and health, and even residential location can effect career outcomes in even
technical fields. I thought we put this canard to rest.

~~~
marcinzm
Nothing is perfect, saying something is atrocious because it is not perfect is
irrational.

Also, if you define "merit based" as "capable of improvement through own
effort" then both "professional network size and health" and "residential
location" fall under that.

~~~
bsanr2
I would define "merit-based" as "measuring and ranking based on directly
relevant capability." There are limited cases where the first contributes to
merit, but both are becoming less and less justifiable in a global, or even
national, job market.

------
boxed
The hope is that grit can be developed and nurtured in children, while we're
pretty sure intelligence is genetic to a super high degree.

~~~
sdinsn
> while we're pretty sure intelligence is genetic to a super high degree.

No? There are tons of studies about childhood development that focus on
intelligence being affected by their environment / upbringing.

------
marcinzm
This seems to be the full paper for those noting the paywall:

[https://www.gwern.net/docs/iq/2020-zissman.pdf](https://www.gwern.net/docs/iq/2020-zissman.pdf)

~~~
enkid
I think the most interesting thing in this paper is the fact that grit seems
to be extremely important in certain circumstances, and much less important in
others. It makes sense that getting through a challenge like going through
West Point is going to require a sense of the importance of your goals. Most
of the time, though, we're not in those circumstances and other factors are
more important. Grit, as described, could even lead to a lack of flexibility
in some circumstances, which could inhibit success.

~~~
marcinzm
I agree, looking at the questions they seem to penalize changing your goal
which I view as a vital part of success. Work hard towards your goal but stop
if you realize your goal is wrong. Then work hard at whatever you consider the
better goal.

------
fmajid
Neither intelligence nor grit hold a candle to luck and having chosen the
right parents.

------
Konohamaru
That 10% that comes from grit matters more than the 90% that comes from
intelligence, because it's the only aspect that reflects the sovereignty one
has over oneself and the ability to define oneself in spite of physical
reality (supernatural). The 90% is just an inheritance.

~~~
betenoire
Matters more to whom? I'll take your inheritance if you are offering.

~~~
Konohamaru
Why would you choose what's natural (inheritance, physical development,
parent's DNA) over what's supernatural (your willpower)?

~~~
betenoire
I get what you are saying philosophically, but I don't think it holds up in
reality. I'll take good healthy genes over whatever life lesson you are
offering.

~~~
Konohamaru
It only matters if your happiness is in this-life-only. If you don't then it's
just another part of staying in a bad inn.

~~~
betenoire
You seem to have a clear view of what will make you happy, and I genuinely
commend you for that. My initial question was "To Whom (does it matter)?" and
you seem to have answered it with "To me and my beliefs about who I am". Fair
enough.

But I wonder, \- Do you want a brilliantly gifted surgeon or one who gets by
on grit to operate on your loved one? \- Do you want to listen to the
effortless musician who seems to have some gift from god, or one that
practices 6 hours a day even though they know they kinda suck?

~~~
Konohamaru
Kind of a theoretical question since you cannot observe grit. Only the surgeon
can know if he has grit based on his own self-reflection.

> or one that practices 6 hours a day even though they know they kinda suck?

Again, we know that practice does not make perfect, but rather perfect
practice makes perfect. Just because he practiced for six hours a day doesn't
mean he practiced well.

~~~
betenoire
this is all rhetorical, especially when you brought in the afterlife.

grit == effort, not results, and not even effective effort, since that would
take an inheritance to understand what is and what isn't effective. you dodged
the question

~~~
Konohamaru
Reframe it! Instead of thinking results matter, reframe your life as the
following:

1\. Pretend that only effort exists

2\. Assume beforehand that all your results are equal to <insert first-rate
role model>'s results in <insert your desired field>

3\. Only exit this make-believe world by looking at your actual results in
order to measure the quality of your effort

------
lambdatronics
Here's a non-paywall copy:
[https://www.gwern.net/docs/iq/2020-zissman.pdf](https://www.gwern.net/docs/iq/2020-zissman.pdf)

~~~
D-Coder
"...a major concern about the studies which argued that grit has a strong
impact on success is that they were based on selected samples such as West
Point cadets (Duckworth et al., 2007), employees in technology companies
(Jachimowiczet al., 2018) or students in inner city schools (Eskreis-Winkleret
al., 2014). These samples were drawn from institutions which are characterized
by a homogeneous socioeconomic population (and perhaps even by population that
are homogenous in their intelligence)—a characteristic which is likely to
impose a range restriction on important predictors of success."

Ow!

~~~
bsanr2
You mean, they... controlled variables?

Between those 3 cohorts, I see a phenomenon that cuts across a wide slice of
the population.

~~~
ivanbakel
If you're arguing for particular significance of a variable (grit), you can't
go around controlling for all other variables with the same effect. It's not
meaningful to see significance of a variable in a homogeneous cohort and
translate that to the the variable being significant in the general
population.

It might make sense that West Point students differ largely by grit,
especially if all other variables are controlled for. That does not prove that
the success of people generally is significantly affected by their grit.

------
jaylittle
TLDR: Stupid is as stupid does.

------
iandanforth
Standard warnings: Social Psychology is not a science and should probably be
ignored. "Grit" and "Intelligence" are so poorly and inconsistently defined as
to be meaningless. This is controversy for controversy's sake

~~~
programmarchy
Not true but typical belief of STEM types. If you go by reproducibility rates
you could say the same about medicine and economics. Hell, gravity is poorly
and inconsistently defined, but we don’t throw away physics.

~~~
boxed
Gravity is extremely clearly defined. And yes, we do bash on economics quite a
bit. We bemoan medicine, but we don't make fun of it for having reproduction
rates in the low single digits. Because it doesn't.

~~~
programmarchy
Even the Standard Model does not explain gravity, hence it is not "extremely
clearly defined".

~~~
beisner
There’s a famous saying in the sciences: “All models are wrong, some are
useful.” Many of our physics models have been “wrong” historically, but yet
they describe the world well-enough to be useful in making very good
predictions. Newtonian physics is a perfect example of this - it tells an
incomplete story of reality, but it’s still extremely useful in making
predictions about how the world works.

The reproducibility crisis across the social sciences (and sometimes even in
biology and medicine and - hot take - machine learning) is evidence that many
of these models are both wrong AND useless for prediction, as opposed to just
being incomplete.

~~~
programmarchy
Many sociological and psychological models are useful as well. All sciences
are affected by the reproducibility crisis because of the incentive structure
in the peer review process. Physics has perpetuated useless models also so
it’s not immune from criticism.

Back to my original point, it’s misleading to claim that social psychology is
not a science and should be ignored.

------
RickJWagner
Henry Ford would disagree. Does today's economy make some difference?

------
nickthemagicman
That's hilarious. Extremely counter to the prevailing narrative of how hard
work and grit get one to the top.

~~~
gridlockd
I'd say that's still better than the other fake narrative: The game is rigged,
the cards are stacked against you, they're holding you down, don't even try.

~~~
fwip
Is it? For the question "what predicts your pay", your "grit" beta coefficient
is only 0.09, compared to a sum of .55 determined by your gender, background,
and intelligence.

That is, raising your "grit level" by 1 standard deviation will only increase
your pay by 0.09 standard deviations, when compared to your age-matched peers.
Somebody only one standard-deviation down on each of the other factors would
have to be 6 standard-deviations higher on grittiness in order to get the same
pay. That's the difference between the 0.1th percentile of grittiness and the
99.9th percentile.

The data shows that increasing your grit enough to compensate for even small
levels of adversity is impractical.

~~~
gridlockd
> Is it?

Yes, because if you buy into that narrative, you're going to be a loser,
guaranteed. If you don't, you might still be unsuccessful, but you will earn
(self) respect. You gave it your best shot.

 _Nobody_ respects the person who never tries, who always makes excuses, who
blames everyone and everything but themselves. It's _bad_ for you.

~~~
fwip
Maybe we shouldn't base our respect for people on how much grit they seem to
show, if it doesn't seem to matter anyways.

~~~
gridlockd
It _does_ matter even by the numbers shown in the paper. It matters more than
agreeableness, emotional stability and even conscientiousness. It just doesn't
make up for intelligence.

Also, if you're lazy and smart, you're still going to be a loser. You'll still
need some amount of grit.

~~~
fwip
It matters half as much as conscientiousness, was the conclusion.

------
aSplash0fDerp
Using the term "tools" loosely, fork and spoon comparisons digest better than
apples and oranges.

Eating soup with a fork is still a success with enough intelligence.

Pfffft. Spoon-fed geniuses raised with only forks was a cruel joke.

Let me guess... For an encore, they'll do a study that says AI is more
successful.

A suckers iterated ever minute on the Internet with a path/information like
that.

~~~
aSplash0fDerp
I hope the downvote was for being incomplete.

A fork, spoon and a knife are required tools for most meaningful
job/professional successes.

It may even mirror the mind, body and soul tools needed to achieve personal
successes. 2 outta 3 sucks. 1 outta 3 is worse..

