Hacker News new | past | comments | ask | show | jobs | submit login
Psychological Constellations Assessed at Age 13 Predict Success 35 Years Later (nih.gov)
92 points by michalu 46 days ago | hide | past | web | favorite | 105 comments



A couple of funny tidbits from the paper:

> Of the 677, we were unable to find 27 males and 15 females in our Internet searches (we could not confirm their identities in our contact-information database). Given our method and criteria (Table 1), however, we believe it is unlikely that we missed an individual with a truly illustrious career. ... Our reasoning was that, if someone were truly eminent, then that person should be publicly conspicuous and therefore discoverable through online searches.

And a finalist for the irony in naming things award:

> We used Publish or Perish software (Harzing, 2007) to collect information on each participant’s number of publications, number of patents, and h-index.


>> if someone were truly eminent, then that person should be publicly conspicuous

rather doubtful, for a variety of reasons, but probably unlikely to affect their numbers


I try to keep a low profile, stay off of social media, and leave a minimal fingerprint. I was a "gifted child" of the 80's. I took the SAT when I was in 7th grade, and did better than above average high school seniors.

I have "stealth wealth", most of it from savings and stock market investments. I don't like to draw attention to myself.


"Impactful" individuals are those that impact society "e.g., full professors at research-intensive universities, Fortune 500 executives, distinguished judges and lawyers, leaders in biomedicine, award-winning journalists and writers".

"stealth wealth" is not impact.


Ignore the stealth wealth part. I think many "eminent" individuals would also not want to draw unnecessary attention to themselves.


My rule of thumb is eminent people do not mention their PSAT scores as a reason why they are eminent.

It's like, say, bringing up your Mensa membership in a job interview.


Eminent means famous.


It has a different connotation. For example, almost no CS researchers or professors are "famous" in the traditional sense. Yeah, definitely a few here on HN, but not outside their profession, and certainly not to the general public. Even if you look at "Fortune 500 executives", outside of a small subset of CEOs, they're not "famous."


And those researchers would publish papers, which would have turned up in their searches.


Do you think there are Fortune 500 executives that have managed to remain so obscure that somebody who knows their name and life until age 23 would be unable to find out that they held the position?


What's an executive? C-level? Some places everyone is a vice president, right?

It occurs to me that the Fortune 500 is not the S&P 500, so I would question whether information about them is necessarily even as accessible as with a public company.


this study isn't gauging wealth at all. which I think is refreshing, but limiting. and I think it also highlights how we are trained that "success" is some arbitrary form of linear or exponential wealth advancement that is assumed to be permanent


Sorry if I burst a bubble but that's not even close to "eminent".


Well since they aren't just trying to determine if people got rich at some point in time but only looking for public accolades, this is a self-fulfilling outcome

I'll add it to the gradient of life, but I'm more interested in socioeconomic movement


This is not the astrology paper I was looking for.



The rest of the SMPY papers can be found at https://www.gwern.net/SMPY btw.


\o/

Thanks!


Summary: If you did well on the SAT or attended graduate school, you were more likely to become a professor or receive NSF grants.


>Summary: If you did well on the SAT or attended graduate school, you were more likely to become a professor or receive NSF grants.

This seems to measure people who have bought into "the system" in a particular way, rather than measuring exceptional people.

Paul Graham's essay today (http://paulgraham.com/lesson.html) seems especially apt. These people have learned to hack certain standardized tests.


You can't just hack your way into being a STEM professor at a top institution. You actually have to do stuff, discover new things. It's analogous to actually building a good product if you're a startup founder.


"Hack" is a bad word, because it definitely involves work

But it's 30% research/academic work, 70% playing the system.


>You can't just hack your way into being a STEM professor at a top institution. You actually have to do stuff, discover new things. It's analogous to actually building a good product if you're a startup founder.

From my friends with PhDs, it seems like it's about 30% doing real stuff, 10% fucking around with grant funding paperwork and 60% trying to turn a single result into as many papers as possible in order to maximize impact scores and other metrics.


I'm getting a PhD and it's been 90% real stuff, with no regard towards impact metrics. It probably depends a lot on the subfield. My subfield is less "hot" than most, which some would take as a sign of unhealth, but it means we're spared all these other problems.


Statistically, most people who score low on standardised tests don’t do it because they’re too independent-minded to be part of the system, man!

Statistically most of them just aren’t that bright.


> Statistically, most people who score low on standardised tests don’t do it because they’re too independent-minded to be part of the system, man! Statistically most of them just aren’t that bright.

As measured by standardized tests?


No, just intrinsically.


The next question is how do we build as many people with traits like this as possible, and I assume it to be high quality education with a low stress home life. This is something that the US economy is not currently achieving, based on happiness and education metrics.

What should not be learned from this is that there are high caliber people, but instead that these traits call be learned, and I'm certain that there are schools teaching them, but I assume that those are quite foolishly only offered to specific subsets of people here, and that we will see further advancement in other societies as they find solutions to that exact problem by advancing educational practice through scientific experimentation on a national scale.


The real question is how can we take something that meets all the conditions necessary to satisfy the central limit theorem but that doesn't result in a normal distribution?

If you can solve that you can have your nation of smart people.


There absolutely are high caliber people. We know that IQ is the most powerful predictor of success and that you can't train it into people by any known means so it would be a waste to treat everyone like these geniuses.


>We know that IQ is the most powerful predictor of success and that you can't train it into people by any known means

One very simple way of increasing IQ is to repeatedly take IQ tests. There is a noticeable uptick in recorded score for people who take the same type of test the second time within a twelve month period.


That suggests a method for finding the "true" IQ; keep taking the same kind of tests until your score plateaus.


In order to understand my position on "IQ", you should read the book "Hacking Darwin", which actually covers the question of what IQ is at length. I hope that it alters your position on IQ a little bit, what aspects of it are hereditary, and how the nature of life can alter it.


Wouldn't the larger the study, the less statistically-possible success is? There's a set number of fortune 500 executives, for example.


Hah! I was a subject in that first study.


Please say more.


Are you “successful”?


He reads hackernews how much more success does one achieves?


Successful enough.


Looking at the comments, there needs to be a distinction between the "eminent" career vs what people define as "success" in the title.

The success in this paper is the equivalent of the Forbes 30 under 30 rankings except it's when you're 48. There will be genuinely eminent people who run a fortune 500 company, illustrious careers in academia, but the methodology of simply searching for someone years later has clear problems.

What if you were to win a Nobel prize at age 55 or 60 (very common). You may have contributions earlier but didn't meet the criteria.

On the reverse, like with Forbes 30 under 30, there are always people who master personal branding and PR over substance. Lot's of news articles about you doesn't equate to being eminent in the usual definition of the world- just well known.

FYI, I'm not disputing their thesis or suggesting their results are wrong, I just find it odd that evaluating people at age 48, they couldn't find an algorithm much better than Forbes 30 under 30 which we all know has a lot of BS.

I'd be curious to the results at age 75.


The methodology is much much better than Forbes 30 under 30. In Forbes you pay to nominate yourself. The list is accordingly mostly people in obscure startups seeking to use it to attract publicity. The "eminence" in this study is based on what people actually did.


Don't most people who attain the highest levels of executive positions peak around 60-65, i.e. near retirement age? Without definitely knowing the typical age of a CEO, that's my stereotype. And the candidates for president of the US seem to be trending even older these days.


It's interesting how "success" means "selective, high-workload, white-collar career" in this study. In other cultures, it just means having friends/relationships and not being stressed all the time. I suspect many of these successful types are failures in this regard.


What's more interesting is that the word "success" doesn't appear in the title or abstract at all, the functional word is "eminent", and their definition is definitely more complex than your summary:

"Eminent individuals were defined as those who, by age 50, had accomplished something rare: creative and highly impactful careers (e.g., full professors at research-intensive universities, Fortune 500 executives, distinguished judges and lawyers, leaders in biomedicine, award-winning journalists and writers)."


Yep, I really don't think that makes it better (can anyone explain why it is always psychologists who invent these kind of measures? Is there some test we can perform at age 13 so we can identify people who will go on to do this kind of research? A follow up paper perhaps, this will surely add to the "eminence" of the authors).

I have met my fair share of Fortune 500 executives, the majority are borderline braindead. Equally, I am unclear what "award-winning" journalist/writer might mean...it clearly doesn't mean success in any general way..."eminent" is just a very weird definition. Equating "eminent" with "highly impactful" and "creative"...dear God, are the authors actually human?

What is especially bizarre is this is a paper from psychologists who, presumably, should understand that equating success with other people thinking you are successful (which "eminence" clearly measures) is a fairly dysfunctional habit...the irony never ends.


>Eminent individuals were defined as those who, by age 50, had accomplished something rare

Absolutely every single person who achieves age 50 has done something rare.

The authors of the study (as professors themselves) simply measure their own biases here.


oooh touché


> In other cultures, it just means having friends/relationships and not being stressed all the time.

I’d love to watch you have a conversation with my Bangladeshi mom.


Bangladesh is not the only country with a culture!


I’d love to watch you have a conversation with a Chinese, Korean, Indian, Japanese, or Pakistani mom.


As a fellow-Asian, I usually like your contrarian commentary which brings 'our' experiences to this forum.

But I've gotta say, this is starting to seem like some variant of the good ole' days fallacy.

I had brown parents who spent countless hours teaching me maths, and stressing me out quite thoroughly when I didn't get it and when I had less than stellar grades.

Now an adult, and under medication to manage ICD/ADHD, I realize how all for naught all that stress was. More than that, I attribute my issues of having intimate healthy relationships to my eager brown parents who were never afraid to hand it to me when I fell short.


I think you're calling happiness success. Success in the form of an impactful career is important, not because it makes the person happy, but because it does more good for others than mere personal happiness. We've had thousands of years of people making friends with each other but almost never curing any diseases or improving quality of life in any lasting way until recently. The friends thing produces value that doesn't scale, so it's not as highly valued as, say, discovering new science which may help all future generations forever.


"In other cultures, it just means having friends/relationships and not being stressed all the time"

Sure, many people would say that's better or sufficient or whatever, but who calls that "success" who is not contrasting it with the grubby alternative of fame and fortune?

To me, you have defined basic mental health and implied that Americans don't value it, which is silly and offensive.


> In other cultures, it just means having friends/relationships and not being stressed all the time.

Could you share examples of such cultures that still exist?


Many European countries are like this, go visit somewhere like the south of Spain and it's a completely different lifestyle. There was an article here the other week about a country (Netherlands?) where you can take up to six months off work paid (at a lower salary) for burnout.


pretty much describes latin america. In colombia they have a saying "you don't need money when you have friends"


I suspect that psychometric tests would also be able to predict success in terms of having friends/relationships and living a low stress life.


I would guess that giving the establishment the finger would not bestow eminence.


It did fine for George Carlin


There are always multiple establishments. If one establishment doesn’t suit you, you can always go right next door to the other establishment labelled “the anti-establishment” and enjoy the company of like-minded individuals as you sneer at the other establishment and congratulate yourself on your independent-mindedness.

Nowadays the anti-establishment is bigger and richer than the actual establishment, which is why we all wear t-shirts to work and sneer at those who have to wear suits.

Are there groups that exist outside both the establishment and anti-establishment? Sure, but they’re weird and scary and if you join them you’ll definitely be banned from polite society.


I'm waiting to join the anti-anti-establishment group once they get stronger.


Good advice. Being banned from polite society is one of the best of all goals in this world.


Well, OK, there's an exception to every rule... although that may be prominence not eminence.

I can thank George for helping us to understand why our 'eminent' leadership have been so remarkably silent about the many pressing issues of our time. Silence and eminence have always been such good buddies.


Paywalled. It's on sci-hub.

https://sci-hub.tw/10.1177/0956797618822524


Ok so I see no controls for income/family wealth, etc

If anything I see selection for wealth: the samples in one of the bigger studies selected from “the 15 best STEM graduate training programmed”, which is broadly a proxy for wealth.

More over they don’t have negative samples: afaict their “study” was to take the top X% of students from some already wealth biased institutes and compare their performance to the overall average for everyone. What they should have done is compare the performance of the X% of the class they sampled from with the performance of the remainder of that class.

Then we could compare more similar starting points.

Obviously we would generally expect some correlation between skill when younger to 35 years later, but I would expect the entire group to do significantly better than the general average. Eg a person who got into Harvard because of their family history, and gets a C, is likely to do better than a person with a 4.0 from a state university. (General averages, obviously there can be outliers in every direction)


This is junk science that can best be used to reinforce existing stratification or create/justify a self-fulfilling prophecy of success. Best to ignore it and move on.


Their method would miss the likes of Blake, Van Gogh, Galileo, ...


SMPY identifies outlier ability via standardized testing taken before age 13 (for cohorts other than the very last one). All of Study 1 was selected this way. This may have some very loose correlation with wealth.

Is there a correlation between SMPY eligibility and getting into elite universities? Probably, but it doesn’t seem like an improper one. Is there a correlation between elite universities and life outcome? Sure.

Does that make the longitudinal study from age 11/12 into adulthood useless or content-free? Not in my opinion.

Only Study 2 was done on cohort 5, which was selected as grad students already at elite universities.


Cohort 4 was the only state university sample, but even that one used students in the summer programme (which is tied to wealth).

I also realized that SAT scores are heavily impacted by SAT specific training/tutoring that costs a reasonable amount (because as far as I can tell - as a foreigner - the SAT tests rely more on having been taught the exact way - and tricks - to answer SAT questions in the time available. Honestly if your test is good it should not have any real time pressure for the vast majority of participants)


The evidence says that SAT prep courses do not make much of a difference:

"From 1981 to 1990, three separate analyses of all the prior studies were published in peer-reviewed journals. They found a coaching effect of 9 to 25 points on the SAT Verbal and of 15 to 25 points on the SAT Math. In 2004, Derek Briggs, using the National Education Longitudinal Study of 1988, found effects of 3 to 20 points for the SAT Verbal and 10 to 28 points for the SAT Math. Donald Powers and Donald Rock, using a nationally representative sample of students who took the SAT after its revisions in the mid-1990s, found an average coaching effect of 6 to 12 points on the SAT Verbal and 13 to 18 points on the SAT Math. Many studies tell nearly identical stories. On average, coaching raises scores by no more than a few dozen points, enough to sway college admissions in exceedingly few cases."

(from https://www.aei.org/articles/abolish-the-sat-2)


Most people scoring 700+ on SAT-M experience no time pressure on the test. (That could be read as tautological, but I’m fairly convinced that there is no real time pressure that would dramatically shift the “SMPY/SET eligible” from not. The questions are straightforward and there’s more than ample time provided for those with a realistic chance of scoring 700+.)

The extent of my tutoring was a public school guidance counselor explaining to our whole class who was taking the test that there was statistically no penalty for guessing and how the “A is greater, B is greater, C means equal, D means indeterminate” questions worked. That was 30 minutes at most.


My high school did intense drilling for SATs as an after school thing. The average kid in my class had a 180 point improvement versus the PSAT as a result.

High SAT scores are evidence of high SAT score.


I got a 760 on SAT-M and I definitely felt tremendous time pressure. I am convinced I could have gotten 800 with no time limit. At the same time someone much smarter than me could probably get an 800 with half the time given.


> I also realized that SAT scores are heavily impacted by SAT specific training/tutoring

It's difficult to realize something that isn't true. The effects of training on SAT scores have always been negligible, less than the measurement error associated with taking the test one time.


When I was tested in that first cohort, I received no coaching whatsoever. I was 12 years old.


ah yes, the exorbitant cost of $35 for an SAT prep textbook


Well, even better, you don’t even need to purchase them—most public libraries have SAT prep books available for borrowing.

When I was in high school, I would check out whichever ones were available at the time and would photocopy the whole book. I drilled myself on the tests until I scored 1600s every time. Shows how flawed they are as a testing vehicle.


That's not evidence the tests are flawed. I was just having a very similar conversation with a friend who aced the physics olympiad. She said, "it's too easy to game, all I had to do was try old tests on my own". Well, hundreds to thousands of others were putting in more hours doing the same thing, in addition to enrolling in expensive prep programs, and they didn't do as well as her.

That is evidence in favor of the test's quality, not against it. There's no paying to win.


That’s a good point. You’re right if the aim of the test is to measure information retention, and not raw intelligence.


Never mind what it measures. My main point is that whatever it is, it's not simply "dollars spent on prep". Good tests provide an opening for the less well-off to succeed, despite the more well-off doing everything they can to rig it in their favor. My parents (and by extension me) owe a lot to them.


> Ok so I see no controls for income/family wealth, etc

There is no reason to control for those, and doing so would be an 'Everest regression': "controlling for altitude, Mount Everest is actually warm today". What you are asking for is like reading about IQ and income and saying "ah, but did they control for education?" Controlling is not magic pixie dust.


> What you are asking for is like reading about IQ and income and saying "ah, but did they control for education?"

that thing that literally raises the measured IQ number?

you know that practice can raise your score, right


Right - but there is also a relationship between IQ and educational attainment. So controlling for it means that you're measuring something fundamentally different. That's fine, but you can't just control for things without having a valid theoretical construct to justify it. (Well, OK, you can, and people do, and it's great for getting p<0.05 if you don't care about getting meaningful answers about the underlying questions.)


also, to clarify things a bit: in my first comment i got a bit mixed up (fuck yeah lifelong learning disability) trying to make my point about lifelong attainment and the IQ analogy and made it sound like the paper in question was about IQ, which is incorrect.

The point stands however: controlling for educational background is a thing you can do in such a study as according to the factors I pointed to, and gwern's glib comment is wrong


No, your comment was glib. You suggest controlling for variables which are going to create overcontrolling and the sociologist's fallacy, and you know perfectly well that family background is not independent of IQ and attenuates the relationship and your suggestion creates only an uninterpretable mishmash of a marginal effect which includes some causal effects but quite arbitrarily excludes others which run through controlled variables. I did not choose my example of education/IQ/income casually, but picked one which is very common in the literature which people commit because they think they are being rigorous when in fact they're only fucking up their own results and making them less meaningful. One's intelligence is not independent of one's family or genetic background! This matters to testing and things which can change! (Because it gives you residual confounding, if nothing else.) By the way, the proper way to do what you may or may not want to do (because your comments are so glib it's unclear to me what DAG you have in mind or what causal effect you want) is sibling controls, not family background controls, and you will be unhappy as to the results when they have been done.


I mean jesus, of course family background is not independent of IQ, but assessments of family background can obviously be informative in understanding how we get from there to here. To say that, quote, "One's intelligence is not independent of one's family or genetic background" is a flaccid truism. Nonetheless particular comparisons within assessments of family background can obviously in principle be informative even if they are hard to do.

Your suggestion - with this fringe talk about "the sociologist's fallacy - is to eliminate any explanation from any study which does not assign success to invariant psychological metrics.


> I mean jesus, of course family background is not independent of IQ

Then what exactly do you think you are doing when you rant about how they didn't control for it?

> Your suggestion - with this fringe talk about "the sociologist's fallacy - is to eliminate any explanation from any study which does not assign success to invariant psychological metrics.

Nothing fringe about it, I'm afraid. This is well-understood in the statistics fields like causality you seem so loathe to actually learn about.

The point is: when you control for something and then go on to make causal claims, you are not making a minor tweak. You are assuming an extremely implausible causal model and making strong inferences from there about what are real causes and what are not. You have entirely failed to argue for a causal model which would justify simply controlling away family background etc, and it seems from how your comments handwave it away in favor of nothingburgers, you don't even understand what you're arguing for.


This is a total nothingburger, a long paragraph attacking a strawman of an argument you haven't even solicited the details of


The burden is on the person saying that 'such-and-such a thing should be done' to explain themselves. You failed to do that, and you have still failed to do that, and you spend your comments complaining about being criticized, so your comments remain nothingburgers.


honestly it's disappointing when i see people with training in stats make these comments: I'm a fucking poet if I'm anything at all and yet I find myself drawn into these ridiculous discussions about stats (in which I have no formal training) where I feel like I'm one of the only people in the room who knows how to divy up numbers


it's a triviality to say that you can control for educational attainment (and opportunities for education as well as class) when accounting for differences in IQ: it's hard to do in practice because social science is complicated and requires ACTUALLY DOING THE SCIENCE so gwern's glib comment is just wrong here

clearly we're talking about the impact on IQ of whatever factors we make up to describe IQ, and obviously, and provenly, education (and therefore the ability to get education) is in play here, which gwern's glib rebuttal ignores


Really Gerard, are you going to lecture me on this? I am aware of practice, and more importantly, I am aware the IQ gains from education are hollow, from the Milwaukee Project on to Ritchie's paper on education increases. The practice that raises your score does not affect all subtests the same: increasing your vocab does not affect your backwards digit span or reaction time, or increase your longevity or income or...

And that's not relevant anyway. Are you trying to suggest people only get into SMPY because their parents made them practice the SAT? In the 1970s and 1980s, hardly anyone would have even heard of SMPY before their kids took the test in school.


you know as well as I do that zip codes are more heritable than IQ

and predict results pretty well too


No, I don't, because they're not: are identical twins more likely to live in the same zip code as fraternal twins? Seriously, think about it for a second. Or I'll put it this way: how much more likely are identical twins adopted out to live in the same zip code?

Nor does zip code predict remotely as well as IQ.


Are we just making up definitions of heritability now?

http://bactra.org/weblog/520.html


The way I read the study is that within the top N% group, higher scores predict "eminence". Not the top N compared to everyone else.


> If anything I see selection for wealth: the samples in one of the bigger studies selected from “the 15 best STEM graduate training programmed”, which is broadly a proxy for wealth.

Not everything is about wealth.

STEM graduate school doesn't select for wealth at all. Undergraduate admissions sometimes do, but essentially zero of the wealth-based admits major in STEM. In fact in my experience the effect is the exact opposite of what you say: STEM takes in whoever can do the work, regardless of wealth, and helps produce new wealth. In every institution I've been in, STEM people had the lowest average family wealth by far.


>Obviously we would generally expect some correlation between skill when younger to 35 years later

Though this is obvious, it wasn't obvious that the tests given were actually showing who had some amounts of skill. Though, this should more accurately be called "some amount of ability to become prominent in a field," as like you say things other than skill may cause both high scores and success later.


The justification for using the SAT-M in underage testing is in fact that: because the material would not have yet been covered in any classes, it tests ability to come up with a correct answer to a novel problem rather than skill. (Remember, educators have long been convinced that any kind of acceleration of gifted kids is poison, and SMPY's results have been a major reason that gifted kids can do any acceleration these days, so they're not just covering it in some accelerated math class.)


Uh oh, that seems to be rather out of date. When I took the SAT-M in 7th grade I recall having already seen harder versions of all the questions, because of math competitions. The same went for almost everyone I knew that took it.


When you took it, you weren't part of this SMPY cohort starting in the 1970s. Most of these kids certainly were not taking 'math competitions' or 'private tutors', they were languishing in regular classes, and that's why Stanley used the SAT-M.


FWIW I took it in the 80s (also around 7th grade though I forget exactly) and it was not like that for me. I wish I'd had more contact with real math that early -- school was such a waste of time.


They could be covering the material with private tutors or similar outside the classroom experience. That's why we can't say for certain the test is measuring skill.


It would be interesting to compare the trajectories of the bottom 5% of Harvard grads with the top 5% of UMass grads.

As a thought exercise, you can ask yourself: would you prefer that your doctor be from the former or the latter group?


Definitely the latter. Bottom 5% of Harvard are likely legacies with little motivation - and unlikely to get into medical school.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: