Hacker News new | past | comments | ask | show | jobs | submit login
Schooling in adolescence raises IQ scores (2011) [pdf] (pnas.org)
54 points by lainon on Dec 14, 2016 | hide | past | favorite | 46 comments



Doesn't this just mean that IQ Scores don't measure what they're supposed to? My understanding is that IQ scores measure how smart you COULD be, all things considered. But if you can increase this with schooling, isn't it just how smart you are right now?


I think the way to understand IQ scores is that they are proxies for measuring general intelligence (general intelligence being ability to solve novel, abstract cognitive problems). The art of designing an IQ test is in making it as good as proxy as possible. Usually this is done by either testing stuff that everyone has exposure too under a time limit (general knowledge, basic math, vocab) or by testing stuff that people generally don't spend time practicing (Raven's matrices). But if a person deliberately tries to cram for an IQ test, they may increase their score, without increasing their general intelligence. And so with any policy that increases IQ scores, you have to try and figure out if it really increased general intelligence, or just increased their familiarity with the kinds of questions that appear on an IQ test. This can be tricky to figure out.


Also, "general intelligence" ("G) is a hypothesized abstract concept, not well-defined. The non-existence of a non-gameable IQ test shows that G is not well defined.


From other quotes/pdf

"The measure of IQ used in the analysis was obtained from tests of cognitive ability administered by the Norwegian military to all draft-eligible men at approximately the age of 19 y as part of the universal military draft in Norway. "

So they weren't even IQ tests.

So there are issues with the results, but the results as they stand seem to contradict current understanding.

Is the pdf significant or not? You'd need an expert to say.


IQ tests really need to be take cold with no cramming before hand.


That would mean only your first IQ test ever can be meaningful. Every IQ test also teaches you a bit about the kind of thing IQ tests are looking for. I don't think any kind of a training is ever going to get you more than 10 points extra, but I'd be very surprised if at least a few points isn't doable.

There's also the fact that IQ tests need to be designed for a specific culture. Taking an IQ test for a very different culture is likely to cost you a couple of points. They're unavoidably subjective.


> I don't think any kind of a training is ever going to get you more than 10 points extra

I think it could easily be a lot more than 10 points. I once (ages ago) did an IQ test from a book and then 2 days later did another test from the same book and scored 27 points higher.

From this I concluded that either I had become vastly cleverer over those 2 days or that IQ tests weren't a good measure of intelligence.


From my own experience, you're able to prepare for IQ-tests. I scored slightly below average (below 90 actually) on the IQ-test I did with 7. Average (104) on the one I did with 16. And actually made it in to Mensa with a result of 144 on the IQ-test I did aged 18. I was in a good mood the day I did that IQ-test, and learned math, memory techniques etc. a few weeks before. So, IQ-tests really aren't that reliable.


Were your IQ tests run on a 1:1 basis with a trained practitioner over the course of several hours? If not, they're not one of the actual formal IQ tests, but either some form of proxy or some random unrelated test that someone has slapped the label on.

This is why there's so much confusion about "how can the IQ test be considered studied and reliable, bah!" - because the ones that are properly performed are labour-intensive and require a lot of training - and this is why results are all over the place. It's just not the case that $RANDOM_YAHOO[1] on the internet knows more about the caveats around IQ tests that the professionals who apply and study them.

Disclaimer: have formally studied 'intelligence' (and cognition, perception etc) and been given a partial IQ test myself by a properly trained professional, which came with the caveat "the results can't be properly used, because you're familiar with the process, which confounds the results".

[1] I don't mean you, I just mean that this is a common complaint


The Mensa tests are generally done in a room with an experienced practitioner. The "take home" tests are just used as a marketing tactic and/or as a way to decide if you should even bother with a formal test.


Maybe your actual IQ was varying, and the tests were accurate enough to show that ?

We should consider the real possibility that both a) IQ tests are a reasonably good proxy for general intelligence AND b) preparation by training on problem solving improves IQ

Id really like to know how much IQ is improvable. If the brain is plastic and physically adapts to training, what are the most effective methods? It seems an important issue, not least for schooling.


Depends on what you mean by IQ. If IQ is simply the result from the test, then it's clearly varying. If you define IQ as some sort of intrinsic and mostly unchanging ability for intelligence, then IQ tests are clearly very imperfect.

Part of the problem is that there are too many slightly different definitions of IQ and intelligence, and what's worse: we're not even sure what exactly we mean by "intelligence".


Ya, turns out people can learn just about anything with practice


Around here those tests seems to be careful to avoid maths, memory etc and only try to test your ability to recognize patterns or find missing solutions.


>Around here those tests seems to be careful to avoid maths

I was required to do mental calculation during the WAIS-IV I did.


So you practice patterns to raise your score.


No, not really. Never felt I had time.

I also think they try to introduce elements that makes it hard to prepare for the test.

That said if someone has never taken one of these then getting used to how the test works and doing a sample test could help them get a better score right away.


> Doesn't this just mean that IQ Scores don't measure what they're supposed to?

No, it does not mean that IQ scores don't measure general intelligence.

It does mean that things some people believe about general intelligence may need to be reevaluated.

> But if you can increase this with schooling, isn't it just how smart you are right now?

That intelligence (and particularly IQ) can change over time, and that there were environmental contributors to that change has been well-know for some time.

That education in adolescence is a relevant factor is perhaps new, and certainly challenges the treatment of education and intelligence as orthogonal factors that contribute to outcomes.


For those of us with only a layman's understanding of the term "general intelligence", would you mind posting a definition?


The best way to understand general intelligence is to understand why general intelligence is around and what phenomenon is explains.

It started long ago when everyone thought you could be good at Math, or good at History, or good at Science, and sometimes good at all 3.

One day Suzy(named changed and story overly simplified for explanatory purposes), a very smart math teacher, was promoted to principal. So she finally had the chance to look at students grades from every class. And she found something curious. Her students that succeeded in her Math class also succeeded at Science. And kids that did well at History, scored highly in English. So Suzy decided to call these newly discovered traits "Verbal Ability" and "Quantitative Ability" respectively.

But Suzy didn't stop there, her investigations found that Verbal and Quantitative were correlated with each other. So she invented a third trait which was the combination of Verbal and Quantitative ability and she called that g for General Intelligence. Of course Suzy wanted to measure this g with a test so she could predict(though by no means perfectly) a student's Math, Science, English, and History grades from just one score. She settled on IQ as the measure of this test, and thus IQ was born.

Basically when we give someone a whole bunch of different tests we find that if someone does well on one, they will probably do well on others. For instance if we know someone is really good at theoretical physics we know they'll be good at working memory, and solving silly matrix puzzles. But if we know someone is really good at poetry we don't know as much about how good their working memory is, or how good they are at solving silly matrix puzzles.


I strongly believe someone's conformity to obedience, authoritarianism and conventionalism plays a substantial factor in their performance.

I've dealt with many people with advanced degrees from top tier schools who seem to be completely unable to grasp subversive or non dominant thought patterns.

For years I was stunned, thinking "how on earth could they have such impressive academic credentials and be struggling like this."

The entire institution continually fosters and rewards a set of specific things with limited application.


Rebelliousness, while may be a valuable skill, is different from intelligence.


Presume there's a multiple choice with two decent answers. A "rebellious" person would see the merit in both and essentially coin toss. A conformist would mimic what the professor sees and pick his right one.

The rebellious person did a deeper, objective, thoughtful analysis but did worse than the obedient mimic.

Often it's a D student who knows the material the best, has the most passion for the subject, is the smartest in the class, and puts the most work in. I saw it all the time while I was TAing.


> A "rebellious" person would see the merit in both and essentially coin toss.

Perhaps a smart person is a rebel roleplaying as a conformist. In a game theoretic sense you get the most advantage without the penalty. It is one explanation I suspected but haven't seen for why contradictory traits might be expressed by somebody in or close to power.

> A conformist would mimic what the professor sees and pick his right one.

Less Wrong calls this 'Guessing the teacher's password'.

It's the art of mimicry. That there is no penalty for this in the education system is an indictment.

> Often it's a D student who knows the material the best

Even most A students will agree with you. There is an argument made by Alfie Kohn that grades should be abolished.

http://www.youtube.com/watch?v=lfRALeA3mdU


Sorry but I completely disagree. The D students I knew were dumb as rocks. I got As through my engineering degree with a few Bs in a few of the classes I was less interested in and I have to say, it's completely ridiculous to suggest the D students knew more than I did.

You could either solve the problems using techniques taught or you couldn't. Grades were a direct measurement of this ability. Maybe grades don't matter in Psychology or Sociology class, but I don't want to rely on an electrical system designed by the D students in my classes. It would probably kill me.


The claim wasn't about all D students or that there was an inverse correlation. There's certainly slackers with Ds...


Unorthodox thinking is a cognitive ability. what kind is it if not intelligence?


Creativity.


> Doesn't this just mean that IQ Scores don't measure what they're supposed to? My understanding is that IQ scores measure how smart you COULD be, all things considered.

Actually there is no precice definition of intelligence outside IQ tests. My psychology professor had a nice saying: "Intelligence is what an IQ test measures".

IQ tests in general have very low prognostic reliability, e.g. they neither say how smart you could be (neglected children can easily rise more than 20 points after living a a few years with a nurturing foster family) nor how successful you'll be in your professional life (where motives and motivation play a more important role).


> Doesn't this just mean that IQ Scores don't measure what they're supposed to?

The most cautious defensible position, in my opinion, is that "we can crudely measure a quantity that is related to the folk notion of intelligence"[1]. IQ tests are good, not great, at measuring something which 1) correlates to performance on just about any task that we regard as "mental", and 2) which we are thus far unable to divide any further – it appears to be impossible to split this "something" into a bundle of uncorrelated disparate abilities[2]. This "something" is called general intelligence, or g. However, all IQ tests are trainable and have different emphases; there is no bulletproof g test.

> My understanding is that IQ scores measure how smart you COULD be, all things considered.

The g factor is not a potential. It is defined by statistically extracting a factor common to all (or all minus epsilon) measures of intelligence, which factor is then found – empirically – to predict future performance. It does appear to measure a kind of intellectual potential[3], which is one reason psychologists are interested in it.

So, does this notion of general intelligence contradict the findings linked in this post? Logically, not at all; there is no reason a priori why the general factor should not be malleable. The question is if the general factor is in fact altered. Not an easy question to answer, due to the fact that g, being a statistical construct, cannot directly be measured, but must be inferred from other observations and tests.[4]

Since the linked Brinch-Galloway paper was published in 2011, others have attempted to determine whether education changes g[5]. Briefly, the authors measure a set of other cognitive skills, only partially correlated with g, but which cognitive tests also measure, and compare three models:

A) that education affects these skills via g; B) that education partially affects these skills via g, but also directly affects some of these skills; C) that education does not affect g, but only these domain-specific cognitive skills. In this last model, the positive effects of education on cognitive test scores are fully mediated by domain-specific improvements.

They find that Model C is favoured (p=0.04). The state of play is that education appears to improve performance in things like "Logical Memory" (the capacity to recall ideas and the logical connexions between them), and that this is what causes gains on IQ tests.

Practically, this means that education is good, very good even, but that there's something that it doesn't seem to touch. At this point one may wonder what exactly is so great about g compared to domain-specific cognitive skills. For probably the majority of jobs out there, domain-specific skills are exactly what are required; provided one has some moderate level of general intelligence to cover exceptional situations, the rest is trainable. If this becomes the received wisdom I wouldn't exactly be upset, as it would likely take a good deal of the heat out of "the IQ controversy". Perhaps instead of "general intelligence" we would refer to "abstract generalization capacity" or "cross-domain transfer ". With the great g threat on the back-burner for the time being, the discussion on trends towards increased demand for generalization skills on the decadal scale would then have room to breathe. Some of the proposed solutions[6] are rather radical for the current political climate...

[1] Steve Hsu, http://infoproc.blogspot.co.il/2013/04/myths-sisyphus-and-g.... [2] Not for want of trying – most prominently Howard Gardner's theory of multiple intelligences, which is quite popular outside of psychology, but which lacks empirical evidence (Waterhouse (2016), http://dx.doi.org/10.1207/s15326985ep4104_5) [3] With a few exceptions, by far the most significance of which is childhood neglect, as commented by bildung at https://news.ycombinator.com/item?id=13183521 . But his point on motivation as a factor is misplaced – yes, motivation matters, but on average g will also have its effect. It is the combination of g and other specific factors which predicts any given facet of an individual's life. [4] All this hairsplitting is an unfortunate consequence of the lack of a firm reductionist model of g. If we had such a model (grounded in neuroscience, for instance) we could just study the effect of educational interventions on the brain directly. Of course we can (and do) do this anyway, but it tells us a lot less without the model. [5] Ritchie, Bates and Deary (2015), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4445388/ [6] http://www.businessinsider.com/how-to-genetically-engineer-i...


IQ scores have to do with a lot of things. IQ can lower due to:

- not meeting basic nutrition requirements during gestation and early life

- not consuming enough iodine. iodine gets added to salt to ensure people consume it.

- exposure to lead and other toxic substances.

...and mental stimuli.

Some people (notably alt-right Twitter accounts), promote that idea that certain races have significantly lower IQ, spreading maps of average IQ per country. It is my duty to call out that as wrong. Having the aforementioned factors in consideration quickly refutes that idea.

Some people point out that there are genetic markers associated with higher IQ. To be honest, we haven't listed all of them yet. So I would stay quiet before pointing out that certain population doesn't have certain SNP.


>not consuming enough iodine. iodine gets added to salt to ensure people consume it.

source?

Also, it's ridiculous to argue that genetics don't affect IQ. There are many genetic disorders that impair cognitive skills.


The first one I found: http://www.medscape.com/viewarticle/804569

So beware of feeding kids and pregnant women with sea salt only.


You just rebutted an argument he didn't make.


Woops, you're right.


What really raises the score is practising on IQ test type of riddles and questions.


It would be interesting to see some sample tests. From the data section: The measure of IQ used in the analysis was obtained from tests of cognitive ability administered by the Norwegian military to all draft-eligible men at approximately the age of 19 y as part of the universal military draft in Norway. The data from these tests have been widely used and interpreted as IQ scores for research purposes.

Also interesting, the historical summary of the "distributed" nature of the Norway educational system: The reform [...] required that all municipalities provide their youths with an additional 2 y of standardized education in the eighth and ninth grades. [...] The reform was introduced at the municipality level, the lowest of three administrative levels in Norway, with the other two levels being the national level, which has responsibility for higher (tertiary) education, and the county level, which has responsibility for secondary education. [...] Following the reform, the new type of middle school was administered by the municipalities. Each separate municipality was able to introduce the full compulsory schooling reform after local officials submitted a reform plan to a national committee, provided national funds [...]

It sounds like they used a bottom-up approach rather than a plan forced from on top, which seems like a good thing.

This makes me want to learn more about how school systems / school boards work. It seems to be a city-level organizational structure, but there are provincial/state, and federal rules too. This is a lot of APIs: you pay federal, provincial, municipal, and school taxes, and then some portion of your taxes comes to the schools. Surely there's room for simplification...


IQ scores usually increase with education in youth but as an individual becomes an adult the child age scores drop off.


Guys before you take this too seriously, PNAS is a publication repeatedly criticized by stats experts like Andrew Gelman. NHST techniques are bad, and there are many papers showing almost zero effects of education.

Plus your prior on this should be pretty low given Turkheimer's 3 laws of behavioural genetics. Especially since education is more like a shared environment variable rather than a unique environment variable.


You shouldn't invoke Turkheimer's three laws paper (which is a good paper) to make a statement with which Turkheimer would disagree. Turkheimer certainly agrees with the factual observation that formal education (which is part of the environment not necessarily shared by siblings in the same family) can raise IQ. You can look up a paper on which he is a co-author for more details. (I correspond with Turkheimer from time to time, and know many of his co-authors and colleagues.)

The link to the paper here is from Eric Turkheimer's faculty webpage at the University of Virginia, which I visit often to read his latest publications.

http://www.people.virginia.edu/~ent3c/papers2/Articles%20for...


Turkheimer's laws are not the property of the man himself! I think it's pretty safe to say there's no clear winner on what the effects of schooling is on the margin at least.

You're right to correct me on school being part of the nonshared environment that's my mistake, you probably know more than me how complicated it is to extract meaning from the nonshared environment. Also the temporary nature of the improvements are very confusing. Thanks for the paper.

Here's a summary by SSC (not really an expert but worth reading): https://slatestarcodex.com/2016/05/19/teachers-much-more-tha...


Professor Gelman's preferred term is "PPNAS". You mustn't forget the prestige.


I'm curious why this was downvoted.


Please don't change the subject to downvotes. It never works, which is why HN has not one but two guidelines against it: https://news.ycombinator.com/newsguidelines.html. Also, as typically happens, that comment is no longer net-downvoted.

We detached this comment from https://news.ycombinator.com/item?id=13180415 and marked it off-topic.


But I wasn't complaining about being downvoted, nor was I inviting anyone to downvote me. A comment not written by me was downvoted by someone who didn't seem to be represented in the replies to it, and I really did want to know why, out of actual curiosity.


I appreciate the curiosity part, but it really is off topic.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: