Out of interests sake: how does one validate an IQ test?
Thanks for asking the important follow-up question that is so rarely asked. To validate an IQ test (a test that purports to estimate "general intelligence"), one must first reach a consensus among test designers about some sign of intelligence that is detectable outside the testing room. Over the years, psychologists have proposed various behavioral characteristics of human beings as signs that those human beings are "intelligent," with entering challenging, high-income occupations that require a lot of higher schooling being one criterion proposed for adult IQ tests, and being precocious in school and having good grades and good teacher ratings being one criterion that is proposed for child IQ tests.
One finds a sample of persons to take a new brand of test in its norming administration, and rates those persons by external criteria of "intelligence," weighting those criteria by consensus, and then checks the rank-order correlation between the ranking of the test-takers yielded by the IQ test and the ranking of the test-takers yielded by the validation criteria. There will NEVER be a perfect ("1.0") correlation between the test and the validation criterion, just as there is never a perfect correlation between IQ scores on one occasion and IQ scores on another occasion on the same brand of IQ test by the same group of test-takers.
There is enough play in the joints in both IQ test scoring (whatever the brand of test) and ranking people by other validation criteria (whatever they are), that strictly speaking one can't say that there is any all-time, universally significant ranking of human beings by intelligence. But a close-enough-for-government-work validation study would show an IQ test having correlations above .80, and perhaps even above .90, in comparison with previous brands of IQ tests, or in comparison with subsets of its own item content, or in comparison with some well regarded external validation criterion.
For reasons mentioned in another comment in this thread (above?), there is especially little reliability, and hence especially little validity, for IQ scores far above the population mean, and thus it's very hard to devise a validation criterion that would sort, say, members of Mensa
Thanks for asking the important follow-up question that is so rarely asked. To validate an IQ test (a test that purports to estimate "general intelligence"), one must first reach a consensus among test designers about some sign of intelligence that is detectable outside the testing room. Over the years, psychologists have proposed various behavioral characteristics of human beings as signs that those human beings are "intelligent," with entering challenging, high-income occupations that require a lot of higher schooling being one criterion proposed for adult IQ tests, and being precocious in school and having good grades and good teacher ratings being one criterion that is proposed for child IQ tests.
One finds a sample of persons to take a new brand of test in its norming administration, and rates those persons by external criteria of "intelligence," weighting those criteria by consensus, and then checks the rank-order correlation between the ranking of the test-takers yielded by the IQ test and the ranking of the test-takers yielded by the validation criteria. There will NEVER be a perfect ("1.0") correlation between the test and the validation criterion, just as there is never a perfect correlation between IQ scores on one occasion and IQ scores on another occasion on the same brand of IQ test by the same group of test-takers.
There is enough play in the joints in both IQ test scoring (whatever the brand of test) and ranking people by other validation criteria (whatever they are), that strictly speaking one can't say that there is any all-time, universally significant ranking of human beings by intelligence. But a close-enough-for-government-work validation study would show an IQ test having correlations above .80, and perhaps even above .90, in comparison with previous brands of IQ tests, or in comparison with subsets of its own item content, or in comparison with some well regarded external validation criterion.
For reasons mentioned in another comment in this thread (above?), there is especially little reliability, and hence especially little validity, for IQ scores far above the population mean, and thus it's very hard to devise a validation criterion that would sort, say, members of Mensa
http://www.mensa.org/
or members of the Study of Exceptional Talent
http://cty.jhu.edu/set/index.html
or members of the Davidson Young Scholars program
http://www.davidsongifted.org/youngscholars/
into their "true" rank order by IQ, not to mention that IQ scores for the same individual can and do change over the course of life.