
Reliably measuring IQ through commercial puzzle games - davidiach
http://www.sciencedirect.com/science/article/pii/S016028961500104X
======
jasode
Well, reading the PDF was anticlimactic. Why? Because page 2 of the PDF
describes the 12 games that were used. They were games that test memorization,
puzzle completion, and pattern matching.

In other words, the "commercial video games" used in their research were
"brain games" and not Pac-Man, Mario Bros, and Call of Duty, etc.

If you've ever taken a _real_ IQ test where the proxy shows you cards to match
or pictures with "missing" data, the 12 video games they used are very similar
in spirit. It's not surprising that there is high correlation between the
scores of those particular types of video games and real IQ tests.

However, that's not to say there is zero correlation between blockbuster video
games like Call of Duty and IQ. Maybe another study will pursue that.

~~~
alexvr
Thanks for the warning. I was really hoping to see Halo or Quake or something,
because in my experience practicing these games in a casual-competitive way, I
hit a performance ceiling. In high school I played Halo nearly every day, and
a bright friend of mine, who didn't even own the game, was a surprisingly good
player for his limited experience.

On a similar note, I once read that reaction time and IQ are correlated (not
sure how strongly), which is interesting because you might expect motor
functions like that to be orthogonal to higher-order cognitive abilities.

~~~
ansible
_On a similar note, I once read that reaction time and IQ are correlated (not
sure how strongly), which is interesting because you might expect motor
functions like that to be orthogonal to higher-order cognitive abilities._

Speed is good, in many contexts. Say you've two people, and person A has
overall cognitive speed 25% greater than the other.

That doesn't just help with Jeopardy!, it gives you 25% more time to think in
normal conversations, on SAT tests, while playing video games, at work, etc.

Even a 10% slower speed differential from a baseline human is a big, big
disadvantage.

~~~
freyr
> _Even a 10% slower speed differential from a baseline human is a big, big
> disadvantage._

This would make sense if all thinking is equal and only the _rate of thinking_
varies. In practice, the quality of thought processes seems much more
important than the the rate at which such processes are carried out.

For example, attempting to assess "cognitive speed" can be very dangerous in
interviews. It seems like such a promising metric. A candidate who answers
questions 10% faster than average will generally be much more impressive than
a candidate who answers questions 10% slower than average, and it's tempting
to think person A will be 22% more productive. Of course, over time, it may
turn out that he occasionally provides more _bad_ solutions to problems, or
can only solve superficial problems, or solves the wrong problems. And you
realize the person who consistently solves problems correctly (albeit at a
slower pace) is a better choice than the person who introduces new problems as
quickly as he solves existing problems.

It could always be argued that person B is actually thinking faster, but
thinking through the problem much more carefully, causing him to verbalize his
answer later. Or perhaps both people are thinking at the same speed, but
person B has simply thought _more_. Maybe person B is actually thinking slower
but more effectively. From an outsider's perspective, we can't know what's
happening under the hood, which makes "cognitive speed" a weak metric for
judging effectiveness in practice.

~~~
setpatchaddress
This. You want some easy proof? Watch a video of Steve Jobs answering
questions, like the WWDC 1997 Q&A video. Time how long it takes him before he
starts speaking after the questions are asked, and note the quality of the
answer.

~~~
veidr
Yes, but presumably in those long pauses, a faster thinker would have time to
consider more possible replies, make predictions on how the audience would
receive them, and generally make more adjustments and improvements than a
slower thinker.

In other words, faster cognition would probably help you formulate a slow,
considered reply just as much as it would help you answer more quickly. That
might be especially helpful in a public speech to a group asking unplanned
questions.

I agree with freyr, though, that quality of cognition/ideation matters more
than speed, and doubt that 'thinking speed' is necessarily correlated to
better thought output.

Like IQ itself, faster basic cognition probably means _something_ , but I
don't think anybody knows _what_ it means or how (or even if) it relates to
"intelligence" (whatever that means).

In my personal experience, I have seen more great ideas and solutions come
from "weird thinkers" than "fast thinkers". I realize that sentence does
reduce to the cliche, "Think outside the box, bro," but it seems true over the
course of all that I have observed in my own life.

~~~
ghaff
In that example, it may also simply have been that Jobs had been prepped (or
prepped himself) thoroughly with answers to a lot of likely questions.

I don't put a lot of credence to the speed thing here. I know one senior
executive who made a point of writing down question on cards when he was asked
them live at an event. Which I always thought was a rather clever approach to
1.) Make sure he understood the question correctly and 2.) To give himself
some time to formulate the best response.

------
Uptrenda
These academic pay walls really are a cancer to learning and human knowledge.

~~~
semi-extrinsic
In proper scientific fields, authors post preprint PDFs on freely available
sites like arXiv.

------
mohnishc
A year and a half back, when I was in the 10th grade, I did some similar
research on the effects of video games on the cognitive skills of students. I
specifically designed games for this research. I wanted to further this study
to test commercially available video games, but due to lack of support and
finance, I could not (I'm still in high school).

You can go through the paper here if you want:
[http://www.emerginginvestigators.org/2015/01/gaming-
cognitiv...](http://www.emerginginvestigators.org/2015/01/gaming-cognitive-
skills/) OR [http://arxiv.org/abs/1504.01665](http://arxiv.org/abs/1504.01665)

TL;DR; of the paper: I divided the class into two groups, one that played the
video games and one that did not. Two tests were taken - one before the
children were allowed to play games and another at the completion of one week.
Children were made to play games for an hour daily. The results showed that
gaming does improve cognitive skills. I was particularly surprised by the
substantial improvement that it led to in scores.

PS: Anyone who wants to take this further or maybe has ideas for implementing
this, please feel free contact me. I would love to help you out on this.

~~~
palguay
I am the founder of [http://brainturk.com](http://brainturk.com) . I would
like to talk to you on this . What's your email ?

~~~
mohnishc
You can send me a connection request on Linkedin
([https://in.linkedin.com/in/mohnish7](https://in.linkedin.com/in/mohnish7))
and once I accept it, you can check out my email address. I'm a bit skeptical
about sharing personal data on public forums.

------
kragen
Maybe a better summary of this paper would be "IQ tests presented on a
computer screen produce scores highly correlated to IQ tests presented on
paper," since they didn't use what you would normally call "video games."

------
ryanmarsh
Was thinking this knowledge could be used as a recruiting tool, then
immediately this came to mind...

"Greetings, Starfighter. You have been recruited by the Star League to defend
the frontier against Xur and the Ko-Dan armada..."

~~~
reustle
I don't know if this is what you're referring to, but this is exactly what
patio11 is working on now
[http://www.starfighters.io/](http://www.starfighters.io/)

~~~
DaCapoo
He's referring to the (1984) movie "The Last Starfighter", in which an arcade
game is used to recruit a fighter pilot.

It's also where patio11 + ptacek got the name for that project.

[http://www.imdb.com/title/tt0087597/](http://www.imdb.com/title/tt0087597/)

------
stared
They are already used for that purpose:

"The largest human cognitive performance dataset reveals insights into the
effects of lifestyle factors and aging" Daniel A. Sternberg, Kacey Ballard,
Joseph L. Hardy, Benjamin Katz, P. Murali Doraiswamy and Michael Scanlon,
Front. Hum. Neurosci., 20 June 2013
[http://dx.doi.org/10.3389/fnhum.2013.00292](http://dx.doi.org/10.3389/fnhum.2013.00292)

------
hyperion2010
One addendum here is that you can use them to measure g, but as soon as you
start trying to use them to _improve_ g any value of g you measure using them
will be incorrect due to learning effects and there is no reliable evidence
that playing such games can improve g. Source: wrote my prelims on this.

------
gaelow
reliably <> accurately

Wow, some psychology professor who read Ender's Game (Probably watched it
since, come on, psychology is science at its laziest and the timeframes match)
decided to make a quick buck by publishing an online paywalled paper on the
"discovery" "for the first time" that people give up signs of their
intelligence when observed performing an activity that requires them to use it
(which matches the description of an infinite number of activities...).

Hey, I can reliably measure your intelligence by reading your comments on
Hacker News. May I have your cash-money bucks, please? $_$ I have been working
hard with hundreds of my students to prove it! For the first time ever! $_$

The fact is that you need specifically designed games for an accurate test.
Most video games just won't make the cut. Even a combination of them: take the
top 10 from steam. I dare anyone to accurately measure the IQ of their players
just by looking at them them play in whatever lab environment they want. An
those would be pretty boring. They would be in fact, tests.

~~~
stuxnet79
Publish or perish mentality ... or ... good science hindered by a hyperbolic,
misleading title. The former is cancerous but the latter can be easily
rectified. I myself can't call it because I didn't click the link hoping I
would get a TLDR in the comments.

------
payne92
Is there a link to the paper?

~~~
ingenter
Fuck academic paywalls, here's a pdf.

[https://mega.nz/#!510ngJYY!S2_PGUIrprdnc5Byf5y1MNIbonm9-v3Y2...](https://mega.nz/#!510ngJYY!S2_PGUIrprdnc5Byf5y1MNIbonm9-v3Y2zCmdsSMJ4k)

~~~
vengadora
I haven't a credit card, i am a penniless person studying on a public
university that does not teach (almost) anything good, and wants to become a
game dev based on free software philosophy, currently researching about
edutainment and... zomg... you pwn, thanks! really thanks!

every article i want to read about this, is via payment only, so it is
becoming really hard to research about this subject :( but today, you saved
the day!!

~~~
delecti
I don't know what specific university you're going to, but many university
libraries have free access to journals for students.

You should do some research on your school's library's website.

~~~
kbutler
Sad that we need to research how to get access to research.

How much impediment to the progress of the arts and sciences should we allow
copyright to impose?

------
nevinera
We can estimate a metric of intelligence by observing people performing tasks
that require thought and problem solving?

O.O

------
tyrion
> Here we show, for the very first time, that commercial video games can be
> used to reliably measure individual differences in general intelligence (g).
> One hundred and eighty eight university undergraduates took part in the
> study.

Is this a joke? _reliably measure_ ? _188 students_? So are you trying to tell
me that if I could play Call of Duty with A. Einstein I would be humiliated
despite him never having played a video game in his entire life?

~~~
psycr
You would do well to learn about effect size and p values. Your common sense
guess at what might be reliable is incorrect, which means your doing a
disservice to your own understanding of the world.

