
SAT: Getting the lowest score possible - solipsist
http://www.colinfahey.com/sat/sat.html
======
Steko
Spoliers (it's very long): he got one question correct, sadly.

I feel for him given the preparation and detail he put into it. I missed a few
SAT questions my senior year but helpfully, The College Board decided to make
the test easier the following year and magically recalibrate my scores to
perfect. It was too late for colleges to care and the expected thousand
girlfriends never materialized but it did make me feel warm and fuzzy inside.

~~~
Evbn
I didn't get the recalibration :-( one off perfect forever. same as my LSAT.

------
swordswinger12
I honestly don't know whether to be impressed or frightened by the level of
obsessive attention to detail on display here.

~~~
devcpp
I visited some of the article from his homepage. This guy is an all-around
genius! He writes every page on his website in this manner, on so many
different fields. Bookmarked.

I hope no one will only look at his SAT.

------
zheng
This is possibly the funniest thing I have read this year. As an aside, the
amount of thought and focus this guy obviously can devote to a single topic is
amazing. I really hope someone is paying him lots of money (or whatever he
prefers) to use his talents for good.

~~~
jobu
It was funny, but in a somewhat disturbing, manic way. I skimmed much of it,
but the reference to Weekly World News caught my eye and still can't figure
out the point of the tangent even after going back and thoroughly reading most
of the article.

Section 8.3.10 starts with possible numeric encodings for answers, but then
jumps into a philosophical question about admission tests for religion and
ends with the reference to WWN and the headline: "10 MORE COMMANDMENTS FOUND!
YOU WON'T BELIEVE WHAT THEY SAY!"

------
nateberkopec
A lot of the SAT-apologist comments here missed one of the most amazing parts
of the post:

> "The correlation between [...] combined verbal and math scores and freshman
> GPA is .52;"

.52! And it's pulled _straight from the College Board's Terms and Conditions!_
And later on, it goes on to explain that high school GPA's correlation is just
.54! The graph he produced to visualize the scatter involved with a .52
correlation is both hilarious and horrifying.

~~~
tzs
Ideally, shouldn't there be weak or no correlation between score and college
grades, when your sample is all SAT takers? My reasoning (not entirely non-
serious) is that people who score high on the SAT get into hard schools and so
get their GPA smacked down to average, whereas those with poor SAT scores get
into easy schools where they can earn an average GPA.

~~~
yummyfajitas
Harder schools don't necessarily have harder grading. I taught at both Rutgers
and NYU - if anything, Rutgers was tougher than NYU.

At Rutgers we failed people regularly. At NYU this would raise red flags and
generate huge numbers of complaints (from students and parents).

~~~
rdl
I thought Rutgers was actually a harder/better school than NYU, at least for
engineering. Rutgers is obviously better for film studies.

------
beatgammit
I admit, I didn't read the whole thing, it was super long.

I personally think the ACT (and by association the SAT) is a better measure of
intelligence than a GPA, and I would love it if companies used that more often
as a filtering metric. Obviously, grades in general aren't the best metric,
but they're simple and generally pretty reliable.

If I were hiring somebody, I'd look at a combination of test scores and
personal achievements. If I were hiring a programmer, I'd filter by test
scores, then look at examples of projects the applicant works on/is associated
with. Grades aren't as important in my opinion.

If that's the premise of the post, I totally agree. If not, I'm not willing to
read a forever long post that forces me to relive the horror of the testing I
went through before I got into college.

~~~
thaumasiotes
You're looking for the canned tokenadult post which summarizes as follows:

[paraphrased] "Decades of research have established that the most effective
indicators of whether someone will be a good hire are (on the one hand) a
general intelligence test, and (on the other hand) a work sample. Outside the
US, the use of intelligence tests in hiring is common; within the US, it
subjects the hiring agent to legal difficulties. If you are hiring outside the
US, use both. If you are hiring inside the US, use work samples."

~~~
tokenadult
Actually, the effective hiring procedures of either kind are rare everywhere,
but, yes, that is otherwise a fair summary of my FAQ post on company hiring
procedures,

<http://news.ycombinator.com/item?id=4613543>

which I too thought seems to fit in with several of the comments in this
thread. Regrettably, most companies miss out on opportunities to use the best
available hiring procedures, preferring the traditional methods to methods
validated by research.

------
mehulkar
This reminds of me playing Hearts on a PC. You can try really hard to get a
good score, or you can get the worst score possible and win the round[1]. I
didn't read the whole article, but it was pretty clear that to get the lowest
possible score you have to know the correct response to every question. (A
little less black & white for the essay section, but you get the gist.)

Colleges should have a lottery admission available to people who _can_ get a
perfect score on the SAT/ACT. Students would inadvertently study harder and
learn proportionally more than if they were to study hard enough to get a
perfect score.

[1] <http://en.wikipedia.org/wiki/Hearts#Shooting_the_moon>

~~~
gizmo686
> but it was pretty clear that to get the lowest possible score you have to
> know the correct response to every question.

Not exactly. To get the lowest possible score, you only need to know a
incorrect response to every questions. This is a very different thing, as it
is not uncommon for a question to have some obviously incorrect answers.

~~~
mehulkar
Knowing an absolutely incorrect answer to every question is difficult to
ensure. It leaves things up to chance since you cannot prepare for the exact
questions you get. On the other hand, knowing all the correct answers is
perfectly doable.

~~~
bagosm
This is not true. For example you know that the result of 5.123.443 x
9.999.999 can't be 42. But you dont immediately know the correct answer, which
is harder.

~~~
mehulkar
Making the answer hard to deduce/calculate is only one side of the difficulty.
The other side is making the question hard to decipher. I don't think the SAT
has any questions that are as straightforward as your example, but I could be
wrong.

------
imjk
Isn't it considerably easier to get every question wrong than it would be to
get a perfect score (every question right)? For each question, there is only
one correct answer but four incorrect ones. If you're aiming for a perfect
score, you have to choose the only one correct answer among the five choices.
But if you're trying to get every question wrong, you can choose one of four
different answers to get the outcome you want (much better margin for error).

~~~
spicyj
That would be true, but you are in fact allowed to miss a couple of questions
and still get a perfect score.

~~~
Serplat
That depends on the section of the test. Critical reading you can get several
wrong, writing depends on your essay score, and math you typically can't get
any wrong.

~~~
ohashi
I made 1 mistake and math and ended up with 790. I was quite annoyed with
myself when I walked out knowing exactly which question I messed up too.

~~~
Evbn
Did it hurt your future prospects?

~~~
ohashi
Ruined my life.

Just kidding. It would be impossible to tell what, if any, effect it had. Do I
feel like it actually had any effect? Not really.

------
ghshephard
I had to laugh when reading about the Fantasy "Calculators" - included as a
fantasy is a "Slide Rule" - when I took my grade 12 physics provincial finals
in 1987, I didn't have a calculator with me, but that was not unexpected, so
students were also provided with a booklet of log tables - a predecessor, of
course, to the slide rule.

The log tables, of course, were more than sufficient for whatever complex
multiplication and division that had to be done.

~~~
narsil
These are still used frequently in high schools across the world. For example,
Indian high schools following the CBSE system do not allow calculators, but
log tables are permitted for some courses.

~~~
archangel_one
Yep, at my high school in New Zealand we were required to learn how to use log
tables, even though we all had calculators. I don't know if it was a
compulsory part of the curriculum though, and I wouldn't be surprised if they
don't do it any longer.

~~~
Evbn
Sad. Learning to use of tables helps you learn the concept of logarithm. Way
better than a calculator.

------
dbaupp
Mostly off topic, but do USA schoolschildren get taught to use things like
"number of degrees of arc", "measures in degrees" and "inverse logarithm"
instead of just "degrees" and "exponentiation"/"exponential" (and the use of
the last one is the geometric mean, which is a much pithier explanation and
concept).

And, a correlation of .52 isn't the same as a coin toss: that would be a
correlation of 0.

~~~
grecy
Yes, they do.

After 35 years teaching in Australia, my parents are now in NYC running
seminars for teachers there.

They are constantly shocked at the amount of time spent in classes memorizing
lexicon, dates, etc. without actually learning to apply any of it. The
students can very confidently regurgitate what the "number of degrees of arc"
means, but they have no idea how to apply that to anything. They'll also spit
out the exact date of some war, but have no idea who fought, why, or what any
outcome was.

~~~
Evbn
I have never seen a teacher or student like you describe. The ones who don't
know the (purported) reason for the Civil War don't know when it started
either.

------
ekm2
Isnt amazing that at the time you apply to Harvard,Yale and other top
schools,admission officials brag at just how many perfect SAT scorers they
have turned down,yet a few years down the line turn back and claim to have
sifted out the best and the brightest and gladly hint at scores as the measure
of brilliance.I even remember reading an article about Harvard adcoms pointing
out how they do not want to turn their school into an Ecole Nomale Superieure
.Either they lie about inputs or the outputs.They cant have it both ways.

~~~
Evbn
Can you praphrase? I couldn't follow all your pronoun referents.

------
tokenadult
A lot of the comments here are related to the idea of whether or not the SAT
can be regarded as being much like an IQ test. It can, and psychologists
routinely think of the SAT that way. Despite a number of statements to the
contrary in the various comments here, taking SAT scores as an informative
correlate (proxy) of what psychologists call "general intelligence" is a
procedure often found in the professional literature of psychology, with the
warrant of studies specifically on that issue. Note that it is standard usage
among psychologists to treat "general intelligence" as a term that basically
equates with "scoring well on IQ tests and good proxies of IQ tests," which is
the point of some of the comments here.

<http://www.iapsych.com/iqmr/koening2008.pdf>

"Frey and Detterman (2004) showed that the SAT was correlated with measures of
general intelligence .82 (.87 when corrected for nonlinearity)"

<http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144549/>

"Indeed, research suggests that SAT scores load highly on the first principal
factor of a factor analysis of cognitive measures; a finding that strongly
suggests that the SAT is g loaded (Frey & Detterman, 2004)."

[http://www.nytimes.com/roomfordebate/2011/12/04/why-
should-s...](http://www.nytimes.com/roomfordebate/2011/12/04/why-should-sats-
matter/the-sat-is-a-good-intelligence-test)

"Furthermore, the SAT is largely a measure of general intelligence. Scores on
the SAT correlate very highly with scores on standardized tests of
intelligence, and like IQ scores, are stable across time and not easily
increased through training, coaching or practice."

[http://faculty.psy.ohio-
state.edu/peters/lab/pubs/publicatio...](http://faculty.psy.ohio-
state.edu/peters/lab/pubs/publications/2012_Peters_Beyond_comprehension_The_role_of_numeracy_in_J_and_D.pdf)

"Numeracy’s effects can be examined when controlling for other proxies of
general intelligence (e.g., SAT scores; Stanovich & West, 2008)."

As I have heard the issue discussed in the local "journal club" I participate
in with professors and graduate students of psychology who focus on human
behavioral genetics (including the genetics of IQ), one thing that makes the
SAT a very good proxy of general intelligence is that its item content is
disclosed (in released previous tests that can be used as practice tests), so
that almost the only difference between one test-taker and another in
performance on the SAT is generally and consistently getting all of the
various items correct, which certainly takes cognitive strengths.

Psychologist Keith R. Stanovich makes the interesting point that there are
very strong correlations with IQ scores and SAT scores with some of what
everyone regards as "smart" behavior (and which psychologists by convention
call "general intelligence") while there are still other kinds of tests that
plainly have indisputable right answers that high-IQ people are able to muff.
Thus Stanovich distinguishes "intelligence" (essentially, IQ) from
"rationality" (making correct decisions that overcome human cognitive biases)
as distinct aspects of human cognition. He has a whole book on the subject,
What Intelligence Tests Miss, that is quite thought-provoking and informative.

[http://www.amazon.com/What-Intelligence-Tests-Miss-
Psycholog...](http://www.amazon.com/What-Intelligence-Tests-Miss-
Psychology/dp/0300164629/)

(Disclosure: I enjoy this kind of research discussion partly because I am
acquainted with one large group of high-IQ young people

<http://cty.jhu.edu/set/>

and am interested in how such young people develop over the course of life.)

~~~
imjk
How do you reconcile the large leaps in SAT scores that people achieve through
preparing for the test? For example, while I had a fairly respectable score in
high school, after I spent a little time working as an SAT tutor in college, I
was regularly scoring perfectly on all practice tests and recently release new
tests. Surely my IQ hadn't jumped drastically. Just curious for your take on
this.

~~~
yummyfajitas
According to studies not performed by a company selling test prep, there is no
large leap in SAT score.

<http://online.wsj.com/article/SB124278685697537839.html>

~~~
Evbn
Yet practicing old tests clearly does increase scores, and token adult's
comment above did not account for that. He assumed that everyone practices and
so the measurements are unbiased, or that practice is highly correlated with
IQ (which may be true)

------
scotty79
Instead of points for correct answers the result of the test should be defined
as probability of such set of responses to be achieved by chance only. :-)

------
dangoldin
I've thought of doing this but never had the courage unfortunately. Maybe when
I retire I'll have the time to go back and try this out.

------
protomyth
My high school required students to take the ASVAB in addition to the ACT (SAT
wasn't offered, so I had to go to a testing center for it).

I know a guy who honestly tried on the math section. He got the single point
for signing his name, but missed all the questions. The first question is
"2+2".

------
pirateking
I have done this before on high school and college exams (not the SAT
unfortunately), along with other similar experiments like writing answers
backwards, using red pen to write my answers to muddle grading attempts, and
writing essays on completely different subjects than the one assigned. The
level of disbelief expressed by peers and teachers when you challenge their
value system led to perhaps one of the most important lessons I learned in my
time spent locked in the school system. Thinking outside the box means you are
still stuck in a box.

------
ekm2
>Scores on the SAT correlate very highly with scores on standardized tests of
intelligence, and like IQ scores, are stable across time and not easily
increased through training, coaching or practice.

That is interesting.How would you account for the fact that i increased my
score by 130 points upon a retake without the help of a tutor(Too bad my new
score was useless since i already had a fullride)?These tests can be
gamed.Heck ,even iQ scores arent stable over a lifetime if anyone has bothered
to read current research(Fynn Effect blah blah)

~~~
Evbn
Flynn effect compared genereations, not individual's changes.

~~~
ekm2
Arent generational changes just the average of individual changes in the same
way that the character of a nation emanates from that of individual nations?

------
jgeralnik
Tangentially related, on one of my (national) highschool math finals, I
decided to bring an abacus instead of a calculator. The proctor gave me a
weird look, I explained I was using it as a calculator, and he gave me no
problems.

~~~
Evbn
Did you use it? Aren't abaci rather loud?

~~~
jgeralnik
No, it was just for show.

------
unimpressive
While reading this, I slowly started to tune out and pressed the back button
after realizing just how much material was on the page.

I realized that this must be what my friends go through when I explain stuff
I'm interested in.

Ouch.

------
nitrogen
For anyone wondering what is meant by the "12700-choice" questions mentioned
starting in section 5.1, they are the questions in the "student-produced
response" format analyzed in section 8.

------
cyphersanctus
Collin Fahey has won the internet today in my book. Every single thing he
wrote had amazing attention to detail and in many cases was charged with extra
meaning. Genius.

------
pbiggar
tl;dr: he got one wrong, discussed in section 14.2.4.

------
maeon3
The SAT is an important measure of a humans intelligence and performance in
the real world, that's why the best companies always filter candidates and
select the ones with highest SAT scores.

It's a good thing we have tests like these and pay large sums of money to the
people who maintain it. Otherwise interviewing might be totally screwed up and
completely fail at its intended purpose in this country.

~~~
hkmurakami
The SAT is good for measuring how good one is at taking the SAT, which in turn
is an indicator of how much the test taker took preparing for the exam
seriously, which is an indirect indicator of the person's work ethic and
attitude towards putting in effort for a means to an end (yes, despite the
pointlessness of this exam). It does do a decent job at finding such people
who strive to excel within the limits of a (flawed) system, whether or not
they realize the pointlessness of said structure. If an institution is looking
for an indicator for such people, the SAT is an acceptable tool.

Of course, what it's truly good for is to measure how affluent the student's
family is.

~~~
seacond
Well said. Except I'd make the following corrections:

    
    
      s/[(]yes.*exam.//;
      s/.flawed.//;
      $d
    

I'd add that sometimes the "end" may be pointless, though you might not see it
as such while working toward it; years later, in hindsight you might realise
it was the "means", i.e., the process of working toward it, i.e. the "hard
work", that was the point.

Interestingly, there's no "hard work" in taking a test you do not prepare for,
e.g., an IQ test.

~~~
Evbn
Everyone I know with a high IQ did plenty of IQ test practice.

------
oofabz
Web Page

Making the ugliest one possible

Colin Fahey

