
The Big Problem with the New SAT - jack_axel
http://www.nytimes.com/2015/05/05/opinion/the-big-problem-with-the-new-sat.html
======
com2kid
There is a fundamental misunderstanding here.

The SAT is not designed to rank students. The SAT is designed to solve a
supply/demand problem. There are less spaces available at top universities
than there are applicants. The SAT creates a bell curve, allowing schools to
go a standard deviation or two out to pick their incoming class.

If the test was an absolute ranking, you'd end up with a large number of
people scoring 100%, and the test becomes worthless to colleges as a filter.

~~~
majormajor
The article is making the claim that the test is already pretty worthless to
colleges as a filter because the differences that separate, say, a 99.9th-
percentile score from a 98th-percentile score (number picked by myself at
random, as the article didn't mention any that I saw) don't actually correlate
to much of anything of use for selecting who should get into the top schools.
Especially if those differences are largely "had more time and money to spend
on preparation." It reports that research shows that grades are a better
predictor. (Though high schools are also hardly immune to grade inflation
pressures, for the same reason that the huge test prep industry has sprung
up—parents don't care about merit or fairness, they care about doing whatever
they can to get the best-perceived college for their own kids. I find it
doubtful that changing the SAT in the way suggested would do anything to
address this root issue.)

It also makes an interesting side point about year-to-year results for
tracking overall education progress.

~~~
com2kid
> The article is making the claim that the test is already pretty worthless to
> colleges as a filter because the differences that separate, say, a 99.9th-
> percentile score from a 98th-percentile score (number picked by myself at
> random, as the article didn't mention any that I saw) don't actually
> correlate to much of anything of use for selecting who should get into the
> top schools.

Apply multiple filters. The more bell curves you have, the easier this gets.
Bonus if those bell curves are not correlated to each other! (I'm not going to
comment on how useless this becomes)

Want to be an exclusive college? Use SAT scores to remove 87% of applicants
right off the bat. (OK most people below the left half of the curve likely
don't bother to apply.)

Now filter out kids who didn't go to enough AP classes. (You've instantly
removed children with parents who "didn't care enough" to move to an area with
a good AP program! bonus!) Now filter out kids whose parent's didn't have
enough discretionary free time to shuttle them to and from multiple extra-
curricular activities.

Alright, next up, who can afford to pay for a good essay? Drop anyone who
can't.

Congrats, you've just cut down the number of papers you have to read by a
_lot_.

In fairness, once the number of applications become's unmanageable, what else
is a college going to do? Spend 1/2 of tuition money on entrance examiners?

~~~
VLM
"what else is a college going to do?"

Competitive bidding instead of fixed cost tuition. This is what they already
select for anyway, this would just be more honest and fair about it.

Nobody seems offended that tuition cost seems to have no relationship with
prof/TA salaries, facilities, future income... so removing the charade and
opening for bidding seems the most fair.

I suspect the end result is freshie year at MIT will be like $5M to buy a
spot, but sophomore year transfer students will probably only have to bid $5K
once the less talented flunk out. I pity the instructors trying to teach the
wash-out classes. The tests will still exist but for classroom placement,
someone belonging in remedial algebra will not be allowed into calculus
without passing algebra, geometry, trig, first.

------
danieltillett
I am surprised how nobody here seems to have drawn the obvious conclusion here
which is these changes (making the test less g loaded) is a massive bias in
favour of the rich over the poor. Kids who are naturally smart, but who attend
bad schools will struggle with any test that is not heavily g loaded.

~~~
gwern
Rich/poor is itself heavily g-loaded. What's the net effect here?

~~~
danieltillett
It is not that rich/poor is directly g loaded, just that g is highly
correlated with income. Quality of schooling is directly related to the
income. The more you make the tests reflect past education quality the more
you bias it is favour of the rich and dumb against the poor and smart.

~~~
gwern
> It is not that rich/poor is directly g loaded, just that g is highly
> correlated with income.

What's the difference? We say an item is g-loaded when it correlates with the
latent factor g; this is true whether it's an obscure vocab word on the SAT or
whether it's not blowing your income and planning ahead.

~~~
danieltillett
I was trying to make point that one of the major reasons income is so g loaded
is that education allows the poor-but-smart to rise out of their poverty (and
vice versa) via things like the old SAT. Block this and you end up with
something like the Middle Ages where the nobility and peasants had the same
mean g.

I should say that changes like this are going to have most effect in the
middle rather than the extremes. If you are born with a g in the 99.9
percentile band then you will still be likely to escape poverty. It is people
born with a g say in the 80th percentile band that will be affected the most
by these changes.

------
exelius
This highlights a number of problems with our public education system in
general. I think the problems with the SAT highlighted in the article are
reacting to these problems, not the other way around as the author seems to
suggest.

* There is no standardized high school curriculum across schools, and even when curriculum is standardized, it is taught unevenly. It's hard to have a college entrance exam that covers a significant amount of content as a result, because it puts kids from poor areas at a huge disadvantage in even getting in to college. It's also very hard to identify which students are from good schools and which are from bad ones, so adjusting test scores to compensate isn't a solution.

* Teacher-student ratios in public schools do not lend themselves to the kind of one-on-one work it takes to become a good writer. Unless we're prepared to say "you have a better chance getting in to college because you went to a suburban school district", this can't be a huge part of the test. Writing ability as a high school senior is largely correlated to the amount of attention you got from your English teacher. If you're a good student in a bad school, you probably get very little attention because you're not acting up.

* There's already too much "teach to the test" as it is. Making the SAT yet another test where students have to cram and memorize content is not the solution.

There are plenty of standardized tests that cover content. The problem is that
they are far too basic for college entrance purposes. Turning the SAT into a
content-focused test does not solve a problem that colleges have; many
colleges are not looking for the best students, but for a racially-diverse
selection of the most teachable students. What schools should do (and many
already do) is require students from lower quality schools to take a semester
of remedial classes free of charge to get them up to speed with whatever the
base assumption of prior knowledge is at that school.

~~~
CamperBob2
_Writing ability as a high school senior is largely correlated to the amount
of attention you got from your English teacher._

There's no way I'd ever buy that. Writing ability is largely correlated to the
amount of time the student spends _reading_... reading novels, instruction
manuals, technical books, Web pages, and cereal boxes. Teachers have almost
nothing to do with it in my experience, and I think you'll find that any
successful author will tell you exactly the same thing.

~~~
exelius
If you're talking about the ability to write a novel, I agree with you. But
you have to learn how to get your thoughts down on paper in a way that is
intelligible by other humans, and that's a technical skill that must be
learned through repetition and critique. There's a basic technical skill to
writing long form content, and many kids don't get it because their teacher
doesn't have time to grade 140 papers.

------
qmalxp
The purpose of the SAT is to be a unbiased barometer; say what you will about
it, but everyone takes the same test. Unfortunately, a 4.0 GPA from an obscure
high school may tell you very little about a candidate.

I don't know how much admissions officers care about 96th versus 99th
percentile, but if someone has an otherwise amazing resume yet ranks in the
40th percentile on their SAT, that's a bit of a red flag.

~~~
falsestprophet
You can see from class profiles of leading schools that they admit at least
75% of their class from the 99th percentile.

~~~
JoeAltmaier
Only reason I got into Stanford is, they make (made?) a point of offering
admission to all the 99th percentile each year. I was one of them.

However there're only a few hundred of these nationwide, right? Not enough to
fill several leading schools.

~~~
epmatsw
3.3 million high school graduates this year, so wouldn't the 99th percentile
be the top 33k of them? Seems like that would be more than enough to fill
Stanford and the Ivys' classes.

~~~
JoeAltmaier
Ah. There were two rankings when I took the tests - math and language. Those
ranking 99th in both were offered admission. That was a smaller group.

~~~
epmatsw
Interesting. They must have stopped that before 2009 haha.

~~~
JoeAltmaier
1983

------
Aloha
The problem with the SAT is it mostly judges what you've learned, not how well
you can learn. If we're going to have a test for college, it should judge your
future potential and ability, not just how well your high school taught you.

------
cowardlydragon
The SAT doesn't correlate to grades...

And grades don't correlate to much either.

~~~
steveeq1
There's a small correlation between SAT scores and college performance, but
that's mainly because the types of questions asked for the SAT are the types
of questions that are asked in college exams. SATs have ZERO correlation with
success in life though.

~~~
fiatmoney
"SATs have ZERO correlation with success in life though"

That is wildly untrue for any value of "success" that includes socioeconomic
status.

