I could consistently help even the poorest students move from below 50th percentile to 75th percentile. Moving from < 600 to mid 700s is totally doable with sufficient tutoring [1]. Even for pretty dumb students.
I think SAT/ACT are pretty good tests [1], but they're horrendously over-gamed at this point. I have very little faith in either as anything other than a demonstration of how badly the student wants to be admitted to a good school and has money for tutoring.
[1] edit: i.e., SAT/ACT are not easy to game wit short-term coaching, but sustained tutoring can substantially increase students' performance... see thread below for further elaboration and discussion of "coaching doesn't help" studies.
Nearly all of those studies focus on short-term and test-specific interventions; i.e., "coaching", not "tutoring".
I worked with students throughout the school year with a focus on the underlying content, and only switched to "coaching" the last few weeks before the exam. For many students, I tutored them weekly or biweekly, for 1-3 hours per week, for multiple years!
NONE of the studies on the effect of coaching consider the effect of this sort of longer-term individualized instruction.
I'm willing to believe that short-term coaching only has small effects, but sustained individual instruction has a huge impact on mathematical ability. And as I explicitly said in my original post, SAT/ACT do a good job of measuring that ability.
But claiming that sustained access to individualized high-quality teaching doesn't effect performance on subject-specific tests that require nontrivial content knowledge and practice is, on face, absurd. At the very least, the studies you're citing say absolutely nothing about this sort of sustained intervention.
(Also, College Board loves amplifying those studies. I wonder why...)
> For many students, I tutored them weekly or biweekly, for 1-3 hours per week, for multiple years!
This... is a feature, not a bug. The SAT is a tool to measure educational attainment, and you boosted scores by legitimately educating students. The SAT is not a test to measure natural ability (I can't believe this needs to be said, but so many people claim that no intervention should be able to boost SAT scores, and the only logical conclusion is that they want the SAT to measure some sort of unchangeable inborn ability? Of course, I think the actual problem is that they haven't realized that if you eliminate all environmental differences, all you're left with is the genetic lottery.)
But anyway, I think this is absolutely fine. Would you expect someone who hasn't gone to high school to do well on the SATs? Then why in the world would you think legitimate education shouldn't boost SAT scores?
There's a huge spectrum between "raw ability" and SAT/ACT. A good test would measure somewhere in the middle of that spectrum. It's not impossible to do, but is really hard.
The SAT (and other standardized tests) make a lot of sense when you're comparing people who have spent more-or-less the same amount of time and money preparing. They also make sense as one component of a holistic picture, weighed appropriately.
The the true value of these tests for predicting potential is a lot less useful otherwise.
The huge problem, from a predict-success perspective, is that you can't tell the difference between:
1. a brilliant person;
2. a kind-of-smart person who's very driven; and
3. an average person with no work ethic who was forced to sit with a tutor for many hours each weekend.
> Then why in the world would you think legitimate education shouldn't boost SAT scores?
It should. That's what the SAT is for. As I've said twice now, the SAT is a well-designed test. I don't think the SAT should change. I'm just now sure how useful it is, especially as a holistic measure.
To be really concrete about this: colleges should shy away from the SAT because I won't be holding those students hands forcing them to study and custom-designing their course of study at their first job!
At some point soon after graduating college, the hand holding disappears and you sink or swim. Academic preparation helps, but work ethic and the ability to learn on your own is really important. Colleges are, or at least should be, attempting to select people who are more likely to "swim".
If I were a college admissions officer, I'd probably weigh "good enough scores to know you're not an idiot, plus a compelling demonstration of grit and work ethic" WAY over "great scores with no demonstration of independent drive".
(FWIW I think we're now completely disconnected from the actual topic of the article, since that's not what the hardship score is measuring)
Oh okay, I think we mostly agree. People ragging on the SATs and other standardized exams for being teachable is a pet peeve of mine and I overreacted.
Back to the hardship score, I just don't think that the College Board should be in this business at all. Individual colleges certainly know where an applicant is coming from, and what high school they went to, and they have a lot more additional information not available to the College Board. So they have a much better idea of what hardships the applicant went through. Furthermore, different colleges want different things from their students which would and should lead to them weighing different kinds of hardships differently. Reducing all of this to a single number based on very coarse data is exactly the opposite of what holistic admissions is supposed to achieve.
