
Should the U.S. Make Standardized Tests Harder? - tokenadult
http://www.theatlantic.com/education/archive/2014/11/should-the-us-make-standardized-tests-harder/383084/?single_page=true
======
exDM69
I find it very strange that one of the most important tests that US students
take is a bombardment of multiple choice questions.

The equivalent where I am from is written essays and/or math, physics,
chemistry exercises (ie. calculation) where you have to have all intermediate
steps visible (which means you have an chance of scoring a few points even if
the final answer is incorrect).

You take four to eight exams during a period of two weeks (opportunity to
participate is twice a year), six hours per exam.

In language exams, there's a listening comprehension part (in addition to the
six hours of written exam) where you listen to a recording and answer multiple
choice or with a few sentences.

The method used here is not entirely problem-free either but it's a whole lot
more practical than having a long string of multiple choice questions where
you have a 25% chance by guessing and you get to know all the possible
answers. This is not really representative of what you need in life or what
you'll encounter in later studies in College/University.

~~~
rtpg
A lot of countries do this, and the biggest issue is a practical one: it takes
a long time to correct. Multiple choice you can just stick it into a machine.

Just another example of how most guiding principles in US school systems are
due to a lack of resources. We can only hope that some innovative people will
be able to get around this (big advantage of US system is experimentation at
the local level)

~~~
ghshephard
Re: "lack of resources"

The United States spends more per student on education than any other country
in the world, by a large margin. See [http://www.oecd.org/education/skills-
beyond-school/48630868....](http://www.oecd.org/education/skills-beyond-
school/48630868.pdf), page 206.

Lack of resources has never been a problem with the US education system, it's
been the management of them.

------
analog31
The present day regime of standardized testing was pitched as a way to make
school more like "business" by providing a numerical metric for driving things
like funding and teacher promotions. But just like in business, we know that
people will figure out how to game the metric.

Speaking out of fairly narrow self interest, the effect of the tests on my
kids is that it dumbs down the curriculum, because drilling the test material
becomes more important than moving on to more advanced topics or more complex
types of assignments. They are both consistently in the 98th or 99th
percentile on every test, and it seems like the testing is constant.

For my kids, more sophisticated and varied tests would mean a more
sophisticated and varied curriculum.

~~~
spacecowboy_lon
In the UK Schools agressivly game the system to get sligtly dim kids to the
required Grade C which is what they are scored on. Of course thise means
ingnoring realy bright kids and dumping the ones needing help.

------
logfromblammo
Before addressing the difficulty factor, the testing companies should be
refining the testing methods to better judge student mastery of the subject
matter rather than their test-taking ability.

This involves automated production of questions unique to a particular test-
taker, and automated scoring of the answers.

Once that has been done, the difficulty of the test can even be adjusted on
the fly to better determine the exact ability level of the student. Rather
than having separate static tests for an expected range of ability levels,
everything from preschool readiness to SAT and GRE equivalents can be tested
by the same software.

If a child takes state-mandated standardized tests, and it shows a perfect
score in math, the teachers may put that student in the most advanced math
class taught at that grade level. But what if the test could show that the
student's ability exceeded that of the typical student four or five years
older, without becoming less precise for typical students the same age?

It's really a shame that standardized tests haven't yet evolved beyond
multiple choice answers for human-written questions and scored by machines
with optical scanners. But in general, I do think that test should be harder.
They should be so hard that no one can achieve a perfect score, even by
chance. And they should also be so easy that no one can score zero. It's the
only way to get a good idea of the population distribution of the people you
are testing.

~~~
Torgo
>It's really a shame that standardized tests haven't yet evolved beyond
multiple choice answers for human-written questions and scored by machines
with optical scanners.

They have somewhat, it's called adaptive, or "branched" testing. It is a
computer-based test that gives you progressively harder questions to find the
limit of your abilities and when you get them wrong it drops the difficulty
again in an attempt to try to find out exactly where your knowledge drops off.
That's an extremely oversimplified explanation, but you can read the Wikipedia
article, which is OK:

[https://en.wikipedia.org/wiki/Computerized_adaptive_testing](https://en.wikipedia.org/wiki/Computerized_adaptive_testing)

The problem is money, and to be honest, the motivations of who is buying the
test (for some clients the testing is just meant to be a professional barrier
to entry or to do the bare minimum to comply with a government requirement.) I
worked in standardized testing for almost a decade on nearly every kind of
test, helping psychometricians and test administrators, so I saw a lot of bad
and good tests.

------
chroma
It's important to consider what we want a standardized test to measure. Is it
to check that students have learned the curriculum? Is it to evaluate
abilities for future prospects? Current tests seem to skew toward the former,
and the article suggests they should do more of the latter.

One wonders what a pure test for abilities would look like. The biggest
factors in predicting long-term success are intelligence, knowledge, and
conscientiousness. A hypothetical "how good you are at life" test could look
something like this:

1\. For intelligence, a straight-up IQ test. Raven's Progressive Matrices[1]
is an excellent candidate. It can measure almost the entire human range of IQ,
and it has little cultural bias. Heck, it doesn't even require literacy.

2\. For knowledge, the format would be similar to existing tests, with one
caveat: it must be very broad; so broad that no one person could know
everything on it. To keep the length reasonable, one could use random
sampling. Just as a vocabulary test need not ask the definition for every word
in the dictionary, so too could a knowledge test remain accurate without
asking for a copy of Wikipedia. To prevent cheating, it would be crucial to
keep the exact questions secret each year.

3\. Conscientiousness is the hardest to test, because it requires measuring
one's ability to work hard over a long time. Besides self-reports, the only
short test for conscientiousness involves telling the test taker that the task
is unimportant. If they know the truth, the jig is up and they'll work hard.

This sort of test would be extremely hard to game. Training helps RPM very
little. Studying for the knowledge section would help one's score, but it
would have the side-effect of increasing one's overall knowledge. The biggest
wrinkle is the conscientiousness evaluation. It either requires honesty on the
part of the test-taker or trickery on the part of the test-giver.

While interesting to ponder, I doubt such a test will ever exist. It's not
just students and parents who want to be able to game tests. School boards and
other organizations want to be able to show improvement over time. If they
can't teach the test, their true effectiveness (or lack thereof) might be
revealed.

1\.
[http://en.wikipedia.org/wiki/Raven's_Progressive_Matrices](http://en.wikipedia.org/wiki/Raven's_Progressive_Matrices)

~~~
shutupalready
> _The biggest factors in predicting long-term success are intelligence,
> knowledge, and conscientiousness_

I wouldn't dispute that those are important factors, but what about:

\- initiative and/or risk-taking (which is very different from
conscientiousness)

\- attractiveness (either physical or non-physical things like charm or
schmoozing)

\- starting capital (it's easier turn $1 million into $2 million than starting
with $0 and getting to $1 million)

\- general health and fitness (successful people are rarely sickly or unfit)

\- leadership

\- number and quality of connections

Looking at the tech sector, it seems that a high level of intelligence,
knowledge, and conscientiousness will get you a good job. But the most
successful people in tech are not those with even higher levels of
intelligence, knowledge, and conscientiousness -- for those people it's the
other factors that dominate.

------
higherpurpose
US should give up on the "leave no one behind" mentality - or at least
seriously overhaul that system. Because it seems to have pushed everyone to
the lowest common denominator. From what I hear, schools even consider
everyone a "winner" now, just for passing the year. That seems a bit
much...What's the incentive for excellence then?

~~~
ausjke
Exactly, race-to-the-top is more beneficial absolutely, the C-grade president
Bush's no-child-left-behind policy is making the whole country's students a
C-level group. The bar is set way too low these days, even communism society
does not have dumb policy like that to make everyone "happy". The no-child-
left-behind policy could drag the whole nation to new lows in the hi-tech era.

~~~
Dewie
> , the C-grade president Bush's no-child-left-behind policy is making the
> whole country's students a C-level group.

The fact that "C" is an insult is a sign of grade inflation, which might be
another symptom of the bar being set too low (and that has to be compensated
for by using grades like A+, etc. to distinguish the top performers).

------
danieltillett
Yes if means you can't teach to the test. The problem with standardised
testing is not testing, it is the design of the tests. Any test that can be
gamed is a bad test.

~~~
trynumber9
While a good sentiment, what kind of tests do not have a slight gaming effect
when you have previous exposure to a similar test?

~~~
danieltillett
Tests that require the student draws on a wide variety of skills and knowledge
to be able to answer.

When I was a professor I used to design my exams in two parts. The first were
pure multiple choice questions that the students could game. The second were
complex question that required integrating knowledge from multiple areas.
Answering these question required understanding - these can't be gamed other
than by learning.

You might be wondering why have the first part? The reason was because the
administration set a quota on the pass rate. If I had only used questions of
the second part then too many of my students would have failed. The bargain
with my students was if you could fog a mirror you would pass, but to get an A
you had to show understanding.

