

Under Pressure, Teachers Tamper With Test Scores - asnyder
http://www.nytimes.com/2010/06/11/education/11cheat.html

======
patio11
_“Educators feel that their schools’ reputation, their livelihoods, their
psychic meaning in life is at stake,” said Robert Schaeffer, public education
director for FairTest, a nonprofit group critical of standardized testing.
“That ends up pushing more and more of them over the line.”_

There is a certain strain of teacher who never felt their reputation,
livelihood, or psychic essence was at danger when they were merely failing to
educate, for decades. After all, to the extent that anyone noticed, it was
poor black kids. Who the heck cares what they think.

That strain of teacher is now motivated by _fear of exposure_. Which is as it
should be.

~~~
count
While completely true, it also glosses over the bigger problem. Not All Kids
Can Learn The Same. Every single child is completely different, and has a
different learning capacity and developmental time line. A child who can
barely read in 3rd grade may be left behind his peers for ever, OR, he may
suddenly 'get it' in 4th grade, and rapidly catch up. K-3, however, he's going
to fail those standardized tests, and there isn't a damn thing his teacher can
do about it. Holding every child to the same standard at the same age
arbitrarily is ridiculous, and ignores the way kids actually grow and develop.
It also has completely eliminated 'teaching' for all of those teachers who
WERE awesome teachers, and who cared and poured their heart and soul into
their lessons and kids. EVERYBODY is now essentially forced to teach to an
arbitrary test, developed by academics (and lawmakers) who don't actually deal
with children (anymore).

Poor black kids are going to stay lower than everyone else, because poor black
kids, statistically, have much less family involvement, and a (again,
statistically), substantially less 'structured' home life. THIS is the number
one problem with the low end of the socio-economic academic performance
ladder. Rich white kids and poor black kids attending the same class, at the
same school, will not perform the same, unless the parents are both involved
to the same amount. Making everybody take tests to prove this out doesn't
accomplish anything. Firing teachers because parents won't do anything to help
their children also does not accomplish anything.

This issue is near and dear to me - I worked in one of the school districts
cited in the NYT article for 7 years, and my wife is currently teaching there.
It's a farce.

~~~
yummyfajitas
_Every single child is completely different, and has a different learning
capacity and developmental time line._

True, but most groups of 30-40 children are roughly the same. There will be a
few smarties, a few morons, and a multitude of people in the middle. It's
called the law of large numbers.

~~~
nooneelse
This presupposes that classrooms can be thought of as random samples of the
child population. We know that they are actually geographically localized
samples, so that presupposition is somewhat strained and needs a bit of
defending, preferably by empirical means.

~~~
yummyfajitas
Obviously a given class is only a random sample from the population of a given
school. So if you want to evaluate a teacher, all you need to do is compare
his/her class to other classes from the same geographic region.

Or even better, you can compute a statistical predictor of his student's
expected performance, and measure the deviation between predicted and actual
performance.

The point is that individual variation in students is irrelevant. It is
extremely unlikely that a teacher will get a class full of dumb kids.

------
RiderOfGiraffes
That cheating occurs is no surprise and is closely related to the question of
assessing a programmer's output. Every time you instigate a method of
measurement, the programmer (or teacher) finds some way to increase the result
without achieving the actual aim.

In the case of a programmer, the number of lines of code output, or the number
of functions written, or the number of bugs detected and fixed, can all be
achieved with less "true value output" (whatever _that_ means). The
measurement can be gamed.

Teachers in the UK have been accused of "teaching to the test" rather than
"teaching for understanding" - but what more can be expected? When exams test
for understanding rather than rote skills, only then will understanding be the
aim of the teachers looking for the easy way out.

Let me add that I know many dedicated teachers who do the right thing
regardless of the threats posed. They encourage and enthuse their students,
ranging widely over their subject to get the students engaged and interested.
They know that this serves them better, and will, mostly, improve the test
scores, even if not as much as "teaching to the test" would. They know they're
doing the right thing, but come under enormous pressure to work more obviously
towards improving test scores.

I'm not surprised that many of them buckle under the strain and take less
honorable courses.

That's part of the tragedy - the destruction of otherwise excellent teachers.

~~~
liedra
See I think that these sorts of tests are looking at the wrong sorts of
things. What they should be looking for is relative improvement rather than
reaching a particular level. Especially in underprivileged areas, where the
kids might be doing much more poorly than more privileged areas.

Don't get me wrong, I think that if there are poor teachers then they should
be dealt with, but I think that these standardised tests of kids across the
board are not the way to go about it. I'm thinking maybe something like a
"peer review" type system could be put in place that takes more into account
than just the output of the kids on one test (the results of continuous
assessment across the whole year, for example; teaching reviews, etc.). Of
course this would come with its own issues, but it would certainly cut down on
the pressure that comes from that one test!

~~~
RiderOfGiraffes
Most teacher and school assessments in the UK are based exactly on that -
achieved improvement versus expected improvement. Schools in underpriveleged
areas and which perform poorly on standardised tests still themselves score
well if the results are "good given the context and circumstances."

The tragedy is that teachers who don't teach to the test and provide an
excellent education that is tailored to the children they have are starting to
worry about the security of their jobs, whereas poor teachers who have no real
ability except to regurgitate the texts and drill the kids on process and
recitation are relatively safe.

~~~
tokenadult
_The tragedy is that teachers who don't teach to the test and provide an
excellent education that is tailored to the children they have are starting to
worry about the security of their jobs, whereas poor teachers who have no real
ability except to regurgitate the texts and drill the kids on process and
recitation are relatively safe._

Evidence for this empirical statement? I have my doubts about this, because
the best teachers I had in childhood (too few, but more than one) were able
both to engage the pupils from the dullest to the brightest with deep,
thoughtful questions and able to cover the fundamental facts of the lessons
with efficiency.

~~~
RiderOfGiraffes
I only have the information based on talking to perhaps 100 or 150 different
teachers over the course of each and every year, and the conference I was just
at raised this very question during a question and answer session.

------
tansey
Seems like there is a need for an independent third-party group to monitor the
tests. Since that is likely prohibitively expensive, here's an idea:

1) Have schools partner with each other, where each school is matched to
another school of similar size.

2) Extend the testing into twice as long periods (e.g., if it was a 1-day
test, now the test period is 2 days).

3) Have partner schools test on opposite period halves.

4) Have partner schools give students the day(s) off on the non-test half.

5) Have schools send their teachers to their partner schools to monitor all
the tests.

Now, this isn't fool-proof. It's possible that teachers across multiple
schools could collude. It would be crucial to mix teacher pairs up so that if
A monitors B then B does not monitor A, thus requiring a multi-way collusion
that involves many participants.

------
count
Cheating is wrong - I don't see any justification otherwise. With that said,
the NCLB and standardized tests are destroying school - the only focus now is
to learn to answer those multiple choice questions at the end of the year.

In Norfolk (one of the cities in the NYT article), the 'SOL' annual
standardized test is a sole determining factor in pupil promotion. If you come
out of 4th grade with a 100% average, but you fail your SOL for whatever
reason (you're nervous and didn't do well, you were sick that day, etc.) - YOU
FAIL 4TH GRADE. That seems ridiculous.

~~~
ktsmith
In schools on the watch list in Nevada where the state has stepped in teachers
are no longer really allowed to teach. In 4th grade students receive grades in
Reading, Writing, Math, Art, Social Studies, Physical Education and a few
other categories. At these schools the teachers are only allowed to teach
reading, writing and math. Every lesson plan must be geared towards meeting a
standard in one of those three categories. The teachers are required to give
out grades in the areas they are not allowed to teach. When pressed on "what"
grade the children should receive the answer is typically "whatever you feel
is appropriate." These children aren't receiving an education, they are being
taught how to memorize information for their CRT's then for their benchmarks,
then for whatever test follows that.

This isn't what teachers want, this is what teachers are forced to do by their
states in order to achieve the largest number of passing schools and keep
federal funding levels high.

------
politicalist
I hope some teachers "cheated" on the tests for principled reasons, not just
for personal reward, like this UMass teacher:

[http://books.google.com/books?id=x3ertj1IcaAC&lpg=PA62&#...</a>

~~~
count
To clarify, MOST school districts are not giving financial or other rewards
for passing the exams at an acceptable level, it's simply expected to occur.

------
samratjp
Obligatory-long-yet-relevant-animation(?) on our educational
system:<http://www.youtube.com/watch?v=A3oIiH7BLmg>

When are _we_ (loosely speaking) going to realize that we're not teaching in
the right medium.

------
karzeem
This shouldn't be a surprise. When you create strong incentives for someone to
do something, they'll probably do it.

~~~
patio11
Like embezzlement shouldn't be a surprise. I agree, this is predictable, and
in a just world our response would be predictable, too: if you falsify test
scores to get a $2,970 bonus, you get kicked out of the profession, just like
if you took $2,970 out of the drawer as a bank teller and tried to pocket it.
(n.b. While acknowledging that 95% of bank tellers are morally upright
individuals, we devise our systems under the assumption that they need to be
caught if they are attempting to cheat. That assumption is instructive.)

In the world we actually live in, you'll have a union rep testify as to how
dedicated of a teacher you are, and you'll probably not miss a day of class.

~~~
gjm11
There's an interesting difference between the situations.

If you're a bank teller, your whole job is to manage the flow of money and
information to and from customers of the bank, the ultimate goal being to make
more money for the bank. If you take $3k out of the drawer, then you have
failed at your immediate task (money has gone to the wrong place) and worked
against the overall purpose of your job (the bank has less money, rather than
more).

If you're a teacher, the heart of your job is (or should be) teaching your
pupils: enabling them to know more and think better and so forth. A secondary
purpose (the real purpose, according to some cynics) is to keep them out of
their parents' hair and off the streets for a few hours every day. Another
(subsidiary to the first, in an ideal world) is enabling them to do as well as
possible in whatever external examinations they take, in the hope that it'll
help them in later life. Arranging for your pupils' standardized test scores
_not to be too high_ comes very low down that list -- and if you get away with
it, it benefits your pupils and your school. (While harming other pupils and
other schools and other people generally, by making the test results less
useful. Tragedy of the commons.)

That's not to say that it's not very bad for teachers to cheat to get better
test results for their pupils. But it's bad in a different way from a bank
teller's embezzlement: peripheral to their job rather than central, and
immediately beneficial rather than harmful to the people you're supposed to be
working for.

A better analogy (though perhaps an overdramatic one, given recent economic
events) would be a mortgage salesperson, working for a mortgage broker, who
sells people mortgages they are too likely to default on. When that happens,
the salesperson gets more commission; his employer, the broker, gets more
commission; the person who buys the mortgage gets a mortgage they want and
otherwise couldn't have had (and, let's say, does manage to keep up the
payments in 90% of cases, fails to do so but not so badly that they lose their
home in 5% of cases, and crashes hard in 5% of cases -- which is probably
kinda parallel to the outcomes for pupils whose test scores are inflated);
other people, less directly connected with the broker, are the ones who get
screwed.

Anyone expect mortgage salespeople who sell a bit too enthusiastically to get
kicked out of their profession en masse? ... No, nor me.

> In the world we actually live in ... you'll probably not miss a day of
> class.

That doesn't appear to be what happened in the cases in the NYT article.

