
Meet the ‘worst’ 8th grade math teacher in NYC - pstuart
http://www.washingtonpost.com/blogs/answer-sheet/post/meet-the-worst-8th-grade-math-teacher-in-nyc/2012/05/15/gIQArmlbSU_blog.html
======
blahedo
I was glad to see this paragraph included in the article:

> _By the time they take the eighth-grade tests in the spring of the year,
> they already know which high school they will be attending, and their scores
> on the test have no consequences. “The eighth-graders don’t care; they rush
> through the exam, and they don’t check their work,” Abbott said. “The test
> has no effect on them. I can’t make an argument that it counts for kids._

Anytime anyone makes an argument that teachers should be evaluated based on
their kids' test scores I have to explain about the problems of misaligned
incentives. If the kids are not motivated to do their best, it's not going to
be a fair evaluation of the teacher even if you manage to solve _ALL_ the many
other problems with high-stakes testing. And a lot of students aren't
motivated to do their best on these kinds of tests, for a variety of reasons.

~~~
icarus_drowning
I teach 6-8th grade music at a local charter school, and while my performance
isn't tested through standardized exams, I can attest to my colleagues'
enormous frustration with this very issue.

For example: during our standardized testing this year, the school put in huge
amounts of effort to align student incentives, most of which were silly and
clearly ineffective. (My personal favorite? Disallowing students to bring
their own books to read after they finished the test to "encourage" them to
check their work instead of hurrying through so they could read the next
chapter of _The Hunger Games_. Talk about easy ways to incite student
frustration and contempt!)

I can attest to students finishing a test in less than 10 minutes despite
being given more than an hour, or refusing to do _any of the test at all_.
I've heard horror stories of students using the multiple-choice answer
documents to make pictures by filling in the bubbles, or simply choosing to
color in bubbles so lightly that they can't possibly be read. I know of
examples in prior years wherein students chose to write essays about how much
they hate testing on the essay portion instead of addressing the infantile
tripe they were assigned.

Now, these are obviously exceptions to the rule (I'm a bit biased, but I think
our students are almost uniformly excellent), but it is amazing how much
damage one student can do by choosing not to care. (Or, in some cases, using
their answers to vent their frustration).

And why should they? As far as I know, rewarding students for higher
performance is disallowed, leaving teachers with very few tools with which to
convince students that these tests matter. Sure, you can go on and on about
how their scores will be "important" in their future (which, to my knowledge,
isn't always true), but that only goes so far. Sometime during the 2 weeks of
intensive testing, the kids just don't care anymore.

I'm all for assessing the performance of teachers, and I'm not sure how to fix
the problem (or even if it can be fixed), but one thing seems clear to me:
they're doing it wrong.

[Updated to fix punctuation errors]

~~~
lupatus
Want an incentive to have students perform better?

Schools need to implement a standard performance-based path for skipping grade
levels so that students can get out sooner.

I was reading Hawking, Nietzsche, and college-level humanities books in my
free time in 8th grade because I was bored out of my gourd with my classes. In
12th grade, I was forced to take classes that taught, for example, how to
balance your check book.

I'm pretty sure that when he's older, I'm going to encourage my son to get a
GED when he's 16 yo or so, just so that he doesn't have to put up with all of
the bullcrap classes.

And, I don't care about your socialization arguments. My son is 2 yo and
already has great table manners, is polite, shares, and is quite empathetic.
Most of the "normal" young people I meet these days behave either like wild
Indians (feather not dot) or slovenly Barbarians.

~~~
hnhg
Clearly your great mind hasn't yet come to realise why some of us are aghast
at stupid racist statements like yours.

~~~
lupatus
hnhg,

Thanks for saying that because it brings up a problem that HN has that I've
been thinking about for awhile. HN seems to be stuck in an upper to upper-
middle class/Leftist/Politally-Correct viewpoint, and I think that is to its
detriment. Why do so many HNer's startups focus on things like iPod Apps,
whose chances for profitability are very questionable, instead of markets like
industrial sales software, which is very under-served?

Shouting "That's racist!" every time someone expresses a non-Leftist opinion
here is a symptom of your limited viewpoint, and frankly is embarrassing for
you.

Would you express the same righteous indignation if I was trying to
distinguish between groups of Europeans and had said "lederhosen not
shamrock"?

I used to live in an Alaskan Native tribal village, and my friends there would
be some of the first ones to tell you that some Indians are wild while some
are not. (Note, in my comment, I am not saying all Indians are wild, I am only
specifically referencing the wild ones.) And, by wild, I mean those who
stereotypically disregard personal and communal property rights and engage in
harmful activities like thievery and poaching for the fun of it.

hnhg, I don't know if my point will get through to you or not. I can only hope
that you will actually go out and experience the greater world sometime.

If you're angry, please go ahead and downvote me. I've been voted down to -80
karma before. It is close-minded people like you who are the reason why PG no
longer shows karma scores for users.

~~~
bilbo0s
I have to say...

hnhg seems correct here your arguments can definitely be 'distilled'. In that
distillation process the references to 'Indians', 'Shamrocks' and 'Barbarians'
would be the first to go as they add no support to your material point.

Of course, hnhg's comment approaches Ad Hominem. It could also be better
worded.

Just trying to be fair.

~~~
lupatus
Thanks. I appreciate the fairness.

I agree that I could have merely said something like, "...wild and slovenly
behavior, such as x,y,and z...." At the time, I was trying to create a vivid
verbal picture that conveyed additional concepts such as the Barbarian's
penchant for tattoos and piercings. Maybe next time I should just spell it all
out instead of refactoring it for the sake of brevity.

~~~
kenrikm
^ Wow surprised at the pushback on that. The term "Wild Indians" is referring
back to the "Wild West" days and is no way indicative of actual modern day
American Indians. Maybe another term could have been used however I think it
was blown out of context.

I think what he was trying to get across by using the term is this:
<http://vimeo.com/25239728>

Edit Never Mind: missed the "Dot" part. Yes that was clearly over the line.

------
peterhunt
I'm surprised to see so many HNers come out against standardized testing.
Sure, it's imperfect and this teacher got screwed, but how are administrators
supposed to make informed, data-driven decisions on what the right course of
action is? Many of us extol the virtues of A/B testing landing page designs
and gathering deep metrics and analytics but turn around and slam educators
for making decisions based on standardized test scores.

Give me a better metric for success that you can measure over the course of a
year.

~~~
raphman
Unlike website visitors, the teachers in a school are not acting completely
independent of each other, but part of a social group. Even if you could
adequately measure their 'teaching quality', you might miss factors that
contribute to the success of the school as a whole.

And measuring 'teaching quality' is not a simple as counting ad click-
throughs. Education is at least as much about helping people learn about their
role in the world, as it is about learning facts, concepts, and tools. Just
measuring the increase of students' scores on standardized tests misses a lot
of the impact teachers have on their students' minds.

Nobody in his right mind would judge a developer solely on the amount of
features he has implemented during the previous year, but also on how robust
and maintainable the code is. I you want to take these factors into account,
you have to actually read the code, understand it, and ideally talk to the
developer about it. Unless you do this, you can not judge the quality of the
code.

The same holds true for evaluating teachers, imho. Administrators need to talk
to students, parents and teachers and form a comprehensive image of a
teacher's ability. I really think it is that simple. Certainly not perfect.
But not less perfect than standardized testing.

As a German, maybe I am misunderstanding some properties of the US educational
system.

~~~
ig1
Doesn't that just imply that tests are failing to measure what we want
teachers to teach and we should fix the tests ?

The whole reason that test results are being used as opposed to manual
assesment, is that test results have proven to be a better predictor of long
term outcomes. Test results aren't perfect but they're better than what was
previously used.

~~~
bgruber
"test results have proven to be a better predictor of long term outcomes"

i'm not sure what long term outcomes you're referring to and how they were
measured, but if they're education related the measurement was probably more
test results. Yes, test results are a good predictor of future test results.

------
tokenadult
As a parent of four children (but living in a different state), I had to think
for a minute about what the real problem is here. If I knew that a particular
teacher was teaching well in advance of the meager United States curriculum,
perhaps even coming close to east Asian standards, I would (as a parent) take
that into account when looking at teacher ratings based on a slower curriculum
pace. My question might be, "How do I sign up?" rather than "What is the
matter with this teacher?" (By the way, the school's answer about how to sign
up is found on its website:

[http://www.ps334school.org/admissions/anderson-school-7th-
gr...](http://www.ps334school.org/admissions/anderson-school-7th-grade-
admissions)

The school accepts new applicants for sixth grade or seventh grade, but not
for eighth grade.)

But, really, the problem with the system is not mostly about how teachers are
rated, but how little power most parents have to shop. If a teacher is doing a
good job for a particular learner, most parents will be glad to seek that
teacher's instruction. And if the parents are shopping, the hard-to-measure
things in the aggregate that are crucially important at the individual level,
for example whether or not a teacher encourages a child to do his or her
personal best, will be given proper weight among all the trade-offs involved
in choosing one teacher over another. But right now the great majority of
pupils, in New York City and elsewhere, are mostly assigned to teachers
without parental power to shop, and teacher advancement in the profession is
based mostly on seniority and degrees attained

<http://www.nytimes.com/2010/11/19/us/19gates.html>

rather than on the basis of the teachers meeting learner needs better than
other teachers. The best way to promote system reform is to let the parents
have more power to shop. They will ask for the information they need and weigh
it in appropriate ways.

~~~
redthrowaway
The 'let's give parents the ability to shop' ethos is what directly led to the
current clusterfuck that is the US public education system, through NCLB and
its state equivalents.

I'm not opposed in principle to the idea of statistically ranking teachers and
giving the power of choice to parents, but you guys have royally fucked it up
in practice.

Signed,

A victim of the evil socialist Canadian public school system.

~~~
tokenadult
I see that this thread has been busy overnight. I'll quote here first another
kind reply I received, and then respond both to it and to your reply.

 _If you give the parents the ability to "shop", what happens when everyone
thinks that teacher X is the best English teacher for 7th grade and they ALL
want their kids in her class?_

For me, this is not a theoretical question, because I live in a state of the
United States where there is actual "power to shop." In the entire state of
Minnesota in the United States, there is public school open enrollment,

[http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.h...](http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.html)

and the school district for the neighborhood in which I live includes open-
enrolled students from the territories of more than forty other Minnesota
school districts, with funding following the students on a per-capita basis.

<http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf>

Aspects of the system in Minnesota that are rigid and suboptimal as they are
in many other states include

lock-step union seniority pay and promotion systems for nearly all public
school teachers,

[http://minnesota.cbslocal.com/2012/05/03/seniority-still-
rul...](http://minnesota.cbslocal.com/2012/05/03/seniority-still-rules-in-
minn-teacher-layoffs/)

but parents with power to shop and funding that follows students allows school
districts with the better programs, overall, to thrive and produce innovative
new programs (e.g., language immersion programs, specialized fine arts
programs, and school-within-a-school programs for highly gifted learners) and
the schools that are forced to lay off teachers (alas, by seniority rather
than by effectiveness) are the schools with laggard overall programs.

 _the evil socialist Canadian public school system_

This is from your comment. I'm not aware what province you grew up in, but
it's my understanding that there have been elements of school choice in some
Canadian provinces in our liftime,

[http://www.heritage.org/research/reports/2010/11/school-
choi...](http://www.heritage.org/research/reports/2010/11/school-choice-in-
canada-lessons-for-america)

[http://www.societyforqualityeducation.org/parents/canada-
cho...](http://www.societyforqualityeducation.org/parents/canada-choices.html)

although just what is going on in Canada in comparison to other countries
receives different degrees of emphasis even in Canadian sources

[http://educhatter.wordpress.com/2012/02/19/school-choice-
and...](http://educhatter.wordpress.com/2012/02/19/school-choice-and-equity-
why-is-canada-not-in-the-game/)

and perhaps right now when Canada is in the world news

<http://www.startribune.com/world/151854375.html>

we might conclude that there are some difficulties with the system in some
Canadian provinces.

Anyway, Canada provides an example, through its system of provincially
administered health insurance programs,

<http://www.hc-sc.gc.ca/hcs-sss/medi-assur/index-eng.php>

of a general public subsidy program that still allows much user choice.
Patients get to choose their doctors in Canada.

<http://www.cwhn.ca/node/40789>

In principle, as already observed in reality, it is perfectly possible for
there to be a general public subsidy for some service that is deemed to have a
positive externality, while still allowing user choice of the provider of the
service.

There are other international examples of school choice. I particularly like
the example of the Netherlands with its very wide array of choices for
parents, all at equal publicly subsidized expense,

[http://www.denhaag.nl/en/residents/to/Want-to-send-your-
chil...](http://www.denhaag.nl/en/residents/to/Want-to-send-your-child-to-a-
Dutch-school.htm)

which has been studied for years as part of broader studies of parental choice
in schooling.

[http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_...](http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED316478&ERICExtSearch_SearchType_0=no&accno=ED316478)

[http://books.google.com/books/about/Choice_of_schools_in_six...](http://books.google.com/books/about/Choice_of_schools_in_six_nations.html?id=FDfXaqcr9qUC)

By the OECD testing program called PISA, the Netherlands does as well as or
outperforms other countries both as to helping students from disadvantaged
families

<http://www.oecd.org/dataoecd/5/5/49603617.pdf>

and as to getting educational achievement results per unit of money spent by
the school system.

<http://www.pisa.oecd.org/dataoecd/50/9/49685503.pdf>

 _I'm not opposed in principle to the idea of statistically ranking teachers
and giving the power of choice to parents, but you guys have royally fucked it
up in practice._

I'll heartily agree with you and with other comments here that the current
state testing systems, which vary state by state, but which are mostly poorly
designed, are a weak basis for publishing ratings of schoolteachers, but as
you correctly point out, that's not to prove that a better system of assessing
students and their academic progress has NO value in helping parents shop for
schools. What I don't want to go back to is the day in which no one had any
idea how well any teacher was teaching, because no one was looking, and no one
was looking because parents couldn't shop for schools anyhow. If parents can
make global evaluations of what is good for each of their children, better
incentives exist to improve teaching, improve school administration, and
improve all other aspects of the school experience.

Returning to the other comment's thoughtful question,

 _If you give the parents the ability to "shop", what happens when everyone
thinks that teacher X is the best English teacher for 7th grade and they ALL
want their kids in her class?_

that is precisely the kind of situation that builds curiosity among other
teachers about "What is that teacher doing that I'm not doing?" and among
administrators about, "What value do families perceive in that class that they
don't perceive in our classes?"

As other comments have already pointed out, parents in all countries of the
world shop for schools at least by how they choose their residence addresses.
But decoupling school choices from residence choices allows schools and
families to respond more efficiently to their own mix of trade-offs. When many
shoppers prefer one grocery store to another, and take their business to the
better grocery store, what usually happens is that the worse grocery store
changes the way it does business and improves its overall customer value
proposition for shoppers. As I noted above, one HUGE problem with schools all
over the United States, even schools in states that do not formally have
"union shops" with mandatory schoolteacher union membership, is that
administrators have little flexibility in reassigning teachers to the work
that they do the best. So today is a somewhat slow process at the margins for
schools to improve (by realigning staff assignments) as families shop for the
best classrooms. But I've seen what Minnesota school districts have been able
to do even within the limitations of the current system, and I'm confident
that adding incentive for school improvement by giving learners more power to
shop makes as much sense (as a matter of basic public policy) as it does for
providing most other services.

AFTER EDIT: Commentary on the link submitted here by a blogger based in New
York City,

[http://nycrubberroomreporter.blogspot.com/2012/05/carolyn-
ab...](http://nycrubberroomreporter.blogspot.com/2012/05/carolyn-abbott-best-
worst-8th-grade.html)

which I learned about from a Facebook friend who lives in New York State and
has been following the controversy on school testing in New York State
closely.

~~~
redthrowaway
From the responses, it seems many people misread my comment, so I'll clarify:

I'm _not opposed to choice_. I love it. I myself went to an out-of-catchment
school so I could participate in the challenge program there (3 years of
certain classes in 2, makes AP easier to take in gr 12). My point is that, _as
implemented_ , you guys borked it.

It's far too simplistic to say, "let's reward the good schools and punish the
bad ones, and the market sort itself out". It results in schools in
disadvantaged neighbourhoods getting even worse, and the least well-to-do
segments of society being put even further into a hole from which they can't
escape. It results in idiotic "teach to the test" systems, instead of good
teachers teaching creatively.

It _might_ be possible that a truly free market school system would give
better results than the norm, but we've seen what your first forays into it
have brought: unmitigated disaster. The US seems particularly poor at
political innovation, so why not simply imitate those countries that do
education well like Korea, Finland, and the like?

Take a look at this list [1] (pdf, pg8) and tell me which of the top scoring
countries have achieved educational success by adopting policies that share
the same principles as NCLB.

[1] <http://www.oecd.org/dataoecd/34/60/46619703.pdf>

------
mturmon
The LA Times published a database of these value-added results two years ago,
for teachers in the LA USD. The LA USD contains a mix of good teachers and
time-servers, with the time-servers well-protected by the teachers' union's
last-hired/first-fired policy. Teacher performance is not allowed to be a
factor in layoffs.

These value-added statistics, which on a per-teacher basis must be incredibly
noisy (they measure a difference, the "value added", of two already-noisy
performance numbers) resulted in the suicide of a low-ranked teacher:

[http://www.msnbc.msn.com/id/39404037/ns/us_news-life/t/la-
te...](http://www.msnbc.msn.com/id/39404037/ns/us_news-life/t/la-teacher-
suicide-sparks-test-score-pushback/#.T7SBw3lYu00)

I've been surprised to hear the credence placed in these statistics when I
listen to presentations by principals and superintendents.

------
Umofomia
If you want to see how basically meaningless these teacher evaluations really
are, check out the graphs in these posts:
[http://garyrubinstein.teachforus.org/2012/02/28/analyzing-
re...](http://garyrubinstein.teachforus.org/2012/02/28/analyzing-released-nyc-
value-added-data-part-2/)

The evaluations of the same teachers teaching the same subjects in two
different classes are almost completely uncorrelated. The distribution of the
scores on the scatterplots look almost random.

It's a travesty that perfectly fine teachers are being publicly shamed based
on these unreliable metrics.

~~~
yummyfajitas
No, it's a travesty that Gary Rubinstein used an inappropriate plotting tool
and then declared the metric flawed.

See my previous post on the topic:

<http://news.ycombinator.com/item?id=3749530>

------
SagelyGuru
> “They’re not accepting answers that are mathematically correct,” Abbott
> notes, “and accepting answers that aren’t mathematically correct.”

I don't believe that 'the professionals' setting up, promoting and
administering a system like this can possibly be ignorant of this fact and/or
well meaning.

Of course Abbott had to be eliminated as a teacher. She was interfering with
the program of _deliberate_ dumbing down.

------
ShabbyDoo
We just sent in our six year-old's Montessori application for first grade next
year. Our suburban public school system has an amazing reputation, and the
passing rates for standardized tests are some of the highest in the state.
Sadly, our little Aspie with ADHD issues has already expressed his dread of
first grade. He doesn't want to do "seatwork", "paperwork", or any of the
other drudgery that our local district tells him is "his job". [Quotes
indicate words used by teachers with the best of intentions.] I can't figure
out the value of this pseudo-rigor. Maybe I'm unusual in my autodidactism, but
I can't think of anything I know which I (1) find valuable and (2) learned by
force.

I bought a K'NEX roller coaster kit and helped him put it together. At first,
he didn't grok the pseudo 3D instructions or the way the pieces fit together.
However, after an hour or so, he was rotating our partially assembled segments
in his head to compare them to the illustrations. But, this doesn't count as
rigorous seat work. Oh well.

Our school actually does an excellent job of avoiding the binary incentives of
No-Child-Left-Behind and attempts to provide somewhat individualized
instruction to students. Ohio has yet to adopt a value-added measurement
scheme, and I wonder how our district would fare, especially if achievement
was normalized for socio-economic status (the average household income is over
$100K, a high number for Ohio).

We hope that the Montessori school's motivated teachers can use their freedom
from mandated curriculum requirements, standardized testing, etc. to
facilitate our son's exploration of what interests him. I don't care if his
education is seen as incomplete, unbalanced, or otherwise non-standard so long
as he grows to become a curious, thoughtful, and creative person. When has
someone with these qualities ever failed because he had some perceived
weakness in his childhood education?

~~~
pbhjpbhj
> _When has someone with these qualities ever failed because he had some
> perceived weakness in his childhood education?_ //

Define "failed". Failed to get a job, too right. Failed to fit in to society,
check. Failed to be happy, seems to go with the territory(!?).

Will you excuse a short aside:I'm interested in the diagnosis of a 6yo as
having ADHD - what procedures and tests were performed to come to that
decision. In my, albeit limited, experience of primary aged (4-11yo) boys if
they're not running around half the time like some sort of sugar rushing
lunatic then they're the exception to the rule. Can he sit and watch a TV show
for 20 minutes without leaving to do something else?

~~~
btilly
If you think that ADHD boys are _running around half the time like some sort
of sugar rushing lunatic_ then you do not know what ADHD actually is.

It is not generally a problem of too much energy and lack of focus. Rather it
is a problem of inappropriate focus, frequently very intense. For instance my
nephew would get so interested in whatever he was doing that he would fail to
realize that he needed to pee, and then after he peed his pants he would get
very upset that people were making him change his pants when he wanted to do
something else.

It becomes a problem in school when children are unable to follow repeated
directions because they are unaware that directions have been given, because
their interest has been caught by something - anything - else.

Disclaimer: I am the parent of a 7 year old who has been identified as likely
having ADHD, though I have not yet done the official screening.

~~~
pbhjpbhj
Sounds like I have ADHD.

> _identified as likely having ADHD_ //

Yes this is the part I'm interested in, what's the diagnostic process you've
gone through?

~~~
btilly
He was identified as having a front lisp, so went in for speech therapy
evaluation. During that evaluation he was found to have the speech impediment,
and was flagged as requiring further screening for occupational therapy and
ADHD. We have not yet done that screening.

My understanding is that an important part of that screening is a patient
history and impressions from caregivers. Every caregiver has identified the
same thing. He's plenty smart and has a great attention span for what
interests him, but simply doesn't follow direction. Not in a, "kids don't
listen" sort of way. But in the kind of way which causes the teacher in
parent-teacher conferences to get all serious and spend the session talking
about how she has more problems with him than with the rest of a class of 24
kids.

Having talked with my sister, whose son had very severe ADHD, I've been forced
to admit that the symptoms fit. (My son is clearly not as severe though.) But
it isn't official until he gets the official screening for it.

As for what to do about it, my sister spoke highly of cognitive therapy. She
understands our resistance to drugs, had the same herself, but points out that
drugs are optional and only needed as a temporary stopgap to give therapy a
chance. If you don't need them to make therapy work, you don't do them. If you
do need them, once therapy has progressed far enough, you can drop the drugs.

~~~
pbhjpbhj
Thanks for a full response. Really appreciated. Just one more quick question
if you will:

Do you recognise yourself in the symptoms your son has, does it look like an
inherited trait to you (I know that's not at all scientific). Do you have any
suspicions for why this affliction might appear to be so prevalent amongst
children now (other than observer bias or similar discrepancies).

~~~
btilly
There is definitely a hereditary component to it. And yes, the shoe might have
fit for me as well.

My suspicion is that over time we're adjusting to having fewer kids and more
attention/kid, and so are noticing and diagnosing problems that in previous
generations went unnoticed.

------
awbraunstein
This is what happens when education law is created by politicians. Politicians
who get their information from lobbyists lobbying on behalf of the test
makers. They believe that more testing is good, mostly because it means more
money for them. In NY, the ELA is mandatory for students in grades 1-10 iirc.
And these are the scores we are judging teachers on. There are many teachers
in schools who are given classrooms of students who need more attention and
won't perform as well on exams.

We need to seriously rethink education and how we are testing students and
teachers.

~~~
yummyfajitas
_Politicians who get their information from lobbyists lobbying on behalf of
the test makers._

Yes, the politicians are clearly in the pockets of test makers. After all, the
test makers are the #5 and #10 biggest politicals donors ever.

<http://www.opensecrets.org/orgs/list.php?order=A>

Oops, my mistake - it's actually teachers (generally opposed to
accountability) who are the big political donors, not the test makers.

------
Tsagadai
Sometimes, in professions like engineering and accounting, it is easy to miss
the meaning that numbers actually have. If you only look at the numbers
(especially when the numbers are a representation for a data model and not
physical data) you can miss the forest for the trees. Reading this article
makes me think their statistical model is broken.

~~~
Drbble
And in social science, as we have here.

------
wisty
For those who don't bother to read the article - the school in question has an
accelerated program, and the students are ahead of the test. They no longer
care about 8th grade curricular, because they are up to calculus by then.

I'd guess that this means they understand 8th grade curricular, but the test
isn't testing deep understanding, just speed and accuracy of the basic skills.
Most tests are badly flawed like this, and create a misaligned incentive for
teachers to grill students on a very narrow subset of the syllabus.

~~~
Drbble
Algebra, not calculus, but yes. And not accuracy on basic skills, but
regurgitating keywords and choosing plausible multiple choice options.

------
simonh
The sad truth is that schools actually only have a minimal effect on the
educational outcome for students, the dominant effect is the home environment
and parental attitudes. I cared deeply about how well I did in exams when I
was at school, and so do my children.

Politicians won't say this because they answer to voters and telling parents
it's their fault wins no votes. It's the great fat elephant in the room nobody
wants to talk about. Forget incentives for children and bad teacher witch
hunts. Better education for parents about how to meet their responsibilities
is the way to go.

There are plenty of books on good parenting and how to raise happy and
motivated kids. I know this because I have read several of them. In this, and
other public forums we aren't beholden to political constraints. We can tell
it like it is. There are no excuses.

~~~
koide
I agree. Although I think that the right combination of parenting and
schooling can take a child much further than good parenting and bad schooling
only. After all, the children spend lots of time at school.

By the way, care to share a list of the books on parenting you found more
useful or interesting?

~~~
simonh
A few to start with. To be honest, even one good book is a great start.

Raising Happy Children - Jan Parker, Jan Stimpson, Dorothy Rowe

Playful Parenting - Lawrence J. Cohen

The Well-Trained Mind - Jessie Wise, Susan Wise Wise Bauer

I don't agree with everything they say, but they are pointed in the right
direction. I don't home school our kids, but we do homework with them and I go
out of my way to talk to them about anything they're interested in and look
things up with them.

It does help that my mother was head teacher of a primary school and did an
Open University degree in child psychology when I was a teenager.

We're engineers, right? When you've got a new piece of equipment or software
or a new language to get to grips with we read the manuals and find out about
best practices. It's the same with kids.

~~~
koide
You actually read manuals?! Thanks for the pointers, I have read some myself,
but I'm in the search for new ones.

------
snikolov
Gary Rubinstein wrote an excellent series of posts debunking the value added
metrics [http://garyrubinstein.teachforus.org/2012/02/26/analyzing-
re...](http://garyrubinstein.teachforus.org/2012/02/26/analyzing-released-nyc-
value-added-data-part-1/)

~~~
johnhess
I don't think that's a debunking. yummyfajitas covered the plotting tool, but
that's only the start.

Clearly, the metric is far from perfect -- you can tell that without his
analysis. Year-over-year variation is high, indicating that there's likely a
lot of noise in the measure. That said, year-to-year scores _do_ correlate.
This means that the measure is at least a little bit reliable in the sense
that if you measure the same thing twice you'll get an answer that's close to
the original.

All he's shown is that these are "noisy". There's no reason not to use a noisy
metric, you just need to know that's what you've got, and make responsible
decisions.

For example, don't fire teachers who rank poorly, but do review their
performance. In that case, we'd find that the "Worst Teacher" is
underperforming on this metric because it wasn't designed to test great
students, and we can move on. Another example of the same phenomena in
education is looking for teachers who help students cheat exams. When there
are suspicious correlations found, administrators investigate.

One last thing: What he doesn't investigate -- and this is much more important
-- is the (lack of a) correlation between value added ratings and other
metrics (say, peer evaluations).

------
KaoruAoiShiho
Is this a bug in the value add system or is it unfixable?

~~~
adpirz
I honestly think, as benign as it sounds, this is the most pertinent comment.

First off, I don't think that a teacher's performance eval should be entirely
rooted in state tests, nor do I believe that the current method of testing is
by any means perfect, but I think value-added measurement is a decent measure
of how teachers are performing and how much knowledge students have gained.

What we're looking at here is an extreme case where that value added measure
has no more meaning and the actual evaluation has no statistical meaning (the
students performance was already top notch, they were probably more likely to
score a little worse.)

What SHOULD have happened is that they should have been evaluated using a more
rigorous / advanced curriculum that was aligned to what the teacher was
teaching (ie, high school algebra). The system is not perfect, but it's not
terrible--they just need to evaluate the system, adjust, and iterate,
something many here are plenty familiar with.

(Disclosure: I teach at a charter school)

~~~
colomon
How about giving the test on a computer and having it automatically change the
difficulty level on the fly? (Obviously this may be a lot easier for math than
for other subjects.) Test every grade level using the same program, just with
different entry points based on the knowledge they are presumed to have
learned at that point.

Right now it sounds like they are awarding the teachers based on the
percentile level of their students compared to other students in the same
class level, which is pretty much guaranteed to be useless noise if your
students all start at the 98th percentile. It would be much more meaningful to
be able to say, "Each of your students clearly knows more algebra than he did
last year."

The other issue is how to motivate the children to actually want to do well on
the test. Somehow rewarding a kid who does better than he did the previous
time he took the test seems like a good first step...

~~~
adpirz
All valid points, and there are testing suites that do exactly what you say
that are widely used (this one is pretty standard:
[http://www.nwea.org/products-services/computer-based-
adaptiv...](http://www.nwea.org/products-services/computer-based-adaptive-
assessments/map)), but they aren't the assessment that's measured by state
departments. There are probably lots of reasons for this, (cynical: they were
outlobbied by Pearson, which has million dollar contracts with many states to
write their state tests.), but nonetheless they exist.

The other problem is that this is a problem at which we should be throwing our
best data scientists, and in many cases we're lucky if those involved have an
intro stats course under their belt. Evaluating student performance is no
trivial task (ESPECIALLY when trying to create some accurate measure of
reading level), but we're not putting the brainpower behind it to figure out.
And as much as I love it, I don't think Khan Academy will be solving those
problems, but I'd love to be wrong.

------
chubbard
This is a disheartening story about education in the US. People aren't going
to continue to work a job if you call them terrible at it. I feel I have
strong self esteem, but I don't think I could handle that.

What's sad is our over reliance on these statistics. First thing you have to
do when using stats is setup checks and balances to verify you are getting
what you expect. This is a counter example that might indicate you aren't
measuring the right thing. Does this value add measurement of prior year
improvement even predict anything about the students comprehension? It's like
the BCS system of education where you just scratch your head over the
outcomes.

America has a long time fascination with measurement and quantifying. We
created Scientific Management Theory dating back to 1880s, but it ended mostly
because workers rebelled. However, its influence is still felt everywhere in
American society so much so I have a hard time understanding how the Finish
model achieves its results without measuring. I know its better, but I can't
explain it.

------
pstuart
It seems almost libelous the way it was done.

------
basseq
"You can’t just measure what teachers do and slap a number on it.”

I think you _can_ , it's just that the Teacher Data Report doesn't seem to be
doing a good job of it. It's a broken model, not a broken idea.

------
mathattack
Standardization helps aim everyone to B quality. It tries to pull up the
underachievers, but drags down those at the top. On average it may be helpful,
but not at the very top.

The standards of Walmart, P&G, and similar companies help the companies
achieve economies of scale. They allow C level performers to overachieve.
But... You don't want to be the only free thinker in Mayberry.

Similarly, standardizing education may help the masses, but it's not for the
best students or teachers.

------
mrobataille
This is the natural result of a commoditized profession. Talented people will
not be rewarded to the extent they deserve, and will leave.

------
astrofinch
That multiple-choice remark was a copout. Knowing material well seems like a
sufficient condition to do well on multiple-choice exam.

~~~
kijin
The students actually did do well. 89th percentile is nothing to scoff at.
Especially when, as the article mentions, students had zero reason to care
about that exam. (They already knew which schools they were going to.)

------
Zarathust
This treats teachers the same way I treat cpu benchmarks. Teaching is not a
mechanical process!

~~~
maxerickson
So is it never appropriate for teaching to be a mechanical process, or is it a
failure of current techniques that they cannot be applied mechanically?

I don't mean to be flip, I'm thinking about trying to teach the same lesson to
15 or 30 people and how some of them won't need the lesson and some of them
won't have the base knowledge to understand it.

