

Problems in Education - JumpCrisscross
http://lesswrong.com/r/discussion/lw/h5z/problems_in_education/

======
tokenadult
I'm not sure this collection of anecdotes will be news to someone who has been
following education policy for a while but I do think the topic is important.
With the importance of the topic in mind, I'll suggest here some better things
to read on improving education.

[http://stuff.mit.edu:8001/afs/athena/course/6/6.969/OldFiles...](http://stuff.mit.edu:8001/afs/athena/course/6/6.969/OldFiles/www/readings/ma-
review.pdf)

(review of a very important book that essentially every parent in the English-
speaking world ought to read)

<http://hanushek.stanford.edu/publications/teacher-quality>

(a collection of articles on teacher quality by an experienced education
policy researcher)

<http://educationnext.org/>

(website of a journal on education policy issues)

~~~
Alex3917
For what it's worth Hanushek is most famous for his theory that a student
having a teacher in the 85th percentile or better for test score improvement
for five years in a row would be enough to close the gap between low-SES and
high-SES kids, a theory which has been soundly disproven. I have no idea about
the quality of his other research, but I'd tread with extreme caution there.

edit: C.f. The Death and Life of the Great American School System by Diane
Ravitch, the paperback edition, esp. chapters 6 - 9 and the epilogue.

~~~
tokenadult
I'd have to look up the reference you kindly shared, but I sure hope Ravitch
had help with her mathematics in that analysis, as she is the person who
doubts the general usefulness of even beginning algebra as a high school
graduation requirement. (Her higher education was in history, not any
quantitative subject as Hanushek's was.)

~~~
Alex3917
She is citing other studies that attack Hanushek's analysis, but she isn't
citing any mathematical arguments that she created herself. There are a number
of problems with Hanushek's argument, among them:

\- The ratings are completely unstable, so a teacher who is in the bottom 20%
one year has a pretty good chance of being in the top 20% the next year.

\- Hanushek doesn't account for forgetfulness when evaluating the differences
between teachers. So if you retest the students on the same material two years
later, it turns out that the difference between a great teacher and a mediacre
teacher is only about 1 / 8th as much as Hanushek had supposed. And the
teachers that do the best in the short term (at the end of the school year)
have little correlation with the teachers that are actually the most effective
in terms of getting kids to know the most after two years, which is what is
actually important.

\- Even if the teacher ratings were stable, which they aren't, they're not on
an interval scale, meaning that the comparisons between different teachers are
largely meaningless.

\- Students aren't assigned randomly to classrooms, and the differences
between their abilities and backgrounds can't fully be corrected for for
various reasons. (One of which being that again the tests are not on an
interval scale.)

\- In order for the VAM to become at least fairly stable it takes about 7
years, by which point about 60% of teachers have already been fired or quit
anyway.

\- VAM only works for math and literacy and not the other subjects that aren't
tested.

\- The VAM ratings are extremely test specific, and if you give a different
test of the exact same material to the same kids then the teachers that are
'effective' or 'ineffective' are often wildly different.

\- VAM only works if you have data for kids from the previous year. However
the data tends to be missing for about a third of kids, especially in low-SES
communities where people move around much more, which means that VAM is much
less accurate in minority communities due to having less data to draw from and
therefor less statistical power. Which means that it's effectively
firing/disciplining/rewarding teachers at random in these communities, and
therefor discouraging good teachers from teaching there.

There are other issues too, I'm happy to email you my notes if you'd like,
which are mostly from the sources that Ravitch cites and not from Ravitch
herself. (The sources are mostly things like Department of Education studies.)

------
edtechdev
"It's pretty hard to argue with the data."

Would be nice if there were any actual data in this article.

~~~
rednum
Last paragraph from the article:

> I can provide citations for any statements I've made, but since nearly
> everything I said is NDA'd I'd like to be careful to anonymize my citations,
> making sure to find national studies instead of state or county specific
> studies.

So I believe that if you are interested in any particular claim the OP made,
you can contact him for some resources.

------
bromang
This article has a pretty bad smell. The author makes a lot of fairly extreme
claims but these are mostly based on his own anecdotes or non public data
sets. The claims that

"you can separate all advanced math teachers easily into two categories: Okay
with blacks in their classroom [...] Not okay with blacks in their classroom.
Whites end up succeeding, blacks end up failing"

and

"the evidence shows that teacher recommendations have zero correlation with
aptitude in a field"

are really quite dubious and almost certainly false. How did they measure
whether they were "okay with blacks in their classroom"? As many others have
said in the original thread, this is probably a hoax.

~~~
Alex3917
Actually those claims are basically true. If you look at the graph from my
blog post here:

[http://alexkrupp.typepad.com/sensemaking/2009/02/the-most-
im...](http://alexkrupp.typepad.com/sensemaking/2009/02/the-most-important-
graph-in-education.html)

What you see is that the data points from all three ability groups are
basically overlapping, meaning that the students get sorted into ability
groups based on factors like race rather than on actual ability.

That being said, while you might be able to model the bias of teachers and
school systems by using an algorithm, I've never heard of schools actually
using an algorithm to explicitly sort students based on race, which he implies
that they are doing.

However, assuming he doesn't actually mean this, I think the post is actually
legit but just poorly written, perhaps purposely to promote his startup.

------
TheFuture
The big assumption that everyone tends to make when talking about public
education is that the politicians, school boards, principals, and teachers are
advocates for students.

Sure, some of them are, but the vast majority of them are looking out for
themselves. (I don't blame then, the system pushes them to this) They're
making decisions on what will be best for their situation, and they are not
really held directly accountable to those they serve, the students.

If anyone has tried to fight the system (like the author), they quickly
realize this. And the only effective way I've seen to balance the power is
school choice vouchers. Otherwise you're condemned by your zipcode unless
you're wealthy enough to afford private school tuition.

------
lucb1e
I never even heard of a system that automatically selects students for any
classes. You choose what you want to do, and you can go do it. At least until
you go to university or something.

And I thought our school system was bad...

------
lsiebert
This is why evaluation is so important.

Also, I think there is an analogy in how startups talk about number of users,
instead of dollars earned.

People need to use the correct metrics for success.

------
jimhefferon
Almost all of this article, importantly starting with the "background"
material in the first paragraph, is wrong. It is so wrong that I wonder: is it
intended as a joke?

~~~
untothebreach
Care to elaborate on what is wrong about it? I don't have a lot of knowledge
about the inner workings of the public education system (outside of going to
one for 12 years), but from my observations as an outsider, the article seemed
plausible enough. So, I am interested in anything you can dispute.

~~~
jimhefferon
Sure. Just for a start, sentence 3 is wrong.

------
Hyrum_Graff
This is not an issue in the UK. In our schools teachers are provided with data
based on prior achievement and demographics of all their students. If a cohort
of students fail to reach their expected target then the teacher is usually
considered responsible. Of course, rigorous adoption of a target based system
also leads to teachers, schools and exam boards gaming the system and
subsequent grade inflation, but that is a different problem entirely.

~~~
keithpeter
" _If a cohort of students fail to reach their expected target then the
teacher is usually considered responsible._ "

Or at least the alarm bells start ringing and there is a conversation (UK
based myself). A 'cohort' is the word I usually use for a whole set of
classes, e.g. a year group. More than one teacher is usually involved. We also
have independent information on teaching through QA observations and OFSTED.
Your picture is fundamentally accurate but it is a tad more complex in my
experience.

