
Bias in student evaluations of teaching - mariedm
https://threader.app/thread/1066721849734615041
======
slededit
This happens in reverse too. Teachers assume high achieving students will
continue to achieve and give them more slack.

The trick to all my "writing" classes was to simply put the most effort in the
first assignment. It was pretty clear that future marks were a function of
prior marks, and I tested this hypothesis in different classes by modulating
my effort on various assignments throughout the year.

------
bluGill
I know I tended to rate my foreign" instructors poorly. Not that I want to be
dislike them, but when I have trouble understanding their accent it is hard to
learn from them. If I ever have to give a lecture in a foreign language I'm
sure my students to have the same reaction (of course as a human you can be
sure that if I actually spoke a language well enough for this to be reasonable
I'd consider my accent perfect)

------
ballenf
Would it be that hard to judge professors based on students success (in
subsequent classes) using something like a pagerank algorithm? But that
wouldn't necessarily correlate with institution popularity and success.

Academia seems to be becoming more of a customer service-oriented business
where customer (student) perception is more important than the value/quality
of the service provided.

~~~
eesmith
It's called "Value Added Method" (VAM) and is used in many school systems.
It's also unreliable, says the American Statistical Association
[https://www.educationdive.com/news/statisticians-reject-
valu...](https://www.educationdive.com/news/statisticians-reject-value-added-
teacher-evaluations/250785/) or original ASA statement at
[https://www.amstat.org/asa/files/pdfs/POL-ASAVAM-
Statement.p...](https://www.amstat.org/asa/files/pdfs/POL-ASAVAM-
Statement.pdf) .

Details about the difficulties in the implementation, at
[http://vamboozled.com/](http://vamboozled.com/) , which as you can infer is
against VAM. Eg:

* Class composition varied so much that comparisons of value-added scores of two teachers were only valid if both teachers are assigned students with similar characteristics.

* Annual fluctuations in results were so large that they lead to widely varying conclusions from one year to the next for the same teacher.

* There was strong evidence that results were often due to the teaching environment, not just the teacher.

------
ryanwaggoner
I know how this is going to sound, but I'm genuinely curious: how do they
measure whether the evaluations are gender biased, or whether the quality of
the instruction is not equally distributed between the genders?

Before I get a bunch of hate and downvotes, I'm not suggesting that female
teachers get lower evaluation scores because they're not as good on average as
their male counterparts. But I assume that researchers can show _why_ that's
not the underlying source of the difference in ratings, since that would be an
obvious objection to these findings. Just curious how they separate that out,
that's all.

EDIT: nevermind, this was just laziness on my part. Here's the study, and it's
actually not that hard to understand:
[https://www.scienceopen.com/document_file/25ff22be-8a1b-4c97...](https://www.scienceopen.com/document_file/25ff22be-8a1b-4c97-9d88-084c8d98187a/ScienceOpen/3507_XE6680747344554310733.pdf)

TLDR: In one experiment, they randomly assigned students to a TA and compared
how well the students did in the class to determine whether the instructors
were of comparable quality. In another experiment they secretly switched the
TA identities in an online course so the students who had the female TA
thought they had the male TA, and vice versa. Very nice.

~~~
trjordan
Did you read the linked papers? They go into a lot of detail about how they
did it.

~~~
ryanwaggoner
Did you read my comment? I know yours wasn’t there when I added that edit :)

------
analog31
I spent one semester teaching at a major state university, many years ago.
Here are some anecdata:

* I shared an office with a guy who was team-teaching a course with a professor. My office mate was an adjunct, the professor was coming up for tenure. They fought bitterly over the grading of the course. The prof wanted to give easy grades because he needed to boost his student evaluation scores.

* It was common knowledge that the best predictor of evaluation scores was the grades that the students expected to receive. It was as if the system was rigged to encourage grade inflation. That was 20 years ago.

* Female teachers received in their evaluations, among other things, violent threats.

------
btilly
I knew one CS professor who always had very good evaluations.

His trick? He brought free cookies on the day he was going to get evaluated.

He swore that it worked.

------
olliej
A really simple indicator of this the long term relevance of "hotness" in
things like ratemyprofessor - mercifully they killed that, but it should never
have been present in the first place - which showed that "hotness" was
strongly correlated to a professor or TAs rating.

Couple with other studies that hotness ratings of women are more extreme
(tending towards a binomial hot vs. not), and more negative (I would assume,
but am not sure of actual real study to back it up, because "average" hotness
gets demoted).

~~~
philipodonnell
I'm kind of conflicted about that. A strongly explanatory variable is valuable
to have in order to control for its effects. Voluntarily removing it reduces
the usefulness of the derived statistics and in this case I guess seems to
make ratemyprofessor just a niche version of hotornot.

Perhaps a middle ground would have been to allow students to rate "hottness"
and explicitly controlled for it when ranking professors, but never making a
given professor's "average hotness" viewable to users?

------
shittyadmin
Is there an actual solution proposed here or is this just full of hot air?

He says adjusting can't work and that explaining bias doesn't do the job, so
what is he suggesting? Don't consider a professors performance at all? Add
some sort of standardized testing system?

------
adamnemecek
Are students of color harsher on instructors of color as well?

~~~
olliej
Unsure, but given this thread is mostly reporting on bias against women
instructors it's not a super relevant question.

------
lacker
Education is screwed up in many ways. Colleges would rather look good in the
US News rankings than educate their students well. Students would rather have
attractive professors than ones who teach them effectively. And yet the whole
thing is massively popular and expensive.

------
AlexCoventry
Worth noting that all four links in his thread are to the same paper.

------
mcherm
Why do I keep encountering people who apparently think that Twitter is an
appropriate medium for conveying thoughtful, considered information?

~~~
olliej
I have wondered about that recently as well - I assume it's because "everyone"
already has a twitter account, so if you do have some one off occasion that
you want to say a pile of stuff, you already have that account that is easily
sharable. Any other blog like service requires you signing up for another
account, and sharing your email address again.

So you use twitter. The next time you want to say something, you still don't
have a blog account so you take the same logic and end up posting through
twitter again.

Things like threader seem to make it easier for your thread to be shared as a
single document now, which makes it less annoying to share and/or read.

Then, with the death of Reader no one really has a non-FB, non-Twitter style
mechanism for following people any more so if you want to say something and
encourage people to follow you then twitter is your best bet. Likewise if you
want to hear what other people say (ugh, internet commentators ;) ) twitter
makes it possible to easily get replies and you don't need to worry about
comment spam continuously appearing.

