
Students Evaluating Teachers Doesn’t Just Hurt Teachers, It Hurts Students - chmaynard
https://www.chronicle.com/article/Students-Evaluating-Teachers/245169
======
skywhopper
This strikes me as a very whiny article, not worthy of publishing in the
Chronicle. A lifetime of hurt feelings and resentments bubbling up into a rant
full of straw men and frankly irresponsible misinformation about the role
student evaluations play in instructor assessment.

It’s too bad, too, because there certainly are criticisms to be made, and this
article hints at some of them. But a well designed evaluation instrument along
with critical interpretation of the results and comments can be an invaluable
tool for instructors who want to improve their teaching.

But unfortunately this article also spreads untruths about the weight given to
student evaluations, and dismisses common-sense security protocols like having
a student deliver the results instead of the instructor (not perfect of
course, but the student has much less incentive to tamper) as demeaning rather
than a sensible precaution.

If the author had presented any evidence whatsoever to support her claims
linking student evals to grade inflation (possibly related, and I’m sure
studies have been done, but evals are very far from the only obvious
contributing factor), or even made a passing attempt to explain what
percentage of weight is given to raw eval scores for actual tenure
considerations (very little in the broad scheme of things), and cut the junk
out about how universities seeing students as customers is a new thing (it’s
not new at all as any skim of the history of universities as an institution
will make clear, but someone at a US state university should realize that it’s
skyrocketing tuition rates, and not whiny students, that give students a
bigger sense of entitlement) the article would have been much better.

~~~
BugsJustFindMe
> _a well designed evaluation instrument along with critical interpretation of
> the results and comments can be an invaluable tool for instructors who want
> to improve their teaching._

An instructor who actually wants to improve their teaching solicits and
engages with feedback from the very first day of the term and doesn't wait
until end of term evaluations roll in. Why? Because end of term evaluations,
by definition, cannot help the students who wrote them.

~~~
newen
That's not easy at all. Instructors design their course typically before the
first class starts. Trying to incorporate suggestions from students --
students who have no experience teaching -- is just going to lead to disaster.

~~~
BugsJustFindMe
Feedback isn't for suggestions, it's for understanding problems that your
students are having. If you aren't normalizing the process of students telling
you when they are struggling, then they either won't tell you when they're
struggling at all or you'll only find out too late. It could be pedagogical
stuff like they don't understand your explanations, or it could be emotional
things like they're having panic attacks and don't know what to do, or it
could be simple technical details like what fonts you use on your slides. And
if you know right away then you can do something right away and actually help
someone.

------
avani
Student reviews are also _ridiculously_ sexist in the average case. In a
recent study, the students rated identical teachers differently enough by
gender that actual teaching performance was lost in the noise:
[https://academic.oup.com/jeea/advance-article-
abstract/doi/1...](https://academic.oup.com/jeea/advance-article-
abstract/doi/10.1093/jeea/jvx057/4850534?redirectedFrom=fulltext)

Another pair of studies showed that identical courses where students thought
instructors were male rated higher than those that had a female professor, and
their qualitative answers were also very different:
[https://www.insidehighered.com/news/2018/03/14/study-says-
st...](https://www.insidehighered.com/news/2018/03/14/study-says-students-
rate-men-more-highly-women-even-when-theyre-teaching-identical)

I'm grateful to be a computer scientist, where the consolation prize for not
getting tenure is a cushy industry job, but I feel deeply for my pre-tenure
colleagues in the life sciences who have to work twice as hard for their
evaluations while still maintaining their research and service requirements.

~~~
YorkshireSeason

       students rated identical teachers 
    

Did you actually read critically the papers you cite?

[1] claims to be comparing "identical teachers" but [1]'s sole claim to
courses taught by male and female instructors is " _neither students’ grades
nor self-study hours are affected by the instructor’s gender_ ". Clearly those
are hardly the only factors relevant to teacher quality. Moreover [1] claims
to use "objective measure of the instructors’ performance", and which includes
-- I kid you not -- " _self-reported_ number of hours students spent studying
for the course".

[2] is similarly vague, and claims that " _the courses were identical: all
lectures, assignments, and content were exactly the same in all sections_ "
only to to state in the next sentence that the "only aspects of the course
that varied between Dr. Mitchell’s and Dr. Martin’s sections were the course
grader and _contact with the instructor_ ". Well, isn't "contact with the
instructor" significant?

Both [1, 2] use p-values [3], which doesn't increase confidence in the
results.

As an aside, neither paper discusses potential bias the authors might have, in
particular their own _social desirability bias_ [4].

[1] [https://academic.oup.com/jeea/advance-article-
abstract/doi/1...](https://academic.oup.com/jeea/advance-article-
abstract/doi/10.1093/jeea/jvx057/4850534?redirectedFrom=fulltext)

[2] [https://www.insidehighered.com/news/2018/03/14/study-says-
st...](https://www.insidehighered.com/news/2018/03/14/study-says-students-
rate-men-more-highly-women-even-when-theyre-teaching-identical)

[3]
[https://en.wikipedia.org/wiki/P-value](https://en.wikipedia.org/wiki/P-value)

[4]
[https://en.wikipedia.org/wiki/Social_desirability_bias](https://en.wikipedia.org/wiki/Social_desirability_bias)

~~~
snailletters
> Please don't insinuate that someone hasn't read an article. "Did you even
> read the article? It mentions that" can be shortened to "The article
> mentions that."

~~~
vonmoltke
This has nothing to do with the article. This has to do with sources _the
poster chose_ as evidence to support a position. It is definitely relevant to
question whether a poster has read and understood their own citations,
particularly when those citations are not very good or contradict the poster.

------
fossuser
I'm pretty skeptical of the claims in this - it'd be interesting to see actual
data.

Evaluations are one of the extremely few levers students actually have to pull
when dealing with a terrible teacher. I'd expect that given an entire class'
worth of evaluations you'd be able to strip outliers and get some genuine
useful responses.

Though the worst professor I had at RPI held our grades hostage until the
evaluations were in, so they're not perfect.

~~~
olliej
In America teachers get tenure so your feedback has no impact on them
improving.

Also if you look at literally any study of them TAs for harder courses get
harsher ratings, they get harsher ratings from weaker students.

I also spent a number of years as a TA I can say anecdotally that is my exact
first hand experience - good reviews in easy courses like graphics, atrocious
scores in design of concurrent systems. In fact for concurrent systems it was
well known that the TA for the course would literally always get the lowest
review rating in the department, and that was independent of their rating in
any other courses that they TAd.

TAs also get punished if the actual instructor is bad - because students blame
everyone in the course.

I mean the American system has its own slew of problems - lecturers run the
tests themselves, they by design don’t provide prior exams (hint: if knowing
the prior years exam questions tells you enough to be considered your exams
are bad and you should feel bad)

~~~
Y7ZCQtNo39
Very few professors are granted tenure, as was the case in the past. It's
weakly non-zero: times have changed. There are far more adjunct professors
today. Universities are more run like businesses than in decades past.

~~~
bocklund
I don’t have data, but I think most tenure track professors get tenure (if not
at their current university, than another).

Adjunct professors are not tenure track, so there is really no expectation
that they would get tenure. Though I have seen the numbers that say the number
of adjuncts is increasing.

~~~
vonmoltke
The point is that far fewer professors are tenure track to begin with,
compared to past decades.

~~~
olliej
Almost every lecturer my wife has is tenure track or already tenured.

One of them just plays videos of himself in his lectures. He doesn't actually
go to them. He is actually around during less than half of his scheduled
office hours. He complains to (and at least once has shouted at) the TAs about
how students keep asking him questions. He has tenure.

------
katsura
To be honest, I'm not sure how teachers get and keep their jobs, but as
students are graded on their studying skills, I feel like teachers should be
graded every fews years on their teaching skills.

My girlfriend is studying in a distance university, so she basically never
meets any of her teachers. This year she has to do practices at a chosen
company, and she got a teacher assigned to her who is supposed to "help" her.
In her study guides it is written that the teacher has to give her classes
every week. After first contact the teacher said that she will give every
information by email (or other kind of online communication), and won't waste
the students' precious time with classes. At first this sounded awesome,
because my girlfriend has a lot to do in her last year. BUT... Since then 2
months have passed, the teacher has been repeatedly asked to keep at least a
few classes, because things are not moving forward. She ignores her messages
for days, answers in very short sentences, and although it is mandatory for
her to keep the classes, she is always "busy" at those times and doesn't offer
any other dates, so it is impossible to meet her. My girlfriend spends crazy
hours combing through PDFs hoping to find information on how to do her
practices, what she should prepare, how to write her work diary, etc.

And the other day, when she tried to contact the teacher's higher-ups to do
something about this, one guy basically shouted at her, on the phone, saying
that she is making her teacher look bad, even though the said theacher works
really hard, and she should just listen to the teacher and let her be. After
some of this preaching he just put the phone down.

It has been a really frustrating experience for us.

~~~
flashman
> as students are graded on their studying skills, I feel like teachers should
> be graded every fews years on their teaching skills

If a teacher grades a student poorly, there can be repercussions. They have an
incentive to grade fairly and accurately. The same isn't true of students, who
can grade on feelings and grudges.

Teachers are also subject matter experts in the material being graded.
Students aren't trained as educators, so why would they be any good at
evaluating their teachers?

Student evaluations lack even the smallest part of the rigor that they should
contain. They are practically worthless.

~~~
ergothus
> If a teacher grades a student poorly, there can be repercussions.

Yes, but the above poster was talking about TEACHING poorly. It is easy enough
to expect students to know things at the end of class, but harder to be of use
in them learning that material.

> Teachers are also subject matter experts in the material being graded.

Some are. Maybe even most. But definitely not a universally true statement,
particularly in areas where teacher pay is dramatically out of line with
industry wages.

> Students aren't trained as educators, so why would they be any good at
> evaluating their teachers?

Fair point. But as pointed out by others, there are few other forms of
evaluation/accountability. And while a student may not be able to identify WHY
they struggled to learn, they can certainly evaluate IF they did. If many
students agree, that's a problem best addressed quickly.

I teach a university class (just one, my day job is coding) and I've been
utterly amazed at the lack of accountability applied to me and my peers.
Meanwhile I have friends that teach teens, and the bureaucracy and policies
they must follow seem just as bad, but in the opposite direction.

If you have a good alternative to student evaluations, please share, otherwise
I'll agree with the flaws you listed but still find them better to have than
not.

~~~
chithanh
>> Teachers are also subject matter experts in the material being graded.

> Some are. Maybe even most. But definitely not a universally true statement,
> particularly in areas where teacher pay is dramatically out of line with
> industry wages.

So you say that if we find out which teachers are incompetent and actually
kick them out, we would need to raise teacher wages to the point of being
competitive with the industry?

Sounds like a win-win to me.

------
weliketocode
Let me a add a wrinkle to the mix specifically in terms of grade inflation.

The value of most coursework and any learnings gleaned from it, has dropped
significantly.

The best value from entering education institutions is instead from job
opportunities after. Unfortunately, these opportunities are often based on
grades.

If the grades are more important than the coursework, then _of course_
students will optimize for it.

Students should choose institutions & teachers based on both quality of
opportunities and ease of coursework.

Going a step further, teachers that don't understand the changing interests of
their students should be reprimanded.

At my university, most major recruiting for internships and such happened in
one semester. Now, I specifically remember the dichotomy of two of my
professors' approach to this.

Prof John - If you miss a class/deadline/exam due to recruiting, you get a 0.

Prof George - I understand recruiting is happening, reach out to me ASAP if
you feel overwhelmed or need a date changed.

Now, the coursework has largely been forgotten. But, I will _never_ forget
John's idiocy.

~~~
analog31
The grading just gets kicked down the road. Now, because we don't trust your
credentials, we give you a coding exam, that is effectively a grade.

Likewise, an open secret is that college work is a simulation of the
workplace. Those college students who develop through practice the ability to
turn in work on time that makes their teachers happy, will also be able to do
the same in the workplace. If you give up the chance to learn it while it's
available to you in college, where your only risk is a Zero on an assignment,
you will learn it later in the school of hard knocks. Your performance review
will be a grade. There are people who come to work knowing how to get good
grades, and others who don't.

This is coming from one who struggled to get good grades in college. A goose
egg on an assignment is rarely an isolated occurrance.

Knowing educators, and having kids in high school, I know that the kids who
miss a deadline during recruiting tend to have already missed many deadlines
over the years.

I strongly believe that while credentialism and college brand recognition are
real and probably regrettable, if those are your only reasons for attending
college, then you are wasting your money and your life, whereas someone who
has other reasons for attending the same college is getting a lot more value
out of it. But colleges assume they get the same money either way, so it's
really your choice.

~~~
SketchySeaBeast
> Likewise, an open secret is that college work is a simulation of the
> workplace.

I don't think this is the case, and hasn't been for a long time. Assuming
you're not looking at graduate classes college is largely a game of keeping
your head down and doing the work you need to when you need to and the
minimizing effort other than that. Cramming is totally a viable option as the
metrics for success in college are totally different than the workplace. The
skills one learns to survive in college are extremely maladaptive to a long
term functional career.

If the criteria for success are orthogonal, how can one be a simulation of the
other?

~~~
analog31
In my view treating college as largely a game or whose metric is sheer
survival, is wasting money, or more likely, paying to support the students who
are getting something else out of it. This is not necessarily a personal
judgment, because there may be people who are predisposed to see everything as
a game, but who deserve to get the best education possible anyway.

Here's my stupid parable. Give two people shovels and send them into a mine.
One of them comes back with a bag of gold. The other comes back with an
intense hangover. What's broken, the shovel or the mine?

Colleges, despite the image of paternalism, are actually designed to let you
fail. People will leak clues on how to _survive_ , but are less likely to tell
you explicitly how to _succeed_. But everybody has the same chance of getting
a degree in _something_. As a result, the people who benefit from the
signaling and branding of the college outnumber the people who got a good
education out of it, creating the impression that college is largely about
signaling and branding. But college is really about deciding what you want to
get out of it.

There are doubtlessly many things wrong with college education today, but in
spite of that, taking college at face value and pursuing it in a relatively
straightforward way may still be a better strategy than trying to figure out
what game to play in order to survive.

~~~
SketchySeaBeast
> But college is really about deciding what you want to get out of it.

If college is really a simulation of work, does that apply?

------
xtrapolate
> "A teacher (also called a school teacher or, in some contexts, an educator)
> is a person who helps others to acquire knowledge, competences or values."

The target audience is a crucial part of the feedback loop. Removing the
students from the equation sounds counter intuitive. What are the proposed
alternative metrics? Who delivers the feedback? And what is the feedback based
on, if not on the direct opinions of the people on the receiving end of the
service?

~~~
JacKTrocinskI
I think the main idea here is that student evaluations only make sense if we
assume that students are actually interested in acquiring knowledge,
competences, or values, however if their main aim in a course is a good grade
then their evaluations suddenly hold little value since they will rate "easy"
instructors highest.

~~~
dual_basis
When I was a college instructor I found that when I tried hard - putting a lot
of thought into pedagogy, having weekly (open notes) quizzes, assigning
challenging but fair homework - I got terrible evaluations, even though the
average grade in my class was higher than other teachers who were teaching the
same course. When I phoned it in, didn't really try, was very lax with the
homework and often made the quizzes take-home, I got excellent reviews but the
average grade in the class was worse.

~~~
xtrapolate
> "When I was a college instructor I found that when I tried hard - putting a
> lot of thought into pedagogy, having weekly (open notes) quizzes, assigning
> challenging but fair homework - I got terrible evaluations, even though the
> average grade in my class was higher than other teachers who were teaching
> the same course. When I phoned it in, didn't really try, was very lax with
> the homework and often made the quizzes take-home, I got excellent reviews
> but the average grade in the class was worse."

It must've been a frustrating experience.

At the same time - you're making a very broad statement here based on a rather
personal experience. You went from a certain regimen yielding certain results,
to a different regimen yielding different results. There are way too many
parameters here to draw conclusions.

~~~
dual_basis
Sure, but that's the sort of data we have to deal with in this arena. Who's
going to run a large randomized trial where students are purposely assigned to
different classes (keeping in mind that schedule conflicts already add
additional constraints to this which may bias these assignments) and then,
furthermore, have the instruction fixed apart from how easy the assignments
are? Is it even fair to the students to knowingly assign students to
relatively poorer teaching? Clearly it happens all the time, every department
has that professor who is known for being a bad teacher yet they still have to
assign classes.

In speaking with other grad students this seemed to be a well-known
phenomenon, to the point that most other grad students intentionally didn't
put much time into their teaching and basked in the positive reviews as a
result. It was suggested many times to me that I was spending too much time
thinking about my teaching. In my case, the lax teaching was not intentional,
I simply was overcommitted that semester and had less time to prepare.

------
RcouF1uZ4gsC
An interesting way to evaluate professors would be to contact alumni who have
graduated and ask them about the professors that they remember. Chances are
that the really good and the really bad will stand out. I still remember my
calculus professor and statistics professor who both did a great job. I
remember some other professors on the other hand who were bad at teaching
undergraduates. My opinions on my college professors seems independent of the
grade I received or how hard the class was.

Often times, time and experience give a different perspective on how valuable
something was.

~~~
Fomite
> Often times, time and experience give a different perspective on how
> valuable something was.

Indeed. I'm a professor in a STEM field.

One of the most valuable courses I ever took was in medieval feminist art
history.

------
DanielBMarkham
This is the age-old doctor question. You go to the doctor's office because you
have a medical problem.

Doctor 1 comes in on-time, she's smart, capable, friendly, and spends a good
amount of time concerned about you and your situation.

Doctor 2 comes in late, she's irascible, pedantic, hurried, and only listens
long enough to tell you what to do. She also insults you and your family in
various ways and doesn't seem to care.

After the visit, you go to Yelp. Which one was the better doctor?

You have no idea! But do you think that's going to stop somebody from forming
and posting an opinion anyway?

You can make the argument that these other qualities are good to have in a
doctor. I agree. I want a doctor like the first one. But people go to doctors
first and foremost to _get better_ , not to make new social acquaintances. If
the first doc is incompetent and hurting people, and the second doc a genius,
and her attitude is actually causing more people to comply with what's best
for them out of a sense anger at her style, perhaps? That's the thing. Who the
hell cares if the outside looks cool/sexy/friendly or not?

So when we ask for reviews from students who have never worked in the field
they're studying for, they're like that Yelp doctor reviewer. I don't see how
this situation is good for anybody. In fact, it'll probably lead to a bunch
friendly, good-looking morons with great people skills entertaining bunch of
kids who aren't learning anything. That's the only natural consequence here.

~~~
pas
It's not like there can't be an accurate way to measure this.

Pre-register patient problems, and see how fast doctors solve problems on
average/median. Then it doesn't matter how jovial/amicable the doc is.

With enough data you can fish out potential signals (and then test them
properly), maybe bedside manner really doesn't matter, maybe being on time is
more important. (Because then people spend less time in the clinic among other
sick people.)

And as others mentioned the teacher eval problem can be solved by splitting
the lecturer, exam writer and grader roles. (Grading should be done by a book
written as edge cases accumulate, edge cases should be handled without names
to prevent bias, as much as possible.) And then aggregate stats can be
published about each class and school.

~~~
DanielBMarkham
Sorry no, it doesn't work like that. In either case.

Doctors, like professors, are evaluated by former patients based on a mix of
factors.

In your doctor example, doctors don't "solve problems". They see patients.
Many times these patients present with conflicting and hard-to-resolve
symptoms. I imagine with the internet this has gotten worse. They use a mix of
Occam's Razor, deductive reasoning, and social psychology for _the patient 's
long-term health_.

You don't want a doctor that finds the answers to complex problems and gives
out morphine. You want a doctor that over decades has patients with the best
outcomes. In my example, the one doctor may have ticked you off so much you
decide never to go back -- and quit smoking just to prove her wrong. That's a
win for both you and the doc, although there's no problem being solved.

You can speculate various ways to go at this. Perhaps the doctor is evaluated
by other doctors working at that location. Or perhaps it's a matrix of various
things. As long as you're just pulling stuff out of the air, you can invent
any kind of system you'd like. And tech will deliver it for you.

Human interventions are messy affairs, and thank goodness for that. We are not
robots. Your proposal might do a great job of sorting out professors who teach
students well enough to take the test, but heck if I see where that is the end
of the analysis. That's just the beginning. We're just getting started talking
about professors who can lead a class to mostly pass a test. That should be a
baseline for any professor.

This is the problem we see in software development. You can come up with all
sorts of metrics to stand in place of evaluating whether a team/person is good
or not, but whatever you come up with mostly seem to work -- and has very bad
effects on those being measured. (With the possible exception of business-
value-loop feedback time)

I don't think this is tractable in the way you seem to want it. If I were
picking a college or professor, I might look at five or six variables and
weigh them in various ways depending on my personal needs and goals at the
time -- which might change tomorrow. I might want to go interview them. Meet
some current and former students.

There is a severe and onerous "platform danger" in tech. It starts with the
conceit that we can build platforms for anything. It ends up with people not-
as-smart-as-they-used-to-be having Google tell them where to go for dinner. Or
Wally in Dilbert coding himself up a minivan. [0]

I'm not saying the metric definitions and accumulations aren't _interesting_.
They deserve to be considered. I'm saying that by reducing human judgment to
single vector, it makes us as a species dumber. That's one of those intuitive
things that sound good but are in reality quite bad.

0\.
[https://dilbert.com/strip/1995-11-13](https://dilbert.com/strip/1995-11-13)

~~~
pas
Over decades, yes, but that is moving the goalpost.

The vast majority of healtcare problems are dependent on using the right known
technique (medication, surgery, etc.) and of course the problem is finding
that, so diagnosis. Handing out morphine is not a solution in the genral case.

Long-term health is sadly largely independent of acute healthcare providers'
work. It depends on genetics, environmental factors, behavior (exercise, diet,
lifestyle) and a stochastic factor of acute problem solving (as the body
breaks down more and more depends on doctors, and the fast and correct
identification and application of the best healthcare technique).

So healthcare has two categories, the first is the common flu, I have some
gastro issues, refer to specialist, manage chronic illnesses (diabetes, pain,
low/high blood pressure) and the other is usually the problem for specialists,
when they have no or wrong idea what's going on and thus an acute problem
becomes a year(s) long trail of tears.

Now, that said, I'm not advocating for ignoring the edge cases and just use
this one simple magic formula to decide who can play doctor. But we are still
stuck in the dark, because we focus on the complexity, the long term, the
human component, the expertise, yet fail to do the most basic thing to get at
least somewhat reliable data.

And I'm aware of the problem of fishing for signals in data, but without it we
can't even start the trial and error process, we can't differentiate between
the two cases, and on average let's say we end up doing nothing.

Furthermore, we know that the obvious solution is to refactor the system. But
that rarely presents a real possibility for education and human long term
health. (You know, the small stuff :)) It'd be easy to ban smoking and alcohol
and give less damaging stuff for people instead and spend more to treat
addicts, and introduce negative income tax as a form of basic income to help
poverty to reduce stress and help mental health to reduce number of smokers.
Similarly it'd be easy to reform schools, and do all of the above to help
parents to have more time for their kids so they get a better nurturing youth
overall, which makes them ambitious and motivated to learn, study, explore and
understand. And it'd be easy to do all this, but that's not likely. Hence the
proposal for simple but maybe vastly effective changes.

------
enriquto
As a teacher, I disagree strongly with the conclusions of this article.
Anonymous student evaluations are an invaluable tool to improve my teaching.
The students say so many fair and useful things!

~~~
kiriakasis
I think the problem is when student evaluation have an affect (positive or
negative) on the teacher career.

~~~
enriquto
Yes, when that measure becomes a target it may have some unintended
consequences. But the device is too useful to be discarded for that reason.

~~~
narag
The teachers can do it voluntarily and for their own eyes, without making it
an evaluation form.

------
waterpigcow
It is my suspicion that the attitude of american students towards their
professors is perhaps a function of the privatization of schools. since
students are paying for their education they feel entitled to a good grade
regardless of performance. I would like to know others thoughts on this. I
don't know much about post secondary education in Europe but it seems
different from the American collegiate system somehow.

~~~
sdinsn
> privatization of schools

Hmm? There has always been private schools. Nothing is new about them

~~~
dragonwriter
Over the last several decades, US public colleges and universities have
generally seen a declining share of funding from the chartering governments,
relying more on business activity plus student fees.

This is probably what is being referred to.

~~~
travisoneill1
The student fees and tuition are mostly paid for by government backed student
loans. There really isn't any private education in the US past high school.

~~~
waterpigcow
I don't see the government paying for my college... In my experience a
majority of my friends have or are on track to have student debt from having
to pay for college.

~~~
Fomite
Technically you wouldn't see it even if it was there. For most public
universities, it comes as a direct allocation from the state legislature to
the university.

------
olliej
No shit basing compensation on reviews by people who have an overt bias to
blame the instructor.

The purpose of the review should be purely to provide feedback to the
instructor and TA. It should not be available to the administration - the
purpose is to help the TA and instructors improve their teaching. Especially
if the instructors have tenure and so can’t be fired for failing to teach.

Ratings should also be scaled to control for biases - what are the average
grades given by each student, how do they compare to other students in their
classes, and to their course grade.

~~~
eastWestMath
My university explicitly rewards TAs with high reviews. I swear I’ve walked in
on people handing out assignment solutions.

~~~
olliej
In the US at least you're essentially expected to be a TA as part of most PhD
programs, so if your ability to complete your PhD is dependent on the will of
students who you might be causing to fail - or you might just not be a good
teach - why would you not do that?

------
gdsimoes
One of the main problems with the educational system is that students are
evaluated by the same person that teaches them. If the teacher wasn’t the one
grading the mean grade of the students would be a decent evaluation of the
performance of teacher.

~~~
khawkins
If another professor wrote the exams for a class they didn't sit in, you'd
have the students screaming in unison, "this wasn't discussed in class!" Not
exactly disagreeing with the idea, but students have been normalized to the
idea that, with enough studying and concentration, any exam can be aced. When
they feel like they've been cheated by the system they get discouraged, making
a harmful learning experience.

~~~
michaelt

      If another professor wrote the exams for a class they
      didn't sit in, you'd have the students screaming in
      unison, "this wasn't discussed in class!"
    

In standard testing at other levels, this is solved by a syllabus.

For example, in electronic engineering I would expect a second-year class in
switch mode power supply design at any university to look extremely similar.

Admittedly there may be classes in cutting-edge research, opinion-based
subjects or extremely niche topics, where you can't find two academics in the
country who could agree on a year's syllabus and set of answers. In my degree
I'd say less than 25% of courses were that way, though.

------
blakesterz
This is a short commentary piece, not exactly much there. There's some
interesting comments, the one at the top right now says

"They are no doubt imprecise, but for a majority of the faculty the scores
form a tight bunch around a mean. Newer faculty are likely to have lower
scores, so it is often times useful to sit down with them and review the
feedback and talk about ways to improve. A few faculty members continually
score higher than average, and I want to know why--perhaps they can help do
some mentoring with the lower scoring faculty."

~~~
jimhefferon
> mentoring with the lower scoring faculty

Of course this comment assumes that higher scores means better teaching.

FWIW, my experience both as a prof and as an evaluator of profs is that when
students are reasonable good their comments, in general, are reasonably
useful. After all, they are in the room and they see what is happening.

The problem is that when students diverge from reasonably well-prepared and
reasonably hard-working, their responses diverge from valuable. It is a kind
of Dunning–Kruger, where you get evaluations that are hard to reconcile with
any kind of learning goals standard.

All this would be mitigated if there were multiple measures. But there are
not, at least in practice that I've seen.

It is a problem. As an evaluator you want to give people credit for doing a
good job. But it can be hard to tell.

As a prof for many years, I know how to increase evaluations. (I don't agree
that it is grades; I recently had a fifth year review and calculated the
correlation between my grades and class evaluations for those years and r^2
was basically zero.) But the things that increase evaluations are not very
tied to increasing learning and are certainly not tied to increasing the
amount of material covered.

------
tamaharbor
I had taught two semesters of a basic pharmacy course at a local community
college. Of the 20 or so student reviews I received, 80% of them mentioned my
'ta-tas'. Enough said.

~~~
Fomite
I've read the evaluations of a few of my female colleagues. Without fail, a
good portion of them are toe-curlingly sexist.

~~~
jhasse
Care to give an example? Are you male or female?

~~~
Fomite
I'm male.

 _Lots_ of students under the impression that commenting on their dress is
appropriate. A large chunk of those being suggesting they dress in ways more
visually interesting. Some direct references to their appearance (usually
"positive").

One particularly egregious one speculating on who she slept with to have her
position.

 _Tons_ of gendered expectations in language. Expectations to "be more
nurturing" and things like that. Far more comments about them being "young",
"inexperienced" and "new" than my own evaluations, despite being considerably
more experienced than I am (and no, there was no way my class was just more
polished - it was thrown together last minute).

~~~
belorn
> Tons of gendered expectations in language.

That would make for a rather interesting study if they look at gender
expectations overall for teachers. From Swedish studies it has been clear that
male teachers leave the professions significant more and earlier than female
teachers, and same with male students for the teacher master program in higher
education. The numbers is very similar if not almost identical to master
programs in STEM except for the genders being reversed, a fact that is rather
unexplored in gender studies but noted in a somewhat recent government study.

Just looking at evaluations, I wonder how height, build and wealth symbols
(expensive car, clothing, jewelry) impact the rating for male teachers. Do a
non-typical male traits contribute more negatively to the score for male
teachers than non-typical female traits do for female teachers? Same question
for typical male traits and female traits. Is it correlated to leaving the
profession, and is there a difference in abuse tolerance?

------
wisty
There's a much better way to hold educators to account - have other educators
sit in on classes. But this is often very strongly resisted, apparently it's
'unprofessional'.

~~~
AndyMcConachie
I'd be curious to hear who thinks this is unprofessional. Not any educators I
know.

~~~
jcagalawan
While I can't find any specific sources in the field of education, I was
taught in engineering school that engineers reviewing the work of another
engineer, unless their job specifically required them to review work, was
against the code of ethics in Ontario. It looks like this is true as well in
the USA which has this tidbit in their code of ethics.[1]

"a. An Engineer in private practice will not review the work of another
engineer for the same client, except with the knowledge of such engineer, or
unless the connection of such engineer with the work has been terminated."

[1][http://ethics.iit.edu/ecodes/node/4099](http://ethics.iit.edu/ecodes/node/4099)

~~~
nickloewen
This is different though, because of the “except with knowledge of said
engineer” line. This code is trying to prevent clients from, for example,
getting a “second opinion” on a project if they are unhappy that the first
engineer deemed it unsafe. This isn’t really an issue in teaching. I think
that it would be impolite to just drop in on a colleagues class unannounced,
especially if it is a bit lecture where you might go unnoticed, but I don’t
see why a prearranged visit would be unprofessional per se. (Perhaps because
it would cause students to doubt the quality of their instructor?)

FWIW my understanding is that at my university the response to poor evals is a
visit from a colleague, to assess whether the teacher really is weak or if the
material is just difficult. Depending on that, the instructor may get
additional coaching in teaching technique. This seems like a sensible approach
to me.

------
Y7ZCQtNo39
We pushed very hard at my alma mater to publicize course/teacher evaluations.
We eventually got the University to accept iff a minimum participation rate
was met (on a per course basis-- so it wasn't all or nothing. We'd get a
subset of the data when enough participation occurred. We considered this
reasonable, since if very few students participate, the data isn't all that
meaningful).

Regarding professors wanting to keep review data tightly sealed: in my view,
if you can't by public disclosure of your evaluation, then you either don't
feel you're meeting expectations, or have no desire to improve in areas where
students feel improvement could be made.

Also, the biases pointed out in these reviews aren't unique to academia.
Gender and age biases exist everywhere. This article sounds like it's just
pushing the idea that students should have less influence in the hiring and
promotion decisions of professors. You know, the very people that the teachers
first and foremost serve at a university.

~~~
cortesoft
> Regarding professors wanting to keep review data tightly sealed: in my view,
> if you can't by public disclosure of your evaluation, then you either don't
> feel you're meeting expectations, or have no desire to improve in areas
> where students feel improvement could be made.

There is also the third option, which the author is arguing for: that the
professors do not think the ratings are an accurate reflection of their
teaching skills.

~~~
mcguire
I'm sure many professors would agree.

Perhaps the engineering statistics professor I met who bragged that no one
ever got an A on his final, because teaching wasa competition between him and
the students. And the government professor I had who was inordinately proud of
the fact that his course was required because three soldiers from Texas stayed
in China after the Korean war and spent the rest of the first class going
around the room having students introduce themselves and then mocking them. (I
dropped the class the next day.) And a number who were just disorganized and
incompetent, but protected by their relationship with other faculty. And the
professor who was given tenure for political reasons, after threatening to
fail an entire class (of a required course) of computer science undergrads
because they weren't electrical engineering great students.

~~~
WalterSear
I bet the adjunct whose english was so bad that she would just answer a
question kind of close to what you asked, as quickly as possible and try and
move on would too.

She also answered any questions asked in Mandarin, in Mandarin. When asked to
translate an exchange for the rest of the class, she blushed and said 'it was
complicated'. At the end of the semester, the class was so far behind, 1/3 of
the materials for the course exam were delivered during an optional study
session.

And so would the electrophysiology professor who spent over an hour of a grad
seminar explaining how to use a floppy disk. Because he had trouble with
computers.

------
duchenne
A possible solution would be to split the job, to avoid conficts of interest.
One teacher teaches. Another one grades. When I was studying in France in
"prépa", all students wanted to get a real evaluation of their level, because
the really important exam to enter the best universities was organized nation-
wide. But, this is hard to organize in private universities where students are
also clients. Which parent would pay 40k$/y to be told that his kid does not
study well? Maybe, here also, to avoid conflicts of interest, the job can be
split. One university teaches. Another organization grades.

~~~
chmod775
This is precisely what some private schools in Germany do (yes, there are a
few of those).

They might've started doing so to combat the notion that private education is
worth less because it just might be easier.

------
rb808
My best advice I got about college teaching:

University is a place of higher learning, not higher teaching.

If you're relying on being spoon fed the course, you're doing it wrong.

~~~
invalidOrTaken
What exactly are universities selling in that case?

From an individual standpoint, sure, a student is always well-advised to take
charge of their own learning process. But if you can do _that,_ what do you
need a university for?

At this point I'm pretty sure the answer boils down to "a piece of paper and a
dating pool."

~~~
baroffoos
Honestly I wouldn't be surprised if uni for tech becomes a thing of the past.
Uni didn't teach me anything I didn't already know from working on personal
projects and im fairly sure my public git repos had far more effect in job
interviews than uni did. I have talked to others who have finished and they
hardly know anything compared to those I know who are self taught.

------
sonnyblarney
In my experience this has been good for weeding out really bad profs. At my
Grad school they turfed a prof 1/2 way through that was bones bad.

That said - it's perverse at the other end and I do believe that there is
unconscious sexism.

We had a really smart bunch of people, and some of the female teachers were a
little more apprehensive in front of us which I think signals to people
subconsciously.

Another way: the feedback has to be _interpreted_.

I suggest that the profs should not be 'graded' by students - rather there
should just be an open ended opportunity for feedback.

------
booleandilemma
It’s been awhile since I was a student, but I remember the reviews on
ratemyprofessors.com to be pretty accurate.

There were some asshole teachers employed at my college and they were called
out for it accordingly.

------
nocobot
I'm a CS student in Germany and contrary to the authors claim we actually do
evaluate teachers (to be fair that might not have been common place when she
was teaching).

This is obviously anecdotal but in my experience these ratings seem to reflect
my perceived quality of the lectures quite well.

------
chuckgreenman
Sure, evalutations are flawed, but it appears that universities are unable or
unwilling to develop other quality measures. For example, this semester I've
had a networking professor that would get half way into explaining something
interesting and then state that he didn't care about the topic so he didn't
prepare to talk about it and we'd be moving on to something else.

I've got an A in that class, but I wouldn't say that I learned much from it
due to that attitude. I don't think that the CS department here has any way to
tell that something like that is happening without evaluations.

~~~
bane
Right. I think anybody reading comments in these review sites can see if
students are ranking professors well for being "easy" or if they're ranking
them well for being great educators.

The universities I went to for my undergrad and graduate programs both had an
informal rating system for professors and I definitely encountered some of
both.

One of the most highly rated professors I encountered was also one of the best
educators at the school. He had a reputation as being a brutally hard grader.
But most of the students who left his classes with some bruises and (what
would have been in any other class) a mediocre grade, ached to take more from
him because you felt the immense value of his teaching.

I had other professors who taught poorly, tolerated cheating, or gave non-
sequitur exams on material they never covered and so on. There was absolutely
no recourse or way of providing feedback on these professors and even ones who
received universally poor reviews at the end of the semester stayed on and
even attained tenure in some cases.

The only _real_ recourse was for students to rank professors amongst
themselves and simply starve out poor professors with lack of registrations.
_That 's_ the only signal universities seem to respond to.

~~~
chuckgreenman
I got the impression that the author was talking about department run
evaluation sheets rather than review sites. Student evaluations shouldn't be
seen as an evaluation metric but a smoke detector that indicates a staff
member might need some more direction.

Teaching is an art, just because someone isn't doing it well doesn't mean they
can't do it well. In fact in some cases I imagine they are doing poorly
because they would rather be working on their research area. Providing the
opportunity to course correct rather than starve out is good.

------
danmaz74
There could be sexism at work or not, but how can you even define "identical"
teachers? I never saw two teachers that were identical (even if they had the
same gender, ethnicity, age and sexual orientation).

------
ccleve
If the school would force teachers to grade on a curve, the problem would go
away. Teachers would have only a limited budget of high grades to give away.

Grading on a curve is not without problems, particularly when class sizes are
small, but it has a lot of benefits. It makes it easier to compare students
across schools when the schools use the same curve.

~~~
azhenley
That is a horrible idea. Then students will try to take courses with weak
students. Oh, that smart group is taking CS340 next semester? Guess I'll wait
until the following semester. Do not turn it into even more of a competition
than it already is.

Why should someone else's ability influence my grade???

~~~
thaumasiotes
Because the meaning of your grade is that it gets compared to their grade.

~~~
azhenley
Says who? Why can't it be used to evaluate my individual learning/effort?

Grades only make sense to compare _within_ the same class at the same
university during the same semester.

~~~
the_jeremy
Grades are relative to other people. That's the point of places asking for
GPA. If you were just evaluating your effort / learning, your grades could
just say "learned a lot" or "made a good effort". But GPA is a way to
discriminate between potential candidates for schools/businesses, and thus
needs to be quantifiable and comparable.

~~~
closeparen
No, not at all. GPA tells the difference between "mastered the material" and
"did well enough not to get expelled" which can be a useful signal for an
employer, but universities and professors don't view it as their
responsibility to sort students for industry's needs.

For starters, the quality of students is not uniform across universities.
Middle of the pack at Harvard is a very different thing from middle of the
pack at a party school. To get a useful ordering, you'd need to give every CS
graduate the same standardized test, like the bar exam.

~~~
wahern
Grades are relative to other people in your school and program; it's why you
mention the institution on the resume. And IME you don't often communicate
your grade but your class rank, most notably if you're at the very top
(valedictorian, etc); even with grade inflation schools still must be able to
differentiate the very top students from everybody else so they can actually
provide these honors. You can't graduate an entire class of "top" students, no
matter how earnestly they studied.

With few exceptions there is no mastering the material, or at least there
shouldn't be. Not in college. There's always more complex material you can use
to differentiate students. When you grade on a curve the test should be
difficult enough that even the best students will typically get at least one
question wrong, with most students falling along a nice bell distribution. (I
say most because it's probably better to err on the side of a handful of
students clustering at the top rather than at the bottom, especially if the
bottom means flunking out.)

This can be more difficult for smaller, seminar style classes. One solution is
giving the professor leeway to shift up and tighten the curve so he's not
forced to give Fs or Ds. These classes are usually in the latter years of a
program, particularly in undergraduate, where it's less important to winnow
students out and ensure challenging curriculums.

That latter point is important. If you don't enforce some sort of statistical
distribution how can you gauge the quality of your curriculum? If everybody is
getting As is that because all your students are smart and disciplined, or
because the material is too simple? If you require a bell distribution, then
the curriculum will by _necessity_ be a good fit for your cohort of students.
In this way a university can maintain a quality curriculum without having to
resort to external metrics or comparisons with other schools.

~~~
closeparen
I'm aware that this philosophy of education and grading exists, and it may
well be superior, but it's not how things worked in any part of my education.
Grades indicated how well you met the instructor's expectations. Most classes
gave mostly As and Bs: if you weren't going to do well relative to the course,
you wouldn't let it get to the point of a letter grade.

Instead the differentiation was between departments and courses. It was well
understood that X majors were Y majors who couldn't hack it, and that within a
major, certain classes were for high-achieving masochists while others were
schedule padding. (This was crucial information, not just dick-measuring. You
could easily put yourself in crisis by taking a class beyond your abilities,
or multiple hard classes at once. Friends helped each other avoid such
nightmares). If you asked me to evaluate a classmate's transcript, I wouldn't
even look at their grades, only at the classes they allowed themselves to have
grades in.

No one has ever asked me that, or to see my transcript.

------
cyberpanther
This is also a problem with developer bootcamps since they get a lot of
"sales" from reviews. They can often cater to students to the detriment of the
learning experience. At times it is about catering to creature comforts, but
can bleed over into lessening the learning experience.

And especially in bootcamps since things are so fast paced and often times
emotional, the reviews are not objective. Many are too positive or too
negative.

------
killjoywashere
Teachers should be judged on the longitudinal performance of their students.
Khan Academy and BlackBoard must have some utterly fascinating data in this
regard.

------
augbog
As someone who had a teacher who literally was tenured, had the absolute worst
reviews ever on the website and wherever, literally recorded his lectures once
and now goes to lectures just to answer questions (which btw he barely answers
em he sometimes just posts to the lecture video) I strongly feel evaluating
teachers is a necessary thing.

~~~
wahern
Is something preventing you from communicating your concerns directly to the
administration (in person or anonymously) outside the context of a
perfunctory, after-the-fact review?

I would imagine a truly poor teacher would standout for the number of
unsolicited complaints.

~~~
augbog
Truthfully I have no idea what he even does with the required review papers
that are given to us a the end of the class. Lots of people have reviewed him
on Askmyprofessor.

------
Overtonwindow
Possible alternative method is to hire and train students to be observers,
both covert and overt. Put them into classes as if they were students, and
those students Would then Evaluate that professor. It could work especially
cross colleges, where those students will never naturally take classes by that
professor.

~~~
pas
Around here the student associations have seats on the accreditation bodies
and they do indeed sit on in classes as they tour the county.

I don't think this is effective, but it happens.

------
nijave
This article seems to suggest larger problem with grades than reviews. It
seems like reviews are just aggravating the grade problem.

------
starbeast
I would also assume that the students, taken as an average, would also be some
of the worst qualified people to evaluate educators, pretty much by
definition.

------
tgsovlerkhgsel
As I expected, this is an opinion piece from a teacher whining because they
got bad ratings:

> I got a terrible rating, and its publication humiliated me.

While this doesn't mean that the article can't make valid points, large parts
of it also complain about "today's youth".

I suspect this got upvoted mostly based on the contrarian headline...

------
epx
I was a visiting professor in a local university for a while. The student
evaluations were directly proportional to the grades - easier courses = better
grades. I refused to see the evaluations and refused to discuss them with my
supervisor, it was pointless. BTW my long-term relationship with ex-students
is great, at least with the ones I care about.

