
Flawed Algorithms Are Grading Millions of Students’ Essays - elorant
https://www.vice.com/en_us/article/pa7dj9/flawed-algorithms-are-grading-millions-of-students-essays
======
Balgair
> That combination of results, Williamson and the other researchers wrote,
> suggested many students from mainland China were using significant chunks of
> pre-memorized shell text.

Well, that's one heck of a way to game an AI!

It reminds me of the confucian exam system [0] where you had to just straight
up memorize a huge book of Master Kong's scripts and hope the person that re-
copied your work did so too.

> ... they found that the machine boosted students from China by an average of
> 1.3 points on the grading scale and under-scored African Americans by .81
> points.

Note that this is on a 0-5 point scale. So that says that the mean for the
Chinese students was ~26% higher than the human grade and for African American
students it was ~16.2% lower. The Chinese students could be considered (very
roughly) to be boosted from a C- to an A+, and the African American Students
from an A+ to a B-, for comparison.

What a mess.

[0]
[https://en.wikipedia.org/wiki/Imperial_examination#Details_o...](https://en.wikipedia.org/wiki/Imperial_examination#Details_of_the_imperial_examination)

