Hacker News new | past | comments | ask | show | jobs | submit login

A subtle point you may have missed, amazon knew about and accounted for the gender bias, the scrapped it because of all of the biases that they couldn't identify and were leery of. Most of your suggestions seem to be solving for the known biases, which I believe they did.

Also knowing some people who worked on this, they were VERY cognizant of re-encoding biases from the start of the project, it was one of the main reasons they thought the project might fail.




I did not at all get from the article "amazon knew about and accounted for the gender bias".

"Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory." I read that as a very different statement - as written, Amazon corrected two specific instances of keyword gender bias by hand, but couldn't reliably prevent further bias (including gender bias) from arising. That's where tricks like "ask the system to classify gender, and then un-train via that data" come in.

(I don't mean you're wrong, just that if gender bias was accounted for more generally, the article should have said so.)

That said, I think our disagreement might just be a miscommunication on what went wrong in the first place. If you know some people involved, maybe you can help clarify the situation?

The article totally fails to explain why "most engineering resumes are from men" led to an algorithm that downrated female resumes. "Most applicants had brown hair" does not produce a system that downrates blondes if you tell it hair color. So the question is - was the training data biased against female applicants (in which case why wasn't it caught before specific outputs needed modification?), or did something else altogether cause this issue (in which case what?)


I believe the system was trained on successful candidates. And Successful means: They were retained and likely promoted after hire over the next 3 years.

If they only trained on who was hired they wouldn't really know if those were good hires.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: