> Fascinating tech. imagine if human element is removed from critical decision making, meaning bias is drastically reduced (if not completely eliminated in this case)
I mean, we can get a more real example easily. Amazon had an AI recruiting tool that was biased against women. The explanation is very simple, the training dataset of successful hires mostly consisted of males so the AI though nothing better than penalizing women.
The question is, how do you detect such a bias, and how do you prevent it in the first place? We have some ideas when it comes to humans, but for code it's much more challenging.
I feel like this is a very disingenuous way to put this issue. It's not that people write a program that actually have lines of code like this in them but rather machine learning algorithms operate on our current reality rather than any ideal. With few exceptions a pregnant employee is worse than a non pregnant one so if all employee data is fed into a nice ML black box with all things being equal it's going to start firing the pregnant ones.
And then an engineer has to correct it by determining the weight of pregnancies in general. Some will call it fixed, some will call it stupid. The latter one is far more correct though.
algorithm could not be based on that. what you described is highly personal and doesn't make sense when optimizing for profits. When optimizing for profits all that matters is value created/delivered (or loss reduced, risks averted).
One can speculate about possible (and/or highly personal) correlations while not realizing that the outcome is completely fair.
> algorithm could not be based on that. what you described is highly personal and doesn't make sense when optimizing for profits.
That's irrelevant. My point is that no bias is eliminated at first place, since the algorithm is still man made. You could argue that since there is no human intervention the same input would lead to the same output, but again, neither you or I have read the code to claim that it is somehow "bias free" or even completely deterministic, especially if the input relies on computer vision.
1. "no bias is eliminated at first place, since the algorithm is still man made" -
This is a straw man. if specific criteria are being automatically evaluated bias is most certainly reduced (if not eliminated).
2. "especially if the input relies on computer vision" - evaluation criteria can be infinitely improved upon
3. "neither you or I have read the code" -
I highlighted transparency concerns in my original comment
4. Unavoidable, unintentional implementation flaws (bugs) are expected but so are fixes and continuous improvements.
I would certainly trust an algorithm to make a decision about my employment over an unpredictable, flawed, and (often times) incompetent human management
The algorithm could be biased at first place.
I'm being hyperbolic but you get the gist. Somebody wrote that program.