That is probably the exact opposite of what you really want. If the problem is that someone's name is Bob and the AI thinks Bobs are evil, what you want is for there to be 100,000 other factors for Bob to show the system that it isn't so, as many factors as possible, so that the one it gets wrong will have a very low weight.
You don't understand. My point is that institutions making such decision should not be able to make decisions according to these 100,000 unexplained factors. If you're a lender, you can look at employment history, records of payment and other objective related criteria. You can't look at, say, eye color, however useful you might think it is. Institutions should be able make these decisions arbitrarily, at the level that they effect lives. There should legal provisions for auditing these things (as there are, on occasions, provisions of auditing affirmative action, environmental protection behaviors, insurance decisions, etc).
> My point is that institutions making such decision should not be able to make decisions according to these 100,000 unexplained factors.
But how does that help anything? The objective factors have the same potential for bias as the seemingly irrelevant ones. All you get by excluding factors is to increase bias by not considering information that could mitigate the bias in the factors you are considering.
Suppose that 80% of black men would be rejected for a loan based on your preferred set of objective factors. Of that 80%, many would actually repay the loan, but none of the objective factors can distinguish them from those who wouldn't, and when averaged together the outcome is to refuse the loan. If you used some seemingly arbitrary factors that happen to correlate for unknown reasons, you could profitably make loans to 60% of them instead of 20%.
Let's assume we push it from 20% to 23% (I don't think we can expect the huge gains you posted) by using various weird features such as whether you like to purchase odd or even number of bananas.
People's live will depend on the decisions of these machines so people will start trying to game them. They will make sure to always purchase odd number of bananas, they will wear hats but only on Thursdays etc etc.
Now two things happen. As more people game the system the rules need to be updated. Suddenly it's all about buying bananas divisible by three and wearing hats on weekends. The people who tried to follow the previous advice got screwed over, and what's more they have nothing to show for it. Instead of making people do useful things like paying bills on time and saving up some money, it made them follow some weird algorithmic fashion. Because of this expenditure of energy on meaningless things we may see that now only 18% of people would manage to pay back loans on time.
> People's live will depend on the decisions of these machines so people will start trying to game them. They will make sure to always purchase odd number of bananas, they will wear hats but only on Thursdays etc etc.
But that's just another reason to use 100,000 factors instead of twelve. If someone's income is a huge factor in everything, people will spend more than the optimal amount of time working (instead of tending to their family or doing community service etc.), or choose to be high paid sleazy ambulance chasers instead of low paid teachers, because the algorithm makes the returns disproportionate to the effort.
If buying an odd number of bananas is a factor but the effort of learning that it's a factor and then following it is larger than the expected benefit from changing one factor out of thousands, they won't do any of it.
> But how does that help anything? The objective factors have the same potential for bias as the seemingly irrelevant ones.
Given a choice between observable, identifiable and modifiable rules or hidden, poorly understood rules integral to a whole model, I'll take the former every time.
Bias will continue to exist for now. What we need to do is make sure we always build processes to appeal and review our systems, preferably in a public way.
The whole problem is that single cases are not statistics yet people would live to apply global generalized statistics to single cases.
What you touched upon is the accuracy/ bias trade-off. To have evidence in particular case you need to attempt to debias the particular system and see how it affects accuracy. Sometimes, it may even vastly improve it.
What is more important is that the systems are not benchmarked properly. As in compared against very simple metrics and systems. Such as: against random decision. Against simple recidivism prevention (grudge system). Against plain math metrics with constants.
To add, they're opaque and it is impossible to easily extract the factors that went into any given single decision. This means they act fully irrationally. Intelligently but irrationally.
The idea of society moving forward is removing systemic bias and other entrenched forms of discrimination and prejudice that have occurred in the past and continue to occur. And this requires human thinking, intelligence and sustained effort.
Doubling down based on historical and backward looking data does not seem like the way forward and can only perpetuate entrenched bias.
All the inferences and correlations will reflect that. This is not intelligence and can only take you backwards.
You don't understand. My point is that institutions making such decision should not be able to make decisions according to these 100,000 unexplained factors. If you're a lender, you can look at employment history, records of payment and other objective related criteria. You can't look at, say, eye color, however useful you might think it is. Institutions should be able make these decisions arbitrarily, at the level that they effect lives. There should legal provisions for auditing these things (as there are, on occasions, provisions of auditing affirmative action, environmental protection behaviors, insurance decisions, etc).