The thing about this discussion is that it aims to balance social welfare goals and profit maximizing goals but without any criteria for attaining basic fairness on a general level.
To wit, suppose a machine learning algorithm is looking at twenty or fifty piece of data about a given individual, all of which are entirely irrelevant to the individual's chance of repaying a loan. But by random chance, one of those pieces of data, say handedness, happens to be correlated with a group's repayment history. So, some individuals with a good handedness are given loans more frequently and some individuals, with a bad handedness, are given loans less frequently. This situation doesn't matter to the company since the data is irrelevant and they only give out so many loans anyway and shitting on, say, left-handed people, gives them no grief. Moreover, if this trend is noted by all the companies soon it will be made "true".
Which to say, companies making life-defining decisions like mortgages or parole-granting, just should be prohibited from the using a lasagna of random data to make their decisions and instead should be required to use specific rules with specific reasons behind them.
And cry me a river about missed chances for optimization. This about the structure of society and optimization doesn't benefit society here imo.
That's what the article is about.
Hold on, excuse me?
It's particularly insidious too since a model that accidentally is making credit decisions based on race helps to perpetuate the inequity in the future data.
If you want to keep commenting here, please (re-)read the guidelines and use this site as intended from now on. The intention is intellectual curiosity, and that is the first casualty of ideological war.
and i didn't say that he said that racism is correlated with anything.
However, if something like "race" or "gender" is actually being used as a feature input, then the output from most ML strategies is highly likely to pick up on correlations between racial and gender groups and certain outputs. That is undoubtedly going to lead to negative discriminatory outputs.
So while I see absolutely no problem with the first scenario I mentioned, and I clearly see a problem with the second scenario...I'm inclined to believe that the politics of many people would lead them to have a problem with both.
Machine learning doesn't necessarily do what you want it to do; it probably will "cheat" and use an easier proxy variable, unless you watch out for it.
There is the urban legend about the machine learning program that learned to "recognize" tanks based on time of day in the training set. It probably never happened  but it gets the point across.
As for whether police attention creates feedback loops, we can check by looking at crime victimization surveys, which generally show the same patterns of racial disparities as arrest records. For that matter, the black / white murder gap has been stable at about 6-8:1 for as long as we've been keeping track, and you can't change that by selectively hassling certain people for weed.
Put another way, if you took the first-order strategy you'd come up with by looking at the body count, and then layered the most moustache-twirlingly racist jaywalking policy that you could think of on top of it, the two strategies wouldn't look that much different.
That's all before questioning why it is a bad thing for neighborhoods to be policed for misdemeanors. I lived in West Baltimore for years and I can assure you that my neighborhood was, if anything, chronically underpoliced.
>> There might be unfair feedback loops, but they would be effectively negligible.
First of all, it is curious to me that you mention the 8x disparity in murder rate but don't mention that the violent crime survey disparity is less than 0.2x. Secondly, you just claim that the policing has nothing to do with people being arrested at different rates but do not offer any evidence.
Here are the actual numbers if you really care about facts. The victimization rates are 20.5% whites vs 24.1% blacks. Less than 18% difference. On the other hand incarceration rates are 0.7% whites vs 4.5% blacks. That is more than 540% difference. This means the blacks are being arrested at about 32 times the rate at which they commit violent crimes. So your claim that:
>> "we can check by looking at crime victimization surveys, which generally show the same patterns of racial disparities as arrest records"
is not only ignorant and false. It is so far from the truth that it is laughable.
Discrimination isn’t solely measured by input but also by outputs.