
Revealing Algorithmic Rankers - nkurz
https://freedom-to-tinker.com/blog/jstoyanovich/revealing-algorithmic-rankers/
======
yummyfajitas
I've been thinking all the recent hostility towards algorithms - including
ProPublica's deliberate lies about them which are cited by this very article
([https://www.chrisstucchio.com/blog/2016/propublica_is_lying....](https://www.chrisstucchio.com/blog/2016/propublica_is_lying.html))
- and what might be motivating it.

I have the following theory. Consider the proposition of formalism - the idea
that the stated public rules must, in fact, drive the expression of political
power. It's pretty clear that we don't live in a world governed by formalism.
As one example, intimately related to many of the critiques of algorithmic
decisionmaking, we claim to favor individual fairness (the proposition that
individuals should be judged on their own individual merits) yet in practice
we push for equality of outcomes (the proposition that processes should make
decisions that do not vary across certain reference classes).

However, algorithmic decisionmaking _actually implements formalism_. If you
tell an algorithm to predict which student is most likely to graduate college,
or which loan applicant is most likely to repay, the algorithm will do exactly
that to the best of it's ability given the available data.

In many cases, the algorithm will even correct bias in it's inputs:
[https://www.chrisstucchio.com/blog/2016/alien_intelligences_...](https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html)

But the thing is, the formalist output of algorithms is not necessarily in
agreement with what political power _actually_ does in the real world. For
example, political power in the world strives to have proportional
representation of different racial groups in college, yet algorithms aiming to
predict grades/graduation probabilities would actively penalize blacks (who
underperform relative to what a race-neutral model predicts).

As such, algorithms are a huge threat to existing power structures, because
they both implement formalism and they also reveal the lies behind our
existing stated principles.

I'd love to hear what others think about this theory.

~~~
sp332
There's no reason to think that the bulk of women or black people would have
different merits than the bulk of white men. So when we see those outcomes
diverging, we assume that some something other than individual merits caused
it and we act to bring the outcomes back in line with the assumed-equal
merits. In other words, forcing equality of outcomes is just supposed to be
correcting for some external thing that does not treat people according to
merits.

~~~
yummyfajitas
I'm aware of the theory. But algorithms are another way to formally provide
equality of opportunity.

The algo formally encodes "admit the best students" or "give out loans to the
best credit risks". That's what it does. Maybe your ad-hoc procedure will be
reproduced, maybe not.

My belief is that "equality of opportunity" is a lie, just a smokescreen to
gain equal outcomes, and automated formalism is a threat to this.

~~~
sp332
Your algorithm would take literally any form of discrimination that had been
applied to a student so far in their lives as a signal to discriminate against
them even further. It wouldn't look at the reasons a student's grades were low
and consider whether it was their "merits".

~~~
yummyfajitas
Yes, if your actual goal is some sort of compensation for historical wrongs,
you need to openly admit that and formally encode it in your algorithm.

That's formalism.

