
Machine learning is fundamentally conservative - macawfish
http://blog.lareviewofbooks.org/provocations/neophobic-conservative-ai-overlords-want-everything-stay/
======
jonnypotty
Teach AI using our behaviour, AI learns our behaviour. A bit like our
children. I'm genuinely confused as to the alternative.

The objection seems to be based on the falasy that technological progress
equals social or political "progress". Why on earth would we expect AI
descision making to display a lack of prejudice when human decision making is
suffused with it.

The only people who expect technology to act like a benevolent god are the
ones who have replaced their god with it. All technological progress does is
to increase the power and influence of human beings. The progress the writer
seems to want is socio- political, not technological.

~~~
MaxBarraclough
> Why on earth would we expect AI descision making to display a lack of
> prejudice when human decision making is suffused with it.

Particularly when we consider what we mean by prejudice, which is presumably
something like _Making a decision on grounds which we deem it important to
ignore_. This is a very complex concept. It's a function of society, and
changes with society. It's not something with a rigorous definition.

Obvious example: reasonable modern people know it's indefensible to make an
engineering hiring decision on the grounds of ethnicity, regardless of whether
there are any correlations associated with ethnicity. This is even enshrined
in law in many countries.

To make a decision on the grounds of someone's qualifications and job
experience, however, does not count as prejudice.

We should expect a machine learning system to act as a correlation-seeker
(that is after all what it is designed to do), without a nuanced understanding
of what prejudice means.

We've seen this issue crop up in the context of an AI having a say in parole
decisions. [0] also relevant discussion at [1]

[0]
[https://www.forbes.com/sites/bernardmarr/2019/01/29/3-steps-...](https://www.forbes.com/sites/bernardmarr/2019/01/29/3-steps-
to-tackle-the-problem-of-bias-in-artificial-intelligence/)

[1]
[https://news.ycombinator.com/item?id=13156153](https://news.ycombinator.com/item?id=13156153)

~~~
marcosdumay
> We should expect a machine learning system to act as a correlation-seeker

We should expect machine learning systems to dismiss accidental indirect
correlation for the benefit of the variables that are directly correlated.
There are plenty of algorithms that achieve that, it's only that gradient
descent doesn't.

The fact that our AIs are becoming biased is a bug. It should be fixed.

~~~
muldvarp
> The fact that our AIs are becoming biased is a bug. It should be fixed.

In many cases, it's the data that is biased. In that case it's impossible to
differentiate between bias that the AI should learn and bias that the AI
shouldn't learn.

Let's assume we have a database of incidents where a person was found to have
cannabis. This database has the following data items: a timestamp, the persons
name, the persons ethnicity and the amount of cannabis that was found. Now
assume further that black and white people have the same base rate of cannabis
use (which according to the studies I found seems to be the case). The last
thing we have to assume in this case is that this database was created by
racist policemen who arrest more black people for cannabis consumption.

An AI trained using this data would assume a higher base rate of cannabis
consumption by black people. It's impossible for this AI to differentiate
between correlations it should learn (for example that people who used
cannabis multiple times and were found to have much of it are worth looking
at) and (untrue) correlations that it shouldn't learn (that black people have
a higher base rate).

The correct solution here is to use a dataset that is not biased, but it's
hard to tell whether a dataset is biased.

~~~
tlb
The data can't tell you how the groups differ, since you can't tell the
difference between criminal behavior and policing behavior. So you have to add
some priors. The most progressive approach is to assume that there are no
intrinsic differences between protected groups, and any difference in the data
is the legacy of past discrimination.

You can add such a prior by adding a term to the loss function that penalizes
any difference between the way the groups are treated. The math isn't hard,
only the political decision of what is protected and what isn't.

~~~
MaxBarraclough
> assume that there are no intrinsic differences between protected groups, and
> any difference in the data is the legacy of past discrimination.

I don't see why we should assume that this would reflect reality.

If a law has racist roots, i.e. if it is written to target a particular
ethnicity, then we should expect that certain ethnicities really do break that
law more than others.

------
reilly3000
Conservative in this sense means something like 'resistant to deviation from
established norms'. I think a lot of the headline-only readers take
conservative to mean 'of the character of a specific political movement' which
ironically seems more activistic than change resistant.

~~~
TeMPOraL
Perhaps you could call it "integral controller" (like the I in "PID
controller")? Because systems that have a memory behave like ones, and we are
definitely in a feedback loop with those systems.

And, from what I remember from my control theory classes, the integral part of
a controller introduces lag, inertia, generally making the output more
resistant to input changes.

(Also note that the "non-conservative" Proportional and Derivative components
in a PID by definition don't learn - they react to input and its change.)

~~~
reilly3000
That's a pretty interesting corollary! I suppose I could extrapolate that in
such a system, too much change over too little time would invite a strong
conservative response / lag to future input.

Our 20 year old millennium has seen a tremendous rate of change in both
technology and social norms. Perhaps politics has some similar inertial
dynamics.

------
bordercases
> Data analysis is as old as censuses of the tax collectors of antiquity —
> it’s as old as the Book of Numbers! — and it is unquestionably useful. But
> the idea that we should “treasure what we measure” and a reliance on
> unaccountable black boxes to tell us what we want and how to get it, has
> delivered to us automated systems of reaction and retreat in the guise of
> rationalism and progress. The question of what the technology does is
> important, but far more important is who it is doing it for and who it is
> doing it to.

See "The Cult of Information" by Theodore Roszak.

~~~
intuitionist
I think the idea of ML as “unaccountable black boxes” is a bit of a bait-and-
switch from the problem being described. The problem is that ML has _no_
political biases, it just minimizes some loss function on the data you give
it. So it can’t correct for implicit biases in how that data was collected, or
how the model’s outputs are used.

If you fitted a decision tree to predict recidivism risk, it would be
extremely easy to interpret. But if black men are rearrested more often in
your dataset, then black men will likely have a higher predicted risk on
average—no matter the causes of that feature of your dataset.

~~~
Iolaum
Your example demonstrates a political bias in a dataset that can lead to a
biased ML model. This is what some people mean when they say ML can be biased.

~~~
strken
If I slap my friend Tom once a day, train an ML model to detect which of my
friends is going to be slapped next, and then find out it always predicts Tom,
the model isn't biased against Tom: the model is correctly showing me my own
anti-Tom bias.

I don't get to stand there and blame the model when I'm the one doing the
slapping.

~~~
pelario
For us, technologists, yes, the distinction between "AI bias", and the bias in
the data is clear. The point however, is when it comes to the general public,
"AI" is the whole thing, and actually the public has absolutely no saying
(perhaps even no knowledge) about the data; nevertheless, technocrats will
argue that "data doesn't lie".

Edit: the auto correct had written "data doesn't like"

~~~
strken
It's not just biased data, though, it's an objective function optimising a
biased metric.

We've picked a metric, recidivism rate, that is believed to be inherently
biased because cops arrest a lot of protected minorities. The model has
correctly predicted that cops will arrest a lot of protected minorities. The
general public has then turned around and shot the messenger rather than hold
cops accountable for all that arresting they're doing.

------
knzhou
The article calls machine learning "conservative" because it only tells us
what is, and not what should be. I don't think that's a useful framing. It's
more accurate to say that, like all statistical techniques, machine learning
is _descriptive_ , not prescriptive. Not everything in the world has to be
prescriptive.

~~~
visarga
We generally want ML to imitate us (or human generated data), thus we teach it
to do so. But there are also generative ML (like GANs) and simulation based ML
(like AlphaGo) which can be more creative. There is nothing stoping us from
letting agents evolve in a complex environment and be creative. It's just not
commercially useful to do that yet. Doctorow writes like he doesn't understand
much math behind ML, yet has strong opinions on it.

Every time a random number is involved in the training process (stochastic
selection of mini batches, noise injection, epsilon-greedy action selection),
a form of exploration is taking place, finding novel paths of solving the
problem at hand. The mixing of noise with data is paradoxically useful, even
necessary in learning.

~~~
knzhou
Sure, but no matter what, you can't derive an "ought" from an "is". At the end
of the day we merely tell these algorithms what is the case. No matter what
goes on inside, they cannot output moral prescriptions.

~~~
logicchains
>Sure, but no matter what, you can't derive an "ought" from an "is".

The same applies to humans, no?

~~~
Iolaum
No it doesn't. When a humam gets wronged by what "is" they can likely feel or
imagine a better "ought".

~~~
jmmcd
They can feel or imagine an "ought", but the point of Hume is that they can't
argue for it.

~~~
goatlover
And yet humans do, all the time. It's similar to Hume's attack on causality.
You can't show that A caused B, but yet we all act like it's the case, when B
always follows A. Kant's critique comes next.

------
krick
Examples are good and on point, the conclusion is not. When he tries to frame
it all in some grand political/quasi-philosophical manner, it becomes outright
wrong and stupid, but I won't argue with that part, because it won't be useful
to anyone.

What I want to point out is that nothing he says should be attributed
specifically to "machine learning". Machine learning is a set of techniques to
make inferences from data automatically, but there is no implicit restriction
on what the inferences should be. So machine learning is not "conservative" —
almost all popular applications of it are. There is no inherent reason why an
ML-algorithm should suggest the most similar videos to the ones you watched
recently. The same way you can use (say) association learning to find most
common item sets, you can use it to find the least common item sets with the
given item, and recommend them instead. Or anything in between. But
application designers usually choose the less creative option to recommend
(understandably so) stuff similar to what you already got.

Sometimes it's ok: if the most popular thing to ask somebody to sit on
nowadays is "my face" it's only logical to advice that, I see nothing wrong
with this application. But many internet shops indeed could benefit from
considering what a user has already bought (from this very shop), because it
isn't likely he will want to buy a second similar, but different TV anytime
soon. Or, when recommending a movie, you could try to optimize for something
different than "the most popular stuff watched by people that watched a lot of
stuff you did watch" — which is a "safe" thing to recommend, but at the same
time not really interesting. Of course, finding another sensible approach is a
lot of work, but it doesn't mean there isn't one: maybe it could be "a movie
with unusually high score given by somebody, who also highly rated a number of
movies you rated higher than the average".

~~~
platz
The point is that ML today is based on pattern recognition and memorizing a
stationary distribution.

This stationary distribution is the source of the conservativeness and central
to algorithms that we call "machine learning". ML always tries to replicate
the distribution when it makes infererences, so it is fundamentally biaed
against changes that deviate from that distribution

The Future and the Past are structurally the same thing in these models! They
are "missing" but re-creatable links.

AI is the broader term, but in pop culture AI and ML are very much synonymous.

~~~
krick
Well... no, not really. It is kinda hard to discuss in general, because it
depends so much on the details: the application and the algorithms in
question. But there is nothing inherently conservative about ML algorithms.

I see why you assume that "stationary distribution is the source of the
conservativeness", so maybe I should clarify this moment. It is kind of true
in the most general sense: sure, when querying the stationary distribution we
can only ever ask how things _are_ in a timeless universe. How anything _new_
can be obtained this way? The problem is, that if we are this general, then
the word "conservativeness" loses any meaning, since _everything_ in the
[deterministic] Universe can be framed as a question of "how things are",
_everything_ is conservative, _nothing new_ can be obtained anywhere, ever.

And we don't even need to get this general for the word "conservativeness" to
lose practical sense. When you ask another human for an advice, all he ever
does is, in essence, pattern recognition and querying his internal database of
"how things generally are to the best of his knowledge". Yet you don't call
every human advice ever "conservative": only the kind of advice that
recommends the safest, most popular thing, thing that everybody likes, pretty
much ignoring the nuance of your personal taste. In fact, even then, you call
it "conservative" only if you can notice that, which means that the
recommendation isn't _new_ for you personally (and by that criteria alone most
humans would lose to a very simple, absolutely currently implementable music
recommendation algorithm, since they probably know much lesser number of
"artists similar to what you like" than Spotify knows: the only thing Spotify
has to do to win is not to recommend the most popular choice almost every
time).

One more thing. I could probably convey to you that assuming "ML =
conservativeness" is wrong much faster by invoking the example of
reinforcement learning, since it is sort of intuitive: there is the obvious
existence of "time" in that, you can imagine a "dialogue" where it adapts to
what user wants using his feedback, etc. It is easy to see how it could behave
"not conservatively". I intentionally avoided doing that, since it can lead to
the false impression that RL is somehow different and less conservative than
other algorithms. On the contrary, the point I'm trying to make is that
_every_ algorithm, even the purest form of memorizing the stationary
distribution (like Naïve Bayes) is not inherently conservative. It all depends
on what exactly you ask (i.e. how you represent inputs and outputs) and how
you query the distribution you have (e.g. how much variability you allow, but
not only that).

So, when you see the application that uses ML algorithm and behaves
"conservatively", it isn't because of the ML algorithm, it is because of the
application: it asks the ML algorithm wrong questions.

------
ramraj07
While Cory is talking about a slightly different facet of conservatism, I
found it quite ironic that he made a rather conservative statement himself:

>Nor is machine learning likely to produce a reliable method of inferring
intention: it’s a bedrock of anthropology that intention is unknowable without
dialogue. As Cliff Geertz points out in his seminal 1973 essay, “Thick
Description,” you cannot distinguish a “wink” (which means something) from a
“twitch” (a meaningless reflex) without asking the person you’re observing
which one it was.

Was this mathematically proven? It's definitely an interesting statement,
since a lot of "AI" systems try to predict intention and do a piss poor job of
it, but to quote that the anthropological ancestors have proclaimed for
eternity that a computer can never know even the slightest fraction of
intention from just observation seems hypocritical.

~~~
jmmcd
This was one of the stupidest parts of a fairly weak essay. Of course you can
distinguish a wink from a twitch without asking, otherwise we wouldn't use
winks for surreptitious communication.

~~~
robotbikes
As an anthropologist you are in theory studying humans from out of a shared
cultural context and it is easy to argue that a machine which hasn't even the
slightest semblance of a shared cultural context would not be capable of
distinguishing a twitch from a wink with 100% reliability. Not all human
gestures are universal either so even if you trained your AI to read faces it
would likely be trained on a small subset of human cultures and would produce
varying results. In regards to the greater question of knowing intention I'd
agree that algorithms do a much poorer job of guessing my intention than me
directly communicating it and I am annoyed that so much of our on-line
communication is filtered by algorithmic social engagement that an encourages
shallow interactions.

~~~
madsbuch
This sounds the the classical "intelligence being something people have".

It is futile to talk about 100% correctness in this field

> ... machine which hasn't even the slightest semblance of a shared cultural
> context ...

Why is this universally true?

> ... it would likely be trained on a small subset of human cultures and would
> produce varying results.

As it has when you travel to foreign countries.

There is not such hing as understanding artificial intelligence. There is a
task of understanding _intelligence_ and attempting to implement it without
giving birth.

------
daenz
He specifically used the example of predictive policing. I have a question: is
it racist to send police patrolling areas of high crime, if those areas have a
majority ethnic demographic? Should that information be discarded, and all
places patrolled equally? Should they be patrolled less? It seems like there
is no winning. You're either wasting resources, ignoring a problem, or being
perceived as racist.

~~~
wsxcde
The problems of appropriate policing in so-called "high crime" neighborhoods
are well understood. Many academic studies, as well as popular non-fiction
like Jill Leovy's Ghettoside and Chris Hayes' Colony in a Nation, have
discussed the issue. To sum up the literature in a few sentences, the problem
is that minority neighborhoods are both overpoliced and underpoliced. There
are a lot of useless arrests for minor crimes like jaywalking which makes the
residents of these neighborhoods hostile to the police. (These arrests are
driven by the debunked theory of broken window policing.) Simultaneously,
there's not enough effort put into solving serious crimes like murder. In this
context, the actual effect of predictive policing is that it ends up doing
more of the useless over-policing, which unfortunately makes these
neighborhoods even worse.

So, how does this relate to your question? The point is that predictive
policing is solving the wrong problem. What's needed are not more accurate
neural nets predicting crime, but techniques for addressing the underly
sociological factors that cause crime.

Taking a step back and speaking broadly, Cory's point is that the focus on
data and quantitative analysis is causing problems in two ways: (i) people are
using quantitative methods to solve the wrong problems, and (ii) they seem to
be oblivious to (and in some cases actively hostile to acknowledging) the
harms being perpetrated by their methods. Both of these problems seem to be
driven by a lack of understanding of well understood (but non-quantitative)
social science literature.

~~~
quotemstr
Broken windows policing is not "debunked". A lot of people would prefer for
ideological reasons that it not work, but it does. Anyone who lived in NYC can
tell you that.

~~~
wsxcde
Speaking of NYC specifically, a lot of cities across the US, as well as
worldwide, experienced a major decrease in crime at the same time that NYC
did. Most of the cities were not practicing broken window policing. That's one
reason to be skeptical.

From an academic perspective, it is true that there is some debate about the
efficacy of broken windows policing, but even the most supportive academic
studies find only a small correlation between violations of "order" (like
jaywalking and graffiti) more serious crimes. There's just isn't any evidence
at all that the way to reduce serious crimes is by going after jaywalkers.

------
xpe
Sure, many of the examples ring true.

However, I would refine the claim "Machine learning is fundamentally
conservative" to say "Reducing a distribution of predictions to only the most
likely prediction is conservative."

~~~
jrumbut
Also similarity is conservative. There are alternatives.

For a product/content recommendation you could include the product that's
maximally different, the second most similar, you could exclude products
within the same category, you could choose randomly and weight by conversion
rate. We need more creativity and experimentation here I believe.

~~~
Pamar
* you could include the product that's maximally different, the second most similar, you could exclude products within the same category, you could choose randomly and weight by conversion rate*

This is interesting. Has anyone tried that or knows of a website that uses
this?

~~~
jrumbut
I have personally implemented discarding the top-n (small n, like 3-5) most
similar results in a recommendation system as well as adding a totally random
item in the results but it was a small scale system.

The top-n removal was a small but noticeable amount better. That was in part
due to the specific content in the database as well as the audience, but it's
a thing to try I believe.

------
kitsuac
It isn't /fundamentally/ conservative, it is just typically programmed to
choose the most conservative (highest probability) predictions. You could
integrate a liberal aspect by fuzzing the decision process to choose from
lower probability predictions.

More creativity, and ability to escape local minima, but at some cost when
dealing with 'typical' cases and when making particularly damaging
mispredictions.

~~~
l0b0
I think the point is rather that you _can 't_ get a more useful prediction by
choosing a lower probability description _unless_ you have AGI. Only an AGI
could tell that you're not in the mood for "Hey" to be followed by "darling",
and only a superhuman AGI could realistically compensate for human bias in
data sets.

~~~
kitsuac
Without AGI there are still cases when the lower probability prediction will
be better, and will lead to escaping a local minima. I'd argue that the
potential benefits of calibrating that axis dynamically exist with or without
AGI.

~~~
ben_w
Are you describing the explore/exploit tradeoff or simulated annealing in this
case?

------
hprotagonist
_Mom: If all your friends jumped off a bridge, would you, too?

ML Algorithm: yes!!_

~~~
TulliusCicero
If all my friends are jumping off a bridge, it's probably bungee jumping.

I mean they could have all simultaneously gone insane, or be infested by alien
mind parasites, but going bungee jumping is the much more likely reason.

------
hacknat
This is a problem in almost every academic field right now. Peoples’ lack of
sophistication with understanding the mathematical/philosophical constraints
of their tools is incredibly scary.

For example, people throw around the low labor productivity stat all the time
to prove that no automation is happening, not realizing that GDP is the
numerator of that stat. Well, GDP is a pretty terrible gauge of who is
benefitting from technological innovation, as it is distributed incredibly
unevenly, probably more so than income even. The problem with automation isn’t
that it’s not happening, it’s that only a very small number of people are
capturing the wealth that it is generating. Also, it is generating a small
number of professional jobs, but mostly the jobs it is generating suck (Uber
driver, AMZN warehouse worker, etc).

~~~
alexmingoia
Likewise, looking only at nominal wage increases is a “pretty terrible” way of
gauging who benefits from increased wealth. If improvement in productivity
(automation) results in lower prices (or better or more goods for the same
price), that increases the purchasing power of a worker’s earnings, even if
their wage has not risen.

In other words, a worker is richer if they can buy more and better stuff even
without any rise in their wage.

The fact that everyone can afford a smartphone, including those with jobs that
“suck”, does not reconcile with “only a very small number of people are
capturing the wealth that it is generating.“

~~~
0xfaded
This argument forgets that where people spend most of their money, housing,
education, healthcare, costs are rapidly raising in real terms and have not
followed Moore's law in usefulness.

Yes technology has made cool gadgets for consumption, but that isn't what
wealth is about. From Wikipedia: "Wealth is the abundance of valuable
financial assets or physical possessions which can be converted into a form
that can be used for transactions."

The average worker hasn't seen a real pay ride in 40 years, and their ability
to build wealth through savings has been greatly diminished.

~~~
makomk
So basically, improvements in productivity are only benefitting people in
areas where productivity has improved? That seems more like a tautology to me
than something which can be fixed by gnashing teeth about the distribution of
wealth

~~~
xg15
How did you get from

> _The average worker hasn 't seen a real pay [rise] in 40 years_

to

> _So basically, improvements in productivity are only benefitting people in
> areas where productivity has improved?_

?

Worker productivity has increased enormously in the last decades.

~~~
makomk
I didn't? Obviously, I got that from the "housing, education, healthcare,
costs are rapidly raising in real terms and have not followed Moore's law in
usefulness" part...

------
rsync
I am open-minded to the idea that "Machine Learning" is conservative in the
manner Doctorow describes.

However, I do wish we would not use the word "conservative" as an epithet. I
think it's quite likely that "conservative" is exactly what we should be
looking for in algorithms and prediction engines.

The fact that their properties are misused and infused with _reactionary_ (not
necessarily conservative) biases by humans should not make us attach morally
negative properties to being conservative.

FWIW, my conservatism leads me to be suspicious of employing these tools in
the first place ...

------
azinman2
When you have a dumb machine that simply solves for maximum likelihood, using
past data to predict the future is what you’re going to get. Why is this
surprising?

I don’t understand what he’s arguing for, exactly. That we should have AGI all
of the sudden to understand intent, and detangle causation vs correlation? I
don’t think anyone in the machine learning community would argue against that,
but the question is how.

What’s new here?

~~~
onion2k
I don't think Cory is writing for the machine learning community.

Cory is trying to inform people that AI and ML _aren 't_ a magical solution to
all our wasteful systems because in some cases we need to throw away the
system and start again, and AI can't tell us that. It can only tell us the
best way to run the current system. That's what he means when he says AI is
conservative.

It's not surprising at all if you understand AI but most non-tech people
don't.

~~~
azinman2
It has the same flavor generally of people who criticize something that’s hard
but offer no solutions — a style that has become increasingly popular IMHO.
This also isn’t a new angle for the non-informed audiences. It bears repeating
for those audiences, but I’d wish he’d at least point to others who have
already echoed such concerns.

~~~
licyeus
Paragraph two cites the inspiration for this piece, a 2017 article by Molly
Sauter: [https://reallifemag.com/instant-
recall/](https://reallifemag.com/instant-recall/)

And he points to three other sources right in the first paragraph.

------
sgt101
There are systems of algorithmic inference that are not conservative.
Constraint satisfaction and logical deduction can create novel insights.

~~~
gyulai
...also, as soon as you bring a network element or social element into it, it
may no longer be this conservative/self-reinforcing thing. For example, if the
algo behind spotify were to identify a "music taste clone" of yours somewhere
in the world, they could present you with music you've never heard about that
your clone has liked, and vice-versa. So you actually start discovering new
stuff that you end up liking.

Furthermore, there is a psychological element at play around mirroring back
your own intelligence at you (see Eliza / Rogerian Psychotherapy) in a way
that will lead you to new thought.

------
AndrewKemendo
_Nor is machine learning likely to produce a reliable method of inferring
intention: it’s a bedrock of anthropology that intention is unknowable without
dialogue. As Cliff Geertz points out in his seminal 1973 essay, “Thick
Description,” you cannot distinguish a “wink” (which means something) from a
“twitch” (a meaningless reflex) without asking the person you’re observing
which one it was._

I see this and other examples of "explainability" from time to time as proof
why humans are not a "Black Box."

However it rests on two faulty assumptions.

1\. The explanation will be truthful

2\. The explainer can always reliably describe the actual cause of their own
actions

For the purposes of theory, you could explain away #1. However a minute of
introspection will make you realize that you would fail at #2 the vast
majority of the time - using story telling and retroactive explanations to
explain your behavior.

------
anjc
> Search for a refrigerator or a pair of shoes and they will follow you around
> the web as machine learning systems “re-target” you while you move from
> place to place, even after you’ve bought the fridge or the shoes

> ...

> This is what makes machine learning so toxic.

I'm saddened to learn about our toxic contributions to society and can't wait
to hear about alternative mind-reading approaches for fridge recommendation in
the next article.

Words like 'conservative' and 'toxic' are misleading because they imply that
there are better alternatives that are not being chosen. Far better are the
terms by commenters here, 'descriptive' and so on. That the article is not
written for machine learning practitioners makes it even more misleading.

------
at_a_remove
I am not a fan of this article. I have seen many critiques in this vein, this
is just another car in the train. None of them have quite reached the point of
confronting what is bothering many: What will we do when machine learning (or
science, or anything, really) comes to a conclusion we find unpalatable, for
whatever reason?

It could be any conclusion, not just those conservatives dislike. Using myself
as a target, what if we eventually have enough sampling data to show that
people of Irish extraction _are_ more prone to alcoholism, and people of
Scottish extraction _tend_ to be statistically more thrifty? (This suggests
that I would be a cheap drunk). How will we cope with that?

It is true, ML is prone to some black-boxiness, but it could be any
statistical extraction. We might very well use other methods besides ML to
show the correlates once suggested.

Will we simply put in a hard override to get the answers we want to get, the
answers we find comfortable? History shows we have seen whole governments
subscribe to this idea before. Ignore the results, publish what pleases.

I've no easy answers here, but my guess is that history will repeat itself.

~~~
moduspol
> What will we do when machine learning (or science, or anything, really)
> comes to a conclusion we find unpalatable, for whatever reason?

What makes you think it hasn't already happened? Numerous times?

Biological differences between sexes? IQ differences between differentiable
subsets of people? The gender equality paradox?

We already know what will happen when findings are unpalatable. We'll think of
ways to explain why they don't matter and can't possibly be right, or that we
weren't measuring the right thing in the first place. And these are just the
more obvious cases!

~~~
bsanr2
>or that we weren't measuring the right thing in the first place.

Which has been true, actually. For example, human facial recognition systems
that can't properly distinguish dark-skinned faces. What is your conclusion in
those cases? That dark-skinned people don't have faces? All look the same?
Aren't human? Or is it more likely that the system is flawed in some way? Is
that flaw in line with existing biases? What then is the chance that the flaw
is BECAUSE of our existing biases, in some way or fashion?

From what I've read on HN, many people involved in developing ML technology
seem to be overly concerned with their systems spitting out "uncomfortable
truths", and less concerned with flaws in their data capture, training, or
system design processes. We're putting the cart before the horse, and the
giddiness with which it's being done is troubling in and of itself.

~~~
moduspol
Sure--nobody's arguing that we're always measuring the right thing.

It may be more relevant in developing ML technology because readers here are
the people who have to (at some point) decide whether they've found an
uncomfortable truth or falsehood. Or that it's potentially biased, but may or
may not be relevant. If I'm doing this at work, I may need to be able to
explain to my boss whether or not what we're doing is going to be called
racist or sexist, and that's regardless of whether what's being done actually
is racist or sexist.

It's easy if you're using ML to detect whether an image contains a bird or
train. And it's easy when you point to an obvious example with bad data (e.g.
inability to differentiate dark faces). But that's not all cases. Sometimes
it's good data and the findings are unpalatable. That was the root of the
question.

~~~
bsanr2
We say "unpalatable," but what we really mean is "politically incorrect."
There are segments of our society which would be overjoyed to have such
"concrete" evidence of their terrible beliefs. Therefore, it is important to
observe OP's thesis: that ML "findings" are conservative; that they tend to
reproduce preexisting biases; and that we should be interrogating the input
data, systems, and output of ML-applied tasks that return "controversial"
findings.

I agree that it's important to be aware of the potential for these sorts of
findings, but I continue to disagree that our takeaway should be an impetus to
gird ourselves for the backlash when The Machine spits out, "Black people are
dumb," rather than an impetus to observe the bedrock notion of scientific
inquiry (you know, skepticism).

>It's easy if you're using ML to detect whether an image contains a bird or
train. And it's easy when you point to an obvious example with bad data (e.g.
inability to differentiate dark faces). But that's not all cases. Sometimes
it's good data and the findings are unpalatable. That was the root of the
question.

I think the "Newspaper Error Rule" (or, I guess, the "Cockroach Rule") applies
here: if you recognize one when you're looking, how many are you missing when
you're not, particularly when you're in less familiar territory? The _less_
obvious examples are the ones we should be _more_ wary of. And if your boss is
expecting a quick, cut-and-dry answer to questions that have vexed society for
centuries, just because we've applied a sophisticated (but still limited)
statistical model to it... I'm just saying, maybe you don't want to go down in
history as the 21st century's version of a phrenologist.

~~~
moduspol
I think we mostly agree on premise but are disagreeing on emphasis.

> I agree that it's important to be aware of the potential for these sorts of
> findings, but I continue to disagree that our takeaway should be an impetus
> to gird ourselves for the backlash when The Machine spits out, "Black people
> are dumb," rather than an impetus to observe the bedrock notion of
> scientific inquiry (you know, skepticism).

There is no shortage of pressure to avoid ML conclusions like "black people
are dumb." We are already girded there. This isn't the first HN article about
recognizing ML biases, and it's not like we're wading through headlines about
how racial stereotypes are justified due to some all-knowing ML algorithm. We
know we should be interrogating biases.

That said, it will always be possible to dismiss any evidence-based science by
claiming there are unfalsifiable flaws with data or methods. That was the
point of the comment I responded to. Outside the context of ML, this already
happens, so we can already predict how it will happen with ML. And it is.

Nobody's saying, "just trust the machine if it says black people are dumb."
I'm saying, "if the machine says sickle-cell anemia is most common among those
with sub-Saharan African ancestry, it's still worth checking your methods and
data, but it shouldn't be ignored just because unfalsifiable reasons why the
analysis could be wrong can be postulated."

~~~
bsanr2
The tendency to say, "Don't worry, we've got it covered," is precisely where
my worry stems from. It is better than not having any awareness of the
potential problems, to be sure, but only so much so.

Your last analogy is an example of this. The link between sickle cell anemia
and ancestry is a causal link that could be shown conclusively with a genetic
test for a SNP. Finding that gene may have been an involved effort undertaken
with a great many resources, but that link was established with near-perfect
data and a clear conclusion.

The most promising are applications for ML are situations on the exact
opposite end of the spectrum: problems where data is imperfect, noisy, or
incomplete, where conclusions are not certain, but simply likely, to some
measure of confidence.

We want to use ML to come to answers approaching conclusive, when a solid
conclusion is impossible or unlikely, or would too long to obtain by
conventional means. And we won't know how that conclusion is reached. So, in
terms of conclusions that could spark wars, genocide, untold suffering, maybe
we need to be more serious about our vigilance in regards to what we can
control.

To put it bluntly, these concerns people have seem akin to NASA being more
worried about handling post-catastrophe PR and internal messaging than keeping
the shuttle from exploding in the first place.

~~~
moduspol
Your perspective doesn't lead to the discovery of the true causal link for
sickle cell anemia, though.

It's what leads to us pretending there can't be any unpalatable differences
between ethnic groups, clutching to presumptions about maybe the health care
system being too racist, and avoiding looking deeper into it because we think
we've already decided what the answer is (racism). Meanwhile real people are
suffering because they aren't getting treated as effectively as they could.

We don't know what we don't know, so it's impossible to tell how many people
are suffering or for what reasons.

~~~
bsanr2
We know what racism begets. We've seen the way that it has been used to warp
inquiry, ethical guidelines, and clinical outcomes, often unintentionally and
through processes that diffuse blame to below the level of individual bigotry.
It would be nice if we lived in a world without a track record of pursuing the
ramifications of assumed "unpalatable differences," such that what you're
suggesting would be sound, but we don't. We, at this moment, are trying to
climb out of a hole of ignorance - in medicine, in social science, in
economics, in much of the quantifiable and model-able world - dug by
presumptions of "unpalatable differences."

So, I say again, the the priorities, as you've described them, are out of
wack. We are much more at risk of jumping to damaging conclusions than we are
of missing helpful breakthroughs. Our vigilance should be tuned in regard to
this.

------
YeGoblynQueenne
>> Empiricism-washing is the top ideological dirty trick of technocrats
everywhere: they assert that the data “doesn’t lie,” and thus all policy
prescriptions based on data can be divorced from “politics” and relegated to
the realm of “evidence.”

Well data _doesn 't_ lie. Because data doesn't _say_ anything. People
interpret data and they do so according to their own inherent biases. And if
the data is already biased (i.e. gathered according to peoples' biases) its
interpretations end up far, far away from any objective truth.

------
naveen99
Alpha zero is not conservative at chess or go. It doesn’t have to have seen a
position before to evaluate it.

You can always train a model to reject a class just as easily as you train it
to accept a class. So train it to reject a common class and accept a mutant
and it will function more like an evolutionary algorithm that protects against
bad luck like bacteria with antibiotics.

I am way more optimistic for AI I guess.

------
ALittleLight
The author writes:

>Machine learning systems are good at identifying cars that are similar to the
cars they already know about. They’re also good at identifying faces that are
similar to the faces they know about, which is why faces that are white and
male are more reliably recognized by these systems — the systems are trained
by the people who made them and the people in their circles.

But this seems like an absurd claim. Surely it's more likely that darker skin
reflects less light than paler skin and so features are harder to detect on
darker skin people. This explains why facial recognition works better on white
people than on black people. Likewise women are more likely to wear makeup
than men are and therefore are more likely to have a visually different face
which could cause problems for facial recognition.

To assume that dark skinned people aren't involved in creating or testing
facial recognition, or that women aren't, seems bizarre and vaguely
racist/sexist. It's also a strange and I think poor assumption to think that
the different accuracy rates for facial recognition on different demographics
are due to a bunch of white dudes being too dumb to train their models on the
faces of anyone but themselves.

The author also writes about how recommendation systems are "conservative" but
to me it actually seems the opposite. Recommendation systems get you to stuff
that you haven't tried before. They take what they know about you and progress
you to new content or products.

For example, I used to listen to listen to a lot of country music on YouTube.
Over time my recommendations evolved to include Irish folk music - which I
quite like but would've never discovered without YouTube gradually testing and
expanding my musical tastes.

~~~
onion2k
_To assume that dark skinned people aren 't involved in creating or testing
facial recognition, or that women aren't, seems bizarre and vaguely
racist/sexist._

It's unlikely anyone making a facial recognition system would choose to
release a version that didn't work, _at the very least_ , for everyone on the
team.

If the team is only white men then you can see why they might accidently
release something that only works on white men's faces. If there was a woman
or a PoC on the team they'd have someone saying "Hang on, it doesn't work on
_me_!"

The important thing to learn here is that any system is only ever as good as
the data used to test it. If your test data sucks then the system you build is
also going to suck.

~~~
ALittleLight
This might be a valid theory if facial recognition was a single niche product
made by unsophisticated or under resourced developers. That's incredibly far
from reality though. There are many well funded groups of serious
professionals and academics working on facial recognition with data sets of
millions of faces.

It's just objectively not correct and not reasonable to think that the reason
facial recognition works better on some demographics than others is because
nobody thought of training models on those demographics. If you believe that
is the case, then do you think people working on facial recognition reading
this blog post are slapping themselves on the forehead saying "Of course.
Train the models on someone besides the dev team!"?

~~~
onion2k
I was talking about testing the application rather than training the model.
You can train a ML model on a diverse range of faces, but if, for example, an
application that uses the model doesn't calibrate the camera properly it still
won't work for darker faces.

~~~
ALittleLight
I don't understand.

The different accuracy of facial recognition by demographic group is common
across multiple facial recognition systems. It's highly unlikely that all of
the affected systems owe their inaccuracy to a post-training configuration
issue.

To me it seems more like you are arguing for the sake of arguing rather than
that you actually believe in your point. You are just trying to conceive of
imaginative possibilities that preserve the "dumb facial recognition
developers forgot non-white people exist" interpretation. The problem is that
your imagined possible alternatives are not plausible explanations.

~~~
onion2k
I gave a single example of why an app might fail at facial recognition in some
cases. _Obviously_ I wasn't suggesting that would be the case for every app
that doesn't work. Suggesting I was is a strawman of ridiculous proportions.

I also didn't say anything about ML training data in the post you replied to.
My point was about facial recognition app developers not testing with a
diverse range of inputs, which is why we see apps failing in public when
people try to use them with inputs the app hasn't been tested against. If the
apps were well tested those cases would have been caught before the public
ever saw them.

My original point stands - if your testing sucks then it's very likely that
your app will suck too. There will be problems with it that you don't know
about, and the users will find them very quickly.

~~~
thu2111
Do you have any data showing facial recognition doesn't work well on black
people? My understanding is that this is an urban legend - it's not true, the
algorithms work fine assuming a decent quality camera and video (same for
everyone).

------
macawfish
Here's the Molly Sauter essay that this article takes core inspiration from:
[https://reallifemag.com/instant-recall/](https://reallifemag.com/instant-
recall/)

------
buboard
the opposite would be also described as terrifying: machine learning making up
the future of humanity. I m not sure if the author's point is particularly
important. ML learns from the past, it doesn't have enough personality or
'intent' to be a 'conservative'. On average it will keep doing what people
did. Also, not all ML systems do, a RL system may be trained with another
objective, e.g. radicalism.

------
lowdose
Conservative or is not yet full context aware?

------
fastball
Can we stop trying to place complex topics and people and positions onto a
one-dimensional scale?

------
solipsism
Why is the HN title so different from the article's title?

~~~
grzm
(a) The submitter changed the title when submitting.

(b) A mod changed the title after it was submitted.

(c) The site changed the article title after it was submitted.

I suspect (b), given the baity-ness of the article title: "Our Neophobic,
Conservative AI Overlords Want Everything to Stay the Same", in accordance
with the guidelines:

> _" please use the original title, unless it is misleading or linkbait; don't
> editorialize."_

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

If you think a title has been changed inappropriately (or should be changed),
you can let the mods know directly using the Contact link in the footer.

~~~
lidHanteyk
Let the mods know publically, by posting comments. Don't trust the mods to be
genuine in private conversations.

~~~
grzm
The mods don’t see every comment, so if the goal is to let the mods know,
email is the most reliable way. Of course it’s up to them how to act on it.
Just commenting isn’t going to improve anything, as they legitimately might
not see it. It just increases the likelihood of inaction.

------
naveen99
i was hoping for a roast of AI similar to when he beat the semantic web into a
pulp last time... now I wonder if the semantic web messed him up too much in
revenge.

------
api
This headline alone seems like a shark jumping moment.

------
zozbot234
And I for one welcome our neophobic, conservative AI overlords! /s

------
jevgeni
Doctorow is a smart guy, which is why the weasel words he uses seems
disingenuous. For example "machine learning is fundamentally conservative, and
it hates change".

Mate, it doesn't hate anything. It's math.

~~~
yipbub
He wasn't being literal...

~~~
jevgeni
So? He is still attaching emotional value to a technology. You could say the
same thing with "machine learning doesn't handle change robustly". Same
meaning, different emotion.

~~~
kick
Math isn't technology, and like you said yourself, AI is just math. Attaching
emotional value to math is done _incredibly_ frequently for all sorts of
reasons.

Attaching emotional value to technology is also fine. "Machine learning is
fundamentally conservative" gets the point across better than "machine
learning doesn't handle change robustly," which is vague and doesn't give a
person looking for something to read any reason _why_ that would matter.

~~~
jevgeni
> Attaching emotional value to math is done incredibly frequently for all
> sorts of reasons.

Writing alarmist articles for clicks being one of those reasons.

> Attaching emotional value to technology is also fine.

Why??? Doing this with other things would be ridiculous. For example, instead
of "The poor are disproportionately affected by climate change" a journalist
could write "The climate hates the poor". Which subtly plays down the human
cause in all of this.

> "Machine learning is fundamentally conservative" gets the point across
> better

It demonstrably doesn't. If you just read this thread, you'll find enough
people discussing whether "conservative" has a political meaning or just a
"deviation from the norm" meaning.

