
Machines Taught by Photos Learn a Sexist View of Women - nsgi
https://www.wired.com/story/machines-taught-by-photos-learn-a-sexist-view-of-women
======
Eridrus
ML, even when perfect, learns the world as it is, not as people want it to be.
There's nothing inherently sexist about a machine noticing that the background
of an image can be used to identify a class label, e.g. noticing that a mug is
more likely to contain coffee than tea in a coffee shop, but given how shitty
we have been to each other, some correlations are considered taboo. The only
way for machines to learn our taboos is to explicitly teach them.

~~~
rixed
Have you read the article?

The problem is not that ML reproduces the bias that is present in the training
set, it is that it amplifies it (a lot).

~~~
Eridrus
The article is garbage, but anyway.

If you see that an algorithm has issues with imbalanced classes, you could try
to address that, or you could frame your problem as sexism, write a paper with
_that_ in the headline and then call Wired.

And in any case, if you read the paper, they examine a single algorithm
(Conditional Random Fields) on two datasets (which Wired extrapolates to the
entire field), and their own solution is to add constraints saying that it
should preserve the ratio of woman:man cooking as in the original dataset. And
while there is no loss of accuracy, it also has no improvement, so it's just
shifting the errors around. And it has absolutely no analysis of why a CRF
would exhibit these properties anyway.

But this is kind of my point, even after you solve these issues that arise
from class imbalance (which ML practitioners & researchers are highly
motivated to solve these already because they lead to better average
performance), you are still left with a bias that is taboo that society will
say must be fixed, which cannot be fixed by simply more accurate ML or more
accurate data.

------
angmarsbane
This is something I know very little about, but will machine learning evolve
over time?

I mean imagine if this technology was being created before the civil war and
it was fed images of slavery the assumptions it would make!

Our world and what is normalized changes over time, will machine learning
reflect that?

The world I've experienced is different than the world a generation below me
has experienced but if it's my generation feeding the machine the images what
happens?

Images from my parents' generation would have shown women who didn't go to
college unless it was for an MRS., nursing, or teaching. My Mom's only sport
option was cheer leading. This is drastically different from my sister's life
experience.

------
erroneousfunk
There are really a few problems here that I think the article is getting a
little mixed together, and I wanted to lay them out.

First, that the image corpuses used for machine learning have a strong gender
bias, perhaps more than exists in the "real world." More images of men than
women, more images of men working on computers, more images of women in
kitchens, etc.

These images are sourced from [http://imsitu.org/](http://imsitu.org/) which
is sourced from [http://image-net.org/](http://image-net.org/), which (after
some digging) looks like it gets most of its images from Flickr, stock
photography sites, and random corporate sites. Are these representations of
the "real world"? I would argue not. Professional photography, stock
photography, and photos taken for the purpose of being used in an unknown
future context, and/or to appeal to the most people, tends to err on the side
of being "universally applicable" and emphasizing the "common idea of a thing"
rather than how the thing actually is, with all its variations. An image of a
man in a kitchen be perceived as more controversial and may be less
universally usable than an image of a woman in a kitchen. So if you want to
take a photo with as many possible uses as possible, you'd tend to fall back
on established social norms MORE often than they might actually occur.

Second, machine learning tends to emphasize small differences when it has
nothing else to go on, or is improperly trained. If you have a dataset
featuring people in kitchens where 75% of the time the person in a photo is a
woman, you could get an algorithm that is 75% accurate simply by saying "the
person in the photo is a woman" every single time. While the dataset reflects
75% women, the algorithm reflects 100% women. It emphasizes small differences
in order to gain accuracy.

This isn't just hypothetical. Many times, I've worked on a
categorization/labeling dataset that turns out to have no _actual_ underlying
pattern, but I wind up, after many hours, getting a best fit algorithm that,
say, predicts the dataset correctly 85.166667% of the time... only to realize
that my dataset is spectacularly unbalanced and exactly (EXACTLY) 85.166667%
of the dataset is in a single category. It's amazing how it just sort of snaps
to that when you start layering the machine learning algorithms and you
realize that the real problem is that there's no real pattern in the data
(something data scientists don't often like to admit).

Third, sometimes the algorithms just get it wrong in ways that seem minor and
rare from a data science perspective, but have large social consequences. Like
improperly labeling a black couple as gorillas. It might actually be the case
that the algorithm was improperly trained because it lacked photos of black
people and photos of gorillas and didn't have much to go on (an example of the
first issue) but I don't know enough about the situation to say for sure.

And fourth, of course, is that these patterns DO exist to some degree in the
"real world," and this is a point that's been hammered on over and over again
on Hacker News. The problem is that machine learning is a sort of big leveler
that finds these patterns wherever it can and applies them universally (and
often while emphasizing the differences for the reasons stated above). I mean,
that's the point of it, after all! But knowing this fact, I think it makes
sense to be careful where and how it's used.

