
Autonomous Cars Can't Recognise Pedestrians with Darker Skin Tones - kaboro
https://interestingengineering.com/autonomous-cars-cant-recognise-pedestrians-with-darker-skin-tones
======
czr
The headline here is not a good description of the actual paper
([https://arxiv.org/pdf/1902.11097.pdf](https://arxiv.org/pdf/1902.11097.pdf)).

The difference in average precision in bounding boxes between the LS (light
skin) and DS (dark skin) observed by the researchers was around 1-10%. Not
"can't recognize". Notably, people with dark skin were underrepresented in the
dataset that the researchers used.

When they compensated for the dataset imbalance by reweighting examples, the
issue was resolved.

Notably, the research itself is on publicly-available object detection
systems, not any actual autonomous vehicles. Any company seriously building an
AV 1) has their own diverse datasets collected from real-world driving and 2)
has almost certainly already handled this issue (skin color is a blessedly
minor source of variation in pedestrian detection, compared to the tremendous
headaches caused by clothing, costumes, carrying other objects, all of which
need to be accounted for...).

The paper is good a reminder of the importance of fine-tuning any pedestrian
detection system to fix imbalances in your dataset, and avoid false negatives.
But it's not what the title claims.

------
krapp
OK.

Can we suspend identity politics and admit that systemic homogeneity bias in
software development is as much a technical as social and political problem?

I mean, why is it _ok_ to release facial recognition software that has only
been trained on white faces? Why is that considered a minimum viable product?

When we get to the point that facial-recognition systems don't recognize black
people, autonomous cars won't even stop for them and law enforcement risk
assessment AIs automatically tag them with a x2 modifier for criminality, this
stuff stops looking like naive techie ignorance and starts looking like
malice.

And, all else aside, it's _bad design._

~~~
toufiqbarhamov
Agreed, although I’d add to: _When we get to the point that facial-recognition
systems don 't recognize black people, autonomous cars won't even stop for
them and law enforcement risk assessment AIs automatically tag them with a x2
modifier for criminality, this stuff stops looking like naive techie ignorance
and starts looking like malice._

I strongly suspect that it looked like malice long before this point if you
were for example, black yourself. In the same way that people familiar with
the history of TLA’s and tech were concerned about the behavior of the NSA
long before Snowden confirmed our worst fears, I’m sure that black people saw
this coming based on warning signs typically dismissed by people who can
afford to do so.

------
tropo
Lighting matters, and thus Daylight Saving Time matters. Could it be racist to
make one choice or another for Daylight Saving Time?

Background matters. Should be use concrete roads or asphalt roads? Could that
choice be racist?

Maybe we shouldn't go looking for racism _everywhere_ and always assuming it
exists.

