
Lyft’s algorithm is trying to block people with names like ‘Dick’ and ‘Cummings’ - danso
https://www.theverge.com/2019/12/19/21030601/lyft-algorithm-block-names-inappropriate-obscene
======
danso
Besides the humor of how Lyft's filter flagged traditionally Caucasian names
like "Cummings" – _in addition_ to the usual issues with non-Western names,
e.g. Pimpong and Poon – I'm fascinated/confused how this made it into
production? The user database _already_ exists and was currently being used by
the live application. Before deploying this new filter onto the production
database, wouldn't you do a dry run to get not _only_ the count of users who
will be flagged and notified, but a listing of frequently flagged names? Which
you could easily manually eyeball to make sure there weren't obvious false
positives?

With a userbase as big as Lyft's, I'm sure there were a ton of obvious true
positives (anyone named "Fuck", ostensibly). I just can't believe they didn't
notice a surname as relatively common as Cumming/Cummings. Yes, the apparent
naivety of the regex is an issue, but this seems like a system for which a lot
of testing on actual data would be easy and natural to do as part of the QA
process.

~~~
spacechild1
> traditionally Caucasian names like "Cummings"

It always amazes me when US Americans talk in terms of human races like it's
the most normal thing in the world. And no, “Cummings“ is not a “Caucasian
name“ (whatever that is supposed to mean), it's an English name. I don't want
sound too harsh, it's just that as a Central European with all our recent
history, things like this really rub me the wrong way.

~~~
coldcode
Caucasian is a funny word to apply to Europeans since its a part of the area
around present day Georgia and is a highly diverse ethnic mix of people which
most people would classify as Central Asian (like Turks and Armenians for
instance). Not exactly what you would think would be "white" which is what
many Americans assume it means.

~~~
isquared23
It does mean white/of European origin in American English. Just like entree
means main dish. Both don’t correspond to the original meaning of the words.
You can find similar shifts in meaning in other languages too (e.g., German
calls a mobile/cell phone “Handy” which makes no sense to a native English
speaker).

I question the wisdom of telling native speakers that their language uses a
word wrong/that doesn’t mean what “they assume it means”. Stuff in AE (and
other language) means exactly what AE speakers (and speakers of other
languages) assume it means.

Cau·ca·sian /kôˈkāZHən/ NORTH AMERICAN white-skinned; of European origin.

~~~
kps
‘Handy’ comes from the WWII ‘handie talkie’, a transceiver small enough to
hold in the hand¹, which followed the ‘walkie talkie’, a transceiver small
enough to walk around with².

¹
[https://en.wikipedia.org/wiki/File:Portable_radio_SCR536.png](https://en.wikipedia.org/wiki/File:Portable_radio_SCR536.png)

²
[https://en.wikipedia.org/wiki/File:Scr300.png](https://en.wikipedia.org/wiki/File:Scr300.png)

~~~
Izkata
Meanwhile, at least in the modern US, "walkie talkie" is the name of the
handheld unit and "handie talkie" would get weird looks because it's not
really known.

~~~
detaro
Same in Germany when you're talking about radios.

------
minimaxir
It seems like tech companies have been doing DIY profanity filters. Apple has
a profanity filter for engraving text onto a device in the online store
([https://twitter.com/minimaxir/status/1213188371452841984](https://twitter.com/minimaxir/status/1213188371452841984)
); intended to be your name, so terms like "Dick" have to not be filtered.

I learnt that Apple has a validation endpoint. Since the engraving service was
recently updated, I took a profanity wordlist and checked it against the
endpoint just for fun. The results are...counterintuitive:
[https://pastebin.com/mzpECiQw](https://pastebin.com/mzpECiQw) (NSFW language,
obviously)

~~~
ben_w
The profanity filter in the Apple App Store was [0] amusingly wrong, but
fortunately also a soft filter rather than a hard filter.

What happened was, the German localisation of the app description included the
word “Knopf”. Knopf is not a rude word, according to any German I’ve discussed
this with — it is one translation of “knob” in the sense of “button”, but
Apple’s naughty word detector clearly thought it was “knob” in the sense of
the euphemism for a body part.

It didn’t stop the app passing review, but the automatic warning was still a
regular part of updates for that particular app.

[0] Back in 2012

~~~
dathinab
Honestly if filter try to filter out euphemisms all is lost because:

1\. People who use them for that just come up with new ones all the time

2\. People still use the word in the regular sense, like would I now have to
come up with a euphemisms to describe a door knob ?!

3\. How I wore is my decision. As long as I don't hurry anyone intentionally
or knowingly no person and even less company had the (moralic) right to
constraint me. (Through wrt. Minirs, Y least younger ones, their parents
opinion matters, too.)

------
kstenerud
This is what happens whenever someone decides that they have enough expertise
in something they don't.

We see the same phenomena with encryption, authentication, and password
security. People just tend to have a myopic understanding of the topic and
vastly overestimate their ability in something they've barely even studied,
which is the worst kind of ignorance. And they very much deserve to be named
and shamed when they inevitably screw it up.

When dealing with names & addresses, consult an expert.

~~~
Razengan
> _When dealing with names & addresses, consult an expert._

And phone numbers, and dates.

------
hprotagonist
[https://en.wikipedia.org/wiki/Scunthorpe_problem](https://en.wikipedia.org/wiki/Scunthorpe_problem)

the scunthorpe problem will never die.

~~~
shalmanese
You could almost say it's a clbuttic problem.

~~~
userbinator
At least they did not try to modify the names... imagine if every Dick got
renamed to Penis!

------
jamestimmins
Patio11 has a good post about this topic from 10 years ago.
[https://www.kalzumeus.com/2010/06/17/falsehoods-
programmers-...](https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-
believe-about-names/)

As a programmer, it's difficult how little of the world follows consistent
"rules" that you need to build straightforward conditional logic.

Really, the only surprising thing is that Lyft didn't cross this bridge years
ago.

------
jasonjei
My last name “Hung” was blocked by Apple’s Genius Bar at one point when they
displayed the names queue on the digital board. They don’t like certain
methods of Chinese romanization (particularly the one used in Taiwan).

------
joostdecock
This sort of thing happens to me all the time. (Bad) Spam filters are a big
problem too. And how do you even try to fix this?

I ended up taking my name off (the emails and website) of an open source
project I maintain just to please the spam filters:
[https://twitter.com/j__st/status/1210559583934189568?s=19](https://twitter.com/j__st/status/1210559583934189568?s=19)

I can appreciate the funny side of it, but it's also profoundly saddening
somehow. I was born with this name, what do you expect me to do?

~~~
kps
If you want to have fun with the English slang, change it to ‘Joost De Tip’.
[https://www.urbandictionary.com/define.php?term=Just%20the%2...](https://www.urbandictionary.com/define.php?term=Just%20the%20tip)

~~~
gefh
Joost ryhmes with toast though

~~~
kps
Doesn't start with an English _J_ [dʒ] sound, either. Not important to the
casual English reader.

------
llarsson
How do they not realize what an obvious shitstorm this will cause?

And who is bothered by this, really?

I find it hard to imagine that it's the drivers calling themselves Dick
Cummings out of immaturity. So what does it really matter what the customers
call themselves?

~~~
cmdshiftf4
>I find it hard to imagine that it's the drivers calling themselves Dick
Cummings out of immaturity. So what does it really matter what the customers
call themselves?

That was my thought too. It's one thing if you're using obscenity in an
anonymous online alias, as a great many Redditors seem to love doing, but I'm
struggling to believe that it's something people do in signing up for real-
life ride-sharing apps, either as a driver or rider.

Obviously there must be some amount of people doing it, otherwise Lyft
wouldn't bother with this at all, but it's just so... _unexpected_.

~~~
manigandham
> " _Obviously there must be some amount of people doing it_ "

There's no data that this is true. It could simply be a project manager
deciding that it's a good thing or the result of a tiny fraction of
complaints.

~~~
vernie
Complaints about what

------
blindgeek
A few years back, Facebook and others were banning People with the name Isis.
Anyone with more than two brain cells who had spent some time studying the
humanities would not have made that mistake. I'd think that people who write
code and design algorithms would at least be aware of the concept of
overloading, too.

~~~
brabel
Wow, that's ridiculous! Isis is a common name in many countries!
[https://www.behindthename.com/name/isis](https://www.behindthename.com/name/isis)

------
js2
"Falsehoods Programmers Believe About Names":

[https://www.kalzumeus.com/2010/06/17/falsehoods-
programmers-...](https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-
believe-about-names/)

Discussed most recently just 63 days ago:

[https://news.ycombinator.com/item?id=21492464](https://news.ycombinator.com/item?id=21492464)

~~~
progval
Poor Mr Graham-Cumming. Both an hyphen and a word banned by Lyft.

------
jFriedensreich
The US has a serious problem with saying normal things like fuck, cunt and
shit. I find it extremely offensive to hear beeps in series and even the verge
author self censors his article. Fucking stop being hypocrites.

~~~
teh_klev
Twitter seems to filter through the prism of US standards of speech. Woe
betide you if you're Scottish and use the word cunt. In Scotland there's quite
a lot of nuance in the usage of cunt. In fact I go so far as to say that the
use of "profane" language is a bit of an art form in Scotland, but sadly not
appreciated by our Facebook and Twitter overlords.

[https://www.youtube.com/watch?v=qdCmFg4xIPI](https://www.youtube.com/watch?v=qdCmFg4xIPI)

~~~
dragonsngoblins
Ditto in Australia. In fact we even have a saying about it: "Australia: where
you call your mates cunt and you call cunts mate".

Obviously context and tone play a lot into it and both "cunt" and "mate" can
be friendly or threatening/derogatory depending on delivery and circumstance,
but I imagine it would be remarkably difficult for people who aren't immersed
into Australian culture to figure out which is which based on text alone

------
hysan
And my name was filtered by Spectrum, who at the time was the only ISP in my
area. Made chat support incredibly difficult whenever I had to type in my
name. Explaining why I wrote it with spaces with an explanation to remove
which spaces was quite the test in reading comprehension apparently.
Especially funny was when the customer support tried to type it back to
confirm only to see nothing! Confused them even though I just explained why I
was doing it!

I hope to one day be able to see the feature spec and QA that led to these
types of hiccups.

------
cs702
I hereby propose the term "AS" for _Artificial Stupidity_ : The use of
software to automate obviously stupid behavior.

Sample usage: "AS is sometimes an unintended result obtained when insanely
smart people work with vast resources on very complex problems for long
periods of time."

~~~
nullc
'Automated Stupidity' would be more descriptive and correct. :)

------
dathinab
It's always funny (not) when companies don't slice realize that there is a
certain overlap between "bad" words and last names (many because of a shift
over time wrt. What words are bad, and differences between languages and
dialects).

------
soulofmischief
I got randomly "banned" from nexus mods two months ago because they forced a
password change, which wouldn't proceed because it said my name wasn't
allowed. Apparently having "fucker" in your name is against the community
guidelines for a website which lets you download adult content such as
hyperviolent and hypersexual game mods. I've also had this account for almost
a decade. I ended up getting in quite a spat with customer support until they
stopped responding.

I expect I will be receiving a similar notification from Lyft soon as the
email address I am registered with definitely has a name which would
incorrectly trigger this algorithm.

------
ravenstine
Doesn't Lyft know the real names of these drivers by the time they're on the
road? If people are offended by other people's given names then they're the
ones in the wrong, not the people named "Dick Cummings".

When are we going to grow up and realize that the mere appearance of words
like "dick" and "cum" don't hurt anybody?

------
sfgweilr4f
Religious, especially Puritan, values still echoing down through the decades.
Organizations reinforcing the idea that words about body parts are inherently
offensive are even worse.

I remember getting sent home for using the word "Penis". My parents both went
to the school for a parent-teacher interview. That principal had brought the
teacher in and were serious about my need for detention etc. The whole thing
ended in farce as my father laughed in the principal's face then angrily
demanded no consequence for me.

I doubt today's generation of teachers would tolerate this. Probably react to
being humiliated like this by calling it harassment or similar nonsense. But I
also think my father if he was part of this generation would have twittered
the whole incident instead and let the twitterverse do it for him.

------
squarefoot
Can't wait for their algorithm to expand to city names too.

[https://www.travelettes.net/funny-town-names-from-around-
the...](https://www.travelettes.net/funny-town-names-from-around-the-world/)

------
toyg
I guess it's one less high-profile customer for Lyft in London:
[https://en.wikipedia.org/wiki/Dominic_Cummings](https://en.wikipedia.org/wiki/Dominic_Cummings)

------
interestica
What happens if someone after-the-fact makes the term 'lyft' offensive a la
'santorum'?

------
vertis
This is a problem not just in profanity filters but also on marketplaces like
Etsy. Heaven forbid anyone named Harry Potter wanting to sell on Etsy. They
would more likely than not find themselves banned with little recourse.

In this case for 'trademark' reasons, but the problem is the same. Filter for
a set of things, and eventually legitimate examples of said thing are going to
get caught up in it.

------
tempsy
Sounds like a bad product manager. Could’ve talked to literally any company
that has a real name policy on common pitfalls and this would be near the top.

------
flatiron
Man butplugg that’s rough. Would not want that name in middle / high school.

~~~
libria
My first impulse was that was a troll name and I wondered if the journalist
verified it. I'm sure it's possible, I just haven't heard it or variants of it
before.

------
0x8BADF00D
Ah the good old Scunthorpe problem. Funny to see people still run into it.

------
elfexec
What about the Wangs, Coxes and Butts of the world. That's a significant
portion of the population Lyft is shutting out.

Also, is it really "the algorithm" or does Lyft just maintain a list of
entries of banned names? I know "the algorithm" is the hip new thing, but I
have my suspicion it's the latter.

~~~
thaumasiotes
Hey, the Wangs of the world probably only number about 100 million. Almost
certainly less than 200 million. Even at the high end, that's only 3% of the
world population. ;p

------
haunter
A german user was banned on Twitter for writing "Die Boomer" = "the boomers"

[https://www.dw.com/en/german-grammar-in-ok-boomer-tweet-
gets...](https://www.dw.com/en/german-grammar-in-ok-boomer-tweet-gets-twitter-
account-banned/a-51225631)

Also my all time favorite when Facebook banned people posting about faggots
which is a traditional food in the UK Midlands

[https://www.telegraph.co.uk/technology/facebook/10419598/Man...](https://www.telegraph.co.uk/technology/facebook/10419598/Man-
banned-from-Facebook-for-liking-faggots.html)

[https://en.wikipedia.org/wiki/Faggot_(food)](https://en.wikipedia.org/wiki/Faggot_\(food\))

~~~
noodlesUK
I have to say, when I first moved back to the U.K. after a long time in the
states, I absolutely could not control my shock followed by giggling when
people asked if they could bum a fag off me at a pub or suchlike. (Fag is
slang for cigarettes for those outside the U.K.)

~~~
reaperducer
I've always known the word "fag" to also mean a bundle of sticks, like you'd
use to start a fire. I expect the meanings are related.

Life seems to be peppered with problems with people with limited vocabularies
who would rather make themselves "offended" by something, rather than think.

See also:
[https://en.wikipedia.org/wiki/Controversies_about_the_word_n...](https://en.wikipedia.org/wiki/Controversies_about_the_word_niggardly)

------
uyuioi
Well I guess Benedict Cumberbatch is really in trouble.

------
sytelus
<speculation>

It's not an algorithm, it's some overjealous dev/PM with not many important
tasks on hand except for a little incident which he made big deal out of it by
downloading all dirty words and do string contains.

</speculation>

------
foxyv
Names are funny, you can't really tell people what names they can use because
they are THEIR NAMES. You try to censor certain names and you will trample
someone who has a name you didn't expect just because of their background or
race.

Example:
[https://en.wikipedia.org/wiki/Marijuana_Pepsi_Vandyck](https://en.wikipedia.org/wiki/Marijuana_Pepsi_Vandyck)

------
dschuetz
There must be some very smart people behind Lyft making decisions, as if they
have never met a person named "Cummings" or "Dick". I'm astonished by this. I
mean, those people have degrees in CS and software engineering. What were they
thinking? I suspect that they chose the _easy_ solution, not the smart
solution. In that case, because the "Community Guidelines" and the filtering
system disrupt business operation, I would fire everyone involved and replace
them with smart people.

~~~
6510
yes, fire them.

------
allovernow
FromSoft has a similar problem with totally inane censorship of names in
online play in the Dark Souls series. E.G. you would think they'd whitelist
the word "knight" in a game about knights, but it's always censored to k
__*ht. There 's some other wacky stuff going on but in this case I don't quite
blame them because the devs probably don't speak English natively or at all,
and you can tell they had a tight budget/schedule.

------
OrgNet
Not being from the US originally, I always found it very strange that someone
would name their child Dick... what's the reasoning?

Not a good reason to block them though...

~~~
thaumasiotes
That would be rare; the child would usually be named Richard and nicknamed
Dick.

It's not clear to me why Dick is the standard nickname for Richard.

~~~
klyrs
Better question; why is Jack short for John?

~~~
thaumasiotes
Middle English Jan -> Jankin (diminutive) -> Jackin -> Jack.

Compare manikin, a little man.

Apparently unrelated to French Jacques, despite the similarity.

~~~
klyrs
Wow, that's a much better answer than I've ever seen; I'm so glad I asked
here. Thanks!

------
brohee
Facebook did the same with Native American names
([https://newsmaven.io/indiancountrytoday/archive/facebook-
nam...](https://newsmaven.io/indiancountrytoday/archive/facebook-name-police-
native-american-names-aren-t-authentic-enough-zOU6k2YnK0ST5s09nFRl7A)), anyone
remember if they were eventually sued for that?

------
sjcsjc
Words change their meaning over time.

[https://metro.co.uk/2015/03/10/people-living-on-butthole-
lan...](https://metro.co.uk/2015/03/10/people-living-on-butthole-lane-say-
they-would-never-change-their-street-name-5096585/)

------
TigeriusKirk
What happens with towns like Cumming, GA?

~~~
interestica
Lyft just avoids operating in the city at all?

------
kerzol
[https://en.m.wikipedia.org/wiki/Pick_operating_system](https://en.m.wikipedia.org/wiki/Pick_operating_system)

Dick Pick created an operation system named GIRLS !

------
jklein11
Maybe the Danes are on to something here.

[https://www.worldatlas.com/articles/the-strict-name-laws-
of-...](https://www.worldatlas.com/articles/the-strict-name-laws-of-
denmark.html)

------
partlysean
I’ve run into something similar when naming loadouts in Call of Duty.
“Assault” is never accepted presumably because it has “ass” in it. It’s not
even something other players can see, like a username.

------
Piskvorrr
A clbuttic mistake.

The underlying story is getting tragic, however: "you can't blacklist
impropriety" is apparently not learnable, despite decades of failed attempts.

------
sbassi
My lastname is bassi and some sites doesn't like the "ass" part.

------
ceejayoz
The good old Scunthorpe problem.
[https://en.wikipedia.org/wiki/Scunthorpe_problem](https://en.wikipedia.org/wiki/Scunthorpe_problem)

------
mcintyre1994
I remember in secondary school the terrible filtering stuff they used would
block “weightwatchers”. Apparently annoyed a lot of the teachers :)

------
DrScump
This reminds me of a story arc on "Hill St. Blues" where LaRue comes across a
great stand-up comic and invests in his management... only to find that his
full name is "Vic Hitler"... and further finds that Vic had made his father a
deathbed promise that he wouldn't deny his surname or family "on account of
that one unrelated lunatic".

------
al_form2000
So what year are we in, again?

------
pfdietz
And what about Gary Null?

------
The_rationalist
It's funny because Verge in french is a synonym for dick!

------
abductee_hg
so ... tough luck for people like Whitney Cummings?

[https://www.whitneycummings.com/](https://www.whitneycummings.com/)

------
vanniv
Tech companies will never learn.

How many times must we go through "real names" idiocies before these supposed
geniuses stop being imbeciles

~~~
minimaxir
Real name policies are not a good analogy here, as being on Lyft requires in-
person interaction.

~~~
vanniv
What does an in-person interaction have to do with the idiocy of building
software to decide what names people are allowed to have?

------
donmcronald
Clbuttic

------
tmpz22
Its funny to me how they try to apply christian values like avoiding vaguely
sexual sounding words while also making significant amounts of money off late
night debauchery, picking people up from clubs at 3am.

~~~
vxxzy
How does this directly relate to “Christian” values? Could “Islamic” values be
applied? Afterall, the Prophet Mohammed made his opposition to crude language
clear in the Hadith.

Follow up: Also appears there are some misgiving regarding crude language
within the Hindu religion as well. [0]

[0]:
[https://www.hinduismtoday.com/modules/smartsection/item.php?...](https://www.hinduismtoday.com/modules/smartsection/item.php?itemid=4005)

~~~
qwerty456127
Ok, let's call that puritan values of abrahamic religions.

~~~
klipt
Ironically a lot of "puritan" values are now considered "social justice"
values.

See eg "shirtgate" where an astronomer was castigated by feminists for wearing
a shirt with skimpily clad figures, on the basis that it was offensive to
women. Ironically, the shirt had been given to him by a feminist woman friend
who thought it was liberating.

~~~
wutbrodo
There's always some subset of the population whose conception of "morality"
consists of blindly bellowing the dogma of the day. It's not weird that the
behavior looks identical even though the dogma itself is nominally different.

------
Havoc
Bit of a dick move

------
petre
Philip K. Dick would get a dystopian feeling about this if he were alive.

------
rdiddly
Tell me again why anybody cares? Is it because the Church is so central to our
lives? We're masters of rationality and technology, who believe in a
retributive super-ape in the sky?

Is it because we're medieval scribes and we really want to use the Latin
_faeces_ to indicate our scholarly intentions, instead of the Anglo-Saxon
_shit_ that everyone around us uses?

There is no reason I can think of that comports with a technological or
innovative worldview.

Including cybersecurity. ("Shit" is just an identifier or a guessable string
with 13 bits of "chaos" completely equivalent to "Dave" or "Phil.")

Edit: Well I see small-minded superstitious dunderheads have shown up to
downvote me, so FUCK CUNT SHIT ASS

~~~
falcor84
Words have meanings. These particular words have offensive meanings, in the
sense of them being used to intentionally offend others. There's no word I'm
aware of that is only ever used to offend and they all have (and generally
have stated with) inoffensive meanings.

In pretty much every community, we seek to reduce the friction between
members, and as a part of that, different communities would often choose a
point on that chance-to-offend spectrum and discourage use of anything more
(potentially) offensive than that point. Despite myself generally preferring
more open (and more offensive communities), I find it very rational for others
to set the bar at other points.

~~~
rdiddly
Sure, agreed, and Lyft's apparent priorities are a big part of why I'm not in
a relationship with them or part of their "community" (read: "customer base."
_Community_ is a lie and therefore offensive, despite being totally a "clean"
word... great and unexpected example... I digress).

Edit: sorry, digressed so far that I forgot to finish. By "priorities" I mean,
Lyft, pull your people off of pointless projects like messing with people's
identities, and put them on QA of basic functionality like accepting a credit
card payment via your website. Last and only time I tried it, it failed
utterly. Now I don't know how much anti-sales other people need to de-persuade
them before they'll not-buy, but in my case that was enough. The person for
whom I was trying to buy a gift card, got something nicer with a lifecycle
that doesn't burn such an obscene/offensive quantity of fossil fuels.

