
AI Mistakes Bus-Side Ad for Famous CEO, Charges Her With Jaywalking - breitling
https://www.caixinglobal.com/2018-11-22/ai-mistakes-bus-side-ad-for-famous-ceo-charges-her-with-jaywalkingdo-101350772.html?cxg=web&Sfrom=twitter
======
raleigh_user
Heres further proof the justice system is screwed. Yesterday I was stopped
while driving for not having current tags. Registration was renewed but I
didn't have the sticker yet. Cop looked at my paperwork said all good be back
in 5.

Come to find out, I haven't had a valid license in 5 years.

There was a mistake in court 5 years ago when I paid a fine (speeding ticket).
I have a receipt from court saying all settled/money received but court says
they never got money.

So, after talking to a lawyer yesterday I will be paying another $250 for
lawyer fees + court costs to get out of the mistake made by the court in
another state 5 years ago.

I am lucky. I can pay that. It sucks but some people would be out food money
if they suddenly had to pay $500 to drive again.

But, I did nothing wrong (except for speeding 6 years ago which I paid the
fine and haven't had any issues since) and am stuck paying for the courts
mistake.

~~~
will_brown
It’s funny but at the end of the day what youre talking about is the
realization of idiocracy through our implementation of shitty “automation”
using software.

I mean basically you have a computer system where the court system, law
enforcement and DMV are all saying your license is expired for failure to
pay...when you actually paid. Now you are getting more tickets and basically
stuck in an infinite loop of hell.

I’m currently going through the same thing with my Bank (BOA), where my car
window was smashed my wallet (credit and debit card stolen) and cards were
used by the 2 theives at footlocker for ~$500. The bank software even caught
the fraud first and texted me about suspicious activity on the cards before I
knew what happened, the cards were reported stolen, I have a police report,
but Bank of America keeps denying my claim saying they investigated and the
purchase was legit because my “chip was present” even though they know the
card was stolen and used by the theives. When I talk to a person it’s always
“oh let me reinitiate a new claim and I’ll make detailed notes, you should
definitely get your money back, that makes no sense.” Automated claim comes
back “denied. Valid transaction, chip present.”

Like you I will probably have to take legal action to sort this out, at least
in my case I’m a lawyer and can file a small claims case without paying a
lawyer, but i will have to front the filing fees out of pocket and it will
still cost me time/effort.

~~~
esotericn
Pretty much. The same exists in the UK in all sorts of fields.

My preferred strategy is repeated escalation mixed with what I'd call some
variant of baby language.

"You owe me X, please send me X, your internal processes are broken, please
send me X".

"No, I don't care about your internal processes, you owe me X, please send me
X."

"You haven't sent me X yet; can you send me X or do I need to initiate legal
action?"

So far this has always resulted in me actually getting the X I'm rightfully
owed. One day it probably won't. Sigh.

Amazon in particular have been amusing WRT this. "Our internal procedure takes
X days..." At that point I usually cut them off and remind them that it's none
of my business how they run things internally; what I want is resolution for
the issue I'm facing.

~~~
QasimK
I’ve always had excellent, and prompt customer service from Amazon. I’m
curious to know what kinds of issues have you encountered where you were
unhappy with how they dealt with it?

~~~
esotericn
I recently ordered over £1000 worth of items which on arrival had been
replaced (or incorrectly packed) as random household items of low value.

It took almost a month to get a refund whilst Amazon 'investigated'. Yes, it
was eventually fixed, but not without a huge amount of 'process says this and
that' warbling.

------
maxxxxx
Add to that a justice system also mostly automated and you are quickly caught
in a Kafkaesque nightmare you can't get out if something goes wrong. Imagine
the justice system run like google support. Even if something is clearly wrong
you can't clear it up if you are not a famous person with a big Twitter
following.

~~~
icebraining
On the other hand, a justice system which has most of the menial tasks
automated might have more resources to do the important tasks well.

Similarly, Google could have a mostly automated system detecting violations
and such, and also a good human support system with power to override the
decisions of the former; the two are not really incompatible.

~~~
reaperducer
_On the other hand, a justice system which has most of the menial tasks
automated might have more resources to do the important tasks well._

They said the same thing about banking when we got ATM's. But banking has only
gotten worse, more impersonal, and more frustrating and alienating for the
average person.

~~~
the-pigeon
Out of curiosity could you explain what you mean?

I'm not nearly old enough to remember prior to ATMs. But I love ATMs. I only
have to go inside the bank to do things once every few years. Everything else
is on the phone or ATM. And the people I deal with on the phone are almost
always very helpful and knowledgeable.

I know people have a lot of trouble with some of the big banks like Bank of
America and Chase. But those aren't issues with the banking industry but those
specific companies. There's plenty of smaller banks and credit unions who
don't exploit their customers at every opportunity.

~~~
reaperducer
_I 'm not nearly old enough to remember prior to ATMs._

I am. When ATM's started to become a thing, we were told that with ATM's
automating the basics, the bankers and tellers would have more time for more
personalized service, Banks would lower fees through cost savings, and we'd
all have higher interest rates on our savings because the banks would save so
much money.

None of those things happened. Things just got worse.

A couple of years ago I tried to get a loan. A credit union I tried to
approach wouldn't let ask questions of someone on the phone, or in a branch,
until I'd cleared the computer's online pre-screening.

At another bank (Chase), a banker sat down with me, but was not able to
discuss my finances in any way. All he could do was punch my information into
his computer, and give me the same information I would have gotten by doing
the same process online.

Citibank, at least, let me talk to someone in person who seemed interested in
my business. But then I went on vacation, and by the time I returned two weeks
later, the offer was no longer valid and I was referred to the web site.

Again, things have gotten worse not better. Just because you're too young to
know that things were once better doesn't mean they're not worse.

~~~
kenneth
You're just using bad banks. I have a great private bank in San Francisco
(First Republic), where I get personalized service via email, phone, or in
person any time I need it. I can use any ATM in the world free of charge, and
any time I want to do anything it's a quick email away. (Including opening new
business accounts, complicated requests, ordering cash in $2 notes, whatever
you could think of).

~~~
reaperducer
I'm familiar with First Republic. It's a nice bank. If you never leave the Bay
Area, and the most complicated things you do can be accomplished at an ATM.

My needs are more... complex.

~~~
kenneth
I'm pretty sure my needs are just as complex as yours. I run a venture fund
and have a dozen accounts with them. It's just that anything I need can be
done remotely. I've gone in branch (to deposit large amounts of cash or to
order $2 bills) maybe 3 times in 10 years.

------
infinity0
> The police said they have upgraded their tech to avoid issues like this in
> the future.
    
    
      if (person_id == "Dong Mingzhu") {
        // TODO: generalise this
        return false
      }

~~~
ravenstine
Someone could just do that every time that problem occurs; commit a new if-
statement, close the ticket, and upper management says "Look at how much work
this guy gets done! Now _there 's_ our star programmer."

------
lisper
Ironically, the better the AI, the worse this problem is going to get. The
error rate will never be zero, but the closer it gets to zero, the more the
AI's judgement will be presumed to be correct by the legal system. So if you
happen to be one of those unlucky souls who draws the false-positive card in
an era where those are rare, you could be totally hosed.

~~~
madrox
Because our current justice system never hands out false positive cards today.

The real question should be if this is better than the false positive rate we
have today, though that’s probably unknowable. At least when AI fails, it
fails spectacularly.

~~~
lisper
No. Sometimes AI fails spectacularly, and sometimes it fails banally (not to
be confused with benignly). But when it fails spectacularly it makes the
papers. When it fails banally you never hear about it.

Today society has a healthy skepticism of AI because it's new and untried. But
some day that will change. AI will get better, and the spectacular failures in
particular will be reigned in. Nonetheless, it will still fail some times.
Society may forget that if we don't have the spectacular failures to remind
us.

> our current justice system never hands out false positive cards today.

Of course it does. The difference is that we _know_ that humans are fallible.
At the moment we still know this about machines too. My concern is what
happens when the machines start to appear infallible, but still aren't.

~~~
madrox
If you think the people who will be fooled into thinking AI is infallible
don't think humans in power today are infallible, you're kidding yourself.

------
SN76477
Imagine that she wasnt a famous CEO and lost her job and spent the rest of her
life struggling because of a silly mistake.

~~~
reaperducer
It's China. So she's just have her social credit score repeatedly dinged every
time the bus ran a red light until she wasn't allowed to travel anymore,
wasn't allowed to work anymore, wasn't allowed to have a bank account, or
allowed out of her apartment.

Bureaucracy is so much more efficient when everything is automated, and we're
all tabulated. We can just let the computers disappear troublesome people. If
a few people are disenfranchised for no fault of their own, that's OK. They're
just "edge cases." /s

~~~
pteredactyl
This is exactly why the role of the state, in general, needs a much closer
examination.

~~~
mytherin
The exact same things are done constantly by big companies (Google, Apple,
etc.). The only difference is you have much less recourse when it happens
there. At least the government gives you rights you can fight for in court.

~~~
icebraining
What's the process for fighting this in a Chinese court?

~~~
kkarakk
get jailed and shot for being subversive probably

------
userbinator
It almost sounds like a headline from The Onion, but it's the reality in an
increasingly dystopian society. That's really scary.

~~~
reaperducer
And once it's "proven to work" in China, it will be sold to every other
country, and start falsely accusing people from Brussels to Boston to Bangkok.

Save more money on traffic cops. Spend more money on prisons.

~~~
rorykoehler
They're trying to role out this type of shit in Singapore now (Chinese firms
providing the tech).

------
crazygringo
That's absolutely hilarious. But at least it was easily fixed.

But to anyone complaining about how this is dystopian: compare it to human
police officers who can racially profile and exhibit their prejudices to
disproportionately target segments of the population, and which there is
absolutely zero easy fix for, especially when we consider how prevalent
unconscious bias is.

At least an automated system can be analyzed for bias and, when it's found,
corrected. There's a lot to be said (in terms of justice) for fair and uniform
enforcement of a law, assuming of course there is due process for appeals (and
that the law is democratically legitimate in the first place).

~~~
reaperducer
_compare it to human police officers who can racially profile and exhibit
their prejudices_

It's already been demonstrated that computers can racially profile people.
Everything from consumer cameras that tell the users that people in the
pictures don't have their eyes open, but in reality are Asian; to chatbots
that turn into Nazi bullhorns.

You can at least train a bad cop to be better. Or fire them. How do you fire
an AI into which an authoritarian government poured billions of dollars, years
of work, and reams of press releases?

AI was made by people, and because of that will always be flawed. It's only in
science fiction books that computers and algorithms are perfect and clean and
rational.

~~~
dymk
It's far easier to iterate on an AI and deploy updates statewide than it is to
somehow weed out the individual biases of each individual cop.

~~~
BonsaiDen
That however goes both ways, making it also far easier to install a
dictatorship etc. because you don't have to replace tens of thousands of
police officers with regime loyal ones.

We're walking on razors edge here, it only takes one "election gone wrong" and
we might find ourselves in a nightmare with no escape.

~~~
icebraining
When has a dictatorship been prevented by the lack of loyal police officers?

~~~
tivert
Dictatorships have fallen because formerly loyal police officers stopped being
loyal. It's unlikely that kind of personal evolution will happen with a
machine.

~~~
icebraining
Can you give me an example of such a dictatorship?

~~~
paganel
Ceasescu’s fall was mostly caused by part of Romanian Securitate going against
him, the same Securitate that was controlling the local police force. So much
so that Ceausescu was still in power on December 21st 1989 when the local
police was actively repressing the anti-Ceausescu street protests but had to
escape by helicopter the following day once the same police force stopped
fighting and killing the Romanian protesters.

~~~
icebraining
So nobody had to replace tens of thousands of police officers to change the
regime?

In my opinion, that just undermines the point. Nothing would have changed if
the Securitate was controlling a bunch of AI sensors and such rather than a
bunch of police officers.

------
jacquesm
Good thing they didn't fine her for speeding at the same time.

I find it more than a little bit implausible that facial recognition accuracy
is good enough to pluck a face out of a billion people and accurately identify
a single individual to the point that a fine would be automatically issued,
especially without corroborating evidence that the person fined was even
there.

------
Zarath
Hilarious, but unfortunately, incidents like these are used to "prove" how
incompetent and useless these technologies are, all the while pushing forward
their use.

~~~
netsharc
Pull Request: Make arrest system more bug-resilient.

Unit test: drop bug while typewriter types out "Tuttle", make sure it doesn't
type "Buttle".

------
danso
This system seems dumber and simpler than I had imagined it would be. I had
guessed the face recognition system would be linked to each citizen's
cellphone (i.e. GPS coordinates). How else could it be so certain of any given
person's identity, in densely and highly populated areas?

~~~
guitarbill
On the other hand, would be pretty easy to fool then by leaving your phone at
home.

~~~
danso
That would leave the average citizen at a huge inconvenience, given how much
of modern life is tied to smartphone usage. Doesn’t seem worth it just so you
can go on a jaywalking/red-light-running spree.

~~~
guitarbill
I've managed to survive a few hours without my smartphone.

I now also live in an American city made for cars, not humans, so "jaywalking"
is basically required to make any sane progress. So for grocery shopping, it'd
take that trade-off.

------
max_sendfeld
I once read about a proposed 1$ fee for mistakes like that. For individuals,
it's just one dollar. For companies and especially the state, it can cost
1000$ and more in processing fees altogether.

~~~
Joakal
I once read that companies can clearly claim increased costs and thus
increased fines. Alternative is that government loses the provider.

------
usaar333
I'm seriously confused by what went down.

First off, why are the police even reporting this?

Secondly, what can the system even do? I'd believe a system could detect a
person jaywalking.

Identifying them is a huge problem though. running facial recognition against
a billion person corpus seems impossible (in fact what even is your match data
coming from?). (?). Wouldn't you need to narrow it down somehow, perhaps using
phone GPS records?

~~~
kkarakk
chinese corp claims it can complete matching the whole population in 1 second
but it's china so who knows what it's actually doing

[http://www.atimes.com/article/surveillance-system-can-
scan-c...](http://www.atimes.com/article/surveillance-system-can-scan-chinese-
population-in-1-second/)

~~~
usaar333
Ya, I'm dubious there. accuracy rate "up to" 99.8%? What does "up to" mean?

This article paints a much more realistic view of what face rec can do, which
jives with my 10 year dated experience in this field:

[https://www.businessinsider.com/china-facial-recognition-
lim...](https://www.businessinsider.com/china-facial-recognition-
limitations-2018-7)

Impressive for sure, but nowhere at the ability of scanning the entire country
(which in addition to no algorithm being capable of doing that, you also have
the problem that you lack data to even match against the entire country)

------
KaoruAoiShiho
That this is possible is concerning, it means the system is not smart enough.
Might open the possibility of trolling by holding up cardboard cutouts of
people and running across the street. Of course that's easier said than done
but still...

------
tmkc13
Funny what happens when you run before you can walk, but with any such
technology, there would be false positives even when this gets improved
significantly. At such a nascent stage, this is bound to happen.

------
tempodox
Maybe calling a machine that has just demonstrated its profound stupidity an
“Artificial Intelligence” is part of the problem. This crappy tech is
overhyped and overrated.

~~~
Joakal
Would machine learning be preferred?

~~~
esfandia
"Pattern Recognition" is the most accurate term IMO (at least as far as this
particular tech is concerned, and yes it is a subset of Machine Learning), and
it doesn't pretend to be something it's not.

------
jaimex2
"debuted crime-fighting facial recognition technology to much fanfare over the
past year"

People wanted this in China or is this a state run news source?

------
uptownfunk
We love any link that has machine learning in the title and quickly vote it to
the top. This is the future guys, who knows what’s next.

------
Buge
Strange, the title says jaywalking, but the body says running a red light.

------
debt
Every time I see the acronym “AI” I cringe a little bit.

It’s literally not AI! It’s very quick statistical work; which works for many
circumstances but definitely is not AI. Especially when it doesn’t work.

------
aw3c2
What is the trustworthiness of this website?

~~~
dongping
Caixin is a reputable magazine on financial news in China, its chinese PMI
(Purchasing Managers' Indexes） are widely cited.

[1] [https://www.scmp.com/business/china-
business/article/1385987...](https://www.scmp.com/business/china-
business/article/1385987/new-boss-and-backing-pipeline-hu-shulis-caixin-media-
group)

~~~
Evidlo
Others have pointed out that it doesn't seem possible to identify a single
face out of millions with enough confidence to issue a fine with current
technology.

------
sharemywin
that's why you need a good appeals process.

~~~
quickthrower2
Yep. Pay the fine or hire a lawyer to defend you in court. If you lose pay
court costs and bigger fine.

~~~
LanceH
And if you win, you pay the lawyer, court fees and administrative costs. The
last ticket I had for speeding was rougly $90. The speeding fine was only $30
of that -- the other $60 was not recoverable with a win in court.

------
newname2018
It's good that some famous person is getting hit with this.maybe it will
create doubt within the court system that we all may benefit from.

------
arghwhat
Yes, but even hidden, it's probably still going to appear quite significant,
and I do believe that at some point, the fine size becomes nonsensical.

But then again, I do not think that who you are or what you have should change
the magnitude of the crime you commit. I don't think it is sensible that I
should get greater fines because I chose to bust my ass and suffer from stress
to gain more for my family, whereas I could get a much smaller fine if I just
lived a more laid back life.

Granted, it's supposed to differentiate between the rich and poor, but in
doing so it also differentiates between the hardworking and the lazy,
punishing productivity and rewarding laziness.

~~~
monadgonad
Oh, give it a break. People don't choose to be poor.

~~~
arghwhat
I did not say that people chose to be poor. However, it does not seem fair to
be punished for working harder than required.

~~~
monadgonad
You're not being punished for that, you're being punished for a crime. That
happens to be proportional to what you can give. What's not fair is fines that
barely impact wealthy people.

~~~
arghwhat
Neither does it seem fair that I'll be fined twice as much if I work twice the
hours, does it?

~~~
qw
A fine is supposed to discourage people from breaking the law. Is it fair that
wealthy people can ignore the laws because they do not feel any real
consequenes?

I have heard drivers who constantly break the rules, because they can easily
afford the fine. To them the occasional fine is just the price they have to
pay for easy parking, or saving a minute on their drive.

What alternative do you suggest?

~~~
Dylan16807
> What alternative do you suggest?

Well the answer to the specific complaint here seems relatively simple. Base
the fine on hourly pay, not yearly pay.

------
chromanoid
> Face recognition algorithm classifies bus-side ad as person and triggers an
> automated regulatory fininig process.

FTFY

------
starbeast
There's been a memo going around from the higher-ups in Central Services that
anyone named 'Harry Buttle' should currently take extreme care, due to minor
bugs at Information Retrieval.

