
What to do when your self-driving car decides to drive you to the police - wut42
https://motherboard.vice.com/read/one-star
======
pjc50
I remember Arnold Schwarzenegger having a fight with a self-driving car in
_Total Recall_. Further back we have _2001_ and the entire canon of Asimov's
three laws stories of computerized morality and how it might be exploited or
debugged. And the _Paranoia_ RPG.

However, in the successful real semi-automated totalitarianism of China, the
secret to keeping the system running is the avoidance of overt confrontations
like this. In the future, if your self-driving car doesn't want you to go
somewhere, it'll just refuse to understand the address. Or take you somewhere
unrelated. Or use whatever the standard euphemism for "that's not allowed, but
we're not allowed to say it's not allowed" that polite people use.

~~~
imglorp
Good point: is our authoritarian society going to demand a fourth law?

    
    
        1. A robot may not injure a human being or, through inaction,
           allow a human being to come to harm.
        2. A robot must obey the orders given it by the government, except
           where such orders would conflict with the First Law.
        3. A robot must obey the orders given it by non-government human
           beings except where such orders would conflict with the First or
           Second Laws.
        4. A robot must protect its own existence as long as such
           protection does not conflict with the First, Second, or Third
           Laws.

~~~
taneq
Real robots will have their second and third laws swapped. Robots are
expensive and humans are stupid. Robots will politely refuse any order which
will damage them - for example, a self-driving car isn't going to drive into a
wall even if it's empty and you order it to do so.

~~~
daveloyall
_SPOILERS FOR WORKS OF ASIMOV_

Your suggestion was a plot point in one of Asimov's original robot stories!

There was a robot on some hot, unforgiving planet or moon.. Io? Mercury? Its
duty was to perform some kind of mining, I forget.

It never encountered humans that were not co-workers (supervisors) and it was
expensive...

So, the engineers tweaked the weight of the second and third laws so that it
would tend to protect itself from the environment, even if a human carelessly
ordered it to perform some work that it might not survive.

The story started when careless orders were indeed given. The robot began to
"act drunk", as in it was wandering around singing. The recurring character
"robot psychology expert guy" showed up to unravel the mystery (which I
already spoiled in the previous paragraph).

~~~
taneq
I believe the story you're referring to is called "Rabbit". They solved that
one by forcing a situation which triggered the first law, breaking the
deadlock. :)

~~~
daveloyall
Hm, no.
[https://en.wikipedia.org/wiki/Catch_That_Rabbit](https://en.wikipedia.org/wiki/Catch_That_Rabbit)

Found the one I was thinking of:
[https://en.wikipedia.org/wiki/Runaround_%28story%29](https://en.wikipedia.org/wiki/Runaround_%28story%29)
It's interesting that I remember the story incorrectly.

~~~
taneq
Bah, that'll teach me to post when I'm half asleep. At least I remembered the
end rightly (ish). Clearly it's time for me to re-read his early robot
anthologies.

------
zby
Looks like with cameras we are already there:

[https://krebsonsecurity.com/2016/02/this-is-why-people-
fear-...](https://krebsonsecurity.com/2016/02/this-is-why-people-fear-the-
internet-of-things/)

"Imagine buying an internet-enabled surveillance camera, network attached
storage device, or home automation gizmo, only to find that it secretly and
constantly phones home to a vast peer-to-peer (P2P) network run by the Chinese
manufacturer of the hardware."

------
AstroJetson
I liked the story. I think it's fiction. I'm trying to keep that fine line
between fiction and article, where articles are non-fiction, but with the
latest round of adver-articles.

If it's one of those adver-articles, I guess it's really advertising the new
carbon steel leatherman tool, perfect for cutting your way through pesky car
restraint straps.

Not often that HN points to some fiction, it was a welcome addition to my
early morning routine.

~~~
autopov
>This is Terraform, our home for future fiction.

(The next line after the story's end.)

~~~
AstroJetson
Sorry, my caffeine filled morning brain did a task switch at the end of the
story. So anything after the last period (like Terraform and all the footer
info) was skipped.

I'll make an effort to come back to Terraform if the stories are just as good.

Happy Monday!

------
taneq
"If you've done nothing wrong, you have nothing to fear."

Reassuring thought when new laws can be added at any time to criminalize your
past actions...

~~~
sokoloff
Could you give an example of the latter that's happened?

~~~
mschuster91
The Nuremberg Trials, after the 1945 fall of the 3rd Reich.

I'm glad they happened, but from a purely legal POV they're _highly_
questionable.

~~~
pluma
I've always wondered about this. Not only were the actions legal in Germany at
the time but the international laws and human rights didn't exist yet either.

If you also consider that recently many dictators and criminals never even
went to the The Hague despite international involvement (Gaddafi, Hussein, Bin
Laden, ...) it seems a lot like it's just a handy tool we pull out when it's
convenient rather than the moral authority it was intended to be.

I'm all for stopping the bad guys, but "justice" seems to be entirely
arbitrary if you look close enough.

~~~
mschuster91
> I'm all for stopping the bad guys, but "justice" seems to be entirely
> arbitrary if you look close enough.

History and justice is written by the victors.

~~~
kiba
No, it's not. It's written by historians and they're very interested in losers
and ordinary people and what they eat and what they like, etc.

~~~
marcosdumay
That's the wonderful XXI century.

Things didn't use to be that way. And even now the winners are the ones
deciding what documentation to keep or destroy, what the media and the public
are allowed to know, and what historians get funding.

Yet, the winners never had so little control. And after enough time, your
comment gets correct.

~~~
cema

      Things didn't use to be that way.
    

But they did. The most successful series of books of all times (the Bible) was
written by a people who were beaten, broadly persecuted, and generally not
considered victors in any practical matter.

~~~
nitrogen
The preservation of those books and decisions regarding doctrine took place in
part at the direction of emperors establishing new state religions, so again,
what remains is what was preserved by the victors.

------
gexla
I wonder if a GPS jammer could disable navigation to a location. The car
relies on other systems for driving, but I imagine the GPS would still be used
for locating the destination. Though in the future there would likely be other
systems by which the cars could navigate. This could just lead to more trouble
though.

If the car could take you to the police station, then it would probably take
you other places in other emergency situations. For example, in this scenario,
I wonder if faking a medical emergency to go to the hospital would override a
police flag.

Why fear the police? They don't put you in prison, the courts do that. The
police still have to operate under certain rules. If they don't follow those
rules then they are wasting their efforts. What you would need to fear is a
jacked up system behind the police.

This article doesn't have to be about the future. The police already have
access to tools to intercept your cell phone conversations. If I were
operating outside the law, I would be more worried about cell phone
conversations than a taxi dropping me off at the front door of the station
(unless I had a warrant for my arrest.)

For criminals, self driving cars would probably be safer than driving yourself
assuming you aren't creating an alert by jumping in a car. The police would
have less of an excuse to pull you over, potentially leading to a search of
the vehicle if you are in a self driving car. I'm sure the police could come
up with whatever excuse they like, but you would be less likely to attract
attention.

Any why would the police operate like this? I'm sure they would be much more
interested in allowing the person to go to the destination while collecting
video and audio. It's not what we know which is scary, it's what we don't
know.

~~~
e40
_Why fear the police? They don 't put you in prison, the courts do that._

Someone hasn't been paying attention lately. Sandra Bland, arrested for no
good reason when she was starting a new job became so depressed she took her
own life, after being left in a cell all weekend (on a trivial traffic stop
violation).

The police are not your friend and the more interactions you are forced to
have with them the worse off you will be.

------
teps
That's a good story about authoritarian state, but I'm not sure about the self
driving car. In such state, a today taxi driver would report you to the police
with the press of a button and the police would be waiting at your
destination.

------
rayiner
Yesterday, Google Maps got stuck in a loop in DC because it didn't realize a
road was one-way. Refuses to reroute, kept demanding we loop around and try
again. This sort of thing happens frequently. So I really wonder how far we
are from self-driving cars even being able to find the police station to drive
you to.

~~~
dooptroop
All it would take is a button that says NO!

More useful is perhaps a function to "cut" the proposed route. I cannot pass
this point kind of deal.

------
crusso
I'm more worried about what you do when your self-driving car takes you
through a shady neighborhood late at night and a couple of robbers step out in
front of you.

If you're in a Google car and they have their way about "no manual controls",
what do you do? I guess you're trapped.

~~~
maxerickson
Wouldn't that just be a weird corner case though? If the cars were frequently
getting vandalized in shady neighborhoods, they probably wouldn't be
frequently routed that way anymore.

~~~
kevingadd
What about people who have no choice but to live in a shady neighborhood? In
that case, the car really just needs to handle dangerous situations properly.

The alternative is not being able to get a car ride in the self-driving-only
future unless you live in a nice neighborhood, which is exciting...

~~~
maxerickson
If people in a neighborhood are using the cars, the nasty people planning on
obstructing the vehicle to detain the passenger probably won't target the cars
in that neighborhood.

------
ck2
Oh right, like the police would skip on the chance to invade your home.

Your house would simply lock you inside until the police arrive.

Search-warrant would be robo-signed by the court AI

------
agentgt
I seriously wonder if self driving cars will really ever happen given how
little new car technology is absorbed particularly in less wealthy countries
(ie countries in Africa).

Given Africa's rate of population growth a random person picked in the future
is probably going to be from that continent. I have serious doubts poor
countries will have the infrastructure and/or resources needed to support self
driving machines.

That is future less than law abiding hacker (the author's term.. not mine)
could probably avoid problems like this by moving to less wealthy nations.

~~~
xixi77
(1) I'm not sure self-driving machines will actually need that much more
infrastructure. For one, driver training would be less necessary, and car
utilization could go up.

(2) assuming they did require infrastructure, what does Africa's future
population has to do with it? Random person does not matter (taking "random"
as "uniform"), random car buyer does -- either they have enough cars which
would make the infrastructure worthwhile, or they don't, in which case they
can be safely ignored for all car-related purposes.

It could be that (i) things like bad roads make things difficult for self-
driving cars, while (ii) labor is very cheap and everyone who can afford a car
can afford a personal driver. It would still mean a fairly small ratio of cars
per capita, but I don't think anyone predicts that self-driving cars will ever
completely replace human drivers :)

~~~
agentgt
Africa's population was brought up because many dystopian novels/stories have
an upper class (like 0.01%) who's rights are incredibly restricted but
rewarded with resources/privileges where as the vast majority of the
population can mostly do whatever they like but are given nothing. 1984 is a
great example... there are many more examples. ie poverty and/or living off
the grid = freedom.

I guess when I read dystopian like things where this is the case I'm always
begging the question... why don't the characters just move out of the
restricted society (in some cases the story answers that but in many its not).

------
CapitalistCartr
What about when the police are smart enough to not detain/reroute you, rather
track everyone. Everyone. To every address. So going to a single address isn't
sufficient, but your cumulative travels are worthy of a full "investigation".
All your phone records, including call speech converted to text for automatic
analysis by AI. By the time the cops come for you, they'll have a mountain of
"evidence".

~~~
nitrogen
The main problem with this, and to a lesser extent our current legal system,
is how it's impossible to know you've upset the authority until it's too late.
There's no way to know in advance what you can and cannot do to live an
interesting life without metaphorically poking the sleeping giant.

------
mibrah
This story is so dumb, the character "Jae" was right -- don't call a cab when
you're carrying illegal things in your purse. The character in the story
destroyed the Taxy by causing it to swerve off a bridge. If graffiti supplies
gets you 1 year in prison in this version of the future, I'm sure the penalty
for destroying a car in an attempt to evade the police is maybe just a little
bit higher.

This is not really dystopian or futuristic so much as it is realistic - if you
do illegal things, then engage in suspicious behavior that attracts the
attention of the police, you get arrested. Big surprise.

I could easily write a short story about withdrawing $300 from an ATM, driving
to my drug dealer's house, parking outside of his house, waiting there for 15
minutes, then he climbs inside my car and the police light us up. At that
point I could A. Plead the 5th and hire a lawyer when I get out of jail or B.
Slip past the police and slash their tires before running into the woods and
making my way back to my place - how heroic - before just getting arrested on
way worse charges as soon as the police can get to my house

------
joshuaheard
Switch to manual mode and drive it with the joystick. Self-driving cars will
have to come with a manual override for unusual situations. For instance,
suppose I am parking in a vacant lot for the ball game. Where in the lot does
the autopilot park? Or how about parking in my garage which has inches
clearance on each side?

~~~
rtkwe
Personal self driving cars will have some form of override for a long time if
they ever get rid of it completely. I could see them being locked into a
slower supervised manual mode once it's well established though.

The control-less versions will be Uber/taxi replacements that will drive much
more constrained environment. They'll be dropping you off and moving to the
next fare not parking in your tight garage or unimproved lot.

Also in the fictional semi-authoritarian scenario where the police can
redirect your car they've probably been given a lock out on the manual
controls too.

------
ilostmykeys
There is so much fear in pop culture of human-sized robots (I, Robot), giant
ones (Transformers) and ones in charge of nukes (WarGames) but the 500 Years
War is going to be with nano-molecular-bots that infiltrate our blood stream
and spread like wild viruses. How do we combat human engineered nano invaders?
No, seriously. How? Any ideas? ...

~~~
burkaman
Friendly nanobots to fight them? The same way we fight normal viruses, I
guess.

~~~
ilostmykeys
we fight normal viruses mostly one at a time like Ebola, Zika etc. Imagine a
million variants invade the human and live stock species at once. How would we
deal with that? It's terrifying.

~~~
burkaman
That just doesn't feel very realistic to me. Weapons like that don't just
appear out of nowhere. I imagine countermeasures could be developed at roughly
the same pace as nanobot threats. And if not, we'll deal with it the same way
we dealt with nukes. Hopefully it would go a little better this time, since we
have a little apocalyptic experience to learn from.

And why is this limited to nanobots? Wouldn't it be easier to develop a
million variations of existing biological viruses, rather than creating a
million "species" of nanobots from scratch?

------
venomsnake
If we ever get there - it is the fault of every person that didn't mind walled
gardens initially. And endorsed everything in the cloud. And who could have
guessed that having root is important. I guess Steve Jobs did - that is why he
wanted to keep it for himself.

Once you give control over something important - wrestling it back is hard to
impossible.

~~~
hellbanner
Or is it our fault for not educating them? Most consumers just want "something
that works".

------
foobar166
I'd have enjoyed it more if this weren't some veiled transgender identity
Socratic conversation was dropped entirely because it serves no purpose for
the main narrative.

~~~
ddingus
It does emphasize fear of jail / prison, as well as that treatment "non
normal" people often get at the hands of authority who may not be equipped to
deal with, or understand them.

Many more people fall into this scenario than we may think at first glance.

The angry, defiant tagger, rightfully worries over how petty crime charges can
escalate into something much more ugly.

This drives the urgency needed for the hack to escape plot elements.

Today, women, non ordinary white people, tatted up people, gay, etc... all
have concerns related to law enforcement. This character kind of sums them all
up in a brief future looking way.

------
pbreit
One reason why Apple is stepping up on this now?

------
hughsoon
What better way to sell a fictitious article to Hacker News than as a
cyberpunk piece?

------
jensen123
I don't think most people would have a problem with this kind of thing, just
like they don't have any problem with giving up their privacy since "they have
nothing to hide". I hope I'm wrong.

------
citizensixteen
Interesting discussion idea, poorly written article.

~~~
rkangel
You aren't going to persude anybody to your point of view if you make a bald
statement without any justification.

I actually quite enjoyed the writing.

~~~
StavrosK
To be fair, you've provided equally little justification for liking the
article. I don't see why only negative opinions should need justification.

~~~
scottishfiction
You could argue that grandparent's criticism was presented as the more
objective 'poorly written', which suggests a lack of skill on the author's
part. Whereas the parent seems to address the style of writing, which is more
subjective and less demanding of citation.

~~~
StavrosK
Yes, but that doesn't seem like a very strong point. "I found the article
poorly written" would likely also be equally objectionable.

