
What Self-Driving Cars See - natejackdev
https://www.nytimes.com/2017/05/25/automobiles/wheels/lidar-self-driving-cars.html
======
acomjean
I walk/drive/bike in a city. I'm wondering if we'll ever get to fully
autonomous.

The thing is I'm often looking at other drivers and when driving pedestrians
look at me. You can send them across a crosswalk you are stopped at in a car
with a wave or a tilt of your head. You can tell that person isn't going to
cross the street and is waiting for a ride because they are looking at their
phone and then cars. Sometimes pedestrians wave me though when I'm in my car.
I had a situation where an adult waved me across at an intersection but her
kids kept going. I stopped, kids stopped after mom told them to and we were
back to square one of our intersection dance. Things that sensors are going to
have a hard time doing.

I think we'll get driver assisted as sensors can look 360 and further ahead.
highways and less dense areas seem ideal for self driving.

~~~
rockinghigh
Self driving on highways is already working. There are solutions for
interacting with periastrians. Mercedes had a prototype where a virtual cross
walk was displayed in front of the car to show it was waiting for the
pedestrian.

~~~
Piskvorrr
#1 cause of pedestrian injury right there: car in lane 1 stops and {driver
waves at pedestrians/car displays crosswalk}; pedestrians start walking; car
in lane 2 doesn't stop.

A protocol incompatibility issue? Oh well, can't make an omelette without
killing a few people :-/ In other words, this requires precise cooperation
from _multiple_ parties involved; one fancy car implementing this is
insufficient.

------
theprop
1\. There are supposedly new lidar systems coming out in the hundreds of
dollars.

2\. You can be "superhuman" with just image sensors (i.e. without lidar).

That said, I believe lidar is effective for precise object determination e.g.
whether a small thing on the road is soft or hard which would help determine
if it's a stuffed animal or a real one or a plastic bag in the shape of an
animal.

~~~
MrQuincle
What is difficult for lidar are the following:

* not-reflective / dark surfaces, i.e. [http://www.thedrive.com/tech/9293/luminar-says-its-lidar-tec...](http://www.thedrive.com/tech/9293/luminar-says-its-lidar-tech-will-help-self-driving-cars-see-farther) has 10 percent reflectivity at 50 meters. As stated in the article Luminar is able to do the same at 200 meters, but violating eye safety standards. We might introduce paint restrictions - [http://blog.caranddriver.com/why-better-paint-coatings-are-c...](http://blog.caranddriver.com/why-better-paint-coatings-are-critical-for-autonomous-cars/) \- but it's not really acceptable to paint everything I think. :-)

* fog and dust particles (already known since lidar use for aerial imaging).

* interference with other cars not so much (much worse with radar).

I don't know if soft/hard objects are so easy to recognize for lidar though!
That's pretty challenging!

------
friedman23
I really do not believe lidar is going to be the right answer for self driving
cars. We already know that you can be successful at driving a car in various
weather conditions with two stereoscopic cameras on a swivel and limited sonar
(humans). Cameras, infrared cameras, and sonar will likely be much cheaper and
just as capable as lidar.

~~~
irq11
If you think humans are "two stereoscopic cameras and a limited sonar", you
need to learn more about humans.

Not only are our eyes far more versatile than the most sophisticated camera
equipment, we constantly maintain a mental model of the world that allows us
to focus only on the things that matter, and accurately infer state. This
affects everything from vision, to hearing, to proprioception and situational
awareness.

Machines don't come close, which is why the quality of the sensors matters so
much more. This trope gets repeated in nearly every thread about self-driving
cars, and reveals mainly that engineers don't know much about biology.

~~~
chrismcb
OK, but we developed or mental model of the world through those two cameras. I
agree that we still have aways to go, the fact is only two cameras and
processing is all that is needed. But we can do better with more sensors

~~~
argonaut
If a company's solution to self driving cars with just two cameras requires
developing a machine learning "model of the world" (I don't think it does, but
it does make it a much harder research problem), then they are going to be
years behind everyone else in shipping a self-driving car.

~~~
harperlee
If a company's solution is able to maintain a real-time model of the world on
top of which reasoning and reaction at human-level speeds is possible, never
mind the driving cars - that's priceless!

------
DonHopkins
Will a bunch of self driving cars all using LIDAR interfere with each other?

Busy intersections could have strategically placed high resolution LIDAR "base
stations" that wirelessly transmit the model to cars as they near the
intersection.

~~~
deevolution
That sounds like a centralized approach to the problem. Which ive thought
about, too. If the government provided more infrastructure to assist
driverless cars it would probably accelerate adoption and the installation and
maintenance costs of LIDAR turrets and other infrastructure would ideally be
offset by the number of lives saved. Im also wondering if a something similar
to air traffic control could be applied to self driving cars.

~~~
DonHopkins
If you really want to decentralize, take down all the "DON'T WALK" signals,
and let pedestrians use a smartphone app to tell them when to cross the road.

"It's like Uber, but for crossing the road!"

Hans Monderman would have loved it!

[https://en.wikipedia.org/wiki/Hans_Monderman](https://en.wikipedia.org/wiki/Hans_Monderman)

------
jweir
Reflections. How many self driving cars can see these? Every day that is what
I have to look for when leaving a garage - the reflections, or shadows, of
people and pets, as I come out of a parking garage.

The edge cases for self driving cars are massive - unless we dictate our roads
and street to be machine friendly.

~~~
frodeopdahl
I don't know the exact state of recognition used in the state-of-the-art self-
driving cars, but I wouldn't you be able to handle reflections if you use
normal camera image recognition combined with LADAR? From a simple image, you
can infer the distance to an observed object by looking at the size of its
bounding box, and compare that to what the size of a bounding box of that type
of object is at some known distance. After that, you can validate this
distance using LADAR. If there are large discrepancies, you can either infer
that it is a reflection, or you can maybe have trained a system specifically
to classify if something is a reflection or not. That system would probably
look at how much distortion there is on an object, compared to the rest of the
scene, which could be caused by e.g. a non-flat reflective surface.

Like I said this is just some thinking out loud on my part, and I am by no
means an expert. Does anyone that are more knowledgeable about this topic know
if this strategy could be feasable?

~~~
argonaut
I'm not an expert either, but what if the object-size pair is not in your
database of known object-size-distances.

~~~
frodeopdahl
I would assume that you would have this data about every object that the
system can recognize. It would just be trained like any other system would be
trained, because you can use the ladar for finding out the real distance. It,
of course, will be difficult to guess the distance on objects the main system
haven't been trained for.

~~~
AstralStorm
The interesting part is that distance is not really needed, only good estimate
of presence or absence of an object and a real easy model of how a mirror
works.

------
candiodari
This video convinced me that self-driving cars are definitely going to beat
humans at safety. Freaky how early the thing gets that there's going to be a
crash and the distance at which it responds:

[https://www.youtube.com/watch?v=WZ-d9k6JFA8](https://www.youtube.com/watch?v=WZ-d9k6JFA8)

~~~
joering2
Can Lidar distinquish between an empty plastic white bag dancing on the wind
in the middle of the highway, and a baby that fell out of a stroller?

In both scenarios whether it can or cannot, its a distaster.

~~~
tbabb
I don't know why this is getting downvoted. That's on of the harder problems
in vehicle autonomy.

~~~
joering2
I wouldn't worry much. I have a few "fans" that follow my comments regularly
and mass down-vote me. I also realized lately that my own upvote/downvote does
not get accounted for.

------
deepnotderp
LIDAR is quite interesting. If there's anyone who has more information about
automotive lidar, I'd love to chat with them, my email is visible on my tag :)

It'll be interesting to see how SDC makers handle the computational complexity
of deep nets operating on these massive point clouds generated from LIDAR. It
seems like these aren't getting any easier with Luminar claiming ~10 million
points. Running that through a hefty 3D ConvNet could easily soak up a
petaflop...

~~~
argonaut
To my limited knowledge, they don't. Deep nets are mostly used by the self-
driving companies for processing camera data for image detection and
segmentation tasks like detecting other vehicles, recognizing street signs,
etc. Presumably, having 3D point clouds lets you hand-code solutions to a lot
of these tasks without having to use ML.

~~~
deepnotderp
That's been my experience as well, I was just wondering if anything has
changed in the year or so that's passed.

------
cmurf
For as much as Americans complain about regulations, there's very little
standardization in automobile driving environment (rules of the road, road
paint and signage) or in the driver (uncle Joe can teach you how to drive in a
few weeks, and essentially once licensed you're licensed for life as long as
you pay a fee and don't become legally blind). Anyone who drives in the U.S.
and either is a pilot or has driven in another industrialized country knows
how non-standard automobile driving is in the U.S. This might make us fairly
defensive and adaptive drivers, because our fellow drivers are so
unpredictable due to non-standardization.

I think that makes the task of integrating autonomous cars a difficult task,
not for the human driver, but for the computer. It has to be more adaptive and
responsive with a lower error rate than the best American driver, while
simultaneously sharing the exact same physical environment with non-
deterministic actors.

Once we solve the pollution problem, all the HOV lanes can become autonomous
car lanes. Only qualified cars use those lanes and they must be in autonomous
mode. And conversely in mixed zones they can only be in some kind of hybrid
mode. How insurance sorts out liability in the hybrid case will be
interesting.

~~~
Piskvorrr
For all the non-regulation of US roadspace, I don't see any significantly
_different_ driving in Europe: having driven enough on both continents, it
feels the same w/r/t unpredictable drivers. IMHO it doesn't matter whether the
unpredictability comes from "has no training" or "has training but ignores
it".

------
ge96
Is there a reason most lidars still use the rotating design and not phased
array (faster).

\- cost \- redundancy \- depth

Just a thought, not a scientist/engineer

edit: then imagine it could be made into strips and that goes around the
car... no rotating-delay and full-view all the time.

------
kleiba
Lidar is pretty much my favorite animal. Bred for its skills in magic.

------
nerfhammer
Does any of this work in rain or snow?

~~~
keerthiko
LIDAR is supposed to have low attenuation in water, however it will still
suffer refraction. Hopefully it will function about as well as a human. The
vision processor will have to understand that it's raining and correct for
refraction where possible.

Snow is a different challenge, but a sufficiently smart system should be able
to composite frames and appropriately fill in the holes created by being
obscured by snow. It won't be trivial, but it should all be doable.

------
omarforgotpwd
It sounds to me like they are discounting the possibility that an entirely
camera based solution could be developed that bypassed the whole lidar cost
issue altogether.

------
wnevets
> You can’t afford to miss a single object because that object could be a
> person.

Humans do all the time and we accept that risk, why are self driving cars held
to a higher standard?

~~~
D-Coder
It's an argument, IMHO not a very good one.

It's basically the same as "but they don't cure cancer!" argument.

------
ruleabidinguser
Sorry, I know very little about this stuff, but how does LIDAR avoid
interference with other cars emitting LIDAR?

~~~
brians
Coded pulses: rather than emitting a uniform beam, they emit a pulse and
listen for that back. You can think of it a little like a bar code, a sequence
of short and long, on and off. Then select for that on receive.

~~~
rubicon33
what happens if another source uses the same pattern?

~~~
DaiPlusPlus
LIDAR units could cycle through randomly-generated patterns for each pulse -
if they can be made unpredictable then it would be impossible for someone to
confuse a LIDAR unit by copying its pulses. Assuming it's a time-of-flight
LIDAR then the time required for an attacker to read a pulse and send back
their own signal would be greater than the time of the original signal to
return - and if the sending LIDAR has a sufficiently short timeout then even
an attacker using Vanta-black to absorb all of the victim's lasers still won't
be able to respond quickly enough.

