
How Google's Self-Driving Car Works - ma2rten
http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/how-google-self-driving-car-works/
======
revelation
You may giggle at Googles attempts to catch up with Facebook, but looking at
this video.. boy, they are way ahead. Google isn't so much a search company,
its an innovation company. They are already building the next trillion dollar
industry while Facebook is optimizing sharing cat pictures.

~~~
kamaal
>> _They are already building the next trillion dollar industry while Facebook
is optimizing sharing cat pictures._

What makes you think that sharing cat pictures isn't a trillion dollar
industry?

Last time I check photo sharing apps and bird-in-pig games fetch and sell for
more money than life saving cancer drugs.

------
omegant
It is normal for pedestrians to look for eye contact in order to know if a
driver will yield the right of way or simply will accelerate and pass (in
light less crosswalks). In dense pedestrian cities (I am from Spain) it is
very usual to have some guessing and false start stop between the pedestrian
and the driver, nobody seems to know who is going to yield finally. I wonder
if this is been thought and how you'll signal the car's intention to a
pedestrian?, maybe a forward braking light?. It could be that automatic cars
will be much better yielders than a human driver.

~~~
pilgrim689
It makes sense for the self-driving car to yield to pedestrians all the time.
So if you see that there is no driver, you assume it will stop until you are
out of its way.

There will be no more awkward social guessing games!

~~~
omegant
There is a second part on it. How will cars behave when they are driving along
a crowded street in a place where people crosses anywhere and changing
directions and intentions very fast. I don't think there will be dangers to
pedestrian. But it's going to be a tought problem avoiding false emergency
breaking. Once more you have to rely on visual contact to know what are the
real intentions of people. Crossing, waiting for the car to pass, simply
unaware of the car coming, going to pick something that fell at the border of
the street but no intention to cross. Not an easy task!

Also it is strange they are not working on a kind of transponder, that could
be used between nearby cars to coordinate maneuvers(maybe I missed it on this
article, but I think I read about somebody was developing it) . It will be
usefull once robocars are more common.

~~~
solutionyogi
My guess is that originally self driving cars will be mainly used on highways
and take you to the metro hubs. Once you reach a very populated metro area,
you will use the local public transportation.

~~~
Joeri
They would probably BE the public transportation. We need large buses because
drivers aren't flexible enough to deploy, and you need to have enough capacity
to meet peak demand. If you can have smaller driverless vehicles, and just put
more of them on the road during peak hours, it's much more cost-effective (and
environmentally friendly) to have smaller capacity vehicles. I doubt we'll see
more than 10 people per public transit vehicle once driverless transport
becomes the norm.

~~~
njs12345
A diesel bus gets 6mpg --- you only need 10-15 people on one to match even the
most fuel-efficient of cars in terms of environmental friendliness, unless
you're referring to something else..

~~~
jiggy2011
That's assuming there is only 1 person in the car. A car that does 60mpg with
5 passengers can move people 300mpg (of course you will lose some mpg due to
the extra weight)

------
ctdonath
_Everything_ Google does feeds back, somehow, to data mining.

How does self-driving cars fit in to Google's plans?

ETA: Downvoted for asking a serious question? WTH?

~~~
Game_Ender
What do you think a bunch of people in self driving cars with smartphones or
personal wifi hotspots are going to do? Surf the internet, and therefore use
more of Google's products. It's similar to Chrome or Android in that it
enables more people to use Google's core services which they make money off
of.

~~~
ctdonath
Sensible answer, tnx.

Anyone play with the numbers? average web use vs. average AdWords profit vs.
typical commute?

------
islon
I want to believe this is the future. Maybe not from Google, maybe from other
companies, but self-driving cars are an amazing idea. Put a 100mph giant piece
of metal in the hands of humans, on the other side, is not a smart idea...

------
div
Once legislation is updated to allow self-driving cars in most parts of the
world, I wonder how profitable self-driving cars could be for Google. About 77
million cars were produced in 2010. With the appropriate licensing, a
percentage of that could make a very nice profit. I should buy some Google
stock :)

~~~
dangrossman
I don't see what Google's role is going to be, aside from pushing forward
legislation for unknown reasons. The auto manufacturers don't need them for
the technology and research... they've all had access to that for just as
long. We'd have self-driving cars today if the public (and regulators) were
ready for it.

But they're not and aren't going to be overnight; what makes more sense is to
gradually introduce the features a little at a time so that the move to
complete autonomy is a small step instead of a leap. First, automatic parking.
Then, collision detection warnings. Next, lane assist that steers you back
into the lane when you drift. In the next few years, you'll start seeing
"smart cruise control" that will drive for you on well-marked roads in mapped
areas.

"Cadillac To Release Self-Driving Cars By 2015"

[http://www.inquisitr.com/227496/cadillac-to-release-self-
dri...](http://www.inquisitr.com/227496/cadillac-to-release-self-driving-cars-
by-2015-video/)

Maybe 10-15 years from now we'll be at the complete autonomy point.

~~~
huherto
I don''t know. It is my impression that traditional companies are not able to
produce complex software. I guess there are many paradigms that you have to
break to become a software company. It seems to me that is the reason why cell
phone companies ended up buying their O.S. from google.

~~~
jacquesm
> It is my impression that traditional companies are not able to produce
> complex software.

What gave you that impression?

What do you define as a non-traditional company?

~~~
div
I think OP defines non-traditional in this instance as software companies.

While I believe there's room for other companies to develop self-driving
navigation systems, I think the Android example is pretty apt.

It's interesting to think about when self-driving cars will actually be in
stores. By then, all navigation systems will meet strict safety guidelines and
some standardised tests.

If all self-drivers are equally safe, the AI's driving style will become a
differentiating factor. It seems to me like Google has an advantage in this
type of development, although the future can certainly prove me wrong.

------
ajays
I would love to see these in a city like SF. It's a small city
(geographically), but we spend $1B/year on MUNI, which carries about 700K
passengers/day. Put together a fleet of, say, 5000 such autonomous vehicles
that take you door to door, for the price of a MUNI ticket. Wow. That would be
something.

~~~
Someone
If I do the math, you will need many more than 5000 vehicles. 700k trips with
5000 vehicles is 140 per vehicle per day, say 50 in morning traffic and 50 in
evening traffic, the other 40 in-between. Even assuming 5 people in e car at
each trip, that means 10 round trips from suburb to the office per car.

I guess you will need at least 100k vehicles, but with that number of
vehicles, that $1B per year starts to look cheap ($10k times 100k = $1B)

~~~
im3w1l
Maybe cars for the long tail routes, and hours?

------
ChuckMcM
Given the amount of stuff going on it made me skeptical of GM's claim they
would have a self driving car available for sale in 2015.

The Google stuff is pretty cool, I notice they don't show the early videos of
people running around the parking lot trying to get it to stop :-) But such
things are to be expected in development. This is a technology I look forward
to as it would simplify a lot of things.

------
cpeterso
I'd like to see some self-driving cars in a high-speed chase or race.

Google and NASCAR produced a joke video about self-driving race cars:

* Video: <http://www.nascar.com/video/none/none/120331/cup-mar-google/>

* More info: <http://www.google.com/racing/>

~~~
sp332
Already done, 2 years ago by Audi
[http://www.engadget.com/2010/11/21/autonomous-audi-tts-
scale...](http://www.engadget.com/2010/11/21/autonomous-audi-tts-scales-pikes-
peak-in-27-minute-climb/) Some competitive jockeying would be fun though!

------
daenz
> With its full 360° horizontal field of view by 26.8° vertical field of view,
> 5-15 Hz user-selectable frame rate and over 1.3 million points per second
> output rate, the HDL-64E provides all the distancing sensing data you'll
> ever need.

That's pretty interesting. I would have thought that they needed data at a
faster rate than 15fps.

~~~
JackC
That is interesting! Looking into it, the estimated response time to an
unexpected event for an average human driver is 1.9 seconds (i.e. time to
notice, look, evaluate, decide, react).[1] Of that, 50 ms or 1/20th of a
second goes to just processing the visual chemical stimulus into a usable
signal.[Awesome Dinosaur Comics link] So if the robocar is getting a frame
every 66 ms, that's probably not a significant factor making it better or
worse than a human driver -- it's certainly not the key factor to optimize. On
the plus side, the robocar is constantly looking in all directions rather than
having to refocus on unexpected events, and can send signals to the car
instantly, so it saves a couple hundred ms on each end. It probably ends up
with a good bit more time than an average human to make decisions.

Of course the robocar isn't allowed to drive like an average human. It'll have
to drive like a perfect human. Seems like that should be doable.

[1]
[http://books.google.com/books?id=YkNY6gjxnMIC&lpg=PA396](http://books.google.com/books?id=YkNY6gjxnMIC&lpg=PA396)
[Awesome Dinosaur Comics link]<http://www.qwantz.com/index.php?comic=1806>

------
yread
I wonder how long will it take till the price of something like this gets to
reasonable levels. The lidar and other sensors probably won't get cheap
anytime soon. Imagine the costs of Google having to drive through all the
places first, mapping them. Then the massive insurance until its proven that
self-driving cars are safer (and then it will probably get expensive again
after a first accident)

Perhaps it would make sense for government to step in like it did with
electrical cars.

Edit: It seems the LIDAR they use costs 75000$. :-/
[http://eole.ecam.be/claroline/backends/download.php?url=L0dv...](http://eole.ecam.be/claroline/backends/download.php?url=L0dvb2dsZV9jYXItc3VtbWFyeS5wZGY%3D&cidReset=true&cidReq=AN52)

~~~
joezydeco
Not to speak like a party-pooper, but could someone potentially mess with this
LIDAR system by aiming a very bright IR beam at the device, if not a matching
laser itself?

Bringing down the cost of the LIDAR will be one major task, the other will be
making this untouchable in the environment, IMO.

~~~
daenz
> Not to speak like a party-pooper, but could someone potentially mess with
> this LIDAR system by aiming a very bright IR beam at the device, if not a
> matching laser itself?

You could probably screw up the samples that would be taken from wherever your
beam hits on the mirrors. But you could also shine a laser in someones eyes
while they're driving and seriously impair them as well.

~~~
Qworg
You _would_ screw it up - but only that mirror. The systems are designed for
safety - the car would likely stop.

~~~
daenz
But what about reflective objects? The roadway is covered with different
degrees of weirdly-reflective surfaces. Surely those beams land in places
where they shouldn't (on the mirrors), and surely the car doesn't stop every
time.

~~~
xxbondsxx
LIDAR works _through_ reflections. It essentially paints out the surrounding
area with laser pulses and then measures the time for those pulses to come
back. Since lasers travel at the speed of light, this time-to-return is quite
small and hence they need quite accurate clocks (which is where a lot of the
expense comes in).

That being said, laser range-finding works well on diffuse surfaces; this is
because when diffuse surfaces reflect, they reflect the incoming light in a
broad hemisphere (or cone) which sends the laser pulse out in many directions.
Consequently, the surface to measure can be at a variety of angles and still
be picked up by the LIDAR sensors (the sensor doesn't care about strength,
only time-to-return)

So in terms of "weirdly reflective" surfaces out there, almost everything is
diffuse enough for LIDAR to work well. Car hoods, carbon fiber, chrome wheel
covers, etc. The only exception is glass, where generally lasers travel
straight through and don't return to the sensors. So LIDAR actually detects
"holes" in these situations, as if other cars were driving with no windshield
an all their windows down.

So the only real risk would be a large plane of glass in the middle of the
highway with completely normal road behind it. LIDAR would miss the glass, and
the cameras would not be able to see it either. Most real drivers would fail
at that too though :D

~~~
daenz
I guess what I meant was, surely there are weird surfaces that have multiple
bounces. Or light being emitted could bounce between 2 cars and back to the
sensor...or off a shop window (at a high angle, fresnel reflection), back onto
something else, and back into the sensor. This data would come back into the
sensor and it wouldn't be expected.

So surely the automated car, when it sees data it does not expect, does not
stop, because it must see data it does not expect often through multiple
bounces, right?

~~~
xxbondsxx
Sure, the situation you described certainly happens and is just considered a
general noisy measurement. The car could detect empty road one second and all
of the sudden some small object _right_ in front of the grill at the next
frame. To avoid this "freakout" situation where the car slams on the brakes
every time a noisy measurement comes in, all the data is passed through a
particle filter (or Kalman filter) first before being processed by the AI.

The transition model of the cars environment is known, so it can reason that
"there is a very small chance this reading represents a real object and is not
noise, because i did not detect anything near this position over the last 20
frames, so I'm going to assign a very low probability to it." Hence you can
clean up the data really well because you're measuring an outdoor environment,
not a meteor shower (or anything else where objects could appear and disappear
every frame due to high velocity).

------
sambeau
I wonder if they will eventually license the self-driving software in an
Android-like model?

~~~
Qworg
Doubtful. Most of these companies want to own the tech. I'm actually concerned
they won't be able to sell it if they tried - Chrysler is actively developing
a self-driving system that they hope to deploy in 2 years.

~~~
jsnell
Really? I'm no expert, but this seems like a market where it'd be natural for
there to be only a few major players, rather than every manufacturer having
their proprietary implementations. Why would you build an in-house system
rather than buy one? Generally because buying is more expensive in the long
run, because you can build a better one, or because keeping the expertise in-
house allows for more chances of integration.

The expense idea makes no sense. This is software we're talking about, so the
marginal cost is always going to be 0. It's also software that's likely going
to have to through very expensive certification procedures to be allowed on
the roads (or to be allowed by insurance companies). How could it possibly be
economical for a manufacturer with 2% market share to do this on their own?

So could a smallish manufacturer at least build a better one, if not cheaper?
It's hard to see why you'd expect that. Maybe if 20 of them tried, a couple of
them would end up with a better technology than what could be bought. But even
if that's true, it's not going to brighten the day of the remaining 18
manufacturers who ended up with substandard software. There's one obvious
exception (regional differences - maybe Japanese drivers have very different
preferences from German ones, or something).

Are there any integration benefits from making the software for self driving
cars in-house? I can't think of any, but I'm not too familiar with the
automotive industry. Maybe there's something obvious I'm missing.

Of course there's no guarantee that even if the future ends up as one of a few
major in-house systems and a couple of successful publicly available ones,
Google's solution would be one of the successful ones. Still seems like it's
worth a shot.

~~~
Qworg
There are very few standardized systems in the automotive industry, especially
for "luxury items". Recently, that's changed with some manufacturers using the
same undercarriage to build different cars. It would take a huge change in the
mindset of a car manufacturer to buy something like this that could be a
competitive advantage.

~~~
tomkarlo
Granted, but given that most of them haven't even been able to get their in-
dash software systems right (look at Ford's debacle), how likely are they to
be able to make dependable self-driving cars?

------
mflanagan
Does anyone know what languages are used to build the software brains for the
car? I haven't found any information on the actual software architecture, more
on the hardware.

~~~
eru
<http://en.wikipedia.org/wiki/DARPA_Grand_Challenge> has a bit on the
languages.

------
pbw
The technology is impressive, but humans drive okay without LIDAR, radar, or
GPS. Maybe some day self-driving cars can drive using only two small visible-
light sensors located above and behind the steering wheel. It might just be an
AI challenge to operate that way, and commercial systems will always employ
advanced sensors for better safety. But I don't think true parity has been
reached until they can drive like we do.

~~~
jonknee
But people aren't very good at driving, it would be silly to try and handicap
computers with the features we evolved. Every new car made already has lots of
advanced technology to make up for our two eyes.

~~~
RobAtticus
Agreed, I don't think we should strive for parity. I'd much rather see
driverless cars that are better than humans at driving, even if they need more
tools to do it. The more poor drivers we can replace with driverless cars, the
better.

~~~
pbw
Commercial self-driving cars should use every sensor modality which is cost
effective and which improves safety. I would never argue we should hamstring
our driverless cars, let's give them the best shot to be uber-safe and
reliable, no question.

In the mean time, on a completely non-commercial separate track, AI
researchers should try to do more with less. Driving using only visible-light
sensors is a challenge. AI is pushed forward by taking on challenges exactly
like this, let's see the push continue.

These two tracks may in fact intersect. When your LIDAR and radar are caked
with ice and mud, you'll want the car to be able to drive visually at least to
a safe stopping point.

