
Drone Uses AI and 11,500 Crashes to Learn How to Fly - kasbah
http://spectrum.ieee.org/automaton/robotics/drones/drone-uses-ai-and-11500-crashes-to-learn-how-to-fly
======
cr0sh
This is interesting. It really shows how deep learning has become almost
"Lego-like".

1\. Imagine the problem, add a camera or two (or more).

2\. Build/use a pre-trained ImageNet model as a starting point (probably using
TensorFlow/Keras).

3\. Build a dataset, split it into test, train, validation sets.

4\. Train the model further.

5\. Test and validate the model. Lower the error rate (don't overfit though!).

6\. Profit?

As far as what language to use, depending on the speed of whatever you're
trying to do, Python would likely work fine in the majority of cases. If you
need more than that, C/C++ is around the corner.

Oh - and OpenCV or some other vision library will probably be used (but just
to grab the images, maybe a little pre-processing).

You wouldn't have to use this exact pipeline (you could substitute other deep
learning libs, other vision libs, other languages, etc) - but the basics are
to start with a well-known CNN model, preferably "pre-trained", then apply
your own dataset(s) to the task to get it to work better. Not much more
tweaking needed, the biggest thing is to get (or be able to synthesize from
what you do have) enough data to throw at it (and have a fast enough system to
train it in reasonable time).

We've seen this approach many, many times; it seems to work well for a ton of
domains and problems. Again - very "Lego-like"...

~~~
thearn4
It concerns me a bit that at some point, there are fundamentals concerning
simple yet effective approaches that may be forgotten (or a least not taught)
in favor of black box neural nets, like Kalman filtering, PID control, etc.

Institutional memory is something of a weak point in the software industry
compared with other fields.

~~~
markvdb
Similar "institutional memory loss" happened in mechanics. On many occasions,
it's not so much the theory as the fluency putting it in practice that might
be lost...

------
flaviuspopan
"...success is simply a continual failure to crash"

My programming methodology has been validated at last!

~~~
marsRoverDev
"There is an art to flying, or rather a knack. The knack lies in learning how
to throw yourself at the ground and miss. Clearly, it is this second part, the
missing, that presents the difficulties."

~~~
backtoyoujim
That describes my view of the wingsuit flying to a T.

~~~
itsdevlin
Wingsuit instructor here. Can confirm.

~~~
alanfalcon
Is there a more stressful job than instructor for an extreme sport with such a
high risk of death or serious injury? (I literally cannot imagine, though I
hope I at least overstate the risk in my mental model.)

~~~
jdietrich
Jumping out of a plane wearing a wingsuit is reasonably safe. If something
goes wrong, you deploy your parachute. If your parachute fails, you deploy
your reserve. You have plenty of time to control with a spin or stall.

Jumping off a rooftop or a mountain wearing a wingsuit is practically
suicidal. If something goes wrong, you die. The margin of error is simply too
small.

The latter form of wingsuit flying is relatively new and highly controversial,
even within the wingsuit and BASE jumping communities.

~~~
itsdevlin
Off of a rooftop _would_ be practically suicidal. Wingsuits need time to
inflate and start flying, which for the best guys on the planet is around 300
feet. Normal humans require about 400. 'Margin' is a word that has a bunch of
different contexts, most of which still put wingsuit base in a reasonably safe
range. Terrain flying, which is the sub-discipline of wingsuit base where
you're goalposting trees, is indisputably the most dangerous sport on the
planet, and I've lost six friends to it in the last year.

I don't know that i'd describe it as 'controversial' but would rather describe
it as 'that thing a bunch of people with nowhere near enough experience or
currency to be doing it keep doing and fucking killing themselves.'

------
soared
Similar to the rc rally car that learns to drift by driving and crashing
itself [1]. I'm collecting articles/videos where you can see machines teaching
themselves things like flying/driving/video games [2].

[1] [http://spectrum.ieee.org/cars-that-
think/transportation/self...](http://spectrum.ieee.org/cars-that-
think/transportation/self-driving/autonomous-mini-rally-car-teaches-itself-to-
powerslide)

[2]
[https://www.reddit.com/r/WatchMachinesLearn/](https://www.reddit.com/r/WatchMachinesLearn/)

~~~
intrasight
I know there are lots of rules with racecars. Is "can't have AI" one of them?

~~~
averagewall
There's a driverless race. I'm doubtful if it'll be as powerful a force as the
real world race to the market that has a bigger prize.

[https://electrek.co/2015/11/30/formula-e-will-launch-a-
new-r...](https://electrek.co/2015/11/30/formula-e-will-launch-a-new-race-
series-with-driverless-electric-cars-starting-in-2016/)

------
stupidcar
I know people have trained AIs to do more sophisticated things than this, but
something about watching from the drones perspective as it scans its
environment and moves around really makes you feel like you're watching a real
intelligence at work.

~~~
notahacker
It's certainly cool to watch and in area of research which is proving
fruitful, but isn't ascribing true intelligence to it just anthropomorphising
the point of view of the camera? When I watch an insect essentially using
trial and error to try to find its way towards the light on the other side of
an open window, _intelligence_ isn't the first thing that springs to mind

~~~
sondr3
I find that in many cases the goalpost of what is intelligence moves further
away each time we get somewhere close to what was previously considered
intelligence. If you asked someone you could train a computer to fly on it's
own and learn from its mistakes, say, ten years ago, they would probably be
more inclined to say it's intelligent than we are.

------
Clanan
This is how Uber is going to train it's self-driving cars, isn't it?

~~~
mhb
It's how they're training their business.

------
rep_movsd
Would it not be cheaper and faster to simulate a drone and fly it through
virtual 3d environments and still learn?

Or would the physics be too complex to model well for simulation?

~~~
ef4
It's not that the physics can't be modeled, but figuring out the appropriate
models is probably harder than it looks. Every sensor and actuator has quirks.
Does it matter how much your airframe flexes? Does the turbulence caused by
some little protrusion matter enough to model? You aren't going to model every
individual molecule, but how much detail is enough to be right?

The elegant thing about using machine learning is that you don't need to build
any models at all. And you can develop the ML technique once and then reuse it
to train different hardware configurations, instead of incurring the cost of
modeling every one.

~~~
taneq
One way around that is to make small random variations in the simulation
(sensor calibration, vehicle performance etc.) when you generate the training
data, so that your system learns to drive a generic vehicle rather than a very
specific one.

------
gene-h
I wonder how well the learned policy generalizes to other environments. Places
like an art gallery, outside, or a cave. Could the network have learned
something fundamental about monocular vision?

It would also be interesting to see if the learned policy corrects for
perturbations. If we tilt the drone by hitting it, will the policy stabilize
it again?

While this is a really cool result, I suspect that this approach might not be
the best way to control UAVs. Dragon flies are ready to fly, avoid obstacles,
perch on stuff, hunt down prey right after warming up their wings for the
first time. This implies that a good amount of the flight behavior is 'hard-
coded.'

Although I really can't wait until someone expands upon this approach. So
instead of outputting left or right, the network could output 'stick vectors,'
which translate to control stick commands. Maybe even have the network take in
some sensor data and a 'move in this direction' vector. Add in a pinch of
sufficiently fast video processing and we could probably learn how to do fly
through an FPV course or do aggressive maneuvers to fly through someone's
windows[0]

[0][https://www.youtube.com/watch?v=MvRTALJp8DM](https://www.youtube.com/watch?v=MvRTALJp8DM)

~~~
jon-wood
> If we tilt the drone by hitting it, will the policy stabilize it again?

My understanding of the way this is being done is that the output from the
machine learning model is already a simple "left", "right", "straight on", so
it's not really responsible for stabilization anyway.

That side of things is likely being handled by the drone's control software
which takes those inputs, translates those into what angle the propellers need
to be at to achieve it, and then translates that into the correct rotor
speeds. If you hit the drone the gyroscope will pick up that it's at the wrong
inclination, feed that information into the control software, and the control
software will adjust rotor speeds to correct.

------
rcarmo
This reminded me of the Douglas Adams books, in which Arthur Dent eventually
learns how to fly by "throwing himself at the ground and missing".

Also, the flight had an almost organic quality to it somehow. Spooky, but
cool.

------
vanjoe
What determines autonomous flying? Couldn't the drone just hover in the middle
of the room and not crash? Would that count? Don't you need some sort of
increased score for moving around?

~~~
stupidcar
In the video it shows that the AI is only able to do one of three things: "go
left", "go forward", and "go right". It can't choose to hover in place because
that isn't an intent it can express. Although I don't know why it couldn't
just keep going around in tight circles?

Funnily enough, if you've ever implemented a Pacman clone, this is also how
the ghost AI works.

~~~
WillKirkby
Fun fact: all of the Pacman ghosts actually behave differently:

[http://www.gamasutra.com/view/feature/3938/the_pacman_dossie...](http://www.gamasutra.com/view/feature/3938/the_pacman_dossier.php?print=1)

------
peter303
Imagine if Google taught their self-driving cars that way: 10,000 crashes. I
think self-driving was completely procedural back when it started 15 years
ago. But now with faster and better understood neural nets, some parts like
recognizing objects have been replaced by deep learning.

------
fredley
> There is an art to flying, or rather a knack. The knack lies in learning how
> to throw yourself at the ground and miss. ... Clearly, it is this second
> part, the missing, that presents the difficulties.

------
udkl
Why would they not use a general purpose classifier [1] instead ?

Sure, the tagging of objects in the field of view in this model may be
unnecessary but you leverage an existing model that should allow the drone to
'think' beyond the current limited "obstruction here". It could at-least have
been used as a base model to build upon.

[1] [https://www.youtube.com/watch?v=_wXHR-
lad-Q](https://www.youtube.com/watch?v=_wXHR-lad-Q)

Personally, I'm also looking forward to neural networks modeled after real
brains [2] .... but the tech to accurately scan the complex interconnections
in larger brains seems far away.

[2] [http://www.smithsonianmag.com/smart-news/weve-put-worms-
mind...](http://www.smithsonianmag.com/smart-news/weve-put-worms-mind-lego-
robot-body-180953399/)

~~~
mifeng
Why would an object classifier matter?

If my plane is about to crash, do I really care whether it's a mountain or a
building?

~~~
MrQuincle
You would have really cool behaviour.

\+ Not going through glass (although humans do not always exceed in that
themselves)

\+ Going through a fly door curtain

\+ Going through smoke

\+ Not going through a mosquito net.

\+ Not going through a fountain.

\+ Pushing through soft objects.

\+ Pushing a half open door.

~~~
zild3d
The method used would also end up with many of these behaviors. If it didn't
crash, that behavior is enforced. So flying through smoke would not be avoided
if it didn't crash

------
bluetwo
Does it really predict 'weather' to move forward or not?

I think you mean 'whether'.

~~~
blorenz
Does it really 'more' forward or not?

I think you mean 'move'.

~~~
bluetwo
Fixed. Good thing I'm not published in IEEE.

~~~
caio1982
You need a few doses of
[https://twitter.com/respectfulmemes](https://twitter.com/respectfulmemes) I
think

------
lenkite
Its a pity that it was programmed to split for a decision only between left
and right images. It could have avoided the chairs by flying higher if there
were top and bottom images. Ideally, the number of decision-point images
should be the area of the FOV divided by the drones forward surface area.

------
folli
Cool! Do I understand correctly, that the splitting into the part where the
drone was doing fine and the part where the drone is crashing (i.e. the
annotation of the dataset) was still done manually?

A similar approach using unsupervised learning would be even cooler...

------
otto_ortega
Pretty cool.

Why did they use an input that does NOT provide any information about
depth/distance from objects?

~~~
sqeaky
I can only guess because I am not on their team, but:

They clearly didn't need it. Human pilots clearly don't need it.

Extra custom sensors might produce more noise than they are worth.

Image processor is a hot topic of research in CNNs.

~~~
otto_ortega
Except humans derive distance/depth information from their sight (depth
perception), that's why we have a pair of eyes and not a single eye.

The equivalent on this setting would be adding a second camera.

I'm not criticizing, their experiment is pretty cool, I was just wondering why
they chose to use only the camera on board.

~~~
sqeaky
> that's why we have a pair of eyes and not a single eye.

This is really overstated. It really only matters for about 3 or 4 meters of
distance. We do depth perception well enough at distances to drive cars.

We also do just fine at perceiving depth in video games and single lens
camera.

How often do you have trouble determining the depth of something in a movie?
Only about as often as the filmmakers want you to.

------
lottin
Somebody should do a cost-benefit analysis of all this machine learning
business. For instance, how much did this project cost and what did they get
in return? I'm not suggesting it's not worth it, just curious to know how the
numbers turn out.

~~~
rch
Also, IIRC early neural net research covered this approach with Hannibal &
Attila decades ago.

Edit: I guess there are some differences -- I think this is what I was
remembering:

[http://people.csail.mit.edu/brooks/papers/AIM-1091.pdf](http://people.csail.mit.edu/brooks/papers/AIM-1091.pdf)

...but maybe there was further work that is more closely related.

------
artursapek
Like watching a baby learn to walk! We're truly in an exciting age for
technology.

~~~
logicallee
Yep! And the best thing is that nothing could possibly go wrong and there is
no conceivable universe in which humanity comes to regret the exact way we're
building intelligent machines right now. :)

~~~
hinkley
Oh come on. If that were even remotely possible, there would be books about
it. Maybe even movies. That would make a good summer blockbuster.

Luckily, that possibility is too remote even for Hollywood.

~~~
justinpombrio
Regrets about building small autonomous drones that later end up killing
people? That was a Black Mirror episode (albeit not the best episode):

[https://en.wikipedia.org/wiki/Hated_in_the_Nation_(Black_Mir...](https://en.wikipedia.org/wiki/Hated_in_the_Nation_\(Black_Mirror\))

------
bluetwo
That's silly. It took me no more than 1,000 crashes to learn to fly my drone.

~~~
averagewall
But hiring you to fly their drone doesn't scale. You're not nearly as good at
replicating your ability as a computer is, or at working for free.

~~~
bluetwo
True, but I was kidding.

------
Para2016
Pretty cool!

I was curious, in the article it mentions difficulties navigating through
glass environments, could they combine visual information with sonar to avoid
crashing into glass and other transparent barriers?

~~~
gene-h
Now the question is, do human quadcopter pilots have the same problems running
into glass? Although birds tend to be bad at handling 'glass environments'

------
jayeshsalvi
Drone Uses AI and 11,500 _casualties_ to learn How to _kill terrorists_ ...
coming soon /s

~~~
mooresedge
After 10,000 civilian casualties, it finally got a terrorist.

------
sumoboy
Learn to fly scenarios vs how tesla would build an autonomous drone is my
question for something viable.

------
return0
Cant they use a simulation to do the training and then finetune it using the
poor drone?

~~~
jahewson
They're training on video so their simulator would have to produce thousands
of hours of realistic video. On top of simulating the aerodynamics and
performance of the done itself. All to produce something which is going to be
inferior to real-world data.

~~~
amelius
Human pilots also train using simulators first.

~~~
yathern
Well because if a human crashes a plane, that human doesn't stick around long
enough to learn from that experience and do better next time.

Also the cost of a human crashing a plane is a bit more than a drone crashing
itself, such that it's probably better to save on planes, and invest in
simulators - whereas developing an accurate physics simulator for the purpose
of training a drone might take more time/money than just letting it crash, and
figure it out itself.

------
rodionos
A coding drone uses AI and 11,500 bug reports to learn how to code. Are we
there yet?

------
monksy
In other words, they're just learning not to crash/failing to crash.

------
kevin2r
what happens if after trained we move the drone to another location, will
those learned "abilities" be reused? making easier to fly at the new location.

------
sc0tfree
Oh, great. Next step: weaponization and AI target analysis.

------
sheeshkebab
Pretty close to 10000 hours to get good at something?

~~~
udkl
I don't really believe in the 10,000 hours number. Granted I've never read the
book but it 'feels' faux.

The amount of time needed to master is more a combination of deep practice,
motivation to master & the individual learning rate.

'The talent code' talks about deep practice. Deep practice is the 80 in the
80/20 principle.

~~~
peter303
Some essays have refuted this hypothesis. Some people have natural talent,
i.e. prodigies, which may become proficient in far shorter time.

------
tormeh
I think this is how babies learn to walk.

------
contingencies
This really is the beginnings of SkyNet.

------
albanium
I don't understand why it didn't learn to fly; using a flight simulator.

