
Perceptiv’s Shift (YC W15) Turns Drones into Smart Dollies for Filmmakers - uptown
http://blog.ycombinator.com/perceptivs-shift-yc-w15-turns-drones-into-smart-dollies-for-filmmakers
======
Animats
That's a very nice control system. Smooth tracking, and no visible jitter in
the shot. Tracking reliability will be the big issue; losing track on the hero
during a shot is unacceptable.

I wonder how they do tracking. The tracker in OpenCV isn't that good, but then
it's about fifteen years old.

Movie production will soon be using fewer Louma cranes and manned helicopters.

They'll eventually need to put an ADS-B transponder (there's a 100g unit
available for drones) on the thing and get the procedures in place with the
FAA so productions can use this in areas near airports. That makes the drone
radar-visible to air traffic control and other aircraft.

~~~
pj_mukh
Founder here :). Thanks for the great comments! OpenCV has very rudimentary
tracking capabilities. This is why we went back to the drawing board and
developed something that is robust in a wide number of cases including in busy
environments or where saliency is not high.

You are correct about tracking reliability, however, most of our early
customers were more worried about creative control and smoothness. So, we've
worked hard to make sure they get that level of control while not having to
worry about the minutiae of controlling a drone/gimbal.

In cases of occlusions or odd situations where it may lose track of the actor,
they are more than happy to quickly re-initiate tracking.

------
abbasmehdi
Heck yes! Alas not a filmmaker but I have wanted something very nearly like
this forever. I go horseback riding regularly, mostly through relatively open
areas (county parks / regional preserves / city trails etc). I have wanted a
drone that can carry a GoPro (or something) to capture footage of our rides
(sometimes we go through some gorgeous country that we really wish we could
capture on video).

However, here is the kicker, because we ride at a fast trot or canter, I can't
actively control the quad. Instead, I must be able to passively control it. It
must perform specific, pre-defined actions based on my position, speed, and
heading while flying, steering, and maneuvering without receiving any input
directly from me. It should continually adjust its position and flight path
based on my speed according to the rules I pre-define. For example, after
takeoff with the camera turned on, the drone ought to stay ahead of me, say 30
feet ahead and 20 feet off the ground. Then if I start moving faster than 10
mph it should come to my side about 30° (10 o'clock). If I continue at this
speed for more than a minute it should do a slow semi circle around me and
come to the 2 o'clock point for a minute. If I keep going it should... etc
etc. I don't care if I have to program the drone in code or through a UI, I
just want to be able to define lots and lots of rules (based on the terrain of
the day). Of course it would need to protect itself from running into trees
and walls, or any other objects while trying to follow my rules as closely as
possible.

You have my attention.

~~~
pj_mukh
Wow. This is great information! You have a very well laid our requirement :).

"Of course it would need to protect itself from running into trees and walls,
or any other objects while trying to follow my rules as closely as possible."

And that is the kicker. A lot of the features you've outlined can be done with
current technologies simply using GPS (though framing would be wrong unless
you have a SHIFT-like product). However, none of it is safe to use. Even if
you were in "mostly" open areas, to constantly watch out and make sure the
drone doesn't crash is very cumbersome.

Comprehensive sense-and-avoid is the golden egg of the drone community,
precisely because of the features you've mentioned!

------
zk00006
Nice application! Based on what I have seen so far, I am pretty sure they are
using a tracker based on correlation filters [1]. These are very fast and
robust. Naive implementation does not provide scale, which is visible in the
video. but this is rather trivial to do. OpenCV 3.0 has also implemented this
MOSSE [2] tracker.

Overall, these trackers are not able to learn and will always fail in same
situation. It would be great if they had re-detection capability and improve
with every touch from the operator.

[1]
[http://home.isr.uc.pt/~henriques/circulant/](http://home.isr.uc.pt/~henriques/circulant/)
[2]
[http://www.cs.colostate.edu/~bolme/publications/Bolme2010Tra...](http://www.cs.colostate.edu/~bolme/publications/Bolme2010Tracking.pdf)

~~~
pj_mukh
Wow! The author of TLD, that's quite the honor :). You are right about a lot
of the issues we've been facing. However, our focus is squarely on getting
tracking results VERY fast, so we can get smooth operation from our
controllers. We are working hard to incorporate some global re-detection (and
scale), however, we are balancing this with running the algorithm fast online,
onboard the drone.

I'd love to talk and bounce more ideas off of you! Ping me,
pj(at)perceptivlabs.com

------
Jack000
I wonder if this is based on OpenCV or if it's a proprietary system. Looking
at the demo it could be using CAMShift with Kalman control?

My experience with OpenCV is that object tracking depends heavily on hue
differences between the tracked object and the background - eg. I start out
tracking my face, and wave my hand in front of my face, the tracker then
starts to track my hand instead. Have you found a good way to mitigate this
issue?

~~~
pj_mukh
Very good question! a) We've found that OpenCV's boilerplate algorithms have
much bigger problems than the ones you have mentioned in deployment scenarios.
So, we've had to develop something better from the ground up (while still
using the OpenCV infrastructure). b) For the tracker to go back to tracking
your face, it would require global re-detection which is usually a much slower
process. Moreover, the tracker would have to know what your face looks like
under different lighting conditions and angles. This is something we are
investing in heavily. Convolutional neural networks have shown a lot of
promise!

------
outericky
This is fantastic. I've been wanting to see something like this!

I'd love to see a partnership with GoPro and get these into live events (once
GoPro can do live feeds). I'd love me some Red Bull Rampage, or Supercross
footage.

Supercross has this spider cam concept they started this year, but it still
requires a camera on wires zigzagging the stadium, this would solve the issue.
if you can get live footage out.

awesome stuff!

~~~
yanma
Perceptiv founder here. Thanks for the support! Would love to talk to GoPro
but we haven't been able to do that, so please talk to us if anyone has leads
:) Great idea about live streaming! We're definitely trying to figure out how
to do that.

------
jsnk
Filmmakers?

This is law enforcement and military's wet dream. Chasing suspects or hostile
force in an urban setting. Possibilities are endless.

~~~
pj_mukh
Hahah! I'm sure it is, but it is far less interesting to us. To know that
someone is creating fantastic works of art using something you've made is much
more rewarding :).

------
dba7dba
Never thought we could possibly have self flying video camera drones that I
used to see in animation films, in real life...

------
anigbrowl
Any prospect of prototype availability for an upcoming project?

~~~
pj_mukh
Hey! We can definitely talk about this. We do have early prototypes. Ping me,
pj(at)perceptivlabs.com

~~~
anigbrowl
OK, I'll be in touch this week. Thanks!

------
Anisa_Mirza
amazing product and kickass team

