
FPV with Oculus Rift and a Quadcopter - arijit91
https://github.com/Matsemann/oculus-fpv
======
matsemann
Wow, cool that someone is sharing our project! It's for a course at my
university called "experts in teams" where they combine master students from
many programs in teams and give them tasks. A video with more shoots is
available[1].

We got UAVs for the Norwegian oil industry as a task, and explored how they
can be flown better in the future. Other teams with the same task made other
cool stuff as well, for instance a custom drone for geological mapping. [2]

[1]:
[https://www.youtube.com/watch?v=ANSjwWomIJ8](https://www.youtube.com/watch?v=ANSjwWomIJ8)
[2]:
[https://www.youtube.com/watch?v=5di01L1mot8](https://www.youtube.com/watch?v=5di01L1mot8)

~~~
mabbo
As one commenter on youtube said: "Shut up and take my money".

But seriously, have you guys considered commercializing this? You're offering
people the ability to experience flight. Not just a video of flight, but
controlled flying in their own neighbourhood.

~~~
jodrellblank
Here's one from a year ago - FPV system with QuadCopter, commercialised
because people were asking for 'affordable' versions:
[http://www.youtube.com/watch?v=aWPrf4pw6V8](http://www.youtube.com/watch?v=aWPrf4pw6V8)

It's not Oculus Rift based, but it is FPV flight.

Slightly off toic, NB. You need to also be a bit careful with flying
regulations, as radio controlled aircraft are still regulated. In the UK for
example, flying an FPV radio controlled aircraft can't be done on your own,
you must have a competent observer keeping it in line of sight at all times
and watching for risk of collisions, so no flying it behind buildings or
hills. You must also not fly within 50 meters of any building or vehicle you
don't own, or above any congested or crowded area, or takeoff/land within 50
meters of other people. So where you say "experience flight in their
neighbourhood" \- they might not be allowed to:

[http://www.fpvuk.org/fpv-law/](http://www.fpvuk.org/fpv-law/)

[http://www.caa.co.uk/docs/33/ORS4%20number%20956.pdf](http://www.caa.co.uk/docs/33/ORS4%20number%20956.pdf)

~~~
zobzu
theres hd flights from fpv machines from the day youtube started. ppl have
being doing this for decades in analog and a good 10 years in HD. its not like
if one year ago was a special feat or something.

its having the oculus vr with it which is cool, with analog cameras. the
previous attempt had a big laptop, webcams and wifi.

------
baddox
This idea has been buzzing around the FPV community since the Oculus Rift was
first announced. There have been a few barebones setups, but this is the first
fully-fledged implementation I've seen.

It's a fun usage for VR goggles and a great build project, but it's honestly
not super useful, even for hobby flying. Any quadcopter you're flying with a
head tracker is going to be so far away from other objects that the 3D effect
will be minimal. The 3D might be great for zipping quickly through
trees/obstacles like a pod racer [0], but for that you don't dare use head
tracking.

[0] [http://youtu.be/xlKrabm5Exg](http://youtu.be/xlKrabm5Exg)

~~~
pedrocr
They're not controlling the quad with head tracking, just the camera
orientation. So for "pod racing" this should work making it more realistic
both because of the 3D effect and the fact that you can turn your head like
you would if you were sitting inside it.

~~~
baddox
I know. It would still be disastrous to be able to turn your head when doing
fast moves like this. To make it work there would need to be something in the
frame of the camera, or overlaid on the video, to indicate the center of the
frame, because orientation is extremely important.

~~~
pedrocr
Yeah, having a HUD in a fixed orientation relative to the quad would be
awesome.

------
ohadron
Youtube here:
[https://www.youtube.com/watch?v=ANSjwWomIJ8](https://www.youtube.com/watch?v=ANSjwWomIJ8)

Very cool idea. I could think of some indoor applications for it.

For outdoors, since the distance between the cameras and the object they're
looking at is so far, and the distance between the two eyes is so close the
difference between the two images is almost meaningless. I'd put the two
cameras further apart and see what happens.

~~~
bigiain
I remember looking through this thing:
[http://eyestilts.com/intro.html](http://eyestilts.com/intro.html) at
Burningman many years ago - it makes things look very un-real - much like
tilt-shift photos. I suspect it might become pretty disorienting to fly using
such odd optics...

~~~
omegant
The lens in this device are pretty close to the looking subject. If you have
cameras with a wider separation on a drone they'll be farther asay from the
subjects and the effect will be less extreme, they are also very difficult to
mount with 3 meters of separation.

~~~
baddox
Use two GPS guided quadcopters, each with their own camera. Problem solved,
and now you can dynamically vary the intensity of the depth perception.

~~~
bigiain
"Problem solved", says - I suspect - the guy who's never tried to make even
_one_ quad copter go where he wanted, never mind a pair of them controlled to
optically-accurate positions! ;-)

------
montag
Awesome project. Have you thought about using a spherical camera, so that the
pan/tilt can be done entirely in software? Make it a bandwidth problem,
instead of a mechanical problem.

~~~
sopooneo
I think one major issue there is that all of the available video transmitters
for FPV transmit in standard definition. You'd have a hard time spreading that
over a sphere and being able to coble anything useful together.

Most of the FPV videos you see on youtube are _not_ showing what the pilot saw
in real time. They're showing you the HD that was recording on board the plane
and retrieved after landing.

~~~
xeeton
You don't really have to send the entire video feed from the camera, only the
part your looking at. That way, most of the "software" is running remotely,
and you also don't have a mechanical component to worry about.

------
mambodog
As it appears to be wireless I would imagine the latency between head movement
to camera movement and then video capture to video output would likely be
enough to make users feel ill.

~~~
ohadron
Wireless 2.4ghz radio latency is measured in milliseconds which is quick
enough not to notice. Latency also depends on how quick the gimbal servos are
but that too may be quick enough.

~~~
cvbncvbncgbc
Since the movement and change of speed of the copter isn't very fast you can
apply time warp to this; or just make the camera film in 120 fps.

------
DanHulton
This is going to be a job someday.

People are going to train for this.

Very likely police, field agents, military, but there are absolutely going to
be people whose sole jobs will be piloting drones like this.

(Until those people are replaced by AI algorithms, but still.)

------
leoc
Apparently telepresence like this was the original application of HMDs (like
the Philco Headsight); the concept was only generalised into VR (and AR) by
Ivan Sutherland a few years later.

Somewhat related:
[https://www.youtube.com/watch?v=SQ2tCMXOd_w](https://www.youtube.com/watch?v=SQ2tCMXOd_w)
[https://share.oculusvr.com/app/hiyoshi-
jump](https://share.oculusvr.com/app/hiyoshi-jump)

------
gcr
I want to turn this into a sport.

Imagine this with some augmented reality thing where multiple contestants
could shoot each others' copters with an IR laser or something.

Heck yes.

~~~
spullara
This game exists for the Parrot AR drones.
[http://singularityhub.com/2010/06/17/parrots-ar-drone-
gets-c...](http://singularityhub.com/2010/06/17/parrots-ar-drone-gets-cool-
new-augmented-reality-games-video/)

------
elliottkember
I'd love to know how well this works. I always assumed the pan and tilt setup
would be too slow for the Rift.

~~~
maaaats
The video latency is low enough that you don't notice. We tried with moving
our hand in front of it, it feels weird too see yourself real-time!

The rotations of the cameras are a bit too slow though, but mainly because of
our setup, so this can be a lot better. Our custom protocol couldn't really
handle dropped packets, so if we filled the sender's or receiver's buffers,
the servos would start to spasm. To counter that we just didn't send head
position as often as we could.

Of course, by fixing that you still get the round-trip time of turning your
head -> moving the cameras -> getting updated image back. But we think it's
feasible.

~~~
elliottkember
I do a bit of FPV myself so I know what you mean about the real-time! The
round-trip is what I meant - especially with all the work to get the latency
down on the Rift.

I've always thought fixed, super-wide-angle lenses and software would be the
ideal way to go - the groundstation does all the work.

~~~
maaaats
Yeah, it's been suggested to us a few times after people have seen the
project. Someone should definitely explore that way as well.

------
higherpurpose
Looks like the cameras could use some good OIS.

~~~
baddox
Good old fashioned vibration dampening would go a long way. That's a big area
of concern for all FPV pilots, and it's fairly easy to get good results. The
ultimate solution would be to also add a brushless gimbal for stabilization as
well as low reaction latency:

[http://youtu.be/Dr5CnIA52zY](http://youtu.be/Dr5CnIA52zY)

------
SkyDrone
Great to see more projects using the Oculus Rift for FPV. We are using it in
out Sky Drone FPV system.

------
spiritplumber
[http://spiritplumber.deviantart.com/art/dancers-in-the-
dark-...](http://spiritplumber.deviantart.com/art/dancers-in-the-
dark-390629203) Here, I wrote you a little story.

~~~
spiritplumber
Geez, why the hate? It's a story about FPV quadcopters dancing.

