

Wayland 3D Compositor with Oculus Rift and Razer Hydra [video] - jmgrosen
https://www.youtube.com/watch?v=Dgtba_GpG-U

======
noonespecial
I love the idea of coding on a giant virtual wraparound screen but I'm not
sure even a DK2 is going to be up to it. Think about it: The text in your
windows can't look "fuzzy" (antialiased or rendered) at _all_. It really has
to be "pixel perfect". We go nuts around here over fonts that get this or that
pixel in just the right place on a given serif.

If you doubt it, try setting your monitor to something other than its native
resolution and coding for a while using the interpolated resolution. I'm
afraid text in floating 3d windows on vr sets is going to look like that for a
long time to come.

I'm still going to try, dammit.

~~~
akurilin
Luckily these goggles won't stay at the current resolution for long, we should
be seeing 4k and higher in the next few years.

------
jmgrosen
(No, I'm not the author.)

Code: [https://github.com/evil0sheep/qtwayland-motorcar-
compositor](https://github.com/evil0sheep/qtwayland-motorcar-compositor)

Thesis:
[https://github.com/evil0sheep/MastersThesis/raw/master/thesi...](https://github.com/evil0sheep/MastersThesis/raw/master/thesis.pdf)

Defense Slides:
[https://docs.google.com/presentation/d/1svgGMxxbfmcHy_KuS5Q9...](https://docs.google.com/presentation/d/1svgGMxxbfmcHy_KuS5Q9hah8PQOsXqvjBKOoMIzW24Y/edit?pli=1)

------
wmf
Heh. Reminds me of
[http://www.blendogames.com/qc/](http://www.blendogames.com/qc/)
[http://en.wikipedia.org/wiki/Project_Looking_Glass](http://en.wikipedia.org/wiki/Project_Looking_Glass)
[http://www.nngroup.com/articles/2d-is-better-
than-3d/](http://www.nngroup.com/articles/2d-is-better-than-3d/)

------
jonhohle
This is really cool. I ordered a DK2 to try out some similar things in a
similar domain. The idea of a 120° workspace which can be expanded
indefinitely seems like a reasonable alternative to getting bunches of
physical screens on a desk.

~~~
XorNot
This is also 100% the reason I ordered one.

I've spent quite a deal of time thinking about how you make a 3D interface
actually work for usability, and its interesting to see some of those ideas
implemented in this example (allowing 3D views to either pop out in the
workspace, or be pushed back into their own world).

------
malandrew
The beauty of this approach is that it frees up window managers from managing
desktop spaces and instead allows you to manage windows and the relation of
the windows relative to other windows. Being freed of borders and boundaries
presents amazing opportunities. Defining things in absolute space is very
confining. Defining things in relativistic space, gives you the opportunity to
move into multi-view/multi-screen modes with multiple participants much more
easily. It's not longer about desktops, but about taskspaces.

I don't want to say "put window A on the lefthand side of the desktop and put
window B on the righthand side of the desktop." Instead I want to be able to
say "put A to the left of B in when mode X" or "put C on top of D when
performing task Y."

It's simply much more natural and more intuitive to tell the computer where
your focus should be (or if the focus should be split equally between 2 or
more windows), and then defining all the peripheral windows that should appear
around the focused window. Ideally these wouldn't even separate windows, but
simply a rectangular stream of pixels from the buffers of your choice, so that
the same application could be multiplexed to layout modes. This would
basically be tmux for windows/apps in 3D space.

A prolog-like declarative language with relations and predicates for
describing windows and how they are organized relative to one another under
variable conditions would be awesome. Type one command to switch to the window
you want to have focus in the center of your screen. Once that window is in
place, bring into focus the last used "mode" for that window, with all the
peripheral windows coming in from behind you or zooming in from the back (time
machine style, but with your primary focused window held constant). Then you
can activate switching between modes with another key stroke combination.

------
zan2434
This is pretty cool as far as tech goes, but also seems rather ridiculous
practically. Productivity with a HMD will likely amount to a lot more than the
same 2D dynamic images we already have(screens) being placed arbitrarily in 3D
space. What will it look like instead? one may ask, to which I have no
definitive response.

What we have here seems like an extrapolation of what we have now, which may
prove fruitful if we have nothing else to go off of (and is the nature of
incremental progress). If I had many more monitors I'd place a class (or any
other abstraction) on every screen, so maybe every class will have its own 2D
plane of text and they'll be organized semantically & dynamically in 3D space.
Also resolution here needs to be really really high. Idk, just interesting
food for thought.

~~~
radiac
As you say, it's an interesting tech demo (which looks like it would be even
more fun if you coupled it with a physics engine btw), but not particularly
practical.

The way I could see it working though (without a significant change in OS UI)
is by arranging your 2D desktop across multiple virtual 2D screens in 3D
space, ie a bank of monitors angled towards you. You could use a conventional
keyboard and mouse to navigate the 2D space (with perhaps mouse movement
assisted or replaced by hand/head/eye tracking), and drag windows between
screens, set programs to open on certain ones, with full screen clipping to
screen edges - all basically as many OSes do already.

But because they're virtual, you could resize, split and merge screens, curve
large ones, have ones which float in front of your vision to use for toolbar
windows and HUDs. You could even use existing technology like VMs or VNC to
use multiple OSes at the same time.

Mount your keyboard and mouse to a swivel chair which tilts, and you could
comfortably have a 360 degree working area (assuming you can touch type) 4 or
so screens deep, which would put traders' setups to shame (and at a fraction
of the price). Throw in some panoramic backgrounds and you can move your
office to a forest or a beach. Take it further you could even set up some sort
of virtual office, being able to virtually get up from your desk and visit the
workspaces of remote-working colleagues and friends to collaborate.

The exciting thing is that none of this is a massive leap from where we are
now, and it would map pretty seamlessly to the software and UI we already use.
The main thing holding it back would seem to be the resolution of 3D goggles,
but with 4k rising, even that can't be too far off.

------
akurilin
It'd be fantastic to have an Oculus-based desktop environment! Imagine not
having to drag giant monitors around. Combine it with xmonad and you're
golden. I hope this becomes reality soon, at least for development purposes.

Shut up and someone take my money already.

~~~
Qantourisc
Drop xmonad, what you want is no-longer the virtual desktops, but the virtual
rooms! (As a bonus it has been proved humans memory interact with entering and
leaving rooms.) We will be hanging applications on the walls like pictures.

~~~
XorNot
I always thought what you need is instead a kind of body model.

So you have things which are attached to your head, things which follow
position relative to your body, and things which follow your absolute position
in the world.

Most people would work in body-space, and maybe have several different
overlays they switch between for that.

On top of that, you probably need the concept of the private, shared and
public objects in all these spaces - you can interact or allow people into a
limited view of your 3D environment.

------
mentos
I feel like jumping straight into floating resizable screens is too great a
leap.

I think we have to start off by simulating a desk and two fixed size monitors
so the user understands the metaphor, nail that experience and then evolve
from there.

~~~
GhotiFish
I'm _sure_ The Onion has some mockups.

------
shmerl
Another 3D Wayland compositor example:
[https://www.youtube.com/watch?v=_FjuPn7MXMs](https://www.youtube.com/watch?v=_FjuPn7MXMs)

------
dsplatonov
nice one. It reminds me The Lawnmower Man movie. Soon, you will be able to
physically dive into the code.

------
XorNot
The one effect I really really want is when you switch from 2D -> 3D mode with
the Goggles I kind of want the 2D desktop to do the laser activation effect
from Tron Legacy.

------
joeevans
I hope the author ports this to something else than Oculus, since I bailed on
it when it sold out.

~~~
spullara
The next version is up for pre-order.
[http://www.oculusvr.com](http://www.oculusvr.com)

~~~
XorNot
What he means is he's one of those people who absolutely can't stand Facebook
but for some reason still buys gasoline.

~~~
joeevans
Actually... I don't!

