
Show HN: Hacking on a VR desktop inside of a VR desktop [video] - ddevault
https://spacepub.space/videos/watch/f60bee0e-31d3-4aca-9e49-6fcdc87ad40d
======
daeken
This is awesome to see. I've been working on something similar: a remote
Wayland compositor for the Oculus Quest, Magic Leap, and other AR/VR devices.
Last night I hit a milestone, getting a terminal running on another machine to
render inside Unity, then on the Magic Leap itself.
[https://twitter.com/daeken/status/1205762441285685248](https://twitter.com/daeken/status/1205762441285685248)

I really believe that 2020 is going to be the year of VR/AR productivity, and
I'm committing myself to working full-time from within the environment.

~~~
dvt
I love your enthusiasm so you got my upvote :) With that said, it's hard for
me to see _why_ some people are so bullish on VR/AR productivity. The headset
seems heavy, taking it off-and-on is cumbersome, AR fidelity seems like it
leaves a lot to be desired, etc.

My sister has a VR headset, and so does my cousin. I think it's a cool toy,
but I'm not really sold... maybe you can convert me. After all, Christmas is
just around the corner.

~~~
daeken
So, I've been working much of the time (30-50%) through the Quest, using
ImmersedVR to have 3 screens from my Mac in VR. The headset is super
comfortable, even for long periods of time; the only downside comfort-wise is
that if I'm breathing out of my nose, it'll end up blowing upwards into my
eyes, drying them out (but I'm going to be making a little buffer piece to fix
that).

The fidelity is at the low end of what's usable for productivity without
causing discomfort, but it's just going to get better from here. To me, $400
for the Quest is a total no-brainer; the Magic Leap .... not so much. I'd
rather stick entirely to VR, but I have a kid to watch much of the time, so AR
it is!

~~~
paulryanrogers
> ...but it's just going to get better from here.

Ever increasing improvements are not guaranteed. My hope is it will improve
too, and ideally be done with a lower carbon footprint. But it's still an
unknown.

~~~
daeken
Ever-increasing improvements aren't required; the eye has a (more or less)
fixed resolution. In fact, HMDs with extremely high resolutions already exist,
e.g. the Varjo VR-2 Pro and the XR-1, they just cost an arm and a leg. In a
few years we'll get them at consumer-level prices instead of having to drop
$6-12k.

------
mschuetz
Somewhat similar project I've been working on: Live Coding in JavaScript and
VR [https://github.com/m-schuetz/Fenek](https://github.com/m-schuetz/Fenek)

Video:
[https://www.youtube.com/watch?v=xhGNhKVAqW0](https://www.youtube.com/watch?v=xhGNhKVAqW0)

This allows you to modify the code of the VR render engine directly at runtime
while you're in VR.

~~~
gfodor
You might be interested in RiftSketch:

[https://github.com/brianpeiris/RiftSketch](https://github.com/brianpeiris/RiftSketch)

~~~
mschuetz
I am! RiftSketch is great. We just rolled our own because we needed OpenGL 4.5
and access to visual studio code within the VR window. So our live coding
framework is a native desktop app that uses V8 to bind javascript calls to C++
and OpenGL 4.5, and the windows desktop mirror API to render the desktop, and
visual studio code along with it, in VR. Apart from the C++ core, virtually
all aspects of the application can be modified at runtime by modifying the JS
files and simply saving them. JS files are executed on save, and structured in
a way so that they replace previously loaded functions with new ones.

------
shafyy
Working in VR is a cool glimpse of the future. I also sometimes do it with
Immersed VR (similar to VR Desktop, but works with Mac). I say glimpse,
because I still kind of need to "force" myself to do it.

There are some advantages (like the big screen and more focus), but today the
disadvantage will overweigh for most people (friction of putting on the
headset and connecting to your computer over WiFi, the weight of the headset,
lower resolution, starting to sweat over time and leaving marks on your face,
etc.).

However, these are very solvable engineering problms and I'm exteremly
optimistic that working in VR will be better than looking at a "small" flat
screen sooner than most people realize.

------
Trisell
This excites me. I’ve wanted to build a renderer like this for a while. The
current display model is outdated. We no longer have a reason to be limited to
a monitor. Either VR or AR is the visual path forward to provide us with a
better work environment as we continue to become more and more of a digital
workforce.

~~~
cjbprime
Unfortunately the current display model also has several times more resolution
than any available VR headset.

~~~
gpm
That will change though - or at least resolution in both will increase to the
point where it doesn't matter.

~~~
theptip
Exactly, I believe I read a Carmack article saying that you need 4K pixels per
eye to get to a resolution where pixels are imperceptible (though presumably
those with better than 20/20 would need higher).

That’s probably a few years away, but I’d be surprised if we don’t get there
in 5-10.

Meanwhile, we need to build all the ui and OS code to drive such displays.

That said I’m not sure I’d sign up for using the current generation as my
primary display device.

------
mncharity
Nice, but...

Resolution is a bottleneck. Notice only 30-ish lines are visible here during
editing. How many do you normally use? And VR videos can mislead, as inside a
VR HMD, only the center ~1/3 region is as clear as the video, with pixels
blurring together as you glance further away.

With a custom stack and subpixel rendering one can squeeze out almost a
hundred lines on some consumer VR HMDs, but it's non-trivial and not a happy
thing.

Terminals in spaaaaace... remember skeuomorphic user interface design? Cell
phones were new, so to ease on-boarding novice untrained users, things were
made to resemble familiar real-world objects. A calendar app might perhaps
show lined paper, flipped around a spiral binding, with a textured leather
surround. We don't do that now - it would be silly. Because resembling the
physical world is so unlikely to align with desired UI and domain affordances
and tradeoffs. Windows tumbling in 3-space? For professionals and experienced
users in AR/VR, how long before resembling "R"eality is design smell?

So I've used drone googles in preference to VR HMDs, for 1080p of mostly
readable pixels. Shallow 3D rather than virtual world, so long hours in fixed-
focus HMDs isn't eye pain. Arbitrary non-physical mapping from head and hand
tracking to input, because good ergonomics is using a touch pad, not something
silly like constantly lifting hand from keyboard to tap a laptop screen.
Arbitrary non-physical "space" and "physics", because slavishly emulating the
oppressive constraints of the physical world would be... just why? Intensively
avoid immersion. And so on. Arguably not vR. But is "R" really the best
possible professional use of AR/VR hardware?

Nreal light 1080p AR glasses recently became available for $1.2k dev kit
preorder. Which is months late, but recent PR is still quoting $500 "early
2020" consumer availability. We'll see if pixel blur and alignment is good
enough to render a small font from corner to corner. If so... I hope to be
using my laptop screen rather less soon. But not to see windows floating in
spaaaaaace.

EDIT: Oops, I just noticed this was a ShowHN, rather than a generic post. So
on a more upbeat note... it's great to see the envelope being pushed, with
non-toy real extended demos, inspiring people, and encouraging them to reflect
on the very non-trivial changes which are inbound. And PeerTube worked pretty
well.

------
C14L
This will be very useful when traveling as a digital nomad (again). Instead of
being limited to a small laptop screen, just use as many screens as needed,
with no extra luggage.

~~~
leoc
One thing that I'm not sure is generally understood is that Samsung etc. are
advancing folding-screen phones, and MS is jumping back into phones with a
dual-screen model, for just the same reason that Apple is apparently working
hard on AR—and the reason isn't simply to have something new to sell
smartphone users. It's because the usability of the smartphone as an all-round
personal computing device is now massively bottlenecked by its small screen,
and if you simply make that screen bigger then as soon as you go beyond
phablet size you simply end up with a tablet, too bulky to carry all the time
in your trousers pocket. And when the alternative is an overly-small screen
too close to the eyes then (present and near-future) AR/VR drawbacks like low
pixel density are not so bad in comparison. (One snag here is the bulk of the
AR HMD itself: we'll have to see what Apple has done on that front.) Laptops
have that problem to a lesser extent, being unequivocally on one side of the
screen-too-small/device-not-ultraportable tradeoff, but AR/VR is going to
displace (or at least supplement) the laptop clamshell screen too, and it will
eventually come for desktop monitors as well.

------
moron4hire
Since others are sharing their VR productivity apps, I will do the same:

I wrote the first WebVR framework in the world. With it, I built a very simple
live-programming environment:
[https://primrosevr.com/demos/editorVR.html](https://primrosevr.com/demos/editorVR.html)

That was 2, almost 3 years ago now. I'm no longer working in WebXR, but I am
still working in VR productivity. I'm currently working on an app for language
and cultural learning in VR.

Unfortunately, there is nothing public to see, yet. Though we did just give
our first semi-public demo last week: [https://dlsdc.com/blog/dls-hosts-tenth-
annual-open-house/](https://dlsdc.com/blog/dls-hosts-tenth-annual-open-house/)

------
jv22222
It’s a good experiment but from a productivity point of view, I don’t get how
it’s better than a regular desktop (especially considering you have to wear
those big goggles!).

But maybe that wasn’t the point.

Anyway, nice to see it in action.

~~~
ehnto
Imagine if program windows weren't constrained to the flat planes of your
monitor, but could be anywhere in your field of view at your desk. That's the
dream, and this is a stepping stone I think. Soon enough the VR headset will
be slim and ergonomic, seamlessly combining reality and the virtual workspace
so you don't feel separated (if that's what you want).

------
postpawl
I don’t think this would work very well on the Oculus Rift S. I’ve tried using
BigScreen to navigate webpages and it’s just not that easy to read normal
sized text. Maybe it’s better on the Valve Index?

~~~
daeken
I've been working on the Oculus Quest (which has the same resolution as the
Index) and it's the lowest resolution I find functional. In my case, it's just
barely good enough to use for long periods of time, without absolutely killing
my eyes. Pretty excited to see what the next generation looks like.

------
trevordev
Cool! I love seeing tools like this. I've also done a bit of research into VR
desktop experiences
[https://twitter.com/trevorjbaron/status/1197043776197251074](https://twitter.com/trevorjbaron/status/1197043776197251074)
mainly built using webGL + other web tech and running on Quest.

------
haolez
This brings us one step closer to Heavy Rain's AR desktop:
[https://youtu.be/SsQT3mbvVWY](https://youtu.be/SsQT3mbvVWY)

I wonder if it's possible to get the same outcome of the game using such thin
glasses, though. The current VR headsets look a little ridiculous to me :)

------
cjbprime
How cool! I remember Mozilla announced some work on improving text rendering
under 3D transformations, do you think that could be integrated?
[https://github.com/servo/pathfinder](https://github.com/servo/pathfinder)

------
corysama
Awesome!

There’s a quiet little subreddit devotee to this idea
[https://www.reddit.com/r/HMDprogramming/](https://www.reddit.com/r/HMDprogramming/)
They’d really appreciate this.

~~~
gpm
I went ahead and posted it there... hope you don't mind Drew

[https://old.reddit.com/r/HMDprogramming/comments/eandil/hack...](https://old.reddit.com/r/HMDprogramming/comments/eandil/hacking_on_a_vr_desktop_inside_of_a_vr_desktop/)

------
drudru11
The text looks really good in the terminal in the video. Is the display as
good as the video capture?

Also, what is the hardware used for the video card and HMD?

~~~
zhynn
It's because it is huge. Compared to a normal desktop monitor, he is looking
at like 30 point text. If you were to render text as the same size as a normal
monitor, it would be illegible on a vive (which is what it looks like he is
using). The thing is, when you have near infinite surface area, having big
text is no big deal.

~~~
drusepth
It'd be interesting to see how the eye damage/strain caused by reading small
text on a big monitor compares to the eye damage/strain caused by reading big
text on monitors inches away from your eyes.

------
Exuma
Can someone explain the appeal of japanese pop music? I mean this in an
entirely non-sarcastic way, and I'm looking for a serious answer.

It can't be the lyrics unless you know japanese... maybe it's just me but I
don't find the melody appealing at all (even from a more general POV where I
acknowledge the wide range of melodies from country to classical to death
metal).

Is there some part more to it that goes beyond melody or lyrics? I'd say
something similar about extremely hardcore death metal which is essentially
just noise, but I imagine to someone who has that personality type it's almost
cathartic, so even that I can understand the appeal, but not japanese pop
music, especially while coding lol

~~~
justtopostthis3
_Not_ understanding the lyrics can be part of the appeal, because it makes
them less distracting.

Understanding a little can make embarrassingly cheesy and saccharine pop
lyrics take on a profound poetic quality.

Growing up on anime and Japanese video games is probably a big contributor to
finding the melodies appealing.

Also, dude's got a giant concert video in view pretty much constantly, so
image is probably part of it.

hope this helps

~~~
Exuma
Ok thanks, the japanese video games + anime answer puts it into perspective, I
hadn't thought of that. I could see how growing up with either of those would
mean listening to that music then would bring good memories or feelings back.

~~~
ddevault
It's not nostalgia, I didn't grow up with Japanese culture. And it's not that
I can't understand it - I can. I just like the music. Japan has the second
largest music industry in the world after the US. They produce a huge, diverse
group of interesting music which stands fine on its own merits.

