Neat, I didn't know for Ghost in the Shell, I'll be watching it in few weeks then! Thanks for sharing.
PS: so weird to watch the trailer while working in XR. I never imagined as a kid I'd be programming in a headset but now it's banal. I even buy 2nd hand HMDs for 100 bucks. Weird times.
+1 to this, I’ve never heard of someone actually doing that. I was hopeful the Vision Pro would become that device but Apple seems to have decided they don’t care about the future.
I clarified a bit in a longer post. I do actually like the Vision Pro... but even though I own one I barely use it. It's a good device but it's just too many hoops for what I do. I find a much affordable Quest, even older generations, with adb and a proper WebXR support (including with AR support, which Apple refuses to implement) fits better my need. I have 0 problem popping a Quest 2 in my backpack without even thinking about it, only protection being a literally piece of cardboard on the lenses. I can work with it anywhere and give demos to total stranger with no worries about them dropping it or heck even running away with it.
FWIW I'm also not in the Apple ecosystem so all the wall garden hoops just tire me. I want the device I own to let me do my work.
I find that, probably counter-intuitively, super high resolution and refresh rate matter... but only up to a point. As I mentioned in the longer post if you are "just" looking for a monitor with another form factor, sure, maybe it's OK-ish, but to me it's very uninteresting and not worth all the challenges, including social, of having an HMD strapped to your face. Only when it's properly immersive does it shine, and then resolution and refresh rate don't have to be that high to be usable.
It seems that Apple doesn't care about a device for an audience of nerds/gamers that pretty much no one else is is interested in--given the current technology.
Ah my bad I probably created false hopes here but let me clarify :
I program in VR for VR, and I help kids (and adults) learn programming thanks to VR.
What I do NOT do is writing my bash scripts or things unrelated to VR in VR. I technically can but I just have so much tooling and habits outside of VR it wouldn't make sense.
I'll answer your question as the end, you can skip to the last paragraph if you do not need the context.
Still do program in VR I can explain a bit more, or even show with a 1min video https://video.benetou.fr/w/iuJUP5oNnRAfH2xC3AW9s4 . Basically you can see ~20s that using the left pinch I can execute code in VR. Those specific code snippets are part of triad with querySelector() to get an element, setAttribute with it's attribute name then the last the value, e.g. here querySelector('a-sky').setAttribute('color', 'black') which sets the sky color to black. The idea here is to make a small change with a big impact.
If you are not a developer, it feels pretty cool (from the feedback I received) but if you are a developer there are 3 things you can notice :
- this is all done in a browser, as you can see from the beginning of the video, despite being properly immersive. I'm relying on eval() to provide a no-install no-build programming environment
- the black plane on the right is the browser console, literally. There is some visual feedback with green for console.log() yellow for console.warn() and red for console.error()
- the black plane on the left (hard to notice, ~34s in) is the code of the page
- the code snippets can be combined
- there is some syntax highlighting in order to hint that different attributes and there values are not compatible, e.g. scale black does not make sense or color 2 2 2 is also not correct
- the modifications are saved server side (but can also be done solely on device via local storage) so that others can build on the modifications you saved
and things you can't notice
- if you plug a keyboard, bluetooth or just USB-C, you can type code directly, it will become floating text snippet
- there is page reload code snippet at your feet, if you mess up you can recover from a proper state
- there is a non immersive console via eruda that is richer and expandable in case your changes break immersive mode, you still have a way to try to recover
- you can self-host that immersive Web on the device, it's just a Web server with HTTPS, if you want work properly offline, which is good when traveling, e.g. train or planes (that's how I do it on the go)
So... yes IMHO it's a good way to program in VR, today. It's also a good way to learn programming. If you program in VR chances are you quickly get frustrated by having to put the headset, work, hit a bug then remove the headset, try again, etc. That cycle can be frustrating so trying to find ways to do the spatial things in VR makes sense. The non spatial work though, e.g. backend, that can be properly tested outside of VR, IME should not be done in the headset, even when it can be done. Of course it depends on your workflow.
Finally which is a good headset depends on what kind of programming you want to do. If it's nothing spatial and you "just" want a 2D screen ... honestly I don't think it's worth it. I did try a lot of headsets. I worked with Varjo and Quest Pro, have a Vision Pro, Lynx XR1, Quest 3, Quest 2, Quest 1, Oculus Go, Valve Index, etc. I never spend 8hrs/day in any of those. It's short sessions or while on the move. I don't expect any headset to be "good enough" compared to a basic 4K 32" monitor, ever. What I also don't expect is any monitor, no matter how fancy (and same here, I worked with Looking Glass 8K, ~15k EUR monitor) to provide any proper sense of depth supporting manipulation. So if your programming work involve 3D work... then VR headsets are amazing. If so which one to buy? Depends on your programming language. If you are into JavaScript then any of the Quest will do but the latest generation (Quest 3) has color passthrough and higher resolution. It's also Android based so you can put termux to setup NodeJS, Python, etc and serve the content as notebooks. If you are into the Apple wall garden then obviously the Vision Pro but you'll have to go through all the usual hoops to publish anything.
TL;DR: high res with WebXR support and adb (or equivalent), e.g. Quest 3 (ideally rooted but that's hard to come by)
A note on programming languages : you can program with any language in VR.
There are few languages that make it easier, e.g. JavaScript as in my case, or any other language with a VR library, e.g. Lua LÖVR or also tooling like Godot which does have an in-VR live reloading mode (sorry discussed in French only at https://video.benetou.fr/w/g111LSRGfj6HmNxa91au3q ).
If your favorite language isn't very convenient for VR you can get a WASM build via e.g. QEMU. If that's not practical you can get the heavy load in a container then communicate to it via ssh or a Web API.
My point being that you can program in any language in VR quite conveniently and the power of the device itself is typically not relevant. For example I started some simulation for swarm robotics from the HMD to explore the result in VR but the runs themselves where not done on the device.
TL;DR: your HMD can host everything or can be a thin terminal to your heavier workload.
I did try the Valve Frame ... and I'll give recommendation only when it's officially out, but I can freely repeat what everybody else already said, the ergonomics are on point. You basically forget you are wearing an HMD. Sadly AFAIK they won't officially support WebXR and their focus, which is fair, is on their own store, Steam. So I wouldn't wait on it unfortunately.
PS: so weird to watch the trailer while working in XR. I never imagined as a kid I'd be programming in a headset but now it's banal. I even buy 2nd hand HMDs for 100 bucks. Weird times.