Since it sounded to me like the low hanging mobile VR fruit has been picked, I went ahead and pre-ordered my very first HMD afterwards. Time to see for myself if this is the computing future or an evolutionary dead end.
I had the opposite impression. He did seem to care about VR as a general purpose system (alternative to traditional monitors), even if it's not quite there yet, both due to hardware and software limitations. He also seemed proud that they were actually shipping stuff.
My impression of pessimism came from his skepticism that Moore's law would continue long enough to get them to where they really wanted to be on mobile VR, his struggles with latency probably making cloud rendering a poor experience for most, recognition that they were already at the ceiling of their heat and power budget, his belief that foveated rendering was not going to save the day, his recognition that the lenses were now a limiting factor vs the screen for image quality and that other lens designs would be fragile and suboptimal in a shipping product.
I didn't mean to say that I thought he was pessimistic about VR overall, just that he didn't seem to think there were any large gains in hardware quality coming soon and it was down to the software to make full use of what was there.
As far as the limits of mobile computation goes, I think the thing to consider is that Qualcomm is already providing Wi-Fi 6E chips that have VR-class latency (<3ms) - this is bog standard, but of there are of course companies like Amimon that have been offering proprietary ultra-low latency (<1ms) solutions for a while now. We'll see if the PS5/PSVR2 offers a wireless solution, but I don't think it's too much to ask for those looking for cutting edge experience to use untethered, local compute - why do you need to strap an RTX 3080 to your face? There's already more than enough power on mobile to provide the basic I/O for presence and engagement in VR.
While I'm less sure on foveated rendering (although I'm really impressed by DeepFovea https://research.fb.com/publications/deepfovea-neural-recons... I wonder what kind of rendering pipeline will be required), I think basic eye tracking will be coming w/ Quest 3 as well - it's not only needed for DeepFocus, but also their Codec Avatars.
The stuff that FB has been showing off the past few years is insanely impressive - I think their mixed reality approach beats the pants out of what MS has been doing. I'm really curious to see what Apple has been whittling away at in secret - I'm doubtful if it can compete, but who knows? Google seems a bit out to lunch on all this.
There's a third possibility - VR is finally here and consumer market is what it is. It's not going to take over the world but it isn't going away either.
We are far past tech demos and concept games.
There's no non-nerd out there who thinks "maybe I'll try VR", and goes into a store to buy... whatever it is, goes home to have a great time with it, and immediately tells their friends. No, it's Linus Tech Tips upgrading his water-cooled living room PC with a special USB3 card just to get enough bandwidth to all the peripherals: https://www.youtube.com/watch?v=HFuLvGf0g0c
In fact, so many people literally went to Best Buy or Walmart to purchase a Quest that it has been sold out since the beginning of the year, online and off, with eBay pricing at 50%+ premiums.
You literally open the box, run an app on your phone (for configuration), put the HMD on your face and you're in VR. People not only tell their friends, but buy multiple versions to give to friends and family.
With Quest 2 at a $299 price point, the non-nerds saying "maybe I'll try VR" will happen even more often. The OculusQuest subreddit has a constant stream of questions about this very behavior. It is currently the ~2,000th most popular subreddit, with an impressive growth curve. Some are literally buying the Quest 2 instead of the PS5 (though clearly a small number).
So far, 35 games have made over $1m on Quest alone, with one clearing that figure in 4 days and another in 8 days. Top games have made over $10m, not even including revenue from Steam, PSVR, or the Oculus PC VR platform. These aren't Epic / Apple level mega-wins, but many mobile developers would be happy with $5-10m revenue figures.
Upcoming VR games are coming from top tier publishers like Respawn (Medal of Honor), Ubisoft (Assassin's Creed, Splinter Cell), and Rockstar (unnamed AAA open world game), Crytek (The Climb 2), and EA (Star Wars: Squadrons).
Additional games include Sniper Elite, Warhammer 40,000, Star Wars: Tales from the Galaxy's Edge, Jurassic World Aftermath, and Myst. And this is just the tip of the iceberg from brands/publishers you may know.
Point being, stand-alone VR has really addressed many of your (and the market's) concerns.
And, to be fair, in your linked video Linus was trying to use optional, expensive peripherals for full-body tracking while streaming - not exactly a mass-market use case.
1. Facebook funds many of these games; think of it like a first-party game made to drive sales of a new console. The developer gets paid up front, Facebook recoups the revenue via incremental sales of HMDs and software on their store.
2. Note that many games are VR offshoots of existing IPs. This means the opportunity for asset re-use and consequently lower costs.
3. "High quality" and "big budget" are not equivalent. There is a huge spectrum between a tech demo and a flagship title with 400-500 people working on it (as is the case of AC). AAA-caliber, lengthy games can be made by teams of 50-75.
4. Again, those revenue numbers are on Quest only. PSVR is an even larger market, and PC VR (Vive, Index, WMR, Rift) is probably about the same size.
The most important point though is that if your entry point requires Cyberpunk 2077, Red Dead Redemption 2, or The Last of Us, then it is indeed too early.
However, the two hottest non-VR games right now are Fall Guys and Among Us - great, high quality games come in all sizes.
It does feel like the goalposts are being placed wherever VR isn't.
This is definitely a thing that happens with the Oculus Quest already. The hardware is easy to use and the software library is pretty much in place now. A major marketing campaign could make it really take off.
I don't have any insight into FB VR's decision to require the login on Oculus devices now, but my guess is that they consider the device to be ready for a mass market, and believe that the gains from the social graph will be worth the short-term pain of leaving early-adopter money on the table from here on.
I think the Quest showed FB that they had a potential winner on their hands and the Quest 2 is moving out of MVP phase to mass market phase. They are gambling that this is going to take off in a big way and they pushed hard to get an improved version out there and ramp up their manufacturing faster than most thought they would. They clearly want to be first to the "Oasis" with Horizon if that's ever going to be a thing and the FB login is a piece of that. If this is a flop, it will be a huge disappointment to the execs. If it's a win then FB may have been first to plant their flag in our VR future, for better or for worse.
(It's popular enough that Sony actually has made a second iteration of the headset, but it's not an upgrade.)
It's not as accessible as the Oculus Quest, but the PS4 has sold roughly 106 million units to date, and gamers are probably the biggest market at present for VR devices.
VR needs more than that.
Is there anything in VR apps/games that’s even passingly familiar to 1% of average people?
Personally, I know Myst has been ported to VR and that there’s some way to watch NBA games in VR but that’s about it.
I played too much half-life in my college days and think of it as fun campy sci-fi. Stuff like press e to move this dead body is suddenly this viscerally horrific experience.
You picked up a box of matches accidentally? Read the label. Found a pen draw something.
oh and myst is coming, MYST.
Games with that constraint are limited. You could do ping-pong. Shooting galleries.
Also cockpits seem to reduce nausea thus opening up another broad category.
It's mostly smooth first person motion with strafing and smooth turning that is problematic.
I hate that we live in a world so dominated by network effects, economies of scale, corporate consolidation, and shareholder value that there are fewer and fewer realms where "small success" still exists.
You're right that some sort of ready player one metaverse thing is probably on that level, but creating genuinely interesting, fun VR experiences that may in many ways be more interesting and unique does not to me seem beyond the scope of small scale business.
In a sense VR is one of the purest versions of that sort of cyberspace idealism, it's an escape hatch away from curated, corporate products to whatever people want to make. So I'm not pessimistic that creative people find things to built.
Real wealth is, you know... actual wealth. It's being able to start a high 6-figure to 8-figure business (or even smaller if you are satisfied) and not being laughed at as a failure. It's being able to buy rather than rent everything you want to buy. It's having productive assets.
I suspect this is a bit of a SV bubble viewpoint. I don't think that in the general population, if you tell people you are pulling 8-figures in your business, you would be considered a failure.
If by "productive assets" you mean income/revenue producing, then I'm not sure we can all have that. Many assets like that require some else to pay rent.
That you personally care more about starting a business than about using a VR headset is not really relevant.
It's not cut-and-dry bad the way we do things, but there is a lack of balance. There's a lot wrong that could be better if it wasn't someone's perverse bread and butter. Extreme wealth and power inequality, whether for people or companies, don't make for a just, healthy society.
If you want to strengthen workers, create more demand for jobs, so that workers have options. By for example creating a successful VR industry, you create demand for more jobs that haven't previously existed, thereby empowering workers. Somebody who has previously been exploited to manufacture fridges can now give their boss the finger and go manufacture chips for VR headsets instead. One easy way to create more jobs is to hire a cleaning person, by the way. Everybody who is concerned about "living wages" and things like that should start by hiring cleaners and possibly gardeners for their homes.
I personally think the whole concept of "workers" is bunk. Everybody is a free agent entering contracts with other market participants. That's it. "Worker" is a social construct, actually, a socialist construct. More so, being a "worker" does not sound like a very desirable state of affairs, so I think society should work towards getting rid of workers.
The success of mega projects (iphone, etc.) creates a massive supply chain (mostly in China) of people putting things together in different ways to make random stuff.
E.g.: http://www.arcadeguns.com/index.php?main_page=product_info&p... I was just looking at this to turn our entire wall into a shooting arcade with Aimtrak lightguns to help kill time during the winter pandemic.
This was not possible ten years ago.
As a society we need a way to determine where we invest resources. Some of those metrics you mentioned, though not perfect, are proxies for evaluating whether or not we should be investing more in certain tech. Otherwise, how would we determine where we should spend our limited resources?
For another example, to make a modern movie can cost hundreds of millions of dollars. It is only possible to make them because there are many people who will possibly watch it and pay for it. Nobody would be able to film the Lord of The Rings trilogy in their own basement just for their own entertainment. Not yet, anyway - tools of the future may make it possible ("GPT-300, make me a good movie based on the LOTR books").
It won't likely ever be as big of a platform as smartphones are, though. So I overall agree with you.
Don't forget that the current generation of VR hardware has roots back to a Kickstarter project. Billions of dollars are investment aren't needed.
Half Life: Alyx is a fantastic AAA game that in no way feels like a proof of concept. Strongly recommend playing it. (Beat Saber too)
John is known for being completely candid in his talks. It comes with the territory. It’s like, you get carmack’s mind, but you also get his mouth... I for one, would accept that. I listened to his quakecon talks about the physics of light many many times. The guy is fascinating how his mind works. But more importantly, he speaks his mind because he has the chops to back it up and ACTUALLY FIX the things he complains about.
EDIT: if he's moving towards other projects, actions speak louder than words. I didn't get that from his talk, or maybe I just missed it.
But if anything, Carmack expressed overall optimism. He said that we pretty much have everything we need to implement his ideal vision of VR. It’s just a matter of putting all the pieces together.
what is he moving onto?
So why put a limit?
So John is actually still working on the next phase of the Facebook AR project in my opinion.
Let's start with cost. Both headsets alone run around $1000 - $1200. If you build your VR ready PC, it's about $1500. If you pay Dell or HP, then it costs about $2000. We're already at $2500 minimum and we haven't even added the costs for Vive's wireless option which runs at an extra $350, so with wireless the cost is $2850 minimum; for the non-technical it could skyrocket to $3250 vs the Quest's $299.
Now let's talk about the "initial hill to climb" that most techies forget about. If you must deal with wires, then you have to do ceiling based wire control, where you mangle your room with ceiling hooks. For base stations, unless you have the perfect room with perfect furniture and you don't mind drilling into walls, it's a pain to set it up just right. It took me a lot of trial and error before I got it working flawlessly all the time. Let's also not forget that you need to install a wireless card for Vive's wireless accessory. Not everyone has both the skill and patience to deal with these time consuming annoying things.
Sure, there's no option for full body tracking, visual fidelity isn't as good, and nothing beats external tracking performance; but with Quest, it's $299. You put it on your head and you can play. I can buy a Quest for my mom.
Don't get me wrong, I really love both the Index and Vive Pro, but I am not forgetting about the work and costs associated with either platform
EDIT: I forgot the ritual I have to go through to get Vive and Index to work
1. Turn on computer
2. Put wires on ceiling
3. Turn on each base station
4. Turn on VR HMD
5. Turn on controllers.
1. Turn on Oculus
2. Turn on controllers
It's bad enough where my SO "doesn't want to hassle" me just to play VR.
Oculus link gives you very good latency, but with the default settings the quality leaves a lot to be desired. You can tweak the render settings though, which fixes the issue, and it ends up looking about as good as Virtual Desktop. Only real unsolvable problem is the cable, but it's not a huge issue for me.
Alyx is quite playable through both of these setups (assuming good latency for a virtual desktop setup).
The first time I heard him, almost 10 years ago, speaking to a journalist at E3, I was shocked. So much detail, but so well communicated. I can happily listen to him talk on almost any subject.
Up until that point, you are terrified of it happening. Once it does and you get past it, everything else seems much easier.
Watching recordings of yourself sounds like a mild form of exposure therapy.
But you know what? You learn to (1) tune it out quick, (2) maintain your confidence, and (3) move on - a skill that's been invaluable for me going forward.
In a big picture way, it's also interesting to hear a prominent Facebook employee being frank about the limitations of the product. Perhaps they're just not allowed to do that when it comes to privacy issues.
I suspect a lot of the problems with videochat (and VR) has to do with this.
Bad latency is frustrating and wears you down, at some point it's not worth attempting to speak up because it's tiring to deal with.
In real life communication transitions between people are fast and you can overlap a bit.
VR could solve the issue of what people are looking at least which I think is the other issue, though I'm not sure how well avatars will fit in for that.
So, add a 3d space with spatial audio, and it's natural to form groups. Add really good spatial audio, and the system can handle the different unconscious movements we do to pick up snippets of activity around us and figure out which conversations we want to be closer to. (You still can't replicate the free food, though, so it's probably doomed anyway, but it's worth exploring).
His bit about how you can't really control volume in that kind of experience was really interesting to me.
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%"
People often ignore that second half.
> "I mean how many live on a 10 Mbps home connection with high latency yet they make perfectly fine calls."
I'm not sure I agree with 'perfectly fine', people just suffer through things being terrible because they don't know they could be better. Latency on a 1:1 call is bad, but tolerable - things get a lot worse as you move towards group interaction (dealing with group video chats often because of covid it's clear how much worse this is than in a real life room).
It's not premature optimization to get the latency down when that's probably the most critical feature tied to usable performance and better experience, arguably everything else should be lower priority.
My smartphone hasn't been working, so I've been using the headset a lot more as a secondary computing device. The keyboard is not nearly as practical as it could be task switching requires quitting out of each application. The pass-through environment is only available in the base applications, and it's difficult to use when you have multiple guardians and are switching rooms. Also, the hand-tracking interface is really bad. If there were some way to just grab controls directly in 3D space instead of using the pinch-as-a-mouse mechanism, that'd be way nicer. it borrows the poor windowing support of mobile that was driven by screen real estate constraint, even tho there is less constraint on real estate even than desktop.
If Oculus engineers have any notions of making the Quest 2 (or later headsets) into a general use platform, they should try doing all their work in one. Get rid of their laptop and smartphone and do everything in the headset. I'm sure we'd soon see better text rendering, better support for keyboard and mouse, better switching between applications and guardians, better copy/paste behavior between applications, ability to take pictures with the tracking cameras, more control over how the headset goes to sleep or content pauses (or doesn't) when donning and doffing the headset, etc.
The camera access is a huge privacy issue. I would also love to get access to cameras. It's in OpenXR standard, so we can hope they will implement it at some point.
That said, Quest is already useful for productivity. There's a modern browser, keyboard support (usb or bluetooth), Termux app that gives you capable posix shell, also apps for streaming your PC monitors.
I've built an in-VR live coding environment that I use to prototype and debug Lua 3D .There is also another environment by MrDoob . I'm actually surprised how much Quest is open platform and usable as standalone headset. Very unhappy about FB account though.
Yeah, Carmack mentioned some official tracked physical keyboard being announced soon, which is definitely the way to go for useful work. Good tracked keyboard and mouse support plus some decent windowing will make it dramatically better than a cellphone and perhaps even a typical tablet as far as productivity whereas by default (i.e. just controllers or default hand tracking) the Quest 1 is slower for precise input than a smartphone.
I actually don't think it'll replace phones. More likely to replace large laptop displays for on-the-go productivity as resolution approaches retinal quality in the next few years.
You mean virtual controllers?
EDIT: Sorry, I was typing the above in VR using my controllers. (Honestly!)
Occulus is tracking motion with camera which is way more CPU intensive than calculating position based on electromagnetic signals.
One trouble they had though that it wouldnt work well with high energy electric items nearby.
So dooes anyone know if Occulus or HTC vive or other well in market consumer VR device ever tried that or has plans to do so? is this even possible?
EDIT: They also made controllers for Razer. https://support.razer.com/console/razer-hydra/
Adding compute power to compensate is a much easier problem to solve.
One thing to realize is also that cameras (or lighthouses in the case of the Vive) are not the primary sensors. What really do the job are accelerometers and gyroscopes. Cameras only provide reference points to correct for drift.
It'd be sweet if someone eventually made a headset with pass-through view with high resolution and ability to choose wavelength (UV to IR, maybe even a phased audio array to visualize sound sources for the deaf).
If you could make the tracking cameras work in the Sun and figured out a way to harden the display (perhaps using a protective electronic shutter system) to sunlight, it'd be closer to a general platform.
What is next? Facebook login required to use a pointing device, keyboard, printer? Before you know it you will be completely oppressed.
They're launching an official project for this, called Infinite Office. You see the world in black and white through the tracking cameras, but you have colorful VR screens around you.
See this pic for an example:
I love it for learning, or working out a coding challenge.
The benefit to "code inside a void" is immense for people who really struggle with ADHD. I've never felt the ability to zone in like I can in VR.
But I have to blow the text size up quite large. And I don't do it for more than an hour, because the resolution isn't quite there for an all day reading experience.
I have a lot of fun playing my 2D games in VR this way as well. Starcraft is crazy fun on a screen that feels like its the size of a movie theater.
It's easy to imagine how useful this could be eg: when working with complex web apps where having 4 screens (IDE, terminal, browser, browser console) actually has great utility.
At best VR somehow is a perfect argument on why people should play escape games or paintball/airsoft. It sound like a second version of the google glass.
You just need to use some common logic: a video game works on a screen. It's too difficult to add new sensors. You cannot bridge reality and virtuality. You either have a video game or a normal game/toy/puzzle, you cannot have both.
Maybe the military will come up with new sensors, but until then, I would not bet on VR.
I was really surprised and delighted at how cool it was. It felt raw and fresh- almost like when I first got on the internet, there was just a whole new world to explore, and different types of content and games- some were cool, some were not, but were at least interesting in their mechanics. Especially during covid, I found it super neat that I could watch a basketball game in a "section" of a stadium and chat with those around me, and similarly watch a band perform.
The biggest drawback is that the form factor isn't all that comfortable, and battery life is an issue, but I think there is a tremendous amount of potential in terms of experiences.
Is playing real paintball more immersive and real than playing VR paintball in Rec Room? Of course it is. It's also an expensive hobby with a bunch of equipment that requires you to set aside hours of time and drive to a paintball arena.
You can't just jump from "This isn't a 100% faithful reproduction of real life activities" to "This is pointless and will never catch on."
Pong is clearly inspired by ping pong, but you wouldn't have played it in the 70's and said "Pong is not the same as ping pong, video games aren't going anywhere."
Funnily enough, ping pong is one of the more convincing VR experiences because you only need to move in a small area, and the small haptic feedback from the controller is close enough to a ball hitting the paddle. Just don't try to lean on the table, it's not there.
I just have more faith in augmented reality. Less latency problems, more possibilities of games like pokemon go.
I'm curious why they didn't try a version with transparent screens.
I’m not sure I understand your argument at all - what is the logic you are following?
Since it sounded to me like the low hanging mobile VR fruit has been picked, I went ahead and pre-ordered my very first HMD afterwards. Time to see for myself if this is the computing future or an evolutionary dead end.