Linden Lab went through a period when they envisioned a 3D web, where the same client would talk to multiple virtual worlds. Around 2011, they backed off on that. But others went on, and to the surprise of Linden Lab, Open Simulator was developed. The Second Life simulator is proprietary, while the client is open source. Both are in C++. Open Simulator is in C#, and it's roughly compatible, although totally different internally.
I've been writing a Second Life / Open Simulator client in Rust for the last year or so, so I know the wire protocol. VWRAP seems to be an attempt to define a more portable wire protocol with roughly the same information.
The current Second Life wire protocol has two parts, events via binary UDP messages, and assets via HTTP. The UDP message formats are defined here.[1] This is a painful system. Both ends have to use exactly the same message template file, because messages do not contain length information. You have to parse each message perfectly just to find the next message. This makes adding new messages very difficult. No new messages have been added in years. That's one problem VWRAP was intended to solve.
Assets come in from ordinary web-type servers, and complex ones are encapsulated in Linden Lab Serial Data Protocol, which is documented and has multiple implementations. Those are reasonably straightforward to deal with.
The conceptual problems of interoperability are much tougher. There are some metaverse standards organizations starting up, but they're working more on diversity, inclusion, and safety than wire protocols. Content standardization is making progress. Content standardization seems to be settling on some subset of GLTF/USD formats. NVidia is pushing that, and the major tools (Maya, Blender, Unity, Unreal) all speak it. That's a file format, not a wire protocol.
Enough standardization to allow portals between virtual worlds would be a big win. There's talk about this, but not much is happening, except in the Open Simulator ecosystem, where it works now.
One company, Ready Player Me, has an interesting concept - outsourced avatars. Avatars are created on the Ready Player Me site. Cooperating games have a module which renders such avatars, going out to Ready Player Me for the assets. This gives Ready Player Me a lock on avatar appearance and wearable items. For now, this is free. For now. "Wolf3D has the right to amend the Terms unilaterally." Some smaller games are using this, but the big studios have mostly stayed away. If that catches on, those guys have control over the air supply of the Metaverse.
If I remember rightly it was even messier than that. You see the note at the top of that file about how the Version 2.0 template requires preservation of message numbers? Well, I think in the original version most of the messages were assigned IDs based on where they ended up in some internal hashtable used by the code, or something equally fragile and dependent on the exact contents of the message template file - so even unused messages would change around the message IDs. The new format did at least make it possible to add messages without instantly breaking all compatibility...
I'm curious why they would have chosen to make their UDP messaging so inflexible. Is it just because it was written when bandwidth was so scarce they couldn't afford to reserve a couple bytes for length and start/termination?
Games programmers in the 90s had this belief that UDP was faster than TCP. And in some cases it was. I don't think there was a standard way to change your stack's behavior for startup and congestion back then (we have CUBIC, BIC, FAST START and all sorts of options now.)
The commonly held belief was that if you want speed, you have to use UDP. But then the first thing people do is they re-implement TCP poorly at the application layer to get fragmentation, reliability, windowing or large datagrams.
OGP and "Assets Over HTTP" were introduced later and used TCP to carry some of the application protocol, but by the time they came around, UDP was wedged very far up in the application code and it was difficult to get it all out.
I haven't looked at the SL Viewer code in over a decade. Maybe it's better now.
It's still true that UDP is necessary for realtime communication. I recently worked on an MMO project in which a sub-team attempted to use TCP as the main layer, but ran into huge issues, all of which were cured by switching to UDP. TCP is good at what it's good at, but nearly every realtime audio and video protocol is gonna keep using UDP. You have to be able to ignore dropped packets instead of blocking the entire channel waiting for a full RTT, it very quickly spirals out of control.
Of course, TCP is a much better protocol for infrequent and request/response type interactions, so while I was at Linden we converted as much traffic as possible to TCP/XML. But a lot of the "realtime" stuff like object updates had to remain UDP, for empirical reasons.
Second Life didn't reimplement fragmentation -- messages just got truncated if they went over the MTU, lol.
UDP is not required for real-time communication. I have 5k+ concurrent channels of RTP over TLS (not DTLS) going most of the day. If you solved a throughput problem by switching from TCP to UDP, it means you had a middle-box doing bad things.
TCP is not slow unless you encounter congestion or packet loss. But you have to deal with those problems if you're sending data over UDP.
When I was at Linden, we activated fast start and bic on ADITI when testing OGP and Assets over HTTP and things worked MUCH BETTER.
OTOH, limited congestion and packet loss do not affect things much if you have UDP-based event/object updates. With TCP there there is no way to avoid latency due to head-of-line congestion, but with UDP you just drop a message and move on.
I think TCP works reasonably well today because the network is better and there is often very little packet loss, but as soon as you get some, TCP will optimise for throughput, like designed and at the expense of latency.
actually... we DID do fragmentation in one of the messages. but i think that one didn't do windowing or something. this is the problem of using UDP, everyone thinks they're Van Jacobson and re-invents TCP poorly. In the case of the viewer, we had at least three implementations of "random transport over UDP," each talking with a different part of the back-end. Each used the same algorithm to deal with packet loss, however: dump core and wait five minutes. (I jest. I think we mostly fixed that, but it did happen occasionally.)
I haven't looked at the SL Viewer code in over a decade. Maybe it's better now.
I have. It's a little better. Bulk file transfers over UDP are finally gone, moved to HTTP.
The commonly held belief was that if you want speed, you have to use UDP. But then the first thing people do is they re-implement TCP poorly at the application layer to get fragmentation, reliability, windowing or large datagrams.
Yes. The protocol Second Life uses has both unreliable and reliable UDP datagrams. "Reliable" means they have retransmits, on a fixed timer with a fixed retry count. Some get lost on overload, because the viewer's one-thread implementation discards if too many packets arrive in a frame time. In order delivery, reliability, no head of line blocking. - pick two. You can't have all three. TCP picks the first two. The Second Life protocol picks the second two. This results in occasional trouble where there really is an in-order requirement imposed by the way the state of the system changes.
Unreliable messages are common in game protocols. "Unreliable" means that lost messages should be superseded by the next one of the same type. The intent is to be consistent-eventually. Second Life is almost consistent-eventually, but not quite, which results in occasional viewer out of sync errors which make some objects in the world look wrong.
(Personally, I'd do this with one UDP path of unreliable messages, plus two TCP connections, one for high priority, low-traffic stuff, and one for lower-priority bigger stuff. Get rid of "reliable UDP". Only a very few message types, such as user mouse activity and movement updates, should be unreliable, because those can tolerate a lost message.)
It was basically "throwaway code" which was written in a hurry for a prototype that was not expected to have a long life. This kind of parsing was (is?) very common in the video game industry, and quite easy to author in C++, so it would have been the first tool the developers reached for to implement a real-time network protocol.
The evolution was hardcoded -> message template -> "Liberación" which allows the protocol to tolerate unknown fields, and thus be future-compatible, much like Protobuf does.
Yeah. Linden was this weird mixture of Games Programmers and Enterprise Software Architects. Games programmers all know the absolute most important thing in the world is ship date and since you never re-use your code, who cares if it's easy maintain, you're never going to see it again! Enterprise Software Architects know performance takes a back seat to correctness and ease of extension. Since the enterprise never throws any code away, it's important to make sure everything can be updated by the next group of contractors you hire, even if that means you take a performance hit.
Turns out both communities didn't have a perfect idea of what they should be doing.
Consider the web today. We have web pages that display some text and a few pictures. Yet viewing one requires downloading tens of megabytes of data, perhaps a hundred network transactions, a few gigabytes of RAM, a CPU or two running at upwards of 3GHz, a little help from a GPU, and several seconds of loading time. This is an awesome level of inefficiency.
Today's web browsing uses at least an order of magnitude more resources than it should based on what appears on the screen.
We cannot afford that level of inefficiency in good quality 3D worlds. Just doing a good 3D world with every known optimization is at the upper limits of what available hardware can do.
Watch the Unreal Engine 5 Matrix demo to see what's possible with the best known techniques.
Then read up on what's done to put those meshes into Nanite format, an appallingly complex, insanely optimized, and very clever representation. One full of internal offset fields, so hostile content would be a huge problem.
That's the problem with adapting web technology to virtual worlds. If you make the system as general, portable, and robust as web services, it's too bulky, too slow, and too low-rez.
The "web" today is effectively a basic virtual machine, and pages are containers deployed onto it.
Which means you get all the bloat of re-shipping an OS (js libs), because the ecosystem as a whole would rather have diversity, speed of framework iteration, minimized developer time, and absolutely minimal client friction.
And by and large, those have been good choices! Look at how far the web's come in just the last 30 years.
But as you note, many of those goals are incompatible with game engines, which have seriously tight minimum performance requirements.
So what do you think are the most important priorities for a 3D world format? I'm assuming performance (rendering and network) comes first, but what gets optimized for after that?
Adding on, it's got to either be fun or useful, or it's going continue to linger in this sounds cool in books, doesn't make sense in real life (or movies, really, either). There's not a lot of usefulness either.
Maybe, but you can also end up with standard cached assets and people leveraging optimized frameworks like unity. Already the tooling is there, and I as a react dev am having an easy time building up a vr project, that I see as a next evolution of a website. Some concerns have to be spent on optimization, but 3d environments are a well practiced area, and we can get some great experiences for the same cost of some of these bloated Js websites.
This is just because of where we currently are on the hardware curve. All of the relevant capacities are still growing exponentially. You can run any UE3 game on a cellphone. Just wait a couple generations.
They were well ahead of their time. Or even if they weren't, the market simply wasn't ready for them.
As an aside, I have a few anecdotes about Second Life:
- I remember IBM paying money for virtual real estate in second life. They set up a marketing presence, too. It seemed like such an IBM thing to do.
- My university was giving class credit to a grad student for building a virtual mock up of our campus in Second Life. It was a thesis project, I think. I remember thinking he was rather late to the game, as Second Life had already dropped in popularity versus MMOs such as WoW.
I wonder what those events would have been like today, with VRChat and the emergence of the "metaverse". Everything old is new again.
I think the biggest issue was that bandwidth wasn't ready for them, and their hype came early because it was one of the first instances of people profiting from, well, the equivalent of NFTs with the real estate land rush. By the time bandwidth caught up, that was old hat and they had a rep for largely being a place for "alternative communities" to meet up.
I was a super early adopter of things like virtual communities and MMOs, myself (and a huge Neal Stephenson fan to boot, so immediately recognized that they were using the Metaverse as their inspiration) and Second Life held very little attraction to me because of the performance. I tried it a few times, but there wasn't much immersion happening at the standard 1.5Mbps DSL speed, never mind modem.
If they'd come out at a point where the asset streaming worked smoothly for most customers, and the hype came after that--and especially if they'd come out after things like Dreams or even Little Big Planet or Mario Maker had prepped people for the idea of crowdsourced assets--I think they'd have absolutely smashed the market.
I tried it a few times, but there wasn't much immersion happening at the standard 1.5Mbps DSL speed, never mind modem.
It needs about 50Mb/s and a GTX 1060 or better to work well. With gigabit networking and a GTX 3070, it's pretty good. Now once the viewer becomes multi-threaded...
Before the era of MtGox, you would buy Linden dollars for SecondLife, and then trade them for bitcoin on VirWoX.
SecondLife was well positioned to become a gateway into the world of cryptocurrency back in 2010, but instead they decided to start banning people doing anything Bitcoin related, and everyone had to move to MtGox.
Pretty sure Linden had FINCEN people breathing down their collective necks. The threat of federal prosecution for enabling money laundering tends to tamp down your excitement for crypto.
SecondLife is a cautionary tale for anybody wanting to start a "metaverse" type project. If you give your users unlimited freedom and allow people to make money you end up with endless strip malls full of Cybersex attachments for avatars. Anybody who tries to do "legit" events is in danger of being spammed by helicopter penises.
If you want anything else you need to incorporate and enforce standards and practices right from the start, but then you'll be perceived as corporate and lame.
That happens very rarely in Second Life. Being a jerk in Second Life has an annoyance radius of about 100 meters. The world is the size of Los Angeles or Greater London. This gives the system considerable resistance to annoyances. Space is what keeps everything from being in the same place. Unlike the 2D web, there's not much ability to annoy large numbers of people. There's no way to broadcast spam. There are no "retweets".
If you hold an event on your own land, you can block people, or require they be in a your group just to get in. You can prevent objects owned by others from entering. You can kick people out and send them to their home location. You can even use weapons they can't use on your property to apply large physical forces to their avatar and kick them up a few thousand meters. That's usually considered overkill. Property owners have more than enough tools to deal with problems. This is why Second Life needs only a tiny "governance" unit.
I have a virtual motorcycle shop in an adult area of Second Life and I'm not seeing annoyances like that. All the action is in nightclubs and people's own houses, just like real life.
Maybe before saying furries are "jokes" and "weirdos" you should talk to a furry.
Calling people names says a lot more about you than it does about furries and "weirdos." I am happy you are in no way responsible for defining what the metaverse is or isn't.
You have to judge people, but if you make all of your judgements on the basis of who's gay or furry or whatever you won't be making your judgements on the basis of stuff that actually matters. You can only have one set of criteria and every item competes with the others for weight.
This is exactly the corporate kiss of death I've seen over and over. If I can't use your product/community/whatever without being assaulted by the sexual fetishes of others, then it's just ultimately chase away customers who don't subscribe to those fetishes. Now your growth and future outlook is just much less than a competitor.
A common con in capitalism is to sell people on "free-speech" and "anti-corporate" agendas in your capitalist corporate product. I think most sensible people see through this, but there's a segment who just take to it pretty well and they tend to have marginalized and very questionable ideas about politics, race, religion, age of consent, boundaries, and justifiable terrorism.
Every "free speech" equivalent to the big players has failed and badly. Voat can't touch reddit. Parlour can't touch twitter, etc.
The real problem with these companies is that ultimately they want growth and profitability, thus when they try to expand out you now have the "Nazi Bar" problem, where the marginalized end up owning the establishment and chasing away everyone else.
Good moderation, protection of minorities, protection of LGBTQ people, protection of women, curtailing all hate speech, strict penalties, etc is the formula for success whether the tech libertarian crowd likes it or not. This is why your company has HR policies like this too. Once you let in the first Nazi into your bar, its over.
I suspect the protocols defined in the link provided do not require protocol users to be assaulted by the sexual fetishes of others. If you don't want to be assaulted by the content on Porn Hub, don't go to Porn Hub.
The irony is that complete freedom ends up being quite limiting. You can't do kids events or anything where you expect the major press to show up. Sponsored events end up being pretty hard to do. Your platform won't even be available in many countries due to restrictive laws regarding pornography, including highly populous countries like China.
I think FB and a lot of VCs are going to learn that virtual worlds are boring, if not awful, unless they're tied to an addictive game loop. It'll be six years since the Vive came out, possibly the first "realistic," open-ish, mm tracking, high performance, modern VR set and there's still no killer app for VR. Worse, the VR market has decided its all nonsense anyway and is fine with buying headsets that just run mobile phone chipsets because that performance and graphics is "good enough" and no one is demanding a photo realistic world to lose yourself in. All they care if the games are good enough, that is to say if the loop addictive.
Those pushing out past games in VR are brave and I'm thankful they are there. Maybe Meta will figure it out, but I'm skeptical.
Even a decade before I was “playing” Worlds, Active Worlds, and 3D Anarchy. Those were just 3D chatrooms like SL. The charm had already worn by the late 90s though.
Even harder... Convincing corporate entities to open even just a small bit of their metaverse / virtual world. At Linden, we pitched OGP/MMOX/VWRAP as "If the Metaverse is the next big thing, then its better to be Google than AoL."
That message resonated well with Linden, who saw the "Open Metaverse" as a place where their existing customers building UGC assets could find new markets for their goods. Linden could then jump in and take a percentage for cash transactions or hosting or whatever other services people would pay for.
I honestly don't know how FB/Meta sees their market / business model. Selling ads in a virtual world? Content to justify selling head-mounted-hardware? But it does sort of seem like walled gardens are in their DNA.
Would love to see Meta step into the "child labor" controversy. Isn't that an episode of Silicon Valley? Their videochat platform is a hit but all their users are underage - they agree to sell to the bad-guy-VC just before the lawsuits show up.
The thing is, most Roblox players that I've seen have FB Messenger (the kids version) video/audio calls running in the background so they can talk. With the new focus on metaverse the combo really makes sense.
On the other hand Roblox could really pull a coup by replicating this functionality natively.
I assume they don't do it because they are scared of being responsible, not because it's technically difficult. It probably helps that the communication is separate from Roblox because it's harder to manipulate a kid on two separate channels together (in the same ways it's hard to actually get two kids connected that want to be connected).
Though Facebook also has some advantage because they are already maintaining a bunch of relationship information and they use that to bootstrap Messenger For Kids since one of the common patterns is parents become Facebook friends with clear identities and then they connect their children.
Linden Lab went through a period when they envisioned a 3D web, where the same client would talk to multiple virtual worlds. Around 2011, they backed off on that. But others went on, and to the surprise of Linden Lab, Open Simulator was developed. The Second Life simulator is proprietary, while the client is open source. Both are in C++. Open Simulator is in C#, and it's roughly compatible, although totally different internally.
I've been writing a Second Life / Open Simulator client in Rust for the last year or so, so I know the wire protocol. VWRAP seems to be an attempt to define a more portable wire protocol with roughly the same information.
The current Second Life wire protocol has two parts, events via binary UDP messages, and assets via HTTP. The UDP message formats are defined here.[1] This is a painful system. Both ends have to use exactly the same message template file, because messages do not contain length information. You have to parse each message perfectly just to find the next message. This makes adding new messages very difficult. No new messages have been added in years. That's one problem VWRAP was intended to solve.
Assets come in from ordinary web-type servers, and complex ones are encapsulated in Linden Lab Serial Data Protocol, which is documented and has multiple implementations. Those are reasonably straightforward to deal with.
The conceptual problems of interoperability are much tougher. There are some metaverse standards organizations starting up, but they're working more on diversity, inclusion, and safety than wire protocols. Content standardization is making progress. Content standardization seems to be settling on some subset of GLTF/USD formats. NVidia is pushing that, and the major tools (Maya, Blender, Unity, Unreal) all speak it. That's a file format, not a wire protocol.
Enough standardization to allow portals between virtual worlds would be a big win. There's talk about this, but not much is happening, except in the Open Simulator ecosystem, where it works now.
One company, Ready Player Me, has an interesting concept - outsourced avatars. Avatars are created on the Ready Player Me site. Cooperating games have a module which renders such avatars, going out to Ready Player Me for the assets. This gives Ready Player Me a lock on avatar appearance and wearable items. For now, this is free. For now. "Wolf3D has the right to amend the Terms unilaterally." Some smaller games are using this, but the big studios have mostly stayed away. If that catches on, those guys have control over the air supply of the Metaverse.
[1] https://vcs.firestormviewer.org/phoenix-firestorm/files/tip/...