Hacker News new | past | comments | ask | show | jobs | submit login
Linux in a Pixel Shader – A RISC-V Emulator for VRChat (blog.pimaker.at)
428 points by PiMaker 57 days ago | hide | past | favorite | 98 comments

Definitely watch the presentation by the developer in VR Chat:


I haven't played VR Chat in a few years, and holy cow this makes me want to jump back in. Seeing fully captured motion avatars of people is SO immersive. It's absolutely insane how the experience of interacting with people can feel so natural even when they're on the other side of the planet.

I just love watching "body language" of the presenter, it's such a precise rendering of what a "real life" presentation like this would be.

Plus, you meet people like the writer of this blog post, who are so knowledgeable and interesting and fun to talk to.

Why are the majority of avatars small children or furry-things?

I'm really not one of those people who bashes nerdy or alternative culture but I can't deny that it weirds me out a bit. It's a clear barrier of entry for me.

Most of them aren't actually all that small, it's just that the perspective it's being filmed from (the narrator's head-camera) is on a stage, and VR warps things in your periphery pretty hard.

There's also the fact that it's easier to be immersed if you aren't leaning heavily on realism; Oculus has a lot of research on this, actually. For example, if you have a photorealistic avatar but aren't doing eye-tracking, it's actually more jarring and discomforting than a cartoon avatar without it. That's why the initial avatars they did had force-obscured eyes and no color, and the new ones are literally cartoons: your mind can adapt to something very different, but it can't adapt to something that's just subtly different.

However, some people do choose to do weird stuff. That's mostly a tech thing though. If you go to a normal place, you'll find a lot of actual kids and actual teenagers, and they're relatively normal. Very few people as weird as the tech community.

Really, technical people in digital spaces kind of get a bit wild with their self-expression. Some theorize it's because most of them don't express themselves much in real life.

I've tried this particular app only a handful of times (I don't like using proprietary software, but it's important to keep yourself updated with the state of it), and when I wasn't visiting explicitly tech-related things, it was pretty normal. Lots of copyright infringement of Nintendo characters, as you might expect.

Should it be surprising that in a world where you can look however you want, lots of people would not choose to look how the real world forces them to?

It's just surprising that everyone wants to look like a mix of a 16yo anime schoolgirl and a fox.

It may or may not be a symptom of the masculinity crysis the western world is l experiencing and also sweeping under the rug every time discussions pop up about it.

Also there is something seriously wrong with the world when a people playing virtual pretend immidiately makes others, even in such thoughtful places such as HN suspect that they are horrible child molesters etc etc.

I think maybe because those are both things that are significantly different than reality.

Nah, it's more a combination of memeing and that the avatar interface in VR Chat is horrible. Picking an avatar takes a lot of work. Some of the ones listed can crash some machines. Many of the ones listed are only available on the PC version, so you get some "PC-Master-Race" posturing. And there aren't a lot of good, default options, so you end up with a lot of people picking the same avatars. There are lots of other avatars, but most of them are very unhuman-like and several are just plane bad.

EDIT: for what is worth, I tangentially work on the same sort of software. My system is a training environment not intended for the public, but a lot of the same concerns are involved. Currently, we have no avatar selection system. Everyone gets the same avatars, differentiated by a name tag. This was 50% a deliberate decision to avoid avatar mayhem and maintain an egalitarian environment, 50% pushing off the problem until I can figure out how to give some options without devolving into avatar mayhem.

There are lots of things that are different from reality, the questions is why so many of them chose that particular way to be unrealistic.

Pretty easily answerable question...

As mentioned in another comment there's no difference between someone choosing say a furry avatar vs someone choosing say a star trek character as their avatar. Same reason: they enjoy that world/universe.

Why do people make pop culture jokes? Because it's something they familiar with and enjoy and that gives 'em the seratonin.

The question is not "how can anyone choose to be an anime fox", it's "why do so many of them do that".

If say 40% of all quarterbacks were named James, you are not explaining that by saying "James is a perfectly normal name, just like Richard or John".

It's probably tied to internet culture/subculture - VR Chat is still a pretty niche thing, most people don't own VR headsets. A lot of people owning a headset and interested in participating in a virtual world intersect with online subcultures and Otaku subculture.

Personally, I don't understand why there is so much surprise. If you really think about it, your avatar is something you will look at a lot. So you naturally choose something you like. And people have tendency to chose live objects rather than say, a gigantic 3D number, particle system, or white light. There is not really much choice; humans (humanoid characters, like elfs), animals (& imaginary creatures & hybrids, four-legged animals are typically weird to control in 3D environment) and maybe robots.

I feel seriously old since I've been using furry fox avatar in SecondLife back in the 2006, and even then, this was fairly common.

This is one of those situations where having the conversation in person would be way easier. I have no idea why this point is not landing lol.

People with similar intrests tend to clump and early adopters of a technology tend not to represent the general population. Not to mention there are very obvious reasons why people drawn to roleplaying as something other than they are would be drawn to a technology that lets you easily do that in a relatively risk free environment.

I also wonder if its really as many as you think it is, or if they just stand out to you because you don't like them. You also don't have a good basis of comparison since you have no idea how many people in regular life are like that (i imagine its still much higher in vr space, but maybe the gap is not as vast as you think)

well, it's all about different perspective and respect each other right? Because those who don't like anime girl/fox and think every fiction character should look over 30 are _also_ weird to some other group of people.

I think that's a little understating, certainly anyone with experience of fringe groups attaching themselves to Pride each year (Minor-Attracted Persons and furries well represented), "perspective and respect" is probably the least interesting aspect.

Sexually developed adults suffering from age or species dysphoria and ultimately demanding rights to match their self image is something that should not be encouraged in civil society, as those rights are inextricably linked to an implied requirement for consent which cannot be given.

It might be tempting to try and draw a line between the kind of folk above and the folk from the video, but I find that difficult given the variety of clearly sexually exaggerated avatars on display. The one that sticks most in mind being a fox with a pair of impractically large breasts.

I don't get why you're linking those two groups.

As far as i understand, furries just want to dress up as animals and maybe have sex with other people dressed as animals, not have sex with actual animals. As long as everyone involved is a consenting adult i don't see anything wrong with that. I might think its weird, but that's true of a lot of "sex" things. Seems very very different from the people attracted to children.

The reason being solely that people of those two groups declare themselves and justify their behavior under the LGBT umbrella.

Saying you're LGBT doesn't inherently make you LGBT - nor does it grant you any particular credibility. The community is insanely diverse these days and at least in American politics, most of the large issues have been resolved (with the exception of trans individuals, who still face struggles w.r.t. insurance and healthcare).

MAPs tend to latch on saying their "attractions" (pedophilia) are part of their unique sexual identity. This is not true, and GP is correct in pointing out these two groups cannot adequately acquire consent from the other party - one is incapable, and other has determined to be not of mature enough mind and body to do so.

Both groups were mentioned due to their presence in the video and parent comments, it is not an association I made.

There is definitely a cute and innocent side to the furries particularly among younger folk, it seems for many there is no sexual element to it at least initially. It'd be a much happier world if the two were more easily separable, but even then the question would remain whether the gentler group acted as a pathway for the uglier group. That both exist undifferentiable and intermingled in the same spaces is extremely problematic, and a priority issue for those folk to solve.

In the early gay community it was also the case that desires for otherwise healthy and innocent rights were abused to mask much more insidious causes. Today it is unlikely you'd see homosexuality and paedophilia mentioned in the same breath without provoking a surprised response, but relatively recently that perception was still common. I think this is probably a good parallel to where furries are now

Yup. Maybe it’s that the IRL self is nothing more than one ended up that everyone can’t care less about.

There has to be something to be argued with respect to human model of consciousness, cognition, and self identity about almost complete lack of male avatars in actual VR scene.

It can’t be like “they’re all young _men_ so they want cute anime”. Maybe it’s not just about VR, or anime, but the concept of masculine dude just seem to disperse and disappear once you stop subjecting yourself to the baseline reality.

Selection bias: the majority of the male population using VR are in no way representative of the general male population outside of VR.

I say this as a gay furry, who knows that there are like, a million gay furries on VR, but constitute only a small portion of the general population.

The number of people into furry or anime in the software community is large, especially among millennial generation. But unless we find someone of similar interests at work, we're not going to talk about it (or only describe it in vague terms) because it can creep some people out. Further if you're using VRChat I think you already self-select as unusual beyond that. (Anecdote, my college anime club almost a decade ago was almost all software engineers with an assortment of other science or engineering fields.)

StraszFilms made a video touching on this subject : https://www.youtube.com/watch?v=5v_Dl7i4Bcw

( he also made a very nice documentary about VRChat nightclubs https://www.youtube.com/watch?v=R1wUg9HCODU )

I bought an Oculus Quest 2 earlier this year to check out the whole VR situation. It was basically a ghost town with pockets of children and, IMHO, sketchy folks having childlike conversation.

I don't really know what I expected but it wasn't that. Aside from some focus issues the goggle hardware was pretty great but the VR world was not a good experience at all. :/

It's worth noting for anyone curious: the Quest 2 can't actually interact with or enter the majority of worlds because of technical constraints (they aren't even offered to Quest 2 users unless the author of the world explicitly opts into it and designs their world for it). Most adults are on private, PC-focused servers.

The application is not a stunning technical achievement, but it's unfair to judge the program by jcim's summary, because he seemingly went into it without knowledge of how it worked and wasn't able to access any of the interesting places. It would be like judging Skyrim by an SNES backport.

While all of this is true, one should also note that none of this is obvious at any point while using VR Chat. It might be the most popular social VR app right now (Altspace has even more problems, and RecRoom is pretty much strictly children), but that doesn't mean the software is actually any good. We're in the Friendster/MySpace/Neopets days of social VR, with Facebook seemingly interested in making the LinkedIn.

Honestly, the game starts at the point where you meet a group of people you enjoy interacting with and stop visiting public instances. Much easier to get in to with a friend to guide you along the way.

Yes, this is absolutely true. It's probably true for most people of most social software, but for me the immediacy and intimacy of VR and voice chat definitely amplify it a lot.

The popular public worlds are like going onto AOL general chatrooms when those were popular. You get a lot of random people sticking around and often some obnoxious people. It's possible to find good conversation buddies and friends through them but it takes a lot of hopping between public worlds.

The real magic is when you either A) have friends who are already into it invite you directly to worlds with them and their friends, or B) find the niche public worlds that have a more mature and invested userbase. (Sadly, a lot of the niche worlds aren't compatible with VRChat on the Quest when running in standalone mode; you have to connect the Quest to a computer to get the full experience.)

Because anime. It's cute. People want to be cute.

It seems representative of VR's demographics ime. Weird fantasy and wish fulfillment. It's kept me from the platform as well.

Why are all Peanuts characters children despite dealing with adult issues regularly?

Artistic license. But mostly because when you are an adult, you still feel like a child in many situations.

The tech might be a bit of a factor. If you don't have good lighting tech western style models will look like cartoons too.

Why does it weird you out?

A few reasons I guess.

For as long as I can remember, we have had adult predators going online posing as young girls/boys with different levels of malicious sexual intent. I understand that that isn't the objective here, but I can't help making that connection. Like "Oh! You weren't actually a 15 year old girl, you are a 30 year old guy pretending to be". That makes me uncomfortable.

The anime stuff always seem to be overly sexualized. Skimpy outfits. Traditionally sexy bodies (Legs, boobs, ass, etc. Yes, that is highly subjective but I think you can see my point).

Same goes for the furry stuff. Always the more or less "sexy", more or less "naked" looks. As an outsider it seems to be a sexual fetish more than anything else.

I have absolutely nothing against people wanting to be more fluid with genders. Or wanting to be sexy. Or roleplaying as whatever. I welcome that. It's just a bit weird that it's such a common choice to be [sexy child] or [sexy animal]. And in a setting that, to me, is not where sexy things happen, or where sexy things are even really wanted?

You should try being in one of the rooms, it's extremely creepy haha. Apparently my poor Quest 2 experience was due to me being a n00b, but when you roll into a room with 5-10 avatars bopping around and you can hear conversations between 4th graders and suddenly realize you're 'alone' with them, it gets extremely uncomfortable. I'm not saying it's a rational response, but I never said a word while I was on there, boxed the thing up after a few hours and shipped it back to Facebook.

Unfortunately true, but also not the experience that most people who continue to play the game have with it. Public instances, especially those in popular worlds, are pure cancer. Anyone who has played more than a few hours will tell you. The reason they are so full of screaming kids is because everyone who isn't one has moved elsewhere.

I very very rarely encounter minors, and hanging out with them makes me uncomfortable too, even having thousands of hours played. Sucks that you had a bad experience, the platform can be a completely different experience if you are brought into it by someone who already knows their way around.

My community actively bans and blocks all kids and child avatars, is quest-incompatible, and boots creepy people aggressively. It's definitely a very enjoyable experience that's kept me sane throughout COVID.

Its very common for men to play girl avatars and for women to play men avatars in online game. Very rarely have I ever met anyone that was actually pretending to be that character.

Sure. I have the same experience in that regard.

I think there's a big difference between "online games" in general, and this, though. If you don't see that difference then I understand that you don't think that this is weird.

Out of interest, what would you choose to be?

It's hard without knowing how freely I could pick. Maybe something from a video game I like or a caricature-y version of a movie character I like. Those are fun choices for a one-off thing though. Maybe I would reconsider if I really got into it and wanted something that actually represented me as a person. I don't think I would swap biological gender in any of those cases.

You got a timestamp? You linked to an almost 3 hour long video.

Edit: Presentation starts around 1:24:00

this is atrocious. Bunch of visual distractions made by avatars, horrible and disrespectful background noise/chatter, graphics like it's 1998..

I'm kinda torn between the current top comment pointing out how immersive everything is, the current top reply remarking about the eyebrow-raising amount of furry representation, and this perspective.

The graphics are indeed really bad.

The background noise also seems to be poorly managed.

And the avatars are visually striking just to the point of being very distracting.

But I have not the first clue how to fix any of these issues. The graphics have to exceed 70fps cannot drop frames or you risk users getting nausea (and not using the platform!), the audio chat works just like every other audio chat out there - badly, and the I looked into the avatar situation and discovered that, contrary to my own intuition (where avatars would incorporate lightweight, aggressively CPU-throttled control programs that could arbitrarily direct how emoting works etc), you basically get a restrictive SDK that's just *upgraded* to the point where you have like 16 different things your avatar can do.

It's simultaneously utterly inspiring and an absolute dumpster fire. I "got" VRChat for the first time when I discovered this last week and have been awkwardly conflicted about it ever since.

Idk why people would complain about furry representation in VR chat, if at all furries are driving things like VR chat and in 20-30 years or so everyone will be using it and it will "be cool".

Not really much difference between an avatar of any other character and a furry avatar, just people are fans of their own character/universe than say a character from Star Trek or something.

It's really no weirder than Rick Sanchez and Superman avatars dancing to Martin Luther King Jr's "I have a dream" speech in Fortnite. The metaverse is strange.



I think a lot of people that have never used VR have a preconception that VR is necessarily about being photo-realistic enough to trick people, but VR is exciting enough just for giving you a first-person perspective and bodily control within a 3d space. There is plenty of VR content that looks photo-realistic enough to be real, but many of my favorite VR experiences are in places that are clearly polygonal. It doesn't take photo-realistic models to get engrossed into talking with people. Body language shines through most avatars well; in VR your brain easily intuits how the motions of an avatar you see are linked to the motions of a human. (There are a couple people in the video whose avatars are very rigid without the arms ever moving or their heads ever tilting, but those people are the ones not in VR and instead just using the desktop version of VRChat.) VRChat is all user-made content and the draw is definitely in seeing how people express themselves and the extremely wide range of content rather than being about photo-realism or experiencing a single creative vision.

On VRChat, look up Albarracín, a Spanish old town of the former city. Astounding recreation.


Is it fully captured motion? It seems like it was random or button controlled.

There is a walking animation that applies when someone uses a joystick to move around, but otherwise generally all movement of the avatar is directly driven by the positional tracking of the user. (All VR users have the positions of their hands and head tracked, but some users also have trackers on their waist and feet too.)

There are also a couple people in the video that aren't using VR but are using VRChat in desktop mode. It's pretty obvious because their hands don't move from their sides while standing, their head motions are limited, their body never sways, etc.

I nearly fell off my chair discovering this just under a week ago, and immediately posted it: https://news.ycombinator.com/item?id=28255821

I commented underneath:

> More details and demonstration, naturally presented natively in VRChat: https://www.youtube.com/watch?v=G2u7NOpzcBQ&t=5052s

> Video found via https://twitter.com/_g1fan_/status/1427073177142939648

> Source: https://twitter.com/fuopy/status/1427051048032620544

> Just discovered Twitter doesn't let you copy a link to a retweet reference D:, but I discovered this on https://twitter.com/whitequark/ 's timeline.

This is outrageous. I'm making a text editor / game where you have a mechanical typewriter, and I've mused quite a few times about giving someone a beige computer running linux inside the game at some point, but it's all seemed unreasonable / impossible..

With this it's suddenly reasonable (if expensive performance wise) Thank you for this, haha.

When you have a full programming language at your disposal, you can compile the linux kernel for the same CPU as your game and run it paravirtualised, skipping all the performance cost of having a full emulated machine.

Unreasonable how?

That sounds kinda interesting/fun.

FWIW, given that you're running a game, and most people run games fullscreen, without anything else going, you can probably get away with not using virtualization, aka eating 100% CPU (or at least 100% of one core)... and even then, with most systems having >4 cores nowadays, you probably needn't worry at all.

But just to clarify the corollary, this implementation is running at around 250kHz (!). You wouldn't viably run Linux like this in a practical context anytime soon.

> you can probably get away with not using virtualization, aka eating 100% CPU (or at least 100% of one core)...

That's not how virtualization works, it's time sliced which is why you can have more vCPUs than physical CPUs. You can also limit the slice utilization to an arbitrary percentage or give it a priority level for scheduling.

> That's not how virtualization works, it's time sliced which is why you can have more vCPUs than physical CPUs. You can also limit the slice utilization to an arbitrary percentage or give it a priority level for scheduling.

At that point though, why bother virtualizing? If you're going for the beige PC look, why not go all the way and emulate an 80486 (or 80386 even) on an IBM PC clone in software running linux 0.01? Performance isn't going to matter too much, so you could limit it to as much or little CPU utilization as you want.

Plus, virtualizing is going to potentially require additional permissions that users may not want to grant your game (or may not be available on the platform).

I'd definitely go the emulation route for these and other reasons for sure, it was just the stated reason to do so wasn't among them.

MrLeap! As soon as I read mechanical typewriter I knew it was you

You found me!

How's island life treating you?

If you're making the actual game, and can therefore implement the virtual machine in native code on the host machine (instead of in a shader on the GPU as here) then you can very easily get 10 to 100 MIPS performance in an emulated machine with a very simple emulator. Such as https://github.com/brucehoult/trv

Bear in mind that the original Mac was roughly a 2 MIPS machine and an early Pentium or PowerMac 100 MIPS.

That's a classic. While the technical stuff was neat, I still remember the line "Despite being around 90% complete, the last 90% still remained to be done."


If I would want to solve that problem, first thing I would have tried is bypassing text representation of these shaders. For a few things in the past I have generated, and dynamically patched, D3D11 shaders directly in DXBC byte code, without HLSL anywhere. The byte code format is even documented by Microsoft. Not the complete DXBC files though, but some people on the internets have reverse engineered the missing pieces.

Even if you use byte code, the driver will have to compile your shader for the target hardware. It won't be nearly as fast as what the game does on console - loading a blob in GPU memory. It will definitely be faster than compiling a text representation, but still not fast enough to skip the uebershader.

Right, I know about the JIT compiler in the user mode half of GPU drivers. It’s just much faster than the source code compiler.

It’s possible the DXBC byte code generation might work fast enough for their application without the overhead of the ubershader, or the complexity of their hybrid approach.

Now you have a new set of problems: getting the compiler that consumes the bytecode not to crash. :)

I wouldn’t expect that. DXBC is not terribly complex. The instruction set only has 207 of them, each one is really simple. Take a look: https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/... https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/...

That the instructions are simple doesn't mean that what the compiler does with them will be! It's still a long way from the actual hardware.

"One of the many problems with shader code is that HLSL doesn’t support arrays in a meaningful way. Pointer math (and thus array indexing) just isn’t a thing on the GPU, so writing to a non-constant index of an array is impossible. To work around this, there are several places in the code with patterns like this:"

This is weird, AFAIK pixel shaders have had arrays since DX10 and the OpenGL equivalent through Buffers (and also Unordered Access Views for reading/writing pixel data) https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/...

Are VRChat shaders stuck to Direct3D9/ps_3_0 level functionality?

No, we can use SM 5.0. It's all DX11. It's not that the framework the game is written in doesn't support such features, we just have no way to bind UAVs for example. What you can do on the CPU is very very limited, while GPU code is completely unrestricted, so we are at the mercy of whatever Unity lets you do on GPU with minimal setup.

Can I have it in WebGL now too?

I wonder if the slow speed can be partially redeemed by having many emulated cores.

No, because in all their wisdom, WebGL never supported compute shaders, Intel has tried multiple times to add it to the standard and now it has been officially dropped for WebGPU.

So when the browsers bother to finally replace their 2011 GPU model by WebGPU (stable), which is still MVP 1.0 compared to Vulkan/Metal/DX 12 Ultimate, then you might have it.

It wasn't an oversight/mistake rather an intentional choice for WebGL to support mobile devices with OpenGL ES 2.0 (and later 3.0) which doesn't support compute shaders amongst many other things. Since it was the baseline exposure of GPU capability they wanted it to work universally not just on desktops which were on the way to being the minority of web users. We GPU is getting it because WebGL is already there as a baseline plus it has been nearly a decade and mobile GPUs are more capable.

I know, the problem was never moving past ES 3.0, and now we have to wait yet another 10 years for widespread deployment of WebGPU MVP 1.0, given past history.

So that, alongside the usual blacklisting issues, makes native the only option for anyone that wants to exploit the full potential of the underlying hardware.

The shader in question isn't a compute shader.

I guess I misunderstood "Compute Shaders in VRChat" then.

It's about using the classic fragment shader (also sometimes referred to as pixel shader) + texture approach as a workaround for compute shaders not being available if the body wasn't clear.

The classical workaround, yeah I missed that, should have read it properly instead of just skimming through.

Thanks for calling me out on that.

I can totally see a “show HN: webapp that turns your GPU into a discount xeon phi” appearing…

Projects like this are so inspiring.

ha, it reminds me of the Croquet Project


Sounds like a headline optimized for HN success!

Interesting. Also, I would like to test VR walks such as the one from Albarracín, but my GPU only supports GL 2.1.

This is insane

I loaded it up in game and it proceeded to instantly make my VR experience go down to 10 FPS. I did manage to get a selfie of the Linux boot though: https://twitter.com/theprincessxena/status/14310429360457154...

so if you run VRC in 2D you could get Doom working ;}

There's no way someone hasn't implemented DOOM in a pixel shader already.

nice but I mean running Doom in the emulated linux instance so its a play within a play within a play .... ?

...Every pixel is an Operating System?

This is the kind of madness/genius that'll take us places.

Every 2048x2048 state texture is the state of an operating system, including CPU and memory. The pixel shaders operate on this state to advance different parts of the state of the operating system as a whole.

Thank-you _pi_, I loved this when I first saw it, and I love it now.

This is a thing that should not ("in the name of all that is holy") have been done; but being done, cannot but be admired.

can't wait until i see kubernetes running on risc-v in a job description


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact