
John Carmack: Steps to Avoid Aliasing in VR - phodo
https://www.facebook.com/permalink.php?story_fbid=1818885715012604&id=100006735798590
======
nighthawk454
Post content:

 _In almost all of the public VR critiques I have made, I comment about
aliasing. Aliasing is the technical term for when a signal is under sampled,
and it shows up in many ways across graphics (and audio), but in VR this is
seen most characteristically as "shimmering" imagery when you move your head
around, even slightly. This doesn't need to happen! When you do everything
right, the world feels rock solid as you look around. The photos in the Oculus
360 Photos app are properly filtered -- everything should ideally feel that
solid.

In an ideal world, this would be turned into a slick article with before-and-
after looping animated GIFs of each point I am making, along with screenshots
of which checkboxes you need to hit in Unity to do the various things. Maybe
someone in devrel can find the time...

All together in one place now, the formula for avoiding aliasing, from the
most important: Generate mip maps for every texture being used, and set
GL_LINEAR_MIPMAP_LINEAR filtering. This is easy, but missed by so many titles
it makes me cringe.

If you dynamically generate textures, there is a little more excuse for it,
but glGenerateMipmap() is not a crazy thing to do every frame if necessary.

Use MSAA for rendering. Not using MSAA should be an immediate fail for any app
submission, but I still see some apps with full jaggy edges. We currently
default to 4x MSAA on Mali and 2x MSAA on Adreno because there is a modest
performance cost going to 4x there. We may want to go 4x everywhere, and it
should certainly be the first knob turned to improve quality on a Qualcomm S7,
long before considering an increase in eye buffer resolution.

In general, using MSAA means you can't use a deferred rendering engine. You
don't want to for performance reasons, anyway. On PC, you can get by throwing
lots of resources at it, but not on mobile.

Even 4x MSAA still leaves you with edges that have a bit of crawl, since it
only gives you two bits of blending quality. For characters and the
environment you just live with it, but for simple things like a floating UI
panel, you can usually arrange to use alpha blending to get eight bits of
blending quality, which is essentially perfect. Blending requires sorting, and
ensuring a 0 alpha border around your images is a little harder than it first
appears, since the size of a border gets halved with each mip level you
descend. Depending on how shrunk down a piece of imagery may get, you may want
to leave an eight or more pixel cleared alpha border around it. To avoid
unexpected fringe colors around the graphics, make sure that the colors stay
consistent all the way to the edge, even where the alpha channel is 0. It is
very common to see blended graphics with a dark fringe around them because the
texture went to 0 0 0 0 right at the outline, rather than just cutting it out
in the alpha channel. Adding an alpha border also fixes the common problem of
not getting CLAMP_TO_EDGE set properly on UI tiles, since if it fades to
invisible at the edges, it doesn't matter if you are wrapping half a texel
from the other side.

Don't use techniques like alpha testing or anything with a discard in the
fragment shader unless you are absolutely sure that the contribution to the
frame buffer has reached zero before the pixels are discarded. The best case
for the standard cutout uses is to use blending, but if the sorting isn't
feasible, using ALPHA_FROM_COVERAGE is a middle ground. Avoid geometry that
can alias. The classic case is things like power lines -- you can model a nice
drooping catenary of thin little tubes or crossed ribbons, but as it recedes
into the distance it will rapidly turn into a scattered, aliasing mess of
pixels along the line with only the 1 or 2 bits of blending you get from MSAA.
There are techniques for turning such things into blends, but the easiest
thing to do is just avoid it in the design phase. Anything very thin should be
considered carefully.

Don't try to do accurate dynamic shadows on GearVR. Dynamic shadows are rarely
aliasing free and high quality even on AAA PC titles, cutting the resolution
by a factor of 16 and using a single sample so it runs reasonably performant
on GearVR makes it hopeless. Go back to Quake 3 style and just blend a blurry
blob underneath moving things, unless you really, really know what you are
doing.

Don't use specular highlights on bump maps. This is hard to get really right
even with significant resources. Specular at the geometry level (still
calculated in the fragment shader, not the vertex level!) is usually ok, and
also a powerful stereoscopic cue. This applies to any graphics calculation
done in shaders beyond just sampling a texture -- if it has any frequency
component greater than the pixel rate (a clamp is infinite frequency!), there
will be aliasing. Think very carefully before doing anything clever in a
shader.

Use gamma correct rendering. This is usually talked about in game dev circles
as part of "physically based rendering", and it does matter for correct
lighting calculations, but it isn't appreciated as widely that it matters for
texture sampling and MSAA even when no lighting at all is done. For photos and
other continuous tone images this barely matters at all, but for high contrast
line art it is still important. The most critical high contrast line art is
text. It is hard to ease into this, you need to convert your window, all
textures, and any frame buffer objects you use too the correct sRGB formats.
Some formats, like 4444, don't have an equivalent sRGB version, so it may
involve going to a full 32 bit format.

Avoid unshared vertexes. A model edge that abruptly changes between two
surfaces has the same aliasing characteristics as a silhouette at the edge of
a model; you only get the MSAA bits of blending. If textures are wrapped
completely around a model, and all the vertex normals and other attributes are
shared, you only get the edge effects along the "pelting lines" where you have
no choice but to have unmatched texture coordinates. It costs geometry, so it
isn't always advisable, but even seemingly hard edged things like a cube will
look better if you have a small bevel with shared vertexes crossing all the
edges, rather than a knife-sharp 90 degree angle. If a surface is using baked
lighting, the geometry doesn't need normals, and you are fine wrapping around
hard angles as long as the texture coordinates are shared.

Trilinear filtering on textures is not what you ideally want -- it blends a
too-aliased sample together with a too-blurry sample to try to limit the
negatives of each. High contrast textures, like, say, the floor grates in Doom
3, or the white shutters in Tuscany, can still visibly alias even with
trilinear. Reducing the contrast in the image by "prefiltering" it can fix the
problem, or you can programmatically add an LOD bias.

Once aliasing is under control, you can start looking at optimizing quality._

~~~
sbierwagen
fold --spaces --width 75 | sed -e 's/^/ /'

    
    
      In almost all of the public VR critiques I have made, I comment about
      aliasing. Aliasing is the technical term for when a signal is under
      sampled, and it shows up in many ways across graphics (and audio), but in
      VR this is seen most characteristically as "shimmering" imagery when you
      move your head around, even slightly. This doesn't need to happen! When
      you do everything right, the world feels rock solid as you look around.
      The photos in the Oculus 360 Photos app are properly filtered --
      everything should ideally feel that solid.
    
      In an ideal world, this would be turned into a slick article with
      before-and-after looping animated GIFs of each point I am making, along
      with screenshots of which checkboxes you need to hit in Unity to do the
      various things. Maybe someone in devrel can find the time...
    
      All together in one place now, the formula for avoiding aliasing, from the
      most important:
    
      Generate mip maps for every texture being used, and set
      GL_LINEAR_MIPMAP_LINEAR filtering. This is easy, but missed by so many
      titles it makes me cringe.
    
      If you dynamically generate textures, there is a little more excuse for
      it, but glGenerateMipmap() is not a crazy thing to do every frame if
      necessary.
    
      Use MSAA for rendering. Not using MSAA should be an immediate fail for any
      app submission, but I still see some apps with full jaggy edges. We
      currently default to 4x MSAA on Mali and 2x MSAA on Adreno because there
      is a modest performance cost going to 4x there. We may want to go 4x
      everywhere, and it should certainly be the first knob turned to improve
      quality on a Qualcomm S7, long before considering an increase in eye
      buffer resolution.
    
      In general, using MSAA means you can't use a deferred rendering engine.
      You don't want to for performance reasons, anyway. On PC, you can get by
      throwing lots of resources at it, but not on mobile.
    
      Even 4x MSAA still leaves you with edges that have a bit of crawl, since
      it only gives you two bits of blending quality. For characters and the
      environment you just live with it, but for simple things like a floating
      UI panel, you can usually arrange to use alpha blending to get eight bits
      of blending quality, which is essentially perfect. Blending requires
      sorting, and ensuring a 0 alpha border around your images is a little
      harder than it first appears, since the size of a border gets halved with
      each mip level you descend. Depending on how shrunk down a piece of
      imagery may get, you may want to leave an eight or more pixel cleared
      alpha border around it. To avoid unexpected fringe colors around the
      graphics, make sure that the colors stay consistent all the way to the
      edge, even where the alpha channel is 0. It is very common to see blended
      graphics with a dark fringe around them because the texture went to 0 0 0
      0 right at the outline, rather than just cutting it out in the alpha
      channel. Adding an alpha
      border also fixes the common problem of not getting CLAMP_TO_EDGE set
      properly on UI tiles, since if it fades to invisible at the edges, it
      doesn't matter if you are wrapping half a texel from the other side.
    
      Don't use techniques like alpha testing or anything with a discard in the
      fragment shader unless you are absolutely sure that the contribution to
      the frame buffer has reached zero before the pixels are discarded. The
      best case for the standard cutout uses is to use blending, but if the
      sorting isn't feasible, using ALPHA_FROM_COVERAGE is a middle ground.
    
      Avoid geometry that can alias. The classic case is things like power lines
      -- you can model a nice drooping catenary of thin little tubes or crossed
      ribbons, but as it recedes into the distance it will rapidly turn into a
      scattered, aliasing mess of pixels along the line with only the 1 or 2
      bits of blending you get from MSAA. There are techniques for turning such
      things into blends, but the easiest thing to do is just avoid it in the
      design phase. Anything very thin should be considered carefully.
    
      Don't try to do accurate dynamic shadows on GearVR. Dynamic shadows are
      rarely aliasing free and high quality even on AAA PC titles, cutting the
      resolution by a factor of 16 and using a single sample so it runs
      reasonably performant on GearVR makes it hopeless. Go back to Quake 3
      style and just blend a blurry blob underneath moving things, unless you
      really, really know what you are doing.
    
      Don't use specular highlights on bump maps. This is hard to get really
      right even with significant resources. Specular at the geometry level
      (still calculated in the fragment shader, not the vertex level!) is
      usually ok, and also a powerful stereoscopic cue. This applies to any
      graphics calculation done in shaders beyond just sampling a texture -- if
      it has any frequency component greater than the pixel rate (a clamp is
      infinite frequency!), there will be aliasing. Think very carefully before
      doing anything clever in a shader.
      
      Use gamma correct rendering. This is usually talked about in game dev
      circles as part of "physically based rendering", and it does matter for
      correct lighting calculations, but it isn't appreciated as widely that it
      matters for texture sampling and MSAA even when no lighting at all is
      done. For photos and other continuous tone images this barely matters at
      all, but for high contrast line art it is still important. The most
      critical high contrast line art is text. It is hard to ease into this, you
      need to convert your window, all textures, and any frame buffer objects
      you use too the correct sRGB formats. Some formats, like 4444, don't have
      an equivalent sRGB version, so it may involve going to a full 32 bit
      format.
    
      Avoid unshared vertexes. A model edge that abruptly changes between two
      surfaces has the same aliasing characteristics as a silhouette at the edge
      of a model; you only get the MSAA bits of blending. If textures are
      wrapped completely around a model, and all the vertex normals and other
      attributes are shared, you only get the edge effects along the "pelting
      lines" where you have no choice but to have unmatched texture coordinates.
      It costs geometry, so it isn't always advisable, but even seemingly hard
      edged things like a cube will look better if you have a small bevel with
      shared vertexes crossing all the edges, rather than a knife-sharp 90
      degree angle. If a surface is using baked lighting, the geometry doesn't
      need normals, and you are fine wrapping around hard angles as long as the
      texture coordinates are shared.
    
      Trilinear filtering on textures is not what you ideally want -- it blends
      a too-aliased sample together with a too-blurry sample to try to limit the
      negatives of each. High contrast textures, like, say, the floor grates in
      Doom 3, or the white shutters in Tuscany, can still visibly alias even
      with trilinear. Reducing the contrast in the image by "prefiltering" it
      can fix the problem, or you can programmatically add an LOD bias.
    
      Once aliasing is under control, you can start looking at optimizing
      quality.

~~~
vacri
TIL 'fold', thank you.

I've always been a big fan of including the command you use to get the output
you're quoting - people sometimes pick up even subtle tricks or flags from
doing this.

~~~
taeric
Oddly this is one of the tricks in Programming Pearls that I love, but can't
get others to want to do.

It doesn't help that json prohibited comments. So, anything that modified json
is incapable of adding a breadcrumb of what it did.

------
post_break
I remember reading that the Galaxy phones have trouble with pixel refresh
rates. In that blacks can ghost because the nature of amoled.

~~~
sp332
The Rift and Vive also use AMOLED screens, so I don't think it's the pixel
tech that's the issue. LEDs can turn on and off extremely fast. It's probably
that the screen's electronics were made for power-sipping mobile devices
running at most 60 FPS and not 90+ FPS gaming.

~~~
cma
OLED is the issue. Look into the black smear problem on Oculus DK2 and how
they had to half-way fix it by overdriving for a frame and then falling back.

CV1 of the Rift headset is OLED and has the same issue. They have to run the
panels at a notch above minimum brightness to work around it and as a result
they lose out on true blacks.

[https://vrwiki.wikispaces.com/Black+smearing](https://vrwiki.wikispaces.com/Black+smearing)

~~~
sp332
None of those threads mention the Rift CV1, the Vive, or the Samsung Galaxy S6
or S7 (the ones that fit in the current Gear VR headsets). I tried the test
image from this thread [http://forum.xda-
developers.com/showthread.php?t=2765793](http://forum.xda-
developers.com/showthread.php?t=2765793) on my S6 and didn't see the issue.
That thread and this one
[https://forums.oculus.com/community/discussion/10924/true-
bl...](https://forums.oculus.com/community/discussion/10924/true-black-
smearing-in-dk2) say it's only an issue around areas where the screen is
actually black, so keeping things very dark would seem to be worse?

~~~
cma
I think I wasn't clear. It isn't a notch above minimum brightness as in
running the panel very dim.

It is a notch above minimum as in: minimum is pure black, and instead of every
allowing pure black they set it to one notch higher, because pure black causes
the smearing.

It isn't an visible issue in Rift CV1 or Vive because they do that as a
workaround.

------
tgb
What's the problem with specular lighting on bump maps? I think I see what the
issue can be, specifically, that when the bump map texture is smaller than 1-1
texels-to-pixels, then the normal chosen will 'shimmer' between the possible
texel values. But can't one use mipmaps for bump maps as well to reduce this?

~~~
Ono-Sendai
I think the problem with mimmapping bump maps is that it effectively smooths
the surface, so you loose the bump effect.

~~~
tgb
Can't the same be said for mipmapping textures? Losing detail at low LOD is
expected, though I can't picture if bump maps lose 'more' perceptually.

------
Joof
Anyone here doing HTC vive dev? I'm really interested myself, but as a poor
student I'll settle for hearing other's stories.

~~~
kristianp
I guess you could try google cardboard?

------
SteveWatson
unclosable popup

~~~
natthub
There is a "Not now" link to partially hide the popup.

~~~
theoh
Upvoted you, but the substantive point surely is that Facebook is a bad web
citizen for doing this. OTOH every sane person already has a (negative)
opinion about Facebook and this is not the forum to air those.

------
sintaxi
In an ideal world this article would not be posted on Facebook.

~~~
bertil
That article is shared to 'Public': it should be visible to non-logged-in
users. Not sure what changed — could be an issue with excessive crawlers…

I’ve very convinced that Facebook is very comfortable to let the minority who
are not comfortable having an account read a public post -- especially when
it’s about engineering. Those are usually on the company blog (without log-in
block); Carmack probably was just using internal tool, and didn’t want to
bother.

It’s probably less high on the priority list than “give affordable internet
access to 1 billion people”, but in an ideal world, posts that are meant for
“everyone” should not be gated.

~~~
swang
i can read it logged out of facebook

unfortunately there is an annoying box covering up 1/3 of the page asking you
to login or signup. it does not "block" any content in that you can still read
all of it. but it is highly annoying (and i assume purposely so).

~~~
cowsandmilk
The main question is why Carmack doesn't use the Facebook Engineering Blog.
There are no popups or other annoyances there. For example:

[https://code.facebook.com/posts/265413023819735/surround-360...](https://code.facebook.com/posts/265413023819735/surround-360-is-
now-open-source/)

~~~
bertil
Facebook employees generally communicate internal results or opinion with
Facebook posts, a lot like this, using a Company-only privacy setting. Posts
like this one sound like something he posted internally first, and just
reposted, or changed the privacy setting. I wouldn’t read any more into it
than marginal laziness.

------
pipio21
Carmack: Some times you read HN. If you are reading this, please consider
writing on a blog or something neutral.

It is extremely disgusting for your readers to have to log in or else...half
of your screen is occupied with spam.

There is people out there that prefer not to report everything they read,
when, where, how, how long to a big multinational associated to the US secret
services.

~~~
munchbunny
"Extremely disgusting" sounds like an overreaction. "Not ideal" maybe, or "not
helpful to those of us who don't want to read things in Facebook for privacy
or security reasons". But "extremely disgusting" is over-exaggerating the
importance of the choice of medium for a text post.

~~~
tripzilch
I was going to say "criminally atrocious" but then the popup div literally
made me vomit excrement.

------
jd3
there is literally nothing wrong with jaggies or aliasing

------
qwertyuiop924
New project: A script that extracts public posts from Facebook for a given
user, and formats them for consumption via RSS, or generates content/markdown
to be served by Jekyll/Lektor. I should get on that...

~~~
FlailFast
Might I humbly suggest a project name? FFSS. For F(acebook|uck)'s Sake
Syndication. :)

~~~
8note
You just wrote Facebook Fuck's Sake Syndication with 1 extra character, and
with more reading difficulty!

~~~
teach
Looks like you need to brush up on your regular expressions.

> For F(acebook|uck)'s Sake Syndication

The pipe (|) means or, so the regex is indicating that FFSS can stand for
either:

For Facebook's Sake Syndication, or

For Fuck's Sake Syndication

I thought it was clever, anyway.

~~~
qwertyuiop924
I don't know about clever, but it's a common idiom on tech forums.

