Hacker News new | comments | show | ask | jobs | submit login
GPUs prefer premultiplication (realtimerendering.com)
132 points by dahart on Jan 16, 2016 | hide | past | web | favorite | 25 comments

In addition to the texturing & filtering argument Eric makes in this article, I think learning and using pre multiplied blending is extra important in WebGL because WebGL canvases are always composited over the page, and the default compositing mode is pre multipled.

I mocked up a fiddle to demonstrate this point. https://jsfiddle.net/dahart/m4z61tz1/

*edit: a little explanation, the fiddle shows, in the top half, blending using the most commonly learned method for non-premultiplied colors/textures, with gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);

In the bottom half, blending is done with pre-multiplied colors, and gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA).

The main point is how, using WebGL's default settings, you can see through what should be opaque areas on top, and the stuff on the bottom looks and behaves correctly.

The fiddle has enough toggles and comments in the code and fragment shader to get the non-pre multiplied blending & compositing to work correctly.

Do you have any recommended resources for learning WebGL?

http://webglfundamentals.org is a fantastic place to start.

The author of this blog post has a course on Udacity where he teaches the basics of real time rendering using three.js

In my experience, you should always turn off compositing in WebGL. The performance differences are sometimes rather extreme.

I've been in the "business" for almost 15 years, always hand waved my way through premultiplication. Excellent article, thanks to the author I've seen the light. That being said, the bleeding trick mentioned is widely in use, as it doesn't require modification of all shader pipelines.

Important paper by Alvy Ray Smith on this issue: http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf

Not sure why you're downvoted. This paper is great and understanding compositing math is something that comes way too late for most graphics programmers. Also great is Tom Forsyth's article on the subject, if you can figure out his blog UI: http://eelpi.gotdns.org/blog.wiki.html

Premultiplied alpha also means you don't have zero-alpha pixels with colour data.

That's basically the extreme end of the destructiveness of premultiplication.

You also lose precision in general, which can lead to a grey-brown soup at semi-transparent fringes. He mentions 16bit colors further down, but they're not always available, e.g. in webgl.

Or you use floating point textures.

But even the smallest floating point format still has 16 bits per channel.

R11F_G11F_B10F, RGB9_E5

Both pack floating point RGB into 32 bits per pixel. Combine that with a separate alpha channel and you're only at 40.

Don't ask me how spectacular the support is.

This confused me for a while, because of the poor example. The mushroom and flower textures have transparent black pixels and exhibit black fringe.

With transparent black pixels, premultiplying by alpha does nothing.

It's changing the shaders to interpret a value like (128, 0, 0, 128) differently that fixes the rendering.

Every non-zero color avoids this coincidence, and you have to change both image and shader to get the correct result. Just thought I'd post in case anyone else had the same confusion.

Read the article again. You've completely misinterpreted it.

I have not. When your number is already 0, multiplying it by 0 does not change anything. That's just a fact.

So in the case of this specific color, invisible black, you only need to change a shader to fix the problem. This, plus the author constantly talking about 'premultiplying' as the fix, made it harder for me to understand that the fix is equal parts the texture change and the shader change.

No, it's not about the black pixel at all. Of course zero times anything is zero. It's actually about the red pixel next to it.

The method the article details is the correct implementation for a variety of reasons that it goes into. Yes, there are different ways you can handle 100% transparent pixels on the edges, but only premultiplied alpha handles all edge cases that we're concerned about.

The red pixel is (255, 0, 0, 255). Premultiplying it by alpha makes it (255, 0, 0, 255).

It's not that the article says anything wrong, it's that it uses an example texture where there are no translucent pixels and no hidden colors. In a texture like that, premultiplying by alpha does nothing. In a texture like that, you only need to add a division by alpha to your shader.

The combination of using premultiplied textures and altered shaders solve the general case.

Using only altered shaders solves only an edge case.

But the article only shows that edge case.

That made it harder to follow.

The author of the article here. I know what you mean about my example. I in fact started with an example of red (255,0,0,255) texel next to fully transparent (0,0,0,0) texel, but found that example unconvincing - you don't really see the effect of premultiplication directly, as both texel values look exactly the same whether you premultiply or not. Really, another way to write the article would be to say, "whether or not you want it, the GPU will return premultiplied-looking values when you bilinear interpolate or mipmap." The interpolated value for (255,0,0,255) and (0,0,0,0) is (127.5,0,0,127.5). The only logical way to interpret this value is as a premultiplied value. I hope this helps. I'll add a sentence or two to the article along these lines, as it might help others to get what I'm trying to describe. Thanks for pointing out the confusion.

Eric, I've had this point in my head, but I missed that you were making it, and furthermore I missed the true meaning of your post's title!

It is true, and interesting, that whether you use unassociated colors and blending functions or you use premultiplied colors & blending functions, either way, what comes out is a premultiplied color!

This is true of any software compositor or renderer too, right? It's a fact of rendering, not just of GPUs, that output colors are premultiplied. The over operator multiplies colors by the source alpha values while you blend - that is where "premultiplication" happens when you render or composite something. Some software renderers have an "un-premultiply" flag to do a divide right at the end, so you can spit out files with unassociated colors in them, but otherwise anything rendered comes out premultiplied.

I believe this is exactly why WebGL defaults your canvas to premultiplied compositing over the rest of the page. Your output is premultiplied even when your input isn't. It is also a good reason to think in pre-multiplied colors and use the blending functions that assume pre-multiplied colors, and pay attention to, and convert/premultiply on the fly, when you need to accept unassociated color sources like hand-painted textures.

Is there a reason why GPUs can not do alpha-aware interpolation?

GPUs don't know whether the texture even has alpha data. All the GPU knows is that there is an image with 4 channels (and the data type of those channels), but it's the shader that gives meaning to those channels. Having three colour channels and one alpha channel is a common case, but you can also use the four channels for entirely different purposes, like 4 grayscale images, or encoding 4d coordinates as "image".

I thought everyone knew that PNG and insisted alphas in general wasn't exactly "friendly" to GPU's.

does this effect deep learning at all?


Applications are open for YC Summer 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact