
GPUs prefer premultiplication - dahart
http://www.realtimerendering.com/blog/gpus-prefer-premultiplication/
======
dahart
In addition to the texturing & filtering argument Eric makes in this article,
I think learning and using pre multiplied blending is extra important in WebGL
because WebGL canvases are always composited over the page, and the default
compositing mode is pre multipled.

I mocked up a fiddle to demonstrate this point.
[https://jsfiddle.net/dahart/m4z61tz1/](https://jsfiddle.net/dahart/m4z61tz1/)

*edit: a little explanation, the fiddle shows, in the top half, blending using the most commonly learned method for non-premultiplied colors/textures, with gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);

In the bottom half, blending is done with pre-multiplied colors, and
gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA).

The main point is how, using WebGL's default settings, you can see through
what should be opaque areas on top, and the stuff on the bottom looks and
behaves correctly.

The fiddle has enough toggles and comments in the code and fragment shader to
get the non-pre multiplied blending & compositing to work correctly.

~~~
TACIXAT
Do you have any recommended resources for learning WebGL?

~~~
dahart
[http://webglfundamentals.org](http://webglfundamentals.org) is a fantastic
place to start.

------
badlogic
I've been in the "business" for almost 15 years, always hand waved my way
through premultiplication. Excellent article, thanks to the author I've seen
the light. That being said, the bleeding trick mentioned is widely in use, as
it doesn't require modification of all shader pipelines.

------
theoh
Important paper by Alvy Ray Smith on this issue:
[http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf](http://alvyray.com/Memos/CG/Microsoft/4_comp.pdf)

~~~
chadaustin
Not sure why you're downvoted. This paper is great and understanding
compositing math is something that comes way too late for most graphics
programmers. Also great is Tom Forsyth's article on the subject, if you can
figure out his blog UI:
[http://eelpi.gotdns.org/blog.wiki.html](http://eelpi.gotdns.org/blog.wiki.html)

------
TazeTSchnitzel
Premultiplied alpha also means you don't have zero-alpha pixels with colour
data.

~~~
the8472
That's basically the extreme end of the destructiveness of premultiplication.

You also lose precision in general, which can lead to a grey-brown soup at
semi-transparent fringes. He mentions 16bit colors further down, but they're
not always available, e.g. in webgl.

~~~
wolfgke
Or you use floating point textures.

~~~
wongarsu
But even the smallest floating point format still has 16 bits per channel.

~~~
Dylan16807
R11F_G11F_B10F, RGB9_E5

Both pack floating point RGB into 32 bits per pixel. Combine that with a
separate alpha channel and you're only at 40.

Don't ask me how spectacular the support is.

------
Dylan16807
This confused me for a while, because of the poor example. The mushroom and
flower textures have transparent black pixels and exhibit black fringe.

With transparent black pixels, premultiplying by alpha _does nothing_.

It's changing the shaders to interpret a value like (128, 0, 0, 128)
differently that fixes the rendering.

Every non-zero color avoids this coincidence, and you have to change both
image and shader to get the correct result. Just thought I'd post in case
anyone else had the same confusion.

~~~
moron4hire
Read the article again. You've completely misinterpreted it.

~~~
Dylan16807
I have not. When your number is already 0, multiplying it by 0 does not change
anything. That's just a fact.

So in the case of _this specific color_ , invisible black, you only need to
change a shader to fix the problem. This, plus the author constantly talking
about 'premultiplying' as the fix, made it harder for me to understand that
the fix is equal parts the texture change and the shader change.

~~~
moron4hire
No, it's not about the black pixel at all. Of course zero times anything is
zero. It's actually about the red pixel next to it.

The method the article details is the correct implementation for a variety of
reasons that it goes into. Yes, there are different ways you can handle 100%
transparent pixels on the edges, but only premultiplied alpha handles all edge
cases that we're concerned about.

~~~
Dylan16807
The red pixel is (255, 0, 0, 255). Premultiplying it by alpha makes it (255,
0, 0, 255).

It's not that the article says anything wrong, it's that it uses an example
texture where there are no translucent pixels and no hidden colors. In a
texture like that, premultiplying by alpha does nothing. In a texture like
that, you only need to add a division by alpha to your shader.

The combination of using premultiplied textures and altered shaders solve the
general case.

Using only altered shaders solves only an edge case.

But the article only shows that edge case.

That made it harder to follow.

~~~
erich666
The author of the article here. I know what you mean about my example. I in
fact started with an example of red (255,0,0,255) texel next to fully
transparent (0,0,0,0) texel, but found that example unconvincing - you don't
really see the effect of premultiplication directly, as both texel values look
exactly the same whether you premultiply or not. Really, another way to write
the article would be to say, "whether or not you want it, the GPU will return
premultiplied-looking values when you bilinear interpolate or mipmap." The
interpolated value for (255,0,0,255) and (0,0,0,0) is (127.5,0,0,127.5). The
only logical way to interpret this value is as a premultiplied value. I hope
this helps. I'll add a sentence or two to the article along these lines, as it
might help others to get what I'm trying to describe. Thanks for pointing out
the confusion.

~~~
dahart
Eric, I've had this point in my head, but I missed that you were making it,
and furthermore I missed the true meaning of your post's title!

It is true, and interesting, that whether you use unassociated colors and
blending functions or you use premultiplied colors & blending functions,
either way, what comes out is a premultiplied color!

This is true of any software compositor or renderer too, right? It's a fact of
rendering, not just of GPUs, that output colors are premultiplied. The over
operator multiplies colors by the source alpha values while you blend - that
is where "premultiplication" happens when you render or composite something.
Some software renderers have an "un-premultiply" flag to do a divide right at
the end, so you can spit out files with unassociated colors in them, but
otherwise anything rendered comes out premultiplied.

I believe this is exactly why WebGL defaults your canvas to premultiplied
compositing over the rest of the page. Your output is premultiplied even when
your input isn't. It is also a good reason to think in pre-multiplied colors
and use the blending functions that assume pre-multiplied colors, and pay
attention to, and convert/premultiply on the fly, when you need to accept
unassociated color sources like hand-painted textures.

------
zokier
Is there a reason why GPUs can not do alpha-aware interpolation?

~~~
wongarsu
GPUs don't know whether the texture even has alpha data. All the GPU knows is
that there is an image with 4 channels (and the data type of those channels),
but it's the shader that gives meaning to those channels. Having three colour
channels and one alpha channel is a common case, but you can also use the four
channels for entirely different purposes, like 4 grayscale images, or encoding
4d coordinates as "image".

------
cbsmith
I thought everyone knew that PNG and insisted alphas in general wasn't exactly
"friendly" to GPU's.

------
toisanji
does this effect deep learning at all?

~~~
shock-value
No.

