
Adventures with ffmpeg and color ranges - doppp
https://www.facebook.com/permalink.php?story_fbid=2413101932257643&id=100006735798590
======
hiisukun
Not sure if this kind of meta-comment is appropriate, but I find it difficult
to follow a link like this because of the lack of human readable information
contained in the URL.

If the story was hosted on a personal blog I recognised (like Julia Evans to
take a random example), then perhaps the domain would be enough for me to
choose to read the article to see if the content is interesting.

If the link was to a broader domain that hosts many writers, I might use the
parts after the domain to filter my interest: example.com/news-opinion vs
example.com/science-blog gives me more information that might not be clear
from the post title.

With a site like Facebook, which hosts so much different value levels of
content, it is harder for me to visit facebook.com/233555323 rather than
facebook.com/ffmpeg-interest-group/post6665443. For whatever reason, the URL
remains something I use to filter my internet consumption.

Perhaps it has something to do with the amount of scams or other general
reasons not to trust non-human readable URLs, or perhaps it is more due to my
distrust of article titles (sensationalism) and seeking a second source to
verify my interest. Perhaps I simply bias against Facebook - I don't know.

Regardless, I visited the story after thinking about this, and was very
surprised to see it was an article by John Carmack - an author I certainly
respect. It is a shame for me that both the URL and the title don't reflect
that.

~~~
dtf
Another site that comes to mind is the Financial Times, which uses UUIDs for
article urls, with no other indication of content.

[https://www.ft.com/](https://www.ft.com/)

[https://www.ft.com/content/13be9132-8149-11e9-b592-5fe435b57...](https://www.ft.com/content/13be9132-8149-11e9-b592-5fe435b57a3b)

~~~
nikkwong
Oh wow, you really have to hope these are not the canonical links. If so, they
are doing themselves an incredible disservice and the fact that no one has
brought it to management's attention speaks gravely to the workplace culture
there.

~~~
saurik
Not only does the full URL not show in my browser (Safari), the full URL
doesn't even show on this forum (for me it is cut off after "content"). When
you receive a URL via chat or on social networks, it is in the form of a
preview and might not even show the domain name. The user also isn't typing
these URLs in, with or without human readable text in the path. Essentially
all of the arguments for making URLs more brittle (which is what you are doing
by adding human readable text) are obsolete in a world where URLs have been
relegated back to what they were always supposed to be: permanent identifiers
to a semantically stable unit of content.

~~~
shawabawa3
> Essentially all of the arguments for making URLs more brittle (which is what
> you are doing by adding human readable text)

Not if done right, like e.g. stack overflow, see
[https://stackoverflow.com/questions/517355/string-
formatting...](https://stackoverflow.com/questions/517355/string-formatting-
in-python)

The human readable text is actually meaningless, and anything after the id
will redirect there, e.g. [https://stackoverflow.com/questions/517355/this-
could-be-any...](https://stackoverflow.com/questions/517355/this-could-be-
anything)

~~~
saurik
This technique (which I almost commented on in my original comment, but
didn't) is brittle in a different way (making it so that attempts to search
for the URL have no well-known canonical representation), and quite simply
totally undermines the entire argument being made (which is why I didn't
comment on it) that someone can use the slug of a URL to decide whether or not
they want to click on the content: you can change the URL to say anything you
want. At this point, there is no value in bothering with having done this: it
is just extra text in the URL for no concrete benefit.

meta.stackexchange.com/questions/148454/why-does-stack-overflow-use-title-
slugs/

------
ajxs
Funnily enough, I actually ended up giving myself a crash course in this very
topic when encoding frag movies for Quake 3/Quake Live. It was/is common
practice to use 'brightskins' on your enemy models in competitive play to make
them easier to see, typically being a colour like '#00FF00'. I noticed that
when uploading to online media hosting sites like Youtube these bright colours
would become noticeably washed out and degraded. My investigation of why this
happened led me all the way down this rabbit hole. This was the better half of
ten years ago now so I have no idea if it still occurs, but I ended up just
rendering the videos using a better brightskin colour that appeared less
degraded after conversion... Going through the cycle of
'rerender/reupload/check' to find a replacement colour again and again to
experiment sure was a major pain!

------
m0dest
For all of our sanity, please don't start producing 8-bit H.264 videos with
full 0-255 range.

If you have enough control over the _decoder and renderer_ to make that hack
work, you should just be producing 10-bit HEVC/AV1 video – which does not
suffer from these 8-bit banding issues – with a modern color space (Rec 2020,
PQ, etc.).

And if you _don 't_ have choice or control over the decoder and renderer, then
it's a good sign that this would be a fragile hack.

~~~
JohnCarmack
My primary concern is for immersive VR media on our Go and Quest platforms,
which use Snapdragon 821 and 835 SoC, respectively. The 821 doesn’t support 10
bit on anything, and the 835 only supports it on h265. Software decoding a
4k60 video on mobile isn’t an option, and you can see every bit transition
with dark adapted eyes in a vr headset, so getting full 8 bit range is pretty
valuable.

~~~
thrwaycolor
You forget to mention which colorspace you're using. It matters.

From Rec. 2020 wikipedia article: Since a larger color space increases the
difference between colors an increase of 1-bit per sample is needed for Rec.
2020 to equal or exceed the color precision of Rec. 709.

The "correct" solution is to use a modern colorspace (rec 2020) with 10 bits
encoding (or 12 bits, if you can). UHD standard mandates at least 10 bits.

On the '821 you _could_ gain 1 bit by degrading to an older color space - 709.

Full 0-255 range should never be used - they are invalid according to the
standard. None of the hardware codecs are validated for 0-255 - you're asking
for trouble here. Expect ghosting and weird motion artifacts accumulating from
one I-frame to the next I-frame.

~~~
the-dude
Are you sure about that? Logitech sells H264-encoding cameras which do
yuvj420p ( which is full range as I understand it ).

YT and other platforms have no problem ingesting these videos. Also the RPi
Broadcom chipset does not seem to care.

------
zaroth
> _This is usually visible as ugly banding or blocking in dark scenes._

I was just complaining about this a couple weeks ago with some friends after
the “Dark” episode of GoT. I was saying the codecs don’t seem to have caught
up or they are just being fed at too low a bit rate to really excel at dark
scenes.

It’s hard to imagine this is just a scaling issue with the color ranges, and
if so, how could it possibly not have been fixed?!

I have a high end OLED screen in a very dark room, and I was very disappointed
by the picture quality. Enough to wish I hadn’t watched it through HBO Go and
that I had a better source.

I would love to see some before/after scenes showing the effect of the patch.

~~~
stevenwoo
The Amazon 720P version had noticeable banding several times during that
episode that was distracting even the day after so it's probably not a
bandwidth issue, the Dothraki charge into the dark might have been a cool idea
on paper but what a disappointment in execution. I think the director and
cinematographer are just ignorant of how stuff gets delivered to consumers.

[https://news.avclub.com/was-game-of-thrones-the-long-
night-t...](https://news.avclub.com/was-game-of-thrones-the-long-night-too-
dark-to-see-t-1834416256)

~~~
zaroth
From a tweet linked from that article:

[https://pbs.twimg.com/media/D5SmDG7WwAAMhdf.jpg:large](https://pbs.twimg.com/media/D5SmDG7WwAAMhdf.jpg:large)

I think that particular screenshot makes it look even worse than it was, but
it was definitely sub-par. I assume that a hardcopy 4K Bluray would not
exhibit the same effects?

------
brownbat
Interesting find.

I have encoded video for over a decade, and I don't think a two month period
has gone by where I haven't learned something that convinces me to completely
change how or what flags I'm using.

This is way beyond that but still.

~~~
PorterDuff
At the risk of telling you something you know, there's a lot more things to
trip on with proper color space conversion than just going from scaled to full
range color in 601. It's something of rat's nest to cover all the cases.

~~~
dylan604
After ProRes became a thing, there was a lot of confusion where the metadata
saved within the MOV started to include the HD vs SD color primaries. Some
programs were able to correctly read this data. Most did not. Most programs
did not include the data including Final Cut. The iTunes team started
rejecting files exported from FCP because they claimed they were not made
correctly and not a valid file. The iTunes team were just reading the spec and
built their automation based on that, but not what the real world output from
FCP was doing.

------
Quarrelsome
I still find it kinda odd that this is still the best tool out there for this
sort of work. Archaic parameters and options and exceptionally hard to read
documentation.

Maybe I'm just dumb and not in the domain enough to get a decent handle but
everytime I use ffmpeg I feel like a Norse Shaman trying through trial and
error to locate the correct orientation of runes for the blessing.

~~~
Hello71
there are two problems here. the main problem is that nobody reads the manual,
which actually starts out half-decent. the description is very clear, concise,
and even contains the oft-whinged-about examples.
[https://ffmpeg.org/ffmpeg.html](https://ffmpeg.org/ffmpeg.html)

the second problem is that ffmpeg has a _lot_ of features, and nobody wants to
go through and fully document hundreds of formats, codecs, and filters.

~~~
Daemon404
Part of the problem is that the documentation is so vast (the page you link is
only one of many, many pages; the documentation has been split out into
multiple pages for a while now) that as a user, it's fairly insane to try and
consume and comprehend it all, especially if all you want to do is something
quick.

------
airstrike
[http://archive.is/K5Fpv](http://archive.is/K5Fpv)

~~~
nvartolomei
TIL: EE, a mobile provider, blocks archive.is as unsafe content for people
under 18.
[https://twitter.com/nvartolomei/status/1133644305812381696](https://twitter.com/nvartolomei/status/1133644305812381696)

~~~
cr3ative
That sounds correct to me, since it could contain NSFW content, and by
default, mobile providers in England have content controls switched on.

------
slavik81
The ffmpeg documentation for YouTube videos[1] suggests using `-pix_fmt
yuv420p` with H.264 and there are warnings against using other pixel formats
(such as yuv444p) for compatibility reasons. Should the documentation instead
be suggesting `-pix_fmt yuvj420p`? What are the risks associated with choosing
that output format?

[1]:
[https://trac.ffmpeg.org/wiki/Encode/YouTube](https://trac.ffmpeg.org/wiki/Encode/YouTube)

~~~
_Gyan_
No, limited range is the most common and expected range for YUV videos. Many
players don't look or respect the container metadata such as nclx in MP4 or
the bitstream VUI flags and assume limited range.

~~~
bscphil
This is correct. In fact, the following statement from the post is nonsense:

> For arcane reasons related to TV broadcast limitations, many video formats
> restrict the YUV color components to be in the 16..235 or 16..240 range
> instead of the full 0..255 range. Losing 14% of the already-barely-enough 8
> bit dynamic range is bad enough, but it also often results in the black
> range starting at a quite visible grey value because most players don’t
> rescale the range. This is usually visible as ugly banding or blocking in
> dark scenes.

Of _course_ players "rescale" the range. (It's not exactly rescaling, it's
just that a value of 235 _IS_ reference white in Rec. 709.) In fact, the
thought if them _not_ doing this is kind of crazy, as Rec. 709 (used in
Blurays and elsewhere) as well as Rec. 601 explicitly use limited range YUV.
Players that don't use limited range correctly would be out of spec and
couldn't play a single Bluray disc correctly. So for the overwhelming majority
of use cases, you want regular old yuv420p.

The author _is_ picking up on something that actually happens, however. There
are Blurays out there that have incorrectly converted YUV values. This happens
when for example something is in limited range YUV, and then that data is
passed to a converter that treats the data as _full_ range YUV, and compresses
it _again_ to fit in limited range YUV. This "doubly compressed" data is
embarrassingly common (maybe 5% or so of Blurays have this problem). If you
get a Bluray like this, or any video with the same issue, the blacks will be
bright grays and the whites will be dull (similar to how the author describes
limited range YUV). The player on the end _is_ reading the YUV values
correctly, but the data has been doubly compressed so it only undoes one round
of this.

The issue is well known in the pirate communities as an example. Reliable
release groups always fix these broken Blurays in their encodes. This is
usually described as "fixing levels" or something like that.

~~~
josteink
Interesting post!

I have one question about terminology though: when you refer to “players“, do
you mean all kinds of players or just physical hardware devices like Blu-ray
players?

I ask because I haven’t had/observed any issues or defects with colour in
software/PC-based desktop serious at all.

Disclaimer: not in the Blu-ray-authoring business.

~~~
layoutIfNeeded
It’s pretty easy to verify: create a uniform white png and convert it to
yuv420p h264 via ffmpeg:

ffmpeg -i input.png -pix_fmt yuv420p movie.mp4

Open in VLC or your favourite player and take a screenshot. Now check with a
color picker whether you have white as 255 or 235.

Update: checked with VLC and QuickTime Player, both handled it correctly and
displayed white as 255.

------
waivek
I see a lot of people confused about the host (Facebook). Carmack initially
posted a lot of high quality articles on AltDevBlogADay. However, that site
went down. Carmack said he moved to Facebook because he feels it's more
reliable than other (cleaner) third-party hosts, which might not be around 10
years from now.

~~~
Arkanosis
I've seen a lot of similar posts hosted on Google Plus and shared on HN in the
past. They are all gone, now.

Sure, Google has a horrible track record when it comes to service durability,
but I wouldn't trust Facebook as a reliable host either. Anyone having relied
on their @facebook.com email address (2010-2016) may agree.

------
forgotmypw5
John Carmack 7 hrs ·

Adventures with ffmpeg and color ranges.

A video legacy issue that can cause a lot of problems is the issue of
“limited” versus “full” component range. For arcane reasons related to TV
broadcast limitations, many video formats restrict the YUV color components to
be in the 16..235 or 16..240 range instead of the full 0..255 range. Losing
14% of the already-barely-enough 8 bit dynamic range is bad enough, but it
also often results in the black range starting at a quite visible grey value
because most players don’t rescale the range. This is usually visible as ugly
banding or blocking in dark scenes.

For ffmpeg, the trick to avoid this is to use ‘-pix_fmt yuvj420p’, which says
to use the j-for-jpeg full range in a 420 YUV subsampled p-for-planar format.

If you are starting with either RGB images, a 10/12 bit format, or a yuvj420p
format video as input, then with the libx264 codec, you would get a full range
output. Note that any video processing tools used along the way could also
limit the range, and once it is gone, there is no getting it back, so you must
be very careful and check your entire pipeline!

When you us this format, ffmpeg complains about ‘deprecated pixel format used,
make sure you did set range correctly’, but you should ignore this warning.

Ffmpeg would like the world to move to specifying the range independently from
the YUV channel subsampling and layout:

Setting ‘-color_range 0 -pix_fmt yuv420p’ makes the output format yuv420p
Setting ‘-color_range 1 -pix_fmt yuv420p’ makes the output format yuv420p(tv)
Setting ‘-color_range 2 -pix_fmt yuv420p’ makes the output format yuv420p(pc)

Unfortunately, this isn’t yet uniformly handled throughout all the internal
format tests.

The libx265 integration in ffmpeg didn’t support the deprecated yuvj420p pixel
format, only the basic yuv420p one, and no matter what I did, my test videos
were always coming out limited range. It didn’t matter if you add a
’-color_range 2’, or ‘-x265-params range=full’. Those will change the settings
in the VUI (Video Usability Information) section of the output, but the values
are still compressed to the limited range.

Regardless of the input data, any 8 bit h265 video coming out of ffmpeg was
limited range!

I walked through the libx265 code looking for range compression, but it turned
out that all the damage was being done by ffmpeg before it got to x265. Ffmpeg
will automatically convert formats when the output differs from the input, and
since x265 only supported yuv420p, any full range input will be processed.

Adding ‘-v 48’ to ffmpeg will dump more information, including this
auto_scaler invocation, which is what is killing the full color range:
[auto_scaler_0 @ 000001f55ecd1a40] w:2048 h:2048 fmt:bgr24 sar:0/1 -> w:2048
h:2048 fmt:yuv420p sar:0/1 flags:0x4

I was about to start hacking the code to at least do what I wanted for my use
case, but I tried an appeal to Twitter:

[https://twitter.com/ID_AA_Carmack/status/1131715388067274753](https://twitter.com/ID_AA_Carmack/status/1131715388067274753)

My suspicions were confirmed, but Derek Buitenhuis went ahead and submitted an
official patch to get yuvj420p accepted by libx265, and windows builds are
already available at
[https://ffmpeg.zeranoe.com/builds/](https://ffmpeg.zeranoe.com/builds/).

I suspect there was probably some way of working around this involving
explicit format conversion filters with -src_range and -dst_range overrides,
but this is now working as you would expect it:

ffmpeg -i source.mp4 -c:v libx265 -pix_fmt yuvj420p dest.mp4

Bravo to ffmpeg!

------
Hello71
also see "Falsehoods programmers believe about [video stuff]":
[https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-
be...](https://haasn.xyz/posts/2016-12-25-falsehoods-programmers-believe-
about-%5Bvideo-stuff%5D.html). previously discussed:
[https://news.ycombinator.com/item?id=13259686](https://news.ycombinator.com/item?id=13259686).

------
bArray
Version you can view with JS disabled:
[https://mbasic.facebook.com/permalink.php?story_fbid=2413101...](https://mbasic.facebook.com/permalink.php?story_fbid=2413101932257643&id=100006735798590&_fb_noscript=1)

------
dearrifling
Fun fact: sRGB also has a non-black black point that is (fortunately)
virtually ignored by everyone.

~~~
devit
Source?

As far as I can tell in XYZ black is (0,0,0) and the only sRGB value that maps
to that is (0,0,0)

~~~
klodolph
This is incorrect.

If sRGB (1,1,1) is XYZ (1,1,1), then sRGB (0,0,0) is XYZ
(0.0025,0.0025,0.0025).

Or if you are using absolute XYZ values, sRGB (0,0,0) is XYZ
(0.1901,0.2,0.2178).

------
VectorLock
Its weird to think how John Carmack went from .plan files to Facebook Posts. I
think it might be safe to say he had the most widely read .plan file in
existence. Wish he would go back to it.

~~~
jakear
I suspect it’s not his decision to make.

~~~
cookiecaper
That's totally absurd. Luminaries like Carmack are anything but prisoners; any
firm in SV would pay big bucks to say "John Carmack works here".

Facebook may be a big conglomerate these days, and they may not be the cool
kid on the block anymore, but that doesn't mean that they're idiots. If
anything, they treasure the presence of someone like Carmack all the more,
because it's no longer reputionally-free for people like him to come on board.
I'm sure that Facebook is _appreciative_ when he blogs on their platform, but
it's patently ridiculous to pretend that anyone would -- or could -- force it.

In fact, Facebook has already been forced to pay at least dozens of millions
of dollars essentially as a direct consequence of employing Carmack [0], and
I'd be willing to bet they'd happily pay (at least) dozens of millions more.

If you want your mind blown even harder, look up some of the paperwork around
the Waymo / Uber debacle [1]. SV companies pay big bucks for this type of
talent, even the ones you haven't heard of, like Anthony Levandowski.

If the roll of the dice is in your favor, SV stardom is at least as valuable
as Hollywood stardom (and, for the most part anyway, you still get to walk
around in public unharrassed).

Disclaimer: I have no special connection with any of the companies or people
mentioned, I don't know the inside baseball, and this is all conjecture. But
at least it's better supported than yours. ;)

[0]
[https://en.wikipedia.org/wiki/ZeniMax_v._Oculus](https://en.wikipedia.org/wiki/ZeniMax_v._Oculus)

[1] [https://www.cnbc.com/2017/04/03/waymos-uber-lawsuit-
reveals-...](https://www.cnbc.com/2017/04/03/waymos-uber-lawsuit-reveals-
anthony-levandowskis-120m-pay.html)

~~~
wavefunction
Picturing The Zuck leading two mutant mandogs on leashes: the audience is
horrified to realize they're Yann LeCun and John Carmack, though 'heavily
modified' and almost unrecognizable in their snarling ferocity.

"This fist holds the Ego. This fist holds the Id. I bring you a new existence:
the GlobalCoin. I came up with that myself. As Super-Ego."

~~~
spongo
I appreciate your imagination.

[not the person you responded to]

------
carmacks_fb
The best part about these blog posts is that people discuss the hosting
service more than the content of the article at this point.

No one is nearly as interested in Carmack's thoughts on codecs as they are
about the domain name of the server he's posting to.

And yet, we'd all read his stuff no matter where he puts it. But maybe he
reaches more outsiders this way? I doubt it.

