
High Quality Video Encoding at Scale - hepha1979
http://techblog.netflix.com/2015/12/high-quality-video-encoding-at-scale.html
======
spdustin
It's a fascinating look into how they produce multiple encoded formats from
source video, but something stuck out at me... One feature in their ingestion
system is meant to reject source inputs that would lead to a poor viewer
experience. This doesn't seem to include interlaced NTSC video as a "poor
experience" metric. I'm willing to bet that the majority of the screens on
which Star Trek: Deep Space Nine is played are progressive scan, yet that show
is interlaced at NTSC resolution.

In cases like this, where the source material is analog and, I would have to
assume, not available in progressive scan, is there a technical reason why
Netflix doesn't de-interlace the source before encoding? DS9 seems big enough
of a catalog to be worth encoding in a less jarring scan rate.

Not complaining, mind you. The DVDs are interlaced too. Seems that the
original recordings were NTSC only. Honestly curious if anyone has any
insight.

~~~
speakeron
TNG was like this (sourced from a video version and with a very nasty
interlace look) on Netflix up to a couple of months ago. Now they stream the
magnificent remastered blu-ray HD version which is sourced from the original
film with fresh CGI since the original CGI was tied to the video.

It's not clear whether DS9 will get the same treatment.

~~~
dbcooper
I prefer to watch SD rips from the TNG blu-rays. The merciless clarity of
1080p just kills the magic.

~~~
devonkim
I've wondered if this could be fairly easily emulated with some playback
filters locally if a viewer has a preference of experiencing material with
lesser fidelity (CD v. vinyl comes to mind where it's almost more about
ceremony and romanticism than technical merits 95%+ of the time to reject the
hi-fi version, "remastering" insanity notwithstanding).

~~~
baldfat
CD vs Vinyl is not a fidelity issue but actually a compression that is a part
of analog which is what people call "warm" sounding. A good analog recording
has just as much or more fidelity then digital. An analog recording can
actually reproduce a accurate sound wave while a digital can technically never
be able to do that. (Former sound engineer)

~~~
nitrogen
Actually digital recordings can reproduce signals up to Nyquist perfectly.
Watch this surprising demo:
[http://xiph.org/video/vid2.shtml](http://xiph.org/video/vid2.shtml)

~~~
devonkim
In some defense of vinyl purists, there's a lot of signal intensity above
audible human hearing range in sibilants and especially percussions.
[http://www.cco.caltech.edu/~boyk/spectra/spectra.htm](http://www.cco.caltech.edu/~boyk/spectra/spectra.htm)

However, similar to how we perceive ultra low frequency more as a physical
vibration than as sound, these harmonics above 20 kHz are probably merely
annoying and subtly felt and impact one major issue of enjoyability -
listening fatigue. So maybe we don't want those frequencies from a musical
perspective anyway.

Regardless, there's hardly any equipment in use by even "audiophiles" on full-
blown analog setups that can faithfully reproduce sounds beyond 30 kHz because
of electronics design limitations in themselves rather than an analog-digital
end user recording format distinction. In fact, a lot of vinyl historically
had to be mastered with a low-pass filter cutting off a lot of the high
frequencies because historically with sufficiently high enough energy in high
frequencies, the needle would be tougher to control and fly right off the
track sometimes (one explanation I read from an audio engineer - really not
sure about that logic, but there's definitely low AND high pass filtering on
vinyl that makes it lower fidelity in many respects than the master).

Heck, most vinyl produced since the 70s comes from digital masters in the
first place.
[http://wiki.hydrogenaud.io/index.php?title=Myths_(Vinyl)#Myt...](http://wiki.hydrogenaud.io/index.php?title=Myths_\(Vinyl\)#Myth:_Vinyl_is_better_than_CD_because_it_reproduces_higher_frequencies_than_CD_and_avoids_anti-
aliasing_filter_issues_at_the_frequencies_CDs_can_reproduce)

Bugs the crap out of me to see people claim vinyl is superior on technical
merits rather than aesthetic ones (sound preference / taste is real). You'd
think they're climate change deniers with their insistence and rhetoric. But
this is what I meant by purists about the "original" \- higher fidelity and
clarity is oftentimes not what people desire.

~~~
nitrogen
Yeah, I love vinyl, not because it's good, but because it's so bad. I'll still
listen to digital recordings a majority of the time.

------
camperman
I'm envious of these capabilities. I wrote the backend for a streaming service
that streams about forty channels to a quarter of a million mobile users in
Africa. We get source video from our content providers that has given me grey
hair. The problem is that I simply can't spin off the encoding and source
quality checks to the cloud because of bandwidth costs here. So I do the
quality checking and compression on local servers and then upload the
compressed output to the servers.

I'd kill for a 100 megabit line at a decent price.

~~~
noir_lord
That sounds fascinating, what is the video used for?

~~~
camperman
It's literally a bunch of TV channels: sport, soaps, educational, news and
music videos.

------
whatever_dude
As someone who encodes video for work with a ton of ffmpeg-based batch files,
this article makes me feel like a child.

~~~
Splines
I've done a bunch of playing with ffmpeg at home, and I imagine the tech stack
is probably similar at Netflix, at least for the Source-->Chunk,
Chunk-->Assemble, and Assembled-->Encode steps.

The validation done during all of these is interesting. Netflix's early years
are probably exactly like what you're doing - single file in, transcode,
single file out and deploy.

Chunking the pieces up is clever. Getting it right must have been challenging.
How do you write an oracle for something that complex?

~~~
jfb
_How do you write an oracle for something that complex?_

You start with a much smaller problem and incrementally build up.

------
n0us
Fascinating article. Forgive me for being ignorant about this, but how often
do they need to encode video?

I would think that it's only done when new titles are added to streaming, and
once the video has been encoded into all the required formats they would be
done with it. Sure, there is a lot of video content out there to be encoded
but it isn't unlimited. Is serving the content and providing the
recommendation engine to users at scale not a greater challenge than encoding
the video?

~~~
mkagenius
Yes, encoding needs to be done only once.

Generally it would take 2-3x of original duration of video to encode a source
into a 1080p, so I am not sure why they take full 1 day? unless they do each
bitrate serially which I think is not as hard to parallelize as it is to
parallalize single bit rate by chunking.

Yes, I believe serving is lot harder, but serving is almost a solved problem
since people are dealing with for long time.

~~~
mkaufmann
For x264 that is true, HEVC which is also mentioned is much slower. For a 4k
source transcoding can take more than a second per frame. For a normal movie
this can quickly result in encoding times of more than a day.

Another problem is that you have to encode the movie for each codec profile
times the number of different bitrates per profile. The article mentions four
profiles (VC1, H.264/AVC Baseline, H.264/AVC Main and HEVC) and bitrates
ranging from 100 kbps to 16 Mbps. Assuming now there are 20 different bitrates
per code you already get 4*20 => 80 encoded copies per source. But of course
this can be solved by parallelism.

~~~
sorenjan
Are there any codecs that can output multiple versions of an input at the same
time? Seems like a lot of the encoding process (like motion estimation) is the
same every time, so why do it once for every output instead of reusing it?

~~~
JustSomeNobody
That would be interesting to know. A lot of transcoders can make multiple
passes over the source, so being able to reuse the meta data generated for
subsequent passes at different output qualities might help speed up the
process. I dunno, not my forte, just thinking out loud.

------
mkagenius
Interestingly, not even a single mention of ffmpeg.

~~~
astrange
It's possible their AVC encoder is not x264, and their VC1 encoder might be
from Microsoft, in which case it would be a pretty modified ffmpeg. And some
of the input formats might not be using it either.

But you can be sure it's involved somewhere, since there's no other ProRes
decoder that works on Linux!

~~~
coalescence
FFmpeg supports Prores decode and encode? FFMBC for sure last time I checked

~~~
garblegarble
FFmpeg does, it has different decoder implementations with their own quirks
(btw, moot AFAIK because I think they use EyeIO):

    
    
      # ffmpeg -codecs | grep prores
      ffmpeg version 2.8.1 Copyright (c) 2000-2015 the FFmpeg developers
        built with Apple LLVM version 7.0.0 (clang-700.1.76)
        configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-indev=jack --disable-outdev=xv --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --enable-vda --enable-videotoolbox --arch=x86_64 --enable-yasm --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libsmbclient --enable-nonfree --enable-libfdk-aac --enable-libfaac
        libavutil      54. 31.100 / 54. 31.100
        libavcodec     56. 60.100 / 56. 60.100
        libavformat    56. 40.101 / 56. 40.101
        libavdevice    56.  4.100 / 56.  4.100
        libavfilter     5. 40.101 /  5. 40.101
        libavresample   2.  1.  0 /  2.  1.  0
        libswscale      3.  1.101 /  3.  1.101
        libswresample   1.  2.101 /  1.  2.101
        libpostproc    53.  3.100 / 53.  3.100
       DEVIL. prores               Apple ProRes (iCodec Pro) (decoders: prores prores_lgpl ) (encoders: prores prores_aw prores_ks )

------
JustSomeNobody
Interesting post. Netflix's posts usually are.

And then you scroll to the comments on the page...

------
contingencies
I personally worked on a similar system for a major mobile video solutions
provider six years ago: none of this is particularly new.

------
Keyframe
Has Netflix started with HEVC h.265 support? I see they mention HEVC briefly.

~~~
virtuallynathan
I think their 4K streaming to TVs uses HEVC.

