
Optimizing Content Quality Control at Netflix with Predictive Modeling - hepha1979
http://techblog.netflix.com/2015/12/optimizing-content-quality-control-at-netflix-predictive-modeling.html
======
binxbolling
So much (great) emphasis on viewing experience, so little emphasis on browsing
experience. I've been a customer for over a decade and am just perpetually
frustrated; e.g. on the Netflix PS3 app titles automatically start playing
when you view their info. So if you wanted to watch that title, it saves you
the agony of clicking "Play." If you were just browsing, you now have this
thing playing while you're trying to read the synopsis, director, actors, etc.
Then, it'll show up in Account Activity as "started" and also affects
recommendations ("Based on your interest in...").

~~~
JonLim
Have had the same issue on my PS4. It's mildly annoying when searching for
something new, and kinda nice when you're getting back into something you were
watching yesterday.

I wish it was selective about autoplay, for certain conditions. Chances are, I
don't want it to play when I've never watched a second of any episodes in my
life, but it's nice that it autoplays when I click it from "Continue Watching"
section.

~~~
endymi0n
Same thing for the Amazon Fire Stick... the Netflix app is extremely laggy and
playback stutters, it feels like even they have very capable engineers,
there's a lack of love at the very last mile.

Also, discovery of new films is awful, as they basically want to be helpful
and guessing your taste all the way, but it's severely lacking filtering
options for power users. Which I don't mind, because it's driving people to
our site in droves recently :)

(shameless self-plug: [https://www.justwatch.com](https://www.justwatch.com) )

The sheer amount of systems they are trying to cover with a performance and
security (DRM) sensitive video player component isn't that easy to support on
the other hand.

~~~
binxbolling
I don't even think basic filtering is only appealing to power users. Just some
way to find films apart from their recommended, too-cute "genres" would be
appreciated.

------
_dark_matter_
If this kind of control system interests you, check out Chimera [0]. It is a
similar approach that the output of the machine learning approach can go to a
human, but is interesting because the human can feedback to the machine
learner. That is, not only can the human see that the machine learner was
wrong and write rules to correct it next time, but the learners can utilize
that new training data to better classify in the future as well.

Of course its a tradeoff, because it seems like false negatives (movie had QoE
problems but ML system did not flag it) should be avoided at all costs in this
case.

[http://pages.cs.wisc.edu/~anhai/papers/chimera-
vldb14.pdf](http://pages.cs.wisc.edu/~anhai/papers/chimera-vldb14.pdf)

------
tryitnow
My experience with Netflix's video quality has been pretty positive, so they
seem to have been doing something right. I hope this predictive model will
only improve things and won't deteriorate quality (consider they tuned for low
false negatives quality shouldn't decline).

This is a nice write-up of the business case for data science. It would have
been nice to have a follow up with the technical details - but maybe that's
deemed proprietary?

My problem with Netflix is that I rarely find streaming films I would be
interested in watching. It's so bad that I don't even bother looking there any
longer, I'd rather just pay the $2.99 and rent a film I'd like to watch from
Play or Amazon.

~~~
binxbolling
Just curious: what are you interested in that Netflix doesn't have? My queue
always hovers around 350 unwatched titles, so this always baffles me a little
bit.

~~~
adenadel
They have a lot of great television, but the film selection always seems
limited.

~~~
veb
I discovered this thing the other day,
[https://www.smartflix.io/](https://www.smartflix.io/) and lo behold, imagine
my surprise when nearly everything I searched for... was in fact, streaming in
my own country (New Zealand) and nowhere else. That was an eye opener.

With that said, I quite like the Smartflix UI -- it's great for discovering
things.

------
quadrature
It's interesting but they could have given more details into the features,
types of models they tried, how successful it is etc etc. right now it just
reads as "we used ML to fix a problem we have".

~~~
oakenshield
Likely they're holding their cards close to the chest as it's a competitive
market.

~~~
Zikes
Historically they've been very open about their tools, though, even open-
sourcing a lot of them.

Not to say this can't be the exception, just that it does seem a little out of
character.

