
Netflix and the Napoleon Dynamite Problem (2008) - ImpressiveWebs
http://www.nytimes.com/2008/11/23/magazine/23Netflix-t.html
======
quotedmycode
From what I understand, they never implemented any of the algorithms, because
their existing one was good enough, and the premise has changed. It used to
be, you'd make suggestions so people could get DVDs sent to them, and you'd
know they would enjoy it. Now with streaming, the cost to send something you
don't like is cheap. So the recommendation engine just has to come up with
something you'd try to watch and perhaps enjoy. If you start watching
something, it's not your taste, you switch streams. So there's not a lot to
gain from improving recommendations 10% unless they were pretty low quality
ratings to begin with.

~~~
raisedbyninjas
I think the cost of postage/fulfillment/bandwidth should be a secondary
concern to how many users can Netflix attract by providing content they
actually enjoy. A bad recommendation for streaming or DVD both waste n minutes
of viewing time. A bad DVD recommendation additionally wastes 2-6 days of
waiting on USPS.

They already have the recommendation engine built and they need every
differentiation they can get to compete with the crowd of streaming providers.

~~~
JTon
I completely agree. Real hype is generated when Netflix consistently nails
recommendation. Content is still king, but curation of that content is the
engine that drives the whole thing

~~~
jaawn
I agree about your "real hype" statement. There is always the potential for a
provider to build business value by nailing recommendation, _especially_ when
it comes to lesser known content. Most viewers have heard of big-budget films
they plan to watch, or they'll watch any movie with <insert actor>. However,
if someone created a system that did this well enough for users to trust it
with unknown content, it would go a long way toward meaningful
differentiation.

Right now, I don't know of anyone who has limited spare time who would risk
trying something unknown just because it shows up in their Netflix
recommendations. If Netflix got those right frequently, people would rely on
it more and probably enjoy it more. A 10% increase in accuracy has a decent
chance to push beyond the invisible "good enough to trust" threshold and
create an experience other streaming services just don't have.

------
mrspeaker
"If you use a computerized system based on ratings, you will tend to get very
relevant but safe answers"... I call this my "Radiohead metric": for a given
music recommendation system, how long do I have to use it before it tries to
recommend me Radiohead.

One of the problems I find in music algorithms is they can't know how I've
changed - suggesting music based on what I play or input - but that's stuff I
liked in mid-2000s when I was in peak music consumption mode. Most suggestions
then get picked from the same pool.

I'd love these algorithms to take my age into consideration: "If you liked X
when you were 25, you're going to love Y now that you're 35!"

~~~
redthrowaway
Applying weight to each rating based on its age seems like such an obvious fix
that someone must have tried it and realized that it makes things worse.

~~~
s_kilk
Maybe, but I'd think you'd have to take the age weight with a pinch of salt.

The are 14 year olds who are obsessed with jazz and blues, while some 70 year
olds only listen to hardcore punk and deathgrind

~~~
redthrowaway
Age of the vote, not age of the voter.

------
AlwaysBCoding
All those hours and millions of dollars into perfecting their recommendation
system, and Popcorn Time still has a better UI for viewing movies than Netflix
does. Just show me all the top movies and let me pick one, why partition them
into drama/comedy and filter out the one's you don't think I'll like? it makes
the experience worse.

~~~
jandrese
Or you know, just let me use scrollbars instead of having to wait for your
stupid javascript to SLOOOOOWLY pan the movie list. And then I have to hover
and usually click through to get details about a movie.

What's wrong with vertical lists with relevant information next to the
picture? Something like every shopping interface ever? Using Netflix is like
going to a grocery store, deciding that you want some canned corn, and having
to walk up to a conveyer belt and wait while canned beans, turnips, tomatoes,
mushrooms, etc... slide by before the corn comes around. And it is like this
for every single item.

~~~
Pxtl
Seriously, Netflix' UI is a complete trainwreck. The searching is painful too.
I still cannot find a way to say "give me all your View Askew movies" or "all
your DC Animated movies". The search is woefully primitive and there is
functionally no UI for categories... can't I just pick multiple category tags?
Or see _all_ the categories you're using under the hood?

~~~
cwbrandsma
I agree there are some problems with the ui, but at the same time the number
of devices they design for is quite large (desktop web, mobile phone, set-top
boxes, Apple TV, Google TV, Amazon TV, etc). Creating a UI that is consistent
between all of them must be a bit daunting.

~~~
jblow
So don't do that. Make a few UIs. They are a big company, they can do that if
they actually have anyone who knows how to program.

------
rogerbinns
I've been getting increasingly frustrated at Netflix, because it keeps
recommended things I have already seen, or definitely do not want to see.
Having to keep scrolling through unwanted content like that really reinforces
just how dumb it is, despite all these claims about their intelligence.

Eventually I did figure out that on the website you can say you aren't
interested in particular items, but Android/Roku etc do not have that option.
Of course those Netflix employees who work on this stuff deal with the PC/web
based interface all the time, but apparently don't realise that not everyone
does that.

Android/Roku etc do let you set ratings on items, but to me there is a big
difference between "I am not interested in this item" and "I have watched it
and give it one star".

~~~
ohitsdom
> Eventually I did figure out that on the website you can say you aren't
> interested in particular items

I didn't know you could do this. Looking forward to telling Netflix that no, I
do not want to watch The Interview.

~~~
talmand
I haven't looked into in a while, but from what I remember it lets you say
whether you are interested in categories of content and not specific content.

~~~
JupiterMoon
You can say not interested specific content as well (on the web interface at
least). It doesn't always seem to get the message straight away though.

------
tyingq
I'm not sure there's a reasonable way past the "Napoleon Dynamite Problem".
They mentioned, in the article, other movies where it's difficult to predict
if someone would like them or not.

The list was “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,”
“The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

Personally, the common thread I see across those is that the viewer would be
more likely to enjoy the movie if they had the right context.

Kill Bill, for example, is far easier to enjoy if you've seen (not necessarily
liked...but have seen) cheesy old Kung Fu / Karate movies.

Napoleon Dynamite is easier to appreciate if you were socially awkward
yourself in high school. There's also some humor in there that's more relevant
if you were a kid in the 80's, as well as some humor that might make more
sense if you lived somewhere rural, or even more specifically, rural
Idaho/Utah/Wisconsin.

All that to say that Netflix may not have enough data at their disposal to
know if you have the right context to enjoy movies that almost require it.

~~~
BillPond
Yea, a user's expectations surely plays a role here. I am unsure how computers
would be able to notice someone "gets it" while someone else just stumbled
upon the movie.

------
mozumder
To be fair, even people aren't able to judge a movie properly. People often
change their opinion of movies on second viewing.

Just watching a movie at different times of day can significantly alter your
opinion of a movie.

~~~
elijahmoore
Agreed here. I also think that the overall vagueness of the five-star rating
system makes it hard for users to express their true opinion. Algorithms are
"garbage-in, garbage-out", and I think a five-star rating response represents
"garbage-in". Unfortunately it would be risky and expensive for Netflix to
change its rating system at this point.

~~~
JoshM33k
Interesting. What do you think is vague about the five star system? Or is it
any rating system that would be "garbage-in"?

Being interested in video games, I've thought a lot about rating systems.
Older magazines used to use 10 (or even 100) point scales, and you quickly
realize all the rating ends up taking place in the top 30-40 percent of your
scale anyway... so why not limit the gradations. For my money, 5 stars makes
sense... you've got a middle of the road "average" 3 star movie, and then two
stars above and below.

~~~
elijahmoore
My thoughts are generally in line with yours - big problem from the consumer's
perspective is tendency for most titles to end up in the 3.8 to 4.2 range. Not
very helpful when I'm browsing titles.

The stars are also just a snapshot of feeling, and don't account for any
context like expectations, identity, mood etc. Maybe I'd give Black Swan five
stars because it was impressive and won awards, but it doesn't mean I
necessarily enjoyed it or would recommend it to others. Perhaps a system based
loosely on Net Promoter Score ideas could be more helpful

------
VLM
A controversial observation might be they maximize profit by having the most
people sign up and then stream the least possible, correct? And they have a
semi-monopoly, correct, where theres no real competition? I mean, I have and
use Prime but I don't seriously watch videos and I'm in it for the shipping...

Anyway there's this assumption that they'd want to present results that
encourage you to watch as much as possible, yet they have a financial
incentive to present results that are just barely good enough that you won't
cancel and/or sign up for a minor competitor.

This is NOT like a public library card catalog where next years funding
depends on increasing circulation counts.

In some ways I think my proposed search result criteria are more difficult
than presenting result I'd actually want.

------
Animats
Netflix, having crushed the video store industry, no longer needs this. What's
the alternative? Redbox?

 _" Even though Net­flix has a good deal of demographic information about its
users, the company does not currently use it much to generate movie
recommendations; merely knowing who people are, paradoxically, isn’t very
predictive of their movie tastes."_

"Targeted" advertising suffers from that model. Demographics matter only a
little. What you've bought (which Amazon knows) matters far more.

------
kenjackson
I feel like one thing Netflix doesn't do a good job of noticing is what movies
I actually order/watch. That is, the mere fact that I ordered a movie means a
lot, whether or not I rate it.

The biggest manifestation of this issue is that I tend to order new movies,
but I seem to get a lot of recommendations for older movies. Netflix should
see that I've ordered maybe 2 out of 200 movies that are older than the year
2000.

------
pasbesoin
Netflix doesn't show user comments in the app version of its (streaming)
product.

I'm sure there are reasons for this. Nonetheless, it continues to strike me
that, in that context, their own, internal information on product offerings is
restricted to a two sentence blurb and an unqualified single five start rating
metric.

------
JoshTriplett
There's a related problem: if you enjoy both good movies and some so-bad-
they're-good movies, how do you rate the latter?

------
davisclark
In Notes from Underground, Dostoevsky nailed the reason computers and science
and algorithms will never be able to replicate humanity or predict the actions
of man, when he wrote:

“What is to be done with the millions of facts that bear witness that men,
consciously, that is fully understanding their real interests, have left them
in the background and have rushed headlong on another path, to meet peril and
danger, compelled to this course by nobody and by nothing, but, as it were,
simply disliking the beaten track, and have obstinately, wilfully, struck out
another difficult, absurd way, seeking it almost in the darkness. So, I
suppose, this obstinacy and perversity were pleasanter to them than any
advantage...

The fact is, gentlemen, it seems there must really exist something that is
dearer to almost every man than his greatest advantages, or (not to be
illogical) there is a most advantageous advantage (the very one omitted of
which we spoke just now) which is more important and more advantageous than
all other advantages, for the sake of which a man if necessary is ready to act
in opposition to all laws; that is, in opposition to reason, honour, peace,
prosperity -- in fact, in opposition to all those excellent and useful things
if only he can attain that fundamental, most advantageous advantage which is
dearer to him than all. "Yes, but it's advantage all the same," you will
retort. But excuse me, I'll make the point clear, and it is not a case of
playing upon words. What matters is, that this advantage is remarkable from
the very fact that it breaks down all our classifications, and continually
shatters every system constructed by lovers of mankind for the benefit of
mankind. In fact, it upsets everything...

One's own free unfettered choice, one's own caprice, however wild it may be,
one's own fancy worked up at times to frenzy -- is that very "most
advantageous advantage" which we have overlooked, which comes under no
classification and against which all systems and theories are continually
being shattered to atoms. And how do these wiseacres know that man wants a
normal, a virtuous choice? What has made them conceive that man must want a
rationally advantageous choice? What man wants is simply independent choice,
whatever that independence may cost and wherever it may lead. And choice, of
course, the devil only knows what choice.

Of course, this very stupid thing, this caprice of ours, may be in reality,
gentlemen, more advantageous for us than anything else on earth, especially in
certain cases… for in any circumstances it preserves for us what is most
precious and most important -- that is, our personality, our individuality.
Some, you see, maintain that this really is the most precious thing for
mankind; choice can, of course, if it chooses, be in agreement with reason… It
is profitable and sometimes even praiseworthy. But very often, and even most
often, choice is utterly and stubbornly opposed to reason ... and ... and ...
do you know that that, too, is profitable, sometimes even praiseworthy?

I believe in it, I answer for it, for the whole work of man really seems to
consist in nothing but proving to himself every minute that he is a man and
not a piano-key! …And this being so, can one help being tempted to rejoice
that it has not yet come off, and that desire still depends on something we
don't know?

You will scream at me (that is, if you condescend to do so) that no one is
touching my free will, that all they are concerned with is that my will should
of itself, of its own free will, coincide with my own normal interests, with
the laws of nature and arithmetic. Good heavens, gentlemen, what sort of free
will is left when we come to tabulation and arithmetic, when it will all be a
case of twice two make four? Twice two makes four without my will. As if free
will meant that!”

I think it comes down to authenticity. Humans can predict the actions of
humans, so long as humans act by and large as they think they're supposed to
act or like the things they think they're supposed to like. The moment an
individual walks in, an individual who does what they truly want to do when
they want to do it, all correlations and inferences and predictors fall to
waste.

~~~
pcmaffey
The point of any system should never be to choose for a person, but rather, to
improve the quality of available choices.

~~~
davisclark
In that case, aren't you, in effect, choosing for me the set of choices
available to me for selection, based upon a preconceived notion of "quality"?

The Underground Man goes on to say, “I agree that two times two makes four is
an excellent thing; but if we are dispensing praise, then two times two makes
five is sometimes a most charming little thing as well.”

Would two times two makes five be included in a set of high-quality choices?

~~~
saalweachter
And Rush said, "If you choose not to decide, you still have made a choice.".

There is no alternative to curation. If you display all the possible options,
there is still the order -- humans, being humans, will not thoroughly read and
equally consider even a list of a thousand options, so the order will affect
their choice. Choosing to display the results in random order, or in
alphabetical order, or in chronological order, is still a decision which
affects the choice of the user.

Curation cannot be avoided, as long as we are men.

I am reminded of Stanislaw Lem's Demon of the Second Kind.

