

The fallacy in the Netflix algorithm - cwan
http://scienceblogs.com/cortex/2009/08/netflix.php

======
larryfreeman
He's not correct about the Netflix Prize algorithms. The leading ones all
assume time-effects so that your rating on one day is not the same as on
another day.

For the explanation of the winning team's view on time-effects as of 2008, see
here: <http://research.att.com/~volinsky/netflix/Bellkor2008.pdf>

For a simpler example of a time-based global effect, (see User x Time Effect,
for example): [http://algorithmsanalyzed.blogspot.com/2008/05/bellkor-
algor...](http://algorithmsanalyzed.blogspot.com/2008/05/bellkor-algorithm-
global-effects.html)

~~~
ovi256
I would also recommend Yehuda Koren's (Bellkor leader, if I recall correctly)
KDD 09 excellent paper, "Collaborative Filtering with Temporal Dynamics"
<http://research.yahoo.com/pub/2824> which speaks exclusively of how to handle
temporal dynamics.

------
frossie
Let me get this right. Dude doesn't bother to rate movies, and then complains
that an algorithm based on movie ratings fails to read his mind?

If you keep putting on Criterion discs on your queue and then fail to watch
them or rate them, what would you like the poor software to do exactly?

Netflix has an _excellent_ method for dealing with the highbrow/lowbrow
problem - it is called multiple profiles. If you want some brainless mind
candy for Friday night with your mates, make a mindcandy profile on your
account. Make a Criterion queue for your cinephile Sunday group. You will find
the software does an excellent job of providing you with ratings tuned to your
preferences.

~~~
jpwagner
You are way off here.

The article was not a customer support complaint about his personal account.
Sure he could've written this article a helluva lot better, but his point is
100% valid which you implicitly agree with by recommending he open multiple
profiles.

Remember that this guy is a writer not an engineer. His solution is likely
bad! So throw out the idea of basing ratings on times of day or mood, etc. The
problem statement is still there and valid:

My preferences change over time. How can a good recommendation algorithm
account for that?

Brainstorm away...

~~~
brg
I don't think the gp is far off. The author of the article clearly says that
he places movies in his queue which he does not watch. If he provided truthful
responses to Netflix (not interested, not-enough-stars-to-order-again)
regarding these then the algorithm could be given a chance.

But he doesn't use the algorithm, and instead chooses movies the he won't
watch. And then complains that Netflix will not populate his recommendations
with more movies he doesn't want to watch.

Non-linear functions (such as mood over time) and non-static behavior
(attitude adjustment to one title due to exposure to another) can be accounted
for in the Netflix recommendation algorithm.

~~~
jpwagner
Ok, the author is a bit of a moron. He said stupid things and detracted from
what I am guessing his point is. He also didn't read about the Netflix
algorithms and how they actually work (all around terrible job of reporting
indeed.)

However, to circle back, what I think is his point is interesting: how do you
model "mood over time" and "non-static behavior".

You say they "can be accounted for" and don't talk about how. In the winner's
documentation they say they treated "user preference" variation over time the
same as chunked up "user bias" over time (they define these terms in the
paper.) While that obviously performed well numerically, isn't it believable
that this is not an ideal solution, but one done under severe time
constraints?

The problem is by no means solved...that's all I'm saying.

~~~
roundsquare
If we just take his examples though, then the problem is on its way to being
solved. So he puts movies on his queue that he doesn't watch? Great, he should
give them low rankings. If the algorithm is good, that problem is solved.

He likes some movies at night, some on the weekends and some with his wife?
Well, okay, rate the movies how you thought they were when you saw them. The
algorithm will recommend some movies for each situation.

Your preferences change over time? Thats a bit harder, but was also out of the
scope of the challenge. They were given static data. If you want to account
for changes over time, then give the ratings some weighting that decreases
over time or only use the last x months of data.

But, like you said, he probably doesn't know much about the algorithm.

------
wglb
Thoughtful article, but is "the software tries to predict how I'll feel about
movies based on my past ratings" this strictly true? Might it not be a broader
predictor of what you might like in a changing landscape?

And most seriously, it is not about _you_ , it is really about _netflix_. That
is to say, it may not totally please your every whim, but if it improves
netflix predictions over all of its users, it is a big win for them.

------
sh1mmer
I disagree with the article. If his point has merit it might be that the data
for the Netflix algorithm is the wrong or, more probably, incomplete. That
doesn't invalidate it, rather it means the factors are more complex.

Netflix can't know that he leaves Chaplain DVDs on his player unwatched for a
week, but it does know that he returns low-brow stuff faster. That faster
turn-around probably indicates a behavior. If that data wasn't exposed in the
prize, then maybe Netflix will look at exposing it for the next prize, or
adding it themselves.

------
kqr2
As Netflix streams more movies, it might give them the ability to better
predict when you might like to watch certain types of movies.

------
hc
im glad i saw this. the linked paper [
[http://www.csbmb.princeton.edu/ncc/PDFs/Neural%20Economics/M...](http://www.csbmb.princeton.edu/ncc/PDFs/Neural%20Economics/McClure%20et%20al%20\(Science%2004\).pdf)
] is quite intriguing and seems relevant to a lot more than netflix

