

Pragmatic Theory, Commendo, Yahoo and AT&T hit the Netflix grand prize - derwiki
http://www.netflixprize.com/leaderboard?att

======
profquail
I'm not sure it's still the case (and the website is down), but the rules used
to state that once someone hit 10%, they would wait a month before awarding
the prize, to see if anyone else could beat their solution.

In any case, I'd love to see Netflix start another challenge round. A million
dollars isn't a whole lot of money for them, it brings in a lot of publicity,
and provides a 'fun' way to get some research done in the data-mining field.

~~~
kqr2
I believe you are correct:

<http://www.netflixprize.com//rules>

 _To qualify for the Grand Prize the RMSE of a Participant’s submitted
predictions on the test subset must be less than or equal to 90% of 0.9525, or
0.8572 (the "qualifying RMSE"). After three (3) months have elapsed from the
start of the Contest, when the RMSE of a submitted prediction set on the quiz
subset improves beyond the qualifying RMSE an electronic announcement will
inform all registered Participants that they have thirty (30) days to submit
additional candidate prediction sets to be considered for judging._

------
rottencupcakes
AT&T confirms their accomplishment here:
<http://www.research.att.com/~volinsky/netflix/bpc.html>

This starts a 30 day period during which all the teams can submit their final
versions. After that, I believe Netflix will run the algorithms against the
unreleased set of movie rating data.

~~~
Eliezer
Since the current teams were using the squared error ratings on the standard
test data (not this new test data) to combine their old algorithms, _it's
quite possible that in one month we will learn that the prize has not yet been
won_. As I understand the rules, anyway.

~~~
elq
No. from the rules - "The RMSE for the first "quiz" subset will be reported
publicly on the Site; the RMSE for the second "test" subset will not be
reported publicly but will be employed to qualify a submission as described
below. The reported RMSE scores on the quiz subset provide a public
announcement that a potential qualifying score has been reached and provide
feedback to Participants on both their absolute and relative performance."

The final call announcement would not go out unless a team beat 10% on the
test set. That announcement went out at 4:30pm PST today.

It is still quite possible that the entries can be rejected if the reviewers
find the system description lacking or if we are unable to reproduce their
result. That clock starts after the 30 days runs out.

------
grandalf
I'd love to see if their work stands up to all the new ratings data generated
by netflix since the prize was announced.

Is it possible that there is over-fitting going on?

~~~
elq
(disclaimer: I work on the cinematch team)

it's rather important to us that the solutions aren't over fit so we've worked
very hard to make it hard if not impossible to do.

~~~
Aron
1) There's not enough information returned by the submission process (a single
RMSE number per day). 2) The information you do get is on half of your
submission, but it is the other half that counts. So you get no information
back on that.

------
Aron
The only candidate I think to make this a surprising finish is the 'Grand
Prize Team' which has another of the long-time leaders 'Gravity' involved and
has structured itself to accept any and all team members who care to join.

Probably no one would have won this contest if it wasn't for the fact that
many of the top teams published their results, and so there was cross-
pollination of ideas. Pragmatic Theory didn't contribute in this manner. Kudos
to them for making I think a significant improvement on the best of the
published models however.

------
FiReaNG3L
I hope someone will take the time to code an open source version of the
published algorithm, or something close to it!

~~~
dave_au
Netflix gets a non-exclusive license from the winner, so they can sell it to
other parties, and it seems like something that various parties would be
willing to pay for.

An open source version would be a significant act of charity. It would be nice
to see older versions or versions that were crippled enough to be education
without watering down the reward that the winnners receive.

------
geuis
Its not just AT&T, its 4 groups working together including
Commendo(<http://www.commendo.at/>), Pragmatic
Theory(<http://pragmatictheory.googlepages.com/>), Yahoo
Research(<http://research.yahoo.com/>), and AT&T Statistics Research
Team(<http://www.research.att.com/%7Estat>)

~~~
derwiki
You're right, I tried to update the headline but it wouldn't take -- sorry
about that

~~~
derwiki
hey! who changed my title?

~~~
kqr2
Probably one of the HN moderators.

~~~
quizbiz
HN moderators?

~~~
staunch
There exists a mysterious group of HN members, appointed by PG, who have admin
abilities. They kill submissions, edit titles, etc..

~~~
joeycfan
They dwell upon Mt. Olympus....

------
dmfdmf
Did anyone here take a crack at this? I had some ideas that I wanted to try
this summer but now its too late. If anyone already has a model built and made
submissions using the KNN and SVD methods perhaps we can talk. I thought of a
different approach that was not discussed in any of the papers, forums or
websites that I read. I know its a long shot but it could happen... yhfin at
yahoo dot kom.

------
mitko
Congratulations to BellKor's Pragmatic Chaos!

Actually receiving the prize will stimulate more people to do ML research. I
would be happy to see some other AI competitions similar to IOI, IMO or
TopCoder with big publicity.

Netflix are pioneers in such challenges and have the opportunity to be some of
the founders of such future competitions.

------
kqr2
Very cool. For a long while, 10% improvement seemed like an asymptote.

Has Netflix announced any follow-up challenges?

~~~
zitterbewegung
Technically completing 10% gives you 30 days to submit another algorithm to do
better than this algorithm.

------
Sidnicious
So, all the submitted algorithms are going to be open sourced now, right?
Right?!

In all seriousness, can anyone enlighten me as to who owns the submissions
now?

~~~
a-priori
The teams who created the solutions own them, but they're obligated to license
them to Netflix. It doesn't have to be an exclusive license though. Quite
generous on Netflix's part if you ask me.

~~~
vang3lis
> but they're obligated to license them to Netflix

Technically they are obliged to license them only if they are about to receive
the prize.

------
lallysingh
Plus points on the baby-blue matrix background.

