On: Pragmatic Theory, Commendo, Yahoo and AT&T hit the Netflix grand prize

[grandalf]

I'd love to see if their work stands up to all the new ratings data generated by netflix since the prize was announced.

Is it possible that there is over-fitting going on?

[elq → grandalf]

(disclaimer: I work on the cinematch team)

it's rather important to us that the solutions aren't over fit so we've worked very hard to make it hard if not impossible to do.

[Aron → elq]

1) There's not enough information returned by the submission process (a single RMSE number per day). 2) The information you do get is on half of your submission, but it is the other half that counts. So you get no information back on that.

[elq → grandalf]

I forgot to answer your first question. We released a new prediction engine a few months ago (http://blog.netflix.com/2009/05/more-accurate-star-predictio...)...

I think it's safe to assume that we wouldn't have used new algorithm(s) if they didn't work well on our ~3B ratings.