
Ask HN: Where do I start to build Netflix like personalizations? - foundmyselfa
I work at BigCorp. We have tons of good user usage data and we have an opportunity to provide recommendations the same way amazon or Netflix does personalizations. But I am not sure what pieces are needed to build such a recommendation engine. I am talking from trying to understand what I need from server side model generation or can this all be done client side. I am a total n00b here so any suggestions?
======
PaulHoule
Let's look first at the problem where "the user is looking at item X, what
item Y is related to item X".

This could be done on a textual basis by using a tfidf scoring applied to the
words in the text for items X and Y

[https://en.wikipedia.org/wiki/Tf–idf](https://en.wikipedia.org/wiki/Tf–idf)

An alternate approach is to look for "people who looked at X also looked at Y"
and mathematically this can be done exactly the same way as for documents,
except that instead of having a set of "does word W appear in this document X"
relationships you have "did user U look at (up vote/express interest) in
document X".

The latter works really well if you have a large number of documents. On a
large collection of scientific papers, I found that a paper with 10,000 views
would generate astonishingly good recommendations. With 1,000 views it was
hit-or-miss, with fewer than 100 views it was mostly missed.

The problem here is that you are sampling a huge matrix, and most cells are
not going to get sampled enough to get accurate results. One answer to that is
dimensional reduction

[https://en.wikipedia.org/wiki/Dimensionality_reduction](https://en.wikipedia.org/wiki/Dimensionality_reduction)

which gives better results with less data because there are many fewer matrix
cells which are better sampled.

Please send an email to the address in my HN profile and I'll be glad to talk
more.

------
edimaudo
This might be of help to you ([https://www.ibm.com/developerworks/library/os-
recommender1/](https://www.ibm.com/developerworks/library/os-recommender1/))

