
Deep Neural Networks for YouTube Recommendations - os7borne
http://research.google.com/pubs/pub45530.html
======
dsco
One of the very best recommendation engines I've encountered is the "Discover
Weekly" playlist from Spotify. It's helped me reconsider my relationship to
music which I basically thought was dead since I had hit a rut on exploring
new artists.

There's an interesting presentation of how it's created on SlideShare

[http://www.slideshare.net/MrChrisJohnson/from-idea-to-
execut...](http://www.slideshare.net/MrChrisJohnson/from-idea-to-execution-
spotifys-discover-weekly/8-Discover_Weekly_Started_in_2006)

~~~
seanwilson
> One of the very best recommendation engines I've encountered is the
> "Discover Weekly" playlist from Spotify.

The addition of Discover Weekly really confused me. Shouldn't the features
that create a radio station from an artist or a playlist fill this need
already? Why is it only updated weekly? I haven't tried other services much
but it feels like Spotify isn't doing as much as they can with
recommendations.

~~~
makeee
I think it's because people are used to listening to their Playlists. It feels
more natural to check your "Discover Weekly" playlist then a whole new section
within Spotify. And I think the decision to only update it ever Monday was
pretty genius. Most people aren't particularly excited when Monday rolls
around.. but when they think about the fact that it's Monday they are likely
to remember they have a brand new Discover Weekly playlist to listen to. It's
one of the good things about their Monday and becomes a habit over time.

~~~
2bitencryption
I have Android Auto in my car, and my favorite part of the terrible, traffic-
ridden commute from Redmond to Bellevue is listening to my Discover Weekly
playlist. Even though I only really like about 10% of what it picks, usually
in that 10% I find an artist I really enjoy and dig into that.

------
kbart
I'm skeptic about recommendation feature. My reaction to them, in these rare
occasions when not get ignored all together, in all sites ranges from 'huh?'
to 'wtf?'. Specifically Youtube gives geographic location too much weight --
just because I live in a certain country, it doesn't I'm interesting in all
these trending local shit-pop music or stupid 'fun' videos.

~~~
thr0waway1239
I can confirm this. I saw a lot of region-based, really stereotyping videos
when I got a new computer and visited YouTube. While I understand they need to
fill up the home page with something, I just prefer they can be a little more
creative about the whole thing.

After glancing through the top few, I quickly went for the search box. To
their credit, once I do a few searches, the recommendations drastically
improved when visiting next time.

A side effect of this, of course, is that you can study all kinds of
stereotyping and biases by repeating my experiment in various regions I
suppose.

~~~
genofon
if they don't have any good information about you other than your location
what else would you recommend rather than the most popular in that location?

~~~
Ntrails
When you lack the information to make a good recommendation, "the best we can
guess from really generic or sparse information" tends to be annoying. That's
really bad.

Yes, I watched a bunch of Dota replays during a recent tournament. No, I don't
normally go on youtube. So I watched a daily show video clip that was linked.
All my "watch next" and "recommended" are Dota. That's not smart, that's
aggravating. I would have been ok watching a couple more ds clips, but instead
exclusively bad recommendations were made based on poor data.

I dislike the idea that my world gets filtered by algorithms, but I really
hate when they're obviously bad at it. Although I suppose I should be grateful
that it's easily spotted when it's bad?

------
pcovington
Author here - happy to answer questions about the techniques in the paper.
We're super excited to finally share this work externally. Feedback about
YouTube recommendations in general also welcome.

~~~
hacker42
Do you study the phenomenon of _information bubbles_ at Google? Let's say, a
German user just happens to watch some right-wing populist video claiming that
we need to stop Merkel's refugee politics. The next day the user might receive
plenty of recommendations in their feed that confirm the message in the first
video. They happen to stumble upon a video of some party convention by an
uprising German populist party, and everything makes sense now! Video by video
the user gets dragged into a right-wing ideology.

That is an information bubble. The algorithm cannot detect low quality or
populism, neither can it recommend opposite standpoints, and at the end of the
day it has a real effect on a country's politics and the well-being of many
people.

Do you have means of quantifying such effects? What are possible
countermeasures?

If you cannot talk about that, then this would be my feedback: Perhaps you
could train a language model to find opposing views in video titles and tags
and then diversify the video recommendations based on that.

~~~
sevenless
What about the reverse scenario, though? Should someone who watches videos
about refugee suffering be given anti-refugee video recommendations, lest they
be dragged into a 'left wing ideology'? I don't see how that would be
acceptable. Would Holocaust documentaries be 'diversified' with Holocaust
denial videos?

'Information bubbles' have existed as long as people have had a choice of
newspapers to buy and TV channels to watch. Calling for Youtube to
artificially 'balance' videos seems like political interference.

~~~
cscurmudgeon
Ah good old moral relativism.

~~~
force_reboot
Isn't the concept of an "information bubble" already inherently relativistic?
sevenless was simply pointing out the relativism cuts both ways.

~~~
cscurmudgeon
I was just pointing out the obvious.

------
Eliezer
I noticed that YouTube's recommendations had suddenly gotten better! I
wondered if they were using a new statistical approach, or had just started
really optimizing at all because the old recommendations were extremely naive.
I'm actually a little disappointed to find out that it might just be another
deep learning thing. (Yes, it works, but I feel like you learn a little less
about problem structure when what you read is mostly "we threw a generic
function approximator and 30,000 hours of GPU time at it".)

~~~
thomasahle
Mine seem like they have gotten a lot more click-baity. I wonder if that's an
artifact of their optimization goal.

~~~
artifaxx
If people fall for the click bait(they do), then they get more views from
optimizing for it. It would be more surprising if an optimization algorithm
for views didn't favor this.

------
catnaroek
During the last few months, YouTube has consistently recommended me videos I
wasn't interested in (to put it in polite terms), in spite of the fact by now
Google knows enough about me to answer quite reliably what I'm likely to be
interested in. The only explanation that I can find is that their need to show
me specific videos (what do they call it nowadays? “sponsored content”?)
prevails over other considerations.

------
rainy-day
I don't feel like anyone has gotten recommendations right, even though one
seemingly obvious approach has not been tried by anyone: allow ratings of
favorite works across all media: movies, tv shows, books, music, radio
programs, youtube videos. Make a very easy, efficient UI to add ratings. This
way you will avoid superficial matches: if I just watched an excellent
steampunk cartoon, let's offer a zillion of throwaway crap steampunk. It's not
the steampunk part that I liked, it's that it was amazingly done.

If I was a huge fan of books, movies, music, youtube picks of another user, it
may be there is a deeper connection of the kind of quality we are both looking
for, and so his or her recommendations would be highly relevant.

~~~
firasd
You're kind of suggesting people go in reverse. Ratings were the initial way
these things worked but then they moved to more implicit signals. Netflix used
to be all about star ratings back in the day; now they want to measure what
you're actually watching.

I think the issue of a system determining whether you like the steampunk genre
vs the quality of only that particular steampunk video is separate from the
issue of ratings.

~~~
pastullo
But also view-time or view-count don't tell the full story of how much you
liked that video.

I am not happy with YT recommendations because they suggest crap videos to me
and not the finest one available for that topic, just as he said.

The system should rather suggest me a different topic but with the best
quality/content available, rather than a super similar video with crappier
quality/content.

------
erichocean
Recommendation systems are a really interesting topic to study/engineer on. I
think there's a lot of unexplored/undiscovered techniques still remaining.

~~~
taeric
Oddly... I have grown rather weary of them. I have yet to get a surprise
recommendation that I cared for. At best, I have seen people surprised that
some good recommendations came out of a system.

Which usually just leads to curation systems being key. And they work well,
until they are gamed. And they will be gamed.

~~~
brianberns
I felt the same way until I tried Spotify's Discover Weekly recommendation
system. I don't know what they're doing, but I've found many songs I really
enjoy that way.

~~~
iverjo
Spotify's Discover Weekly is based on collaborative filtering. Some of the
techniques they use are described in these slides:
[http://www.slideshare.net/MrChrisJohnson/collaborative-
filte...](http://www.slideshare.net/MrChrisJohnson/collaborative-filtering-
with-spark/6)

~~~
erichocean
That's a major technique in one of the social platforms I'm working on, too.
Glad to see it working so well at scale!

------
fatman13gg
Go through the comments of this video
[https://www.youtube.com/watch?v=tGe4uWEvwe8](https://www.youtube.com/watch?v=tGe4uWEvwe8)

People were literally bizarred by youtube, saying they were there by
recommendation. (I have this video in the recommendations also...)

~~~
eva1984
Ahh...This happens to me also, there is one day, my recommendations is flooded
with this CandianMum, and I didn't really watch that much, if any, video of
relationship on Youtube.

------
qd6pwu4
I hate that one day I happened to click one video and watched it, then youtube
starts to recommend videos on the same topic day after day, even if I marked
them as not interested, they still show up time to time...

~~~
aedron
Or you watch something in privacy and now you can't open up Youtube in front
of anyone anymore.

------
intoverflow2
I'd be more interested if YouTube recommendations took into account users I've
blocked.

Sometimes YouTube recommends me videos with clickbaity gross thumbnails or
from YouTubers I dislike or have no desire to watch but there is nothing I can
do to to stop it recommending these to me, why can't I just go to these users
profiles and block them and have them removed from my YT experience?

Block just seems to stop people from messaging you, not from you being shown
their videos by an algorithm.

------
noiv
I use YT mostly for new music, interesting documentaries and the occasional
fun. Beside that there are one-off searches for random topics. Regarding the
latter recommendation won't help, because it's not fast enough to tell me
which aspect I'm missing. Regarding the former three I'd love to know whether
there are users who liked the same videos. So please, recommend users not
videos and let me do the rest.

------
londons_explore
Your recommender seems to be trying to predict (from logs) which video the
user will watch next/soon, and how much watch time it will lead to.

If you used to have a bad recommendation system, and then you switch over to
this system, then it will still be trained with data generated by users who
saw the old recommendations, leading it to have a bias towards the same bad
predictions.

Is there any way around that?

------
supergirl
Like others here I am also disappointed by the youtube recommended videos. So
I was investigating building a better recommender myself. I was actually
searching for how the youtube recommender works yesterday but could only find
the 2010 paper. Now I am starting to believe that it is not the recommender
that is the problem. It is that youtube consists of 99% low quality videos.

------
bryanrasmussen
Is this the source of all those Recommendations that I look at Ben and Holly
cartoons in the middle of the day when my daughter is at school? Or more
Italian daytime television when I've just watched a video my wife never would?
In short, is this the reason why there's never anything interesting for me
when I go to the frontpage of youtube?!?!

------
londons_explore
You seem to be doing all the right things in this paper, yet user sentiment
seems to still be negative. Do you think that's because maximizing watch time
and impressing users are conflicting goals, or are there perfect
recommendations out there which both impress users and maximize watch time,
yet they haven't yet been found?

------
spynxic
would it be legal to re-use this technique on another commercial project
without getting consent?

