A bit unrelated, but: anyone remember in the "early" days when Netflix held competitions to develop the most accurate recommendation and ratings engines? Giving out million dollar prizes? And held possibly state-of-the-art tech for this?
And then one day completely gutted the star rating system, replacing it with a much-despised "thumbs up/down" system, giving up their "so good it's magical" recommendations for something that feels less accurate than a coin flip...
Netflix has ~6,000 items to recommend to streaming customers in the US . This is about 6% as much content to recommend as the DVD-by-mail service has. Their streaming catalog size is actually shrinking over time . Fine grained rating systems and better recommenders aren't going to move the needle much for Netflix's current business, because they now have a content availability problem much more than a discovery problem.
I am reminded of the show Tuca & Bertie. It was a new Netflix original from the same people who created BoJack Horseman. You would think it would be obvious to surface the new show to anyone who watched the old show. Except that never happened. The show's creator even commented that Netflix's algorithm never recommended her her own show. She had to manually search to find it.
The only real explanation is that Netflix believes it is more important for customers to think the catalog is large rather than the catalog is good. Netflix wants people to spend 10 minutes scrolling through crap to find something to watch instead of simply listing all the recommended content at the top and for people to notice that there are only a few good options.
But I have the same problem with the music services -- I listen to a lot of "quiet" music playlists for sleep and work. And so, all the playlists Spotify build for me are quiet. But I also follow and listen to a lot of rock/noise/loud music. But the "made for you" formulas won't budge.
I jokingly believe I need two services -- one where I listen to ambient/quiet/etc, and another where I never listen to it.
But the probability is I'll stop caring about music services and live without them. I don't need to curate my relationship with services like they're tomagachi's. I pay for Youtube Premium, but when I subscribe to a certain number of music artists, it chokes my video subscription feeds.
Moving to willfully not caring for content, music and videos alike.
Meanwhile I can pay the same for amazon and get more Picard, Mr Robot, The Expanse... and if the TV sucks there are plenty of movies on rotation.
But I realized they might also be found elsewhere, and it turns out they're carried by Amazon Prime Video. That was the last straw for my Netflix subscription.
I'm amazed by how thoroughly Netflix has changed me from a lover to a hater of their service and company in recent years. It's not just a matter of cancelling because they don't have a catalog I care about. It's that I feel lied to and manipulated as well.
Last weekend I tried to find To Live and Die in L.A. -- from the 1980s but hardly obscure. I couldn't find it legitimately available to stream anywhere, for any amount of money.
I even tried Kanopy, which has been my go-to source for films I can't find on paid streaming services, but they didn't have it either.
"Page Eight" was a great little spy movie with Bill Nighy and Rachel Weisz on Netflix. Now you can't rent from anywhere, for any price afaict.
How does this optimize revenues for the studios?
* The Name of the Rose
* The Draughtmans Contract
I spose I'll have to find them at an old fashioned DVD store, eg. https://filmclub.com.au/
The Name of the Rose (2019 series): https://dvd.netflix.com/Movie/The-Name-of-the-Rose/81093444
The Name of the Rose (1986 movie): https://dvd.netflix.com/Movie/The-Name-of-the-Rose/70000552
The Draughtsman's Contract: https://dvd.netflix.com/Movie/The-Draughtsman-s-Contract/459...
Your spelling of "Draught[s]man" might have misled you for that one. It's a little sad that Netflix's search returned no results for such a close spelling. I'm not sure why you didn't see Name of the Rose.
(although maybe from your example link you are in Australia and thus don't have access to Netflix DVD?)
This makes me wonder just how much other content they do have that would really appeal to me. It feels like rather than perhaps not spend _quite as much_ time on perfecting their recommendation algorithm, they've just completely given up. For a while now, the only real content that Netflix successfully "recommends" to me are things that I've already started watching.
Edit: I feel like I should add something besides a snarky comment:
I was searching for a mid-2000s Best Picture Nominee a few days ago (something like Munich or Capote, but don't remember). Results 1 & 2 were The Irishman and Marriage Story. None of the other non-Netflix best picture nominees were anywhere in the results if that was the connection they were making.
/I'm OTA, Netflix, and the library. But I mostly only use Netflix for the DVD service. I'm fortunate enough to have enough going on in my life that I watch very little video.
I think I'd easily bet 3:1, $1000 that if you polled people they'd rather have targeted end-of-show ads than untargeted mid-show interstitials.
I much prefer textual descriptions; for example "Terrible", "Don't like", "Okay", "Like", "Favourite", which has the same options as a 5-star rating system, but choosing between 3/4 and 4/5 is much easier IMHO.
I don't really know of any system that uses this, except this one music player I wrote myself (which has "Crap", "Meh", "Okay", "Super").
In other words, it not only expects me to define my own scale, it also expects me to stay consistent with it over time. I don't think that's realistic.
I honestly wouldn't mind if from time to time, a service asked me to stack-rank movies. Because that would be a question I feel confident I can answer. Give me two movies and ask me to say I prefer A, prefer B, or don't have a clear preference. Or give me 5 movies and have me put them order from best to worst, allowing me to say two of them tied or exclude some. Maybe the UI on this would be too weird for average users, though.
I don’t have any specific knowledge of Netflix per se, but I suspect the granularity of a 5 point rating system just proved to be superfluous. Even a +1 / -1 rating is probably sufficiently proxied by a simple measure of completion percentage (appropriately normalized).
I'm just hoping they're not making the same mistake here that they're making in the UI: showing movies I watched to the end as "continue watching", because I closed them on the end credits roll.
Effectively, this would be a "proof of emotional work": it's not a measure of your own subjective experience of your reaction to the product, but rather measuring your level of objectively-observable motivation to [dis]recommend the product. Which, I would think, would neatly get around all the arguments about "what it means" for something to be 3/4/5 stars—provoking a large number of people to click a like (or dislike!) button 10 times seems like a pretty good predictor of some objective property that the product has. (Whether that property is "quality" is up for debate.)
Scenario: I'm considering some odd looking sci-fi movie to watch that I never heard of before. Ratings between 2-4 stars might not tell me much, but very reliably titles with only one star were terrible movies. Now Netflix happily recommends any and all sci-fi titles, saying they are a "98% match" for me! Sure by category, but when the movie is a low budget dumpster fire I no longer have that instant signaling that the previous rating system gave me.
What's to stop you from applying whatever textual descriptions you want to the numbers?
If each star rating had a textual description of what it meant, and drivers didn't have to maintain something like a 4.3 (or whatever it is) in order to stay on the platform, the ratings would actually mean something.
I think star systems are a waste of time.
I'm honest with him and tell him exactly why I won't be filling it out. If I do ever fill one out, he'll know exactly why before I send it in.
: I want to give this more than 5-stars
: This is pretty good, but not MORE than 5 stars
: I've enjoyed this.
: I need to keep this for completist reasons
*: Delete this.
Oh wait, that's my iTunes rating system
Except for a couple of shows, I've almost completely switched to Prime.
Statistically, if each show is 8 episodes, then 88% of the time when I come to the homepage, I want to "continue watching" not "find a new show". Why the hell is continue watching not always the very first row.
Other things I don't like about Prime:
- they used to have a setting for "Show Prime only", because I am never going to pay for TV. They removed that and now you have to see all their pay crap.
- if I run across a Prime thing that looks interesting and add it to my watchlist, it might no longer be free when I get around to watching it. This sucks so bad, and I was bitten by it so often, that I no longer use the watchlist feature. It made me too mad.
I have given up trying to watch TV. I was spending half an hour scrolling through menus of crappy movies, then giving up. I won't cancel Prime because I split that with a friend and we both get the 2-day shipping, and I probably won't cancel Netflix because I have 4 other friends on my subscription, but I haven't used either video service for over a month.
Self-reported ratings fail in a lot of ways. Users can upvote what they think they're supposed to upvote. They'll downvote something, then go watch the sequel, because "It's terrible but I love it". And since they're seeking out stuff that they suspect they're going to like, they either compress all of the ratings into the top tier, or they end up "hating" things that they actually thought were merely so-so.
I suspect that even the up-down ratings get little attention in their recommendation engine. They have far more information, especially in the streaming service. Did you watch all of it? Did you watch it all at once? Did you watch it more than once? Did you watch it immediately after discovering it?
That's all stuff that users can't fake or be confused about. You don't get users saying, "Citizen Kane is the greatest film of all time, but I really don't want to see it." It's more likely that you can tease out what it is that a user actually wants to see, rather than what they tell you they want.
Netflix did run a competition back in the DVD days, when they had less interaction with the user. They did get a slight improvement, but at about the same time, they introduced streaming, and a whole lot of Big Data techniques appeared that could take advantage of that new data.
So it wouldn't surprise me if the up/down buttons were completely ignored by the recommendation engine. Maybe it's just a reminder to you: "Oh, yeah, I did see that. It sucked."
The new system clearly misses some things, but a lot of the problem is that the DVD catalog is immense and full of classics, but the streaming catalog is weak and full of crap. The whole thing may be back-ported into the DVD recommendation engine, but that's the redheaded stepchild at Netflix these days. They tried to spin it off entirely.
There are classic, important, society-changing movies from the previous century that were highly praised by contemporary critics and won scads of awards. But then people today watch them with zero context or understanding of what was happening or what the world was like and give them bad ratings because they can only compare them to the latest shoot-em-up sequel of a sequel of a sequel.
It's one of the reasons Rotten Tomatoes is worthless for anything more than 20 years old, and why the voices of movie critics are needed so badly today.
See also: Yelp.
The situation gets even worse from a diversity and internationalisation standpoint. You have to consider what different national and language groups want.
That's the opposite of "worthless".
The consequence is that you don't care about things like masses of teenage girls upvoting the latest vampire fad, people downvoting a classic because they don't understand the context or anything, because only your bubble matters to you. And if you fear that the bubble will keep you from discovering new things, there's a way to set up the algorithm to calculate your similar users so that they aren't exactly close to you - or you can go to a specific users profile if you know them and see what they're voting.
I find it amazing compared to IMDb or rottentomatoes, but for some reason I think it's mostly unknown outside of the Spanish speaking world.
Let's use a food analogy - I may go to a very fancy restaurant with exquisite food, and see that it holds up to its standards and is quite okay, and deserving all its stars. And in comparison I may have a basic hotdog or a sloppy burger, and admit that it's really nothing special, the fancy stuff was better, and that hotdog deserves less stars.... but at the same time that definitely does not mean that I always prefer the fancy stuff, and I can reasonably want that hotdog right now.
So that's the same with movie star ratings. I can at the same time agree that I liked movie A much better than movie B (and rank them accordingly), and I might re-watch movie A the fourth time and wouldn't bother with movie B ever... and at the same time not be in the mood for a complex movie like A, and want a simple, cheap, stupid movie similar to B, but with more explosions.
It's clear that 1 star sucks, but "5-star movies" and "4-star movies" perhaps simply indicate different genres of something roughly like "fancy" and "non-fancy" movies; and a fancy movie that's not really good doesn't get 4 stars but much worse; much like in the food analogy if the fancy restaurant serves something comparable in quality to that basic hotdog, it's not getting the same rating as the hot dog stand but much worse because of the unmet expectations.
Not only did the winning entry only bring in a slight improvement, and the payout followed suit, but they never actually used the winning algorithm.
Netflix's big winner for this contest, unplanned as it was, was their own recruiters and dev teams: Netflix became a company great developers wanted to work for because it cares about "hard problems".
They used to be alone in the space. Now they're competing with a lot of other services. Not in the DVD space, but I think they don't care about that space, and there are fewer people using it.
A lot of people suggest that there's another factor, that for streaming they pay different amounts for different content, and may not aim you at a movie that's expensive for them to show you. I have no idea if that's true.
2. They didn't want users to see that highly rated content was slowly disappearing from the service
Not rating does not mean the same thing as rating something as average.
The only case I can see where this incentive doesn’t exist is for shows that Netflix pays a fixed price to stream rather than a unit price per stream.
Yeah, it's extremely annoying. Netflix essentially mocks its customers by keeping the best works just out of their reach. Almost every time I search for a classic film, I find its sequels instead. The film I actually wanted to watch was apparently too good for Netflix's pay grade.
Netflix has Terminator 1 and 3 but not 2; the first Mad Max but none of the others; Spider-Man 1 and 3 but not 2; several Hannibal movies and a TV series but not Silence of the Lambs; some Star Wars movies but not others; the list goes on...
2016 blog post: https://netflixtechblog.com/selecting-the-best-artwork-for-v...
2018 presentation: https://www.youtube.com/watch?v=UjQMEjkrUGo
I’m not sure whether you’re trying to imply that non-ML based, non-personalised simple average ratings are better or worse than ML. Regardless, I have nothing nice to say about anecdotes.
I feel like netflix tends to recommend a lot of bottom of the barrel movies along with movies that I've already seen, there's no fine grained control.
I just doubt people actually care so much that they use them. It's a minor nice-to-have feature. But the law of triviality will suggest that everyone will argue about it every time Netflix comes up, and we do, even though I doubt we really care.
I don't think it's any more useful than searching "best $genre $year" when you're looking for new things to watch.
If anything a 3 star system is better, 1 - bad, 2 - neutral, 3 - good. Simple.
Often the 5 star system just devolves into 1, 4, or 5 anyways. Or in the case of averaged ratings like your ride share apps, you get stuck as a floating point. Suddenly 4.3 is significant vs 4.8. It's all silly.
I fully support the thumbs up or down. It's effectively a 3 star system anyways. 1 - Thumbs down, 2- neutral enough to not warrant a response, 3 - Thumbs up
edit: specifically, that people only give 1 or 5 stars
It's mostly just looking at trends. A few things I've purchased from Amazon come with a little advertising slip that says "if this product is anything less than 5 stars, let us know what we can do to make it better!" I've seen plenty of stories from people complaining that their less than 5 star reviews got removed for X reason (often because it's "dishonest").
Maybe that'll be my next learning exercise: scraping the count of reviews from Amazon to support this hypothesis.
“Netflix settled on “thumbs up/down," which is widely understood to imply that you are training an algorithm to know what you like, [Netflix’ Cameron] Johnson said.
“That simple change led to an over 200% increase” in ratings, Johnson said. The inclusion of a “percent match” number also reinforces the idea that these recommendations are personalized, he added.”
If you could only watch one movie at a time, sure you want to make it good. But when it's streaming, people mostly just want to binge watch garbage. Who needs million dollar prizes when you can do just fine recommending 50 different shows, until the user gives up finding anything good and watches a Netflix original.
Probably getting rid of recommendations led to some massive increase in viewership that couldn't be ignored.
I'm guessing that the definition of "better" changed at some point, perhaps when they went from 1-5 stars to +-1 thumbs.
In Netflix's early years, I was confident that their classifier's goal was to predict my future rating for a given video. I.e., it was part of a larger system designed around my viewing pleasure.
Around the time they switched to thumbs, I started suspecting the classifier was being used to manipulate me towards optimizing Netflix's profitability, instead of my viewing pleasure.
Why not? Every big platform that I know of has eventually done this with either 1p or “sponsored” content (Google, Amazon, Apple, etc.). In this case their incentive would be to push their own content above 3p content. What matters to them is their revenue, not maximizing the accuracy of their recommendations model.
Giving a boost to first party content is very different from making it "just worse".
As has been mentioned before, there is at least some motivation for doing so. One, it means their originals don't rate below third party titles (as far as we can tell), and two, as they lose content the average rating of what's left will likely fall.
While I'm ranting about Netflix, I absolutely hate the feature where if you leave a show paused for more than five seconds, it starts showing spoilers (the episode description). I often open Netflix before I eat a meal, get it to the start of the next episode of a show I'm watching, and pause it until I'm ready to watch. And then I carefully avoid looking at the screen to avoid the episode description/spoilers. The descriptions are only useful after I've watched the show and want to find a specific episode. I don't want to see them before I've watched the episode!
The Netflix app just seems so actively hostile. I'm used to apps being unpleasant because of bugs, but being unpleasant because of features is unusual.
My guess is that for some majority of users who weren't super bothered by it it genuinely caused them to watch more things. Even though a vocal minority was extremely upset about it, the numbers still worked out overall. Of course adding a preference is easy, and people who aren't upset won't bother to change the default, so this is probably a win/win for them.
In other words, could you separate out:
- folks who naturally came to a stopping point between shows.
- others who ragequit because of annoying previews?
I do like the feature. If I'm actively looking for something to watch, I like the fact that I can quickly see a preview. They normally don't start playing until I've finished reading the excerpt anyway, so it's not some massive annoyance. By the time the preview starts to play, there is a high probability that I want to see it. And if not, I just continue scrolling. It is the tiniest of inconvenience for an overall improvement to the experience of finding something to watch.
I suspect there is a high correlation between people that hate autoplay and people that hate the open office. Some people find tiny little distractions absolutely unbearable, but I don't.
If I am actively looking for something to watch on TV, why would I be so bothered that a video starts playing on my TV? I don't really understand the use case of someone that wants to leave the Netflix home page up just hovering over a title.
Liking to see a preview is fine - and there could be a button for that, so that I could tell the app to show it. But the way it is (was) I don't have the control, it's the UI who decides where my attention should be, and that's wrong.
The best comparison I can make it's an overeager shop attendant asking "can I help you?" When you're looking at a product - except in this case it doesn't matter how many times you tell them you're just looking, they come back 8 seconds later to ask again.
I watch on my Roku tv, where there is only a few titles on screen and it makes sense to scroll threw them one by one. Maybe that's the difference?
Maybe now that they have competition, we'll see more stuff like this.
Yes, we have more than one account. Mistakes happen.
Whether they make `autoplay=true` default will answer this.
Absolutely terrible. I'm really happy Netflix has given us a way out.
It shows up in "continue watching" and I believe probably stuff like "because you watched A..." (...here are B, C, D, and E).
Which is just silly because even if they want to steer me into watching it because I'm too lazy to tell them to stop, that doesn't mean it makes meaningful data for recommendations. They can tell whether I pushed a button or not, but which seems like a pretty meaningful signal to ignore.
I literally finished Bojack Horseman after years of watching 30 minutes ago. End it’s left hanging there in a poignant moment but I have to jump up to stop the trailer for god knows what, and the a minute later I have to do it again after the credits have finished. Years of investment in a show, but I’m not allowed 10 seconds to stop and digest it. (PS amazing series)
This option is to turn off autoplay when browsing. It would be great if it also disabled the auto play on trailers at the end of a show.
If you use greasemonkey/tampermonkey then the script below will disable the post-episode playback (along with a couple other UI changes)
So all I can say is "about time"!
From feedback like this, they might have just made a policy to not allow cursing in the preview whether that was by choosing different scenes or bleeping them.
Actually, I hate most of the current design (UI/UX). It's really streamlined when all you want to do is push "play", but everything else is, quite frankly, laborious.
Good user interfaces use timeless concepts. This might imply that conservative folks make the best UIs, which might not be as sexy.
And a good browsing experience shouldn't be held hostage to "featured promotions" or worse, advertisements.
Maybe there's a market for nice experiences.
I think of costco vs other stores -- at costco, you go get your stuff, pay for it and get out. At other stores there are featured products and advertising influencing the store layout and cluttering the checkout line.
Bought a new one late last year, and the whole UX is a big step backwards, but the Netflix app and the automatically playing previews is high on the list of annoyances.
It feels like one of those "Do you actually dogfood this? Have you actually compared it to previous releases?"
These two complaints are actually two manifestations of the same complaint. With a shrinking catalog, it is vital that the UI be nerfed in order to keep it from being too obvious just how little content there is.
Goes to show how hated autoplay was.
Maybe a month back, they started cutting off playback as soon as closing credits started to roll. I can understand the motivation because sometimes credits are boring.
But other times, they are part of the enjoyment of watching the show. For example, every episode of "Grace and Frankie" ends with a different song, carefully chosen to fit in thematically with that episode. People have written articles about these song choices, made Spotify playlists, etc. Yet Netflix cut them off. (And that's a Netflix original show.)
Anyway, in the last few days, it seems like they've stopped doing this. They now have a button in the UI that will stop playing the credits, but the button looks different, and it doesn't cut the credits off by default.
Goddamnit, I just woke up every mammal in the house.
Netflix; your jingle/call sign thing doesn’t need to be the loudest thing on on my TV. Get some group therapy, find your self esteem and chill the fuck out.
Perhaps we can thank the fact that streaming services now have competition, and the quality of their UX matters quite a bit.
> Some people find this feature helpful. Others not so much.
> We’ve heard the feedback loud and clear — members can now control whether or not they see autoplay previews on Netflix.
But happy it's optional for those who never warmed up to it.
If I watched 2 episodes and marked show thumbs down I DO NOT WANT TO SEE IT AGAIN!
Instead my continue to watch is a list of shows i didn't like!
How hard is to figure it out, or at least give me an option to black mark a show. Anything really. They spend millions on show and then package then is such user unfriendly manner .
I honestly stopped watching most netflix due to this feature.
I would go there to watch something specific but I would not browse around because of that annoying auto-play crap
I am still getting AutoPlay on Roku
I have 2 Roku;s. 1 is working (no autoplay) and 1 is not (still autoplay)..
the one with the AutoPlay still enabled is my primary one, and the newest one, the other is a older Roku Streaming stick.
On a serious note, I get why they added it.
For some, it saves a few button presses.
But boy was that an annoying feature.
Literally made browsing Netflix stressful.
Next feature request:
And to disable auto-play of next episodes (now happily obsolete, perhaps) and "promoted post-play preview content".
There is no UI so you'd have to toggle the script on/off if you wanted to resume watching intros and recaps.
If they do a clip of someone we thought died six episodes ago, guess what, they aren’t dead.
Of course, a weakness of this testing is failing to account for negative metric impact outside of the one you are optimizing.
update: looks like 1 of my 2 Devices is working correctly (not AutoPlay), so here hoping the other starts soon
I consider the previews spoilers. And autoplay destroys pacing because it skips the preroll/credits.
> Note: There may be a delay before the setting takes affect.
"Inigo Montoya: You keep using that word. I do not think it means what you think it means."
Now, I'd like to see them do this:
1) Quit emailing me/sending notifications to keep watching shows I start and don't like.
2) Bring back a purposeful, decent recommendation system.
3) Allow filters so only movies/shows in your language or dubbed in your language are shown. (...or just some kind of filter.)
Email prefs are here: https://www.netflix.com/EmailPreferences
Marketing prefs are here: https://www.netflix.com/MarketingCommunications
I should have clarified my response. When I contacted netflix they told me you cannot unsubscribe to one part of the notifications ... it is all or nothing. So, if you turn it off you aren't alerted when new shows are added, etc.
This is what it looks like with an episode list at the bottom.