
Al Gore’s New Movie Exposes the Big Flaw in Online Movie Ratings - mcone
https://fivethirtyeight.com/features/al-gores-new-movie-exposes-the-big-flaw-in-online-movie-ratings/
======
jcfrei
The following explanation is probably implied but I still want to mention it:
When a movie (or any form of media really) discusses such a polarizing issue
like climate change doesn't the rating always lose its meaning? When a film,
book, etc. treats a polarizing issue, most people probably abuse ratings as a
(binary) vote on whether or not they agree with the narrative. And not whether
it's a good movie based on film theory (ie. characters, visuals, story arc,
etc.). Does this issue have a name? We see it on HN too, where people use up-
and downvotes for agreeing/disagreeing, rather than separating informative,
from non-informative comments.

~~~
trgv
Another thing to keep in mind here is that these kinds of review systems will
often result in high amounts of polarization regardless of whether the topic
is controversial. My feeling is that people often vote based on how they want
the current average to change rather than based on how much they liked the
movie.

E.G. Sometimes I'll watch a movie, I'll think it's an 8/10 but then see it has
4/10 on imdb, so I'll give it a 10.

Presumably this may affect another person's vote. Maybe someone saw the film,
hated it, wondered why the review was so high, and gave it a 0 to bring the
average down.

IMDB recognizes this and uses a "secret formula"
([http://www.imdb.com/help/show_leaf?ratingsexplanation](http://www.imdb.com/help/show_leaf?ratingsexplanation))
as a result.

~~~
euyyn
> Sometimes I'll watch a movie, I'll think it's an 8/10 but then see it has
> 4/10 on imdb, so I'll give it a 10.

Do you really do this?

------
maldusiecle
A good context to this piece would be the recent NYTimes article ("Attacked by
Rotten Tomatoes") that claims Rotten Tomatoes is damaging sales for films.
Everyone is to blame except for the studios, who are producing bad movies that
no one wants: endless sequels, remakes, and adaptations of successes in other
media.

If An Inconvenient Sequel really were controversial, that would be all the
more reason for its success. Controversy sells. In fact, it's the least-risky
kind of film to make: a sequel to a widely-lauded, successful film (An
Inconvenient Truth has 93% Fresh on Rotten Tomatoes). Hollywood execs can cry
me a river.

~~~
dagw
_who are producing bad movies that no one wants: endless sequels, remakes, and
adaptations of successes in other media._

How do you square that belief with the fact that 8 of 10 top grossing movies
so far this year (and 7 out of 10 for last year) is a sequel, remake or
adaptation? Obviously someone seems to want them

~~~
corysama
Everybody hates sequels when they sit down and think about it. But, when they
don't... When it's "Eh... Let's go see a movie. What's playing?" They are
faced with two investment choices: One selection of high-risk, unknown,
unproven, original movies Vs. another selection of low-risk, familiar, proven
productions --some of which are well-known to be liked by their group and
others are well-understood to be skippable.

The "Eh... Let's go to a movie" scenario makes up a far larger portion of the
total revenue than the carefully researched, highly anticipating planners. It
should be no surprise that they spend so much on sequels and adaptations.

~~~
luord
> Everybody hates sequels when they sit down and think about it.

As someone whose only hobby is watching and discussing movies: No, this isn't
true.

------
some-guy
I suspect that Al Gore's name had the biggest impact on the polarization of
the score. He's been a polarizing figure (for whatever reason) for a very long
time. Not to mention it's a "sequel".

I'm not attacking the man in any way, but I do wonder if furthering the
polarization of the issue using his name (which he cannot reconcile with the
public at this point) does the movement a disservice. I place more blame on us
as a species than Al Gore himself, but here we are.

~~~
mikeash
I doubt that leaving his name off of it would really change much. The people
giving it the lowest possible score are reacting to what they see as political
propaganda pushing the hoax of climate change. That Al Gore is involved
certainly doesn't help, but they'd still hate it even if he had nothing to do
with it.

~~~
Brakenshire
At some point a significant proportion of the people who have been ridiculing
climate change (or to a large extent just going along with a political
movement which incidently ridicules it) will shift their beliefs to accept
moderate action to tackle climate change, and it's important they can do that
without being forced to admit they are actually changing their minds.

That means leaving open, for people who previously dismissed it, the narrative
of "I don't believe in climate alarmism pushed by radical leftists, but I do
believe in following the science, and now that the evidence is in, I support
reasonable action". I think it is true that a figure like Al Gore acts as an
anchor to make those sorts of changes more difficult.

~~~
exclusiv
Great points and I think you're spot on.

I'm personally a skeptic of the claims made by the AGW side and the alarmism
pushed out by so many. The proposed AGW solutions are not feasible worldwide
either which is what would be necessary to solve the problem they outline. I'm
saying this not to get into a discussion about it at this time but to lay out
why I would STILL support the agenda: essentially there are several great
secondary benefits from doing what they recommend.

This is the biggest failure of the AGW side - they have terrible positioning
and think you're stupid if you question science. If you tell people they're
stupid (even if they are) they aren't changing their minds.

This is how you push the AGW agenda. "Let's say we're wrong. Here are the
benefits of doing what we're suggesting and making some changes &
investments":

\- Better air quality

\- It's good for national security - we no longer need to send large sums of
money to hostile countries, many of which support terrorist organizations and
other groups which are Anti-American / Anti-Western values.

\- Cheaper energy

That's hard for anyone to argue, assuming the legislation is sensible.

~~~
koube
That argument is already being made, if people were open to this line of
reasoning they would have heard it a long time ago.

I think the longer term and overarching issue is that we don't have a shared
reality. Science is not a method of finding truth for a large proportion of
the American electorate. Academia, scientific institutions, and journalism
might as well not exist if people don't believe in them.

If that's the case, then it's requires a much more in depth answer than "well
look, we might do some good if we spend billions in (perceived) cost". I don't
have the answer and perhaps there isn't one.

------
jmcdiesel
More evidence supporting the claim that, as a society, we have completely lost
the capability to have a conversation on anything. Everything is polarizing
and everyone feels the need to pick (and defend) a side.

~~~
thrill
Well, in reference to the subject of this particular movie, one "side" of the
"conversation" is frequently in the news as wanting to put nonbelievers of
their view of the issue in prison. That might tend to create some animosity.

~~~
city41
Can you give some examples? I have never heard of this.

~~~
kbutler
[https://www.google.com/search?q=climate+change+deniers+%28de...](https://www.google.com/search?q=climate+change+deniers+%28death+penalty%2Cjail%29)

Death penalty, tried for crimes against humanity, jail, fines for
disagreeing...

I don't think any of these are mainstream ideas yet, but there was failed bill
in the California legislature to have the AG sue those who "spread
disinformation" on climate change.
[http://www.washingtontimes.com/news/2016/jun/2/calif-bill-
pr...](http://www.washingtontimes.com/news/2016/jun/2/calif-bill-prosecutes-
climate-change-skeptics/)

Can't reply to responder below - but I am not your research assistant. The
parent asked a question, I showed where they could find answers. The google
search above links to a professor speculating on death penalty for skeptics,
and others speculating on trying them for crimes against humanity.

For spokesmen, Al Gore didn't mention jail, but said we should "punish" and
politicians should "pay the price" for differing perceptions (and note that
even climate change advocates who are not extremists get branded as
"deniers").

[http://www.chicagotribune.com/bluesky/originals/chi-sxsw-
al-...](http://www.chicagotribune.com/bluesky/originals/chi-sxsw-al-gore-
penny-pritzker-bsi-20150313-story.html)

""" Former Vice President Al Gore on Friday called on SXSW attendees to punish
climate-change deniers, saying politicians should pay a price for rejecting
“accepted science.” ... we need to put a price on denial in politics """

~~~
SippinLean
Re: your edit:

>but I am not your research assistant

Correct. Also correct: you need to cite specific examples to back up the
claim, which you have not done. "A professor speculating" and the nebulous
"others" are, again, not in such a prominent position as to speak for their
"side" as a whole.

>For spokesmen, Al Gore didn't mention jail

Correct, which would make this _not_ an example of "wanting to put
nonbelievers of their view of the issue in prison," OP's only claim.

~~~
kbutler
>>but I am not your research assistant >Correct.

Glad we cleared that up.

>Also correct: you need to cite specific examples to back up the claim, which
you have not done.

Hmm. See above.

I chose to contribute to the conversation by pointing to examples of behavior
that poison the discussion, in the spirit of thrill's comment.

I personally thought "death penalty" and "war crimes" were stronger examples
than just "prison".

I did not attempt nor do I need to prove the ancestor literally correct, but
just for fun, here are a couple of links that do - you'll have to decide if
Bill Nye and Robert F. Kennedy, Jr. meet your criteria as "prominent" (which
was not in thrill's post - just that they are in the news).

Rfk, jr. [http://m.washingtontimes.com/news/2014/sep/23/robert-
kennedy...](http://m.washingtontimes.com/news/2014/sep/23/robert-kennedy-jr-
we-need-laws-punish-global-warmi/) Koch brothers should be in prison in the
Hague

Bill Nye [http://m.washingtontimes.com/news/2016/apr/14/bill-nye-
open-...](http://m.washingtontimes.com/news/2016/apr/14/bill-nye-open-
criminal-charges-jail-time-climate-c/) compares it to jailing guys from Enron

Examples, plural, specific, people in the news claiming their opponents should
be imprisoned.

Good enough for me.

~~~
SippinLean
I agree, you did not prove them correct, indeed that was my only point.

Now you have presented _new_ evidence. Simply linking to a Google search was
not sufficient after all?

The RFK quote gets thrown around a lot, in context it's much more clear:
[https://www.youtube.com/watch?v=41yJTxrPFhM](https://www.youtube.com/watch?v=41yJTxrPFhM)

He says the Koch brothers are treasonous, and for _that_ reason should be
arrested. He goes so far as to clarify that there is no law politicians can be
punished under (the opposite of the ancestor's claim).

Bill Nye compares them to the guys at Enron. He does not suggest jailing them.

~~~
kbutler
SippinLean> The RFK quote gets thrown around a lot

Thrill> one "side" of the "conversation" is frequently in the news as wanting
to put nonbelievers of their view of the issue in prison

Yeah, what you both said.

(And Rfk said the Koch's "treason" was co2 "pollution", and he wished there
were a law, which pretty much matches thrill's statement)

So we can agree that the idea is "thrown around a lot", even apparently by
someone prominent enough to be recognized as a spokesman, but (thankfully or
regrettably) there isn't yet a law to enforce it.

------
deanCommie
I'm surprised the article makes no reference to the worst of this phenomenon:
2016's The Promise:
[http://www.imdb.com/title/tt4776998/ratings?ref_=tt_ov_rt](http://www.imdb.com/title/tt4776998/ratings?ref_=tt_ov_rt)

* 52.9% 10/10 * 42.6% 1/10

For a total of 95.5% polarizing votes!

Interesting that The Ottoman Lieutenant is top of the 538's list and it deals
with exactly the same subject matter.

Turks really need to grow up and stop being such babies about their history.
Face it, don't go spamming down the ratings for movies that dare talk about
your history.

------
jpfed
See also Zoe Quinn's "Crash Override"\- totally polarized reviews, with
notable difference between verified purchasers and not.

[https://www.amazon.com/Crash-Override-Gamergate-Destroyed-
Ag...](https://www.amazon.com/Crash-Override-Gamergate-Destroyed-
Against/product-
reviews/1610398084/ref=cm_cr_getr_d_paging_btm_1?ie=UTF8&reviewerType=avp_only_reviews&pageNumber=1)

~~~
favorited
Someone writes a book about how anonymous people on the internet tried to ruin
her career by spreading fake stories. People don't like that, so they post
fake reviews in an attempt to ruin the book's sales.

...making the author's point for her.

------
baron816
I wish someone would create a more personalized algorithm for movies. Use the
movie critics' reviews, and compare the other movies they liked to the other
movies you liked. Even bad movies get a few good reviews, and there are people
out there who would still enjoy those generally poorly received movies. Why
not show me the reviews of people who liked a lot of the same movies I liked?

------
dmix
I found the fact "The Emoji Movie" getting 74% 1-star reviews in their ranked
chart to be a more interesting fact. Climate change and Al Gore being
controversial and highly politicized is nothing new. South Park has been
satirizing Gore's hyperbolic approach for years. An approach which tends to
draw tribalistic audiences who have opinions formed well before hearing the
arguments.

But the fact Hollywood spent $50 million on such a disaster as T.J. Miller's
big feature film is what is somewhat shocking. Especially given it had some
big names like Columbia Pictures and Sony Pictures Animation behind it.

You'd think with the data-driven cookie-cutter focused-grouped-to-death
process that is the modern film industry would be able to avoid producing one
of the worst rated films in the history of online film reviews. I thought
modern Hollywood was extra careful? Or are those resources only limited to
hedging their financial performance rather than producing enjoyable content?

~~~
minimaxir
The primary KPI of any movie by a major studio is _box office revenue_. Making
the movie good is a correlation but not causation. (Especially for children's
movies.)

Case in point, the Emoji Movie made way more than its budget. ($150M)

------
helicon
Begs the question why does the IMDb voting system allow reviews before the
movie's release date?

~~~
jhbadger
Because many movies are actually screened months before the official release
date in film festivals and the like.

------
bluetwo
I'm curious if there was an organized campaign to give the movie low votes or
if it was simply an army of similarly minded trolls.

------
suckerburg
I would like to point out that there are times when critics pay more attention
to the hype surrounding the movie when it fits their agenda, "XX" the horror
anthology film directed by women directors for example was hated by the
audience because it's bad. Check out the "I Hate Everything" review for the
film.

------
acd
Filter bubbles ie when you consume content by your previous content to provide
confirmation bias on your already existing ideas.

Filter bubble
[https://en.wikipedia.org/wiki/Filter_bubble](https://en.wikipedia.org/wiki/Filter_bubble)

Confirmation bias
[https://en.wikipedia.org/wiki/Confirmation_bias](https://en.wikipedia.org/wiki/Confirmation_bias)

On rating systems including how the IMDB rating system works
[https://stats.stackexchange.com/questions/6418/rating-
system...](https://stats.stackexchange.com/questions/6418/rating-system-
taking-account-of-number-of-votes)

------
Florin_Andrei
> _men hate the film way more than women do_

The "climate change is bullshit" narrative is engineered to appeal (among many
other things) to this whole mythology of rugged individualism. If you "believe
in that global warming stuff" it must be because you're a weakling - as
opposed to the strong manly individuals who choose to reject it.

It's gut instinct politics.

I've a hunch (heh) that gut instinct politics correlates with falsehood (and
perhaps specifically with engineered, purposeful falsehood) more strongly than
intuition would guess. E.g. the whole rise of the National Socialist Party in
Germany in the 1930s was entirely predicated on a backwards, anti-
intellectual, think-with-your-guts ideology.

~~~
ankushnarula
That's just one of the counter-narratives. Unlike the "97% narrative", the
counter-narratives are not monolithic.

Another counter-narrative says that climate change is happening but the models
to date are inaccurate and overblown.

Another counter-narrative says that man-made or not, it's hubris to assume
that humans can do something about it.

And yet another counter-narrative acknowledges climate-change but says that
the prescriptions for fighting climate change aren't guaranteed to work and
will hurt humanity - in that we should spur innovations that continuously
adapt to climate change and boost economic prosperity for all mankind while
avoiding worldwide austerity controlled by an earth-worshipping misanthropic
orthodoxy. (or something like that ;) )

~~~
Florin_Andrei
All that is technically true.

What's missing is that Reality doesn't care about human narratives. It is what
it is. Narratives align with it in varying degrees. Some align with it quite
well, and we call them true. Others don't align with it at all, and we call
those bullshit.

------
redm
In general, I find 1-10 rating systems too much, it's just too much
granularity when the measure is purely subjective anyway.

I like 1-5 better, but the descriptions are the best, "Hated, Didn't Like, No
Opinion, Liked, Loved".

At the end of the day, if your sample is big enough, I think the new Netflix
approach is really the best, liked or didn't like, and then you can draw from
that what you want based on demographics, etc.

------
simulate
This is why YouTube went from a 5 star rating system to a thumbs up / thumbs
down system back in 2009. See: [https://youtube.googleblog.com/2009/09/five-
stars-dominate-r...](https://youtube.googleblog.com/2009/09/five-stars-
dominate-ratings.html)

------
tim333
I'm not sure the ratings are that broken in this case. Googling it I get:

78% Rotten Tomatoes 5.5/10 IMDb 3/5 The Guardian

above the fold, which is enough to guess it's quite good but not amazing. It's
kind of obvious IMDB will be polarized if you look or just use common sense.

------
RickJWag
Gore's response to critics: "You don't have to get snippy about this!"

------
Isamu
This is a worthwhile analysis - interesting to see the graphs and breakdown
along different angles.

But I don't see the "big flaw" exposed. You've democratized movie rating and
got an accurate reflection of the polarized feelings on the topic. Are you
arguing people should be "objective"? Are they objective about any movie?

People voting before really seeing the movie - a problem yes, but the outcome
wouldn't change much. And it is an implementation problem, not a problem with
online rating per se.

~~~
favorited
The "big flaw" is that a film that people either love or hate is
indistinguishable from a film that everyone thinks is OK.

~~~
Isamu
So on Amazon page we see the distribution in addition to the average score.
People clearly either love it or hate it.

[https://www.amazon.com/Inconvenient-Truth-Al-
Gore/dp/B00AB0S...](https://www.amazon.com/Inconvenient-Truth-Al-
Gore/dp/B00AB0SHRW)

Plus it skews toward the positive, since we would expect a verified buyer is
more likely to be someone that already approves.

On the other hand, films that everyone thinks are OK are still
indistinguishable from a film I may either love or hate. This is not a flaw,
but inherent in the idea of ratings itself.

I don't see bimodal distributions as a problem, but hiding them might be.

------
wnevets
Climate change conspiracy theorist really hate the truth.

