

The Impact of the Megaupload Shutdown on Movie Sales - valgaze
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2229349

======
dude_abides
The authors are from a prestigious school, CMU. Still, I was curious to know
who funded this research. Alas, not surprisingly, it is MPAA.

The CMU lab is called Initiative for Digital Entertainment Analytics (IDEA),
and according to CMU's press release _"The creation of IDEA was made possible
through a gift from the Motion Picture Association of America (MPAA)"_

[http://www.cmu.edu/homepage/society/2012/fall/entertainment-...](http://www.cmu.edu/homepage/society/2012/fall/entertainment-
analytics.shtml)

 _"The MPAA was interested in CMU in part because of our unique strengths in
the performing arts, computer science and technology, and management," noted
Smith._

Looks like MPAA also had something else in mind, when they funded this lab.

~~~
maxmcd
Not disagreeing, but whenever there is a study posted where the funding comes
from a source that has an obvious interest in the outcome of a study there is
always a comment like this one pointing out the conflict.

I understand providing information beyond that would be likely very difficult,
but having never been in a similar research situation, could someone outline
what this influence is like? If there an underlying pretense during the
research that they have to reach a specific outcome? Is it more
subtle/obvious? Any additional insight would be appreciated.

~~~
dude_abides
I became suspicious because the writing style of this paper was very
superfluous, yet the conclusions were very strongly worded. Here is an example
in the very first page:

 _for each additional 1% pre-shutdown Megaupload penetration, the post-
shutdown sales unit change was 2.5% to 3.8% higher, suggesting that these
increases are a causal effect of the shutdown._

This does not suggest causality. You need more rigorous inductive reasoning,
and randomized trials, if you want to conclude causality.

Edit: Just found out that SSRN is not even a peer reviewed journal! (
[http://www.ssrn.com/update/general/ssrn_faq.html#no_paper_su...](http://www.ssrn.com/update/general/ssrn_faq.html#no_paper_sub)
)

~~~
andrewfelix
I agree it would be hard to draw sound conclusions from the study. The changes
probably aren't even outside variability.

But are these the claims really that outrageous? ie. _The biggest piracy site
in the world has a slight effect on sales?_

~~~
drucken
It is meant to be a scientific study. What's the use of an unsubstantiated
claim? In fact, its been proven to be highly counterproductive (at least in
other fields) as others go on to build works and further claims around it,
only to later discover the errors and depth of uncertainties.

Using the same principle, I could posit (probably quite defensibly), that the
amount of alleged copyright infringment of Megaupload Limited's limited access
and centralized services would be dwarfed by that from Bittorrent and other
P2P methods.

------
jowiar
Personally, I don't pirate movies/music, but as a CMU (SCS + Heinz) alum, the
lack of rigor and potential conflicts of interest displayed in this paper are
disheartening. More than that, they cheapen the value of my degrees.

One of my big pet peeves in data analysis is making major inferences and
broad, sweeping statements based on a very small sample size. This paper
appears to be a prime offender.

Looking at this case, there's a pretty good argument that Netflix and the
LoveFilm app launching in the UK, QuickFlix Streaming launching in New
Zealand, The Artist (with a French director and star actor) getting a lot of
attention and being released to rental, a particularly chilly winter in
western Europe, and god knows what else all contributed in whole or part to
the observed phenomenon. I'd love to have whatever data the authors of the
paper had, to facet the data by movie studio and country - did different
countries peak on different weeks, and the like. Alas, the data consists of
the sales and rentals of two anonymous major motion picture studios.

That none of these potentially seriously confounding factors were mentioned or
explored before reaching for a conclusion is somewhere on the scale between
negligence and dishonesty, and the lack of data transparency renders the the
paper useful for nothing more than kindling.

Any fellow Tartans here on HN (I know there are a bunch of us) want to sign on
to a letter requesting an investigation into the integrity of the paper and
IDEA?

~~~
nonamegiven
"Personally, I don't pirate movies/music, but as a CMU (SCS + Heinz) alum, the
lack of rigor and potential conflicts of interest displayed in this paper are
disheartening. More than that, they cheapen the value of my degrees."

It's probably also worth writing a letter stating exactly that to your school,
your department, and the department involved, as well as a letter to your alum
magazine.

------
caublestone
Does this study account for the fact that 2011 was the worst year for movies
(in terms of quality) ever? Look at the academy award winners and the top box
office performers of that year and compare it to 2012.

Side note: I would like to see a study on the quality of movies in 2011 vs
2008 based on economies 3 years previous. So when movies were getting funding
in 2005 for release in 2008, we got some real gems. In 2008, when the economy
took a down-turn, we see some bad ones in 2011.

<http://boxofficemojo.com/yearly/chart/?yr=2011>
<http://en.wikipedia.org/wiki/84th_Academy_Awards>

[http://boxofficemojo.com/yearly/chart/?yr=2012&p=.htm](http://boxofficemojo.com/yearly/chart/?yr=2012&p=.htm)
<http://en.wikipedia.org/wiki/85th_Academy_Awards>

~~~
sturadnidge
Exactly this. The 'study' hasn't controlled for anything. There precisely no
discussion of the weather (the summer months are suspiciously absent from the
data set, nor the weather patterns commented on - Sydney, Australia had one of
the wettest summers on record over 2011/2012), the economies of the countries
involved, let alone a year over year data comparison.

The only thing "gone in 60 seconds", is the credibility of the authors of this
study.

------
waterlesscloud
Is it really so far-fetched that closing down a major piracy site would
increase legitimate downloads by 4-6%?

~~~
andrewfelix
No.

Quite interesting looking at the comments here furiously looking for flaws in
the study.

If the results had found the opposite, I doubt the study's legitimacy would
have been so vigorously questioned.

~~~
jeremyswank
I don't think it would work very well if it were the people who agreed with a
study to somehow also have the obligation to point out its flaws. Could we
believe that they were being thorough in finding problems if the conclusions
supported their own beliefs? It works better, I think, if the challenger takes
an adversarial, but not actually hostile, position. And, of course, for both
sides to have open minds, but I realize that is often too much to ask.

~~~
anigbrowl
Well, that actually happens a lot with scientific studies where people are
interested in the result but want to point out the limitations of the study so
that it doesn't get over-hyped and lost in a backlash. I much prefer a
qualified conclusion.

------
DigitalSea
This is most certainly not a biased funded study with links to the MPAA guys.
Without-a-doubt it is definitely an independent study performed by real people
with no links to the entertainment industry. Now if you'll excuse me guys I'm
just going to go to the cinema to watch a $30 per ticket blockbuster movie
because the entertainment industry needs our support. I'll be back later.

Definitely not suspicious!

~~~
fragsworth
If piracy didn't actually cost the entertainment industry anything, wouldn't
you think they would stop wasting any money trying to prevent it?

I would assume they'd try to be fairly objective with this study, because
they'd be very happy to discover that piracy is a good thing for them.

~~~
vacri
On favourite tactic of the movie houses is to report on decline in US domestic
sales (and they are, or at least were last year), but report on piracy
globally. They don't report on global sales, because they're raking it in.

~~~
anigbrowl
Half-true. Foreign distributors get a large cut of that on a hit, but lots of
films don't play to a foreign audience while still suffering from the decline
in domestic sales. Said decline hits theaters and the people who work there,
who are not subsidized by a film's overseas profits.

~~~
vacri
Global piracy also does not contribute to the local cinemas demise - it's an
intentional apples vs oranges comparison.

------
zacharyvoase
It says that they analysed 12 countries. Is it possible that they selected
these countries specifically, and rejected others that may have refuted their
hypothesis?

I'm trying to say this with as little judgement as possible. I just know that
deciding to look only at certain countries to support a hypothesis has been
done before (c.f. Ancel Keys's Seven Countries Study).

~~~
andrewfelix
They were looking at data from two major studios. I assume those studios had
good penetration/data for those particular countries.

~~~
lessnonymous
I am concerned that the MPAA, in commissioning the study, provided the
researchers with data from two studios who saw a big lift in sales in the
period following the shutdown.

I've not read the paper so I've no idea if this is an issue or not, but only
using the data of two studios seems to be a low sample size.

Imagine if the week after the shutdown "Lord of the Rings" was released, or
"Twilight", or any other blockbuster. Find two studios with big releases in
the period after the shutdown and it looks like some significant causation.

~~~
sambe
Is it completely unreasonable to also suggest that they would simply
manipulate the data? I know, it's a strong claim without evidence. However,
I'm under the impression that sales figures are routinely manipulated for
other reasons, and it definitely wouldn't be beyond these companies morally-
speaking to do so here.

------
kevinwuhoo
There are a few points from this publication that are suspicious:

1\. There's no mention of funding sources for this study.

2\. It's a small sample size of "two major studios".

3\. Most importantly, this study isn't peer reviewed. [1]

[1]: <http://www.ssrn.com/update/general/ssrn_faq.html>

~~~
a_bonobo
I think one can assume that the funding comes from the IDEA from this
sentence:

>This research was conducted as part of Carnegie Mellon University’s
Initiative for Digital Entertainment Analytics (IDEA).

The IDEA is apparently a think-tank to help "groups like MPAA and policymakers
[to] navigate the changing digital media landscape."[1]

Interestingly, funding for the IDEA has been made possible by "an unrestricted
gift from the Motion Picture Association of America." [2]

Edit: From one of the author's blog it looks like the study has been submitted
for peer review: "The study is currently under peer review at a good econ
journal." [3] It seems that the SSRN where this paper is currently hosted is
to social sciences and economics as to what arxiv is for natural sciences: a
repository for yet unreviewed drafts of papers.

[1]
[http://www.bizjournals.com/pittsburgh/blog/innovation/2012/1...](http://www.bizjournals.com/pittsburgh/blog/innovation/2012/10/mpaa-
teams-with-cmu-to-fund-new-center.html)

[2] [http://www.techtaffy.com/carnegie-mellon-launches-digital-
en...](http://www.techtaffy.com/carnegie-mellon-launches-digital-
entertainment-analytics-initiative/)

[3] [http://brettdanaher.wordpress.com/2013/03/12/megaupload-
shut...](http://brettdanaher.wordpress.com/2013/03/12/megaupload-shutdown-
increased-sales-of-digital-movies/)

------
jared314
Brett Danaher:
[http://scholar.google.com/citations?user=w7W3zn0AAAAJ&hl...](http://scholar.google.com/citations?user=w7W3zn0AAAAJ&hl=en&oi=ao)

Michael D. Smith:
[http://scholar.google.com/citations?user=rzPlMfQAAAAJ&hl...](http://scholar.google.com/citations?user=rzPlMfQAAAAJ&hl=en&oi=ao)

From the paper:

Our [Megaupload] Internet penetration data is based ... Google AdWords
estimates of the total number of unique visitors ... [and] ... the total
number of Internet users in each country in our sample...

As such, the shutdown of Megaupload influenced the policies of several other
cyberlockers focused on piracy, and our results necessarily measure the “net
impact” of the Megaupload shutdown across the cyberlocker industry, as opposed
to just measuring the impact of Megaupload.

------
dmak
"Aggregating these increases, our analysis across 12 countries suggests that,
in the 18 weeks following the shutdown, digital revenues for these two
studio’s movies were 6-10% higher than they would have been if not for the
shutdown. "

How can you compare with something that is nonexistent? If they had a control,
a timeline with piracy, then this comparison would be valid right? I mean, the
effects of 6-10% higher could be just the hottest titles that came out in the
summer such as The Avengers, etc...

~~~
anigbrowl
You can compare year-on-year because studios spread their releases to maximize
revenue. I'm having problems accessing the site to DL the paper right now but
I have a hunch you're arguing with the adstract, not the methods.

------
joshka
I read the article and have a few questions to the authors:

How was the megaupload estimate calculated?

Various parts of the article use per week data to measure sales / rental
trends, however the megaupload estimate for each country was held constant,
given there's a likely correlation (with a potential time lag - i.e people buy
stuff on the advice of people that have already seen it and recommended), how
would this variation effect the modelling?

What sort of measurement error does the google adwords estimate have? How does
this effect the model?

Are there any factors that might impact the adwords estimate on a per country
basis (such as prevelance of use of particular browsers, adblocking plugins,
etc.)? How does this effect the model?

Do you have year on year data to compare this data with?

Do you have any data to show correlation between sales and MPR over time?

Does your data differentiate on movies that were available on megaupload
before it closed vs those that were released afterwards? How does this effect
your model?

------
t4nkd
So, using their extremely scientific method, did anyone else experience
increased site traffic, or sales? I'm sure we can link other industries who
deliver digital goods(software) so the closing of a digital locker site.
Correlation === Causation; the pinnacle of statistical research!

~~~
qompiler
One of the authors of this 'paper' is Brett Danaher. He is a member of the
SERCI (The Society for Economic Research on Copyright Issues) which directly
gets funding from SGAE (Sociedad General de Autores y Editores) which is a
large collecting society (copyright collective) for songwriters, composers and
music publishers in Spain.

~~~
t4nkd
So, what you're saying is, the next published paper will be on how Megauploads
shutdown has increased the sales of music?

I wonder what dinner theatre ticket sales look like over this term, maybe
digital piracy is killing the stage!?

------
iambibhas
6-10% increase should be the normal sales amount variation, right? Any given
month/week, this much of variation can occur without any reason.

~~~
Evbn
That's why multiple data points are used.

~~~
AnthonyMouse
There was only one shutdown. That isn't multiple data points. They could be
measuring the effect of _anything_ that occurred during the same time period
as the shutdown: The recovering economy, which specific films were new
releases during that period of time, the popularity of other goods or
activities that serve as substitutes for movies during that period of time,
the weather, etc.

To get multiple data points you need multiple events.

------
tudorconstantin
Assuming that the 5% increase in sales is real - I wonder if the total costs
invested in lobbying, bribering politicians to create favorable laws and all
the anti piracy pr campaigns are worth it in the end

~~~
waterlesscloud
VERY rough ballparking to follow.

Call it 200,000 digital sales per week total, 350,000 digital rentals total.
That's for 2 studios, so let's multiply by...I dunno, 4 sounds good to account
for the other distributors (this might be way off).

So 800,000 sales, 350,000 rentals. Say the sales net an average of 8 bucks per
sale and the rentals 2 bucks each. I made these numbers up, but they match my
ancient retail experience.

So that's 6.4 million in sales and 700,000 in rentals per week.

Times 52 and that's 333 million in sales, 36 million in rentals. So, call it
370 million total for a year.

So 5% is 18.5 million. Be generous since we're ballparking and call it $20
million.

Hmm.

Digital alone would probably be a close call. But factor in dvd/bluray and
theatrical and it would certainly be worth doing.

EDIT- I didn't do too bad!
[http://latimesblogs.latimes.com/entertainmentnewsbuzz/2011/0...](http://latimesblogs.latimes.com/entertainmentnewsbuzz/2011/02/online-
movie-downloads.html) says in 2010 the digital market for movies was $385
million. But then tv was an additional $366 million, so double that $20
million estimate to $40 million. And it's going to grow fast.

------
nextstep
Even if the result of this study is accurate, the point is irrelevant. Piracy
cannot be stopped, and eventually free methods of consuming content will
become so prevalent that content creators will be forced to adopted entirely
new business models. I don't believe there will ever be a silver bullet that
can block piracy; DRM schemes will only serve to ruin the experience for
paying customers, while free, pirated alternatives will continue to
proliferate and present better and better user experiences.

Perhaps a revenue model based solely on product placement could work, but most
likely entirely new business models that haven't yet been dreamt up will
resurrect entertainment for profit. Perhaps digital sales will serve purely to
advertise performances or other tangible art experiences. Regardless, I
imagine the entertainment industry will look vastly different in 10-15 years.

~~~
anigbrowl
_eventually free methods of consuming content will become so prevalent that
content creators will be forced to adopted entirely new business models_

...or just exit the market and do something else. Musicians can play live;
that doesn't work very well for film directors or actors. Sure, there's the
theater, but but the size of the audience you can play to is limited to the
distance within which actors can be easily seen and heard. You can't really
stage a play in a stadium the way you can stage a music event.

 _Perhaps a revenue model based solely on product placement could work_

Producers and screenwriters already make the most of their opportunities
there. It's possible, up a point; if I'm writing a script based in the 19th
century I can look for some premium brands with a long legacy for placement or
cross-promotion opportunities (you'd be surprised how many there are). But
that won't work for earlier periods, science-fiction, or contexts that brands
typically prefer not to be associated with, like war movies or stories with
gloomy themes.

Entirely new business models are pretty thin on the ground. 'Make something
and charge admission to people who want to see it' has been around for a
while. The latest attempt is the current crop of 3d movies, since that doesn't
really work on the sofa, but it's a big increase in production costs for
relatively little box-office gain.

------
DanBC
> Governments spend resources in order to win court cases against such sites
> and to mount shutdown operations.

Why, if there is no criminal offence, are governments getting involved? Why
aren't the rights holders (who claim to be the ones losing money) suing for
lost earnings?

------
wizawuza
Anyone that read the 26 pages (sorry, I'm lazy), can you comment on whether
they take into account timing of releases? In other words, if the best and
biggest movies come out right after the shutdown, sales could have increased
due simply to that.

~~~
anigbrowl
I can't access the site right now (HN effect??) but it would be surprising if
they didn't, given the widespread understanding of release scheduling within
and without the industry. Whole books are written about release timing, and
the film industry supports two daily trade newspapers, whose economic analysis
is as sophisticated as those in any other industry.

------
HunterV
Just from a completely personal viewpoint, I find that in my extended social
groups it takes more than a few months for "non geeks" to find a new site to
stream/download new illegal content from. It takes more than 18 weeks for the
word to get out about a new site and then trickle down to the normal
population. The study should be done after at least a year, and not with MPAA
funds, duh.

------
y1426i
The conclusion seems to be written before the research.

------
mtgx
Says the research lab with "unlimited" funds from MPAA. Okay.

------
youngerdryas
I wish HNers got half as upset about pay walled research papers as they do
about movies. If your time has no value you can get any film for free, and
that will never be stopped. If you are openly profiting from someone else's IP
you will get smacked down, until the apocalypse comes, which some would
welcome in exchange for free movies.

