The authors are from a prestigious school, CMU. Still, I was curious to know who funded this research. Alas, not surprisingly, it is MPAA.
The CMU lab is called Initiative for Digital Entertainment Analytics (IDEA), and according to CMU's press release "The creation of IDEA was made possible through a gift from the Motion Picture Association of America (MPAA)"
I actually read this paper without knowing it was sponsored by the Motion Picture Association of America and without realising that it wasn't peer reviewed. I cringed on the thought that this is the level of social science.
In my profession (law), proving loss of revenue is so difficult that it is in many cases considered impossible, even in civil cases. You have to take into consideration many factors and you can't just automatically ascribe declining or increasing revenues to one fact, even if it seems plausible. You can't just make up a model full of assumptions and attach fancy names and mathematical symbols to it. No decent judge would fall for that. To start with you would have to make a year-on-year comparison spanning 5-10 years. Then you would have to make a convincing argument why the revenues didn't just decline or increase due to one of the many other possible factors such as popularity of the movies, internet penetration, broadband access, weather, macro economic trends, changes in taste, normal seasonal changes, supply of alternatives (cable tv shows, sport games etc.), changes in payment options for the legal movie outlets, other changes in business model or marketing that could explain why the legal movie outlets sold more after the shutdown.
If the product had been more generic that might have been possible but with movies the popularity of the supply varies so much that changes could often be ascribed to differences in popularity.
Not disagreeing, but whenever there is a study posted where the funding comes from a source that has an obvious interest in the outcome of a study there is always a comment like this one pointing out the conflict.
I understand providing information beyond that would be likely very difficult, but having never been in a similar research situation, could someone outline what this influence is like? If there an underlying pretense during the research that they have to reach a specific outcome? Is it more subtle/obvious? Any additional insight would be appreciated.
> Edit: Just found out that SSRN is not even a peer reviewed journal!
It's not really a journal at all. It is a repository for papers in the social sciences, kind of like arXiv is for some other fields.
Most of their target audience is well aware of this, but it might seem confusing if you're coming from outside academia. They're not trying to pretend they're something they're not. Putting pre-prints of papers on SSRN is considered good practice.
SSRN is actually a pretty important source for open access research.
Just found out that SSRN is not even a peer reviewed journal!
Nobody claimed it was. It's a platform for sharing preprints/working papers, like arxiv. Obviously you don't like this study or its conclusions much, but throwing the kitchen sink at it doesn't strengthen your argument either.
It is meant to be a scientific study. What's the use of an unsubstantiated claim? In fact, its been proven to be highly counterproductive (at least in other fields) as others go on to build works and further claims around it, only to later discover the errors and depth of uncertainties.
Using the same principle, I could posit (probably quite defensibly), that the amount of alleged copyright infringment of Megaupload Limited's limited access and centralized services would be dwarfed by that from Bittorrent and other P2P methods.
You need more rigorous inductive reasoning, and randomized trials
This is usually impossible in social science research studying real-world phenomena that can't be reproduced in the laboratory. It's a cargo cult attitude to think all science must look like an FDA drug application. Part of science is making do with the information you can collect.
I think you have it backwards, economics is for the most part a cargo cult science because it pretends it's OK not to test hypothesis. Many people thought Phrenology was also scientific and they measured things and made predictions. However, if you apply the rules of Feynman's lecture they both fail.
Self-censoring is in my view an important, but largely ignored phenomena that should be more researched.
There is some evidence that people stay away from research that is considered controversial by the public . In my own experience (biologist), a lot of biologists shun away from public debates on more controversial stuff like GMs or creatonism out of fear of being associated with the fringe-elements participating, or are scared of "loosing" their reputation.
There was a Nature survey in which 15% of 3247 scientists confessed of having changed their study-design, methodology or findings because of pressure from a funding agency .
More indirectly, this study checked out studies on a fat substitute from Procter & Gamble and found that "supportive authors were significantly more likely than critical or neutral authors to have financial relationships with P&G"... 
These are just glimpses, from my own experience I would say that the problem is larger than it's currently being described.
Edit: There is little to no research on what's happening directly with the scientists in a situation in which funding influences findings. Does the researcher self-censor out of fear of loosing funding? Is there a slightly threatening phone-call from the funding agency? No-one really knows!
In my experience, a lot of biologists avoid debates on creationism because it is not a scientific theory. Getting into a debate as a scientist would suggest there is something to discuss scientifically, there is not. Richard Dawkins voiced this publicly many times.
Furthermore, and more importantly, avoiding public debate is very different from changing the results of your research. If you are implying that some people have results that support creationism and keep it to themselves, I would like to hear more about this. I cannot even imagine how you would conduct scientific research on creationism.
Another point: you are using the word "confess" which has negative connotations. It is perfectly normal for a funding agency to voice their concerns on, e.g., the design of an experiment, if they can think of an improvement. There is nothing wrong with this, there is nothing to "confess". In some cases the agency might be altering the design to game the results. That is wrong, of course; but your phrasing is too general.
If noone really knows, than maybe nothing substantial is happening. "slightly threatening phone-call from the funding agency" sounds near paranoid. And again, your wording is way too general. For example, it would include NASA (as the funding agency) ask a laboratory that is designing a detector to make modifications to allow higher resolution in higher energies.
Actually, it's pretty well known how it can happen.
It's fairly easy for people to rationalize not publishing a null result not favorable to their funding source when null results are often not published anyway.  However, in aggregate, this means that studies that show unfavorable results to a funder are less likely to see publication.
Imagine you're testing the efficacy of a new drug, which has a close-to-neutral or neutral effect (to keep things simple), and there are n labs testing it. You might expect there to be a normal distribution of effect around a mean of zero (whatever quantity we're measuring). However, the groups who didn't find any effect decided to not publish their null results for any minor technicality or due to some pressure like (e.g. are you sure you don't want to repeat this experiment?)
Left as an exercise: if you observe only at what got published, what do you expect to see?
That being said, there's obviously more than just your garden variety publication bias and conflict of interest at play here.
If you expect or are looking for specific results you are likely to unconsciously manipulate the experiment in order to find them. And when big corporations are involved uncounscious manipulation is often the smaller problem..
For me, it was "statistically significant" that lowered their credibility: Oh yeah? under what p-value?
The data is hard enough to analyse, we don't need misleading statistics on top of that. (To clarify my point: "statistical significance" is indicative of frequentist statistics. While the changes in the frequency properties of sales prior to and after the shutdown are relevant information, they are probably not the best way to use available information.)
Personally, I don't pirate movies/music, but as a CMU (SCS + Heinz) alum, the lack of rigor and potential conflicts of interest displayed in this paper are disheartening. More than that, they cheapen the value of my degrees.
One of my big pet peeves in data analysis is making major inferences and broad, sweeping statements based on a very small sample size. This paper appears to be a prime offender.
Looking at this case, there's a pretty good argument that Netflix and the LoveFilm app launching in the UK, QuickFlix Streaming launching in New Zealand, The Artist (with a French director and star actor) getting a lot of attention and being released to rental, a particularly chilly winter in western Europe, and god knows what else all contributed in whole or part to the observed phenomenon. I'd love to have whatever data the authors of the paper had, to facet the data by movie studio and country - did different countries peak on different weeks, and the like. Alas, the data consists of the sales and rentals of two anonymous major motion picture studios.
That none of these potentially seriously confounding factors were mentioned or explored before reaching for a conclusion is somewhere on the scale between negligence and dishonesty, and the lack of data transparency renders the the paper useful for nothing more than kindling.
Any fellow Tartans here on HN (I know there are a bunch of us) want to sign on to a letter requesting an investigation into the integrity of the paper and IDEA?
"Personally, I don't pirate movies/music, but as a CMU (SCS + Heinz) alum, the lack of rigor and potential conflicts of interest displayed in this paper are disheartening. More than that, they cheapen the value of my degrees."
It's probably also worth writing a letter stating exactly that to your school, your department, and the department involved, as well as a letter to your alum magazine.
Does this study account for the fact that 2011 was the worst year for movies (in terms of quality) ever? Look at the academy award winners and the top box office performers of that year and compare it to 2012.
Side note: I would like to see a study on the quality of movies in 2011 vs 2008 based on economies 3 years previous. So when movies were getting funding in 2005 for release in 2008, we got some real gems. In 2008, when the economy took a down-turn, we see some bad ones in 2011.
Exactly this. The 'study' hasn't controlled for anything. There precisely no discussion of the weather (the summer months are suspiciously absent from the data set, nor the weather patterns commented on - Sydney, Australia had one of the wettest summers on record over 2011/2012), the economies of the countries involved, let alone a year over year data comparison.
The only thing "gone in 60 seconds", is the credibility of the authors of this study.
I apply a higher degree of skepticism any time the results of a study funded by an organization with a viewpoint support exactly the conclusion you'd expect for that viewpoint; if nothing else, I'd expect a high degree of selection bias, if not a predetermined conclusion in mind during methodology selection and data collection.
I'd strongly question the results of a study funded by The Pirate Bay that said the ready availability of downloadable films increased movie sales.
I don't think it would work very well if it were the people who agreed with a study to somehow also have the obligation to point out its flaws. Could we believe that they were being thorough in finding problems if the conclusions supported their own beliefs? It works better, I think, if the challenger takes an adversarial, but not actually hostile, position. And, of course, for both sides to have open minds, but I realize that is often too much to ask.
Well, that actually happens a lot with scientific studies where people are interested in the result but want to point out the limitations of the study so that it doesn't get over-hyped and lost in a backlash. I much prefer a qualified conclusion.
The presence of amazing startups finally punching through the incredible awkwardness and extreme hassle that is consuming content legally is certainly the driving force behind increased sales.
I've never understood why people think piracy is some form of theft. It makes more sense to think about it as a form of advertising/marketing. You see, media companies NEVER have sales. The closest you'll get is watching 20 minutes of ads per hour on cable TV while footing a pretty intense bill in addition.
For over 100 years, it's been sound business sense to give things away to increase market penetration and drive increased sales. Sales bring buyers who also tend to make other purchases, which leads them to advertising to their friends. Giving things away to people who broadcast the products to a large audience is advertising 101. Loss-leading is a modern strategy that is working very well for expensive electronics. It works by essentially giving people technology so that they can consume more easily, expecting them to compensate you for the difference over time.
This is all identical to how I have actually seen piracy affect the media industry. It's literally just marketing. People who pirate stuff typically aren't people who would buy it anyway. I really do not see a negative impact whatsoever. There's so much evidence to the contrary that I find someone making a claim that "piracy made more difficult, content sales up" is anything other than a context-free juxtaposition of facts absolutely inane. Piracy is easier today than it ever has been. For < $200 in hardware you can have your own personal Netflix but with all the latest movies and TV shows. Yet sales continue to rise, and the failed and hassle-filled pre-internet business models are falling away, leading to a minimal-cost sampled environment where the need for something like piracy is growing smaller all the time.
Then there are some industries that are devolving and paying for it severely. Remember demo disks when we were all kids? I could, every week, get a demo disk with all the latest games on it. Maybe I got to play 10% of the game, but I didn't have to go pay $60 to see if I liked it. Now that digital purchases are non-resellable, fewer and fewer companies offer demos of their games to people so they can try before they buy. Piracy is useful here, and still tends to be an advertisement rather than a hindrance. And these companies want us to buy into always-on, in-your-face, and anti-consumer DRM? They're making it harder to consume legitimately! That's the exactly wrong thing to do! I haven't pirated a game in a long, long while. But I haven't bought any either (except a few indie games and the humble bundles) because the industry seems to be incapable of making any good business decisions.
So yes. It really is so far-fetched that closing down a "major" (what, <5% of all piracy?) piracy site would increase legitimate downloads by even 1 sale.
It is. There are so many alternative sources of pirated content, that thinking that someone that normally downloads movies would simply switch to buying it if just one of these sources shuts down it's just naive.
It seems far fetched to me, because the shutdown did little to limit the access to pirated material. I imagine the next stop for someone who found himself unable to use Megaupload was another similar site, not a legitimate download site.
The study posted in favor of MU (or at least neutral effect) got pretty much no comments at all. Might have been because it was a post linking a 3rd source (secondary source being Torrentfreak and primary the study on SSRN).
No, but is it reality to think that people that would steal begin paying? Most pirated content is content that would have otherwise not been paid for in the first place.
How much did it affect the promotion of movies though if people weren't talking and looking for it more? I actually think it ends up benefitting in the end or close to even. Exhibit A: The success of Windows Exhibit B: The success of Photoshop. Both gained a foothold via piracy initially. Piracy gets them hooked then later they buy or they buy it with budgets at work.
Is it worth recovering 1-2% for losing 98% of a user base? Does that eventually lead to more sales? It is a tough problem.
Movies have less network effect than software, and most is not all. Why does MPAA need a bunch of people watching movies but not paying? These viewers aren't tastemakers, mass market advertising is taste maker.
People watching more movies gets them interested in actors, series, artists, directors etc, this later does lead to sales and is directly a network effect. It is advertising and content is being consumed, if the content is good it has a lasting impact. Same effect it has on software in some cases.
These are also people that have the time to pirate and were never going to buy right now. Later when they have no time and more money they will buy products, they weren't buying if they were taking the time to pirate it. So is it better for an artist or piece of content to be seen/heard or not? $10-$20 for a movie is cheap compared to the time it takes to get it at a certain point and is more convenient. Compared to software prices, music and movies have it locked up in the convenience factor if prices are low and content is actually available. Users will choose convenience and have on those types.
Piracy will be won via convenience, not against the current. Yes it can never be legal or the whole system breaks but it will never go away, use it. Piracy is a 10x+ modifier on user base (freemium is a response to this).
>These are also people that have the time to pirate and were never going to buy right now. Later when they have no time and more money they will buy products, they weren't buying if they were taking the time to pirate it.
That was me with games. When I was a teenager with no money, it didn't bother me to copy games because there was no lost sale - I couldn't afford to buy no matter how badly I wanted the game. Now that I'm older I have a lot of money (relative to the cost of a game, anyway) and I don't have the time or inclination to pirate anything.
This is most certainly not a biased funded study with links to the MPAA guys. Without-a-doubt it is definitely an independent study performed by real people with no links to the entertainment industry. Now if you'll excuse me guys I'm just going to go to the cinema to watch a $30 per ticket blockbuster movie because the entertainment industry needs our support. I'll be back later.
>I would assume they'd try to be fairly objective with this study, because they'd be very happy to discover that piracy is a good thing for them.
You assume their goal is to stop piracy rather than to promote "anti-piracy" legislation which just so happens to harm their competitors in markets related to content production (like content distribution and consumer devices) or otherwise attempts to make things more like the bad old days when reaching a large audience required the backing of a major studio.
I mean why do you think they keep defending the DMCA prohibition on circumventing DRM? It's obviously nothing to do with piracy because the pirates all just comprehensively ignore it. But what it does is give them a veto over consumer technology. If you have to interact with their DRM then they get to veto your device if they don't like it. Which they don't hesitate to do.
Well the RIAA still firmly believes that litigation is what slowed down music piracy. However in reality it was probably the advent of easy to use and cheap (or even free) music streaming services that really hurt music piracy.
On favourite tactic of the movie houses is to report on decline in US domestic sales (and they are, or at least were last year), but report on piracy globally. They don't report on global sales, because they're raking it in.
Half-true. Foreign distributors get a large cut of that on a hit, but lots of films don't play to a foreign audience while still suffering from the decline in domestic sales. Said decline hits theaters and the people who work there, who are not subsidized by a film's overseas profits.
It says that they analysed 12 countries. Is it possible that they selected these countries specifically, and rejected others that may have refuted their hypothesis?
I'm trying to say this with as little judgement as possible. I just know that deciding to look only at certain countries to support a hypothesis has been done before (c.f. Ancel Keys's Seven Countries Study).
I am concerned that the MPAA, in commissioning the study, provided the researchers with data from two studios who saw a big lift in sales in the period following the shutdown.
I've not read the paper so I've no idea if this is an issue or not, but only using the data of two studios seems to be a low sample size.
Imagine if the week after the shutdown "Lord of the Rings" was released, or "Twilight", or any other blockbuster. Find two studios with big releases in the period after the shutdown and it looks like some significant causation.
Is it completely unreasonable to also suggest that they would simply manipulate the data? I know, it's a strong claim without evidence. However, I'm under the impression that sales figures are routinely manipulated for other reasons, and it definitely wouldn't be beyond these companies morally-speaking to do so here.
"Our sales data are provided by two major U.S. movie studios and include all digital purchases and rentals through their major digital channels aggregated at a weekly level from September 2, 2011 until May 31, 2012. These data include each of the 12 countries where these digital channels were available as of September 2011."
Quote from page 8 of the paper (First paragraph of section 4 if you care to look it up in context).
My concerns are that (1) the study was funded by the same individuals who provide the data showing immediate conflict of interest and (2) the data does not span a 12 month spread. So even though they claim to make consideration for seasonal attributes like Christmas sales we don't have a full data set to know if the rise/fall pattern they display is typical for this timeframe. The release of "summer blockbusters" or other studio phenomenon cannot be compared to the data unless we have data spanning multiple years to know when peak seasons of sales are.
Heck, a third issue is their lack of physical sale data. So although they have source showing digital piracy to digital sale correlation is likely it doesn't account for digital adoption of newcomers to the market.
I think one can assume that the funding comes from the IDEA from this sentence:
>This research was conducted
as part of Carnegie Mellon University’s Initiative for Digital Entertainment Analytics (IDEA).
The IDEA is apparently a think-tank to help "groups like MPAA and policymakers [to] navigate the changing digital media landscape."
Interestingly, funding for the IDEA has been made possible by "an unrestricted gift from the Motion Picture Association of America." 
Edit: From one of the author's blog it looks like the study has been submitted for peer review: "The study is currently under peer review at a good econ journal."  It seems that the SSRN where this paper is currently hosted is to social sciences and economics as to what arxiv is for natural sciences: a repository for yet unreviewed drafts of papers.
Our [Megaupload] Internet penetration data is based ... Google AdWords estimates of the total number of unique visitors ... [and] ... the total number of Internet users in each country in our sample...
As such, the shutdown of Megaupload influenced the policies of several other cyberlockers focused on piracy, and our results necessarily measure the “net impact” of the Megaupload shutdown across the cyberlocker industry, as opposed to just measuring the impact of Megaupload.
"Aggregating these increases, our analysis across 12 countries suggests that, in the 18 weeks following the shutdown, digital revenues for these two studio’s movies were 6-10% higher than they would have been if not for the shutdown. "
How can you compare with something that is nonexistent? If they had a control, a timeline with piracy, then this comparison would be valid right? I mean, the effects of 6-10% higher could be just the hottest titles that came out in the summer such as The Avengers, etc...
You can compare year-on-year because studios spread their releases to maximize revenue. I'm having problems accessing the site to DL the paper right now but I have a hunch you're arguing with the adstract, not the methods.
I read the article and have a few questions to the authors:
How was the megaupload estimate calculated?
Various parts of the article use per week data to measure sales / rental trends, however the megaupload estimate for each country was held constant, given there's a likely correlation (with a potential time lag - i.e people buy stuff on the advice of people that have already seen it and recommended), how would this variation effect the modelling?
What sort of measurement error does the google adwords estimate have? How does this effect the model?
Are there any factors that might impact the adwords estimate on a per country basis (such as prevelance of use of particular browsers, adblocking plugins, etc.)? How does this effect the model?
Do you have year on year data to compare this data with?
Do you have any data to show correlation between sales and MPR over time?
Does your data differentiate on movies that were available on megaupload before it closed vs those that were released afterwards? How does this effect your model?
So, using their extremely scientific method, did anyone else experience increased site traffic, or sales? I'm sure we can link other industries who deliver digital goods(software) so the closing of a digital locker site. Correlation === Causation; the pinnacle of statistical research!
One of the authors of this 'paper' is Brett Danaher. He is a member of the SERCI (The Society for Economic Research on Copyright Issues) which directly gets funding from SGAE (Sociedad General de Autores y Editores) which is a large collecting society (copyright collective) for songwriters, composers and music publishers in Spain.
There was only one shutdown. That isn't multiple data points. They could be measuring the effect of anything that occurred during the same time period as the shutdown: The recovering economy, which specific films were new releases during that period of time, the popularity of other goods or activities that serve as substitutes for movies during that period of time, the weather, etc.
To get multiple data points you need multiple events.
Assuming that the 5% increase in sales is real - I wonder if the total costs invested in lobbying, bribering politicians to create favorable laws and all the anti piracy pr campaigns are worth it in the end
Call it 200,000 digital sales per week total, 350,000 digital rentals total. That's for 2 studios, so let's multiply by...I dunno, 4 sounds good to account for the other distributors (this might be way off).
So 800,000 sales, 350,000 rentals. Say the sales net an average of 8 bucks per sale and the rentals 2 bucks each. I made these numbers up, but they match my ancient retail experience.
So that's 6.4 million in sales and 700,000 in rentals per week.
Times 52 and that's 333 million in sales, 36 million in rentals. So, call it 370 million total for a year.
So 5% is 18.5 million. Be generous since we're ballparking and call it $20 million.
Digital alone would probably be a close call. But factor in dvd/bluray and theatrical and it would certainly be worth doing.
Even if the result of this study is accurate, the point is irrelevant. Piracy cannot be stopped, and eventually free methods of consuming content will become so prevalent that content creators will be forced to adopted entirely new business models. I don't believe there will ever be a silver bullet that can block piracy; DRM schemes will only serve to ruin the experience for paying customers, while free, pirated alternatives will continue to proliferate and present better and better user experiences.
Perhaps a revenue model based solely on product placement could work, but most likely entirely new business models that haven't yet been dreamt up will resurrect entertainment for profit. Perhaps digital sales will serve purely to advertise performances or other tangible art experiences. Regardless, I imagine the entertainment industry will look vastly different in 10-15 years.
eventually free methods of consuming content will become so prevalent that content creators will be forced to adopted entirely new business models
...or just exit the market and do something else. Musicians can play live; that doesn't work very well for film directors or actors. Sure, there's the theater, but but the size of the audience you can play to is limited to the distance within which actors can be easily seen and heard. You can't really stage a play in a stadium the way you can stage a music event.
Perhaps a revenue model based solely on product placement could work
Producers and screenwriters already make the most of their opportunities there. It's possible, up a point; if I'm writing a script based in the 19th century I can look for some premium brands with a long legacy for placement or cross-promotion opportunities (you'd be surprised how many there are). But that won't work for earlier periods, science-fiction, or contexts that brands typically prefer not to be associated with, like war movies or stories with gloomy themes.
Entirely new business models are pretty thin on the ground. 'Make something and charge admission to people who want to see it' has been around for a while. The latest attempt is the current crop of 3d movies, since that doesn't really work on the sofa, but it's a big increase in production costs for relatively little box-office gain.
Anyone that read the 26 pages (sorry, I'm lazy), can you comment on whether they take into account timing of releases? In other words, if the best and biggest movies come out right after the shutdown, sales could have increased due simply to that.
I can't access the site right now (HN effect??) but it would be surprising if they didn't, given the widespread understanding of release scheduling within and without the industry. Whole books are written about release timing, and the film industry supports two daily trade newspapers, whose economic analysis is as sophisticated as those in any other industry.
Just from a completely personal viewpoint, I find that in my extended social groups it takes more than a few months for "non geeks" to find a new site to stream/download new illegal content from. It takes more than 18 weeks for the word to get out about a new site and then trickle down to the normal population. The study should be done after at least a year, and not with MPAA funds, duh.
I wish HNers got half as upset about pay walled research papers as they do about movies. If your time has no value you can get any film for free, and that will never be stopped. If you are openly profiting from someone else's IP you will get smacked down, until the apocalypse comes, which some would welcome in exchange for free movies.