Large sample size over as much types of media possible (game, music, movies, and software). Afterward, I would start digging down further by separating producers, market groups and market targets, and then do statistics to find correlations. I would ask questions like: Do a indy game being released on steam have similar economic characteristics of a indy movie being released on netflix.
Then I would collect data on piracy rate vs initial buy rate vs long tail and find product types that share those. Then, with all that data, one can start looking into DRM and individual products from groups with shared economic traits but which differ primarily on the DRM question. With a decent sample size, some kind of answer should pop out.
That would be my guess of a simplified study. Statisticians and economics has surely more to say on this and where the corner cases are. There might also be smaller studies to do if one accept some early assumptions.
I'm far from a statistician, but the difficulty in the study seems to be that you have products which are radically different from each other in demand structures.
How do you tell whether the sales delta is driven by DRM or a much, much higher quality artist?
If someone is out there on HN with some ideas, I'd love to hear them, not for this problem (I don't really care), but for other problems that may be difficult to solve later on.
> I'm far from a statistician, but the difficulty in the study seems to be that you have products which are radically different from each other in demand structures.
>How do you tell whether the sales delta is driven by DRM or a much, much higher quality artist?
I'm not going to weigh in on DRM vs. non DRM, but you'd get around that problem by explicit randomization. If you have enough observations then differences in quality, etc., literally average out. If you have more information up front you can randomize conditional on the observed characteristics, which reduces the number of observations you need.
They only average out if they're uncorrelated. I'd say it's pretty likely that high-budget, expensive games are more likely to be released with DRM than low-budget indie ones. You could try randomizing on budget, dev size, etc..., but there's really so many random variables to account for in game development (expected sales? platforms released on?) that you'd need a ton of data before you really got something that's statistically significant.
Yeah, I should clarify that I meant you could do it if you control whether they're released under DRM or not. If a group of indie developers wanted to pool together and run an experiment, they could do it in a way that was really informative about its effect on revenues (the conclusions might not hold up if you tried to extrapolate too far beyond indie games though).
I agree that the existing data aren't much like the random experiment I was talking about.
Maybe, but what if indie games are less susceptible to piracy than blockbuster games.
There are really just massive statistical hurdles to overcome with this type of analysis.
You could prove that indie games with strict DRM will perform differently than indie games without strict DRM, but extrapolating that to feature, blockbuster games is problematic.
The same game released with and without DRM? Different games with and without DRM?