Hacker News new | comments | show | ask | jobs | submit login

I'm with you on A/B testing being hilariously inefficient (and have founded a startup to do better: http://mynaweb.com) but I disagree you can't test creative content. There are two things:

1. Delivering content in movie or TV episode sized chunks is an artefact of inefficient distribution mechanisms. Smaller units of content are easier to test, easier to produce, and less risk if they tank.

2. Any script contains numerous decisions. You can test the main ones. It's viable to do this if you aren't making big bets on big chunks of contents.

I'm not sure if content chunking is an artifact of inefficient distribution mechanisms. It would be impossible to tell the Illiad through tweets, for example, for anything other than an intellectual exercise. But to be less radical, consider that there don't really exist ads longer than 1 minute.

Content dictates length, not the other way around...

The Iliad consists of many scenes. Each scene could probably be a five-ten minute piece. If you read the Iliad it will probably take eight or more hours. Movie's rarely exceed three hours and any movie based on the Iliad would be abridged (director's cut of Troy is just over three hours). It's not clear to me that content dictates length. It seems medium does to a large extent.

Well, to be specific, a lot of Hollywood producers looked at H+ as a test of whether or not people really wanted to watch short episodic content. After the first episode, few people came back. So while we can say "probably" in abstract, in practice short episodic series are hit or miss, for reasons that have nothing to do with length, etc. There's no formula, I guess is my point.

I agree, and we find the same thing in A/B testing. There is stuff that has worked historically for other people's products and their customers. It might work for you and your customers but it might not. You have to test it. The idea with A/B testing is to follow what actually works for your product and your customers, not what you think will work. The idea of producing smaller chunks (reducing batch size in lean startup speak) is to make testing cheap enough to be viable.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact