

Mixpanel (YCS09) - How Real-Time Data is Changing Business Optimization - suhail
http://mashable.com/2010/07/14/real-time-business-optimization/

======
mortuus
fta: "One area where we are seeing real-time analytics improve content
companies is in article headlines. For most of us, an article headline is all
we use to decide whether or not to read on, so having a good one is definitely
important. The Huffington Post is ahead of the game here. They use analytics
to run A/B split tests on their important articles – in real-time. The
Huffington Post initially shows 2 headlines for the same story, after 5
minutes of testing they discard the less popular one."

How common is this kind of behavior on the web? Are many news outlets doing
this kind of split testing?

Can anyone recommend a 'getting started' with A/B testing guide (for
blogging)?

~~~
physcab
5 minutes seems sort of arbitrary. How do they know they have a representative
set that describes the whole population? What if in the next 5 minutes they
get a rush of users who prefer the other headline? On the other hand, if they
are getting second by second updates and they are looking at the standard
deviation of the entire minute, then maybe it would be easier to tell which
dataset was more statistically significant...no?

~~~
jrich
It all depends on the amt of traffic they get. I'm sure they've experimented
enough to realize that 5 minutes is enough time to get a representative sample
of their general audience.

------
hackerbeegood
This is only natural because the Internet age has and continues to
significantly shorten users' attention span. What's considered new gets
shorter life spans as information travels at a faster rate. It would only make
sense for businesses to take full advantage of real-time analytics (for ad
dollars) before whatever data gets stale fast.

