

Denial, anger, bargaining, depression, & acceptance: The A/B testing lifecycle. - bjpless
http://benplesser.com/2012/09/20/denial-anger-bargaining-depression-and-acceptance-the-ab-testing-lifecycle/

======
btilly
Getting emotionally invested in your tests is a good recipe for an emotional
rollercoaster. Don't do that.

Instead try a bunch of things, and see what sticks. Be curious, not invested.

~~~
suneel0101
That's a great point, echoing the part of this post about not being results-
oriented. Process-driven is the way to go!

------
thenomad
This is a great piece. I'm doing a lot of testing for advertising right now,
and the emotional rollercoaster - no matter how sanely you attempt to approach
it - can be pretty intense. It's good to see someone writing about that side
of the process!

~~~
bjpless
thanks thenomad. There's a funny disconnect between the rational, detached
self and the insane subconscious self present in most people.

------
bjpless
What is the most frustrating A/B test that you've run?

~~~
forgot_password
Off-topic but what do you guys use to run A/B tests?

~~~
bjpless
We're currently using Google Content Experiments. The Pros are that it's
incredibly easy to get a test started thanks to the presence of Google
Analytics scripts in the rest of the website. The major Con is the lack of
true multivariate testing.

We experimented with Optimizely but had a bad experience. It seemed as though
their javascript was synchronous and blocking. We may have been able to
customize that but did not dig deeper.

~~~
scoremotive
Another flaw in Google Content Experiments (which was mentioned in the
article) is that it doesn't distribute traffic evenly. This is a big bummer
with seasonal products/services where having the control start getting 80% of
the traffic during a high conversion period makes the test useless. Overall,
even though testing is easier in some ways, I definitely miss Google Website
Optimizer.

