

How Growth Hackers Get A/B Testing Wrong - hemancuso
http://blog.klaviyo.com/2014/06/02/why-growth-hackers-get-ab-testing-wrong/

======
iamlikeaninja
You have to wonder if its even possible for the same tools that you use to A/B
test small changes (i.e. in UI, menus, branding language) can also be applied
for these more broad systemic changes. What it sounds like a lot of the
software mentioned in this article does is to maximize locally. Is it even
possible to then use the same process to maximize globally? If the answer is
yes, then companies should ask themselves if they're willing or able to make
such large systemic changes on such little data. I wonder how many people
would be able to make such game changing decisions off of the results of these
A/B testing surveys. My guess is probably not a lot.

~~~
edhallen
I think it probably is, it just takes more planning. Take email A/B testing -
if email is a main driver of your usage activity, you could split your users
or list into two groups and then send them different emails over a period of
months. It'd be slightly more manual (you'd have to segment the list) and then
leave one group off for a few months, but it would let you easily gather very
significant data.

------
jacquesm
Off-topic: there isn't a single tech term that has been such a turn-off for me
as the whole 'Growth Hacker' thing.

~~~
normloman
Yes, for the love of god, just call it marketing. All marketing is supposed to
be data driven anyway. We don't need a new buzzword to describe a marketer
behind a computer.

