

Ideas about A/B Testing Methodology - pathdependent
http://37signals.com/svn/posts/3169-ab-testing-its-not-about-the-results-and-its-definitely-not-about-the-why

======
pathdependent
I'm disappointed that this did not generate a discussion on HN. About two
years ago, I was made accurately aware that methodology is one of the most --
_if not the most_ \-- important factor in conducting research, and that my
methodology was shit. I've spent that time almost obsessed with methodology,
hoping to strengthen this weakness.

This is the first post I've seen about the importance of methodology in A/B
testing. Given that the underlying distributions can be difficult to estimate,
there is a susceptibility to both Type-I and Type-II statistical errors.
Consequently, I think this post was much more important then it seemed when
looking at the lack of discussion.

~~~
ForrestN
I agree. My partner is about to start A/B testing for his small business, and
a robust discussion of these issues would be quite helpful!

