This is the strangest warning against A/B tests I've heard. "Be careful because you might end up producing something that is #1 internet retailer". The Amazon home page might not be as sexy as the latest whizz-bang startup, but success doesn't lie.
Thank you. I can't count the times I've tried to explain this to people. The purpose of amazon's site is not that visitors think it's pretty, or even easy to use. It's to get you to buy as much as possible. And there's convincing evidence that it's quite successful in fulfilling that purpose.
Why do supermarkets put the most frequently sold items at the back, when it would be so much easier for customers if they were right by the entrance?