P.S. If you're using Rails, download A/Bingo. If you're not, email me and I will write you design documents sufficient to create an A/B testing framework in your language of choice.
It should not take godhood to do A/B tests. I want A/B tests to be like for loops: not only will you know them and use them automatically every single time, you will laugh with your buddies about that one time somebody applied to your company without ever having written an A/B test before.
Unfortunately that project doesn't look that active, but it's the best out there now.