A/B testing is useful stuff. Everyone agrees on it, but most people don't get the opportunity to do it. Sometimes there's engineering concerns where the coders are working on a big feature push, but the product person wants to do some more testing.

Here's a post from Dan Siroker, the Director of Analytics for Obama's very successful internet campaign, explaining the mechanics and value of A/B testing. In this case, that value is $60 million dollars, plus the added human value from the nearly 300k volunteers who were gained from effective use of A/B testing. Read the post for the how to, but here's three key points.

1. Every visitor to your website is an opportunity. Take advantage of that opportunity through website optimization and A/B testing.
2. Question assumptions. Everyone on the campaign loved the videos. All the videos ended up doing worse than all the images. We would have never known had we not questioned our assumptions.
3. Experiment early and often. We ran this experiment in December of 2007 and reaped the benefits for the rest of the campaign. Because this first experiment proved to be so effective we continued to run dozens of experiments across the entire website throughout the campaign.