What’s this A/B testing all about?

First off, A/B testing is “a simple way to test changes to your page against the current design and determine which one produce positive results.” For example, I may want to know what color font people prefer on my webpage. In order to do this I must show visitors two versions of the same page and let them determine which page they like better. One versions could be normal black colored font (Control: A test), and the other maybe a purple colored font (Variation: B test). Whichever site ends up reaching the ultimate goal the company wants, being statistically significant, will be the “winner”, the site they choose to use. Just changing the color of something or the size isn’t the only thing that A/B testing can do. There are many different variations of webpages that companies can check. Do we add a picture? Do we change the phrasing of the sentence? Is there enough information? Is there too much information? Every single aspect of the webpage is being tested. A/B testing can test these questions to tell you what on your page is working and what is not. In the article, Practical Guide to Controlled Experiments on the Web: Listen to Your Customers not to the HiPPO, Doctor FootCare compared two webpages with multiple changes (Figure 1). When tested, most of the upgrades were positive but they lost 90% of their revenue because adding the coupon code was too critical. People started to think twice about whether they were paying too much for the product because of the discount codes that were out there. Removing this code but sticking with everything else in version B gave an increased conversion rate.
Figure 1:
Another example of how A/B testing works was during the Obama reelection campaign. Amelia Showalter and her team sent out emails and experimented with different versions of a single message to see what got the most clicks. Some subject lines were: “If you believe in what we’re doing…”, “Would love to meet you”, “Change” and the winner: “I will be outspent”. They did A/B testing on subject lines, amount of money they would ask for, and even what was in the message itself, including the formatting. I think it was good of them to experiment on all these different aspects. However, I think it is odd that they would try testing things that had cuss words, ugly things, and even links they wouldn’t even click on. Having all of this information and data on the people was good for the campaign because there was probably a lot of people working on just this one campaign. In a normal business there is probably not as much data or employees able to put all their time into one project. Another thing the campaign did was bombard their customers by sending many emails, but this is not good for a normal business because their will want to unsubscribe right away. No one likes to get their inbox flooded everyday… unless it’s a campaign. A good note I took from the results of the The Obama Campaign’s Legacy: Listen, Experiment, and Analyze Everything article was that “people don’t like when politicians target an identity that they don’t really hold”. This is good information to know about who you are targeting in the market because if a question is biased towards any type of person/group than that someone generally shows a strong negative or positive affiliation. Another business that does this well is Adore Me, they A/B tested the world’s hottest women to see who makes you click “buy”. For each bra, they shoot multiple versions of images (different models, same position, different position, etc.) for the website. Adore Me subjects all their images to testing every month. They have found that although gentlemen have said to prefer natural blondes, nobody seems to want to buy lingerie from them. The model matters more than the price and the pose of the model is important too. Hermand-Waiche says the beauty of A/B testing is it doesn’t cancel out an ethnicity and a genre; its all about emotion.
The article MailChimp: 7 Years of Successful Experiments explains their A/B testing tool that has been a huge success to their companies because they know how good having this testing available is for companies. It’s good for their customers because they can see what pages are working best for them and improve their macro and micro conversions, the hits on their site and overall user experience all by testing which parts of their sites work the best.
All around I really think that A/B testing can help companies. It is good to track what is working on your website. In the world we live in, everyone is online. You need to be tracking everything you can to improve the user experience. If you don’t, another company will.
Thanks for reading, katielomas

What’s this A/B testing all about?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s