1..2..3….A…or…B? [Testing]

Today, I would like to introduce you to something known as A/B testing.

What is A/B Testing?

A/B testing is comparing two versions of a webpage to see which one performs better.  The functionality of the two webpages is measured by any sort of metric, for example, the conversion rate.  In this case, the webpage that correlates with a higher conversion rate between option “A” and option “B” is the optimal winning format.

Now that you know what A/B testing is, let’s take a look at a published journal article titled, A Practical Guide to Controlled Experiments on the Web: Listen to Your Customers not the HiPPO.

HiPPO: Highest Paid Persons Opinion.

This article summed up in three words:  Data trumps intuition.  Forget the HiPPO’s when you can provide statistical evidence. Controlled experiments are valuable and embody the best experimental design for establishing a causal relationship.  This brings me back to A/B testing.  A/B testing is a means of testing a randomly selected controlled group against an experimental group to determine whether there is a statistically significant difference between the two.  Based on observations collected, overall evaluation criterion (OEC) is derived for each variant.  It’s important to remember, A/B testing provides the “what”; however, it does not necessarily explain the “why”.

Once again, A/B testing is cool! If you do not believe me, I would like to introduce you to a young, successful women who goes by the name of Amelia Showalter. She can teach you a few things about A/B testing and why it is so damn useful.

First off, Amelia is kind of a freak, in a good, complementary sense.  She has had many successes in her life.  She was featured in Elle Magazine, and more notably was the digital analytics director of the 2012 Obama campaign.  Showalter used a series of extensive A/B testing throughout the elections to formulate the perfect subject lines, how much money they should ask for, to the content and formatting of the email themselves.  Needless to say she’s a pro.  For example, in this case specifically, they found a casual tone was usually most effective.  I found it especially interesting how they found the uglier they made the emails look, the more successful they ended up proving to be.  For example, ugly links versus “donate” buttons and highlighting specific sections of the letter that they wanted people to read.  At one point they began thinking of ways to they could create the least attractive emails possible.  Also little subtle details like profanity here and there, proved to be effective.

Important to note:  There is no perfect email.  Always retest, because novelty does wear off.

The next article by techpresident goes more in depth about the 2012 presidential election. The strategy was to take an evidence-based approach to person-to-person contact.  If there’s anything you can take away from this post (besides A/B testing), it’s that having real conversations with people by going door-to-door still matters.  Functionally, not much has really changed when it comes to campaigns, just the number of available tools to help analyze data has multiplied.

Anyways, after the election, Amelia eventually went on to found her own digital and quantitative consulting agency. Amelia, you are a true boss!

All of this A/B testing and importance of direct clientele relationships, sparked my interest in todays digital analytic industry.  Who else is using A/B testing?

This prompted me to stumble across an agency known as L2 Interactive. L2 Interactive, specializes in many services, one of which is what they call “split testing” aka, A/B testing.  Basically, they provide the right tools to collect and gather data from your site. In turn, allowing them to apply the results to improve your website.  Pretty straightforward, but definitely necessary and an easy way to help determine how to improve your sight, backed by real data.