For those of us who work in marketing, we know the value of a hunch. That gut-feeling. We use it a lot to brainstorm new campaigns, solve problems, and direct our social media posts. There’s merit to the gut-feeling technique, it’s true. But there’s a downside too – your hunch might not always be right. In reality, it may always be wrong. That’s where A/B testing comes in.
What is A/B Testing?
A/B testing is a way to compare two versions of something (an email, a landing page, an ad, etc.), so that you can use analytics to determine which version is most successful. A/B testing isn’t unique to online marketing, but it is particularly helpful in this field, and in eCommerce especially. Here’s why.
When you’re putting together a website or ad campaign, there are many subtle changes and tweaks to make along the way. You choose what color to make the “Add to Cart” button. You opt for one image over another for your PPC ads. You change the copy on your landing page slightly. These choices might seem minor, but studies have shown again and again that customer behavior is majorly affected by them.
How Can A/B Testing Help?
A/B testing is actually ideal for trying out these kinds of changes. It’s a test for refining your online marketing, not completely overhauling it. Any eCommerce whiz can tell you that even small changes to the sales funnel can make big waves on the other end. Do you want to make those changes based on your hunch of what color looks best? Or your personal opinion on which text flows better? Or would you rather test out both options so you know for a fact that you’ve optimized in every way you can?
The real problem with relying on your gut is that it’s yours. Other people react differently to colors, words, images, even fonts. Your feeling about a certain choice might not be the majority’s reaction to it, and that’s who you should be trying to please! Even minute changes can be the difference between a considerable sale and an abandoned shopping cart, and I know which you’d prefer.
Email Marketing: A/B Testing in Action
Let’s look at an example of how A/B testing can help your email marketing strategy.
For many of us, our email marketing follows a standard schedule: weekly mailers with sales/offers, monthly newsletters, follow-up emails to encourage returning customers, etc. This probably means you’re pretty set in your ways – do your emails always go out at 11 a.m. on a Monday? Do they always have roughly the same subject line? Is the format the same?
You could be missing out on a chance to maximize the sales benefit for each email you send – but you’ll never know unless you use A/B testing to try out some changes and analyze the results.
Landing page test
First, you’ll need to decide which metrics matter the most to you for determining ‘success.’ You might want to track whatever constitutes a conversion for your business, whether it’s a sale or just downloading a free white paper. Click through rates are another good measure of your recipients’ interest, or you could track how many use a discount code included in the email. If you’re trying out different subject lines or timing, your open rate might be all you need to see results. Related Class: Metrics-Driven Demand Gen in a Multichannel World
The next step is splitting your mailing list into two groups – let’s call them A and B, for obvious reasons. If you’d like to compare two new ideas, you could send new, different versions to both groups. Or if you’re just looking to see how your current newsletters stack up against a slightly updated version, you could keep group A as your control group, and send group B the new email.
The key when making changes is to keep it minimal. Choose ONE aspect to change. This is important because at the end of the day, you’re looking for actionable results that can guide you going forward. If you change half the email, you won’t be able to put your finger on the one thing that really matters for increasing sales, conversions, CTR, etc.
A few easy things to test: if you usually email at 4 p.m., try 10 a.m. instead. If you usually use a standard greeting in the subject line, try something more discount/offer oriented. If you usually include a voucher code for a percentage off, try a buy-on-get-one voucher code instead. If you usually offer 15% off for returning customers, try 20% off instead.
The last step is sitting down with a big cup of coffee and all your analytics. Did the changes improve your key metrics? Or make them worse? Did offering a deeper discount pay off overall, or did you shrink your profit for nothing? Read over your results, compare outcomes, and then form steps or recommendations based on your data. It shouldn’t be surprising that continuing your A/B testing over a longer period will help you get more accurate data, less likely to be affected by random highs or lows on a certain day.
Goodbye to the Marketing Hunch
Overall, the clear lesson here is that if you’re not A/B testing changes to your website, sales funnel, or marketing techniques, you’re just relying on your own feedback to make decisions. Gather more feedback through testing, and it’s like a free focus group for your online marketing efforts! If you don’t have the time or personnel to manage A/B testing yourself in-house, there are many tools and services that will do the dirty work for you, for a fee.
Want to learn more about A/B testing for website design changes? See it in action and play along in this class, “10 A/B Test Studies.”