What You Need To Know About A/B Testing Email Campaigns

What You Need To Know About A/B Testing Email Campaigns

You’ve spent hours crafting the perfect message for an email you’re about to send to your customers, but just before you click send, ask yourself this: is this piece of work what’s going to get you the number of opens and conversions that you want? Of course, there’s no sure way to find out except to hear from your recipients directly. But how can we go about doing that? Definitely it won’t be possible to reach out to every single customer and directly ask if they liked it, but what we can do is hear from them through A/B testing.

What is A/B Testing

A/B testing is the process of comparing two or more versions of a single marketing asset to see which one performs best. In this case, we’ll be looking at your email campaigns and how the different elements might fare against one another.

Each version would fundamentally be the same thing, with slight differences depending on what you would like to test. For example, you want to see which version of your subject line generates more open-rates than the other, making it somewhat like a competition between the two variations.

Knowing which marketing assets work better for your audience can help your brand make more informed decisions in the future, and better cater to the likes of your audience. With the audience group being special and unique to each brand, it would be good to know which set of elements work best for them, and ultimately work best for you. Even the smallest of changes (like using ‘we’ instead of ‘you’) could result in significant differences in your ROIs.

Some Jargon You Should Know

🏆 Champion

This is your marketing asset that has performed well in the past. Thinking about it as a physical fight, this is the contestant that has emerged as champion the previous round. It doesn’t only work for emails as well and can be applied to other mediums, be it a web page, blog copy, Facebook advertisement etc.

💪🏼Challenger

The champion will then be pit against a challenger, a variation of the champion with an element that’s different. This is the element that you’ll be testing.

📈 Variant

After the test, you’ll see which version or variant worked better than the other. The one that performs best then becomes the new champion. The testing cycle then continues as you go on to further optimize your marketing assets.

Things to Test

The first step is to know what you want to test, and the elements that you’ll be changing in order to find out.

Realistically speaking, you could test virtually anything you want to. From the wording of your subject line, down to the colour of the links in your email. If it’s something that you can change, it’s something that you can test. But of course, this doesn’t mean that you should test everything! As much as it would be nice to uncover the ultimate combination of elements that drive maximum conversions, it’s going to take way too much time.

What you want to do is to focus on the elements that have a bigger impact. Here are some of our recommendations:

1. Subject line

This is the text that your customers will see first, making it crucial for your open-rates and conversions.

Some A/B tests you might see include:

Generic vs specific

You could test to see if a generic subject line gets more attention and love than a longer subject line that contains more details about the content of your email.

Gain-framing vs loss-framing

In this test example, some people might be more motivated to open your emails when they get the impression that they stand to gain from it. However, some others might be risk-seeing when presented with a message that indicates a loss of some sort. This ultimately depends on your audience, but the nature of the service or product you are trying to sell as well!

Adding personalisation

This can be done to see if greeting your readers by their first name could result in a better performing email. Of course, it depends on the relationship between your brand and the recipient of the email as well.

Here’s an example of weekly roundups that I receive in my mailbox from Tech In Asia where they address me by name. The personal tone of the message makes it seem apt as the writer briefly mentions their “3-year-old niece” as though I were their friend. Figure out if this suits the look and feel of your brand, or if something less intimate would work better.

Types of promotions that work best on your customers

See what kind of promotions capture the attention of your customers, be it an offer of “Free Delivery” or a short and sweet “20% Off”.

2. Copies or product descriptions

With diminishing attention spans, it’s natural to assume that short-form copies would work best! However, this may not be the case for all businesses, where long-form copies could generate better quality leads as compared to their shorter counterpart. This could also be the case for more formal and serious brands. The only way to find out which works best is to test!

Here are two examples we grabbed from the local online restaurant Grain, and the ways in which they made use of a succinct copy for one of their collaterals and a lengthier one for the other, catering to the different messages and objectives.

With this, you can see that you aren’t limited to just one style all the way as well, and can experiment depending on the occasion.

3. Call to action

Besides the text used, things that can be considered can include where the call-to-action is located, the colour of the call-to-action box, and the surrounding space given to the call-to-action. Does it stand on its own, or is it wrapped along with the rest of your content?

4. Visuals

You want to make sure that the visuals you use, be it images or videos, resonate with your audience and help build a sense of identification for them to want to engage with your brand further! Test your images!

For example, would you be more likely to buy this?

Or this?

Images from IKEA

Test to see what works best! Especially if your brand has a specific aesthetic associated with it. Otherwise, you’d truly be leaving money on the table.

5. Layout

Many different elements come together to make the layout of your emailer

Things that you would want to consider would be the number of columns that your emailer has. Having multiple columns might spice things up a little as compared to a single string of words, however, the goal here is to reduce the friction for your readers, and to keep them coming back for more. These work together with the other elements of your email campaigns as well, such as whether it’s text-heavy or chock-full of images. You would want to avoid clutter in a way that makes your reader want to stay on.

A/B testing limits your test down to specific elements, allowing you to pinpoint exactly what works and what doesn’t. When you figure out which elements are bringing in the conversions, it’s a potential sale and a win for your brand.

You’re already creating content, so let’s not let it go to waste.


Written By

Yukie Miyazaki

With a keen interest in visual communication, Yukie believes in weaving strategies and narratives together to create special experiences for people. She spends most of her time producing social media content with Clickr, and she hopes to be able to dip her toes into other creative mediums in the future. Outside of work, she also enjoys illustrating and animating to bring bizarre ideas to life.