A/B Test Email Campaign

A/B Test Campaigns make it possible for you to test up to six different versions of an email campaign with your recipient list, and then send the winning variation to the remaining contacts. By testing multiple variations of your email campaigns, you can gain insight into which materials your recipients may be more apt to react positively to. This allows you to craft content and messages that resonate with your customers and ultimately lead them to deliver the desired result – conversions!

Setting Up an A/B Test Email Campaign

To begin setting up an A/B test email campaign, head to Campaigns in the Emails section within Marketing, select a drafted campaign from the list, or click the +New button in the upper right-hand corner to create a new campaign.

Choose whether to use a system template, a template you created or to start from scratch.

Enable A/B Testing

Once you are satisfied with the content and design of your email, go to the A/B Testing tab and toggle the switch to enable the feature.

A/B Test Settings

Decide whether to test the subject line or the email content. Testing the subject line focuses on increasing the open rate while testing the content aims to drive clicks or encourage specific actions. 

The subject line is critical as it’s the first thing recipients see and often determines whether the email is opened. Alternatively, testing various elements of the email content, such as section titles, article length, call-to-action buttons, images, or videos, can provide valuable insights into what resonates with your audience, helping you create more effective campaigns in the future.

Test Duration

Next, determine the duration for your A/B test, during which the different versions of your email will be sent to your audience. Consider the typical response time in your industry to emails when setting an appropriate timeframe for the test.

You can set the test duration from 30 minutes to 24 hours.

If no results for click or open rates are achieved within the set duration, the first variation will be sent as the winning variation by default.

Variations

Decide how many versions of your email you would like to test, with a maximum of six variations. 

📌Note: A minimum of 10 contacts are required for each variation. This means that creating four emails to split tests would require 40 contacts, with 10 for each test.

Use the slider to determine the percentage of contacts that will receive the different variations and view the size distribution below. In this example, Variations A and B will be sent equally to half of the contacts (25% each), then the winning variation will be sent to the remaining 50%.

Subject Variations

If you choose to test using subject variations, enter the various subject lines in the provided fields.

Content Variations

If you choose to test using Content Variations, switch between each variation to add different elements to each.

Winning Criteria

Choose the metric you will use to decide which version of the email is the winner. You can pick either "Unique Open Rate" or "Unique Click Rate."

Testing Your A/B Split Test

Once the A/B Test emails are prepared, and you are happy, you may test the email campaign or preview the campaign by clicking on the 3 dots.

Send

To launch your campaign to the selected receipts, please set the delivery method to either 'Send Now' or 'Schedule For Specific Date/time'.

Once the email campaigns are sent out, you may check the enhanced statistics and reports of variations and overall email campaigns.

✍️Important Notes

• A minimum of 10 contacts are needed for each variation. So creating 4 emails to split the test would require 40 contacts, 10 for each test.
•  A/B test campaigns can be only sent via using Send Now or scheduling the campaign. It will not work for batch schedules or RSS schedules.
• During the duration that is set, if there were no results for click or open rate as chosen, by default the first variation will be sent out as the winning variation.