A/B or split testing Follow

Running an A/B test on your email or SMS is a great way to determine what works best in getting you the best results from your campaign. You can use A/B testing to test many aspects of your campaign. Some of the common items tested are:

  • Subject lines
  • Pre-header text
  • Sender details (from name and sender email address)
  • Personalisation (using wildcards)
  • Content placement in the email (placement of call to action button, such as 'Register')

This article will give you the run down on how to run an A/B test - so let's get started!

Step One: Create your emails

In this first step, you will want to create two emails with varying content. As an example, you may want to test whether including someone's first name in the subject line leads to a higher open rate. So you will create your first email in the system which has no name in the subject line. Once this email is created, you can follow these steps to easily create a second copy and adjust the subject line:

  1. Go to 'Messages'.
  2. Select the first message.
  3. Choose 'Copy' from the Actions menu.

  4. Give the message a new name (e.g. Message name, version 2) and click Copy.

  5. Select the newly created message from the list.
  6. Choose 'Edit Details' from the Actions menu.

  7. Edit the subject line to insert the first name of the recipient by clicking the pencil icon to the right of the subject field and choosing the 'first name' field in your recipient list. Click 'Insert Wildcard' button.

  8. Save the changes.

You should now have two copies of the message; one with a personalised subject line, and the other without. Now we can move onto step two.

Step Two: Separate your list for testing

The easiest way to separate your list for testing is to use our spreadsheet view feature. Here's the steps you will need to follow:

  1. Go to 'Contacts'.
  2. Choose the recipient list from the drop down menu's on the left side of the page.
  3. When the correct list is selected, choose the 'Spreadsheet' icon from the top right corner of the contacts table. 

  4. Click the 'Add Field' option at the top left of your spreadsheet.

  5. Choose a 'Drop down' field and click the 'Add' button to add the field.
  6. You can now insert a field name (e.g. AB Test Field).
  7. For the drop down values, you will want two different values. These can be as simple as 'Group A' and 'Group B'. Once the values have been inserted, save the new field.

  8. The new field should now appear in your spreadsheet. Select the first cell in the column of the AB test field, and double click to access the values. Select a value for this contact (e.g. 'Group A). Press 'Enter' when complete to update your selection into the spreadsheet.

  9. There should be a small square in the bottom right of the selected field which now says 'Group A'. Click and hold this little square and drag down the column. This will apply the value 'Group A' to the contacts you select below as well. So - depending on how big your contact list is, you might want to assign 50 contacts to Group A.

  10. You will then repeat this process, but assign the next 50 contacts to 'Group B'.

That's it! The rest of your contacts who don't have anything assigned you can send whichever message yields the best result from your A/B test. So, let's move onto step three, conducting the actual test.

Step Three: Sending your A/B test and viewing the results

To send your A/B test, we will do an advanced search for the contacts with the correct values. Here are the steps to do the sends:

  1. Go to 'Messages'.
  2. Select the message which will go to Group A.
  3. Choose 'Send' from the Actions menu.
  4. Choose the correct list from the drop down menu's, and then select 'Search for contacts (Recommended for large lists)'. Click 'Next'.

  5. Now click on the grey 'Advanced Search' button.
  6. Search for contacts who have 'Group A' in the AB test field.

  7. Click the 'Send message' button.
  8. Repeat steps 1-7 for the message to go to Group B. At step 6, you will choose 'Group B' instead of 'Group A'. 

Once the send is completed, you can view the results by going to the 'Reports' page and viewing both message sends.

Something to keep in mind is that it may take a few days for the opens to reach completion, so you should conduct your A/B test a few days ahead of when you want to send out your email to the rest of your database.

Once you have obtained the results, you can then use steps 1-7 above to search for and send to contacts who 'Does not include' 'Group A' or 'Group B' in the AB test field (so you will only be sending to contacts who haven't received a copy of the message already).

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request