Can I do an A/B test for a campaign?

Do you want to run an A/B-test to compare two versions of your campaign, and find out which has a higher conversion? Laposta does not (yet) have a standard functionality for this, but you can arrange it yourself in your Laposta account. To do so, create extra fields in your subscriber list, use segments, and send the best-converting campaign to the largest section in your list.

What is an A/B-test?

With an A/B-test, you compare the results of your campaign's version A and version B for a small portion of your subscriber list, to send the best results to the remain part of that list. You can for example sample 20% of your list, to find out which subject line results in the highest open ratio, and send the 'winning' version to the remaining 80% of your subscriber list.

You will find that this is only useful and reliable if you have a large subscriber list of at least 10.000 subscribers. Of course, you can just run the test to see if you find a large discrepancy in the results. Though the delivery time can cause a variation in the open ratio, you can very well set up an A/B-test to see if your subscriber list responds more if the subject line is different, or the if call-to-action (link to a relevant action that your subscriber should take in your newsletter) is worded or designed differently.

Doing an A/B-test using segments

You can set up an A/B-test yourself by working with segments. In this example, we are using test groups A and B. Both contain 50% of the subscriber list. After the test, the remaining 50% will receive the newsletters with the best result.

1. The subscriber list

You start with your subscriber list, to select a representative part of the list for your test, and to mark them as A or B.

Export your subscriber list to Excel, and open the file in Excel (or open this Excel file if you want to import a new list for this test).
Add a 'version' column, and add 'A' or 'B' for a relevant percentage of subscribers. Leave this field empty for the remaining subscribers.
Tip: make sure at least 10% of your subscribers has an 'A' in this field, and and identical number has 'B'.
Save your (altered) subscriber list and import it to your Laposta account. During the import, create a new multiple choice field called 'version', where multiple answers are possible. If necessary, check out: how do I import a large subscriber file?.

2. Create segments in your list for the A/B-test

Next, you will divide the entire list in segments to test with later.

Go to Subscribers > Edit settings (for the relevant list) > Segment.
Click 'Create a new segment', and select 'subscriber data'.
Select the 'Version' field, and immediately indicate that it should be equal to A. Save this segment as 'Version A'.
[screenshot]
Create a segment for B in the same way.

3. Two campaigns for testing

You have a campaign for which you want to test a certain variation. Now, you will duplicate the campaign, and edit it for the test. In this example, we assume an A/B-test for trying out a different design.

Create a campaign. In Step 1, at the Campaign name (for internal use), indicate that this is version A. In Step 2, only select the 'Version A' segment as a recipient.
If you are satisfied with the campaign, do not send it yet. Instead, head back to the general campaigns page.
Copy version A, and make it into version B (in the Campaign name), for instance by changing the design. Select only the 'Version B' segment in Step 2.
If you are satisfied with both campaigns, determine the moment of delivery in Step 6 (Schedule delivery), and add both campaigns to the planner. Ideally, choose the exact same date and time, so you only test what you want to.

4. And the winner is...

Wait for the test results for a representative time period. Give your subscribers in both segments at least 2 hours time to open the newsletters. Subsequently create a segment to judge which tested as the winning version.

Go to Subscribers > Edit settings (for the relevant subscriber list) > Segment.
Click 'Create a new segment', and select 'Campaign results'.
Select subscribers who have: B.6 - opened a certain campaign.
Next, select Version A for this campaign.
Save this segment as 'Opened A'.
Follow these same steps to create the segment that Version B opened.
Now look at the page Segment to see which segment is larger. If the group for 'Opened A' is larger than 'Opened B', then you know that version A has been opened more than version B. This way, you determine the 'winner' of your A/B-test.

The best version for everybody else

Now that you know which version yields a better result, you will send that version to the remaining subscribers in your list.

Create a new segment in your list where you put subscribers who have not yet received version A or B.
1. For this, go to Subscribers > Edit settings (for the relevant list) > Segment.
  Click 'Create a new segment', and choose 'Subscriber data'.
2. Select the 'Version' field, and then select the 'Empty' option. Save this segment as 'Remaining subscribers'.
Go to the Campaigns page and copy the 'winning version'. In Step 2, select only the 'Remaining subscribers' segment as recipient. You can immediately send this newsletter in Step 6.