Test multiple email variations and automatically launch the best-performing one with A/B email testing. For example, you can test:
- Two different subject lines
- Two different calls to action
- Combinations of these (for example, 2 subject lines × 2 calls to action = 4 variations to test)
You can add up to five email messages to a single A/B email test.
Set Up an A/B Test
- Go to Outbound > A/B Test Messages and click New A/B Test.
- Enter a Test Name and Description.
- Under Distribution List(s) > Send To, choose the lists or segments to target. If needed, add Suppress Against segments to exclude contacts.
- If you have a connected CRM, select Attach sent notes to contacts or leads from my CRM lists.
- Under Messages To Test, choose the emails to test. You can select both regular and legacy emails.
- Under Test Run, choose your Sample Size:
- # of contact records (minimum 10)
- % of your total list (33% or less)
- Pick a scheduled date and time for the initial A/B test.
- Under Final Run, choose when the winning message should be sent (immediately after the test, at a specific time, or not scheduled).
- Click Save, then Schedule Test to begin testing.
Final Run
Act-On selects the winning message based on the highest number of clicks (not opens). If there is a tie, the message with the higher click-through rate is chosen.
The winning message is sent to the remaining contacts who did not receive a test email. Contacts who received test versions appear in the Not Sent/Suppressed section of the final email report with the reason listed as Suppression List.
Manage Your Tests
Important: You must cancel an active A/B test to make edits to the test run time or the messages included in the test.
Drafts
Saving a new A/B email test creates a draft. Access drafts from Messages > A/B Testing > Drafts.
Winning Messages
After all test runs are complete, you can manually select a winning message and override Act-On’s choice. Open the test, hover over the message you want, and select the check box that appears. Be sure to save the test for the change to take effect.
Distribution List Issues
If your test includes a large number of suppressions, the final run may send to fewer contacts than the test messages.
To reduce this risk, follow good list maintenance practices, remove hard bounces and opt-outs, or refine your segments. If needed, choose a smaller test sample size.
Active Tests
The Active section shows the status of running tests and provides access to each test’s dashboard.
Messages sent as part of a test run also appear under Outbound > Triggered Messages > Other.
Canceling a Test
When canceling an active A/B email test:
- If the test has not run, it returns to Drafts and can be edited and scheduled again.
- If the test has run, canceling stops the final run and moves the test to Completed. This action cannot be undone.
- To remove only the final run schedule, edit the test from the dashboard, set Final Run to Not Scheduled, and save.
Completed Tests
Completed tests (including those canceled after the test run) appear in the Completed section. Open a test to view its report dashboard.
Final-run messages also appear under Messages > Sent. Contacts who received test messages appear as suppressed with the reason Suppression List.
FAQ
Why did email A win when email B appears to have more activity?
Activity shown in reports includes interactions that occurred after the test deadline. If email A had more clicks at the moment the test ended, it is selected as the winner, even if email B received more clicks later.