The A/B test condition in Act-On's Automated Programs allows you to see which variations of your messages perform best against a random and equally-distributed set of program contacts. After a specified test interval, the winning message (determined by the highest clickthrough rate) is then sent to future contacts entering that stage of the program.

The ability to select unlimited messages during the test allows you to test simple variables across two messages or design larger multivariate tests.

Instructions

Create an A/B test

You can create A/B tests with the Send Message program step in both new and existing automated programs. To do this:

1. Go to Automation > Automated Programs
2. Create a new automated program
   OR
   Hover over one of your existing programs and click Edit
3. In the Automated Journey Builder, go to Program Flow
4. Click the plus icon to add a Send Message step
5. Click on the new Send Message step to edit it
6. In the Send Message editor, select the email message you'd like to use
7. Click Add Conditional or A/B Test Message
   
   
8. Select A/B Test
9. Specify the number of clicks you'd like the program to count before determining the winning message
   • The test will run until your program receives the specified number of unique clicks across all test messages. The message with the highest clickthrough rate will automatically be determined the winner and will be sent to any contacts who enter that program step in the future. You can also manually select a winner once the test begins.
   • The default value and recommended minimum sample size is 100 clicks.
10. Select an email message for Test 1
11. Click Add A/B Message to add additional messages for your A/B test – you may select as many messages as you'd like
    
12. Click Submit to save your step changes, then Save your automated program

Once you've created your A/B test program step and saved your program, view your A/B test's status and results in the Automated Programs Dashboard.

Run your A/B test

When you're ready, start your automated program to begin the A/B test you've set up. To do this:

1. Click Start Program in the Automated Programs Dashboard
   OR
   Hover over your program in the Automated Programs listing page and click Start
2. Your program's A/B Test step will send each message in your test to program contacts in an equal and random distribution. For example, if there are two messages in the step, each message will be sent approximately 50% of the time. 

   Heads up! If you're using a very small sample set, the randomizer may result in a slightly different distribution between all messages.

3. In your Dashboard, click on the A/B Test step to expand its description and see more information on its status. The description will show the test in progress, how many clickthroughs are needed to complete the test, and the current test counts for each message. 

Select a message winner

Your A/B test can select a winning message in one of two ways:

• System-defined winner – Once your test has reached the specified number of unique clickthroughs, it will end automatically. In the future, only the message with the highest clickthrough rate will send to each contact entering the program step. Your Automated Programs Dashboard will refresh to show that your A/B test has finished and confirm that the winning message has been selected for use in your Send Message program step. A green checkmark will display to the right of the winning message. If the clickthrough results are the same for each test message, the first message will be the default winner. 
• User-defined winner – You may also manually end the test at any time and choose any message to be the winner. To do this, click Mark as Winner next to the message you'd like to officially use for your Send Message step.
  
  

Pause or edit an A/B test

If you'd like to pause your test, you'll need to pause your overall automated program. To do this, go to the Automated Programs Dashboard and click Program is Running > Pause Program. To resume your program and test, click Start Program. Your test counts will stay intact.

You may edit the A/B Test step at any time by clicking Edit in the top-right corner of the Automated Programs Dashboard. When editing, keep in mind:

• Running programs must first be stopped before making changes.
• Messages can be added or removed from the test by editing the step in Program Flow. Modify message content in Program Messages. 
• If you replace any of the test messages with new messages, your test results for those messages will be reset.

Any messages you've added in Conditional mode for a particular step will be preserved when switching to A/B Test mode and vice versa.

Restart a finished A/B test

You can restart a completed A/B test at any time. Click on the Send Message step with an A/B test configured, then click Restart Test. 

• This will clear the previous test counts and start a new test. Although test counts will clear, you can still view the message’s total count history from Program Messages in the Automated Programs Dashboard.
• If you wish to modify, add, or remove messages before restarting your test, you'll need to pause your automated program first.