A/B testing is a fast and statistically relevant way to optimize desired end-user actions by testing content variations and recording which content variant results in the largest response rate.
Depending on website traffic, a marketer may be able to optimize click rates in as little as 30 minutes, optimizing lead generation opportunities, increasing content relevance, or simply maximizing funnel conversion stages.
The Act-On A/B web page testing feature is accessible from the Start page under Content > A/B Test Content. Once you click on the link, you will be directed to the A/B Testing Draft tab view.
Managing A/B tests
Overview
The A/B Testing page allows you to manage and create A/B web page tests. The page includes three tab views:
- Drafts tab
- Running Tests tab
- Completed Tests tab
Drafts Tab
The Drafts tab lists tests that are not complete or currently active. The Drafts tab is the administration page default. Individual tests can be selected by moving the cursor over the test line item exposing options to Start Test, Edit, or Delete.
If a draft test has not been fully configured to run as a test, the Start Test option is not available and the test is listed as an Incomplete Test.
Running Tests Tab
The Running Tests tab lists the A/B tests that are currently active. The default view of a running test lists the campaign URL, which can be copied and pasted to link campaign assets, and a summary progress bar toward the end campaign threshold value.
Selecting a specific test with a mouse-over reveals the following options: Dashboard, End Test, Edit, and Delete.
- The Dashboard option displays a statistical view of the test results, including page views, success/click-through rates, and the identification of a test winner.
- The End Test option stops the test from collecting additional A/B test metrics and moves it to the Completed Tests tab.
Completed Test Tabs
The Completed Tests tab lists the tests that have been manually or automatically moved to a completed state. By default, an A/B campaign is automatically moved to a completed state when 100 combined conversions have been recorded. Highlighting a specific test reveals the following options: Dashboard, Edit, and Delete.
The Edit option provides the ability to modify where the campaign URL is redirected once an A/B test is over. The default setting is to direct campaign traffic to the winning alternative web page. This view also provides a time indication as to when the campaign was last viewed.
Creating an A/B Test
Overview
To create an A/B web page test, click the New A/B Test button located at the top-right corner of the A/B Test Content page.
Test Name and Test Description
Each A/B test requires a name to be entered in the Test Name field. Use a unique name for each test you create to keep them organized. An optional test description can be added to provide more information.
Alternatives
At least two web page alternatives must be selected. An alternative is a variant of the web page you want to test. An alternative is either a landing page or web form that has already been created within Act-On; therefore, these pages must be created in advance of running an A/B test.
Alternatives are selected by clicking the Add Form or Add Landing Page buttons; this displays a pop-up menu where you may select from a list of landing pages or forms within your Act-On account.
- Forms – It is not necessary to select a call-to-action or conversion link for forms. By default, the submit action of the form is recorded as that variant’s conversion.
- Landing Pages – If you’ve selected a landing page you must also select a conversion or call-to-action contained within the web page. Examples of calls-to-action include hyperlinks, images, and buttons.
To select a conversion call-to-action:
- Highlight the added alternative row and select the Edit option to open the Conversion Picker.
- The Conversion Picker provides an HTML view of the landing page and highlights the available call-to-action elements within the page with a dotted, red border. Highlighting, or clicking, a specific call-to-action will turn the dotted red line into a solid red line.
- Click the desired conversion element to select it as the call-to-action.
- Once selected, the picker overlay will be closed.
Once an alternative has been selected and a call-to-action has been set, there are two additional options available to manage an alternative.
- Preview – The Preview button displays an HTML version of the selected landing page or form alternative
- Delete – The Delete button permanently deletes the selected alternative from the test
A/B test methods
- Distribute Evenly – The Distribute Evenly test option is a classic A/B split, randomly displaying each variant an equal amount of times over an appropriate timeframe. For example, with a two-variant alternative A/B test 50% of the users will see Alternative A and 50% of users will see Alternative B.
- Optimize for Conversions – The Optimize for Conversions model is a variant of the classic A/B test model. The difference between the two test methods is how traffic is directed to a variant. With this model, more traffic is directed to the best-performing page more frequently than poorer performing alternatives. The model splits traffic in the following manner:
- For 10% of the time, traffic is equally split between the test alternatives. This is referred to as the 'exploration phase'.
- For the remaining 90%, test traffic is directed to the highest conversion page. This is referred to as the 'exploitation phase'.
The Optimize for Conversions model maintains a higher conversion rate over the test period by directing traffic to the highest performer. However, under this method an A/B test should be run for a longer period of time to ensure the test data is statistically valid.
A/B Test Completion
Overview
- Completed Test Threshold – By default, A/B test campaigns are automatically moved to a completed state once 100 conversions are recorded. This value can be increased or decreased by individual campaign to reflect anticipated traffic directed through the campaign URL.
- Handling Legacy Campaign URL Traffic – Once an A/B test has been completed, the legacy campaign URL traffic is redirected by one of three methods:
- Redirect to the winning alternative (the best performing alternative).
- Redirect to another page created within Act-On.
- Redirect to an external URL.
Save Button
To save a newly-configured A/B test click this 
button. Saving a new test sets it to a draft state and lists it under the Draft tab view.
Start Test Button
Clicking this 
button sets the A/B test state to active, generates a campaign URL and lists the test under the Running Tests tab.
Close Button
Clicking this 
button closes the added test form and loads the applicable tab view.
A/B Test Campaign URL
Once an A/B test is created and started, the Act-On platform generates a unique campaign URL. The URL is then used by the marketer to drive web traffic to the A/B test using web pages, emails, banner ads, search engine ads, partners or other online campaigns. The URL is constructed to include information relevant to a specific A/B test so test data can be recorded and reported.
To copy and paste a campaign URL, open the Running Tests tab view, drag the selected campaign URL located under the test name and press Ctrl-C (or Command + C on a Mac) to copy it to the clipboard.
Editing an A/B Test
To edit an A/B test, highlight the test you wish to edit and click the Edit option 
. This will open a pre-filled A/B test web form with the selected A/B test data settings. Make your changes and click the Save button to update the test. The Edit option is visible regardless of which tab view the A/B test is listed in.
Deleting an A/B Test
To delete an A/B test, mouse over the test and click the Delete option. This option will permanently delete the test record including all campaign data. The Delete feature is primarily a clean-up option to discard temporary, unused, or old tests
Ending an A/B Test
In addition to the automatic ending of an A/B test, an A/B test can also be manually stopped or ended.
To manually end an A/B test, open the Running Tests view, mouse over the test you want to end, and then select the End Test option.
Although there may be many reasons to end a test, the primary function is to allow the marketer to proceed with the campaign quickly once a clear winner has been determined. Ending a test early will maximize the exposure and click-through rate for the winning page.
A/B Testing Results Dashboard
Overview
The dashboard displays the most recent results associated with a particular A/B test. The dashboard provides the following data results:
- Total Visitors – The sum of all visitors across all alternatives.
- Total Clicks – The sum of all call-to-action success clicks across all alternatives.
- Test Started – The date and time the A/B test became a running test.
- Progress – A visual indicator of progress toward the completed test threshold.
- Click Threshold – The completed test threshold for the test.
- Views – The number of views recorded for a specific test alternative.
- % of Views – The alternative’s percentage of all views (Views / Total Visitors) x 100.
- Clicks – The number of call-to-action clicks recorded for a specific test alternative.
- CTR – The percentage of clicks to visits for an alternative (Clicks / Views) x 100.
Winner "Green" Icon
The winning alternative, based on the alternative determined to have the highest CTR, is visually identified by a green badge with a white checkmark.
Refresh Button
Clicking this 
button reloads the dashboard and displays any data changes that may have occurred since the last page load.
End Test Button
Clicking this 
button ends the test and moves it to the Completed Tests tab. Once a test is ended, no additional data will be recorded or reported within the dashboard for that test.
Listing Button
Clicking this 
button loads either the Running Tests or Completed Tests list view, depending on which tab the A/B test resides in.
Limitations
The following is a list of limitations:
- Landing pages and forms that are to be tested must be hosted by the Act-On platform and need to be created before an A/B campaign can be run.
- When using the Optimize for Conversions A/B test method, the test should be run for a longer period of time to ensure the test data reaches a point of statistical relevance.
- "IFRAME" embedded forms or other embedded items within a landing page are not available to the call-to-action picker. Only "A HREF" elements within the parent web page can be selected.