Test and Conquer: Tips for Smart Experimentation

As a marketer, and often with many stakeholders to a campaign, it may be difficult at times to come to a consensus on one message or style for the creative campaign. But why choose one based on feelings and opinions when you can test multiple creatives and base your decisions on data (you can find out how to track your campaigns here)!

Momentum Media offers several options to test creative across all our platforms which we will describe later in this piece, but first let’s explore what A/B split testing is.

What is A/B split testing?

A/B split testing is a process of comparing two or more versions of your campaign, typically testing one element or variable while maintaining all other variables the same. For example, you may decide to test completely different creatives of the same ad unit, while keeping the landing page the same and, where possible, running both over the same period of time.

What is a multivariate campaign and how does it differ from an A/B split test?

A multivariate campaign refers to a type of testing available within a solus eDM booking. While an A/B split test sends two or more variations to our entire audience split equally between the variations, a multivariate campaign will send the variations to a small portion of the audience (typically around 20 per cent) as a test, then send the winning variation to the rest of the audience at a set time later (typically four-plus hours after the initial send). 

To decide on a winning variation, you can have our platform automatically choose the better version by the highest “click rate” or “open rate”, or alternatively, you can decide to manually revise the results of each and decide for yourself which campaign will go to the rest of the audience.

Platform: Website Advertising

Website advertising is the easiest method to run a split test with, and has many options for running your split test. Some examples are:

• Time-based split: Have your creatives swap out over different periods of time. This method isn’t recommended for testing different campaigns, as you are adding another variable to the test (i.e. time).
• Even rotation: All variations are evenly split over the same period of time, delivering equal number of impressions for each. Typically, you would manually monitor the results of these campaigns regularly and make alterations to the campaign based on the data.
• System optimised: Creatives are rotated evenly for a period of time. As results come in, our platform increases the frequency of those creatives with higher CTRs (click-through rates). 
• Weighted: Creatives are rotated based on a weighting or frequency that you specify.

Platform: Email Advertising 

When we say email advertising, we are referring to banners, towers or natives within our eNewsletters. This section does not cover Solus eDMs, which will be covered in the next section. Unfortunately, this form of advertising is a little limited on testing options due to the limitations within the email body. At this stage, the only means to test a campaign is a time-based split, but this can be split daily, weekly or monthly.

Platform: Solus eDM

As a solus eDM gives you full control over the entire email being sent out, we can split the audience and campaigns far more easily. The two options we offer to test your campaigns are: 

• A/B split test: The audience is split evenly between the variations and each sent at the same time. Insights and decisions can be made for future campaigns based on the results at any time after the send (best to pull the report at least 24 hours after sending to give the audience time to see the eDM before making any decisions).

• Multivariate campaign: Similar to an A/B split test, but the variations are tested on a percentage of the audience (typically 20 per cent), with one variation sent to the rest of the audience based on a higher predetermined metric two to four hours after the test. These campaigns must be advised at the time of booking and incur a higher fee, as the campaign is blocking out a larger part of our schedules and requires further testing.

Summary

Testing your marketing campaign is always a great idea whenever you have the opportunity to, and it will help you consistently learn more about what works best for your brand or product. It’s always important to first look at your testing options and then apply previous test outcomes to future marketing plans.