A/B testing, also known as split testing, is a tool that site owners and business owners can use in order to manage their sales pages and landing pages and to optimize them to make the most possible sales. Theoretically, this is a ‘fool proof’ system that should turn even the least effective sales strategy into a honed system.
What is A/B Testing?
A/B testing is a multivariate test in web analytics that involves two variants to determine which version of a web page yields better results. Two versions of a web page are randomly assigned to web visitors, statistical analysis is performed, and the results of that analysis determines which variation of a web page is better.
A/B testing (also known as bucket testing ) refers to a process that involves comparing different versions of a web page in order to find out which version is performing or converting better. Though the name “A/B” might suggest that only two pages at a time can be compared, in reality, you can compare as many pages as you would like. The web pages are compared by showing two different variants (let’s say A and B) to visitors that have similar qualities and habits at the same time. The test results of A/B testing will reveal the page that produces the better results and converts visitors more successfully, and that page is considered the better option.
Running an A/B test that compares a direct variation of a page against one that is currently being used allows you to learn what time of changes, if any, you should make to your webpage in order to improve its effectiveness. By using AB testing, the guesswork that surrounds conversion optimization and website optimization can be eliminated. It allows you to make decisions that are based on data rather than guesses or projections. Through measuring the impact that changes have on metrics, you can be sure that the changes you make will yield positive results.
How Do A/B Testing Tools Work?
The process of conducting an A/B test is pretty straightforward. You start by taking a specific webpage and making changes to it to create a second (or third, or fourth, or however many you wish) version of it. The changes you make can be as simple as altering the headline or moving a button, or it can be as complex as a complete overhaul of the design of the page.
As visitors are shown either the control or variation of the page, their engagement with each one is measured and collected through an analytics dashboard. That information is then analyzed via a statistical engine and you can see if changing the page provide to provide a more positive experience, if it created a negative experience, or if there was no effect on user behavior at all.
How to Perform A/B Testing
So, the next question becomes: how do you perform A/B testing?
One option is to simply create two different versions of your landing page yourself and then to create a third page that will redirect visitors to one of those two sites. The redirect should be random and not based on geography or anything else like that. Ideally it will simply take it in turns. This will create a random sample and avoid sample bias (you don’t want all the early risers or people in one country being sent to one site for instance).
If you don’t have the technical skills to do this, then you can find plugins and tools that will handle the process for you. These include sites like Optimizely which will then allow you to see a calculation of the significance as well – making everything simple and easy for you.
The other option, regardless of which method you use to employ it, is to send only a very small portion of your audience to the two versions of your site. For instance, you have your main site and then you have your A/B test off to ‘the side’ where you will send a portion of your visitors.
What can be Tested?
Virtually any element on your website that has an effect on the behavior of your visitors can be A/B tested. Some examples of elements that can be tested include:
- Headlines
- Sub headlines
- Testimonials
- Text used in a call to action
- Buttons used for call to action
- Text used in paragraphs
- Links
- Social proof
- Images
- Mentions in the media
- Badges
A/B Testing Examples
Let’s say for instance that you are selling an ebook that promises to help you lose weight and get into shape. The way that you would normally go about doing this is to create a ‘sales page’ (also sometimes called a ‘landing page’) and fill that page with lots of text and images attempting to convince the reader that this is the book they have been looking for and that they need.
These are the long narrow pages that tend to say things like:
“Hey, are you sick and tired of looking overweight and feeling low on energy? I was like you once. I once believed that it was normal to feel that way as you get older. Heck, so many of my friends felt that way…”
You know the ones.
These pages are meticulously designed and crafted to be as highly effective as possible and as highly ‘optimized’ for sales. The aim is to increase the statistic known as the conversion rate. This is the number of people who open the page and then go on to buy.
Something like 0.1% is a low conversion rate, whereas 10% would be considered an incredibly high conversion rate. Seeing as you generally pay for clicks by using PPC advertising, you want to try and make the most of each of those clicks as possible by driving up your conversion rates. This will increase your profits.
Other tricks to increase your conversion rate include using a narrow layout (this encourages people to keep scrolling down), using lots of facts and figures along with social proof and testimonials, and using color tricks – such as the color red which is known to encourage impulsive action. It’s common to see lots of very short sentences, lots of bold text and underlined text and lots of headings.
The good news is that once you’ve designed your sales page, you aren’t ‘stuck’ with what you’ve built. The web works in such a way that you can continuously update and tweak the things you’ve put out.
So, if you build your landing page and find that it’s not converting any of your visitors into paying customers, then you can make changes in order to try and drive up that number.
Normally, you might do this by simply following your ‘gut’. Perhaps a bolder title would work? Maybe you need to increase the size of the font? Maybe you need to be more ‘in your face’ with the marketing talk. Or then again, maybe it’s too in your face and you need to be more subtle so as not to put off the more cynical visitors you get.
And herein lies the problem: there’s no way to be sure what the problem is or what the best thing to fix that problem would be.
Which is where A/B testing comes in!
With split testing then, you instead take a more systematic approach to the problem and create two different versions of your site and upload these automatically.
So, you have one page where your ‘Buy Now’ button is red and one page where your ‘Buy Now’ button is blue.
This will then allow you to directly compare both pages and see which one does best out of the two. From there, you can then choose to adopt the one that brings in the most sales and you can forget the one that is less effective.
This is important because we can otherwise make the mistake of drawing conclusions. As they say in science, correlation does not mean causality. In other words, just because two things happen at the same time, that doesn’t necessarily mean that one is caused by the other.
Just because you changed your buy button to red and that increased the number of clicks, that doesn’t mean that red buttons drive more clicks! It could simply mean that you changed it on a Saturday and people are more likely to buy on a Saturday. Likewise, it might simply mean that by sheer random chance, you had more clicks on that day.
By doing a split test and seeing the two different versions performing side-by-side, you can rule out this possibility and you can be absolutely certain that it was this change that ultimately led to the improvement in performance.
The Process of A/B Split Testing
The process of conducting an A/B test includes the following:
- Gather data. Analytics can provide great insight into where you can start optimizing. Starting with high traffic areas on your site will allow for much quicker data collection. Pages to consider include those that have high traffic, yet have low conversion rates and/or high drop-off rates that you want to improve.
- Determine conversion goals. Conversion goals are the metric that you will be using to find out if the variation of a web page is more successful than the original version of the page. Conversion goals can vary and can include anything from users clicking on a button or a link, to signing up for an email list, or to making a purchase.
- Develop a hypothesis. After you’ve established a goal, you can start generating ideas for your A/B testing, as well as hypotheses regarding why you think variations will perform better than the existing version of a web page. Once you’ve created a list of ideas, start prioritizing them in terms of the impact you expect they will make and how difficult the changes will be to implement.
- Develop variations. Using whichever A/B testing tools you have chosen, make the changes that you desire to a web page. These changes could include changing the color and location of a button, making navigation elements more or less visible, or changing the arrangement of elements on the page. It could also be something that is completely customized. Most reputable A/B testing tools will offer a visual editor which makes it easier to make the changes. After you make your changes, conduct quality assurance to ensure that the changes work as you expect them to.
- Run the experiment. Get your experiment up and running. Site visitors will be randomly assigned either the control or the variation page. The interaction of users with each experience will be measured, counted, and compared to see which one performs better.
- Assess the results. After the experiment is complete, analyze the results. The A/B testing software you are using will show the data from the experiment, as well as the differences between how the versions of your web page performed, allowing you to see if there is a difference that would be considered statistically significant.
Whatever the outcome of your experiment may be, make sure that you use your experience as a way to improve your web page and improve the experience of your users.
A Word of Caution – Sample Size, Statistics & Significance
The very cynical among you may have already spotted a potential problem with this plan however. That is that there is still the possibility that any change will simply be caused by chance and not by anything you did.
Because if you have two separate websites and one is performing better than the other, that might just mean that the random selection of people that you sent to the other site happened to include more people who would buy from you!
So how do you avoid this mistake?
The answer is that you need to make sure that the data you are collecting is statistically significant. This is another term used in scientific studies and it’s very important that you take it into account.
Because if you have 100 visitors, 50 on each site, and 10 people buy from site number one versus none for site two… that could very well be chance.
However, if you have 1,000,000 visitors, 500,000 on each site, and you have 10,000 sales on one site and 53 on another… well then the odds of that being a coincidence are now dwindled to almost zero.
This number will never be zero. However, the longer you run the test, the lower you can make the number and the more certain you can be that you should adopt the change.
Testers also need to look out for other confounding variables. These are other factors that can unnecessarily complicate the results. For instance, if you have more success with your site with the red button and that is proven to be statistically significant, there is still an outside chance that there is a ‘confounding variable’ you haven’t thought of. For instance, maybe red is in fashion right now and this will only hold true until that fashion changes?
Or perhaps the red simply looks better with the design of button you’ve chosen. Maybe blue would be better if you changed the font.
You can never be 100% sure that you have removed all confounding variables. But what you can do is to keep all elements as precisely the same as possible in order to at least reducing the likelihood of these playing a role.
As you can see, A/B testing is not perfect and you always need to consider how you are using it and if it is indeed the right tool for the job at any given time.
Google A/B Testing and Search Engine Optimization
Major search engines, including Google, not only allow A/B testing, but they encourage it. In fact, search engines have stated that conducting A/B test or other types of multivariate tests will not do any damage to the ranking of your website. However, with that said, it is possible to put your page ranking at risk if you use A/B testing tools too frequently, especially if you are using them for the purpose of cloaking.
In order to avoid damaging your page rank, make sure to follow these best practices, which are presented by Google:
- Don’t cloak. Cloaking refers to the practice of showing search engines content that is different than an average visitor to a website would see. If you conduct cloaking, your site can end up being demoted or even completely removed from search engine results.
- Only run experiments for as long as they are necessary. Running A/B tests for a longer period of time than is actually necessary might be viewed as an attempt to trick search engines. This is particularly true if you are showing one variation of your web page to a large amount of users. According to Google, you should update your site and remove all test variations as soon as your A/B test is completed. Furthermore, you should not run tests longer than you need to.
- Opt for 302 redirects, not 301s. If you running a test that redirects the original URL of a web page to a variation URL, make sure you use a 302 redirect instead of a 301. A 302 redirect is temporary, while a 301 is permanent. Using the 302 redirect will let search engines know that a redirect it only temporary and that they should retain the original URL for the web page indeed instead of the test URL.
By following these best practices offered by Google, you can ensure that conducting A/B tests won’t have a negative impact on the ranking of your site.
The Benefits of A/B Testing on Website / Landing Page
Through A/B testing, you can clearly see which variation of a web page customers interacted with more and had a better experience with. With that information, you can improve your strategy to create a more appealing web page that your visitors will enjoy interacting with, which can improve your conversions, your page ranking, and the overall success of your business.
In addition to these benefits, here’s a look at some of the other notable benefits that A/B testing can provide:
- Better engagement with content. The information gathered from A/B testing can help you improve the structure of your content. When your content is better designed and structured, customer engagement with that content will improve.
- Lower bounce rates. High bounce rates are a real bummer. Through A/B testing, you can significantly reduce your bounce rate and help keep your visitors on your site long enough so that you can provide them with valuable information, which can ultimately lead to more conversions.
- Better conversion rates. A/B testing is the simplest and most effective way to transform a web page and turn it into a site that gets more conversions. Through test results, you can see which elements your visitors interact with more and make the necessary changes to your web page in order to encourage more engagement with the page, which will lead to more conversions.
- Better content. Content is one of the most important – if not the most important – element of a website. Through A/B testing, you can create better content for your site, which can improve the experience of your users and improve the overall ranking and success of your site.
These are just some of the benefits that A/B testing provides. In short, this type of testing provides valuable information that you can use to improve your website, and thus improve your page ranking and your overall success.
Why A/B Testing Works
Through A/B testing, you can see which content on your website leads visitors to:
- Engage more with your site
- Spend more time on your site
- Click through your site to additional pages
- Take certain actions, such as signing up for mailing lists and making purchases
In other words, A/B testing allows you to see which version of a web page leads your targeted audience to take the actions that you want them to take. For example, if your goal is to have more people sign up for your mailing list, through A/B testing, you can try out different designs or content for your call to action and find out which one leads to the most amount of signups.
Other Uses for A/B Testing
There are many more potential uses for A/B testing. For instance, as we’ve already discussed, this is a technique that is used in science all the time. For instance, in medicine if you were trying to test the effectiveness of a new drug, then you would normally split your participants into two groups. Group A will be given the medication and group B will be given a placebo. The results will then be tested for significance, in order to try and ‘prove’ beyond reasonable doubt that the cause for the observed effect was the new drug.
In any scenario, an A/B test allows the user to see which changes have positive effects and which don’t. That way, they can only adopt any new changes once they are 100% certain that they should be effective, allowing the product, service or sales page to gradually develop and get better and better over time.
In this way, the product, site, service or sales page can ‘evolve’ in order to become the most optimized version of itself – with little risk of failure.
Bottom Line
A/B testing is a valuable tool for web analytics. It can drastically improve the success of a website and lead to more conversions, and ultimately help you reach the goals that you have set out to achieve.
