What is A/B Testing?
A/B Testing is a statistical hypothesis testing where it determines whether a new design bring improvement to a Web page, and it is also called "split testing". As the name recommends, this is essentially a randomized investigation with two parameters A and B.
This testing is also done to estimate population parameters based on sample statistics.
A/B testing (also known as split testing or bucket testing) is a method of comparing two versions of a web page or app against each other to determine which one performs better.
AB testing is essentially an experiment where two or more variants of a page are shown to users at random, and statistical analysis is used to determine which variation performs better for a given conversion goal.
A/B Testing Process
The following is an A/B testing framework you can use to start running tests:
Identify Goals: Your conversion goals are the metrics that you are using to determine whether or not the variation is more successful than the original version. Goals can be anything from clicking a button or link to product purchases and e-mail signups.
Generate Hypothesis: Once you've identified a goal you can begin generating A/B testing ideas and hypotheses for why you think they will be better than the current version. Once you have a list of ideas, prioritize them in terms of expected impact and difficulty of implementation.
Collect Data:Your analytics will often provide insight into where you can begin optimizing. It helps to begin with high traffic areas of your site or app, as that will allow you to gather data faster. Look for pages with low conversion rates or high drop-off rates that can be improved
Create Variations: Using your A/B testing software (like Optimizely), make the desired changes to an element of your website or mobile app experience. This might be changing the color of a button, swapping the order of elements on the page, hiding navigation elements, or something entirely custom. Many leading A/B testing tools have a visual editor that will make these changes easy. Make sure to QA your experiment to make sure it works as expected.
Run Experiment: Kick off your experiment and wait for visitors to participate! At this point, visitors to your site or app will be randomly assigned to either the control or variation of your experience. Their interaction with each experience is measured, counted, and compared to determine how each performs.
A/B Testing & SEO
Google permits and encourages A/B testing and has stated that performing an A/B or multivariate test poses no inherent risk to your website’s search rank. However, it is possible to jeopardize your search rank by abusing an A/B testing tool for purposes such as cloaking. Google has articulated some best practices to ensure that this doesn’t happen:
No Cloaking - Cloaking is the practice of showing search engines different content than a typical visitor would see. Cloaking can result in your site being demoted or even removed from the search results. To prevent cloaking, do not abuse visitor segmentation to display different content to Googlebot based on user-agent or IP address.
Use rel="canonical" - If you run a split test with multiple URLs, you should use the rel="canonical" attribute to point the variations back to the original version of the page. Doing so will help prevent Googlebot from getting confused by multiple versions of the same page.
Use 302 Redirects Instead Of 301s - If you run a test that redirect the original URL to a variation URL, use a 302 (temporary) redirect vs a 301 (permanent) redirect. This tells search engines such as Google that the redirect is temporary, and that they should keep the original URL indexed rather than the test URL.
Run Experiments Only As Long As Necessary - Running tests for longer than necessary, especially if you are serving one variation of your page to a large percentage of users, can be seen as an attempt to deceive search engines.
A media company might want to increase readership, increase the amount of time readers spend on their site, and amplify their articles with social sharing. To achieve these goals, they might test variations on:
Email sign-up modals
Recommended content
Social sharing buttons
A travel company may want to increase the number of successful bookings are completed on their website or mobile app, or may want to increase revenue from ancillary purchases. To improve these metrics, they may test variations of:
Homepage search modals
Search results page
Ancillary product presentation
A technology company might want to increase the number of high-quality leads for their sales team, increase the number of free trial users, or attract a specific type of buyer. They might test:
Lead form components
Free trial signup flow
Homepage messaging and call-to-action
A/B Testing Examples
These A/B testing examples show the types of results the world's most innovative companies have seen through A/B testing with Optimizely:
Discovery Communications : Discovery A/B tested the components of their video player to engage with their TV show 'super fan.' The result? A 6% increase in video engagement.
Secret Escapes : Secret Escapes tested variations of their mobile signup pages, doubling conversion rates and increasing lifetime value.