Shopify Blogs
Ecommerce
The Beginner's Guide to Simple A/B Testing
by Mark MacdonaldConversion Rate OptimizationFeb 20, 2014
6 minute read30 comments
Email Pinterest Facebook Facebook LinkedIn
For nearly a decade now, A/B testing has been the not-so-well-kept secret of the web world only known to the pros, the initiated and the curious. It’s the method of choice for startups and growth hackers when it's comes to improving online conversions and achieving growth.
But more and more ecommerce merchants are making the leap and starting to use A/B testing to improve sales and get the most out of their hard earned traffic.
You can use it to test anything from landing pages and web copy to emails and search ads. The advantages are substantial, and more than enough to compensate for the time it takes to run an experiment on your site.
Fortunately or unfortunately, depending on how you look at it, most people aren’t as aware of this measurement tool as they should be. What this article hopes to do is walk you through the processes and advantages of using A/B testing in your business.
But, before we get ahead of ourselves, the question needs to be asked:
What is A/B Testing?
Simply put, it’s a way to figure out how you can get your website users to click that shiny red button instead of yesterday’s green one. Or help you discover if the green one was the better choice to begin with, thus forcing you back to the drawing board.
Put another way, it’s a method for measuring two versions of one element so you can determine which one is more successful. Is the winner version A or version B?
Clear so far?
To discover which version is a better bet, you have to subject both to experimentation at the same time. After which, you’re left with a more successful option to put into real-world use.
This is not unlike your high school science experiment. Except, instead of dissecting a frog you’re analyzing which is the better scalpel to slice it open. Which one cuts more smoothly, is easier to handle, saves time and increases overall dissection efficiency (so to speak of course). The only thing you’ll be slicing with this particular tool is the competition!
(Image credit Optimizely)
Translated to the web world, the process can be quite simple. For example, you might have two "Add to Cart" button designs for your website. Usually, A is the existing button you’re currently working with. This is also known as the “control”. B is the new button you want to implement, aka the “variant”. You then divide your live traffic into two equal groups and direct them to either page A or B. After a sufficient period you can measure performance by seeing which button generated more clicks and ultimately sales.
Common metrics for websites are things like conversion rate, bounce rate, CTA (call to action), sales, and CTR (click-through rate).
Other elements to test can include:
LayoutDesignStylePricingPromotionsHeadlinesCopywritingImages
When all is said and done, you select the version that has produced the best results and you’re on your way to a brave new world of online success.
If you think this is all too simple, or wish to generate more advanced results, then this next level of A/B testing may be for you.
A/B Split Testing vs. Multivariate Testing
If you're the type of person who wants to explore your traffic’s behavior at a deeper level, then multivariate testing is for you.
While more complex, the process of multivariate testing is similar to the standard A/B split test with only one difference. There are more variables to measure the outcomes you desire. If you have read and understood the previous section, then you may have already deduced the control stays the same. However, there is more than one variable, which paves the way for more specific results.
A simple example could be changing your headline, images, button colors, button text, and surrounding copy so you can come up with the perfect combination of variables. This refines your results and allows you to be more informed about the behavior of your traffic.
(Image credit Optimizely)
The downside to going the multivariate route is the increased complexity, and possibly the duration, of the testing period. However, you'll be able to generate some revealing data about your website performance.
This can be a cheap way to increase conversion rates. Why invest more hard-earned dollars into expensive ad spending when you can reduce marketing budgets by converting more people already visiting your site? Perhaps you want to optimize your site for existing and new visitors? Whatever your agenda for world domination, consistently testing your website ensures it’s always fully optimized.
You might be thinking, well this all sounds great in theory but what kind of results can I expect?
Successful A/B Test Case Studies
To get an idea of how you can apply A/B testing to your business, let's take a look at some mini case studies.
Single Page vs. Multi Page Checkout
Elastic Path wanted to change a two-page checkout process into a one-pager. They tested out the process on the Official Vancouver 2010 Olympic Store with some interesting results. Using A/B split testing they directed one set of traffic to the original two-page checkout while the other half was directed to the one-page checkout. The experiment was quickly ended with a clear winner after only 606 transactions. The gold medal went to the single page checkout, which outperformed its counterpart by 21.8%.
Conversely, the folks at Crazy Egg ran a similar test but found the exact opposite results. Originally they had a two step checkout process in which the user would first select their plan and create an account, and then be taken to a page where they would enter their payment information.
They decided to add another step and test a three-step checkout process. After selecting a plan, the user was now taken to a second page where they would create their account, and then be taken to yet another page to enter their payment information. The total number of form fields was the same as the two-step checkout process, but instead they decided to just break it out into three separate pages.
After a total of 817 conversions, the three-step checkout process was the clear winner and had a 10% increase in conversions.
The lesson here is that you need to experiment. Looking to what has worked for others is great for getting ideas, but ultimately you need to run your own tests to see what works for your site.
Fab’s Multivariate Add-to-Cart Experiment
Fab.com decided to test their purchase process by making three variations of an “Add to Cart” button to increase sales. One being icon focused, one with larger text, and one with a simple “+Cart”.
The test had a major impact on “Add to Cart” clicks on Fab. Variation 2 increased conversion by 15% compared to the original and Variation 1, the larger button with the clear call-to-action text, increased “Add to Cart” clicks by a staggering 49%.
Two Magical Words Increased Conversion Rate by 28%
In this experiment, two simple words increased the sign-ups on this button by 28%, demonstrating the significance of measuring call-to-action buttons. Apparently, even the smallest alterations can have surprisingly effective results. The two magic words in question? “It’s Free!”
Changing a Sign-Up From Green to Red Led to 34% Growth
Sometimes even the most trivial change to a website can fuel substantial results. CareLogger increased their conversion rate by 34% by changing the color range of their sign-up button from green to red.
Goes to show fashionistas have been saying for years, red never goes out of style.
“Mad Libs” Style Form Increased Conversion by 25-40%
A/B testing proved conventional wisdom isn’t always right. This test revealed paragraph-styled forms with inline input fields work more effectively than traditional form layouts.
While it’s important to mention the results were more specifically tied into the offer, this case still makes a strong argument for A/B testing as a trend mining tool.
A/B Testing Tools
Here are a variety of resources available to the many professional and budding A/B testers out there.
Google Analytics Content Research: A helpful and free A/B testing tool from Google. What it lacks in advanced features it makes up for as an ideal starting point.
Pricing: Free
Optimizely: An easy to use platform that let's you perform simple and advanced A/B tests via a point and click WYSIWYG editor.
Pricing: 3 packages - 3 packages - $17/mo, $71/mo, $359/mo (Shopify App)
Visual Website Optimizer: A simple A/B testing tool with advanced WYSIWYG editing, click maps, visitor segmentation and tag-less integration.
Pricing: 3 packages - Free, $49/mo, $129/mo
Unbounce: This simple landing page creator features integrated A/B testing.
Pricing: 3 packages - $49/mo, $99/mo, $199/mo
Webtrend’s Conversion Optimizer: A tool geared for testing enterprise clients.
Pricing: Varies
Further Reading
For those who want to delve deeper into the subject, here are a few resources to help you become an A/B testing pro.
Introduction to A/B Testing: A practical guide for implementing your tests.Effective A/B Testing: A variety of advice for increased test quality and overall effectiveness.101 A/B Testing Tips: A great go-to for ideas for your next A/B test, this site features a collection of tips.Which Test Won?: A simple game that challenges you to guess which variant won in a test.
Have you experimented with A/B testing in your business? What results have you found? Let us know in the comments.
About the Author: Mark Macdonald is the Content Manager at Shopify. Get more from Mark on Twitter.
ليست هناك تعليقات:
إرسال تعليق