THE BEGINNER’S GUIDE TO A/B TESTING

BY
SARAMON
Dec 24, 2018
DESIGN

Ever feel like you hit a brick wall when it comes to designing your perfect website? It seems that even after conducting usability tests, you just can’t quite put your finger on what it is that isn’t getting you enough subscribers or users.

Maybe it isn’t usability or user experience that’s causing the problem, what if it’s the layout, the images you’re using or the pricing on your website?

But how do you test that? A not-so-secret weapon online business pros use is A/B Testing.

WHAT’S A/B TESTING?

In short, A/B Testing is making two versions of one element in your design and seeing which one gets you better results.

For example, it’s like testing to see if the ‘SUBSCRIBE’ button written in upper-case letters gets you more subscribers than the ‘subscribe’ button written in lower-case letters. If you find out that the ‘SUBSCRIBE’ button gets you more clicks than the other ‘subscribe’ button, then you’ll simply go for the better option.

The way it works is that you make two versions of one design, one is the design you’re already using, which is design ‘A’, and the new design you want to test, which is design ‘B’. You run both of them at the same time, then you divide your live traffic between the two and see where both versions take you. After a sufficient amount of time, you compare the analyses of the two versions and see which one gets you more clicks or subscribers.

Alright, so now you know the basic idea behind A/B Testing, but what good will this do for your business?

The first obvious benefit of A/B Testing is that it will help you figure out the ideal marketing strategy for your product or company. Also, you’ll have a clearer understanding of your target market and what catches their attention the most. With that said, it’ll become easier for you to make decisions regarding design and you won’t even have to take it out for a test run because you’ll know what gets your users going… or buying in this case.

Now the question remains, how do you go about A/B Testing without messing it up?

SO YOU WANT TO RUN AN EFFECTIVE A/B TEST? ASK YOURSELF THESE 4 IMPORTANT QUESTIONS…

Q1: WHAT’S YOUR PLAN?

 

img

 

As tempting as it is to just start testing right away, you need to make sure you have a plan worked out. How will you know what ‘good’ results are to you when you don’t even know what you want exactly? Therefore, it’s crucial that you have a clear idea of what kind of results you’re looking for before you start. To do that, you need to figure out what kind of results you’re getting now, what kind of results you want to see, and list the objectives of the test.

Another important thing you should look into is what A/B Testing tool suits your needs the most. There are many testing tools out there for you to check out, some are even for free; there’s Google Optimize, CrazyEgg, Optimizely, etc. Don’t be afraid to explore your options!

Q2: WHAT DO YOU WANT TO TEST EXACTLY?

 

img

 

This question should be answered based on the list of objectives you made during ‘the planning phase’. You first need to identify whether you’re planning on running an off-site or on-site test. If you want to test an ad design or marketing copy or anything that happens anywhere outside your website, then that’s off-site testing. However, if you’re testing the headlines or any kind of graphic you use to direct your user to your website, then that’s on-site testing.

Once you figure that out, make a list of the variables you want to test. For example, if you want to test the headline on your website, you would probably want to test font color, font style, and alignment.

Q3: ARE YOU READY TO KEEP A STEADY SCHEDULE?

 

img

 

 

That’s right, you need to make sure you’re keeping a controlled testing schedule for your website. Be consistent and always focus on your main objective. Avoid jumping from one element to another; don’t go running tests on one of your ad campaigns and then randomly start testing one of the headlines on your site, this will only confuse you and derail your progress.

Q4: SO HOW LONG IS THIS GOING TO TAKE TO DO?

 

img

 

This highly depends on the amount of traffic you get, so one test may take a few days while another could take weeks. This is why it’s important to set a time limit for your tests, this can be tricky though because giving a test insufficient time could lead to skewed results because you didn’t get enough people for the test to be statistically accurate. The same goes for running tests for too long. Always know when to draw the line, so set a minimum and maximum duration, number of clicks, bounce rate, sales, etc.

CONCLUSION

One thing worth mentioning is that it’s easy to take things at face value and just pick the version that works best for you, but remember, analytics is your friend, study it thoroughly and carefully. Otherwise, you’ll end up eliminating certain aspects of the design that were actually working for you. You’ll find yourself on an endless loop, not knowing what your audience wants or where the problem is anymore. For effective A/B Testing, devise a plan and keep your goals in sight.

x