background preloader

A/B test

Facebook Twitter

Most of your AB-tests will fail. Jitbit Blog - please Subscribe if you like this post Jun 19 2013.

Most of your AB-tests will fail

20 lines of code that will beat A/B testing every time. is a drop-in solution that lets users draw on your web site.

20 lines of code that will beat A/B testing every time

A/B testing is used far too often, for something that performs so badly. It is defective by design: Segment users into two groups. Show the A group the old, tried and true stuff. Show the B group the new whiz-bang design with the bigger buttons and slightly different copy. After a while, take a look at the stats and figure out which group presses the button more often. In recent years, hundreds of the brightest minds of modern civilization have been hard at work not curing cancer.

With a simple 20-line change to how A/B testing works, that you can implement today, you can always do better than A/B testing -- sometimes, two or three times better. It can reasonably handle more than two options at once.. The Multi-armed bandit problem The multi-armed bandit problem takes its terminology from a casino. Like many techniques in machine learning, the simplest strategy is hard to beat.

Why does this work? More blog entries. A/B test case study: how two magical words increased conversion. Posted in A/B Split Testing on May 26th, 2010 When we saw the results Soocial had got from their latest A/B test, we were astonished!

A/B test case study: how two magical words increased conversion

They added just two words next to the Sign up button and the conversions shot up by 28%. If we say the phrase was one of these: “Sign up for Free”, “It’s Free” or “Free Signup”, can you guess which one did the trick? That is precisely the beauty of A/B testing, you can never guess what works – they only way out is to actually test it. This case study has direct inputs from Soocial CEO, Stefan Fountain. Background Soocial is an online address book that helps you keep your phone, computer and online services contacts sane.

What was tested Their homepage features a large Sign up button. Results As you can read above, the changes on the page were extremely minor and, on the surface of it, look quite trivial. Control: 14.5% conversion rate Variation: 18.6% conversion rate Why “It’s Free” Worked How valuable was Visual Website Optimizer for the A/B Test? Invaluable. A Beginner's Guide to A/B Testing: Better Pay-Per-Click Ads. Pay-per-click advertising is a key component of many online marketing campaigns.

A Beginner's Guide to A/B Testing: Better Pay-Per-Click Ads

It can also be one of the most expensive ongoing costs in a campaign. Therefore, it’s key that you test your ads regularly, to make sure you aren’t letting conversions slip through the cracks. To an extent, PPC testing is simpler than many other kinds of A/B tests, partly because there are fewer things to test. But that doesn’t mean any less care and planning should go into preparing and executing these tests. This is the fourth installment in our A Beginner’s Guide to A/B Testing series.

Deciding What to Test Pay-per-click ad testing is a bit more streamlined than the other topics we’ve covered in this series. The headlineThe body textThe linkThe keywords the ad displays for The headline is the part that’s going to show up as a link (in blue) in search results. The body text is the equivalent to your page’s description meta tag in organic search results. A Beginner’s Guide To A/B Testing: Exceptional Web Copy. Optimizing the copy on your website is at least as important as optimizing the design, especially if the primary goal of that site is to convert visitors.

A Beginner’s Guide To A/B Testing: Exceptional Web Copy

A pretty design can only get you so far. If you really want to gain new customers, you need to optimize the text on your site to instill trust in visitors and make them want to purchase from you. We often spend hours or days reading about the best techniques to use to sell products. But that doesn’t give us a complete picture, and what works great for one company or one product might not work at all for another. A/B testing is the simplest type of testing you can do to figure out which variations of copy, headline, and other factors are most effective in direct relation to your site and your offerings. A Beginner's Guide To AB Testing: An Introduction. A/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for your business.

A Beginner's Guide To AB Testing: An Introduction

It can be used to test everything from website copy to sales emails to search ads. And the advantages A/B testing provide are enough to offset the additional time it takes. Well-planned A/B testing can make a huge difference in the effectiveness of your marketing efforts. Narrowing down the most effective elements of a promotion, and then combining them, can obviously make your marketing efforts much more profitable and successful.