background preloader

20 lines of code that will beat A/B testing every time

20 lines of code that will beat A/B testing every time
Zwibbler.com is a drop-in solution that lets users draw on your web site. A/B testing is used far too often, for something that performs so badly. It is defective by design: Segment users into two groups. Show the A group the old, tried and true stuff. Show the B group the new whiz-bang design with the bigger buttons and slightly different copy. After a while, take a look at the stats and figure out which group presses the button more often. In recent years, hundreds of the brightest minds of modern civilization have been hard at work not curing cancer. With a simple 20-line change to how A/B testing works, that you can implement today, you can always do better than A/B testing -- sometimes, two or three times better. It can reasonably handle more than two options at once.. The Multi-armed bandit problem The multi-armed bandit problem takes its terminology from a casino. Like many techniques in machine learning, the simplest strategy is hard to beat. Why does this work? More blog entries Related:  generalA/B testOthers

How do you review code? I'm hoping to find ways to improve the code review process at the company where I work. My team has a fairly has a fairly standard github PR-based process. When you have some code you want to merge into the master branch you open a PR, ask another developer or two to review it, address any comments they have, and then wait for one of the reviewers to give it an LGTM (looks good to me). The problem is that there can be a lot of lag between asking someone to review the PR and them actually doing it, or between addressing comments and them taking another look. Worst of all, you never really know how long things will take, so it's hard to know whether you should switch gears for the rest of the day or not. Over time we've gotten used to communicating a lot, and being shameless about pestering people who are less communicative. So, has anyone else run I to similar problems?

A Beginner's Guide to A/B Testing: Better Pay-Per-Click Ads Pay-per-click advertising is a key component of many online marketing campaigns. It can also be one of the most expensive ongoing costs in a campaign. Therefore, it’s key that you test your ads regularly, to make sure you aren’t letting conversions slip through the cracks. To an extent, PPC testing is simpler than many other kinds of A/B tests, partly because there are fewer things to test. But that doesn’t mean any less care and planning should go into preparing and executing these tests. This is the fourth installment in our A Beginner’s Guide to A/B Testing series. Deciding What to Test Pay-per-click ad testing is a bit more streamlined than the other topics we’ve covered in this series. The headlineThe body textThe linkThe keywords the ad displays for The headline is the part that’s going to show up as a link (in blue) in search results. The body text is the equivalent to your page’s description meta tag in organic search results. What Are You Testing For? Track and Analyze Your Results

The Codist: My Biggest Regret As A Programmer A little over 20 years ago I was at a crossroads. My second company was petering out when our 5 years of building Deltagraph for the publisher ended (they wanted to move into the nascent internet space). At that point I had 13 years experience as a programmer but also 9 years or so experience running a company (at the same time). I no longer wanted to do both. My first company 85-87 not only built a new kind of spreadsheet program but also published it ourselves. I led the company, did all the press interviews, managed the investors, did all the usual business stuff and also was one of the three programmers and the UI designer. So at that point in 1994 I could have gone either into technical management or continued as a programmer. I was in the Bay area for a year around 1995 and worked at Apple for the last half. Not only did Apple begin a huge turnaround a year later when Steve came back, but the whole Dotcom explosion happened. And today I am still just a programmer.

Nick Craver - Stack Overflow: How We Do Deployment - 2016 Edition This is #3 in a very long series of posts on Stack Overflow’s architecture. Previous post (#2): Stack Overflow: The Hardware - 2016 Edition We’ve talked about Stack Overflow’s architecture and the hardware behind it. The next most requested topic was Deployment. How do we get code a developer (or some random stranger) writes into production? I’m going ahead and inserting a set of section links here because this post got a bit long with all of the bits that need an explanation: Source This is our starting point for this article. A Little Context We deploy roughly 25 times per day to development (our CI build) just for Stack Overflow Q&A. The Human Steps When we’re coding, if a database migration is involved then we have some extra steps. And here’s the local %Repo%\StackOverflow.Migrations\ folder: You can see both in chat and locally that 726 was the last migration number taken. Now let’s add some code — we’ll keep it simple here: A \StackOverflow\Models\User.cs diff: .. So far, so good. Tiers

A Beginner’s Guide To A/B Testing: Exceptional Web Copy Optimizing the copy on your website is at least as important as optimizing the design, especially if the primary goal of that site is to convert visitors. A pretty design can only get you so far. If you really want to gain new customers, you need to optimize the text on your site to instill trust in visitors and make them want to purchase from you. We often spend hours or days reading about the best techniques to use to sell products. But that doesn’t give us a complete picture, and what works great for one company or one product might not work at all for another. A/B testing is the simplest type of testing you can do to figure out which variations of copy, headline, and other factors are most effective in direct relation to your site and your offerings. Things to Test There are a ton of things you can test on your site to see what’s most effective. Here’s a brief list of the web copy elements on your site you might test: Tools for A/B Testing A/B Testing Best Practices

A little thing about release notes — Several People Are Typing — The Official Slack Blog — Medium A little thing about release notes Why they’re important to us, and how we approach the writing of them at Slack As long as there’s been software to update, people have been writing notes to tell you what’s gone into those updates. So, when we started working on Slack, and updating the service constantly, we thought about what we could do to make release notes more useful than the default… March 23rd: v2.0.8 Bug fixes and Minor Improvements. Because why stick to that, when they could contain real value, useful information, and the opportunity to connect a little bit, human to human, with the people who use Slack the most. The funny thing about release notes is that they’re disposable, and temporary. Or you can read them. So we try to make them worth it for you. It’s not about moving them too far away from what they are. This is how we go about it. Make a list As our teams work on a release, they keep a list of everything that goes into it. Check it twice Know when to stop

Rule of Silence definition by The Linux Information Project The rule of silence, also referred to as the silence is golden rule, is an important part of the Unix philosophy that states that when a program has nothing surprising, interesting or useful to say, it should say nothing. It means that well-behaved programs should treat their users' attention and concentration as being valuable and thus perform their tasks as unobtrusively as possible. That is, silence in itself is a virtue. There is no single, standardized statement of the Unix philosophy, but perhaps the simplest description would be: "Write programs that are small, simple and transparent. This philosophy has been fundamental to the the fact that Unix-like operating systems have been thriving for more than three decades, far longer than any other family of operating systems, and can be expected to see continued expansion of use in the years to come, particularly in the form of Linux. The rule of silence is one of the oldest and most persistent design rules of such operating systems.

A Beginner's Guide To AB Testing: An Introduction A/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for your business. It can be used to test everything from website copy to sales emails to search ads. And the advantages A/B testing provide are enough to offset the additional time it takes. Well-planned A/B testing can make a huge difference in the effectiveness of your marketing efforts. How Do You Plan an A/B Test? The first thing to do when planning an A/B test is to figure out what you want to test. With off-site tests, you’re probably testing either an ad, or a sales email. Once you know what you’ll test, make a list of all the variables you’ll test. the location of the call to actionthe exact text usedthe button color or surrounding space It’s a process, and it’s common for multiple A/B tests to be carried out prior to making a final decision or final change. Make sure that before you start testing you have a clear idea of the results you’re looking for. Things to Test

Related: