Articles on Multiple variations testing

Articles on the topic of multiple variant testing (MVT) in online experiments, a.k.a. A/B/n tests, dealing mostly with the different statistical methods, adjustments, and estimators related to the case of testing multiple variants versus a control.

Statistical Significance in A/B Testing – a Complete Guide

Statistical Significance P Value

The concept of statistical significance is central to planning, executing and evaluating A/B (and multivariate) tests, but at the same time it is the most misunderstood and misused statistical tool in internet marketing, conversion optimization, landing page optimization, and user testing. This article attempts to lay it out in as plain English as possible: covering […] Read more…

Also posted in A/B testing, Conversion optimization, Statistical significance, Statistics | Tagged , , , , , , , , ,

Multivariate Testing – Best Practices & Tools for MVT (A/B/n) Tests

Multivariate AB Tests / MVT Testing

Let’s get this out of the way from the very beginning: most “A/B tests” are in fact multivariate (MVT) tests, a.k.a. A/B/n tests. That is, most of the time when you read about “A/B testing” the term also encompasses multivariate testing. The only reason to specifically differentiate between A/B and MVT is when someone wants to […] Read more…

Also posted in A/B testing, AGILE A/B testing, Statistical significance, Statistics | Tagged , , , , ,

Overview of Books on Conversion Optimization, Web Analytics, SEO & PPC

Books on online A/B testing

This overview is a little different than most overeviews out there. It will focus on three specific basic statistical concepts and would show if and how well they are explored in 18 of the top books on Conversion Optimization, A/B testing, web analytics, SEO & PPC/AdWords. The three concepts this review focuses on are statistical […] Read more…

Also posted in A/B testing, Conversion optimization, Google Analytics, Internet marketing, Statistical significance, Statistics | Tagged , ,

Why Every Internet Marketer Should Be a Statistician

Marketing as Fortune Telling

I’ll start this post right with the punch line: Internet marketing at its current state is dominated by voodoo magic. There, I’ve said it, and I’ve said it in bold italic so you know it must be true. Internet marketing is not as data-driven as most of us have been led to believe, it is […] Read more…

Also posted in A/B testing, Conversion optimization, Internet marketing, Statistical significance, Statistics | Tagged , , , , , , ,