Category Archives: Multiple Variations Testing

Statistical Significance in A/B Testing – a Complete Guide

Statistical Significance P Value

The concept of statistical significance is central to planning, executing and evaluating A/B (and multivariate) tests, but at the same time it is the most misunderstood and misused statistical tool in internet marketing, conversion optimization, landing page optimization, and user testing. This is not my first take on the topic, but it is my best […] Read More…

Also posted in A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Tagged , , , , , , , , , | Leave a comment

Multivariate Testing – Best Practices & Tools for MVT (A/B/n) Tests

Multivariate AB Tests / MVT Testing

Let’s get this out of the way from the very beginning: most “A/B tests” are in fact multivariate (MVT) tests, a.k.a. A/B/n tests. That is, most of the time when you read about “A/B testing” the term also encompasses multivariate testing. The only reason to specifically differentiate between A/B and MVT is when someone wants to […] Read More…

Also posted in A/B Testing, AGILE A/B Testing, Statistical Significance, Statistics | Tagged , , , , , | Leave a comment

Overview of Books on Conversion Optimization, Web Analytics, SEO & PPC

This overview is a little different than most overeviews out there. It will focus on three specific basic statistical concepts and would show if and how well they are explored in 18 of the top books on Conversion Optimization, A/B testing, web analytics, SEO & PPC/AdWords. The three concepts this review focuses on are statistical […] Read More…

Also posted in A/B Testing, Conversion Optimization, Google Analytics, Internet Marketing, Statistical Significance, Statistics | Tagged , , | Comments closed

Why Every Internet Marketer Should Be a Statistician

Marketing as Fortune Telling

I’ll start this post right with the punch line: Internet marketing at its current state is dominated by voodoo magic. There, I’ve said it, and I’ve said it in bold italic so you know it must be true. Internet marketing is not as data-driven as most of us have been led to believe, it is […] Read More…

Also posted in A/B Testing, Conversion Optimization, Internet Marketing, Statistical Significance, Statistics | Tagged , , , , , , , | Comments closed