Tag Archives: statistical power

Efficient AB Testing with the AGILE Statistical Method

AGILE AB Testing

Don’t we all want to run tests as quickly as possible, reaching results as conclusive and as certainĀ as possible? Don’t we all want to minimize the number of users we send to an inferior variant and to implement a variant with positive lift as quickly as possible? Don’t we all want to get rid of […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Conversion Optimization, Statistics | Also tagged , , , , , , , , , , , | Leave a comment

The Importance of Statistical Power in Online A/B Testing

Statistical Power and Test Sensitivity

What is Statistical Power? In null-hypothesis statistical testing (NHST) – the procedure most commonly applied in A/B tests, there are two types of errors that practitioners should care about, type I and type II errors. Type I is the probability of the test procedure to falsely reject a true null hypothesis. Type II error is […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Also tagged , , , , | Leave a comment

Why Every Internet Marketer Should Be a Statistician

Marketing as Fortune Telling

I’ll start this post right with the punch line: Internet marketing at its current state is dominated by voodoo magic. There, I’ve said it, and I’ve said it in bold italic so you know it must be true. Internet marketing is not as data-driven as most of us have been led to believe, it is […] Read More…

Posted in A/B Testing, Conversion Optimization, Internet Marketing, Multiple Variations Testing, Statistical Significance, Statistics | Also tagged , , , , , , | 15 Responses