Tag Archives: sequential testing

The A/B Testing Guide to Surviving on a Deserted Island

AB Testing Survival Guide Desert Island

The secluded and isolated deserted island setting has been used as the stage for many hypothetical explanations in economics and philosophy with the scarcity of things that can be developed as resources being a central feature. Scarcity and the need to keep risk low while aiming to improve one’s situation is what make it a […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistics | Also tagged , , , , , , , , , , , , , | Leave a comment

20-80% Faster A/B Tests? Is it real?

Percent Runs and Stopping Stage 1Delta

I got a question today about our AGILE A/B testing calculator and the statistics behind it and realized that I’m yet to write a dedicated post explaining the efficiency gains from using the method in more detail. This despite the fact that these speed gains are clearly communicated and verified through simulation results presented in our AGILE […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Statistics | Also tagged , , , , , , , | Leave a comment

Statistical Significance in A/B Testing – a Complete Guide

Statistical Significance P Value

The concept of statistical significance is central to planning, executing and evaluating A/B (and multivariate) tests, but at the same time it is the most misunderstood and misused statistical tool in internet marketing, conversion optimization, landing page optimization, and user testing. This is not my first take on the topic, but it is my best […] Read More…

Posted in A/B Testing, Conversion Optimization, Multiple Variations Testing, Statistical Significance, Statistics | Also tagged , , , , , , , , | Comments closed