Tag Archives: multiple testing

Statistical Significance in A/B Testing – a Complete Guide

Statistical Significance P Value

The concept of statistical significance is central to planning, executing and evaluating A/B (and multivariate) tests, but at the same time it is the most misunderstood and misused statistical tool in internet marketing, conversion optimization, landing page optimization, and user testing. This is not my first take on the topic, but it is my best […] Read More…

Posted in A/B Testing, Conversion Optimization, Multiple Variations Testing, Statistical Significance, Statistics | Also tagged , , , , , , , , | 2 Responses

The Bane of AB Testing: Reaching Statistical Significance

Illusory Results AB Testing

Now, this is not a new topic on this blog. I’ve discussed the issue of optional stopping based on “achieving” or “reaching” statistical significance┬áin my Why Every Internet Marketer Should be a Statistician post more than two years ago and others have touched on┬áit as well – both before and after me. However, the issue […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Also tagged , , , , , , , , , | Leave a comment