Tag Archives: multiple testing

The Google Optimize Statistical Engine and Approach

Frequentist vs Bayesian A/B testing - Google Optimize

Google Optimize is the latest attempt from Google to deliver an A/B testing product. Previously we had “Google Website Optimizer”, then we had “Content Experiments” within Google Analytics, and now we have the latest iteration: Google Optimize. While working on the integration of our A/B Testing Calculator with Google Optimize I was curious to see […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistics | Also tagged , , , , , , , , , , , | Leave a comment

Statistical Significance in A/B Testing – a Complete Guide

Statistical Significance P Value

The concept of statistical significance is central to planning, executing and evaluating A/B (and multivariate) tests, but at the same time it is the most misunderstood and misused statistical tool in internet marketing, conversion optimization, landing page optimization, and user testing. This is not my first take on the topic, but it is my best […] Read More…

Posted in A/B Testing, Conversion Optimization, Multiple Variations Testing, Statistical Significance, Statistics | Also tagged , , , , , , , , | 6 Responses

The Bane of AB Testing: Reaching Statistical Significance

Illusory Results AB Testing

Now, this is not a new topic on this blog. I’ve discussed the issue of optional stopping based on “achieving” or “reaching” statistical significance┬áin my Why Every Internet Marketer Should be a Statistician post more than two years ago and others have touched on┬áit as well – both before and after me. However, the issue […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Also tagged , , , , , , , , , | Comments closed