Tag Archives: statistical significance

The Bane of AB Testing: Reaching Statistical Significance

Illusory Results AB Testing

Now, this is not a new topic on this blog. I’ve discussed the issue of optional stopping based on “achieving” or “reaching” statistical significance┬áin my Why Every Internet Marketer Should be a Statistician post more than two years ago and others have touched on┬áit as well – both before and after me. However, the issue […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Also tagged , , , , , , , , , | Leave a comment

Bayesian AB Testing is Not Immune to Optional Stopping Issues

Fantasy vs real world in bayesian ab testing

Fantasy vs the Real World: Naive Bayesian AB Testing vs Proper Statistical Inference This post is addressed at a certain camp of proponents and practitioners of A/B testing based on Bayesian statistical methods who claim that outcome-based optional stopping, often called data peeking or data-driven stopping, has no effect on the statistics and thus inferences […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Also tagged , , , , , , | Leave a comment

Overview of Books on Conversion Optimization, Web Analytics, SEO & PPC

This overview is a little different than most overeviews out there. It will focus on three specific basic statistical concepts and would show if and how well they are explored in 18 of the top books on Conversion Optimization, A/B testing, web analytics, SEO & PPC/AdWords. The three concepts this review focuses on are statistical […] Read More…

Posted in A/B Testing, Conversion Optimization, Google Analytics, Internet Marketing, Multiple Variations Testing, Statistical Significance, Statistics | Also tagged , | Leave a comment

Why Every Internet Marketer Should Be a Statistician

Marketing as Fortune Telling

I’ll start this post right with the punch line: Internet marketing at its current state is dominated by voodoo magic. There, I’ve said it, and I’ve said it in bold italic so you know it must be true. Internet marketing is not as data-driven as most of us have been led to believe, it is […] Read More…

Posted in A/B Testing, Conversion Optimization, Internet Marketing, Multiple Variations Testing, Statistical Significance, Statistics | Also tagged , , , , , , | 15 Responses