Category Archives: A/B Testing

Bayesian AB Testing is Not Immune to Optional Stopping Issues

Fantasy vs real world in bayesian ab testing

Fantasy vs the Real World: Naive Bayesian AB Testing vs Proper Statistical Inference This post is addressed at a certain camp of proponents and practitioners of A/B testing based on Bayesian statistical methods who claim that outcome-based optional stopping, often called data peeking or data-driven stopping, has no effect on the statistics and thus inferences […] Read More…

Also posted in Conversion Optimization, Statistical Significance, Statistics | Tagged , , , , , , , | Leave a comment

Using Content Groupings for User Flow Analysis in Google Analytics

Order Chaos Content Groupings

In the first part of this article we talked about how to set up content groupings in Google Analytics and how to make sure it’s working as expected. In this part we’ll discuss the 4 main benefits of having content groupings defined. If I have to sum it up in one word, I would go with […] Read More…

Also posted in Google Analytics | Tagged , , , , | Comments closed

Setting up Content Groupings in Google Analytics

Planning

It’s been 2 years since content groupings first made it into Google Analytics, but it seems like this incredibly beneficial feature is still quite underused and poorly understood even among people otherwise familiar with the software. That’s why I decided to write a two-part guide of which you’re reading the first one – how to setup […] Read More…

Also posted in Google Analytics | Tagged , , , , | Comments closed

Overview of Books on Conversion Optimization, Web Analytics, SEO & PPC

This overview is a little different than most overeviews out there. It will focus on three specific basic statistical concepts and would show if and how well they are explored in 18 of the top books on Conversion Optimization, A/B testing, web analytics, SEO & PPC/AdWords. The three concepts this review focuses on are statistical […] Read More…

Also posted in Conversion Optimization, Google Analytics, Internet Marketing, Multiple Variations Testing, Statistical Significance, Statistics | Tagged , , | Comments closed

Should you do A/A, A/A/B or A/A/B/B tests in CRO?

A/A Split Test

This question pops up often in more “advanced” discussion forums and blogs on Conversion Optimization and usually one (or several) of the following advices are given: Do an A/A test first in order to test your split testing framework. If the difference between the two is statistically significant at the decided level, then your framework is […] Read More…

Also posted in Conversion Optimization, Statistics | Tagged , , | Comments closed

Why Every Internet Marketer Should Be a Statistician

Marketing as Fortune Telling

I’ll start this post right with the punch line: Internet marketing at its current state is dominated by voodoo magic. There, I’ve said it, and I’ve said it in bold italic so you know it must be true. Internet marketing is not as data-driven as most of us have been led to believe, it is […] Read More…

Also posted in Conversion Optimization, Internet Marketing, Multiple Variations Testing, Statistical Significance, Statistics | Tagged , , , , , , , | Comments closed