Sequential statistics are gathering interest and there are more and more questions posed by CROs looking into the matter. For this article I teamed up with Lucia van den Brink, a distinguished CRO consultant who recently started using Analytics Toolkit and integrated frequentist sequential testing into her client workflow. In this short interview she asks […] Read more…
A central feature of sequential testing is the idea of stopping “early”, as in “earlier compared to an equivalent fixed-sample size test”. This allows running A/B tests with fewer users and in a shorter amount of time while adhering to the targeted error guarantees. For example, a test may be planned with a maximum duration […] Read more…
Have you heard how there is a much greater probability than generally expected that a statistically significant test outcome is in fact a false positive? In industry jargon: that a variant has been identified as a “winner” when it is not. In demonstrating the above the terms “False Positive Risk” (FPR), “False Findings Rate” (FFR), […] Read more…
Google Analytics 4 has been a let down in many aspects based on every discussion I’ve seen and had with professionals of all stripes – marketers, advertising specialists, CROs, GA professionals, online experimentation experts, etc. One of the less discussed issues it brought with it is the default heavyweight GTAG library integration it comes with. […] Read more…
Analytics Toolkit was conceived in 2012 as a set of tools that automate essential Google Analytics-related tasks and augment the GA functionalities in various ways. This goal was achieved in the years since with the release of over a dozen tools utilizing the Google Analytics API. These were accompanied by dozens of in-depth technical articles […] Read more…
Take your A/B testing program to the next level
Want to take your A/B tests to the next level?
If you enjoyed this article and want to read more great content like it make sure to check out the book “Statistical Methods in Online A/B Testing” by the author, Georgi Georgiev, and take your experimentation program to the next level.