Monthly Archives: October 2020

Bayesian Probability and Nonsensical Bayesian Statistics in A/B Testing

Bayesian Probability and Bayesian Statistics in AB Testing

Many adherents of Bayesian methods put forth claims of superiority of Bayesian statistics and inference over the established frequentist approach based mainly on the supposedly intuitive nature of the Bayesian approach. Rational thinking or even human reasoning in general is Bayesian by nature according to some of them. Others argue that proper decision-making is inherently […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistics | Tagged , , , , , | Leave a comment

The Perils of Using Google Analytics User Counts in A/B Testing

Google Analytics User Data in AB Testing

Many analysts, marketers, product managers, UX and CRO professionals nowadays rely on user counts provided by Google Analytics, Adobe Analytics, or similar tools, in order to perform various statistical analyses. Such analyses may involve the statistical hypothesis tests and estimations part of A/B testing, and may also include regressions and predictive models (LTV, churn, etc.). […] Read More…

Posted in A/B Testing, Google Analytics, Statistics | Tagged , , , , , , , , | 4 Responses

The Effect of Using Cardinality Estimates Like HyperLogLog in Statistical Analyses

This article will examine the effects of using the HyperLogLog++ (HLL++) cardinality estimation algorithm in applications where its output serves as input for statistical calculations. A prominent example of such a scenario can be found in online controlled experiments (online A/B tests) where key performance measures are often based on the number of unique users, […] Read More…

Posted in A/B Testing, Google Analytics, Statistics | Tagged , , , , , , | Leave a comment