“Observed power”, “Post hoc Power”, and “Retrospective power” all refer to the statistical power of a statistical significance test to detect a true effect equal to the observed effect. In a broader sense these terms may also describe any power analysis performed after an experiment has completed. Importantly, it is the first, narrower sense that […] Read more…
The above is a question asked by some practitioners of A/B testing, as well as a number of their clients when examining the outcome of an online controlled experiment. It may be raised regardless if the outcome is statistically significant or not. In both cases the fact the observed effect in an A/B test is […] Read more…
Observed power, often referred to as “post hoc power” and “retrospective power” is the statistical power of a test to detect a true effect equal to the observed effect size. “Detect” in the context of a statistical hypothesis test means to result in a statistically significant outcome. Some calculators aimed at A/B testing practitioners use […] Read more…
Take your A/B testing program to the next level
Want to take your A/B tests to the next level?
If you enjoyed this article and want to read more great content like it make sure to check out the book “Statistical Methods in Online A/B Testing” by the author, Georgi Georgiev, and take your experimentation program to the next level.