Articles on risk-reward ratio

Statistical Power, MDE, and Designing Statistical Tests

Statistical Power and MDE Demystified

One topic has surfaced in my ten years of developing statistical tools, consulting, and participating in discussions and conversations with CRO & A/B testing practitioners as causing the most confusion and that is statistical power and the related concept of minimum detectable effect (MDE). Some myths were previously dispelled in “Underpowered A/B Tests – Confusions, […] Read more…

Posted in A/B testing, Statistics | Also tagged , , ,

Risk vs. Reward in A/B Tests: A/B testing as Risk Management

Risks vs Rewards in AB Testing

What is the goal of A/B testing? How long should I run a test for? Is it better to run many quick tests, or one long one? How do I know when is a good time to stop testing? How do I choose the significance threshold for a test? Is there something special about 95%? […] Read more…

Posted in A/B testing, Conversion optimization, Statistical significance, Statistics | Also tagged , , , , , ,

Costs and Benefits of A/B Testing: A Comprehensive Guide

Costs and Benefits in AB Testing

This is a comprehensive guide to the different types of costs and benefits, risks and rewards related to A/B testing. Understanding them in detail should be valuable to A/B testers and businesses considering whether to engage in A/B testing or not, what to A/B test and what not to test, etc. As far as I […] Read more…

Posted in A/B testing, Conversion optimization | Also tagged , , , , , ,