Tag Archives: roi

Risk vs. Reward in A/B Tests: A/B testing as Risk Management

Risks vs Rewards in AB Testing

What is the goal of A/B testing? How long should I run a test for? Is it better to run many quick tests, or one long one? How do I know when is a good time to stop testing? How do I choose the significance threshold for a test? Is there something special about 95%? […] Read More…

Posted in A/B Testing, Conversion Optimization, Statistical Significance, Statistics | Also tagged , , , , , , | Leave a comment

Futility Stopping Rules in AGILE A/B Testing

Stopping for Lack of Effect (Futility)

In this article we continue our examination of the AGILE statistical approach to AB testing with a more in-depth look into futility stopping, or stopping early for lack of positive effect (lack of superiority). We’ll cover why such rules are helpful and how they help boost the ROI of A/B testing, why a rigorous statistical rule […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Conversion Optimization, Statistics | Also tagged , , , , , | Leave a comment