Articles on lift

What Can Be Learned From 1,001 A/B Tests?

Meta Analysis

How long does a typical A/B test run for? What percentage of A/B tests result in a ‘winner’? What is the average lift achieved in online controlled experiments? How good are top conversion rate optimization specialists at coming up with impactful interventions for websites and mobile apps? This meta-analysis of 1,001 A/B tests analyzed using […] Read more…

Posted in A/B testing, AGILE A/B testing, Conversion optimization | Also tagged , , , , , , , ,

Analysis of 115 A/B Tests: Average Lift is 4%, Most Lack Statistical Power

Observed Percent Change Significant

Oct 2022 update: A newer, much larger and likely less biased meta analysis of 1,001 tests is now available! What can you learn from 115 publicly available A/B tests? Usually, not much, since in most cases you would be looking at case studies with very basic data about what was tested and the outcome of […] Read more…

Posted in A/B testing, Conversion optimization | Also tagged , , , , , , , , , ,