Tag Archives: agile ab testing

Updates to Our A/B Testing Statistical Calculators

Meta Analysis Statistical Calculator

This short post presents several changes, updates, and new tool functionalities released in the past couple of months – some very recent, some less so. The updates concern all statistical calculators for design and analysis of A/B tests at Analytics-Toolkit.com and consist of the following: New tool: Statistical Calculator for Meta Analysis Quality of life […] Read More…

Posted in Analytics-Toolkit.com | Also tagged , , , | Leave a comment

AGILE A/B Testing Update: Custom API & Support for Non-Binomial Data

AGILE A/B Testing Tool Updates

I’m happy to announce the release of two long-awaited features for our AGILE A/B Testing Calculator: Support for non-binomial metrics like average revenue per user A new custom API for sending experiment data to the calculator Below is an explanation of each of these new features in some detail. Support for Non-Binomial Data While our […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Analytics-Toolkit.com | Also tagged , , , , , , | Comments closed

Affordable A/B Tests: Google Optimize & AGILE A/B Testing

The problem most-often faced by owners of websites who want to take a scientific approach to improving them by using A/B testing is that they might have relatively small revenue. Thus, when the ROI calculation for the A/B test is done it might turn out that it is economically unfeasible to test. In some cases, […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Analytics-Toolkit.com, Conversion Optimization | Also tagged , , , , , | Comments closed

20-80% Faster A/B Tests? Is it real?

Percent Runs and Stopping Stage 1Delta

I got a question today about our AGILE A/B testing calculator and the statistics behind it and realized that I’m yet to write a dedicated post explaining the efficiency gains from using the method in more detail. This despite the fact that these speed gains are clearly communicated and verified through simulation results presented in our AGILE […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Statistics | Also tagged , , , , , , , | Comments closed

Futility Stopping Rules in AGILE A/B Testing

Stopping for Lack of Effect (Futility)

In this article we continue our examination of the AGILE statistical approach to AB testing with a more in-depth look into futility stopping, or stopping early for lack of positive effect (lack of superiority). We’ll cover why such rules are helpful and how they help boost the ROI of A/B testing, why a rigorous statistical rule […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Conversion Optimization, Statistics | Also tagged , , , , , | Comments closed

Efficient AB Testing with the AGILE Statistical Method

AGILE AB Testing

Don’t we all want to run tests as quickly as possible, reaching results as conclusive and as certain as possible? Don’t we all want to minimize the number of users we send to an inferior variant and to implement a variant with positive lift as quickly as possible? Don’t we all want to get rid of […] Read More…

Posted in A/B Testing, AGILE A/B Testing, Conversion Optimization, Statistics | Also tagged , , , , , , , , , , , | Comments closed