Online glossary of A/B testing terms and abbreviations

We are happy to present a brand new addition to our website: a comprehensive A/B testing glossary containing terms and abbreviations used testing as part of conversion rate optimization (CRO).  Definitions start from very basic things such as “A/B test“, “mean“, “conversion rate” and “revenue per user“, go through “hypothesis“, “null hypothesis“, “standard deviation“, “p-value” and “confidence interval” and go into depth with “generalizability“, “alpha-spending“, “statistical model“, “statistical design” and so on. All term definitions and explanations are lavishly cross-referenced so one can start anywhere and learn on the go. Of course, there is also an alphabetic index.

The aim of the glossary is not only to explain each concept, but to put it in the context of A/B testing and to even provide some best-practice advice and warnings against common misuses, sort of as a guide. Naturally, the goal of covering all these aspects in full depth is far too ambitions for a simple glossary, but I hope it delivers to a satisfactory extent. If you have any suggestions for improving certain glossary entries or want to suggest an important one we’ve missed, feel free to drop us an e-mail or find me on LinkedIn.

Basic Statistics

Since the glossary is all about stats, it makes sense to present some rudimentary statistics related to the project. At release (Dec 3, 2018) it contains 148 unique definitions which have a total of 65 aliases and abbreviations. The total length of the definitions is about 228,000 symbols, which is about 32,000 words or about a 100-page book in A-5 format.

The mean entry length is 1544 while 50% of entries are over 1428 symbols long. The length distribution is:

We see sort of a power-curve where most terms require significantly less words to explain while some need multiple times more words (graph is for symbols, not words, but an average word count of symbols / 7 should be a good approximation). I believe the current distribution offers a good balance between offering concise and on-point descriptions and advice while not going overboard and becoming a full encyclopedia entry.

Another curious statistic is the number of terms starting with different letters:

“S” is a natural leader due to all terms that start “statistical” and “significant/significance”. Curiously, none of the current entries start with “J”, “Q”, “W”, “X” and “Y”.

We always love to hear from our readers, so let us know if you found value in the glossary. Also, share with us if a term entry surprised you by presenting a new angle to a concept you thought you are pretty familiar with or by any of the auxiliary info presented (@AnalyticsToolki on Twitter).

Georgi is an expert internet marketer and statistician working passionately in the areas of SEO, SEM and Web Analytics since 2004. He is the founder of Analytics-Toolkit.com and owner of an online marketing agency & consulting company: Web Focus LLC and also a Google Certified Trainer in AdWords & Analytics. His special interest lies in data-driven approaches to testing and optimization in e-commerce and internet advertising. Georgi is the author of the book "Statistical Methods in Online A/B Testing".

Online glossary of A/B testing terms and abbreviations by
This entry was posted in A/B Testing, Conversion Optimization, Statistics and tagged , , , . Bookmark the permalink. Trackbacks are closed, but you can post a comment.
0Shares