# A/B Testing
Confidence: high
Last verified: 2026-05-22
Generation: human_only


## TL;DR

A/B testing (split testing) compares two variants to determine which performs better. Users are randomly assigned to version A (control) or B (treatment); results measured via a Key Performance Indicator (conversion rate, click-through rate, revenue). Statistical significance ensures results aren't due to random chance.

## Core Explanation

Sample size calculator determines minimum users needed (for given effect size, power, significance level). p-value < 0.05 typically indicates statistical significance. Pitfalls: peeking (checking results early leads to false positives), multiple comparisons (Bonferroni correction), novelty effect (new performs better initially), Simpson's paradox (aggregated results differ from segmented). Common test duration: minimum 1-2 full business cycles.

## Further Reading

- [undefined](undefined)