A/B test significance
The standard two-proportion z-test, two-tailed — the same maths every major testing tool runs, so you can check their verdicts. No uploads; nothing leaves the page.
Variant A (control)
5.00%
Variant B (challenger)
6.50%
Not significant yet (p = 0.150) — this could easily be noise
- Relative uplift
- +30.0%
- Confidence
- 85.0%
- z-score
- 1.441
- 95% CI of diff
- -0.54% … 3.54%
To detect this size of uplift reliably (95% confidence, 80% power) you would need about 3,780 visitors per variant. Peeking early and stopping at the first significant reading inflates false positives — decide the sample size first, then read once.