Free A/B test significance calculator Did your variant actually win?
A two-proportion Welch’s z-test, computed in your browser. Plug in two variants: get the lift, p-value, z-score and a clear ship/keep-running call.
Inputs
Two variants. Conversion = any goal event (purchase, add-to-cart, sign-up).
Result
FAQ
What test does this use?+
A two-proportion z-test (Welch’s) with a normal-approximation two-tailed p-value.
When is a result significant?+
Conventionally when p < 0.05. We highlight that threshold in green; any larger p value gets the amber “keep running” state.
How big a sample do I need?+
The calculator outputs the per-arm sample size required to detect the observed effect at α=0.05 and 80% power, using meanP across the two arms.
Is the math the same as in Idukki?+
Close, but not identical. The dashboard pools the variance across both arms. It also adds an mSPRT p-value and a Šidák correction for three or more variants. The two read close when both arms are a similar size.
More free tools
Try one of these next
Attribution model comparator
See how first-click, last-click, linear and U-shaped models split credit across a real purchase path, and how much last-click under-credits your UGC discovery.
Open toolUGC revenue calculator
Pick your industry, drag your annual online sales, and see the revenue you’re leaving on the table: a low–likely–high band across conversion, discovery and repeat-purchase lift. Two inputs, instant, no signup for the headline number.
Open toolWidget speed checklist
Eight checks every widget should pass for CLS under 0.05, INP under 200 ms, no layout damage.
Open tool
Run this test on your own product pages.
Experiments split traffic between gallery layouts and report the lift for you.
On Shopify? Install from the App Store.