A/B testing
Monday stand-up. Someone moved the gallery above the fold last month, and sales went up. Someone else says it was the weather. Nobody can prove either.
Split the traffic, declare the winner, move on.
One running experiment per widget. Live KPIs every 30 seconds. Declare a winner once each variant clears 1,000 impressions.
No code. On Shopify, the Free plan needs no card.
What you get
Split a widget’s traffic between two versions and watch add-to-cart on each. You call a winner once each side has enough views to trust.
How it works
Split the widget’s traffic, hold one variant back, and only call a winner once each side has enough impressions to mean something.
01
Control versus one variant
Control (A) is your current layout. Variant B changes the layout, the CTA copy, the source filter or the overlay style. Traffic splits 50/50 when you hit Start; weighting is configurable.
Widget builder
Pick sources and a layout; the preview is the real renderer
Sources
- TikTok
- YouTube
- Hashtag
- Reviews
Layout
Products
AI tagging on
Add to cart inline
Live preview
02
Live counters, not a weekly export
Impressions, clicks, conversions and revenue per variant refresh every 30 seconds while a test runs. You watch it live.
Engagement
Every user interaction event, click, watch, add to cart
Visitors
284.1K
Clicks
7.9K
Click rate · 2.78%
Add to cart
154
ATC rate · 1.95%
Orders
248
Conv · 161.04%
Revenue
₹843.4K
Overall · 0.09%
Conversion funnel
Click → ATC
1.95%
Overall conv
0.09%
Daily add-to-cart · 30 days
Conversion by platform
| Platform | Add to cart | Checkout | Revenue |
|---|---|---|---|
| 153 | 12 | ₹620.4K | |
| TikTok | 38 | 2 | ₹104.2K |
| YouTube | 21 | 1 | ₹38.6K |
| Reviews | 14 | 1 | ₹52.1K |
| Manual | 22 | 2 | ₹28.1K |
03
Significance that survives peeking
The engine runs a pooled two-proportion z-test and an always-valid sequential test (mSPRT). You can check results early without inflating false wins.
One
event taxonomy
Per widget
impressions to orders
Illustration
not live data
04
Declare the winner, then apply it
Once each variant has 1,000 impressions, one click stops the test. “Apply to gallery” then copies the winner’s settings to the widget.
Attribution
One order, three ways to credit it
Instagram reel
Day 1
Gallery click
Day 3
PDP video watched
Day 3
Checkout
Day 5
First touch
Last touch
Linear
Order value $118. Credit shown per touchpoint under each model.
How it works, from switched on to paying for itself.
- Step 1
Pick the widget
Any widget (PDP, PLP, homepage, collection, lookbook) hosts one running experiment at a time.
- Step 2
Define Control and B
Keep your current layout as A; change one thing for B.
- Step 3
Watch the counters
Per-variant impressions, clicks, conversions and revenue, refreshed every 30 seconds.
- Step 4
Declare the winner
Past 1,000 impressions per variant, declare the winner. Then apply it to the gallery.
“For fashion brands sitting on a lot of creator/UGC content, this has been a very effective way to turn that content into something far more measurable and conversion focused.”
What ships, and what does not.
Each variant’s numbers and the significance figure sit in the dashboard. The free calculator gives a close check from the raw counts.
- 01
One experiment per widget
Enforced in the dashboard so test scopes never collide. Starting a second test on the same widget stops the first.
- 02
Pooled z-test plus mSPRT
Each variant is read with a pooled two-proportion z-test. An mSPRT p-value stays valid when you check early. Three or more variants get a Šidák correction.
- 03
Up to six variants per test
Each variant is a whole widget setup. Single elements are not tested in combination.
The questions to ask before you switch it on.
How many experiments can run at once?
One per widget, enforced in the dashboard so two tests never overlap on the same surface. You can run experiments on different widgets in parallel.
What can Variant B change?
Layout, CTA copy, source filter or overlay style. Keep the change to one thing per test so the result is readable.
When can I declare a winner?
Once each variant has 1,000 or more impressions. The counters and the significance test are visible the whole time.
Which plans include A/B testing?
Idukki pricing is impression-based rather than feature-gated: you pick a monthly impression band, and the comparison table at /pricing lists what every band ships. If you need a specific capability confirmed for your tier, ask us and we will show it to you on a call. The significance calculator at /tools/ab-significance is free for anyone.