Synetix SystemsIt depends on two numbers: how often visitors convert now, and how big a change you want to detect. At a 3% conversion rate, a test needs about 1,500 visitors to detect a doubling, about 5,000 to detect a 50% lift, and about 28,000 to detect a 20% lift.
Most "minimum traffic" figures you read online skip the second number. That is why they disagree. The math below is the standard two-group test, and you can check every figure in a free calculator such as Evan Miller's sample size calculator.
The assumptions
- Two versions of one page, with traffic split 50/50.
- A significance level of 5% (two-sided) and statistical power of 80%. These are the usual defaults, and most testing tools start from them.
- The conversion is one action, such as a form sent or a booking made.
- "Lift" is relative. A 50% lift on a 3% rate means 3% becomes 4.5%.
The sample size for each version is about 7.85 × (p1(1 − p1) + p2(1 − p2)) ÷ (p2 − p1)², where p1 is the current rate and p2 is the rate you want to detect. The 7.85 comes from the two defaults above.
Visitors needed for one test
| Current rate | Lift to detect | Visitors per version | Total visitors | At 1,000 visitors a month |
|---|---|---|---|---|
| 3% | 20% (to 3.6%) | 13,911 | 27,822 | about 28 months |
| 3% | 30% (to 3.9%) | 6,452 | 12,903 | about 13 months |
| 3% | 50% (to 4.5%) | 2,515 | 5,029 | about 5 months |
| 3% | 100% (to 6%) | 746 | 1,492 | about 6 weeks |
| 5% | 20% (to 6%) | 8,155 | 16,310 | about 16 months |
| 5% | 50% (to 7.5%) | 1,468 | 2,936 | about 3 months |
| 5% | 100% (to 10%) | 432 | 864 | about 4 weeks |
Two things stand out. Small lifts need a lot of traffic. And a low conversion rate needs more traffic than a high one, for the same relative lift.
What this means for a small site
At 1,000 visitors a month, a test can find big differences in a season. It cannot find small ones in any useful time. So a low-traffic site should test changes that can plausibly move the result by half or more: a different offer, a different first screen, a shorter form, a different call to action. It should not test a button color.
Three other ways to get an answer sooner:
- Test the page with the most traffic. For most firms that is the home page or the main practice-area page, not a deep page.
- Measure an action that happens more often. A click on "Book a consultation" happens more often than a finished booking. Use it as a guide, and confirm with the real goal later.
- Do not stop early. If you check a test every day and stop at the first "significant" result, you will often keep a version that is not better. Decide the sample size first, then wait for it.
Where Synetix SmartSite fits
Synetix SmartSite runs these tests for you, keeps the version that wins, and only accepts sites with 1,000 or more visitors a month, for the reason in the table above.