A/B tests on low traffic site – is it possible?

Question

Did you have any success with A/B testing on low traffic landing page?Without for weeks or even months to see conclusive test results (95% statistical confidence).

softwaredoug · Accepted Answer

Instead of A/B testing, why not try and figure out why the traffic is so low on the landing page? Seems like there’s SEO work to be done?
In my last job we struggled with this. It was more useful, frankly, to go with our gut and qualitative information from sales inquiries we got (questions like “how did you learn about us” or “what made you think to contact us”). We also had a PR firm interview clients to understand how they perceived our brand.
For better or worse we used these qualitative bits of info to try and build useful landing pages. But a lot of it was unfortunately subjective and involved debate over how our marketing actually worked.

bmallerd · Answer

It really depends what you mean by "low". It's certainly possible at the hundreds-per-day type volume. The key is to just look for bigger effects. If 200 people show up, 10/100 convert on A, and 25/100 convert on B then B wins (p < .01)As your traffic scales, you can look for smaller and smaller effects. But the whole idea of testing variants can be applied pretty damn early.

jonny383 · Answer

If you flip a coin four times and get tails every time, do you conclude that flipping a coin isn't random?Obvious answer: the more traffic, the better your results.

brudgers · Answer

Is this a bigger priority than increasing traffic?

A/B tests on low traffic site – is it possible?

Did you have any success with A/B testing on low traffic landing page?
Without for weeks or even months to see conclusive test results (95% statistical confidence).

If you flip a coin four times and get tails every time, do you conclude that flipping a coin isn't random?
Obvious answer: the more traffic, the better your results.

Is this a bigger priority than increasing traffic?