Playbook
Email A/B Testing Guide
Most A/B tests in email marketing produce no useful result. Not because testing is broken, but because the test was too small, too vague, or measuring the wrong thing. Here's how to run tests that actually change what you do next.
This is part of the Supercharged Email playbook, built to help you get better results without guesswork.
TL;DR
- Only test things that will change what you do next.
- One variable at a time. Always.
- Subject tests need ~1,000 per variant. Click tests need ~5,000.
- A 1% lift is almost always noise. Want 10%+ before calling a winner.
- Log every result in a single spreadsheet.
Do this today
- Pick one variable to test on your next send
- Write down the decision the result will change
- Confirm your sample size is large enough
- Run the test and log the result
- Plan the next test using what you learned
Frequently asked
How long should I run a test before calling it?+
Until you hit the sample size, not a clock. Subject tests usually resolve in 2–4 hours; click tests can take 24–48 hours.
Can I test on a small list?+
Yes, but pool results across multiple sends. One test on 500 people is too noisy to act on alone.
What's a winning threshold?+
10%+ relative lift with a meaningful sample. Anything less, pick whichever is easier to keep doing.
Should I A/B test images?+
Rarely. Image differences are swamped by other variables. Test copy first.
What if my test is inconclusive?+
That's a result too, the change doesn't matter, move on to a bigger lever.
Put these strategies into action
Start free with Constant Contact
Opens Constant Contact's free trial, where you'll see our partner name, Synergy.The account is yours and the price is the same. We may earn a commission.