|
It’s Anna from Selzy here — with the sudden (and wrong!) question. Do you want to learn more or earn more?
That’s the (very wrong) question I was hit with during an A/B testing debate. I usually test on 20% of my list and let Selzy send the winner to the rest. My teammates go full 50/50 — all for long-term insights. So why and what’s better? If you’re just starting out, this split of splits can get confusing. So I asked Selzy email team to break it down.
First of all, what’s the difference?
|
|
•
|
50/50 split: Half of your list gets version A, half gets version B (you don’t wait for a winner, instead analyzing the results later). This split gives you larger groups, with more opens, clicks, and conversions per version — the margin of error is lower and the comparison is more reliable.
|
|
|
•
|
20% test group (10% A, 10% B): Only a small portion of your list is used for testing. Once a winner is determined (usually based on opens or clicks in a short amount of time), the remaining 80% of your list gets the better-performing version. That means fewer data points and more noise — the winner might be determined too early, every click (even a random one) counts, so confidence is lower here.
|
|
|
Now, when to use them? We have checklists! Save it, keep it in your phone — one quick look, if you are in doubt, and all set!
First, 50/50:
|
Whichever one you choose, Selzy has A/B test feature, by the way (on a Lite plan and beyond). After all, you can’t split the quality or the right answer — because with the right strategy, you really can learn more and earn more. Just don’t forget to analyze your A/B test results correctly (What? We are all learning here!) |
|
