Help Center › Email Marketing › A/B subject testing

A/B subject testing

Test two subject lines side by side and let Asan automatically send the winner to the rest of your audience.

How A/B testing works

When you enable A/B testing on a campaign, Asan splits your audience into three groups: a test group for Variant A, a test group for Variant B, and a holdback group. After a set wait period, Asan compares performance and automatically sends the winning subject line to the holdback group.

A/B testing only changes the subject line. The email body remains identical for both variants, so you are measuring the true impact of subject line wording on open rates.

Setting up an A/B test

Create or open a campaign in the campaign editor.
When you are ready to send, click A/B Test Subject instead of sending directly.
Your original subject line becomes Variant A. Enter a second subject line as Variant B.
Choose a split percentage (10% to 50%) — this is the total portion of your audience used for testing.
Choose a wait duration (2 to 24 hours) — how long Asan waits before picking the winner.
Click Start Test to begin sending.

How the split works

The split percentage determines how many leads receive the test emails. The rest are held back for the winning variant.

Split %	Variant A	Variant B	Holdback	Example (1,000 leads)
10%	5%	5%	90%	50 A, 50 B, 900 holdback
20%	10%	10%	80%	100 A, 100 B, 800 holdback
30%	15%	15%	70%	150 A, 150 B, 700 holdback
50%	25%	25%	50%	250 A, 250 B, 500 holdback

A 20% split is a good default. It gives each variant enough recipients for meaningful data while reserving most of your audience for the winner.

How the winner is chosen

After the wait duration expires, Asan compares the unique open rate of Variant A against Variant B. The variant with the higher open rate wins, and its subject line is automatically sent to the holdback group.

If both variants have identical open rates, Variant A is used as the default winner.

You receive a notification when the winner is selected. The holdback group begins receiving the winning subject line immediately after the wait period ends.

Viewing results

Once the test completes, you receive a notification with the winning variant. You can also view the full comparison in Campaign Analytics:

Open rate per variant — side-by-side comparison
Click-through rate per variant — deeper engagement comparison
Winner badge — clearly marks which variant won
Holdback results — performance of the winning subject line on the larger audience

Tips for better tests

Change one element at a time. Test a question vs. a statement, or including a number vs. not. Changing multiple things makes it impossible to know what drove the difference.
Keep both variants roughly the same length so length alone does not bias the result.
Use a longer wait duration (12-24h) if your audience spans multiple time zones.
Review past A/B results to build a library of what works for your specific audience.

Frequently asked

Is there a minimum list size for A/B testing?+

There is no hard minimum, but A/B testing works best with at least 200 recipients. With smaller lists, the test groups may be too small to produce statistically meaningful differences in open rates.

What metric determines the winner?+

The winner is determined by unique open rate — the percentage of recipients who opened the email at least once. This is the most reliable indicator of subject line effectiveness.

Can I cancel an A/B test while it is running?+

Yes. Go to the campaign and click Cancel Test. The test emails already sent to the A and B groups cannot be recalled, but the holdback group will not receive any email. You can then choose to send to the holdback group manually with either subject line.

What happens if it is a tie?+

If both variants have exactly the same open rate after the wait period, Variant A is used as the default winner and sent to the holdback group. You can view the tie result in Campaign Analytics.

Was this article helpful?

Thanks — we'll use this to make the docs better.

Last updated 16 Jun 2026 · Applies to: Asan CRM