Testing Email Campaigns

Sections of this topic

    Never take anyone’s advice on how to run an email campaign (except this advice from me)!

    Instead, take the good ideas you hear from others and test them within your campaign. Testing online is similar to testing in direct mail (or with your favorite recipe, for that matter).

    Test one subject line against another, or one landing page against another. Start with your favorite (or the one the boss likes most) or the one that did the best last time. That’s your “control.” Then find one variable that might improve results, and that new message is your “test.”

    Easier said than done, you say? True. Most basic email tools do not have sophisticated testing capabilities. However, you can get some pretty reliable results with a few tricks.

    If your list is less than 10,000 email addresses, the best you can do is to split the list into two groups and send one of the halves the test email and the other the control.

    If the list is larger, you can create two test groups of 5,000 email addresses each – one for the control, and the other for the test. Within 24 hours, you should know which list did best, and send that message to the rest of the list.

    How do you create valid test segments? The only one sure data element you have for each address is the email address itself. Most email tools will let you create a segment of a list based on the content of the email address.

    So, if you create a segment that includes every email address with an ‘s’ or ‘t’ or ‘k’ in it, you might end up with 5,000 names. Avoid using a, o, l, c, o, and m, or else you’ll get every AOL.Com address in one segment or the other. That’s never a good idea, since AOL subscribers might very well perform, as a group, differently than others. For the same reason, avoid ‘y’ (Yahoo!) and ‘g’ (Gmail) as well as any other letters that might skew your results.

    Now you can test different subject lines, long copy against short copy, different designs, anything – and get pretty valid results.

    How do you test landing pages? Send the exact same email message to both groups, but in one message, change the link so it goes to the test landing page.

    As an added way of ensuring valid results, and to maximize your overall response, see if you can swap the segments: mail message B to those on list A who didn’t respond, and message A to those on list B who didn’t respond. If your lists are equal, whatever message did the best last time should do better this time. And, you’ll gain more responses from people who didn’t see or respond to the message the first time.

    Do your test, tell us your results, and ask us your additional testing questions!

    We’re taking a break until after the New Year.
    Be back on January 4, 2011

    Best wishes to you and yours from Hank,
    Natalie, Andrew & Rick. ☺


    Rick Christ has been helping nonprofit organizations use the internet for fundraising, communications and advocacy since 2009, and has been a frequent writer on the subject. He delights in your questions and arguments. Please contact him at: RChrist@Amergent.com or at his LinkedIn Page