Email subject lines are stressful. Get them right, and you’re 80% on your way to a successful campaign. Botch them, and you’re sunk.
I used 80% for a reason. David Ogilvy, granddaddy of copywriting and direct response marketing, famously said, “When you have written your headline, you have spent 80 cents out of your dollar.” Ogilvy was talking about headlines, but the 80 cents still adds up. Subject lines are headlines for emails.
Subject lines just might be even more important than headlines. When people scan headlines in a magazine they’re looking for something that interests them. It’s a relatively friendly selection process.
Not so in the inbox. Your subscribers are not browsing a magazine to pass time. They’re at work or tearing from errand to errand. They may even be driving. They’re scanning their emails to find which ones to kill (ahem, "delete"). They’re looking for a reason to delete your precious little message.
It gets worse. 69% of subscribers will mark an email as spam based on the subject line, according to Convince and Convert. A bad subject line doesn't just risk your message getting deleted. It might cause you to never get another message through to that subscriber again.
Like Scheherazade of The One Thousand and One Nights, your email subject line needs to capture and hold a reader’s interest, if you want to live to tell another day.
So it’s no wonder we get a little stressed about writing subject lines. They’re a matter of life and death… for emails.
If you’re going to test anything, test the subject line
UK agency Howling Mad did an interesting survey of email marketers earlier this year. They wanted to know which elements of an email marketers felt made the most difference in their campaigns.
Howling Mad surveyed 304 email marketers. 60% of them were US-based, 25% were UK-based. The remaining 15% came from other parts of the world. The survey queried both agency and in-house marketers. 55% of those surveyed worked for brands. 45% of worked for agencies.
Here’s what they found:
For more information about what to split-test in your campaigns, see our recent post, 52 Email Split Testing Ideas, or watch our webinar, Improve Email Responses with Split Testing.
5 tools to pre-test your email subject lines
What if you’ve only got 10 minutes to craft a subject line? You can’t test, and you can’t spend an hour pouring over subject line testing data. What then?
Well, try playing around with any one of these free email subject line testing tools. They all use data from major subject line studies. They’ll give you quick feedback about which subject line to run with.
1) Touchstone Subject Line tool
Touchstone pulls data from six different sectors: Your database, B2B, Media & Entertainment, Retailers, Travel, Financial Services, Consumer Services, Telecom & Tech and Price Comparison. Based on emails from those areas, it will show expected metrics like opens, clicks and delivery rates for whatever email subject line you give it.
Touchstone will also score your email subject line as better or worse than the average email from that industry. Here’s what results look like:
If you take only one thing away from this post, try playing around with this tool for a few minutes before you send your emails. There’s a good chance you could increase your results by 10-20%.
2) Litmus’s Subject line checker
This email subject line tool shows a preview of how your subject line looks in various email clients. It won’t give you statistics on which subject line is best, but it's worth a quick check before you send your next campaign.
If nothing else, Litmus’s tool will show you where your subject line could get cut off. It will also remind you how visible the pre-header text of your email is. You are paying attention to your pre-header text, right?
3) Adestra’s Email Subject Line Keyword Checker
This tool uses data from Adestra’s popular email subject line report. You can download the 2013 Adestra Subject Line Analysis Report from the bottom of the page this tool uses. While the tool won’t assess a full subject line, it’s still helpful for optimizing one, especially the first word of the subject line.
The Adestra tool is also sophisticated enough to pull from different sectors, so if you’re in B2B Publishing, B2C Publishing, B2B Events, B2C events, Retail/Ecommerce or Not for Profit / Charities, you can get results suited to your audience.
According to the tool's website, “Results from this tool are based on 1 billion+ email messages that have been sent and tracked via SubjectLine.com's partners and clients. Over 400 unique rules are tested that evaluate filtering and deliverability issues as well as marketing and performance related concerns.” This is what the results looked like after I gave it a whirl:
5) CoSchedule’s Headline Analyzer
Maybe it’s cheating to add a headline analyzer to this list, but this tool from CoSchedule is probably the online tool I use most. In a snap second it will give you a score for your headline and some analysis of each word. Any headline that scores above 70 is good enough to use. I owe at least part of the success of any post I write to this tool. It chose the headline for this post.
Recent studies about which email subject lines work best
I’ve already mentioned one of the best-known email subject line studies, but Adestra’s analysis is not the only subject line report in town. However, if you only have time to read one huge email subject line report, it’s worthwhile, even if it is more than a year old. Here’s just a sip of what’s in the report:
Adestra also did a webinar to explain and highlight their findings. If you’ve got the time, it’s worth a listen.
Alchemy Worx just published some new subject line analysis, largely based on data gleaned from its email subject line tool, mentioned above. The Alchemy Worx data was drawn from 21 billion marketing emails sent by 2,500 brands. It says words like ‘content’, ‘offer’, ‘benefits’ and ‘premium’ are most likely to increase the open rates. The words ‘Friday’, ‘Monday’ and ‘double’ are most likely to suppress open rates.
Alchemy’s data also looked into using special characters in subject lines. These were all the rage a few years ago, but many marketers are still using them. They still work, too.
The report also broke out best and worst words for subject lines by industry:
A few resources for subject line ideas
Suffering from data overload? Subject line research studies can do that. If you just want a list of fresh ideas for subject lines, try these resources:
Marisa Murgatroyd wrote this clever post right around Valentine’s Day. As she says in the post, “A word of warning: if you end up in an email exchange with a copywriter, you may end up married… ”
– AutomizeIt.com has a great lead magnet, “49 Killer Subject Lines.” They don’t share click-through or testing information, but it should still spark a few ideas.
– 557 Email Subject Line Hacks to Get You Noticed in the Inbox from the Daily Egg blog. If 557 ideas isn’t enough, I’m not sure what would be.
– The 17 Best Email Subject Lines for Increasing Open Rates [+ Video]. Some of the subject lines may sound a little “late night TV”, but these formulas have worked a thousand times over. Who else wants to increase their open rates?
How to split-test your email subject lines in Pinpointe
While tools are helpful and research studies are interesting, unfortunately they’re all too general. None of them applies specifically to your list. The only way to really be sure you’ve got the perfect subject line is to test.
Fortunately, setting up a test in your Pinpointe account is easy. Once you know how to do it, it takes about ten minutes to set up. There’s a detailed explanation, with screenshots, for how to set up a split-test in our recent post, 52 Email Split Testing Ideas. Scroll about two-thirds down the page to see the instructions.
I’ve borrowed the first screenshot from that explanation here, just for some context.
One tip: Before you set up your subject line test, you’ll need to have already created the two email campaigns you want to test. Those email messages (aka “campaigns”) should be identical except for their subject lines. Usually I just create the first email complete with subject line, then duplicate it. I’ll change the subject line on the second email, and voila – I’m ready to set up the test.
How to send a statistically valid a/b split test to part of your list, then roll out with the winning subject line to the remainder of your list
Split-testing has a serious shortcoming. Splitting your list in half and sending one version to each half of your list is fine, but it’s not a perfect solution. Why not? Because you’re never going to use that subject line again. Sure, you’ve gained some insight on which subject line your audience liked best. That’s good. You’ve also optimized your subject line for at least half of your list for this mailing. That’s good, too. It’s definitely better than not optimizing any part of your list.
But what if you could send out just enough emails to know which subject line would do best? Then you could send out the rest of your email messages – with the winning headline – to the remainder of your list. That would be pretty cool, huh?
That is exactly what the people with big lists, who make big sales from their lists, do. If you want to play with the big kids, you’ll mail your two test subject lines to two small segments of your list. Once you’ve got statistically valid results, you’ll mail the winning subject line (or email creative, or offer) to the remaining subscribers.
This is a great technique, but it’s got two major snags:
- You need statistically valid results from those test segments
- You need a list big enough to give you statistically valid results
This is where things get dicey. Dicey, but not impossible. To work through this we’ll have to delve into a bit of statistics. Stay with me, math-phobes. I promise to keep this simple.
As you probably know, you want statistically valid results in all your split-tests. Otherwise you could be choosing a winning email message that’s not actually a winner – it just looks like a winner.
For example, if we flipped a coin three times and got three heads and no tails, that doesn’t mean the coin is heads on both sides. It just means we haven’t tested enough times to know for sure. We’d have to flip the coin another dozen or two dozen times to be sure. How many times we flipped the coin would be based on the confidence level we’ve chosen to apply to the test. An 80% confidence level would mean we were 80% confident in our test results.
Testing wonks know there’s actually no way to be 100.0% perfectly confident you’ve got a winner in any test. You’d have to flip the coin a long, long time. Even after thousands of flips, you might “only” be 99.9999999% confident. So we have to choose a level of confidence we can live with. For most marketers, that’s a 95% confidence level.
Each flip of that coin counts toward our sample size. If we want to test subject line A against subject line B, each email message is kind of like a flip of the coin.
Unfortunately, it gets more complex. If we’re declaring a winner based on, say, click-through rate, we have to acknowledge that not every email earns a click.
This is where sample size calculators come in. There are two really good ones: Optimizely's Sample Size Calculator and Evan Miller’s Sample Size Calculator. Use whichever one flips your coin, but I’m going to use Optimizely’s for this example.
In this screenshot, I’ve chosen a 1.5% “conversion rate”. My conversions are clicks. 1.5% is a reasonable click-through rate for unique clicks. I’m using Optimizely’s default 20% detectable effect. As their instructions state, this is the amount of change in conversion rate that I want this test to detect. If you decrease the detectable effect, your sample size gets much larger. For this example, when I dropped the detectable rate to 10%, my required sample size shot up to 112,092 emails per variation.
I’m also using Optimizely’s default 95% confidence level we talked about. And so… according to this I need a list size of 22,570 x 2 to even do a basic split-test with these settings. That’s a list of 45,140. That’s a pretty big list.
Let’s say I’m really lucky and get to work with a list that’s 100,000 subscribers. I’m going to mail email subject line A to the first list segment of 22,570 subscribers. I’ll also send an email with subject line B to another segment of 22,570 subscribers. I will be careful to pick these subscribers randomly, and to mail to them at the same time. If I didn’t, I might mess up the test because, as you know, mailing at different times can get you different results.
So I send my emails, wait 24 hours for results, and hopefully see a statistically valid winner emerge. Keep in mind that you may not see a clear winner every time… some tests end up being a wash. But we’re lucky this time: Subject line B crushes subject line A. So I happily send out emails to my remaining 54,860 subscribers, because 100,000 – 45,140 = 54,860. I make three times as many sales than if I had just sent subject line A. My boss is amazed. I get a raise. Bluebirds sing.
Except… I realize that if I had been really smart, I would have noticed that my Pinpointe account can do all this for me.
D’oh! I just needed to read the second option, “Autopilot”, in one of the screens I went through when I was setting up my test.
Rats. But I can test again next week.
