When I read Tony Wagner’s latest blog post for Only Influencers (OI), ‘Email Quality Score: Your Secret Weapon for Post-MPP Success’ (disclosures: he works for Strategic America; I am the General Manager of OI), I mostly loved it.
I loved that he gave credit where credit was due for the idea (shout out to Lucas Chevillard at Litmus!). I love that he was codifying something I’ve looked at with my own clients, but never gave a formal name. I love the clarity with which he presented it.
But I didn’t love the name. More on that shortly.
How to Calculate Email Quality Score
From Tony’s article, here’s the formula to calculate the Email Quality Score (EQS):
Email Quality Score = 1 – (Unsubscribes / Unique Clicks)
How to read it:
- A perfect score (no unsubscribes) is 1.00
- A positive score (>0.00) means that there are more unique clicks than there are unsubscribes
- A score of 0.00 means that the number of unsubscribes is equal to the number of unique
- A negative score (<0.00) means that there are more unsubscribes than there are unique clicks
Here’s a sample calculation:
Data:
- Unique Clicks: 34,120
- Unsubscribes: 3,300
EQS = 1 – (Unsubscribes / Unique Clicks)
EQS = 1 – (3,300 / 34,120)
EQS = 1 – (0.0967175)
EQS = +0.90
This (+0.90) would be considered a pretty good EQS.
Why I Take Issue with the Name
Using misleading names for metrics is nothing new in the email world. For instance, many people and platforms refer to the quantity of email sent which doesn’t bounce as a ‘delivery rate.’ But we all know that there are times when email that doesn’t bounce also doesn’t reach the inbox. ‘Non-bounce send’ or ‘assumed delivered’ would be a more truthful name for this.
And when people refer to ‘click rate’ I’ve always got to ask whether they’re referring to a unique click-through rate (CTR), some metric related to total clicks (which can be useful at times), or click-to-open rate (CTOR). Sometimes they don’t know, which is a whole other issue (here’s a primer, in case you’re uncertain about the different click metrics).
I think EQS falls into the same category; my issue is with the term ‘quality.’
Unsubscribes are an important negative signal – I love the work that Tom Wozniak and Optizmo have done analyzing and reporting on this.
But in most cases, unsubscribes are few and far between. Most people who no longer find your email content of interest will just stop opening your email messages. If they’re really annoyed with your content, they’ll report you as spam. But if their perception of your content is just ‘meh,’ they’ll just start ignoring you.
Over the years I’ve often had people present low unsubscribes to me as a positive, as in ‘they aren’t really opening and clicking much, but they haven’t unsubscribed!’ I still love to use relationship metaphors for email – this is basically ghosting, right? They haven’t officially broken up with you but they’re no longer responding when you reach out to them.
So back to EQS.
I agree that if you have a negative EQS score, that’s bad.
But a positive EQS score isn’t necessarily good.
A Client Case Study to Illustrate
Here’s an excerpt from a monthly report I do for a client (I’ve isolated some key sends to illustrate my points). Here’s a brief ‘legend’ for what you’re looking at:
- I’ve sorted the sends highest to lowest by click-through rate (CTR)
- The sends highlighted in darker green have a CTR in the top quartile of all sends
- The sends highlighted in lighter green have an above-median-but-not-top-quartile CTR
- The sends highlighted in paler peach have a below-median-but-not-bottom-quartile CTR
- The sends highlighted in darker peach have a CTR in the bottom quartile of all sends
- The cells highlighted in pink are ‘pink flags’ – things that may be problematic which we need to investigate, like:
- A high unsubscribe rate. My rule of thumb has always been that if you are losing more than 0.5% of your recipients via unsubscribe on an email send there’s an issue you need to look into (could be content, could be list source, could be something else).
- A high spam complaint rate. Recent Yahoogle guidelines codified what many of us have used as a rule of thumb for years:
- a spam complaint rate above 0.10% puts you at higher risk of deliverability issues
- a spam complaint rate above 0.30% makes deliverability issues almost unavoidable
- A negative EQS. This is bad, as we discussed earlier
- A positive but low EQS. Not negative, but still not great.
Let’s start at the top.
The first send (first row beneath the column labels) has an EQS of 0.81 (right-most column), which is positive and actually pretty close to a perfect 1.00. And this send is highlighted in darker green, meaning that the CTR is in the top quartile of all sends. All good, right?
But look at the unsubscribe rate – it’s 0.73%. So that’s a pink flag for me, even if the EQS looks good thanks to a healthy number of unique clicks (see the click-through rate of 3.8%).
Now let’s look at the three EQSs highlighted in bright pink (right-most column) toward the top of the table. They’re all negative, which is bad, right? But two of these have CTRs which are above the median (note the light green shading) – including the one with the incredibly low EQS of -4.00.
All of these sends are to prospects, with the goal of generating meetings for the sales reps. At least one of these went to a list of visitors to a booth they had at a conference; I’m not sure about the other two. It’s not unusual to see high unsubscribe rates on lists like this. Often booths offer swag or the chance to win something in return for an email address (or, more likely today, a badge scan). So while it’s a bummer that the unsubscribe rates and EQSs are so high, it’s not as big an issue as it would be with an email sent to long-time subscribers.
Would we like to see if we can increase or at least maintain the CTR while we decrease the unsubscribe rate? Yes, of course. But it’s not as alarming as the EQS would lead you to believe.
And the other EQS cells highlighted in pink? Oddly enough, these are all webinar emails, In fact, most of the sends that are highlighted in paler peach are webinar emails. You can see that the CTRs are pretty low, 0.5% or less, which is unusual for an offer of free content which is educational, not just promotional. When we looked into this, it appears the issue is a mix of:
- Hidden redundancy – the subject line changes for each send, but the body copy remains exactly the same. So subscribers who opened it the first time and weren’t interested continue to receive basically the same email again and again, but they don’t realize this until they open it, since the subject line changes significantly (and isn’t always transparent about the email content). That’s a reason to unsubscribe.
- Over-mailing – I know, I know. Sending more email often results in more conversions and/or revenue initially. But after a while it’s a bell curve and your bottom-line results start to decrease. In this case, the standard webinar series they’re using has a lot more efforts than what I traditionally recommend. As the series continues, the CTR goes down and the unsubscribes increase. We’ll be diving in to address this in the near future.
I agree there’s an issue with the EQS -4.00 send (it was a low quantity send, probably a cold outreach, as its goal was to generate meetings for the sales reps).
Wrapping Up
So it’s not the formula I take issue with, it’s the name. In my head I can hear future prospects espousing the health of their email programs and attributing it to having high EQSs. Which basically just means that very few people are unsubscribing. It doesn’t mean that their conversions and, if appropriate, revenue, are positively contributing to the bottom-line of their organization. Which is what I really care about.
So do use the EQS calculation as an additional metric to evaluate your email marketing campaigns and let me know how it goes – just don’t call it an ‘Email Quality Score.’ Maybe we could rename it the ‘Unsub-to-Click-Rate’ or UTCR… Let me know your thoughts on the name!
Until next time, be safe, stay well,
jj
Photo by Shannon Baldwin on Unsplash
