You’ve probably read about personalization and how it boosts the bottom-line performance of your email marketing. Or maybe you’ve read about how it can be creepy. Or maybe both.

Here’s a case study, the first of a few I’ll publish, about a journey into personalized email that one of my clients took. You’ll learn what they did, how it performed, and what we learned.

As always, don’t just take these case study test results and apply them to your own program – do your own testing to see how your email subscribers respond to personalization.

Case Study: Testing Personalization

Background

I’m a big believe in relevance, and personalization is one way to make your email marketing messages more relevant to your audience. At least that’s what the conventional wisdom says. But you better be sure you’re helping, not hurting, your email program before you go ‘all in’ on personalization.

Here’s a case study to get you started. Watch this blog for more.

Test Set-up

For our first personalization test, we took an existing email that had a product which could be purchased in bulk and personalized with a company name. This is a popular holiday gift for customers and/or employees and is also often used as conference swag.

Our control is what we usually send – it has an image of the product at the top; the product is imprinted with a generic company name and has a ‘your company name here’ starburst next to it.’

For the test version, we were able to digitally add the recipient’s company in place of the generic company name – which also allowed us to remove the starburst.   

Wireframes of both versions appears below.

For a test like this, I would typically do a simple 50/50 random split of the list. In this case, the numbers weren’t exactly even, but close enough. Each segment had at least 250,000 in it. This is well above my minimum cell size of 20,000, which almost always allows me to get statistically significant results. See the split below. 

As you can see, 46% of the list received the test/personalized version, while the balance (54%) received the control/not personalized version.

As always, we kept everything except the personalization the same, so we could get clear results.

Which version do you think won? Was it the test with the personalization? Or the control without the personalization?

I’ll give you a minute to think about it… then scroll down to get the answer.

Results

Here’s are the results!

That’s right. The personalized version bested the control.

How many of you guessed correctly?

Our key performance indicator (KPI) here is revenue-per-thousand-emails-sent (RPME), because we are looking to optimize revenue.

Note: We used RPME instead of revenue-per-email (RPE) because the RPE numbers were small. By increasing the magnitude it makes it easier to see variances – and the relative variance between the cells remains the same.

Here are the full results.

As you can see, the personalized test version generated nearly twice the RPME that the not personalized control version did. Nearly $20 versus just over $10 is a dramatic difference. What kind of a rock star would you be if you could double the revenue your email generated with a test?

Going deeper, we see that the test version equaled or bested the control in every metric we typically look at.

The reason the test version won is primarily because it bested the control in conversions (CR) – the personalized test version drove nearly 85% more orders than the non-personalized version. Note that conversion rate is calculated from the non-bounce send quantity, not from clicks.

There’s also a slight lift, nearly 7%, in average order value (AOV) for the personalized version.

Were we surprised by the result?

No. We had anticipated that personalizing the product imprint would boost performance.

But we were surprised by the magnitude of the win. We didn’t project that the personalization would nearly double revenue.

Take-aways

So, does this mean that you should always personalize your email messages?  

No. In fact watch this blog, because not every personalization test I’ve done recently has resulted in a list in performance.

This test just means that you should investigate personalization options for your own email program and test to see if boosts bottom-line performance.  

Give it a shot and let me know how it goes!

Be safe, stay well,