Back in late January 2023, when Chat-GPT was just beginning to take off, an acquaintance invited me to try his company’s new AI product, which was in Beta. You input information about the content of the email and it provides subject lines and preheader texts for your email message.  

We had an Only Influencers (OI) webinar that we were pulling together for late March, so I decided to input information on that and see what subject lines and preheader text I got back.

Here’s the information I provided:

Free Webinar, Tuesday, March 21, 2023, 12:00 Noon to 1:00 PM ET Title: Testing to Boost Your Email Marketing Performance in an Economic Downturn Hosted by the OI Metrics Committee

Yeah, not super detailed. It was early in the process and we didn’t really have a webinar description yet (and we have since changed the title). But anyway…

Sidenote: This is a real webinar, feel free to register if you’d like to attend.

The AI asked me to choose a ‘style’ from a dropdown list of options – so I chose “Direct Response” since the goal of the email is registrations.

Here’s what I got back:

Subject: Last Chance: Unlock Email Marketing Success Preview Text: See how our experts can help you unlock email marketing success with a free webinar. Tuesday, March 21, 12-1pm ET. 2. Subject: Get Ready for an Economic Boost Preview Text: Join us for a free webinar to learn how testing can help you achieve email marketing success in an economic downturn. Tuesday, March 21, 12-1pm ET. 3. Subject: Beat the Downturn: Learn Email Testing Preview Text: Don’t let the economy slow you down. Join us for a free webinar to learn how email testing can help you beat the downturn. Tuesday, March 21, 12-1pm ET. 4. Subject: Harness the Power of Testing Preview Text: Learn how to use testing to your advantage. Join us for a free webinar to learn more. Tuesday, March 21, 12-1pm ET. 5. Subject: Get Your Edge with Email Testing Preview Text: Join the OI Metrics Committee for a free webinar to learn how email testing can give you an edge. Tuesday, March 21, 12-1pm ET.

So, readers, what do you think?

Are all of these worth a test? Are any of these work a test?

Here’s my take, which I shared with my acquaintance.

“Thanks for sharing the link! Interesting… I think the subject lines are a bit too generic.”

All of the subject lines are benefit-oriented, which is good.

The first two don’t mention testing, which I found odd, since that is what the webinar is about. These could really work for just about any product or service that promises to help you improve your email marketing.

The first one also said “Last Chance” – which I found odd, as I might use that on the last email in the series but would certainly not use it in the first of the series.

The second and fourth subject lines didn’t mention email, which again seemed odd. These benefit-oriented statements could apply to, well, just about anything.

“And the preheaders are too long—and the first 25 characters (all you can guarantee will be seen with truncation) didn’t include the key information.”

It’s not that a preheader (or a subject line, for that matter) should be short. It’s just that these preheaders were excessively long.

And more importantly – while the key information I had provided is there, it’s at the end of the preheader. Which means that it’s unlikely to be seen, as both subject lines and preheaders are usually truncated in the inbox. You can really only guarantee that the first 25 characters can be seen, so you want to be sure that your key messages are in there.

One thing the AI did well was not making the subject line and preheader text redundant. They didn’t say the same thing. I wish that the subject lines were more ‘meaty’ — and that the preheaders weren’t long sentences (most of my subject lines and preheader text copy is phrases, not full sentences). But at least the AI didn’t just repeat the same thing twice. That’s a mistake I still see some human email marketers making.

“Just my $0.02 after one try, but will keep playing with it!”

My acquaintance didn’t seem daunted by my feedback.

He thanked me for sharing my exact inputs and outputs, then said:

“I hadn’t yet instructed it to try to place the most important pieces of info at the beginning – that’s a great point; I can integrate that into the prompts and see how it handles it. For some reason, it keeps selectively ignoring my instruction on preview text length. We’ll get there!”

Thoughts

I have industry friends who feel they are having great success with some of the AI-driven subject line tools out there. That’s great! This doesn’t make me think any worse of any of them.

But it does give me pause.

I’ve spoken with marketers who seem to believe that AI is smarter than they are at creating subject lines. I actually spoke to a marketer today who took at look at the results of a recent AI vs. human subject line test she had done – and was genuinely shocked to see that her subject line had driven more sales than the one the AI had written.

My take is this. The AI is only as smart as the people programming it. If they know how to write great subject lines and preheader text, then maybe they can use what they know to program the AI to do the same.

But if they aren’t in the weeds on truncation and other details that can make or break subject lines and preheader text, then how can they program the AI to know these things. And can the AI on its own figure those things out? I guess it would if the results of each test were provided to the AI so it could learn – or would that be machine-learning?

Use AI, but don’t automatically assume that the AI – or the people programming it — is smarter in your area of core competency than you are.

Testing the Subject Lines and Preheader Text the AI Provided

Did I test what the AI suggested?

I really struggled with this. But in the end, I decided not to.

Based on what I know about crafting effective subject lines and preheader text, I didn’t feel that any of those provided by this AI were worth testing. I just don’t see them effectively driving registrations for the webinar.

But stay tuned… there’s more to this story…

What’s your experience been with AI subject lines? Let me know…