Generative AI can take on simple writing tasks and produce readable copy. But just like any copywriter, it needs expert oversight to ensure its writing is fit for purpose.
No writer is infallible. But ongoing issues like hallucinations and outdated training data mean you need to check especially carefully before staking your reputation on AI – as Google found to its cost.
So if you’re one of the growing number of marketers tempted to publish AI-generated content, it’s important to have a clear, repeatable review process – and enforce it rigorously. (Studies at Chicago’s Quest Academy revealed that the greater the role AI had in producing a piece of work, the less engaged the humans involved were in checking it.)
Also, there’s a lot more involved in an effective piece of copy than just reading well.
At Radix, we use a 16-point checklist to assess each piece of our writers’ work before we send it to a client; it keeps our quality high, and it’s a great way for our writers to learn from each other. If you’re planning to experiment with ChatGPT or its competitors, it could come in pretty handy (you can get a copy of your own right here).
Here, I’ve whittled down our content quality checklist to the five big questions to ask when you’re reviewing generative AI outputs.
Question 1: Is it accurate?
The most basic requirement of any piece of B2B content is that it’s correct.
It’s tempting to assume that computer-generated output will at least be factually accurate, but that’s not always the case. Although new-gen algorithms like GPT-4 are ironing out some of the creases, generative AI is still prone to making up “facts” that have no basis in (or even contradict) the training data. So, you’ll need to go through its copy with a fine-toothed comb to make sure it’s not invented things.
You’ll also want to double-check grammar and spelling. Despite the hype, AI isn’t yet intelligent in any meaningful sense; ChatGPT doesn’t understand what it’s writing, it just mimics things it’s read. AI outputs are only as good as the inputs – which is why it also repeats common grammatical errors and widely misused phrases.
Question 2: Is it clear?
I’ve never yet heard about a B2B decision-maker who has tonnes of spare time to read marketing content. That means your copy needs to make a compelling, logical argument – and do it in prose that’s quick and easy to read.
At first glance, AI-generated copy often appears to make a plausible case. But, as we discovered in our ChatGPT copy experiment, the logic of the argument doesn’t always stand up to scrutiny.
A combination of not wanting to admit it doesn’t know the answer, not understanding the things it writes, and a tendency towards verbose and repetitive prose can lead AI to generate meandering arguments that end up in a logical tangle.
And because so much B2B content is needlessly complex, AI risks mimicking this flaw too. As Doug Kessler put it: “The point is not whether Generative AI is good enough today. The point is that most B2B writing isn’t.”
Question 3: Does it speak with authority?
Not only does AI not understand what it’s writing, it also doesn’t understand the audience it’s writing for.
That’s a problem, because it can undermine the authority of your piece. It’s pretty clear when you’re reading something if the writer doesn’t have a strong grasp of the topic and understand which aspects are most relevant to you. That’s why professional copywriters interview subject matter experts and work hard to understand the critical pain points and priorities – and the existing knowledge level – of the target audience.
Pre-trained models like GPT also rely on old data, so they’re not up to speed with the latest developments in fast-moving industries, making it very difficult for them to sound authoritative.
Another thing that instantly torpedoes authority in B2B marketing copy (whether human or AI-written) is cliché. And that’s a problem for large language models like ChatGPT, because they’ve been fed a solid diet of the cliché and banality that constitutes most writing on the web. A marketer recently told me that their agency is using ChatGPT to help rule out hackneyed phrases in headlines; AI writes the crap, so they don’t have to.
So, when you’re reviewing AI-generated copy, be ready to make heavy edits to give the writing the sense of authority, credibility, and authenticity it needs to be effective.
Question 4: Does it demonstrate empathy?
Again, AI is fighting a losing battle here. With no understating of what they write or the audience they’re writing for, and no lived experience to draw on, machines are a long way from being able to empathise with human readers. And without the human connection that empathy brings – that sense of a shared understanding of the world – marketing copy sounds very hollow and won’t engage its audience.
Another thing to watch out for is that AI can’t apply critical thought to assess the validity, authority, and relevance of its sources. That absence of critical thinking also means AI can’t understand that just because something has been said often, doesn’t mean it must be said again. Nothing displays a lack of empathy quite like telling people things they already know – or worse, things they don’t even care about.
Question 5: Is it insightful, engaging, and compelling?
It’s impossible to offer original insight if you’re just mimicking things you’ve read. So for thought leadership pieces, you’ll need to add that insight to the copy the AI generates, which will probably involve extensive rewriting.
Alternatively, you could try being very prescriptive in your prompt about the points you want to make. This might mean you need an interview with a subject matter expert before you set the AI to work. But, however you do it, you’ll need to do the legwork to ensure your reader gets some original, unique value from your piece – that’s not something ChatGPT can help you with.
Finally, remember that AI’s tendency to use repetitive phrasing and cliché means its writing can be quite dull to read. B2B marketing copy must be engaging to achieve its objectives, so you’ll need to spend some time polishing whatever the AI comes up with to give it a bit more sparkle.
Progress is good – but check, check, and check again
Generative AI is getting better at completing low-value writing tasks, like simple emails (although they still need to be reviewed and edited by humans). But it’s nowhere close to delivering high-value content like lively, well-researched ebooks and white papers that put an original spin on a topic, or video scripts that strike an emotional chord with the audience.
In the future, it’s likely that professional copywriters will use AI tools to do the heavy lifting before applying their craft to create content that helps marketers achieve their goals. But until then, use our content quality checklist to help ensure whoever (or whatever) writes your copy is doing the best job possible.
