Finding magic words

Every experienced marketer can recall a time that tweaking just a few words of adcopy had an astounding impact on performance. Mark Twain called the difference between the almost right word and the right word “the difference between the lightning bug and the lightning”. It’s actually fairly difficult to determine which of the elements of an ad are responsible for its success, and which are holding it back. To prove one word performs better than another you need to run an experiment, but we can’t test each and every word. Instead we look for patterns in the data caused by natural differences in where ads were used.

Adcopy is regularly replicated across multiple campaigns, but rarely is it uniformly copied. Some ads will have been altered, edited, optimized. It’s these differences that can throw out clues as to what’s working. Much like in the process of genetic evolution, small mutations can add up to large differences in the survival of a species, and we can identify which genes gave them the advantage (or disadvantage). In adcopy analysis, in an account with enough variation, we can identify our top and bottom performers, and look to see what the pattern tells us about what words are work best.

To conduct this kind of analysis, it’s not enough to look at performance by ad: it’s not granular enough. To spot the patterns you have to deconstruct the ads into their component parts, or memes, to find which words or phrases contribute the most to sales. Knowing which these words are lets you double down on what works. If you don’t do this analysis you risk accidentally abandoning what’s working the next time you refresh your creative. Do this analysis enough times, for many different advertising accounts, and you’ll find a set of key words that are universally performant. Words like – ‘foolproof’, ‘ultimate’, ‘exclusive’ – what Leo Burnett called them “magic words”. Once you find them you can use them everywhere to immediately improve effectiveness.

Functionally the method for word extraction that’s the most practical is called “NGrams”. These are 1, 2, or 3+ word combinations or phrases appearing consecutively. First break the text up into NGrams, and then aggregate the performance data for the ads where those combinations appear. Calculate common KPIs (Key Performance Indicators) to get an idea of relative performance. The word ‘star’ may show a low ROI (Return on Investment), skewed downwards by ads for ‘2 star’ hotels, but the phrase ‘4 star’ might be a top performer. Edit out poor performers to eliminate wasted ad spend, and apply your best phrases liberally to dramatically improve performance.

It’s common practice to clean ‘stop words’ – common words like ‘the’, ‘is’, or ‘at’ that don’t add anything to the analysis – from the data first. Advanced techniques include lemmatization or stemming – grouping related words together – to make the analysis easier to interpret. These techniques are freely available open source NLP (Natural Language Processing) libraries such as NLTK, spaCy and Scikit-learn. It’s also possible to use more advanced AI models like GPT-3 to ‘unbundle’ the memes from a passage of text, by asking it to describe its characteristics or generate a comma-seperated list of tags.

You need variance to identify what’s correlated with performance. If your adcopy never changes, the data can’t tell you anything. An NGram that appears in every text isn’t helpful to analyze, nor is an NGram that only appears once very interesting. It’s the middle of the distribution where the gold is to be found. Words that are almost always used but weren’t on a few occasions. Phrases that have only been used a handful of times. Competing taglines which are alternated, and occasionally used together. When you identify these situations and there’s a noticeable performance difference, you’re onto something. It could still be correlation not causation, but the patterns you find today, are the hypotheses you should be testing tomorrow.

About the Book 📕

← Back to Essays 📝