Scientific marketing

You’d be forgiven for thinking that evidence-based medicine had been common practice since the dark ages, but it might shock you to learn that the first randomized controlled trial (RCT) study was published in 1948! Of course attempts were made before then. The first reported clinical trial was conducted by James Lind of the Royal Navy, to establish a treatment for Scurvy. An early double-blind experiment, was the Nuremberg Salt Test in 1835, performed by a ‘society of truth-loving men’ to disprove Homeopathy. However modern randomized trials appeared first in Psychology in the 1880s, and recognized as standard in Medicine until the late 20th Century.

Despite the lack of life or death consequences, the Advertising industry was remarkably far ahead in this respect. Scientific Advertising by Hopkins, which covers techniques like split testing and coupon-based tracking, was published in 1923 – a quarter century earlier. Hopkins was building on work done by Stanley Resor at J. Walter Thompson, the first to commission an advertising effectiveness study using coupon response rates. In the Great Depression Era, fully a third of magazine ads featured a coupon, customers could return to get a free sample, or discounted price. By randomly assigning unique coupon codes to different versions of adcopy, advertising effectiveness could scientifically tested for what drove effectiveness. The ad industry immediately and eagerly adopted a surprising amount of scientific rigour, given the industry’s reputation today. Ogilvy wrote of the book "Nobody should be allowed to have anything to do with advertising until he has read this book seven times”.

In the digital age, we no longer have to rely on coupon clipping (though promo codes are still a common measurement technique), because we can use off-the-shelf software to randomly assign visitors to one version or another. Random assignment – the hallmark of scientific testing – isn’t always possible. For example some marketing channels, like Search Engine Optimization, have structural limitations: Google doesn’t offer A/B testing functionality for marketers hoping to test what helps their websites rank on the first page or number one spot. In these instances great progress has been made with Causal Inference, winner of the 2021 Nobel Prize in Economics, which offers techniques – Instrumental Variables, Regression Discontinuity, Natural Experiments – for synthesising the conditions of a controlled trial, where running one isn’t practical.

RCTs, or “A/B tests” as marketers know them, still sit at the top of the evidence pyramid – the gold standard for proving one thing caused another. To ensure your results are valid, change one variable at a time, carefully setting out your plan in advance. The group exposed should be a large enough for statistical significance, and randomly assigned the treatment or control. Preregistration of your hypothesis and methods ahead of the experiment, to avoid “twisting facts to suit theories instead of theories to suit facts”, as Sherlock Holmes warned of. Publish results openly to build trust in the methodology, and let others build on your work. It’s not always possible to maintain this level of rigour – we’re in business, not in a lab – but without documented proof of what causes success, an organization risks abandoning what's working, or failing to stop doing what’s not. Growth is a function of the number of experiments you run. As Ogilvy said: “Never stop testing and your advertising will never stop improving”.

About the Book 📕

← Back to Essays 📝