A news site running a headline test shows a small share of its readers one version of a page and another share a different one. It then tracks what each group does and keeps the version that performs better, often while the story is still live. The headline you read may be the one that won.
more
How big the habit is
An outside analysis in 2021 found that The New York Times A/B tests 29% of its headlines, Nieman Lab reported.
The practice spread from Google and Amazon. By 2014 Digiday called it table stakes for publishers chasing audience growth. Optimizely, one of the firms selling the tools, then had more than 8,000 customers, and about a third of them were media companies. CNN, CBS and Forbes were among them.
Most publishers started by testing versions of their adverts, according to Helen Phung of Optimizely. The editorial side came later.
What a single word can do
The Dodo used testing to pick headlines for social media. It found that adding “epic” to a story about acrobatic dogs doubled the click-throughs and shares from Facebook.
BuzzFeed built its own tool so editors could test headlines and thumbnail images, said Dao Nguyen, its publisher. The site also tested design. When it gave its share buttons a flatter colour, sharing rose 20%. Readers who arrived from Pinterest saw fewer share buttons, and pinning rose even after the lost Facebook and Twitter shares were counted.
Upworthy, a viral site then two years old, built its business on the method. One of its tests found that recommendation boxes placed to the right of an article lifted social sharing by 28%.
The catch
Testing rewards small changes, so it pushes a site towards steady tweaks rather than bold redesigns, Digiday noted. Nguyen warned against assuming that one result holds across a whole site. Her line on data was that BuzzFeed uses it to inform decisions, not to dictate them.
