Which Headlines Attract Most Readers?

Times Insider delivers behind-the-scenes insights from The New York Times. In this piece, Mark Bulik, a senior editor, writes about recent tests designed to see what kind of headlines attract most readers.

I’d like to think we haven’t started any domestic spats. I’d like to think there’s never been an occasion when two partners were sitting across from each other, looking at their laptops, with one saying, “Hey, did you see this Times headline about Trump getting $2 billion worth of free publicity?” and the other replying, “You’re crazy. It doesn’t say anything at all about $2 billion.”

I’d like to think that, but I could well be wrong — because they could both be right.

In one effort to increase readership, The Times is using a tool that allows us to simultaneously present two different headlines for the same article on its home page. Half of readers on the page see one headline; half see the other. The test measures the difference in readers clicking on the article and lets us know if the numbers are statistically significant. If so, the winning headline goes on the home page for all readers.

And so, for a short while on March 15, one reader might have seen this:

$2 Billion Worth of Free Media for Trump

While another saw this:

Measuring Trump’s Media Dominance

Any guesses on which won the test, and by how much?

The top one got nearly three times as many readers, which underlines the crucial role of headlines in the digital age.

A story might be 1,000 words long, but tweaking the tiny handful of words that promoted this one on our home page gave us 297 percent more readers.

In other cases, headline tests have increased readership by an order of magnitude.

When this:

Soul-Searching in Baltimore, a Year After Freddie Gray’s Death

was paired against this:

Baltimore After Freddie Gray: The ‘Mind-Set Has Changed’

The test showed a 1,677 percent increase in readership for the second one.

To some degree, there’s nothing new about changing headlines. Editors regularly tweak them in print for any number of reasons — updates, greater clarity, a change in the layout. The difference with digital media is that there is no need to wait for the next edition, and more crucially, it’s possible to see which headline attracts more readers.

We’ve learned a number of lessons from months of testing, but the most important one is pretty obvious — clear, powerful words and a conversational tone make a big difference.

Can you tell which of the headlines below resulted in a tenfold increase in readers?

Is Everything Wrestling?

It’s Not Just Wrestling That’s Fake. It’s the World.

It was the conversational two-sentence version, with “fake.”

The Times generally uses the tests when a story isn’t performing as well as expected, and we have a few rules. Both headline options have to meet our standards; we don’t test clickbait versions because no headline should leave readers feeling cheated when they get to the story. We avoid running two tests at the same time, to ensure that the only factor changing on the home page is the headline in question. The alternative headline is vetted, in writing, by a colleague — often editors end up batting around alternatives.

In writing headlines, as in testing them, two heads are better than one.