One of the many ways generative AI is assisting the news media industry is by creating story summaries — something that is fast, simple and can increase readers’ engagement.  

During this week’s Webinar, Vebjørn Nevland, a data scientist at Verdens Gang (VG) in Norway, told INMA members how the company has leveraged AI to write summaries for every article it publishes. Nevland walked the audience through how Schibsted brand VG used OpenAI’s GPT-3 model to generate that content, but first, Smart Data Initiative Lead Ariane Bernard noted that as publishers look at ways to benefit from AI, summaries could be a great starting point.

“It’s one of the somewhat less risky uses of this technology that is scalable and useful,” she said. “The big reason it is less risky and more advanced and more mature is that summaries, by definition, work from a very specific content base. They’re not just building something from nothing.”

That minimises the risk of things like hallucinations and could be a good way to introduce the technology to newsrooms because journalists may be more receptive to using AI for mundane tasks: “No one really loves summarising,” Bernard said. “Yes, humans can do it, but maybe they don’t love doing this. This is a place to introduce technology to offload something that takes up their time.”

How VG did it

Nevland explained how VG began to classify new summaries and then implemented them, including how it approached journalists with the new tools.

“That’s a huge thing in itself,” he said. “Basically [we need to know] does it really work? Do people like it? Is it usable? And things like that.”

VG’s article summaries have a small blurb at the top of the article; then users click a button to see the entire summary. The summaries are created using OpenAI’s ChatGPT-3 model, which generates eight different summaries for each article.

For the new summaries, it was important to create new classifications based on how accurate each one was: “Classification is how we can try and make sure that we are as close to the truth as possible,” Nevland said. “We take those eight article summaries and classify [each one] to say if it’s good or bad and we take the best one.”

Finding the best summary is also done by using GPT-3 and asking if it is the best one. Then, the summary that GPT-3 selects is implemented into the article.

What users say

User feedback was critical to the success of implementing AI-generated summaries, so VG began asking readers if they liked having a short version of the story.

That provided valuable insight into how different readers used the summary, with a majority — 74% — saying the summary provided an introduction and made them want to read the article.

At the same time, 12% of readers said it provided enough information that they didn’t need to read the entire article.

“That might be bad if you want them to read the article, but on the other hand, we’re giving the user information so they will still be satisfied,” Nevland said. “So the product will make the user happy in a way.”

Overall, what VG discovered was “a lot of users like this feature, they’re using it frequently, and they miss it when articles don’t have them.”

Taking it to the journalists

In addition to passing the test with readers, VG needed its journalists to embrace the AI summary tool. Experience told Nevland that it had to be fast and simple for journalists to use — or it would be ignored.

VG’s tool for creating articles automatically generates summaries. An article summary field appears at the top of the article, and journalists can generate a summary by pushing a button. This is a significant improvement from its previous system, where articles had to be published in order to generate a summary.

Now, once AI creates a summary, the journalist becomes responsible for reading and approving a summary — and making any needed edits.

“It’s really easy to do changes,” Nevland said. “As a journalist, you have to read through it to see that yes, this is good or no, this is wrong, but then they can just change the wrong thing. Then you just publish it as normal.”

For the journalist, even though it’s an extra step, it’s a matter of hitting one button and reading, approving, or editing it. They can also delete the summary if they don’t want it.

The tool has been so successful that Schibsted has onboarded all its brands to use article summarisation.

“With this tool, you don’t have to be a data scientist or an analyst; you can just go in here, push experiments and try different prompts. And since we are just using ChatGPT for this, it’s really easy for everyone to just change the prompt.”

How it’s working

One of the most common questions Nevland hears from journalists and editors is how the summaries affect the performance of articles. He said newsrooms want to know if the reading times or the amount of news people read are affected.

“The short answer is people that read the short summary read the article just as much and they [spend] a little bit longer time. We think that they use a longer time because it takes time to read the short version,” he said.

“This is fantastic news for us because the journalists want them to read the articles.”

As of August, about 19% of all readers were using the short version feature. Young users, defined as people between 15 and 24 years old, delivered a click-through rate of 27%.

“That means that almost 30% of our young people use this feature. And if you ever worked in development or something like that on having a new feature and having these numbers as fast as we did, it’s insanely high and click to rate,” he said.

That’s exciting, considering the global struggle for newspapers to attract young users.

“When we create a feature that works better for young readers and everyone likes it, we push that feature extra hard to make sure that our journalists and our editors know that this feature is something young people want,” he said. “So this is a huge success on all metrics.”

If you’d like to subscribe to my bi-weekly newsletter, INMA members can do so here.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .