AI-generated article summaries have quickly become a standard feature on our news sites. What started as hack-day experiments is now built into the CMS and being adopted across Schibsted’s newsrooms.

The best part? Users love them. We see average opening rates in the range of 20% to 30%, with even stronger engagement among younger users. And, contrary to what one could think, split tests show the article read time is increasing when summaries are added.

Succeeding with generative AI requires editorial, product, data, and tech to come together. Creating space for collaboration and experimentation needs to be at the top of every media leader’s agenda in this technological shift.

Schibsted’s approach to AI encourages both experimentation and local initiative within the brands. We also encourage collaboration across the group to make promising tools and products available to all. This is easy to say, but harder in practice. The AI-generated summary solution is a beautiful example of this playing out.

A blueprint for collaboration

The article summary solution started as a hack-day idea at VG earlier this year. The product team quickly prioritised and launched a first version of the feature. At the offset, they were curious to see how users adapted to the summaries, slightly worried they would negatively impact time spent reading the actual article.

A few design iterations later proved the team had come up with a great new addition to the product. A valuable insight was how easy it can be to create useful products with AI these days: 80% of the software engineering was done by their talented apprentice, Edvard Høiby.

VG joined forces with colleagues at Aftonbladet and data scientists from Schibsted’s AI enablement team working on a similar initiative. The content management systems (CMS) team turned work around quickly, and the collective effort has resulted in a scalable solution for generating summaries now at the hands of hundreds of journalists across Norway and Sweden.

How it works

Journalists generate the summary when the article is ready to be published. As a means to ensure it is of high quality, multiple summaries are actually being generated by the AI. With additional prompts, these summaries are automatically scored based on accuracy, and then the best-performing summary is served back to the journalist as an editable suggestion. The journalist can either approve or edit the summary, and hit publish.

Each newsroom can customise and experiment with different prompts to suit their style. The solution builds on GPT-4 from OpenAI, with a fallback to GPT-3.5 for longer articles and token sizes. This INMA Webinar by VG provides further details.

Newsrooms onboard

All major newsrooms at Schibsted have established guidelines to ensure responsible adoption of AI. “Human in the loop” is an important principle.

Aftonbladet initially observed great quality of the summaries, which allowed the team to somewhat relax quality checks. However, this summer, they experienced a sudden drop in quality for a few days, despite no known changes to the solution. Then it returned back to normal. The experience highlighted the importance of having journalists review the summaries and making the publishing decisions.

Like most large news organisations, Aftenposten has a group overseeing their language across platforms. One of their language experts is now heavily involved in prompt design, tailoring summaries to Aftenposten’s tone of voice and language standards. It’s a great example of tradition meeting new capabilities.

Another learning is to make AI tools super easy to use and accessible for the newsroom in day-to-day workflows. Early versions of the summary solution involved additional steps, in yet another system. Newsroom adoption improved a lot when it became available right in the CMS.

Accelerating responsible adoption of AI

Schibsted has been investing in AI for many years. Our Futures Lab and AI Enablement team offer training, expertise, and capacity to support our brands. Through the NorwAI research centre, we are collaborating with academia and industry partners to develop a Nordic-based large-language model. We also develop fine-tuned models in-house.

In response to the breakthrough development of generative AI, we ran a strategy project called Nextgen Newsrooms this spring. It outlined how we will leverage AI for better products and efficient newsrooms, and to innovate new journalistic methods. A key element of the strategy is taking action to strengthen trust.

Each newsroom has now appointed someone to be responsible for AI, and most newsrooms have groups actively working on the adoption of AI. We meet regularly across newsrooms to form a news media wide AI task force. The goal is to share knowledge and to identify the most promising applications of AI to be made available to all. The AI-generated article summaries is one such successful application, and it has boosted our motivation to find more.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .