Revolutionizing Travel Reviews: Tripadvisor's AI-Driven Summaries

How the Tripadvisor Machine Learning team is using generative AI & Large Language Models (LLMs) to distill user reviews into a helpful and intuitive set of review summaries

By Varuni G. , Raja H.

With over 1 billion reviews and opinions and counting, Tripadvisor is unquestionably the world’s largest and most popular travel review site. Each month, hundreds of millions of visitors rely on Tripadvisor to provide personal insights and to give recommendations on almost 9 million different restaurants, hotels, and attractions. Our huge collection of long-form user reviews are why our travelers trust us, and the advancements in AI models help summarize that content in an effective, transparent, and impartial way.

We’re pleased to announce, new AI-powered feature to Tripadvisor.com, Review Summaries, which provides concise summaries of recent traveler reviews on a hotel based on user-identified quality attributes. You can read a summary that describes how clean a hotel is, then skim through review quotes that were used to generate the summary itself. We’ve taken the content that you care the most about, and we’ve made it much easier to consume, all while staying true to the voice of the original traveler.

All of this was done using state of the art generative AI and Large Language Models (LLMs). 

Simplifying Your Search with AI

Imagine that you’re looking for a hotel for your next vacation. You appreciate a clean room, and you need a quiet environment to help you sleep. If you’re the kind of traveler that takes great care when booking a vacation, then you might spend hours skimming reviews and switching between properties looking for that perfect place to stay. With review summaries, all of those great traveler insights are now surfaced in clear, and practical format with easy access to original traveler quotes.

Starting this quarter, when you click on one of our more frequently visited hotel properties, you may notice a Traveler Insights section that is similar to the following:

On the left side of the Traveler Insights section, under the title, we have an AI-generated overview summary for the property itself. This summary was generated, with care, based on the attribute summaries that we created for each quality attribute. In this way, the summary reflects the overall, unbiased opinions of each user, according to what they appreciate the most. Our summaries are crafted to be easy to read, of optimal length, and must be strictly driven by quotes from reviews. This ensures that they remain impartial and truly reflective of our users' experiences.

On the right side of the Traveler Insights section, we render a set of quality attributes, along with a one word opinion (Cleanliness: Immaculate), to help you get a true sense of how users feel about a specific property. These attributes were chosen because they represent the values that our users care most about when evaluating an accommodation. The one-word opinion, a concise representation of the majority view, offers a quick, unbiased snapshot of each property, reflecting the collective judgment of our users.

You can click any of the attributes to read a summary that describes how this attribute, for example cleanliness, value, or location, applies to this property. You can drill down even further and read actual quotes and the complete reviews that were used to generate the attribute summary:

Building AI-Driven Summaries

A popular hotel on Tripadvisor.com may have thousands of existing reviews; surfacing this content to the user in a meaningful way is no easy feat. We relied on extensive user research and a recent, large scale customer survey to create two types of summary: a detailed overall summary and a succinct attribute summary. Making the effort to listen to our users and analyzing their feedback, helped us cater to our diverse set of user preferences. 

Our goal with Review Summaries was to make as much of this content as easy as possible to digest, but also to capture the broad spectrum of positive and negative opinion without favoring one side or the other, and without biasing the AI. We know from extensive interviews with real travelers that there's a tremendous amount of trust in reviews on Tripadvisor, so we made an effort with this new feature to uphold that integrity by prioritizing transparency and impartiality. This process reflects our unwavering commitment to delivering trustworthy, insightful content that resonates with the real experiences of travelers worldwide.

Choosing the right attributes 

We started by identifying the review elements based on the results of our Power of Reviews report. The report describes the results of an exhaustive survey on 6000 users across five continents to determine how they perceive our review content and how the reviews help them when they plan a vacation. It was instrumental in deepening our understanding of travelers' decision-making processes. 

Most importantly, the report identified the top review elements most valued by users for accommodations. The results show a very clear picture about what our users value when it comes to choosing a place to stay. These were the elements that we chose when we created our list of review attributes for summarization. 

Choosing the right data 

In the Power of review report users shared that they highly value the latest information from fellow travelers. Therefore we only create summaries for properties with enough recent reviews to capture a broad range of feedback. 

In addition, we only create review summaries for properties that contain an adequate number of attributes discussed in its reviews. We let the user reviews speak for themselves and we do not create an Insight section for a property if the reviews do not sufficiently support it.

Generating the summaries

Equipped with the most relevant attributes and fresh reviews, our team dove into the exciting task of crafting these concise summaries

To achieve this, we utilized a cutting-edge machine learning technique known as Retrieval-Augmented Generation, or RAG. RAG is an AI framework for retrieving facts from a knowledge base to ground large language models (LLMs) in the most accurate, up-to-date information and improve the quality of its responses. The process of retrieval-augmented generation involves several key stages, including document retrieval, relevance scoring, contextual integration, and text generation. TripAdvisor's cloud infrastructure, which is underpinned by Kubernetes-managed clusters and optimized for machine learning workflows through Kubeflow enabled the scalable application of such Natural Language Processing (NLP) techniques to provide travelers with relevant, easy to read, and comprehensive collective guidance from tripadvisor reviews.

To extract meaningful quotes that discuss specific review attributes, we utilized BART  (Bidirectional and Auto-Regressive Transformers), an advanced language model developed by Meta. By leveraging BART's text classification capabilities, we were able to systematically sift through extensive textual data, identifying and categorizing quotes that precisely align with the targeted review attributes.

While we successfully retrieved relevant sets of quotes using this approach, it was crucial to evaluate their relevance based on user expectations and insights derived from the power of reviews research. To achieve this, we applied a secondary classification process using transfer learning. Specifically, we fine-tuned DistilBERT model, an advanced language model from Google AI using a manually labeled dataset. We established stringent scoring thresholds to ensure that only sentences pertinent to the specified attributes were selected. 

We next utilized the relevant sentences as input for GPT-4 and enabled us to generate responses that are not only contextually accurate but also aligned with the specific user expectations and attributes identified earlier. Our approach included giving the model detailed prompt, specifying the desired style of summaries. The aim was to fine-tune the model's output to not only mirror the user's unique voice but also to ensure the summaries are presented to travelers in a format that is clear, engaging, and rich with information. 

This process, known as prompt engineering, was intensively conducted with human-in-the-loop methodology. We took great care to ensure that GPT-4 maintained neutrality and did not introduce any bias or subjective color into the content. We invested significant time and effort in iterative refinements, ensuring that the summaries generated by GPT-4 accurately mirrored the essence of the original user review. Our goal was to faithfully reflect the genuine experiences and perspectives of our real-life travelers, preserving the authenticity and integrity of their voices in the process.

We performed similar prompt engineering to further summarize attributes to create our overview summary. 

What’s next?

It was clear to us that our users valued and trusted our long form review content. Pairing that source content with cutting edge generative AI and machine learning concepts to create a simple and highly usable interface built around attribute descriptions was an obvious choice. The real trick was in how we stayed true to the spirit and voice of the real-world traveler, without introducing bias or hallucination. 

This is just another step in an exciting journey to provide travelers around the world with the most relevant guidance to help them plan the best trip possible. With such a wealth of great content, and the latest AI-powered technologies, the possibilities are endless. We're currently gathering feedback from users, iterating on the feature, and look forward to expanding into restaurants and attractions. In the meantime, we hope you enjoy exploring hotels around the world more effortlessly with our AI-powered Review Summaries, designed to make your travel planning smoother and more insightful.

Acknowledgements

Improvements like this are always a collaborative effort, and while here we focus on the machine learning portion, our SEO team, Design, User Research, Analytics, QA teams, MLOPs and Web Engineering teams all play a big part in making this project and projects like this happen.