In this practical talk, we share how Booking.com built its AI Trip Planner - an LLM-powered experience that personalizes travel planning at scale. We’ll walk through real-world design decisions, technical challenges, and infrastructure optimizations involved in delivering real-time hotel and destination recommendations using large language models (LLMs).
We’ll cover key challenges like moderating user input, classifying intent, structuring dialogues, and generating grounded responses. Through prompt engineering and custom model development, we tailored LLM interactions to our product needs while ensuring speed and relevance.
To address inference latency, we implemented speculative decoding and integrated Medusa-1, a novel architecture that predicts multiple tokens in parallel, achieving a 1.8x speedup with no loss in quality. We’ll detail its design and training trade-offs.
Beyond acceleration, we’ll highlight our move toward agentic AI systems - modular components that orchestrate LLMs, retrieval services, and Booking.comAPIs to solve complex travel queries. For example: A Question-Answering Agent that fuses LLMs, real-time data, and APIs for context-aware answers.

Finally, we’ll show how we evaluate quality in production using LLM-based evaluations, including Judge LLMs for automatic assessment, dialog quality and more.