top of page

The Best LLM Call is the One You Never Make

Abstract

A successful demo is only the beginning. Drawing on lessons from three production LLM systems at Adyen, this session explores what it takes to turn a promising prototype into a reliable application. We’ll focus on three principles: keep it deterministic by moving predictable logic out of prompts; keep it simple by starting with straightforward LLM calls instead of agents; and keep it measurable by continuously evaluating retrieval and generation in production. Through practical examples involving citations, orchestration, and automated evaluation, attendees will learn how simpler architectures improve reliability, speed, and debuggability.

Topics To Be Covered

  • Build reliable LLM applications beyond prototypes

  • Reduce costs with deterministic application logic

  • Simplify architectures before adopting AI agents

  • Measure LLM quality with continuous evaluation

  • Improve reliability through production-ready orchestration

Perfect For

  • AI Engineers

  • ML Engineers

  • Platform Engineers

  • AI Product Managers

  • Enterprise Architects

Meet Your Speakers

Bjorn van Dijkman

Bjorn van Dijkman

Adyen

AI Engineer, Adyen

Björn van Dijkman is an AI Engineer at Adyen, specializing in transforming promising LLM prototypes into reliable production systems. He develops AI applications for payment investigations, customer-facing content generation, and internal commercial support. His work spans retrieval, orchestration, evaluation, and workflow integration, with a strong focus on combining deterministic software engineering with AI to build robust, high-quality systems through rapid feedback loops. Before moving into AI engineering, Björn worked as a Data Engineer in Adyen’s in-person payments team and previously built data platforms and delivered data-driven solutions across multiple industries. He is also an active contributor to the AI community, regularly speaking at developer events and organizing hackathons to share practical lessons from building and operating LLM systems.

Yasmin Levens

Yasmin Levens

Adyen

AI Engineer, Adyen

Yasmin Levens is an AI Engineer at Adyen, where she builds and deploys LLM-powered applications for production use. Her work focuses on designing reliable AI systems that create measurable business value, with experience spanning retrieval, orchestration, evaluation, and real-world deployment. Before transitioning into AI engineering, Yasmin worked as a Data Engineer at Adyen's In-Person Payments team, building scalable data solutions. She previously collaborated with the Data Science team at Albert Heijn on AI-driven recommendation systems, contributing to research published on the company's technology blog. Passionate about practical AI engineering, Yasmin enjoys sharing lessons learned from building production-grade LLM applications and helping teams bridge the gap between prototypes and enterprise-ready AI systems.

ADDITIONAL INFORMATION

Time & Place

Thu, Nov 26

14:30 - 15:00

Matterhorn III

Limited to 40 participants.

Secure your seat – registration required.

Notes

Agenda for this session

  • 20 min presentation + Audience Q&A

REGISTRATION

In order to register to this session you must hold a Pro or Max Pass.

Limited Seating Still Guaranteed

WhatsApp button (66 x 66 px).png
bottom of page