Abstract
A successful demo is only the beginning. Drawing on lessons from three production LLM systems at Adyen, this session explores what it takes to turn a promising prototype into a reliable application. We’ll focus on three principles: keep it deterministic by moving predictable logic out of prompts; keep it simple by starting with straightforward LLM calls instead of agents; and keep it measurable by continuously evaluating retrieval and generation in production. Through practical examples involving citations, orchestration, and automated evaluation, attendees will learn how simpler architectures improve reliability, speed, and debuggability.
Topics To Be Covered
Build reliable LLM applications beyond prototypes
Reduce costs with deterministic application logic
Simplify architectures before adopting AI agents
Measure LLM quality with continuous evaluation
Improve reliability through production-ready orchestration
Perfect For
AI Engineers
ML Engineers
Platform Engineers
AI Product Managers
Enterprise Architects
Meet Your Speakers
Bjorn van Dijkman
AI Engineer, Adyen
Björn van Dijkman is an AI Engineer at Adyen, specializing in transforming promising LLM prototypes into reliable production systems. He develops AI applications for payment investigations, customer-facing content generation, and internal commercial support. His work spans retrieval, orchestration, evaluation, and workflow integration, with a strong focus on combining deterministic software engineering with AI to build robust, high-quality systems through rapid feedback loops. Before moving into AI engineering, Björn worked as a Data Engineer in Adyen’s in-person payments team and previously built data platforms and delivered data-driven solutions across multiple industries. He is also an active contributor to the AI community, regularly speaking at developer events and organizing hackathons to share practical lessons from building and operating LLM systems.
Yasmin Levens
AI Engineer, Adyen
Yasmin Levens is an AI Engineer at Adyen, where she builds and deploys LLM-powered applications for production use. Her work focuses on designing reliable AI systems that create measurable business value, with experience spanning retrieval, orchestration, evaluation, and real-world deployment. Before transitioning into AI engineering, Yasmin worked as a Data Engineer at Adyen's In-Person Payments team, building scalable data solutions. She previously collaborated with the Data Science team at Albert Heijn on AI-driven recommendation systems, contributing to research published on the company's technology blog. Passionate about practical AI engineering, Yasmin enjoys sharing lessons learned from building production-grade LLM applications and helping teams bridge the gap between prototypes and enterprise-ready AI systems.
ADDITIONAL INFORMATION
Time & Place
Thu, Nov 26
14:30 - 15:00
Matterhorn III
Limited to 40 participants.
Secure your seat – registration required.
Notes
Agenda for this session
20 min presentation + Audience Q&A



.png)