Your Agent Works - So Why Is It Failing in Production?
Product & UX
Your agent passed every test, so why is it failing your users? The gap between demo success and production reality is where most agentic AI initiatives quietly break down, and traditional evaluation methods weren’t built to catch it.
Drawing on real-world experience building and validating agentic systems at Toyota Motor Europe, this session introduces an adaptive approach to evaluating non-deterministic agents, where no single framework fits all use cases. Attendees will leave with practical thinking tools to define what “production-ready” means in their context, and the confidence to scale systems that deliver real value.
Time & Place
Thu, Nov 26
14:30 - 15:00
Matterhorn I
Limited to 45 participants.
Meet Your Intructors

Nour Eid
AI Product Analyst, Hazelheartwood
Nour Eid is an AI Product Analyst specializing in Generative AI and agentic systems, with a focus on evaluation, reliability, and scalable deployment. Her experience is rooted in the automotive sector, where she has contributed to one of the largest AI initiatives at Toyota Motor Europe.
Nour's work centers on testing and validating agentic systems in highly variable, non-deterministic environments, where each use case introduces unique behaviors and challenges traditional evaluation approaches. To address these challenges, she developed a comprehensive evaluation framework that has been adopted across multiple teams, enabling the delivery of more robust and production-ready AI systems.
She also designs AI maturity assessments and contributes to AI strategy initiatives, helping organizations define and measure AI value, identify high-impact use cases, and scale solutions that deliver measurable business outcomes.
What To Expect
Who Is This For?
AI Product Managers
AI Architects
Data Scientists
AI Engineering Leaders
Enterprise AI Teams
Pre-Requisites
No Prerequisits
What You'll Learn & Do?
Why agents fail in production
Evaluating non-deterministic AI systems
Building adaptive evaluation frameworks
Defining production-ready AI success
Scaling reliable agentic AI solutions
Agenda & Activities
Agenda for this session
20 min presentation + Audience Q&A
.png)