Abstract

Can GenAI judge like a human?

As LLMs take on roles like scoring chatbot responses and assessing risk, their reliability, transparency, and fairness become critical. This ‘no-code’ workshop explores the paradigm of LLM-as-a-Judge, offering practical strategies and hands-on tactics for deploying LLMs as scalable, consistent evaluators. Participants will examine how LLMs replicate human judgement and where they fall short - especially around ambiguity, bias, and drift. Through a real-world use case and interactive exercise, you’ll learn to design robust evaluation frameworks, engineer effective prompts, and build reproducible systems that combine GenAI and human oversight.