When AI Goes Off the Rails: Lessons from Public Failures and the Power of AI Observability & Evaluations

AI systems can behave unpredictably in production, from agents drifting off task to models confidently inventing answers. In this session, we'll revisit several memorable public AI failures, including some legendary moments from Microsoft's early chatbot experiments, to illustrate how quickly things can go sideways when teams lack visibility into what their AI systems are doing. 

Then the Progress product team will demonstrate Progress AI Observability and show how developers can trace agent behavior, detect model drift and hallucinations, monitor performance and costs, and use evaluations to continuously improve AI systems. 

What you'll learn 

• Why AI systems can behave unpredictably in production
• What public AI failures can teach engineering teams
• How developers can trace agent behavior
• How to detect model drift and hallucinations
• How to monitor performance and costs
• How evaluations support continuous improvement of AI systems 

Through practical examples and evaluation workflows, you'll learn how to diagnose problems, reduce risk, and ship AI-powered applications with greater confidence. 

Continue exploring after the webinar 

Attendees will receive access to a developer resource hub with sample applications, evaluation workflows, setup guidance, test data and trial information. 

These materials will help teams continue exploring hallucination analysis, cost monitoring, changes in model behavior and ongoing evaluation practices after the webinar.