Oct 13, 2026

Summary

Evals are key to defining quality and preventing regressions, but they only test what teams already know to look for. Once agents reach production, new behaviors and failure modes emerge across more traces than any team can review manually.

This talk introduces active observability: a continuous quality loop that connects production traces, pattern discovery, debugging, and evals. Attendees will learn how they can help their teams turn production evidence into prioritized improvements, validate changes, and measure whether agent quality improves over time.

Speakers

Ameya Bhatawdekar

Ameya Bhatawdekar

VP & Field CTO @ Braintrust

Ameya serves as VP, Field CTO at Braintrust, the Agent observability platform for shipping quality agents. He has spent his career using AI to help people be more effective and stay safe, and has driven multiple 0→1 initiatives to internet-scale, shipping intelligent systems for Dropbox Dash, Microsoft Office, Facebook, Windows, and major web browsers. At Dropbox, he led ML Engineering for universal AI search in Dash (search, discovery, recommendations, agentic conversational AI). Before that, he led Microsoft's Office AI team shipping AI features across Word, Excel, and PowerPoint, and created Office Copilot.

Stay In The Loop