As AI agents increasingly act autonomously in production systems, devops expands into evaluation, observability, and all of the above. This talk explores how to observe, improve, and debug AI agents in ways we did not have to think about for standard software services. I'll cover AI agent eval, monitoring, sandboxing best practices, and failure modes unique to agent-driven systems. Because when your software makes decisions on its own, someone still needs to be watching (it's agents all the way down).

Diamond Bishop is a Director of Engineering and AI at Datadog, where he leads applied AI teams and experimental product efforts focused on AI agents and developer productivity. A longtime techno optimist, he has spent over 15 years building AI and ML systems across startups and large platforms, including Amazon, Meta, and AWS. Diamond is the co founder of Augmend, an AI company acquired by Datadog, and splits his time between New York and San Francisco thinking about how AI can meaningfully extend human capability.