Building, Operating, and Securing AI Applications in Production

AI agents have moved from prototypes and internal experiments into customer-facing products, support workflows, developer tools, and business-critical systems. But building AI applications introduces a different set of engineering challenges than traditional software. Failures can be harder to reproduce, quality can be harder to measure, and a poor response can quickly become a customer experience and brand trust issue.

At the same time, teams still need to apply the fundamentals of production engineering and balancing these concerns forces engineering organizations to rethink not only their architectures, but also their team topologies and operational processes.

In this panel, Maxime Agostini, Group Product Manager at Datadog, will discuss with Datadog customers the practical realities of building, running, and securing AI-powered applications in production. The conversation will cover how teams design agent architectures, evaluate response quality, monitor LLM calls and tool usage, debug unexpected behavior, mitigate hallucinations, manage customer-facing risk, and organize engineering teams around fast-moving AI capabilities.

No matter what type of AI application you’re building, this session will surface real-world lessons from teams bringing AI applications beyond demos and into production. Attendees will leave with practical perspectives on how engineering practices, observability and security strategies, and organizational structures are evolving to support reliable, trustworthy, and production-ready AI systems.