Fireside Chat: LLM Observability and AI Evaluations

Iterating on your AI-powered applications can be difficult when LLMs are constantly updated and small prompt changes can lead to wildly different outputs. Join Shriram Subramanian (Group Product Manager, Datadog), Charles Jacquet (Product Manager, Datadog), and Rahul Iyer (VP of Engineering) from Vercel as they share best practices for reliably evaluating and improving your AI products. The session will explore how innovations like Vercel v0, an AI-powered UI generation tool, are not just transforming development workflows, enabling rapid prototyping, but also leading to rapid experimentation on path to production. Come hear and learn real world examples from working with Datadog customers and answer your questions so you can measure what matters most when developing and deploying AI applications.