Building Bits Detection

When Datadog started building Datadog Bits Detection, the idea was simple: send a signal to an LLM and get a decision back. But the team quickly learned that detecting an anomaly and deciding whether it's a real incident are two very different problems. While tools like Watchdog use statistical methods to detect anomalies, determining their impact requires understanding deployments, dependencies, and user experience. The real challenge wasn't the AI model itself, but engineering the system around it.

In this session, Samantha Scaglione, Product Manager for Bits Detection, will take you behind the scenes of building Bits Detection. She’ll share how they tackled the non-deterministic nature of LLMs using evaluations, datasets, and guardrails to measure reliability and control costs. She’ll also cover why deterministic rules remain essential for filtering and enriching context before LLM reasoning, and how Datadog combined these approaches into a hybrid architecture designed to balance cost, latency, and scale.

Attendees will walk away with practical lessons for building reliable, cost-efficient AI-powered systems, combining deterministic logic with LLMs, and designing evaluation and observability practices that support better decisions in production.