Datadog Detect Chicago

Every detection engineering team eventually hits the same unanswered question: a threat hunt turns up something real — should it become a standing detection rule? Get it wrong toward "yes" and you get alert fatigue and rules built to chase ATT&CK coverage metrics rather than catch anything real. Get it wrong toward "no" and repeatable adversary behavior stays trapped in a hunter's head or a Slack thread — exactly the memory loss problem modern hunting frameworks were built to solve.

Two open source frameworks currently bracket this decision without answering it. The Agentic Threat Hunting Framework (ATHF) gives hunts structure and persistence through its LOCK pattern, and its own hunt template literally asks the hunter, at the end of every investigation, "could this become an automated detection?" — with no criteria for answering. The Adversarial Detection Engineering (ADE) Framework and its Detection Logic Exposures (DLE) catalog pick up on the other side: a rigorous taxonomy for auditing detection logic that has already been written and shipped. Neither framework touches the moment in between.

This talk introduces GATES: a five gate promotion model — Generalizable, Actionable, Tunable, Exposure tested, Sustainable — that sits between ATHF's KEEP stage and ADE's quality gate, and gives every hunt finding a defensible, repeatable path to production or a defensible reason to stay a hunt. Each gate is tied to a documented, citable failure mode, not an abstract principle.

Rather than present this in the abstract, we'll run it live against a real, already shipped detection rule and expose a genuine gap hiding in its own documentation — then show exactly how that gap becomes next sprint's hunt hypothesis, closing a loop most "hunt vs. detect" advice only gestures at.

Attendees will leave with the five gates, the routing model for findings that fail them, and a worked template they can run against one of their own shipped detections before the end of the week.

A detection rule is only as good as its ability to catch the behavior it was written for, and most teams never actually check. We built a system that does the checking: safely emulating real attacker behavior in our own cloud environments and confirming the right alarm goes off as new detections are authored, and every time existing ones change.

This talk covers how our team put that system into practice across 600+ detection rules, dozens of log sources, and six cloud platforms (AWS, Azure, Entra ID, GCP, and Kubernetes). We combined two open-source Datadog tools — Stratus Red Team, which emulates realistic attacker behavior on demand, and Threatest, which runs the test and checks the result — and this talk will focus on the engineering we added around them to make this reliable at scale.

The hard part is scale. Someone has to decide, for each detection, which emulated behavior should trigger it, and then write that test. Doing that by hand for hundreds of rules, and keeping it current as rules change, is unsustainable, so we handed that job to an AI. When a rule is authored or changed, a language model reads the detection, works out what it's designed to catch, picks the matching behavior from our catalog, and generates the test automatically. Rules that can't be reliably tested this way are identified and set aside up front, so we never waste a run.

Each test then runs in isolation: it logs into the target cloud, executes the attacker behavior against the live environment, and verifies that the expected alert is fired. For a brand-new detection that isn't live yet, there's no alert to wait for, so we verify one level down, confirming the emulated behavior actually showed up in the logs the rule will watch. To keep results trustworthy, every emulation is tagged with a unique ID that we match inside the resulting alert or log, so unrelated activity can never fake a passing test.

Our most important design decision was what the system doesn't do: these tests inform, they don't block. A failing test alerts the engineer, but never stops their work from shipping. We'll make the case that this limitation is precisely what lets continuous, real-world emulation survive in a busy team, and why the more obvious approach, treating every test as a hard gate, kills the practice.

You'll leave with a practical blueprint: how to connect these tools into an automated testing pipeline, where AI genuinely helps versus where it gets things wrong, how to trust your results against noisy real-world data, and the honest trade-offs of continuously emulating adversary behavior in live cloud environments.

As information systems get more complex by the day, so does the quantity and variety of logs organizations collect, parse, ingest and build threat detections upon. As a result, security teams are increasingly turning towards log normalization to help unify detection logic across disparate log sources from endpoints, servers, cloud environments, SaaS providers.

In this talk, we showcase how Datadog utilizes the Open Cybersecurity Schema Framework (OCSF) to normalize logs, allowing practitioners to efficiently build and maintain detection content. We demonstrate how Datadog uses agentic harnesses to accelerate and expand the normalization of logs. LLMs allow us to classify and map logs at machine speed, while the OCSF framework provides a tight validation loop that can be used to ensure a high quality bar.

In turn, OCSF normalization enables both human and LLM security practitioners to more efficiently detect, triage, and respond to security incidents. We’ll demonstrate how OCSF allows for the easy correlation of indicators like domain names, usernames, and session tokens across sources. By ensuring a common taxonomy of log attributes, OCSF-driven normalization allows practitioners to focus purely on detection logic without having to worry about the particular quirks of individual sources

Attendees will leave with an understanding of the OSCF-driven log normalization, and how it both impacts and is impacted by continuing improvements in AI security tooling.

Cloud environments span three layers: the cloud control plane, containers, and code. In this workshop, you'll have access to a real AWS environment that an attacker has just compromised by exploiting a vulnerability in an internet-exposed web application. It appears the threat actor didn't stop there. Your mission, should you choose to accept it, is to investigate the full extent of the compromise across all three layers before eradicating the attacker from your environment and implementing remediations.

To fulfill your duties, you'll have several data sources at your disposal: CloudTrail logs, application logs, process execution events, and access to the web application's source code. You'll walk away with a practical understanding of how to investigate and remediate a full cloud compromise end to end.

This session is a hands-on workshop. A laptop is required to participate.

MFA phishing (multi-factor authentication phishing) has rapidly evolved over the last few years from cool research into a mainstream initial access vector, largely due to the open-sourced adversary-in-the middle (AiTM) frameworks such as Evilginx and Tycoon. While defenders/vendors may have improved detection of MFA phishing attempts at the identity layer, a group of attackers have been flying under your radar using a much less scrutinized cloud control plane.

This talk presents a highly evasive technique observed in the wild that is getting no publicity: MFA phishing delivered entirely within Microsoft Purview Message Encryption. By compromising a single Microsoft 365 mailbox, adversaries can send encrypted, restricted-permission messages that effectively blind secure email gateways, evade content inspection, and exploit user trust in “Microsoft-protected” communications that can target anyone with a mailbox.

We will walk through how attackers chain business email compromise (BEC) access with AiTM phishing kits, using Purview as a recursive delivery mechanism to propagate access across organizations, even when targeted recipient(s) do not themselves use Microsoft 365. This technique shifts the detection battleground from traditional email filtering and signature rules to identity telemetry, audit logs, and behavioral signals that many SOCs are not yet monitoring.

Attendees will leave with a concrete understanding of: • How Purview Message Encryption is being abused as a phishing transport • The evolution of Purview MFA Phishing TTPs • How to build detections that adapt to this threat

This session is designed for detection engineers, cloud security practitioners, and incident responders who want to stay ahead of adversaries abusing “trusted” cloud features as weapons.