In this section
DE0.1 Detection Engineering — What It Is and Why It Is Important
You work in security — triaging alerts, investigating incidents, managing a SIEM, or leading a team that does. You've seen detection rules fire. You've seen them miss. This section defines detection engineering as a discipline, explains where it sits in the security operations landscape, and makes the case for why it matters — for organizations and for the people who practice it.
A definition, not a buzzword
Detection engineering is the discipline of building, testing, deploying, and maintaining the detection rules that determine what a security operations program can see. Not the alerts themselves — the system that produces the alerts. Not the KQL queries — the methodology that makes the queries effective. Not the SIEM — the engineered content that turns a data platform into a detection capability.
Think about what happens when a new employee clicks a phishing link. The email arrives in their inbox. The click generates a URL click event. The credential harvesting page captures their password. The attacker authenticates with the stolen credential. They create an inbox rule to hide evidence. They read three months of email. They send a fraudulent wire transfer request to the CFO.
At every step, the Microsoft security stack generates telemetry. EmailEvents records the inbound email. EmailUrlInfo records the URL click. SigninLogs records the authentication. OfficeActivity records the inbox rule creation and the email access. The data exists. The question is whether any detection rule examines that data for the specific pattern that distinguishes this attack from legitimate activity. In most organizations, the answer is no — not because the data isn't collected, but because nobody wrote the rule.
Detection engineering is the discipline that writes the rules. But the definition matters because "detection engineering" gets used loosely. A vendor calls their template rules "detection engineering." A SOC analyst writes a KQL query and calls it "detection engineering." A consultant enables Sentinel content hub solutions and calls the engagement "detection engineering." None of those are detection engineering in the sense this course teaches it, because none of them involve the systematic process of modeling threats, hypothesizing detection signals, building rules against those signals, testing rules against real attack data, deploying rules through a versioned pipeline, and maintaining rules on a cadence that keeps them current as the environment and the threat landscape change.
Detection engineering is an engineering discipline. Engineering means applying systematic methods to build reliable systems. A detection engineer doesn't write ad-hoc queries — they design, test, document, version, and maintain detection rules the same way a software engineer designs, tests, documents, versions, and maintains code. The output is a detection program — a measured, maintained system — not a collection of rules that happened to accumulate over time.
Estimated time: 40 minutes.
Figure DE0.1 — The four security operations functions form a cycle. SOC operations, threat hunting, and incident response all depend on the detection engineering function that most organizations lack. Without it, the cycle breaks — hunt findings stay in reports, IR post-mortems produce recommendations nobody automates, and the SOC operates against rules nobody tests or maintains.
The Sentinel deployment project enabled 30 template rules, connected four data sources, and handed the workspace to the SOC. The project closed. The SOC triages alerts, the managed SOC partner monitors after hours, and security leadership reports "24/7 monitoring" to the board. Nobody's job description includes evaluating whether the rules detect the right things, testing rules against real attack data, tuning rules based on false positive analysis, or measuring coverage against the techniques that actually target the organization. The detection library is a deployment artifact, not an engineered system. It was configured once and never revisited.
Where detection engineering sits in security operations
Scenario
Your threat hunting team discovers that an attacker used OAuth consent grants to maintain persistent mailbox access for three weeks. The hunt finding goes into a report. The report recommends "implement detection for suspicious consent grants." Six months later, the next hunt discovers the same technique — still undetected. The recommendation is identical. What's missing?
Security operations has evolved into four distinct functions. Each does something different. Each depends on the others. Most organizations have three of them in some form. The fourth — detection engineering — is the one most organizations are missing, and its absence constrains everything else.
SOC operations
The function you're most likely to know already. The SOC monitors the alert queue, triages incidents, performs initial investigation, and escalates confirmed threats. The SOC analyst is the first responder — they receive the alert, evaluate whether it's real, determine its severity, and either close it or hand it off.
SOC operations is reactive by design: it responds to what the detection rules produce. The quality of SOC work is bounded by the quality of the alerts it receives.
A SOC operating against a detection library with a 60% false positive rate spends more time dismissing noise than investigating real threats. The SOC can't fix this imbalance by working harder. It can only be fixed by improving the detection rules upstream.
Threat hunting
Proactively searches for evidence of compromise that detection rules have missed. Where the SOC reacts to alerts, the threat hunter operates in the silence — the techniques and behaviors that produce telemetry but no alert.
Hunting is hypothesis-driven: "I believe attackers may be abusing OAuth consent grants to establish persistent access — let me query CloudAppEvents for anomalous consent patterns."
When a hunt produces a finding, that finding should become a detection rule. But in most organizations, it becomes a paragraph in a hunt report that nobody converts into automation.
Incident response
Investigates and contains confirmed compromises. The responder takes an escalated alert, determines the full scope, identifies every system the attacker touched, contains the threat, and leads remediation.
Incident response operates at the end of the detection pipeline — if detection is late, response is late. Every hour of dwell time the attacker gains before detection increases the blast radius.
IR teams routinely discover during post-mortems that telemetry existed at every phase of the attack chain. If a detection rule had queried that telemetry, the incident would have been caught days or weeks earlier.
Detection engineering
Builds and maintains the detection rules that the SOC consumes, that threat hunting evaluates for gaps, and that incident response depends on for early warning.
The detection engineer takes threat intelligence, hunt findings, incident post-mortems, and ATT&CK coverage analysis and turns them into production detection rules. Tested, tuned, documented, versioned, deployed, maintained.
The cycle
These four functions form a cycle. Threat intelligence and coverage analysis inform what the detection engineer builds. The detection rules feed the SOC. The SOC's triage results inform tuning. Threat hunting searches for what the rules miss. Incident response identifies what the rules should have caught.
When the detection engineering function is absent, the cycle breaks. Hunt findings become reports instead of rules. Incident post-mortems produce recommendations instead of deployed detections. The SOC operates against stale rules that nobody maintains.
Before reading further, check your own environment. This PowerShell shows you every analytics rule in your Sentinel workspace — when it was created, when it was last modified, and whether anyone has touched it since deployment:
If your output looks like this — rules created on the same day, never modified, averaging over a year old — you're looking at a template deployment that nobody has maintained. Those rules are not a detection program. They're a configuration that was completed once and then forgotten. The rest of this module quantifies the gap that creates.
What a detection engineer actually does
The daily work is not writing KQL all day. KQL is the implementation tool — maybe 25% of the work. The other 75% is the thinking that makes the implementation effective.
Threat modeling and prioritization
Which attack techniques are relevant to your organization? Not all 208 ATT&CK techniques. The subset that your industry's threat actors use, that your technology stack enables, and that your crown jewels make impactful.
The detection engineer maintains a prioritized backlog of techniques to build rules for, ranked by probability, impact, and data availability. This backlog drives the development sprint — what gets built next is determined by the threat model, not by the latest vendor advisory or the most recent incident.
Hypothesis development
For each prioritized technique, the detection engineer develops a hypothesis about what the technique looks like in the organization's telemetry.
Not "detect password spray" — that's a category, not a hypothesis. "Detect more than 15 failed authentication attempts from a single IP address against more than 5 distinct user accounts within a 10-minute window in SigninLogs" — that's a hypothesis. It's specific enough to implement, specific enough to test, and specific enough to be wrong.
Being wrong is useful: a hypothesis that fails testing reveals something about the technique or the environment that the next hypothesis incorporates.
Rule specification
Before writing any query, the detection engineer documents the rule: the hypothesis, the ATT&CK technique mapping, the data source and specific fields, the detection logic in plain language, the entity mapping rationale, the severity rationale, the expected false positive sources, and the analyst response procedure.
This specification is the engineering document. Without it, the rule is an undocumented query that only the person who wrote it understands.
Rule building
The KQL query. The entity mapping configuration. The scheduled rule parameters — how often it runs, how far back it looks, what threshold triggers an alert, how alerts are grouped into incidents.
This is where platform-specific skill matters: you need to know KQL well enough to express the detection logic efficiently, and you need to know Sentinel's rule architecture well enough to configure the rule correctly. Module 1 teaches rule architecture. Modules 3 through 8 build 71 rules.
Testing. Does the rule fire on the attack it claims to detect? Does it fire on legitimate activity that resembles the attack? These are two separate questions with two separate test methods. Historical testing runs the rule against past telemetry to see what it would have caught and what noise it would have produced. Simulation testing generates the specific attack behavior (using tools like Atomic Red Team or manual techniques) and verifies the rule fires with the correct entities, severity, and enrichment. A rule that hasn't been tested against the attack it claims to detect is a configuration. It is not a detection.
Tuning. The rule operates in production. Some alerts are true positives — the rule caught real attacker activity. Some are false positives — the rule caught legitimate activity that matched the pattern. Some are benign positives — the rule caught activity that is technically suspicious but authorized (an admin running PowerShell scripts during a maintenance window). Tuning classifies each false positive by root cause and applies the targeted fix: threshold adjustments for rules that fire too broadly, watchlist entries for specific authorized accounts, logic revisions for patterns that match unintended behaviors. Tuning is monthly, evidence-based, and documented.
Measurement. Four metrics track program health: ATT&CK coverage percentage (what proportion of relevant techniques have at least one detection rule), mean time to detect (how fast rules fire after an attack executes), false positive rate (what proportion of alerts are noise), and rule health score (what proportion of rules are operationally functional). These metrics are the detection program's accountability. They answer the question "is the program improving?" with data, not assertions.
Communication. The board report. The ATT&CK heatmap. The before-and-after comparison. Detection engineering produces measurable outcomes, and communicating those outcomes to leadership is part of the role. A detection engineer who builds excellent rules but can't demonstrate their impact to the CISO runs a program that loses funding. The capstone in Module 11 produces the board report.
Why this matters now
Detection engineering is not new as a concept. Organizations have been writing custom SIEM rules since SIEMs existed. What's changed is three things that make it a distinct, formalized discipline rather than an ad-hoc activity.
The threat landscape outpaced templates. Five years ago, enabling vendor-provided detection rules covered a reasonable proportion of the threats most organizations faced. Commodity attacks — brute force, known malware signatures, basic phishing — were the majority, and template rules detected them adequately. That's no longer true. AiTM phishing kits that bypass MFA, cloud-native persistence through OAuth consent grants, LLM-assisted social engineering that evades content filters, abuse of workload identities and AI agents, living-off-the-land techniques that use legitimate system tools to avoid endpoint detection — these techniques require environment-specific detection because the signals that distinguish attack from legitimate depend on the organization's specific traffic patterns, admin behaviors, and technology stack. A template rule for "suspicious PowerShell execution" that fires on any encoded command line produces 200 false positives per day at an organization whose admin team uses encoded PowerShell for legitimate automation. The same rule produces zero false positives and catches real attacks at an organization where nobody uses encoded PowerShell. The template can't know which environment it's in. The detection engineer does.
The Microsoft stack unified the data. Sentinel and Defender XDR place identity telemetry, endpoint telemetry, email telemetry, cloud application telemetry, and infrastructure telemetry in a single queryable workspace. For the first time, a detection engineer can write a single KQL query that joins a compromised authentication (SigninLogs) with an inbox rule creation (OfficeActivity) within 30 minutes — tracing two phases of an attack chain in one rule. This cross-family correlation was impossible when identity data lived in one SIEM, email data in a separate email gateway console, and endpoint data in an EDR platform with its own query language. The unified data model created the opportunity for detection rules that trace full attack chains across the kill chain. Detection engineering is the discipline that exploits it. The organizations that benefit most from Sentinel's unified model are the ones that write cross-family detection rules. The organizations that benefit least are the ones that enable templates and leave the cross-family data unexamined.
The profession formalized. Detection engineering is now a named role with a career path, a community, and an established body of practice. SpecterOps codified detection engineering methodology through their training and research programs. SANS developed SIEM engineering coursework. The ATT&CK framework provides the common language — a shared taxonomy that detection engineers use to describe what they build and measure what they cover. Detection-as-code practices borrowed from software engineering provide the operational model — version control, CI/CD deployment, automated testing, code review. Organizations from mid-market enterprises to Fortune 100 companies post detection engineer job listings. The role has a hiring pipeline, a defined skill set, competitive compensation, and a professional community that shares research, tooling, and methodology. Five years ago, the person writing custom SIEM rules was "the SOC lead who also does detection." Now the discipline has a name and a practice, and learning it is a career-defining investment.
The convergence of these three factors means that detection engineering is not an optional specialization — it's a necessary function for any organization that operates a SIEM. The alternative is template detection that covers a diminishing fraction of the threat landscape while the SOC operates against rules that nobody tests, tunes, or maintains. That alternative is the default state of most organizations today. This course exists because of the gap between that default state and what detection engineering makes possible.
Detection Engineering Principle
Detection engineering is the discipline that decides what your security operations program can see. The SOC, threat hunting, and incident response all depend on the rules that detection engineering builds. Without the function, the other three operate with degraded visibility — responding to what template rules happen to catch rather than what the threat landscape requires.
Get weekly detection and investigation techniques
KQL queries, detection rules, and investigation methods — the same depth as this course, delivered every Tuesday.
No spam. Unsubscribe anytime. ~2,000 security practitioners.