DE0.1 The Detection Coverage Illusion

The number nobody wants to hear

Twenty-three analytics rules. That is the detection library for Northgate Engineering — an 810-person precision manufacturing company with an M365 E5 license, Microsoft Sentinel deployed, and Defender XDR protecting endpoints, email, identity, and cloud apps. They have a managed SOC partner providing 24/7 monitoring. They passed their ISO 27001 surveillance audit. They have a CISO, a security team, and a budget.

Twenty-three rules. Twelve of them are Microsoft-provided templates, enabled as-is during the initial Sentinel deployment in 2023, never tuned, and never reviewed. Eleven are custom rules written by the CISO during the first year, covering specific scenarios she encountered — password spray, suspicious inbox rules, and a few identity anomalies. Basic threshold logic. No entity mapping beyond the minimum required to create an alert.

The MITRE ATT&CK enterprise matrix contains 201 techniques across 14 tactics (as of ATT&CK v15). Not all of them are relevant to every organization. Northgate Engineering operates primarily in a Windows and M365 environment, which narrows the relevant technique set. After removing techniques that require physical access to systems Northgate does not operate (mainframes, ICS protocols they do not use, mobile-specific techniques for platforms they do not deploy), approximately 145 techniques remain relevant to their threat landscape.

Twenty-three rules cover approximately 15 distinct ATT&CK techniques. Some rules detect the same technique through different data sources (two rules detecting password spray — one in SigninLogs, one via Defender for Identity alerts). Some rules do not map to any ATT&CK technique at all (a template rule that fires on “high severity alerts from any Microsoft product” is a pass-through, not a detection).

Fifteen techniques covered out of 145 relevant. That is 10.3% detection coverage.

The other 89.7% is invisible. No query runs against that telemetry. No alert fires when those techniques execute. The data exists in the workspace — DeviceProcessEvents captures the process execution, IdentityDirectoryEvents captures the LDAP enumeration, CloudAppEvents captures the OAuth consent grant — but no rule looks at it.

Figure DE0.1 — Northgate Engineering ATT&CK coverage baseline. Red bars show the proportion of techniques with at least one detection rule. The remaining space is unmonitored.

What “coverage” actually means

Detection coverage is not a count of analytics rules. An organization with 100 rules that all detect variants of password spray has 100 rules and coverage for one technique. An organization with 15 rules that each detect a different technique across different tactics has fewer rules and vastly better coverage.

Coverage is measured against the techniques that matter for your environment — not the entire ATT&CK matrix. A hospital does not need detection for techniques targeting industrial control systems. A manufacturing company does not need detection for techniques targeting mobile banking applications. The relevant technique set is determined by your industry, your technology stack, your threat intelligence, and your crown jewels.

The coverage assessment asks three questions for each relevant technique. First, does at least one analytics rule query for indicators of this technique? Second, does the rule query the correct data source with sufficient fidelity to distinguish the technique from normal activity? Third, has the rule been tested against real or simulated attack data to confirm it fires when the technique is executed?

A rule that exists but has never been tested is assumed coverage, not confirmed coverage. A rule that fires on every instance of a common Windows process (cmd.exe execution, for example) without filtering for the specific suspicious context (cmd.exe spawned by a Microsoft Office application with an encoded command line) is noise, not detection. The coverage assessment counts confirmed detection only.

Most organizations that conduct this assessment for the first time discover their confirmed coverage is lower than their assumed coverage. Rules that looked comprehensive on paper fail when tested. Template rules designed for generic environments fire constantly on legitimate activity in their specific environment — and the analysts have learned to ignore them.

Why template rules fail

Microsoft provides analytics rule templates in Sentinel’s content hub. These templates are valuable — they encode detection logic developed by Microsoft’s threat intelligence team and they cover common attack techniques with tested KQL. The problem is not the templates themselves. The problem is enabling them without adaptation.

A template rule for detecting impossible travel uses a fixed time threshold and a fixed distance threshold. It does not know that your organization has 120 remote workers who connect via VPN from residential ISPs that geolocate to different cities than their actual location. It does not know that your field engineers fly between Bristol and Sheffield three times a week and that their sign-in logs show two cities 200 miles apart within 30 minutes. The template fires. The analyst investigates. The analyst marks it as a false positive. The template fires again. The analyst stops investigating impossible travel alerts entirely.

That is the false positive tax. Every untuned template rule that generates false positives actively degrades your detection capability — not because the rule is wrong, but because the analyst response to the rule becomes “ignore it.” When the real impossible travel event occurs (an attacker signing in from São Paulo with credentials stolen from your Bristol-based finance director), the analyst ignores that alert too. The template rule has trained the analyst to not respond to the technique it was supposed to detect.

The second failure mode is the missing context. A template rule that detects “new inbox rule created” fires on every inbox rule creation in the organization. In an 810-person company, that is dozens of alerts per day — every user who creates a “move newsletters to a folder” rule triggers it. The rule detects the correct event type but lacks the contextual filtering that distinguishes an attacker creating a forwarding rule to an external address from a user organizing their inbox. The detection engineer adds that context: filter for rules that forward to external domains, exclude known internal distribution patterns, and correlate with recent sign-in risk signals. The template cannot do this because it does not know your environment.

The three layers of detection failure

Detection failure is not binary. Organizations do not have “detection” or “no detection.” They operate across three layers of failure, each progressively harder to identify.

Layer 1: No rule exists. The technique executes and no analytics rule queries for it. The data may exist in the workspace — DeviceProcessEvents captured the process, AuditLogs captured the directory change — but no rule looks at it. This is the most obvious gap and the easiest to close. You build a rule.

Layer 2: The rule exists but does not fire. The rule queries the correct data source but the KQL logic does not match the specific variant of the technique the attacker uses. A rule that detects Mimikatz by process name fails when the attacker renames the binary. A rule that detects password spray by counting failed logins per IP fails when the attacker distributes the spray across 500 residential proxy IPs at one attempt per IP. The rule exists. The technique executes. The rule does not fire. This is harder to identify because the rule’s existence creates the illusion of coverage.

Layer 3: The rule fires but the analyst does not respond. The rule detects the technique correctly. The alert appears in the incident queue. The analyst has seen 47 false positive alerts from this rule in the past month. The analyst closes the alert without investigation. The attacker proceeds. This is the hardest failure to identify because every system — the SIEM, the rule, the alert, the incident queue — functioned correctly. The failure is operational, not technical. It is also the most dangerous because it is invisible in any metric that counts rules or alerts.

Detection engineering addresses all three layers. Building rules closes Layer 1. Testing and precision engineering closes Layer 2. Tuning and false positive management closes Layer 3. A detection program that only builds rules without testing and tuning is solving one-third of the problem.

The cost of the gap

The detection gap has a dollar value. The 2025 IBM Cost of a Data Breach Report measured the relationship between detection capability and breach cost. Organizations that detected breaches through their own security tooling (internal detection) had significantly lower costs than organizations where the breach was disclosed by the attacker (ransomware notification) or by a third party (law enforcement, customer report). The difference is measured in hundreds of thousands of dollars and weeks of dwell time.

The mechanism is straightforward. An attacker who operates undetected for 30 days has 30 days to establish persistence, move laterally, identify crown jewels, stage data, and exfiltrate. An attacker detected at initial access — within the first hours — has accomplished initial foothold and possibly one lateral movement. The blast radius is smaller. The remediation is simpler. The business impact is contained.

Detection engineering compresses dwell time by expanding the set of techniques that generate alerts. Instead of detecting the attacker at the ransomware deployment phase (when the damage is irreversible), detection at the credential theft phase (when containment means revoking one session token) changes the outcome entirely.

This is the argument you make to the CFO. Not “we need more security tools” but “our current tools see 10% of what attackers do, and the 90% they miss is where the damage happens. Detection engineering expands that 10% to 35% in 90 days, targeting the techniques that cause the most expensive incidents in our industry.”

Troubleshooting: Common obstacles to coverage assessment

“We do not know which techniques are relevant to our environment.” Start with your industry. The MITRE ATT&CK website provides threat group profiles that show which techniques specific threat actors use. If you are a manufacturing company, look at threat groups that target manufacturing (APT41, Sandworm, APT28 for defense supply chain). The techniques those groups use are your relevant technique set. DE2 teaches this threat modeling methodology in depth.

“Our rules do not have ATT&CK mappings.” Many template rules include ATT&CK mappings in their configuration. Custom rules often do not. Start by reviewing each custom rule’s KQL query and manually mapping it to the technique it detects. If the rule queries SigninLogs for failed authentication counts, it maps to T1110 (Brute Force). If it queries DeviceProcessEvents for encoded PowerShell commands, it maps to T1059.001 (PowerShell). This manual mapping is time-consuming but essential — you cannot measure coverage without it.

“We have rules but we do not know if they work.” This is Layer 2 failure. The only way to validate is testing: either replay historical attack data through the rule, simulate the technique using tools like Atomic Red Team, or review past incidents to confirm the rule fired when the technique was used. If you cannot confirm a rule has ever fired on a true positive, it is assumed coverage, not confirmed coverage. DE9 teaches the testing methodology.

“Our managed SOC partner handles detection.” Your managed SOC partner triages alerts from the rules you deploy. If you deploy 23 rules, they triage alerts from 23 rules. They do not build new rules for your specific environment unless that service is explicitly contracted and scoped. The detection coverage is yours to own. The managed SOC partner is an operational resource — not a detection engineering function.

References used in this subsection

• MITRE ATT&CK Enterprise Matrix v15 (attack.mitre.org)
• Course cross-references: DE2 (threat modeling and prioritization), DE9 (testing methodology), DE0.2 (attack chain walkthrough)

Quantifying the coverage gap

1
2
3
4
5
6
7

// COVERAGE BASELINE: How many analytics rules exist in your Sentinel workspace?
SecurityAlert
| where TimeGenerated > ago(30d)
| summarize AlertCount = count() by AlertName
| summarize TotalRules = dcount(AlertName), TotalAlerts = sum(AlertCount)
// NE baseline: 23 rules, ~180 alerts/month
// After DE program: 66+ rules, structured alert pipeline

This baseline query is the STARTING POINT for every detection engineering program. Run it in your Sentinel workspace — the number of distinct AlertNames is your current rule count. Compare against the ATT&CK matrix: how many of the 208 enterprise techniques have at least one rule?