ES0.4 Key Metrics: MTTD, MTTR, and What Actually Matters

Figure ES0.4a — Microsoft Secure Score provides a posture assessment across Identity (67.82%), Data (100%), Device (48.11%), and Apps (44.19%). The top recommended actions are endpoint security controls: ASR rules (block Office child processes, block credential stealing from LSASS, block unsigned processes) and AV real-time protection. A Device score of 48% indicates that over half of available endpoint security controls are not yet deployed — the gap this course closes.

Why MTTD and MTTR are necessary but insufficient

Mean Time to Detect (MTTD) and Mean Time to Respond (MTTR) are the most commonly reported SOC metrics. They appear in every vendor pitch, every analyst report, and every board-level security presentation. They matter because they measure speed — and speed matters during an active incident. A SOC that detects an AiTM attack in 8 minutes and contains it in 15 minutes produces a fundamentally different outcome than one that takes 4 hours for each.

The problem is that MTTD and MTTR can be optimized without improving security. Auto-closing low-severity alerts reduces MTTR to near zero for those alerts — but the alerts are closed, not investigated. Detecting only commodity threats (for which MDE has built-in detections) produces a fast MTTD for those threats — but says nothing about detection capability for targeted attacks. A SOC that handles 95% of alerts in under 10 minutes but misses the 5% that represent actual compromise has excellent MTTD/MTTR numbers and a catastrophic security posture.

The metrics that drive endpoint security engineering decisions are different. They measure what the stack actually prevents, what it actually detects, and what coverage gaps remain.

The metrics that drive endpoint security engineering

Prevention metrics measure whether your prevention controls are configured and effective:

The percentage of ASR rules in block mode across the fleet tells you how much of the prevention layer is actually preventing. If 3 of 18 ASR rules are in block mode on 60% of devices, your prevention coverage is approximately 10% of its potential. The target is not 100% of rules in block mode — some rules have legitimate business conflicts that keep them in audit or warn mode permanently. But the metric tracks progress.

AV cloud protection level across the fleet reveals how many devices are operating with enhanced protection. Default level is functional. High+ catches significantly more unknown threats. The metric shows you the gap between what your license provides and what your configuration delivers.

Detection metrics measure whether your detection layer finds real threats:

The total custom detection rule count matters because zero custom rules means you depend entirely on Microsoft’s built-in detections. Built-in detections are good for commodity threats but do not cover your organisation’s specific patterns, your environment’s normal baselines, or the targeted techniques that your threat model identifies as highest risk.

The hunting cadence — how frequently analysts execute proactive hunting queries — measures whether the detection layer extends beyond automated rules. A SOC that hunts weekly finds threats that scheduled rules miss because the hunt can adapt to emerging intelligence in real time.

Forensic readiness metrics measure whether evidence will exist when needed:

The percentage of endpoints with advanced audit policies, PowerShell ScriptBlock logging, and Sysmon configured tells you whether your IR team can reconstruct attacker actions after containment. This metric is binary for each endpoint — either the logging is configured or it is not. Partial logging (audit policy configured but PowerShell logging missing) creates evidence gaps.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

DeviceEvents
| where ActionType startswith "Asr"
| summarize
    BlockEvents = countif(ActionType endswith "Blocked"),
    AuditEvents = countif(ActionType endswith "Audited")
    by DeviceName
| summarize
    DevicesWithBlocks = dcountif(DeviceName, BlockEvents > 0),
    DevicesAuditOnly = dcountif(DeviceName, BlockEvents == 0 and AuditEvents > 0),
    TotalDevices = dcount(DeviceName)

1
2
3
4
5
6
7
8
9

AlertInfo
| where Timestamp > ago(30d)
| where DetectionSource == "CustomDetection"
| summarize
    TotalAlerts = count(),
    TruePositives = countif(Classification == "TruePositive"),
    FalsePositives = countif(Classification == "FalsePositive"),
    BenignTruePositives = countif(Classification == "BenignTruePositive"),
    ByRule = dcount(Title)

1
2
3
4
5
6
7

DeviceInfo
| where Timestamp > ago(1d)
| summarize arg_max(Timestamp, *) by DeviceId
| summarize
    TotalDevices = count(),
    HealthyDevices = countif(SensorHealthState == "Active"),
    OnboardedDevices = countif(OnboardingStatus == "Onboarded")

Troubleshooting

“Our CISO only cares about MTTD/MTTR — they do not want to hear about ASR enforcement percentages.” Present both. Lead with the metric the CISO cares about (MTTD/MTTR), then show the metric that explains it. “Our MTTR improved by 15% this quarter because we moved 6 ASR rules from audit to block mode, preventing 340 incidents that would have required analyst response.” The CISO gets the number they want. You get the engineering investment you need. The ASR enforcement metric explains the MTTR improvement, creating a clear cause-and-effect narrative that justifies continued investment.

“We do not have enough data to calculate meaningful metrics — we only have 3 months of MDE telemetry.” Three months is sufficient for most endpoint security metrics. ASR audit data stabilises within 2-4 weeks (you can identify false positive patterns and make enforcement decisions). Device health metrics are real-time. Exposure score is calculated daily. The only metric that benefits from longer data is hunting effectiveness, which requires baseline establishment. Start measuring now with whatever data you have. The trend matters more than the absolute number.

“Our managed SOC provides metrics in their monthly report — why do we need our own?” Because your MSSP’s metrics measure their performance, not your security posture. MSSP reports show: alerts handled, mean response time, incidents escalated. They do not show: ASR enforcement coverage, custom detection rule TP rate, technique coverage gaps, forensic readiness completeness. You need both sets of metrics — the MSSP’s operational metrics and your own engineering metrics — to have a complete picture.

Building the measurement cadence

Metrics without cadence are snapshots that never become trends. The measurement cadence for NE’s endpoint security program:

Daily (automated): Device health summary query (ES2.8) — total devices, active sensors, inactive sensors. AV health query — devices with stale signatures. These run as saved queries or Sentinel automation rules and alert on threshold violations. No analyst time required unless an alert fires.

Weekly (15 minutes): ASR enforcement summary — how many rules are in block mode across the fleet, how many block events occurred this week (by rule). Hunting cadence check — were the weekly hunting queries executed? Any findings? This is a quick status check during the security team standup, not a deep analysis.

Monthly (2 hours): Custom detection performance review — for each active custom detection rule, calculate the TP rate over the last 30 days. Rules below 30% TP rate are flagged for tuning. Rules above 80% TP rate are candidates for automated response actions. Exposure score trend — is the score improving (hardening efforts reducing attack surface) or degrading (new vulnerabilities being introduced faster than remediation)? Generate the executive dashboard for the CISO’s monthly report.

Quarterly (half day): ATT&CK technique coverage mapping — which techniques in the threat model are covered by detections (built-in + custom)? Which gaps remain? This drives the detection engineering backlog for the next quarter. Maturity model re-assessment — score each layer of the architecture against the maturity model from ES0.9. Compare to the previous quarter’s scores. The delta is the progress metric that justifies continued investment.

The cadence ensures metrics are not a one-time assessment but a continuous measurement system that tracks improvement and catches degradation before it becomes a security incident.