NF0.3 The NSM Philosophy

The Core Principle: Collect Before the Incident

NSM is not a response capability. It's a preparation capability. The data collection happens before anyone knows an incident is occurring, and the investigation happens after.

The fundamental problem with reactive network forensics — "we'll capture traffic when we see something suspicious" — is that it can't travel backward in time. When the SOC analyst sees a Defender XDR alert at 14:00 and decides to start a packet capture, the attacker's initial access at 09:00 is already five hours in the past. The C2 beacon that's been running since last week is only captured from now forward. The DNS query that resolved the attacker's phishing domain happened three days ago. Reactive capture gives you evidence of the investigation, not evidence of the attack.

NSM inverts this. The Zeek sensor was running at 09:00 when the attacker sent the phishing email. It was running last week when the C2 beacon started. It was running three days ago when the phishing domain was resolved. The conn.log, dns.log, and ssl.log entries for all of these events were captured as they happened and stored on the sensor. When the SOC analyst gets the alert at 14:00, the evidence is already there — waiting to be queried.

This isn't a novel concept. You already do this with endpoint forensics: Windows Event Logs collect continuously, SIEM forwarders run continuously, EDR telemetry streams continuously. Nobody suggests starting event log collection only after an incident is detected. NSM applies the same principle to the network layer.

The Three Data Categories

Richard Bejtlich's original NSM framework organizes network evidence into three categories. These map directly to the five evidence types from NF0.2, but the categories clarify the relationship between them.

Full content data is the raw packet capture — the PCAP files that record every byte on the wire. This is the most detailed evidence and the most expensive to store. You use full content data when you need to reconstruct exactly what happened in a specific session: extract a downloaded file, read an unencrypted HTTP request, or prove in legal proceedings that specific bytes were transmitted. In the NF0.2 taxonomy, this is Type 1 (Full PCAP).

Extracted content is objects pulled from the network streams — files transferred over HTTP, email attachments sent over SMTP, certificates presented during TLS handshakes. Zeek's file extraction capability and tools like NetworkMiner produce extracted content from full PCAP. The value is that extracted content is pre-processed: instead of searching through a 2 TB PCAP file for a malware binary, you have the binary already extracted and hashed. The cost is that extraction requires parsing, and parsed extraction can miss content that uses uncommon encoding or non-standard protocol implementations.

Session and transaction data is the structured metadata about network activity — Zeek logs, NetFlow records, IDS alerts. This is the evidence you query first and query most. Session data tells you that a connection existed, how long it lasted, how much data was transferred, what protocol was used, and what the IDS thought about it. It doesn't tell you what the bytes contained. In the NF0.2 taxonomy, this covers Types 2 through 5 (Zeek metadata, IDS alerts, NetFlow, DNS logs).

The practical relationship: session data answers most investigation questions. When it doesn't, extracted content narrows the search. When you need byte-level proof, full content provides it. You work from the cheapest evidence to the most detailed, and you stop as soon as your question is answered.

What Makes NSM Work in Production

Deploying tools is the easy part. The operational practices below determine whether your NSM deployment produces useful evidence or expensive noise.

The first operational practice is sensor health monitoring. A Zeek sensor that crashes at 03:00 and isn't restarted until the next business day creates a gap in your evidence. If the attacker's lateral movement happened during that window, you have nothing. NSM sensors need monitoring — process watchdogs, disk space alerts, packet loss counters. The sensor is an evidence source. Treat it with the same availability requirements as your SIEM.

The second practice is retention policy alignment. Your retention periods need to match your detection capability. If your average time to detect an intrusion is 21 days (the industry median for organizations without active threat hunting), your Zeek log retention needs to be at least 21 days — ideally 60–90 days to cover the full investigation. Retaining 7 days of metadata when your detection takes 21 days means you've lost the first 14 days of every investigation. PCAP retention is shorter (days to weeks), but Zeek metadata should cover the full expected dwell time.

The third practice is regular query exercises. If nobody queries the Zeek logs between incidents, two things happen: the team loses proficiency with the query tools, and configuration drift goes undetected. A Zeek parser that stopped handling HTTP/2 traffic after an update goes unnoticed for months — until the investigation needs HTTP/2 data that was never parsed. Regular hunting exercises (weekly or biweekly) keep both the team and the sensor sharp.

The fourth practice is baseline profiling. You can't identify anomalous network behavior without knowing what normal looks like. How many DNS queries does a typical workstation make per hour? What's the normal connection duration for HTTPS traffic to your SaaS providers? What internal servers normally communicate with external IPs? Baselines are built from Zeek logs over time, and they're what make network threat hunting possible. Module NF11 covers baseline building and hunting methodology.

Common Failure Modes

These are the patterns that reduce NSM from a monitoring philosophy to an expensive log generator. Recognizing them is the first step to avoiding them.

Deploy-and-forget. The Suricata sensor was installed two years ago. The rules haven't been updated since. Zeek was deployed but the JSON output format was never configured, so the logs aren't ingesting into the SIEM. The PCAP disk filled up six months ago and nobody noticed. Deploy-and-forget is the most common NSM failure mode because it's invisible — the tools appear to be running until someone actually needs the evidence.

Alert-only monitoring. The organization monitors Suricata alerts but doesn't collect or query Zeek metadata. This means they detect known-bad signatures but can't investigate anything that doesn't match a rule. When the alert fires for "ET MALWARE Cobalt Strike Beacon," they know it's bad, but they can't use conn.log to find every other host that communicated with the same C2 server, because they don't have conn.log.

No baseline, no hunting. The Zeek data exists but nobody queries it between incidents. Without baseline profiling, the team can't tell whether 500 DNS queries per hour from a workstation is normal or anomalous. Without regular hunting, attackers with low-and-slow techniques go undetected for months. The data was there the whole time — nobody looked.

Retention mismatch. The organization detects incidents an average of 30 days after initial compromise, but their PCAP retention is 5 days and their Zeek retention is 14 days. Every investigation starts with a 16-day evidence gap. The first two weeks of attacker activity — the initial access, the credential theft, the early lateral movement — are gone before the investigation begins.