5.8 Threat Hunting and KQL Generation with Copilot

Threat Hunting and KQL Generation with Copilot

Introduction

Threat hunting is the proactive search for threats that have not triggered detection rules. It requires three skills: forming hypotheses (what might an attacker be doing in my environment?), translating hypotheses into queries (KQL that searches for the hypothesised activity), and interpreting results (do the query results indicate a real threat or normal activity?). Copilot accelerates all three.

Hypothesis generation

The hunting process starts with a hypothesis — a specific, testable statement about potential attacker activity. Without Copilot, hypotheses come from threat intelligence reports, MITRE ATT&CK framework review, and analyst experience. With Copilot, you can generate hypotheses conversationally.

Broad prompt: “Based on current threat intelligence, what are the top 5 threat hunting hypotheses for a Microsoft 365 environment with Defender for Endpoint and Sentinel?”

Copilot generates hypotheses such as:

“Look for OAuth application registrations with suspicious redirect URIs — this is the consent phishing technique (T1528) used by threat actors like Storm-1167 to establish persistent access.”

“Search for users with MailItemsAccessed events from IPs that differ from their sign-in IPs — this indicates session token theft or AiTM proxy access.”

“Check for PowerShell execution on servers that do not normally run PowerShell — this detects living-off-the-land (LOLBin) activity.”

“Search for bulk file downloads from SharePoint by users who are flagged as departing employees in HR systems — this detects pre-resignation data theft.”

“Look for DNS queries to recently registered domains from internal hosts — this detects C2 communication with newly stood-up infrastructure.”

Each hypothesis is specific enough to translate into a KQL query. The analyst evaluates which hypotheses are relevant to their environment and proceeds with the most promising ones.

Natural language to KQL translation

For each selected hypothesis, Copilot translates the hunting concept into executable KQL.

Prompt: “Write a KQL hunting query for hypothesis 2: find users where MailItemsAccessed events come from different IPs than their sign-in events in the last 7 days.”

Copilot generates:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

// Hunting: MailItemsAccessed from IPs not matching sign-in IPs
let signinIPs = SigninLogs
| where TimeGenerated > ago(7d)
| where ResultType == "0"
| summarize SigninIPs = make_set(IPAddress, 50) by UserPrincipalName;
CloudAppEvents
| where Timestamp > ago(7d)
| where ActionType == "MailItemsAccessed"
| join kind=inner signinIPs on $left.AccountId == $right.UserPrincipalName
| where not(IPAddress in (SigninIPs))
| summarize SuspiciousAccessCount = count(),
    SuspiciousIPs = make_set(IPAddress, 10),
    FirstSeen = min(Timestamp),
    LastSeen = max(Timestamp)
    by AccountId
| where SuspiciousAccessCount > 5
| order by SuspiciousAccessCount desc

Analyst validation: The query logic is sound — it finds MailItemsAccessed events from IPs that are not in the user’s sign-in IP set. Potential issue: the join condition uses AccountId vs UserPrincipalName, which may not match if the formats differ (UPN vs account ID). The analyst tests the join and adjusts if needed.

Result interpretation: If the query returns results, each row is a user whose mailbox was accessed from an IP they did not sign in from. This could be: a session token replay attack (the attacker captured a token and is using it from their own IP), an API-based access using a stolen OAuth token, or a legitimate scenario (a user accessing email through a VPN that uses a different exit IP than their browser sign-in).

The analyst investigates each result: check the suspicious IPs against threat intelligence, check the user’s recent risk events, and check whether the access pattern matches a known attack technique. Copilot can assist with this triage: “Investigate IP 203.0.113.47 — check threat intelligence, geolocation, and whether it appears in any other alerts.”

Query explanation and optimisation

Copilot can explain existing KQL queries — a capability that is valuable when reviewing queries written by other analysts, inherited from previous team members, or found in Sentinel Content Hub solutions.

Prompt: “Explain this KQL query line by line:” [paste a complex query]

Copilot provides: a plain-language description of what each line does, the overall purpose of the query, any potential performance issues (inefficient joins, missing time filters), and suggestions for optimisation.

This is particularly valuable for junior analysts who are building KQL proficiency. Instead of struggling to decode a complex query, they ask Copilot to explain it, learn from the explanation, and build their understanding incrementally. Over time, they need Copilot’s explanations less frequently — because they have learned KQL through the explanations.

Optimisation prompt: “This query takes 45 seconds to run. How can I optimise it?”

Copilot analyses the query for common performance issues: missing time filters (scanning more data than necessary), inefficient join strategies (full table scans vs filtered joins), unnecessary columns in intermediate results (projecting unused fields), and suboptimal operator ordering (filtering after joining vs filtering before joining). The analyst applies the suggested optimisations and re-runs.

Iterative hunting with Copilot

The most effective Copilot-assisted hunting workflow is iterative: generate a broad query, review results, refine based on what you find, and repeat until you either find a threat or exhaust the hypothesis.

Iteration 1: “Show me all OAuth application registrations in the last 30 days.” Results: 47 applications. Too many to review individually.

Iteration 2: “Refine to only show applications with redirect URIs to non-corporate domains.” Results: 8 applications with external redirect URIs.

Iteration 3: “For these 8 applications, show me the registration date, the registering user, and the permissions requested.” Results: 6 were registered by IT admins (expected). 2 were registered by a user in Marketing with mail.read and files.readwrite permissions.

Iteration 4: “Check the sign-in activity for the Marketing user who registered these applications. Were there any risky sign-ins before the registration?” Results: A medium-risk sign-in from an unusual location 2 hours before the first application registration.

The hunt has found something: a possibly compromised user registered OAuth applications with data access permissions. This warrants a full investigation. Copilot accelerated each iteration — the analyst provided the investigative direction, Copilot provided the KQL.

KQL explanation for inherited and complex queries

SOC teams inherit queries from multiple sources: Sentinel Content Hub solutions, analytics rules created by previous team members, community hunting queries from GitHub repositories, and vendor-provided detection logic. Understanding these queries is essential for maintenance and tuning — but complex queries with multiple joins, nested lets, and advanced operators can take 30 minutes to parse manually.

Prompt: “Explain this KQL query line by line. What does it detect? What are the key tables, filters, and logic?”

Copilot deconstructs the query into sections: the let statements (defining intermediate datasets), the main query body (the detection logic), the join operations (how tables are correlated), the where filters (what is included and excluded), and the project/extend operations (what output columns are produced). The explanation identifies the detection intent: “This query detects users who authenticate from two countries within 30 minutes by joining sign-in events with a self-join filtered by time window and geographic distance.”

For exam preparation: The SC-200 exam includes questions where you are shown a KQL query and asked what it detects or what the output represents. Practising with Copilot’s query explanations builds the skill of reading KQL fluently — even if you do not use Copilot during the exam itself.

Hunting for false positives with Copilot

Proactive hunting is not only for finding threats — it is also for finding false positive patterns that degrade detection quality. Analytics rules that generate excessive false positives waste analyst time and create alert fatigue.

Prompt: “Analyse the alerts generated by analytics rule ‘Brute force SSH attempt’ in the last 30 days. How many were true positives vs false positives? What patterns do the false positives share?”

Copilot queries the SecurityAlert table for alerts from this rule, cross-references with incident outcomes (were the incidents classified as true positive or false positive?), and identifies common characteristics of false positives: “78% of alerts from this rule were closed as false positive. The most common false positive pattern: the source IP 10.0.1.254 (your VPN concentrator) generates 40 alerts per day because of authentication retry behaviour. Excluding this IP would reduce alert volume by 78% without affecting true positive detection.”

This analysis — which would take an analyst an hour of manual review — takes Copilot 30 seconds. The analyst validates the finding by checking a sample of the identified false positives, then creates the exclusion rule.

The Copilot-assisted hunting cycle

Threat intelligence enrichment in hunting

During hunting, you frequently encounter entities (IPs, domains, file hashes) that need threat intelligence context. Copilot queries the Microsoft Defender Threat Intelligence plugin to enrich these entities automatically.

Prompt: “Check these 5 IP addresses against threat intelligence: 203.0.113.47, 198.51.100.77, 192.0.2.15, 45.33.32.156, 104.26.10.229”

Copilot returns enrichment for each IP: reputation score, associated threat actors (if known), geographic location, whether the IP appears in known C2 infrastructure databases, and recent malicious activity attributed to the IP.

Without Copilot, this enrichment requires navigating to a threat intelligence portal, querying each IP individually, and recording the results. With Copilot, all 5 IPs are enriched in a single response — saving 10-15 minutes of portal navigation and manual lookup.

Hunting workflow integration: When a hunting query returns suspicious IPs or domains, feed them into Copilot for immediate enrichment. If any return positive TI matches, the hunting finding is elevated from “suspicious” to “high confidence” and warrants escalation to a formal investigation.

From hunting finding to analytics rule: the Copilot-assisted pipeline

The most valuable outcome of a successful hunt is not just finding a threat — it is turning the hunting query into an analytics rule that detects the same pattern automatically in the future. Copilot accelerates this conversion.

Step 1: Hunting query produces results. Your hunting query for OAuth applications with suspicious redirect URIs found 2 suspicious applications registered by a potentially compromised user.

Step 2: Convert to analytics rule. Prompt Copilot: “Convert this hunting query into a Sentinel scheduled analytics rule. Set the rule to run every 4 hours with a 4-hour lookback. Set severity to High. Map the user entity to AccountId and the application entity to the application name.”

Copilot generates the complete analytics rule configuration: the KQL query (adapted from the hunting query with appropriate time filters), the schedule (every 4 hours), the lookback (4 hours), the severity (High), and the entity mapping (user and application entities mapped to the correct fields).

Step 3: Validate and test. Run the rule query against 30 days of historical data to check for false positives. If the query produces too many results, refine: “Modify the rule to exclude applications registered by members of the IT Admin group, and only alert when the redirect URI is to a domain not in our approved domain list.”

Copilot adjusts the query to add the exclusion conditions. The analyst provides the IT Admin group name and the approved domain list. The modified query is tested again.

Step 4: Activate. After testing confirms acceptable false positive rates, activate the rule. What started as a manual hunt is now an automated detection — and the entire conversion took 5 minutes with Copilot assistance versus 20-30 minutes of manual rule configuration.

Hunting cadence with Copilot

Copilot changes the practical hunting cadence for SOC teams. Without Copilot, threat hunting is a time-intensive activity that most teams schedule weekly or monthly — one analyst dedicates a full day to hunting, running 5-10 queries, and investigating results.

With Copilot, hunting becomes a daily micro-activity. Each analyst spends 15-30 minutes per day running Copilot-assisted hunts: generate 2-3 hypotheses from current threat intelligence, translate each to KQL, run the queries, triage results with Copilot enrichment, and bookmark anything interesting. The daily micro-hunt catches threats faster than the weekly marathon hunt because the detection gap (time between threat arrival and hunting discovery) is reduced from 7 days to 1 day.

Recommended daily hunting workflow:

Start of shift: ask Copilot “Based on recent Microsoft Threat Intelligence, suggest 3 hunting hypotheses relevant to my environment.” Copilot generates hypotheses from current TI. Select the most relevant hypothesis. Copilot generates the hunting KQL. Run the query. If results appear, triage with Copilot enrichment. If a finding warrants investigation, escalate. If the hypothesis is exhausted, move to the next. End of the 15-minute hunting window.

This daily cadence is only practical with Copilot — without it, generating hypotheses, writing queries, and enriching results takes too long for a 15-minute window. With Copilot, the entire cycle fits into a brief daily session.

Natural language query refinement: the conversation that narrows

The most effective Copilot hunting pattern is the natural language refinement loop — starting broad and narrowing through conversation.

Round 1 — broad: “Show me all users who accessed more than 100 files in SharePoint yesterday.” Result: 47 users. Too many — most are legitimate heavy users.

Round 2 — filter by context: “From those 47 users, show only those who also sent emails to external recipients yesterday.” Result: 12 users. Better — cross-referencing file access with external email narrows to users who both accessed data and potentially exfiltrated it.

Round 3 — add anomaly detection: “From those 12 users, show only those whose file access volume yesterday was more than 3x their 30-day daily average.” Result: 3 users. These are the genuine anomalies — users who both accessed unusual volumes of data AND sent external emails.

Round 4 — enrich: “For these 3 users, check their recent risk events in Entra ID, their HR status (if available through the HR connector), and whether they triggered any DLP alerts yesterday.” Result: 1 user flagged — submitted resignation 2 weeks ago, 5x file access volume, DLP alert for sending a document labeled “Confidential” to a personal Gmail address.

Four conversational rounds, each building on the previous, narrowed from 47 users to 1 high-confidence insider risk finding. Each round took 30 seconds with Copilot (prompt + response + quick review). Total hunting time: 2 minutes for a finding that would take 30+ minutes of manual KQL work.

The key insight: each refinement prompt was a natural language question, not a KQL modification. The analyst thought in investigation terms (“who also sent external emails?”), not in query language terms (“add a join to the EmailEvents table”). Copilot translated the investigative thinking into query operations. This natural language refinement loop is how Copilot makes hunting accessible to analysts who think clearly about threats but are still developing KQL fluency.

Knowledge check