In this section
EI1.5 Conditional Access Evaluation in Sign-In Logs
The conditional access evaluation record
Every sign-in log entry contains a ConditionalAccessPolicies field — a JSON array where each element represents one conditional access policy that was evaluated during this sign-in. The evaluation record includes the policy's display name, its ID, whether the policy's conditions matched this sign-in, and what the enforcement result was.
The three possible enforcement results for each policy are:
Reading the ConditionalAccessPolicies array in KQL
The ConditionalAccessPolicies field is a JSON array that requires specific KQL patterns to parse effectively. Here is the fundamental extraction pattern:
// EI1.5 — Extract conditional access evaluation details per sign-in
// Shows every policy that evaluated and what it decided
SigninLogs
| where TimeGenerated > ago(24h)
| where ResultType == 0 // Successful sign-ins
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| extend PolicyName = tostring(CAPolicy.displayName)
| extend PolicyResult = tostring(CAPolicy.result)
| extend PolicyId = tostring(CAPolicy.id)
| extend GrantControls = tostring(CAPolicy.enforcedGrantControls)
| extend SessionControls = tostring(CAPolicy.enforcedSessionControls)
| project
TimeGenerated,
UserPrincipalName,
AppDisplayName,
PolicyName,
PolicyResult, // "success", "failure", "notApplied", "reportOnlySuccess", etc.
GrantControls, // What the policy required: ["mfa"], ["compliantDevice"], etc.
SessionControls // Session controls applied: sign-in frequency, token protection, etc.
| where PolicyResult in ("success", "failure", "reportOnlySuccess", "reportOnlyFailure")
| order by TimeGenerated desc
// mv-expand flattens the array — each row is now one policy evaluation
// Filter for success/failure to see policies that matched (exclude notApplied)The mv-expand operator is essential. It takes the array of policy evaluations and creates one row per policy per sign-in. Without it, the ConditionalAccessPolicies field is an opaque JSON blob. With it, you can filter, aggregate, and analyze individual policy evaluations across thousands of sign-ins.
Verifying specific policies
The most practical use of the conditional access evaluation data is verifying that your policies are working as designed. For each policy you deploy, you should run a verification query within the first 24-48 hours to confirm it is matching the intended sign-ins and producing the intended results.
Verify: "Require MFA for all users" policy
// EI1.5 — Verify MFA-for-all-users policy is evaluating
let policyName = "Require MFA for All Users"; // Replace with your policy name
SigninLogs
| where TimeGenerated > ago(24h)
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| where tostring(CAPolicy.displayName) == policyName
| extend PolicyResult = tostring(CAPolicy.result)
| summarize
Evaluated = count(),
Success = countif(PolicyResult == "success"),
Failure = countif(PolicyResult == "failure"),
ReportOnly = countif(PolicyResult has "reportOnly")
by bin(TimeGenerated, 1h)
| order by TimeGenerated asc
// Expected: Evaluated should be non-zero every hour
// Success should be high (users completing MFA)
// Failure should be low (users blocked because they could not complete MFA)
// If Evaluated is zero, the policy is not matching any sign-ins — check assignments// EI1.5 — Verify legacy auth block is working
// Should see failures (blocks) for legacy protocols and zero successes
let policyName = "Block Legacy Authentication"; // Replace with your policy name
SigninLogs
| where TimeGenerated > ago(7d)
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| where tostring(CAPolicy.displayName) == policyName
| extend PolicyResult = tostring(CAPolicy.result)
| summarize count() by PolicyResult
// Expected: "failure" count = legacy auth attempts that were blocked (good)
// "success" should NOT appear — if it does, the policy has an exception gap
// "notApplied" should appear for modern auth sign-ins (expected — they don't match)// EI1.5 — CRITICAL: Find sign-ins evaluated by ZERO conditional access policies
// These sign-ins bypassed your entire CA architecture
SigninLogs
| where TimeGenerated > ago(24h)
| where ResultType == 0 // Successful sign-ins only
| where ConditionalAccessStatus == "notApplied"
| summarize
GapCount = count(),
Users = make_set(UserPrincipalName, 10),
Apps = make_set(AppDisplayName, 10)
by bin(TimeGenerated, 1h)
| where GapCount > 0
| order by GapCount desc
// EVERY result here is a gap in your Zero Trust coverage
// Common causes: applications not targeted by any policy,
// user groups excluded from all policies, service accounts
// not covered by workload identity CAReport-only mode evaluation
Conditional access policies can be deployed in report-only mode, which evaluates the policy against every matching sign-in but does not enforce the controls. The evaluation is recorded in the sign-in log with result values "reportOnlySuccess" (the user would have satisfied the controls) and "reportOnlyFailure" (the user would have been blocked).
Report-only mode is essential for safe deployment — you can see exactly which sign-ins would be affected before turning on enforcement. EI8 covers the full report-only testing methodology. For this subsection, the key skill is reading the report-only results in the sign-in logs:
// EI1.5 — Analyze report-only policy impact
// Shows what would happen if you enforced a report-only policy
SigninLogs
| where TimeGenerated > ago(7d)
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| where tostring(CAPolicy.result) has "reportOnly"
| extend PolicyName = tostring(CAPolicy.displayName)
| extend PolicyResult = tostring(CAPolicy.result)
| summarize
WouldSucceed = countif(PolicyResult == "reportOnlySuccess"),
WouldBlock = countif(PolicyResult == "reportOnlyFailure")
by PolicyName
| extend BlockPercentage = round(100.0 * WouldBlock / (WouldSucceed + WouldBlock), 1)
| order by BlockPercentage desc
// Review: WouldBlock = users who would lose access if you enforce this policy
// If BlockPercentage is high, the policy scope may be too broad or users need
// remediation (device enrollment, MFA registration) before enforcement
// If BlockPercentage is zero, the policy is safe to enforce — all users complyThis query is the safety check you run before promoting any report-only policy to enforcement. If the BlockPercentage is 15%, that means 15% of matching sign-ins would be blocked — which means 15% of affected users would lose access. You need to investigate those users (are they on unmanaged devices? do they need an exception?) before enforcing the policy. EI8 covers this workflow in detail.
Combining CA evaluation with other signals
The real power of the conditional access evaluation data emerges when you combine it with other sign-in log fields. Here are three patterns that experienced identity security engineers use daily:
Pattern 1: Risky sign-ins that bypassed CA. A sign-in flagged as risky by Identity Protection that was not evaluated by any conditional access policy — meaning the risk signal was detected but no enforcement action was taken:
// EI1.5 — Risky sign-ins with no CA enforcement
SigninLogs
| where TimeGenerated > ago(24h)
| where RiskLevelDuringSignIn in ("medium", "high")
| where ConditionalAccessStatus == "notApplied"
| project TimeGenerated, UserPrincipalName, AppDisplayName,
IPAddress, RiskLevelDuringSignIn, RiskEventTypes_V2
// Every result is a risky sign-in that your CA architecture did not evaluate
// Fix: ensure risk-based CA policies cover all users and all applications// EI1.5 — MFA satisfied but with phishing-capable method
SigninLogs
| where TimeGenerated > ago(24h)
| where ResultType == 0
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| where tostring(CAPolicy.result) == "success"
| where tostring(CAPolicy.enforcedGrantControls) has "mfa"
| extend AuthDetail = parse_json(AuthenticationDetails)
| extend PrimaryMethod = tostring(AuthDetail[0].authenticationMethod)
| where PrimaryMethod == "Password" // Not phishing-resistant
| extend MFAMethod = tostring(AuthDetail[1].authenticationMethod)
| project TimeGenerated, UserPrincipalName, AppDisplayName,
PolicyName = tostring(CAPolicy.displayName), MFAMethod
| where MFAMethod !in ("FIDO2 security key", "Passkey (Microsoft Authenticator)")
// Results: users meeting MFA requirement with phishing-capable methods
// These users are protected against spray but vulnerable to AiTM// EI1.5 — Successful sign-ins from non-compliant/unmanaged devices
SigninLogs
| where TimeGenerated > ago(24h)
| where ResultType == 0
| where DeviceDetail_dynamic.isCompliant != true
or isempty(DeviceDetail_dynamic.isCompliant)
| where AppDisplayName in ("Office 365 Exchange Online",
"Office 365 SharePoint Online", "Microsoft Teams")
| project TimeGenerated, UserPrincipalName, AppDisplayName,
DeviceOS = tostring(DeviceDetail.operatingSystem),
IsCompliant = tostring(DeviceDetail.isCompliant),
IsManaged = tostring(DeviceDetail.isManaged),
ConditionalAccessStatus
// Results: users accessing sensitive apps from unmanaged devices
// If your policy should require compliance, these are the exceptions to investigateSigninLogs | where TimeGenerated > ago(24h) | where ResultType == 0 | where ConditionalAccessStatus == "notApplied" | countTry it yourself
Try It — Find Your CA Coverage Gaps
Environment: Your M365 developer tenant with Sentinel workspace.
Exercise: Your developer tenant likely has no conditional access policies configured yet (unless you created some in a previous session). Run the gap-finding query:
The result should be the total number of successful sign-ins — because without any CA policies, every sign-in has ConditionalAccessStatus "notApplied." This is your baseline: 100% gap. By the time you complete EI3 and EI4, you will have policies in place and this number should drop dramatically. Run this query again after deploying policies to measure the improvement.
The myth: We configured conditional access policies. All sign-ins to our M365 environment are evaluated and protected.
The reality: Conditional access policies only evaluate sign-ins that match their scope — the combination of user assignments, application targets, conditions, and exclusions. A policy that targets "All Users" but only applies to "Office 365" applications will not evaluate sign-ins to Azure Portal, Azure DevOps, Power BI, or custom applications. A policy that excludes a "Break-Glass Accounts" group will not evaluate sign-ins from those accounts. A policy that only applies to Windows and macOS will not evaluate sign-ins from Linux or mobile devices if they are not included. The ConditionalAccessPolicies array in the sign-in log reveals exactly which policies evaluated each sign-in — and the gap query reveals which sign-ins were not evaluated by any policy. EI8 teaches the systematic approach to closing these gaps.
A sign-in log shows a successful authentication from an IP in a country where NE has no employees. MFA was satisfied by push notification. The user says they approved the MFA prompt while traveling. Do you accept this explanation?
Verify, do not accept at face value. Check: does the user have a travel request on file? Does the IP geo-location match the claimed travel destination? Does the device fingerprint match the user's enrolled device? Are there other sign-ins from NE's corporate IP in the same time window (which would indicate the user is NOT traveling)? An attacker who stole credentials and is bombarding the user with MFA prompts (MFA fatigue) gets the same 'I approved it' response from a confused user. The investigation confirms or refutes the travel explanation within 5 minutes.
You've mapped the identity threat landscape and learned to read sign-in logs.
EI0 established that every cloud attack starts with identity. EI1 took you through the signal that matters most — interactive, non-interactive, service principal, and managed identity sign-ins. Now you engineer the defences.
- 17 engineering modules — authentication methods, conditional access architecture, Identity Protection, PIM, token protection, application governance, and detection rules
- The Defense Design Method — the six-step framework applied to every identity control you'll build
- EI18 Capstone — Identity Security Architecture Design — design complete identity architectures for three realistic organisations (SMB, mid-market, regulated enterprise)
- Identity Security Toolkit lab pack — deployable conditional access policies, PIM configurations, and Identity Protection risk rules
- Cross-domain detection (EI16) — email-to-identity correlation and the full phishing-to-inbox-rule attack chain