AD5.8 Sign-In Log Review for Early Detection

The weekly sign-in log review

Navigate to entra.microsoft.com → Monitoring → Sign-in logs. Set the date range to "Last 7 days."

Filter 1 — Risky sign-ins. Add filter: "Risk level: High OR Medium." These are sign-ins that Entra ID Protection flagged as suspicious based on behavior analysis. Each one needs investigation: click the entry, check the IP geolocation, check the device, and verify with the user if the sign-in is not obviously legitimate.

Filter 2 — CA failures. Add filter: "Conditional access: Failure." These are sign-ins blocked by your CA policies. Most are CA003 blocks (non-compliant device) — legitimate users on personal devices being redirected to app protection. But blocks from unfamiliar IPs or at unusual times warrant investigation.

Filter 3 — MFA method changes. Navigate to entra.microsoft.com → Monitoring → Audit logs. Filter by Activity: "User registered security info" OR "User deleted security info." These show MFA method registrations and deletions. An attacker who compromises an account often registers a new MFA method (a phone number they control) to maintain persistent access. A new MFA registration for a user who didn't initiate it is a red flag.

PowerShell for a comprehensive weekly check:

Connect-MgGraph -Scopes "AuditLog.Read.All"
$weekAgo = (Get-Date).AddDays(-7).ToString("yyyy-MM-ddTHH:mm:ssZ")

# Risky sign-ins
Write-Host "=== RISKY SIGN-INS ===" -ForegroundColor Red
Get-MgAuditLogSignIn -Filter "createdDateTime ge $weekAgo and riskLevelDuringSignIn ne 'none' and riskLevelDuringSignIn ne 'hidden'" -Top 50 |
    Select-Object CreatedDateTime, UserPrincipalName, RiskLevelDuringSignIn,
        @{N="IP";E={$_.IpAddress}},
        @{N="Location";E={"$($_.Location.City), $($_.Location.CountryOrRegion)"}},
        @{N="MFA";E={$_.MfaDetail.AuthMethod}} |
    Format-Table -AutoSize

# CA failures from unfamiliar sources
Write-Host "`n=== CA FAILURES ===" -ForegroundColor Yellow
Get-MgAuditLogSignIn -Filter "createdDateTime ge $weekAgo and conditionalAccessStatus eq 'failure'" -Top 50 |
    Where-Object { $_.Location.CountryOrRegion -notin @("GB","United Kingdom") } |
    Select-Object CreatedDateTime, UserPrincipalName,
        @{N="IP";E={$_.IpAddress}},
        @{N="Country";E={$_.Location.CountryOrRegion}} |
    Format-Table -AutoSize

# After-hours successful sign-ins (22:00-06:00)
Write-Host "`n=== AFTER-HOURS SIGN-INS ===" -ForegroundColor Cyan
Get-MgAuditLogSignIn -Filter "createdDateTime ge $weekAgo and status/errorCode eq 0" -Top 200 |
    Where-Object {
        $hour = ([datetime]$_.CreatedDateTime).Hour
        $hour -ge 22 -or $hour -le 6
    } |
    Select-Object CreatedDateTime, UserPrincipalName,
        @{N="IP";E={$_.IpAddress}},
        @{N="App";E={$_.AppDisplayName}} |
    Format-Table -AutoSize

Investigating a suspicious sign-in

When you find a sign-in that looks suspicious, follow this 5-minute investigation:

1. Check the IP. Copy the IP address and look it up: whatismyipaddress.com or ipinfo.io. Is it a residential ISP (might be the user's home), a cloud provider (VPN or attacker infrastructure), or a hosting provider (likely attacker)?

1. Check the location. Does the location match where the user is? Check the user's calendar for travel. If the user is in London and the sign-in is from Lagos, that's suspicious.

1. Check the device. Is the device managed (Intune-enrolled)? Is it compliant? A sign-in from a managed, compliant device is more likely legitimate than one from an unknown device.

1. Check the timing. Is 02:00 a normal sign-in time for this user? Some users legitimately work late. Others never sign in outside 09:00-17:00.

1. Contact the user. If the first four checks are inconclusive, a 30-second Teams message or email resolves it: "Did you sign in to M365 at [time] from [location]? Just checking for security purposes." If the user says "no," treat it as a credential compromise and execute AD1.9.

Most suspicious sign-ins turn out to be legitimate — VPN exit nodes creating unfamiliar IPs, travel creating unusual locations, insomnia creating unusual times. But the one time the user says "that wasn't me" is the incident you caught early because you checked.

Detecting token theft beyond AiTM

AiTM is the most common token theft technique, but it's not the only one. Your sign-in log review should also catch:

Stolen browser cookies. Infostealers (like Raccoon, RedLine, or Lumma) harvest browser cookies from infected devices. The stolen session cookie is replayed from the attacker's machine. The sign-in log shows a successful authentication from a device that isn't the user's normal machine — typically with a different operating system, browser version, or device ID. Filter the sign-in log for a specific user and compare the "Device info" fields across recent sign-ins. A sudden switch from "Windows 11 / Edge 122" to "Windows 10 / Chrome 108" may indicate a replayed cookie from a different machine.

# Check for device inconsistencies for a specific user
$user = "j.morrison@northgateeng.com"
Get-MgAuditLogSignIn -Filter "userPrincipalName eq '$user' and createdDateTime ge $weekAgo" -Top 50 |
    Select-Object CreatedDateTime,
        @{N="OS";E={$_.DeviceDetail.OperatingSystem}},
        @{N="Browser";E={$_.DeviceDetail.Browser}},
        @{N="DeviceId";E={$_.DeviceDetail.DeviceId}},
        @{N="IP";E={$_.IpAddress}},
        @{N="MFA";E={$_.MfaDetail.AuthMethod}} |
    Format-Table -AutoSize

If the same user appears with two different device profiles signing in simultaneously or in close succession from different IPs, one of those sessions is likely stolen.

Service principal sign-ins. Attackers who compromise an admin account sometimes create or modify service principals (application registrations) to maintain persistent access. Service principals authenticate without MFA and don't appear in the standard sign-in log — they appear in a separate log. Navigate to entra.microsoft.com → Monitoring → Sign-in logs → switch the tab from "User sign-ins (interactive)" to "Service principal sign-ins." Check for new or unfamiliar service principals signing in during the last 7 days.

# Service principal sign-ins in the last 7 days
Get-MgAuditLogSignIn -Filter "signInEventTypes/any(t:t eq 'servicePrincipal') and createdDateTime ge $weekAgo" -Top 50 |
    Select-Object CreatedDateTime, AppDisplayName, AppId,
        @{N="IP";E={$_.IpAddress}},
        @{N="Status";E={$_.Status.ErrorCode}} |
    Format-Table -AutoSize

Any service principal you don't recognize signing in from an unfamiliar IP warrants investigation. Check entra.microsoft.com → Applications → App registrations to see who created it and what permissions it has.

Legacy authentication attempts. Legacy protocols (IMAP, POP3, SMTP AUTH) don't support MFA. If legacy auth isn't blocked by conditional access, an attacker with stolen credentials can sign in without any MFA challenge. Filter the sign-in log: Client app → filter by "Exchange ActiveSync," "IMAP4," "POP3," "SMTP AUTH," "Other clients." Any successful legacy auth sign-in is a red flag if you've deployed CA policies that should block these — it means either the CA policy has an exclusion or the legacy client is bypassing the policy.

# Legacy auth sign-ins (should be zero if blocked by CA)
Get-MgAuditLogSignIn -Filter "createdDateTime ge $weekAgo and clientAppUsed ne 'Browser' and clientAppUsed ne 'Mobile Apps and Desktop clients'" -Top 100 |
    Where-Object { $_.ClientAppUsed -match "IMAP|POP3|SMTP|ActiveSync|Other" } |
    Select-Object CreatedDateTime, UserPrincipalName, ClientAppUsed,
        @{N="Status";E={if($_.Status.ErrorCode -eq 0){"SUCCESS"}else{"BLOCKED"}}} |
    Format-Table -AutoSize

If any legacy auth sign-ins succeed, either add a CA policy blocking legacy auth or extend your existing CA001/CA002 to block "Other clients" and "Exchange ActiveSync." Legacy auth is the most common MFA bypass — blocking it is a quick Secure Score improvement too.

Building your sign-in baseline

After 4 weeks of Monday reviews, you'll know what "normal" looks like for your environment: which IPs appear regularly (office, VPN, employee home ISPs), which users sign in after hours (on-call staff, executives), and which service principals authenticate routinely (backup tools, marketing automation). Document these expected patterns in your security log so that next month's reviewer (whether it's you or a colleague covering your holiday) can distinguish normal from abnormal.

Create a simple reference:

=== EXPECTED SIGN-IN PATTERNS ===
Office IPs: 203.0.113.10, 203.0.113.11 (main + backup circuit)
VPN exit: 198.51.100.50 (Prisma Access London)
After-hours users: s.chen (CEO, often 20:00-23:00), m.thompson (remote, US timezone)
Service principals: MarketingAutomation (daily 06:00 from Azure), BackupService (hourly)
Known travel: s.chen travels monthly, triggers impossible travel — confirm via calendar

This baseline turns the sign-in log review from "scan everything and guess" to "compare against known patterns and investigate deviations." The first month of reviews builds the baseline. Subsequent months use it. Update it when patterns change (new VPN provider, new employee with unusual hours, new service principal deployed).

Monthly sign-in metrics for quarterly reporting

On the first of each month, pull summary metrics for the quarterly report:

$monthAgo = (Get-Date).AddDays(-30).ToString("yyyy-MM-ddTHH:mm:ssZ")

$total = (Get-MgAuditLogSignIn -Filter "createdDateTime ge $monthAgo" -Top 5000).Count
$risky = (Get-MgAuditLogSignIn -Filter "createdDateTime ge $monthAgo and riskLevelDuringSignIn ne 'none' and riskLevelDuringSignIn ne 'hidden'" -Top 500).Count
$caBlocked = (Get-MgAuditLogSignIn -Filter "createdDateTime ge $monthAgo and conditionalAccessStatus eq 'failure'" -Top 500).Count

Write-Host "=== MONTHLY SIGN-IN METRICS ==="
Write-Host "Total sign-ins: $total"
Write-Host "Risky sign-ins detected: $risky"
Write-Host "Sign-ins blocked by CA: $caBlocked"
Write-Host "Block rate: $([math]::Round(($caBlocked/$total)*100,2))%"

The quarterly report includes: "Identity monitoring: X risky sign-ins detected, Y sign-ins blocked by conditional access (Z% block rate). All risky sign-ins investigated during weekly review — N confirmed compromises, M false positives from travel." These numbers demonstrate active monitoring with measurable outcomes.