8.12 Module Summary

Module 8 Summary: Connect Logs to Microsoft Sentinel

What you learned in this module

This module filled the workspace you configured in Module 7 with data from every source category in your environment. Without data connectors, Sentinel is an empty database. With them, every security event across identity, endpoint, email, cloud, network, and custom applications is queryable, detectable, and actionable.

Subsection 8.1 — Ingestion Strategy and Connector Architecture. The decision framework for prioritising data sources (identity + XDR first, network second, custom third). The four connector categories: service-to-service (Microsoft first-party), Defender XDR, agent-based (AMA), and API-based (Logs Ingestion API). The AMA migration from the deprecated Log Analytics Agent.

Subsection 8.2 — Microsoft First-Party Connectors. One-click connectors for Entra ID (SigninLogs, AuditLogs), Azure Activity (AzureActivity via Azure Policy), Microsoft 365 (OfficeActivity), and Defender for Cloud (SecurityAlert). Verification queries for each connector. The cost consideration for AADNonInteractiveUserSignInLogs.

Subsection 8.3 — Connecting Microsoft Defender XDR. The most important single connector: incidents + alerts + Advanced Hunting tables. Selective table ingestion for cost control. Bi-directional incident sync. XDR data tier benefits. Operational considerations: disabling redundant Microsoft Security incident creation rules to avoid duplicates.

Subsection 8.4 — Connecting Windows Hosts. Azure Monitor Agent deployment (Azure Policy for scale, Azure Arc for on-premises). Data Collection Rules for Windows Security Events. Collection levels: All Events vs Common vs Minimal vs Custom XPath. The SecurityEvent table and key Event IDs. The critical importance of command-line auditing.

Subsection 8.5 — Common Event Format (CEF) Connectors. CEF architecture: device → Syslog → Linux log forwarder → AMA → CommonSecurityLog table. Log forwarder deployment and configuration. CEF field mapping and structured columns. High availability with load-balanced forwarders. The difference between CEF and plain Syslog.

Subsection 8.6 — Syslog Data Sources. Direct collection (AMA on the Linux host) vs forwarded collection (network devices to a forwarder). Facility and severity level selection for security-relevant data. The Syslog table and KQL parsing patterns for unstructured messages. ASIM parsers for vendor-specific normalisation.

Subsection 8.7 — Data Collection Rules: Filter, Transform, Route. DCR architecture (sources, transformations, destinations). Ingestion-time transformations: row filtering, column removal, field parsing, aggregation. Cost impact of transformations. Multi-destination routing. The irreversibility warning: filtered data is permanently lost.

Subsection 8.8 — Custom Logs and API Ingestion. The Logs Ingestion API pipeline: application → DCE → DCR → custom table. Custom table creation and schema design. Entity-compatible column naming for analytics rule integration. Common use cases: custom web apps, SaaS webhooks, legacy systems.

Subsection 8.9 — Connector Troubleshooting and Validation. Universal troubleshooting checklist (status, data flow, freshness, daily cap, health events). Connector-specific diagnostics for service-to-service, Defender XDR, AMA-based, and API-based connectors. The comprehensive connector health query. Post-deployment validation: data completeness, field correctness, latency measurement, analytics rule compatibility.

Subsection 8.10 — Ingestion Cost Optimisation at the Connector Level. The five-layer cost reduction hierarchy: do not connect low-value sources, select right collection level, apply DCR transformations, use XDR tier, then workspace-level optimisation. Connector-specific techniques for each data source. The cost optimisation report query. Knowing when to stop optimising.

Subsection 8.11 — Building the Complete Ingestion Pipeline. The five-phase deployment plan (core identity → M365 → Windows hosts → network → custom). The production-ready checklist. Ongoing operational monitoring (daily, weekly, monthly). Scaling the pipeline as the organisation grows.

SC-200 exam objectives covered

Domain 1 — Manage a Security Operations Environment (40-45%): Connect data to Microsoft Sentinel using data connectors. Connect Microsoft services to Microsoft Sentinel. Connect Microsoft Defender XDR to Microsoft Sentinel. Connect Windows hosts to Microsoft Sentinel. Connect Common Event Format logs to Microsoft Sentinel. Connect syslog data sources to Microsoft Sentinel. Connect threat indicators to Microsoft Sentinel.

This module covers a significant portion of the exam’s data connector questions. The concepts — connector categories, AMA deployment, DCR configuration, CEF vs Syslog, and troubleshooting — appear frequently in scenario-based exam questions.

What comes next

Module 9 — Create Detections and Perform Investigations. The data is flowing. Module 9 builds the analytics rules that turn that data into security alerts and incidents. You will create scheduled rules, NRT rules, and automation rules. You will configure incident management. And you will build the detection engineering workflow that continuously improves your detection coverage.

Module 10 — Threat Hunting in Sentinel. Proactive threat hunting using the data you connected in this module. Hunting queries, bookmarks, livestream, and hunt management — finding threats that analytics rules miss.

Together, Modules 7-10 complete the Sentinel operational capability: workspace (M7) → data (M8) → detection (M9) → hunting (M10).

Skills checklist

Before moving to Module 9, verify you can perform each of these tasks.

Strategy and architecture. Prioritise data sources using the ingestion priority framework (detection value per cost). Identify visibility gaps by mapping data sources to MITRE ATT&CK kill chain phases. Explain the four connector categories and when to use each. Describe the AMA migration path from the deprecated Log Analytics Agent.

Microsoft connectors. Enable and verify the Entra ID connector (all log types). Deploy the Azure Activity connector via Azure Policy. Identify and resolve the Entra ID diagnostic setting conflict. Enable the Microsoft 365 connector and assess overlap with CloudAppEvents. Enable the Defender for Cloud connector with bi-directional sync.

Defender XDR connector. Configure bi-directional incident sync. Select Advanced Hunting tables based on detection value and cost. Disable redundant Microsoft Security incident creation rules. Verify incident sync and table ingestion with diagnostic queries. Understand the XDR data tier cost benefit.

Windows hosts. Deploy AMA via Azure Policy (for scale) and Azure Arc (for on-premises). Create a DCR for Windows Security Events at the appropriate collection level. Explain the difference between All Events, Common, Minimal, and Custom XPath. Identify the critical Event IDs (4624, 4625, 4648, 4688, 4720, 4728, 4697, 1102). Enable command-line auditing and PowerShell Script Block Logging via GPO.

CEF and Syslog. Deploy a Linux log forwarder for CEF devices. Explain the CEF message format and CommonSecurityLog field mappings. Configure Syslog collection with appropriate facility and severity levels. Parse unstructured Syslog messages with KQL (parse, extract). Explain the difference between CEF and plain Syslog data. Describe ASIM parsers and their normalisation benefit.

Data Collection Rules. Create DCRs for Windows Security Events, Syslog, and CEF. Write ingestion-time transformations (row filtering, column removal, field parsing). Explain the difference between DCR transformations and workspace transformations. Test transformations against historical data before deployment. Configure multi-destination routing.

Custom logs. Create a custom table with the Logs Ingestion API. Configure a Data Collection Endpoint and Data Collection Rule. Send test data via the API. Design custom table schemas with entity-compatible column names. Handle API errors (429, 400, 403, 413).

Troubleshooting and validation. Systematically troubleshoot each connector type using the diagnostic flowchart. Verify data completeness, field correctness, and ingestion latency. Build and use the comprehensive connector health query. Identify and resolve common failure patterns (DCR disassociation, AMA network blocks, CEF parsing failures).

Cost optimisation. Apply the five-layer cost reduction hierarchy. Estimate monthly cost from daily ingestion volume. Write and interpret the cost optimisation report query. Calculate ROI for connector-level optimisations. Conduct the monthly optimisation review.

Key decisions recap

Ingestion priority — identity + XDR first (day 1), M365 + Defender for Cloud second (day 2-3), Windows hosts third (week 1), network fourth (week 2), custom fifth (week 3+).

Collection levels — Common for Windows Security Events. Security-relevant facilities at info level for Syslog auth. Deny logs only for firewalls (where possible).

Table selection — all investigation-critical Defender XDR tables enabled. Inventory tables (DeviceInfo, DeviceNetworkInfo) excluded to reduce cost.

DCR transformations — filter service account logons, filter routine non-interactive sign-ins, remove unused columns. Test before deploy. Document every transformation.

Connector documentation — every connector documented in the operations runbook with verification queries and troubleshooting steps.