7.4 Log Types: Analytics, Basic, and Archive Tiers

Log Types: Analytics, Basic, and Archive Tiers

Introduction

Not all log data has equal investigation value. Sign-in logs that you query during every investigation are more valuable than verbose firewall connection logs that you check once a month. Performance counter data that supports capacity monitoring has minimal security investigation value. Sentinel’s three-tier log model lets you assign each table to the tier that matches its value — paying full price only for high-value data and significantly less for lower-value data.

The tier decision has three consequences: what you can do with the data (query capability), how long you keep it (retention), and how much you pay (cost). This subsection teaches you the three tiers, when to use each, and how to assign tables correctly.

The three-tier model

Analytics tier: full capability

The Analytics tier is the default for all Sentinel tables. It provides full KQL query capability (all operators including join, union, summarize, make-set, arg_max, and every other operator from Module 6), full analytics rule evaluation (scheduled rules, NRT rules, and anomaly rules can query Analytics-tier tables), 90 days of interactive retention included in the Sentinel per-GB price (no additional retention cost for the first 90 days), and support for all Sentinel features (hunting, workbooks, entity pages, UEBA).

Use Analytics tier for: Every table that you actively query during investigations, that analytics rules evaluate for detections, or that you need for real-time hunting. This includes all core security tables: SigninLogs, SecurityAlert, SecurityIncident, DeviceProcessEvents, DeviceNetworkEvents, EmailEvents, CloudAppEvents, OfficeActivity, AzureActivity, and any custom table used in detection rules.

The cost: Analytics tier is the most expensive per GB. The exact price varies by region and commitment tier but is typically $2-4 per GB/day ingested (pay-as-you-go). For context: 10 GB/day at $3/GB = $30/day = $900/month. At 100 GB/day, this is $9,000/month. Cost management (subsection 7.5) becomes critical at scale.

Basic tier: reduced cost, reduced capability

The Basic tier costs approximately 60% less than Analytics for ingestion. The trade-off: limited KQL operators. Basic-tier tables support only: where, extend, project, project-away, project-keep, project-rename, project-reorder, parse, parse-where. They do NOT support: join, union, summarize, make-set, make-list, arg_max, arg_min, top, sort, distinct, count, take, and most aggregation operators.

The practical impact: You can filter and project Basic-tier data (find specific events by timestamp, user, or IP), but you cannot aggregate it (count events per user, find the user with the most events) or correlate it with other tables (join with SigninLogs). Analytics rules cannot query Basic-tier tables — so you cannot write detection rules against Basic-tier data.

Use Basic tier for: High-volume, low-investigation-value data that you query infrequently and only for specific lookups. Common candidates: verbose firewall connection logs (you query them for specific IP lookups during investigations but do not run analytics rules against them), raw Windows Security Events at the verbose level (most events are routine — you search for specific event IDs during investigation), network flow data (high volume, used for post-incident traffic analysis but not for real-time detection), and application trace logs (debugging data that is occasionally relevant to security investigation).

Archive tier: long-term retention at minimal cost

The Archive tier stores data for up to 12 years at a fraction of the Analytics/Basic cost. Archive data cannot be queried directly — it must be restored to an Analytics-tier table first. Restoration takes minutes to hours depending on the data volume and creates a temporary table that exists for a configurable period.

Use Archive tier for: Data that must be retained for regulatory compliance (7 years for financial regulations, 10 years for some healthcare requirements) but is rarely accessed. After the interactive retention period expires (90 days for Analytics, 30 days for Basic), data can transition to the Archive tier automatically.

Archive + search jobs: While you cannot run live KQL queries against archived data, you can run search jobs. A search job is an asynchronous query that searches archived data and writes the results to a new table. This is useful for historical investigations: “Did this IP address appear in our firewall logs 6 months ago?” The search job takes minutes to hours (depending on the data volume being searched) but provides access to data that would otherwise require a full restore.

Changing a table’s tier

Table tier assignment is configured per table in the workspace settings.

Navigate to Log Analytics workspace → Tables → select the table → Configure. Choose the log plan: Analytics or Basic. (Archive is configured as an extension of the retention policy, not as a separate tier selection.)

Important constraints: Not all tables can be changed to Basic. Microsoft-managed tables that are essential for Sentinel functionality (SecurityAlert, SecurityIncident, and some Defender XDR tables) are locked to Analytics tier. Custom tables and some high-volume standard tables (Syslog, SecurityEvent, WindowsEvent) can be changed to Basic.

Tier changes are not retroactive. Changing a table from Analytics to Basic affects new data ingested after the change. Existing data retains its original tier capability for the remainder of its retention period. You do not lose the ability to query existing Analytics-tier data when you change the table to Basic for future ingestion.

The Auxiliary tier (preview)

Microsoft has introduced an Auxiliary tier for extremely high-volume, low-value data (network flow logs, verbose application telemetry). Auxiliary is even cheaper than Basic and supports only search operations (not standard KQL queries). It is designed for data that you retain only for compliance and access only through search jobs.

The Auxiliary tier is in preview and may change before general availability. For the SC-200 exam, understand the concept (cheapest tier, most limited capability) but focus your study on Analytics, Basic, and Archive — these are the established tiers tested on the exam.

The cost impact of tier assignment

Tier assignment is the single most impactful cost management lever in Sentinel. Moving a high-volume table from Analytics to Basic reduces its ingestion cost by ~60%. For a table generating 20 GB/day, this saves approximately $36/day or $1,080/month.

Example cost comparison for a 100 GB/day environment:

If all 100 GB is Analytics tier: ~$300/day = $9,000/month. If 60 GB is Analytics and 40 GB is Basic: ~$180 + $48 = $228/day = $6,840/month. Savings: $2,160/month (24% reduction) — by moving 40 GB of low-investigation-value data to Basic.

The savings are significant, but the trade-off is real: Basic-tier data cannot be used in analytics rules and supports limited query operators. Only move data to Basic if you are confident it is not needed for detection or complex investigation queries.

Tier selection decision tree

For every table in your workspace, walk through this decision tree to determine the correct tier.

Question 1: Does any analytics rule query this table? Check by searching your active rules for the table name. If yes → the table must be on Analytics tier. Analytics rules are your automated detection capability — moving their data source to Basic silently disables the detection.

Question 2: Do any investigation queries join this table with other tables? Review your saved queries and the cross-product join map from subsection 7.6. If yes → the table must be on Analytics tier. Basic tier does not support join, and moving the table breaks cross-product investigation.

Question 3: Is this table queried frequently during active investigations? If analysts query this table multiple times per week during incident investigation, keep it on Analytics for full KQL capability and fast query response. If the table is queried only during rare, deep forensic investigations → Basic tier may be acceptable, with the understanding that investigation queries will be limited.

Question 4: What is the daily ingestion volume? If the table ingests less than 0.5 GB/day, the cost difference between Analytics and Basic is negligible — keep it on Analytics for maximum capability. The cost optimisation only matters for high-volume tables (5+ GB/day).

If the answer to questions 1, 2, and 3 is “no” AND the volume is significant → move to Basic tier. Common candidates: AzureMetrics, ContainerLog, AzureDiagnostics (non-security resources), AppTraces, AppRequests.

Search jobs: querying archived data

When data moves to Archive tier (after the Analytics/Basic retention period), it is no longer queryable through standard KQL. To access archived data, use a search job.

Creating a search job: Navigate to Sentinel → Search → create a new search job. Specify: the table to search, the time range (within the archive period), and a KQL query to filter the results. The search job runs asynchronously — it may take minutes to hours depending on the data volume being scanned.

How search jobs work: The search job scans the archived data and writes matching results to a new Analytics-tier table (named <YourSearchName>_SRCH). This temporary table has full KQL capability — you can query, join, and analyse the results using standard KQL. The results table is available for the search job’s configured retention period (default: 30 days).

When to use search jobs: Compliance audit requests (“provide all sign-in activity for user X from January to March 2024”), legal hold investigations (“retrieve all email activity for this user during the litigation period”), and retroactive threat hunting (“check if this newly discovered IOC appeared in our data 6 months ago”).

1
2
3
4
5
6

// Example: after a search job completes, query the results table
SearchResults_SRCH
| where UserPrincipalName =~ "j.morrison@northgateeng.com"
| summarize SignInCount = count() by bin(TimeGenerated, 1d),
    Country = tostring(LocationDetails.countryOrRegion)
| order by TimeGenerated asc

Cost of search jobs: You pay for the data scanned during the search (per GB scanned) plus the ingestion cost for the results written to the temporary Analytics table. For large time ranges across high-volume tables, search job costs can be significant. Use the KQL filter in the search job definition to narrow results and reduce both scan cost and result volume.

1
2
3
4
5
6
7

// Identify high-volume tables that are candidates for Basic tier
Usage
| where TimeGenerated > ago(30d)
| where IsBillable == true
| summarize TotalGB = sum(Quantity) / 1000 by DataType
| order by TotalGB desc
| take 15

Knowledge check