Sign In
In this section

LX0.8 The Investigation Toolkit

3-4 hours · Module 0 · Free
Operational Objective
The Toolkit Question: the incident is confirmed and evidence collection must begin immediately. You need to acquire memory, triage the filesystem, image the disk, analyze logs, build a timeline, and examine a suspicious binary. Each task requires a specific tool — and if the tool is not already installed, compiled, and tested, you lose hours during the investigation preparing what should have been ready before the call came. You need a complete toolkit map: which tool does what, when you use it, what it requires, and what alternative exists if the primary tool is unavailable.
Deliverable: A complete investigation toolkit reference organized by investigation phase — collection, filesystem analysis, log analysis, memory forensics, and malware analysis — with installation commands, usage examples, prerequisites, and alternatives for each tool.
⏱ Estimated completion: 25 minutes

The Investigation Toolkit: What Each Tool Does and When You Use It

Toolkit philosophy: free first, commercial when justified

This course teaches every technique with at least one free or open-source tool. When you get the call at 03:00 that a server is compromised, the tool you need must be available immediately, not waiting for a procurement approval. Free tools are available now, on every forensic workstation, without budget or licensing delays.

Commercial tools are referenced where they provide significant advantages: faster processing, better GUI workflows, cross-platform correlation, or enterprise deployment. But the investigation methodology does not depend on them. Every finding can be reached with free tools. Commercial tools get you there faster when you have them.

Phase 1 tools: evidence collection

# Deploy and run UAC from external media
# UAC requires no installation — it is a self-contained shell script
cd /mnt/usb/uac
./uac -p ir_triage /mnt/usb/output/
# -p ir_triage    : fast triage profile (3-8 minutes)
#                   captures: volatile data, key logs, user artifacts, bodyfile
# /mnt/usb/output : save output to external media (not the compromised disk)
# Output structure:
#   live_response/   → volatile data (processes, network, open files)
#   logs/            → log file copies
#   system/ + user/  → configuration and user artifacts
#   bodyfile/        → filesystem metadata for timeline generation
#   hash.txt         → SHA256 of every collected file
# Sleuth Kit core commands for Linux forensics

# List all files (including deleted) from a disk image
fls -r -m "/" /path/to/disk.raw
# -r          : recursive (all directories)
# -m "/"      : output in bodyfile format with "/" as mount point
# Deleted files appear with an asterisk (*) prefix
# Pipe to mactime for timeline: fls -r -m "/" disk.raw | mactime -b -

# Display inode metadata (all four timestamps + permissions)
istat /path/to/disk.raw 12345
# 12345 = inode number (from fls output)
# Shows: file type, permissions, owner, group, size,
#        atime, mtime, ctime, crtime, block addresses

# Extract file content by inode number (even if deleted)
icat /path/to/disk.raw 12345 > recovered_file
# Recovers the file content even if the directory entry was removed
# Works as long as the data blocks have not been overwritten

# Display partition layout
mmls /path/to/disk.raw
# Shows: partition table with start/end sectors and types
# Use the start sector as offset for fls: fls -o 2048 disk.raw
# ausearch examples for common investigation queries

# Find all command executions during the compromise window
ausearch -m EXECVE --start "03/28/2026" "03:00:00" --end "03/28/2026" "06:00:00"
# -m EXECVE    : filter to process execution events only
# --start/end  : time window matching the compromise timeline
# Output shows every command executed with full arguments

# Find privilege escalation events
ausearch -m USER_CMD -ua root --start "03/28/2026" "03:00:00"
# -m USER_CMD  : sudo/su command events
# -ua root     : events involving root user

# Generate a summary authentication report
aureport --auth --start "03/28/2026" "03:00:00"
# Shows: count of successful/failed authentications per user
# Quick way to identify brute force patterns
Expand for Deeper Context

UAC (Unix Artifact Collector) — the automated triage tool covered in depth in LX1. Open-source, runs as a shell script, collects over 200 artifact categories. Use it as the first tool on every compromised system.

LiME (Linux Memory Extractor) — kernel module for memory acquisition. Pre-compile for every kernel version in your infrastructure. Covered in depth in LX1.6.

dc3dd — forensic disk imaging tool with simultaneous hashing. Covered in LX1.7.

Standard Linux commands — ps, ss, lsof, lsmod, mount, df, ip, find, stat, cat, grep, journalctl, ausearch, aureport, last, lastb, who, w. These commands are available on every Linux system. For live response, always supplement standard commands with direct /proc reads to bypass potential rootkit hooks.

Commercial alternative: Velociraptor — open-source with enterprise features. Provides agent-based remote collection. You deploy an agent on your infrastructure and can collect artifacts from any managed system through a web console without SSH access. It runs VQL queries that can collect specific artifacts, hunt across multiple systems simultaneously, and monitor for ongoing attacker activity. The key advantage: no SSH access needed. The key prerequisite: agent must be deployed before the incident.

Phase 2 tools: filesystem analysis

The Sleuth Kit (TSK) — the foundational open-source filesystem forensics toolkit. Works with disk images — you point it at a dd or dc3dd image and it reads filesystem structures directly, without mounting.

plaso / log2timeline — the unified timeline generation tool. Open-source, maintained by the Google Forensics team. plaso parses hundreds of artifact types from a disk image or evidence collection and produces a unified, time-sorted timeline. The timeline is the single most powerful analysis artifact in any investigation — it shows exactly what happened, in what order, across all evidence sources.

mactime (from Sleuth Kit) — generates a timeline from a bodyfile (filesystem metadata). Simpler and faster than plaso but limited to filesystem metadata only. Use mactime for a quick filesystem timeline during initial triage. Use plaso for the comprehensive timeline.

extundelete and photorec — deleted file recovery. extundelete recovers deleted files from ext3/ext4 by scanning the journal and inode table. photorec carves files from unallocated space based on content signatures — works even when inodes are overwritten.

Phase 3 tools: log analysis

journalctl — the systemd journal query tool. Available on all systemd-based distributions. Queries binary journal files with powerful filtering: by time, service, priority, PID, and boot session. The journal's binary format makes it harder to tamper with than plaintext logs.

ausearch and aureport — the auditd log query tools. ausearch searches by event type, user, time range, system call, and audit key. aureport generates summary reports. These tools transform the raw audit.log into readable investigation-focused output.

grep, awk, sed — the universal text processing tools. For unstructured log files (Apache, Nginx, application logs), these remain the fastest way to extract investigation-relevant entries. Available on every Linux system and process log files at disk read speed.

Phase 4 tools: memory analysis

Volatility 3 — the primary memory analysis framework for Linux. Reads a memory dump (from LiME) and extracts running processes, network connections, loaded kernel modules, bash history from process memory, open files, and much more. The critical prerequisite: requires an ISF (Intermediate Symbol Format) profile matching the target kernel exactly. Building ISF profiles is covered in LX12.

Phase 5 tools: malware analysis

file, strings, readelf — static analysis basics. file identifies the file type. strings extracts readable strings (C2 addresses, file paths, hardcoded credentials). readelf -a displays ELF binary metadata.

strace and ltrace — dynamic analysis tools. strace traces system calls made by a running process. ltrace traces library calls. Both used in a sandbox to observe malware behavior.

INVESTIGATION TOOLKIT — BY PHASE COLLECT UAC (triage) LiME (memory) dc3dd (disk) ps, ss, lsof, find Alt: Velociraptor FILESYSTEM Sleuth Kit (fls, istat) plaso / log2timeline mactime (timeline) extundelete, photorec Alt: AXIOM Cyber LOG ANALYSIS journalctl ausearch / aureport grep / awk / sed Timesketch (GUI) Alt: Elastic/Splunk MEMORY Volatility 3 ISF profiles /proc direct reads (live alternative) Alt: Rekall (legacy) MALWARE file / strings readelf / ldd strace / ltrace VirusTotal / YARA Alt: Cuckoo Sandbox TOOLKIT PREPARATION CHECKLIST □ UAC cloned + tested on USB □ LiME compiled per kernel □ Volatility 3 + ISF profiles □ Sleuth Kit + plaso installed □ dc3dd tested with hashing □ Forensic VM analysis-ready
Figure LX0.8 — The Linux IR toolkit. UAC for automated triage, LiME for memory acquisition, Volatility 3 for memory analysis, Sleuth Kit for filesystem forensics, plaso for timeline generation, and Velociraptor for enterprise-scale collection. Each tool serves a specific phase of the investigation.
Figure LX0.8: The investigation toolkit organized by phase. Each phase maps to specific modules in this course. The preparation checklist at the bottom must be completed before the next incident.

Worked artifact — Forensic workstation setup script:

Run this script on your forensic workstation or analysis VM to install the complete toolkit. Adapt paths and package names for your distribution.

`` # Forensic workstation setup — Ubuntu 22.04 sudo apt update && sudo apt install -y sleuthkit dc3dd wireshark python3-pip python3-venv git python3 -m venv ~/tools/vol3 && source ~/tools/vol3/bin/activate && pip install volatility3 && deactivate python3 -m venv ~/tools/plaso && source ~/tools/plaso/bin/activate && pip install plaso && deactivate git clone https://github.com/tclahr/uac.git ~/tools/uac git clone https://github.com/504ensicsLabs/LiME.git ~/tools/lime echo "Verify: fls -V && dc3dd --version && ~/tools/vol3/bin/vol -h | head -1" ``

Beyond this investigation: LX13 (Malware Analysis) uses these tools in a real investigation: static analysis with readelf and strings, dynamic analysis with strace and ltrace, and memory analysis with Volatility 3. LX16 (IR Readiness) covers pre-staging these tools on every host.

Myth: "You need commercial forensic tools like EnCase or AXIOM to conduct a proper Linux investigation."

Reality: Every investigation technique in this course uses free, open-source tools. The Sleuth Kit, Volatility 3, plaso, UAC, LiME, and dc3dd provide complete forensic capability — from evidence collection through timeline generation to memory analysis. Commercial tools provide workflow efficiency (GUI interfaces, automated parsing, case management) but do not provide investigative capabilities that the open-source toolkit lacks. The investigator's skill — knowing where to look, what to extract, how to interpret, and what it proves — is what determines investigation quality, not the tool's price tag.

Decision points: when to use which tool

Quick triage (time-constrained, need answers fast): UAC ir_triage + mactime for filesystem timeline. Skip plaso (too slow for initial triage). Skip memory acquisition if LiME is not pre-compiled.

Full investigation (time available, legal proceedings possible): LiME for memory, dc3dd for disk image, UAC full profile, plaso for comprehensive timeline, Volatility 3 for memory analysis. Dual-hash everything.

Multi-system hunt (looking for indicators across infrastructure): Velociraptor for agent-based remote collection and VQL hunting queries. If Velociraptor is not deployed, use SSH-based UAC deployment in parallel across all target systems.

Troubleshooting: common tool issues

Volatility 3 cannot parse the memory dump: Wrong or missing ISF profile. The ISF must match the target kernel exactly. Check: vol -f memory.lime banners.Banners to identify the kernel version in the dump, then build or download the matching ISF.

plaso takes hours to process a disk image: plaso is thorough but slow on large images. For faster results, use mactime on just the bodyfile (filesystem metadata timeline) while plaso runs in the background. The mactime result covers most investigation questions within minutes.

Sleuth Kit fls shows no files: Wrong partition offset. Use mmls disk.raw to find the correct start sector, then fls -o [start_sector] disk.raw.

UAC fails to collect auditd logs: auditd is not installed or not running. UAC collects what exists — if audit.log does not exist, that artifact is skipped. This is normal on systems without auditd and is documented in the UAC output log.

Try it yourself

Build the foundation of your Linux IR toolkit.

Build the foundation of your Linux IR toolkit. On your forensic workstation, run the setup script from the worked artifact above. Verify each tool: fls -V (Sleuth Kit), dc3dd --version (dc3dd), ~/tools/vol3/bin/vol -h | head -1 (Volatility 3), ./uac/uac -h (UAC), ~/tools/plaso/bin/log2timeline.py --version (plaso). If any tool fails to install, troubleshoot now — not during an incident at 03:00.

Beyond this investigation

This toolkit maps to the entire course. LX1 uses UAC, LiME, and dc3dd. LX2 uses Sleuth Kit, plaso, mactime, and extundelete. LX3 uses journalctl, ausearch, and grep. LX12 uses Volatility 3. LX13 uses file, strings, strace, and ltrace. Each tool is taught in depth in the module where it is used — this subsection provides the complete map.

Check your understanding:

1. You need to acquire memory from a server running kernel 5.15.0-91-generic. Your LiME toolkit has modules for 5.15.0-88 through 5.15.0-91. Can you acquire memory? What happens if you try to load the wrong module? 2. What is the difference between using mactime and using plaso for timeline generation? When would you use each? 3. You recovered a deleted binary using extundelete. What three tools would you use for initial characterization before executing it in a sandbox? 4. The compromised system does not have auditd installed. What command execution evidence sources remain available?

Subsection artifacts: Investigation toolkit diagram organized by phase (Figure LX0.8). Toolkit preparation checklist. Forensic workstation setup script (worked artifact). Annotated command blocks for Sleuth Kit and ausearch. Install the tools and complete the checklist before the next incident.

Decision point

You are investigating a Linux server and discover evidence of both a cryptominer (resource abuse) and an SSH key theft (lateral movement preparation). The cryptominer is consuming 95% CPU and impacting production. Which do you address first?

Address the lateral movement first. The cryptominer is visible, noisy, and contained to this server — it is causing performance impact but not spreading. The SSH key theft is silent, potentially already exploited, and may have given the attacker access to additional servers. Contain the lateral movement risk: rotate the stolen SSH keys, check the target servers for unauthorized access, and apply network restrictions. Then address the cryptominer: kill the process, remove the binary and persistence mechanisms. Prioritizing the noisy but contained threat over the silent but spreading threat is the most common Linux IR prioritization mistake.

© 2026 Ridgeline Cyber Defence™ Ltd. Content may not be reproduced or redistributed. Worked artifacts may be adapted for use within your organization.
Unlock the Full Course See Full Course Agenda
← Previous Next →