In this section

LX1.4 Evidence Integrity and Chain of Custody

3-4 hours · Module 1 · Free
Operational Objective
The Credibility Question: You collected auth.log from BASTION-NGE01 three weeks ago. The CISO asks: "How do we know this file has not been modified since you collected it?" If you cannot answer with cryptographic proof, your investigation findings are assertions — not evidence. The attacker's defense counsel, the regulator, the insurer, or the CISO themselves can challenge every finding built on unverified evidence. Even internal investigations without legal proceedings require integrity verification because your remediation decisions depend on the evidence being accurate.
Deliverable: The ability to hash all collected evidence at the point of collection, verify integrity before analysis, maintain chain of custody documentation from collection through storage to reporting, and understand the additional requirements when legal proceedings are anticipated. A complete evidence collection log template that you deploy on every investigation.
⏱ Estimated completion: 30 minutes

Evidence Integrity: Hashing, Documentation, and Chain of Custody

Evidence integrity is not just a legal requirement. It is a professional standard that distinguishes reliable investigation from guesswork. When you present findings to a CISO, a legal team, or a regulator, the first question is not "what did you find?" — it is "how do you know the evidence has not been modified since you collected it?" If you cannot answer that question with cryptographic proof, your findings are assertions rather than evidence.

The mechanism is simple: cryptographic hashing. When you collect a file, you compute its SHA256 hash. When you analyze the file days or weeks later, you recompute the hash. If the hashes match, the file has not been modified — not by you, not by anyone. If the hashes differ, the file was changed after collection, and you need to determine when and how.

CHAIN OF CUSTODY — EVIDENCE LIFECYCLE 1. COLLECT Acquire artifact Hash immediately (SHA256) Record: who, when, how Hash A = original truth sha256sum file > file.sha256 2. TRANSFER Move to storage/analysis Verify hash at destination Record: from, to, method Hash B must = Hash A Mismatch = transfer corruption 3. STORE Encrypted, access-controlled Original is immutable Record: location, access list Never analyze the original Create working copy for analysis 4. ANALYZE Work on COPY only Verify copy hash = original Record: tools, dates, findings Hash C = Hash A always If mismatch → investigate why At every stage: SHA256 hash must match the original collection hash Any mismatch = evidence was modified → investigate when and how → document the discrepancy STANDARD: Internal investigations SHA256 hash at collection + verification before analysis Collection log with timestamps and actions ELEVATED: Legal proceedings anticipated Dual hashes (SHA256 + MD5/SHA1), witness signature Hardware write blockers, tamper-evident evidence bags
Figure LX1.4 — The evidence lifecycle from collection to analysis. Every transition requires hash verification. Standard investigations require SHA256 and a collection log. Elevated handling for legal proceedings adds dual hashing, witness signatures, and tamper-evident storage.

Hashing Collected Evidence

# Hash a single file immediately after collection
sha256sum evidence/auth.log > evidence/auth.log.sha256
# Output: e3b0c44298fc1c149afb... evidence/auth.log

# Hash all files in a directory recursively
find evidence/ -type f -not -name "*.sha256" -exec sha256sum {} \; > evidence/hashes.sha256
# Creates a manifest of every file — verify the entire collection in one command

# Verify hashes before beginning analysis (days or weeks later)
cd evidence/
sha256sum -c hashes.sha256
# Output: each file shows "OK" or "FAILED"
# Any "FAILED" = the file was modified after collection → investigate

# Hash comparison for a specific file
sha256sum evidence/auth.log
# Compare output against the hash recorded in the collection log
# Must match character-for-character
# During disk acquisition — hash simultaneously with dc3dd
dc3dd if=/dev/sda hash=sha256 log=evidence/disk_acquisition.log of=evidence/bastion-nge01.raw
# dc3dd computes SHA256 during the copy — no second pass needed
# Hash appears in the log file alongside acquisition metrics

# Standard dd + separate hash (if dc3dd is not available)
dd if=/dev/sda of=evidence/bastion-nge01.raw bs=4M status=progress
sha256sum evidence/bastion-nge01.raw > evidence/bastion-nge01.raw.sha256
# Two-pass approach: slower but achieves the same result

# Verify the image before mounting for analysis
sha256sum evidence/bastion-nge01.raw
# Must match the hash recorded during acquisition
# Dual hashing for legal-standard evidence
sha256sum evidence/bastion-nge01.raw > evidence/bastion-nge01.raw.sha256
md5sum evidence/bastion-nge01.raw > evidence/bastion-nge01.raw.md5
# Both hashes recorded in the collection log and on the evidence bag label

# Verify both hashes before analysis
sha256sum -c evidence/bastion-nge01.raw.sha256  # Must show OK
md5sum -c evidence/bastion-nge01.raw.md5          # Must show OK
# Both must pass — if one fails, the evidence may have been modified
Expand for Deeper Context

Every evidence file should be hashed immediately after collection. UAC does this automatically — it generates a hashes.sha256 file containing the SHA256 hash of every collected artifact. For manually collected evidence, hash each file as you collect it:

For disk images, the hash is critical because a multi-gigabyte disk image cannot be visually inspected for modifications. The forensic imaging tool should compute the hash during acquisition:

The dc3dd tool (a forensic version of dd) computes the hash during acquisition — you get the hash at the same time as the image, with no additional pass over the data. This is the preferred method for disk imaging because it saves time and provides the hash in the acquisition log. If dc3dd is not available, standard dd followed by sha256sum achieves the same integrity guarantee — the two-pass approach is slower but equally valid.

Chain of Custody Documentation

The chain of custody records who handled the evidence, when, and what they did with it. For a Linux investigation, the chain of custody document should include four categories of records.

Collection record — for each evidence artifact: what was collected (filename or description), when it was collected (UTC timestamp), where it was collected from (hostname, IP, path), who collected it (investigator name), how it was collected (tool and command used), and the SHA256 hash at the time of collection.

Transfer record — for each handoff: who transferred the evidence, who received it, when, and how (USB drive, network transfer, cloud storage). If evidence is transferred electronically, hash verification should occur at both ends. A gap between transfer and verification is a weakness — record verification immediately upon receipt.

Storage record — where the evidence is stored (encrypted disk, evidence locker, cloud storage with access controls), who has access, and what access controls are in place. The original evidence must never be modified after collection. All analysis is performed on verified copies.

Analysis record — who analyzed the evidence, when, what tools were used, and whether the analysis was performed on the original evidence or a working copy. Best practice: always analyze a copy, never the original. Hash the copy and verify it matches the original before beginning analysis.

Worked artifact — Evidence collection and chain of custody log:

Start this log at the beginning of every investigation. It becomes part of the IR report (LX14).

Case: INC-2026-XXXX System: [hostname] IP: [address]

Investigator: [name] Authorization: [who authorized the investigation]

COLLECTION RECORD

03:55 UTC — System time recorded using date -u → timestamp.txt (SHA256: e3b0c442...) 03:56 UTC — Processes captured using ps auxf via SSH → processes.txt (SHA256: 7f83b165...) 03:57 UTC — Network connections using ss -tnp + /proc/net/tcp → network.txt (SHA256: 2c26b468...) 03:58 UTC — /proc deep reads using custom loop → proc_data.txt (SHA256: fcde2b2e...) 04:02 UTC — Auth logs copied using scp → auth.log* (3 files, per-file hash) 04:10 UTC — UAC triage run using ./uac -p ir_triage → uac_output/ (manifest hash) 04:32 UTC — Collection complete

Modifications caused by investigator activity: - SSH session created auth.log entry at 03:55 (IP: 192.0.2.10) - SSH session created wtmp entry at 03:55 - UAC execution created process entries in /proc at 04:10-04:18

TRANSFER RECORD

04:35 UTC — From BASTION-NGE01 to Forensic WS (fw-01) via SCP over SSH. Hash verified at destination: ☐ Yes ☐ No 04:40 UTC — From Forensic WS to Evidence share via Encrypted USB. Hash verified at destination: ☐ Yes ☐ No

STORAGE: Evidence stored at: ___ Encrypted: ☐ Yes ☐ No Access restricted to: ___

ANALYSIS: Working copy created: ☐ Yes Copy hash verified against original: ☐ Yes ☐ No Analyst: ___ Analysis start date: ___

For investigations that may involve law enforcement, regulatory action, or civil litigation, evidence handling standards are stricter. The key additions:

Write blockers for bare-metal disk acquisition. A hardware write blocker physically prevents any write operation to the evidence disk. Software write blockers (mounting read-only) are acceptable for cloud disk snapshots and copied evidence, but hardware write blockers are the standard for original physical media.

Two independent hashes (SHA256 + MD5, or SHA256 + SHA1). If one hash algorithm has a collision weakness (MD5 has known collision attacks), the second hash provides independent verification. SHA256 alone is sufficient for internal investigations, but dual hashing is the standard for evidence that may be challenged in court.

Witness documentation. A second person witnesses the evidence collection and signs the collection log. This prevents challenges based on the investigator fabricating evidence.

Evidence sealing. Physical media (USB drives, hard drives) are sealed in tamper-evident bags with the case number, date, and hash written on the seal. If the bag is opened, it is visibly obvious.

Decision points: choosing the right integrity standard

Internal investigation, no legal proceedings anticipated: Standard handling. SHA256 hash at collection, verification before analysis, collection log with timestamps. This covers 80% of Linux investigations — SOC incident analysis, threat hunting confirmations, security posture assessments.

Investigation may escalate to legal proceedings: Elevated handling from the start. Dual hashes (SHA256 + MD5), witness signature on the collection log, hardware write blockers for physical media, tamper-evident storage. It is far easier to handle evidence to the elevated standard from the beginning than to retroactively prove integrity after the fact. If there is any doubt about whether legal proceedings are possible, use the elevated standard.

Law enforcement involvement or regulatory investigation: Consult with legal counsel before beginning collection. Law enforcement may have specific requirements for evidence handling, or may want to perform the collection themselves. Preserving the evidence (not shutting down the system, not overwriting logs) takes priority over analyzing it. Your role may shift from investigator to evidence custodian.

Employment tribunal or internal disciplinary action (insider threat): The evidence standard for employment proceedings varies by jurisdiction. In the UK, the balance of probabilities standard is lower than criminal proceedings, but the evidence must still be demonstrably unmodified. Use the elevated standard: dual hashes, witness signatures, documented chain of custody. The insider threat investigation in LX15 covers this in detail.

Troubleshooting: common evidence integrity issues

Hash verification fails after transferring evidence via SCP. The most common cause is an interrupted transfer — the file was partially copied and the SCP session timed out or was interrupted. Re-transfer the file from the original source. If the source system has been modified since collection (system rebooted, logs rotated), the re-transferred file will have a different hash. Document the discrepancy and note that the original collected version (with the matching hash) is the authoritative copy.

You forgot to hash a file at the time of collection and are now beginning analysis. Hash it now and document the gap. The hash you compute now verifies integrity from this point forward — it cannot prove the file was unmodified between collection and now. Note in the collection log: "Hash computed [date] — not at time of collection. Integrity between [collection date] and [hash date] cannot be cryptographically verified." This is a weakness, not a disqualification — but it gives opposing counsel or auditors something to challenge.

Two investigators collected evidence from the same system at different times and the hashes differ. This is expected if the system was still running between collections — log files grow, process states change, timestamps update. The difference does not indicate tampering. Compare the hashes of files that should not have changed (binaries, configuration files) — if those match, the collection is consistent. If even static files differ, investigate whether the system was modified between collections.

You need to analyze evidence but only have the original (no working copy). Create a copy now: cp -a evidence/ evidence-working-copy/. Hash the copy: find evidence-working-copy/ -type f -exec sha256sum {} \; > evidence-working-copy-hashes.sha256. Verify the copy matches the original: compare the hash manifests. Perform all analysis on the working copy. The -a flag (archive mode) preserves timestamps and permissions — critical for forensic analysis where file metadata matters.

UAC's hashes.sha256 file shows one or more FAILED entries after collection. A file was modified between the time UAC collected it and the time the hash was computed. For volatile data (process listings, network connections), this is expected — the system is still running and the state changed during collection. For log files and static configuration, a failure warrants investigation: was the file modified by the attacker during collection, or by a system process (log rotation, automatic updates)?

Beyond this investigation: LX14 (IR Reporting) covers evidence handling documentation for legal proceedings: chain of custody forms, evidence integrity statements, and regulatory compliance requirements for evidence preservation.

Myth: "If you did not use a hardware write blocker, the evidence is inadmissible in court."

Reality: Hardware write blockers are the gold standard for physical disk acquisition, but their absence does not automatically render evidence inadmissible. Courts evaluate the totality of the evidence handling: was the collection documented, were hashes computed, can the investigator explain what modifications occurred and demonstrate they did not affect the evidence being analyzed? Cloud disk snapshots, for example, have no concept of a hardware write blocker — they are created by the cloud provider's API and are immutable by design. Software write blocking (mounting read-only) combined with hash verification is accepted for analysis of copied evidence. The standard is not perfection — it is documented, defensible handling with cryptographic verification.

Try it yourself

Create a test evidence directory on your forensic workstation.

Create a test evidence directory on your forensic workstation. Copy a few log files into it. Hash the directory: find evidence/ -type f -exec sha256sum {} \; > evidence/hashes.sha256. Now modify one file (add a blank line to the end of a log file). Verify the hashes: sha256sum -c evidence/hashes.sha256. One file should show "FAILED" — the file you modified. Now restore the file and verify again — it should show "OK". This is the integrity verification workflow you will perform on every investigation. Practice it until the commands are automatic: hash on collection, verify before analysis, document any discrepancy.

Beyond this investigation

Evidence integrity is not a one-subsection topic — it is a discipline that runs through every module in this course. In LX2 (Filesystem Forensics), you will hash disk images before and after timeline generation to prove the analysis did not modify the evidence. In LX14 (IR Reporting), the chain of custody log becomes a required section of the IR report. In LX15 (Insider Threat), the elevated evidence handling standard is mandatory because the investigation may lead to employment tribunal proceedings. In every investigation scenario (LX4-LX13), the collection phase begins with hashing and the analysis phase begins with verification — this subsection teaches the discipline that the rest of the course assumes you practice.

Check your understanding:

1. You collected auth.log from a compromised system three weeks ago and hashed it (SHA256: abc123...). You are now beginning analysis. What is your first step before opening the file? 2. Why does dc3dd provide a more reliable acquisition process than standard dd? 3. The chain of custody log shows a gap — evidence was transferred on March 28 but the recipient's verification hash was not recorded until March 31. Why is this a problem, and what should you document? 4. You need to analyze a 50GB disk image. Should you analyze the original or a copy, and why? 5. You are investigating an insider threat that may lead to an employment tribunal. What evidence handling standard do you use, and what additional steps does it require beyond a standard internal investigation?

Decision point

You are investigating a Linux server and discover evidence of both a cryptominer (resource abuse) and an SSH key theft (lateral movement preparation). The cryptominer is consuming 95% CPU and impacting production. Which do you address first?

Address the lateral movement first. The cryptominer is visible, noisy, and contained to this server — it is causing performance impact but not spreading. The SSH key theft is silent, potentially already exploited, and may have given the attacker access to additional servers. Contain the lateral movement risk: rotate the stolen SSH keys, check the target servers for unauthorized access, and apply network restrictions. Then address the cryptominer: kill the process, remove the binary and persistence mechanisms. Prioritizing the noisy but contained threat over the silent but spreading threat is the most common Linux IR prioritization mistake.

Unlock the Full Course See Full Course Agenda