Interactive

File Investigator

Everything worth asking about a single file, in the order you would ask it. Put a path in and every command below rewrites itself for that path — ready to copy, nothing to substitute by hand at three in the morning.

What is it, really

Why

Before anything else, establish what kind of object you are looking at. A symlink, a socket, a device node and a regular file all behave differently, and `ls` alone will let you assume wrongly.

Run

$ stat {p}

Type, size, inode, device, links, and all three timestamps

$ file {p}

What the content actually is, regardless of the extension

$ readlink -f {p}

Resolve every symlink to the real path

$ ls -la {p}

Mode, owner, size and name, including a trailing + for an ACL

$ df -h {p}

Which filesystem it lives on — often not the one you assumed

When was it touched

Why

The three timestamps answer different questions. mtime is when the content changed, ctime is when the inode changed — permissions, ownership, link count — and atime is when it was last read, if the mount even records it.

Run

$ stat -c "atime %x%nmtime %y%nctime %z" {p}

The three timestamps, labelled

$ find {d} -maxdepth 1 -newer {p} -printf "%T+ %p\n" | sort

Everything in the directory modified after this file

$ find {d} -maxdepth 1 -mmin -60 -type f

Neighbours changed in the last hour

$ ls -lu {p}

Show atime instead of mtime

$ ls -lc {p}

Show ctime instead of mtime

Caution

ctime is not creation time. Linux mostly does not record one at all, though `stat` will show Birth on filesystems that do. A ctime newer than mtime means metadata changed — often a chmod or chown you were not told about.

Who can reach it

Why

Reaching a file requires execute permission on every directory above it, so the mode on the file itself is only the last gate. ACLs and SELinux deny independently and neither appears in a normal listing.

Run

$ namei -l {p}

Every component of the path with its mode and owner

$ getfacl {p}

POSIX ACLs — the mask entry is the usual surprise

$ ls -Z {p}

SELinux label

$ lsattr {p}

Extended attributes, including the immutable bit

$ sudo -u <user> test -r {p} && echo readable || echo denied

Settle it empirically as the user that matters

Caution

A `+` at the end of the mode in `ls -l` means an ACL exists and the mode you are reading is not the whole story.

Who has it open

Why

An open file descriptor keeps a file alive after deletion, holds its disk space, and explains why a truncate or a rotation had no effect. This is where missing disk space hides.

Run

$ lsof {p}

Every process holding this file open

$ fuser -v {p}

The same question, with the access mode

$ lsof +L1 | head -20

Deleted files still held open, across the whole system

$ ls -l /proc/*/fd 2>/dev/null | grep {n}

Find the descriptor when lsof is not installed

Caution

If a deleted file is holding space, truncate it through the descriptor — `: > /proc/<PID>/fd/<FD>` — or restart the holder. Deleting it again does nothing; it is already unlinked.

What is inside, and has it changed

Why

A checksum settles the "is this the same file" argument in one line, and looking at the head and tail of a file is faster than opening something that turns out to be 8 GB.

Run

$ sha256sum {p}

The checksum to compare against a known-good copy

$ head -20 {p}

The start, safely

$ tail -50 {p}

The end, which is where a log tells you what happened

$ wc -lc {p}

Lines and bytes

$ xxd {p} | head

When `file` says binary and you want to see why

$ diff <(sha256sum {p} | cut -d" " -f1) <(sha256sum /known/good | cut -d" " -f1)

Compare against a reference copy

Where it came from

Why

On a package-managed system, most files belong to a package that knows what they should look like. That turns "has this been modified" from a guess into a query.

Run

$ dpkg -S {p}

Which Debian package owns it

$ rpm -qf {p}

Which RPM owns it

$ dpkg -V $(dpkg -S {p} | cut -d: -f1)

Verify the package — lists files that differ from the manifest

$ rpm -V $(rpm -qf {p})

The same verification for RPM

Caution

A file that belongs to a package but fails verification has been edited by hand or by a configuration tool. That is worth knowing before you spend an hour on the contents.

What is producing it

Why

When a file is growing, changing or reappearing after deletion, the useful question is which process is responsible — and watching it is more reliable than reasoning about it.

Run

$ watch -n1 "ls -l {p}"

See it grow in real time

$ inotifywait -m {p}

Every open, write and close as it happens

$ auditctl -w {p} -p wa -k filewatch

Audit writes and attribute changes

$ ausearch -k filewatch -ts recent

Read back what the audit rule caught

$ lsof -r2 {p}

Repeat the open-handle check every 2 seconds

Caution

Remove the audit rule when you are done — `auditctl -W {p} -p wa -k filewatch` — or it will quietly fill the audit log.