Explore file identity and rename operations without retransmitting file bytes

[adamziel]

The planner being developed in #367 compares the fresh local index with the local index at the previous push by path. When the same path exists in both indexes, file and symlink changes are determined by type, size, and ctime. When a large file disappears at one path and appears at another, the plan emits a deletion and a full upload even when the only user action was a rename.

Explore whether Reprint can recognize an unchanged local file at a new path and stage a rename operation so the target moves its existing file during commit instead of receiving the same bytes again.

This is a design and measurement issue. It should end with a concrete recommendation before protocol or durable-layout code is added.

File identity choices

Compare at least these approaches:

1. Device id and inode id. Keep the lstat() device id and inode id in each local index entry. Treat the pair as local-machine information, not an identity shared with the target. Target-side commit checks already read both values, but the local push index currently keeps only path, type, size, ctime, and the empty-directory marker.
2. Cryptographic content digest. Hash files while indexing, or hash only rename candidates after comparing the indexes. This costs local reads but can avoid sending file bytes.
3. Hybrid matching. Use device id and inode id to find candidates, then validate unchanged bytes with a digest or another concrete check before emitting a rename.
4. Filesystem-specific identity. Determine whether inode generation, birth time, filesystem UUID, or statx() values add useful protection and whether PHP exposes them on supported platforms.
5. Open file handles. Determine whether retaining an open handle helps within one process and why it cannot be the only mechanism across cancellation or process death.

The analysis must cover:

• An inode id is unique only together with its device id.
• Inode ids may be reused after unlink, and device ids may change after remount or when push state moves to another machine.
• Hard links put the same device id and inode id at several paths. An old path which remains while a new path appears is not necessarily a rename.
• Rename may change ctime. Size and timestamps alone cannot establish unchanged bytes, and mtime can be backdated.
• Identical files at several paths make digest-only pairing ambiguous.
• APFS, ext4, overlay/container filesystems, network filesystems, Windows filesystems used through PHP, and filesystems which return zero or unstable inode ids may behave differently.
• Files, directories, and symlinks need separate conclusions. Directory rename can save many uploads but has more path-conflict consequences.

Any uncertainty must fall back to the existing deletion plus upload behavior. A mistaken rename can move the wrong target bytes; sending a file again is expensive but correct.

Planner shape

Rename candidates do not arrive next to each other in path-sorted indexes. A solution must preserve the processor contract from AGENTS.md: bounded memory, durable cursors, one bounded event per step, and no scan of completed work after resume.

Explore disk-backed ways to pair old and new paths, for example:

• a second index sorted by device id, inode id, and path;
• a candidate list externally sorted by identity and then by path;
• a digest index built only for unmatched files;
• a second resumable planning phase which produces a path-sorted rename list before the ordinary path merge.

Measure the extra index bytes, local reads, temporary disk use, and steps for a large tree. Do not replace the streaming merge with an in-memory map.

Target rename operation

Define what the target must receive and confirm for one rename. At minimum the operation needs the old path, new path, expected type, and whatever check establishes that moving the old target value is safe.

Answer these questions:

• How does the target know the old path still contains the bytes from the previous successful push? A local inode match says nothing about independent target changes.
• Should the sender provide a digest for the target to check locally, or should the target retain something from the previous commit?
• Is a rename staged as durable work and applied only by push_commit, rather than changing the document root during upload?
• What target-confirmed cursor makes a repeated request and a later sender run safe?
• What happens when the old path is absent, the new path already exists, or either path changed after the rename was staged?
• How are rename chains, swaps, case-only renames, directory moves, and moves across mounted filesystems ordered?
• How do excluded paths apply to the old path, new path, their ancestors, and their descendants?
• How do renames interact with work files and work deletes for the same paths?
• Which checkpoint is written before the filesystem rename, and how does commit distinguish interruption immediately before it from interruption immediately after it?
• How does an older target reject or decline the operation so the sender can use deletion plus upload instead?

The target must never apply the same rename twice, follow a symlink while checking either path, move an excluded path, or turn one bounded commit step into an unbounded copy.

Scenarios to test in a prototype

• Rename one unchanged multi-gigabyte file.
• Rename and then edit the file before the next index.
• Delete a file and create a different file which receives the same inode id.
• Create a hard link while keeping the old path.
• Rename one of several files with identical bytes.
• Rename a directory containing many descendants.
• Swap two paths and create a three-path rename cycle.
• Perform a case-only rename on case-sensitive and case-insensitive filesystems.
• Change or remove the old target path independently before commit.
• Stop the sender after staging a rename, and stop the target immediately before and after applying it.
• Resume every interrupted case without retransmitting unchanged bytes or skipping required work.

Expected outcome

Produce a short design note which:

• recommends the local identity components and states exactly when they are trusted only as a candidate;
• defines the rename operation, target checks, durable cursor, commit ordering, and fallback behavior;
• shows how planning remains streaming, bounded, single-pass per durable stream, resumable, and recoverable;
• reports measurements for the candidate algorithms and representative filesystems;
• lists the interruption and ambiguity tests required before implementation.