Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions docs/PUSH-SYNC.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,15 +135,14 @@ A push plan is an internal part of the sender lifecycle:
create stores the exclusions in `excluded_paths.json` after creating the
active `plan/` directory.
2. The sender starts one internal `PushPlan`. The plan copies the exclusions to
`plan/excluded_paths.json`, then opens
`plan/fresh_local_index.jsonl` and a `FileIndexProcessor`. Each internal
`indexing` step advances one traversal event, appends its JSONL entries when
applicable, and updates its traversal cursor and fresh-index byte offset.
`plan/excluded_paths.json`, then starts a `FreshLocalIndexProcessor`. Each
internal `indexing` step advances one filesystem traversal event. The
processor writes `plan/fresh_local_index.jsonl` and updates its cursor.
One sender step runs at most 256 internal planning steps without crossing
an internal phase boundary, flushes their output, and stores the resulting
cursor in `sender.json` before returning.
3. Once traversal is complete, the plan enters `starting_diff`. The next step
starts the index diff and enters `diffing`.
sorts the fresh local index, starts the index diff, and enters `diffing`.
4. Each later `next_step()` compares at most one path represented by either
index. It returns true while another planning step remains and false when
both indexes reach EOF. It
Expand Down Expand Up @@ -175,10 +174,10 @@ sender run. Keeping another cursor and retained handle for this post-commit copy
is not justified until measurements from materially larger installations show
that it matters.

The cursor contains the plan directory, filesystem root, local index file,
document root's local relative path, and current planning position. During
indexing, that position contains the `FileIndexProcessor` cursor and committed
fresh-index byte offset. During diffing, each step flushes only the path list or
The cursor contains the plan directory, local index file, document root's local
relative path, and current planning position. During
indexing, that position contains the complete `FreshLocalIndexProcessor`
cursor. During diffing, each step flushes only the path list or
append-only active deletion roots file changed by that step before updating
the two output byte offsets and the nested FileSyncPatchPlanner cursor. Each
active deletion root links to the preceding one, so continuation reads only
Expand Down Expand Up @@ -348,9 +347,9 @@ without closing, the next process uses the preceding sender boundary and
receiver-confirmed cursors to account for later remote work.

During PushPlan's internal `indexing` phase, the plan retains one
`FileIndexProcessor` and the open fresh local index across steps. A
newly opened plan truncates that file to the byte offset stored with the
processor cursor before continuing. The sender lazily opens
`FreshLocalIndexProcessor` across steps. A resumed processor truncates the
fresh local index to the byte offset stored with its filesystem traversal
cursor before continuing. The sender lazily opens
`local_paths_to_push.jsonl`, `local_paths_to_delete`, and the current local file.
It retains those handles across `next_step()` calls, lets each handle advance
with the work, and seeks only when a newly opened or receiver-confirmed offset
Expand Down
19 changes: 9 additions & 10 deletions docs/PUSH-TERMINOLOGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ local relative path to a document-root-relative path.
state directory. Files-pull advances only the entries for completed local
mutations; files-push atomically replaces the local index only after the
target confirms commit. Use `$local_index_file`.
- A **fresh local index** is the current filesystem-root scan created while
planning a push. Use `$fresh_local_index_file`.
- A **fresh local index** is the current filesystem-root scan used for sync
planning. Use `$fresh_local_index_file`.
- An **index entry** records one path, type, size, and ctime. Use
`$index_entry`.
- A **file sync patch planner** compares a patch base index with a patch result
Expand Down Expand Up @@ -312,12 +312,11 @@ active deletion roots file. That file remembers directory deletions which
cover index paths the planner has not processed yet.

The PushPlan cursor is stored in `sender.json`. It contains `plan_directory`,
`filesystem_root`, `local_index_file`, and the current
planning position. During `indexing`, that position contains the
FileIndexProcessor cursor and the committed byte offset in
`fresh_local_index.jsonl`. During `diffing`, it contains the output offsets
and the complete FileSyncPatchPlanner cursor. PushPlan
stores that nested cursor without unpacking or rebuilding it. The active
`local_index_file`, and the current
planning position. During `indexing`, that position contains the complete
FreshLocalIndexProcessor cursor. During `diffing`, it contains the output
offsets and the complete FileSyncPatchPlanner cursor. PushPlan stores either
nested cursor without unpacking or rebuilding it. The active
deletion roots file is append-only; each entry links to the preceding active
directory. The exclusions have a maximum of 100 paths. The `sender.json`
phases are `creating`, `finishing_previous_commit`,
Expand Down Expand Up @@ -524,8 +523,8 @@ Use these names verbatim inside `PushPlan`:
| Index entry and shape | `$index_entry`, `$local_index_entry`, `$local_index_entry_shape`, `index_entry_shape()` |
| Cursor | `$cursor`, `get_cursor()` |
| Plan-owned excluded paths | `$excluded_paths_file` |
| Fresh local index processor | `$file_index_processor`, `next_file_index_step()` |
| Fresh local indexing cursor | `IndexingCursor`, `file_index_cursor` |
| Fresh local index processor | `FreshLocalIndexProcessor`, `$fresh_local_index_processor` |
| Fresh local indexing cursor | `fresh_local_index_cursor`, `$fresh_local_index_cursor` |
| Fresh local index byte offset | `$fresh_local_index_byte_offset` |
| Open fresh local index | `$fresh_local_index_handle` |
| Combined index bytes | `$index_bytes_total` |
Expand Down
Loading
Loading