Skip to content

[PHP] Extract resumable fresh local index processing from PushPlan - #576

Closed
adamziel wants to merge 1 commit into
trunkfrom
codex/share-fresh-local-index
Closed

[PHP] Extract resumable fresh local index processing from PushPlan#576
adamziel wants to merge 1 commit into
trunkfrom
codex/share-fresh-local-index

Conversation

@adamziel

Copy link
Copy Markdown
Collaborator

Moves fresh local index construction out of PushPlan so pull and push can use the same resumable filesystem scan.

FreshLocalIndexProcessor owns FileIndexProcessor, the fresh local index file, its saved byte offset, and the final sort. Its cursor contains every value needed by resume():

$processor = FreshLocalIndexProcessor::start(
    $fresh_local_index_file,
    $filesystem_root,
    $storage_path
);

do {
    $has_next_step = $processor->next_step();
    $processor->flush_pending_output();
    save_cursor($processor->get_cursor());
} while ($has_next_step);

$processor->close();

PushPlan now stores that cursor unchanged. It keeps the existing phase boundary between traversal and sorting, then starts FileSyncPatchPlanner exactly as before. The raw traversal, JSONL writing, output truncation, and sorting code are removed from PushPlan.

Testing

The focused tests cover sorted local-relative index entries, empty directories, resume after an uncommitted output tail, the separate sorting step, PushPlan behavior, bounded sender steps, and process-death resume.

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Pull pipeline performance — large-directory

Site: large-directory · 2,000+ plus targeted file-transfer scenarios files · 10,000 posts · 25,000 postmeta · PHP 8.5.9

Stage PR trunk Δ Status Details
playground-sqlite-db-pull 10.12 s 9.71 s ⚪ +404 ms (+4.2%) condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
trunk: condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
playground-sqlite-db-apply 3.85 s 3.81 s ⚪ +37 ms (+1.0%) condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
trunk: condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
Total 13.96 s 13.52 s ⚪ +441 ms (+3.3%)

Numbers carry runner noise; treat single-run deltas as directional, not authoritative.

📈 Trunk performance history — commit-by-commit timeline.

@adamziel
adamziel force-pushed the codex/share-fresh-local-index branch from a007aea to 1adfe02 Compare August 13, 2026 19:45
@adamziel

Copy link
Copy Markdown
Collaborator Author

Superseded by a single minimal PR based directly on trunk.

@adamziel adamziel closed this Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant