Skip to content

[PHP] Rewrite URLs in live databases with resumable record cursors - #578

Open
adamziel wants to merge 1 commit into
trunkfrom
codex/rewrite-live-database-urls
Open

[PHP] Rewrite URLs in live databases with resumable record cursors#578
adamziel wants to merge 1 commit into
trunkfrom
codex/rewrite-live-database-urls

Conversation

@adamziel

@adamziel adamziel commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Adds db-rewrite-urls to rewrite URL-bearing values in a live MySQL or SQLite database without exporting or parsing SQL.

DatabaseUrlRewriteProcessor owns the DatabaseRowsReader introduced in #606, receives structured records directly, and performs one bounded record step at a time. Changed records use primary-key-scoped prepared UPDATE statements with positional bindings. No table lock or lifecycle-long transaction is taken.

The positional Reprint API URL is optional, and this command does not use --fs-root. Omitted database options fall back to the target recorded by db-apply when available.

Examples

reprint db-rewrite-urls --state-dir=./state \
  --rewrite-url https://old.example https://new.example

reprint db-rewrite-urls --state-dir=./state \
  --target-engine=sqlite --target-sqlite-path=./wordpress.sqlite \
  --rewrite-url https://old.example https://new.example

Resume semantics

Progress is saved after each record decision. SIGINT or SIGTERM stops after the active step and exits with status 2; rerun the command to resume. Completed records are not scanned again. An update interrupted before its cursor is saved may be read again, but the already-rewritten value produces no second update.

A non-empty table without a primary key stops the command. Resume also rejects a changed target or URL mapping, and a concurrent change to the selected row stops rather than overwriting it.

Testing

The importer and MariaDB suites cover structured records, bound updates, identifiers, saved targets, primary keys, batch boundaries, and cursor resume. An E2E test sends SIGTERM mid-table, resumes in a new process, and confirms each content row is processed and updated once.

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Pull pipeline performance — large-directory

Site: large-directory · 2,000+ plus targeted file-transfer scenarios files · 10,000 posts · 25,000 postmeta · PHP 8.5.9

trunk baseline unavailable — showing PR numbers only.

Stage Wall time Resume attempts Status Details
playground-sqlite-db-pull 9.79 s 1 condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
playground-sqlite-db-apply 3.88 s 1 condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
Total 13.67 s

Numbers carry runner noise; treat single-run deltas as directional, not authoritative.

📈 Trunk performance history — commit-by-commit timeline.

@adamziel
adamziel force-pushed the codex/rewrite-live-database-urls branch from 6ccfe30 to 270331d Compare August 14, 2026 12:03
@adamziel
adamziel changed the base branch from trunk to codex/extract-database-row-reader August 14, 2026 12:04
adamziel added a commit that referenced this pull request Aug 14, 2026
Extracts `DatabaseRowsReader` from `MySQLDumpProducer` so database
records can be consumed without generating SQL first.

`MySQLDumpProducer` now owns a reader and remains responsible for SQL
formatting, oversized values, and the dump lifecycle. The reader owns
table discovery, metadata, bounded primary-key queries, exclusions, and
cursor state.

This does not change dump output. The child PR #578 uses the reader to
process structured live database records directly.

## Example

```php
$reader = new DatabaseRowsReader($pdo, [
    "batch_size" => 250,
]);
$reader->initialize_tables_to_process();

while ($reader->move_to_next_table()) {
    while ($reader->next_record()) {
        $table = $reader->get_current_table();
        $record = $reader->get_current_record();

        process_record($table, $record);
        $reader->clear_current_record();
    }
}
```

## Testing

The existing dump producer tests cover batch boundaries, composite keys,
exclusions, data types, and cursor resume. PHP 7.2 compatibility,
PHPStan, and CI pass.
@adamziel
adamziel force-pushed the codex/rewrite-live-database-urls branch from 270331d to e00a860 Compare August 14, 2026 12:16
@adamziel
adamziel changed the base branch from codex/extract-database-row-reader to trunk August 14, 2026 12:16
@adamziel
adamziel force-pushed the codex/rewrite-live-database-urls branch from e00a860 to 15bc621 Compare August 14, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant