Skip to content

perf(scan): read the album-covers pass's stored images in one batch - #755

Merged
mangoleaf merged 1 commit into
release/v1.5.0from
fix/album-artwork-image-infos
Nov 6, 2026
Merged

mangoleaf merged 1 commit into
release/v1.5.0from
fix/album-artwork-image-infos

Conversation

@mangoleaf

@mangoleaf mangoleaf commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

What

A library scan's album-covers pass now reads its albums' stored images in one batched read instead of one statement per album. It runs on every scan, unchanged albums included. Which albums keep their cover and which have it published again is unchanged, and so are the images stored.

Upgrade note: none

Why

gl-1277. After the walk, finish_album_artwork checks each album the scan saw still stores the cover the walk gave it, and skips the album when it does. It read each album's image rows with get_image_infos(id), one statement per album on every scan. That is thousands of point reads on a large music library, each repeated on every routine rescan. The batched read the issue points at (preload_album_artwork's) runs before the walk, and the walk writes album rows after it, so it cannot be reused: the pass needs a fresh read of its own.

finish_album_artwork is Ferrofin's own closing pass (it publishes the shared, content-addressed album cover the track AudioImageProvider extraction gives its album, and backfills the tracks). It is not a port of one C# method. The change keeps every decision it makes, so no handlers::VERIFIED or VERIFIED_SUBSYSTEMS row changes.

How

crates/ferrofin-core/src/library_scan.rs:

  • The new albums_keeping_their_cover reads the image rows of the albums the pass could skip (not in albums_needing_backfill, with a walk cover) in one get_image_infos_batch. The repository already splits the ids into chunks (padded_in_chunks). It returns the albums whose stored Primary is still their walk cover.
  • finish_album_artwork builds that set on its first album that could be skipped and skips the albums in it, as the per-album check did. Every other album goes to inherit_album_cover as before.
  • The pass serves the priority lane between albums (item refreshes requested during the scan), and the old per-album read came after it, so a served refresh's writes were always seen. When serve_lane reports that a refresh touched anything, the set is dropped and the remaining albums are read again in one batch, so no album is judged by rows a refresh may have rewritten.
  • A failed batch read logs a warning and counts as no album keeping its cover, so every album has its cover published again. That is what a failed single read did before, without the log line.

Verification

  • New test crates/ferrofin-core/tests/library_scan_album_artwork_statements.rs, a_rescan_reads_no_album_images_one_album_at_a_time. It records the statements a rescan of a three-album library runs. Before the fix:
    assertion `left == right` failed: no album's images are read on their own: [...]
      left: 3
     right: 0
    
    It runs three SELECT * FROM "BaseItemImageInfos" WHERE "ItemId" = ?1, one per album. After the fix it runs none, and every album and track keeps the same image path.
  • Live check: a scratch server with a three-album music library (a cover.jpg and two FLAC tracks per album, remote fetchers off), created with POST /Library/VirtualFolders and rescanned twice with POST /Library/Refresh. GET /Items/{id}/Images for every album and track returned the same rows after all three scans (Primary, the shared album cover, 64x64, the same blurhash). In the server's sqlx debug log, a rescan's album-covers pass now makes one batched BaseItemImageInfos read and no single-album read.
  • cargo nextest run -p ferrofin-core: 5123 passed. cargo clippy -p ferrofin-core --all-targets --all-features -- -D warnings and cargo fmt --all --check: clean.
  • ci_gates.sh on 08d0ebe9: fmt, clippy, doctests and ferrofin-core coverage (95.70 % lines) pass. nextest: 16529 of 16530 pass. The one failure is ferrofin-wasm::wasm_hello_guest, which fails locally before it builds the guest (sccache: error: path must be shorter than SUN_LEN, the local sccache socket path). That is an environment problem, not this change. GitHub CI on the same head passes all 14 checks, wasm_hello_guest included.
  • Perf path: touches ferrofin-core (library scan closing pass). Performance is benchmarked per release.
  • Start: release/v1.5.0. No open PR changes finish_album_artwork (pr_stack.py plan).
  • Merge danger
    • Conflicts left for merge time: none.
    • Door: two-way. A revert restores the per-album reads. Nothing stored changes.
    • Blast radius: music scans. If wrong, an album's tracks could miss a cover update until a later scan.

Checklist

  • Commit subjects follow Conventional Commits (type(scope): subject) and are signed off (git commit -s)
  • cargo fmt --all --check clean
  • cargo clippy --all-targets --all-features -- -D warnings clean
  • cargo nextest run --workspace + doctests pass (locally, all but the sccache-bound wasm_hello_guest; GitHub CI runs that one green)
  • Per-crate coverage still ≥80% for touched crates (CI's command, see CONTRIBUTING.md)
  • If a handler or core path changed: compared against the upstream C# at UPSTREAM_TAG, and the operation's handlers::VERIFIED row or the subsystem's handlers::VERIFIED_SUBSYSTEMS row added or updated with its divergences (a re-checked row also leaves its earlier-pin list), or the description says why the touched behaviour is unchanged
  • If a perf-sensitive path (ferrofin-core, ferrofin-db, ferrofin-api) was touched: noted in the description (performance is benchmarked per release, not per PR)
  • Ran the server and exercised the change over real HTTP (for data/auth/stateful paths)

After the walk, finish_album_artwork checks each album the scan saw
still stores the cover the walk gave it, and skips the album when it
does. It read each album's image rows with its own statement, on every
scan and for unchanged albums too: one read per album, thousands on a
large music library (gl-1277).

The albums the pass could skip (not waiting for a track backfill, with
a walk cover) are now read in one get_image_infos_batch, which splits
the ids into chunks itself, and the pass skips the ones whose stored
Primary is still their walk cover. Every other album has its cover
published by inherit_album_cover as before. preload_album_artwork's
batch could not be reused: it runs before the walk, which writes album
rows after it.

The pass serves the priority lane between albums, and the old
per-album read came after it. A lane refresh that touched anything now
drops the set, and the remaining albums are read again, so no album is
judged by rows a served refresh may have rewritten. A failed read is
logged and sends the albums to inherit_album_cover, as a failed
single read did.

The new library_scan_album_artwork_statements test pins a rescan of
three unchanged albums to no single-album image read (three before)
and checks every album and track keeps its image.

Signed-off-by: Ken Smith <ken@mangoleafstudios.com>
@mangoleaf
mangoleaf marked this pull request as ready for review November 6, 2026 04:55
@mangoleaf
mangoleaf merged commit 7a2703b into release/v1.5.0 Nov 6, 2026
14 checks passed
@mangoleaf
mangoleaf deleted the fix/album-artwork-image-infos branch November 6, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant