refactor(canvas): admit edits before they mutate, own raster memory, and add a color picker loupe by joshistoast · Pull Request #392 · invoke-ai/InvokeAI-7

joshistoast · 2026-10-01T02:56:59Z

Summary

Reworks the canvas engine for correctness, clear ownership and bounded memory, adds a color picker loupe, and fixes layer-panel controls that went stale. Before, edits could mutate first and fail to record undo, background work could evict pixels in use, undo could replay onto an evicted cache, frames recomposited everything, focused controls fought the canvas for keys, and layer actions such as Filter stayed disabled after a fill until another layer was selected. Now:

Raster ownership: one raster owner accounts base, derived, group and decoded surfaces, pins what operations read, evicts only outside the frame's working set, and keeps dirty or session-held pixels. Project close/export/duplicate cross a paint barrier and never upload a hole.
Admitted edits: every recorded edit runs as a transaction that reserves its undo bytes (and raster prep bytes) before it changes anything. Oversized edits are refused up front with a notice, not applied and then lost. Steps verify reducer and mirror postconditions and roll back on failure.
Failure-atomic history: undo/redo are async; a step moves only when its replay completes, and pixel replays rasterize an evicted or stale cache first. Edits and new strokes wait while a replay runs.
Destructive operations: merge down (now undoable), rasterize, crop, copy, boolean, extract, mask clear/invert, selection fill/erase, transform bake, float commit and every result adoption report distinct outcomes (over-budget, not-ready, stale, busy) as notices. Their edit steps, replay reservations and outcome mappings live in one module: layer operations report a step that did not land as failed, guarded results as stale.
Floating selections: a float lifts only after a drag threshold, is admitted for its undo entry before it cuts anything, and grows that admission with each move, scale or rotation. A transform that would not fit keeps the last accepted one and shows one notice; a refused lift moves nothing (never the whole layer). Its cut layer is not persisted until it lands; cooldown banks it, and every other end restores the hole before persistence resumes.
Filter errors: the filter session publishes typed error codes that the Filter footer translates when shown, so a visible error follows a language change; unexpected failures show a translated summary with the original message as technical detail.
Composition: damage is explicit, frames reuse one description, group surfaces refresh in place, previews and floats compose inside their group, overlay chrome follows any device-pixel ratio, and export applies adjustments layer-locally like the display.
Input: Enter belongs to the canvas only when it has focus; Space/Alt/C still pan or switch after a toolbar or layer click, never over a keyboard-focused control, field or open menu. Tool switches end holds cleanly.
Color picker loupe: hovering with the color picker (the tool, Alt-hold or "Pick a color from the canvas") shows a 120 px loupe of the composited pixels around the pointer, magnified without smoothing, with the sampled pixel boxed as the target (the system cursor is hidden over the canvas) and its RGB values below. It samples through the same compositor as a pick, so the readout is exactly what a click picks; it stays under the pointer through pans and zooms, appears at once over a still pointer, and shows fewer, larger pixels past 8× zoom. Hover chrome (the loupe and the brush ring) leaves the canvas with the pointer unless a gesture owns it.
Live layer actions: the React Compiler cached render-time engine reads, so Filter stayed disabled after a fill until another layer was selected, and merge, group and delete items, merge visible and the control-LoRA/Z-Image position warnings could show stale state (all also on main). useCanvasEngineRead reads them fresh on every render, pixel publish and document change (a new engine documentEpoch, published once the mirror has reacted).
Layer pane tabs: a collapsed editor or color pane keeps every divider in its tab strip; only the tab whose panel is shown hides the dividers beside it.
Document updates: value edits travel through the document index (no full-tree diff), strokes render at most once per frame, and the color picker reuses its scratch.
CI: the workflow generator browser test loads its lazy editor chunk before running, so it no longer times out against a cold CI dev server.

QA Instructions

From invokeai/frontend/webv2 with Node 22.14.0 and pnpm 10:

pnpm run check:release passed end to end with the loupe (tsc, oxlint, oxfmt, the architecture policy, the node suite at 599 files / 9,290 tests and the Chromium suite at 268 files / 2,014 tests, the performance build and browser gates, and the accessibility journeys).
The later commits (hidden picker cursor, loupe cleared on window blur, pane tab dividers) re-ran tsc, oxlint, oxfmt and their affected node and Chromium suites; CI runs the full set on the final head.
Environment note: on this machine, two browser tests in src/features/workflow/ui/editor/InvocationFlowNode.browser.test.tsx ("exports authored source images…") can fail when run in isolation, on unrelated branches too, and some browser tests time out only while other suites run concurrently. They fail on main too; they passed in the final full run.
Real browser (mock backend, Edit layout):
- Click the Brush button, then hold Space over the canvas: the tool switches to View and returns to Brush on release, and the button is not re-activated.
- A brush stroke records one "Brush stroke" step; undo and redo replay it.
- Merge down onto an undecodable fixture layer is refused with a notice and records nothing.
- A filled 1024² selection lifted with Move and dragged past the 256 MB undo budget at 10% zoom stops at its last accepted position with one notice; Enter records one "Move selection" step, and undo/redo restore the pixels and the selection.
- Fill a new raster layer: Filter in its properties enables at once, without selecting another layer first; Merge down on it enables the same way.
- Arm "Pick a color from the canvas" and hover an anti-aliased stroke edge: the loupe shows hard magnified pixels, the boxed center pixel and its RGB values, with no system cursor over it; leaving the canvas hides it and Escape restores the brush cursor.
- Collapse the editor panes: the Properties | Transform | Overview strip shows both dividers.
- A filter run whose enqueue fails shows "The filter could not run." with the server's message under Technical details; switching the language to German updates the visible error, and Cancel clears it.

Measurements (node unless noted):

Measure	Before	After
One value edit, 10k nodes	0.33 ms (`updateNodesTracked` + derive)	0.02 ms (`updateNodeValues`)
Reducer + mirror per value edit, 10k nodes	—	0.6 ms, zero forest diffs, zero nodes materialized
Live stroke renders per frame (dense trace)	one per input event	at most one per presented frame (counted)
Group surface while painting a member	full rebuild per stroke frame	0 rebuilds, one region refresh per frame (counted)
Selection mask replacement, 4096²	64 MB RGBA copy	GPU blit, no copy
Export of an adjusted, scaled layer	16 levels off the display	within 1 level of the display

Export work stays bounded by the output size for downscaled layers.

Review

Resolved findings:

Composition: group refresh margins under scaled members and floats; preview residency; document-size bounds memo; staged outline culling.
Persistence: fail-fast paint barriers for open edits.
Input: Space ownership after a mouse click; restoring the held tool on a switch; the eyedropper during a hold.
Admission:
- materialized pixel edits were admitted twice;
- a pixel edit's own publish cancelled it, a double-consume bug;
- refused previews left unrecorded values behind;
- admission leaked on paint-tool early returns;
- extents were not restored after refused shapes and inverts;
- transform replays were unreserved;
- a composed stroke redo was not failure-atomic;
- copies were allocated before admission.
Export: the downscaled export cost regression.
Tests: assertions weakened in the engine-test rewrite.
Floats and filters: renewing a float's admission after a lock counted its bytes twice; a failed landing could leave the layer cache enlarged; filter-source refusals still reached the UI as English text; rollback paths lacked coverage.
Live layer actions: merge selected, merge visible and the control-layer position checks shared the same compiler staleness and now read through the hook; merge visible stops at its second contributor so the extra reads stay cheap.
Loupe: the pointer-leave handler was rebuilt per call, so each detach left a listener (and the engine) behind; the loupe was anchored in document space and drifted under keyboard zoom; it stopped magnifying past 8×; its checker cost about 450 fills per frame; it waited for a move after an Alt hold; it stayed drawn after a window blur.

All of these have regression tests checked against the original defect.

Remaining limitations:

A live float holds its largest accepted undo reservation until it lands or is dropped.
The engine registry still disposes dirty engines belonging to another Workbench, which predates this PR. Background reservations are now refused while raster memory is in overage.

Compatibility / Rollout

Persisted data: no project or document format changes.
Canvas capability API:
- history.undo/redo/stepBy return promises with a replay status.
- Merge down, rasterize, copy to raster and the mask edits return result unions.
- Structural previews use beginStructuralPreview sessions.
- interaction.set accepts only setting keys.
- surface.attach takes the keyboard root.
All callers in webv2 are updated.
Performance baselines: performance/browser-baseline.json pins the new startup modules explicitly. After merging main, route resources were re-recorded: startup "other" bytes grew 4.7 KB (the English catalog, from main and the new filter messages) and editor routes make nine fewer startup requests.
Canvas capability API (addition): interaction gains a read-only documentEpoch signal.

Checklist

The PR has a short but descriptive title, suitable for a changelog
Meaningful regression coverage added / updated where needed; obsolete tests/code removed
Persisted-state and API changes include required migrations / compatibility validation
Relevant performance/efficiency opportunities considered; material claims have evidence
Material review findings resolved and relevant checks rerun
Documentation added / updated (if applicable)
Updated What's New copy (if doing a release after this PR)

… surface Base, derived, group and decoded stores keep running byte totals; working-set eviction keeps pinned and unsaved pixels as accounted overage. Raster reads return pinned RasterReadLeases; a dirty layer without a cache now fails the paint barrier instead of saving nothing. Decodes refuse pre-aborted requests, cooldown/disposal release group surfaces, and closing a project crosses its paint barrier first.

A genuine tool switch cancels the active drag through the outgoing tool and drops its tail; reset cancels before restoring a held tool. Enter and Space act only while the canvas surface owns focus, ignoring controls, prevented events and modifier chords. Freehand lasso/shape spacing is measured in CSS pixels, and the cursor ring follows size changes only.

Value edits locate nodes through the index, copy only the sibling arrays on each edited path and record the replaced nodes; the mirror folds those records instead of diffing forests, and outermostNodes and move-overlay overrides resolve by id. The engine prunes group surfaces only on restructuring.

Stroke samples append immediately with incremental decimation; the accumulated stroke renders in a scheduler task that runs before the frame's flags are taken, and pointerup renders the final stroke.

The readback only feeds the coverage scan and outline trace; the mask takes its pixels by drawImage.

Layer caches expose a revision; the sampler skips repeated pixels of an unchanged document and cache revision, and compositeDocument accepts a prepared draw plan.

Merge Down of empty layers keeps an empty cache; pixel-edit rollbacks reinstate caches through the store; cooldown keeps pinned and held caches. Read leases capture guards before pinning and release the source pin once a copy exists; pinned layers persist untrimmed. Open-project flushes cross the paint barrier through the renamed LiveCanvasEngines registry, and close re-crosses it after the push.

…de group scopes Damage is full, none or bounded: offscreen damage draws nothing and bounded repaints cull layers, groups and previews; one cached frame description (matrices, bounds, scopes) feeds frame demand, compositing and sampling. Grouped filter previews and floats compose inside the group surface, which now refreshes only member-damaged regions in place; compositing requires the raster backend. Overlay chrome draws in CSS pixels under a DPR base transform, and the viewport owns sub-one and fractional ratios for both backing size and view.

…t on open edits Close, export and duplicate cross the barrier through the open-project handle; rename keeps its tolerant flush. Document-write barriers fail fast with a "finish or cancel the current edit" reason instead of polling held pixels.

…float drags Group damage pads one source pixel before the member matrix; a moved float damages its old and new landing. Previewed layers stay resident, leaf bounds follow document size, and staged outlines and overlay shade survive bounded repaints.

…fore tool switches Space, Alt, C and Escape reach the canvas when focus arrived by pointer; keyboard-reached controls, fields and open overlays keep them. The widget passes its keyboard root to attach; a genuine switch restores the held tool first, and cancelled drags keep the cursor tracking.

Batched layer configs apply as one value edit; folding stops past a quarter of the forest; tools must schedule stroke frames through the engine. Adds coverage for group lock/hidden flips, group prune on delete, incremental decimation, override order and shared preparation reuse.

Float landings expand one source pixel before the placed matrix; float drags on scaled members are covered.

A colour-sample request ends any temporary hold before recording the tool to return to; the pointer-focused control is held weakly.

…ring a hold The surface test starts on brush and owns Space only through its keyboard root; choosing the held tool again deactivates nothing.

Export adjusts the region a layer's output samples before resampling, matching the display; downscaled layers stay bounded by the output size.

…ry failure-atomically Edits run as admitted transactions: history and raster bytes are reserved first, steps are verified, and refusals (over-budget, busy, stale) reach users as notices. Undo/redo are async and failure-atomic, pixel replays rasterize untrustworthy caches first, previews run as owned sessions, and merge down, rasterize, mask clear/invert and transform bake are undoable steps. Removes no-op bitmap tracking and a pass-through engine hook, narrows interaction writes to settings, renames the pixel epochs, and lets export reuse the display's adjusted copies.

Edit-step builders, the refusal notice adapter and the damage constant module join the editor startup graph; the removed surface budget and engine-hook re-export leave it.

A failed rasterization keeps the step unpersisted and reported; an empty source replaces a stale cache before the patch. Clarifies that export's adjusted copy is built on a miss.

editSteps.ts owns byte accounting, replay reservations with one refusal message, layer add/replace/remove steps and new paint layers. Rejected publications map explicitly: layer operations report failed, guarded results report stale.

A float is admitted for its commit footprint before it cuts anything and grows that admission, from geometry, with each accepted move, scale or rotation; a transform that does not fit is refused once and the float stays put. A refused lift no longer falls back to moving the layer; cooldown banks a live float, and every terminal path returns the admission after the hole is refilled.

The filter session publishes typed error codes for refused applies, unreadable or stale sources and unexpected failures, keeping a failure's message as technical detail. FilterOptions translates them through an exhaustive key map, so a visible error follows a language change; English catalog entries cover every code.

The React Compiler cached render-time engine reads, so Filter stayed disabled after a fill until another layer was selected. useCanvasEngineRead takes those reads fresh on every render and pixel publish (Filter, layer menu merge/group/delete, merge visible, control-layer checks); merge-visible stops at its second contributor.

The engine publishes a documentEpoch once the mirror has reacted to a document change, and useCanvasEngineRead re-reads on it. Control-layer position warnings and the other routed reads now follow edits to other layers, such as reordering or toggling a parent group.

… into refactor/canvas-engine-overhaul

The English catalog grew with main's strings and the new filter refusal messages; startup requests dropped by nine on editor routes.

The first generator test waited one second for a lazy chunk that CI's dev server could still be transforming.

The picker shows a loupe of the composited pixels around the pointer with the sampled pixel boxed and its RGB values, sampled once per overlay frame through the pick's own sampler. Hover chrome leaves the canvas with the pointer, and input handlers stay stable so detach removes every listener attach added.

The color picker hides the system cursor over the canvas, so the boxed center pixel is no longer covered by a crosshair.

SegmentTabs hides the dividers beside a tab only while its panel is shown, so a collapsed strip keeps every divider.

joshistoast added 19 commits September 30, 2026 18:31

perf(canvas): render live strokes at most once per frame

dd0e590

Stroke samples append immediately with incremental decimation; the accumulated stroke renders in a scheduler task that runs before the frame's flags are taken, and pointerup renders the final stroke.

perf(canvas): blit replacement selection masks instead of copying RGBA

b38f70d

The readback only feeds the coverage scan and outline trace; the mask takes its pixels by drawImage.

perf(canvas): reuse color picker scratch, draw plan and last sample

12bb38e

Layer caches expose a revision; the sampler skips repeated pixels of an unchanged document and cache revision, and compositeDocument accepts a prepared draw plan.

fix(canvas): pad a moved float's landing like member damage

d1aa01b

Float landings expand one source pixel before the placed matrix; float drags on scaled members are covered.

fix(canvas): arm the eyedropper from the tool under a held key

a11beee

A colour-sample request ends any temporary hold before recording the tool to return to; the pointer-focused control is held weakly.

test(canvas): prove keyboard-root ownership and same-tool switches du…

c787a45

…ring a hold The surface test starts on brush and owns Space only through its keyboard root; choosing the held tool again deactivates nothing.

perf(canvas): adjust exported layers in layer-local space

ea3554b

Export adjusts the region a layer's output samples before resampling, matching the display; downscaled layers stay bounded by the output size.

test(perf): pin the canvas engine's new startup modules

64ba594

Edit-step builders, the refusal notice adapter and the damage constant module join the editor startup graph; the removed surface budget and engine-hook re-export leave it.

test(canvas): cover pixel undo when the cache cannot be restored

895aecc

A failed rasterization keeps the step unpersisted and reported; an empty source replaces a stale cache before the patch. Clarifies that export's adjusted copy is built on a miss.

joshistoast requested review from JPPhoto, Pfannkuchensack, blessedcoolant and lstein as code owners October 1, 2026 02:57

github-actions Bot added the frontend label Oct 1, 2026

joshistoast added 6 commits September 30, 2026 21:20

Merge branch 'main' into refactor/canvas-engine-overhaul

24419e0

joshistoast added 6 commits September 30, 2026 22:55

Merge remote-tracking branch 'origin/refactor/canvas-engine-overhaul'…

9f533d1

… into refactor/canvas-engine-overhaul

test(perf): re-record route resources after merging main

669d360

The English catalog grew with main's strings and the new filter refusal messages; startup requests dropped by nine on editor routes.

test(webv2): load the generator editor before its browser tests

9327c7f

The first generator test waited one second for a lazy chunk that CI's dev server could still be transforming.

feat(canvas): let the loupe's center box be the picker's target

b257b7e

The color picker hides the system cursor over the canvas, so the boxed center pixel is no longer covered by a crosshair.

fix(webv2): restore pane tab dividers while the panes are collapsed

61e9f2c

SegmentTabs hides the dividers beside a tab only while its panel is shown, so a collapsed strip keeps every divider.

joshistoast enabled auto-merge October 1, 2026 06:35

joshistoast disabled auto-merge October 1, 2026 06:35

joshistoast changed the title ~~refactor(canvas): own raster memory, admit edits before they mutate, and make history failure-atomic~~ refactor(canvas): admit edits before they mutate, own raster memory, and add a color picker loupe Oct 1, 2026

joshistoast merged commit 8188a75 into main Oct 1, 2026
14 checks passed

joshistoast deleted the refactor/canvas-engine-overhaul branch October 1, 2026 06:49

Navigation Menu

Repository navigation

refactor(canvas): admit edits before they mutate, own raster memory, and add a color picker loupe - #392

refactor(canvas): admit edits before they mutate, own raster memory, and add a color picker loupe#392
joshistoast merged 31 commits into
mainfrom
refactor/canvas-engine-overhaul

joshistoast commented Oct 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

joshistoast commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

QA Instructions

Review

Compatibility / Rollout

Checklist

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

joshistoast commented Oct 1, 2026 •

edited

Loading