Skip to content

feat(generation): restore hosted-provider options, Canvas modes and model record editing - #9811

Open
joshistoast wants to merge 4 commits into
mainfrom
feat/parity-external-providers
Open

joshistoast wants to merge 4 commits into
mainfrom
feat/parity-external-providers

Conversation

@joshistoast

@joshistoast joshistoast commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Hosted image providers (OpenAI, Gemini, Seedream and the other external generators) lost several legacy capabilities in webv2.

Provider options and named sizes. Generate offers the options each provider's node accepts (OpenAI quality, background and input fidelity; Gemini temperature and thinking level; Seedream watermark and optimize-prompt) only for models whose provider and capabilities support them, plus the provider's named image sizes and preset resolutions. They compile into the external invocation with the node's own field names, are recorded in metadata and restored by Recall All and Remix. A named size is sent only while it still matches the current width and height, so a hand-typed size is sent as is.

Canvas. External models can now generate from Canvas in the modes they advertise: text-to-image, image-to-image with the composited frame as the init image, and inpaint with the composited mask, sized to the bounding box. Outpaint, unadvertised modes, control layers and regional guidance are rejected with named reasons; the render-size slot and strength control are hidden for providers because the provider renders the whole frame.

Model record editor. External models gain an editor for the provider (a select over the configured providers), provider model id, supported modes, seed and reference support, image and reference limits, and default width/height/image count, saved through the model-records update contract with untouched capability tables preserved. The backend overlays the starter catalog's capabilities and defaults onto every model installed from it, so for those models the editor shows capabilities, limits and defaults read-only with a notice and saves only the provider ids, which do persist. Models added outside the starter catalog get the full editor.

Related Issues / Discussions

Follows up the legacy parity audit behind #9803.

QA Instructions

From invokeai/frontend/webv2 (Node 22.14, pnpm 10.12): pnpm check:release passes. No Python changes.

Mock backend (representative profile; three fixture models are external):

  1. Models → "Fixture GPT Image": the External Provider section shows provider (select), model id, modes, limits and defaults. Toggle "Accepts a seed", Save: toast, unsaved marker clears, and a reload shows the saved value. A model installed from the starter catalog shows the same values read-only with a "Managed by the starter catalog" notice; its provider ids stay editable.
  2. Editor → model picker → External → "Fixture Gemini Image": the Size section shows Provider size presets (pick "16:9 (2K)" → 2048×1152); Provider options show Temperature override and Thinking level. "Fixture GPT Image" shows Quality, Background and Input fidelity; "Fixture Seedream" shows the two toggles.
  3. Canvas with an external model: the Size section shows the frame size only and the Render section has no strength. Invoke and the queue cannot be exercised on the mock (no enqueue route); graph shape is covered by compile tests. A real provider key is needed to see output and recall end to end.

Review

Material findings resolved: the Canvas size badge disagreed with what external models received (now the bounding box, with the render-size slot hidden); starter-catalog models appeared to save capability edits that the backend's catalog overlay then reverted (those fields are now read-only for starter models; an attempted backend change was dropped because install-time copies of the catalog would have stopped receiving catalog updates); a vacuous provider-options browser assertion; a hand-typed size still submitting the previous named size. Also applied: validation reasons for control layers and regional guidance, a provider select instead of free text, fixtures aligned with provider capabilities.

Limitations: no real provider execution was performed; OpenAI input fidelity is offered only when the model takes image inputs (the backend uses it only on the edits endpoint); if the starter catalog cannot be loaded the editor falls back to the full form, where capability edits to a starter model would be reverted by the backend overlay (reopening the section retries the load).

Compatibility / Rollout

New generate settings are optional with defaults and normalize on load; no project-document migration. No backend or API changes.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…record editing

Add OpenAI/Gemini/Seedream options and provider named sizes to the Generate panel, compile them into the provider node and metadata, and recall them from images.
Let external models generate from the Canvas in the modes they advertise (txt2img, img2img, inpaint) and reject outpaint with a named reason.
Add an External Provider editor to the model detail for provider ids, modes, support flags, limits, and defaults; extend the mock backend with external fixtures and model PATCH.
… consistent

Starter overlays fill only the capability, default-settings, and panel-schema fields a record never set, so saved external model edits persist through responses and generation; the router and service share one helper.
Canvas asks a provider for the bounding box itself and hides render size and denoising strength for it; control layers and regional guidance are rejected by name.
A named provider size is sent only when the current dimensions match its preset; the provider id is a select over the registered providers, falling back to text while the list is unavailable.
…of overriding the catalog

The backend keeps overlaying the starter catalog, so starter records save only their provider ids
and show capabilities, limits and defaults read-only; custom records keep the full editor.
Unit tests follow the pixel-exact external grid and the mode-specific canvas rejection.
Request counts and chunk sets are unchanged; the provider core joins the editor's generation
sources and the provider sections join the Generate widget's.
@github-actions github-actions Bot added frontend PRs that change frontend files docs PRs that change docs labels Oct 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs PRs that change docs frontend PRs that change frontend files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant