Skip to content
This repository was archived by the owner on Oct 4, 2026. It is now read-only.

feat(ltx2): choose LTX-2.5 clip length with the duration head - #367

Merged
lstein merged 3 commits into
mainfrom
feat/ltx2-duration
Sep 29, 2026
Merged

lstein merged 3 commits into
mainfrom
feat/ltx2-duration

Conversation

@lstein

@lstein lstein commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

LTX-2.5 ships with a small duration head that reads the encoded prompt and predicts how long the shot it describes should run. This lets a generation choose its own length.

  • Model: new model type ltx2_duration_head, with an optional 3.6 MB starter, LTX-2.5 Duration Head.
    • The head exists only in Lightricks' license-gated Lightricks/LTX-2.5-Diffusers repo, not in the ungated mirror the other LTX-2.5 starters use. It is therefore its own record rather than a role in the components folder, and the starter description says it needs a token whose account has accepted the license.
    • Identification requires the exact 19-tensor set at the widths the loader builds.
    • The loader reads it through the guarded single-file reader (quantized builds are refused) and widens it to fp32.
  • Node: new ltx2_duration node. It reads the positive prompt's post-connector embeddings, as upstream's pipeline does. It calls upstream predict_num_frames to clamp to the given bounds and snap to the 8k+1 grid. Equal or inverted bounds are refused.
  • Video panel (webv2):
    • an optional Duration head slot, auto-selected when a head is installed;
    • an Auto duration switch above Frames, off by default.
  • With Auto duration on, Frames stays editable and becomes Max frames: the ceiling the head chooses beneath. That number sizes the run's memory, so the head can never pick a clip larger than the one the user planned for.
    • The help text states the reachable range, e.g. "up to 121 frames (5.0 s)".
    • A ceiling too short to leave a choice skips the head, and the clip runs at that length.
    • It is off for audio-to-video, video-to-audio and extend, where the source clip decides the length.
  • Graph: the head's num_frames feeds the base pass, the two-stage refine pass and the recorded metadata. Last-frame keyframes use a tail-relative index, so they follow automatically.
  • Recall restores the length that ran and turns Auto duration off, so the recalled clip is reproduced exactly. Removing the head, by uninstalling it, switching model family or clearing the slot, turns Auto duration off rather than letting it come back on later.
  • Docs: an "Auto duration" section and an install-list entry in the video generation page.

Related Issues / Discussions

PR 8 of the LTX-2.5 integration plan: automatic duration, split from the enhancer work in #352.

QA Instructions

Automated:

Check Command Result
Backend tests uv run --no-sync pytest tests/app/invocations tests/backend/model_manager/configs tests/backend/model_manager/load/test_ltx2_loader.py tests/backend/model_manager/test_ltx2_identification.py tests/backend/model_manager/test_starter_models.py tests/backend/model_manager/test_starter_models_package.py tests/app/services/model_install 2916 passed
Unit tests pnpm -C invokeai/frontend/webv2 test 8972 passed
Lint pnpm -C invokeai/frontend/webv2 lint clean
Browser tests pnpm -C invokeai/frontend/webv2 test:browser src/features/video 37 passed
Performance gate pnpm -C invokeai/frontend/webv2 test:performance:architecture passed, before the final merge from main
Legacy type check lint:tsc clean

openapi.json and schema.ts were regenerated after merging main, with no drift.

New tests:

  • a loader round trip on a random-init head saved as bf16: fp32 load, output equal to a reference, predict_num_frames called with the node's keywords;
  • a float8 head refused;
  • identification: exact key set and widths;
  • node: clamp, a non-default floor, equal and inverted bounds, one prompt only;
  • graph: the head feeds denoise, refine and metadata; fps and max_seconds equal the Frames ceiling; the head is skipped for a short ceiling and in clip modes;
  • bounds arithmetic;
  • recall, from an LTX-2 panel with a head and Auto on: Auto is turned off and the length restored;
  • healing on uninstall and on a family switch; auto-selection; the comparator;
  • a browser test of the new VideoLengthControls: switch presence and label, Frames staying editable as "Max frames" with its range, and the disabled state in clip modes.

Real file (CPU): the gated head identifies as ltx2_duration_head and loads strictly. bf16 tokens into the fp32 head give identical output, because upstream casts them.

Live app (2x AMD W7900, int8 Dev transformer, 24 fps):

Prompt Max frames Predicted Frames
"a cat jumps onto a table" 121 4.45 s 105
drone flight over mountains 241 4.18 s 97
dog chases a cat up a tree 241 5.22 s 121
long enhanced prompt 241 10.54 s 241 (clamped to the 10.04 s ceiling)
  • Recall restored the exact length and turned Auto duration off.
  • Two-stage: the final clip ran at the head's length, with no mismatch at the refine pass.
  • Last frame: the keyframe landed at the end of the chosen length.

Review

Material findings resolved:

  • Unbounded length: the head could pick up to the family maximum (481 frames), several times the memory the panel was sized for, with no warning. Frames is now the ceiling, and the panel states the range.
  • Recall: a recalled length was silently overridden while Auto was on. Recall now turns Auto off.
  • Auto re-enabling: Auto duration could switch itself back on after the head was removed and later re-added. It is now healed wherever the head is cleared, and the comparator and persisted-settings guard cover both fields.
  • Loader and identification: the loader skipped the quantization guards, and identification ignored tensor widths.
  • Tests that could not fail: no real-loader test; parameter tests at upstream's defaults; bounds assertions that always held; a recall test masked by model selection. Each now fails under the corresponding mutation.

Remaining limitations:

  • The head's trained 1–20 s range is defined in both Python and TypeScript, linked by comments, with no test that they match.
  • The loader builds the head with diffusers' default configuration. The installed file carries no config.json, and the pooler's head count does not show in any tensor shape.
  • The duration node itself has no frame ceiling; webv2 bounds it. A hand-built workflow can ask for up to 20 s at up to 120 fps, which ltx2_denoise does not cap either.

Compatibility / Rollout

  • New model type ltx2_duration_head: legacy web's enums and tables are updated, and the generated openapi.json and schema.ts are regenerated.
  • Saved Video panel state without the new keys loads with no head and Auto duration off.
  • The starter is license-gated. Installing it without an accepted license fails with the usual 401/403 install error, which the starter description anticipates.
  • No database migrations.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

Adds the LTX-2.5 duration head as a model type (gated starter), an ltx2_duration node, and an
Auto duration switch in the Video panel where Frames becomes the ceiling the head chooses beneath.
Recall restores the realised length and turns Auto off; the head's num_frames feeds both passes.
@lstein
lstein merged commit 05a814d into main Sep 29, 2026
16 checks passed
@lstein
lstein deleted the feat/ltx2-duration branch September 29, 2026 02:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant