fix: preserve CLIP hidden states after first skipped forward

[wunianze666-netizen]

Summary

A fresh CLIP encoder first used with positive CLIP skip can keep returning incomplete hidden states after its skipped layers are restored. Subsequent SDXL conditioning can therefore select the wrong penultimate layer even though the final encoder output is unchanged.

Initialize Transformers' existing capture hooks on the complete, actual CLIP capture owner before temporarily removing layers. For CLIPTextModelWithProjection, this is the inner text_model, not the projection wrapper. The existing helper is idempotent; skip-zero handling and the context manager's layer restoration are unchanged.

Related Issues / Discussions

Closes #9722.

The dependency/layout update in #9723 is already in this branch's baseline; this fixes the separate first-forward capture lifetime problem reported by @lstein.

QA Instructions

Repository-relative command for the application-context checks:

python -m pytest -o addopts= -p no:cov -p xdist.plugin -n 0 -q tests/backend/test_clip_skip_first_forward.py tests/backend/test_clip_text_model_wrapper.py

Windows, Python 3.12.14, Torch 2.13.0+cpu, Transformers 5.10.4:

• On current upstream aa46003581e2c482e03e62116f68ef3c3a6f0233, replaying the exact unmodified apply_clip_skip method against the 26 new cases gives 16 failed / 10 passed. This is an isolated owning-method replay with real Torch/Transformers, not a full-application baseline run.
• Applying this unchanged patch to that same current upstream and importing the real production module with the project's root application fixtures gives 34 passed / 4 existing deprecation warnings (26 new cases plus 8 existing CLIP-wrapper/LoRA/SDNQ cases).
• The submitted b47066e8c6 candidate was also validated with the new file alone in the application context: 26 passed. Earlier isolated compatibility checks on Transformers 5.5.4 gave 16 failed / 10 passed before and 26 passed after; these do not claim full-application backward compatibility.
• Ruff 0.11.2 check/format and git diff --check pass. Only the production patcher and the new regression file are submitted.

The new tests use small, real, random CPU float32 encoders and independent weight-matched references. They compare every hidden state, final and pooled/projected outputs, layer identity/order, fresh/warmed/repeated skip contexts, and recovery from both an error inside the forward and an error after it. No model download is required.

Not run: accelerator/compiled inference, full SDXL generation, concurrent generation, complete repository test/type/build checks. No speedup or full-platform claim is made.

Review

Initializing the outer projection wrapper after an earlier forward can double-record hidden states. The warmed-model numerical controls exposed this during development; the final implementation initializes only the inner capture owner. Exceptions retain the existing finally restoration behavior.

The alternative of warming the model first requires an extra inference. This patch instead reuses the pinned dependency's capture installation helper without changing output selection or temporarily replacing the forward implementation.

Implementation, diagnosis and regression design were AI-assisted. The reported commands and results were executed; internal checks are not maintainer approval.

Compatibility / Rollout

No API, serialized model or database migration. The helper is an internal Transformers API, validated for the currently pinned >=5.10.4,<5.11 range and separately checked at 5.5.4. Future dependency upgrades require revalidation.

The fix prevents corruption on a fresh encoder's first skipped forward; it does not repair an encoder already poisoned in a running process. Such instances must be reloaded/restarted.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation (no persisted-state or API change)
• Relevant performance/efficiency opportunities considered; material claims have evidence (no additional warm-up inference; no measured speedup claim)
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable) (not applicable)
• Updated What's New copy (if doing a release after this PR) (not a release PR)

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}