fix(rack-controller): recover power-blocked firmware updates [backport 2.1]

[kunzhao-nv]

Backports #5031 to release/v2.1.

Rack firmware maintenance can leave a rack permanently stuck in Maintenance(FirmwareUpgrade(WaitForComplete)) when a scoped machine is Ready with desired_power_state == Off. The rack writes host_reprovisioning_requested, but the machine power-manager gate prevents the request from being consumed, so the rack controller waits forever and later maintenance requests remain blocked.

This backport rejects that condition before RMS submission and recovers racks already stuck by it by marking the firmware job failed, clearing maintenance_requested and credentials, conditionally clearing unconsumed rack-owned host requests, and transitioning the rack to Error. It also includes the accepted review follow-ups from #5031: one canonical rack-maintenance initiator formatter, non-panicking persisted-job handling, and removal of a shadowed binding. Pending device IDs are included in wait outcomes for diagnosis.

Related issues

• Backport of fix(rack-controller): recover power-blocked firmware updates #5031
• [nico/state-control] Rack fw update deadlocks when a target tray is powered off | No state SLA

Type of Change

• Add - New feature or capability
• Change - Changes in existing functionality
• Fix - Bug fixes
• Remove - Removed features or deprecated functionality
• Internal - Internal changes (refactoring, tests, docs, etc.)

Breaking Changes

• This PR contains breaking changes

Testing

• Unit tests added/updated
• Integration tests added/updated
• Manual testing performed

Additional Notes

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ac678c26-ce9e-445b-bbdd-68ef898a6f67

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔐 TruffleHog Secret Scan

✅ No secrets or credentials found!

Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉

🔗 View scan details

_{🕐 Last updated: 2026-08-17 19:14:37 UTC | Commit: 551f738}

[nv-dmendoza]

LGTM, Backports needed fix