Skip to content

fix(orchestration): quarantine completed workers without an explicit role lease - #9946

Closed
connfy wants to merge 2 commits into
stablyai:mainfrom
connfy:fix/orch-r15-role-lease
Closed

fix(orchestration): quarantine completed workers without an explicit role lease#9946
connfy wants to merge 2 commits into
stablyai:mainfrom
connfy:fix/orch-r15-role-lease

Conversation

@connfy

@connfy connfy commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • Add an authoritative ORCH-R15 role guard: active dispatched workers stay worker-scoped; after an accepted worker_done the pane is post-completion quarantined and cannot run coordinator lifecycle/task-control ops (task-create/update, dispatch, reset, run/run-stop, gate-*, coordinator-shaped sends, self role-lease grant).
  • Coordinator promotion requires a durable orchestration role-lease --role coordinator ceremony (or never having been a worker / fresh redispatch). Ordinary owner chat is not promotion; denial is fail-closed with no mutation.
  • Bind post-dispatch RPC identity claims to the runtime's live pane map, preserve quarantine evidence through reset, and consume coordinator leases on fresh redispatch so stale grants cannot reactivate.
  • Pane-leaf identity follows reminted handles; legitimate worker_done/heartbeat/escalation/ask paths, empty-DB bootstrap, and dry-run dispatch remain available where appropriate.

Integration with open PRs

Test plan

  • pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/orchestration/role-lease.test.ts src/main/runtime/rpc/methods/orchestration-role-lease.test.ts src/main/runtime/orchestration/preamble.test.ts src/main/runtime/rpc/methods/orchestration.test.ts src/main/runtime/orchestration/db.test.ts src/main/runtime/orchestration/lifecycle-reconciliation.test.ts src/cli/handlers/orchestration.test.ts src/main/runtime/orchestration/db-empty-dispatch-shortcircuit.benchmark.test.ts (276 passed)
  • pnpm run typecheck:node
  • pnpm run typecheck:cli
  • pnpm exec oxlint --quiet
  • Executable harness: completed worker task-create / gate-create / coordinator-shaped send are denied and row counts remain unchanged.
  • Executable harness: coordinator role-lease --role coordinator persists across DB reopen and allows the promoted pane to create work.
  • Executable harness: redispatch consumes the coordinator lease, restores worker ask/decision-gate scope, and keeps task-create denied.

Full pnpm run lint reaches a pre-existing unrelated switch-exhaustiveness-check failure in src/renderer/src/components/skills/skill-freshness-group.tsx; the branch's own switch error was fixed, and oxlint --quiet plus the max-lines ratchet pass.

Made with Cursor

…role lease

Prevent post-worker_done panes from executing coordinator control ops or
self-promoting from ordinary user chat; require a durable role-lease ceremony
or fresh redispatch (ORCH-R15).

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds durable coordinator role leases and authority resolution based on dispatch, handle, and pane identity. Orchestration RPC methods now validate coordinator control and accept caller identity metadata. CLI commands propagate terminal and pane identity, with a new role-lease command. Database migrations and reset paths include lease records. Tests cover worker permissions, quarantine, redispatch, explicit handoff, RPC guards, and preamble role-boundary guidance.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 8.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description covers summary, integration, and testing, but it omits required Screenshots, AI Review Report, Security Audit, and Notes sections. Add the missing template sections, including screenshots/no visual change, AI review report, security audit, and notes.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: quarantining completed workers unless they have an explicit coordinator role lease.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
src/main/runtime/rpc/methods/orchestration-role-lease.test.ts (1)

168-183: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Test claims heartbeat coverage but doesn't exercise it.

The test title says "worker_done / heartbeat sends" but only sends a worker_done message; no heartbeat-type send is issued. Since the PR objectives explicitly call out heartbeat as a path that "remain[s] available," add an assertion (or a second test) that actually sends type: 'heartbeat' and confirms it isn't blocked by the role-lease guard.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: fc73e470-8021-49ed-8b79-f4dfb6723dbb

📥 Commits

Reviewing files that changed from the base of the PR and between 334027c and 1f41da4.

⛔ Files ignored due to path filters (1)
  • src/main/runtime/orchestration/__snapshots__/preamble.test.ts.snap is excluded by !**/*.snap
📒 Files selected for processing (13)
  • src/cli/handlers/orchestration.ts
  • src/cli/specs/orchestration.ts
  • src/main/runtime/orchestration/db.ts
  • src/main/runtime/orchestration/preamble.test.ts
  • src/main/runtime/orchestration/preamble.ts
  • src/main/runtime/orchestration/role-lease.test.ts
  • src/main/runtime/orchestration/role-lease.ts
  • src/main/runtime/orchestration/types.ts
  • src/main/runtime/rpc/methods/orchestration-gates.ts
  • src/main/runtime/rpc/methods/orchestration-role-lease.test.ts
  • src/main/runtime/rpc/methods/orchestration-role-lease.ts
  • src/main/runtime/rpc/methods/orchestration.test.ts
  • src/main/runtime/rpc/methods/orchestration.ts

Comment on lines +184 to +194
// Why: ORCH-R15 role checks must see the live pane, not a spoofable --from alone.
async function resolveCallerRoleIdentity(
client: Parameters<CommandHandler>[0]['client']
): Promise<{ callerTerminalHandle?: string; callerPaneKey?: string }> {
const callerTerminalHandle = await resolveTaskCreatorTerminalHandle(client)
const envPane = process.env.ORCA_PANE_KEY
return {
callerTerminalHandle,
callerPaneKey: envPane && envPane.length > 0 ? envPane : undefined
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Fail closed when live caller identity is unavailable.

resolveCallerRoleIdentity() can return no callerTerminalHandle, while the RPC contract falls back to callerTerminalHandle ?? from. Consequently, Line 786 and other --from-accepting commands can let a user-controlled handle become the authorization identity, defeating the “not spoofable by --from alone” guarantee. Reject identity-less protected calls, or prevent from from being used as the caller identity.

Also applies to: 786-786

Comment thread src/cli/handlers/orchestration.ts Outdated
Comment on lines +57 to +64
function isEquivalentPaneKey(a: string, b: string): boolean {
if (a === b) {
return true
}
const aLeaf = parsePaneKey(a)?.leafId
const bLeaf = parsePaneKey(b)?.leafId
return Boolean(aLeaf && bLeaf && aLeaf === bLeaf)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Duplicate isEquivalentPaneKey — extract to shared module.

Identical logic already exists as a private helper in db.ts (lines 22-29). Both files already import parsePaneKey from ../../../shared/stable-pane-id, so this is a good candidate to move there once and import in both places — avoids two independently-maintained copies of identity-matching logic that authority resolution and dispatch lookups both depend on.

♻️ Proposed fix
-function isEquivalentPaneKey(a: string, b: string): boolean {
-  if (a === b) {
-    return true
-  }
-  const aLeaf = parsePaneKey(a)?.leafId
-  const bLeaf = parsePaneKey(b)?.leafId
-  return Boolean(aLeaf && bLeaf && aLeaf === bLeaf)
-}
+import { isEquivalentPaneKey } from '../../../shared/stable-pane-id'

And remove the duplicate private copy from db.ts, importing it there too.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 7350682a-4ae1-4684-90c3-44fee6d43d6e

📥 Commits

Reviewing files that changed from the base of the PR and between 1f41da4 and 2c0f34b.

📒 Files selected for processing (10)
  • src/cli/handlers/orchestration.test.ts
  • src/cli/handlers/orchestration.ts
  • src/main/runtime/orchestration/db-empty-dispatch-shortcircuit.benchmark.test.ts
  • src/main/runtime/orchestration/db.test.ts
  • src/main/runtime/orchestration/db.ts
  • src/main/runtime/orchestration/role-lease.test.ts
  • src/main/runtime/orchestration/role-lease.ts
  • src/main/runtime/rpc/methods/orchestration-role-lease.test.ts
  • src/main/runtime/rpc/methods/orchestration-role-lease.ts
  • src/main/runtime/rpc/methods/orchestration.test.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • src/main/runtime/rpc/methods/orchestration-role-lease.ts
  • src/main/runtime/orchestration/role-lease.ts
  • src/main/runtime/rpc/methods/orchestration-role-lease.test.ts
  • src/main/runtime/orchestration/db.ts

Comment on lines +272 to +285
const second = dispatchWorker(d, {
handle: 'term_worker',
paneKey: 'tab_w:leaf_w',
spec: 'redispatched work'
})
d.updateTaskStatus(second.task.id, 'completed')
d.completeDispatch(second.ctx.id)

expect(
d.findActiveCoordinatorLease({
handle: 'term_worker',
paneKey: 'tab_w:leaf_w'
})
).toBeUndefined()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Assert lease consumption before completing the redispatch.

The current assertion runs after completeDispatch(second.ctx.id), so it cannot prove that redispatch revoked the lease. Assert immediately after dispatchWorker, then retain the quarantine assertion after completion.

Proposed test adjustment
     const second = dispatchWorker(d, {
       handle: 'term_worker',
       paneKey: 'tab_w:leaf_w',
       spec: 'redispatched work'
     })
-    d.updateTaskStatus(second.task.id, 'completed')
-    d.completeDispatch(second.ctx.id)
-
     expect(
       d.findActiveCoordinatorLease({
         handle: 'term_worker',
         paneKey: 'tab_w:leaf_w'
       })
     ).toBeUndefined()
+    expect(
+      resolveOrchestrationAuthority(d, {
+        handle: 'term_worker',
+        paneKey: 'tab_w:leaf_w'
+      }).role
+    ).toBe('worker')
+
+    d.updateTaskStatus(second.task.id, 'completed')
+    d.completeDispatch(second.ctx.id)

@connfy

connfy commented Jul 22, 2026

Copy link
Copy Markdown
Author

Withdrawn at owner direction. The motivating event was an operator mistaking a completed worker terminal for the coordinator, not a product defect that should constrain Orca role changes. Intentional worker-to-coordinator promotion is a legitimate workflow, so this role-lease/quarantine design and all follow-up hardening are being fully withdrawn without merge.

@connfy connfy closed this Jul 22, 2026
@connfy
connfy deleted the fix/orch-r15-role-lease branch July 22, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant