Skip to content

feat(dispatch): guard task start/assign against routing dev work onto a review slot - #575

Merged
jlong merged 1 commit into
mainfrom
jlong/dispatch-guard-refuse-assigning-a-normal-task-to-a-review-tagged-workspace-task-start-assign-with-force-escape
Aug 17, 2026
Merged

feat(dispatch): guard task start/assign against routing dev work onto a review slot#575
jlong merged 1 commit into
mainfrom
jlong/dispatch-guard-refuse-assigning-a-normal-task-to-a-review-tagged-workspace-task-start-assign-with-force-escape

Conversation

@jlong

@jlong jlong commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Task

An orchestrator dispatched a normal dev task onto a review-tagged workspace. Review slots exist only to load a handed-off branch and serve it for a human; they should never be selected for normal dev work. Today nothing enforces this — shelbi task start / task assign don't validate workspace tags, so any orchestrator (or human) can route dev work onto a review slot. The orchestrator instructions have been updated (live + this task updates the shipped default) to exclude review slots from auto-dispatch, but instructions are soft; this task adds the hard CLI backstop.

Design decision (user, 2026-08-16): reject by default, with a --force escape hatch (logged).

Current Behavior

shelbi task start <task> --workspace <review-slot> and shelbi task assign <task> --to <review-slot> succeed with no tag check, placing a dev task on a review workspace.

Expected Behavior

  • task start / task assign targeting a workspace whose tags include review is rejected with a clear error, e.g.: workspace 'review' is a review slot (tagged 'review') — review tasks load via the review queue, not direct dispatch; pick a non-review workspace (or pass --force to override).
  • A new --force flag on both task start and task assign overrides the guard for deliberate human use. When used, the override is recorded on ~/.shelbi/events.log (auditable) — e.g. a workspace=<ws> review-slot-override task=<id> reason=user:force line.
  • The legitimate review-load path is unaffected: review slots are filled by the poller's maybe_autoload_review_queueshelbi_orchestrator::load::autoload_review_queue, which does NOT go through these CLI handlers, so it needs no change and no --force.

Technical Details

  • Guard lives in crates/shelbi-cli/src/commands/task.rs in the start and assign handlers. Detect a review slot with the existing helper project.effective_tags(&workspace).contains("review") (tags are the union of workspace + machine tags; review is the canonical marker — see config_upgrade_apply which migrates legacy role: reviewtags: [review], and poller.rs:832 which gates review handling on effective_tags(...).contains("review")).
  • Add --force (bool) to the TaskCmd::Start and TaskCmd::Assign variants; thread it to the guard.
  • Emit the override event through the same events.log path other task events use, so /activity and the orchestrator can see it.
  • Shipped default orchestrator instructions template (crates/shelbi-state/src/default_orchestrator.md.template): mirror the live edit to the "Free-workspace selection" section — exclude review-tagged workspaces from the dev auto-dispatch pool (an idle review slot is not available dev capacity). This reaches NEW projects.
  • Config-upgrade sniffer (NeedsJudgment) in crates/shelbi-cli/src/commands/config_upgrade.rs over the orchestrator instructions surface (project.<name>.agent.orchestrator.instructions): flag instructions whose "Free-workspace selection" section is present but lacks the review-slot exclusion clause (an absence sniff — header present, exclusion phrase like review-tagged / review slot absent), handing the orchestrator a boot finding to refresh its own copy. Existing projects self-heal. (Per the AGENTS.md "Changing shipped defaults" guardrail — a default change needs a sniffer to reach existing installs.) Include a drift-guard test that the shipped default template does NOT trip this sniffer.

Acceptance Criteria

  • task start / task assign targeting a review-tagged workspace is rejected (non-zero exit) with a message naming the tag and pointing to the review queue + the --force option.
  • --force on task start / task assign performs the assignment and writes an auditable override line to ~/.shelbi/events.log.
  • Assigning to a non-review workspace is unchanged.
  • The autoload review-load path still places a review-routed task onto a review slot without --force (test asserts it's unaffected).
  • Shipped default orchestrator instructions template excludes review-tagged workspaces from the free-workspace dev-dispatch pool.
  • A project whose orchestrator instructions lack the review-exclusion clause produces a NeedsJudgment config-upgrade finding on boot; the shipped default does not trip the sniffer.
  • cargo build/test/clippy green; unit tests cover the reject, the --force override, and the non-review pass-through.

Auto-opened by Shelbi — review at: /Users/jlong/.shelbi/projects/shelbi/tasks/dispatch-guard-refuse-assigning-a-normal-task-to-a-review-tagged-workspace-task-start-assign-with-force-escape.md

… a review slot

Review slots exist only to load a handed-off branch and serve it for a
human (filled by the poller's review autoloader, never by direct
dispatch). Nothing enforced this: `task start` / `task assign` accepted
any workspace, so an orchestrator or human could route a normal dev task
onto a review slot. The orchestrator instructions already exclude review
slots from auto-dispatch, but that's soft — this adds the hard CLI
backstop.

Decisions (per user, 2026-08-16):
- Reject by default, with a logged `--force` escape hatch. Both `task
  start` and `task assign` gain `--force`; without it a `review`-tagged
  target is rejected with a message naming the tag and pointing at the
  review queue + `--force`. With it, the override is recorded on
  events.log via a new `review-slot-override task=<id> workspace=<ws>
  reason=user:force` line (best-effort log; a logging failure warns
  rather than blocking the deliberate human action).
- Detect a review slot with `effective_tags(ws).contains("review")` (the
  canonical marker), matching the poller. The guard sits in the CLI
  handlers only; the autoload path (`autoload_review_queue`) never calls
  them, so the legitimate review-load path is unaffected (test added in
  load.rs asserting it still plans a review task onto a review slot with
  no `--force`).
- Mirror the live orchestrator-instructions edit into the shipped
  default template so NEW projects exclude review slots from the
  dev-dispatch pool.
- Add a config-upgrade `NeedsJudgment` sniffer over
  `project.<name>.agent.orchestrator.instructions`: an absence sniff that
  flags a "Free-workspace selection" section present but silent on review
  slots, so EXISTING projects get a boot finding to self-heal (per the
  AGENTS.md "changing shipped defaults needs a sniffer" guardrail).
  Included a drift-guard test that the shipped template does not trip it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
shelbi Ready Ready Preview Aug 17, 2026 12:45am

Request Review

@jlong
jlong merged commit 697135e into main Aug 17, 2026
3 checks passed
@jlong
jlong deleted the jlong/dispatch-guard-refuse-assigning-a-normal-task-to-a-review-tagged-workspace-task-start-assign-with-force-escape branch August 17, 2026 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant