Skip to content

Latest commit

 

History

History
463 lines (385 loc) · 26.7 KB

node-maintenance-operator.md

File metadata and controls

463 lines (385 loc) · 26.7 KB

Maintenance service: checks and recovery

This reference is for ongoing maintenance of the shared GitHub App and Cloudflare service. Repository maintainers install the shared App; they do not host it.

The receiver verifies signed webhook deliveries and reads canonical repository metadata with a token restricted to one repository. For managed PRs it also reads the canonical PR and displays the selected package versions in the App's Dependency repair Check. A maintainer can select fixes in the PR description and tick Run selected fixes, or post /repair selected in the Conversation tab. The App replies there with the request status and exact Actions link. The native Check action remains an optional secondary entry point. D1 retains the active request and returned Actions run identity. Completed runs update that same reply and release their own request lock. The supported caller invokes preparation, separate validation and publication. The App reports workflow status and the exact run URL. The PR's published report and native diff establish which fixes actually landed.

Installation release gate

Repository users follow the main setup guide. The existing shared App is Security Maintenance by CLI.dev. Consumers copy node/repair-caller.yml to the fixed default-branch path .github/workflows/dependency-repair.yml. Keep its five inputs unchanged and use the same reviewed full workflow revision as the coordinated audit caller. Consumers need no service database or App private key.

The fresh-consumer receipt records successful installation acceptance for workflow source 42fe6d163e7f3152b8ef55ed3b0a842a999564a9 and Worker version 309baf0d-f485-4e69-bb9b-cf2bba604ee1. It covers the existing organization-wide App installation, an isolated private consumer, and the real Conversation flow. It does not establish a new installation permission grant or Hydra acceptance.

Before announcing a later installation revision:

  1. Review and test the source. Before deploying this description-trigger revision, obtain Issues-write installation consent and apply migration 0002 after 0001 through the service owner's authorized process. Deploy the exact reviewed Worker before releasing workflow reports that offer the new trigger. A green caller PR alone authorizes none of these live actions.
  2. Follow the user guide in a fresh isolated repository. Install the App and both callers, run an audit, and submit one eligible /repair selected request. Verify actual preparation, separate validation, bounded publication, truthful reply, original advisories and native evidence. A demo receipt does not prove a fresh installation works.
  3. Record actual source/deployment revisions, App/installation/repository IDs, caller hashes, request/Check/run/artifact IDs, PR URLs, outcomes and cleanup. Record the consumer receipt after observation, never from planned values. Store no token, private key or webhook body.
  4. Keep stable-tag changes and any Hydra pilot behind separate explicit authority. Fresh-consumer success does not authorize either action.

Check the selected batch

The PR must be open, bot-authored, from maintenance/node-security in the same repository, and target its current default branch. Its description contains one selection block:

<!-- node-repair:selection:v1 -->
- [x] Allow `launch-editor` `2.14.1 → 2.14.2` by updating its explicit pin.
<!-- /node-repair:selection -->

Select at most 20 distinct packages. Each change must increase the patch version within the same major and minor. Unchecked rows must also be valid and unique. Malformed or missing selections produce a Check that asks for attention.

Editing selections or changing the PR head refreshes the Check from canonical GitHub data. The Check displays the exact selected versions, bound to the PR number, head and base. With D1 configured, a nonempty valid batch shows Run selected fixes as a native Check action. Package-selection edits only refresh that Check; the separate description run toggle submits a request. Refresh tokens grant Contents read, Pull requests read and Checks write. Submission tokens also grant Actions write; conversation submissions need Pull requests write and Issues read, or Issues write for comment receipts. All tokens are restricted to the same repository. The service never updates another App's Check.

Request the selected batch from the PR conversation

The report includes the head and base revisions in one hidden line outside the selection block (replace these illustrative hashes with the exact PR revisions):

<!-- node-repair:revision:v1 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb -->

Tick the allowed package upgrades in the PR description, then tick the separate Run selected fixes checkbox. Package selections alone start no work. The App replies with the accepted batch count, an active-request indicator and the final verified workflow result. Nothing merges automatically.

The App does not rewrite the description to reset the run checkbox or update a counter: GitHub has no conditional body update that protects concurrent checkbox edits. After completion, untick and re-tick the run checkbox to submit another request if eligible upgrades remain. An active request prevents another dispatch.

Alternatively, post a new comment containing only /repair selected using GitHub's Comment button. A 👍 on that comment means the App received it; the reply determines whether it was accepted. The reaction is best effort and cannot block a valid repair. Existing native Check actions remain supported.

The receiver accepts only a newly created human comment on a managed PR. It re-fetches the comment and verifies its author, exact body, PR association and unchanged creation/update timestamps. It checks current maintainer permission, the selections captured in the signed comment event, the report's head/base binding, and the current canonical PR/Check fingerprint. A stale report or selection starts no run and produces a reply asking for a fresh command. Unrelated comments and edits are ignored. Missing or edited/deleted commands and unmanaged PRs do not authorize an attempt.

The description trigger accepts only the exact unchecked-to-checked run-control edit by a current maintainer. It verifies the signed PR head/base, exact body, edit timestamp, revision marker and canonical selection Check. A retained edit fingerprint deduplicates the same signed edit even with a new delivery ID and after completion. No App status path edits the PR description or audit evidence.

The shared renderer includes the revision line in bound reports. Existing reports without it cannot authorize comment dispatch; rerun the security audit.

Existing Check action

Review the exact versions in the Check summary, then click Run selected fixes. The service reads current write, maintain or admin permission and verifies the canonical PR and Check again. A stale selection or head refreshes the Check and requires another click; it starts no run.

Shared admission and recovery

The request UUID is the signed GitHub delivery ID. A single D1 insert reserves one active request per installation, repository and PR before dispatch. Replayed deliveries and competing clicks cannot admit another active request. The Check loses its action and retains the request identity across PR edits. A new head can have its own selection Check, but the existing PR lock still blocks dispatch.

Only .github/workflows/dependency-repair.yml on the current default branch is dispatched, with the five C2 inputs. The API must return HTTP 200 with the exact run ID and matching repository URLs. The known run link is saved in D1 and shown in the Check. The supported caller invokes the real repair workflow. Workflow success alone does not identify published dependency changes; inspect its publication result, PR report and native branch history.

Comment requests additionally retain the source comment ID, reply ID and exact selections. A unique source-comment index prevents the same comment from authorizing a second attempt, even under a different delivery UUID. Only the winning reservation creates a status reply. Its ID is saved before dispatch; a lost reply POST or failed save retains the lock and starts no run. Replays, losing admissions and PR refreshes do not repaint request output or retry an ambiguous reply POST. Exposed idle Checks are reused without PATCHing them, or a fresh idle Check is created. Admission still validates current selections and revisions; an older idle control cannot authorize a stale batch.

The control Check stays completed/neutral while its text changes. Real GitHub did not reopen the previously completed Check when asked for in_progress. The pending text stays Request accepted, with a workflow-page link, until verified completion. No request-status writes occur after dispatch except completion updates using the immutable verified outcome. An uncertain pending write prevents dispatch and retains the reservation for operator recovery. The reply says Workflow succeeded, Workflow failed or Workflow cancelled and retains the exact run link and selected versions. The App never rewrites the dependency report or infers an upgrade from workflow success. Only the trusted publisher updates that evidence after actual validation and publication.

Existing service setup and migration

These steps are for service operators whose installation predates comment delivery. They are not extra setup for repository users:

  1. Set Issues and Pull requests to read/write on the App. Subscribe to Issue comment events. Existing installations must accept the new permissions. Keep Contents read, Checks write and Actions write.
  2. For an existing slice-3 database, apply repair-app/migrations/0001_comment_requests.sql exactly once. It preserves all existing request rows and active locks. New databases use schema.sql instead; do not apply both initial schema and migration to one database. Before deploying description-trigger support, apply repair-app/migrations/0002_description_requests.sql once after migration
    1. It adds the nullable edit fingerprint and unique index without changing existing request rows or locks. Every request path needs the updated schema. Before deploying direct selection-control lookups, apply repair-app/migrations/0003_lookup_indexes.sql. It adds request lookup and cleanup indexes and a replaceable control locator for each installed repository. Existing requests and replay identities remain unchanged. The first refresh discovers existing Checks; later refreshes read the saved Check directly and verify its current GitHub identity. A changed selection creates a new control without scanning old Checks. Exposed Checks are never repainted. This migration does not alter the 90-day request retention policy.
  3. Deploy the reviewed Worker. Put the correct revision marker and selections in the pilot description, then verify a real command, exact inputs, reply link, replay and stale-command rejection in the isolated repository.

No live permission, database, deployment or repository change is performed by these instructions. The local/native tests cannot replace that acceptance.

Candidate preparation boundary

The reusable .github/workflows/node-repair.yml enables guard, prepare and validate. It accepts the existing five request inputs and returns candidate-artifact-id for the immutable artifact created in that run. It does not install a caller or enable repairs in an existing repository.

The read-only guard fetches the canonical repository and managed PR again. It checks bot authorship, the same repository, maintenance branch, default branch, exact head/base, revision marker and selected versions. It does not require the old idle Check identity after the App has admitted the request.

Preparation checks out only the expected head without persisted credentials. It uses the configured toolchain, existing audit engine, release cooldown and parent/direct constraint evidence. Lifecycle and local/global pnpmfile hooks are disabled during metadata reads, audit and lockfile-only resolution. New unselected versions, unrelated lockfile metadata and changed peer suffixes fail preparation. Obsolete transitive versions may be removed only when the candidate has no remaining dependency or peer reference to them. Remaining unselected entries and edges must stay unchanged. No project validation script runs in this job.

The preparation artifact contains exactly three regular files:

  • candidate.patch: only the selected project's pnpm-workspace.yaml and pnpm-lock.yaml changes, exported with external diff and text conversion off.
  • audit.json: a version-1 envelope containing the exact request, normalized project directory, structured before audit and candidate audit.
  • result.md: the shared package report. It states that project validation and dependency publication have not run.

The envelope rejects missing/extra fields, mismatched request or project, unsupported versions, symlinked project parents, unconfirmed selected fixes and new findings. Later consumers must download by the returned artifact ID from the same run. A preparation success is not proof of a validated or published repair.

Validation checks out the same expected head on a separate read-only runner, without persisted Git credentials or App credentials. It downloads only the preparation artifact ID from the current run. It rejects extra members and unapproved patch paths or file modes before applying the candidate. Frozen installation and the configured validationScript run against that candidate; include any project-specific local-patch tests in this script. Changes to dependency files or configuration fail validation. A fresh audit must confirm all selected advisories are removed and no new finding is present.

The separate node-repair-validation-<request-id> artifact contains human evidence only: install/check logs, the fresh audit and a validation result. It never replaces the preparation artifact. Project code runs with the repository write maintainer as its trust principal; validation cannot attest intentionally dishonest tests. The publisher uses the original preparation artifact and actual validation job success, never validation-local patches, reports or logs. Normal-audit retention and the full isolated live repair gate remain required before general rollout.

Check the receiver locally

Use Node 24 and run these commands from repair-app/:

npm ci --ignore-scripts --no-fund
npm test
npm run test:runtime
npm run check

The unit suite reports coverage for the receiver, selection parser and Check updates, action authorization, fixed dispatch and request-state failures. selection-fixtures.json holds the shared C1 cases and expected fingerprints. The runtime test executes the receiver in local Cloudflare Workers with synthetic credentials and intercepted GitHub requests and an actual local D1 binding for concurrency and replay checks. The final command bundles the preview with --dry-run; it does not deploy. The CI job runs the same checks without App or Cloudflare credentials.

Stop deliveries or recover

Disable the App webhook or uninstall the affected installation to stop new deliveries. Restore a previous approved Worker version when one exists. This service does not cancel an already dispatched Actions run; cancel it separately. Restoring Worker code does not roll back the D1 database.

Webhook processing shares an eight-second budget across input and GitHub reads. Timed-out requests fail without an automatic dispatch retry. Before a reservation exists, a failed delivery can be redelivered after recovery. After reservation, inspect the request record and Actions first:

  • reserved: a crash or Check update failure may have interrupted the attempt.
  • unknown: the dispatch response was lost, invalid or unexpected. A run may exist.
  • dispatched: the exact returned run ID and URLs are retained.

Active locks never expire automatically. Replaying the same delivery never repeats its dispatch POST. Do not delete an ambiguous row or start another attempt until an operator proves no run remains active and reconciles any observed run with the exact request inputs, repository, workflow and actor.

For a known run, redeliver its workflow_run.completed event after correcting an output-update failure. The service fetches the canonical run and checks the saved installation, repository, run ID, workflow path, dispatch event, original dispatched branch and App actor. It never searches by run title. An early event that arrives before the run ID is saved can be redelivered afterward.

The service saves the verified conclusion and completion time before updating the recorded App-owned reply and request Check. These fields can be populated while the state is still active. A transient GitHub failure retains that lock; redelivery retries safe output updates without dispatching or posting another reply. Conflicting conclusions require inspection. A missing or replaced old reply/Check is not overwritten or recreated; the verified run can still finish its own record. A new PR head's Check is left alone.

After safe output updates, state becomes success, failure or cancelled. Other unsuccessful GitHub conclusions map to failure and retain the exact conclusion. The request Check stays neutral and request-bound. A separate idle Check exposes the current selection action when the same PR head is eligible. A later deliberate command can start a fresh attempt without manual SQL recovery. Duplicate completion does not dispatch or create another conversation reply.

The daily Cron Trigger runs at 03:17 UTC and deletes terminal records older than 90 days. It never expires active or unknown requests, including those with a saved outcome awaiting output recovery. Retention bounds replay history; it does not promise permanent deduplication after a record is deleted.

Keep App credentials in operator storage and Worker secrets. Keep verification receipts outside the repository, without credentials or webhook payloads. A local test or dry-run bundle does not replace live delivery evidence.

Sealed normal-audit evidence

The normal audit producer prepares dependency changes on a fresh read-only runner. It disables lifecycle scripts, local/global pnpmfile hooks and dependency materialization. Its parser installer runs in Python isolated mode. Project validation runs later on another read-only runner with the exact prepared files. That runner can retain check diagnostics; it cannot replace the producer artifact. A failed proposal check still permits a PR report with failed validation. A committed-state validation run fails when its project checks fail.

The existing dependency-audit-pr.md report includes one bounded canonical version1 record with original audit advisories, severities, links, package rows, before/candidate dependency hashes and deduplicated counts. After the report upload succeeds, the producer emits one Node audit evidence v1 notice on its existing Actions job Check. The notice binds the numeric artifact ID and archive and content SHA256 hashes. It requires no Checks write permission.

The normal publisher resolves the expected producer job in the exact current run, then verifies its server-derived Check URL, immutable workflow revision, completed seal steps, unique matching annotation and immutable artifact digest. The provider run, job, Check and artifact heads must agree on the triggering revision. The sealed checkout revision is checked separately against the planned audit revision, which can advance before the coordinated planning checkout. It rejects sibling-job notices, duplicate/ambiguous producer jobs, expired or altered artifacts and mismatched project/dependency state. Native commit trailers retain exact run/job/Check/artifact IDs and hashes for the later read-only repair consumer. Editable PR prose is not historical audit authority. The immutable artifact retains the canonical record while the human PR body omits that opaque record.

Only the publisher reads Actions metadata/artifacts and Check annotations; grant actions: read and checks: read to its calling job and any publisher App token. The producer and project-validation jobs keep Contents read only. The isolated pilot must provide these read grants. Both supported caller templates include them, and fleet setup checks require them. Updating callers in existing repositories and the installation rollout remain later delivery work.

Before the cumulative repair consumer begins, verify a distinct-run hosted readback against the reviewed source and prove a sibling-job forgery rejects. Local API fixtures do not satisfy that hosted gate. Missing or expired history must never be reported as an empty original advisory list.

Cumulative repair preparation

Selected repair preparation reads the nearest native producer receipt on the PR's first-parent history. The initial receipt names the immutable normal audit; later receipts name the original repair preparation artifact. Editable PR text is never used as historical audit evidence. The consumer verifies the producer job, its Check annotation, workflow revision, artifact ID and archive/report hashes, then checks the exact project and dependency-state binding.

Each sealed result.md retains the earlier canonical audit records and renders one cumulative table. Earlier rows keep their recorded outcomes with a historical evidence qualifier. A current unresolved finding takes precedence. An advisory missing from today's audit is only "not observed in the current audit"; absence is not evidence of withdrawal or a newly confirmed repair. The preparation notice names only the currently selected repair.

The preparation runner needs Contents, Actions and Checks read access. It emits one Node repair evidence v1 notice after uploading exactly candidate.patch, audit.json and result.md. The notice binds the immutable artifact and report hashes. Project validation runs on a separate runner and cannot replace this artifact. Missing, expired, ambiguous or mismatched history stops preparation before sealing. Inspect and finish the existing proposal, or explicitly close it and create a new normal-audit proposal; do not reset a branch containing repairs.

This checkpoint prepares cumulative evidence. It does not publish dependency changes or authorize merging. The later publisher must retain the validated repair marker and exact native Node-Repair-* locator trailers for another read-only attempt; producer job outputs alone do not persist across runs.

Publish the validated selected repair

The reusable repair workflow now has a fresh publisher after successful preparation and validation. Only that job receives Contents/Pull requests write permission; its caller must grant those scopes plus Actions/Checks read. It downloads the original three-member preparation artifact by its exact ID and verifies the producer job, Check seal, immutable archive/content digests, source, request, project and exact candidate dependency state. Missing cumulative history or stale evidence stops before a branch push. Project scripts and package-manager commands never run on this writer.

Immediately before pushing, it rechecks the canonical PR, selected batch and head/base. The native push uses an explicit expected-head lease. Its commit contains Node-Repair: validated-selected-pins-v1 plus the original preparation run/job/Check/artifact IDs, archive/content digests and trusted workflow SHA. These trailers locate the immutable cumulative report for the next attempt; editable PR prose is never historical audit input.

The PR body uses the sealed cumulative rows and verified candidate choices, adds the actual successful-validation run link and binds choices to the new head/base. It preserves the existing recognized platform section byte-for-byte, including CRLF and per-release CVE links. A missing or ambiguous platform section or an oversized body stops before push; inspect or replace an old proposal through the normal audit flow rather than dropping evidence.

The push and PR-body edit are separate operations. A successful push followed by an edit failure leaves the repair commit published and reports the incomplete body update with its commit ID. After push, the writer rereads only an exact unchanged pre-push PR view up to four times, with waits of one, two and four seconds. It never repeats the push. An unexpected head or changed base, body, open state, branch or repository stops immediately; an old view at the limit also leaves an explicit partial result. The report is edited only after the expected published head and unchanged remaining PR state are observed. An uncertain push reports the attempted commit and requires branch inspection before another attempt. Nothing merges automatically.

Retain an unmerged repair during normal audits

Before checkout, staging or PR edits, the normal security publisher reads native commit trailers in the maintenance head's history that is not reachable from the audited default branch. An exact Node-Repair: validated-selected-pins-v1 trailer retains the existing PR head, report, open state and draft status. It also protects a repair below a later maintainer commit and a published repair whose report update failed. A marker in the editable body supplies no authority.

Missing branch history or a failed Git inspection stops publication. A failed new audit still fails its Actions job while leaving the repaired PR intact. The repair must be reviewed or handled by a maintainer; there is no automatic reset or replacement. Repairs already reachable from the audited default branch do not block later publication. Upgrade and legacy proposals keep their existing publication behavior. Complete live repair acceptance remains required before general rollout.

History storage bounds

New audit and repair artifacts use storage v2: identical audit objects, repeated strings and dependency-path prefixes are stored once inside the same artifact. The reader still accepts existing v1 artifacts. Both formats reconstruct the same complete logical history, including every attempt, historical field value, path string, duplicate and ordering. No old artifact or native receipt is rewritten.

The report stays bounded at 1 MiB; normal audit history retains its 512 KiB encoded bound. The v2 reader checks backward-only references and computes complete expansion costs before allocating historical containers. It rejects histories above 8 MiB canonical logical JSON, 200000 values/table entries or depth 64. Repeated mutable containers are rebuilt independently. Invalid, cyclic, noncanonical or oversized history fails preparation; it never drops old evidence or skips producer, job, Check, artifact-digest or publication verification.