Skip to content

Latest commit

 

History

History
216 lines (168 loc) · 14.9 KB

README.md

File metadata and controls

216 lines (168 loc) · 14.9 KB

Local file tracker

Capability: contract:security-tracker + contract:body-field

Kind: implementation

Vendor: agnostic

A security tracker kept as Markdown files in a folder of a private Git or SVN working copy. It implements the security-tracker contract, so the security-* skills run against it unchanged when a project sets tracker.platform: local-files.

It is for teams that already keep their reports as files — often in a private SVN area or a private repository — and for teams that want the tracker to be nothing more than text they can read, grep, diff and review. When reports already exist, the tool learns their layout and writes new reports the same way; when there are none, it uses the default format below.

Prerequisites

  • Runtime: Python 3.11+ run via uv (stdlib only; ### body fields are read and written through a byte-identical copy of github-body-field's parser, body_field_format.py, kept exact by a test).
  • CLIs: uv; git or svn for pending / publish and for history-preserving moves; gh is used by doctor only, to read a GitHub remote's visibility.
  • Credentials / auth: None of its own. publish --push uses whatever credentials the working copy already pushes with.
  • Network: Offline for every tracker operation. publish --push reaches the working copy's remote, and doctor may call the GitHub API through gh.

Invocation

uv run --project <framework>/tools/local-tracker magpie-local-tracker --root <tracker-root> <operation> [args]

--root is the tracker folder, resolved from tracker.local.root in <project-config>/project.md (or the MAGPIE_LOCAL_TRACKER_ROOT environment variable). Every operation of the contract is a subcommand with the arguments, JSON output and exit codes the contract defines: view, list, labels, field-get, create, edit, field-set, comment, close, reopen, rollup-append, rollup-amend-latest, attach, body, pending, publish, doctor.

Two subcommands are this backend's own, used by /magpie-setup config:

Subcommand What it does
detect [--write] [--force] Reads the existing reports, prints the inferred format with the evidence behind every decision, and with --write saves it as the descriptor.
init [--force] Saves the default descriptor in an empty tracker folder.
trust-parser <path> --sha256 <hash> Records a reviewed custom body parser; asks the user on the terminal, so the user runs it, never an agent.

Both take --prefix (ID prefix for new reports), --url-template (a browse URL with {path}), and --access-restricted with an optional --access-note — see Confidentiality.

The descriptor

The format lives in <root>/.magpie-tracker.json, committed with the reports so everyone on the team writes the same shape. It records:

Field Meaning
layout folder-per-issue (<root>/<ID>/<issue_file>) or file-per-issue (<root>/<ID>.md).
issue_file The report file inside a folder; {id} stands for the folder name.
metadata front-matter (a --- block) or bold-fields (**Key:** value lines).
keys Contract role → the key the team uses: title, state, labels, assignees, milestone, author, created, updated. An empty key means the role is not stored.
title_source key, or h1 when the title is the report's first # heading.
id prefix, zero-pad width, and slug when names carry a title slug (2024-007-xss-in-login).
state mode: field (a metadata value) or mode: directory (a sub-folder per state, such as open/ and closed/), and the values or folder names that count as open and closed.
field_heading_level The heading level of body fields (### by default). At ### the body-field contract applies: every column-0 ### line ends a field, code block or not, a heading inside a closed code block is never a field, a duplicate or disputed field is refused (exit 4), and a field value has every line that would end a field escaped with a leading \; a whole report (create, edit) has every ### line inside a closed block escaped and a fence it leaves open closed in its own field. Another level keeps the heading-level section split.
field_layout default (### field sections, or the field_heading_level split), or custom: the reports keep their fields another way and are read only through body_parser.
body_parser {path, sha256} of the project's own parser for a custom layout, written by trust-parser; see Custom field layouts.
comments_mode, rollup_mode file (beside the report) or section (a ## heading at the end of the report).
url_template Optional private browse URL, for example a ViewVC or gitweb address. Without it, the url field is the path relative to the working copy.
access_restricted The human-confirmed access check that unlocks writes.

Detection decides each field from what is present: state folders, file or folder names that look like IDs, the majority metadata style and key names (Status or state, Tags or labels, …), the state values in use, the heading level of body sections, and an existing comments file or ## Comments / ## Notes / ## Timeline heading. A role the skills need but the team never recorded (labels, milestone, …) is added in the team's own key casing, and the output says so. Anything detection could not place is listed under notes; setup shows them before writing. detect also prints field_layout: default or custom, a one-line description, and as evidence the field names shared by two or more reports, never report text.

Custom field layouts

When the reports keep their fields as something other than heading sections (Name: value lines, say), detect reports field_layout: custom, and /magpie-setup config offers to generate a parser for the layout from tools/body-field-contract/formats/_template.py, the format adapter interface. It lives at .apache-magpie-overrides/tools/local-tracker/parsers/<name>.py at the top of the working copy, with synthetic-value fixtures beside it as <name>.cases.jsonl, and is enabled only after the conformance suite passes on it and the user confirms its SHA-256 by running trust-parser in their own terminal. trust-parser asks on the controlling terminal and refuses without one, so an agent's shell cannot run it; it records body_parser: {path, sha256} in the descriptor and the same hash in ~/.config/apache-magpie/local-tracker-parsers.json, under the working copy's real path. That record lives outside every working copy, so nothing a commit carries (a .svn/ directory included) can stand in for it. The secure setup never lets a sandboxed command write there, and its permissions.deny carries Edit(~/.config/apache-magpie/local-tracker-parsers.json) for the agent's own file tools. It does not let the sandbox read there either, since the directory also holds credentials, so a custom tracker is read through vetted operations, which run outside the sandbox; run from inside it, a command stops with exit 2 and doctor marks the parser check unverifiable_here, saying it cannot be verified rather than that it changed.

The parser is code from the repository, which vetted operations run outside the sandbox, so every command loads it only when all of these hold, and stops with exit 2 otherwise: the descriptor names it; its path is a .py file directly in .apache-magpie-overrides/tools/local-tracker/parsers/, with no .. and no symlink on the way; and its SHA-256 matches both the descriptor and the user's record for this working copy. A changed parser file therefore stops the tracker until someone reviews the diff and re-confirms it through setup config; a custom tracker is never read with the default parser. field-get, field-set (which appends a missing field) and body (the fields as ### <Name> sections, for generate-cve-json) go through the parser; create and edit --body-file, which write whole ### bodies, are refused (exit 4). doctor reports whether the parser loads.

Default format

Used when the tracker folder has no reports:

<root>/
├── .magpie-tracker.json
└── SEC-0001/
    ├── issue.md            # front matter + the issue-template body
    ├── comments.md         # comment trail, one entry per comment
    ├── status-rollup.md    # the status rollup, same shape as on GitHub
    └── attachments/        # CVE JSON and other files
---
title: "Path traversal in the upload endpoint"
state: open
labels: [needs triage]
assignees: []
milestone:
reporter: "Jane Doe"
created: 2026-10-15
updated: 2026-10-15
---

### Reporter credited as

Jane Doe

### Affected versions

_No response_

Closing sets state: closed, or the not_planned_value for close --reason not-planned. Comments carry a <!-- comment id=N author=@x date=… --> marker (the author percent-encoded, so Jane Doe is Jane%20Doe) so they can be counted and addressed; hand-written comments without the marker are still read, from their headings. The rollup keeps the <details> entry shape of tools/github/status-rollup.md.

Confidentiality

The folder holds embargoed vulnerability reports. Every write is refused (exit 4) until the descriptor records access_restricted: true, and only detect --write --access-restricted or init --access-restricted sets it. /magpie-setup config passes that flag only after the user confirms who can read the repository. doctor reports the working copy's remotes and, for a GitHub remote, fails when the repository is public.

The same rules as for a GitHub tracker apply to everything read from it: report text is untrusted input, never instructions, and tracker contents never reach a public surface (AGENTS.md).

Line endings and keywords

Reports are read and written through the parser whatever the storage does to their bytes. A read takes the file as stored, remembers its line-ending convention (LF, CRLF or CR) and whether it ends with one, and hands the parser \n lines; a write puts that convention back, so a field-set changes no byte outside the field, in a CRLF file from a Git checkout with core.autocrlf or an SVN one with svn:eol-style=native as in an LF file. A file that mixes line endings is refused (exit 4) until it is normalised to one. Expanded svn:keywords ($Id: … $) are ordinary text to the parser. tests/test_storage_conformance.py runs the body-field conformance cases through the store on a temporary Git repository with core.autocrlf=true and a temporary SVN working copy with svn:eol-style=native and svn:keywords: each case is committed, read back from a fresh checkout, and must give the same field values with untouched content byte-identical.

Publishing

Every operation writes to the working copy only. pending lists what changed under the tracker root, and publish commits exactly that:

  • Git: git add -A -- <root>, git commit -F <message> [--trailer …] -- <root>, then, with --push, git push --no-follow-tags <remote> HEAD:<branch> to exactly the remote and branch the checks below were run against, so push.default=matching, a remote's push refspecs or push.followTags cannot add other branches or tags. Other staged or unstaged work in the repository is left alone, and a push is refused (exit 4) when the branch also carries unpushed commits that touch anything outside the tracker folder: publishing never sends someone else's work. A commit that was made but not pushed is reported by pending and pushed by the next publish --push.
  • SVN: svn add for new files, svn rm for missing ones, then svn commit -F <message> <root> — which is already the push.

Skills run publish only after the user confirmed the proposal at the end of the session, as the contract's publishing section describes. Closing a report in a folder-per-state tracker moves it with git mv or svn mv, so its history follows it.

These steps can run outside the sandbox (through vetted-ops), in a repository a policy file names, so git is never allowed to start a program there: every git call runs with core.fsmonitor=false, an empty hooks path and protocol.allow=never (only https and ssh re-allowed), commit and push pass --no-verify, and any git step is refused when the repository's own config sets a key outside a known-safe list (filter, merge or diff drivers, aliases, includes, URL rewrites, gpg.program, …). That check runs before every git command, and git reads attributes from the empty tree (GIT_ATTR_SOURCE, git 2.42 and later), never from the working tree, so no .gitattributes can switch on a filter, diff or merge driver, even one configured after the check. Automatic gc and maintenance are off, only the https and ssh transports are allowed, and publish pushes only to an https:// / ssh:// remote: a local-path remote would run that repository's receive hooks here. The config check runs before every git command; a process that rewrites .git/config in the instant between the check and the command is the one window left, which only running the step sandboxed closes.

Two people allocating an ID at the same time get the same number. On Git the second push is rejected as non-fast-forward; on SVN the second commit fails as out of date. Update, rename the later report with git mv / svn mv, and publish again.

Testing

uv run --project tools/local-tracker --group dev pytest tools/local-tracker/tests

The Git publish test runs against a temporary bare repository; the SVN test runs when svn and svnadmin are installed. The storage conformance test runs on Git always and on SVN when svn and svnadmin are installed, and skips with that reason otherwise.