Add a 'reports' feature

[adamhjk]

We need to add the ability to have 'reports', which produce markdown and json based on model method runs, model changes, and workflow runs. These reports would run after the work, and allow for custom, domain specific output for workflows.

For example, you could have an 'aws infra' report that looks at all the infrastructure deployed in aws via the @swamp/aws extensions, and produces a markdown report of all the relevant information.

Or the cfgmgmt extensions can report if they updated resources.

[adamhjk]

Reports Feature Design

Context

Swamp needs a post-execution reporting system. Reports take information about a
method run, a model, or a workflow and produce both markdown (human-readable) and
JSON (machine-readable) output. Reports are stored as versioned data artifacts
with full GC support, enabling historical tracking.

Reports follow the validation checks pattern for filtering (labels, require/skip)
but differ in key ways: they run after execution (post-flight), they
produce stored data (not ephemeral), they are registry-based (not
model-coupled), and failures are non-fatal but affect exit code.

---

1. Domain Model

ReportRegistry (new, src/domain/reports/)

Reports are a first-class concept with their own registry, similar to
ModelRegistry. A report is defined in TypeScript (typically in
extensions/reports/), registered globally, and referenced by name from model
definitions and workflow YAML.

// src/domain/reports/report.ts

type ReportScope = "method" | "model" | "workflow";

interface ReportResult {
  markdown: string;
  json: Record<string, unknown>;
}

interface ReportDefinition {
  description: string;
  scope: ReportScope;
  labels?: string[];
  execute(context: ReportContext): Promise<ReportResult>;
}

// src/domain/reports/report_registry.ts
class ReportRegistry {
  register(name: string, report: ReportDefinition): void;
  get(name: string): ReportDefinition | undefined;
  getAll(): Array<{ name: string; report: ReportDefinition }>;
  getByScope(scope: ReportScope): Array<{ name: string; report: ReportDefinition }>;
}

ReportContext (discriminated by scope)

All scopes provide dataRepository for historical access AND current execution
dataHandles as pointers. Report authors choose what to use.

// src/domain/reports/report_context.ts

interface BaseReportContext {
  repoDir: string;
  logger: Logger;
  dataRepository: UnifiedDataRepository;
  definitionRepository: DefinitionRepository;
}

interface MethodReportContext extends BaseReportContext {
  scope: "method";
  modelType: ModelType;
  modelId: string;
  definition: { id: string; name: string; version: number; tags: Record<string, string> };
  globalArgs: Record<string, unknown>;
  methodName: string;
  executionStatus: "succeeded" | "failed";
  dataHandles: DataHandle[];
}

interface ModelReportContext extends BaseReportContext {
  scope: "model";
  modelType: ModelType;
  modelId: string;
  definition: { id: string; name: string; version: number; tags: Record<string, string> };
  globalArgs: Record<string, unknown>;
  methodName: string;              // method that triggered this
  executionStatus: "succeeded" | "failed";
  dataHandles: DataHandle[];       // current execution results
  // Historical data accessed via dataRepository.findAllForModel()
}

interface WorkflowReportContext extends BaseReportContext {
  scope: "workflow";
  workflowId: string;
  workflowRunId: string;
  workflowName: string;
  workflowStatus: "succeeded" | "failed";
  stepExecutions: Array<{
    jobName: string;
    stepName: string;
    modelName: string;
    modelType: string;
    methodName: string;
    status: "succeeded" | "failed" | "skipped";
    dataHandles: DataHandle[];
  }>;
}

type ReportContext = MethodReportContext | ModelReportContext | WorkflowReportContext;

ReportSelection (definition/workflow YAML)

Parallel to CheckSelectionSchema in src/domain/definitions/definition.ts,
but richer — each entry can be a string (all methods) or an object with method
scoping:

// src/domain/reports/report_selection.ts

// Each entry: string shorthand or object with optional method scoping
const ReportRefSchema = z.union([
  z.string(),
  z.object({
    name: z.string(),
    methods: z.array(z.string()).optional(), // if omitted, all methods
  }),
]);

export const ReportSelectionSchema = z.object({
  require: z.array(ReportRefSchema).optional(),
  skip: z.array(z.string()).optional(),  // skip is always by name
}).optional();

export type ReportRef = string | { name: string; methods?: string[] };
export type ReportSelection = {
  require?: ReportRef[];
  skip?: string[];
};

Referenced from model definitions:

# definition YAML — string shorthand (all methods)
reports:
  require:
    - cost-summary
  skip:
    - verbose-audit

# definition YAML — with method scoping
reports:
  require:
    - name: cost-summary
      methods: [create, update]
    - compliance-report        # string shorthand = all methods
  skip:
    - verbose-audit

Referenced from workflow YAML:

# workflow YAML
reports:
  require:
    - deployment-summary
  skip:
    - debug-report

Method scoping is a deployment concern (controlled in YAML), not a report
definition concern. The report's execute() function always runs when selected —
it receives the method name in context and can produce scope-appropriate content.

---

2. Report Output Storage

Reports are stored as two data artifacts per report execution using the
existing data infrastructure (versioned, with GC):

• Markdown: data name report-{reportName}, contentType text/markdown
• JSON: data name report-{reportName}-json, contentType application/json

Tags:

{
  type: "report",
  reportName: reportName,
  reportScope: "method" | "model" | "workflow",
}

Default lifetime/GC (configurable later):

• lifetime: "30d"
• garbageCollection: 5 (keep last 5 versions)

Method/model scope: stored under the model's data path
(.swamp/data/{model-type}/{model-id}/report-{name}/...)

Workflow scope: stored under a synthetic workflow path
(.swamp/data/workflow/{workflow-id}/report-{name}/...)

Accessible via existing swamp data list --tag type=report and
data.latest("model-name", "report-{name}") CEL expressions.

---

3. Report Execution Service

New file: src/domain/reports/report_execution_service.ts

Responsibilities:

• Filter applicable reports (require/skip/labels/appliesTo/CLI flags)
• Build ReportContext for each scope
• Execute reports sequentially
• Persist markdown + JSON artifacts via DataWriter
• Return results (success/failure per report) for rendering
• Non-fatal: failures logged as warnings, but tracked for exit code

Filtering logic (mirrors check filtering in method_execution_service.ts:389-410):

1. Definition/workflow-level skip always wins
2. Respect YAML method scoping (if report ref has methods: [...], only run for those methods)
3. Required reports immune to CLI skip flags
4. Honor CLI: --skip-reports, --skip-report <name>, --skip-report-label <label>
5. Inclusion filters: --report <name>, --report-label <label> (narrow to subset)

---

4. Integration Points

Method Execution (method_execution_service.ts)

After method completes and ModelOutput is saved (after line ~653), before
deletion markers:

1. Collect method-scope and model-scope reports from the ReportRegistry that
   match the current model type
2. Filter by definition-level reportSelection + CLI flags
3. Execute each, persist artifacts
4. Track failures for exit code

Workflow Execution (execution_service.ts)

Per-step: Method/model scope reports run as part of the step's method
execution (handled by method_execution_service integration above).

Post-workflow: After run.complete() (line ~862):

1. Collect workflow-scope reports from the ReportRegistry
2. Filter by workflow-level reportSelection + CLI flags
3. Build WorkflowReportContext with all step execution data from WorkflowRun
4. Execute each, persist artifacts under workflow data path

Report events (new, added to existing event unions):

// Events carry content so renderers can display reports inline:
| { kind: "report_started"; reportName: string; scope: ReportScope }
| { kind: "report_completed"; reportName: string; scope: ReportScope;
    markdown: string; json: Record<string, unknown>; dataHandles: DataHandle[] }
| { kind: "report_failed"; reportName: string; scope: ReportScope; error: string }

The report_completed event carries both markdown and json content. The log
renderer displays syntax-highlighted markdown delineated per report. The JSON
renderer accumulates json into a reports object keyed by report name.

---

5. CLI Interface

Flags on model method run (after existing check flags)

--skip-reports               Skip all post-run reports
--skip-report <name>         Skip a specific report (repeatable)
--skip-report-label <label>  Skip reports with this label (repeatable)
--report <name>              Run only this report (repeatable, inclusion filter)
--report-label <label>       Run only reports with this label (repeatable)

These map to new fields on MethodContext:

skipReportNames?: string[];
skipReportLabels?: string[];
skipAllReports?: boolean;
reportNames?: string[];     // inclusion filter
reportLabels?: string[];    // inclusion filter

Same flags on workflow run

Threaded through to method execution for per-step reports and to workflow-level
report execution.

Standalone model report command (new)

swamp model report [model_id_or_name]
  --label <label>       Only run reports with this label
  --method <method>     Simulate method context
  --format md|json|both Output format to display (default: both)

Modeled on model validate command. Runs reports without executing a method
(useful for generating reports from existing stored data).

Report Display

Reports are displayed to the user after execution, with each report clearly
delineated. The display behavior differs by output mode:

Log mode (human-readable)

Each report's markdown content is rendered to the terminal with syntax
highlighting when colors are enabled (NO_COLOR not set, --color not false).
When colors are disabled, plain markdown text is output.

Generating 3 report(s)...

── Report: cost-summary ──────────────────────────────────
# Cost Summary                        ← highlighted markdown
Estimated monthly cost: **$42.50**
| Resource | Cost |
| -------- | ---- |
| EC2      | $30  |
| EBS      | $12.50 |
──────────────────────────────────────────────────────────

── Report: compliance-check ──────────────────────────────
# Compliance Check
All resources pass compliance requirements.
──────────────────────────────────────────────────────────

✗ Report: drift-report → template evaluation failed

Markdown rendering uses npm:marked + npm:marked-terminal (see below).
Reports that fail show the error inline and do not render content.

JSON mode (machine-readable)

Output is a single JSON object where each report name is a key and its produced
JSON data is the value:

{
  "reports": {
    "cost-summary": {
      "totalCost": 42.50,
      "currency": "USD",
      "resources": [...]
    },
    "compliance-check": {
      "passed": true,
      "checks": [...]
    }
  }
}

Failed reports include an error field instead of data:

{
  "reports": {
    "drift-report": { "_error": "template evaluation failed" }
  }
}

This reports object is included in the existing ModelMethodRunView (for method
runs) and WorkflowRunView (for workflow runs) so it's part of the standard
JSON output.

Markdown rendering: npm:marked + npm:marked-terminal

Use the existing marked-terminal library (source at ../marked-terminal) via
npm:marked-terminal import. It provides a full terminal markdown renderer with
chalk-based syntax highlighting (headings, bold, tables, code blocks with
cli-highlight, emoji, hyperlinks).

// src/presentation/markdown_renderer.ts
import { marked } from "npm:marked";
import TerminalRenderer from "npm:marked-terminal";

export function renderMarkdownToTerminal(markdown: string): string {
  marked.setOptions({ renderer: new TerminalRenderer() });
  return marked(markdown);
}

export function renderMarkdownPlain(markdown: string): string {
  return markdown; // pass-through when NO_COLOR is set
}

chalk (used by marked-terminal) auto-respects NO_COLOR env var, so when
colors are disabled it degrades to plain text automatically. Add npm:marked and
npm:marked-terminal to deno.json imports.

---

6. Extension Support

Report definition in extensions

// extensions/reports/cost-summary.ts
export const report: ReportDefinition = {
  description: "Summarize estimated costs for provisioned resources",
  scope: "method",
  labels: ["cost", "finops"],
  async execute(context: MethodReportContext): Promise<ReportResult> {
    // Access current data handles and/or historical data
    const data = await context.dataRepository.findAllForModel(
      context.modelType, context.modelId,
    );
    return {
      markdown: "# Cost Summary\n...",
      json: { totalCost: 42.50, currency: "USD" },
    };
  },
};

Loading

A new report loader (parallel to user_model_loader.ts) discovers and registers
reports from extensions/reports/. Reports are loaded alongside model extensions
during startup.

Model types can also define inline reports

For convenience, ModelDefinition gains an optional reports field:

interface ModelDefinition {
  // ... existing fields ...
  reports?: Record<string, ReportDefinition>;  // inline reports, auto-registered
}

These are auto-registered in the ReportRegistry with the model type as a
namespace prefix to avoid conflicts.

---

7. Exit Code Behavior

• Report failures are non-fatal: they never block method/workflow execution
• Report failures affect exit code: if any report fails, the command exits
  with a non-zero code
• This is tracked via a reportFailures counter returned from the report
  execution service, bubbled up to the CLI command handler

---

8. Files to Create

File	Purpose
src/domain/reports/report.ts	ReportDefinition, ReportResult, ReportScope types
src/domain/reports/report_context.ts	ReportContext union type (method/model/workflow)
src/domain/reports/report_registry.ts	ReportRegistry class
src/domain/reports/report_execution_service.ts	Filtering, execution, persistence logic
src/domain/reports/report_selection.ts	ReportSelectionSchema, ReportSelection type
src/domain/reports/mod.ts	Barrel exports
src/domain/reports/report_registry_test.ts	Registry tests
src/domain/reports/report_execution_service_test.ts	Execution service tests
src/cli/commands/model_report.ts	Standalone model report command
src/presentation/markdown_renderer.ts	Thin wrapper around npm:marked + npm:marked-terminal
src/presentation/markdown_renderer_test.ts	Markdown renderer tests
src/presentation/output/model_report_output.ts	Report output formatter
src/presentation/output/model_report_output_test.ts	Output formatter tests

9. Files to Modify

File	Change
src/domain/models/model.ts	Add reports?: Record<string, ReportDefinition> to ModelDefinition
src/domain/models/model.ts	Update ModelRegistry.extend() to accept reports parameter
src/domain/models/method_execution_service.ts	Add post-flight report execution after method completes
src/domain/definitions/definition.ts	Add reports: ReportSelectionSchema to DefinitionSchema, add reportSelection getter to Definition
src/domain/workflows/execution_service.ts	Add workflow-scope report execution after run.complete()
src/domain/workflows/execution_events.ts	Add report_started/completed/failed event kinds
src/domain/workflows/workflow.ts	Add reports: ReportSelection field to Workflow
src/cli/commands/model_method_run.ts	Add 5 report CLI flags
src/cli/commands/workflow_run.ts	Add 5 report CLI flags
src/cli/commands/model_create.ts	Register model report subcommand
src/libswamp/models/run.ts	Add report events to ModelMethodRunEvent, add report fields to input
src/libswamp/models/model_method_run_view.ts	Add ReportResultView to view
src/presentation/renderers/model_method_run.ts	Add report event handlers
src/presentation/renderers/workflow_run.ts	Add report event handlers
design/models.md	Document reports feature (after Pre-flight Checks section)

10. Verification

1. deno check — all new types compile
2. deno lint && deno fmt — passes
3. deno run test — all new and existing tests pass
4. Manual: create a test report in extensions/reports/, reference it from a
   model definition, run swamp model method run and verify:
   • Report executes after method
   • Markdown + JSON artifacts stored in .swamp/data/
   • swamp data list shows report artifacts
   • --skip-reports flag suppresses execution
   • Report failure causes non-zero exit code without blocking method
5. Manual: reference report from workflow YAML, run workflow, verify
   workflow-scope report runs at completion
6. deno run compile — binary compiles successfully