Skip to content

Test suite: error and edge paths have zero coverage #16

Description

@baijum

Summary

tests/test-cli.scm passes and is well-built (self-counting, string-contains? for help output rather than brittle exact strings), but it exercises none of the behaviors that turned out to be buggy in the audit. List of concrete gaps, to be closed alongside the fixes:

  1. --flag=value on a flag spec — would have caught the crash in Parser crash: --flag=value on any flag spec aborts with 'car: expected pair, got ()' #2
  2. Malformed specs: empty long name, long without --, non-string command name, duplicate long names (No spec validation: malformed long names crash every run or corrupt option keys; duplicate names cross-wire #11)
  3. parsed-ref on an unknown name → #f; parsed-flag? on an unset flag and on non-#t values (only the #t cases are tested today)
  4. Missing positional → (("input" . #f)) — the or ... "World" idiom in README and the greeter depends on it, untested
  5. Missing option value at end of argv; extra positionals silently dropped (Option value handling: greedily consumes flag-like tokens; missing value silently keeps the default #7)
  6. Coercion fallbacks: --count abc"abc", --count="" (Numeric option coercion silently falls back to string — contradicts README and crashes user code far from the cause #8)
  7. The 3-arg option form (no default) — both existing option tests use explicit defaults
  8. run-cli dispatch paths: "Unknown command" (known command, missing handler), no-#f-handler fallback, and exit codes (run-cli error paths exit 0: 'Unknown command' and missing-handler fallback never set a nonzero exit #4, Unknown options and typo'd commands are silently ignored; the 'Unknown command' message is unreachable #5)
  9. generate-help with an unknown subcommand (generate-help with a nonexistent subcommand fabricates a plausible help page #15)
  10. An option named --help colliding with the built-in (A user-defined option named --help silently disables dispatch (name collision) #10)
  11. Help short-circuit result shape: positionals parsed before --help are discarded from parsed-args

Notes

  • Keep the suite self-contained: check helper + run-cli-parse/dispatch, no test framework (per AGENTS.md).
  • Cosmetic while in there: the suite dumps the full generated help twice to stdout at the end (lines 169-174), which pollutes non-verbose runs.

Context

Found during a multi-agent code review (2026-09-06); verified on Kaappi v0.26.2.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    auditFound during systematic auditpriority: mediumShould fix — plan for upcoming work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions