feat: add daily CLAUDE.md audit command#185
Conversation
Add `/claude-md-audit-daily` meta-orchestrator that: - Runs `/audit-claude-md` via sub-agent on a daily schedule - Compares report against previous baseline for regressions - Creates a PR only when issue count increases - Follows established patterns from security-audit-daily and docs-audit-daily Update canon/autoclaude.md to mark all 4 steps as complete. Co-Authored-By: Claude <noreply@anthropic.com>
📝 WalkthroughWalkthroughIntroduces a new meta-orchestrator for automated daily CLAUDE.md audits with a five-phase workflow including branch management, unattended execution, baseline comparison, and conditional PR creation. Updates project documentation to reference the new daily audit capability. Changes
Sequence DiagramsequenceDiagram
participant Scheduler as Scheduler/Cron
participant Orchestrator as Daily Audit<br/>Orchestrator
participant Git as Git/Branch<br/>Manager
participant SubAgent as Sub-Agent<br/>(Audit Runner)
participant Reports as Reports<br/>Directory
participant BaselineComp as Baseline<br/>Comparator
participant GitOps as Git Push/<br/>PR Handler
Scheduler->>Orchestrator: Trigger daily audit
Orchestrator->>Orchestrator: Phase 0: Preflight validation
Orchestrator->>Git: Phase 1: Create daily branch<br/>(claude-md/audit-YYYY-MM-DD)
Git-->>Orchestrator: Branch created/verified
Orchestrator->>SubAgent: Phase 2: Delegate audit task
SubAgent->>SubAgent: Run full CLAUDE.md audit
SubAgent-->>Orchestrator: Audit complete, return report
Orchestrator->>Reports: Write audit report<br/>(YYYY-MM-DD-audit.md)
Orchestrator->>Reports: Phase 3: Read previous baseline
Reports-->>Orchestrator: Previous audit data
Orchestrator->>BaselineComp: Compare NEW vs PREVIOUS totals
BaselineComp-->>Orchestrator: STATUS & ACTION<br/>(commit-only/create-pr/error)
Orchestrator->>GitOps: Phase 4: Commit changes
alt Regression Detected
GitOps->>GitOps: Create PR with delta details
end
GitOps->>Git: Push branch (or delete if clean)
Git-->>Orchestrator: Push complete
Orchestrator->>Orchestrator: Phase 5: Cleanup & report status
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly Related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Deploying herdctl with
|
| Latest commit: |
1874a8b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://78febcec.herdctl.pages.dev |
| Branch Preview URL: | https://feature-claude-md-daily-audi.herdctl.pages.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/commands/claude-md-audit-daily.md:
- Around line 142-161: The fenced code block that begins with "Use the Task tool
with:" (containing keys like subagent_type: "general-purpose",
run_in_background: false, prompt: "...") is untyped and triggers markdownlint
MD040; update the opening fence from ``` to include a language identifier (e.g.,
```text) so the block becomes a typed fenced code block and linting passes.
- Around line 199-223: The decision order makes CLEAN unreachable because the
NEW_TOTAL == 0 check comes after the numeric comparisons; update the branching
logic that sets STATUS and ACTION (using variables NEW_TOTAL, PREVIOUS_TOTAL,
STATUS, ACTION and the Phase 4/5 flow) so the zero-check runs before comparing
NEW_TOTAL to PREVIOUS_TOTAL (or implement a post-condition override that sets
STATUS="CLEAN" and ACTION="commit-only" when NEW_TOTAL==0). Ensure the
special-case for NEW_TOTAL=="ERROR" still short-circuits to STATUS="ERROR" and
skips Phase 4 as before.
- Around line 264-289: The heredoc is using a quoted delimiter (<<'PREOF') which
disables shell variable expansion so ${PREVIOUS_TOTAL}, ${NEW_TOTAL},
${ISSUE_DELTA}, and ${TODAY} remain literal; change the delimiter to an unquoted
form (<<PREOF) in the --body "$(cat <<'PREOF' ... PREOF )" block so those
variables are interpolated into the PR body.
| ``` | ||
| Use the Task tool with: | ||
| - subagent_type: "general-purpose" | ||
| - run_in_background: false | ||
| - prompt: "IMPORTANT RULES: | ||
| (1) Do NOT create branches or run git checkout — stay on the current branch. | ||
| (2) Do NOT commit any changes — the orchestrator handles commits. | ||
|
|
||
| Run the /audit-claude-md command. Execute a full CLAUDE.md audit: | ||
| 1. Read the gold standard from canon/claude-md-gold-standard.md | ||
| 2. Discover all CLAUDE.md files and .claude/rules/*.md files | ||
| 3. Launch the 4 audit sub-agents (inventory+size, style+content, staleness, coverage) | ||
| 4. Assemble the full report | ||
| 5. Write the report to .reports/claude/{TODAY}-audit.md | ||
|
|
||
| When complete, report back with: | ||
| - Total issues found (number) | ||
| - Breakdown by dimension (inventory, size, style, content, staleness, coverage) | ||
| - The full remediation checklist" | ||
| ``` |
There was a problem hiding this comment.
Add a language identifier to this fenced block.
This block is currently untyped; markdownlint MD040 is triggered. Use a language tag (likely text here since this is Task-tool instruction format).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/commands/claude-md-audit-daily.md around lines 142 - 161, The fenced
code block that begins with "Use the Task tool with:" (containing keys like
subagent_type: "general-purpose", run_in_background: false, prompt: "...") is
untyped and triggers markdownlint MD040; update the opening fence from ``` to
include a language identifier (e.g., ```text) so the block becomes a typed
fenced code block and linting passes.
| If NEW_TOTAL is "ERROR": | ||
| STATUS="ERROR" | ||
| ACTION="error" | ||
| → Skip Phase 4, proceed to Phase 5 cleanup | ||
|
|
||
| If NEW_TOTAL > PREVIOUS_TOTAL: | ||
| STATUS="REGRESSION" | ||
| ACTION="create-pr" | ||
| → New issues found, create a PR for visibility | ||
|
|
||
| If NEW_TOTAL == PREVIOUS_TOTAL: | ||
| STATUS="STABLE" | ||
| ACTION="commit-only" | ||
| → No change, commit report to branch but no PR needed | ||
|
|
||
| If NEW_TOTAL < PREVIOUS_TOTAL: | ||
| STATUS="IMPROVED" | ||
| ACTION="commit-only" | ||
| → Issues decreased (someone fixed things), commit report as new baseline | ||
|
|
||
| If NEW_TOTAL == 0: | ||
| STATUS="CLEAN" | ||
| ACTION="commit-only" | ||
| → Perfect score, commit report as new baseline | ||
| ``` |
There was a problem hiding this comment.
CLEAN status is unreachable in a common zero-issue case.
When NEW_TOTAL=0 and PREVIOUS_TOTAL>0, the < branch is matched first, so status becomes IMPROVED instead of CLEAN. Move the zero check before numeric comparisons (or derive CLEAN as a post-condition override).
Suggested decision-order fix
-If NEW_TOTAL > PREVIOUS_TOTAL:
+If NEW_TOTAL == 0:
+ STATUS="CLEAN"
+ ACTION="commit-only"
+ → Perfect score, commit report as new baseline
+
+If NEW_TOTAL > PREVIOUS_TOTAL:
STATUS="REGRESSION"
ACTION="create-pr"
→ New issues found, create a PR for visibility
@@
-If NEW_TOTAL == 0:
- STATUS="CLEAN"
- ACTION="commit-only"
- → Perfect score, commit report as new baseline🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/commands/claude-md-audit-daily.md around lines 199 - 223, The
decision order makes CLEAN unreachable because the NEW_TOTAL == 0 check comes
after the numeric comparisons; update the branching logic that sets STATUS and
ACTION (using variables NEW_TOTAL, PREVIOUS_TOTAL, STATUS, ACTION and the Phase
4/5 flow) so the zero-check runs before comparing NEW_TOTAL to PREVIOUS_TOTAL
(or implement a post-condition override that sets STATUS="CLEAN" and
ACTION="commit-only" when NEW_TOTAL==0). Ensure the special-case for
NEW_TOTAL=="ERROR" still short-circuits to STATUS="ERROR" and skips Phase 4 as
before.
| --body "$(cat <<'PREOF' | ||
| ## CLAUDE.md Audit Regression | ||
|
|
||
| The daily CLAUDE.md audit found **more issues than the previous baseline**. | ||
|
|
||
| | Metric | Previous | Current | Delta | | ||
| |--------|----------|---------|-------| | ||
| | Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} | | ||
|
|
||
| ### What This Means | ||
|
|
||
| New CLAUDE.md quality issues were introduced since the last audit. Review the | ||
| audit report at `.reports/claude/${TODAY}-audit.md` for the full remediation | ||
| checklist. | ||
|
|
||
| ### Next Steps | ||
|
|
||
| 1. Review the remediation checklist in the audit report | ||
| 2. Fix the new issues (or confirm they are acceptable) | ||
| 3. Merge this PR to update the baseline | ||
|
|
||
| --- | ||
|
|
||
| *Generated by `/claude-md-audit-daily` on ${TODAY}* | ||
| PREOF | ||
| )" \ |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
fd -t f "claude-md-audit-daily.md"Repository: edspencer/herdctl
Length of output: 43
🏁 Script executed:
find . -type f -name "*claude*" | head -20Repository: edspencer/herdctl
Length of output: 237
🏁 Script executed:
wc -l ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 108
🏁 Script executed:
sed -n '264,289p' ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 769
🏁 Script executed:
# Let me check the context around this code to understand the intent
sed -n '250,290p' ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 1120
🏁 Script executed:
# Let me verify the heredoc behavior with variables in the context
cat > /tmp/test_heredoc.sh << 'EOF'
#!/bin/bash
PREVIOUS_TOTAL=3
NEW_TOTAL=7
ISSUE_DELTA=4
TODAY="2024-01-15"
echo "Test 1: Quoted heredoc (no expansion) - what the code currently does:"
result=$(cat <<'PREOF'
| Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} |
Generated on ${TODAY}
PREOF
)
echo "$result"
echo ""
echo "Test 2: Unquoted heredoc (with expansion) - what reviewer suggests:"
result=$(cat <<PREOF
| Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} |
Generated on ${TODAY}
PREOF
)
echo "$result"
EOF
bash /tmp/test_heredoc.shRepository: edspencer/herdctl
Length of output: 345
🏁 Script executed:
# Check if there are other instances of similar patterns in the file
grep -n "<<'" ./.claude/commands/claude-md-audit-daily.mdRepository: edspencer/herdctl
Length of output: 90
PR body variables won't expand due to quoted heredoc delimiter.
Line 264 uses <<'PREOF', which prevents shell variable expansion. Variables like ${PREVIOUS_TOTAL}, ${NEW_TOTAL}, ${ISSUE_DELTA}, and ${TODAY} will appear literally in the PR body instead of being interpolated. Change to <<PREOF (unquoted delimiter) to enable expansion.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.claude/commands/claude-md-audit-daily.md around lines 264 - 289, The
heredoc is using a quoted delimiter (<<'PREOF') which disables shell variable
expansion so ${PREVIOUS_TOTAL}, ${NEW_TOTAL}, ${ISSUE_DELTA}, and ${TODAY}
remain literal; change the delimiter to an unquoted form (<<PREOF) in the --body
"$(cat <<'PREOF' ... PREOF )" block so those variables are interpolated into the
PR body.
edspencer
left a comment
edspencer
left a comment
There was a problem hiding this comment.
This PR adds a /claude-md-audit-daily meta-orchestrator command (a markdown spec of bash steps an agent executes) and marks steps 3–4 of the AutoCLAUDE initiative complete in canon/autoclaude.md.
I read the new command file in full, compared its git/branch/PR logic against the sibling docs-audit-daily.md and security-audit-daily.md commands it says it follows, and traced the shell variables through Phases 0–5. The overall flow (pre-flight, branch isolation, sub-agent delegation, baseline comparison, conditional PR) is sound and consistent with the established patterns. The canon doc edit is accurate. I found one real defect in the PR-body generation and one unreachable branch in the decision logic; no security or injection concerns in the diff.
| --body "$(cat <<'PREOF' | ||
| ## CLAUDE.md Audit Regression | ||
|
|
||
| The daily CLAUDE.md audit found **more issues than the previous baseline**. | ||
|
|
||
| | Metric | Previous | Current | Delta | | ||
| |--------|----------|---------|-------| | ||
| | Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} | | ||
|
|
||
| ### What This Means | ||
|
|
||
| New CLAUDE.md quality issues were introduced since the last audit. Review the | ||
| audit report at `.reports/claude/${TODAY}-audit.md` for the full remediation | ||
| checklist. | ||
|
|
||
| ### Next Steps | ||
|
|
||
| 1. Review the remediation checklist in the audit report | ||
| 2. Fix the new issues (or confirm they are acceptable) | ||
| 3. Merge this PR to update the baseline | ||
|
|
||
| --- | ||
|
|
||
| *Generated by `/claude-md-audit-daily` on ${TODAY}* | ||
| PREOF |
There was a problem hiding this comment.
MEDIUM · bug — Single-quoted heredoc leaves ${...} placeholders literal in the PR body
The PR body is built with cat <<'PREOF'. A quoted heredoc delimiter disables shell expansion, so ${PREVIOUS_TOTAL}, ${NEW_TOTAL}, ${ISSUE_DELTA} (line 271) and ${TODAY} (lines 276, 287) are emitted verbatim — the regression PR would show a table reading | Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} | instead of real numbers. The sibling docs-audit-daily.md avoids this by using a plain double-quoted --body "..." (which expands variables).
Note the fix is not a simple unquoting: the body contains backticked paths (`.reports/claude/${TODAY}-audit.md` and `/claude-md-audit-daily`), so an unquoted heredoc would trigger command substitution on those backticks. The suggestion unquotes the delimiter and escapes the backticks so both variables expand and the literal backticks survive.
| --body "$(cat <<'PREOF' | |
| ## CLAUDE.md Audit Regression | |
| The daily CLAUDE.md audit found **more issues than the previous baseline**. | |
| | Metric | Previous | Current | Delta | | |
| |--------|----------|---------|-------| | |
| | Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} | | |
| ### What This Means | |
| New CLAUDE.md quality issues were introduced since the last audit. Review the | |
| audit report at `.reports/claude/${TODAY}-audit.md` for the full remediation | |
| checklist. | |
| ### Next Steps | |
| 1. Review the remediation checklist in the audit report | |
| 2. Fix the new issues (or confirm they are acceptable) | |
| 3. Merge this PR to update the baseline | |
| --- | |
| *Generated by `/claude-md-audit-daily` on ${TODAY}* | |
| PREOF | |
| --body "$(cat <<PREOF | |
| ## CLAUDE.md Audit Regression | |
| The daily CLAUDE.md audit found **more issues than the previous baseline**. | |
| | Metric | Previous | Current | Delta | | |
| |--------|----------|---------|-------| | |
| | Total Issues | ${PREVIOUS_TOTAL} | ${NEW_TOTAL} | +${ISSUE_DELTA} | | |
| ### What This Means | |
| New CLAUDE.md quality issues were introduced since the last audit. Review the | |
| audit report at \`.reports/claude/${TODAY}-audit.md\` for the full remediation | |
| checklist. | |
| ### Next Steps | |
| 1. Review the remediation checklist in the audit report | |
| 2. Fix the new issues (or confirm they are acceptable) | |
| 3. Merge this PR to update the baseline | |
| --- | |
| *Generated by \`/claude-md-audit-daily\` on ${TODAY}* | |
| PREOF |
| If NEW_TOTAL == 0: | ||
| STATUS="CLEAN" | ||
| ACTION="commit-only" | ||
| → Perfect score, commit report as new baseline |
There was a problem hiding this comment.
LOW · correctness — CLEAN status branch is unreachable due to comparison ordering
The decision logic is evaluated top-to-bottom, but the NEW_TOTAL == 0 → CLEAN case is listed last, after the == and < comparisons. When NEW_TOTAL is 0 it is always caught by an earlier branch first: on a clean baseline (PREVIOUS_TOTAL == 0) it matches NEW_TOTAL == PREVIOUS_TOTAL → STABLE; if issues were previously present (PREVIOUS_TOTAL > 0) it matches NEW_TOTAL < PREVIOUS_TOTAL → IMPROVED. So STATUS="CLEAN" is never assigned, even though the <edge_cases> and <success_criteria> sections document CLEAN as a real outcome. Behavior is otherwise harmless (all three map to ACTION="commit-only"), but the status label is misleading. Move the NEW_TOTAL == 0 check to the top of the ordering to make it reachable.
|
Reviewed 2 changed files (2 hunks); ran the verify pass; 2 findings. |
1 participant
Summary
/claude-md-audit-dailymeta-orchestrator for automated daily CLAUDE.md quality auditingcanon/autoclaude.mdto mark all 4 steps of the AutoCLAUDE initiative as completeWhat it does
The daily audit command:
/audit-claude-mdauditFollows the established patterns from
security-audit-dailyanddocs-audit-daily.Test plan
claude "/claude-md-audit-daily"produces a report.reports/claude/*-audit.md🤖 Generated with Claude Code
Summary by CodeRabbit