Skip to content

Daily AIC guardrail counts completed v0.91.2 runs as zero #66293

Description

@JeffreyCA

Summary

After upgrading to gh-aw v0.91.2, the daily AI Credits guardrail counted completed agent and detection jobs as having no billable usage. Two preceding runs had approximately 48.11 recorded credits, but the guardrail counted zero and allowed another agent to start despite a one-credit daily limit.

Observed example

These two successful runs produced nonzero usage artifacts:

Run Recorded agent and detection credits Guardrail count
37532693758 18.02604 0
37532807067 30.08748 0
Total 48.11352 0

For the next run, GH_AW_DEFAULT_MAX_DAILY_AI_CREDITS was temporarily set to 1. Its activation logs reported reason:"no_billable_jobs" for both preceding runs, followed by:

[daily-workflow-aic] Completed AIC inspection window: {"candidateRunsCount":2,"inspectedRunsCount":2,"countedRunIds":[37532807067,37532693758],"currentAIC":0,"threshold":1,"exceeded":false}

The agent then ran successfully instead of being skipped. The scan artifact also stored both zero-credit values with source:"recorded" and coverage_version:1, making them eligible for reuse by later scans.

The jobs API returns the generated display names Agent and Detection. The released component matcher only recognizes lowercase agent, detection, and evals, so it overlooks these jobs and treats the run as having no billable components.

Reproduction

  1. Compile a PR-triggered agentic workflow with v0.91.2 and let it complete with nonzero agent and detection usage.
  2. Set the daily limit below that recorded usage and trigger another run within 24 hours.
  3. Observe that activation counts the preceding run as zero and allows the agent to execute.

A separate test using actual completed GitHub job metadata also fails because the released accounting code does not recognize the Agent component.

Expected behavior

Completed compiler-generated jobs should contribute their recorded usage to the daily guardrail, regardless of their display names. An agent should be skipped when preceding recorded usage reaches the configured limit. Incorrect zero-credit cache entries should not perpetuate the undercount after accounting is corrected.

Versions

Observed with gh-aw v0.91.2, Copilot CLI 1.0.90, AWF v0.28.31, and GPT-5.6 Sol medium while testing the upgrade in Azure/azure-dev#10345.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions