Skip to content

perf(runtime): add compiled workflow plan cache - #1530

Merged
ShuhaoZhangTony merged 3 commits into
mainfrom
codex/perf-compiled-plan-cache
Aug 15, 2026
Merged

perf(runtime): add compiled workflow plan cache#1530
ShuhaoZhangTony merged 3 commits into
mainfrom
codex/perf-compiled-plan-cache

Conversation

@ShuhaoZhangTony

@ShuhaoZhangTony ShuhaoZhangTony commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a typed structural PlanFingerprint covering DAG, schema, policy, capability, retrieval, resource, and compiler versions;
  • add bounded LRU/TTL and negative-cache behavior with single-flight compilation and metrics;
  • keep request identity, input, evidence, deadline, trace, and cancellation state in an explicit bind step;
  • export the API from sage.runtime and document a business-neutral migration example.

This completes the SAGE runtime portion of #1529: FlowDeclaration now exposes real reusable compilation and request-neutral binding on top of the cache contract.

Validation

  • python3 -m pytest -q src/tests/test_compiled_plan_cache.py src/tests/test_flownet_compiled_plan_integration.py src/tests/test_workflow_product_integration_contract.py — 24 passed;
  • python3 -m ruff check --config tools/ruff.toml ... — passed;
  • python3 tools/maintenance/checks/check_release_tools_boundary.py — total=0, PASS;
  • 20 concurrent identical fingerprints trigger one compile;
  • real minimal FlowDeclaration probe: cold compile 0.202 ms, warm lookup p95 0.037 ms over 2,000 iterations, with one compiled FlowProgram.

The host base Python lacks optional cloudpickle; real FlowNet integration tests were therefore run with the Sage Mate environment that contains the runtime dependency. GitHub install/import smoke and deployment-readiness checks passed in the clean CI environment.

Boundary checklist

  • No dependency from src/sage to tools or sage.tools was added.
  • The implementation is in src/sage/runtime because it is part of the installed runtime API.
  • No repository-maintenance or CI behavior moved into the release package.
  • Default dependencies are unchanged; the implementation uses only the standard library.
  • Roll back if compiler integration demonstrates that cached artifacts retain request-scoped state or changes cold-path semantics.

Closes #1529.

@ShuhaoZhangTony
ShuhaoZhangTony marked this pull request as ready for review August 15, 2026 00:28
@ShuhaoZhangTony
ShuhaoZhangTony merged commit a2f21a8 into main Aug 15, 2026
3 of 4 checks passed
@ShuhaoZhangTony
ShuhaoZhangTony deleted the codex/perf-compiled-plan-cache branch August 15, 2026 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Performance] Add reusable compiled workflow plans for interactive requests

1 participant