Add agent upgrade compatibility workflow - #336
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 48b489d9d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| proc = subprocess.run( | ||
| argv, | ||
| cwd=str(repo_root), | ||
| env={ | ||
| **os.environ.copy(), | ||
| "COMPAT_AGENT_REF": agent_ref or "", |
There was a problem hiding this comment.
When agent_ref is supplied for command-backed agents such as OpenClaw, Claude, OpenCode, or Codex, this path still runs the manifest command in the current checkout and only passes the ref through COMPAT_AGENT_REF. I checked repo-wide for COMPAT_AGENT_REF, and nothing consumes it, so dispatching the workflow from main with agent_ref=<upgrade branch> tests main and can falsely report an upgrade as compatible.
Useful? React with 👍 / 👎.
| profile=env.profile, | ||
| session_id="compat-compact", | ||
| message="/compact", | ||
| output_dir=module_dir / "compact-command", |
There was a problem hiding this comment.
In the OpenClaw full/upgrade suite, openclaw.compact passes a new subdirectory here, but run_agent writes agent.stdout.json and agent.stderr.txt directly under output_dir without creating it. When the suite reaches /compact, this raises FileNotFoundError before the module can finish, so the full OpenClaw compatibility lane fails even if the agent behavior is correct.
Useful? React with 👍 / 👎.
48b489d to
6c9c246
Compare
1 participant
Summary
Adds a repo-owned compatibility harness and manual GitHub Action for upgrade checks across the mem9 agent stack:
The latest workflow separates the two upgrade axes:
plugin_ref: the mem9 plugin candidate ref/version/sourcehost_ref: the host agent/CLI candidate version, tag, SHA, orpath:<bin>This lets us test plugin upgrades by themselves and host upgrades by themselves from GitHub Actions.
What changed
compat/with a manifest-driven runner:python3 compat run <module>python3 compat suite <suite>python3 compat matrix <lane>python3 compat agent <agent|all> --lane <plugin-contract|host-smoke|full>compat/manifest.yamlwith plugin metadata, host metadata, default refs, stable/next channels, and lane plans for all six agents.host_refsupport foropenclaw@<version>orpath:<bin>.mem9-ai/mem9-hermes-pluginandmem9-ai/mem9-dify-pluginviaplugin_ref.session_id,scanAll, and store/search request shape..github/workflows/agent-compat.ymlfor manual upgrade checks..github/workflows/compat-framework-checks.ymlandmake compat-testfor harness PR checks.compat/artifacts/**for per-agent logs and result JSON.How to run locally
Fast harness self-test:
Run plugin contract tests:
Run host upgrade smoke tests:
Use local installed CLIs during development:
GitHub Action usage
Manual workflow:
Actions->Agent compatibility->Run workflowInputs:
agent:openclaw,hermes,claude,opencode,codex,dify, oralllane:plugin-contract,host-smoke, orfullplugin_ref: optional plugin branch/tag/SHA/source overridehost_ref: optional host version/tag/SHA/source overridehost_channel:stableornext;nextis non-blockingplugin_source:manifest,local,npm:<version>,path:<path>, orgithub:<repo>@<ref>Plugin upgrade example:
Host upgrade example:
Full matrix example:
GitHub Action configuration
No secrets are required for:
plugin-contractlaneshost-smokehost-smokehost-smokeRequired for OpenClaw and Hermes hosted/host smoke lanes:
OpenClaw also needs one model key:
Optional Action variables for default host refs:
Verified locally
OpenClaw
host_ref path:<bin>was also verified with the fake OpenClaw harness incompat-test. Live OpenClaw/Hermes hosted checks requireMNEMO_BASE_URLand, for OpenClaw, a model key.