Skip to content

Make the agent integrations actually work, and correct the OpenClaw claim - #372

Merged
samrusani merged 3 commits into
mainfrom
fix/agent-facing-payloads-are-callable
Aug 15, 2026
Merged

Make the agent integrations actually work, and correct the OpenClaw claim#372
samrusani merged 3 commits into
mainfrom
fix/agent-facing-payloads-are-callable

Conversation

@samrusani

@samrusani samrusani commented Aug 15, 2026

Copy link
Copy Markdown
Owner

You asked me to test the OpenClaw skill. Three things came out of it, each verified by running software rather than reading about it.

1. I was wrong about OpenClaw, and the claim was already public

I previously concluded OpenClaw does not support MCP and that a plugin would be required. That came from reading docs.openclaw.ai and never opening their source, which was on the machine the whole time.

$ openclaw mcp probe
- alice: 11 tools
$ openclaw mcp doctor
- alice: ok

OpenClaw 2026.8.1 ships @modelcontextprotocol/sdk 1.30.0 as a production dependency with a stdio transport. One config entry under mcp.servers, no plugin, zero diagnostics. Tools arrive prefixed, so alice_recall is exposed as alice__alice_recall.

The false claim had reached the wiki, the landing page copy, and the announcement drafts, where it sat inside the section arguing for our own candour. All corrected.

2. Neither skill pack was loading, on either host

This is the real defect and it hit Hermes too, which is the platform being promoted on. Three variants of the same pack, side by side in OpenClaw's managed skills directory:

DROPPED  alice-project-memory-skill.md   (what we shipped since July)
DROPPED  SKILL.md, no frontmatter
LOADED   SKILL.md + frontmatter          modelVisible=true

Silent at every log level: a file not named SKILL.md is never a candidate, and a file without frontmatter parses to an empty record with zero issues. Hermes uses the same convention. Both packs were inert from the day they were written.

Both are now <skill-name>/SKILL.md with frontmatter, moved with git mv. Copied straight out of the repo into OpenClaw with no edits, both load with eligible=true.

3. Every documented payload was rejected by the server

Running the skill's own loop against the published v0.15.4 artifact:

MCPToolError: tool 'alice_memory_commit' does not accept additional properties: intent

One cause: the packs name the core eleven-tool surface but carry payloads shaped for the legacy alice_vnext_* tools. intent: "explicit_remember" exists only on alice_vnext_commit_memory, and the core tool rejects unknown properties rather than ignoring them.

defect files
intent on a core commit 4
sprint output sends content/output_type/propose_memory to a core tool, omits required raw_text 2
proposal examples omit required title 6 blocks
proposal_type is not a shipped field 4 blocks
all 11 context-pack recipes nest scope/options; the core tool takes them flat 11 blocks
lifecycle sent to alice_vnext_confirm_memory and /vnext queues, neither on a default server 3
"create open loops with alice_open_loops", whose actions are list/close/edit/reopen/snooze 1
two /ABSOLUTE/PATH/TO/AliceBot paths the rename missed 2

4. Our release gate was blind to a hole the landing page split would open

Found while answering a separate question about putting a public page on the same host. Every check in validate_caddyfile is a file-wide substring test, so it proved some block demands a client certificate, not that Alice's block does. Reproduced against the shipped validator: strip the whole tls/client_auth stanza from the Alice block, leave those strings in any other block, and all four mTLS checks pass. CI goes green on a configuration serving the memory API to anyone.

Harmless while the example had one site block. The moment a second one exists it is live, and a hostname split for a landing page is a second block.

Now scoped per block. A public block is still allowed, and there is a test for it, as long as it cannot reach Alice.

Guards

Three new files, each proven non-vacuous by reverting the fix:

guard reverting fails
test_agent_facing_payloads_are_callable.py 33 of 106
test_skill_packs_are_loadable.py 4 of 11
the two Caddy block-scoping mutations both, against the old validator

The payload guard runs each documented payload through _validate_mcp_arguments_against_advertised_schema, the same function the server calls before any handler runs, against default-surface tools only. It caught two defects I had missed while writing it.

End to end

Payloads parsed live out of the shipped markdown rather than retyped, against the published artifact:

[OK] hermes ambient commit    status=confirmation_required
[OK] hermes explicit commit   status=committed
[OK] hermes low-confidence    status=review_required
[OK] openclaw sprint capture  status=imported
[OK] openclaw project commit  status=committed
--- new session, same data dir ---
[OK] recall count=1, count=1 · context_pack memories=1

Every outcome the docs promise now occurs, including the graded ones.

Full unit suite: 4682 passed, 2 skipped.

Testing the OpenClaw skill pack against the published v0.15.4 artifact over stdio found that
its own example payloads do not work. Nine payloads across four files were rejected outright.
Two more files were found by the guard written to stop it recurring.

The root cause is one thing, not nine. The skill packs and their recipe docs name the CORE
eleven-tool surface but still carry payloads shaped for the legacy alice_vnext_* tools. When
the core surface was introduced the tool NAMES were updated and the PAYLOADS were not.

The clearest case: every documented explicit commit carried intent: "explicit_remember", which
exists only on alice_vnext_commit_memory. alice_memory_commit rejects unknown properties rather
than ignoring them, so the flagship example in both packs failed with the opaque
tool_request_failed and no reason. Deleting that one key was the difference between a hard
failure and status=committed.

What was wrong, all reproduced by execution:

- intent on alice_memory_commit, in all four skill files. Legacy-only property.
- the OpenClaw sprint-output payload sent content/output_type/propose_memory to a core tool.
  Those belong to alice_vnext_ingest_agent_output, and the payload also omitted the required
  raw_text. Retargeted at alice_capture, which is review-gated, which is what a generated
  summary wants anyway.
- six proposal examples omitted the required title.
- proposal_type is not a shipped field. The four values used map onto real memory_type enum
  members: project_state, belief, open_loop, contradiction.
- all eleven context-pack recipes nested scope and options. The core alice_context_pack takes
  those fields flat.
- three docs sent agents to alice_vnext_confirm_memory and /vnext review queues for lifecycle
  work. Neither exists on a default server; alice_memory_manage and alice_memory_review do.
- two /ABSOLUTE/PATH/TO/AliceBot paths the rename sweep missed.

This matters more for Hermes than for OpenClaw. Hermes speaks MCP natively and is the platform
this is being promoted on, and its pack was broken the same way.

tests/unit/test_agent_facing_payloads_are_callable.py runs every documented payload through
_validate_mcp_arguments_against_advertised_schema, the same function the server calls before
any handler runs, against tools on the DEFAULT surface. A payload that only works once an
operator sets ALICE_MCP_LEGACY_TOOLS=1 is not something to hand an agent unqualified. Adding a
JSON block without classifying it fails the suite, so a new example has to be named before it
can be checked, and a deleted block cannot leave a phantom entry checking nothing.

Verified non-vacuously: reverting the doc edits fails 33 of 106 cases. The guard found two
defects I had missed while writing it, an unclassified block and a proposal still missing its
title.

End to end on the published artifact, with payloads parsed live out of the shipped markdown
rather than retyped: ambient commit confirmation_required, explicit commit committed,
low-confidence commit review_required, sprint capture imported, project commit committed, and
both memories recalled from a NEW session with context_pack memories=1. Every outcome the docs
promise now actually occurs.

Full unit suite: 4667 passed, 2 skipped.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Sami Rusani added 2 commits August 15, 2026 20:36
…the OpenClaw claim

Two corrections, both established by running the real thing rather than reading its docs.

OpenClaw supports MCP. I previously concluded it did not, from a docs read, and that claim
reached the README-adjacent docs and the launch copy. It is wrong. OpenClaw 2026.8.1 carries
@modelcontextprotocol/sdk 1.30.0 as a production dependency with a stdio transport that spawns
child processes. Pointing it at the published artifact:

    $ openclaw mcp probe
    - alice: 11 tools
    $ openclaw mcp doctor
    - alice: ok

All eleven core tools, zero diagnostics, no plugin, one config entry under mcp.servers. Tools
arrive prefixed with the server name, so alice_recall is exposed as alice__alice_recall.

Neither skill pack loaded. This is the real defect and it hit Hermes too. Three variants of the
same pack, side by side in OpenClaw's managed skills directory:

    DROPPED  alice-project-memory-skill.md   (what we shipped)
    DROPPED  SKILL.md, no frontmatter
    LOADED   SKILL.md + frontmatter          modelVisible=true

The drop is silent at every log level: a file not named SKILL.md is never a candidate, and a
file without frontmatter parses to an empty record with zero issues. Hermes uses the same
convention; its own bundled skills are <skill-name>/SKILL.md with name and description in
frontmatter. We had no way to notice either pack was inert.

Both packs are now <skill-name>/SKILL.md with frontmatter, moved with git mv so history
survives. Copied straight out of the repo into OpenClaw with no edits, both now load with
eligible=true and modelVisible=true.

Also fixed, found while verifying: the OpenClaw pack told agents to "create open loops with
alice_open_loops". That tool's actions are list/close/edit/reopen/snooze, so the instruction
could only ever error. Unresolved work is now committed with memory_type "open_loop". Both
packs also tell the agent to read tool names from the host rather than assume the bare form.

README gains the OpenClaw config block, the CLI equivalent, the tool-prefix note, and a skill
pack section that says to copy the directory rather than the file.

tests/unit/test_skill_packs_are_loadable.py enforces the filename, the frontmatter, a
description long enough to be chosen from a system prompt, and directory/name agreement. It
also asserts against the open-loop create instruction, deriving the action enum from the
shipped schema so it revisits itself if create is ever added.

Verified non-vacuously: restoring the old pack shape fails 4 of 11 cases.

Full unit suite: 4678 passed, 2 skipped.
The deployment gate proved "some block in this file demands a client certificate", not
"the block serving Alice demands one". Every check in validate_caddyfile is a file-wide
substring or directive-set test, which was sound only while the example had exactly one
site block.

Reproduced against the shipped validator: remove the entire tls/client_auth stanza from
the Alice block, leave those same strings in any other block, and all four mTLS checks
pass. CI goes green on a configuration that serves the memory API to anyone who can route
to it. A second block that proxies straight to 127.0.0.1:8000 with no tls directive also
passed.

This was harmless while the packaged example had one block and opens the moment a second
one exists. That is not hypothetical: the owner is adding a public landing page on the
apex while Alice stays behind mutual TLS, which is a hostname split, which is a second
block. The gate would have stopped protecting the property it names at exactly the moment
the property became non-trivial.

validate_caddyfile now splits the file into site blocks by brace depth, finds every block
that can reach an Alice upstream, and requires client_auth, mode require_and_verify, and
the trust_pool path inside each of them. The file-wide checks are unchanged and still run;
this is an additional constraint, not a replacement.

A public block is still allowed, and there is a test for it, as long as it cannot reach
Alice. That is the supported shape for a landing page on the same host.

Verified non-vacuously: the two new mutation tests fail against the previous validator and
pass against this one. A third test asserts the stanza constant still appears verbatim in
the shipped example, so a reformat cannot turn the mutations into no-ops that pass while
removing nothing.

Full unit suite: 4682 passed, 2 skipped.
@samrusani samrusani changed the title Make every payload we hand an agent one the server will actually accept Make the agent integrations actually work, and correct the OpenClaw claim Aug 15, 2026
@samrusani
samrusani merged commit 188fb55 into main Aug 15, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant