Skip to content

ops(e2e): pin module configs + run-prep punch list for COW-1064 dry run - #46

Closed
brunota20 wants to merge 7 commits into
feat/deployment-guide-cow-1030from
feat/e2e-run-config-cow-1064
Closed

ops(e2e): pin module configs + run-prep punch list for COW-1064 dry run#46
brunota20 wants to merge 7 commits into
feat/deployment-guide-cow-1030from
feat/e2e-run-config-cow-1064

Conversation

@brunota20

Copy link
Copy Markdown

Summary

Reconfigures the M3 example modules' manifests to the pinned
identities for the 2026-06-18 COW-1064 E2E dry run (Bruno's test
EOA + Safe on Sepolia) and adds a run-prep companion doc that
captures every copy-paste-able value the operator needs to
drive the on-chain side of the run without re-deriving any UID,
address, or calldata.

Stacks on #45 (COW-1030 deployment guide). Linear: COW-1064.

What changed

Module manifests:

  • modules/examples/stop-loss/module.toml — owner -> test EOA,
    sell_token -> WETH9 Sepolia (was a stale mainnet KNC address
    that would have failed the orderbook accept), buy_token ->
    COW Sepolia, amounts scaled to fit the 0.01 WETH wrap
    budget, trigger_price set above the Sepolia Chainlink mocked
    ETH/USD answer so the strategy fires on block 1.
  • modules/examples/balance-tracker/module.toml — addresses ->
    EOA + Safe, change_threshold lowered 100x so the small E2E
    gas-side transfers show as Warn diffs.

Regression test:

  • cow_1064_e2e_settings_yield_expected_uid in
    modules/examples/stop-loss/src/strategy.rs constructs
    Settings from the exact same constants as the new manifest
    and asserts the resulting build_creation UID against
    0xc2b9cb4...ffffffff. If anything drifts (manifest values,
    EIP-712 type-hash, domain separator), the test fires before
    the run starts, not during.

Run-prep punch list:

  • docs/operations/e2e-cow-1064-prep.md — ~ 280 lines covering
    every pinned identity (10 addresses verified via
    eth_getCode), the OrderUid decomposition, four on-chain
    actions (wrap ETH, setPreSignature + allowance, TWAP
    create() via Safe TX Builder with the full 516-byte
    calldata pinned, EthFlow swap via cow-swap UI), cast
    validation snippets, Python + cargo test re-derivation
    recipes, and a per-run acceptance checklist.

Why no production code changes

The four production code crates are untouched. This PR is the
operational glue between the COW-1064 scaffold (#44) and the
actual run: pinned config + UID + calldata + procedure. The
regression test lands inside the existing stop-loss test
module — the only .rs diff in this PR.

Verification

  • cargo test -p stop-loss --lib -> 8 passed (+1 from this
    PR).
  • All 10 addresses (EOA, Safe, ComposableCoW, TWAP handler,
    EthFlow, GPv2Settlement, GPv2VaultRelayer, WETH, COW,
    Chainlink ETH/USD) verified via eth_getCode > 0 on
    Sepolia at prep time. The domain separator was queried from
    GPv2Settlement directly (not hardcoded against an EIP-712
    re-derivation).
  • The TWAP create() calldata's selector matches
    keccak256("create((address,bytes32,bytes),bool)")[:4] and
    the inner ABI-encode follows the
    ConditionalOrderParams -> TWAP staticInput layout from
    the ComposableCoW + ext/twap upstream contracts.
  • cargo fmt --all --check clean.

AI assistance disclosure

Claude (Opus 4.7, 1M-context) derived the OrderUid + the TWAP
calldata + drafted the run-prep doc. Cross-checks done:

  • The Python UID derivation was confirmed against the actual
    Rust build_creation via the new regression test (matches
    byte-for-byte).
  • Every contract address was queried for code length on
    Sepolia before being pinned in the doc.
  • The TWAP create() selector + staticInput shape were
    cross-checked against the ComposableCoW + TWAP handler
    source.
  • I (Bruno) reviewed every config field, every URL, and the
    acceptance checklist before the run.

…-1064 dry run

Reconfigures the M3 example modules' manifests to the pinned
identities for the 2026-06-18 COW-1064 E2E dry run (Bruno's test
EOA + Safe on Sepolia) and adds a `docs/operations/e2e-cow-1064-
prep.md` companion to the runbook that captures every
copy-paste-able value the operator needs to drive the on-chain
side of the run without re-deriving any UID, address, or
calldata.

## Module config pinning

`modules/examples/stop-loss/module.toml`:
- owner -> 0x7bF140727D27ea64b607E042f1225680B40ECa6A (test EOA)
- sell_token -> WETH9 Sepolia (was a mainnet KNC address — bug
  that would have failed the orderbook accept regardless)
- buy_token -> COW Sepolia (verified on-chain: name="CoW
  Protocol Token", symbol="COW", decimals=18)
- sell_amount -> 0.005 WETH (fits 0.01 WETH wrap budget)
- buy_amount -> 20 COW (conservative quote)
- trigger_price -> $2000 (above the Sepolia Chainlink mocked
  answer ~$1681 so the strategy fires on the first block)

`modules/examples/balance-tracker/module.toml`:
- addresses -> EOA + Safe (was the hardhat default accounts)
- change_threshold -> 0.001 ETH (was 0.1; lower so the small
  E2E gas-side transfers show as Warn diffs)

## OrderUid pinning + regression test

`modules/examples/stop-loss/src/strategy.rs` gains
`cow_1064_e2e_settings_yield_expected_uid`: an integration
test that constructs `Settings` from the exact same constants
as the new manifest and asserts the resulting `build_creation`
UID against:

  0xc2b9cb4ea1ee5a86d8049ac09d8f494bf04cca0a68407285f31e2e6379800be8
                                                                    7bf140727d27ea64b607e042f1225680b40eca6a
                                                                                                            ffffffff

(orderDigest || owner || validTo per packOrderUidParams.)

If anything drifts — manifest values, EIP-712 type-hash,
domain separator — the test fires before the run starts, not
during the run.

## Run-prep punch list

`docs/operations/e2e-cow-1064-prep.md` (~ 280 lines):

1. **Pinned identities table** — every address the runbook
   references (EOA, Safe, ComposableCoW, TWAP handler,
   EthFlow, GPv2Settlement, GPv2VaultRelayer, WETH, COW
   token, domain separator). All verified via `eth_getCode`
   on Sepolia before commit.
2. **Per-module config pinning** — stop-loss + balance-
   tracker effective values in table form.
3. **OrderUid decomposition** — orderDigest (32) + owner (20)
   + validTo (4) breakdown so an operator reading
   `setPreSignature` calldata can sanity-check the UID
   without redoing the EIP-712 math.
4. **Four on-chain actions** — each as a numbered step with
   the exact contract + function + arguments + Etherscan
   write-UI URL:
     - Action 1: wrap 0.01 ETH -> 0.01 WETH9 (optional, only
       for `submitted:` path; `backoff:` works without).
     - Action 2: setPreSignature + WETH allowance to
       GPv2VaultRelayer (optional, paired with action 1).
     - Action 3: TWAP create() via Safe TX Builder; the full
       516-byte calldata pinned verbatim (selector 0x6bfae1ca
       + tuple-encoded ConditionalOrderParams + dispatch=true).
     - Action 4: EthFlow swap via cow-swap UI on Sepolia
       (UI-driven for the quote endpoint hit; calldata
       fallback link if UI flakes).
5. **Validation snippets** — `cast` invocations to check EOA
   + Safe balances, WETH balance, allowance,
   `preSignature(bytes)` lookup, and a `journalctl + jq`
   one-liner that tails per-module terminal markers in real
   time.
6. **Re-derivation recipes** — Python + `cargo test`
   commands to regenerate every pinned value if config drift
   ever forces a re-run with different identities.
7. **Per-run acceptance checklist** — 9 box-checks that
   double-pin section 7 of the e2e-report template, scoped
   to THIS specific run.

## Workspace impact

- `cargo test -p stop-loss --lib` -> 8 passed (was 7; +1
  for the new pinning test).
- `cargo fmt --all --check` clean.
- No production-code changes outside the test module.

Linear: COW-1064. Twelfth M4 deliverable; stacks on #45.
@linear-code

linear-code Bot commented Jun 18, 2026

Copy link
Copy Markdown

COW-1064

Three-step automation that wraps the COW-1064 runbook +
prep punch list into shell scripts. Operator workflow
collapses to:

  cp scripts/env-template scripts/.env  &&  $EDITOR scripts/.env
  scripts/e2e-run.sh
  scripts/e2e-onchain.sh
  ## … 4-6 h …
  scripts/e2e-finish.sh

Secrets stay on disk. `scripts/.env` is gitignored; the
engine config with embedded RPC URL is rendered into a
gitignored `engine.e2e.local.toml` at boot time; the
PK is read from `scripts/.env` by `cast send` and never
echoed.

## Files

- `scripts/env-template` (committed): every variable the
  scripts read, with comments. Operator copies to
  `scripts/.env` and fills in.
- `scripts/lib.sh`: shared bash helpers — `log/warn/die`,
  `load_env`, `render_engine_config`, `state_value`, and
  the pinned address constants (EOA, Safe, ComposableCoW,
  TWAP handler, EthFlow, GPv2Settlement, GPv2VaultRelayer,
  WETH, COW, expected OrderUid).
- `scripts/e2e-run.sh`: renders engine config, cleans
  data/e2e, builds 5 modules + engine in release, launches
  via nohup, waits ≤ 60 s for `supervisor ready modules=5
  chains=1`, snapshots `metrics-start-<ts>.txt`. Persists
  PID + log path + start-ISO into `scripts/.state`.
- `scripts/e2e-onchain.sh`: derives EOA from
  `OPERATOR_PRIVATE_KEY` + asserts it matches the pinned
  test EOA + asserts balance ≥ 0.02 ETH; then `cast send`s:
    1. ComposableCoW.create() with the 516-byte pinned
       TWAP calldata → `ConditionalOrderCreated` →
       twap-monitor `watch:`.
    2. EthFlow.createOrder() with the tuple built from the
       cow.fi `/api/v1/quote` response (via
       `_ethflow_quote.py`) → `OrderPlacement` →
       ethflow-watcher `submitted:`.
  If `RUN_OPTIONAL_PRESIGN=1` also runs WETH wrap +
  setPreSignature + GPv2VaultRelayer approval (for
  stop-loss on-chain settlement; the `submitted:{uid}`
  marker is produced regardless). Each tx hash appended to
  `scripts/.state` as `TX_<KIND>=<hash>`.
- `scripts/_ethflow_quote.py`: small Python helper that
  POSTs to cow.fi Sepolia, gets feeAmount + quoteId +
  validTo + buyAmount, ABI-encodes the EthFlowOrder.Data
  tuple, and prints the calldata + msg.value for the
  shell script to consume.
- `scripts/e2e-finish.sh`: snapshots
  `metrics-end-<ts>.txt`, sends SIGINT, waits ≤ 30 s for
  `graceful shutdown complete` in the log (COW-1072 path),
  escalates to SIGKILL after 30 s, then invokes the report
  generator.
- `scripts/e2e-report-gen.sh`: parses the JSON-formatted
  engine log + metrics snapshots + state file into the
  e2e-report template's 9 sections. Auto-derives chain
  delta, per-module first marker, every `shepherd_*`
  counter / histogram delta, ERROR/trapped/poisoned
  tallies, and the per-row acceptance checklist
  (block-delta ≥ 1500, all-5-markers, zero traps, zero
  poison, zero ERROR, TWAP+EthFlow txs present). Writes
  `docs/operations/e2e-reports/e2e-report-<date>.md` ready
  for operator review.
- `scripts/README.md`: one-time setup, run sequence,
  troubleshooting table, re-run recipe.

## .gitignore additions

```
*.local.toml          # engine config rendered with embedded RPC key
scripts/.state        # run-state cache (PIDs, paths, tx hashes)
scripts/.env          # operator secrets (redundant with the existing .env / .env.* rules but explicit)
docs/operations/e2e-reports/engine-*.log
docs/operations/e2e-reports/metrics-*.txt
```

The auto-generated `e2e-report-<date>.md` is NOT gitignored
— operator reviews + commits manually with `git add -f` (the
report belongs in history; the raw log + metrics dumps
don't).

## Why a separate render step

`engine.e2e.toml` is committed with a public-placeholder RPC URL
(`wss://ethereum-sepolia-rpc.publicnode.com`); `e2e-run.sh`
substitutes `RPC_URL_SEPOLIA` from `.env` into a local file
`engine.e2e.local.toml` (gitignored via `*.local.toml`) and
points the engine at the local file. This means:

- No secret ever lands in `git diff`.
- The committed config still boots cleanly (against the public
  endpoint) for anyone cloning the repo who doesn't have a paid
  RPC key.
- The render step is idempotent — re-running `e2e-run.sh`
  always overwrites the local file.

## Verification

- `bash -n` syntax check on all 5 shell scripts: clean.
- `python3 -c "ast.parse(...)"` on `_ethflow_quote.py`: clean.
- `render_engine_config` smoke: produced `engine.e2e.local.toml`
  with the rpc_url line correctly substituted; diff showed
  exactly one line changed.

Linear: COW-1064. Stacks on the existing PR #46.
`scripts/e2e-run.sh` was grepping for the pretty-printed
`supervisor ready modules=5 chains=1` flat string. Without
`--pretty-logs` (which production-shape JSON deliberately
omits) the engine emits
  {"message":"supervisor ready","modules":5,"chains":1,...}
so the grep never matched and the script died at the 60 s
deadline even though the engine was already healthy and
dispatching blocks (the nohup'd engine stayed alive
detached; the wrapper just couldn't see it).

Fix: extended the grep to two JSON-field-order alternatives
(`modules` before `chains` and vice versa, since the JSON
serialiser does not guarantee field order across releases).
Bumped the deadline to 90 s because cold-start of the wasm
component compile + first RPC handshake on a paid endpoint
can comfortably take 30-40 s on a fresh checkout.

Linear: COW-1064 (run-prep regression caught live during the
2026-06-18 dry run).
macOS ships /usr/bin/bash at version 3.2.57 due to GPLv3
licensing; `${var,,}` lowercase expansion is bash 4+ only.
The EOA-match check died with `bad substitution` on first
invocation against the live Sepolia run.

Routed both sides of the comparison through `tr '[:upper:]'
'[:lower:]'` which is POSIX-portable.

Grepped the rest of scripts/ for other `${var,,}` / `${var^^}`
constructs — none found, so this was the only impacted site.

Linear: COW-1064 (run-prep regression caught live).
… (COW-1064)

Two fixes caught live during the 2026-06-18 dry run:

1. `_ethflow_quote.py` imports eth_abi + eth_utils +
   eth_hash; these are not in the Python stdlib and the
   script was failing with `ModuleNotFoundError: eth_abi`
   after the TWAP tx had already landed on Sepolia. Added
   a pre-flight `python3 -c 'import eth_abi, eth_utils,
   eth_hash.auto'` at the top of e2e-onchain.sh that fails
   loudly with the exact `pip3 install` command the
   operator needs.

2. Re-running e2e-onchain.sh after a partial failure
   would re-submit the TWAP create() (different nonce →
   new tx → same salt → ComposableCoW reverts) AND
   re-fetch a new EthFlow quote (new feeAmount/quoteId →
   new tx → wastes ETH on duplicate orders). Added
   idempotency: each action is wrapped in
   `if existing="$(state_value TX_*)"; then skip`, so the
   script picks up exactly where it left off using the
   tx hashes already persisted in scripts/.state.

   Acceptance: the dry run had TX_TWAP already in .state
   (manual recovery write); re-running now skips TWAP and
   only attempts EthFlow.

Linear: COW-1064 (run-prep regressions caught live).
The CoW orderbook's `/quote` endpoint rejects the native-ETH
sentinel `0xEeee…EEeE` with `InvalidNativeSellToken`. EthFlow
orders are still quoted with the *wrapped* form (WETH9
Sepolia) as the sell side; the EthFlow contract itself does
the wrap from msg.value on `createOrder`.

Verified end-to-end: `python3 _ethflow_quote.py <EOA>
5000000000000000` returns a 292-byte calldata + VALUE_WEI on
the live Sepolia orderbook (fee_amount ≈ 0.000308 ETH,
buy_amount ≈ 0.192 COW, quote_id 1519204).

Linear: COW-1064.
…ns (COW-1064)

tracing-subscriber's JSON formatter writes `message` /
`module` / `block_number` / `target` at the top level of the
event object (no nested `fields`); the parser was looking
inside `fields` and finding nothing. Added an
`event_field(ev, key)` helper that checks both top-level and
nested-fields shapes.

Replaced the single substring list with a per-module pattern
map, derived from the host.log() call sites inside
modules/*/src/strategy.rs. Specifically:

  twap-monitor    -> "watch:", "indexed watch:", "poll watch:"
  ethflow-watcher -> "ethflow submitted", "ethflow backoff",
                     "ethflow dropped", "already submitted"
  price-alert     -> "TRIGGERED"
  balance-tracker -> "changed +", "changed -"  (per-block
                     "0x<addr> changed +N wei ..." diff log)
  stop-loss       -> "TRIGGERED", "retry on next block",
                     "stop-loss submitted", "stop-loss dropped",
                     "already submitted", "submitted:"

Verified against the live 2026-06-18 dry run's engine log:
all 5 modules surface ≥ 1 terminal marker.

Linear: COW-1064 (run-prep regression caught live during the
T+12-min mark of the run).
@brunota20

Copy link
Copy Markdown
Author

Work landed via dev/m4-base creation @ 64cb6d8 (M4 production hardening + PR #66 rust-idiomatic compliance squashed). The HEAD commit of this PR is now an ancestor of dev/m4-base. Closing as merged-by-ancestor.

@brunota20 brunota20 closed this Jun 24, 2026
brunota20 added a commit that referenced this pull request Jun 25, 2026
Three-step automation that wraps the COW-1064 runbook +
prep punch list into shell scripts. Operator workflow
collapses to:

  cp scripts/env-template scripts/.env  &&  $EDITOR scripts/.env
  scripts/e2e-run.sh
  scripts/e2e-onchain.sh
  ## … 4-6 h …
  scripts/e2e-finish.sh

Secrets stay on disk. `scripts/.env` is gitignored; the
engine config with embedded RPC URL is rendered into a
gitignored `engine.e2e.local.toml` at boot time; the
PK is read from `scripts/.env` by `cast send` and never
echoed.

## Files

- `scripts/env-template` (committed): every variable the
  scripts read, with comments. Operator copies to
  `scripts/.env` and fills in.
- `scripts/lib.sh`: shared bash helpers — `log/warn/die`,
  `load_env`, `render_engine_config`, `state_value`, and
  the pinned address constants (EOA, Safe, ComposableCoW,
  TWAP handler, EthFlow, GPv2Settlement, GPv2VaultRelayer,
  WETH, COW, expected OrderUid).
- `scripts/e2e-run.sh`: renders engine config, cleans
  data/e2e, builds 5 modules + engine in release, launches
  via nohup, waits ≤ 60 s for `supervisor ready modules=5
  chains=1`, snapshots `metrics-start-<ts>.txt`. Persists
  PID + log path + start-ISO into `scripts/.state`.
- `scripts/e2e-onchain.sh`: derives EOA from
  `OPERATOR_PRIVATE_KEY` + asserts it matches the pinned
  test EOA + asserts balance ≥ 0.02 ETH; then `cast send`s:
    1. ComposableCoW.create() with the 516-byte pinned
       TWAP calldata → `ConditionalOrderCreated` →
       twap-monitor `watch:`.
    2. EthFlow.createOrder() with the tuple built from the
       cow.fi `/api/v1/quote` response (via
       `_ethflow_quote.py`) → `OrderPlacement` →
       ethflow-watcher `submitted:`.
  If `RUN_OPTIONAL_PRESIGN=1` also runs WETH wrap +
  setPreSignature + GPv2VaultRelayer approval (for
  stop-loss on-chain settlement; the `submitted:{uid}`
  marker is produced regardless). Each tx hash appended to
  `scripts/.state` as `TX_<KIND>=<hash>`.
- `scripts/_ethflow_quote.py`: small Python helper that
  POSTs to cow.fi Sepolia, gets feeAmount + quoteId +
  validTo + buyAmount, ABI-encodes the EthFlowOrder.Data
  tuple, and prints the calldata + msg.value for the
  shell script to consume.
- `scripts/e2e-finish.sh`: snapshots
  `metrics-end-<ts>.txt`, sends SIGINT, waits ≤ 30 s for
  `graceful shutdown complete` in the log (COW-1072 path),
  escalates to SIGKILL after 30 s, then invokes the report
  generator.
- `scripts/e2e-report-gen.sh`: parses the JSON-formatted
  engine log + metrics snapshots + state file into the
  e2e-report template's 9 sections. Auto-derives chain
  delta, per-module first marker, every `shepherd_*`
  counter / histogram delta, ERROR/trapped/poisoned
  tallies, and the per-row acceptance checklist
  (block-delta ≥ 1500, all-5-markers, zero traps, zero
  poison, zero ERROR, TWAP+EthFlow txs present). Writes
  `docs/operations/e2e-reports/e2e-report-<date>.md` ready
  for operator review.
- `scripts/README.md`: one-time setup, run sequence,
  troubleshooting table, re-run recipe.

## .gitignore additions

```
*.local.toml          # engine config rendered with embedded RPC key
scripts/.state        # run-state cache (PIDs, paths, tx hashes)
scripts/.env          # operator secrets (redundant with the existing .env / .env.* rules but explicit)
docs/operations/e2e-reports/engine-*.log
docs/operations/e2e-reports/metrics-*.txt
```

The auto-generated `e2e-report-<date>.md` is NOT gitignored
— operator reviews + commits manually with `git add -f` (the
report belongs in history; the raw log + metrics dumps
don't).

## Why a separate render step

`engine.e2e.toml` is committed with a public-placeholder RPC URL
(`wss://ethereum-sepolia-rpc.publicnode.com`); `e2e-run.sh`
substitutes `RPC_URL_SEPOLIA` from `.env` into a local file
`engine.e2e.local.toml` (gitignored via `*.local.toml`) and
points the engine at the local file. This means:

- No secret ever lands in `git diff`.
- The committed config still boots cleanly (against the public
  endpoint) for anyone cloning the repo who doesn't have a paid
  RPC key.
- The render step is idempotent — re-running `e2e-run.sh`
  always overwrites the local file.

## Verification

- `bash -n` syntax check on all 5 shell scripts: clean.
- `python3 -c "ast.parse(...)"` on `_ethflow_quote.py`: clean.
- `render_engine_config` smoke: produced `engine.e2e.local.toml`
  with the rpc_url line correctly substituted; diff showed
  exactly one line changed.

Linear: COW-1064. Stacks on the existing PR #46.
brunota20 added a commit that referenced this pull request Jun 25, 2026
…okup (COW-1074)

Closes the gap surfaced by the COW-1064 dry run (2026-06-18):
TWAP orders created through cow-swap UI sign with a non-empty
`appData` hash pointing at a richer JSON document (partner-id,
slippage settings, quote-id). twap-monitor hard-coded
`EMPTY_APP_DATA_JSON` when assembling `OrderCreation`, so the
orderbook rejected every submit with `invalid OrderCreation:
app_data JSON digest does not match signed app_data hash` and
the watch sat in retry-loop forever.

The WIT already exposes `cow-api::request(method, path, body)`
as a generic REST passthrough. We surface that capability on
the SDK trait, wrap it in a typed helper, and use the helper
from the strategy. No new host imports, no WIT ABI change, no
forced rebuild of unrelated modules.

Extended `CowApiHost` with:
```rust
fn cow_api_request(
    &self,
    chain_id: u64,
    method: &str,
    path: &str,
    body: Option<&str>,
) -> Result<String, HostError>;
```

404 responses surface as `HostError { code: 404, kind:
Unavailable }` so callers can distinguish "orderbook does not
have this resource" from genuine upstream failures without
introducing a new `HostErrorKind` variant (the existing enum
is `non_exhaustive`, but adding a variant on the WIT side
would still need an ABI bump).

`resolve_app_data(host, chain_id, hash) -> Result<String,
HostError>` with:
- Short-circuits `EMPTY_APP_DATA_HASH` (`keccak256("{}")`)
  to `EMPTY_APP_DATA_JSON` (`"{}"`) — no host call needed.
- Otherwise GETs `/api/v1/app_data/{hex_hash}` and pulls
  the `fullAppData` field out of the orderbook's envelope
  shape (`{"fullAppData": "<JSON string>", ...}`).
- 5 unit tests pinning the short-circuit, the success path,
  the unexpected-shape fall-through, the 404 propagation,
  and the hex encoder.

Extended `MockCowApi` with:
- `respond_to_request_for(method, path, result)`: per-key
  programmable response.
- `respond_to_request(result)`: catch-all default.
- `request_calls()`: records the (chain_id, method, path,
  body) tuple for every invocation.

The existing `respond` / `calls()` / `submit_order` surface
is unchanged.

`modules/twap-monitor/src/lib.rs`,
`modules/ethflow-watcher/src/lib.rs`,
`modules/examples/price-alert/src/lib.rs`, and
`modules/examples/stop-loss/src/lib.rs` each gained the
trivial 8-line forwarder to the generated
`cow_api::request` binding. Example modules implement
`CowApiHost` purely for the `Host` blanket-impl supertrait
even though some don't actively submit orders — the impl
is symmetrically extended.

`build_order_creation` now takes the resolved
`app_data_json` as an explicit parameter (was hard-coded to
`EMPTY_APP_DATA_JSON`). The resolution itself is lifted into
the caller `submit_ready`, which calls
`shepherd_sdk::cow::resolve_app_data` before assembling the
`OrderCreation`. Two graceful-fallback branches:
- `err.code == 404` → log Warn "appData hash not mirrored
  on orderbook" + leave the watch in place. Operators can
  re-trigger by pinning the document via a future
  orderbook PUT or by re-creating the order with empty
  appData.
- Any other resolver error → log Warn "appData resolve
  failed" + leave the watch. Future retry on the next
  block re-attempts the lookup.

Two new strategy tests:
- `poll_ready_resolves_non_empty_app_data_then_submits`:
  programs MockHost with a known JSON + its hash on the
  order, asserts the full resolve → submit → `submitted:`
  marker flow.
- `poll_ready_skips_submit_when_app_data_hash_not_mirrored`:
  programs MockHost to 404, asserts no submit attempt, no
  `submitted:` / `dropped:` markers, Warn log line present.

Plus one updated test
(`build_order_creation_accepts_matching_non_empty_app_data`)
that pins the new "matching hash → JSON" success path
directly on `build_order_creation`.

- `cargo test --workspace` → 13 + 12 + 16 + 32 + 8 + 8 +
  61 (engine) + 23 (twap-monitor) + 7 doctests + 1
  integration = 181 tests passing (was 174; +5 SDK +2
  twap-monitor).
- `cargo clippy --all-targets --workspace -- -D warnings`
  clean.
- `cargo fmt --all --check` clean.
- All 4 production module .wasm artefacts build cleanly
  with the new SDK trait.

- No WIT changes. Modules built against the prior SDK
  trait will fail to compile (the new method is required),
  but the WIT-generated wasm-side surface is bit-identical.
- No host-impl changes (`crates/nexum-engine/src/host/
  impls/cow_api.rs`). The host already implements `request`
  for the wit-bindgen binding.
- No metric surface drift. The orderbook lookup goes
  through the same `shepherd_cow_api_*` counters via the
  existing `request` path.

Linear: COW-1074. Stacks on the COW-1064 run-config branch
(#46). Validated locally end-to-end via `cargo test
--workspace`; live validation against the running engine
will happen on the next COW-1064 dry run (engine restart
required to pick up the rebuilt modules).
brunota20 added a commit that referenced this pull request Jun 25, 2026
Three-step automation that wraps the COW-1064 runbook +
prep punch list into shell scripts. Operator workflow
collapses to:

  cp scripts/env-template scripts/.env  &&  $EDITOR scripts/.env
  scripts/e2e-run.sh
  scripts/e2e-onchain.sh
  ## … 4-6 h …
  scripts/e2e-finish.sh

Secrets stay on disk. `scripts/.env` is gitignored; the
engine config with embedded RPC URL is rendered into a
gitignored `engine.e2e.local.toml` at boot time; the
PK is read from `scripts/.env` by `cast send` and never
echoed.

## Files

- `scripts/env-template` (committed): every variable the
  scripts read, with comments. Operator copies to
  `scripts/.env` and fills in.
- `scripts/lib.sh`: shared bash helpers — `log/warn/die`,
  `load_env`, `render_engine_config`, `state_value`, and
  the pinned address constants (EOA, Safe, ComposableCoW,
  TWAP handler, EthFlow, GPv2Settlement, GPv2VaultRelayer,
  WETH, COW, expected OrderUid).
- `scripts/e2e-run.sh`: renders engine config, cleans
  data/e2e, builds 5 modules + engine in release, launches
  via nohup, waits ≤ 60 s for `supervisor ready modules=5
  chains=1`, snapshots `metrics-start-<ts>.txt`. Persists
  PID + log path + start-ISO into `scripts/.state`.
- `scripts/e2e-onchain.sh`: derives EOA from
  `OPERATOR_PRIVATE_KEY` + asserts it matches the pinned
  test EOA + asserts balance ≥ 0.02 ETH; then `cast send`s:
    1. ComposableCoW.create() with the 516-byte pinned
       TWAP calldata → `ConditionalOrderCreated` →
       twap-monitor `watch:`.
    2. EthFlow.createOrder() with the tuple built from the
       cow.fi `/api/v1/quote` response (via
       `_ethflow_quote.py`) → `OrderPlacement` →
       ethflow-watcher `submitted:`.
  If `RUN_OPTIONAL_PRESIGN=1` also runs WETH wrap +
  setPreSignature + GPv2VaultRelayer approval (for
  stop-loss on-chain settlement; the `submitted:{uid}`
  marker is produced regardless). Each tx hash appended to
  `scripts/.state` as `TX_<KIND>=<hash>`.
- `scripts/_ethflow_quote.py`: small Python helper that
  POSTs to cow.fi Sepolia, gets feeAmount + quoteId +
  validTo + buyAmount, ABI-encodes the EthFlowOrder.Data
  tuple, and prints the calldata + msg.value for the
  shell script to consume.
- `scripts/e2e-finish.sh`: snapshots
  `metrics-end-<ts>.txt`, sends SIGINT, waits ≤ 30 s for
  `graceful shutdown complete` in the log (COW-1072 path),
  escalates to SIGKILL after 30 s, then invokes the report
  generator.
- `scripts/e2e-report-gen.sh`: parses the JSON-formatted
  engine log + metrics snapshots + state file into the
  e2e-report template's 9 sections. Auto-derives chain
  delta, per-module first marker, every `shepherd_*`
  counter / histogram delta, ERROR/trapped/poisoned
  tallies, and the per-row acceptance checklist
  (block-delta ≥ 1500, all-5-markers, zero traps, zero
  poison, zero ERROR, TWAP+EthFlow txs present). Writes
  `docs/operations/e2e-reports/e2e-report-<date>.md` ready
  for operator review.
- `scripts/README.md`: one-time setup, run sequence,
  troubleshooting table, re-run recipe.

## .gitignore additions

```
*.local.toml          # engine config rendered with embedded RPC key
scripts/.state        # run-state cache (PIDs, paths, tx hashes)
scripts/.env          # operator secrets (redundant with the existing .env / .env.* rules but explicit)
docs/operations/e2e-reports/engine-*.log
docs/operations/e2e-reports/metrics-*.txt
```

The auto-generated `e2e-report-<date>.md` is NOT gitignored
— operator reviews + commits manually with `git add -f` (the
report belongs in history; the raw log + metrics dumps
don't).

## Why a separate render step

`engine.e2e.toml` is committed with a public-placeholder RPC URL
(`wss://ethereum-sepolia-rpc.publicnode.com`); `e2e-run.sh`
substitutes `RPC_URL_SEPOLIA` from `.env` into a local file
`engine.e2e.local.toml` (gitignored via `*.local.toml`) and
points the engine at the local file. This means:

- No secret ever lands in `git diff`.
- The committed config still boots cleanly (against the public
  endpoint) for anyone cloning the repo who doesn't have a paid
  RPC key.
- The render step is idempotent — re-running `e2e-run.sh`
  always overwrites the local file.

## Verification

- `bash -n` syntax check on all 5 shell scripts: clean.
- `python3 -c "ast.parse(...)"` on `_ethflow_quote.py`: clean.
- `render_engine_config` smoke: produced `engine.e2e.local.toml`
  with the rpc_url line correctly substituted; diff showed
  exactly one line changed.

Linear: COW-1064. Stacks on the existing PR #46.
brunota20 added a commit that referenced this pull request Jun 25, 2026
…okup (COW-1074)

Closes the gap surfaced by the COW-1064 dry run (2026-06-18):
TWAP orders created through cow-swap UI sign with a non-empty
`appData` hash pointing at a richer JSON document (partner-id,
slippage settings, quote-id). twap-monitor hard-coded
`EMPTY_APP_DATA_JSON` when assembling `OrderCreation`, so the
orderbook rejected every submit with `invalid OrderCreation:
app_data JSON digest does not match signed app_data hash` and
the watch sat in retry-loop forever.

The WIT already exposes `cow-api::request(method, path, body)`
as a generic REST passthrough. We surface that capability on
the SDK trait, wrap it in a typed helper, and use the helper
from the strategy. No new host imports, no WIT ABI change, no
forced rebuild of unrelated modules.

Extended `CowApiHost` with:
```rust
fn cow_api_request(
    &self,
    chain_id: u64,
    method: &str,
    path: &str,
    body: Option<&str>,
) -> Result<String, HostError>;
```

404 responses surface as `HostError { code: 404, kind:
Unavailable }` so callers can distinguish "orderbook does not
have this resource" from genuine upstream failures without
introducing a new `HostErrorKind` variant (the existing enum
is `non_exhaustive`, but adding a variant on the WIT side
would still need an ABI bump).

`resolve_app_data(host, chain_id, hash) -> Result<String,
HostError>` with:
- Short-circuits `EMPTY_APP_DATA_HASH` (`keccak256("{}")`)
  to `EMPTY_APP_DATA_JSON` (`"{}"`) — no host call needed.
- Otherwise GETs `/api/v1/app_data/{hex_hash}` and pulls
  the `fullAppData` field out of the orderbook's envelope
  shape (`{"fullAppData": "<JSON string>", ...}`).
- 5 unit tests pinning the short-circuit, the success path,
  the unexpected-shape fall-through, the 404 propagation,
  and the hex encoder.

Extended `MockCowApi` with:
- `respond_to_request_for(method, path, result)`: per-key
  programmable response.
- `respond_to_request(result)`: catch-all default.
- `request_calls()`: records the (chain_id, method, path,
  body) tuple for every invocation.

The existing `respond` / `calls()` / `submit_order` surface
is unchanged.

`modules/twap-monitor/src/lib.rs`,
`modules/ethflow-watcher/src/lib.rs`,
`modules/examples/price-alert/src/lib.rs`, and
`modules/examples/stop-loss/src/lib.rs` each gained the
trivial 8-line forwarder to the generated
`cow_api::request` binding. Example modules implement
`CowApiHost` purely for the `Host` blanket-impl supertrait
even though some don't actively submit orders — the impl
is symmetrically extended.

`build_order_creation` now takes the resolved
`app_data_json` as an explicit parameter (was hard-coded to
`EMPTY_APP_DATA_JSON`). The resolution itself is lifted into
the caller `submit_ready`, which calls
`shepherd_sdk::cow::resolve_app_data` before assembling the
`OrderCreation`. Two graceful-fallback branches:
- `err.code == 404` → log Warn "appData hash not mirrored
  on orderbook" + leave the watch in place. Operators can
  re-trigger by pinning the document via a future
  orderbook PUT or by re-creating the order with empty
  appData.
- Any other resolver error → log Warn "appData resolve
  failed" + leave the watch. Future retry on the next
  block re-attempts the lookup.

Two new strategy tests:
- `poll_ready_resolves_non_empty_app_data_then_submits`:
  programs MockHost with a known JSON + its hash on the
  order, asserts the full resolve → submit → `submitted:`
  marker flow.
- `poll_ready_skips_submit_when_app_data_hash_not_mirrored`:
  programs MockHost to 404, asserts no submit attempt, no
  `submitted:` / `dropped:` markers, Warn log line present.

Plus one updated test
(`build_order_creation_accepts_matching_non_empty_app_data`)
that pins the new "matching hash → JSON" success path
directly on `build_order_creation`.

- `cargo test --workspace` → 13 + 12 + 16 + 32 + 8 + 8 +
  61 (engine) + 23 (twap-monitor) + 7 doctests + 1
  integration = 181 tests passing (was 174; +5 SDK +2
  twap-monitor).
- `cargo clippy --all-targets --workspace -- -D warnings`
  clean.
- `cargo fmt --all --check` clean.
- All 4 production module .wasm artefacts build cleanly
  with the new SDK trait.

- No WIT changes. Modules built against the prior SDK
  trait will fail to compile (the new method is required),
  but the WIT-generated wasm-side surface is bit-identical.
- No host-impl changes (`crates/nexum-engine/src/host/
  impls/cow_api.rs`). The host already implements `request`
  for the wit-bindgen binding.
- No metric surface drift. The orderbook lookup goes
  through the same `shepherd_cow_api_*` counters via the
  existing `request` path.

Linear: COW-1074. Stacks on the COW-1064 run-config branch
(#46). Validated locally end-to-end via `cargo test
--workspace`; live validation against the running engine
will happen on the next COW-1064 dry run (engine restart
required to pick up the rebuilt modules).
brunota20 added a commit that referenced this pull request Jun 25, 2026
Three-step automation that wraps the COW-1064 runbook +
prep punch list into shell scripts. Operator workflow
collapses to:

  cp scripts/env-template scripts/.env  &&  $EDITOR scripts/.env
  scripts/e2e-run.sh
  scripts/e2e-onchain.sh
  ## … 4-6 h …
  scripts/e2e-finish.sh

Secrets stay on disk. `scripts/.env` is gitignored; the
engine config with embedded RPC URL is rendered into a
gitignored `engine.e2e.local.toml` at boot time; the
PK is read from `scripts/.env` by `cast send` and never
echoed.

## Files

- `scripts/env-template` (committed): every variable the
  scripts read, with comments. Operator copies to
  `scripts/.env` and fills in.
- `scripts/lib.sh`: shared bash helpers — `log/warn/die`,
  `load_env`, `render_engine_config`, `state_value`, and
  the pinned address constants (EOA, Safe, ComposableCoW,
  TWAP handler, EthFlow, GPv2Settlement, GPv2VaultRelayer,
  WETH, COW, expected OrderUid).
- `scripts/e2e-run.sh`: renders engine config, cleans
  data/e2e, builds 5 modules + engine in release, launches
  via nohup, waits ≤ 60 s for `supervisor ready modules=5
  chains=1`, snapshots `metrics-start-<ts>.txt`. Persists
  PID + log path + start-ISO into `scripts/.state`.
- `scripts/e2e-onchain.sh`: derives EOA from
  `OPERATOR_PRIVATE_KEY` + asserts it matches the pinned
  test EOA + asserts balance ≥ 0.02 ETH; then `cast send`s:
    1. ComposableCoW.create() with the 516-byte pinned
       TWAP calldata → `ConditionalOrderCreated` →
       twap-monitor `watch:`.
    2. EthFlow.createOrder() with the tuple built from the
       cow.fi `/api/v1/quote` response (via
       `_ethflow_quote.py`) → `OrderPlacement` →
       ethflow-watcher `submitted:`.
  If `RUN_OPTIONAL_PRESIGN=1` also runs WETH wrap +
  setPreSignature + GPv2VaultRelayer approval (for
  stop-loss on-chain settlement; the `submitted:{uid}`
  marker is produced regardless). Each tx hash appended to
  `scripts/.state` as `TX_<KIND>=<hash>`.
- `scripts/_ethflow_quote.py`: small Python helper that
  POSTs to cow.fi Sepolia, gets feeAmount + quoteId +
  validTo + buyAmount, ABI-encodes the EthFlowOrder.Data
  tuple, and prints the calldata + msg.value for the
  shell script to consume.
- `scripts/e2e-finish.sh`: snapshots
  `metrics-end-<ts>.txt`, sends SIGINT, waits ≤ 30 s for
  `graceful shutdown complete` in the log (COW-1072 path),
  escalates to SIGKILL after 30 s, then invokes the report
  generator.
- `scripts/e2e-report-gen.sh`: parses the JSON-formatted
  engine log + metrics snapshots + state file into the
  e2e-report template's 9 sections. Auto-derives chain
  delta, per-module first marker, every `shepherd_*`
  counter / histogram delta, ERROR/trapped/poisoned
  tallies, and the per-row acceptance checklist
  (block-delta ≥ 1500, all-5-markers, zero traps, zero
  poison, zero ERROR, TWAP+EthFlow txs present). Writes
  `docs/operations/e2e-reports/e2e-report-<date>.md` ready
  for operator review.
- `scripts/README.md`: one-time setup, run sequence,
  troubleshooting table, re-run recipe.

## .gitignore additions

```
*.local.toml          # engine config rendered with embedded RPC key
scripts/.state        # run-state cache (PIDs, paths, tx hashes)
scripts/.env          # operator secrets (redundant with the existing .env / .env.* rules but explicit)
docs/operations/e2e-reports/engine-*.log
docs/operations/e2e-reports/metrics-*.txt
```

The auto-generated `e2e-report-<date>.md` is NOT gitignored
— operator reviews + commits manually with `git add -f` (the
report belongs in history; the raw log + metrics dumps
don't).

## Why a separate render step

`engine.e2e.toml` is committed with a public-placeholder RPC URL
(`wss://ethereum-sepolia-rpc.publicnode.com`); `e2e-run.sh`
substitutes `RPC_URL_SEPOLIA` from `.env` into a local file
`engine.e2e.local.toml` (gitignored via `*.local.toml`) and
points the engine at the local file. This means:

- No secret ever lands in `git diff`.
- The committed config still boots cleanly (against the public
  endpoint) for anyone cloning the repo who doesn't have a paid
  RPC key.
- The render step is idempotent — re-running `e2e-run.sh`
  always overwrites the local file.

## Verification

- `bash -n` syntax check on all 5 shell scripts: clean.
- `python3 -c "ast.parse(...)"` on `_ethflow_quote.py`: clean.
- `render_engine_config` smoke: produced `engine.e2e.local.toml`
  with the rpc_url line correctly substituted; diff showed
  exactly one line changed.

Linear: COW-1064. Stacks on the existing PR #46.
brunota20 added a commit that referenced this pull request Jun 25, 2026
…okup (COW-1074)

Closes the gap surfaced by the COW-1064 dry run (2026-06-18):
TWAP orders created through cow-swap UI sign with a non-empty
`appData` hash pointing at a richer JSON document (partner-id,
slippage settings, quote-id). twap-monitor hard-coded
`EMPTY_APP_DATA_JSON` when assembling `OrderCreation`, so the
orderbook rejected every submit with `invalid OrderCreation:
app_data JSON digest does not match signed app_data hash` and
the watch sat in retry-loop forever.

The WIT already exposes `cow-api::request(method, path, body)`
as a generic REST passthrough. We surface that capability on
the SDK trait, wrap it in a typed helper, and use the helper
from the strategy. No new host imports, no WIT ABI change, no
forced rebuild of unrelated modules.

Extended `CowApiHost` with:
```rust
fn cow_api_request(
    &self,
    chain_id: u64,
    method: &str,
    path: &str,
    body: Option<&str>,
) -> Result<String, HostError>;
```

404 responses surface as `HostError { code: 404, kind:
Unavailable }` so callers can distinguish "orderbook does not
have this resource" from genuine upstream failures without
introducing a new `HostErrorKind` variant (the existing enum
is `non_exhaustive`, but adding a variant on the WIT side
would still need an ABI bump).

`resolve_app_data(host, chain_id, hash) -> Result<String,
HostError>` with:
- Short-circuits `EMPTY_APP_DATA_HASH` (`keccak256("{}")`)
  to `EMPTY_APP_DATA_JSON` (`"{}"`) — no host call needed.
- Otherwise GETs `/api/v1/app_data/{hex_hash}` and pulls
  the `fullAppData` field out of the orderbook's envelope
  shape (`{"fullAppData": "<JSON string>", ...}`).
- 5 unit tests pinning the short-circuit, the success path,
  the unexpected-shape fall-through, the 404 propagation,
  and the hex encoder.

Extended `MockCowApi` with:
- `respond_to_request_for(method, path, result)`: per-key
  programmable response.
- `respond_to_request(result)`: catch-all default.
- `request_calls()`: records the (chain_id, method, path,
  body) tuple for every invocation.

The existing `respond` / `calls()` / `submit_order` surface
is unchanged.

`modules/twap-monitor/src/lib.rs`,
`modules/ethflow-watcher/src/lib.rs`,
`modules/examples/price-alert/src/lib.rs`, and
`modules/examples/stop-loss/src/lib.rs` each gained the
trivial 8-line forwarder to the generated
`cow_api::request` binding. Example modules implement
`CowApiHost` purely for the `Host` blanket-impl supertrait
even though some don't actively submit orders — the impl
is symmetrically extended.

`build_order_creation` now takes the resolved
`app_data_json` as an explicit parameter (was hard-coded to
`EMPTY_APP_DATA_JSON`). The resolution itself is lifted into
the caller `submit_ready`, which calls
`shepherd_sdk::cow::resolve_app_data` before assembling the
`OrderCreation`. Two graceful-fallback branches:
- `err.code == 404` → log Warn "appData hash not mirrored
  on orderbook" + leave the watch in place. Operators can
  re-trigger by pinning the document via a future
  orderbook PUT or by re-creating the order with empty
  appData.
- Any other resolver error → log Warn "appData resolve
  failed" + leave the watch. Future retry on the next
  block re-attempts the lookup.

Two new strategy tests:
- `poll_ready_resolves_non_empty_app_data_then_submits`:
  programs MockHost with a known JSON + its hash on the
  order, asserts the full resolve → submit → `submitted:`
  marker flow.
- `poll_ready_skips_submit_when_app_data_hash_not_mirrored`:
  programs MockHost to 404, asserts no submit attempt, no
  `submitted:` / `dropped:` markers, Warn log line present.

Plus one updated test
(`build_order_creation_accepts_matching_non_empty_app_data`)
that pins the new "matching hash → JSON" success path
directly on `build_order_creation`.

- `cargo test --workspace` → 13 + 12 + 16 + 32 + 8 + 8 +
  61 (engine) + 23 (twap-monitor) + 7 doctests + 1
  integration = 181 tests passing (was 174; +5 SDK +2
  twap-monitor).
- `cargo clippy --all-targets --workspace -- -D warnings`
  clean.
- `cargo fmt --all --check` clean.
- All 4 production module .wasm artefacts build cleanly
  with the new SDK trait.

- No WIT changes. Modules built against the prior SDK
  trait will fail to compile (the new method is required),
  but the WIT-generated wasm-side surface is bit-identical.
- No host-impl changes (`crates/nexum-engine/src/host/
  impls/cow_api.rs`). The host already implements `request`
  for the wit-bindgen binding.
- No metric surface drift. The orderbook lookup goes
  through the same `shepherd_cow_api_*` counters via the
  existing `request` path.

Linear: COW-1074. Stacks on the COW-1064 run-config branch
(#46). Validated locally end-to-end via `cargo test
--workspace`; live validation against the running engine
will happen on the next COW-1064 dry run (engine restart
required to pick up the rebuilt modules).
brunota20 added a commit that referenced this pull request Jun 25, 2026
Three-step automation that wraps the COW-1064 runbook +
prep punch list into shell scripts. Operator workflow
collapses to:

  cp scripts/env-template scripts/.env  &&  $EDITOR scripts/.env
  scripts/e2e-run.sh
  scripts/e2e-onchain.sh
  ## … 4-6 h …
  scripts/e2e-finish.sh

Secrets stay on disk. `scripts/.env` is gitignored; the
engine config with embedded RPC URL is rendered into a
gitignored `engine.e2e.local.toml` at boot time; the
PK is read from `scripts/.env` by `cast send` and never
echoed.

## Files

- `scripts/env-template` (committed): every variable the
  scripts read, with comments. Operator copies to
  `scripts/.env` and fills in.
- `scripts/lib.sh`: shared bash helpers — `log/warn/die`,
  `load_env`, `render_engine_config`, `state_value`, and
  the pinned address constants (EOA, Safe, ComposableCoW,
  TWAP handler, EthFlow, GPv2Settlement, GPv2VaultRelayer,
  WETH, COW, expected OrderUid).
- `scripts/e2e-run.sh`: renders engine config, cleans
  data/e2e, builds 5 modules + engine in release, launches
  via nohup, waits ≤ 60 s for `supervisor ready modules=5
  chains=1`, snapshots `metrics-start-<ts>.txt`. Persists
  PID + log path + start-ISO into `scripts/.state`.
- `scripts/e2e-onchain.sh`: derives EOA from
  `OPERATOR_PRIVATE_KEY` + asserts it matches the pinned
  test EOA + asserts balance ≥ 0.02 ETH; then `cast send`s:
    1. ComposableCoW.create() with the 516-byte pinned
       TWAP calldata → `ConditionalOrderCreated` →
       twap-monitor `watch:`.
    2. EthFlow.createOrder() with the tuple built from the
       cow.fi `/api/v1/quote` response (via
       `_ethflow_quote.py`) → `OrderPlacement` →
       ethflow-watcher `submitted:`.
  If `RUN_OPTIONAL_PRESIGN=1` also runs WETH wrap +
  setPreSignature + GPv2VaultRelayer approval (for
  stop-loss on-chain settlement; the `submitted:{uid}`
  marker is produced regardless). Each tx hash appended to
  `scripts/.state` as `TX_<KIND>=<hash>`.
- `scripts/_ethflow_quote.py`: small Python helper that
  POSTs to cow.fi Sepolia, gets feeAmount + quoteId +
  validTo + buyAmount, ABI-encodes the EthFlowOrder.Data
  tuple, and prints the calldata + msg.value for the
  shell script to consume.
- `scripts/e2e-finish.sh`: snapshots
  `metrics-end-<ts>.txt`, sends SIGINT, waits ≤ 30 s for
  `graceful shutdown complete` in the log (COW-1072 path),
  escalates to SIGKILL after 30 s, then invokes the report
  generator.
- `scripts/e2e-report-gen.sh`: parses the JSON-formatted
  engine log + metrics snapshots + state file into the
  e2e-report template's 9 sections. Auto-derives chain
  delta, per-module first marker, every `shepherd_*`
  counter / histogram delta, ERROR/trapped/poisoned
  tallies, and the per-row acceptance checklist
  (block-delta ≥ 1500, all-5-markers, zero traps, zero
  poison, zero ERROR, TWAP+EthFlow txs present). Writes
  `docs/operations/e2e-reports/e2e-report-<date>.md` ready
  for operator review.
- `scripts/README.md`: one-time setup, run sequence,
  troubleshooting table, re-run recipe.

## .gitignore additions

```
*.local.toml          # engine config rendered with embedded RPC key
scripts/.state        # run-state cache (PIDs, paths, tx hashes)
scripts/.env          # operator secrets (redundant with the existing .env / .env.* rules but explicit)
docs/operations/e2e-reports/engine-*.log
docs/operations/e2e-reports/metrics-*.txt
```

The auto-generated `e2e-report-<date>.md` is NOT gitignored
— operator reviews + commits manually with `git add -f` (the
report belongs in history; the raw log + metrics dumps
don't).

## Why a separate render step

`engine.e2e.toml` is committed with a public-placeholder RPC URL
(`wss://ethereum-sepolia-rpc.publicnode.com`); `e2e-run.sh`
substitutes `RPC_URL_SEPOLIA` from `.env` into a local file
`engine.e2e.local.toml` (gitignored via `*.local.toml`) and
points the engine at the local file. This means:

- No secret ever lands in `git diff`.
- The committed config still boots cleanly (against the public
  endpoint) for anyone cloning the repo who doesn't have a paid
  RPC key.
- The render step is idempotent — re-running `e2e-run.sh`
  always overwrites the local file.

## Verification

- `bash -n` syntax check on all 5 shell scripts: clean.
- `python3 -c "ast.parse(...)"` on `_ethflow_quote.py`: clean.
- `render_engine_config` smoke: produced `engine.e2e.local.toml`
  with the rpc_url line correctly substituted; diff showed
  exactly one line changed.

Linear: COW-1064. Stacks on the existing PR #46.
brunota20 added a commit that referenced this pull request Jun 25, 2026
…okup (COW-1074)

Closes the gap surfaced by the COW-1064 dry run (2026-06-18):
TWAP orders created through cow-swap UI sign with a non-empty
`appData` hash pointing at a richer JSON document (partner-id,
slippage settings, quote-id). twap-monitor hard-coded
`EMPTY_APP_DATA_JSON` when assembling `OrderCreation`, so the
orderbook rejected every submit with `invalid OrderCreation:
app_data JSON digest does not match signed app_data hash` and
the watch sat in retry-loop forever.

The WIT already exposes `cow-api::request(method, path, body)`
as a generic REST passthrough. We surface that capability on
the SDK trait, wrap it in a typed helper, and use the helper
from the strategy. No new host imports, no WIT ABI change, no
forced rebuild of unrelated modules.

Extended `CowApiHost` with:
```rust
fn cow_api_request(
    &self,
    chain_id: u64,
    method: &str,
    path: &str,
    body: Option<&str>,
) -> Result<String, HostError>;
```

404 responses surface as `HostError { code: 404, kind:
Unavailable }` so callers can distinguish "orderbook does not
have this resource" from genuine upstream failures without
introducing a new `HostErrorKind` variant (the existing enum
is `non_exhaustive`, but adding a variant on the WIT side
would still need an ABI bump).

`resolve_app_data(host, chain_id, hash) -> Result<String,
HostError>` with:
- Short-circuits `EMPTY_APP_DATA_HASH` (`keccak256("{}")`)
  to `EMPTY_APP_DATA_JSON` (`"{}"`) — no host call needed.
- Otherwise GETs `/api/v1/app_data/{hex_hash}` and pulls
  the `fullAppData` field out of the orderbook's envelope
  shape (`{"fullAppData": "<JSON string>", ...}`).
- 5 unit tests pinning the short-circuit, the success path,
  the unexpected-shape fall-through, the 404 propagation,
  and the hex encoder.

Extended `MockCowApi` with:
- `respond_to_request_for(method, path, result)`: per-key
  programmable response.
- `respond_to_request(result)`: catch-all default.
- `request_calls()`: records the (chain_id, method, path,
  body) tuple for every invocation.

The existing `respond` / `calls()` / `submit_order` surface
is unchanged.

`modules/twap-monitor/src/lib.rs`,
`modules/ethflow-watcher/src/lib.rs`,
`modules/examples/price-alert/src/lib.rs`, and
`modules/examples/stop-loss/src/lib.rs` each gained the
trivial 8-line forwarder to the generated
`cow_api::request` binding. Example modules implement
`CowApiHost` purely for the `Host` blanket-impl supertrait
even though some don't actively submit orders — the impl
is symmetrically extended.

`build_order_creation` now takes the resolved
`app_data_json` as an explicit parameter (was hard-coded to
`EMPTY_APP_DATA_JSON`). The resolution itself is lifted into
the caller `submit_ready`, which calls
`shepherd_sdk::cow::resolve_app_data` before assembling the
`OrderCreation`. Two graceful-fallback branches:
- `err.code == 404` → log Warn "appData hash not mirrored
  on orderbook" + leave the watch in place. Operators can
  re-trigger by pinning the document via a future
  orderbook PUT or by re-creating the order with empty
  appData.
- Any other resolver error → log Warn "appData resolve
  failed" + leave the watch. Future retry on the next
  block re-attempts the lookup.

Two new strategy tests:
- `poll_ready_resolves_non_empty_app_data_then_submits`:
  programs MockHost with a known JSON + its hash on the
  order, asserts the full resolve → submit → `submitted:`
  marker flow.
- `poll_ready_skips_submit_when_app_data_hash_not_mirrored`:
  programs MockHost to 404, asserts no submit attempt, no
  `submitted:` / `dropped:` markers, Warn log line present.

Plus one updated test
(`build_order_creation_accepts_matching_non_empty_app_data`)
that pins the new "matching hash → JSON" success path
directly on `build_order_creation`.

- `cargo test --workspace` → 13 + 12 + 16 + 32 + 8 + 8 +
  61 (engine) + 23 (twap-monitor) + 7 doctests + 1
  integration = 181 tests passing (was 174; +5 SDK +2
  twap-monitor).
- `cargo clippy --all-targets --workspace -- -D warnings`
  clean.
- `cargo fmt --all --check` clean.
- All 4 production module .wasm artefacts build cleanly
  with the new SDK trait.

- No WIT changes. Modules built against the prior SDK
  trait will fail to compile (the new method is required),
  but the WIT-generated wasm-side surface is bit-identical.
- No host-impl changes (`crates/nexum-engine/src/host/
  impls/cow_api.rs`). The host already implements `request`
  for the wit-bindgen binding.
- No metric surface drift. The orderbook lookup goes
  through the same `shepherd_cow_api_*` counters via the
  existing `request` path.

Linear: COW-1074. Stacks on the COW-1064 run-config branch
(#46). Validated locally end-to-end via `cargo test
--workspace`; live validation against the running engine
will happen on the next COW-1064 dry run (engine restart
required to pick up the rebuilt modules).
jean-neiverth pushed a commit that referenced this pull request Jun 30, 2026
…okup (COW-1074)

Closes the gap surfaced by the COW-1064 dry run (2026-06-18):
TWAP orders created through cow-swap UI sign with a non-empty
`appData` hash pointing at a richer JSON document (partner-id,
slippage settings, quote-id). twap-monitor hard-coded
`EMPTY_APP_DATA_JSON` when assembling `OrderCreation`, so the
orderbook rejected every submit with `invalid OrderCreation:
app_data JSON digest does not match signed app_data hash` and
the watch sat in retry-loop forever.

The WIT already exposes `cow-api::request(method, path, body)`
as a generic REST passthrough. We surface that capability on
the SDK trait, wrap it in a typed helper, and use the helper
from the strategy. No new host imports, no WIT ABI change, no
forced rebuild of unrelated modules.

Extended `CowApiHost` with:
```rust
fn cow_api_request(
    &self,
    chain_id: u64,
    method: &str,
    path: &str,
    body: Option<&str>,
) -> Result<String, HostError>;
```

404 responses surface as `HostError { code: 404, kind:
Unavailable }` so callers can distinguish "orderbook does not
have this resource" from genuine upstream failures without
introducing a new `HostErrorKind` variant (the existing enum
is `non_exhaustive`, but adding a variant on the WIT side
would still need an ABI bump).

`resolve_app_data(host, chain_id, hash) -> Result<String,
HostError>` with:
- Short-circuits `EMPTY_APP_DATA_HASH` (`keccak256("{}")`)
  to `EMPTY_APP_DATA_JSON` (`"{}"`) — no host call needed.
- Otherwise GETs `/api/v1/app_data/{hex_hash}` and pulls
  the `fullAppData` field out of the orderbook's envelope
  shape (`{"fullAppData": "<JSON string>", ...}`).
- 5 unit tests pinning the short-circuit, the success path,
  the unexpected-shape fall-through, the 404 propagation,
  and the hex encoder.

Extended `MockCowApi` with:
- `respond_to_request_for(method, path, result)`: per-key
  programmable response.
- `respond_to_request(result)`: catch-all default.
- `request_calls()`: records the (chain_id, method, path,
  body) tuple for every invocation.

The existing `respond` / `calls()` / `submit_order` surface
is unchanged.

`modules/twap-monitor/src/lib.rs`,
`modules/ethflow-watcher/src/lib.rs`,
`modules/examples/price-alert/src/lib.rs`, and
`modules/examples/stop-loss/src/lib.rs` each gained the
trivial 8-line forwarder to the generated
`cow_api::request` binding. Example modules implement
`CowApiHost` purely for the `Host` blanket-impl supertrait
even though some don't actively submit orders — the impl
is symmetrically extended.

`build_order_creation` now takes the resolved
`app_data_json` as an explicit parameter (was hard-coded to
`EMPTY_APP_DATA_JSON`). The resolution itself is lifted into
the caller `submit_ready`, which calls
`shepherd_sdk::cow::resolve_app_data` before assembling the
`OrderCreation`. Two graceful-fallback branches:
- `err.code == 404` → log Warn "appData hash not mirrored
  on orderbook" + leave the watch in place. Operators can
  re-trigger by pinning the document via a future
  orderbook PUT or by re-creating the order with empty
  appData.
- Any other resolver error → log Warn "appData resolve
  failed" + leave the watch. Future retry on the next
  block re-attempts the lookup.

Two new strategy tests:
- `poll_ready_resolves_non_empty_app_data_then_submits`:
  programs MockHost with a known JSON + its hash on the
  order, asserts the full resolve → submit → `submitted:`
  marker flow.
- `poll_ready_skips_submit_when_app_data_hash_not_mirrored`:
  programs MockHost to 404, asserts no submit attempt, no
  `submitted:` / `dropped:` markers, Warn log line present.

Plus one updated test
(`build_order_creation_accepts_matching_non_empty_app_data`)
that pins the new "matching hash → JSON" success path
directly on `build_order_creation`.

- `cargo test --workspace` → 13 + 12 + 16 + 32 + 8 + 8 +
  61 (engine) + 23 (twap-monitor) + 7 doctests + 1
  integration = 181 tests passing (was 174; +5 SDK +2
  twap-monitor).
- `cargo clippy --all-targets --workspace -- -D warnings`
  clean.
- `cargo fmt --all --check` clean.
- All 4 production module .wasm artefacts build cleanly
  with the new SDK trait.

- No WIT changes. Modules built against the prior SDK
  trait will fail to compile (the new method is required),
  but the WIT-generated wasm-side surface is bit-identical.
- No host-impl changes (`crates/nexum-engine/src/host/
  impls/cow_api.rs`). The host already implements `request`
  for the wit-bindgen binding.
- No metric surface drift. The orderbook lookup goes
  through the same `shepherd_cow_api_*` counters via the
  existing `request` path.

Linear: COW-1074. Stacks on the COW-1064 run-config branch
(#46). Validated locally end-to-end via `cargo test
--workspace`; live validation against the running engine
will happen on the next COW-1064 dry run (engine restart
required to pick up the rebuilt modules).
lgahdl added a commit that referenced this pull request Jul 10, 2026
…rom subscriptions

- block_chains() and chain_log_subscriptions() now skip modules with
  alive=false, so dead modules never contribute RPC subscriptions
- builder bails early with a clear error when alive_count()==0 after
  boot, rather than silently running an empty event loop for 7 days
- alive_count() no longer gated behind #[cfg_attr(not(test), ...)]
- New test: dead_modules_excluded_from_subscription_lists locks the
  filter invariant using an init-failing price-alert fixture

Closes #46

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lgahdl added a commit that referenced this pull request Jul 10, 2026
…h-level test

Review follow-ups on top of the rebase onto develop:

- builder: when the live subscription lists are empty because every
  declared [[subscription]] belongs to an init-failed module, bail with
  an operator-facing error instead of exiting Ok behind the misleading
  "no [[subscription]] entries" message — the exact scenario in #46's
  title.
- supervisor: new dead_modules_hold_subscriptions() distinguishes
  "no manifest declares subscriptions" (benign) from "subscriptions
  were filtered by module death" (abort).
- builder tests: launch_bails_when_all_modules_fail_init locks the
  headline "all N module(s) failed initialisation" bail, which had no
  test.
- supervisor tests: positive control — one dead + one alive module,
  asserting the alive module's chain survives the filter, so the
  all-dead test can no longer pass vacuously.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TfSsJPYDQ1GQGbnSVFxATx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant