refactor(ethflow-watcher): observe + verify instead of submit (BLEU-855 redesign, COW-1076)#63
Closed
brunota20 wants to merge 1 commit into
Closed
Conversation
…55 redesign, COW-1076) Apply the BLEU-855 strategy.rs / lib.rs split to the observe + verify design proposed in PR #62 (M2 layer). Replaces the submit-based strategy that COW-1074, COW-1075, COW-1076 and COW-1083 were incrementally patching. Why the prior design cannot work (verified empirically during the 2026-06-22 Sepolia soak): the orderbook backend indexes EthFlow `OrderPlacement` events server-side and writes fields the public `OrderCreation` request body cannot carry (onchainUser, onchainOrderData, ethflowData.userValidTo). `POST /api/v1/orders` applies the generic validTo cap and rejects the EthFlow canonical `validTo = u32::MAX` shape every time. The 3 placements observed during the soak all landed as `ExcessiveValidTo` drops in shepherd, while the orderbook held the same UIDs at status `fulfilled`. This commit replaces `submit_placement` with `observe_placement`: 1. Decode the OrderPlacement log (BLEU-832, preserved verbatim). 2. Compute the canonical 56-byte UID from the on-chain order shape. 3. GET /api/v1/orders/{uid} via `Host::cow_api_request` (the COW-1074 trait method) and record: - 200 → write `observed:{uid}` + log Info "orderbook indexed". - 404 → log Info ("not yet indexed; will recheck on re-delivery") and intentionally do NOT write the marker, so the next log re-delivery can recheck. 404 is treated as expected indexer lag, not anomaly. - other err → log Warn with the host error's code+message. Strategy now runs purely through the `shepherd_sdk::host::Host` trait seam; tests use `shepherd_sdk_test::MockHost`. Removed: - `OrderCreation` body construction and JSON encoding (no submit). - `BuildError`, `to_signature`, `build_eth_flow_creation`. - `RetryAction` / `classify_api_error` integration on this path (the helper stays for twap-monitor's use). - `MAX_BACKOFF_RETRIES`, `read_backoff_count`, `apply_submit_retry`, `prior_outcome`, `PriorOutcome::Submitted/Dropped/Backoff`. - The `EXCESSIVE_VALID_TO` / `is_expected_excessive_valid_to` classifier (PR #49 / COW-1076 noise reduction is moot — there is no drop log to silence). - Module Cargo deps: `serde_json`, `thiserror` (the OrderCreation JSON encode path and the BuildError enum no longer exist). Test surface (11 tests, all pass natively + wasm32-wasip2): - BLEU-832 decode invariants kept: well-formed placement, unrelated contract address, wrong topic signature. - New `compute_uid_pins_owner_to_ethflow_contract_and_validto` — asserts the 56-byte UID layout `digest || owner || valid_to` and that the owner suffix is the EthFlow contract (not the sender). - New `compute_uid_returns_none_on_unsupported_chain`. - New observe-path tests via MockHost: 200 marks observed + zero submit_order calls; 404 does NOT mark observed and logs at Info; other err logs at Warn with no marker. - New idempotency test: a placement with `observed:{uid}` already in local store short-circuits before any orderbook call. - New regression guard `strategy_never_calls_submit_order` — the CowApiHost trait still exposes submit_order for twap-monitor, but ethflow-watcher's observe design must never call it. Stack implications (no force-push to other branches in this commit; those are separate operations): - PR #10 (BLEU-833 original submit design) — superseded by PR #62 at the M2 layer and by this PR at the M3+M4 layer. - PR #47 (COW-1074 resolve_app_data) — shepherd-sdk helper stays useful for twap-monitor; this commit removes the now-dead ethflow-watcher consumer. - PR #48 (COW-1075 forward orderbook ApiError) — cow-api error envelope improvement is still useful workspace-wide; no change here. - PR #49 (COW-1076 ExcessiveValidTo log downgrade) — moot, no drop log to silence; close as superseded. - PR #59 (COW-1083 backoff retry cap) — moot, no retry path to cap; close as superseded. Follow-up (not in scope here): block-tick subscription that re-checks `pending:{uid}` markers across N blocks so persistent indexer downtime surfaces as Warn after a configurable threshold. The 404 path today relies on log re-delivery for the recheck. AI assistance disclosure: drafted by Claude (Opus 4.7, 1M context) following the 2026-06-22 soak inspection.
Author
Live validation note (2026-06-23 14:00 UTC)Combined with the rest of the redesign + fix bundle on Companion PRs (same redesign / soak deploy bundle)
Each is single-concern; the deploy branch is the cumulative integration point and will fold into |
This was referenced Jun 23, 2026
Author
|
Superseded by deploy branch commit |
brunota20
added a commit
that referenced
this pull request
Jun 25, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
brunota20
added a commit
that referenced
this pull request
Jun 25, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
brunota20
added a commit
that referenced
this pull request
Jun 25, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
brunota20
added a commit
that referenced
this pull request
Jun 25, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
brunota20
added a commit
that referenced
this pull request
Jun 26, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
jean-neiverth
pushed a commit
that referenced
this pull request
Jun 29, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
jean-neiverth
pushed a commit
that referenced
this pull request
Jun 30, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
jean-neiverth
pushed a commit
that referenced
this pull request
Jun 30, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
jean-neiverth
pushed a commit
that referenced
this pull request
Jun 30, 2026
Cherry-pick of PR #62 + PR #63's redesign onto the M5 host runtime (env-var substitution in engine.toml, healthcheck fixes, etc) for the Sepolia soak VM. The PR review continues on the proper layered branches: - PR #62 — M2/BLEU-833 layer observe design - PR #63 — M3/M4 BLEU-855 split + COW-1074 cow_api_request integration This branch is deploy-only: it lets the soak run on the redesigned ethflow-watcher with the latest host runtime while review iterates on the layered PRs. After merge, this branch can be deleted; CI will republish ghcr.io/bleu/nullis-shepherd:latest with the merged design and the VM rolls forward to the official image. See COW-1076 for the full empirical evidence.
lgahdl
added a commit
that referenced
this pull request
Jul 14, 2026
- "the module fails to load" named the wrong lifecycle stage: traced the actual boot sequence (crates/nexum-runtime/src/builder.rs) and an unconfigured chain fails when subscriptions open (open_block_streams/open_chain_log_streams), which runs strictly after Supervisor::boot (Load+Init), not during Load. Reworded to "the engine bails at boot, before any events dispatch" - accurate without naming a specific state-machine stage. - "Two 0.1 fields dropped..." asserted these were real fields in 0.1 without verification; grepped docs/01-runtime-environment.md and found zero occurrences of either. Reworded to state only what's verified: they're not part of 0.2's schema, no claim about 0.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TfSsJPYDQ1GQGbnSVFxATx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this PR exists
Companion to PR #62 (M2-layer observe design at the BLEU-833 point in the timeline). This PR applies the same redesign at the M3/M4 layer where the BLEU-855
strategy.rs/lib.rssplit already lives and whereHost::cow_api_request(added in COW-1074) is available for the orderbook verification step. Base =feat/resolve-app-data-cow-1074(PR #47's head), same as the chain layer that PRs #48 and #49 build on, so this slots in cleanly as a sibling redesign rather than yet another patch on top.What changes
Replace
submit_placementwithobserve_placementinstrategy.rs. The orderbook-side reality (verified during the 2026-06-22 Sepolia soak) is that the orderbook backend indexes EthFlowOrderPlacementevents natively and the publicPOST /api/v1/ordersendpoint structurally cannot reproduce the indexer's dual-validTo bookkeeping. The 3 placements shepherd ""dropped"" during the soak withExcessiveValidTowere allfulfilledon the orderbook at the same UID. So the right contract with the orderbook is to observe + verify, not submit.GET /api/v1/orders/{uid}→ 200observed:{uid}writtenGET /api/v1/orders/{uid}→ 404code+messageobserved:{uid}makes log re-delivery a no-op once verified. 404 is treated as expected indexer lag (shepherd's WebSocket can deliver the on-chain log a few hundred ms before the orderbook commits) — intentionally no marker so re-delivery rechecks.Strategy now runs purely through the
shepherd_sdk::host::Hosttrait seam —Host::cow_api_requestfor the GET,Host::{get,set}for the marker,Host::logfor diagnostics. Tests drive the same function viashepherd_sdk_test::MockHostwithrespond_to_request_for(""GET"", ""/api/v1/orders/{uid}"", ...).Removed (the submit-design machinery)
OrderCreationbody construction + JSON encoding.BuildError,to_signature,build_eth_flow_creation.RetryAction/classify_api_errorintegration on this path. (Helper stays —twap-monitorlegitimately uses it.)MAX_BACKOFF_RETRIES,read_backoff_count,apply_submit_retry,prior_outcome,PriorOutcome::{Submitted,Dropped,Backoff}.EXCESSIVE_VALID_TO/is_expected_excessive_valid_to(PR feat(ethflow-watcher): downgrade ExcessiveValidTo drops to Info (COW-1076) #49 / COW-1076 silencing is moot now).serde_json,thiserror(no JSON encode path, no BuildError enum).Tests (11 total, all green)
strategy.rsdecodes_well_formed_placementrejects_unrelated_contract_addressrejects_wrong_topic_signaturecompute_uid_pins_owner_to_ethflow_contract_and_validtodigest || owner || valid_to, owner = EthFlow contract.compute_uid_returns_none_on_unsupported_chainplacement_log_marks_observed_on_orderbook_200placement_log_does_not_mark_observed_on_orderbook_404placement_log_warns_on_orderbook_other_errorpreviously_observed_placement_is_skipped_on_redeliveryobserved:{uid}short-circuits the GET.unsupported_chain_logs_warn_without_orderbook_callstrategy_never_calls_submit_ordersubmit_ordercount must stay at 0.cargo clippy -p ethflow-watcher --all-targets -- -D warningsclean.cargo fmt --checkclean.cargo build -p ethflow-watcher --target wasm32-wasip2 --releaseclean.Stack implications (informational — no other branches force-pushed)
resolve_app_data) — shepherd-sdk helper stays useful for twap-monitor; this commit just removes the dead ethflow-watcher consumer of it.Out of scope (follow-up)
[[subscription]] kind = ""block""tomodule.tomland apending:{uid}marker swept on each block, so persistent indexer downtime surfaces as Warn after N blocks.OrderCreationand submitting; the prose needs the observe-not-submit revision. Lands in the M3/M4 doc bundle.AI assistance disclosure: drafted by Claude (Opus 4.7, 1M context) following the 2026-06-22 soak inspection that surfaced the divergence.