[simplex]: Keep posted limit orders alive with heartbeats, renewal and reconciliation

.github/workflows/check-orderbook-schema.yml

@@ -0,0 +1,48 @@
+name: "orderbook schema pin"
+
+concurrency:
+    group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+    cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+# `sdk/packages/simplex/src/tests/fixtures/orderbook-schema.graphql` is a copy of the
+# orderbook's own schema, and `schema.test.ts` validates every document simplex sends
+# against it. A copy that has fallen behind still passes, which is the one failure the
+# test exists to prevent, so the copy is compared with the server here instead.
+on:
+    schedule:
+        - cron: "17 6 * * 1"
+    pull_request:
+        branches: [main]
+        types: [opened, synchronize, reopened, ready_for_review]
+        paths:
+            - "sdk/packages/simplex/src/orderbook/**"
+            - "sdk/packages/simplex/src/tests/fixtures/orderbook-schema.graphql"
+            - ".github/workflows/check-orderbook-schema.yml"
+    workflow_dispatch:
+
+jobs:
+    compare:
+        runs-on: ubuntu-latest
+        steps:
+            - name: Checkout repository
+              uses: actions/checkout@v4
+              with:
+                  ref: ${{ github.event.pull_request.head.sha || github.sha }}
+
+            - name: Compare the pinned schema with the orderbook's
+              env:
+                  GH_TOKEN: ${{ secrets.ORDERBOOK_SCHEMA_TOKEN || secrets.GITHUB_TOKEN }}
+              run: |
+                  pin=sdk/packages/simplex/src/tests/fixtures/orderbook-schema.graphql
+                  # The orderbook is a private repo, and the default GITHUB_TOKEN is scoped to this
+                  # one, so without a token that can read it there is nothing to compare against.
+                  # That is a check that cannot run, not a check that failed.
+                  if ! gh api repos/polytope-labs/hyperfx-orderbook/contents/schema.graphql -q .content \
+                       | base64 -d > /tmp/schema.graphql || [ ! -s /tmp/schema.graphql ]; then
+                    echo "::notice::Could not read the orderbook's schema. Set the ORDERBOOK_SCHEMA_TOKEN secret to a token with read access to polytope-labs/hyperfx-orderbook to enable this check."
+                    exit 0
+                  fi
+                  if ! diff -u "$pin" /tmp/schema.graphql; then
+                    echo "::error::The pinned orderbook schema has drifted. Refresh it with the command in schema.test.ts."
+                    exit 1
+                  fi

.github/workflows/test-simplex-orderbook.yml

@@ -0,0 +1,115 @@
+name: "simplex against a live orderbook"
+
+concurrency:
+    group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+    cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+# Simplex posts limit orders to a HyperFX orderbook, and every other test in the package answers
+# its client from a stub. This runs the published orderbook image and puts one limit order through
+# it for real: created, listed under the solver, heartbeated, resized by a fill, reconciled and
+# withdrawn. No chain is involved — `scripts/fake-indexer.mjs` answers the indexer and the gateways
+# the way the orderbook's own test harness does.
+on:
+    push:
+        branches: [main]
+        paths:
+            - "sdk/packages/simplex/src/orderbook/**"
+            - "sdk/packages/simplex/scripts/fake-indexer.mjs"
+            - "sdk/packages/simplex/src/tests/fixtures/orderbook-ci.toml"
+            - ".github/workflows/test-simplex-orderbook.yml"
+    # No `branches` filter, unlike the other workflows here: the simplex work is a
+    # stack of PRs whose bases are each other rather than main, and filtering on
+    # main would mean none of them ever ran this.
+    pull_request:
+        types: [opened, synchronize, reopened, ready_for_review]
+        paths:
+            - "sdk/packages/simplex/src/orderbook/**"
+            - "sdk/packages/simplex/scripts/fake-indexer.mjs"
+            - "sdk/packages/simplex/src/tests/fixtures/orderbook-ci.toml"
+            - ".github/workflows/test-simplex-orderbook.yml"
+    workflow_dispatch:
+
+jobs:
+    live:
+        runs-on: ubuntu-latest
+        env:
+            ORDERBOOK_IMAGE: polytopelabs/hyperfx-orderbook:v0.1.0
+        steps:
+            - name: Checkout repository
+              uses: actions/checkout@v4
+              with:
+                  ref: ${{ github.event.pull_request.head.sha || github.sha }}
+                  # scripts/build.sh runs protoc over proto/mpcvaultapis to generate src/proto,
+                  # and the test's import graph reaches it through the wallet service.
+                  submodules: recursive
+
+            - name: Setup Node.js
+              uses: actions/setup-node@v4
+              with:
+                  node-version: "22"
+
+            - name: Setup pnpm
+              uses: pnpm/action-setup@v4
+              with:
+                  version: 11
+
+            - name: Install dependencies
+              working-directory: sdk
+              run: pnpm install --frozen-lockfile
+
+            # Straight from GitHub releases, not apt: the runners' default apt mirror has
+            # repeatedly hung apt-get update, and nothing else here needs apt.
+            - name: Install protoc
+              run: |
+                  curl -fsSL -o /tmp/protoc.zip \
+                    https://github.com/protocolbuffers/protobuf/releases/download/v29.3/protoc-29.3-linux-x86_64.zip
+                  sudo unzip -o /tmp/protoc.zip -d /usr/local bin/protoc 'include/*'
+                  protoc --version
+
+            - name: Generate the protobuf client
+              working-directory: sdk/packages/simplex
+              run: pnpm run codegen
+
+            # The test imports `@hyperbridge/sdk`, which resolves to that package's `dist`. A fresh
+            # checkout has none, and vitest fails to collect the file before it reaches the orderbook.
+            - name: Build the SDK
+              working-directory: sdk
+              run: pnpm --filter="@hyperbridge/sdk" build
+
+            - name: Start the fake indexer
+              working-directory: sdk/packages/simplex
+              run: |
+                  node scripts/fake-indexer.mjs --port 4000 --fee-bps 0 &
+                  for _ in $(seq 1 20); do
+                    curl -sf -X POST http://127.0.0.1:4000/graphql -d '{"variables":{}}' && break
+                    sleep 1
+                  done
+
+            - name: Start the orderbook
+              working-directory: sdk/packages/simplex
+              run: |
+                  docker run -d --name orderbook --network host \
+                    -v "$PWD/src/tests/fixtures/orderbook-ci.toml:/app/config.toml:ro" \
+                    "$ORDERBOOK_IMAGE"
+                  # Ready means it answers a query, not merely that the port is open.
+                  for _ in $(seq 1 60); do
+                    if curl -sf -X POST http://127.0.0.1:8080/graphql \
+                         -H 'content-type: application/json' \
+                         -d '{"query":"{ serverInfo { minOrderTtlSecs } }"}' > /dev/null; then
+                      exit 0
+                    fi
+                    sleep 1
+                  done
+                  echo "::error::The orderbook never came up"
+                  docker logs orderbook
+                  exit 1
+
+            - name: Post a limit order through it
+              working-directory: sdk/packages/simplex
+              env:
+                  HYPERFX_ORDERBOOK_URL: http://127.0.0.1:8080/graphql
+              run: pnpm exec vitest run --watch=false src/tests/orderbook/orderbook.live.test.ts
+
+            - name: Orderbook logs
+              if: always()
+              run: docker logs orderbook 2>&1 | tail -50

sdk/packages/simplex/docs/ai/changelog/2026-09-15-limit-orders-posted-to-the-hyperfx-orderbook.md

@@ -28,6 +28,15 @@ creating an order should have to know an asset's decimals, let alone that the or
 everything to 1e18, so the handler scales them while it validates. Anything with more than 18 decimal
 places, or that is not a plain decimal, is refused with the amount named.
 
+Creation also refuses an order the wallet cannot pay out. The orderbook backs an entry with the
+solver's real balance and cuts down what it is not holding, so an order written against money that is
+not there is refused or silently shrunk rather than filled; hearing it while creating the order is
+better. Each order is checked against the whole balance rather than what is left of it after the
+others: one balance backs every order resting on it, which is what quoting both sides of a book is,
+and the orderbook advertises each entry at `min(quoted, balance)` instead of dividing the balance
+between them. Whichever order fills first draws the inventory down and the rest are cut to what is
+left.
+
 `acceptedSources` is required and non-empty: it names the source chains the order accepts swaps
 from, and the orderbook refuses an order that declares none. A create emits `limit-order:posted` or
 `limit-order:rejected`, a cancel emits `limit-order:cancelled`.

sdk/packages/simplex/docs/ai/changelog/2026-09-16-a-fill-resizes-its-limit-order.md

@@ -31,5 +31,5 @@ would advertise the same output twice; the gap between the two calls is at most
 `UNKNOWN_ORDER` on the cancel is as good as cancelled. Every repost signs on a fresh nonce, since the
 orderbook remembers every op hash it has accepted.
 
-`repost` is separate from `settleFill` because renewal before expiry and reconciliation after a
+`repost` is separate from `settleFill` because reconciliation after a
 restart both want the same thing: whatever the order has left, live on the book again.

sdk/packages/simplex/docs/ai/changelog/2026-09-16-a-posting-cannot-undo-a-cancel.md

@@ -0,0 +1,15 @@
+# 2026-09-16 — A posting cannot undo a cancel
+
+`setPosting` and `setStatus` write what the caller asks, so a posting that was already in flight when
+the operator cancelled the order, or when the expiry sweep took it down, wrote the row back to `open`
+with a fresh commitment and a live entry on the book. The cancel was undone and the order started
+matching swaps again. Every posting path could do it: the repost after a fill and
+reconciliation.
+
+Both writes now take an optional list of statuses the row must still hold, and answer null when it
+has moved on. `post` passes `open` and `resizing`, and when the write does not apply it withdraws the
+entry the orderbook has just accepted, since nothing here owns it any more. A refusal arriving late
+is dropped for the same reason: it says nothing about the row the operator left behind.
+
+The guard is a condition on the update rather than a read followed by a write, so two callers racing
+cannot both decide they were first.

sdk/packages/simplex/docs/ai/changelog/2026-09-16-bid-the-ask-and-only-orders-whose-rate-allows-it.md

@@ -0,0 +1,75 @@
+# 2026-09-16 — Bid the ask, and only from orders whose rate allows it
+
+Three findings from review, two of them live before the multi-order work.
+
+## The bid is the ask, not the whole offer
+
+`targetOutput` is not a ceiling the filler stays under, it is `solverAmount` at the gateway. On a full
+fill the gateway sets `fillAmount = totalRequired` and `_splitSurplus` hands everything above it to
+the beneficiary and the protocol, debiting the solver the whole bid; with output calldata all of it
+goes to the protocol. So bidding a limit order's full offer when the swapper asked for less gave the
+difference away, and it fired in the ordinary case, since finding an order that prices better than
+the ask is the point.
+
+Worse, that donation was booked as the profit that justified the fill: `payoutSurplusUsd` counts the
+gap between the offer and the ask, and a partial's whole P&L is that figure.
+
+The bid is now `min(offered, output.amount)`. The escrow released is identical either way, so what is
+not bid is margin kept and `payoutSurplusUsd` describes something real. `maxOverfillBps` stays as a
+warning about a limit order priced well away from the market, which is what an offer far above the
+ask now means. The clamp it once performed was disabled in #964 for curve-priced legs, where the
+computed amount was a curve's answer for that leg; under limit orders it is the operator's rate
+applied to the whole input, which is a different thing.
+
+## An order only takes part if its offer covers the ask
+
+Escrow release is strictly proportional: `Released(filled) = escrowTotal * filled / totalRequired`.
+A fill of `f` out of `T` releases `I * f / T`, so its effective rate is `T / I` whatever `f` is. Every
+fill of an order is therefore paid at the swapper's rate, not its own, and an order may only take
+part where that rate is inside its own terms: `T / I <= price`, which is exactly
+`offer >= requestedOutput`.
+
+That condition was dropped earlier in this stack as a ranking preference. It is not one. Without it a
+single order whose offer fell short would partial-fill above the rate the operator signed.
+
+## One fill may draw on several limit orders, and the total is the ask
+
+Combining levels first summed per-order offers that were each computed against the whole input, which
+bills one input to several orders and overpays by the difference between the best rate and the worst.
+Clamping the bid to the ask removes that by construction: every order in the set already clears the
+ask on rate, so what they add up to is depth rather than price. The total paid is the ask, each order
+funds a slice of it, is drawn down by that slice, and receives that fraction of the input, so every
+one of them settles at `T / I` and stays inside its own terms.
+
+Each order in the set then bids for itself, best offer first, which
+`2026-09-16-one-swap-can-draw-on-several-limit-orders.md` covers.
+
+## Smaller
+
+`orderbook-schema.graphql` is refreshed from `hyperfx-orderbook@main`, which has since removed
+`SwapQuote.priceBucket`, added `Query.chains` with per-chain token decimals, and rewritten the quote
+semantics. It is a pinned copy and would drift again silently, since the test
+validates against the pin rather than against the server, so
+`.github/workflows/check-orderbook-schema.yml` now fetches the schema and diffs it, weekly and on any
+change to this package's orderbook code. `schema.test.ts` carries the one command that refreshes it.
+
+Token decimals come from the orderbook's own registry too. `serverInfo` now carries `chains` with the
+tokens each one registers, `limits()` already caches it for five minutes, and every post and repost
+needs both sides of the pair, so this is two chain reads saved each time. It is also the registry the
+server prices against, which is what makes it the right source rather than merely the cheap one; a
+chain or symbol it does not list still falls back to the token. The matcher docstring no longer claims a cross-chain order reverts on any under-fill.
+
+## A fill settles its holds in one transaction
+
+Claiming what a bid held, working the orders down by what went out and giving the rest back are one
+decision about the same holds, and they were three separate writes. A crash between two of them gave
+a hold back against an order that was never drawn down, which leaves it advertising output already
+paid. §6 asked for them to be one store transaction and this took the weaker route of ordering them,
+on the reasoning that a transaction would have to span two stores. It does not: `SqliteDataStore`
+hands the same `bids.db` connection to the bid store and the limit order store.
+
+`LimitOrderStore.transaction` runs the settlement as a unit, with the same rollback guard the state
+store uses, and the in-memory store restores a snapshot rather than leaving the default backend
+weaker than the configured one. Only store writes go inside. Putting the order back on the book is a
+round trip and now happens after, which is why `settleFill` splits: the draw-down belongs in the
+transaction, `resize` does not.

sdk/packages/simplex/docs/ai/changelog/2026-09-16-one-limit-order-through-a-running-orderbook.md

@@ -0,0 +1,30 @@
+# 2026-09-16 — One limit order through a running orderbook
+
+`orderbook.live.test.ts` takes a limit order through a real HyperFX orderbook: created and accepted,
+listed under the solver, heartbeated, resized by a fill, reconciled, then withdrawn. It is skipped
+unless the environment says where to find a server:
+
+```
+HYPERFX_ORDERBOOK_URL=http://127.0.0.1:8080/graphql   a server already running
+HYPERFX_ORDERBOOK_BIN=/path/to/hyperfx-orderbook      a binary to run on a config the test writes
+```
+
+Build the binary from polytope-labs/hyperfx-orderbook with
+`cargo build --release -p hyperfx-server --bin hyperfx-orderbook`. The test needs no chain: the
+config it writes runs the server the way its own `config.dev.toml` does, with the protocol fee a
+constant and the validation cycle reading nothing, so an order surfaces at its quoted size with no
+balance behind it.
+
+It found two things no other test could.
+
+**`CancelOrder` was signed without the solver.** The server's struct is
+`CancelOrder(address solver, bytes32 commitment, uint64 timestamp)`, and we signed the last two, so
+every cancel came back `SOLVER_MISMATCH`. Withdrawing a limit order left its entry live, and because
+`repost` cancels before it posts, a resize would have put a second entry behind the same liability,
+which is the one thing section 6 sets out to avoid. `Heartbeat` already named its solver and was
+unaffected.
+
+**`backed: false` does not mean under-funded.** It is also false before any balance cycle has read
+the order, and the order surfaces at its full quoted size until one does, so reconciliation was
+stamping `UNDER_FUNDED` on healthy postings seconds after they went up. `PostedOrder` now carries
+`validatedAt`, and only a `backed: false` a cycle actually decided counts, alongside `resized`.