Skip to content

perf(control): prove ownership with one etcd read and keep paged reads stable (3/7) - #546

Merged
wchwawa merged 4 commits into
mainfrom
perf/stack-3-ownership-proofs
Oct 21, 2026
Merged

wchwawa merged 4 commits into
mainfrom
perf/stack-3-ownership-proofs

Conversation

@wchwawa

@wchwawa wchwawa commented Oct 20, 2026

Copy link
Copy Markdown
Member

Related Issue

Relates to #543

Part 3 of 7 of the stack for #543. Stacked on #545 (perf/stack-2-session-transport); review only the commits below.

Summary

  • ControlStore::prove_owner reads the shard record, its recovery state, and the owner session in one etcd transaction at one revision, instead of two reads plus a fresh lease keepalive stream. Periodic renewal still extends the lease; other stores keep their behaviour through the default.
  • A local-only owner proves liveness once per request, after execution, instead of before and after. Shared publication keeps its pre-dispatch work.
  • Fix: a full or ranged artifact read now fences its pages on the immutable revision identity instead of the whole PathMetadata. Previously, any publication to another path of the workspace restarted the read, until it ran out of attempts with "artifact metadata changed during a paged read".

Commits:

  • 4cfb527a09 perf(control): prove ownership with one linearizable etcd read
  • 2198970862 fix(client): keep live paged reads valid while other paths change
  • 8ec1684ac9 perf(server): prove local-only liveness once per request

Scope

  • This PR changes one logical boundary only. Not quite: it groups the two ownership-proof changes with the paged-read fix that concurrent writers exposed; the fix can move to its own PR on request.
  • No unrelated refactor, benchmark, metadata model, Holt layout,
    object-store, agent interface, or docs change is mixed in.
  • The linked issue describes the user-visible problem, design decision, or
    maintenance task this PR resolves.
  • Any breaking change is intentional and documented. (none in this PR)
  • No compatibility shim, deprecated alias, or forwarding wrapper was added without a removal condition.

Change Size And Review

  • I checked GitHub's additions plus deletions for this pull request: +250 / -12 (262 lines).
  • If the total is more than 5,000 lines, one core maintainer other than the
    current-head pusher approved the exact current head commit. Not applicable: under 5,000 lines.
  • This change is still small enough for a focused review; approval count is
    not being used to justify mixed package or lifecycle boundaries.

Code Contract (Code Changes Only)

  • Not applicable; this is a docs/config-only change.
  • Package boundaries follow docs/development/code_contract.md.
  • Shared helpers reuse the standard library or existing repository helpers.
    New generic helper modules are domain-neutral and tested.
  • File names and file placement follow the code contract.
  • New types, interfaces, structs, fields, and functions use domain-specific names.
  • New errors are in the owning package's errors.rs and carry stable error kinds when crossing package boundaries (no new error types).
  • New metrics/stats are owned by the package that reports or serves them (none added).
  • Metadata changes document durability, atomicity, revision-reference
    lifetime, snapshot/commit/event retention, GC epochs, and fail-closed
    recovery boundaries. Not applicable: no metadata format or retention change.
  • Placement or routing changes preserve persisted RootId -> LogicalShardId
    affinity, one active writer per shard, owner-epoch fencing, and explicit
    shard-local atomicity; no split root or cross-shard transaction is implied. (owner-epoch fencing and one active writer per shard are unchanged; the proof still shows no successor can own the shard while this owner's session exists)
  • Agent-interface changes keep the transport-free facade and schemas in
    nokv-agent, routing and workflows in nokv-client, and the concrete
    backend plus native full CLI wiring in nokv. Not applicable: no agent-interface change.
  • User-facing integration guidance orders the native full CLI first, the
    direct Python SDK second, and the Rust SDK third, and does not present the
    deprecated nokv mcp sidecar as a supported NoKV surface. Not applicable: no user-facing documentation change.

Claims and Evidence

  • User-facing documentation distinguishes current, experimental, and
    planned capabilities. Not applicable: no user-facing documentation change.
  • Security claims do not treat a Workbench root or Agent scope as tenant
    authentication or authorization (no security claims).
  • Performance claims state the topology, comparison boundary, cache state,
    run count, and raw-evidence location.
  • Benchmark-only behavior does not alter product semantics.

An etcd ownership proof costs about 0.2 ms per acknowledged request. This PR has no separate end-to-end number; the stack-level harness results are in #543.

Topology for every number: one laptop (Apple M3 Pro, 36 GiB, macOS, APFS, F_FULLFSYNC), one NoKV owner, RustFS in Docker, etcd or a local control directory, fresh storage per trial. The method, run counts, and harness are in the evidence gist.

Validation

  • Command and result: on this head with Rust 1.99.1, the toolchain rust.yml pins: cargo clippy --workspace --all-targets -- -D warnings and the feature clippy runs from rust.yml (nokv-meta/nokv-bench with metadata-read-stats, nokv-meta-holt with read-stats, nokv-server and nokv-bench with restore-crash-test-support) are clean.
  • Command and result: workspace and feature tests ran on the stack top 49949ddea8 (perf/stack-7-time-ordered-ids): 1,209 passed, 0 failed, 13 ignored. They were not rerun on this head.
  • Not run (with reason): the live gates (local-WAL successor recovery, Workbench contract, and restore composition) passed on this stack before its rebase onto fix(transport): send each RPC frame in one write and set TCP_NODELAY #533 and were not rerun after it. Linux was not measured.
  • CI note: rust.yml, DCO, and change governance run only for PRs into main, so this PR gets CI once the PR below merges and GitHub retargets it.

Contributor Sign-off

  • Every commit in this PR includes a DCO Signed-off-by trailer.

Acknowledgement barriers only need to show that no successor can own the
shard yet: a successor cannot acquire it while this owner's
lease-attached session exists. ControlStore::prove_owner reads the shard
record, its recovery state and the owner session in one etcd transaction
at one revision, instead of two reads plus a fresh lease keepalive stream.
Periodic renewal still extends the lease. The default delegates to
renew_owner, so other stores keep their behavior.

Signed-off-by: wchwawa <wch19961116@gmail.com>
A full or ranged artifact read loads path metadata and then range-plan
pages, and required every page to repeat the complete PathMetadata.
PathMetadata carries the workspace revision, which every publication to
any other path of the workspace advances, so under concurrent writers an
unaffected read restarted until it exhausted its attempts with
"artifact metadata changed during a paged read".

Fence pages on the immutable revision identity instead: incarnation,
path, generation, revision id, dependency shape and descriptor. A
changed generation or revision still restarts the read, and callers that
need the exact workspace revision keep read_artifact_at_authority.

Signed-off-by: wchwawa <wch19961116@gmail.com>
A local-only owner proved owner liveness both before and after executing
each request. Only shared publication has work to do before dispatch
(repairing an interrupted upload); for local-only the barrier after
execution already proves liveness after everything the request observed
or wrote, so the earlier proof added a control round trip without
fencing anything more.

Signed-off-by: wchwawa <wch19961116@gmail.com>
@wchwawa
wchwawa merged commit 0dfd525 into main Oct 21, 2026
8 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in NoKV Delivery Oct 21, 2026
@wchwawa
wchwawa deleted the perf/stack-3-ownership-proofs branch October 21, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants