Skip to content

[consensus] gate slim block emission on subscriber lag - #27740

Open
arun-koshy wants to merge 1 commit into
ak/minimal-block-codecfrom
ak/minimal-block-lag-gate
Open

[consensus] gate slim block emission on subscriber lag#27740
arun-koshy wants to merge 1 commit into
ak/minimal-block-codecfrom
ak/minimal-block-lag-gate

Conversation

@arun-koshy

@arun-koshy arun-koshy commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Description

Serves full blocks to a subscriber that has fallen too far behind to rebuild minimal ones, and re-decides as its lag changes. Required before consensus_slim_block_propagation can be turned on.

Reconstruction is all-or-nothing across a block's ~100 ancestors: one unresolvable slot fails the whole block, so a small per-ancestor miss rate compounds into near-total failure once a receiver trails the omission horizon. A lagging cohort measured 76% of blocks failing to rebuild against 16% at the frontier, and sending those blocks wastes both sides' work. Full blocks put such a subscriber back on the ordinary path — verify, suspend, fetch missing ancestors — which is how it catches up.

Thresholds are expressed against gc_depth because that is the horizon reconstruction can resolve within. They are asymmetric: a subscription loses slim blocks at the horizon but earns them back only well inside it. Without that gap a subscriber sitting on a single threshold flapped at ~100 transitions/s in each direction, sustained, while the fleet stalled — each flip changes the wire form mid-stream, so neither mode took effect. A dwell timer bounds how often a subscription may switch at all.

Entry uses the strict threshold, since bootstrap starts every subscriber far from the frontier. The gate then keeps tracking: a node that falls behind mid-stream would otherwise be served slim blocks indefinitely on a decision made while it was healthy — a lagging cohort held a ~450-round deficit while still being sent 347 slim blocks/s, 84% of which it could not rebuild.

Catch-up replay is unchanged and still sends full blocks; it serves rounds where the subscriber's state is unpredictable by construction.

Stacked on #27702 — review that first; this targets its branch.

Test plan

Four tests. The mode decision is extracted to gate_next_mode so the hysteresis is testable without a stream harness: the thresholds leave a real gap (mutation-verified — collapsing it fails the test), the mode holds inside the gap from either direction, and it switches outside it. subscriber_lag is tested against a real DagState to confirm it takes the later of the peer's reported position and its observed one.

With the flag off — every chain today — the gate evaluates once at subscribe time to false and the re-evaluation branch never runs.


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • gRPC:
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:
  • Indexing Framework:

@arun-koshy
arun-koshy deployed to sui-typescript-aws-kms-test-env August 18, 2026 07:57 — with GitHub Actions Active
@vercel

vercel Bot commented Aug 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
sui-docs Ready Ready Preview Aug 21, 2026 9:27pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
multisig-toolkit Ignored Ignored Preview Aug 21, 2026 9:27pm
sui-kiosk Ignored Ignored Preview Aug 21, 2026 9:27pm

Request Review

@arun-koshy
arun-koshy force-pushed the ak/minimal-block-codec branch from 490b089 to 25bf122 Compare August 20, 2026 04:09
@arun-koshy
arun-koshy force-pushed the ak/minimal-block-lag-gate branch from f761ac5 to 4bb8425 Compare August 20, 2026 04:13
@arun-koshy
arun-koshy deployed to sui-typescript-aws-kms-test-env August 20, 2026 04:13 — with GitHub Actions Active
@arun-koshy arun-koshy changed the title [consensus] gate minimal block emission on subscriber lag [consensus] gate slim block emission on subscriber lag Aug 20, 2026
@arun-koshy
arun-koshy force-pushed the ak/minimal-block-lag-gate branch from 4bb8425 to 7c49914 Compare August 21, 2026 19:12
@arun-koshy
arun-koshy deployed to sui-typescript-aws-kms-test-env August 21, 2026 19:12 — with GitHub Actions Active
@arun-koshy
arun-koshy marked this pull request as ready for review August 21, 2026 19:24
@arun-koshy
arun-koshy requested a review from a team as a code owner August 21, 2026 19:24
@arun-koshy
arun-koshy deployed to sui-typescript-aws-kms-test-env August 21, 2026 19:24 — with GitHub Actions Active
Serves full blocks to a subscriber that has fallen too far behind to rebuild
slim ones, and re-decides as its lag changes.

Reconstruction is all-or-nothing across a block's ancestors: one unresolvable
slot fails the whole block, so a receiver trailing the omission horizon fails
most of them and both sides waste the work. Full blocks put such a subscriber
back on the ordinary path -- verify, suspend, fetch missing ancestors -- which
is how it catches up.

Thresholds are fractions of gc_depth, the horizon within which a receiver can
still resolve ancestors: slim is lost above three quarters of it and earned back
at half. The gap keeps lag noise around a single boundary from driving the mode,
and stopping short of the horizon leaves room before the receiver's own GC
collects the slots a reconstruction waits on.

Lag is measured only from the peer's own latest block in the local DAG. The
round a subscriber reports when it connects is its own unverified claim, used
solely as the replay resume point.

Re-evaluation is bounded to once per second per subscriber so the fan-out path
does not take the DagState read lock per block. A dwell bounds how fast a
subscription may return to slim blocks; losing them is never delayed.

With the flag off the handler returns the previous fan-out stream unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant