[runtime]: Glamsterdam (Gloas) support for the sync committee client

[dharjeezy]

Prepares the Ethereum sync committee client for Glamsterdam (CL name gloas, EIP-7732 ePBS), moves the SSZ layer from ssz-rs to Lighthouse's stack, and picks the fork at runtime rather than at build time.

Closes #1011.

The consensus problem

ePBS removes BeaconState.latest_execution_payload_header, the field the verifier proves today for state_root, block_number and timestamp. It is replaced by latest_block_hash, and the real ExecutionPayload moves to a separately gossiped envelope the sync committee never signs. After Gloas there is no SSZ path from the signed header to the execution state root.

The fix is to prove latest_block_hash, have the relayer supply the EL block header, and check keccak256(rlp(header)) == block_hash before decoding the fields out of it. Same trust assumptions as today. Verifying the payload envelope through the PTC was rejected: the PTC attests to payload timeliness, not the state root, so it needs the same block-hash-to-state-root binding anyway while adding a second committee path.

The post-Glamsterdam EL header has 23 RLP fields, not 21: it appends block_access_list_hash (EIP-7928) and slot_number (ePBS). The 21 and 22 field encodings both produce the wrong hash.

No feature flag: the fork is data

Gloas does not extend the pre-Gloas state, it reshapes it. The payload header becomes a block hash, eight builder and payload-timeliness fields are appended, twelve lists become ProgressiveList, and the container merkleizes progressively. Those are two different SSZ types and no single Rust struct is both.

They were briefly one struct behind a glamsterdam feature, which meant the fork was chosen when the binary was built: a relayer had to be rebuilt at the fork, and a binary built for one side could not read a state from the other. The beacon api already reports which shape it is sending, so BeaconStateElectra and BeaconStateGloas are now separate types behind an enum, selected from the version field:

#[serde(tag = "version", content = "data", rename_all = "lowercase")]
pub enum Response<..> {
    #[serde(alias = "phase0", alias = "altair", .., alias = "fulu")]
    Electra(BeaconStateElectra<..>),
    Gloas(BeaconStateGloas<..>),
}

Blocks needed no such split. Only four of their fields are ever read here and none are merkleized, so one lean BeaconBlockSummary deserializes either fork.

The verifier and ISMP client were already fork-agnostic: they dispatch on the epoch of the signed slot, so an update cannot select its own verifier, and a variant that disagrees with the fork is a hard InvalidUpdate.

Generalized indices

Gloas adopted progressive SSZ merkleization (EIP-7688, EIP-7916), so every index moved:

Field	Before	After	Depth
finalized_checkpoint	84	367	8
next_sync_committee	87	2946	11
execution leaf	88	2947	11

Branch depths are no longer uniform. is_valid_merkle_branch now takes the generalized index alone and derives the depth from it, so the per field *_INDEX_LOG2 constants are gone.

Root keeps its wire format

ssz-rs's Node SCALE-encodes as 32 raw bytes. Routing Root through ByteVector<U32> would have encoded it as a Vec<u8>, adding a compact length prefix: 33 bytes leading 0x80. VerifierState.finalized_header carries three of them, so every stored Ethereum and Gnosis ConsensusState would have failed to decode after a runtime upgrade, and relayers still on the old format would have failed with DecodeBeaconClientUpdate.

Root is therefore a newtype whose SCALE impl writes the 32 bytes raw, with tests pinning that standalone and nested inside a struct, which is how it actually arrives in a consensus update. It is Copy again too, as Node was.

The fork dependencies

ssz-rs has no progressive support, so rather than maintain our own the client moved to the crates Sigma Prime maintains for Lighthouse. That stack had gaps, filled in five forks under polytope-labs, all now merged:

• tree_hash already had progressive container merkleization. Added: no_std, gindex addressed proof generation, multiproofs, and a ContainerFields/TreeHashFields split so a progressive container cannot reach the balanced multiproof builder.
• ssz_types gains ProgressiveList<T>, which exists nowhere upstream, plus no_std.
• ethereum_ssz gains no_std, a Default for BitList, and derives that emit core paths so no_std consumers can use the generated code.
• ethereum_hashing and ethereum_serde_utils gain no_std, since the chain has to build for the runtime.

SCALE lives here, not in the forks

The forks originally carried the SCALE impls for the ssz containers and bitfields. That is the one change upstream will never take, since it puts a Substrate dependency inside an SSZ library, and it meant the forks could never be retired: every upstream security patch had to be merged by hand rather than rebased.

The codec now sits on types this repo owns, so the ssz crates need no codec at all.

Type	Where	SCALE encoding
Root	primitives/src/ssz/root.rs	32 raw bytes
ByteVector<N>	bls-utils	the inner Vec<u8>
SyncCommittee<N>	primitives/src/scale.rs	keys as a Vec, then the aggregate
SyncAggregate<N>	primitives/src/scale.rs	bits as Vec<bool>, then the signature

Those are the only types reachable from the runtime boundary that hold an ssz container. That was established by walking the type graph out from VerifierState, VerifierStateUpdate and ConsensusState, not by inspection, and the 26 codec derives that turned out to be unreachable are gone.

The encodings are byte for byte what the derives produced, which in turn matched ssz-rs. A bitfield still encodes as Vec<bool>, one byte per bit, which is wasteful but is what previously encoded updates contain. Decoding re-applies each container's length rule, so a wrongly sized committee or an over long bitfield is rejected at the boundary rather than surviving as an invalid value. That check matters most for ByteVector: an over length value merkleizes to the same root, so a consumer authenticating only by hash_tree_root would not notice.

No crate in the workspace enables a scale feature any more, and parity-scale-codec no longer appears in the resolved dependency graph of either ssz crate.

The pins here point at the fork mains, which carry everything this client needs: no_std, ProgressiveList, the proof generation and the container changes.

Pinning the upstream review branches instead was tried and reverted. Two reasons. They move whenever a maintainer asks for a change, and more decisively, ssz_types#85 was cut down to no_std only at the maintainer's request because #84 already covers EIP-7916, so that branch no longer contains ProgressiveList at all. The upstream branches serve the upstream PRs; the forks serve this client until the crates are released.

While the fork mains still carry the scale code, it is inert: nothing enables the feature, so none of it compiles. ethereum_ssz#2 and ssz_types#2 remove it, which also drops parity-scale-codec from the resolved dependency graph.

Migration notes for reviewers

Most of the diff is one mechanical change with awkward consequences: ssz_types takes its bounds as type level integers where ssz-rs used const N: usize.

• Bound constants gained same named pub type aliases. Rust keeps types and values in separate namespaces, so VariableList<T, MAX_DEPOSITS> picks up the type and vec![0; MAX_DEPOSITS] the constant, and no use site had to change.
• Unused type parameters are an error where unused const parameters were not, which is why the relayer's bounds also needed Send + Sync + 'static: they now flow through a PhantomData into an async host.
• hash_tree_root() returned a Result; tree_hash_root() is infallible.

Validation

Both forks, live chains, from the same binary:

glamsterdam-devnet-8                     kurtosis devnet (pre-gloas)
beacon_state_hashes_to_the_signed_header  fetch_* (5), state_root_and_block_header_root_matches
execution_header_recovers_the_state_root  test_execution_payload_proof, test_finalized_header
verifier_accepts_a_real_gloas_update      test_prover, test_sync_committee_update_proof
verifier_rejects_tampered_gloas_updates
4 passed, 0 failed                        all passed

beacon_state_hashes_to_the_signed_header is the one that matters: it merkleizes the full state and compares against the root the node signed, so it validates our merkleization against an independent implementation rather than against itself. It passes on both devnets from one build, merkleizing an Electra state as a balanced container and a Gloas state as a progressive one, which is the runtime dispatch demonstrated rather than asserted.

Workspace, on this branch:

Check	Result
cargo test --all-targets --locked	109 suites, 456 passed, 0 failed
cargo check --all --benches --locked	clean
gargantua-runtime and nexus-runtime, wasm32-unknown-unknown, no_std	clean
stored ETH0 consensus state decodes and re-encodes byte identically	passes

The stored state test is what guards the wire format. It is 50,412 bytes taken straight from Ismp::ConsensusStates on live Gargantua, so it exercises Root, ByteVector and SyncCommittee against bytes a running chain actually wrote, and it asserts the re-encode matches so a later write cannot corrupt storage either. SyncAggregate is not present in stored state, so it is covered instead by the gloas verifier tests above and by unit tests pinning the Vec<bool> encoding.

The forks pass their own suites (tree_hash 114, ethereum_ssz 123, ssz_types 73, ethereum_hashing 7, ethereum_serde_utils 40) and build for wasm32-unknown-unknown with no_std, verified by a probe crate that declares its own #[panic_handler] and derives the ssz traits, since checking the libraries alone misses both linked-std and derive-emitted-std:: failures.

Not covered: the --ignored integration suites that need their own devnets or RPC secrets (bsc-prover, tendermint-prover, simtests, pharos). BSC's unit tests do run as part of the workspace suite above. test_switch_provider_middleware fails locally because it hardcodes localhost:53001 while our devnet runs elsewhere.

Merge order

This does not block on anything.

• ethereum_ssz#2 and ssz_types#2 delete the now dead scale code from the fork mains. Worth landing, then bumping the pins here, so parity-scale-codec leaves the dependency graph entirely and the forks rebase cleanly onto upstream releases.
• Upstreaming runs in sequence, each blocked on the previous being released: ethereum_hashing#23 and ethereum_serde_utils#21, then ethereum_ssz#86, then tree_hash#53, then ssz_types#85. EIP-7916 is being handled upstream by #84, so ProgressiveList here will follow whatever that lands, including its name. As each crate is released, the matching pin moves from a git revision to a crates.io version and that fork retires.

[Wizdave97]

Remove this feature flag no longer useful

Suggested change

#[cfg(not(feature = "nofulu"))]

[Wizdave97]

This check is not necessary here, if we verify the header hash successfully we don't need this check

[Wizdave97]

@dharjeezy track activation on sepolia here: https://forkcast.org/upgrade/glamsterdam/

[Wizdave97]

Blocking

• Root wire format changed. Old Root was ssz-rs Node([u8; 32]), which SCALE-encodes as 32 raw bytes. New Root is ByteVector<U32> over FixedVector, which encodes as a Vec<u8> with a compact length prefix. I confirmed it with a throwaway test: new encoding is 33 bytes starting 0x80, old is 32. ConsensusState stores VerifierState, whose finalized_header has three Roots, so after a runtime upgrade every stored Ethereum and Gnosis consensus state fails with DecodeConsensusState, and relayers on the old wire format fail with DecodeBeaconClientUpdate. The PR text says encodings match ssz-rs exactly. That holds for ByteVector, BitVector and lists, but not for Root at [constants.rs:10](modules/consensus/sync-committee/primitives/src/constants.rs:10). Fix: make Root a newtype whose SCALE impl writes the 32 bytes raw. That also restores Copy and removes the added clones. The alternative is a storage migration plus a lockstep relayer release, which is far riskier.
• Runtime wasm does not compile. The new seq_of_seq_of_str visitor calls to_string on u64/i64 without importing alloc::string::ToString, at [lib.rs:422](modules/utils/serde/src/lib.rs:422) and [lib.rs:426](modules/utils/serde/src/lib.rs:426). serde-hex-utils is a no_std dependency of the primitives, so cargo check --no-default-features --target wasm32-unknown-unknown fails for the verifier, primitives and the ismp client.
• The relayer does not compile. SyncCommitteeHost still takes const ETH1_DATA_VOTES_BOUND: usize, but the same-named type aliases now shadow the constants in generic-argument position, so [tesseract lib.rs:75](tesseract/consensus/sync-committee/src/lib.rs:75), [:90](tesseract/consensus/sync-committee/src/lib.rs:90) and [:101](tesseract/consensus/sync-committee/src/lib.rs:101) fail with E0747, and the host cannot construct the now type-level prover.
• Three test targets do not compile. [byte_list.rs:148](modules/consensus/sync-committee/primitives/src/ssz/byte_list.rs:148) still uses ByteList::<32> and ssz::serialize. [byte_vector.rs:281](modules/utils/bls-utils/src/ssz/byte_vector.rs:281) uses N + 1 where N is now a type. [bsc prover test.rs:139](modules/consensus/bsc/prover/src/test.rs:139) and [:249](modules/consensus/bsc/prover/src/test.rs:249) call as_bitslice() on the new iterator.

Design gaps for the real fork

Let's investigate if we can do this so we can avoid feature flags

• The prover layout is still selected by the glamsterdam feature. The relayer must be rebuilt at the fork, and a non-Gloas build fails to deserialize a Gloas state. Wizdave97's earlier request to drop the flag is still open. Picking the state type at runtime from the beacon API version field would close both.