Skip to content

framework: guard executor shared state under stateMu - #24

Open
nadahalli wants to merge 1 commit into
mainfrom
tejaswi/executor-statemu-race
Open

framework: guard executor shared state under stateMu#24
nadahalli wants to merge 1 commit into
mainfrom
tejaswi/executor-statemu-race

Conversation

@nadahalli

@nadahalli nadahalli commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

The race

The background refresh loop (EnsureFreshEnclaves) and getLocalNodeAndCapConfig
rewrite shared executor state while Execute reads it concurrently. Only
donMembers/donF were guarded, by donMu.

Left unprotected:

  • nodeID (string): written in getLocalNodeAndCapConfig, read in Execute's
    scoped emitter. A torn string read pairs a new pointer with an old length.
    Memory-unsafe, not just stale.
  • enclaves ([]types.Enclave): written by the refresh loop, read by GetEnclaves.
  • vaultDON.CryptographyThreshold: written by the refresh loop, read in
    GetEncryptedDecryptionShares.

enclaveClient is NOT part of this. It is published once in initLazily under
initializedMutex.

The fix

Renames donMu to stateMu, moves nodeID and enclaves under it, and routes access
through getters/setters. vaultDON.CryptographyThreshold is guarded via
getVaultThreshold/setVaultThreshold; the rest of vaultDON is publish-once at init.

MockMetrics.Emit is guarded so the harness is not itself the race.

Verification

TestExecutor_ConcurrentRefreshAndExecute_NoDataRace: 8 goroutines x 40 iterations
across Execute / EnsureFreshEnclaves / GetEnclaves.

Reverting executor.go to main while keeping the test reproduces it:

WARNING: DATA RACE
Write at 0x00c0001c6fb8 by goroutine 42:
  (*RealExecutor).getLocalNodeAndCapConfig()
  (*RealExecutor).EnsureFreshEnclaves()
Previous read at 0x00c0001c6fb8 by goroutine 53:
  (*RealExecutor).Execute()

Full framework suite green under -race.

The background refresh loop (EnsureFreshEnclaves) and getLocalNodeAndCapConfig
rewrite nodeID, enclaves and vaultDON.CryptographyThreshold while Execute reads
them concurrently. Only donMembers/donF were guarded, by donMu.

A torn read of the nodeID string pairs a new pointer with an old length, which is
memory-unsafe, not just stale.

Renames donMu to stateMu, moves the unguarded fields under it, and routes access
through getters/setters. MockMetrics.Emit is guarded too so the test harness is
not itself the race.

Adds a -race regression test. It fails on main today with a write in
getLocalNodeAndCapConfig against a read in Execute.

Ported from confidential-compute#419, which cannot merge now that the repo is
archived.
@nadahalli
nadahalli marked this pull request as ready for review July 29, 2026 11:50
@nadahalli
nadahalli requested a review from a team as a code owner July 29, 2026 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant