Skip to content

Build the Rust organizational graph platform - #2058

Merged
jonathanhaaswriter merged 33 commits into
mainfrom
agent/organizational-graph-kernel-v2
Jul 24, 2026
Merged

Build the Rust organizational graph platform#2058
jonathanhaaswriter merged 33 commits into
mainfrom
agent/organizational-graph-kernel-v2

Conversation

@jonathanhaaswriter

@jonathanhaaswriter jonathanhaaswriter commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • define sealed Rust types for tenants, provider identities, canonical identities, entities, relationships, collections, and graph deltas
  • compile every checked-in source family into a closed Rust projection class
  • enforce unified workforce identity with authoritative employee anchors and bounded provider matching
  • commit observations, graph state, revisions, parity receipts, authority, and projection outbox work atomically in PostgreSQL
  • project tenant current state into Neo4j
  • consume canonical source events directly with a durable Rust pull consumer
  • ship the Rust platform binary and compiled source catalog in the release image
  • make Rust authoritative for bounded neighborhood reads used by web, Slack, MCP, reports, and the graph agent
  • remove the unused npm toolchain from the production web image

Hard boundaries

  • provider identity and canonical identity are different Rust types
  • general relationships cannot create identity bindings
  • one provider identity cannot bind to two people
  • one authoritative claim cannot produce two people
  • email claims require an employee anchor
  • GitHub requires verified email; Slack can only match an existing claim
  • relations are a closed enum with checked endpoint kinds
  • assertions carry tenant-scoped observation provenance
  • incomplete or incremental collections cannot retract missing assertions
  • PostgreSQL forces tenant row-level security and repeats identity uniqueness checks inside the transaction
  • every entity has one tenant-scoped agent key that can be used as the next graph root
  • agents receive bounded graph operations, not Cypher or graph-write access

Read authority

The Rust API resolves a one-hop root, its bounded edges, and the graph revision in one tenant-scoped Neo4j query. The batch route accepts one tenant and at most 100 roots. Missing roots are omitted, ambiguous aliases fail closed, and the Go adapter rejects cross-tenant entities, duplicate agent keys, unrequested roots, and edges outside the returned neighborhood.

The Go compatibility store remains available only for raw Cypher callers. Product neighborhood reads fail closed if the Rust graph is unavailable.

Exact-image Rust proof

The candidate release now runs one disposable end-to-end scenario against the exact signed image digest before stable promotion:

  • PostgreSQL 16, Neo4j 5, and JetStream receive real canonical append-log events
  • Okta and Slack resolve to one canonical workforce identity
  • Auth0 grants produce a user-to-application-to-API path
  • the cutover gate rejects authority before three parity matches and promotes after the third match
  • bad credentials and cross-tenant reads fail with 401 and 403
  • the consumer is restarted from the same image and resumes from durable state
  • replaying an accepted event does not create a second revision
  • malformed protobuf is rejected

The proof emits cerebro.rust-organizational-e2e/v1, binds the receipt to the image digest, and attaches it to that digest as a signed attestation. Stable promotion verifies the receipt hash, commit, digest, status, ten passed checks, and attestation before publishing the release.

Performance

On the same local five-sample, 300 ms corpus:

  • Rust projection: 2,247 ns/record at 1,000 records
  • equivalent Go raw-record projection: 3,079 ns/record
  • Rust projection plus admission: 2,529 ns/record

The one-hop HTTP path moved from 6.347 ms/request to 1.712 ms/request at concurrency 1. At concurrency 16 it moved from 1,085.7 to 3,939.5 requests/second.

Seven sequential one-hop durable reads took 27.251 ms per set. One seven-root query took 2.588 ms per set.

On an isolated AWS m7g.xlarge with PostgreSQL 16 and Neo4j 5 on loopback, the median 1,000-record durable commit plus projection moved from 1,137.3 ms to 176.1 ms, a 6.46x throughput increase. Eight concurrent 100-record tenant commits moved from 284.3 ms to 151.5 ms, a 1.88x increase. Each revision ran three times on the same warm stores.

Methods and caveats are in docs/engineering/rust-organizational-platform-benchmarks.md.

Validation

  • cargo clippy --locked --workspace --all-targets -- -D warnings
  • cargo test --locked --workspace --all-targets
  • exact release image against disposable PostgreSQL, Neo4j, and JetStream: 10 checks passed
  • exact-image restart recovered revision 4 and committed revision 5
  • Go race suite and 88.5% coverage gate
  • Rust formatting, warning-denied Clippy, full workspace tests, docs, and 91.17% line coverage
  • 58 MCP routing cases plus authenticated ten-tool SDK compatibility smoke
  • 744 finding-policy suites and generated catalog, API, and policy-mapping checks
  • live disposable PostgreSQL and Neo4j tests for conflict rollback, outbox recovery, irreversible cutover, multi-hop reads, and Rust-only promoted projection
  • runtime and web release-image smoke tests
  • release-pinned Trivy CRITICAL scan with zero findings
  • amd64 and arm64 web-image server smoke tests

This pull request contains the portable runtime, contracts, and release proof. It does not claim a production deployment.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant