Skip to content

[3/6] feat(snapshot): template build-context archive store (posixfs + oss) - #71

Open
JoyboyBrian wants to merge 3 commits into
kvcache-ai:mainfrom
JoyboyBrian:e2b-build/03-build-file-store
Open

[3/6] feat(snapshot): template build-context archive store (posixfs + oss)#71
JoyboyBrian wants to merge 3 commits into
kvcache-ai:mainfrom
JoyboyBrian:e2b-build/03-build-file-store

Conversation

@JoyboyBrian

@JoyboyBrian JoyboyBrian commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

What

Add repository-backed template build-context archive storage for POSIX and OSS deployments, including immutable content-addressed archives, single-use upload grants, bounded pruning, and a SnapshotManager accessor.

Layer PR Branch Scope
1/6 #69 e2b-build/01-error-envelope JSON 404 envelope for unmatched routes
2/6 #70 e2b-build/02-alias-rebuild Alias rebuild semantics
3/6 #71 e2b-build/03-build-file-store Build-context archive store
4/6 #72 e2b-build/04-upload-api Upload API and configuration
5/6 #73 e2b-build/05-copy-plan Host-side COPY planning
6/6 #74 e2b-build/06-copy-exec COPY/ADD execution and documentation

Why

The maintainer asked that the full change in #28 be split into smaller reviewable PRs. This is layer 3 of that six-PR stack and provides the shared repository primitive needed for cross-node E2B build-context uploads and later COPY execution.

Related issue

Related: #28

#28 remains the full reference PR containing the design discussion, prior review history, and two-node E2E evidence.

Scope and non-goals

This layer adds only the storage interfaces, POSIX/OSS implementations, module wiring, backend accessors, and storage tests. It also contains the minimal three-line semantic rebase fix that supplies the new store argument at two upstream-added image-export repository construction sites. Layer 4 exposes the HTTP upload contract. Layer 6 consumes stored archives during template builds. Upload limits and TTL configuration intentionally remain in layer 4 because the store implementations do not read runtime config.

Until the preceding PRs merge, the Files view also shows their commits (this branch stacks on them); this layer itself = the last commit, or JoyboyBrian/AgentENV@e2b-build/02-alias-rebuild...e2b-build/03-build-file-store.

Design and behavior changes

TemplateBuildFileStore abstracts content-addressed archive lookup/import/materialization plus issue/verify/claim upload grants and pruning. POSIX publication uses staging, fsync, and no-clobber hard links with first-write-wins behavior. OSS uses repository object keys for the same logical records and grants. Snapshot repositories may return None; POSIX and OSS wire concrete stores. The POSIX concrete store is re-exported crate-locally so upstream's image-export service can construct the now-three-argument repository.

Compatibility and operations

  • Public API or generated protocol: N/A; no HTTP surface in this layer.
  • Configuration or defaults: N/A; limits and TTLs arrive with layer 4.
  • Snapshot manifest, artifact layout, or storage format: Adds build-context archive/grant data beneath the configured snapshot repository; committed snapshot manifests are unchanged.
  • Upgrade and rollback: Additive storage layout. Old repositories remain readable; unreferenced build-context objects can be pruned.
  • Host requirements, permissions, ports, or dependencies: No new crates or host services; POSIX requires the same repository filesystem semantics already used by snapshots.

Validation

  • make fmt
  • make clippy
  • make test-unit
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo test -p agentenv --lib
  • Relevant Rust integration tests
  • make -C services test (required when services/ changes)
  • Generated clients/server regenerated with the documented make target
  • Documentation updated
  • Benchmarks or performance comparison completed

Commands and results:

make fmt
cargo clippy --workspace --all-targets --all-features -- -D warnings
TMPDIR=/dev/shm/agentenv-e2b-tests cargo test -p agentenv --lib

All listed commands pass in the Linux verification environment. The tmpfs TMPDIR avoids a pre-existing ETXTBSY from devmachine's shared /tmp in an unrelated credential-helper test. Store unit tests cover immutable import, first-write-wins behavior, grants, claims, pruning, mtime refresh, no-clobber publication, and failure cleanup. #28 retains the two-node E2E evidence for an upload stored on one node and consumed through the shared repository path.

Skipped checks and reasons: service tests, code generation, docs checks, integration tests requiring the full later API/executor layers, and benchmarks are outside this layer's scope.

Risks and reviewer notes

The important review surfaces are grant single-use behavior, no-clobber publication, and bounded cleanup. The two mixed repository files are restored in full from the reviewed source at this layer, so the stack converges without carrying hand-edited variants forward. Review this layer through the last commit or compare link.

Checklist

  • The PR contains one coherent change and no unrelated formatting or refactoring.
  • New behavior is covered by tests, or I explained why testing is impractical.
  • Logs and examples contain no credentials, tokens, or private registry information.
  • I did not manually edit generated code without updating its source and regenerating it.

@github-actions

Copy link
Copy Markdown
Contributor

OpenCodeReview: Some files could not be reviewed due to errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant