Skip to content

Layer-aware accounting and GC groundwork for shared content - #451

Closed
chruffins wants to merge 2 commits into
hypeship/content-addressed-image-storagefrom
hypeship/layer-aware-gc-groundwork
Closed

Layer-aware accounting and GC groundwork for shared content#451
chruffins wants to merge 2 commits into
hypeship/content-addressed-image-storagefrom
hypeship/layer-aware-gc-groundwork

Conversation

@chruffins

@chruffins chruffins commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Stacked on #449 (content-addressed image storage).

summary

Layer-aware accounting and garbage-collection groundwork that does not depend on runtime composition:

  • Manifest-to-layer references: image metadata now records the layers from the pulled manifest (digest, size, media type) as LayerRef entries, written at finalize from the already-read manifest. No separate manifest-metadata store — this extends the existing imageMetadata only (see sequencing note below).
  • Count shared artifacts once: computeLayerAccounting sums referenced layer bytes with each layer digest counted once regardless of how many images reference it; exposed as hypeman_images_referenced_layer_bytes{scope=unique|shared} from the existing metrics callback. Rootfs disks were already deduped by inode in Move image references to shared content storage #449; OCI cache blobs are content-addressed files counted once by the existing walk.
  • Protect referenced/in-flight artifacts during cleanup: LiveOCICacheDigests returns the manifest digest plus recorded layer digests of every non-failed image; images.OCICacheRoots feeds them into the existing ocicachegc collector as extra roots (wired into the composite roots in cmd/api). Failed images contribute nothing so their blobs stay collectable. Layer digests are marked live as opaque leaves — no manifest parsing needed.

what this does not do (per scope)

  • no OCI layer dedup or runtime composition
  • no hypeman tag API/CLI
  • no VM boot changes

sequencing / dependency note

The layer reference list lives in imageMetadata.Layers and is populated at finalize. If a follow-up introduces a dedicated manifest-metadata store, Layers should migrate there (or be dropped in favor of it); nothing else in this PR depends on the field's location. In-flight protection detail: a queued build's metadata carries the manifest digest immediately, which protects blobs once the manifest blob exists on disk; layer blobs written before the manifest blob during AppendImage remain covered by the collector's existing min_blob_age grace period.

validation

  • go build ./... (embedded hypervisor/caddy/guest-agent binaries stubbed locally; stubs are gitignored)
  • go vet ./lib/images/... ./lib/ocicachegc/... ./cmd/api/... clean
  • New tests (all pass, also under -race):
    • TestLiveOCICacheDigestsProtectsReferencedAndInflightArtifacts
    • TestComputeLayerAccountingCountsSharedLayersOnce
    • TestOCICacheGCProtectsImageReferencedBlobs (real collector sweep: metadata-referenced blob survives, orphan is collected)
    • TestFinalizeImageRecordsLayers
    • TestExtractOCIImageDetailsReturnsLayerRefs
  • Move image references to shared content storage #449's validation set still passes: go test -race -run 'Test(DeleteImagePreservesCrossRepositoryContent|DeleteTagRemovesBothLayoutReferences|LegacyImageIsNotShadowedByContentMetadata|ListAllMetadataDeduplicatesDualLayouts|FailedLegacyImageUsesReadyContent|ReadyContentDoesNotFallBackToLegacyDisk|WriteMetadataUsesContentWhenLegacyDirectoryIsEmpty|ContentLayoutResolvesDiskByDigest|ListAllMetadataContentLayout|TotalReadyImageBytes|ImageMetadata|TagFollowsLastPull)' ./lib/images ./lib/paths
  • ./lib/ocicachegc ./lib/registry ./lib/imagepush ./lib/imageretention tests pass

Pre-existing failures on the base branch (verified identical on #449's head, environment/network related, not caused by this PR): TestImportLocalImageFromOCICache, TestDeleteAndRecreateDuringBuildTail, TestRecoverInterruptedCredentialed*, and the docker.io-pulling tests (TestCreateImage*, TestListImages, TestGetImage, TestDeleteImage*, TestLayerCaching).


Note

Medium Risk
Changes OCI cache garbage collection roots—incorrect digest rooting could delete still-needed blobs, though failed images are excluded and tests exercise a real collector sweep.

Overview
Image metadata now records manifest layers (LayerRef: digest, size, media type) when a pull finishes, replacing a simple layer count on pullResult. That list is what GC and metrics use without re-reading manifests.

OCI cache GC keeps blobs alive from metadata, not only from index.json, registry BuildKit tags, and in-flight pushes. LiveOCICacheDigests / images.OCICacheRoots supply manifest plus layer digests for every non-failed image (failed images contribute nothing); the API wires this into the existing composite RootsProvider. Layer digests are treated as opaque live leaves in ocicachegc.

Observability: computeLayerAccounting dedupes shared layers across images and exposes hypeman_images_referenced_layer_bytes with scope=unique|shared.

Tests cover digest rooting, accounting, finalize persistence, extraction, and an end-to-end GC sweep where metadata-only references survive.

Reviewed by Cursor Bugbot for commit 9b8e641. Configure here.

Record manifest layer references (digest, size, media type) in image
metadata at finalize, count shared layers once in disk accounting, and
protect metadata-referenced and in-flight artifacts from OCI cache
garbage collection by contributing them as extra GC roots.

No runtime composition, tag API, or VM boot changes.
@chruffins chruffins closed this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant