Add per-layer content-addressed erofs artifact exporter

[chruffins]

summary

Reusable per-layer artifact exporter on top of #449's shared content storage:

• images.ExportLayerArtifacts(ctx, paths, imageDigest) reads an image's layers from the existing shared OCI cache (lib/ocicache) and converts each supported layer into a content-addressed EROFS artifact under images/layers/<diff-id>/
• artifacts are keyed by the layer diff ID (sha256 of the uncompressed tar, from the config's rootfs.diff_ids) — a layer shared by any number of images converts once
• the uncompressed stream is hashed during unpack and verified against the declared diff ID (same integrity check umoci does for full unpacks)
• artifacts and their metadata.json install atomically via the existing installAtomically (staged beside the final path, same filesystem)
• reuse is content-addressed: an existing artifact is adopted (healing missing metadata) instead of reconverted
• metadata.json records the filesystem format and options (erofs -z compression via a new ErofsCompression constant, sector alignment) so the artifact contract is complete for readers

Verified against a real pull (public.ecr.aws/docker/library/alpine, since Docker Hub rate-limits here): blobs land at system/oci-cache/blobs/sha256/<hex>, layer descriptors carry …layer.v1.tar+gzip with a matching diffID — exactly the layout and media types the exporter consumes.

Explicitly out of scope, per the parent effort:

• flattened-image behavior and VM boot are unchanged — buildImage still produces one rootfs per image digest; nothing in images/content or tag references is touched (asserted in tests). ExportRootfs/mkfs.erofs is reused, not modified (only a behavior-neutral -z constant extracted)
• no hypeman tag API/CLI, no runtime composition — the image→ordered-artifact mapping is returned to the caller only

explicit OCI whiteout / opaque-directory handling

A raw tar→EROFS conversion does not compose for layers that delete, so the exporter refuses them rather than silently mis-converting:

• Layers carrying deletion semantics are skipped with an explicit reason. OCI whiteouts (.wh.<name>) and opaque-directory markers (.wh..wh..opq) remove content that lives in earlier layers. umoci's standalone UnpackLayer applies those whiteouts against an empty root, which silently drops the deletions — so the exporter scans each layer's tar up front (firstWhiteoutMarker) and reports such layers in Skipped instead of producing a misleading artifact.
• Other skip cases (reported, export continues): unsupported layer media types (e.g. zstd, which the OCI cache cannot decompress) and layers that cannot unpack standalone (e.g. hardlinks referencing files from earlier layers).
• Abort: missing blobs, diff ID mismatches, mkfs.erofs failures. Artifacts installed before an abort remain valid (content-addressed).

documented blockers (no invented types)

Where a correct implementation depends on decisions not made yet, the limitation is documented (doc comment on ExportLayerArtifacts + lib/images/README.md) rather than papered over:

1. Artifacts hold only what a layer adds; layers that delete are skipped. Composing images from per-layer artifacts needs a format that can carry deletions (overlayfs-style vs custom), which isn't chosen yet.
2. The image → ordered artifact mapping is not persisted. imageMetadata gains layer fields once a composition consumer exists. Dependency note: there is currently no separate "manifest metadata" PR; the exporter derives everything it needs from the manifest + config blobs already in the OCI cache, so nothing is duplicated and there is nothing to wait on. If a manifest-metadata change lands later, this mapping is the one field to move.
3. No GC reference counting for images/layers. Safe today — an artifact is self-contained once installed — but a retention policy is future work.
4. Requires Linux + mkfs.erofs; otherwise returns ErrLayerArtifactsUnsupported for callers to soft-skip.

validation

mkfs.erofs and fsck.erofs installed in the test environment, so conversion and artifact-content assertions run for real (not skipped). All tests are synthetic in-memory images written into a temp OCI cache — no live registry required:

GOCACHE=/tmp/hypeman-go-cache go test -race -run 'TestExportLayerArtifacts' ./lib/images/ -count=1 -v

• happy path: two-layer image → two artifacts keyed by diff ID, extracted back with fsck.erofs --extract and contents compared; asserts metadata records format=erofs, compression=lz4, sector_size=4096; asserts nothing written under images/content
• reuse: second export reports Reused, identical sizes, no .unpack-* scratch left, deleted metadata heals
• whiteout / opaque / hardlink layers are skipped with explicit reasons and leave no partial artifact (three dedicated tests)
• zstd layer skipped by media type
• corrupted blob → diff ID mismatch abort with no artifact installed; deleted blob → layer blob missing abort
• empty-PATH run verifies ErrLayerArtifactsUnsupported when mkfs.erofs is absent

Full existing suites (./lib/images ./lib/paths ./lib/ocicache) pass, including PR #449's verification test set. The registry-dependent manager tests (TestCreateImage*, TestGetImage, TestDeleteImage*, TestLayerCaching, …) fail identically on the base branch in this environment: Docker Hub unauthenticated pull rate limits (TOOMANYREQUESTS), unrelated to this change.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 131d5ee. Configure here.}

[cursor]

Restricted layers abort export

Medium Severity

supportedLayerMediaType treats OCIRestrictedLayer as convertible, but ocicache Uncompressed only decompresses standard gzip/tar types. A nondistributable or Docker foreign layer therefore fails at open and aborts the whole export instead of being skipped as an unsupported media type.

Additional Locations (1)

• lib/images/layer_artifacts.go#L235-L241

 

^{Reviewed by Cursor Bugbot for commit 131d5ee. Configure here.}

[cursor]

Diff ID hash may be incomplete

Medium Severity

The diff ID is hashed only from bytes UnpackLayer reads via TeeReader. A tar parser stops at the end-of-archive marker and does not consume trailing uncompressed padding, so the digest can disagree with rootfs.diff_ids and abort export for valid layers.

 

^{Reviewed by Cursor Bugbot for commit 131d5ee. Configure here.}