Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions lib/images/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ Content-addressable storage with tag symlinks (similar to Docker/Unikraft):
rootfs.erofs
latest -> abc123def456... # Tag symlink to digest
3.18 -> def456abc123... # Another tag
layers/ # Content-addressed per-layer erofs artifacts (layer_artifacts.go)
<diff-id-hex>/ # sha256 of the layer's uncompressed tar (rootfs.diff_ids)
layer.erofs # Layer contribution converted with mkfs.erofs -zlz4
metadata.json # Source blob digest, diff id, format, size
system/
oci-cache/ # Shared OCI layout for all images
index.json # Manifest index with digest-based tags
Expand Down Expand Up @@ -93,6 +97,48 @@ Content-addressable storage with tag symlinks (similar to Docker/Unikraft):
- Orphaned digests are automatically deleted when the last tag referencing them is removed
- Symlinks only created after successful build (status: ready)

## Layer Artifacts (layer_artifacts.go)

`ExportLayerArtifacts` converts individual OCI layers from the shared OCI
cache into content-addressed erofs artifacts under `images/layers/<diff-id>/`,
keyed by the layer's diff ID (the sha256 of its uncompressed tar, from the
image config's `rootfs.diff_ids`). A layer shared by any number of images
converts once; the unpacked stream is hashed and verified against the config's
diff ID during export.

Scope and behavior:

- Reusable exporter only: the flattened image build path (`buildImage` -> one
rootfs per image digest) and VM boot are unchanged, and nothing is recorded
in image metadata yet.
- Layers with an unsupported media type (e.g. zstd) or that cannot be
unpacked standalone (e.g. hardlinks into an earlier layer) are reported as
skipped, not errors.
- OCI deletion semantics are handled explicitly: a layer containing whiteout
entries (`.wh.<name>`) or opaque-directory markers (`.wh..wh..opq`) deletes
content that lives in earlier layers, which a standalone read-only artifact
cannot express. Such layers are skipped rather than converted, because a
raw tar-to-erofs conversion would silently drop the deletions.
- Each artifact's `metadata.json` is the contract for reading it back: source
blob digest, diff ID, filesystem format and the options it was built with
(erofs `-z` compression, sector alignment), size.
- Requires Linux and `mkfs.erofs`; otherwise returns
`ErrLayerArtifactsUnsupported`.

Known limitations a future layer-composition step must resolve:

1. Artifacts hold only what a layer adds; layers that delete are skipped.
Composing images from per-layer artifacts needs a format that can carry
deletions (overlayfs-style or custom), which hasn't been chosen yet.
2. The image -> ordered artifact mapping is returned to the caller but not
persisted; `imageMetadata` gains layer fields once a composition consumer
exists. No separate manifest-metadata change exists yet; the exporter
derives everything it needs from the manifest and config blobs already in
the OCI cache.
3. Artifacts are not reference-counted against the OCI cache GC. Safe today
(an artifact is self-contained once installed), but a GC policy for
`images/layers` is future work.

## Reference Handling (reference.go)

Two types for type-safe image reference handling:
Expand Down
7 changes: 6 additions & 1 deletion lib/images/disk.go
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,11 @@ func convertToCpio(rootfsDir, outputPath string) (int64, error) {
// sectorSize is the block size for disk images (required by Virtualization.framework)
const sectorSize = 4096

// ErofsCompression is the compression algorithm passed to mkfs.erofs via -z.
// It is part of the on-disk contract for erofs images: readers (and future
// layer-artifact consumers) rely on the kernel supporting it at mount time.
const ErofsCompression = "lz4"

// alignToSector rounds size up to the nearest sector boundary
func alignToSector(size int64) int64 {
if size%sectorSize == 0 {
Expand Down Expand Up @@ -202,7 +207,7 @@ func convertToErofs(rootfsDir, diskPath string) (int64, error) {
// Create erofs image with LZ4 fast compression
// -zlz4: LZ4 fast compression (~20-25% space savings, faster builds)
// erofs doesn't need pre-allocation, creates file directly
cmd := exec.Command("mkfs.erofs", "-zlz4", diskPath, rootfsDir)
cmd := exec.Command("mkfs.erofs", "-z"+ErofsCompression, diskPath, rootfsDir)
output, err := cmd.CombinedOutput()
if err != nil {
return 0, fmt.Errorf("mkfs.erofs failed: %w, output: %s", err, output)
Expand Down
Loading
Loading