Move image references to shared content storage - #449
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
@cursor review |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6b7d984. Configure here.
| if resolveErr == nil && generationMatches && (current == ref.DigestHex() || current == meta.PreviousTagDigest) { | ||
| if err := createTagSymlink(m.paths, ref.Repository(), meta.RequestedTag, ref.DigestHex()); err != nil { | ||
| fmt.Fprintf(os.Stderr, "Warning: failed to create tag symlink: %v\n", err) | ||
| } |
There was a problem hiding this comment.
Tag generations lost on restart
High Severity
Last-pull-wins uses the in-memory tagGenerations map, but RecoverInterruptedBuilds never restores it. After restart that map is empty, so a recovered pull's TagGeneration cannot match and finalizeImage skips the tag update. A later pull of the same tag also restarts the counter at 1 and can collide with persisted generations, letting a stale build steal the tag.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 6b7d984. Configure here.
| } | ||
| if err != nil { | ||
| return nil, true, fmt.Errorf("create image tag: %w", err) | ||
| } |
There was a problem hiding this comment.
Ready tag claim skips generation bump
High Severity
reuseExistingImage immediately repoints a tag when the digest is already ready, but it does not call nextTagGeneration. An in-flight last-pull-wins pull of a different digest still holds the current generation, so when it finishes finalizeImage treats the ready claim as the previous digest and moves the tag back. The same gap exists in ImportLocalImage.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 6b7d984. Configure here.
promoteImageToContent was implemented but never invoked, so ready per-repository images were never migrated into the digest-keyed content layout. Promote them at manager startup: hardlink the rootfs into shared content, repoint repository tags, and retire the legacy tree. Non-ready images are left untouched and failures only warn.
Tag symlinks now point relatively into the shared content directory, so assert on the resolved digest instead of the raw link target. Deleting a digest whose build is still in flight keeps the shared content and a re-import joins that build, so the recreate-race test must expect the same build id. Recovery writes metadata through the layout resolver, so the credentials scrub check reads the content metadata path. Reuse metadataStatus for content metadata instead of duplicating the parse.
New builds write rootfs and metadata under the shared content directory, so EnsureImageReady must copy from the resolved disk location instead of the legacy per-repository digest directory and create its tag symlink in the repository references layout.
7aac5c0 to
e3cab9b
Compare
1 participant
summary
Move image storage from repository-local digest directories to digest-keyed shared content, while keeping existing image operations and legacy data readable during migration.
images/content/<digest>images/repositories/<repository>/<tag>This change does not deduplicate or compose OCI layers.
pre-existing flows
The existing image APIs continue to support the current pull/import, lookup, wait, list, and delete flows:
CreateImageandImportLocalImagestill reuse an existing digest, return in-progress status, enforce credential matching, and apply Docker-style last-pull-wins tag behavior.GetImage, listing, and disk-path resolution can read both the legacy repository-local layout and the new shared-content layout.DeleteImageremoves a tag without deleting content still referenced by another tag or an active pull. Explicit digest deletion still removes the digest content.new flows
data model and storage changes
The on-disk layout changes from repository-local image data to shared content with separate repository tag references:
Legacy paths remain readable during migration. Image metadata gains fields for tag ownership and stale-build protection:
requested_tagprevious_tag_digesttag_generationThese fields prevent an older asynchronous pull from overwriting a tag after a newer pull or delete. Disk accounting identifies hard-linked rootfs files so aliases count once.
UX and behavior changes
WaitForReadycan observe a tagged pull before its final symlink is installed, reducing races in the registry conversion flow.validation
GOCACHE=/tmp/hypeman-go-cache go test -race -run 'Test(DeleteImagePreservesCrossRepositoryContent|DeleteTagRemovesBothLayoutReferences|LegacyImageIsNotShadowedByContentMetadata|ListAllMetadataDeduplicatesDualLayouts|FailedLegacyImageUsesReadyContent|ReadyContentDoesNotFallBackToLegacyDisk|WriteMetadataUsesContentWhenLegacyDirectoryIsEmpty|ContentLayoutResolvesDiskByDigest|ListAllMetadataContentLayout|TotalReadyImageBytes|ImageMetadata|TagFollowsLastPull)' ./lib/images ./lib/paths -count=1GOCACHE=/tmp/hypeman-go-cache go test -run '^$' ./lib/images ./lib/pathsmkfs.erofs.