Differential Dockerfile fuzzer: docker-vs-kaniko parity and cache-consistency oracles

[mzihlmann]

Draft, not intended for merge. This records the differential-fuzzing approach so the team can see and discuss it.

What it does

A test-only differential fuzzer that generates a random Dockerfile plus a matching build context from a seed, builds it with both docker buildx and kaniko, and compares the results across several oracles. It runs under go test -run TestFuzz with FUZZ_CASES=N or FUZZ_DURATION=30m, against a local registry, and never touches the normal suite unless asked.

Oracles

Each generated case is checked by more than one oracle, so one Dockerfile exercises many code paths at once.

• Docker parity. Build with docker and kaniko, compare the images with diffoci. A diff that matches no known class is a real divergence.
• Cache populate and consume. Build twice into a shared cache and assert the consume build is byte-identical to the fresh build, for both the registry cache and the local OCI-layout backend.
• Determinism. Two --reproducible builds must be byte-identical.
• Cache-lookahead invariance. The aggregate cache key must not change the output.
• Shared-cache collision. Two builds whose composite keys once collided must not contaminate each other.
• Context shapes. The same case is fed through a directory context, a tar context, and an https-served tar context.
• Dockerfile from an http URL, digest-file, two-step build then executor push, and a secret-leak oracle that asserts a mounted secret never lands in any pushed layer or config.

How findings stay signal-rich

• The generator is byte-driven and fully deterministic, so a seed reproduces a case exactly, and coverage-guided admission keeps seeds that reach new blocks.
• A finding is greedily shrunk to a minimal reproducer before it is reported.
• Known, already-filed divergences are matched by shape and counted rather than reported, so a novel bug is never buried under a known one. Each entry is removed once its issue is fixed on main, which re-arms the fuzzer against a regression.

In-process oracles

pkg/util/tar_util.go and pkg/snapshot/snapshot.go add assertions over the layer-writing invariants (an entry is written at most once, an entry never coexists with its whiteout, the kaniko directory never leaks into a layer, a hardlink target is present in the same tar). These do not change behaviour on the correct path. They turn silent layer corruption into a labelled crash the fuzzer can catch, the same idea as a sanitizer.

Outcome so far

The approach surfaced several real divergences. Some already have fixes on main (mz864 WORKDIR ownership, mz879 runv2 mount cache key, mz921 self-referential symlink). Others are filed and open (mz897 named chown on scratch, mz922 chmod on implicit parent dirs, the mz876 snapshot nondeterminism). The branch tracks each with a known-class entry until its fix lands.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7a724e3d-3d3f-4437-a578-66672d2d4212

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/util/tar_util.go	87.50%	2 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!