Skip to content

tracing: nest per-command spans, drop cache-hit source-image spans - #931

Merged
mzihlmann merged 3 commits into
mainfrom
mz-otel-span-nesting
Aug 12, 2026
Merged

tracing: nest per-command spans, drop cache-hit source-image spans#931
mzihlmann merged 3 commits into
mainfrom
mz-otel-span-nesting

Conversation

@mzihlmann

@mzihlmann mzihlmann commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Kaniko's build traces come out flat: pkg/timing parents every span to the root build span and throws away the child context, so operations that are nested in time render as overlapping siblings. The clearest symptom is the per-command initial FS snapshot appearing to run in parallel with the command it belongs to, which is impossible. This threads context through pkg/timing so a Timer can spawn child spans, gives command execution its own span, and hangs both the initial snapshot and the execute span under the command span. A command now reads as a parent bar with its sub-steps nested sequentially inside instead of a pile of overlapping bars.

Stacked on mz-otel-instrumentation.

Summary by CodeRabbit

  • Performance
    • Improved execution timing and tracing for build commands and remote source-image retrieval.
    • Timing data now more accurately reflects active remote operations and nested command activity.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@mzihlmann, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 55 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 61555e3a-bcbe-4bce-80d2-0db5026cf532

📥 Commits

Reviewing files that changed from the base of the PR and between 7630fed and 9d17997.

📒 Files selected for processing (3)
  • pkg/executor/build.go
  • pkg/image/image_util.go
  • pkg/timing/timing.go
📝 Walkthrough

Walkthrough

The timing package now supports child spans. Build snapshot and command execution timing uses those spans. Source-image timing now covers only remote retrieval.

Changes

Timing instrumentation

Layer / File(s) Summary
Child span creation
pkg/timing/timing.go
Start uses shared span-start logic. The new StartChild function creates spans under valid parent spans and preserves existing no-op behavior.
Scoped operation timing
pkg/executor/build.go, pkg/image/image_util.go
Filesystem snapshots and command execution use child spans. Remote source-image timing starts immediately before RetrieveRemoteImage.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the tracing problem and implementation, but it omits the required checklist, reviewer notes, release notes, and issue reference. Add the required template sections, complete the checklists, provide an issue reference when applicable, and state release-note impact or remove that section.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely identifies the two main tracing changes: nested command spans and removal of cache-hit source-image spans.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch mz-otel-span-nesting

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mzihlmann
mzihlmann marked this pull request as draft July 19, 2026 20:22
@mzihlmann mzihlmann changed the title tracing: nest per-command spans under the command span tracing: nest per-command spans, drop cache-hit source-image spans Jul 19, 2026
Base automatically changed from mz-otel-instrumentation to main July 29, 2026 07:12
@mzihlmann
mzihlmann force-pushed the mz-otel-span-nesting branch 2 times, most recently from 1616249 to 6acb868 Compare August 3, 2026 07:10
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mzihlmann
mzihlmann marked this pull request as ready for review August 7, 2026 15:10
@mzihlmann
mzihlmann force-pushed the mz-otel-span-nesting branch from 6acb868 to 66c2886 Compare August 7, 2026 15:10

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/image/image_util.go`:
- Around line 105-107: Ensure timing covers every remote image retrieval,
including the cache-miss path inside cachedImage and the direct
RetrieveRemoteImage call. Move the timing instrumentation to the shared
remote-retrieval boundary or add equivalent timing around the call within
cachedImage, while avoiding duplicate spans for a single request.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6688ac6b-282f-4764-a270-10c66f1f04f7

📥 Commits

Reviewing files that changed from the base of the PR and between 7630fed and 9d17997.

📒 Files selected for processing (3)
  • pkg/executor/build.go
  • pkg/image/image_util.go
  • pkg/timing/timing.go

Comment thread pkg/image/image_util.go
@mzihlmann
mzihlmann requested review from 0hlov3, BobDu, babs and nejch August 9, 2026 11:36
@mzihlmann mzihlmann added the telemetry Everything related to our OTEL exporter label Aug 10, 2026

@0hlov3 0hlov3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mzihlmann
mzihlmann merged commit 5708427 into main Aug 12, 2026
13 checks passed
@mzihlmann
mzihlmann deleted the mz-otel-span-nesting branch August 12, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

telemetry Everything related to our OTEL exporter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants