Skip to content

[I/O profiles 2/4] Add async correlation and io_uring latency - #54

Merged
dalehamel merged 3 commits into
dale/hackdays-io-network-basefrom
dale/io-network-probes-async
Jul 22, 2026
Merged

dalehamel merged 3 commits into
dale/hackdays-io-network-basefrom
dale/io-network-probes-async

Conversation

@dalehamel

@dalehamel dalehamel commented Jul 21, 2026 •

Copy link
Copy Markdown
Member

Stack

  1. LuaJIT unwinding — omnibus snapshot (tarantool + openresty, amd64/arm64) #50: production LuaJIT snapshot (f3e2115a)
  2. Hack Days base: current main, LuaJIT, and Ruby JIT follow-up #59: green Hack Days base: current main + Ruby-JIT ruby: Handle Ruby JIT PC with JIT frame type open-telemetry/opentelemetry-ebpf-profiler#1102 (0a00112e)
  3. [I/O profiles foundation] Integrate upstream ProbeContext API #58: upstream ProbeContext / custom-probes API (a4f279c1)
  4. [I/O profiles 1/4] Add synchronous I/O latency probes #53: synchronous VFS/TCP and block queue latency (9c56b6ec)
  5. [I/O profiles 2/4] Add async correlation and io_uring latency #54 (this PR): async correlation and io_uring latency (f87ae55a)
  6. [I/O profiles 3/4] Add block lifecycle and TCP connect latency #55: block lifecycle and TCP connect latency (1630cf2d)
  7. [I/O profiles 4/4] Add TCP sequence latency profiles #56: TCP send-to-ACK and receive-consumption latency (d046cc66)

Ancestry: #50 ← #59 ← #58 ← #53 ← #54 ← #55 ← #56.

This stack is for the Shopify fork only. No branch or PR is being submitted to upstream OpenTelemetry.

Summary

  • Add a shared bounded asynchronous correlator keyed by (origin, correlation ID).
  • Preserve initiating frames, PID/TID, process/resource attribution, custom labels, and APM IDs until actual completion.
  • Add configurable capacity and TTL plus start, completion, duplicate, orphan, eviction, filtered, and pending metrics.
  • Add io_uring SQE-to-CQE latency profiles with:
    • request-pointer correlation and (ctx,user_data) fallback
    • runtime tracepoint format parsing for Linux layout drift
    • full submitter unwind with kernel fallback for deferred/SQPOLL execution
    • 32/64-bit result handling, cancellation/error labels, completion flags, and multishot retention
  • Add a raw io_uring NOP integration workload.

Base update

Validation

  • Targeted Go tests across tracer, reporter, controller, collector, probes, metrics, support, and process manager.
  • go test ./... -run '^$' compile sweep.
  • golangci-lint: zero issues; vanity-import and eBPF format checks pass.
  • Reproducible amd64/arm64 eBPF builds with clang 17.
  • Privileged arm64 TestAllTracers and io_uring submission-to-CQE integration tests pass on the restacked cumulative tip.

Current CI status

Draft / follow-ups

  • Automated Linux 5.10+ validation is complete; human review and representative SQPOLL/io_uring workload validation remain.

Propagate the green Hack Days and ProbeContext bases without rewriting the asynchronous io_uring commits.
Base automatically changed from dale/io-network-probes-sync to dale/hackdays-io-network-base July 22, 2026 19:06
@dalehamel
dalehamel marked this pull request as ready for review July 22, 2026 19:34
@dalehamel
dalehamel merged commit eb76cb8 into dale/hackdays-io-network-base Jul 22, 2026
39 checks passed
@dalehamel
dalehamel deleted the dale/io-network-probes-async branch July 22, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant