Repository navigation
Conversation
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…metry#1756) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Replay Shopify PR #48 on upstream main after Ruby JIT support merged. Preserve the full LuaJIT interpreter and eBPF implementation while adapting it to current interpreter configuration, process mapping, stack-delta, tracer, metrics, and coredump APIs. Original-Commit: 81b1e8f Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Regenerate Go ABI types and the eBPF error header after adding LuaJIT process data, constants, metrics, and error catalog entries. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Rebuild both embedded tracer objects with clang 17.0.6 in the pinned profiler development image. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
The Loader only matched libluajit-5.1.so*/luajit*/nginx/openresty. tarantool (shopkv) statically links LuaJIT into the main 'tarantool' executable, so no separate libluajit mapping exists and the unwinder never engaged. Add 'tarantool' to the allowlist so the LuaJIT offset extraction runs against the tarantool binary. NOTE: offset extraction was derived from openresty/luajit2; tarantool ships its own LuaJIT fork, so extraction may still need tarantool-specific offsets — this makes the unwinder attempt tarantool and surfaces concrete extraction signal. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…antool) Generalize the executable-detection allowlist: instead of hardcoding which binaries statically link LuaJIT, add a luajit.Config.Executables list so embedders (tarantool, custom static binaries) opt in via the per-interpreter interpreter.Config. embedsLuaJIT() = built-in set (libluajit-5.1.so*/luajit*/ nginx/openresty) + configured Executables. Supersedes the hardcoded 'tarantool' match; this is the patch proposed upstream on open-telemetry#1236. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
… from non-GC64 lua_close) extractor_x86.findOffsetsFromLuaClose derives glref/curL by matching the register that holds global_State across the lua_close disassembly. sameReg only mapped EAX/ECX/EDX/EBX/ESI to their 64-bit forms, omitting EBP/ESP/EDI. Tarantool's x86 (non-GC64) LuaJIT loads G with a 32-bit `movl 0x8(%rdi),%ebp` (greg=EBP) and then stores cur_L via `movl $0x0,0x188(%rbp)` (base RBP). sameReg(RBP,EBP) returned false, so the cur_L store never matched and extraction failed with 'find offsets from lua_close failed' — no Lua unwinding on x86 tarantool (arm64 worked). Complete the 32->64-bit GPR mapping via reg64(). Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…resent
extractInterpreterBounds scans stack deltas for the first large gap matching the
VM unwind pattern. On x86 tarantool that matched an unrelated function (0x164469)
instead of the real lj_vm_asm_begin (0x261ca0), so extractOffsets failed its
start-address sanity check ('unexpected start address') and LuaJIT unwinding was
skipped (only native lj_* frames + ERROR_4012 from unmapped JIT mcode).
When the binary is unstripped (tarantool ships lj_vm_asm_begin), look the symbol
up via ef.LookupSymbol and return the delta gap that starts exactly there;
fall back to the heuristic for stripped binaries (asmBegin==0). Tests pass 0.
Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…m_asm_begin The VM asm FDE starts at lj_vm_asm_begin but the preceding function's unwind info extends to a few bytes below it, so the delta interval starts just under the symbol (exact-address match missed it). Match the large delta interval that contains the symbol and start the range exactly at lj_vm_asm_begin. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
….LookupSymbol ef.LookupSymbol only resolves dynamic symbols; lj_vm_asm_begin is a hidden .symtab symbol, so the lookup returned ErrSymbolNotFound and asmBegin stayed 0, skipping the symbol-anchored interp-range logic entirely. Use scanSymbols (reads .symtab via VisitSymbols), matching how extractOffsets resolves the symbol. Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255 Assisted-By: Claude <noreply@anthropic.com>
Retain focused failure-path diagnostics for invalid or unreadable GCproto objects without emitting the temporary per-frame normal-frame trace logging. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…egression - Hermetic TestExtractInterpreterBoundsAnchor: proves the anchor picks the VM asm over a decoy SP/param gap (x86 bug) and is byte-identical to the heuristic on the arm64 frame-pointer layout; covers start-below-symbol + fallback. - Strengthen the OpenResty integration test to run the anchor on real shared-lib builds (amd64+arm64) and assert it yields identical offsets to the heuristic. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…ARF) Tarantool's LuaJIT inserts a mem_L field in global_State between cur_L and jit_base, so the eBPF assumption that jit_base is adjacent to cur_L read mem_L instead -> garbage frame base for JIT-trace samples -> unresolved <lua> frames. Extract jit_base's offset (g2jitbase) from lj_vm_exit_handler, which clears G->jit_base via the DISPATCH register on trace exit (g2jitbase = g2dispatch + disp). Cheap targeted disassembly, no DWARF. Falls back to cur_L+8 on stripped binaries, correct for OpenResty/luajit2 (no mem_L). arm64 extraction is a TODO (falls back; no regression to the arm64 interpreter path). - support/ebpf: LuaJITProcInfo.g2jitbase; read jit_base from G+g2jitbase. - offsets.go: findG2JitBaseOffset (+ lj_vm_exit_handler symbol); extractor iface. - extractor_x86.go: findG2JitBaseFromExitHandler (movq $0,disp(%r14) -> g2dispatch+disp). - offsets_test.go: assert openresty g2jitbase == cur_L+8 (regression guard, both arches). Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255 Assisted-By: Claude <noreply@anthropic.com>
…t unsigned) x86asm.Decode reports a negative disp32 as its unsigned 32-bit value in the int64 Disp field (0xFFFFF1E0, not -3616), so the g2dispatch+disp candidate overflowed the validation range and the extractor fell back to cur_L+8 (mem_L). Sign-extend the low 32 bits. Verified against the tarantool binary: g2jitbase now extracts 0x1f0 (jit_base) instead of 0x1e8 (mem_L). Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
arm64 tarantool also has the mem_L field, so jit_base is at cur_L+0x10 not +8. On arm64 the VM holds G directly in a register (x22), so the trace-exit clear 'str xzr, [x22, #0x1f0]' encodes jit_base's offset from G directly (g2dispatch unused, unlike x86). Mirrors the setgcrefnull(g->cur_L) matcher in lua_close. Verified against the real arm64 binary: g2jitbase=0x1f0. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…R_4012/4005) After Lua frames resolve, the unwinder hands back to native by stepping over the VM gate's C frame using a hardcoded LUAJIT_CFRAME_SPACE (x86 80, arm64 208). Both are wrong for tarantool: the interp region's own stack delta says x86 CFA=sp+96 and arm64 CFA=fp+16 (frame-pointer based). The mismatch lands the resume PC in an unmapped region -> every tarantool stack carried ERROR_4012 (x86, no_pid_page_mapping) / ERROR_4005 (arm64, stack_delta_invalid), losing the native frames below the VM. Extract the interp CFA offset + base register from the interpreter region's stack delta (info.Deltas covering luaInterp.Start) and use them in unwind_native_frame: SP-based (sp+=param) on x86, FP-based (sp=fp+param) on arm64. Falls back to LUAJIT_CFRAME_SPACE when unavailable. New LuaJITProcInfo.cframe_size_interp/interp_fp. Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255 Assisted-By: Claude <noreply@anthropic.com>
… ERROR_4012) At the bottom of the Lua stack (diff<=2) the code only stepped over the C frame (unwind_native_frame) for is_jit; for the interpreter it set next_unwinder=NATIVE with state->pc still in the luajit-mapped interp region, so the native unwinder re-entered luajit/failed -> ERROR_4012 and the C stack below the VM was dropped. arm64 happened to take the FRAME_CP path (which pokes); amd64 (tarantool, where the bottom Lua frame meets C here) took this path. Now poke for both. Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255 Assisted-By: Claude <noreply@anthropic.com>
…r handback) The JIT native handback resumed 16 bytes short -> into the fiber stack/heap instead of lua_pcall's return -> ERROR_4012, no native C stack. Root cause: cframeSizeJIT = cframeSize(80) + 16, but 80 is OpenResty's VM gate cframe; tarantool's lj_vm_pcall gate cframe is 96 (4 pushes + sub 56 + ret), which we already extract as cframeSizeInterp. Use that base on SP-based (x86) builds: cframeSizeJIT = interp + 16 = 112. Verified against the live fiber stack: lua_pcall's return address sits at new_sp+8. arm64 (FP-based) keeps the arch constant; its JIT path isn't the hot path there. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
…d LuaJIT hosts) Lets coredump test cases probe binaries that statically link LuaJIT (e.g. tarantool) during extraction, and persists the config into the test case so the harness probes the same hosts on replay. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Core's transitional 4.0.5-pshopify3 build reports RUBY_VERSION=4.0.5, but its exported ruby_description identifies source revision 21a2595676. That revision is 31 commits after master_box change 99aac00 and is itself the rb_thread_sched change, so it unambiguously carries both 4.0.6 layout shifts. Treat only Ruby 4.0.5 at that exact description revision as using the new layout. Stock 4.0.0-4.0.5 retain their original offsets, while 4.0.6+ continues using the upstream open-telemetry#1634 version gate. If the description is missing or a pshopify rebuild changes revision, detection fails safe. Assisted-By: devx/38cc1a8a-dd3e-41a2-8b65-28604cba42e1 Assisted-By: Claude <noreply@anthropic.com>
Adapt native-handback cframe discovery to the current block-based IntervalData representation while preserving absolute-address interval matching. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Add the final jit_base and interpreter cframe fields to LuaJITProcInfo after replaying the Tarantool and native-handback fixes. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Rebuild both embedded tracer objects with clang 17.0.6 in the pinned profiler development image after the final native-handback changes. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
|
Closed as inactive. Feel free to reopen if this PR is still being worked on. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-local scope
Important
This integration snapshot intentionally targets this repository's
mainbranch. It is not an upstream pull request and must not be retargeted toopen-telemetry/opentelemetry-ebpf-profiler.The branch provides a reviewable, fork-local integration point for a private downstream consumer while preserving the existing public change history.
Lineage
e52dd663a61f216abaa91ec05eebfb10f6b23b4c625149ae70910346bed828b376ea71df459b617b19aafba388fe12e907f9b4cbb946bc5b204f8f067211a319e088d0944050eb2fe0009eefca86be04f2dfef77f9ab90408efbd53f0f31491e6cff53eaThe branch is 24 commits ahead of fork
main: two intervening upstream dependency/workflow commits followed by the preserved 22-commit LuaJIT/Ruby omnibus series.What is included
mem_L, cframe, and fiber-root handling without changing OpenResty defaults.Source changes, generated ABI/catalog output, and embedded eBPF blobs remain separately reviewable in the commit series.
Rebase equivalence
git range-diffpreserves all 22 omnibus commits. The only substantive replay difference is dependency resolution against the newer upstream Go modules.Every path outside these upstream-owned files is byte-identical to
19aafba3:.github/workflows/codeql.yml.github/workflows/ossf-scorecard.ymlgo.modgo.sumNo eBPF input or generated ABI type changed, so the validated blobs were intentionally preserved:
5d99682a934016028b53260f9066cdfd4a3969f0682f457db0986623d9c6b048f8eeee9fb6a3671f62c847a6defeeca885ab0d1166179479c2cb53d53700b6bbValidation
Deterministic and native validation
Downstream runtime verification
ERROR_8xxxframes.Downstream use
A private downstream consumer pins exact head
7211a319and synchronizes its embedded error catalog. Release and deployment review occur outside this public repository.Review and merge policy
This PR exists for fork-local review and lineage. Do not request upstream review, retarget it, or merge it without an explicit decision about this fork's
mainpolicy.