Skip to content

Merge parca LuaJIT interpreter (#1236) onto Ruby JIT base (#46) - #48

Closed
dalehamel wants to merge 3 commits into
dale/ruby-jit-wide-anon-exec-fallbackfrom
dale/luajit-unwinder-rebase
Closed

dalehamel wants to merge 3 commits into
dale/ruby-jit-wide-anon-exec-fallbackfrom
dale/luajit-unwinder-rebase

Conversation

@dalehamel

@dalehamel dalehamel commented Jun 29, 2026 •

Copy link
Copy Markdown
Member

parca LuaJIT interpreter (open-telemetry#1236) merged onto the Ruby-JIT base (#46)

This PR is the clean base for LuaJIT unwinding: the upstream parca LuaJIT interpreter (open-telemetry#1236) merged onto our Ruby-JIT base (#46), with the LuaJIT plugin ported to the current interpreterconfig.Config API (the API moved after open-telemetry#1236 was written) and the BPF blobs rebuilt.

It intentionally contains no Shopify/tarantool-specific behavior — it's just "#46 + upstream open-telemetry#1236, building against current main's interpreter-config API".

Stacked on top

Keeping this PR as the minimal merge makes it easy to track upstream open-telemetry#1236 and to review the Shopify additions separately in #49.

@dalehamel
dalehamel changed the base branch from main to dale/ruby-jit-wide-anon-exec-fallback June 29, 2026 22:07
@dalehamel

Copy link
Copy Markdown
Member Author

Validated end-to-end on Tarantool (shopkv), arm64

Ran the prophiler built from this branch on a shopkv staging Tarantool storage node (--tracers=go,ruby,luajit):

  • BPF unwind_luajit program passed the kernel verifier and attached cleanly alongside unwind_ruby.
  • First finding: zero Lua frames despite a healthy pipeline (103K native/kernel/daemon samples landed). Root cause: Tarantool statically links LuaJIT into its tarantool binary, and the Loader allowlist only matched libluajit-5.1.so*/luajit*/nginx/openresty.
  • Fix (092939b): added tarantool to the Loader allowlist. With LUA_DEBUG=1:
    lj: interp range {2350992 2366704}
    lj: offsets &{g2Dispatch:4080 g2Traces:1200 currentLOffset:480}
    lj: add JIT region pid(...) ...
    New LuaJIT instance detected
    lj: process_client+116      @.../tarantool/http/server.lua:818
    lj: collect_and_serialize+34 @.../metrics/plugins/prometheus.lua:57
    lj: storage_health_state+25 @.../vshard/storage/init.lua:3647
    
    Interpreter-bounds + offset extraction (openresty/luajit2-derived) worked unmodified on Tarantool's LuaJIT fork — 1 instance detected, JIT region added, real symbolized Lua stacks, zero extraction errors.

Filed an upstream note suggesting the executable allowlist be driven by the per-interpreter interpreter.Config (open-telemetry#1436) so embedders like Tarantool can opt in via config: open-telemetry#1236 (comment).

@dalehamel
dalehamel force-pushed the dale/luajit-unwinder-rebase branch from b2d9889 to 3c4f8bf Compare June 30, 2026 03:10
@dalehamel
dalehamel force-pushed the dale/ruby-jit-wide-anon-exec-fallback branch from 47fb818 to 0810d58 Compare June 30, 2026 15:44
@dalehamel
dalehamel force-pushed the dale/luajit-unwinder-rebase branch from 6eacc37 to 2550eaa Compare June 30, 2026 16:31
@dalehamel dalehamel changed the title Revive LuaJIT unwinder: merge upstream #1236 onto Ruby JIT base (#46) Merge parca LuaJIT interpreter (#1236) onto Ruby JIT base (#46) Jun 30, 2026
@dalehamel
dalehamel force-pushed the dale/luajit-unwinder-rebase branch from 2550eaa to 32dd7c6 Compare June 30, 2026 17:02
dalehamel added a commit that referenced this pull request Jun 30, 2026
…se (#46)

Rebuilt on the current #46 head so the ancestry is clean (#46 -> #48).
Resolved metric-ID space (native errors 295-297, luajit 298-299),
PerCPURecord (usesAnonymousMappings + initialUnwinder), tracemgmt init/collect,
regenerated support/types.go, rebuilt BPF blobs, go mod tidy.
dalehamel added a commit that referenced this pull request Jun 30, 2026
…se (#46)

Rebuilt on the current #46 head so the ancestry is clean (#46 -> #48).
Resolved metric-ID space (native errors 295-297, luajit 298-299),
PerCPURecord (usesAnonymousMappings + initialUnwinder), tracemgmt init/collect,
regenerated support/types.go, rebuilt BPF blobs, go mod tidy.
@dalehamel
dalehamel force-pushed the dale/luajit-unwinder-rebase branch from 32dd7c6 to f17e649 Compare June 30, 2026 17:32
dalehamel added a commit that referenced this pull request Jun 30, 2026
…se (#46)

Rebuilt on the current #46 head so the ancestry is clean (#46 -> #48).

Adapts parca open-telemetry#1236 to interface changes that landed in main after it was
written:
- interpreterconfig.Config port (open-telemetry#1436): luajit.Config + GetLoader + wiring
- EbpfHandler.CoredumpTest: stub on the processinfo test mock
- Instance.UsesAnonymousMappings (open-telemetry#1473): LuaJIT opts in so the now-gated
  per-PID anonymous-mapping sync still covers its JIT mcode; otherwise the
  native unwinder reports ERR_NATIVE_UNSUPPORTED_MAPPING (4018) on every
  JIT-trace sample instead of handing off to the LuaJIT unwinder.

Conflict resolution: metric-ID space (native 295-297, luajit 298-299),
PerCPURecord (usesAnonymousMappings + initialUnwinder), tracemgmt init/collect,
regenerated support/types.go, rebuilt BPF blobs, go mod tidy.
@dalehamel
dalehamel force-pushed the dale/luajit-unwinder-rebase branch from f17e649 to 81b1e8f Compare June 30, 2026 17:41
@dalehamel
dalehamel force-pushed the dale/ruby-jit-wide-anon-exec-fallback branch 2 times, most recently from 4ce17ff to 0e082e4 Compare July 19, 2026 22:35
@dalehamel
dalehamel force-pushed the dale/ruby-jit-wide-anon-exec-fallback branch 2 times, most recently from 431201f to dbe441e Compare July 29, 2026 18:32
@dalehamel
dalehamel force-pushed the dale/ruby-jit-wide-anon-exec-fallback branch 2 times, most recently from 40c28c9 to c1a78fd Compare August 11, 2026 13:31
Replay Shopify PR #48 on upstream main after Ruby JIT support merged. Preserve the full LuaJIT interpreter and eBPF implementation while adapting it to current interpreter configuration, process mapping, stack-delta, tracer, metrics, and coredump APIs.

Original-Commit: 81b1e8f

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Regenerate Go ABI types and the eBPF error header after adding LuaJIT process data, constants, metrics, and error catalog entries.

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
Rebuild both embedded tracer objects with clang 17.0.6 in the pinned profiler development image.

Assisted-By: Claude <noreply@anthropic.com>
Assisted-By: devx/18d54ca0-685e-40de-8b30-f7ba6a0b0255
@dalehamel
dalehamel force-pushed the dale/ruby-jit-wide-anon-exec-fallback branch from c1a78fd to e52dd66 Compare August 13, 2026 22:41
@dalehamel
dalehamel force-pushed the dale/luajit-unwinder-rebase branch from 81b1e8f to 1ce2e70 Compare August 13, 2026 22:41
@github-actions

Copy link
Copy Markdown

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions Bot added the Stale label Sep 11, 2026
@github-actions

Copy link
Copy Markdown

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions Bot closed this Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant