Skip to content

Support CFA saved on the stack and rbp saved at rbp - #130

Open
SSE4 wants to merge 2 commits into
yandex:mainfrom
SSE4:unwinder-deref-cfa-rules
Open

SSE4 wants to merge 2 commits into
yandex:mainfrom
SSE4:unwinder-deref-cfa-rules

Conversation

@SSE4

@SSE4 SSE4 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Depends on #129

Problem

Stacks stop at frames that keep the caller's stack pointer in memory. Two kinds are common:

GCC dynamic stack realignment (DRAP). Functions with over-aligned locals (e.g. __m256i) that also
pass arguments on the stack or use VLAs get this prologue:

lea    0x8(%rsp),%r10
and    $-32,%rsp
push   -0x8(%r10)
push   %rbp
mov    %rsp,%rbp
push   %r10
DW_CFA_expression: r6 (rbp) (DW_OP_breg6 (rbp): 0)     rbp = *(rbp)
DW_CFA_def_cfa_expression (DW_OP_breg6 (rbp): -8; DW_OP_deref)     CFA = *(rbp - 8)

This is widespread in GCC-built code. Counting DW_CFA_def_cfa_expression (DW_OP_breg6 (rbp): N; DW_OP_deref)
entries: 259 in Debian's libhs (Hyperscan), 345 in libx265, 4,023 across /usr/bin and
/usr/lib/x86_64-linux-gnu on the test host.

Hand-written OpenSSL perlasm (sha*_block_data_order_avx2 and others):

DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 152; DW_OP_deref; DW_OP_plus_uconst: 8)     CFA = *(rsp + 152) + 8

ehframe.cpp already recognized the OpenSSL form as RegisterDerefOffset, but fillRule turned it into
UNWIND_RULE_UNSUPPORTED and the eBPF unwinder had no code for it. The DRAP form was not recognized at
all, and neither was its rbp rule, so even with the CFA fixed the next frame would read a wrong rbp.

Fix

  • ehframe.cpp: match *(reg + offset) without a bias as well as with one, and DW_OP_bregN offset
    as a register location, which describes rbp saved at rbp.
  • unwindtable: emit UNWIND_RULE_REGISTER_DEREF_OFFSET (bias 0) and
    UNWIND_RULE_REGISTER_DEREF_OFFSET_PLUS_8. struct cfa_unwind_rule has no room for a bias, so it is
    encoded in the kind; other biases stay unsupported. Emit DWARF_UNWIND_RBP_RULE_AT_RBP for rbp saved
    at rbp.
  • eBPF: evaluate both deref kinds for rsp and rbp. The CFA comes from memory, so a value that is not
    above the current stack pointer is rejected. Restore rbp from *(rbp) for
    DWARF_UNWIND_RBP_RULE_AT_RBP.
  • UNWIND_RULE_REGISTER_DERREF_OFFSET was unused; renamed to fix the typo.

The unwind table layout is unchanged.

Testing

  • agent/preprocessing/lib/ut/ehframe_ut.cpp: the OpenSSL, DRAP and glibc (*(rsp + 160)) CFA
    expressions, the rbp location, and expressions that must not match.
  • agent/collector/pkg/dso/bpf/unwindtable/manager_test.go: fillRule for both biases, an unsupported
    bias, and the rbp rule.
  • Repro below, perforator record -p <pid> -F 999 -d 15s, before and after the change.

Repro

repro-b.c + repro-b-asm.S, gcc -O2 -g -fno-omit-frame-pointer:

repro-b.c
repro-b-asm.S.txt

  • main -> drap_caller -> drap_frame -> drap_leaf: drap_frame has a 32-byte aligned local and calls
    drap_leaf with 8 arguments, so GCC emits the DRAP prologue above.
  • main -> deref8_caller -> deref8_frame -> deref8_leaf: deref8_frame is assembly with the OpenSSL
    CFI, CFA = *(rsp + 8) + 8.

The callers use frame pointers, so reaching them also checks the restored rbp.

Before After
before after
Before After
DRAP samples reaching main 0% 100%
OpenSSL-style samples reaching main 0% 100%
before after

SSE4 added 2 commits November 13, 2026 13:19
TDwarfExpressionPattern::Matches() kept comparing pattern operations
after the expression ran out. A DWARFExpression end iterator still holds
the last decoded operation, so a pattern that repeats the last operation
matched a shorter expression: [breg7 8, deref, deref] matched
[breg7 8, deref].

None of the current CFA patterns repeat their last operation, so this
does not change the unwind tables built today, but it makes adding
patterns that share a prefix safe. Add unit tests for the matcher.
Two kinds of frames keep the caller's stack pointer in memory and could
not be unwound: the stack stopped at them.

- GCC dynamic stack realignment (DRAP), used for functions with
  over-aligned locals that pass arguments on the stack or use VLAs:
    CFA = *(rbp - 8)
    rbp = *(rbp)
  Common in GCC-built code: 259 functions in Debian's libhs, 345 in
  libx265.
- Hand-written OpenSSL perlasm (sha*_block_data_order_avx2 and others):
    CFA = *(rsp + offset) + 8

The unwind table builder already recognized the OpenSSL form, but the
rule never reached the eBPF unwinder. Now:

- ehframe: match *(reg + offset) without a bias as well, and
  DW_OP_bregN offset as a register location, which describes rbp saved
  at rbp;
- unwindtable: emit UNWIND_RULE_REGISTER_DEREF_OFFSET (bias 0) and
  UNWIND_RULE_REGISTER_DEREF_OFFSET_PLUS_8. The bias is encoded in the
  kind because struct cfa_unwind_rule has no room for it; other biases
  stay unsupported. Emit DWARF_UNWIND_RBP_RULE_AT_RBP for rbp saved at
  rbp;
- eBPF: evaluate both deref kinds, rejecting a CFA that is not above the
  current stack pointer since it comes from memory, and restore rbp from
  *(rbp) for DWARF_UNWIND_RBP_RULE_AT_RBP.

UNWIND_RULE_REGISTER_DERREF_OFFSET was unused and is renamed to fix the
typo.
@robot-magpie

robot-magpie Bot commented Nov 13, 2026

Copy link
Copy Markdown

✅ All contributors are covered under a CLA with Yandex

See CONTRIBUTING.md for more info about Yandex Contributor License Agreement.


The following contributors were found:
✅ e2ef2bb Author: @SSE4 (to*****de@gmail.com)

(Only the first commit for a unique contributor is listed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant