Skip to content

fix: preserve hidden visibility when merging symbols and localize dynsym exports#1700

Merged
davidlattimore merged 2 commits into
wild-linker:mainfrom
PritamP20:fix/visibility-merge-dynsym
Mar 28, 2026
Merged

fix: preserve hidden visibility when merging symbols and localize dynsym exports#1700
davidlattimore merged 2 commits into
wild-linker:mainfrom
PritamP20:fix/visibility-merge-dynsym

Conversation

@PritamP20
Copy link
Copy Markdown
Contributor

When a symbol has mixed visibility definitions across object files, the most restrictive visibility should win (hidden > protected > default). Wild was computing the merged visibility correctly but silently discarding it, so symbols that should be local were still leaking into dynsym.

Fixes #1693

Root cause

Two issues in handle_non_default_visibility:

  1. The merged visibility was never passed in, so hidden and protected were treated identically
  2. Only NON_INTERPOSABLE was set, never DOWNGRADE_TO_LOCAL — the flag that actually prevents export into dynsym

A third issue: data5 is declared hidden in one object but defined (with default visibility) in another. Since one is undefined and the other defined, they never appear in the same alternative-definition group, so the merged visibility path never ran.
This is fixed by tracking hidden undefined references during symbol loading and downgrading their definitions after resolution.

Before (Wild dynsym):           After (Wild dynsym):
  data1  ← wrong, should local    data2
  data2                           data3
  data3                           get_data1
  data4  ← wrong, should local    get_data5
  data5  ← wrong, should local
  get_data1
  get_data5

BFD dynsym (correct):
  data2, data3, get_data1, get_data5

Copy link
Copy Markdown
Member

@davidlattimore davidlattimore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. I've benchmarked the change and unfortunately it incurs an 8% slowdown on linking librustc-driver and a 4% slowdown linking the Zed editor.

Comment thread libwild/src/symbol_db.rs Outdated

symbol_db.buckets = buckets;

for name in &symbol_db.hidden_undefined_names {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need to be done from the main thread? Perhaps it could be done in parallel from each object or from each hash bucket? Better yet, could we avoid storing a collection of hidden undefined names at all and update the flags during the resolution phase? - i.e. when we resolve undefined symbols

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, Didn't think about it, so the hidden visibility merge cab be taken care during alternative resolution phase itself, reducing the main thread iteration

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it could probably be done during resolution itself, which is after alternative symbol selection. Resolution is where undefined symbols are looked up. For performance reasons, we want to do just a single hash lookup of each symbol. For defined symbols this is done when building the symbol DB. For undefined symbols, it's during resolution. I suspect you can handle both definitions and undefined symbols in resolution though, since for defined symbols, we know what symbol ID they resolve to without having to do hashmap lookup.

Comment thread libwild/src/symbol_db.rs Outdated
{
let info = self.get_symbol_name_and_version(symbol, local_index)?;
outputs
.add_hidden_undefined_name(UnversionedSymbolName::prehashed(info.name()));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we storing symbol names? Would SymbolIds work?

Copy link
Copy Markdown
Contributor Author

@PritamP20 PritamP20 Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't store it in symbolid because if the symbol is hidden and undefined then there won't be any entry in symbol_definitions vec and can't find the definition ID so the same behaviour might repeat

@davidlattimore
Copy link
Copy Markdown
Member

BTW, to what extent did you use LLMs/AI when writing the PR description? I don't know your usual writing style, but it has an unusual feel to it that reminds me of how LLMs write

@PritamP20
Copy link
Copy Markdown
Contributor Author

PritamP20 commented Mar 17, 2026

@davidlattimore So, I initially I write the PR description myself explaining the issue and the fix, and then use an llm to help polish the wording and formatting

@davidlattimore
Copy link
Copy Markdown
Member

OK, thanks. Going forward, would you be able to just post what you write? Don't worry if it's not polished - if an LLM can understand what you wrote, then I'm sure we can too

@PritamP20
Copy link
Copy Markdown
Contributor Author

PritamP20 commented Mar 17, 2026

@davidlattimore Sure, noted from next time I will take care of it,

I just had a doubt while working on this the hidden visibility, handling was being skipped in the test files, so it was clear why wild's behaviour was wrong for the hidden merge. But the protected visibility seems to be skipped both in the tests and in resolve_alternative_symbol_definitions, Should that also be handled in this PR?

Screenshot 2026-03-17 at 8 02 32 AM

@davidlattimore
Copy link
Copy Markdown
Member

It's good to keep PRs small and focused on one issue where possible. So I'd say fixing hidden symbols in this PR and protected in a separate PR would be best. But certainly good to think about how protected visibility might be handled and take that into consideration when deciding how to solve hidden. The exception, where I'd say it might be reasonable to fix both in the same PR would be if that actually made the PR simpler, but otherwise, probably keep them separate.

@PritamP20 PritamP20 force-pushed the fix/visibility-merge-dynsym branch 4 times, most recently from a29c11c to 2f0f8e5 Compare March 18, 2026 15:10
@PritamP20
Copy link
Copy Markdown
Contributor Author

@davidlattimore any review on this?

Comment thread libwild/src/resolution.rs Outdated
if !local_symbol_attributes.default_visibility {
let visibility = resources.symbol_db.input_symbol_visibility(local_symbol_id);
if visibility != Visibility::Default {
resources
Copy link
Copy Markdown
Member

@davidlattimore davidlattimore Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in order to get acceptable performance, we need to not store into any SegQueues or Vecs. Would it work to get access to the AtomicPerSymbolFlags here and directly update it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we can do it I thought doing it sequentially would affect the efficiency but doing it in parallel can save a lot of time. Thanks for telling me about the atomic symbols, they do exactly what we're trying to do

@PritamP20 PritamP20 force-pushed the fix/visibility-merge-dynsym branch 3 times, most recently from 096aacd to caa5ae7 Compare March 20, 2026 12:26
@davidlattimore
Copy link
Copy Markdown
Member

#1754 moved all test sources into separate subdirectories. The test files were mostly unchanged, so hopefully rebasing should only require moving your changes to the file in the new location.

@PritamP20 PritamP20 force-pushed the fix/visibility-merge-dynsym branch from caa5ae7 to 9b77a43 Compare March 26, 2026 19:08
@PritamP20
Copy link
Copy Markdown
Contributor Author

@davidlattimore I rebased the branch, git handled the rebase itself, so everything is in the right place

@PritamP20 PritamP20 force-pushed the fix/visibility-merge-dynsym branch from 9b77a43 to 21b6246 Compare March 26, 2026 19:14
Copy link
Copy Markdown
Member

@davidlattimore davidlattimore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I can confirm that the performance loss is gone, so this is very close to being able to be merged.

Comment thread libwild/src/resolution.rs
if visibility != Visibility::Default
&& let Some(def_id) = symbol_db.get_unversioned(&pre_hashed)
{
symbol_db::apply_visibility_to_definition(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I comment out both calls to apply_visibility_to_definition, all the tests still pass. Do you think you can come up with an integration test that is fixed by this part of the change?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understood, so you want me to create a integration test, which tests part of the code is working or not, I will do that

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. It looks like the new test successfully fails if this call is commented out, but commenting out the other call below still results in the tests passing. Do you think the test can be extended to cover that as well?

BTW, the failing test:

/usr/bin/ld.bfd: /__w/wild/wild/wild/tests/build/elf/x86_64/hidden-weak-archive-visibility/default/hidden-weak-archive-visibility.c.o: relocation R_X86_64_32 against hidden symbol `get_value' can not be used when making a shared object
/usr/bin/ld.bfd: failed to set dynamic section sizes: bad value

I think might be fixed by adding CompArgs:-fPIC, although I haven't tried.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @davidlattimore , i did try extending the test for the second call but couldn’t make it fail when the second call is commented out i added that call just to be safe thinking the same symbol might show up twice in undefined_symbols but the first call already covers it, i will remove the 2nd call

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about having two objects with references to an undefined symbol, both weak references but only one hidden, then a configuration where the object containing the hidden symbol is first and a configuration where it's second?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

damn, I didn't think in that way, Now i have created one more ref of get_value which is weak and protected which goes through the 2nd function call

@PritamP20 PritamP20 force-pushed the fix/visibility-merge-dynsym branch 2 times, most recently from 06fc429 to 59a2ac9 Compare March 28, 2026 05:34
//#Object:hidden-weak-archive-visibility-2.c
//#NoDynSym:get_value
//#ExpectDynSym:public_func
//#Config:vacant:default
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you put a blank line before each of the configs? That's how all our other tests are formatted since it makes it easier to see where they start

@PritamP20 PritamP20 force-pushed the fix/visibility-merge-dynsym branch from 59a2ac9 to 8312c01 Compare March 28, 2026 08:56
Copy link
Copy Markdown
Member

@davidlattimore davidlattimore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this :)

@davidlattimore davidlattimore merged commit 23e45ec into wild-linker:main Mar 28, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wild exports symbols to dynsym that should remain local after visibility merging

2 participants