Skip to content

Add swift.use_swiftinterface_for_caching feature - #1880

Open
xiemotongye wants to merge 2 commits into
bazelbuild:mainfrom
xiemotongye:feat/swiftinterface-for-caching
Open

Add swift.use_swiftinterface_for_caching feature#1880
xiemotongye wants to merge 2 commits into
bazelbuild:mainfrom
xiemotongye:feat/swiftinterface-for-caching

Conversation

@xiemotongye

Copy link
Copy Markdown

Introduces an opt-in feature swift.use_swiftinterface_for_caching that makes Swift dependency .swiftinterface files (rather than .swiftmodule files) participate in the Swift compile action's cache key when library_evolution is enabled for those dependencies. Swiftmodule files are still provided to the compiler via Bazel's unused_inputs_list mechanism — they remain in the sandbox but do not contribute to the action key.

Motivation

With library_evolution, a Swift module's public ABI is captured in its .swiftinterface. Internal implementation changes (private functions, changes to non-public symbols) only rewrite the .swiftmodule, not the .swiftinterface. Today the swiftmodule of every transitive dependency is part of every downstream compile action's key, so any internal change in an upstream library invalidates the cache for its entire reverse dependency closure — even though nothing observable to those downstream compilations actually changed.

This is especially painful for CI/remote-cache workflows in large mixed Swift/Objective-C codebases: an upstream refactor forces recompilation of thousands of downstream Swift modules that would have produced bit-identical output.

How it works

  • Opt in via --features=swift.use_swiftinterface_for_caching. Requires --features=swift.enable_library_evolution and --features=swift.emit_swiftinterface on the dependencies (this repo's existing features).
  • Only affects the compile action cache key. The compiler still receives the same set of files as before (swiftmodule + swiftinterface); we just tell Bazel via unused_inputs_list that the swiftmodule file's content does not need to trip a cache miss.
  • If any transitive dependency lacks a .swiftinterface (for example, it is not built with library_evolution), that dependency's swiftmodule falls back to being a normal cache-key input. Correctness is preserved: mixing library-evolution and non-library-evolution dependencies is fine, only the library-evolution ones benefit.
  • private_swiftinterface is preferred over swiftinterface where present, matching transitive_swift_dependency_inputs.

Implementation

  1. ConfigResultInfo gains an unused_inputs: List[File] field. _apply_action_configs accumulates it across configurators, and run_toolchain_action wires it into actions.run(unused_inputs_list=...) by writing a per-action listing file (named after <module_name>_<action>_unused_inputs.txt for uniqueness, with fallback to the output path).

  2. compile() and compile_module_interface() collect two parallel lists from transitive_modulestransitive_swiftinterfaces and transitive_swiftmodules_only — using the same private_swiftinterface-preferred rule as transitive_swift_dependency_inputs. Both are placed on prerequisites alongside the existing transitive_swift_dependency_inputs, together with a use_swiftinterface_for_caching boolean.

  3. _explicit_swift_module_map_info also reads the feature and returns struct(file, inputs, unused_inputs). Both call sites propagate explicit_swift_module_map_unused_inputs into prerequisites.

  4. Three configurators consume the split when the feature is enabled and swiftinterfaces are present:

    • _dependencies_swiftmodules_and_swiftdocs_configurator
    • _dependencies_swiftmodules_configurator
    • _explicit_swift_module_map_configurator

    Each falls back to the original behavior (transitive_swift_dependency_inputs going into inputs) when the feature is off or there are no swiftinterfaces to key off of.

Verification

Manually verified against a real large iOS codebase:

  • With the feature enabled, a compile action produces <target>_SwiftCompile_unused_inputs.txt listing its transitive dependencies' swiftmodule files. Their swiftinterface files remain in the action inputs (cache key), while the swiftmodules are marked as unused.

  • Modifying a private function in an upstream dependency (which only changes the swiftmodule, not the swiftinterface) triggers only that upstream module to recompile; downstream Swift compile actions hit the action cache.

  • Disabling the feature with --features=-swift.use_swiftinterface_for_caching under the same change causes downstream compile actions to re-run, confirming the feature is what gates the cache reuse.

  • Mixed dependency graphs (some deps without library_evolution) build correctly; those deps' swiftmodules stay in the cache-key inputs.

@xiemotongye

Copy link
Copy Markdown
Author

Ready for review. External contributor, so the Test workflow appears to be gated on maintainer approval — could someone kick it off when you get a chance?

Rationale and design context in #1881.

Introduces an opt-in feature `swift.use_swiftinterface_for_caching` that makes
Swift dependency `.swiftinterface` files (rather than `.swiftmodule` files)
participate in the Swift compile action's cache key when
`library_evolution` is enabled for those dependencies. Swiftmodule files are
still provided to the compiler via Bazel's `unused_inputs_list` mechanism —
they remain in the sandbox but do not contribute to the action key.

Motivation
----------

With `library_evolution`, a Swift module's public ABI is captured in its
`.swiftinterface`. Internal implementation changes (private functions, changes
to non-`public` symbols) only rewrite the `.swiftmodule`, not the
`.swiftinterface`. Today the swiftmodule of every transitive dependency is
part of every downstream compile action's key, so any internal change in an
upstream library invalidates the cache for its entire reverse dependency
closure — even though nothing observable to those downstream compilations
actually changed.

This is especially painful for CI/remote-cache workflows in large mixed
Swift/Objective-C codebases: an upstream refactor forces recompilation of
thousands of downstream Swift modules that would have produced bit-identical
output.

How it works
------------

- Opt in via `--features=swift.use_swiftinterface_for_caching`. Requires
  `--features=swift.enable_library_evolution` and
  `--features=swift.emit_swiftinterface` on the dependencies (this repo's
  existing features).
- Only affects the compile action cache key. The compiler still receives the
  same set of files as before (swiftmodule + swiftinterface); we just tell
  Bazel via `unused_inputs_list` that the swiftmodule file's *content* does
  not need to trip a cache miss.
- If any transitive dependency lacks a `.swiftinterface` (for example, it is
  not built with `library_evolution`), that dependency's swiftmodule falls
  back to being a normal cache-key input. Correctness is preserved: mixing
  library-evolution and non-library-evolution dependencies is fine, only the
  library-evolution ones benefit.
- `private_swiftinterface` is preferred over `swiftinterface` where present,
  matching `transitive_swift_dependency_inputs`.

Implementation
--------------

1. `ConfigResultInfo` gains an `unused_inputs: List[File]` field.
   `_apply_action_configs` accumulates it across configurators, and
   `run_toolchain_action` wires it into `actions.run(unused_inputs_list=...)`
   by writing a per-action listing file (named after
   `<module_name>_<action>_unused_inputs.txt` for uniqueness, with fallback to
   the output path).

2. `compile()` and `compile_module_interface()` collect two parallel lists
   from `transitive_modules` — `transitive_swiftinterfaces` and
   `transitive_swiftmodules_only` — using the same
   `private_swiftinterface`-preferred rule as `transitive_swift_dependency_inputs`.
   Both are placed on `prerequisites` alongside the existing
   `transitive_swift_dependency_inputs`, together with a
   `use_swiftinterface_for_caching` boolean.

3. `_explicit_swift_module_map_info` also reads the feature and returns
   `struct(file, inputs, unused_inputs)`. Both call sites propagate
   `explicit_swift_module_map_unused_inputs` into `prerequisites`.

4. Three configurators consume the split when the feature is enabled and
   swiftinterfaces are present:

     - `_dependencies_swiftmodules_and_swiftdocs_configurator`
     - `_dependencies_swiftmodules_configurator`
     - `_explicit_swift_module_map_configurator`

   Each falls back to the original behavior (`transitive_swift_dependency_inputs`
   going into `inputs`) when the feature is off or there are no
   swiftinterfaces to key off of.

Verification
------------

Manually verified against a real large iOS codebase:

- With the feature enabled, a compile action produces
  `<target>_SwiftCompile_unused_inputs.txt` listing its transitive
  dependencies' swiftmodule files. Their swiftinterface files remain in the
  action inputs (cache key), while the swiftmodules are marked as unused.

- Modifying a `private` function in an upstream dependency (which only changes
  the swiftmodule, not the swiftinterface) triggers only that upstream module
  to recompile; downstream Swift compile actions hit the action cache.

- Disabling the feature with `--features=-swift.use_swiftinterface_for_caching`
  under the same change causes downstream compile actions to re-run, confirming
  the feature is what gates the cache reuse.

- Mixed dependency graphs (some deps without library_evolution) build
  correctly; those deps' swiftmodules stay in the cache-key inputs.

Tests
-----

- `//test:swiftinterface_for_caching` — a new analysis-test suite covering
  the feature. Fixture in `test/fixtures/swiftinterface_for_caching/` sets up
  a `library_evolution` upstream, a non-`library_evolution` upstream, and two
  downstream clients (one purely library-evolution, one mixed).
- Two `action_inputs_test`s (feature on) assert that the downstream compile
  action still declares upstream `.swiftinterface` and `.swiftmodule` files
  in its `action.inputs`. `unused_inputs_list` files remain in the sandbox
  as regular inputs; Bazel just excludes them from the action cache key.
  Since Starlark's action introspection does not expose the
  `unused_inputs_list` list itself, these assertions cover "we didn't drop
  anything the compiler still needs".
- A mixed-graph test confirms the fallback: `UpstreamLibNoEvolution` has no
  `.swiftinterface`, and its `.swiftmodule` continues to appear in inputs so
  correctness is preserved.
- A baseline test with `-swift.use_swiftinterface_for_caching` documents
  that the input file set is identical when the feature is disabled — the
  difference is only in cache-key membership, which we cannot assert on
  directly.
- A `build_test` sanity check builds the full fixture end-to-end with the
  feature enabled.
@xiemotongye
xiemotongye force-pushed the feat/swiftinterface-for-caching branch from a86f57a to 7c9bf4a Compare August 5, 2026 04:16
@adincebic

Copy link
Copy Markdown
Contributor

I haven't familiarized myself with this change yet. However, I am wondering why wouldn't we have this by default without feature flag? Any downsides? @xiemotongye

@xiemotongye

Copy link
Copy Markdown
Author

Great question — I actually thought about this quite a bit before settling on opt-in. Short answer: I think making it the default is the right long-term direction, but there are a few reasons to gate it behind a flag first.

No correctness regression for the common case. The feature only alters the compile action's cache key; the compiler receives exactly the same file set (swiftmodule + swiftinterface), same command line, same output. And when a transitive dep lacks a .swiftinterface (e.g. built without library_evolution), its swiftmodule falls back to inputs so mixed graphs stay correct. So "correctness" in the coarse sense — will builds compile and produce identical binaries — is fine either way.

The reasons I still went with opt-in:

  1. @inlinable / @usableFromInline / @frozen / C++-interop edge cases. A .swiftinterface captures the declarative public ABI, but the body of an @inlinable function isn't part of that ABI even though it does affect downstream codegen. If an upstream module changes an @inlinable body, the swiftinterface stays byte-identical, downstream compile actions hit cache — but strictly speaking they should re-emit. In our monorepo we accept this trade (the frequency is very low and @inlinable is rare outside a few frameworks), but I'd rather users opt into that trade knowingly than discover it silently.

  2. Silent behavior change for existing library_evolution users. Today folks who enable library_evolution do so for the ABI contract, not expecting any change to their cache invalidation model. Flipping the default would mean their cache reuse jumps overnight, which is usually great but occasionally surprising — "why didn't my downstream module pick up my upstream change?" investigations are painful to debug because unused_inputs_list isn't visible in most tools.

  3. Interaction surface. _explicit_swift_module_map_configurator, swift.emit_private_swiftinterface, per-target library_evolution mixed with global feature flags — there are enough combinations that I'd like to see it get miles in a few external users before making it the default.

  4. rules_swift precedent. swift.cacheable_swiftmodules went through the same pattern: opt-in feature first, defaulted-on later after the caching behavior was well-understood. Following the same rollout feels safer here.

Concretely, my proposal:

  • Land it as opt-in first (this PR).
  • After a release or two, if adoption/feedback is positive, flip the default for targets already built with library_evolution (SWIFT_FEATURE_ENABLE_LIBRARY_EVOLUTION) — those users have already opted into the ABI-stability contract, so the semantic assumption is consistent.
  • Users who want the pre-change behavior can always disable with --features=-swift.use_swiftinterface_for_caching.

Happy to write up the follow-up "flip default" change once this lands, if that's a path you'd support.

@adincebic

Copy link
Copy Markdown
Contributor

Great question — I actually thought about this quite a bit before settling on opt-in. Short answer: I think making it the default is the right long-term direction, but there are a few reasons to gate it behind a flag first.

No correctness regression for the common case. The feature only alters the compile action's cache key; the compiler receives exactly the same file set (swiftmodule + swiftinterface), same command line, same output. And when a transitive dep lacks a .swiftinterface (e.g. built without library_evolution), its swiftmodule falls back to inputs so mixed graphs stay correct. So "correctness" in the coarse sense — will builds compile and produce identical binaries — is fine either way.

The reasons I still went with opt-in:

  1. @inlinable / @usableFromInline / @frozen / C++-interop edge cases. A .swiftinterface captures the declarative public ABI, but the body of an @inlinable function isn't part of that ABI even though it does affect downstream codegen. If an upstream module changes an @inlinable body, the swiftinterface stays byte-identical, downstream compile actions hit cache — but strictly speaking they should re-emit. In our monorepo we accept this trade (the frequency is very low and @inlinable is rare outside a few frameworks), but I'd rather users opt into that trade knowingly than discover it silently.

  2. Silent behavior change for existing library_evolution users. Today folks who enable library_evolution do so for the ABI contract, not expecting any change to their cache invalidation model. Flipping the default would mean their cache reuse jumps overnight, which is usually great but occasionally surprising — "why didn't my downstream module pick up my upstream change?" investigations are painful to debug because unused_inputs_list isn't visible in most tools.

  3. Interaction surface. _explicit_swift_module_map_configurator, swift.emit_private_swiftinterface, per-target library_evolution mixed with global feature flags — there are enough combinations that I'd like to see it get miles in a few external users before making it the default.

  4. rules_swift precedent. swift.cacheable_swiftmodules went through the same pattern: opt-in feature first, defaulted-on later after the caching behavior was well-understood. Following the same rollout feels safer here.

Concretely, my proposal:

  • Land it as opt-in first (this PR).

  • After a release or two, if adoption/feedback is positive, flip the default for targets already built with library_evolution (SWIFT_FEATURE_ENABLE_LIBRARY_EVOLUTION) — those users have already opted into the ABI-stability contract, so the semantic assumption is consistent.

  • Users who want the pre-change behavior can always disable with --features=-swift.use_swiftinterface_for_caching.

Happy to write up the follow-up "flip default" change once this lands, if that's a path you'd support.

Thank you for well written response. Your proposal makes sense to me. I am pinging @keith to give his oppinion too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants