fix(pace-cc): apply -fmacro-prefix-map on clang workers compiling preprocessed input

.agent/rules/compile-cache.md

@@ -362,7 +362,7 @@ same on both — the same defect with no MSVC anywhere near it.
   older launcher keeps resolving to an older object; direct mode is on by default and
   short-circuits before the preprocessed path,
   so without the second bump the re-key never happens where it matters most. The coupling is
-  ONE-WAY, though, and the tags read `objkey-v8` and `manifest-v9`: an `objkey` bump must drag the manifest
+  ONE-WAY, though, and the tags read `objkey-v9` and `manifest-v10`: an `objkey` bump must drag the manifest
   tag with it, while the reverse is free — and free is the direction `v5` itself moved in
   (issue #111), the manifest half forced and the object half taken only to keep the two from
   sitting numerically apart.
@@ -1731,10 +1731,21 @@ stops being one — the same confound that cost #493 a re-run.
   - **On a dispatched line the two spellings part** (`PathValueFlag::remote`, a COLUMN):
     a worker compiles text whose `#line` markers name the client's paths, so without the
     client's rule it would embed `source_location` names unmapped, under the key a mapped
-    one uses. `-fmacro-prefix-map` is FORWARDED verbatim and the worker's allowlist admits
+    one uses. `-fmacro-prefix-map` is FORWARDED and the worker's allowlist admits
     it (`ArgValue::PrefixMapRule`: its halves are paths and it opens neither);
     `-ffile-prefix-map` is REFUSED by name, because it also rewrites debug records the
     worker maps by its own `-fdebug-prefix-map` rules.
+  - **Forwarded is not RECEIVED: clang's driver drops the plain spelling on preprocessed
+    input**, warning `argument unused during compilation` and handing `-cc1` nothing, so
+    every clang worker embedded `__builtin_FILE()` UNMAPPED under the mapped key until
+    `objkey-v9` retired those objects. gcc applies it there. So the spelling a driver needs
+    is a `DriverSpec` column, `preprocessedSourceMapCarrier` (`-Xclang=` for clang, FUSED so
+    the worker still judges one argument at a time), and `CarriedSourcePrefixMap` is the ONE
+    carve-out from the `-Xclang` denial: asked inside `FindDeniedArgument` so both ends
+    agree, admitted by the worker only for the driver whose carrier it is, and exact —
+    `-Xclang=` with anything else stays denied by row. A clang too old for `-Xclang=` fails
+    the remote compile, which is retried locally. `dist-compile-e2e` case 7c compares the
+    dispatched object with a local one; a gcc leg is its control.
   - **The module maps `__FILE__` for the checkout (`<src>/=`) and the dependency cache
     (`<cache>/=deps/`, LAST)** — `_pace_source_prefix_map_rules`, checked by
     `ctest -R debug-prefix-map-rules` — and only when `pace-cc` is the launcher, the

scripts/dist-compile-e2e.sh

@@ -68,6 +68,11 @@
 #                            so the object names the client's checkout and no key sees
 #                            it; the launcher scans a dispatched object as it scans a
 #                            local one. The Windows fixture's case 3b is its twin.
+#  7c. Macro map            — the client's `-fmacro-prefix-map` reaches the worker's
+#                            compiler, so a dispatched `__builtin_FILE()` is the local
+#                            compile's, byte for byte. clang's driver discards the plain
+#                            spelling on preprocessed input; the worker is handed it as
+#                            `-Xclang=`. A gcc leg is the control.
 #   8. Graceful stop        — a worker asked to stop does, promptly, rather than
 #                            waiting for a supervisor to escalate.
 #  12. Cache independence   — a cache the launcher cannot reach does not stop the
@@ -112,7 +117,7 @@ cli=""
 # FAILS without it: the ctest registration passes it whenever the stub is built.
 edit_stub=""
 
-# Which body of assertions to run: `suite` is all sixteen cases (1 to 14, with 2b and 7b;
+# Which body of assertions to run: `suite` is all seventeen cases (1 to 14, with 2b, 7b and 7c;
 # 14 runs only where a DWARF reader is found), and `self-test`
 # drives the pure verdict decisions below with no binary built.
 mode="suite"
@@ -1404,6 +1409,37 @@ if [[ -n "$bound_why" ]]; then
 fi
 echo "   a dispatched object naming its checkout was kept with it, and served back to it"
 
+# --- 7c: the client's macro map reaches the worker's compiler -------------------
+echo "== case 7c: a dispatched compile applies the client's -fmacro-prefix-map"
+# The worker compiles preprocessed text, and clang's DRIVER drops `-fmacro-prefix-map` on
+# such input: it warns `argument unused during compilation` and never hands the rule to
+# `-cc1`. `__FILE__` survived that, being expanded on the client; `__builtin_FILE()` and
+# every `std::source_location` did not, so the worker wrote the client's ABSOLUTE path
+# into an object stored under the mapped key. gcc honours the plain spelling and is this
+# case's control on a gcc leg; on a clang leg it is the regression.
+cat > "${proj}/sevenc.cpp" <<'EOF'
+// unique: casesevenc
+char const* casesevenc_where() { return __builtin_FILE(); }
+EOF
+"$compiler" "-fmacro-prefix-map=${proj}/=" -O1 -c "${proj}/sevenc.cpp" -o "${proj}/build/sevenc-ref.o" \
+    || fail "the case 7c reference compile failed"
+run_launcher "${workdir}/case7c.log" "-fmacro-prefix-map=${proj}/=" -O1 -c "${proj}/sevenc.cpp" -o "${proj}/build/sevenc.o" \
+    || { cat "${workdir}/case7c.log" >&2; fail "case 7c: the mapped compile failed"; }
+sevenc_why=""
+# The control: the local compile DOES embed the mapped name and no checkout path, so a
+# match below is about the mapping rather than about a source that names nothing.
+LC_ALL=C grep -qaF "sevenc.cpp" "${proj}/build/sevenc-ref.o" || sevenc_why+="; the reference object names no source (the case exercises nothing)"
+if LC_ALL=C grep -qaF "$proj" "${proj}/build/sevenc-ref.o"; then sevenc_why+="; the reference object names the checkout (this driver ignores the map)"; fi
+grep -q "DISPATCHED to " "${workdir}/case7c.log" || sevenc_why+="; the compile was not dispatched"
+if grep -qF "argument unused during compilation" "${workdir}/case7c.log"; then sevenc_why+="; the worker's compiler discarded the map"; fi
+if LC_ALL=C grep -qaF "$proj" "${proj}/build/sevenc.o" 2>/dev/null; then sevenc_why+="; the dispatched object names the checkout"; fi
+cmp -s "${proj}/build/sevenc-ref.o" "${proj}/build/sevenc.o" || sevenc_why+="; the dispatched object differs from the local one"
+if [[ -n "$sevenc_why" ]]; then
+    cat "${workdir}/case7c.log" >&2
+    fail "case 7c: the client's macro prefix map did not reach the worker${sevenc_why}"
+fi
+echo "   a dispatched object maps __builtin_FILE() exactly as the local compile does"
+
 # --- 8: a worker stops when it is asked to --------------------------------------
 echo "== case 8: a worker exits on SIGTERM"
 # The property a supervisor depends on, and one that fails in the worst possible

src/apps/pace-cc/ArgumentDenials.hpp

@@ -199,6 +199,13 @@ static_assert(std::ranges::none_of(DeniedArguments, [](DeniedArgument const& row
     if (auto const passed = MatchPassThrough(arg, family); passed.has_value())
         return FindDeniedArgument(passed->inner, DriverOf(passed->into).family);
 
+    // The one `-Xclang=` a client writes itself -- the macro map clang's driver would drop on
+    // preprocessed input -- and only that one (`CarriedSourcePrefixMap`). Asked HERE, inside the
+    // refusal both ends read, so the launcher's pre-lease check and the worker cannot disagree;
+    // which DRIVER may be handed it is the worker's allowlist's question.
+    if (CarriedSourcePrefixMap(arg, family).has_value())
+        return nullptr;
+
     auto const introducers = IntroducersOf(family);
     if (arg.empty() || introducers.empty() || !introducers.contains(arg.front()))
         return nullptr;

src/apps/pace-cc/CacheKey.cpp

@@ -343,7 +343,15 @@ std::string ComputeKey(KeyInputs const& inputs)
     // `dependency-root`. Bumped so the two generations never exchange an entry rather than
     // relying on their keys differing, which they do only for units that read a
     // dependency. `manifest-v9` moves with it, in the lock-step ComputeManifestKey records.
-    KeyDigest digest { "objkey-v8" };
+    //
+    // v9 retires every object a clang WORKER compiled under a `-fmacro-prefix-map`. Its driver
+    // discards that flag on preprocessed input, so `__builtin_FILE()` and every
+    // `std::source_location` in such an object name the client's ABSOLUTE path, filed under the
+    // key a mapped compile uses (`DriverSpec::preprocessedSourceMapCarrier`). The bytes are wrong
+    // rather than merely old and nothing in a value says which ones are -- v6's argument. This
+    // construction is unchanged and the golden vector moved for the tag alone; `manifest-v10`
+    // moves with it, in the lock-step ComputeManifestKey records.
+    KeyDigest digest { "objkey-v9" };
     digest.Field(inputs.compilerId);
     digest.Field(inputs.preprocessed);
     for (auto const& arg: inputs.relativizedArgs)

src/apps/pace-cc/CacheKey_test.cpp

@@ -433,7 +433,7 @@ TEST_CASE("ComputeKey's value is pinned, so changing the construction is deliber
     // to disagree about what was hashed, and the retirement check would go vacuous
     // in the one edit it exists to catch.
     auto const key = ComputeKey(inputs);
-    CHECK(key == "e86dc84efc6cb5de598620cc334f4ee7");
+    CHECK(key == "ccb841c6012c541ddf44ebaefe48132b");
 
     // What the vector alone cannot say: that the tag has not been put BACK. See
     // Generation -- reverting the tag and re-pasting the vector is one edit
@@ -447,6 +447,8 @@ TEST_CASE("ComputeKey's value is pinned, so changing the construction is deliber
         { .key = "objkey-v6", .digest = "e7d074ee4bda35d76e6bf65eba71e7b1" },
         // The v7 -> v8 bump (the dependency root) changed nothing but the tag either.
         { .key = "objkey-v7", .digest = "7323057f60d9ffaca969b16ef293ceaa" },
+        // The v8 -> v9 bump (a clang worker's dropped macro map) changed nothing but the tag.
+        { .key = "objkey-v8", .digest = "e86dc84efc6cb5de598620cc334f4ee7" },
     });
     RequireNoRetiredDigest<std::string_view>(key, Retired);
 }

src/apps/pace-cc/CmdLine.cpp

@@ -315,7 +315,8 @@ namespace
           // `--version` even here. An unrecognised driver is not cached, so this
           // value reaches nothing but a diagnostic -- and asking the one option
           // almost every tool answers beats asking nothing.
-          .versionFlags = VersionProbe },
+          .versionFlags = VersionProbe,
+          .preprocessedSourceMapCarrier = {} },
         { .flavor = Flavor::Cl,
           .family = DriverFamily::Msvc,
           .preprocessFlags = MsvcPreprocess,
@@ -345,7 +346,8 @@ namespace
           // invocation, so it needs no option to be asked -- and it has none, so
           // asking `--version` made it exit 2 and identify as the string `cl` on
           // every MSVC toolset alike. See `DriverSpec::versionFlags`.
-          .versionFlags = {} },
+          .versionFlags = {},
+          .preprocessedSourceMapCarrier = {} },
         { .flavor = Flavor::ClangCl,
           .family = DriverFamily::Msvc,
           .preprocessFlags = MsvcPreprocess,
@@ -370,7 +372,8 @@ namespace
           .fingerprintTree = FingerprintTree::DiscoveredRoots,
           .includeProbeFlags = {},
           .targetProbeFlags = MsvcTargetProbe,
-          .versionFlags = VersionProbe },
+          .versionFlags = VersionProbe,
+          .preprocessedSourceMapCarrier = {} },
         { .flavor = Flavor::Gcc,
           .family = DriverFamily::Gnu,
           .preprocessFlags = GnuPreprocess,
@@ -395,7 +398,8 @@ namespace
           .fingerprintTree = FingerprintTree::OwnTreeWhenTargetingMsvc,
           .includeProbeFlags = GnuIncludeProbe,
           .targetProbeFlags = GnuTargetProbe,
-          .versionFlags = VersionProbe },
+          .versionFlags = VersionProbe,
+          .preprocessedSourceMapCarrier = {} },
         { .flavor = Flavor::Clang,
           .family = DriverFamily::Gnu,
           .preprocessFlags = GnuPreprocess,
@@ -414,7 +418,12 @@ namespace
           .fingerprintTree = FingerprintTree::OwnTreeWhenTargetingMsvc,
           .includeProbeFlags = GnuIncludeProbe,
           .targetProbeFlags = GnuTargetProbe,
-          .versionFlags = VersionProbe },
+          .versionFlags = VersionProbe,
+          // The driver discards the plain spelling on preprocessed input, so the worker's
+          // frontend never sees it. Measured on clang 20.1 and 22.1. A clang too old to
+          // know the fused `-Xclang=` fails the remote compile, which is retried locally:
+          // a lost dispatch, never a wrong object.
+          .preprocessedSourceMapCarrier = "-Xclang=" },
     } };
 
     // --- path-valued flags --------------------------------------------------
@@ -1300,6 +1309,34 @@ std::optional<PassThroughMatch> MatchPassThrough(std::string_view arg, DriverFam
     return std::nullopt;
 }
 
+std::optional<std::string_view> CarriedSourcePrefixMap(std::string_view arg, DriverSpec const& driver)
+{
+    auto const carrier = driver.preprocessedSourceMapCarrier;
+    if (carrier.empty() || !arg.starts_with(carrier))
+        return std::nullopt;
+
+    // Exactly what `RemoteCompileArgs` writes and nothing else `-Xclang` could hand `-cc1`:
+    // a FORWARDED row, its value fused, and a rule rather than a bare path. A bare spelling
+    // would take its value from the next argument, which nothing carries, and a value with
+    // no tail maps nothing -- the worker's allowlist refuses both in their plain spelling.
+    auto const inner = arg.substr(carrier.size());
+    auto const match = MatchPathValueFlag(inner, IntroducersOf(driver.family), driver.family);
+    if (!match.has_value() || match->flag.remote != RemoteLine::Forwarded || match->value.empty()
+        || match->valueTail.empty())
+        return std::nullopt;
+    return inner;
+}
+
+std::optional<std::string_view> CarriedSourcePrefixMap(std::string_view arg, DriverFamily family)
+{
+    // A loop rather than a named `find_if` iterator, for the reason `FindDeniedArgument` gives.
+    for (DriverSpec const& driver: Drivers)
+        if (Overlaps(driver.family, family))
+            if (auto const carried = CarriedSourcePrefixMap(arg, driver); carried.has_value())
+                return carried;
+    return std::nullopt;
+}
+
 bool ProducesSideArtefact(std::string_view arg, DriverSpec const& driver)
 {
     // Judged as what a pass-through carries, by the driver it is handed to (#1680): clang-cl
@@ -1735,8 +1772,18 @@ std::expected<std::vector<std::string>, std::string> RemoteCompileArgs(ParsedCom
             // rewrites debug records the worker maps by its own rules.
             if (match->flag.remote == RemoteLine::Forwarded)
             {
+                // Prefixed with the driver's carrier, where it has one: clang's driver drops
+                // the plain spelling on the preprocessed input a worker compiles, and with it
+                // the `source_location` names (`DriverSpec::preprocessedSourceMapCarrier`).
+                // The carrier is fused to ONE argument, so a rule in the next one cannot ride it.
+                auto const carrier = driver.preprocessedSourceMapCarrier;
+                if (!carrier.empty() && match->value.empty())
+                    return std::unexpected(
+                        std::format("{} names its rule in a separate argument, which this driver cannot be handed "
+                                    "with preprocessed input",
+                                    a));
                 // A separated value travels with its flag, as the build wrote both.
-                out.emplace_back(a);
+                out.emplace_back(std::format("{}{}", carrier, a));
                 if (match->value.empty() && i + 1 < argv.size())
                 {
                     out.emplace_back(argv[i + 1]);

src/apps/pace-cc/CmdLine.hpp

@@ -315,8 +315,10 @@ enum class RemoteLine : std::uint8_t
     /// Left off: the value names something on THIS machine the worker has no use for --
     /// headers already inlined, an object and a depfile the client writes itself.
     Dropped,
-    /// Sent verbatim: a rewrite the worker's compiler must apply to the client's paths in
-    /// the text it is handed, opening none of them (`-fmacro-prefix-map`). Asked AHEAD of the
+    /// Sent: a rewrite the worker's compiler must apply to the client's paths in the text it
+    /// is handed, opening none of them (`-fmacro-prefix-map`) -- behind the driver's
+    /// `preprocessedSourceMapCarrier`, since clang drops the plain spelling on that text and
+    /// gcc does not. Asked AHEAD of the
     /// role's `UndispatchableBecause`, which is what lets one row of a refused role travel:
     /// `-ffile-prefix-map`, which also rewrites debug records a worker maps by its own rules,
     /// stays refused by the role.
@@ -702,6 +704,20 @@ struct DriverSpec
     /// stdin, writes no file, and exits 0. The gate stays as it was; the probe is
     /// what was wrong.
     std::span<std::string_view const> versionFlags;
+    /// What a `RemoteLine::Forwarded` source prefix map is prefixed with on a dispatched
+    /// line, so this driver applies it to PREPROCESSED input; empty when the driver
+    /// applies the plain spelling there itself.
+    ///
+    /// clang's driver discards `-fmacro-prefix-map` when its input is preprocessed text:
+    /// it warns `argument unused during compilation` and never hands the rule to `-cc1`.
+    /// `__FILE__` was expanded on the client and survives that; `__builtin_FILE()` and
+    /// every `std::source_location` are resolved by the worker's frontend, which then
+    /// embedded the client's ABSOLUTE path in an object stored under the mapped key.
+    /// Measured on clang 22.1.8: `-Xclang=` reaches `-cc1` on that input and maps the
+    /// name, and gcc maps it from the plain spelling. FUSED (`-Xclang=<arg>`) because a
+    /// worker judges each argument alone, and the two-argument `-Xclang <arg>` would ask
+    /// it to admit a bare `-Xclang`. See `CarriedSourcePrefixMap`.
+    std::string_view preprocessedSourceMapCarrier;
 };
 
 /// One argument a pass-through hands to a different driver grammar: what it carries, and
@@ -728,6 +744,26 @@ struct PassThroughMatch
 /// @return What it carries, or nullopt when it is no pass-through for that family.
 [[nodiscard]] std::optional<PassThroughMatch> MatchPassThrough(std::string_view arg, DriverFamily family) noexcept;
 
+/// Whether @p arg is a source prefix map in the spelling @p driver's dispatched line
+/// carries it in (`DriverSpec::preprocessedSourceMapCarrier`), and the map it carries.
+///
+/// **The one carve-out from the `-Xclang` denial, and deliberately exact**: the carrier
+/// fused to a `RemoteLine::Forwarded` row with a fused `<from>=<to>` value, and nothing
+/// else `-Xclang` could hand `-cc1`. Read by BOTH ends -- `FindDeniedArgument` asks it
+/// inside itself, so the launcher's pre-lease check and the worker's refusal agree, and
+/// the worker's allowlist admits the carried map only for the driver whose carrier it is.
+/// @param arg    The argument as it appears on a dispatched line.
+/// @param driver The driver that receives @p arg.
+/// @return The carried argument, or nullopt when @p arg is not one.
+[[nodiscard]] std::optional<std::string_view> CarriedSourcePrefixMap(std::string_view arg, DriverSpec const& driver);
+
+/// Whether @p arg is a carried source prefix map for ANY driver of @p family, which is
+/// all the launcher's pre-lease check knows of the driver.
+/// @param arg    The argument as it appears on a dispatched line.
+/// @param family The family of the driver that receives @p arg.
+/// @return The carried argument, or nullopt when @p arg is not one.
+[[nodiscard]] std::optional<std::string_view> CarriedSourcePrefixMap(std::string_view arg, DriverFamily family);
+
 /// Whether `arg` makes a compile write something BESIDES its object file.
 ///
 /// A cache hit reproduces the object and the dependency record and nothing else, so