Skip to content

Commit 90719e0

Browse files
fix: GCC module maps name the BMI directory, keyed in the cache; trim_paths is per package (#844)
A GCC module map now names the BMI directory as its repository ($root) and every path below it, so a BMI built with GCC's own mapper and one built with a map record their imports at the same paths: an indirect import, and `import std.compat`, are found in a unit with a map. A dependency BMI an earlier mcpp cached spells its imports the earlier way, so the cache key of a package that provides a module under GCC carries the spelling (the upgrade lane's `modules` variant). The dyndep step reads $root. The `trim_paths` prefix map left the flags every unit shares: a unit carries the fixed roots and its own package's closure, so switching workspace members no longer recompiles their common dependencies (e2e 872 F). Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
1 parent 5e273a0 commit 90719e0

17 files changed

Lines changed: 395 additions & 22 deletions

‎.github/tools/upgrade_lane.sh‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,31 @@ assert installed.get("revision", 0) == 0, installed
131131
PY
132132
fi
133133
[ "$VARIANT" = orphan ] && { run old-tc-gcc "$W" "$OLD" toolchain install gcc@16.1.0 || fail "the previous release cannot install gcc"; }
134+
# modules_project <dir>: `openkal.timeout` imports `openkal.datagram`, an
135+
# INDIRECT import of the program's unit, found at the path its BMI records.
136+
modules_project() {
137+
mkdir -p "$W/$1/src"
138+
cat > "$W/$1/mcpp.toml" <<'TOML'
139+
[package]
140+
name = "kp"
141+
version = "0.1.0"
142+
143+
[dependencies]
144+
openkal = "0.16.0"
145+
146+
[target.linux.dependencies]
147+
openkal-linux = "0.17.0"
148+
149+
[toolchain]
150+
default = "gcc@16.1.0"
151+
TOML
152+
printf 'import openkal.timeout;\nint main() { return 0; }\n' > "$W/$1/src/main.cpp"
153+
}
154+
if [ "$VARIANT" = modules ]; then
155+
run old-tc-gcc "$W" "$OLD" toolchain install gcc@16.1.0 || fail "the previous release cannot install gcc"
156+
modules_project m-old
157+
run old-modules "$W/m-old" "$OLD" build || fail "the previous release cannot build the module project"
158+
fi
134159
new_project p-old "$OLD"
135160
run old-run "$W/p-old" "$OLD" run || fail "the previous release cannot run a new project"
136161

@@ -192,6 +217,18 @@ if [ "$VARIANT" = snapshot ]; then
192217
grep -q "packaging revision .* available" "$W/new-toolchain-list.log" \
193218
|| fail "toolchain list does not offer the newer packaging"
194219
fi
220+
if [ "$VARIANT" = modules ]; then
221+
# A BMI of the previous spelling is not read: the dependency is rebuilt
222+
# once (its cache key carries the spelling), then served from the cache.
223+
modules_project m-new1
224+
run new-modules1 "$W/m-new1" "$NEW" build -v || fail "the module project does not build over the previous release's cache"
225+
grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules1.log" \
226+
&& fail "a dependency BMI of the previous spelling was taken from the cache"
227+
modules_project m-new2
228+
run new-modules2 "$W/m-new2" "$NEW" build -v || fail "the second module project failed"
229+
grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules2.log" \
230+
|| fail "the rebuilt dependency was not served from the cache"
231+
fi
195232
if [ "$VARIANT" = orphan ]; then
196233
run new-provision "$W" "$NEW" toolchain install "$TC" || fail "toolchain install could not replace the payload"
197234
grep -q "installed asset registration missing" "$W/new-provision.log" \

‎.github/workflows/ci-upgrade.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ jobs:
2929
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: plain }
3030
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: snapshot }
3131
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: orphan }
32+
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: modules }
3233
- { host: linux-aarch64, runs-on: ubuntu-24.04-arm, variant: plain }
3334
- { host: macos-arm64, runs-on: macos-15, variant: plain }
3435
- { host: windows-x86_64, runs-on: windows-latest, variant: plain }

‎CHANGELOG.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,27 @@
1414
`app.wasm`, the edge's implicit output was missing, and `mcpp pack` of a
1515
`wasm32-emscripten` program (mcpp-plugins' `dist-web`) found no stem family.
1616
A WebAssembly link is in place, as a PE link is.
17+
- **A module's indirect import is found whatever map its importer has (GCC).**
18+
A unit that reaches another package's module is compiled with a module map,
19+
and a unit that imports only the root's modules with GCC's own mapper. A BMI
20+
records the paths of its imports as its compile's mapper spelt them, and the
21+
map named no repository, so a unit with a map that imported a root module
22+
compiled without one looked for that module's import beside the build
23+
directory (`failed to read compiled module`, `Bad import dependency`). The
24+
map now names the BMI directory as its repository (`$root gcm.cache`) and
25+
every path below it, as GCC's own mapper does; the dyndep step reads the
26+
same form. The same cause failed `import std.compat` in a unit with a map, in
27+
any project with a dependency. A dependency's BMI that an earlier mcpp put in
28+
the global build cache spells its imports the earlier way, so the cache key
29+
of a package that provides a module under GCC carries the spelling: such a
30+
package is compiled once more after the upgrade, and a package of object
31+
files keeps its entry.
32+
- **Switching workspace members under `trim_paths` no longer recompiles their
33+
shared dependencies** (#844). The prefix map of the whole plan was part of
34+
the flags every unit shares, so a dependency's command line named the roots
35+
of the members being built, and `--release -p cli` then `-p gui` recompiled
36+
the common packages each time. A unit's map is now the fixed roots and the
37+
roots of its own package's dependency closure.
1738

1839
## [2026.10.26.1] - 2026-10-26
1940

‎docs/04-mcpp-toml.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1505,7 +1505,7 @@ debug = false # -g
15051505
lto = true # -flto; /GL + /LTCG on cl.exe (note: some packaged gcc builds ship without the LTO plugin)
15061506
strip = true # -s at link time
15071507
ndebug = true # -DNDEBUG for every unit of the graph (default: on for release/dist)
1508-
trim_paths = true # -ffile-prefix-map for every unit (default: on for release/dist)
1508+
trim_paths = true # -ffile-prefix-map per unit (default: on for release/dist)
15091509
# passthrough escape hatch (fixed keys, open values):
15101510
cflags = ["-fno-plt"]
15111511
cxxflags = ["-fno-plt"]
@@ -1534,7 +1534,10 @@ ldflags = []
15341534
`<name>@<version>/…` (the root package's too, so a report reads
15351535
`app@1.2.0/src/main.cpp`), the store, home, build directory, toolchain and sysroot to
15361536
`/mcpp/store`, `/mcpp/home`, `/mcpp/build`, `/mcpp/toolchain` and `/mcpp/sysroot`. Two
1537-
checkouts of one tree then embed identical paths. It applies to what `build`, `run`
1537+
checkouts of one tree then embed identical paths. A unit's map holds the fixed roots
1538+
and the roots of its own package's dependency closure, so a dependency's command line
1539+
is the same whichever workspace members a build selects (2026.10.26.2+). It applies to
1540+
what `build`, `run`
15381541
and `pack` produce, not to `mcpp test`: a test binary runs where it was built and
15391542
commonly finds its fixtures through `__FILE__`. A build with debug information
15401543
also writes `source-map.gdbinit` and `source-map.lldbinit` beside its outputs, which

‎docs/zh/04-mcpp-toml.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1400,7 +1400,7 @@ debug = false # -g
14001400
lto = true # -flto; /GL + /LTCG on cl.exe (note: some packaged gcc builds ship without the LTO plugin)
14011401
strip = true # -s at link time
14021402
ndebug = true # 整个图的每个单元 -DNDEBUG(release/dist 默认开)
1403-
trim_paths = true # 整个图的每个单元 -ffile-prefix-map(release/dist 默认开)
1403+
trim_paths = true # 按单元的 -ffile-prefix-map(release/dist 默认开)
14041404
# passthrough escape hatch (fixed keys, open values):
14051405
cflags = ["-fno-plt"]
14061406
cxxflags = ["-fno-plt"]
@@ -1429,7 +1429,8 @@ ldflags = []
14291429
映射为它所代表的东西:每个包的源码映射为 `<name>@<version>/…`(根包同样如此,报告读作
14301430
`app@1.2.0/src/main.cpp`),存储、home、构建目录、工具链与 sysroot 分别映射为 `/mcpp/store`、
14311431
`/mcpp/home`、`/mcpp/build`、`/mcpp/toolchain` 与 `/mcpp/sysroot`。同一棵树的两个检出因此嵌入
1432-
相同的路径。它作用于 `build`、`run` 与 `pack` 的产物,不作用于 `mcpp test`:测试二进制在构建处运行,
1432+
相同的路径。一个单元的映射只含固定根与它所属包的依赖闭包中各包的根,所以无论构建选择哪些工作空间成员,
1433+
依赖的命令行都相同(2026.10.26.2+)。它作用于 `build`、`run` 与 `pack` 的产物,不作用于 `mcpp test`:测试二进制在构建处运行,
14331434
且常借 `__FILE__` 找到它的夹具。带调试信息的构建还在输出旁写出 `source-map.gdbinit` 与 `source-map.lldbinit`,
14341435
供调试器把路径映射回来。cl.exe 没有文档化的等价开关,构建会报告一次。在 profile 下写
14351436
`ndebug = false` 或 `trim_paths = false` 可保留以前的产物;升级后第一次发布档构建会提示一次。

‎modules/dyndep/src/dyndep.cppm‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,8 @@ struct DyndepOptions {
6363
// The BMI path `name` takes under `opts`: the module map's, or the flat one.
6464
std::string bmi_path_for(std::string_view name, const DyndepOptions& opts);
6565

66-
// Parse a module map: `<module name> <BMI path>` per line.
66+
// Parse a module map: `<module name> <BMI path>` per line. A `$root <dir>`
67+
// line (GCC's repository) puts every relative path after it below `<dir>`.
6768
std::map<std::string, std::string, std::less<>> parse_module_map(std::string_view body);
6869

6970
// Parse a single .ddi JSON body to a UnitInfo. Returns unexpected on JSON error.
@@ -206,14 +207,21 @@ std::string bmi_path_for(std::string_view name, const DyndepOptions& opts) {
206207

207208
std::map<std::string, std::string, std::less<>> parse_module_map(std::string_view body) {
208209
std::map<std::string, std::string, std::less<>> out;
210+
std::string_view root;
209211
while (!body.empty()) {
210212
auto nl = body.find('\n');
211213
auto line = body.substr(0, nl);
212214
body.remove_prefix(nl == std::string_view::npos ? body.size() : nl + 1);
213215
while (!line.empty() && (line.back() == '\r' || line.back() == ' ')) line.remove_suffix(1);
214216
auto sp = line.find(' ');
215217
if (sp == std::string_view::npos || sp == 0) continue;
216-
out.emplace(std::string(line.substr(0, sp)), std::string(line.substr(sp + 1)));
218+
const auto name = line.substr(0, sp);
219+
const auto path = line.substr(sp + 1);
220+
if (name == "$root") { root = path; continue; }
221+
if (root.empty() || path.starts_with('/') || std::filesystem::path(path).is_absolute())
222+
out.emplace(std::string(name), std::string(path));
223+
else
224+
out.emplace(std::string(name), std::string(root) + "/" + std::string(path));
217225
}
218226
return out;
219227
}

‎src/build/cache_key.cppm‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -249,6 +249,23 @@ struct PackageAxes {
249249
// it) — which is exactly why it is easy to leave out and find later as a
250250
// wrong cache hit. Cheap to close now.
251251
std::vector<std::string> moduleExtensions;
252+
// How a BMI this package writes spells the paths of the modules it
253+
// imports — set by the caller only for a package that provides a module
254+
// under a compiler whose BMIs record those paths (GCC), empty otherwise.
255+
//
256+
// GCC writes into a BMI the path of each module it imports as the
257+
// compile's mapper spelt it, and an importer looks for an INDIRECT import
258+
// at that path below its own repository. Since 2026.10.26.2 a module map
259+
// names the BMI directory as its repository (`$root`, design 2026-10-26
260+
// part 2 §10.2) and spells every path below it; a BMI written before
261+
// spelt `gcm.cache/<pkg>/<m>.gcm`, which an importer under the new map
262+
// reads as `gcm.cache/gcm.cache/...` and cannot find. Measured: the
263+
// first build of any project whose module dependency was cached by
264+
// 2026.10.26.1. The compiler, the flags and the sources of such an entry
265+
// are unchanged, so no other axis moves; this one does, and only for the
266+
// packages whose BMIs carry the spelling. A package of object files keeps
267+
// its key, and `kCacheEpoch` stays where it is.
268+
std::string bmiPathConvention;
252269
// F — keys of direct dependencies, sorted
253270
std::vector<std::string> upstreamKeys;
254271
};
@@ -363,6 +380,8 @@ nlohmann::json to_json(const BuildAxes& b, const PackageAxes& p) {
363380
{"sources", p.sources},
364381
{"module_extensions", p.moduleExtensions},
365382
};
383+
if (!p.bmiPathConvention.empty())
384+
j["language"]["bmi_path_convention"] = p.bmiPathConvention;
366385
j["upstream"] = p.upstreamKeys;
367386
return j;
368387
}
@@ -412,6 +431,8 @@ std::string key_hex(const BuildAxes& b, const PackageAxes& p) {
412431
put_list(s, "srcglobs", p.sourceGlobs);
413432
put_list(s, "sources", p.sources);
414433
put_list(s, "modexts", p.moduleExtensions);
434+
// Only when set, so the key of a package without one is unchanged.
435+
if (!p.bmiPathConvention.empty()) put(s, "bmipath", p.bmiPathConvention);
415436
// F
416437
put_list(s, "upstream", p.upstreamKeys);
417438
return mcpp::toolchain::hash_string(s);

‎src/build/flags.cppm‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,10 @@ struct CompileFlags {
6868
// The link words that name the island's C++ runtime (design 2026-10-26
6969
// §6.3), placed after the build's own runtime in each C++ unit's tail.
7070
std::string ldForeignRuntime;
71+
// `trim_paths` (mcpp#844): the rendered prefix map of each package, by
72+
// `CompileUnit::packageName`, appended to that package's C and C++ units.
73+
// Empty on cl.exe, which has no such switch.
74+
std::map<std::string, std::string, std::less<>> trimByPackage;
7175
// A compile-command prefix stated with a toolchain named by path
7276
// (`launcher = "ccache"`, mcpp#755): written in front of the compiler in
7377
// the build graph, and not into the compile database, whose consumers
@@ -1183,8 +1187,16 @@ CompileFlags compute_flags(const BuildPlan& plan) {
11831187
"paths: the artifact keeps the absolute paths it was built from (clang on "
11841188
"`*-windows-msvc` trims them)");
11851189
} else {
1186-
for (auto const& [from, to] : plan.trimMap)
1187-
opt_flag += " -ffile-prefix-map=" + escape_path(from) + "=" + to;
1190+
// Per package, not here (mcpp#844): a map of the whole plan in
1191+
// the flags every unit shares made a dependency's command line
1192+
// depend on the members the plan builds, and switching members
1193+
// recompiled it. A unit's map is its package's (`packageTrimMaps`).
1194+
for (auto const& [package, map] : plan.packageTrimMaps) {
1195+
std::string rendered;
1196+
for (auto const& [from, to] : map)
1197+
rendered += " -ffile-prefix-map=" + escape_path(from) + "=" + to;
1198+
f.trimByPackage.emplace(package, std::move(rendered));
1199+
}
11881200
}
11891201
}
11901202

‎src/build/ninja_backend.cppm‎

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1421,6 +1421,18 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements,
14211421

14221422
// All compile/link flags are computed once via flags.cppm.
14231423
auto flags = compute_flags(plan);
1424+
// A unit's own flags: its package's, then its package's `trim_paths` map
1425+
// (mcpp#844), so that nothing outside the package's closure reaches it.
1426+
auto unit_trim = [&](const CompileUnit& cu) -> std::string_view {
1427+
auto it = flags.trimByPackage.find(cu.packageName);
1428+
return it == flags.trimByPackage.end() ? std::string_view{} : std::string_view(it->second);
1429+
};
1430+
auto unit_cxx_flags = [&](const CompileUnit& cu) {
1431+
return join_compile_flags(cu.packageCxxflags) + std::string(unit_trim(cu));
1432+
};
1433+
auto unit_c_flags = [&](const CompileUnit& cu) {
1434+
return join_compile_flags(cu.packageCflags) + std::string(unit_trim(cu));
1435+
};
14241436

14251437
// Everything that has to sit beside the artifact: the runtime placement
14261438
// resolver's one answer over the declared deploys, the runtime search
@@ -2849,7 +2861,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements,
28492861
: escape_ninja_path(cu.object)));
28502862
if (auto includes = local_include_flags(cu, dial); !includes.empty())
28512863
append(std::format(" local_includes ={}\n", includes));
2852-
if (auto flags = join_compile_flags(cu.packageCxxflags); !flags.empty())
2864+
if (auto flags = unit_cxx_flags(cu); !flags.empty())
28532865
append(std::format(" unit_cxxflags ={}\n", flags));
28542866
// The scan has the same suffix-recognition problem as the compile:
28552867
// a driver that does not know `.ixx` hands it to the linker and
@@ -2995,7 +3007,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements,
29953007
e += std::format(" sched_cap = {}\n", plan.scheduleCompilerCap);
29963008
if (auto inc = local_include_flags(cu, dial); !inc.empty())
29973009
e += " local_includes =" + inc + "\n";
2998-
if (auto fl = join_compile_flags(cu.packageCxxflags); !fl.empty())
3010+
if (auto fl = unit_cxx_flags(cu); !fl.empty())
29993011
e += " unit_cxxflags =" + fl + "\n";
30003012
append(std::move(e));
30013013
// The join. THE SOURCE IS AN INPUT, and the BMI is only an
@@ -3060,7 +3072,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements,
30603072
e += "\n dyndep = " + it->second + "\n";
30613073
if (auto inc = local_include_flags(cu, dial); !inc.empty())
30623074
e += " local_includes =" + inc + "\n";
3063-
if (auto fl = join_compile_flags(cu.packageCxxflags); !fl.empty())
3075+
if (auto fl = unit_cxx_flags(cu); !fl.empty())
30643076
e += " unit_cxxflags =" + fl + "\n";
30653077
append(std::move(e));
30663078
};
@@ -3100,10 +3112,10 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements,
31003112
if (auto flags = join_compile_flags(unit_asm_flags(cu)); !flags.empty())
31013113
out_line += " unit_asmflags =" + flags + "\n";
31023114
} else if (is_c_source(cu)) {
3103-
if (auto flags = join_compile_flags(cu.packageCflags); !flags.empty())
3115+
if (auto flags = unit_c_flags(cu); !flags.empty())
31043116
out_line += " unit_cflags =" + flags + "\n";
31053117
} else {
3106-
if (auto flags = join_compile_flags(cu.packageCxxflags); !flags.empty())
3118+
if (auto flags = unit_cxx_flags(cu); !flags.empty())
31073119
out_line += " unit_cxxflags =" + flags + "\n";
31083120
}
31093121
append(std::move(out_line));
@@ -3153,10 +3165,10 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements,
31533165
if (auto flags = join_compile_flags(unit_asm_flags(cu)); !flags.empty())
31543166
out_line += " unit_asmflags =" + flags + "\n";
31553167
} else if (is_c_source(cu)) {
3156-
if (auto flags = join_compile_flags(cu.packageCflags); !flags.empty())
3168+
if (auto flags = unit_c_flags(cu); !flags.empty())
31573169
out_line += " unit_cflags =" + flags + "\n";
31583170
} else {
3159-
if (auto flags = join_compile_flags(cu.packageCxxflags); !flags.empty())
3171+
if (auto flags = unit_cxx_flags(cu); !flags.empty())
31603172
out_line += " unit_cxxflags =" + flags + "\n";
31613173
}
31623174
// Clang needs $bmi_out to emit -fmodule-output=$bmi_out

0 commit comments

Comments
 (0)