Skip to content

Commit 37a74f6

Browse files
committed
fix: a payload that starts through its home's loader runs on the home's runtime, read from its programs; a compiler that did not answer says so
Which toolchain payloads run on the C runtime payload was a list (gcc, llvm). The emsdk payload's clang names the home's loader link (<home>/abi) as well; restored from a store cached without that link it did not start, and nothing pinned the runtime again (mtest 0.8.4's Web row). runtime_payload_kind now reads a placed payload's programs: one of bin/ whose PT_INTERP lies inside its own provider home, as the provider's layout gives it. No name and no system loader is written down; a Mach-O or PE program names no loader. A probe that fails with its errors states its last line of output, and clang's std probe keeps why it went unanswered (stdModuleUnanswered), which the refusal of `import std` gives. built_artifact.sh reads the artifact events with grep and sed -E: the hermetic container has no python. Design record 2026-11-14 §5.5 (I-L); docs/91 (en, zh); unit cases RuntimePayload/APayloadStartsThroughItsHome and RuntimePayload/APayloadOnTheSystemsLoaderDoesNot.
1 parent efeb1f2 commit 37a74f6

16 files changed

Lines changed: 210 additions & 54 deletions

‎.agents/docs/2026-11-14-ci-and-release-time-cost-design.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,3 +145,36 @@ graph 778 已改为读取构建图,不依赖输出的完整性。
145145
本地核验:`mcpp test e2e/basics --no-run --no-programs --archive` 的程序阶段 0.96 s(只规划),归档不含程序;
146146
未指定 `MCPP_BIN_MCPP` 的运行以退出码 2 被拒绝;指定 `/bin/false` 时 basics 失败,说明测试使用的是所指定的程序。
147147
用例 1085(重放构建报告产物,包与工作区)与 1086(`--no-programs` 的写出、拒绝、运行与本地运行)通过。
148+
149+
### 5.5 从缓存恢复的 store 中无法启动的工具链(mtest 0.8.4 发布)
150+
151+
**现象。** mtest v0.8.4 的发布运行(43556169215)在 web 行失败:`source imports std but toolchain 'clang 6.0.9
152+
(wasm32-emscripten)' provides no std module source`。同一提交在 PR 运行(43554279064)中通过。
153+
154+
**与此前版本的对比。** 0.8.3 及以前没有 web 行;0.8.4 首次以 emsdk 构建。三次运行的区别只在缓存:PR 运行恢复的
155+
store 不含 emsdk,现场安装后通过;main 运行同样现场安装、通过,并保存了含 emsdk 的 store(键 `722042…`);tag 运行与
156+
mtest#23 的 PR 运行恢复这份 store,emsdk 被视为已就位,均失败。失败与 mtest 的代码无关,是任何恢复含 emsdk 的 store
157+
的运行都会遇到的。在 runner 上诊断:`bin/clang` 存在,执行时 `ENOENT`;其 `PT_INTERP` 为
158+
`<home>/abi/linux-x86_64-glibc/ld-linux-x86-64.so.2`,`RUNPATH` 同样经过 `<home>/abi`,而 `<home>/abi` 不在缓存路径中。
159+
160+
**是否为引入的缺陷。** 是,但不是本 PR 引入,也不在 mtest:
161+
- 本机 2026-09-01 安装的 emsdk(当时 mcpp 自行修补 payload),其 clang 的 `PT_INTERP` 直接指向 store 中的 glibc
162+
payload,store 是自包含的;自运行库钉版本由提供者绑定(设计 2026-10-16 §7.1–7.2)起,xlings 2026.10.26.1 安装的
163+
同一 payload 指向 home 的链接 `<home>/abi`。store 因此不再自包含,链接属于 home 的状态。
164+
- mcpp 在取用工具链时确保运行库存在并保持钉版本(`ensure_declared_runtime` → `pin_runtime`,链接缺失则重建),
165+
但“哪些 payload 运行在运行库上”由一张名单回答(`runtime_payload_kind`:gcc、llvm)。emsdk 的 clang 同样运行在
166+
该运行库上,却不在名单中;新安装时 xlings 安装其 glibc 依赖顺带建立链接,掩盖了这一点,从缓存恢复时便暴露。
167+
缺陷自 store 改为经 `<home>/abi` 链接时即潜伏,emsdk 首次从缓存恢复时显现。
168+
- `em++ --version` 由 emcc 自己回答而不启动 clang,探测因此“成功”;随后两个 std 探测失败的原因被丢弃,报告只剩
169+
“没有 std 模块”,指向了错误的方向。
170+
171+
**设计层面的解决。**
172+
173+
| 编号 | 机制 | 实现 |
174+
|---|---|---|
175+
| I | 运行在运行库上的 payload 由事实回答 | `runtime_payload_kind(pkg, root)`:gcc、llvm 在安装前按配方回答;payload 就位后按其程序回答(`starts_through_its_home`:`bin/` 中第一个指名加载器的 ELF,其 `PT_INTERP` 是否位于该 payload 自身的提供者 home 内,home 由提供者的 layout 给出,两侧按文件系统解析)。不列名字,不写系统加载器路径;Mach-O 与 PE 不指名加载器,答案为否,各系统形式相同;新的工具链族无需登记。各取用点把 payload 目录传入 |
176+
| J | 探测失败如实报告 | 带错误输出的探测失败时,错误信息附上其输出的最后一行;clang 的 std 探测未得到回答时,工具链记下原因(`stdModuleUnanswered`),`import std` 的拒绝信息随之给出“编译器未回答:…” |
177+
| K | 提供者状态整体缓存 | mtest 的 CI 把 `data/xpkgs` 与 `abi` 一并缓存(mtest 0.8.5);mcpp 自己的 CI 缓存整个 home,不受影响。缓存路径改变后旧的缓存条目不再被恢复 |
178+
| L | 不依赖解释器的产物查询 | hermetic 容器中没有 python,`built_artifact.sh` 改为以 grep 与 sed(`-E`,GNU 与 BSD 相同)读取 `artifact` 事件:每行一个产物,其对象只有一个 `path` 键,不依赖键序 |
179+
180+
I 使从缓存恢复的 emsdk 在 mcpp 2026.11.13.1 起自动重建链接;K 使 mtest 在其固定的 mcpp 版本上即可工作。二者互不依赖。

‎.github/tools/built_artifact.sh‎

Lines changed: 9 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@
44
# built_artifact.sh <mcpp> <name> [build arguments...]
55
#
66
# Runs `<mcpp> build --message-format json [arguments]` and prints the absolute
7-
# path of the artifact named <name> (docs/50-machine-output.md §9: an
8-
# `artifact` event per product, and the `artifacts` of the closing record).
9-
# The build's narration stays on standard error. A failed build, or one that
10-
# reports no such artifact, exits non-zero.
7+
# path of the artifact named <name>, from the build's `artifact` events
8+
# (docs/50-machine-output.md §9): one line per product, whose object has one
9+
# `path`. Read with grep and sed (`-E`, as GNU and BSD sed read it), so that a
10+
# machine with no JSON reader (the hermetic container) can ask it. The build's narration stays on standard
11+
# error. A failed build, or one that reports no such artifact, exits non-zero.
1112
#
1213
# Instead of a search of target/: a self-hosting build keeps the engine it
1314
# runs under target/<triple>/<fingerprint>/engine/epoch-N/mcpp, and a search
@@ -19,27 +20,7 @@ mcpp=$1; name=$2; shift 2
1920
report=$(mktemp)
2021
trap 'rm -f "$report"' EXIT
2122
"$mcpp" build --message-format json "$@" > "$report"
22-
py=python3; command -v python3 > /dev/null 2>&1 || py=python
23-
"$py" - "$report" "$name" <<'PY'
24-
import json, sys
25-
report, name = sys.argv[1], sys.argv[2]
26-
found = None
27-
with open(report, encoding="utf-8") as f:
28-
for line in f:
29-
try:
30-
doc = json.loads(line)
31-
except ValueError:
32-
continue
33-
if not isinstance(doc, dict):
34-
continue
35-
records = []
36-
if isinstance(doc.get("artifact"), dict):
37-
records.append(doc["artifact"])
38-
records.extend(a for a in doc.get("artifacts") or [] if isinstance(a, dict))
39-
for a in records:
40-
if a.get("name") == name and a.get("path"):
41-
found = a["path"]
42-
if not found:
43-
sys.exit(f"the build reported no artifact named {name}")
44-
print(found)
45-
PY
23+
found=$(grep -F '"event":"artifact"' "$report" | grep -F "\"name\":\"$name\"" \
24+
| sed -nE 's/.*"path":"(([^"\\]|\\.)*)".*/\1/p' | tail -1 | sed 's/\\\\/\\/g' || true)
25+
[ -n "$found" ] || { echo "the build reported no artifact named $name" >&2; exit 1; }
26+
printf '%s\n' "$found"

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,16 @@
112112

113113
### Fixed
114114

115+
- A toolchain payload whose programs start through its home's loader runs on
116+
the home's C runtime, whatever its name: taken from a store restored without
117+
the home's loader links (`<home>/abi`), it has the runtime pinned and the
118+
links made again, as a gcc or llvm payload had. Which payloads run on the
119+
runtime is read from their programs (`PT_INTERP` inside their provider
120+
home), not from a list; the emsdk payload restored from a CI cache did not
121+
start, and was reported as providing no std module.
122+
- `import std` refused for a toolchain whose compiler did not answer the
123+
question says why (the probe's last line of output), instead of only that
124+
no std module source was found.
115125
- `mcpp build --message-format json` of an unchanged workspace reports the
116126
artifacts of the build it replays, as a package's fast path did; it
117127
reported none, and its envelope carried no record.

‎docs/91-toolchain-internals.md‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -327,11 +327,22 @@ from an upstream home runs on that home's runtime; what it produces is the same,
327327
because the artifact's loader, run paths and sysroot come from the build's own
328328
command lines (§5).
329329

330+
The loader a payload names may be the home's link to it (`<home>/abi/<abi>/`)
331+
rather than the runtime payload itself. The links are state of the home, made
332+
by the pin; a store restored without them (a CI cache of `data/xpkgs` alone)
333+
holds payloads that do not start. So whenever a payload that runs on the
334+
runtime is taken -- installed, or found already placed -- mcpp asks the
335+
provider to keep the pin (`ensure_declared_runtime`), which makes the links
336+
again when they are missing. Which payloads run on the runtime is read from the
337+
payload's programs, not from its name (2026.11.13.1): one of `bin/` whose
338+
`PT_INTERP` lies in its own provider home. A Mach-O or PE program names no
339+
loader, so the question has the same form on every system.
340+
330341
**What mcpp still does here** (`post_install.cppm`, all read-only):
331342

332343
| function | answer |
333344
|---|---|
334-
| `runtime_payload_kind` | whether a toolchain runs on a C runtime payload: `gcc`, `llvm` on Linux; nothing for musl-gcc, a system compiler or a PE toolchain |
345+
| `runtime_payload_kind` | whether a toolchain runs on a C runtime payload: `gcc`, `llvm` on Linux by their recipes; once placed, any payload whose programs start through its home's loader (`starts_through_its_home`, the emsdk payload among them); nothing for musl-gcc, a system compiler or a PE toolchain |
335346
| `select_glibc_payload_lib` | the library directory of exactly `glibc@<version>` (never another installed version) |
336347
| `detect_baked_loader`, `baked_runtime_binding` | the loader and runtime a payload was bound to, read from its ELF headers |
337348
| `write_clean_link_specs` | the per-build `-specs=` file in the build directory (§5) |
@@ -502,8 +513,9 @@ loader coupling, so a host binary that stumbles onto them is at worst confused.
502513
3. **Capabilities** (`src/toolchain/provider.cppm`): stdlib identity, BMI
503514
traits, and feature switches consumed by `flags.cppm`.
504515
4. **Runtime kind** (`post_install.cppm`): whether the payload runs on a C
505-
runtime payload (`runtime_payload_kind`); the provider binds it to the
506-
pinned runtime at install, and mcpp writes nothing into it.
516+
runtime payload (`runtime_payload_kind`), read from its programs once it
517+
is placed; the provider binds it to the pinned runtime at install, and mcpp
518+
writes nothing into it. A new family needs no entry here.
507519
5. **e2e**: a hermetic-link test in the spirit of
508520
`86_llvm_hermetic_link.sh`, and coverage in the no-host-toolchain CI job.
509521

‎docs/zh/91-toolchain-internals.md‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -296,11 +296,17 @@ payload 由提供者(xlings)在安装时写入,此后没有任何程序写
296296
(`glibc@2.44.3`,`xlings config --runtime`,只设一次),提供者把工具链链接到该运行库的加载器与库。从上游 home
297297
借来的工具链运行在那个 home 的运行库上;它的产物不变,因为产物的加载器、运行路径与 sysroot 来自构建自己的命令行(§5)。
298298

299+
payload 指名的加载器可能是 home 指向它的链接(`<home>/abi/<abi>/`),而不是运行库 payload 本身。这些链接是 home 的状态,
300+
由钉版本建立;不带它们恢复的 store(只缓存 `data/xpkgs` 的 CI 缓存)中的 payload 无法启动。因此,凡取用一个运行在该运行库
301+
上的 payload——新安装的,或已经就位的——mcpp 都请提供者保持钉版本(`ensure_declared_runtime`),链接缺失时随之重建。
302+
哪些 payload 运行在运行库上,由 payload 的程序而不是其名字回答(2026.11.13.1):`bin/` 中 `PT_INTERP` 位于其自身
303+
提供者 home 内的程序。Mach-O 与 PE 程序不指名加载器,因此这一问题在各系统上形式相同。
304+
299305
**mcpp 在这里仍做的事**(`post_install.cppm`,全部只读):
300306

301307
| 函数 | 回答 |
302308
|---|---|
303-
| `runtime_payload_kind` | 工具链是否运行在 C 运行库 payload 上:Linux 上的 `gcc`、`llvm`;musl-gcc、系统编译器与 PE 工具链为否 |
309+
| `runtime_payload_kind` | 工具链是否运行在 C 运行库 payload 上:Linux 上的 `gcc`、`llvm` 按其配方;就位之后,任何程序经其 home 的加载器启动的 payload(`starts_through_its_home`,emsdk payload 即在其中);musl-gcc、系统编译器与 PE 工具链为否 |
304310
| `select_glibc_payload_lib` | 恰好 `glibc@<版本>` 的库目录(绝不取另一个已安装版本) |
305311
| `detect_baked_loader`、`baked_runtime_binding` | payload 绑定的加载器与运行库,读自其 ELF 头 |
306312
| `write_clean_link_specs` | 构建目录中每次构建的 `-specs=` 文件(§5) |
@@ -459,7 +465,8 @@ loader,任何环境变量都无法覆盖它,因此一个 `popen()`/`system()
459465
3. **能力**(`src/toolchain/provider.cppm`):stdlib 身份、BMI 特性,
460466
以及 `flags.cppm` 消费的 feature 开关。
461467
4. **运行库类别**(`post_install.cppm`):payload 是否运行在 C 运行库 payload 上
462-
(`runtime_payload_kind`);提供者在安装时把它绑定到钉住的运行库,mcpp 不往其中写入任何东西。
468+
(`runtime_payload_kind`),就位后由其程序回答;提供者在安装时把它绑定到钉住的运行库,mcpp 不往其中写入任何东西。
469+
新的工具链族无需在此登记。
463470
5. **e2e**:按 `86_llvm_hermetic_link.sh` 的思路写一个 hermetic 链接
464471
测试,并纳入无宿主工具链的 CI job。
465472

‎modules/toolchain-model/src/model.cppm‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,9 @@ struct Toolchain {
240240
// GCC/libc++ answer 20; MSVC answers 20 from cl 19.38 (VS 2022 17.8,
241241
// microsoft/STL#3977) and 23 below that.
242242
int importStdMinLevel = 0;
243+
// Why no std module was found, when the compiler did not answer the
244+
// question (its probe failed); empty when it answered that it has none.
245+
std::string stdModuleUnanswered;
243246

244247
// Something about HOW this toolchain was resolved that the user has to be
245248
// told, but which is not a failure. Non-empty ⇒ the caller MUST surface it.

‎src/build/prepare/local_toolchain.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -303,7 +303,7 @@ step2_use_local_toolchain(PrepareState& state, const mcpp::toolchain::ToolchainS
303303
pkg.ximName = family;
304304
pkg.family = family == "llvm" ? mcpp::toolchain::Family::Llvm : mcpp::toolchain::Family::Gcc;
305305
pkg.needsGccPostInstallFixup = family == "gcc";
306-
if (auto provided = state.provide_runtime_payload(pkg); !provided)
306+
if (auto provided = state.provide_runtime_payload(pkg, {}); !provided)
307307
return std::unexpected(provided.error());
308308
}
309309
return {};

‎src/build/prepare/scan.cpp‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -910,8 +910,11 @@ step11_std_module_availability_gate(PrepareState& state, bool needsStdModule) {
910910
state.tc->targetTriple, why, remedy));
911911
}
912912
return std::unexpected(std::format(
913-
"source imports std but toolchain '{}' provides no std module source",
914-
state.tc->label()));
913+
"source imports std but toolchain '{}' provides no std module source{}",
914+
state.tc->label(),
915+
state.tc->stdModuleUnanswered.empty()
916+
? std::string{}
917+
: std::format("\n the compiler did not answer: {}", state.tc->stdModuleUnanswered)));
915918
}
916919
// `std.compat` is a second module of the same library, and not every
917920
// library has it: a package that supplies `std` alone, or a toolset without

‎src/build/prepare/state.cppm‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -616,7 +616,8 @@ struct PrepareState {
616616
std::string runtimePayload;
617617
std::filesystem::path runtimeLibDir;
618618
bool runtimePayloadProvided = false;
619-
std::function<std::expected<void, std::string>(const mcpp::toolchain::XimToolchainPackage&)> provide_runtime_payload;
619+
std::function<std::expected<void, std::string>(const mcpp::toolchain::XimToolchainPackage&,
620+
const std::filesystem::path&)> provide_runtime_payload;
620621
std::string effectiveProfile;
621622
std::vector<std::filesystem::path> storeRoots;
622623
std::optional<std::string> tcSpec;

‎src/build/prepare/toolchain.cpp‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -213,10 +213,11 @@ static std::expected<void, std::string> step1_define_early_toolchain_closures(Pr
213213
// refreshed with the binding, because detection and the fingerprint read
214214
// them afterwards.
215215
state.runtimePayloadProvided = false;
216-
state.provide_runtime_payload = [&](const mcpp::toolchain::XimToolchainPackage& pkg)
216+
state.provide_runtime_payload = [&](const mcpp::toolchain::XimToolchainPackage& pkg,
217+
const std::filesystem::path& payloadRoot)
217218
-> std::expected<void, std::string> {
218219
if (state.runtimePayloadProvided) return {};
219-
if (mcpp::toolchain::runtime_payload_kind(pkg).empty()) return {};
220+
if (mcpp::toolchain::runtime_payload_kind(pkg, payloadRoot).empty()) return {};
220221
state.runtimePayloadProvided = true;
221222
auto cfgP = state.get_cfg(true);
222223
if (!cfgP) return {};
@@ -1813,7 +1814,7 @@ step2_resolve_explicit_spec(PrepareState& state, ToolchainResolveCtx& ctx) {
18131814
// manifest [toolchain] path previously ran none, so a freshly
18141815
// auto-installed payload kept its stale install-time cfg /
18151816
// unpatched runtime libs.
1816-
if (auto provided = state.provide_runtime_payload(pkg); !provided)
1817+
if (auto provided = state.provide_runtime_payload(pkg, payload->root); !provided)
18171818
return std::unexpected(provided.error());
18181819
// Canonical rendering, whatever spelling the manifest/config used:
18191820
// "Resolved gcc@16.1.0 → x86_64-linux-musl → <frontend>".
@@ -2047,7 +2048,7 @@ step2_first_run_auto_install(PrepareState& state) {
20472048
// `mcpp toolchain install` performs — without it a fresh sandbox
20482049
// gcc cannot find the C library (stdlib.h: No such file or
20492050
// directory) and a fresh llvm keeps its stale install-time cfg.
2050-
if (auto provided = state.provide_runtime_payload(defaultPkg); !provided)
2051+
if (auto provided = state.provide_runtime_payload(defaultPkg, payload->root); !provided)
20512052
return std::unexpected(provided.error());
20522053

20532054
// Persist the default so we don't ask again next time.
@@ -2307,7 +2308,7 @@ step2_msvc_abi_without_msvc_repair(PrepareState& state) {
23072308
gnuPkg.target(),
23082309
mcpp::toolchain::payload_frontend_dir(payloadR->root, gnuPkg).string()));
23092310
}
2310-
if (auto provided = state.provide_runtime_payload(gnuPkg); !provided)
2311+
if (auto provided = state.provide_runtime_payload(gnuPkg, payloadR->root); !provided)
23112312
return std::unexpected(provided.error());
23122313

23132314
// Persist both axes so the repair happens once, not on every build.

0 commit comments

Comments
 (0)