diff --git a/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md b/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md new file mode 100644 index 000000000..fef9db772 --- /dev/null +++ b/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md @@ -0,0 +1,642 @@ +--- +subject: cross-platform +status: active +--- + +# 生态缺口补充方案：一个 C 程序走遍全部目标所暴露的通用问题 + +**日期：** 2026-11-15 · **版本：** v9（实施前评审） · **基线：** mcpp 2026.11.15.1、openkal-platform 0.28.0、 +djgpp-platform 0.1.0、mcpp:plugins 0.20.0、dirun 0.3.2、xlings v2026.10.26.1 + +## 1. 来源与方法 + +验证工程是一个 C89 命令解释器（两个程序，约 74 个源文件，自带 System V 可移植层）。它接入 mcpp 只靠清单、 +一个构建程序、一个页面模板与少量 `#if` 守卫，构建 22 个目标；凡有运行环境者，以同一批处理脚本比对输出。运行 +环境：本机（qemu-user、wine、node、okrun、DOSBox、QEMU+OVMF、FreeBSD VM、headless Chromium），GitHub 托管机 +（Linux x86_64/arm64、macOS arm64/x86_64、Windows x64/arm64），模拟器（FreeBSD aarch64、Android x86_64）。 + +编号：G 为 2026-11-11 的缺口，N 为本轮，X 为 xlings 与 xim，D 为 DJGPP，R 为运行器的引擎项，W 为 web，A 为 +运行器架构的缺口（第 10 节）。任务与检查点用小写的仓库缩写加序号（第 7 节），与问题编号不重名。 + +## 2. 原则：基础设施与项目的边界 + +mcpp、openkal、xlings/xim 的包、dirun 与 mcpp:plugins 是通用基础设施。一项修复放在哪一层，按下表判断： + +| 层 | 放什么 | 位置 | 例 | +|---|---|---|---| +| 引擎 | 机制：对所有程序、所有行成立的行为 | mcpp、xlings 本体 | 启动器的系统名映射（N20）、构建依赖的版本统一（N29） | +| 数据 | 事实：某个系统、某个行、某个工具的知识 | builtin.toml、平台包 | wine 运行 Windows 程序、Emscripten 的异常参数、哪些主机能执行哪些架构 | +| 实现 | 一个系统上 openkal 接口的实现 | openkal-* | UEFI 的文件系统、macOS 的 `__bzero` | +| 运行器 | 怎样把程序交给一台机器 | dirun 与随包的配置 | 模拟器的启动、文件的交付、状态的取回 | +| 插件 | 一种分发格式的组装与呈现 | mcpp:plugins | web 目录、以格式命名的 runner | +| 包 | 工具与镜像的获取，以及随包的配置 | xim-pkgindex、mcpp-index | OVMF 经 Shell 传参、Android 镜像与其客体配置 | +| 工程 | 一个工程自己的选择 | 清单、构建程序、模板、CI | 宏定义、页面模板、测试脚本、开启哪些运行器 | + +基础设施只预留扩展点与配置项，不为某个工程写特例；引擎不出现任何系统名、工具名与运行器名（AGENTS.md §2）； +新增的键沿用已有的名字与形状（“一个概念一种拼写”）；每个缺省值是数据，并有四层覆盖（SPEC-004 §2）。 +验证工程中属于工程自身选择的内容（宏、页面模板、测试脚本、对 dirun 的直接调用）留在工程中，不进入本方案。 + +## 3. 结论 + +1. 构建路径已经足够：22 个目标全部构建；主流系统与模拟器上的运行与基准一致（含 Windows arm64）；固件与 + openkal DOS 能启动；浏览器版在 Chrome 中运行。 +2. 剩余问题多数不需要改 mcpp 或 xlings 本体，而在运行器、平台数据、openkal 实现、dirun、xim 包与插件中修复。 + 需要改 mcpp 本体的 11 项在一个 PR 中完成，以 commit 为检查点（第 5 节）。xlings 本体 2 项。 +3. 能放在一个 PR 的放在一个 PR；外部前置只约束检查点与合并的时机，不是拆 PR 的理由；只有内容要等 + **本仓库自己的发布**、或仓库本身是发布渠道时才拆开（第 7 节）：xim-pkgindex 3 个，其余每个仓库 1 个，mcpp 也是 1 个。 +4. v8 依据源码与实测修正了 v7 的运行器规范：mcpp 已有的进程模型足够，不新增信号转发与宽限期（第 10.4 节）。 + +## 4. 问题清单 + +“层”即第 2 节的各行；“文档”指只需写明现有行为。 + +| ID | 现象 | 根因（已核对） | 层 · 归属 | +|---|---|---|---| +| N18 | `x86_64-macos-musl` 发布构建链接失败：`__bzero` 未定义 | clang 在 x86_64 Darwin 上把清零降为 `__bzero`，openkal 的 C 库不提供 | 实现 · openkal-macos | +| N14 | UEFI 程序收不到参数 | 平台的 UEFI runner 命令缺 `--`；ovmf 配置把程序放作 `BOOTX64.EFI` 由固件直启，没有加载参数。**已实测**：不放 `BOOTX64.EFI` 时 OVMF 进入内置 Shell，执行 `startup.nsh`，参数到达程序，状态经 debug-exit 取回（Shell 有 5 秒倒计时） | 数据 · openkal-platform、ovmf 包 | +| N34 | UEFI 程序读不到任何文件：`cd` 为 `/`，`dir` 为 File Not Found | openkal-uefi 未实现 `fs`（其 README 写明 `fs` 待 SIMPLE_FILE_SYSTEM） | 实现 · openkal-uefi | +| N35 | UEFI 参数含引号时切分错误 | openkal-uefi 按空格切分 load options；由 Shell 启动时，Shell 已给出 `EFI_SHELL_PARAMETERS_PROTOCOL` 的 argv | 实现 · openkal-uefi | +| N15 | UEFI 上 mimalloc 警告无安全随机源 | 未接 EFI_RNG_PROTOCOL | 实现 · openkal-uefi | +| N19 | okrun 打不开含 `//` 的路径 | 打开映像前未规范化 | 实现 · openkal-image | +| N12 | dirun 0.3.2 携带的 okrun 落后，仍有 `../` 错误 | dirun 自带一份 okrun 副本 | 运行器 · dirun | +| N13 | DOS 运行器只取回标准输出 | DOSBox 的 shell 无 `2>`；dirun 只有 `output-file` | 运行器 · dirun、dosbox-staging 包 | +| N25 | `mcpp run` 不能把数据文件交给机器类运行器 | 运行协议（docs/41）已规定：搬动产物的运行器复制 `MCPP_RUNTIME_FILES` 所列文件；dirun 未实现 | 运行器 · dirun | +| N28 | `--systems all` 的包缺 macos-x86_64 的 okrun | 平台 runner 的 `hosts`、`built-as` 未列；受 N18 阻挡 | 数据 · openkal-platform | +| N10 | FreeBSD 上 musl 身份的程序取系统名得 `openkal` | hosted 安排报实现名 | 实现 · openkal-musl / openkal-freebsd | +| N17 | openkal DOS 行上 `if … goto` 结束批处理 | 未定位 | 实现 · openkal-dos（技术实验） | +| N22 | 按 `pack_format()` 切换链接参数不生效 | 第二遍按设计不重链；插件文档未说明 | 插件 · mcpp:plugins | +| N31 | 用户打开 `/-index.html`，页面加载不到脚本 | dist-web 的中间页面写在产物目录旁，名字像成品 | 插件 · mcpp:plugins | +| N32 | 从磁盘打开 web 页面，浏览器拒绝取 `.wasm`；pack 不说明产物如何使用 | 浏览器规则；插件未提示 | 插件 · mcpp:plugins（提示与文档） | +| N36 | `mcpp run --pkg-format web` 拒绝：“produced 3 distributables” | dist-web 把每个复制的文件都声明为终端产物；引擎以“未被其他动作消费的输出”为分发物 | 插件 · mcpp:plugins | +| N21 | profile 的 `ldflags` 排在目标块之前 | 顺序未写入文档 | 文档 · mcpp | +| N27 | Emscripten 的 `tcsetattr` 无原始模式 | Emscripten 的行为 | 文档 · mcpp | +| G6 | `mcpp run --target *-freebsd` 只能拒绝 | xim 已有 freebsd-disk-image 14.5（x86_64）；缺 runner 数据与 aarch64 镜像 | 数据、包 | +| G7 | `mcpp run --target *-android` 只能拒绝 | xim 已有模拟器、platform-tools、API 24 镜像；dirun 不能经命令到达客体 | 运行器 · dirun；包；数据 · builtin.toml | +| X1 | Windows 行在 Linux 上需工程自写 `runner = ["wine"]` | xim 已有 wine 11.0；缺 runner 数据 | 数据 · builtin.toml、openkal-platform | +| X4 | openkal-image 0.6.3 的 release 没有二进制；xim 停在 0.6.2 | 发布流程未附产物 | 包 · openkal-image 发布 | +| D1 | DJGPP 的 GCC 为 12.2，无模块 | 上游无 GCC 14 预编译包（build-djgpp 止于 12.2，build-gcc 只有脚本） | 包 · xim（自建） | +| N20 | 打包的 POSIX 启动器以 `uname -s` 原样拼目录，FreeBSD 上找不到包内 okrun | 未小写 | **引擎** | +| N9 | `artifact_name("exe","{name}")` 各行结果不一（`x.exe.exe`、`x.exe`、DJGPP 两份） | B2 规则未统一应用 | **引擎** | +| N16 | `[resources]` 作用于 UEFI 这一 PE 行 | 判据取文件格式而非执行系统 | **引擎** | +| N29 | 工程要 mcpp:plugins 0.21.0、平台要 0.20.0，结果以误导性错误失败 | 同一包的两个版本请求未识别为冲突 | **引擎** | +| N30 | pack 未指名时取第一个程序；插件不提交的原因不进入错误 | 选取与插件的 `target` 互不知晓 | **引擎** | +| N11 | `--all-programs` 只为入口程序生成启动器 | — | **引擎** | +| N23 | i686 映像的载体不构建；`mcpp run` 在 x86_64 上拒绝 i386 映像 | 载体构建未继承平台声明；`same-arch` 不含“能执行该架构”的主机 | **引擎**（执行关系作为数据） | +| N26 | Emscripten 行上 C++ 异常默认不可捕获 | 行未开启 wasm 异常 | **引擎**（行数据与 std 模块参数） | +| N33 | web profile 加 `-sSINGLE_FILE=1` 后，干净构建的打包失败；旧的 `.wasm` 存在时被当作产物打包 | `.wasm` 被声明为无条件的隐式输出；最终错误只说 “no action claimed” | **引擎** | +| R1 | 首次使用代价大的运行器（VM、模拟器、wine）一旦写成数据，`mcpp run` 将在首次运行时下载 GB 级组件或在家目录留下状态 | 决策：默认关闭，可配置（第 5 节） | **引擎**（一个键） | +| R2 | runner 在数据与构建程序中字段不对称；不在包的 `bin/` 下的程序没有可移植的写法；Windows 上脚本型程序的规则未成文（第 10 节 A1–A5） | 两种表达各自生长 | **引擎**（模型与规范） | +| W1 | `mcpp pack --pkg-format web` 的产物没有“打开即可用”的方式 | 页面须经 HTTP 提供；没有以格式命名的 runner | 插件 · mcpp:plugins | +| X2 | Windows 上 `write_file_atomic` 单次 `MoveFileExW` | openxlings/xlings#715 | **xlings** | +| X5 | Windows 上经 xlings 的 shim 启动程序，参数含 `"` 与 `&` 时被拆开，其后的内容被当作命令执行 | shim 经命令解释器转交参数（实测）；影响所有经 shim 启动的工具，含 mcpp | **xlings** | +| X3 | subos 的 `cc` 与主机库错配（`__isoc23_strtol`、`libc.so`） | 已定位：xim `musl` 配方把 11 个与 glibc 同名的链接文件（`crt1.o`、`libc.so` 等）注册进共享 `lib/`，安装 musl 即改变 `cc` 的链接 | xim-pkgindex#1162（已合并）：共享 `lib/` 只留加载器 `ld-musl-.so.1`；链接文件入 `usr/lib/musl/`，头文件在 `usr/include/musl/`；修订号 2 修复已有 home | + +已解决、仅留档：G1、G2、G3、G8。 + +## 5. 需要改 mcpp 本体 + +| ID | 修改 | 机制与数据的分界 | +|---|---|---| +| N20 | 启动器把 `uname -s` 小写后映射包内目录；未知系统列出包内系统后退出，不退到 PATH | 机制 | +| N9 | 统一应用 B2：平台后缀只在平台自身步骤读取时保留；行定义的普通后缀让位于工程命名；DJGPP 只产一份 | 机制 | +| N16 | `[resources]` 适用于执行系统为 Windows 的行 | 判据读行数据的 `execution` | +| N29 | 同一构建中同一包的多个版本请求：可统一时取满足各方的版本，否则报冲突并列出请求者 | 机制 | +| N30 | 插件声明它服务的程序时，pack 据此选取；插件不提交的原因并入引擎错误 | 机制；形状见 5.2 | +| N11 | `--all-programs` 为每个程序生成启动器 | 机制 | +| N23 | 载体构建继承工程的平台声明；“哪个主机能执行哪些架构”写入 builtin.toml | 关系是数据，键为 `-`；形状见 5.2 | +| N26 | wasm32-emscripten 行含 C++ 时启用 wasm 异常，编译、链接与 std 模块一致 | 参数写在行数据中；不新增开关，工程以已有的 flags 覆盖 | +| N33 | 行数据把 `.wasm` 声明为**可选**的同名产物：链接前清除旧文件，存在则暂存；插件不提交时，暂存阶段的警告成为错误的原因 | 同名产物表是数据；形状见 5.2 | +| R1 | runner 的 `enabled`（5.1） | 一个键，四层覆盖 | +| R2 | 运行器的统一模型与规范（第 10 节） | 字段与名字沿用已有的；旧调用保持有效 | + +### 5.1 R1：`enabled` + +- **键。** runner 表（默认与具名）增加 `enabled`，布尔，缺省为真。为假的 runner 存在但不被使用：`mcpp run` 与 + `mcpp test` 同样拒绝，说明所需的包与开启的写法，reason token 为 `runner-not-enabled`，退出状态属 docs/50 + 已有的“拒绝”一类。 +- **它属于哪条陈述。** `enabled` 与 `requires` 一样是“这条命令”的性质：一层替换了 `cmd`，就是一条新的陈述， + `enabled` 回到缺省（SPEC-013 §10 对 `requires` 的现有规则）。所以已经写了 `runner = ["wine", "{}"]` 的工程 + 升级后行为不变。只调整不替换时，覆盖层写不带 `cmd` 的表：`runner = { enabled = true }`（现有读取器要求 + `cmd` 非空，覆盖层放宽为可省；提供者与平台仍必须给出 `cmd`）。 +- **四层覆盖（SPEC-004 §2）。** 引擎缺省为真；提供者或平台的数据标 `enabled = false`；本机 `config.toml`、 + 工程 `mcpp.toml` 以 `[target.] runner = { enabled = true }` 开启；一次性的覆盖用环境变量 + `MCPP_TARGET__RUNNER_ENABLED`，与已有的 `MCPP_TARGET__RUNNER`、`_SYSROOT`、`_RUN_UNDER` 同一 + 构词，不新增命令行开关。 +- **显式指名即同意。** `mcpp run --runner ` 选中一个未开启的具名 runner 时，本次运行视为开启。 + `--pkg-format ` 找到的同名 runner 不算指名。 +- **何时标为关闭（写入 SPEC-016，给数据作者的判据）。** 首次使用要取得整机镜像或模拟器，或在构建树与包仓库 + 之外留下状态。按此：VM、Android 模拟器关闭；wine 关闭（在家目录建立 prefix）；qemu-user、okrun、node 不变。 + +``` +error: x86_64-freebsd runs through a runner that is not enabled. + it needs: xim:dirun@0.4.0, xim:freebsd-disk-image@14.5 (installed on first use) + enable: [target.x86_64-freebsd] runner = { enabled = true } # mcpp.toml, or config.toml for this machine + once: MCPP_TARGET_X86_64_FREEBSD_RUNNER_ENABLED=true mcpp run --target x86_64-freebsd +``` + +### 5.2 三项契约的形状（实现前在各自的检查点中定稿） + +```toml +# N23 builtin.toml：只写已实测的关系；引擎查表，不含任何架构名 +[executes] +"linux-x86_64" = ["i686"] # 已由 CI 验证 +"windows-x86_64" = ["i686"] # 实测后加入 + +# N33 行数据：与主产物同名的附带产物；optional 的缺席不是错误 +[[row.companions]] +suffix = ".wasm" +optional = true +``` + +```cpp +// N30 插件（构建程序）声明它为哪个程序组装分发物；pack 未指名时取它 +mcpp::pack_program("app"); // 一条指令，与 runner()、deploy() 同类；插件以它转达自己的 `target` 选项 +``` + +键名在检查点中对照相邻数据定稿；这里固定的是形状：关系按 `-`，附带产物是行数据的一张表，插件的 +声明是构建程序的一条指令。 + +### 5.3 检查点 + +一个 PR（m），每个 commit 是一个检查点：只做一件事，带自己的测试、规范条款、中英文文档与 CHANGELOG 条目， +CI 在每个检查点上通过。合并保留 commit，不压缩，评审逐个检查点进行。 + +| 检查点 | 内容 | +|---|---| +| m1 构建依赖 | N29 | +| m2 打包 | N20、N11、N30 | +| m3 命名与资源 | N9、N16 | +| m4 行数据 | N23、N26、N33 | +| m5 运行器的启用 | R1；Windows 行在 Linux 上的 wine runner 数据（X1，`enabled = false`） | +| m6 运行器模型 | R2 的字段对齐、`runner_spec`、构建程序中的 `{pkg:}`、数据中的 `longlived` | +| m7 运行器的程序规则 | R2 的查找与 Windows 脚本规则；docs/30 与 docs/41 的矛盾；N21、N27 文档 | +| m8 Android runner 数据 | Android 行提供者条目的 `runner`（dirun 与镜像包，`enabled = false`）；前置：xb 已合并。实现：x86_64 一行（`same-arch`、`exclusive`，提供者 runner 增加 `exclusive` 键）；模拟器包只服务 linux-x86_64，aarch64 行不陈述 runner | + +m1–m7 没有外部前置，立即开工，顺序按价值：N29 与 N20 先。m8 是最后一个检查点，前置在外部（dirun 0.4.0 与 +API 34 镜像已在 xim）。它不拆成另一个 PR：外部前置不妨碍 m1–m7 的开发、评审与 CI，只决定 PR 何时合并；mcpp +的 PR 里也没有任何内容要等 mcpp 自己发布。代价是 m1–m7 的修复随 PR 一起等到 xb 合并后发布，关键路径因此 +经过 mcpp（7.2）。若外部链条因 mcpp 之外的原因停滞，交由评审者决定是继续等待还是先合并 m1–m7，不自行拆分。 + +**Android 的 runner 数据为什么在 builtin.toml。** 运行器陈述的最低一级是提供者条目的 `runner`（SPEC-016 §6）； +Android 行由内置提供者（NDK）服务，没有平台包，所以它的 runner 写在该条目上，与现有的 qemu-user 条目同一 +形式（`cmd`、`hosts`、`requires`）。这不是硬编码：引擎不认识 dirun 与 Android，条目只说“用哪个包里的哪个 +程序”；客体的知识（启动参数、推送与取回的命令）全部在镜像包自带的 dirun 配置中，镜像升级不需要改 mcpp， +只有所需的最低版本变化时才改这一行数据。 + +## 6. 需要改 xlings 本体 + +- X2：替换在共享冲突上有界重试；可用时以 POSIX 语义重命名；自身读者以共享删除打开。修复后关闭 #715。 +- X5：shim 直接创建进程并按 Windows 的参数规则转交 argv，不经命令解释器；加含 `"`、`&`、`%`、`^` 的回归测试。 +- X3：先在干净环境复现；若属 xim 包数据，转为包修复（xb）。 + +## 7. 实施：PR 划分与任务依赖图 + +规则： + +1. 每个仓库尽量一个 PR，以 commit 为检查点；每个检查点自带测试且 CI 通过。外部前置（别的仓库的发布）只 + 约束某个检查点何时完成、PR 何时合并，不拆 PR。拆开只有两种情形：内容要等**本仓库自己的发布**；或仓库 + 是发布渠道（索引），无前置的内容是别人的前置，不能等到最后。 +2. 所有 PR 立即开工。依赖只约束**某个检查点**或**合并**，不约束开工。 +3. 跨仓库的契约先写设计记录、评审后实现：dirun 客体配置的格式（dirun 与镜像包之间），R2 的规范条款与 + 5.2 的三项形状（mcpp 与插件、平台、数据之间）。 +4. 引擎的新接口由使用方以 `requires` 表达式探测（docs/30），旧引擎上退回现有行为。平台包用了新键时声明 + 引擎下限（`[package] mcpp`），旧引擎自动停在上一版本。 +5. 每次发布后按发布规范补齐 GitCode 资源并更新索引；mcpp 发布后不升自身固定版本，mcpp-index 的 CI 指向最新版本。 + +### 7.1 任务 + +| PR | 仓库 | 检查点 | 验收 | 产出 | +|---|---|---|---|---| +| k1 | openkal-macos | N18 `__bzero`；CI 加 `x86_64-macos-musl --release` | 发布构建通过 | 补丁版本 | +| k2 | openkal-uefi | N35 Shell 参数协议；N15 EFI_RNG_PROTOCOL；N34 `fs`（SIMPLE_FILE_SYSTEM，根为程序所在卷） | UEFI 上读写文件；带引号参数正确；无告警 | 次版本 | +| k3 | openkal 的 hosted 层所在仓库 | N10 hosted 安排报宿主系统 | FreeBSD 上报 FreeBSD | 补丁版本 | +| k4 | openkal-image | N19 路径规范化；X4 发布附全部产物，含 freebsd x86_64/aarch64 的 okrun | `//`、`../` 路径可运行；release 有产物 | 补丁版本 | +| k5 | openkal-dos | N17 最小复现 | — | issue | +| m | mcpp | m1–m8（5.3） | 第 9 节对应格；`mcpp run --target *-android` | 一个版本 | +| p | mcpp:plugins | p1 N36（页面一步以复制的文件为输入，页面是唯一的分发物）；p2 N31（中间页面移入隐藏目录）、N32（pack 结束时打印 web 目录与打开方式）；p3 N22 文档与诊断；p4 W1 `web` runner（10.6） | 产物目录外没有可误开的页面；`mcpp run --pkg-format web` 打开页面 | 补丁版本 | +| xl | xlings | X2；X5；X3 定位 | #715 关闭；特殊字符的参数完整到达 | 版本 | +| d | dirun | d1 设计记录；d2 N25 `MCPP_RUNTIME_FILES`；d3 N13 `stderr-file`；d4 N12 发布包携带的 okrun 取自平台版本组（固定 openkal-platform 0.28.1，CI 以含 `//`、`..` 的路径启动发布包）；d5 经命令到达的客体 | 各配置的测试；Android 配置在模拟器中运行 | 0.4.0 | +| xa | xim-pkgindex | ovmf 配置改为 Shell + `startup.nsh`（N14，dirun 0.3.2 已支持，已实测）；freebsd-disk-image 的 aarch64 镜像与配置（G6）；`freebsd-vm` 退役 | 经各包运行 | 无前置，先合并 | +| xb | xim-pkgindex | okrun、dirun 新版本；dosbox-staging 的 `stderr-file`（N13）；android-system-image API 34 x86_64/arm64，包内带 dirun 配置（G7） | 经各包运行 | k4、d 发布后 | +| xc | xim-pkgindex | DJGPP GCC 14 由脚本自建（D1） | 模块工程在 DOS 行构建 | 独立：构建时间长，失败不影响 xa、xb | +| op0 | openkal-platform | 版本组补丁 0.28.1：openkal-image 0.6.4（k4）、openkal-macos 0.23.4（k1）、openkal-musl 0.29.7（k3）；不含新引擎所需的内容 | family-release 工作流；各行构建 | 0.28.1 | +| op | openkal-platform | op1 N14：runner 命令补 `--`，并固定新的 ovmf 包版本；op2 移出本轮（13.1：对齐会使 openkal 构建失败）；op3 各实现的新版本；op4 N28；op5 FreeBSD 行与 windows-musl 行的 runner（`enabled = false`），并声明引擎下限 | 第 9 节对应格 | 0.29.0 | +| ix | mcpp-index | 每次发布一个索引 PR（发布流程本身如此） | 索引 CI | 随发布 | +| v | 验证工程的 CI（fork） | v1 探针、定时与 `workflow_dispatch` 输入；v2 逐格把探针改为断言 | 第 9 节 | — | + +dirun 的 d1 把客体配置分成两个互不相干的部分，三种客体是它们的组合，而不是三种方式： + +| | 生命周期（可缺省） | 传输：控制台、文件交付、状态取回 | +|---|---|---| +| 磁盘镜像的 VM（DOS、UEFI、FreeBSD） | 启动 QEMU 或 DOSBox，结束后关机 | 串口、镜像内的文件、debug-exit 或输出文件 | +| Android | 启动模拟器，等待就绪 | 以命令表达：`adb push`、`adb shell` | +| 已在运行的远程主机 | 无 | 以命令表达：`scp`、`ssh` | + +### 7.2 依赖图 + +箭头表示“前者发布后，后者的该检查点才能完成”。没有箭头的检查点立即可做。 + +``` +k1、k3、k4 发布 ──> op0 0.28.1 ──> d4 ──> d 发布 0.4.0 ──┐ +k4 okrun 发布 ─────────────────────────────────────────────┴──> xb ──┬──> m8 ──> m 合并、发布 ──> op5 `enabled` 生效 + └──────────────────────────> op5 FreeBSD 行 runner +xa 合并 ───────────────> op1 固定新的 ovmf 版本（与 `--` 成对）；op5 freebsd aarch64 镜像 +k1 发布 ───────────────> op4 N28（macos-x86_64 的 okrun 为发布构建） +k1、k2、k3 发布 ───────> op3 各实现的新版本 +p 发布 ────────────────> （op2 移出本轮，13.1） + +每次发布 ──────────────> ix ──> v2 对应格由探针改为断言 + +立即可做、无前置：k1–k5、m1–m7、p1–p4、xl、d1–d3、d5、xa、xc、v1 +``` + +关键路径：k4 → op0 → d（0.4.0）→ xb → m8 → m 发布 → op → ix → v2。op0 与 op 分为两个 PR，理由属于第 7 节的拆分条件：op 的内容依赖 mcpp 新版本的发布，op0 的内容只依赖已发布的成员，且版本组的补丁本是平台既定的发布方式（一个成员的修正即该成员的发布加版本组的一个补丁）。图中没有环：dirun 与 xb 都不需要新的 mcpp。 +op 是汇合点，最后发布。m1–m7 在 xb 之前完成评审，xb 合并后只剩 m8 一个检查点。 + +### 7.3 不推迟的做法 + +- **W1（p4）立即做**：引擎尚无 R2 时，插件以已有接口提供 runner——`mcpp::toolchain_dir()` 定位 emsdk， + `runner_longlived("web")`，参数含 `--hostname localhost`；引擎提供 `runner_spec` 后（`requires` 表达式探测）， + 改用 `{pkg:emsdk}`。N29 发布前，工程把 mcpp:plugins 与平台对齐，不阻挡 W1。 +- **op1 与 xa 成对**：新的 ovmf 配置在旧的平台命令下仍然启动程序（只是没有参数，与现状相同），所以 xa 可以 + 先合并；op1 固定新的 ovmf 版本并补 `--`，两者一起生效。 +- **m 不空等**：m1–m7 逐个检查点评审并通过 CI；p4（W1）、op 的其余检查点都不等 m 发布（7.1、规则 4）。 +- **op 在分支上先验收**：op 发布前，验证工程的 CI 以分支依赖运行对应格（v1 的探针），不等发布。 + +## 8. 验证方式 + +先本地实测；本机缺资源的（Windows arm64、Android 模拟器的长时运行、FreeBSD aarch64）用验证工程 fork 上的 +临时 draft PR 运行 CI，结果记入本文后关闭。本轮已实测： + +| 假设 | 结果 | +|---|---| +| OVMF 无 `BOOTX64.EFI` 时进入 Shell 并带参数运行（N14） | 成立：`startup.nsh` 执行，参数到达，状态经 debug-exit 返回 | +| UEFI 上以 test.bat 验收 | 不成立：openkal-uefi 无 `fs`（新增 N34），验收改为先比对 `/c` 字符串的输出 | +| web 页面的可用性 | 从磁盘打开时取不到 `.wasm`；改用 `-sSINGLE_FILE=1` 被 N33 阻挡；工程侧的页面改为说明原因与打开方式（工程内修复） | +| DJGPP GCC 14 有现成二进制 | 不成立：须自建（D1） | +| okrun 新版有 FreeBSD 二进制 | 不成立：0.6.3 无任何产物（X4） | +| 工程构建程序（插件在其中）能提供具名长驻 runner | 成立：`--list-runners` 显示 build-program、long-lived | +| emrun 服务 web 目录并打开浏览器 | 成立：`.wasm` 以 `application/wasm` 返回；指定浏览器打开页面。默认监听 `0.0.0.0` | +| Windows arm64 批处理超时是生态问题 | 不成立：工程自身的缺陷。批处理以文本模式打开，`CALL` 以 `ftell`/`fseek` 记返回点；Windows C 运行库在文本模式下对只有 LF 的文件 `ftell` 计数错误，`EXIT /B` 回到 `CALL` 行本身。x64 未暴露，因为那里检出的是 CR LF。工程内修复（二进制模式打开），CI 改为两种行尾都运行；修复后全部作业通过 | +| `mcpp run` 在 POSIX 上怎样启动 runner | 读源码（`src/run.cppm`）：mcpp 以 runner 替换自身，runner 拥有终端、信号与退出状态；Windows 上 runner 在与 mcpp 绑定的单元中。v7 的“转发信号、宽限期、报告已结束”与此不符，撤回 | +| emrun 被 Ctrl+C 结束时的状态 | 实测：SIGINT 后退出 0 | +| 旧引擎读到 runner 表中不认识的键 | 读源码（`platform_toml.cppm`）：警告并忽略，runner 照常使用。因此带 `enabled = false` 的平台版本须声明引擎下限（`[package] mcpp`，旧引擎自动停在上一版本，docs/35） | +| 程序查找是否已按 PATHEXT | 读源码（mcpplibs.os `find_executable`）：裸名字在 Windows 上逐个补 PATHEXT 的后缀，且不找无后缀的同名文件；带目录的名字原样使用，不补后缀 | +| mcpp 2026.11.15.1 在 Windows 上启动批处理 runner | 临时 draft PR 实测（windows-2022）：裸名字 `x` 找到 `x.bat`，不取同目录无后缀的脚本；带空格的参数正确到达，退出状态原样返回。带目录而无后缀的名字报 “executable format error”，退出 126。参数含 `"` 时其后的 `&` 被命令解释器当作命令分隔执行，`%VAR%` 被展开，`^` 被吞掉。不经 runner 直接启动的程序参数完整 | +| xlings 在 Windows 上的 shim 怎样转交参数 | 同一次实测：经 shim 启动 mcpp 时，参数含 `"` 与 `&` 即被拆开并执行其后的命令；直接启动 mcpp 本体则不会（新增 X5） | + +## 9. 验收矩阵（验证工程的 CI） + +以同一批处理脚本的输出比对为准。CI 每一格先作为探针（写入摘要，不使作业失败），修复发布后改为断言；接受 +`workflow_dispatch` 的 mcpp 与平台版本输入，并每周定时运行。开启机器类运行器（R1）、传入测试脚本是工程的 +选择，写在工程的 CI 中。 + +| 目标 | 现状（2026-11-15） | 完成后 | 依赖 | +|---|---|---|---| +| Linux x86_64/arm64（gnu、musl）、i686 | 运行 | 运行 | — | +| Windows x64（gnu、musl）、arm64 | 原生机运行（arm64 于修复后经 CI 确认，LF 与 CR LF 两种行尾） | 另加：Linux 上 `mcpp run` 经 wine | X1、R1 | +| macOS arm64 | 运行 | 运行 | — | +| macOS x86_64（musl） | 调试版运行 | 发布版运行 | N18 | +| FreeBSD x86_64/aarch64 | vmactions 运行 | 另加：`mcpp run` 经 freebsd-disk-image；包的启动器直接启动 | G6、R1、N20 | +| Android x86_64/arm64 | 模拟器 action 运行（x86_64） | `mcpp run` 经 dirun 与镜像包 | G7、R1 | +| WebAssembly（node、浏览器） | 运行；pty 交互；Chrome；Pages | 另加：`-sSINGLE_FILE=1` 的页面可从磁盘打开；C++ 异常可捕获 | N33、N26 | +| `mcpp run --pkg-format web` | 拒绝（N36） | 在 Linux、macOS、Windows 上：服务页面（CI 中以 `--no-browser` 的覆盖运行），取回页面与 `.wasm`，中断后结束、无残留进程 | N36、W1、R2 | +| openkal 映像 x86_64/aarch64 | 运行 | 另加：`//`、`../` 路径 | N19、N12 | +| openkal 映像 i386 | 构建 | x86_64 机上运行 | N23 | +| UEFI x86_64 | 启动 | `/c` 字符串带参数运行并比对；之后 test.bat | N14、N35；N34、N25 | +| DOS（DJGPP） | 直接调 dirun 运行 | `mcpp run` 运行，标准错误可见 | N25、N13 | +| DOS（openkal，实验） | 启动 | N17 修复后运行 | N17 | +| 打包 | `--systems all` 的包在 Linux、FreeBSD（直接调 okrun）运行 | 每个程序有启动器；启动器在四种宿主上运行；含 macos-x86_64 的 okrun | N20、N11、N28 | +| 构建依赖 | 须与平台对齐 mcpp:plugins 版本 | 可写较新版本，或得到明确的冲突报告 | N29 | + +## 10. 运行器架构：评估、统一模型与规范 + +### 10.1 两种 runner 的区别：dirun 与 dist-web 的 `web` + +在 mcpp 的模型里两者都只是 runner：一条 argv 加几项性质，引擎不区分种类。区别全在性质上： + +| | dirun | `web`（dist-web 提供，调用 emrun） | +|---|---|---| +| 面对的对象 | 构建产物：一个程序 | 分发物：web 目录（交给 runner 的是页面） | +| 做的事 | 启动整机或模拟器，或到达一台已有的机器；把程序与文件交付进客体，运行，取回输出与状态 | 在本机起 HTTP 服务，打开浏览器，服务到 Ctrl+C | +| 结束 | 自然结束，退出码即结论（`mcpp run`、`mcpp test` 都可用） | 长驻（`longlived`），由操作者结束；状态不是结论 | +| 客体知识在哪里 | 镜像包的配置（`dirun.toml`），数据 | Emscripten 的 emrun；页面由工程模板决定 | +| 由谁声明 | 平台或内置行的提供者数据 | 插件在构建程序中，以格式命名 | +| 入口 | `mcpp run`（默认 runner） | `mcpp run --pkg-format web`（按格式找同名 runner） | +| 首次使用的代价 | 镜像、模拟器，GB 级，默认关闭（R1） | 无：emrun 在已安装的 emsdk 中 | + +两者不合并：dirun 的职责是“把程序交给另一台机器”，web runner 的职责是“把分发物呈现给人”。浏览器中的 +自动化测试（无头、取回输出与状态）需要程序以 `--emrun` 链接、加载即运行，属工程选择。 + +### 10.2 现状评估（2026.11.15.1，源码与实测） + +已经通用、本方案不改的部分： + +- 引擎不认识任何 runner 名字；`flash`、`serve`、`web` 对它都一样（docs/41）。 +- 覆盖层次：提供者条目 → 平台 → 依赖的构建程序 → 本机 `config.toml` → 工程 `[target.] runner` / + `runners` → `MCPP_TARGET__RUNNER` → `--runner`（SPEC-013 §10、SPEC-016 §6）。 +- `mcpp run --pkg-format ` 按格式找同名 runner；插件作为包，经构建程序即可提供 runner（已实测）。 +- **进程模型**（`src/run.cppm`）：POSIX 上 mcpp 以 runner 替换自身，runner 拥有终端、信号与退出状态，如同 shell + 启动它；其他系统上 runner 在与 mcpp 绑定的单元中，mcpp 以它的状态结束，结束请求到来时单元一并结束。 + 无法启动为 127；被信号结束为 128 加信号号。 +- **查找**：程序名先在 `requires` 的包、再在图中各包声明的 payload 的 `bin/`、最后在 PATH 中查找 + （SPEC-013 §10、docs/41）；Windows 上裸名字按 PATHEXT 补后缀，不找无后缀的同名文件，不找当前目录 + （mcpplibs.os）。 +- 运行协议：`{}` 为产物；`MCPP_RUNTIME_FILES` 列出随产物的文件；`exclusive`、`host`。 + +缺口： + +| 编号 | 缺口 | 影响 | +|---|---|---| +| A1 | 同一概念两种表达，字段不对称：数据有 `hosts`、`requires`、`{pkg:}`、`built-as`，构建程序没有；构建程序有 `longlived`，数据没有 | 插件不能声明依赖与适用主机；数据不能声明长驻 | +| A2 | 不在包的 `bin/` 下的程序（emrun 在 `emscripten/` 下）在构建程序中只能写绝对路径；docs/30 写“必须绝对路径”，docs/41 写“写名字，不写路径”，两者矛盾 | 路径在构建时固化；文档相互抵触 | +| A3 | Windows 上带目录的名字不补后缀；批处理型程序怎样启动、参数怎样转义没有成文的规则 | 实测：`../tools/x` 不能启动；批处理 runner 的参数含 `"`、`&`、`%`、`^` 时被命令解释器改写或当作命令执行（CVE-2024-24576 一类） | +| A4 | `{}` 指向目录型分发物时交什么未规定（N36 是其症状） | 插件各自猜 | +| A5 | 是否默认启用（R1）没有键 | 首次使用代价大的 runner 不能写成数据 | + +### 10.3 统一模型（R2） + +一个 runner 是一组字段，数据、构建程序与工程用同样的名字： + +| 字段 | 含义 | 数据（toml） | 构建程序 | +|---|---|---|---| +| 名字 | 缺省为默认 runner；否则为具名 | `runner` / `runners.` | `runner(tok)` / `runner(name, tok)` | +| `cmd` | argv；`{}` 产物或分发物，`{sysroot}`，`{pkg:}` | 已有 | 补：模板在构建程序中同样可用 | +| `longlived` | 无自然结束；状态不是结论 | 补 | 已有 `runner_longlived` | +| `exclusive` | 运行不可重叠 | 已有 | 已有 `run_exclusive` | +| `host` | `same-arch` 或 `any` | 已有 | 已有 `run_host` | +| `hosts` | runner 存在于哪些主机（`-`） | 已有 | 补 | +| `requires` | 所需的包，首次运行时安装，构建不安装 | 已有 | 补 | +| `enabled` | 是否启用（R1） | 补 | 补 | + +默认 runner 与具名 runner 是同一种表：`runner = { ... }` 与 `runners. = { ... }` 形状相同，数组形式是 +只写 `cmd` 的简写。`cmd`、`requires`、`enabled` 是一条陈述，随 `cmd` 一起被替换；`host`、`exclusive` 各自独立 +替换（现有规则）。 + +构建程序一侧用一个类型与表一一对应，写法与已有的 `mcpp::action` 相同（字段、链式添加、`submit()`）；旧的 +逐词调用保持有效： + +```cpp +mcpp::runner_spec web; +web.name = "web"; // 空：默认 runner +web.longlived = true; +web.arg("{pkg:emsdk}/emscripten/emrun").arg("{}") + .require("xim:emsdk"); +web.submit(); +``` + +| 表的键 | `runner_spec` | 说明 | +|---|---|---| +| `cmd` | `.arg(tok)` | 与 `action.arg()` 相同 | +| `longlived`、`exclusive`、`enabled` | 同名 `bool` 字段 | `enabled` 缺省为真 | +| `host` | 同名字段（`"same-arch"`、`"any"`） | 已发布的键，不改名 | +| `hosts` | `.hosts(h)` | 每次添加一项，与键同名 | +| `requires` | `.require(pkg)` | `requires` 是 C++ 的关键字，不能作成员名；取最近的拼写 | + +命名的规则：方法与表的键同名；键是 C++ 关键字时取最近的拼写。类型名取 `runner_spec`：`mcpp::runner` 已是 +函数名；`spec` 是“声明式描述”的通行叫法（Kubernetes 的 `PodSpec`）。数据中的键写作 `longlived`，沿用已发布的 +`runner_longlived`，不另拼 `long_lived`。 + +### 10.4 规范（R2） + +1. **查找。** 沿用现有顺序（`requires` 的包 → 图中 payload 的 `bin/` → PATH）。新增两点：`{pkg:}` 在构建 + 程序中可用，用于不在 `bin/` 下的程序；Windows 上带目录而无 PATHEXT 后缀的名字同样补后缀，规则与裸名字 + 相同（不取无后缀的同名文件）。任何情况下不查找当前目录。 +2. **执行。** argv 不经 shell。Windows 上的批处理（`.bat`、`.cmd`）由命令解释器启动，这是系统的规定；引擎按命令解释器的规则转义每个参数（引号、`%`、`^`、`&` 等），不能安全转义的参数拒绝执行并说明原因（Rust 标准库对 CVE-2024-24576 的处理）。实测现状不转义。 +3. **占位符。** 只替换已知的占位符（`{}`、`{sysroot}`、`{pkg:}`），可以出现在一个词的中间 + （`{}/index.html`、`{pkg:x}/bin/y`），其余花括号原样保留。没有 `{}` 时产物附在末尾（已有规则）。同一名字的 + runner 只能用一种方式声明：`runner_spec` 与逐词调用混用于同一名字时报错。 +4. **进程与状态。** 沿用现有模型（10.2），不新增信号转发与宽限期。runner 的退出状态就是 `mcpp run` 的状态。 + `longlived` 的含义是“状态不是结论”：`mcpp test` 不使用长驻 runner，机器输出标明 `long-lived`，由读者决定 + 如何对待它的状态。实测 emrun 被 Ctrl+C 结束时退出 0。 +5. **分发物。** `{}` 为格式唯一的终端产物；目录型分发物的入口由提供者以 action 的输入输出表达（N36）。 +6. **文件。** 搬动产物的 runner 复制 `MCPP_RUNTIME_FILES` 所列文件（已有条款，dirun 实现 N25）。 +7. **主机名。** `hosts`、执行关系（N23）与打包启动器（N20）都用 `-`，系统名取自系统表。 + +验证：第 1、2 条在 Linux、macOS、Windows 上各有 e2e；`mcpp run --pkg-format web` 一格在三种系统上验证长驻 +runner 的启动与结束（第 9 节）。 + +### 10.5 与行业习惯的对照 + +| mcpp | 同类工具 | 判断 | +|---|---|---| +| `runner`、`MCPP_TARGET__RUNNER` | Cargo `[target.] runner`、`CARGO_TARGET__RUNNER`；CMake `CROSSCOMPILING_EMULATOR`；Meson `exe_wrapper`；Bazel `--run_under` | 与 Cargo 同名同形，保留 | +| mcpp 以 runner 替换自身 | `cargo run` 在 Unix 上同样以 exec 交出终端 | 保留：状态、信号与 shell 启动时一致 | +| 具名 runner（`flash`、`web`） | npm scripts、just 的 recipe | “以格式命名、`--pkg-format` 自动找到”是 mcpp 独有的 | +| `exclusive` | Bazel `tags = ["exclusive"]` | 同名同义 | +| `enabled` | Gradle 任务的 `enabled` | 采用 | +| `MCPP_TARGET__RUNNER_ENABLED` | Cargo 以 `CARGO_TARGET__` 覆盖任一键 | 同一构词 | +| `requires` | mcpp 各处已用 | 同一概念同一拼写 | +| `{}` | `find -exec {}`、`xargs -I{}` | 沿用 | +| 127、128 加信号号 | POSIX shell 的约定 | 已有 | + +已知的不理想，按“已发布的键不改名，历史不一致写成规则”处理：`host`（与产物架构的关系）与 `hosts`（runner +存在于哪些主机）只差一个字母，文档并列说明；`cmd` 只在表中出现，构建程序中对应 `.arg()`。 + +### 10.6 插件接口（dist-web） + +`mcpp::dist::web::options` 增加 `runner`（缺省开启，提供名为 `web` 的长驻 runner）。runner 只监听本机 +（`--hostname localhost`）：emrun 默认监听 `0.0.0.0`，开发服务器不应默认暴露给局域网。浏览器、端口等是本机 +偏好，不进插件选项，用户以已有的覆盖层次替换整条命令（`config.toml` 的 +`[target.wasm32-emscripten.runners] web = ["{pkg:emsdk}/emscripten/emrun", "--port", "8080", "{}"]`）。 +其他 `dist-*` 成员按同一形状提供以格式命名的 runner，由各自决定，不在本方案范围。 + +不做：让 profile 携带 target 与 pkg-format 以缩短命令。命令由 README 给出完整形式。 + +## 11. 决策记录 + +| 序 | 决策 | +|---|---| +| 1 | 第 5 节本体项作为 mcpp 下一版本范围；N9 统一应用 B2；N16 判据改为执行系统；N26 含 C++ 时默认 wasm 异常；N21、N22 只做文档与诊断；以验证工程作金丝雀，定时加按需；32 位新行维持现状 | +| 2 | xim-pkgindex 的生态工具一并补全；验证工程 CI 的全矩阵为整体验收 | +| 3 | Android 运行器在 dirun 中，形式为“经命令到达的客体”；首次使用代价大的运行器默认关闭、提示完善、可在 mcpp.toml 配置（R1）；这类运行器这一轮只保证 Linux 主机（程序规则与 `web` runner 覆盖三种系统）；无前置的先做 | +| 4 | 基础设施只提供机制、数据与扩展点；工程特有的配置留在工程（第 2 节） | +| 5 | W1：dist-web 提供 `web` runner（emrun），只监听本机；不让 profile 携带 target 与格式；运行器的统一模型、插件接口与规范一并设计（R2） | +| 6 | R2 的构建程序接口用一个类型（`runner_spec`）与表一一对应 | +| 7 | 能放在一个 PR 的放在一个 PR，以 commit 为检查点；不能的拆开；任务按依赖图并行，尽量不推迟 | +| 8 | 需要验证的假设用临时 draft PR 实测（本轮：Windows 上的批处理 runner，结果见第 8 节，PR 已关闭） | +| 9 | 撤回此前两项：长驻 runner 结束时由 mcpp 报告“已结束”并退出 0；固定 10 秒宽限期。原因：mcpp 在 POSIX 上以 runner 替换自身，没有可以转发、等待或报告的 mcpp；改为 10.4 第 4 条 | +| 10 | R1 的一次性覆盖用环境变量 `MCPP_TARGET__RUNNER_ENABLED` | +| 11 | Android 的 runner 数据留在 builtin.toml 的提供者条目，是 mcpp 同一个 PR 的最后一个检查点（m8）。mcpp 侧统一为单 PR：只有依赖 mcpp 自己发布的内容才拆；不阻塞开发的外部依赖不拆 | +| 12 | mcpp#932、#936、#938 并入本 PR（m9），按设计规范取最佳方案，不取 issue 的临时建议（13.4）；N39 待评审 | + +## 12. 被否定的方案与自我评审 + +### 12.1 被否定的方案 + +| 方案 | 否定的原因 | +|---|---| +| 引擎为数据文件新增通道（N25） | 运行协议已有 `MCPP_RUNTIME_FILES`；缺的是 dirun 的实现 | +| dirun 增加一种 “android 方式” | 是为一个系统写的特例；改为生命周期与传输两部分的组合 | +| 把“x86_64 可执行 i386”写进引擎；或只按架构写成数据 | 关系随系统而变（macOS 不成立）；写成按 `-` 的数据 | +| N26 增加一个可关闭的开关 | 参数是行数据，工程以已有 flags 覆盖 | +| dirun 自带同版本的 okrun | 改为声明对 okrun 包的依赖，副本与版本漂移一起消失 | +| web runner 并入 dirun | 职责不同：交给另一台机器，与呈现给人 | +| 浏览器、端口作为插件选项 | 属本机偏好，走已有的覆盖层次 | +| `runner_spec` 用指定初始化；方法名 `.requires()` | 前者与 `mcpp::action` 的写法不同；后者是关键字，不能编译 | +| `enabled` 像 `host`、`exclusive` 那样被只写 argv 的覆盖继承 | 已写 `runner = ["wine", "{}"]` 的工程升级后会被拒绝 | +| 为默认 runner 的一次性开启新增命令行开关 | 已有 `MCPP_TARGET__` 的构词，不增加词汇 | +| Android 的 runner 数据随镜像包提供 | 运行器陈述没有“包”这一级；内置行的最低一级是提供者条目，与 qemu-user 相同 | +| mcpp 转发信号、设宽限期、为长驻 runner 改写退出状态（v7） | 与现有进程模型冲突，且会让交互式程序读不到终端；现有模型已满足需要 | +| 平台包直接发布带 `enabled = false` 的 runner，由旧引擎自行处理 | 旧引擎警告后照常使用该 runner，会在首次运行时下载镜像；改为声明引擎下限 | +| xim-pkgindex、mcpp-index 各一个 PR | 索引是发布渠道：无前置的内容不应等关键路径末端的内容 | +| Android 的 runner 数据拆成 mcpp 的第二个 PR（v8） | 外部前置不妨碍其余检查点的开发与评审，只决定合并时机；拆 PR 只为“等本仓库自己的发布” | +| Windows arm64 的超时归为生态问题 | 逐命令定位后是工程自身的缺陷（文本模式下的 `ftell`），在工程内修复 | + +### 12.2 自我评审（v9） + +| 角度 | 检查 | 结论 | +|---|---|---| +| 架构 | 引擎是否出现系统名、工具名、运行器名 | 没有：wine、dirun、emrun、Android 都只在数据、包或插件中 | +| 架构 | 新增的引擎概念 | 四个：`enabled`、`runner_spec`（已有字段的另一种写法）、执行关系表、可选的附带产物；各有数据形状（5.1、5.2、10.3） | +| 规范 | 每个缺省值是否是数据并有四层覆盖 | `enabled` 四层齐全；其余新键是行数据或提供者数据，沿用现有层次 | +| 规范 | “少做不沉默” | 未开启的 runner 拒绝并给出原因、所需的包、两种开启写法与 reason token | +| 通用 | 模型能否覆盖已知的各类 runner | 用户态模拟、解释器、VM 与模拟器、设备、远程主机、呈现、调试服务器：都只用这组字段 | +| 复用 | 是否重复已有机制 | v7 重复了进程模型与 PATHEXT 查找；v8 改为引用现有实现，只补缺口 | +| 语义 | 名字是否一词一义 | 任务编号与问题编号不再重名；`requires`/`.require` 的差异写成规则 | +| 兼容 | 升级后现有工程的行为 | 不变：自写 runner 的工程不受 `enabled` 影响；旧引擎停在旧的平台版本；N9、N16、N33 的产物变化写入 CHANGELOG | +| 简洁 | 文档 | 第 5 节与任务表合并；修改历史压缩为 12.1 | +| 依赖 | 是否有可以提前的任务被后序任务拖住 | xa 与 xb 拆开；op 是汇合点，分支上先验收。mcpp 单 PR 的代价是 m1–m7 的发布等 xb：已写明，并给出停滞时的处理 | +| 依赖 | 单 PR 后是否成环 | 没有：d、xb 不需要新的 mcpp；依赖 m 发布的只有 op5 与 p4 的接口切换，后者经探测自动启用 | +| 未决 | 尚无实测支持的条款 | 10.4 第 2 条的转义算法、`{pkg:}` 在 Windows 上补后缀：实现时在 Windows 的 e2e 中验证；长驻 runner 在 Windows 上被 Ctrl+C 结束后有无残留进程：CI 无法发出控制台事件，m7 中以库的单元测试覆盖 | + +## 13. 实施中的新发现（待评审） + +### 13.1 N37：构建依赖与目标图共用一次版本解析 + +**现象（实测，2026-11-15）。** 工程以 `[build-dependencies.mcpp] plugins = "0.21.0"` 构建 `x86_64-openkal`。 +平台 openkal-platform 0.28.0 的插件请求 `plugins 0.20.0`。N29 的拒绝给出这两个请求者；工程以 +`override = true` 选定 0.21.0 后，构建仍然失败：`mcpplibs.openkal` 由 `mcpplibs.os@0.4.0` 请求 0.24.0，由 +`openkal-musl@0.29.6` 请求 0.25.3。plugins 0.21.0 起 `plugins-core` 依赖 mcpplibs.os，而 os 0.4.x 固定在 +openkal 0.24。 + +**原因。** 构建程序的依赖（主机图，mcpp#797）与目标的依赖在同一次解析中按 `(namespace, name)` 合并。 +主机的子构建从这次解析固定版本（`host_graph.cpp`），因此主机侧的 openkal 与目标侧的 openkal 被当作一个包。 +两者编译给不同的机器，本不需要一致。 + +**影响。** +- op2（平台的插件依赖对齐到 0.22.0）会使所有 openkal 构建失败。本轮 op 保持平台的插件依赖为 0.20.0 + （其 `plugins-core` 不依赖 os），op2 移出本轮，等 13.1 定稿。 +- 工程同时使用 plugins ≥ 0.21 与 openkal 目标时，只能把插件依赖写在只匹配其他目标的 + `[target..build-dependencies]` 下。 + +**建议的设计。** 只经构建依赖到达的包（`buildOnly`）以独立的身份解析：`ResolvedKey` 增加一个“图”维度， +与已有的 `variant` 维度同一方式（两个补丁集已是两个身份）。主机图内部仍按 N29 统一或拒绝；主机图与目标图之间 +不合并、不冲突。主机的子构建取主机身份的版本。锁文件为主机身份另记一节。这是引擎的解析模型变化，需要 +评审后实现，不在本轮的 mcpp PR 中。 + +### 13.2 N38：内置 profile 的部分覆盖整体替换（待评审） + +**现象（实测，2026-11-16，2026.11.15.1 与本分支相同）。** 工程只写 `[profile.dev] ldflags = [...]`，构建报告 +`dev [optimized]`，编译取 `-O2`、无 `-g`：内置 dev 的 `opt = 0`、`debug = true` 不再生效。 + +**现状。** docs/04 §2.9 写明 `[profile.]` 整体替换内置定义，行为与文档一致。Cargo 的 `[profile.dev]` +按键覆盖，未写的键取内置值；只为加一个链接标志而写 profile 的工程因此失去调试构建，且只能从状态行察觉。 + +**建议的设计。** 内置名字的 profile 按键覆盖：未写的键取内置值，与 Cargo 一致；自定义 profile 维持现状（未写的键取 +引擎缺省）。这是已发布行为的改变：依赖“整体替换”的工程需补写键。升级时对写了内置名字、且未写 `opt` 或 `debug` 的 +profile 给出一次提示。需要评审后实现，不在本轮。 + +### 13.3 本轮已实现的范围变化 + +- **N29（m1）** 实现为：构建依赖的两个请求无法统一时拒绝，列出两个请求者、合并失败的原因与 `override` 写法。 + 此前的路径尝试重命名模块，在没有源文件的数据包（平台）上以无关的消息失败。 +- **N30 的插件侧**（dist-web 调用 `mcpp::pack_program`）需要协议 19，随 mcpp 发布后的插件版本采用； + 插件以 `mcpp::warning` 陈述不提交的原因，这一部分不需要新引擎，在 p 中完成。 +- **op0** 见 7.1：版本组补丁 0.28.1 先于 op 发布，dirun 0.4.0 经它取得 okrun 0.6.4。 +- **N27** 实测结论与原记录不同：node 下 Emscripten 对终端的同步读取以 `EIO` 失败（是否调用 `tcsetattr` 相同）， + 管道与文件可读；文档按实测写出（docs/24）。 +- **op（openkal-platform 0.29.0）** 实现的范围： + - op1：UEFI runner 补 `--`，固定 ovmf 9.2.4-2 与 dirun 0.4.0。 + - op3：openkal-uefi 0.11.0。openkal-musl 0.29.7 请求 `^0.10.0`，因此先发 openkal-musl 0.29.8（`>=0.10.0, <0.12.0`）。 + - op4：okrun runner 的宿主加入 `macos-x86_64` 与 `linux-i686`（后者 `built-as = i686-linux-musl`），`--systems all` + 的包实测含两者。 + - op5：按 R1 的规则，虚拟机与机器模拟器的 runner 一律 `enabled = false`，包括 UEFI（QEMU 与 OVMF）与 DOS（DOSBox）， + 不只 FreeBSD 与 windows-musl。hosted 行的 runner 以完整的 `[[platform.provider]]` 陈述：`hosted = true`， + `base = "none"`，toolchain 与 layers 取 `[platform.hosted]` 的值。实测三行的构建图与不加条目时逐字节相同。只写 runner + 的条目会取 ok 映像的约定，所以不能只写 runner。 + - 本项留作评审（N40）：`[platform.hosted]` 若能按行陈述 runner，即可去掉 layers 的重复。这需要改 SPEC-013 §1 的键表， + 不在本轮。 +- **N17（k5）** 定位与原记录不同：批处理静默结束不是 `if … goto` 所致，而是之后的目录列举超出 openkal-dos 的 64 KiB + 首个栈，栈溢出不产生故障而挂起，未写出的输出随之丢失。最小复现（8 KiB 栈帧递归 8 层即挂起）见 mcpplibs/openkal-dos#3， + 修复属 openkal-dos，本轮不实现。 +- **G7（m8、xb）** 实测：API 34 镜像的 system.img 是 GPT 磁盘（vbmeta 与动态分区 super），qemu-user 的根仍只属 API 24 + arm64-v8a；模拟器 37.1.11 以 `-sysdir` 启动时需要一个含 `platform-tools` 的 SDK 根，否则在读取 `-sysdir` 后以 SIGSEGV + 结束，启动脚本给出空的 SDK 根。x86_64 镜像经已安装的配置与 dirun 0.4.0 运行动态程序，19 秒得到状态，参数与标准错误完整， + 运行后不留模拟器与设备目录。 +- **N21** 实测顺序为 `[build]`、profile、`[target..build]`，`cflags`、`cxxflags`、`ldflags` 相同（docs/04）。 +- **R2 的 Windows 脚本规则（与 10.4 第 2 条的差异，2026-11-16 评审确认）** 实现为拒绝含命令解释器语法的参数（系统表 + `command-scripts` 指出哪些后缀经解释器启动），不做转义。转义要求引擎以 `one_string` 自行构造 + `cmd.exe /d /s /c` 的整条命令行（Rust 的做法），其正确性只能在 Windows 上实测；拒绝的集合包含可以安全转义的字符 + （引号内的 `&`、`|`），比 10.4 更严，但满足“不能安全传递的参数不执行”。评审确认以拒绝为本轮的规则，不补做转义。 + +### 13.4 追加：mcpp#932、#936、#938（2026-11-16） + +三个问题都是通用缺陷，与具体工程无关，都在本 PR 实现。下面逐条写出：哪些部分排除了，采用的设计，以及没有采用 issue +原建议的理由。 + +**#938 按确切路径列出的源文件不存在时静默跳过。** +- 设计：`sources` 中不含模式字符（`*?[{`）的条目指名一个文件；文件不存在时，扫描以错误拒绝，并指出陈述它的表。 + - 表按 `[build]`、`[target.''.build]`、`[features.]` 的顺序查找，取自条件合并前的清单 + （`beforeConditionalMerge`）。 + - 扫描器导出 `unnamed_sources(root, manifest)`，只陈述条目，不决定严重性。 + - 严重性由准备阶段第 11 步按包所在的树决定，复用 `state.editablePackage`（与依赖 lint 的划线相同）： + 可编辑的树（根、workspace 成员、`path` 包）报错；取入存储的包报警告。 + - 理由：已发布包的 tarball 不可更改，使用者无从修复。若新引擎拒绝它，此前能构建的发布就会被拒绝。 + 警告仍然说明随后导入失败的原因。 +- 不在检查范围内： + - 构建会写出的文件，即 action 的输出与声明的单元； + - 绝对路径条目，它们来自构建程序； + - 匹配为空的模式，因为一个目标上的选择可以为空。 + +**#932 `mcpp test --guide` 在 workspace 根先规划并构建全部成员。** +- 原因：guide 读取的测试框架是包清单中的声明数据，此前却从测试计划中读取，计划需要工具链、构建程序、扫描和构建锁。 +- 设计： + - 新增 `resolve_graph`，执行与 `prepare_build` 相同的阶段（清单与 workspace、工具链请求、图解析），在图之后停止。 + - 框架的提取抽成 `collect_test_frameworks`，由计划与 guide 共用。 + - guide 在任何位置都不规划、不构建，也不取构建锁：单个包、单个成员、workspace 根都一样。 + - 选中多个成员时，按配置组解析，合并各组的框架。 +- 未采用 issue 的“工作区根打印通用指南”：成员的框架可以从解析图中得到，通用指南会丢掉这一信息。 + +**#936 workspace 测试中，后一个成员的构建重编了已是最新的依赖模块（Windows 上写入失败）。** +- 根因（在 Linux 上复现）：成员的程序（L5，`MCPP_BIN_`）由每个成员各自规划。 + - 该规划只按这一个成员解析共享包的 feature，而测试规划按组内成员的合并结果解析。 + - 两个规划写入同一目录的同一路径，所以一个未改动的模块在一次运行中编译 3 次，每次再运行时又编译 3 次。 + - Windows 不允许覆盖已被映射的文件，这次重写因此失败。 +- 设计：组内成员的程序作为一次规划，与 `mcpp build --workspace` 相同，所以共享包的配置与测试构建一致。 + - 只有构建失败时，才对单个成员重新规划，以指出是谁的程序失败。 + - 实测：首次运行编译 1 次，再次运行 0 次。 +- 未采用 issue 的“写临时文件再改名”： + - 输出由编译器写出，不经过 mcpp； + - Windows 上改名覆盖一个被映射的文件同样失败； + - 根因是不必要的重写，消除重写后，这条路径不再出现。 + +### 13.5 N39：同一包的不同配置共用输出路径（待评审） + +#936 修复后，仍有两种情形使同一个包在一个输出目录中以不同配置交替编译： +- 测试的 dev-dependency 给共享包加了 feature，此时程序规划与测试规划不同； +- `mcpp build -p a` 与 `mcpp build --workspace` 交替执行。 + +在 Linux 上，这两种情形只是重编；在 Windows 上，如果另一个进程映射着旧的 BMI，写入会失败。 + +建议的设计：包的对象与 BMI 路径带上该包的配置身份（feature、标志的摘要），使不同配置并存、互不覆盖，配合 +`mcpp clean --stale` 回收。代价是磁盘占用，这与 #874 的“每个配置一个目录”相互权衡，需要评审后实现，不在本轮。 + +### 13.6 链接没有并发上限（CI 中定位，已修复） + +- 现象：本 PR 的 `unit-linux-archive`（构建全部测试并归档）在每个提交上都失败。runner 被关闭，或与服务端失去通信，没有上传日志。main 上同一作业通过。 + - 先前误判为基础设施故障。按“红色作业先读日志”的规则比对 main 的历史后，确认失败与本 PR 有关。 +- 根因：GCC 且 `bmi_schedule = "on"` 时，ninja 的 `-j` 是编译上限的 6 倍，世代的信号量只约束编译器，链接不受约束。 + - 本 PR 改动了几乎所有模块都导入的 `toolchain-model` 构建程序，因此 26 个测试程序在同一时刻就绪并同时链接。每个 `ld` 占 1–1.8 GB，16 GB 的 runner 内存耗尽。 + - main 上的增量运行没有重链，所以没有暴露。 +- 设计：链接取同一信号量的许可（`$mcpp __compiler-cap --sem .mcpp-sched --output $out -- `），编译与链接合计不超过本机的并发数。 + - 图中仍不写数字，与作业数无关（e2e 861 的规则）；归档只复制成员，不取许可。 + - 许可包住整个发布：被持有的命令结束时，产物已经就位。 + - 世代的事务以命令结束时的输出身份判定提交；若许可只包住链接器，其后的 `mv` 改变了输出，恢复时该输出被判为未提交而删除（实测：构建程序链接后消失）。 + - 因此 POSIX 的临时名发布由 shell 序列（`t=… && … && mv`）改为一个命令：`mcpp link-output --temporary` 以 `/.` 链接（保留扩展名，N9），成功后改名覆盖产物，失败时删除临时文件。 + - 至此每个主机的链接都经 `link-output` 发布（Windows `--aside`，伴随文件 `--companions`，POSIX `--temporary`）；原地链接的目标不经发布。 + - 未采用 ninja `pool`：pool 的深度要写进 `build.ninja`，图就随作业数变化。 +- CI：测试由固定的 bootstrap（2026.11.13.1）构建，它没有这项修复，而发布后不提升自身固定版本。 + - `ci-unit.yml` 按 bootstrap 的版本判断：早于 2026.11.16.1 时以 `MCPP_BMI_SCHEDULE=off` 构建归档，此时 ninja 的作业数（核数 + 2）约束链接；固定版本前移后自动恢复分离调度。 +- 测试：单元 `NinjaBackend/LinksHoldAPermitOfTheScheduleBound`；e2e 938（重链不暴露半个文件）、861（图与作业数无关）。 + - 实测（从 main 的测试构建恢复，`MCPP_JOBS=4`）：bootstrap 同时运行的 `ld` 最多 7 个，超过上限 4，用时 539 s；本分支最多 4 个，用时 360 s。 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index c1290c3a4..dda05bc7b 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -362 records. +363 records. ## By subject @@ -52,6 +52,7 @@ Records that declare one. Everything else is listed by date below. ### cross-platform +- [生态缺口补充方案：一个 C 程序走遍全部目标所暴露的通用问题](2026-11-15-ecosystem-gap-supplement-design.md) — active - [生态方案：跨平台通用基础设施（mcpp + openkal + xim），以一个 C89 工程为验证](2026-11-10-cross-platform-ecosystem-design.md) — active ### design @@ -201,6 +202,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-11 +- [生态缺口补充方案：一个 C 程序走遍全部目标所暴露的通用问题](2026-11-15-ecosystem-gap-supplement-design.md) — active - [发布的发布面与 macOS 链接的 SDK 契约（2026-11-14 补充）](2026-11-14-release-publication-and-macos-sdk-design.md) — active - [CI 与发布的时间成本：按测量调度的测试、分片、并行发布](2026-11-14-ci-and-release-time-cost-design.md) — active - [The three-plane platform architecture: mcpp × openkal × xim](2026-11-12-three-plane-platform-architecture-design.md) — superseded by [2026-11-12-ecosystem-architecture-optimization-design.md](2026-11-12-ecosystem-architecture-optimization-design.md) diff --git a/.github/workflows/ci-unit.yml b/.github/workflows/ci-unit.yml index 2764951b7..a635bb374 100644 --- a/.github/workflows/ci-unit.yml +++ b/.github/workflows/ci-unit.yml @@ -133,6 +133,17 @@ jobs: set -euo pipefail export MCPP_VENDORED_XLINGS="$XLINGS_BIN" mkdir -p "$RUNNER_TEMP/unit" + # LINKS BOUNDED BY THE BOOTSTRAP'S SCHEDULE. Before 2026.11.16.1 a link + # under the detached schedule (bmi_schedule = "on") took no permit, and + # ninja ran six per compiler: a change every test links relinked the 26 + # tests at once, at 1 to 1.8 GB each, and the runner ran out of memory. + # Such a bootstrap builds the archive with the ordinary schedule, whose + # job count bounds links; a later one holds a permit for each link. + v=$("$MCPP" --version 2>/dev/null | awk '{print $2; exit}') + if [ "$(printf '%s\n' 2026.11.16.1 "$v" | sort -V | head -1)" != 2026.11.16.1 ]; then + export MCPP_BMI_SCHEDULE=off + echo "the bootstrap $v holds no permit for a link: the archive is built with bmi_schedule off" + fi # shellcheck disable=SC2086 -- the flag is one word or none # The profile `ci` (mcpp.toml): -O0 and no debug information. The # default profile of this package is release (-O2); the tests built diff --git a/CHANGELOG.md b/CHANGELOG.md index 0979fea39..2933d67cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,132 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [Unreleased] + +## [2026.11.16.1] - 2026-11-16 + +### Added + +- `x86_64-linux-android` runs in the Android Emulator on an x86_64 Linux + machine, once enabled: dirun (>= 0.4.0) boots the API 34 system image + through the image package's profile, and the target's runs do not overlap. + A provider's runner states `exclusive` (G7). +- `mcpp::runner_spec` states a runner in a build program as the table of + data: `.arg()`, `.require()`, `.hosts()`, and `host`, `longlived`, + `exclusive`, `enabled` (protocol 19). `{pkg:}` names a program outside + a package's `bin/`, installed by the first run. A name stated by + `runner_spec` and by the token calls is refused (R2). +- A runner states `enabled = false` when its first use fetches a whole machine + image or an emulator, or leaves state outside the build tree and the package + store. `mcpp run` refuses it with status 2 and `mcpp test` leaves the tests + built and not run, naming `[runner-not-enabled]`, the packages its first use + installs and the ways of enabling it; nothing is installed first. A table + without `cmd` in `config.toml` or the project enables it (`runner = { enabled + = true }`), `MCPP_TARGET__RUNNER_ENABLED=true` enables it once, and + `mcpp run --runner ` enables the runner it names. A runner whose `cmd` + a nearer tier replaces is enabled unless stated (R1). +- The Windows rows on an x86_64 Linux machine state Wine as their runner + (`xim:wine@11.0`, `host = "same-arch"`), not enabled by default (X1). +- A named runner takes the table of the default runner (`runners. = { + cmd = [...], longlived = true, enabled = false }`) and merges by the same + rule; `longlived` states in data what `runner_longlived` states in a build + program. A provider's runner states `host` and `enabled` (R2). +- The execution table (`[[platform.executes]]`, SPEC-016 §2.4): a machine runs + the programs of another architecture where it is stated, `linux-x86_64` + executing `i686`. There a `same-arch` runner is built for the artifact's + architecture and started on this machine, and a launcher whose package lacks + this machine's architecture takes one the machine executes: an i386 image + runs on x86_64 Linux through the i686 runner (design 2026-11-15 N23). +- A row's `companions` (`.wasm` on wasm32-emscripten): removed before each + link, staged when the link wrote one, not an error when absent. A clean pack + with `-sSINGLE_FILE=1` failed, naming the missing module, and an earlier + link's module was packed beside a launcher that embeds its own (N33). + +- `mcpp::pack_program("")` (build-program protocol 19): a pack-format + provider states the program of the package being packed that it assembles + from, and `mcpp pack --pkg-format ` with no program named packs that + program. A program named on the command line outranks it; providers that + state different programs, or a program the build does not have, are refused + by name (design 2026-11-15 N30). +- A pack whose format provider submitted nothing lists, in the refusal, the + warnings the provider stated in that pass (`mcpp::warning`, `mcpp::report`) + as its reasons (N30). + +### Changed + +- A wasm32-emscripten build whose project states no `[abi] exceptions` uses + WebAssembly exception handling, the row's `exceptions = "wasm"`: a C++ + exception is caught, and `setjmp`/`longjmp` follow the same model. The SDK's + default left catching off. `exceptions = "js"`, `true` or `false` states + another model or none (N26). +- A package that carries its runner (`--mode self-contained`) starts only the + runner it carries for this machine, and a machine it holds none for exits + with status 126, naming the machine and the systems the package holds. The + POSIX and the Windows launcher fell back to a runner on `PATH`, which ran + another release's runner (design 2026-11-15 N20). +- The POSIX launcher names the system by the system table's `uname` column, so + a FreeBSD machine finds `libexec//freebsd-/`; it looked for + `FreeBSD-` (N20). + +### Fixed + +- A single file named for a system where the platform's runner does not exist + (`--systems macos` for an x86_64 image whose platform names no + `macos-x86_64` runner host) is refused before the build, by the reading + `--systems all` applies to leave it out; it was refused after the build. +- `mcpp test --guide` reads the frameworks from the resolved graph and plans, + scans and builds nothing; at a workspace root it names the frameworks of + every member. It planned and built the whole workspace first (#932). +- `mcpp test` over several members plans a configuration group's programs + together, as `mcpp build --workspace` does. Each member's programs were + planned alone, with that member's features for the packages it shares, and + rebuilt a shared module into the group's directory: three compiles of one + unchanged module in a run, and on Windows a failed rewrite of a module + another process had mapped (#936). +- Under `bmi_schedule = "on"` with GCC a link holds a permit of the build's + compiler limit. Ninja runs six jobs per compiler there and only compilers + were bounded, so links that became ready together all ran at once: mcpp's + 26 test programs relinked together at 1 to 1.8 GB each and a 16 GB machine + ran out of memory. A POSIX link is published through a temporary name by + `mcpp link-output --temporary`, one command, so the permit is held until the + artifact is in place. +- A `sources` entry without a pattern that names no existing file is an error + naming its table, and a warning in a package fetched into a store. It was + skipped, and the build failed at an importer of its module (#938). +- On Windows a runner named with a directory and no suffix (`../tools/x`) + takes the system's suffixes as a bare name does; it could not be started. + A `.bat` or `.cmd` program, which the system starts through its command + interpreter (the system table's `command-scripts`), is refused an argument + holding `%`, `^`, `&`, `|`, `<`, `>`, `"` or a line end, which the + interpreter rewrote or ran as a command (R2). +- A name a build program, this machine or the project states for a program + (`mcpp::artifact_name("exe", "{name}")`) is followed on a built-in row as on + a platform's: a program named `cmd.exe` was `cmd.exe.exe` on + x86_64-windows-gnu and `cmd.exe` on x86_64-windows-musl (design 2026-11-15 + N9). +- A linked program is written under a temporary name that keeps its extension + (`/.`, then renamed). DJGPP's driver names a second file after the + output's extension, so `.tmp` left two programs in `bin/` on + i586-dos-djgpp (N9). +- The carrier of an image whose row the built-in table keeps planned and the + project's platform lists with its toolchain is built (an i686 image's + i686-linux-musl runner); it was refused as a row this mcpp builds no program + for (N23). +- `[resources]` and the application manifest are compiled only for a PE row + whose execution system reads them, the system table's `resources = true` + (the built-in `windows` entry); the UEFI row, a PE image the firmware + loads, failed the build with them (N16). +- Two `[build-dependencies]` requests of one package that no version + satisfies are refused as such, naming both requesters, the merge's reason and + the `override` line. The engine tried to rename the second copy's modules, + which does not separate a build program's modules, and failed on the + requester's sources with a message about staging (a project asking for + `mcpp:plugins` 0.21.0 over a platform asking for 0.20.0: `stage: no source + files found under ''`) (design 2026-11-15 N29). +- `mcpp pack --all-programs` of an ok image gives every program its own pair of + launchers, `bin/` and `bin/.exe`; only the chosen program had them + (N11). + ## [2026.11.15.1] - 2026-11-15 ### Added diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 2936592f7..b88aff62e 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -568,6 +568,15 @@ belongs at the package/feature boundary, not on an individual target. > mcpp 2026.8.18.1 the two were byte-identical, so there was no spelling for > "nothing" and any file left under `src/` was swept in. +> **A path without a pattern names one file** (2026.11.16.1+). A `sources` +> entry with none of `*`, `?`, `[`, `{` names a file, and a file that does not +> exist is an error naming the table that states it (`[build]`, +> `[target.''.build]`, `[features.]`): it was skipped, and the build +> failed at an importer of the module the file provides. In a package fetched +> into a store, which its consumer cannot change, it is a warning. A file an action or +> a build program of the build writes is the build's. A pattern that matches +> nothing selects nothing, as a target's selection may. + > **`**/` stands for whole directories.** In `sources` and every glob that > shares its grammar, `**/` matches zero or more directories and what follows > it starts a path segment: `**/main.cpp` names every `main.cpp` and not @@ -1063,6 +1072,8 @@ with two behaviours has to be. With GCC and `bmi_schedule = "on"`, ordinary compilation and detached code generation share the same total `jobs` limit (2026.11.7.1+). Additional scheduling and join tasks do not increase the number of compiler processes. +A link shares the same limit (2026.11.16.1+): links that become ready together +run as many at once as compiles do, not all at once. Cancelling the build ends its ordinary and detached compilers. An immediately following build waits for their cleanup before starting another compiler; partial outputs are rebuilt. A missing object from a completed module is @@ -1546,7 +1557,8 @@ runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } | `tools.` | the package that builds a tool for this machine, or `{ program = "" }` | | `link` | `driver`, `flags`, `start`, `script`, `inputs`, `exe`, `shared`, `seal` | | `carrier` | `{ package, section, main }`: the program `--mode self-contained` builds | -| `runner` | an argv, or `{ cmd, host, exclusive }`: `host = "same-arch"` runs an artifact only on a machine of its architecture | +| `runner` | an argv, or `{ cmd, host, hosts, exclusive, requires, enabled, longlived }`: `host = "same-arch"` runs an artifact only on a machine of its architecture; a table without `cmd` adjusts the runner below it, as `{ enabled = true }` enables a runner its data states `enabled = false` (2026.11.16.1) | +| `runners.` | the same argv or table, for a named runner (the table form 2026.11.16.1) | `cfg(platform = "")` selects by platform; `platform` itself is not accepted under such a selector. The same keys are read from @@ -1679,6 +1691,10 @@ anything compiles. - Built-in profiles: `release` (-O2) / `dev`, `debug` (-O0 -g) / `dist` (-O3 + strip; **LTO is not enabled by default**). `[profile.]` can override a built-in definition wholesale. +- **The order of the flag lists.** Each of `cflags`, `cxxflags` and `ldflags` + reaches a command as `[build]`'s, then the profile's, then + `[target..build]`'s: where a compiler or a linker takes the last of + two words, the target block's wins (measured on 2026.11.16.1). - **`ndebug` and `trim_paths` (2026.10.24.1+)** follow the industry's convention: on for `release` and `dist`, off for `dev`, `debug` and a custom profile that does not state them. `ndebug` defines `NDEBUG` for every unit, so `assert` compiles out, as in @@ -2056,8 +2072,10 @@ generates the resource script automatically. | `version-info` | bool | `false` opts out of the generated version resource | | `[resources.version-info]` | table | `company`, `product`, `description`, `copyright`, `original-filename`, `internal-name` | -**Only PE targets *compile* this.** On Linux and macOS the section is -*inapplicable*: no resource units, no diagnostics, byte-identical build. You do +**Only targets whose programs Windows runs *compile* this**: PE rows whose +execution system is Windows. Elsewhere the section is *inapplicable* --- Linux, +macOS, and a PE row another system loads, such as `x86_64-uefi` (2026.11.16.1): +no resource units, no diagnostics, byte-identical build. You do **not** need (and cannot use) a `cfg(windows)` predicate — write it once, unconditionally. diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index 56074e73d..64972cae8 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -265,7 +265,9 @@ mcpplibs.os = { version = "0.4.4", override = true } the defaults, and the project may choose another. - **When it is needed.** Two version requests that no SemVer merge reconciles are refused naming both, and the refusal ends with the line that overrides - the package. A library that pins one release of a package every consumer + the package. Two requests from `[build-dependencies]` tables are refused the + same way and are never renamed apart: a build dependency is a module of a + build program, and one build loads one version of it (2026.11.16.1). A library that pins one release of a package every consumer shares (one that provides a target-side layer or a kernel ABI) is noted to the projects that use it, and `mcpp publish` refuses the pin in a library: a range leaves the choice to the project. diff --git a/docs/08-testing.md b/docs/08-testing.md index 7ba92d85a..e6ca60edd 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -282,7 +282,10 @@ The selection narrows what a pattern or `-p` names and adds nothing to it. the package's programs first, as `mcpp build` does, and gives each test `MCPP_BIN_`: the path of program ``, upper-cased, every character other than a letter or a digit written `_` (`MCPP_BIN_MY_TOOL` for `my-tool`). -A test that runs the program it tests reads it from there. +A test that runs the program it tests reads it from there. Under several +members the programs of a configuration group are planned together, as +`mcpp build --workspace` plans them, so a package they share is built with +one configuration (2026.11.16.1+). `--no-programs` (2026.11.13.1+) builds none of them: the programs are planned for their names, and each test takes each program from the environment's @@ -548,6 +551,10 @@ A framework states its guide to writing tests on its entry, `test-guide = ".exe`, and starts the runner on the image there with its -own arguments, looking for the runner first in `libexec//-/` -and then on `PATH`. The Windows launcher is the engine's program built for +own arguments. A package that carries its runner starts the one in +`libexec//-/`, and on a machine it holds none for exits with +status 126, naming the machine and the systems it holds; a package that carries +none finds the runner on `PATH`. `` is the system whose `uname -s` word the +system table records (`linux`, `macos`, `freebsd`, ...). Under `--all-programs` +every program of the package has its own pair of launchers, `bin/` and +`bin/.exe`. The Windows launcher is the engine's program built for `x86_64-windows-gnu`: it finds the package from its own path, picks the image of the machine's architecture, and starts the runner the same way; no `.cmd` is written. Where the build's platform has a hosted arrangement for that row, the diff --git a/docs/22-target-side.md b/docs/22-target-side.md index e6018ed33..45ec4ee4e 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -1385,7 +1385,7 @@ package needs. | Member | Type | Renders as | Reaches | |---|---|---|---| | `threads` | boolean | `-pthread` on a target that is neither PE nor freestanding; nothing on PE and on freestanding targets | the standard library module prebuild, the dependency scan, every C and C++ translation unit of every package, and the link | -| `exceptions` *(mcpp 2026.9.12.3+)* | boolean, or `"wasm"` / `"js"` *(2026.10.30.1+)* | on `os = "emscripten"` only, on the compile line through the dialect flags and on the link: `"wasm"` is `-fwasm-exceptions`, WebAssembly's own exception handling; `"js"` and `true` are `-fexceptions`, the JavaScript-based model. Nothing on every other target, where exceptions are already the default | the same set `threads` reaches | +| `exceptions` *(mcpp 2026.9.12.3+)* | boolean, or `"wasm"` / `"js"` *(2026.10.30.1+)* | on `os = "emscripten"` only, on the compile line through the dialect flags and on the link: `"wasm"` is `-fwasm-exceptions`, WebAssembly's own exception handling; `"js"` and `true` are `-fexceptions`, the JavaScript-based model; `false` states none. Unstated, the row's own model: `"wasm"` on `wasm32-emscripten` (2026.11.16.1; before, none). Nothing on every other target, where exceptions are already the default | the same set `threads` reaches | | `longjmp` *(mcpp 2026.11.12.1+)* | `"wasm"` or `"js"` | on `os = "emscripten"` only, on the compile line of every C and C++ translation unit and on the link: `"wasm"` is `-sSUPPORT_LONGJMP=wasm`, WebAssembly's own stack switching; `"js"` is `-sSUPPORT_LONGJMP=emscripten`, the JavaScript model and the toolchain's default. Declared on any other row, it is refused. An absent member renders nothing | the same set `threads` reaches | All three members enter the dependency cache key through the flags they render, so a diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index 3cbc29287..862d81cfb 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -363,6 +363,14 @@ returned an error would be the shape the specification forbids — present and always failing, which the caller cannot tell from a condition — and it would move a fact known at link time to run time. +**The terminal under node has no raw mode.** Emscripten reads standard input +with a synchronous read, which node serves from a pipe or a file and refuses on +a terminal: `read(0, ...)` fails with `EIO` there, with or without +`tcsetattr`, which returns 0 and changes nothing; `isatty(0)` is 1 in every +case. A program reads its input piped (`printf 'keys' | mcpp run --target +wasm32-emscripten`), and one that needs key-by-key input on the web reads it in +its page as key events (measured with emsdk 6.0.9 under node 26). + `openkal.task` is carried by a feature for a reason specific to this platform: threads need `-pthread`, which selects a different C library build, a different memory model and a different loader contract. Without the feature the diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index c0e971531..b20ac583e 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -100,7 +100,7 @@ is ignored, so diagnostics may be logged freely. | `mcpp:source=` *(0.0.100+)* | select a **pre-existing** source file into the build (absolute, or relative to the package root). Same downstream effect as `generated=`; use it for files the program *chose* (payload/vendored tree) rather than wrote — e.g. a per-target source selection over a large tarball | | `mcpp:include-dir=` *(0.0.100+)* | add a **private** include directory (`-I`) for this package's own TUs (absolute, or relative to the package root; normalized). Replaces the `cxxflag=-I` + `cflag=-I` double emission | | `mcpp:include-dir-after=` *(0.0.100+)* | like `include-dir`, but searched **after** the system directories (`-idirafter`) — for payload trees that shadow system headers | -| `mcpp:runner=` *(2026.8.19.2+)* | one argv token of the command that EXECUTES this build's artifact, when the host cannot. Emitted once per token, in order; the artifact path is appended (or substituted for `{}`). Reaches the **consumer**. Emit the executable as an ABSOLUTE path, and only **one** dependency may supply it | +| `mcpp:runner=` *(2026.8.19.2+)* | one argv token of the command that EXECUTES this build's artifact, when the host cannot. Emitted once per token, in order; the artifact path is appended (or substituted for `{}`). Reaches the **consumer**. Name the program (located as [41](41-devices.md) states), and only **one** dependency may supply it | | `mcpp:link-flag=` *(2026.9.6.5+)* | add a **linker flag** this program computed, verbatim. The outlet `link-lib` / `link-search` / `link-script` leave open: a generated version script (`-Wl,--version-script=`), `-Wl,--wrap=malloc` for a runtime that takes over a C-library symbol, `-Wl,--exclude-libs,ALL` so a statically absorbed third party does not become part of this package's ABI. Appended after `[build] ldflags`, in emission order. **Reaches the consumer**, exactly as `[build] ldflags` does — see below | | `mcpp:windows-subsystem=:` *(2026.9.12.2+)* | set the PE subsystem (`console` or `windows`) of the executable `` of **this** package, the same field as `[targets.] windows_subsystem` (docs/04). Reaches that target's link and no other, never a consumer, and renders nothing on a target that is not PE. A target the package does not declare with `kind = "bin"`, a value outside the set, and a value that contradicts mcpp.toml are each refused before any directive is applied | | `mcpp:windows-entry=:` *(2026.9.12.2+)* | set the entry function (`main`, `wmain`, `WinMain` or `wWinMain`) of the executable ``, the same field as `windows_entry`; the scope and the refusals are those of `windows-subsystem` | @@ -187,6 +187,7 @@ int main() { | `mcpp::seal(step, token)` *(protocol 17)* | `mcpp:seal=` — one token of seal step `step`, run on the linked file (`{}`) before it is published; `link.seal` | | `mcpp::run_host(h)` *(protocol 17)* | `mcpp:run-host=` — `"same-arch"` or `"any"`; `runner.host` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` — see below | +| `mcpp::runner_spec` *(protocol 19)* | `mcpp:runner-spec=` — a runner as one statement, the table of data field for field; see below | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | the payload directory of a package declared in `[xlings.workspace]` — by this manifest, or by a dependency compiled into this build program *(2026.9.6.6+)*; `""` when it was not declared or is not installed (see below) | | `mcpp::xpkg_source(ns, name)` / `mcpp::xpkg_program(ns, name)` *(protocol 15)* | where that payload comes from — `payload`, `override`, `pending`, or `""` — and the program an override named | | `mcpp::xpkg_request(ns, name)` / `mcpp::xpkg_pending()` *(protocol 15)* | the directory of a payload declared `provision = "on-request"`, asking for it when it is not installed yet; `xpkg_pending()` is then true and the program should return (see below) | @@ -290,9 +291,13 @@ anyway, **it wins** — swapping `-bios default` for `-bios none -semihosting` while debugging is a legitimate thing to want — and mcpp says which dependency it overrode. -**Emit the executable as an absolute path.** A bare name resolves through -`PATH` to a shim that dispatches against its own owner home, which is not -necessarily the home this build uses. +**Name the program, not its path.** The first word is looked up in the +packages of the runner's `requires`, then in the `bin/` of every payload the +graph declares, then on `PATH`, never in the current directory +([41 — Reaching a Device](41-devices.md)). A program outside a package's +`bin/` is written `{pkg:}/` with the package in `requires` +(`runner_spec` below), resolved when the run starts; an absolute path computed +at build time fixes the home and the version into the statement. **Exactly one dependency may supply a runner.** Two board-support packages both claiming to know how to run the artifact is a configuration error, and @@ -307,7 +312,37 @@ chooses between two runners chooses before it emits either. A runner named after a pack format reaches that format's distributable under `mcpp run --pkg-format ` ([41 — Reaching a Device](41-devices.md)). -### Asking instead of declaring: `toolchain_dir` / `sysroot_dir` (2026.8.19.4+) +### `runner_spec` — a runner as one statement (2026.11.16.1+, protocol 19) + +A runner in data is a table (`runner = { cmd = [...], requires = [...], ... }`, +[41](41-devices.md)); `mcpp::runner_spec` is that table in a build program, +field for field, submitted as one directive: + +```cpp +mcpp::runner_spec web; +web.name = "web"; // empty: the default runner +web.longlived = true; // no natural end +web.arg("{pkg:emsdk}/emscripten/emrun").arg("{}") + .require("xim:emsdk"); +web.submit(); +``` + +| Table key | `runner_spec` | +|---|---| +| `cmd` | `.arg(tok)`, as `action.arg()` | +| `longlived`, `exclusive`, `enabled` | fields of the same name (`enabled` defaults to `true`) | +| `host` | the field of the same name (`"same-arch"`, `"any"`) | +| `hosts` | `.hosts(pair)`, one `-` per call | +| `requires` | `.require(pkg)`: `requires` is a C++ keyword | + +`{}` is the artifact and `{pkg:}` the root of a package of `require()`, +installed by the first `mcpp run` or `mcpp test` that uses the runner, never by +a build. A runner is stated one way: a name stated by `runner_spec` and by the +token calls, a `runner_spec` submitted twice for a name, one without a command, +or a `host` other than the two words is refused before anything is applied. +The token calls keep their meaning; overriding, `enabled` and the one supplier +per name follow the rules of data. +: `toolchain_dir` / `sysroot_dir` (2026.8.19.4+) ```cpp const char* tc = mcpp::toolchain_dir(); // the resolved toolchain's payload root @@ -1033,6 +1068,21 @@ pass their own format — and makes the set unknowable for everyone else. mcpp refuses a format nothing submitted for, rather than reporting a pack that produced no package. +**The program a provider assembles from.** A provider that wraps one program of +the package being packed states which, unconditionally, beside the declaration: + +```cpp +mcpp::provides_pack_format("web"); +mcpp::pack_program("app"); // protocol 19, mcpp 2026.11.16.1 +``` + +`mcpp pack --pkg-format web` with no program named then packs `app`; a program +named on the command line outranks it, and a stated program the build does not +have is refused by the provider's name. When a provider submits nothing in the +second pass, the refusal lists the warnings it stated there (`mcpp::warning`, +`mcpp::report`) as its reasons, so a provider explains a decline with either and +not with its standard error, which mcpp shows only for a program that fails. + **`mcpp pack --pkg-format ` prepares twice.** An `artifact` action is a ninja edge and the staged tree is produced by mcpp *after* the link, so the tree cannot be an input of the pass that built it. The first pass collects the diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 91f74e659..24de54710 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -100,8 +100,8 @@ runner = ["my-okrun", "{}"] |---|---| | compile flags, `link.flags` | appended after the platform's; a convention cannot be removed | | `link.inputs`, `link.seal` | replaced whole | -| `layers`, `tools`, `runners` | replaced key by key; ` = false` leaves the layer out | -| `runner` | the argv is replaced; `host` and `exclusive` are kept unless stated | +| `layers`, `tools` | replaced key by key; ` = false` leaves the layer out | +| `runner`, each of `runners` | `cmd`, `requires` and `enabled` are replaced together, and a runner whose `cmd` is replaced is enabled unless stated; `host`, `hosts`, `exclusive` and `longlived` are kept unless stated; a table without `cmd` in `config.toml` or the project adjusts the keys it writes (2026.11.16.1) | | every other field | replaced | The same tables are accepted in `~/.mcpp/config.toml` for one machine. A layer @@ -317,6 +317,16 @@ machine description is the platform's: a target whose machine differs from a built-in row is another row. `[platform]` and `[platform.hosted]` remain the two shorthands of a provider. +**A row's companions and exception model (2026.11.16.1+).** `companions = [".wasm"]` +names the files the link may write beside a program, named after it: they are removed +before each link, staged when the link wrote one, and not an error when it did not. +`exceptions` is the model of the row's builds when the project states no +`[target..abi] exceptions`; under `"wasm"` the long-jump model is `"wasm"` too +(the built-in `wasm32-emscripten` row states `"wasm"`). A machine +runs another architecture's programs only where the execution table states it +(`[[platform.executes]]`, SPEC-016 §2.4); there a `same-arch` runner is built for +the artifact's architecture and started on this machine. + **A defined row is read as a built-in one (2026.11.10.1+).** Its `march` and `link-emulation` reach the compile and an `ld.lld` link (`mcpp why flags` marks them `row`); nothing the toolchain payload built for its own host — its libc++ headers, @@ -401,7 +411,10 @@ it enters below the entries that answer `-` today A system that states a `uname` joins a file that carries one program per system (`mcpp pack --pkg-format single-file`); the system that states none -leads the file as its PE. A platform whose runner is built for several systems +leads the file as its PE. A system whose loader reads a program's resource +section states `resources = true` (the built-in `windows` entry): a project's +`[resources]` compile into the programs of its rows, and not into a PE image +another system loads, such as a UEFI application (2026.11.16.1). A platform whose runner is built for several systems states the identity of each: ```toml diff --git a/docs/41-devices.md b/docs/41-devices.md index a8f3829dc..be63936b2 100644 --- a/docs/41-devices.md +++ b/docs/41-devices.md @@ -68,6 +68,9 @@ mcpp::runner_longlived("monitor"); // no natural end mcpp::run_exclusive(); // this target's runs cannot overlap ``` +`mcpp::runner_spec` states a runner as the table of data, `requires` and +`enabled` included (2026.11.16.1+, [30](30-build-mcpp.md)). + **Name the program, not its path.** mcpp locates it: the `bin/` of a payload declared under `[xlings.workspace]` by **any package in the graph** — the consuming project first, then its dependencies — and then `PATH`. A board-support package @@ -76,7 +79,9 @@ so it declares that payload itself and the consumer declares nothing. Writing an absolute path computed from `mcpp::xpkg_dir` is unnecessary, and it introduces a failure mode — a declaration is not an install, so the lookup can return empty and leave no runner configured with nothing said about why. Naming the program -lets mcpp report exactly which directories it searched. +lets mcpp report exactly which directories it searched. A program outside a +payload's `bin/` is written `{pkg:}/`, its package among the +runner's `requires`, which are searched first. ## The project’s overrides @@ -114,6 +119,53 @@ machine's `config.toml`, the project and `MCPP_TARGET__RUNNER` each replace it in that order, and an argv alone keeps the platform's `host` and `exclusive`. +**A runner enabled on request (2026.11.16.1+).** A runner whose first use +fetches a whole machine image or an emulator, or leaves state outside the build +tree and the package store, is stated `enabled = false` by its data: a virtual +machine, an Android emulator, and Wine for the Windows rows on an x86_64 Linux +machine. `mcpp run` refuses it with status 2, and `mcpp test` leaves the tests +built and not run. The message carries the token `[runner-not-enabled]`, the +packages the first use installs, and the two ways of enabling it; nothing is +installed before it is enabled. + +```toml +[target.x86_64-windows-gnu] +runner = { enabled = true } # mcpp.toml, or config.toml for this machine +``` + +A table without `cmd` adjusts the runner below it. `cmd`, `requires` and +`enabled` are one statement: a tier that replaces `cmd` states another runner, +enabled unless it says otherwise, so a project that writes `runner = ["wine", +"{}"]` behaves as before. `MCPP_TARGET__RUNNER_ENABLED=true` enables +the default runner for one invocation, and `mcpp run --runner ` enables +the runner it names. A named runner takes the same table +(`[target..runners] web = { cmd = [...], longlived = true }`), merged by +the same rule. + +**An Android program runs in the emulator (2026.11.16.1+).** On an x86_64 +Linux machine, once enabled, `x86_64-linux-android` runs through dirun +(>= 0.4.0) and the profile of the API 34 system image package: the emulator +boots a fresh device, the program and its runtime files are pushed through adb, +and its output and status return. The runs of the target do not overlap +(`exclusive`), since the device has one console port. The aarch64 row has no +runner: the emulator package serves no aarch64 machine. + +```bash +MCPP_TARGET_X86_64_LINUX_ANDROID_RUNNER_ENABLED=true mcpp run --target x86_64-linux-android +``` + +## The program a runner names (2026.11.16.1+) + +| | Rule | +|---|---| +| lookup | the packages of `requires`, then the `bin/` of each payload the graph declares, then `PATH`; never the current directory | +| a program outside `bin/` | `{pkg:}/`, its package among `requires` | +| a name with a suffix-less file on Windows | the system's suffixes, `.exe`, `.cmd`, `.bat`, for a bare name and for a name with a directory alike (`../tools/x` is `../tools/x.cmd`); the file without a suffix is not a program there | +| execution | an argv, never through a shell | +| a script the system starts through its command interpreter | `.bat` and `.cmd` on Windows (the system table's `command-scripts`): an argument holding `%`, `^`, `&`, `\|`, `<`, `>`, `"` or a line end is refused before the start, since the interpreter reads it as syntax whatever the quoting (the class of CVE-2024-24576) | +| placeholders | `{}`, `{sysroot}` and `{pkg:}`, inside a word as well (`{}/index.html`); any other brace is kept as written; with no `{}` the artifact is appended | +| status | the runner's status is the run's; a long-lived runner's status is not a verdict, and `mcpp test` does not use it | + ## What a runner receives A runner receives an argv — its own tokens, with the artifact's path appended @@ -202,6 +254,7 @@ and the manifest table that would declare it. |---|---| | default | runs to completion; the exit code is the verdict | | `runner_longlived(name)` | has no natural end; the operator ends it | +| `longlived = true` in a runner table | the same, stated in data (2026.11.16.1+) | `openocd -c "program image.elf verify reset exit"` terminates and `openocd -c "init"` does not, and the two are spelled alike up to the argument the package diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 994abb34c..47aa59e7d 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -295,7 +295,7 @@ and only the first belongs to the program: | `0`–`124` | the program ran; this is its own status, passed through unchanged. The exception is `101`, below | | `101` | `mcpp run` could not build the program: its planning or its build failed, and nothing was started. This is the status Cargo gives a failed `cargo run` build | | `125`–`127` | the spawn was attempted and refused — `127` not found, `126` found but not executable, `125` anything else; `126` also answers `mcpp run --pkg-format ` for a distributable that is a directory and meets no runner, refused before the spawn with the same meaning (2026.9.14.2+) | -| `2` | mcpp refused the request before building anything, or after building it and before starting anything: a usage error, `--pkg-format` with `--no-runner`, a program or a runner that the project does not declare | +| `2` | mcpp refused the request before building anything, or after building it and before starting anything: a usage error, `--pkg-format` with `--no-runner`, a program or a runner that the project does not declare, a runner that is not enabled (the message carries `[runner-not-enabled]`; 2026.11.16.1+) | Until 2026.9.30.2 a failed build exited `1` and a failed planning `2`, so that a compile error and a program that returns `1` could not be told apart by a @@ -410,7 +410,8 @@ of another architecture and a `same-arch` runner, it is the platform's runner for that architecture, located where the first run builds it (`built: false` until then; `self env` builds nothing). `runUnderProgram` is the execution host's program, and `runner.refusal` the reason runs are refused on this -machine (`null` when they are not). For `--host` the host platform's +machine (`null` when they are not; a runner that is not enabled is one, with +`[runner-not-enabled]`). For `--host` the host platform's `runner` and `tools` are reported once the project's build programs have been built under it. A reason the project's sources would be refused for (an `import std` with no C++ run-time in the graph) is a `warning` diagnostic with the code diff --git a/docs/specs/build-plugins.md b/docs/specs/build-plugins.md index 8dde4ffd4..4f0d5bbc0 100644 --- a/docs/specs/build-plugins.md +++ b/docs/specs/build-plugins.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-007 | | 标题 | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | -| 状态 | 草案 v0.12 | -| 版本 | 0.12 | -| 最后修改 | 2026-11-05 | -| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2;注明 mcpp#707、#708、#709、#711 的条款对应 mcpp >= 2026.9.27.1;注明 mcpp#723 的条款对应 mcpp >= 2026.9.28.1;注明 mcpp 2026.9.28.2 的条款对应该版本;§9 与注明 mcpp#734 的条款对应 mcpp >= 2026.9.28.3;注明 mcpp#778 的条款对应 mcpp >= 2026.10.10.1;§10 对应 mcpp >= 2026.10.11.1;R9.10 与协议 17 对应 mcpp >= 2026.10.27.1;§3.1 与协议 18 对应 mcpp >= 2026.11.5.1 | +| 状态 | 草案 v0.13 | +| 版本 | 0.13 | +| 最后修改 | 2026-11-16 | +| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2;注明 mcpp#707、#708、#709、#711 的条款对应 mcpp >= 2026.9.27.1;注明 mcpp#723 的条款对应 mcpp >= 2026.9.28.1;注明 mcpp 2026.9.28.2 的条款对应该版本;§9 与注明 mcpp#734 的条款对应 mcpp >= 2026.9.28.3;注明 mcpp#778 的条款对应 mcpp >= 2026.10.10.1;§10 对应 mcpp >= 2026.10.11.1;R9.10 与协议 17 对应 mcpp >= 2026.10.27.1;§3.1 与协议 18 对应 mcpp >= 2026.11.5.1;协议 19 对应 mcpp >= 2026.11.16.1 | | 相关设计文档 | `.agents/docs/2026-09-26-compile-database-and-issue-699-design.md`(§5)、`.agents/docs/2026-09-28-ecosystem-design-and-optimisation-plan.md`(WS1、WS3)、`.agents/docs/2026-09-28-build-cost-foreign-toolsets-and-library-surface-design.md`(§3、§4)、`.agents/docs/2026-10-10-one-source-one-meaning-design.md`(§4)、`.agents/docs/2026-10-11-build-dependencies-one-pipeline-design.md` | | 相关 issue | mcpp#699、mcpp#701、mcpp#702、mcpp#703、mcpp#707、mcpp#708、mcpp#709、mcpp#711、mcpp#734、mcpp#755、mcpp#778、mcpp#797 | | 使用文档 | [docs/30 - build.mcpp](../30-build-mcpp.md)、[docs/31 - 编写规则包](../31-authoring-a-rule-package.md) | @@ -270,6 +270,7 @@ | 16 | 2026.10.26.2 | 镜像入口:`link_start`(`mcpp:link-start=`,docs/04 `start`) | | 17 | 2026.10.27.1 | 平台的链接与运行器字段(SPEC-013 §12):`link_driver`、`link_input`、`link_input_from`、`artifact_name`、`seal`、`run_host`;平台阶段(`mcpp::phase()` 为 `"platform"`) | | 18 | 2026.11.5.1 | `action.progress` 与可选运行期快照通道（§3.1）；旧载荷默认关闭 | + | 19 | 2026.11.16.1 | 包格式提供者陈述它组装产物所取的程序:`pack_program`(`mcpp:pack-program=`,SPEC-014 §6);一条陈述的运行器:`runner_spec`(`mcpp:runner-spec=`,一个 JSON 载荷,字段与 SPEC-013 §10 的表一一对应;同一名字不能既由它又由逐词调用陈述) | - **R9.3** 构建信息以事实陈述解析出的工具链,不针对任何外部构建系统:`tool(role)`(这一行的 工具)、`abi_tool(role)`(目标 ABI 的原生工具,MSVC ABI 上为工具集的 `cl`、`link`、`lib`、 @@ -351,3 +352,4 @@ | 0.10 | 2026-10-27 | 随 mcpp 2026.10.26.2(mcpp#845):R9.2 的协议表增加协议 16(`link_start`,镜像入口)。 | | 0.11 | 2026-10-28 | 随 mcpp 2026.10.27.1(mcpp#847):R9.2 的协议表增加协议 17(平台的链接与运行器字段);新增 R9.10 平台阶段(SPEC-013),平台阶段可声明包格式与 artifact action(SPEC-014 §6)。 | | 0.12 | 2026-11-05 | 随 mcpp 2026.11.5.1：新增 §3.1 可选 action 进度（协议 18），规定通道、限额、序号、关闭与失败/取消行为。 | +| 0.13 | 2026-11-16 | 随 mcpp 2026.11.16.1(设计 2026-11-15 N30):R9.2 的协议表增加协议 19(`pack_program`)。;协议 19 增加 `runner_spec`(设计 2026-11-15 R2)。 | diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index bd576e98c..1cd38a4af 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -4,11 +4,11 @@ |---|---| | 规范编号 | SPEC-014 | | 标题 | 打包与分发:链接、闭包、容器三个维度与构建腿,腿的合并,暂存树与包元数据,运行覆盖、单文件、容器列表、签名、压缩、合并与分发描述 | -| 状态 | 评审中 v2.4 | -| 版本 | 2.4 | -| 最后修改 | 2026-11-14 | -| 对应实现 | mcpp >= 2026.10.31.1;v2.2 条款 mcpp >= 2026.11.10.1;v2.3 条款 mcpp >= 2026.11.12.1;v2.4 条款 mcpp >= 2026.11.13.1 | -| 相关设计文档 | `.agents/docs/2026-10-30-ecosystem-part2-ide-and-language-service-design.md`(§19.6)、`.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md`(§4、§13)、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7)、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md`(§2.2、§2.6) | +| 状态 | 评审中 v2.5 | +| 版本 | 2.5 | +| 最后修改 | 2026-11-16 | +| 对应实现 | mcpp >= 2026.10.31.1;v2.2 条款 mcpp >= 2026.11.10.1;v2.3 条款 mcpp >= 2026.11.12.1;v2.4 条款 mcpp >= 2026.11.13.1;v2.5 条款 mcpp >= 2026.11.16.1 | +| 相关设计文档 | `.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md`(§5)、`.agents/docs/2026-10-30-ecosystem-part2-ide-and-language-service-design.md`(§19.6)、`.agents/docs/2026-10-29-ecosystem-target-sets-packaging-and-editors-design.md`(§4、§13)、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`(§2–§7、§14)、`.agents/docs/2026-10-28-platforms-design.md`(§4.7)、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md`(§2.2、§2.6) | | 相关 issue | mcpp#847、mcpp#849、mcpp#860 | | 相关规范 | [SPEC-007](build-plugins.md)(包格式提供者、artifact action)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-013](platforms.md)(平台、运行器、载体)、[SPEC-015](target-sets.md)(目标集合与腿) | | 使用文档 | [10 —— 发布打包](../zh/10-pack-and-release.md) | @@ -213,8 +213,13 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` `x86_64`、`amd64`、`AMD64` → `x86_64`,`aarch64`、`arm64`、`ARM64` → `aarch64`;没有对应目录时以状态 126 退出, 写出本机架构与包含的架构。 - 运行 `runner.cmd`,与 `mcpp run` 的展开相同:含 `{}` 的词中 `{}` 替换为所选架构目录中的入口映像,没有词含 `{}` - 时映像追加在末尾;其余词原样传递,随后是启动器自己的参数;运行器先在 `libexec//-/` 中找, - 再找 `PATH`。 + 时映像追加在末尾;其余词原样传递,随后是启动器自己的参数。携带运行器的包(`libexec//` 存在)只运行 + `libexec//-/` 中的运行器,本机没有时以状态 126 退出,写出本机的 `-` 与包所含的各项; + 不携带运行器的包在 `PATH` 上找(2026.11.16.1:此前携带运行器的包也回落到 `PATH`,运行的是另一版本的运行器)。 +- `` 取系统表(SPEC-016 §2.3)中 `uname` 一列与 `uname -s` 相等的系统名;表中没有的系统取 `uname -s` 的小写 + (2026.11.16.1:此前只认 Linux 与 Darwin,FreeBSD 上找的是 `FreeBSD-`)。 +- `--all-programs` 的树中,每个程序都有自己的两个启动器(`bin/`、`bin/.exe`),入口为它在各 + `lib//` 中的映像(2026.11.16.1)。 - POSIX 启动器把每个词写成单引号字符串,沿符号链接找到自身的位置(不依赖 `readlink -f`);`uname -s` 为 `MINGW*`、 `MSYS*`、`CYGWIN*` 时交给 `bin/.exe`。macOS 上运行器带有 `com.apple.quarantine`(浏览器下载的包)时,启动器在启动它之前写出 这一点与去除标记的命令(`xattr -dr com.apple.quarantine <包>`)(2026.10.31.1)。 @@ -400,6 +405,12 @@ openkal 目标的单文件:载体携带一条腿的映像本身(映像不引用 (`mcpp::pack_stage_dir()`、`${mcpp.stage_dir}`)、其中的 `mcpp-package.json`、`MCPP_PACK_*` 环境;它报告的终端 输出即产物。未知的名字被拒绝,消息列出图中可用的格式。 +提供者**可以**以 `mcpp::pack_program("")` 陈述它从被打包的包的哪个程序组装产物(构建程序协议 19, +2026.11.16.1)。陈述与 `provides_pack_format` 一样无条件给出:程序在打包的第二遍之前选定。未指名程序的 +`--pkg-format ` 打包该程序;命令行指名的程序优先;所请求格式的提供者陈述了不同的程序时拒绝并列出各项; +陈述的程序不在本次构建中时拒绝,消息指名提供者。提供者在第二遍中没有提交 `role = "artifact"` 的 action 时, +拒绝消息列出它在这一遍中陈述的警告与降级(`mcpp::warning`、`mcpp::report`),作为它不提交的原因。 + 平台插件(SPEC-013 §12)**可以**在平台阶段声明包格式与 artifact action;平台阶段的指令表因此包含 `pack-format` 与 `role = "artifact"` 的 action(其他角色的 action 仍被拒绝)。该平台的目标打包时,这些格式与引擎的格式并列。 @@ -495,3 +506,4 @@ openkal 目标的单文件:载体携带一条腿的映像本身(映像不引用 | 2.2 | 2026-11-10 | mcpp 2026.11.10.1(设计 2026-11-10 E7,mcpp#911 N8):§1.2 另一系统的产物从文件读取闭包,不运行本机加载器;`--mode static` 链接全静态程序并在打包前核对。 | | 2.3 | 2026-11-12 | mcpp 2026.11.12.1(设计 2026-11-12 C6、C7):§4.4 `--systems all` 取系统表,无程序可构建的对写为未覆盖;三个系统之外的程序按 `runner.built-as` 构建;单文件按系统表的 `uname` 选择程序(SPEC-016 §2.3)。 | | 2.4 | 2026-11-14 | mcpp 2026.11.13.1:§4.4 平台定义的行即该系统的程序之行(行名可带环境段),`built-as` 指向定义的行时在平台之上构建;strip 不读的封装容器按封装结果打包。 | +| 2.5 | 2026-11-16 | mcpp 2026.11.16.1(设计 2026-11-15 N11、N20):§4.3 携带运行器的包不回落到 `PATH`,本机没有时以 126 退出并写出所含各项;`` 取系统表的 `uname`;`--all-programs` 的树中每个程序有自己的启动器;§6 `mcpp::pack_program` 与提供者不提交的原因(N30)。 | diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 381c82f0d..d68f90c85 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -4,11 +4,11 @@ |---|---| | 规范编号 | SPEC-013 | | 标题 | 平台:一类目标的可复用配置,其声明、选择、合并、对依赖图的补全、链接与封装、运行器与载体 | -| 状态 | 评审中 v1.9 | -| 版本 | 1.9 | -| 最后修改 | 2026-11-14 | -| 对应实现 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1;`runner.hosts`、载体列表、`descriptor_keys` 与 `universal` 的弃用 mcpp >= 2026.10.30.1;§7.5 版本组参与解析、§14 平台系列与 §15.1 系列的锁 mcpp >= 2026.11.7.1;§14 内置平台为确切版本与提示 mcpp >= 2026.11.7.2;v1.7 条款 mcpp >= 2026.11.10.1;逐条标注 | -| 相关设计文档 | `.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md` | +| 状态 | 评审中 v1.10 | +| 版本 | 1.10 | +| 最后修改 | 2026-11-16 | +| 对应实现 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1;`runner.hosts`、载体列表、`descriptor_keys` 与 `universal` 的弃用 mcpp >= 2026.10.30.1;§7.5 版本组参与解析、§14 平台系列与 §15.1 系列的锁 mcpp >= 2026.11.7.1;§14 内置平台为确切版本与提示 mcpp >= 2026.11.7.2;v1.7 条款 mcpp >= 2026.11.10.1;v1.10 条款 mcpp >= 2026.11.16.1;逐条标注 | +| 相关设计文档 | `.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md`、`.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md` | | 相关 issue | mcpp#846、mcpp#847、mcpp#848 | | 相关规范 | [SPEC-002](target-side.md)(目标侧的层)、[SPEC-004](manifest-semantics.md)(条件化形状)、[SPEC-006](toolchain-management.md)(工具链选择)、[SPEC-007](build-plugins.md)(构建程序与协议)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-014](packaging.md)(打包) | | 使用文档 | [35 —— 平台](../zh/35-platforms.md) | @@ -205,11 +205,12 @@ plugin = "build.mcpp" | `build` 的四个列表、`link.flags` | 拼接,平台的在前;使用者只能追加,不能删除平台的约定 | | `build` 的 `cfg(...)` 子表 | 追加 | | `link.inputs`、`link.seal` | 整体替换 | -| `layers`、`tools`、`runners`、`versions` | 按键替换 | -| `runner` | `cmd` 替换;`host`、`exclusive` 各自独立替换,只写 argv 的覆盖保留平台的 `host` 与 `exclusive` | +| `layers`、`tools`、`versions` | 按键替换 | +| `runner`、`runners` 的每个名字 | `cmd`、`requires` 与 `enabled` 是一条陈述,随 `cmd` 一起替换,未写的 `enabled` 为真;`host`、`hosts`、`exclusive`、`longlived`、`built-as` 各自独立替换。只写 argv 的覆盖保留平台的 `host` 与 `exclusive`;覆盖层(本机配置、工程)不含 `cmd` 的表只改它写出的键(v1.10) | | 其余 | 按字段替换 | `MCPP_TARGET__RUNNER` 以空白分隔的 argv 替换 `runner.cmd`;值为 `none` 时清除运行器。**已实现** +`MCPP_TARGET__RUNNER_ENABLED` 为 `true`(`1`)或 `false`(`0`)时设定缺省运行器的 `enabled`,位于全部层之上(v1.10)。 ## 7. 依赖图的补全 @@ -319,6 +320,22 @@ plugin = "build.mcpp" | `hosts` | 运行器程序存在的 `-`(`os` 为系统表中的系统,SPEC-016 §2.3):需要该运行器的包能在哪里运行(SPEC-014 §4.4 的 `runs_on.with`)。`host = "any"` 时与产物的架构无关。**mcpp >= 2026.10.30.1** | | `requires` | 运行器从哪些包安装(`xim:<名>@<版本>`)。第一次 `mcpp run` 或 `mcpp test` 时按工具链的同一方式安装;构建不安装。模板的程序先在这些包中查找,再查 PATH。替换 `cmd` 的陈述同时替换它。**mcpp >= 2026.11.12.1** | | `built-as` | `{ "-" = "<身份>" }`:为该宿主对构建的运行器程序是哪个身份。自带运行器的包(SPEC-014 §4.4)对引擎不持有约定的系统按它构建。**mcpp >= 2026.11.12.1** | +| `enabled` | 是否使用该运行器,缺省为真。为假时 `mcpp run` 以状态 2 拒绝,`mcpp test` 使测试已构建而未运行(`unreachableIsBuilt` 的调用为"已构建、本机不运行"),消息带 `[runner-not-enabled]`、`requires` 与两种启用写法;拒绝在安装 `requires` 之前。`mcpp run --runner ` 指名的运行器视为启用;`--pkg-format` 找到的同名运行器不算指名。以 `host = "same-arch"` 到达不了产物的运行器不以此拒绝,而按到达拒绝。**mcpp >= 2026.11.16.1** | +| `longlived` | 运行器没有自然结束,状态不是结论;`mcpp test` 不使用它。与构建程序的 `runner_longlived` 同义。**mcpp >= 2026.11.16.1** | + +运行器的程序(**mcpp >= 2026.11.16.1**): + +1. **查找。** `cmd` 首项先在 `requires` 的包中查找,再在图中各载荷的 `bin/` 与载荷根中查找,最后查 PATH;**禁止**查找 + 当前目录。不在 `bin/` 下的程序写作 `{pkg:}/<路径>`。Windows 上无后缀的名字取系统的后缀(`.exe`、`.cmd`、 + `.bat`),裸名字与带目录的名字相同;无后缀的同名文件不是程序。 +2. **执行。** argv 不经 shell。系统表为本机系统陈述 `command-scripts`(SPEC-016 §2.3)时,后缀属于它的程序由命令解释器 + 启动;参数含 `%`、`^`、`&`、`|`、`<`、`>`、`"`、回车或换行时**必须**在启动前拒绝并指出该参数,**禁止**把它交给 + 解释器改写或执行。 +3. **占位符。** 只替换 `{}`、`{sysroot}`、`{pkg:}`,可位于词中;其余花括号原样保留;没有 `{}` 时产物附在末尾。 +4. **状态。** 运行器的退出状态是 `mcpp run` 的状态;`longlived` 的运行器的状态不是结论,`mcpp test` 不使用它。 + +具名运行器 `runners.` 取与 `runner` 相同的数组或表形式(**mcpp >= 2026.11.16.1**;此前只有数组)。平台与提供者 +陈述的运行器**必须**有 `cmd`;本机配置与工程的表可以只写 `enabled` 等键,调整其下一层的运行器。 模板中 `{}` 为产物,`{sysroot}` 为该目标的基座(SPEC-006),`{pkg:}` 为 `requires` 中该名字的包的根 (**mcpp >= 2026.11.12.1**)。运行器陈述的最低一级是所选提供者条目的 `runner`(SPEC-016 §6),其上依次为平台、构建程序、 @@ -560,3 +577,4 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri | 1.7 | 2026-11-10 | mcpp 2026.11.10.1(设计 2026-11-10 E8,mcpp#911 N6):§4 `link.flags` 在 `driver = "compiler"` 时给出警告。 | | 1.8 | 2026-11-12 | mcpp 2026.11.12.1(设计 2026-11-12 §5.2、§5.5):§10 运行器的 `requires` 与 `built-as`,模板占位 `{sysroot}`、`{pkg:}`,提供者的 `runner` 为陈述的最低一级;`hosts` 的系统取自系统表。 | | 1.9 | 2026-11-14 | mcpp 2026.11.13.1:§7.3 一行所带的工具,`base = "none"` 的提供者只带其自身约定指名的工具,其余身份不带只由这类提供者指名的工具。 | +| 1.10 | 2026-11-16 | mcpp 2026.11.16.1(设计 2026-11-15 R1、R2):§10 运行器的 `enabled` 与 `longlived`;具名运行器与 `runner` 同一种表;§6 覆盖层不含 `cmd` 的表、`enabled` 随 `cmd` 的陈述、`MCPP_TARGET__RUNNER_ENABLED`。;运行器的程序:查找、Windows 的后缀、命令解释器启动的脚本的参数、占位符与状态(R2)。 | diff --git a/docs/specs/targets-and-providers.md b/docs/specs/targets-and-providers.md index 17d2f7742..b98620708 100644 --- a/docs/specs/targets-and-providers.md +++ b/docs/specs/targets-and-providers.md @@ -4,11 +4,11 @@ |---|---| | 规范编号 | SPEC-016 | | 标题 | 目标与提供者：行、提供者、选择、平台定义的行、报告与锁 | -| 状态 | 评审中 v1.2 | -| 版本 | 1.2 | -| 最后修改 | 2026-11-12 | -| 对应实现 | mcpp >= 2026.11.9.1；v1.1 条款 mcpp >= 2026.11.10.1；v1.2 条款 mcpp >= 2026.11.12.1 | -| 相关设计文档 | `.agents/docs/2026-11-09-target-model-design.md`、`.agents/docs/2026-11-10-cross-platform-ecosystem-design.md`、`.agents/docs/2026-11-12-ecosystem-architecture-optimization-design.md` | +| 状态 | 评审中 v1.3 | +| 版本 | 1.3 | +| 最后修改 | 2026-11-16 | +| 对应实现 | mcpp >= 2026.11.9.1；v1.1 条款 mcpp >= 2026.11.10.1；v1.2 条款 mcpp >= 2026.11.12.1；v1.3 条款 mcpp >= 2026.11.16.1 | +| 相关设计文档 | `.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md`、`.agents/docs/2026-11-09-target-model-design.md`、`.agents/docs/2026-11-10-cross-platform-ecosystem-design.md`、`.agents/docs/2026-11-12-ecosystem-architecture-optimization-design.md` | | 相关 issue | mcpp#911、mcpp#913 | | 相关规范 | [SPEC-001](package-identity.md)（引用文法）、[SPEC-002](target-side.md)（层与能力）、[SPEC-013](platforms.md)（平台的语句）、[SPEC-014](packaging.md)（打包）、[SPEC-015](target-sets.md)（目标集合） | | 使用文档 | [21 —— 目标三元组](../zh/21-the-target-triple.md)、[35 —— 平台](../zh/35-platforms.md) | @@ -37,7 +37,7 @@ 行的字段：`triple`（`-[-]`）、`tier`（verified、preview、planned）、`standard`、`exact`、`march`、`mabi`、 `code-model`、`multilib`、`flags`、`link-emulation`、`clang-triple`、`object-format`、`family`、`data-model`、`exe-suffix`、 -`shared-suffix`、`loader`、`execution`、`descriptor-keys`。 +`shared-suffix`、`loader`、`execution`、`descriptor-keys`、`companions`、`exceptions`。 - 一个系统可能用到两种以上 C 库时，身份**必须**含 env；env 即 C 库。业界原生标识无 env 段的系统，其原生身份为两段 （`aarch64-macos`），其余 C 库以 env 区分（`aarch64-macos-musl`）。 @@ -49,6 +49,12 @@ - `loader` 是该行的动态程序所写的程序解释器：运行它的机器上的路径（`/lib/ld-linux-aarch64.so.1`）。内置行与平台定义的行 同一读法；引擎不另持有按架构的加载器表（v1.2）。 - `execution` 是产物运行的系统，未写时为 os 段。内核相同而系统不同的行**必须**写出它（Android 的行为 `android`）（v1.2）。 +- `exceptions` 是工程未陈述 `[target..abi] exceptions` 时该行构建的异常模型；其为 `wasm` 且 `longjmp` 未陈述时， + `longjmp` 亦为 `wasm`。 + `wasm32-emscripten` 为 `wasm`：C++ 异常可被捕获，`setjmp`/`longjmp` 取同一机制；工程以 `exceptions = "js"`、`true` + 或 `false` 陈述其他模型或不启用（v1.3）。 +- `companions` 是链接可能在程序旁写出、以程序命名的文件，后缀替换程序的后缀（`app.js` 旁的 `app.wasm`）。每次链接前 + 删除；链接写出的被暂存；缺席不是错误（`-sSINGLE_FILE=1` 不写）（v1.3）。 ### 2.2 内置行 @@ -70,13 +76,33 @@ |---|---| | `name` | 该系统各行的 os 段 | | `uname` | `uname -s` 在该系统上的回答；每个系统一个程序的单文件按它选择程序 | +| `resources` | 该系统的加载器读取程序的资源节；工程的 `[resources]` 编入其行的程序，未写为 false（v1.3） | +| `command-scripts` | 该系统经命令解释器启动的文件的后缀；在该系统上启动这类文件时，含解释器语法的参数被拒绝（SPEC-013 §10）（v1.3） | - 陈述了 `uname` 的系统可以进入多系统单文件；表中未陈述 `uname` 的系统，其程序是文件开头的 PE；表中没有的系统被拒绝， 并指名 `[[platform.system]]`。 - `mcpp pack --systems all` 取表中的全部系统，加上平台定义了行的系统；其中没有程序可构建的对逐个写为未覆盖（SPEC-014 §4.5）。 - 同一系统陈述两次时取先陈述者；内置数据先于平台。 +- 资源随行的 `execution` 所指系统而定，不随 PE 容器：UEFI 应用是固件加载的 PE，不编入资源。内置数据中 `windows` 陈述 + `resources = true`（v1.3）。 +- 内置数据中 `windows` 陈述 `command-scripts = [".bat", ".cmd"]`（v1.3）。 - 引擎源码中系统名的出现次数由检查持有（`.github/tools/check_system_names.py` 与其基线），只许减少。 +### 2.4 执行关系（v1.3） + +`[[platform.executes]]` 陈述一台机器除自身架构外还执行哪些架构的程序。只陈述 CI 实测过的关系。 + +| 键 | 语义 | +|---|---| +| `host` | 机器，`-` | +| `arches` | 它执行的其他架构 | + +- `host = "same-arch"` 的 runner 在执行该架构的机器上即可运行：引擎为产物的架构构建平台的 runner（载体不携带映像的 + 形式），在本机启动（`mcpp run`、`mcpp test`）。 +- 启动器在包中没有本机架构时，按表取本机执行的架构（POSIX 与 Windows 启动器同一数据）。 +- 内置数据：`linux-x86_64` 执行 `i686`。 +- 载体所在的行若为 planned，而构建它的平台以工具链列出该行，则该行可构建（与工程自身的构建同一规则）。 + ## 3. 提供者 | 键 | 语义 | @@ -166,6 +192,14 @@ 环境变量（SPEC-013 §4.5）。`hosts` 列出该运行器所在的宿主，空为任一宿主。模板中 `{}` 是产物，`{sysroot}` 是该安排的 基座，`{pkg:}` 是 `requires` 中该名字的包的根；模板的程序先在 `requires` 的包中查找，再查 PATH。`requires` 的包 在第一次 `mcpp run` 或 `mcpp test` 时按工具链的同一方式安装；构建不安装它们。 +- **提供者运行器的 `host` 与 `enabled`（v1.3）。** 提供者的 `runner` 取平台运行器的 `host`（`same-arch`、`any`）与 + `enabled`（SPEC-013 §10）。首次使用需取得整机镜像或模拟器、或在构建树与包仓库之外留下状态的运行器，数据**应当**陈述 + `enabled = false`：虚拟机、Android 模拟器与 Wine（在家目录建立 prefix）关闭；用户态模拟器（qemu-user）、okrun、node + 不关闭。内置数据：`x86_64-windows-gnu` 与 `aarch64-windows-gnu` 的 MinGW 提供者在 `linux-x86_64` 上陈述 + `{ cmd = ["wine", "{}"], host = "same-arch", requires = ["xim:wine@11.0"], enabled = false }`；`same-arch` 使 aarch64 的 + Windows 程序不经它运行。提供者的 `runner` 也可陈述 `exclusive`（该目标的运行不重叠）。`x86_64-linux-android` 的提供者在 + `linux-x86_64` 上陈述经 dirun 与 API 34 系统镜像包的配置运行的 runner，`same-arch`、`exclusive = true`、 + `enabled = false`；`aarch64-linux-android` 不陈述 runner（模拟器包不服务 aarch64 机器）。 - **运行器的身份（v1.2）。** 平台的 `runner` 可陈述 `built-as = { "-" = "<身份>" }`：为该宿主对构建的运行器程序 是哪个身份。自带运行器的包按它为各系统构建运行器；引擎只对它一向认识的三个系统保留约定，其余系统无此陈述即无程序。 - **`link.flags` 与链接驱动（v1.1）。** `[platform.link] flags` 只交给 ld.lld；`driver = "compiler"` 的平台陈述它时，构建 @@ -200,3 +234,4 @@ | 1.0 | 2026-11-09 | 初版：行、提供者、选择、平台定义的行、C 环境、执行、报告与锁 | | 1.1 | 2026-11-10 | `clang-triple`；内置行的系统词；FreeBSD 两种身份与 32 位 planned 行；提供者 sysroot 为基座陈述的最后一级；`mingw-llvm` 与 `aarch64-windows-gnu`；C 库由平台描述的行与内置行同一读法；打包读文件闭包；`link.flags` 的警告；构建程序的 `MCPP_KERNEL_ABI_INTERFACES` | | 1.2 | 2026-11-12 | 行的 `loader` 与 `execution` 为内置行所读；系统表（`uname`）；提供者的 `tier`、`system-headers`、`runner`；一条条目一种安排；请求的回答是契约；`glibc-sysroot`；一个执行谓词，另一系统的程序不被启动；程序解释器与基座包的单一读法；运行器的占位、`requires` 与 `built-as` | +| 1.3 | 2026-11-16 | mcpp 2026.11.16.1（设计 2026-11-15 G7、N16、N23、N26、N33、R1、R2、X1）：执行关系表（§2.4）；系统表的 `resources` 与 `command-scripts`（§2.3）；提供者运行器的 `host`、`enabled` 与 `exclusive`（§6）；Android 行的运行器（G7）；行的 `exceptions`（工程未陈述时的异常模型）；行的 `companions`；平台以工具链列出的 planned 行可作载体的行 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 7d0420e40..3b2e39abf 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -538,6 +538,11 @@ required_features = ["gui"] # only built when feature `gui` is > 分发包需要表达的意思。在 mcpp 2026.8.18.1 之前两者逐字节相同，所以 > 没有拼法能表达「什么都没有」，`src/` 下留下的任何文件都会被卷进来。 +> **不含模式的路径指名一个文件**（2026.11.16.1+）。不含 `*`、`?`、`[`、`{` 的 `sources` 条目指名一个 +> 文件，文件不存在时报错，并指出陈述它的表（`[build]`、`[target.''.build]`、`[features.]`）： +> 此前它被跳过，构建在导入该文件所提供模块的地方失败。取入存储的包由其使用者无法修改，其中的这类条目是警告。构建中 action 或构建程序写出的文件属于构建。 +> 匹配为空的模式不选中任何文件，一个目标上的选择可以为空。 + > **`**/` 代表整段目录。** 在 `sources` 以及沿用其文法的每一处 glob 中， > `**/` 匹配零个或多个目录，其后的部分从一个路径段的开头算起： > `**/main.cpp` 选中每个 `main.cpp`，而不选中 `src/domain.cpp`。`*` 只在 @@ -983,6 +988,7 @@ bmi_schedule = "off" # auto (default, = off) | on | off 使用 GCC 且 `bmi_schedule = "on"` 时，普通编译与分离的代码生成共用同一个 `jobs` 总上限 （2026.11.7.1+）。额外的调度和等待任务不增加编译器进程数量。 +链接共用同一上限（2026.11.16.1+）：同时就绪的链接按编译的并发数运行，而不是全部同时运行。 取消构建会结束本轮普通编译器和分离的编译器。紧接着的构建等待它们完成清理后 才启动新的编译器；未完成的输出会重新构建。已完成模块的 object 缺失时， 该次构建会重新生成它。输入和输出均不变的构建仍不进行编译。 @@ -1422,7 +1428,8 @@ runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } | `tools.` | 为本机构建一个工具的包，或 `{ program = "" }` | | `link` | `driver`、`flags`、`start`、`script`、`inputs`、`exe`、`shared`、`seal` | | `carrier` | `{ package, section, main }`：`--mode self-contained` 构建的程序 | -| `runner` | 一个 argv，或 `{ cmd, host, exclusive }`：`host = "same-arch"` 只在产物架构的机器上运行它 | +| `runner` | 一个 argv，或 `{ cmd, host, hosts, exclusive, requires, enabled, longlived }`：`host = "same-arch"` 只在产物架构的机器上运行它；不含 `cmd` 的表调整其下一层的 runner，如 `{ enabled = true }` 启用数据陈述 `enabled = false` 的 runner（2026.11.16.1） | +| `runners.` | 具名 runner 的同一种 argv 或表（表形式 2026.11.16.1） | `cfg(platform = "")` 按平台选择；`platform` 本身不能写在这样的选择器下。同样的键也从 `~/.mcpp/config.toml` 读取。平台包在 `[platform]` 下陈述它们，`[host] platform` 选择构建程序的平台。 @@ -1527,6 +1534,9 @@ cxxflags = ["-fno-plt"] ldflags = [] ``` +**标志列表的顺序。** `cflags`、`cxxflags` 与 `ldflags` 各自按 `[build]` 的、profile 的、`[target..build]` 的 +顺序进入命令：编译器或链接器取两个词中后者时，目标块的生效（2026.11.16.1 实测）。 + **cl.exe 上的调试信息（2026.11.7.1+）。**`debug = true` 以 `/Z7` 编译：每个对象携带自己的调试信息。 `/Zi` 与 `/ZI` 把全部编译的调试信息经由 mspdbsrv.exe 写入同一个 PDB；该服务在启动它的编译结束后继续存在， 并随那个编译的进程单元一同结束。包的标志中的 `/Zi`、`/ZI`、`/FS` 在编译开始前被拒绝。 @@ -1874,8 +1884,8 @@ icon = "assets/app.ico" | `version-info` | 布尔值 | `false` 退出自动生成版本资源 | | `[resources.version-info]` | 表 | `company`、`product`、`description`、`copyright`、`original-filename`、`internal-name` | -**只有 PE 目标会*编译*它。** 在 Linux 与 macOS 上，这一节*不适用*： -没有资源单元、没有诊断、构建逐字节相同。**不需要**（也不能用）一个 +**只有程序由 Windows 运行的目标会*编译*它**：执行系统为 Windows 的 PE 行。在其他目标上这一节*不适用*—— +Linux、macOS，以及由其他系统加载的 PE 行，如 `x86_64-uefi`（2026.11.16.1）：没有资源单元、没有诊断、构建逐字节相同。**不需要**（也不能用）一个 `cfg(windows)` 谓词——写一次，无条件生效即可。 资源属于拥有可执行程序或共享库的包。依赖的 executable artifact 与共享库在通过 diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index 8499eee91..d411f8f68 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -215,7 +215,7 @@ mcpplibs.os = { version = "0.4.4", override = true } - **生效位置。** 根的依赖表与 `[workspace.dependencies]`；后者管辖工作空间所有成员的计划（在那里声明一个包不会让成员依赖它）。`[dependencies]` 管辖构建的图与测试的图；`[dev-dependencies]` 只管辖测试的图；`[build-dependencies]` 只管辖构建程序的图。库写出它时被告知已忽略：这是工程的决定。 - **陈述。** 被替换且本不接受覆盖值的每条请求打印一次：`Overrode <包> <请求> (requested by <请求方>) -> <覆盖> (<表>)`。`mcpp.lock` 为该包记 `override = true`，`--locked` 把该标记视为记录的解析结果的一部分。 - **平台的版本组。** 覆盖版本组所列的包时照样生效，并给出警告，写出版本组的版本：平台给出默认值，工程可以选择另一个。 -- **何时需要。** 任何 SemVer 合并都无法调和的两个版本请求被拒绝时，会写出两者，并在末尾给出覆盖该包的那一行。库精确固定一个所有使用者共享的包（提供目标侧层或内核 ABI 的包）时，使用它的工程会得到提示，`mcpp publish` 拒绝库中的这种固定：范围把选择留给工程。 +- **何时需要。** 任何 SemVer 合并都无法调和的两个版本请求被拒绝时，会写出两者，并在末尾给出覆盖该包的那一行。两个来自 `[build-dependencies]` 的请求同样被拒绝，且不以重命名分开：构建依赖是构建程序的模块，一次构建只加载它的一个版本（2026.11.16.1）。库精确固定一个所有使用者共享的包（提供目标侧层或内核 ABI 的包）时，使用它的工程会得到提示，`mcpp publish` 拒绝库中的这种固定：范围把选择留给工程。 ### `path` 或 `git` 依赖的身份（mcpp 2026.9.14.2+） diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index 6c5167b6e..00d71455d 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -210,7 +210,8 @@ test result ok. 7 passed; 0 failed; finished in 7.77s (plan 2.10s · programs 1. **测试按名字找到包的程序**（2026.10.27.1+）。`mcpp test` 先按 `mcpp build` 的方式构建包的程序，并给每个测试 `MCPP_BIN_`：程序 `` 的路径，名字转为大写，字母与数字之外的字符写作 `_`（`my-tool` 为 -`MCPP_BIN_MY_TOOL`）。运行被测程序的测试从这里读取它。 +`MCPP_BIN_MY_TOOL`）。运行被测程序的测试从这里读取它。选中多个成员时，一个配置组的程序一起规划，与 +`mcpp build --workspace` 相同，因此它们共用的包以一种配置构建（2026.11.16.1+）。 `--no-programs`（2026.11.13.1+）不构建它们：程序只为得到名字而规划，每个测试从环境的 `MCPP_BIN_` 取得各程序，用于在别处构建的程序（一个 CI 作业测试另一个作业构建的二进制）。环境未指定的程序在任何测试运行前 @@ -414,7 +415,9 @@ mcpp test --guide # 框架、它们的指南，以及用例相关的 框架在其入口上陈述编写测试的指南，`test-guide = "<文件>"`（2026.11.8.1+；协议的 P2.5）：包中的一个 文件，从已安装的包读取，与其版本一致。`mcpp test --guide` 列出本包的每个测试框架及其指南与仓库，以及列出、 -选择、解释用例的命令；没有框架时，给出上文的契约与接入 mtest 的写法。失败的运行以 +选择、解释用例的命令；没有框架时，给出上文的契约与接入 mtest 的写法。框架是声明的数据：guide 从解析后的 +图（含 dev-dependencies）读取它们，在任何位置都不规划、不扫描、不构建（2026.11.16.1+）。选中多个成员时（workspace +根或 `--workspace`），列出所有成员的框架。失败的运行以 `how these tests are written: mcpp test --guide (...)` 结尾，`--message-format json` 的记录带有 `guide`。 将要编写测试、或要读懂测试为何失败的人或 agent，从这里开始。 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index db6b7f81f..b5aa45b1e 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -721,8 +721,10 @@ target/dist/hello-0.1.0-openkal/ 其余的腿逐条规划、一起构建（`Packed leg `）。启动器由平台的 `runner` 生成：POSIX 脚本以自身目录的上一级为 包的根，按 `uname -m`（`x86_64`、`amd64` → `x86_64`；`aarch64`、`arm64` → `aarch64`）选择架构目录，Git Bash、MSYS 与 -Cygwin 的 shell 交给 `bin/.exe`，以该目录中的映像与自身的参数启动运行器，先在 `libexec//-/` -中、再在 `PATH` 上查找运行器。Windows 启动器是为 `x86_64-windows-gnu` 构建的引擎的程序：从自身路径找到包，选本机 +Cygwin 的 shell 交给 `bin/.exe`，以该目录中的映像与自身的参数启动运行器。携带运行器的包启动 +`libexec//-/` 中的运行器，本机没有时以状态 126 退出，写出本机与包所含的系统；不携带运行器的包在 +`PATH` 上查找。`` 是系统表中 `uname -s` 一列与之相等的系统（`linux`、`macos`、`freebsd` 等）。`--all-programs` +时包中每个程序都有自己的一对启动器 `bin/` 与 `bin/.exe`。Windows 启动器是为 `x86_64-windows-gnu` 构建的引擎的程序：从自身路径找到包，选本机 架构的映像，以同样的方式启动运行器；不生成 `.cmd`。构建的平台为该行提供宿主安排时，启动器在 mcpp 的 openkal 行固定的 平台上构建，而不是项目选择的平台上，因为它使用的 `mcpplibs.os` 版本固定了该平台的 openkal。包中没有本机架构时以状态 126 退出，写出两者。 `[runtime] deploy` 放置的文件位于每条腿的映像旁边。 diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 54625640a..5b1425efc 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -1160,7 +1160,7 @@ WebAssembly 上，每个目标文件、预编译的标准库模块与链接必 | 成员 | 类型 | 渲染为 | 到达 | |---|---|---|---| | `threads` | 布尔 | 在既非 PE 也非 freestanding 的目标上为 `-pthread`；在 PE 与 freestanding 目标上不产生任何 flag | 标准库模块的预构建、依赖扫描、所有包的每个 C 与 C++ 翻译单元，以及链接 | -| `exceptions` *(mcpp 2026.9.12.3+)* | 布尔，或 `"wasm"` / `"js"` *(2026.10.30.1+)* | 只在 `os = "emscripten"` 上，经方言 flag 进入编译行，也进入链接行：`"wasm"` 为 `-fwasm-exceptions`，即 WebAssembly 自身的异常处理；`"js"` 与 `true` 为 `-fexceptions`，即基于 JavaScript 的模型。在其余每个目标上什么都不产生，因为那里异常本来就是默认开启的 | 与 `threads` 相同的那一套 | +| `exceptions` *(mcpp 2026.9.12.3+)* | 布尔，或 `"wasm"` / `"js"` *(2026.10.30.1+)* | 只在 `os = "emscripten"` 上，经方言 flag 进入编译行，也进入链接行：`"wasm"` 为 `-fwasm-exceptions`，即 WebAssembly 自身的异常处理；`"js"` 与 `true` 为 `-fexceptions`，即基于 JavaScript 的模型；`false` 表示不启用。未写时取行自身的模型：`wasm32-emscripten` 为 `"wasm"`（2026.11.16.1；此前不启用）。在其余每个目标上什么都不产生，因为那里异常本来就是默认开启的 | 与 `threads` 相同的那一套 | | `longjmp` *(mcpp 2026.11.12.1+)* | `"wasm"` 或 `"js"` | 只在 `os = "emscripten"` 上，进入每个 C 与 C++ 翻译单元的编译行与链接行：`"wasm"` 为 `-sSUPPORT_LONGJMP=wasm`，即 WebAssembly 自身的栈切换；`"js"` 为 `-sSUPPORT_LONGJMP=emscripten`，即 JavaScript 模型，也是工具链的默认。声明在其余任何行上都会被拒绝。成员缺省时不产生任何 flag | 与 `threads` 相同的那一套 | 三个成员都经由它们渲染的 flag 进入依赖缓存键，因此没有启用某个成员的构建，不会 diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index d13ff5a69..b2500add6 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -314,6 +314,11 @@ C 库**底下**—— 而必须落在一个 C 库**之上**。规范恰好允许 wasm-ld: error: obj/main.o: undefined symbol: kal_process_spawn ``` +**node 下的终端没有原始模式。** Emscripten 以同步读取读标准输入，node 对管道与文件提供它、对终端拒绝它：在终端上 +`read(0, ...)` 以 `EIO` 失败，无论是否调用 `tcsetattr`；`tcsetattr` 返回 0 且不改变任何东西；`isatty(0)` 在各种情况下 +都为 1。程序经管道读取输入（`printf 'keys' | mcpp run --target wasm32-emscripten`）；在 web 上需要逐键输入的程序在其页面中 +以键盘事件读取（以 emsdk 6.0.9、node 26 实测）。 + 这正是 6.2 条里的第二个时刻。若提供一个 `kal_process_spawn`、只让它返回 一个错误，会是规范所禁止的那种形状 —— 存在却永远失败，调用方无法把它同一个 条件区分开来 —— 并且会把一个在链接期就已知的事实，挪到运行期才能知道。 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index f0d8746f4..c553e11c5 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -92,7 +92,7 @@ mcpp build # compiles + runs build.mcpp, then builds the project | `mcpp:source=` *(0.0.100+)* | 把一份**既有**源文件选入构建（绝对路径，或相对包根）。下游效果与 `generated=` 相同；语义区别在于文件是程序*选中*的（tarball payload / vendored 源树）而非程序写出的——例如对大型源码包做 per-target 源选择 | | `mcpp:include-dir=` *(0.0.100+)* | 为本包自身 TU 增加一个**私有** include 目录（`-I`；绝对路径或相对包根，自动规范化）。取代过去 `cxxflag=-I` + `cflag=-I` 的双重裸发 | | `mcpp:include-dir-after=` *(0.0.100+)* | 同 `include-dir`，但排在系统目录**之后**搜索（`-idirafter`）——用于会遮蔽系统头的 payload 源树 | -| `mcpp:runner=` *(2026.8.19.2+)* | 执行本次构建产物的命令的**一个 argv token**（宿主跑不了它时）。一个 token 一次调用、按顺序；产物路径会被追加（或替换 `{}`）。**到达消费者**。可执行文件要发**绝对路径**，且**只能有一个**依赖提供它 | +| `mcpp:runner=` *(2026.8.19.2+)* | 执行本次构建产物的命令的**一个 argv token**（宿主跑不了它时）。一个 token 一次调用、按顺序；产物路径会被追加（或替换 `{}`）。**到达消费者**。写程序名（按 [41](41-devices.md) 所述查找），且**只能有一个**依赖提供它 | | `mcpp:link-flag=` *(2026.9.6.5+)* | 加一条本程序**算出来的**链接标志，原样传递。这是 `link-lib` / `link-search` / `link-script` 各自命名一类东西之后留下的出口：生成的版本脚本（`-Wl,--version-script=`）、运行时接管 C 库符号用的 `-Wl,--wrap=malloc`、以及 `-Wl,--exclude-libs,ALL`（静态吞入的第三方不得成为本包 ABI 的一部分）。按发出顺序追加在 `[build] ldflags` 之后。**到达消费者**，与 `[build] ldflags` 一致 —— 理由见下 | | `mcpp:windows-subsystem=:` *(2026.9.12.2+)* | 设置**本包**可执行目标 `` 的 PE 子系统（`console` 或 `windows`），与 `[targets.] windows_subsystem`（docs/04）是同一字段。只到达该目标的链接，不到达其他目标或消费者，在非 PE 目标上不产生任何标志。本包未以 `kind = "bin"` 声明该目标、取值不在集合内、取值与 mcpp.toml 的声明矛盾，这三种情形都在应用任何指令之前被拒绝 | | `mcpp:windows-entry=:` *(2026.9.12.2+)* | 设置可执行目标 `` 的入口函数（`main`、`wmain`、`WinMain` 或 `wWinMain`），与 `windows_entry` 是同一字段；作用域与拒绝条件同 `windows-subsystem` | @@ -171,6 +171,7 @@ int main() { | `mcpp::seal(step, token)` *(协议 17)* | `mcpp:seal=` —— 封装步骤 `step` 的一个 token，在链接出的文件（`{}`）发布之前运行；即 `link.seal` | | `mcpp::run_host(h)` *(协议 17)* | `mcpp:run-host=` —— `"same-arch"` 或 `"any"`；即 `runner.host` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` —— 见下 | +| `mcpp::runner_spec` *(协议 19)* | `mcpp:runner-spec=` —— 一条陈述的 runner，与数据中的表逐字段对应；见下 | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | `[xlings.workspace]` 里声明的包的载荷目录 —— 本 manifest 声明的，或编进本构建程序的某个依赖声明的（2026.9.6.6+）；没声明或没安装时返回 `""`（见下） | | `mcpp::xpkg_source(ns, name)` / `mcpp::xpkg_program(ns, name)` *(协议 15)* | 该载荷从哪里来 —— `payload`、`override`、`pending` 或 `""` —— 以及一条覆盖点名的程序 | | `mcpp::xpkg_request(ns, name)` / `mcpp::xpkg_pending()` *(协议 15)* | 一个声明了 `provision = "on-request"` 的载荷的目录；尚未安装时请求它，此时 `xpkg_pending()` 为真，程序应当返回（见下） | @@ -258,8 +259,9 @@ for (auto a : {"-machine","virt","-nographic","-no-reboot","-kernel"}) 调试时把 `-bios default` 换成 `-bios none -semihosting` 是正当需求 —— 且 mcpp 会说明 它覆盖了哪个依赖。 -**可执行文件要发绝对路径。** 裸名会经 `PATH` 解析到一个 shim，而 shim 按**拥有它 -的 home** 派发，那未必是本次构建用的 home。 +**写程序名，不写路径。** 首个词先在 runner 的 `requires` 的包中查找，再在图中各载荷的 `bin/` 中查找，最后查 `PATH`， +不查当前目录（[41 —— 抵达一台设备](41-devices.md)）。不在包的 `bin/` 下的程序写作 `{pkg:}/<路径>`，包写入 +`requires`（见下文 `runner_spec`），在运行开始时解析；构建时拼出的绝对路径把 home 与版本固化在陈述中。 **只能有一个依赖提供 runner。** 两个板级支持包都声称知道怎么跑这个产物是配置 错误；mcpp 会**同时点名两个**并报错，而不是把它们并成一个谁也不是的 argv。 @@ -270,7 +272,32 @@ runner，按发出顺序，无论调用来自这个程序的哪个库。要在 之间选择的程序，在发出任何一个之前完成选择。以打包格式命名的 runner 在 `mcpp run --pkg-format ` 下抵达该格式的可分发物（[41 —— 抵达一台设备](41-devices.md)）。 -### 问，而不是声明：`toolchain_dir` / `sysroot_dir`（2026.8.19.4+） +### `runner_spec` —— 一条陈述的 runner（2026.11.16.1+，协议 19） + +数据中的 runner 是一张表（`runner = { cmd = [...], requires = [...], ... }`，[41](41-devices.md)）；`mcpp::runner_spec` +是构建程序中的同一张表，逐字段对应，作为一条指令提交： + +```cpp +mcpp::runner_spec web; +web.name = "web"; // 空：缺省 runner +web.longlived = true; // 没有自然结束 +web.arg("{pkg:emsdk}/emscripten/emrun").arg("{}") + .require("xim:emsdk"); +web.submit(); +``` + +| 表的键 | `runner_spec` | +|---|---| +| `cmd` | `.arg(tok)`，与 `action.arg()` 相同 | +| `longlived`、`exclusive`、`enabled` | 同名字段（`enabled` 缺省为 `true`） | +| `host` | 同名字段（`"same-arch"`、`"any"`） | +| `hosts` | `.hosts(pair)`，每次一个 `-` | +| `requires` | `.require(pkg)`：`requires` 是 C++ 关键字 | + +`{}` 是产物，`{pkg:}` 是 `require()` 中该包的根，由第一次使用该 runner 的 `mcpp run` 或 `mcpp test` 安装，构建 +不安装。一个 runner 只用一种方式陈述：同一名字既由 `runner_spec` 又由逐词调用陈述、同一名字提交两次、没有命令、或 +`host` 不是两个词之一时，在应用任何指令之前拒绝。逐词调用保持原义；覆盖、`enabled` 与每个名字一个提供者遵循数据的规则。 +：`toolchain_dir` / `sysroot_dir`（2026.8.19.4+） ```cpp const char* tc = mcpp::toolchain_dir(); // the resolved toolchain's payload root @@ -882,6 +909,17 @@ int main() { 所有人，这个集合变成不可知的。对一个谁都没为之提交的格式，mcpp 会拒绝，而不是报告一次 「什么包都没产出」的成功打包。 +**提供者从哪个程序组装产物。** 只包装被打包的包中一个程序的提供者，在声明旁无条件地陈述是哪一个： + +```cpp +mcpp::provides_pack_format("web"); +mcpp::pack_program("app"); // 协议 19，mcpp 2026.11.16.1 +``` + +此后未指名程序的 `mcpp pack --pkg-format web` 打包 `app`；命令行指名的程序优先；陈述的程序不在本次构建中时， +拒绝消息指名提供者。提供者在第二遍中什么也没有提交时，拒绝消息列出它在这一遍中陈述的警告（`mcpp::warning`、 +`mcpp::report`）作为原因。因此提供者以这两者说明为何不提交，而不写标准错误：mcpp 只在程序失败时显示标准错误。 + **`mcpp pack --pkg-format ` 会 prepare 两次。** 一条 `artifact` action 是一条 ninja 边，而那棵暂存树是 mcpp 在链接**之后**产出的，所以这棵树不可能成为构建出它自己 那一次 pass 的输入。第一次 pass 收集声明，并在任何东西被编译之前拒绝未知的格式；随后 diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 75a43333d..d59b3868f 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -86,8 +86,8 @@ runner = ["my-okrun", "{}"] |---|---| | 编译标志、`link.flags` | 追加在平台的之后;约定不能删除 | | `link.inputs`、`link.seal` | 整体替换 | -| `layers`、`tools`、`runners` | 按键替换;` = false` 不提供该层 | -| `runner` | argv 替换;未写出时保留 `host` 与 `exclusive` | +| `layers`、`tools` | 按键替换;` = false` 不提供该层 | +| `runner`、`runners` 的每个名字 | `cmd`、`requires` 与 `enabled` 一起替换,`cmd` 被替换的 runner 未写出时为启用;`host`、`hosts`、`exclusive`、`longlived` 未写出时保留;`config.toml` 或工程中不含 `cmd` 的表只改它写出的键(2026.11.16.1) | | 其余字段 | 替换 | 同样的表也可以写在 `~/.mcpp/config.toml` 中,对一台机器生效。图中已有包提供的层交给该包,报告为 `graph`。 @@ -261,6 +261,11 @@ hosted = true # C 环境有 `main` 未运行」（docs/41）。行的机器描述属于平台：机器与内置行不同的目标是另一行。`[platform]` 与 `[platform.hosted]` 仍是 提供者的两种简写。 +**行的附带文件与异常模型（2026.11.16.1+）。** `companions = [".wasm"]` 指链接可能在程序旁写出、以程序命名的文件：每次链接前 +删除，链接写出时被暂存，未写出不是错误。`exceptions` 是工程未陈述 `[target..abi] exceptions` 时该行构建的异常模型， +为 `"wasm"` 时 long-jump 模型亦为 `"wasm"`（内置的 `wasm32-emscripten` 行为 `"wasm"`）。机器只在执行关系表陈述时运行另一架构的程序（`[[platform.executes]]`， +SPEC-016 §2.4）；此时 `same-arch` 的 runner 为产物的架构构建，在本机启动。 + **定义的行与内置行同一读法（2026.11.10.1+）。** 它的 `march` 与 `link-emulation` 进入编译与 `ld.lld` 链接（`mcpp why flags` 标为 `row`）；工具链载荷为其自身宿主构建的东西——libc++ 头文件、运行时目录、宿主的 `-L` 与 RPATH——不进入该行；本机加载器的 闭包与符号检查只在该行的执行系统为本机时进行。没有内置提供者服务的内置行同样如此。 @@ -325,7 +330,8 @@ uname = "Haiku" # `uname -s` 在该系统上的 （[SPEC-016 §4](../specs/targets-and-providers.md)）。 陈述了 `uname` 的系统可进入每个系统一个程序的单文件（`mcpp pack --pkg-format single-file`）；未陈述的系统，其程序是 -文件开头的 PE。运行器为多个系统构建的平台，陈述各自的身份： +文件开头的 PE。加载器读取程序资源节的系统陈述 `resources = true`（内置的 `windows` 条目）：工程的 `[resources]` 编入其行的 +程序，不编入另一系统加载的 PE 映像，如 UEFI 应用（2026.11.16.1）。运行器为多个系统构建的平台，陈述各自的身份： ```toml [platform] diff --git a/docs/zh/41-devices.md b/docs/zh/41-devices.md index d41ac2cb2..3fcca1fc7 100644 --- a/docs/zh/41-devices.md +++ b/docs/zh/41-devices.md @@ -61,13 +61,16 @@ mcpp::runner_longlived("monitor"); // no natural end mcpp::run_exclusive(); // this target's runs cannot overlap ``` +`mcpp::runner_spec` 以数据中的表陈述一个 runner，含 `requires` 与 `enabled`（2026.11.16.1+，[30](30-build-mcpp.md)）。 + **写程序名，不要写路径。** mcpp 会定位它：先找**图中任何一个包**在 `[xlings.workspace]` 下声明的载荷的 `bin/`（消费工程优先，然后是它的 依赖），再找 `PATH`。板级包正是那个知道哪个模拟器或探针能抵达这台机器 的东西，所以由它自己声明该载荷，**消费者什么都不用声明**。用 `mcpp::xpkg_dir` 拼出绝对路径是多余的，而且会引入一个失败模式——声明 不是安装，查询可能返回空，于是没有配置任何 runner，也没有任何话说明 -原因。写程序名则让 mcpp 能报出它究竟搜过哪些目录。 +原因。写程序名则让 mcpp 能报出它究竟搜过哪些目录。不在载荷 `bin/` 下的程序写作 `{pkg:}/<路径>`， +其包列入 runner 的 `requires`，并最先被查找。 ## 工程的覆盖项 @@ -97,6 +100,41 @@ runner 能从任何机器抵达产物。给出执行宿主（`run_under`）时 [24](24-openkal-cross.md) 中的 `run_under = ["qemu-aarch64"]`）。目标的平台提供它的缺省 runner（[35](35-platforms.md)）；依赖的构建程序、本机的 `config.toml`、工程与 `MCPP_TARGET__RUNNER` 依次替换它，只写 argv 时保留平台的 `host` 与 `exclusive`。 +**按需启用的 runner（2026.11.16.1+）。** 首次使用需取得整机镜像或模拟器、或在构建树与包仓库之外留下状态的 runner， +由其数据陈述 `enabled = false`：虚拟机、Android 模拟器，以及 x86_64 Linux 上 Windows 行的 Wine。`mcpp run` 以状态 2 +拒绝它，`mcpp test` 使测试已构建而未运行。消息带有 token `[runner-not-enabled]`、首次使用安装的包与两种启用写法； +启用之前不安装任何包。 + +```toml +[target.x86_64-windows-gnu] +runner = { enabled = true } # mcpp.toml，或本机的 config.toml +``` + +不含 `cmd` 的表调整其下一层的 runner。`cmd`、`requires` 与 `enabled` 是一条陈述：替换 `cmd` 的层陈述另一个 runner， +除非写出否则为启用，因此已写 `runner = ["wine", "{}"]` 的工程行为不变。`MCPP_TARGET__RUNNER_ENABLED=true` +为一次调用启用缺省 runner，`mcpp run --runner ` 启用它指名的 runner。具名 runner 取同一种表 +（`[target..runners] web = { cmd = [...], longlived = true }`），按同一规则合并。 + +**Android 程序在模拟器中运行（2026.11.16.1+）。** 在 x86_64 Linux 机器上启用后，`x86_64-linux-android` 经 dirun +（>= 0.4.0）与 API 34 系统镜像包的配置运行：模拟器启动一台新设备，程序与其运行时文件经 adb 推送，输出与状态返回。 +设备只有一个控制台端口，因此该目标的运行不重叠（`exclusive`）。aarch64 行没有 runner：模拟器包不服务 aarch64 机器。 + +```bash +MCPP_TARGET_X86_64_LINUX_ANDROID_RUNNER_ENABLED=true mcpp run --target x86_64-linux-android +``` + +## runner 所指名的程序（2026.11.16.1+） + +| | 规则 | +|---|---| +| 查找 | 先 `requires` 的包，再图中各载荷的 `bin/`，最后 `PATH`；不查当前目录 | +| 不在 `bin/` 下的程序 | `{pkg:}/<路径>`，其包列入 `requires` | +| Windows 上无后缀的名字 | 取系统的后缀 `.exe`、`.cmd`、`.bat`，裸名字与带目录的名字相同（`../tools/x` 即 `../tools/x.cmd`）；无后缀的文件在那里不是程序 | +| 执行 | argv，不经 shell | +| 系统经命令解释器启动的脚本 | Windows 上的 `.bat` 与 `.cmd`（系统表的 `command-scripts`）：参数含 `%`、`^`、`&`、`\|`、`<`、`>`、`"` 或换行时在启动前拒绝，因为解释器不论怎样引用都把它们读作语法（CVE-2024-24576 一类） | +| 占位符 | `{}`、`{sysroot}` 与 `{pkg:}`，可位于词中（`{}/index.html`）；其余花括号原样保留；没有 `{}` 时产物附在末尾 | +| 状态 | runner 的状态即运行的状态；长驻 runner 的状态不是结论，`mcpp test` 不使用它 | + ## runner 收到的内容 runner 收到一段 argv——它自己的 token，产物路径被追加或替换 `{}`—— @@ -171,6 +209,7 @@ mcpp run --pkg-format app --runner other # a typed --runner still wins |---|---| | default | runs to completion; the exit code is the verdict | | `runner_longlived(name)` | has no natural end; the operator ends it | +| runner 表中的 `longlived = true` | 同上，以数据陈述（2026.11.16.1+） | `openocd -c "program image.elf verify reset exit"` 会终止， `openocd -c "init"` 不会，两者的拼写直到包所选的那个参数为止都一样。 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 6edbe398c..ef7d80127 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -253,7 +253,7 @@ mcpp cache list --json -> {"root": …, "entries": [ … ]} | `0`–`124` | 程序运行过了；这是它自己的状态，原样透传。例外是 `101`，见下一行 | | `101` | `mcpp run` 无法构建程序：规划或构建失败，没有启动任何东西。这是 `cargo run` 构建失败时 Cargo 给出的状态 | | `125`–`127` | 尝试过启动但被拒绝 —— `127` 是找不到，`126` 是找到了但不可执行，`125` 是其他原因；对 `mcpp run --pkg-format ` 而言，一个是目录、且没有任何 runner 能到达的分发物同样得到 `126`，它在启动之前就被拒绝，含义相同（2026.9.14.2+） | -| `2` | mcpp 在构建任何东西之前拒绝了请求，或在构建之后、启动任何东西之前拒绝了请求：用法错误、`--pkg-format` 与 `--no-runner` 同用、项目没有声明的程序或 runner | +| `2` | mcpp 在构建任何东西之前拒绝了请求，或在构建之后、启动任何东西之前拒绝了请求：用法错误、`--pkg-format` 与 `--no-runner` 同用、项目没有声明的程序或 runner、未启用的 runner（消息带 `[runner-not-enabled]`；2026.11.16.1+） | 在 2026.9.30.2 之前，构建失败退出 `1`，规划失败退出 `2`，脚本因此无法区分 一次编译错误和一个返回 `1` 的程序。现在构建失败是 `101`；`mcpp build`、 @@ -356,7 +356,7 @@ mcpp self env --format json `runner.program` 为该工具，否则为本机找到的程序。对目标图（2026.10.29.1+），该程序就是 `mcpp run` 与 `mcpp test` 会启动的程序：`runner.arch` 为其架构；产物属于另一架构、运行器为 `same-arch` 且给出了执行宿主时， 它是平台为该架构准备的运行器，报告第一次运行构建它的位置（此前 `built: false`；`self env` 不构建任何东西）。 -`runUnderProgram` 为执行宿主的程序，`runner.refusal` 为本机拒绝运行的原因（不拒绝时为 `null`）。`--host` 时，工程的构建程序已在宿主平台下构建过，才报告宿主平台 +`runUnderProgram` 为执行宿主的程序，`runner.refusal` 为本机拒绝运行的原因（不拒绝时为 `null`；未启用的 runner 亦是，带 `[runner-not-enabled]`）。`--host` 时，工程的构建程序已在宿主平台下构建过，才报告宿主平台 的 `runner` 与 `tools`。工程源码会被拒绝的原因 （图中没有 C++ 运行时而 `import std`）是一条代码为 `MCPP_CONFIGURATION_SOURCES_REFUSED` 的 `warning` 诊断，配置照常报告。无法准备的配置带 `MCPP_CONFIGURATION_FAILED`，没有 `data`。 diff --git a/mcpp.toml b/mcpp.toml index 53584709b..0934aeb2c 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.11.15.1" +version = "2026.11.16.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index 70daf5466..9c28a220c 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -206,6 +206,17 @@ enum class Slot : std::size_t { ArtifactNames, Seal, RunHost, + // Protocol 19 (design 2026-11-15 N30): the program of the package being + // packed that a provider of a pack format assembles its distributable + // from. `mcpp pack --pkg-format ` with no program named packs the + // one the format's provider stated. A name, carried and compared with the + // packed member's programs at pack time: the provider is a dependency and + // names a program of the package that depends on it. + PackPrograms, + // Protocol 19 (design 2026-11-15 R2): one runner, default or named, as the + // table it is in data -- `cmd`, `host`, `hosts`, `requires`, `enabled`, + // `longlived`, `exclusive` -- in one JSON payload (`mcpp::runner_spec`). + RunnerSpecs, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -304,7 +315,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -438,6 +449,11 @@ inline constexpr std::array kTable{{ // newer entry already discards the whole record through the unknown-tag // path. {"pack-format", "pack-format", Slot::PackFormats, Scope::Claim, Transform::Verbatim, false, "", "", 9}, + // v19 (N30). Collected on every pass, as `pack-format` is and for its + // reason: program selection reads it from the build that precedes the + // packaging pass. + {"pack-program", "pack-program", Slot::PackPrograms, Scope::Claim, Transform::Verbatim, false, "", "", 19}, + {"runner-spec", "runner-spec", Slot::RunnerSpecs, Scope::RunGlobal, Transform::Verbatim, false, "", "", 19}, // v10 (#618). The value names a target, and `target_directive_error` // refuses a name this package does not declare as an executable before // anything is applied. Persisted like every row but the re-run keys, so a @@ -690,6 +706,11 @@ std::string target_directive_error(const mcpp::manifest::Manifest& m, const Dire // `apply` on both the run path and the cache-hit path, and the message names // both the directive and the declaring package, like `target_directive_error`. std::string deploy_directive_error(const mcpp::manifest::Manifest& m, const Directives& d); +// A `runner-spec` payload that is not a runner (R2): not JSON, cut short, no +// command, an unknown `host`, a name with a colon, a name stated twice, or a +// name the token calls (`runner`, `runner-named`) state as well. Checked +// before anything is applied. +std::string runner_spec_error(const Directives& d); // Resolve an action's paths against `pkgRoot`. // @@ -1105,6 +1126,38 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { } for (auto const& name : d.at(Slot::RunnerLongLived)) bc.namedRunners[name].longLived = true; + // A runner as one statement (`mcpp::runner_spec`, R2), checked by + // `runner_spec_error`. Its command is appended as the tokens are, so one + // supplier per runner is decided by the same rule. + for (auto const& payload : d.at(Slot::RunnerSpecs)) { + auto j = nlohmann::json::parse(payload, nullptr, false); + if (j.is_discarded() || !j.is_object()) continue; + auto words = [&](const char* key) { + std::vector out; + if (auto it = j.find(key); it != j.end() && it->is_array()) + for (auto const& w : *it) if (w.is_string()) out.push_back(w.get()); + return out; + }; + const auto name = j.value("name", std::string()); + const auto cmd = words("cmd"); + mcpp::manifest::NamedRunner fields; + fields.requires_ = words("requires"); + fields.hosts = words("hosts"); + fields.host = j.value("host", std::string()); + fields.enabled = j.value("enabled", true); + fields.exclusive = j.value("exclusive", false); + fields.longLived = j.value("longlived", false); + if (name.empty()) { + bc.runner.insert(bc.runner.end(), cmd.begin(), cmd.end()); + if (fields.exclusive) bc.runExclusive = true; + bc.runnerFields = std::move(fields); + } else { + auto& nr = bc.namedRunners[name]; + fields.argv = nr.argv; + fields.argv.insert(fields.argv.end(), cmd.begin(), cmd.end()); + nr = std::move(fields); + } + } // ANY non-empty value sets it, and nothing can unset it. Exclusivity is a // claim about the DEVICE: if one package knows the target is a mutex, it is // one, and a later package saying nothing must not relax that. @@ -1161,6 +1214,15 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { // engine than a language does, and Slang is already supported without one. for (auto const& f : d.at(Slot::PackFormats)) bc.packFormats.push_back(f); + for (auto const& p : d.at(Slot::PackPrograms)) + bc.packPrograms.push_back(p); + // What the program said, kept with the declaration it explains: a pack + // whose provider submitted nothing names these in its refusal (N30). + for (auto const& sd : stated_diagnostics(d)) + if (sd.severity != "note") + bc.statedReasons.push_back(sd.message); + for (auto const& w : d.at(Slot::Warnings)) + bc.statedReasons.push_back(w); // Sources (mcpp#755). Carried, not interpreted: prepare turns decisions // into the decision record, requests into one installation, and the @@ -1489,6 +1551,38 @@ std::string target_directive_error(const mcpp::manifest::Manifest& m, const Dire return {}; } +std::string runner_spec_error(const Directives& d) { + std::set seen; + for (auto const& payload : d.at(Slot::RunnerSpecs)) { + auto j = nlohmann::json::parse(payload, nullptr, false); + if (j.is_discarded() || !j.is_object()) + return std::format("build.mcpp emitted `mcpp:runner-spec={}`, which is not a runner_spec payload.", payload); + const auto name = j.value("name", std::string()); + const auto shown = name.empty() ? std::string("the default runner") : std::format("the runner '{}'", name); + if (j.value("overflow", false)) + return std::format("build.mcpp's runner_spec for {} was cut short: the allocator refused a list.", shown); + if (name.find(':') != std::string::npos) + return std::format("build.mcpp's runner_spec names '{}'; a runner's name has no colon.", name); + if (auto it = j.find("cmd"); it == j.end() || !it->is_array() || it->empty()) + return std::format("build.mcpp's runner_spec for {} has no command: call arg() for each word.", shown); + if (auto h = j.value("host", std::string()); !h.empty() && h != "same-arch" && h != "any") + return std::format("build.mcpp's runner_spec for {} states host \"{}\"; it is \"same-arch\" or \"any\".", + shown, h); + if (!seen.insert(name).second) + return std::format("build.mcpp submitted a runner_spec for {} twice; a runner is one statement.", shown); + const bool tokens = name.empty() + ? !d.at(Slot::Runner).empty() + : std::ranges::any_of(d.at(Slot::NamedRunner), [&](std::string const& e) { + return e.starts_with(name + ":"); + }); + if (tokens) + return std::format( + "build.mcpp states {} both with runner_spec and with mcpp::runner(...); a runner is " + "stated one way.", shown); + } + return {}; +} + std::string deploy_directive_error(const mcpp::manifest::Manifest& m, const Directives& d) { for (auto const& entry : d.at(Slot::Deploy)) { const auto tab = entry.find('\t'); diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index e270099f5..8332a3050 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -128,7 +128,12 @@ export namespace mcpp::build::program_protocol { // spelling. // v18: optional action.progress; runtime snapshots carry no graph inputs. // Old action payloads default to progress=false. -inline constexpr int kProtocolVersion = 18; +// v19 (design 2026-11-15 N30): `mcpp:pack-program=` -- the program a pack +// format's provider assembles its distributable from, which `mcpp pack` packs +// when no program is named. Same cost as v5's: a package calling +// `mcpp::pack_program()` fails on an older engine at the build.mcpp COMPILE. +// No directive of v18 changes spelling. +inline constexpr int kProtocolVersion = 19; // ── Cache-format epoch ───────────────────────────────────────────────────── // diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index ae0798d59..3fbdc33e5 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -126,7 +126,21 @@ struct Runner { std::map builtAs; // The packages the runner is installed from on the first run. std::vector requires_; + // WHETHER THE RUNNER IS USED (R1, design 2026-11-15 §5.1). A runner whose + // first use fetches a whole machine image or an emulator, or leaves state + // outside the build tree and the package store, is stated `enabled = false` + // by its data: `mcpp run` and `mcpp test` refuse it and say how to enable + // it. It belongs to the command, as `requires` does: a statement that + // replaces `cmd` states it anew (true unless written). + bool enabled = true; + bool enabledSet = false; + // THE RUNNER DOES NOT END BY ITSELF (R2): a server an operator stops. Its + // status is not a verdict, and `mcpp test` does not use it. + bool longlived = false; + bool longlivedSet = false; Origin origin; + // `cmd` is stated. An override table may state `enabled` alone (`runner = + // { enabled = true }`), which adjusts the runner below it. bool set = false; }; @@ -247,9 +261,9 @@ struct Hosted { struct RowSpec { std::string triple; std::string tier = "preview", note, clangTriple, gccTriple, objectFormat = "elf", family, dataModel, - exeSuffix, sharedSuffix, loader, execution; + exeSuffix, sharedSuffix, loader, execution, exceptions; std::string march, mabi, codeModel, multilib, linkEmulation; - std::vector flags, descriptorKeys; + std::vector flags, descriptorKeys, companions; bool standard = true, exact = false; Origin origin; }; @@ -283,6 +297,8 @@ struct ProviderSpec { // A SYSTEM (`[[platform.system]]`, SPEC-016 §2.3). struct SystemSpec { std::string name, uname; + bool resources = false; + std::vector commandScripts; Origin origin; }; @@ -293,7 +309,8 @@ struct Statement { Text sysroot; std::vector sysrootProvides; Runner runner; - std::map runners; + // The named runners (`runners.`), each the same table as `runner`. + std::map runners; // Every carrier entry, in the order stated; `carrier_for` picks one. std::vector carriers; // The descriptor blocks the platform's targets read, in order (design @@ -375,13 +392,13 @@ inline constexpr std::string_view kPlatformKeys[] = { inline constexpr std::string_view kDefineKeys[] = { "triple", "tier", "note", "standard", "exact", "clang-triple", "gcc-triple", "object-format", "family", "data-model", "exe-suffix", "shared-suffix", "loader", "execution", "march", "mabi", - "code-model", "multilib", "flags", "link-emulation", "descriptor-keys", + "code-model", "multilib", "flags", "link-emulation", "descriptor-keys", "companions", "exceptions", }; inline constexpr std::string_view kProviderKeys[] = { "name", "serves", "priority", "hosts", "pairs", "toolchain", "sysroot", "linkage", "base", "hosted", "layers", "link", "execution", "tier", "system-headers", "runner", }; -inline constexpr std::string_view kSystemKeys[] = { "name", "uname" }; +inline constexpr std::string_view kSystemKeys[] = { "name", "uname", "resources", "command-scripts" }; inline constexpr std::string_view kBuildKeys[] = { "cflags", "cxxflags", "asmflags", "ldflags" }; inline constexpr std::string_view kLinkKeys[] = { "driver", "flags", "start", "script", "inputs", "exe", "shared", "seal", diff --git a/modules/manifest/src/platform_providers.cppm b/modules/manifest/src/platform_providers.cppm index fbffb7c7f..ea98963b5 100644 --- a/modules/manifest/src/platform_providers.cppm +++ b/modules/manifest/src/platform_providers.cppm @@ -25,6 +25,9 @@ struct StatedProvider { bool shorthand = false; // `[platform]` or `[platform.hosted]` std::string tier, systemHeaders; std::vector runner, runnerHosts, runnerRequires; + std::string runnerHost; + bool runnerEnabled = true; + bool runnerExclusive = false; }; // `name` is the platform's name (`platform_name`), the name of every provider @@ -86,7 +89,8 @@ std::vector stated_providers(const Statement& s, std::string_vie for (auto const& p : s.providers) out.push_back({ p.name.empty() ? std::string(name) : p.name, p.serves, p.hosts, p.pairs, p.priority, p.toolchain.empty() ? s.toolchain.value : p.toolchain, p.sysroot, p.linkage, false, - p.tier, p.systemHeaders, p.runner.cmd, p.runner.hosts, p.runner.requires_ }); + p.tier, p.systemHeaders, p.runner.cmd, p.runner.hosts, p.runner.requires_, + p.runner.host, p.runner.enabled }); if (s.targets.set) out.push_back({ std::string(name), s.targets.value, {"*"}, {}, s.prioritySet ? s.priority : 100, s.toolchain.value, {}, {}, true }); diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index 544f07b64..35abaf27b 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -37,7 +37,11 @@ struct Reader { std::expected ref(const mcpp::libs::toml::Value& v, std::string_view where, bool allowOff, bool allowProgram) const; - std::expected runner(const mcpp::libs::toml::Value& v, std::string_view where) const; + // `overriding`: the table of an override tier (this machine's config.toml, + // the project), which may adjust the runner below it without a command + // (`{ enabled = true }`). + std::expected runner(const mcpp::libs::toml::Value& v, std::string_view where, + bool overriding = false) const; std::expected flags(const mcpp::libs::toml::Table& t, std::string_view where, Flags& out, @@ -238,7 +242,8 @@ Reader::ref(const toml::Value& v, std::string_view where, bool allowOff, bool al return r; } -std::expected Reader::runner(const toml::Value& v, std::string_view where) const { +std::expected Reader::runner(const toml::Value& v, std::string_view where, + bool overriding) const { Runner r; r.origin = at(v, std::string(where)); r.set = true; @@ -263,6 +268,14 @@ std::expected Reader::runner(const toml::Value& v, std::str if (!val.is_bool()) return std::unexpected(std::format("{} must be true or false", key)); r.exclusive = val.as_bool(); r.exclusiveSet = true; + } else if (k == "enabled") { + if (!val.is_bool()) return std::unexpected(std::format("{} must be true or false", key)); + r.enabled = val.as_bool(); + r.enabledSet = true; + } else if (k == "longlived") { + if (!val.is_bool()) return std::unexpected(std::format("{} must be true or false", key)); + r.longlived = val.as_bool(); + r.longlivedSet = true; } else if (k == "hosts") { auto h = host_pairs(val, key); if (!h) return std::unexpected(h.error()); @@ -287,13 +300,19 @@ std::expected Reader::runner(const toml::Value& v, std::str } else { warn(std::format( "{}: '{}' is not a key of a runner (ignored); the keys are cmd, host, hosts, " - "exclusive, requires, built-as", where, k)); + "exclusive, requires, enabled, longlived, built-as", where, k)); } } } else { return std::unexpected(std::format( "{} must be an array of strings or {{ cmd = [...], host = \"same-arch\" }}", where)); } + // An override table adjusts the runner below it: without `cmd` it states + // the keys it writes and replaces no command. + if (r.cmd.empty() && overriding && v.is_table() && !v.as_table().contains("cmd")) { + r.set = false; + return r; + } if (r.cmd.empty()) return std::unexpected(std::format( "{} has an empty command; an empty template would run nothing and report success", where)); @@ -587,11 +606,11 @@ std::expected Reader::define(const toml::Table& t, std::st {"gcc-triple", &r.gccTriple}, {"object-format", &r.objectFormat}, {"family", &r.family}, {"data-model", &r.dataModel}, {"exe-suffix", &r.exeSuffix}, {"shared-suffix", &r.sharedSuffix}, {"loader", &r.loader}, - {"execution", &r.execution}, {"march", &r.march}, {"mabi", &r.mabi}, + {"execution", &r.execution}, {"exceptions", &r.exceptions}, {"march", &r.march}, {"mabi", &r.mabi}, {"code-model", &r.codeModel}, {"multilib", &r.multilib}, {"link-emulation", &r.linkEmulation}}) if (auto e = text(key, *to); !e) return std::unexpected(e.error()); for (auto [key, to] : std::initializer_list*>>{ - {"flags", &r.flags}, {"descriptor-keys", &r.descriptorKeys}}) + {"flags", &r.flags}, {"descriptor-keys", &r.descriptorKeys}, {"companions", &r.companions}}) if (auto it = t.find(key); it != t.end()) { auto w = strings(it->second, std::format("{}.{}", where, key)); if (!w) return std::unexpected(w.error()); @@ -721,13 +740,11 @@ std::expected Reader::statement(const toml::Table& t) co s.runner = std::move(*r); } else if (k == "runners") { if (!v.is_table()) - return std::unexpected(std::format("{} must be a table of = [argv]", key)); + return std::unexpected(std::format("{} must be a table of = [argv] or = {{ cmd = [argv] }}", key)); for (auto const& [rk, rv] : v.as_table()) { - auto rkey = std::format("{}.{}", key, rk); - auto w = strings(rv, rkey); - if (!w) return std::unexpected(w.error()); - if (w->empty()) return std::unexpected(std::format("{} is empty", rkey)); - s.runners[rk] = Words{ std::move(*w), at(rv, rkey), true }; + auto r = runner(rv, std::format("{}.{}", key, rk)); + if (!r) return std::unexpected(r.error()); + s.runners[rk] = std::move(*r); } } else if (k == "carrier") { auto c = carriers(v, key); @@ -799,6 +816,18 @@ std::expected Reader::statement(const toml::Table& t) co where, sk, list_of(kSystemKeys))); continue; } + if (sk == "command-scripts") { + auto w = strings(sv, std::format("{}.command-scripts", where)); + if (!w) return std::unexpected(w.error()); + sys.commandScripts = std::move(*w); + continue; + } + if (sk == "resources") { + if (!sv.is_bool()) + return std::unexpected(std::format("{}.resources must be a boolean", where)); + sys.resources = sv.as_bool(); + continue; + } if (!sv.is_string()) return std::unexpected(std::format("{}.{} must be a string", where, sk)); (sk == "name" ? sys.name : sys.uname) = sv.as_string(); @@ -907,16 +936,15 @@ Reader::override_(const toml::Table& t, std::string_view selector) const { if (auto r = link(v.as_table(), key, o.fields.link); !r) return std::unexpected(r.error()); } else if (k == "runner") { - auto r = runner(v, key); + auto r = runner(v, key, true); if (!r) return std::unexpected(r.error()); o.fields.runner = std::move(*r); } else if (k == "runners") { if (!v.is_table()) continue; // the table's own reader refuses the shape for (auto const& [rk, rv] : v.as_table()) { - auto rkey = std::format("{}.{}", key, rk); - auto w = strings(rv, rkey); - if (!w) continue; // ditto - o.fields.runners[rk] = Words{ std::move(*w), at(rv, rkey), true }; + auto r = runner(rv, std::format("{}.{}", key, rk), true); + if (!r) return std::unexpected(r.error()); + o.fields.runners[rk] = std::move(*r); } } // Every other key belongs to the table's other readers. diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index fd17bf702..3c348dd8e 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -4242,6 +4242,7 @@ std::expected parse_document(const t::Document& documen // artefact cannot execute here. An ARRAY, so it is neither a // scalar (the unknown-key sweep below skips it by type) nor part // of the conditional sub-table channel. + bool runnerAdjusts = false; // a table without `cmd` (`{ enabled = true }`) if (auto it = body.find("runner"); it != body.end()) { // The table form, `{ cmd = [...], host = "same-arch", exclusive = // true }`, is the platform reader's (SPEC-013 §2.3); its command @@ -4249,9 +4250,10 @@ std::expected parse_document(const t::Document& documen if (!it->second.is_array() && it->second.is_table()) { mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Manifest, &m.schemaWarnings}; - auto r = rd.runner(it->second, std::format("[target.{}].runner", triple)); + auto r = rd.runner(it->second, std::format("[target.{}].runner", triple), true); if (!r) return std::unexpected(error(origin, r.error())); e.runner = r->cmd; + runnerAdjusts = !r->set; } else if (!it->second.is_array()) { return std::unexpected(error(origin, std::format( "[target.{}].runner must be an array of strings or {{ cmd = [...], " @@ -4265,7 +4267,7 @@ std::expected parse_document(const t::Document& documen } e.runner.push_back(el.as_string()); } - if (e.runner.empty()) { + if (e.runner.empty() && !runnerAdjusts) { return std::unexpected(error(origin, std::format( "[target.{}].runner is empty — an empty template would " "run nothing and report success", triple))); @@ -4288,9 +4290,18 @@ std::expected parse_document(const t::Document& documen "\"download\", \"{{}}\"]", triple, triple))); } for (auto& [name, val] : it->second.as_table()) { + // The table form is the default runner's (R2): the same keys. + if (val.is_table()) { + mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Manifest, + &m.schemaWarnings}; + auto r = rd.runner(val, std::format("[target.{}.runners].{}", triple, name), true); + if (!r) return std::unexpected(error(origin, r.error())); + if (r->set) e.namedRunners[name] = r->cmd; + continue; + } if (!val.is_array()) { return std::unexpected(error(origin, std::format( - "[target.{}.runners].{} must be an array of strings", + "[target.{}.runners].{} must be an array of strings or {{ cmd = [...] }}", triple, name))); } std::vector argv; diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 5125dd1d6..927078a8c 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -869,6 +869,13 @@ struct Resources { struct NamedRunner { std::vector argv; bool longLived = false; + // The rest of the runner's table (`mcpp::runner_spec`, protocol 19; R2): + // the packages it is installed from, where it exists, how it reaches the + // artifact, whether runs overlap and whether it is enabled. + std::vector requires_, hosts; + std::string host; + bool exclusive = false; + bool enabled = true; }; // THE C STANDARD OF A PACKAGE THAT DECLARES NONE (#695). @@ -941,6 +948,9 @@ struct BuildConfig : BuildInputs { // gets the same flow. std::map namedRunners; bool runExclusive = false; + // The default runner's table besides its command (`runner_spec` with no + // name); `argv` is unused, the command being `runner` above. + NamedRunner runnerFields; // Was `sources` WRITTEN, as opposed to merely being empty? // @@ -1085,6 +1095,14 @@ struct BuildConfig : BuildInputs { // notarisation each couple a release to a release mcpp does not control, // and a name here is the whole of what the engine learns. std::vector packFormats; + // The program of the depending package this package's pack format is + // assembled from (`mcpp:pack-program=`, protocol 19, design 2026-11-15 + // N30): `mcpp pack --pkg-format ` with no program named packs it. + std::vector packPrograms; + // The warnings and degradations the program stated in its last run, in + // order. A packaging pass whose provider submitted nothing states them as + // the provider's reasons (N30). + std::vector statedReasons; // mcpp#755, protocol 15. What a build program stated about SOURCES, carried // to prepare the way every other directive is: folded in here, read back // after the program ran, replayed with it on a cache hit. diff --git a/modules/toolchain-model/build.mcpp b/modules/toolchain-model/build.mcpp index 8a049bba6..93fd89e1c 100644 --- a/modules/toolchain-model/build.mcpp +++ b/modules/toolchain-model/build.mcpp @@ -17,14 +17,15 @@ namespace { constexpr std::string_view kRowKeys[] = { "triple", "tier", "note", "standard", "exact", "march", "mabi", "code-model", "multilib", "flags", "link-emulation", "clang-triple", "gcc-triple", "family", "object-format", "data-model", "exe-suffix", - "shared-suffix", "loader", "execution", + "shared-suffix", "loader", "execution", "companions", "exceptions", }; constexpr std::string_view kProviderKeys[] = { "name", "serves", "priority", "hosts", "probe", "toolchain", "pairs", "sysroot", "linkage", "platform", "tier", "system-headers", "runner", }; -constexpr std::string_view kRunnerKeys[] = { "cmd", "hosts", "requires" }; -constexpr std::string_view kSystemKeys[] = { "name", "uname" }; +constexpr std::string_view kRunnerKeys[] = { "cmd", "host", "hosts", "requires", "enabled", "exclusive" }; +constexpr std::string_view kSystemKeys[] = { "name", "uname", "resources", "command-scripts" }; +constexpr std::string_view kExecutesKeys[] = { "host", "arches" }; constexpr std::string_view kPayloadKeys[] = { "name", "aliases", "family", "serves", "hosts", "frontend-dir", "frontends", "builds-for", "glibc-fixup", "family-version", }; @@ -123,7 +124,7 @@ std::string row_entry(const toml::Table& t, Arrays& arrays, std::set& rows) { @@ -165,6 +167,8 @@ std::string provider_entry(const toml::Table& t, Arrays& arrays, const std::set< // `runner = { cmd = [...], hosts = [...], requires = [...] }`: how an // artifact of the arrangement is started, where, and from which packages. std::vector runner, runnerHosts, runnerRequires; + std::string runnerHost; + bool runnerEnabled = true, runnerExclusive = false; if (auto it = t.find("runner"); it != t.end()) { if (!it->second.is_table()) throw Failure{std::format("{}: `runner` is not a table", where)}; auto const& r = it->second.as_table(); @@ -173,26 +177,57 @@ std::string provider_entry(const toml::Table& t, Arrays& arrays, const std::set< runner = words(r, "cmd", rwhere); runnerHosts = words(r, "hosts", rwhere); runnerRequires = words(r, "requires", rwhere); + runnerHost = text(r, "host", rwhere); + if (!runnerHost.empty() && runnerHost != "same-arch" && runnerHost != "any") + throw Failure{std::format("{}: host `{}` is not same-arch or any", rwhere, runnerHost)}; + if (auto e = r.find("enabled"); e != r.end()) { + if (!e->second.is_bool()) throw Failure{std::format("{}: `enabled` is not true or false", rwhere)}; + runnerEnabled = e->second.as_bool(); + } + if (auto e = r.find("exclusive"); e != r.end()) { + if (!e->second.is_bool()) throw Failure{std::format("{}: `exclusive` is not true or false", rwhere)}; + runnerExclusive = e->second.as_bool(); + } if (runner.empty()) throw Failure{std::format("{}: `cmd` is empty", rwhere)}; } return std::format( " {{ .name = {}, .serves = {}, .priority = {}, .hosts = {}, .probe = {}, .toolchain = {}, " ".pairs = {}, .sysroot = {}, .linkage = {}, .platform = {}, .tier = {}, .systemHeaders = {}, " - ".runner = {}, .runnerHosts = {}, .runnerRequires = {} }},\n", + ".runner = {}, .runnerHosts = {}, .runnerRequires = {}, .runnerHost = {}, .runnerEnabled = {}, " + ".runnerExclusive = {} }},\n", lit(name), arrays.add(serves), priority, arrays.add(words(t, "hosts", where)), lit(text(t, "probe", where)), lit(text(t, "toolchain", where)), arrays.add(words(t, "pairs", where)), lit(text(t, "sysroot", where)), lit(linkage), lit(text(t, "platform", where)), lit(tier), lit(text(t, "system-headers", where)), - arrays.add(runner), arrays.add(runnerHosts), arrays.add(runnerRequires)); + arrays.add(runner), arrays.add(runnerHosts), arrays.add(runnerRequires), lit(runnerHost), + runnerEnabled, runnerExclusive); } -std::string system_entry(const toml::Table& t, std::set& seen) { +std::string system_entry(const toml::Table& t, Arrays& arrays, std::set& seen) { known_keys(t, kSystemKeys, "[[platform.system]]"); const auto name = text(t, "name", "[[platform.system]]"); if (name.empty()) throw Failure{"[[platform.system]]: an entry without `name`"}; if (!seen.insert(name).second) throw Failure{std::format("[[platform.system]] {}: stated twice", name)}; - return std::format(" {{ .name = {}, .uname = {} }},\n", lit(name), - lit(text(t, "uname", "[[platform.system]] " + name))); + const auto where = "[[platform.system]] " + name; + bool resources = false; + if (auto it = t.find("resources"); it != t.end()) { + if (!it->second.is_bool()) throw Failure{std::format("{}: `resources` is not true or false", where)}; + resources = it->second.as_bool(); + } + return std::format(" {{ .name = {}, .uname = {}, .resources = {}, .commandScripts = {} }},\n", lit(name), + lit(text(t, "uname", where)), resources, arrays.add(words(t, "command-scripts", where))); +} + +std::string executes_entry(const toml::Table& t, Arrays& arrays, std::set& seen) { + known_keys(t, kExecutesKeys, "[[platform.executes]]"); + const auto host = text(t, "host", "[[platform.executes]]"); + const auto where = "[[platform.executes]] " + host; + if (host.empty() || host.find('-') == std::string::npos) + throw Failure{"[[platform.executes]]: `host` is `-`"}; + if (!seen.insert(host).second) throw Failure{std::format("{}: stated twice", where)}; + const auto arches = words(t, "arches", where); + if (arches.empty()) throw Failure{std::format("{}: `arches` names no architecture", where)}; + return std::format(" {{ .host = {}, .arches = {} }},\n", lit(host), arrays.add(arches)); } bool flag(const toml::Table& t, std::string_view key, bool fallback, std::string_view where) { @@ -240,8 +275,8 @@ const toml::Array& entries(const toml::Document& doc, std::string_view path) { std::string generate(const toml::Document& doc) { Arrays arrays; std::set rowNames; - std::set systemNames; - std::string rows, providers, systems, payloads, offers; + std::set systemNames, executesHosts; + std::string rows, providers, systems, executes, payloads, offers; for (auto const& e : entries(doc, "platform.define")) { if (!e.is_table()) throw Failure{"[[platform.define]]: an entry that is not a table"}; rows += row_entry(e.as_table(), arrays, rowNames); @@ -252,7 +287,11 @@ std::string generate(const toml::Document& doc) { } for (auto const& e : entries(doc, "platform.system")) { if (!e.is_table()) throw Failure{"[[platform.system]]: an entry that is not a table"}; - systems += system_entry(e.as_table(), systemNames); + systems += system_entry(e.as_table(), arrays, systemNames); + } + for (auto const& e : entries(doc, "platform.executes")) { + if (!e.is_table()) throw Failure{"[[platform.executes]]: an entry that is not a table"}; + executes += executes_entry(e.as_table(), arrays, executesHosts); } for (auto const& e : entries(doc, "toolchain.payload")) { if (!e.is_table()) throw Failure{"[[toolchain.payload]]: an entry that is not a table"}; @@ -267,15 +306,19 @@ std::string generate(const toml::Document& doc) { "export module mcpp.toolchain.builtin_targets;\n\n" "import std;\n" "import mcpp.toolchain.target_schema;\n\n" - "namespace mcpp::toolchain::builtin::d {{\n{}}}\n\n" + "namespace mcpp::toolchain::builtin::d {{\n{}" + "inline constexpr mcpp::toolchain::target::Executes executes_[] = {{\n{} {{}},\n}};\n" + "inline constexpr std::span executes{{executes_, std::size(executes_) - 1}};\n" + "}}\n\n" "export namespace mcpp::toolchain::builtin {{\n\n" "inline constexpr mcpp::toolchain::target::Row kRows[] = {{\n{}}};\n\n" "inline constexpr mcpp::toolchain::target::Provider kProviders[] = {{\n{}}};\n\n" "inline constexpr mcpp::toolchain::target::System kSystems[] = {{\n{}}};\n\n" + "inline constexpr std::span kExecutes = d::executes;\n\n" "inline constexpr mcpp::toolchain::target::Payload kPayloads[] = {{\n{}}};\n\n" "inline constexpr mcpp::toolchain::target::Offer kOffers[] = {{\n{}}};\n\n" "}} // namespace mcpp::toolchain::builtin\n", - arrays.decls, rows, providers, systems, payloads, offers); + arrays.decls, executes, rows, providers, systems, payloads, offers); } } // namespace diff --git a/modules/toolchain-model/data/builtin.toml b/modules/toolchain-model/data/builtin.toml index ef4cdf9b9..bd3ce5784 100644 --- a/modules/toolchain-model/data/builtin.toml +++ b/modules/toolchain-model/data/builtin.toml @@ -43,6 +43,12 @@ # `-mcmodel` (empty: the driver's default), the multilib # directory of a C library, extra compile flags, and the # `ld.lld -m` emulation when clang's driver cannot reach lld +# exceptions the exception model of the row's builds when the project +# states none (`[target..abi] exceptions`); under `wasm` +# the long-jump model is `wasm` too +# companions the files the link may write beside a program, named after +# it (`.wasm` beside `app.js`): removed before each link, +# staged when written, not an error when absent # # Provider keys # name the provider's name; `builtin:` selects it @@ -63,9 +69,15 @@ # system-headers the package whose headers state the system's own interfaces, # read by the steps that compile against them (resources) # runner how an artifact of the arrangement is started when nothing -# nearer says: `{ cmd = [argv], hosts = [...], requires = [...] }`. -# `{}` is the artifact, `{sysroot}` the arrangement's base; -# `requires` are installed on the first run, never by a build +# nearer says: `{ cmd = [argv], host = "same-arch", hosts = [...], +# requires = [...], enabled = false }`. `{}` is the artifact, +# `{sysroot}` the arrangement's base; `requires` are installed +# on the first run, never by a build. `enabled = false` for a +# runner whose first use fetches a whole machine image or an +# emulator, or leaves state outside the build tree and the +# package store (a VM, an Android emulator, Wine); a project +# or this machine enables it (SPEC-016 §6). `exclusive = true` +# when the runs cannot overlap (a device at one address) # # ONE ENTRY STATES ONE ARRANGEMENT OF PAYLOADS. A native arrangement and a cross # one are two entries, each with its `hosts`; an entry whose facts depend on the @@ -78,6 +90,12 @@ # uname the word `uname -s` answers there; a file that carries one # program per system selects by it, and a system without the # key joins no such file +# resources the system's loader reads a program's resource section: a +# project's `[resources]` compile into its programs (false +# when absent) +# command-scripts the suffixes of the files the system starts through its +# command interpreter; an argument holding the interpreter's +# syntax (`%`, `^`, `&`, `|`, `<`, `>`, `"`) is refused # ── Linux ──────────────────────────────────────────────────────────────────── @@ -438,6 +456,15 @@ triple = "wasm32-emscripten" tier = "verified" note = "wasm" standard = true +# WebAssembly exception handling unless the project states another model +# (design 2026-11-15 N26): a C++ exception is caught, and `setjmp`/`longjmp` +# take the same mechanism. Emscripten's default leaves catching off, and a +# program's `catch` was never reached. `[target..abi] exceptions` states +# another model (`"js"`, `true`) or none (`false`). +exceptions = "wasm" +# The module Emscripten writes beside the launcher (`app.wasm` beside `app.js`), +# none under `-sSINGLE_FILE=1` (N33). +companions = [".wasm"] # OK IMAGES (openkal SPEC clause 12): one artifact per architecture, run by # `okrun` on Linux, macOS and Windows of that architecture. @@ -508,6 +535,12 @@ hosts = ["linux-{arch}", "linux-x86_64"] linkage = "static" # MinGW-w64: gcc, or clang with the payload as its sysroot (SPEC-016 §6). +# On an x86_64 Linux machine a Windows program runs through Wine (design +# 2026-11-15 X1, R1). The runner is not enabled by default: Wine creates a +# prefix in the home directory on its first run. A project or this machine +# enables it (`[target.] runner = { enabled = true }`), once +# `MCPP_TARGET__RUNNER_ENABLED=true`. It runs programs of the machine's +# architecture (`same-arch`): an aarch64 Windows program is not offered. [[platform.provider]] name = "mingw" serves = ["x86_64-windows-gnu"] @@ -516,6 +549,7 @@ hosts = ["windows-*", "linux-x86_64"] toolchain = "gcc@16.1.0" pairs = ["gcc", "llvm"] linkage = "static" +runner = { cmd = ["wine", "{}"], host = "same-arch", hosts = ["linux-x86_64"], requires = ["xim:wine@11.0"], enabled = false } # MinGW-w64 as a base package (E9): the target side of llvm-mingw (MinGW-w64 # on UCRT, libc++, libunwind, compiler-rt) beside llvm, on every host and both @@ -529,6 +563,7 @@ toolchain = "llvm@23.1.3" pairs = ["llvm"] sysroot = "xim:mingw-w64-sysroot@20261104-2" linkage = "static" +runner = { cmd = ["wine", "{}"], host = "same-arch", hosts = ["linux-x86_64"], requires = ["xim:wine@11.0"], enabled = false } [[platform.provider]] name = "msvc" @@ -575,9 +610,27 @@ pairs = ["llvm"] linkage = "static" # bionic is inside the NDK; no package adds it to another compiler. +# +# An x86_64 program runs in the Android Emulator booting the API 34 system image, +# through the image package's dirun profile (design 2026-11-15 G7, R1): the +# emulator boots an image of the machine's architecture only (`same-arch`) and +# starts its device at one console port, so the runs of the target do not +# overlap (`exclusive`). Not enabled by default: the first run fetches the +# emulator and a system image. The emulator package is built for linux-x86_64 +# alone, so the aarch64 row states no runner until it serves an aarch64 machine. +[[platform.provider]] +name = "android-ndk" +serves = ["x86_64-linux-android"] +priority = 100 +hosts = ["*"] +toolchain = "android-ndk@30.0.16248370" +pairs = ["android-ndk"] +sysroot = "toolchain" +runner = { cmd = ["dirun", "--profile", "{pkg:android-system-image}/dirun.toml", "--var", "emulator={pkg:android-emulator}/emulator", "--var", "adb={pkg:android-platform-tools}/adb", "--", "{}"], host = "same-arch", hosts = ["linux-x86_64"], exclusive = true, enabled = false, requires = ["xim:dirun@0.4.0", "xim:android-system-image@34-default-x86_64", "xim:android-emulator@37.1.11-2", "xim:android-platform-tools@37.0.1-2"] } + [[platform.provider]] name = "android-ndk" -serves = ["*-linux-android"] +serves = ["aarch64-linux-android"] priority = 100 hosts = ["*"] toolchain = "android-ndk@30.0.16248370" @@ -646,7 +699,25 @@ uname = "FreeBSD" # A Windows program is not selected by `uname`: it leads the file. [[platform.system]] -name = "windows" +name = "windows" +resources = true +command-scripts = [".bat", ".cmd"] + +# ── Execution relations ────────────────────────────────────────────────────── +# +# Executes keys (`[[platform.executes]]`) +# host the machine, `-` +# arches the architectures whose programs it starts besides its own; +# a runner stated for a machine of the artifact's architecture +# (`host = "same-arch"`) runs there too +# +# Only a relation a CI has measured is stated. An x86_64 Linux kernel runs i386 +# programs (IA-32 emulation), which the i686 rows' CI runs on GitHub's x86_64 +# Linux machines (openkal-platform 0.28.0; mcpp e2e `i686`). Windows' WOW64 is +# not stated until a CI measures an i686 runner there. +[[platform.executes]] +host = "linux-x86_64" +arches = ["i686"] # ── Toolchain payloads ─────────────────────────────────────────────────────── # diff --git a/modules/toolchain-model/src/target_rows.cppm b/modules/toolchain-model/src/target_rows.cppm index 4d91dc566..62699a902 100644 --- a/modules/toolchain-model/src/target_rows.cppm +++ b/modules/toolchain-model/src/target_rows.cppm @@ -51,6 +51,12 @@ const Row* find_row(std::string_view triple); std::vector systems(); const System* find_system(std::string_view name); +// Whether the machine `host` (`-`) executes programs of `arch`: its +// own architecture, or one the execution table states for it (N23). +bool executes(std::string_view host, std::string_view arch); +// The execution table itself, for a launcher that carries it as data. +std::span execution_relations(); + // Whether a defined row names this os, or this env under this os: the words the // parser accepts beyond its own. bool defines_os(std::string_view os); @@ -158,6 +164,16 @@ const System* find_system(std::string_view name) { return nullptr; } +std::span execution_relations() { return mcpp::toolchain::builtin::kExecutes; } + +bool executes(std::string_view host, std::string_view arch) { + if (const auto dash = host.rfind('-'); dash != std::string_view::npos && host.substr(dash + 1) == arch) + return true; + for (auto const& e : mcpp::toolchain::builtin::kExecutes) + if (e.host == host && std::ranges::contains(e.arches, arch)) return true; + return false; +} + bool defines_os(std::string_view os) { // The built-in rows' systems are words of the language too (design // 2026-11-10 E4): a system the data names -- FreeBSD -- needs no word in diff --git a/modules/toolchain-model/src/target_schema.cppm b/modules/toolchain-model/src/target_schema.cppm index af9699f86..7cd59dd50 100644 --- a/modules/toolchain-model/src/target_schema.cppm +++ b/modules/toolchain-model/src/target_schema.cppm @@ -48,6 +48,15 @@ struct Row { // The system the artifact runs on: the row's os by default, `loader` for an // image a loader binds, `none` for bare metal. std::string_view execution; + // FILES THE LINK MAY WRITE BESIDE A PROGRAM, NAMED AFTER IT (design + // 2026-11-15 N33): the suffix replaces the program's (`.wasm` beside + // `app.js`). Removed before each link and staged when the link wrote one; + // their absence is not an error (`-sSINGLE_FILE=1` writes none). + std::span companions; + // THE EXCEPTION MODEL OF THE ROW'S BUILDS when the project states none + // (`[target..abi] exceptions`; design 2026-11-15 N26): `wasm` on + // wasm32-emscripten. Empty: the toolchain's default. + std::string_view exceptions; bool defined = false; // stated by a platform package }; @@ -75,6 +84,12 @@ struct Provider { std::span runner; std::span runnerHosts; std::span runnerRequires; + // `same-arch` or `any` (empty: unstated), as a platform's runner states it, + // and whether it is used before a project or this machine enables it (R1). + std::string_view runnerHost; + bool runnerEnabled = true; + // Its runs cannot overlap (a device the runner starts at one address). + bool runnerExclusive = false; }; // A TOOLCHAIN PAYLOAD (design 2026-11-12 §16, T3): the package a family's @@ -115,6 +130,26 @@ struct Offer { struct System { std::string_view name; std::string_view uname; + // THE SYSTEM'S LOADER READS A PROGRAM'S RESOURCE SECTION (design + // 2026-11-15 N16): a project's `[resources]` compile into the programs of + // its rows. A PE image another system loads (a UEFI application) has none. + bool resources = false; + // THE FILES THE SYSTEM STARTS THROUGH ITS COMMAND INTERPRETER (design + // 2026-11-15 R2, A3), by suffix (`.bat`, `.cmd` on Windows): the + // interpreter reads `%`, `^`, `&`, `|`, `<`, `>` and `"` in an argument + // as its own syntax, so an argument holding one is refused before the + // start rather than passed to be rewritten or run as a command. + std::span commandScripts; +}; + +// AN ARCHITECTURE A MACHINE EXECUTES BESIDES ITS OWN (design 2026-11-15 N23): +// `host` is the machine, `-`; `arches` the architectures whose +// programs it starts as its own. A runner that runs an artifact on a machine +// of the artifact's architecture (`host = "same-arch"`) runs there too. Only +// relations a CI has measured are stated. +struct Executes { + std::string_view host; + std::span arches; }; // THE PATTERN OF `serves`: segment by segment, `*` standing for any one segment diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 22ffc497f..e3c2cc7ec 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.11.15.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.11.16.1"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 4e84a2d53..200a92156 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1334,6 +1334,9 @@ std::expected run_build_program_impl( if (auto derr = dirs::deploy_directive_error(m, d); !derr.empty()) { return std::unexpected(derr); } + if (auto rerr = dirs::runner_spec_error(d); !rerr.empty()) { + return std::unexpected(rerr); + } // GLOB INPUTS OUTSIDE THE PACKAGE (#766). A registry or git dependency's // tree sits in the package store, whose other contents depend on what // else is installed: a pattern leaving that package would make its re-run diff --git a/src/build/command_line.cpp b/src/build/command_line.cpp index a362df580..03509f3ba 100644 --- a/src/build/command_line.cpp +++ b/src/build/command_line.cpp @@ -183,3 +183,21 @@ std::string clean_ld_library_path_prefix(const std::vector args) { + constexpr std::string_view syntax = "%^&|<>\"\r\n"; + for (std::size_t i = 0; i < args.size(); ++i) + if (auto at = args[i].find_first_of(syntax); at != std::string::npos) { + const char c = args[i][at]; + const auto shown = c == '\r' ? std::string("a carriage return") + : c == '\n' ? std::string("a line feed") + : std::format("`{}`", c); + return std::format("argument {} ('{}') holds {}, which the command interpreter reads as its " + "own syntax", i + 1, args[i], shown); + } + return {}; +} + +} // namespace mcpp::build::command_line diff --git a/src/build/command_line.cppm b/src/build/command_line.cppm index 90f5da43c..15d7a92a6 100644 --- a/src/build/command_line.cppm +++ b/src/build/command_line.cppm @@ -27,6 +27,16 @@ std::string quote(std::string_view s); std::string quote_windows(std::string_view s); std::string quote_posix(std::string_view s); +// AN ARGUMENT A COMMAND INTERPRETER WOULD READ AS SYNTAX (design 2026-11-15 +// R2, A3). A script the system starts through its interpreter (`.bat`, +// `.cmd`: the system table's `command-scripts`) receives its arguments +// through that interpreter's parser, which expands `%`, escapes with `^`, +// chains with `&` and `|`, redirects with `<` and `>`, and ends a quoted +// span at `"` -- whatever quoting the caller applied (the class of +// CVE-2024-24576). The first such argument, described for a refusal; empty +// when every argument passes as written. +std::string command_script_problem(std::span args); + // AN ARGUMENT OF A COMMAND PRINTED FOR A PERSON TO PASTE (mcpp#832). Left as // it is when the shell would read it as one word anyway, so the common // message reads as it always did; quoted by this host's shell rules when it diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 84593a04d..dd25f2b32 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1514,6 +1514,13 @@ struct RunnerChoice { // with no base, `{pkg:}` outside `requires`): the launch is refused // naming it, since the word would reach the runner as it is written. std::string unplaced; + // THE RUNNER IS NOT ENABLED (R1): stated `enabled = false` and not enabled + // by this machine, the project, the environment or `--runner `. A + // run and a test refuse before installing anything; `requires` is what + // its first use would install. + bool notEnabled = false; + std::vector requires_; + std::string name; // the named runner; empty: the default // The spelling that names this target in the manifest: the canonical form, // which is also the output directory's name and the key every // `[target.]` reader resolves. Every diagnostic below prints this @@ -1565,6 +1572,39 @@ void settle_runner_places(const BuildContext& ctx, RunnerChoice& c, } } +// MCPP_TARGET__, the spelling of the target's variables. +std::string target_variable(std::string_view triple, std::string_view key) { + std::string out = "MCPP_TARGET_"; + for (char c : triple) + out += (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') + : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; + return out + "_" + std::string(key); +} + +// The refusal of a runner that is not enabled (R1, design 2026-11-15 §5.1): +// what it needs, and the two ways of enabling it. The token is the one a +// program reads (docs/50). +std::string not_enabled_message(const RunnerChoice& c) { + std::string needs; + for (auto const& r : c.requires_) needs += (needs.empty() ? "" : ", ") + r; + const std::string what = c.name.empty() + ? std::format("{} runs through a runner that is not enabled", c.tripleKey) + : std::format("the runner '{}' of {} is not enabled", c.name, c.tripleKey); + std::string out = what + " [runner-not-enabled]"; + if (!needs.empty()) out += std::format("\n it needs: {} (installed on first use)", needs); + if (c.name.empty()) { + out += std::format("\n enable: [target.{}] runner = {{ enabled = true }} # mcpp.toml, or config.toml " + "for this machine", c.tripleKey); + out += std::format("\n once: {}=true mcpp run --target {}", + target_variable(c.tripleKey, "RUNNER_ENABLED"), c.tripleKey); + } else { + out += std::format("\n enable: [target.{}.runners] {} = {{ enabled = true }} # mcpp.toml, or " + "config.toml for this machine", c.tripleKey, c.name); + out += std::format("\n once: mcpp run --target {} --runner {}", c.tripleKey, c.name); + } + return out; +} + // The refusal of a template whose place this build does not have. std::string unplaced_message(const RunnerChoice& c) { return std::format( @@ -1596,9 +1636,13 @@ std::string install_runner_packages(const RunnerChoice& c) { // `which` selects the slot; everything else is shared. RunnerChoice choose_device_action(const BuildContext& ctx, std::string_view which, - bool noRunner = false) { + bool noRunner = false, + // `--runner ` typed: naming a runner + // enables it for this run (R1). + bool named = false) { RunnerChoice c; const bool isDefault = which.empty(); + c.name = std::string(which); const auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); if (ft) c.freestanding = ft->is_freestanding(); c.tripleKey = ft ? ft->str() : ctx.tc.targetTriple; @@ -1612,20 +1656,33 @@ RunnerChoice choose_device_action(const BuildContext& ctx, using T = mcpp::manifest::platform::Tier; return programHad && (o.tier == T::Manifest || o.tier == T::Config || o.tier == T::Env); }; + // A RUNNER THAT IS NOT ENABLED IS NOT SETTLED (R1): its packages are + // what its first use would install, and nothing is installed for it. + // A runner that does not reach the artifact from this machine (`host = + // "same-arch"` and another architecture) is not offered for enabling: the + // run is refused for the reach (`unreachable_execution`). + auto take = [&](const mcpp::manifest::platform::Runner& r) { + c.requires_ = r.requires_; + const bool reaches = !ft || mcpp::platform::runner_reach(r, ft->arch, mcpp::toolchain::platform::host_arch, + !ctx.runUnder.empty(), noRunner) + != mcpp::platform::RunnerReach::Refused; + c.notEnabled = !r.enabled && !named && reaches; + if (!c.notEnabled) settle_runner_places(ctx, c, r.requires_); + }; if (isDefault) { if (pf.runner.set) { c.tmpl = pf.runner.cmd; c.fromManifest = overrides_program(pf.runner.origin, !ctx.manifest.buildConfig.runner.empty()); - settle_runner_places(ctx, c, pf.runner.requires_); + take(pf.runner); } - } else if (auto it = pf.runners.find(std::string(which)); it != pf.runners.end()) { - c.tmpl = it->second.value; + } else if (auto it = pf.runners.find(std::string(which)); it != pf.runners.end() && it->second.set) { + c.tmpl = it->second.cmd; auto nr = ctx.manifest.buildConfig.namedRunners.find(std::string(which)); c.fromManifest = overrides_program(it->second.origin, nr != ctx.manifest.buildConfig.namedRunners.end()); - if (nr != ctx.manifest.buildConfig.namedRunners.end() && !c.fromManifest) - c.longLived = nr->second.longLived; + c.longLived = it->second.longlived; + take(it->second); } // AND THE TOOLCHAIN'S OWN ANSWER, WHEN NOTHING ABOVE GAVE ONE. // @@ -1647,7 +1704,7 @@ RunnerChoice choose_device_action(const BuildContext& ctx, // manifest cannot carry: the triple is native here. On a freestanding // target that leaves nothing to execute, and the caller's existing // no-runner error is the correct answer. - if (noRunner && !c.tmpl.empty()) { c.tmpl.clear(); c.ignored = true; } + if (noRunner && !c.tmpl.empty()) { c.tmpl.clear(); c.ignored = true; c.notEnabled = false; } return c; } @@ -1675,7 +1732,7 @@ export struct RunnerOnThisMachine { nlohmann::json launch_configuration(const BuildContext& ctx) { auto runners = nlohmann::json::array(); for (const auto& [name, entry] : ctx.plan.platform.fields.runners) - if (!entry.value.empty()) runners.push_back(name); + if (!entry.cmd.empty()) runners.push_back(name); return {{"triple", ctx.tc.targetTriple}, {"profile", ctx.profile}, {"platform", nlohmann::json::parse(mcpp::platform::to_json(ctx.plan.platform), nullptr, false)}, {"runners", std::move(runners)}}; @@ -1685,6 +1742,10 @@ export RunnerOnThisMachine runner_on_this_machine(const BuildContext& ctx) { RunnerOnThisMachine out; const auto choice = choose_runner(ctx); out.arch = std::string(mcpp::toolchain::platform::host_arch); + if (choice.notEnabled) { + out.refusal = not_enabled_message(choice); + return out; + } if (auto why = unreachable_execution(ctx, choice.tripleKey, false, !choice.tmpl.empty())) out.refusal = *why; if (!ctx.runUnder.empty()) { auto host = locate_run_under(ctx, choice.tripleKey); @@ -2637,19 +2698,20 @@ export int list_runners(const std::string& package_filter, const std::string key = ft ? ft->str() : ctx->tc.targetTriple; std::println("Target {}", key); - if (!pf.runner.set && pf.runners.empty()) { + const bool anyNamed = std::ranges::any_of(pf.runners, [](auto const& e) { return e.second.set; }); + if (!pf.runner.set && !anyNamed) { std::println(" (none — this project reaches its artifact by executing it)"); return 0; } if (pf.runner.set) - std::println(" {:<12} {} ({})", "(default)", pf.runner.cmd.front(), - mcpp::platform::origin_text(pf.runner.origin)); + std::println(" {:<12} {} ({}){}", "(default)", pf.runner.cmd.front(), + mcpp::platform::origin_text(pf.runner.origin), + pf.runner.enabled ? "" : " [not enabled]"); for (auto const& [name, w] : pf.runners) { - auto nr = bc.namedRunners.find(name); - const bool longLived = nr != bc.namedRunners.end() && nr->second.longLived; - std::println(" {:<12} {} ({}){}", name, - w.value.empty() ? std::string("(no argv)") : w.value.front(), - mcpp::platform::origin_text(w.origin), longLived ? " [long-lived]" : ""); + if (!w.set) continue; + std::println(" {:<12} {} ({}){}{}", name, w.cmd.front(), + mcpp::platform::origin_text(w.origin), w.longlived ? " [long-lived]" : "", + w.enabled ? "" : " [not enabled]"); } if (bc.runExclusive || pf.runner.exclusive) std::println(" note: this target's runs cannot overlap; `mcpp test` " @@ -3469,13 +3531,18 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, // while debugging, or naming `qemu-aarch64-static` for a cross target. const bool isRunSlot = runner_name.empty(); const std::string slotName{runner_name}; - auto choice = choose_device_action(ctx, runner_name, no_runner); + const bool namedRunner = !runner_name.empty() && !runner_from_format; + auto choice = choose_device_action(ctx, runner_name, no_runner, namedRunner); + if (choice.notEnabled) { + std::println(libc::stderr(), "error: {}", not_enabled_message(choice)); + return 2; + } if (!choice.missing.empty()) { if (auto why = install_runner_packages(choice); !why.empty()) { std::println(libc::stderr(), "error: {}", why); return 2; } - choice = choose_device_action(ctx, runner_name, no_runner); + choice = choose_device_action(ctx, runner_name, no_runner, namedRunner); } if (!choice.unplaced.empty()) { std::println(libc::stderr(), "error: {}", unplaced_message(choice)); @@ -4518,8 +4585,10 @@ inline constexpr std::string_view kTestingDocs = using mcpp::build::test_protocol::kMtestRelease; using mcpp::build::test_protocol::kMtestRepo; -static void print_test_guide(const BuildContext& ctx, bool json) { - const auto& frameworks = ctx.plan.testFrameworks; +// `members`: how many members the frameworks were gathered from, when more +// than one (a workspace root, `--workspace`). +static void print_test_guide(std::span frameworks, bool json, + std::size_t members = 0) { if (json) { nlohmann::ordered_json list = nlohmann::ordered_json::array(); for (auto const& f : frameworks) @@ -4535,6 +4604,7 @@ static void print_test_guide(const BuildContext& ctx, bool json) { {"docs", kTestingDocs}, }; if (frameworks.empty()) g["suggest"] = {{"dev-dependency", std::format("mcpplibs.mtest = \"{}\"", kMtestRelease)}, {"repo", kMtestRepo}}; + if (members > 1) g["members"] = members; std::println("{}", nlohmann::ordered_json{{"guide", g}}.dump()); std::fflush(libc::stdout()); return; @@ -4543,7 +4613,8 @@ static void print_test_guide(const BuildContext& ctx, bool json) { std::println("No test framework: a test is a program under tests/, judged by its exit status."); std::println("A framework of cases (mtest): [dev-dependencies] mcpplibs.mtest = \"{}\" ({})", kMtestRelease, kMtestRepo); } else { - std::println("Test frameworks:"); + if (members > 1) std::println("Test frameworks of {} members (one member's: mcpp test -p --guide):", members); + else std::println("Test frameworks:"); for (auto const& f : frameworks) { std::println(" {} {} ({}{})", f.package, f.version, f.entry, f.protocol ? std::format(", test protocol {}", f.protocol) : std::string{}); @@ -4569,6 +4640,53 @@ static void print_test_guide(const BuildContext& ctx, bool json) { std::println("Testing in mcpp: {}", kTestingDocs); } +// `mcpp test --guide` (#932): the frameworks are declared data, read from the +// resolved graph -- with the dev-dependencies, where frameworks are -- and +// nothing is planned or built: no toolchain after the graph, no build program, +// no scan, no build lock. It is the command a failing test names next, and in +// a workspace of 140 members a plan of the tests took minutes before it. +// `groups`: the configuration groups of several selected members, each +// resolved as `mcpp test` plans it, and their frameworks united; empty for one +// package, resolved with `base`. +export int print_test_guide_of(const BuildOverrides& base, const std::filesystem::path& wsRoot, + const std::vector>& groups, bool json) { + std::vector frameworks; + std::set seen; + auto gather = [&](BuildOverrides ov) -> bool { + // The resolution's narration ("Workspace building ...") describes a + // build, which this is not; its refusal is still said. + const bool wasQuiet = mcpp::ui::is_quiet(); + mcpp::ui::set_quiet(true); + auto packages = resolve_graph(/*includeDevDeps=*/true, std::move(ov)); + mcpp::ui::set_quiet(wasQuiet); + if (!packages) { mcpp::ui::error(packages.error()); return false; } + std::vector found; + for (auto const& p : *packages) collect_test_frameworks(p.manifest, p.root, found); + for (auto& f : found) + if (seen.insert(f.package + '\x1f' + f.version + '\x1f' + f.entry).second) frameworks.push_back(std::move(f)); + return true; + }; + std::size_t members = 0; + if (groups.empty()) { + if (!gather(base)) return 2; + } else { + std::vector request; + for (auto const& g : groups) request.insert(request.end(), g.begin(), g.end()); + members = request.size(); + for (auto const& g : groups) { + BuildOverrides ov = base; + ov.package_filter.clear(); + ov.project_root = wsRoot; + ov.workspace_members = g; + ov.workspace_request = request; + ov.member_targets.clear(); + if (!gather(std::move(ov))) return 2; + } + } + print_test_guide(frameworks, json, members); + return 0; +} + // A planned test build, and what the tests run against it need: the plan, the // backend that drives it, how long planning and the build took, and how the // test binaries are executed. One per configuration group of a `mcpp test` @@ -4908,7 +5026,21 @@ static void test_bulk(TestBuild& tb, const TestOptions& testOpts, Keep&& keep) { // holds under a runner exactly as it does on the host. static void test_resolve_runner(TestBuild& tb, const TestOptions& testOpts, bool json) { auto* ctx = &*tb.ctx; - tb.runnerChoice = choose_device_action(*ctx, testOpts.runnerName, testOpts.noRunner); + const bool namedRunner = !testOpts.runnerName.empty(); + tb.runnerChoice = choose_device_action(*ctx, testOpts.runnerName, testOpts.noRunner, namedRunner); + // A runner that is not enabled leaves the tests built and not run, with + // the way to enable it, as an artifact this machine does not reach does + // (below); nothing is installed for it. + if (tb.runnerChoice.notEnabled) { + const auto why = not_enabled_message(tb.runnerChoice); + if (testOpts.unreachableIsBuilt) { + tb.builtNotRunHere = true; + if (!json) mcpp::ui::info("note", std::format("built, not run on this host: {}", why)); + return; + } + tb.invocationNotRunReason = why; + return; + } if (!tb.runnerChoice.missing.empty()) { // A package that cannot be installed leaves the tests not run, said // once for the build as a missing program is. @@ -4918,7 +5050,7 @@ static void test_resolve_runner(TestBuild& tb, const TestOptions& testOpts, bool tb.runnerFatal = 2; return; } - tb.runnerChoice = choose_device_action(*ctx, testOpts.runnerName, testOpts.noRunner); + tb.runnerChoice = choose_device_action(*ctx, testOpts.runnerName, testOpts.noRunner, namedRunner); } if (!tb.runnerChoice.unplaced.empty()) { tb.invocationNotRunReason = unplaced_message(tb.runnerChoice); @@ -6599,7 +6731,7 @@ fast_test_plan(const std::filesystem::path& projectRoot, const TestOptions& test return std::nullopt; }; if (!testOpts.since.empty() || !testOpts.changed.empty()) return declined("a change selects the tests (--since, --changed)"); - if (!testOpts.archive.empty() || testOpts.guide || print_launch_flag()) return declined("an archive, the guide or a launch is asked for"); + if (!testOpts.archive.empty() || print_launch_flag()) return declined("an archive or a launch is asked for"); if (!testOpts.runnerName.empty() || testOpts.noRunner) return declined("a runner is named"); if (mcpp::invocation::get("MCPP_LOCKED").value_or("") == "1") return declined("MCPP_LOCKED=1 asks for a locked resolution"); if (auto v = mcpp::invocation::get("MCPP_MANAGED_ONLY"); v && *v != "0") return declined("MCPP_MANAGED_ONLY asks for the sources to be checked"); @@ -6964,11 +7096,6 @@ export int run_tests(std::span passthrough, } } - if (testOpts.guide) { - print_test_guide(*ctx, json); - return 0; - } - // 4. A package is named by the build that compiles it: Phase A's report // writes its line the first time one of its steps runs, as `mcpp build` // does, and a test binary's line prints in Phase B beside its result. The @@ -7458,50 +7585,81 @@ export void run_workspace_tests(std::span passthrough, } // EACH MEMBER'S PROGRAMS, BUILT BEFORE ITS TESTS AND NAMED TO THEM (L5), // as for one package. A workspace's test plan holds a member's test - // programs only, so its own programs are built by the member's plan; + // programs only, so its own programs are built by a plan of programs; // without this a member's test found no MCPP_BIN_ under // `--workspace` and found it under `-p` (found by mtest's conformance // suite). + // + // ONE PLAN OF THE GROUP, AS `mcpp build --workspace` PLANS IT (#936). + // A plan of one member resolved the features of the packages it shares + // with the others from that member alone, and built them into the + // group's directory, at the same paths: a dependency's module was + // compiled by the group's test plan, again by each member's plan of + // programs, and again by the test pass that followed (three compiles + // of one unchanged module in one run, measured), and on Windows a + // rewrite of a module another process had mapped failed the run. The + // members of the group are planned together, so a package is built + // with the configuration the tests' build gives it; a member is + // planned alone only to say whose programs did not build. + std::vector buildable, withPrograms; for (auto i : who) { if (slots[i].packageFailed || slots[i].session < 0) continue; + buildable.push_back(i); auto m = mcpp::manifest::load(wsRoot / members[i].path / "mcpp.toml", {.insideWorkspace = true}); - if (!m || std::ranges::none_of(m->targets, [](auto const& t) { + if (m && std::ranges::any_of(m->targets, [](auto const& t) { return t.kind == mcpp::manifest::Target::Binary || t.kind == mcpp::manifest::Target::Application; })) - continue; + withPrograms.push_back(i); + } + // The programs of `of`, planned with the members `planFor`: false + // when they did not build, the reason left for the caller. + auto programs_of = [&](const std::vector& of, const std::vector& planFor, + std::string& why) { BuildOverrides po = base; po.package_filter.clear(); po.project_root = wsRoot; po.will_run = false; - po.workspace_members = { members[i].path }; - po.workspace_request = { members[i].path }; + po.workspace_members.clear(); + for (auto i : planFor) po.workspace_members.push_back(members[i].path); + po.workspace_request.clear(); + for (auto const& m : members) po.workspace_request.push_back(m.path); po.member_targets.clear(); auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, std::move(po)); - if (!binCtx) { - slots[i].packageFailed = true; - slots[i].packageOutput = binCtx.error(); - continue; - } + if (!binCtx) { why = binCtx.error(); return false; } // `--no-programs`: planned for their names, and the environment's. if (!testOpts.noPrograms && run_build_plan(*binCtx, mcpp::log::is_verbose(), false, base.target_triple) != 0) { - slots[i].packageFailed = true; - slots[i].packageOutput = std::format("the programs of member '{}' did not build", members[i].path); - continue; + why.clear(); + return false; } - auto& env = tb.memberProgramEnv[slots[i].owner]; - for (auto const& lu : binCtx->plan.linkUnits) { - if (lu.kind != LinkUnit::Binary || (!lu.memberOf.empty() && lu.memberOf != slots[i].owner)) continue; - env.emplace_back(program_env_key(lu.targetName), - testOpts.noPrograms ? std::string{} - : (binCtx->outputDir / lu.output).lexically_normal().string()); + for (auto i : of) { + auto& env = tb.memberProgramEnv[slots[i].owner]; + for (auto const& lu : binCtx->plan.linkUnits) { + if (lu.kind != LinkUnit::Binary || (!lu.memberOf.empty() && lu.memberOf != slots[i].owner)) continue; + env.emplace_back(program_env_key(lu.targetName), + testOpts.noPrograms ? std::string{} + : (binCtx->outputDir / lu.output).lexically_normal().string()); + } } - if (testOpts.noPrograms && !testOpts.noRun) - if (auto why = programs_from_environment(env)) { + return true; + }; + if (!withPrograms.empty()) { + std::string why; + if (!programs_of(withPrograms, buildable, why)) { + for (auto i : withPrograms) { + if (buildable.size() > 1 && programs_of({i}, {i}, why)) continue; slots[i].packageFailed = true; - slots[i].packageOutput = *why; - continue; + slots[i].packageOutput = why.empty() + ? std::format("the programs of member '{}' did not build", members[i].path) : why; } + } + } + for (auto i : withPrograms) { + if (slots[i].packageFailed || !testOpts.noPrograms || testOpts.noRun) continue; + if (auto why = programs_from_environment(tb.memberProgramEnv[slots[i].owner])) { + slots[i].packageFailed = true; + slots[i].packageOutput = *why; + } } // The test goals of the members that can run. std::set runnable; diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 1b07e74b7..5d7059def 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -57,12 +57,13 @@ inline void include_dir_after(const char* dir) { std::printf("mcpp:include-di // carries one value per line. The artifact path is appended by mcpp, or // substituted for a `{}` token if one is present. // -// Emit the executable as an ABSOLUTE path. A bare name resolves through -// PATH to a shim that dispatches against its OWNER home, which is not -// necessarily the home this build uses; measured in CI as -// `xlings: 'qemu-system-riscv64' is not installed` from a job where the same -// bare name had answered `--version` two steps earlier. `xpkg_dir()` is how a -// package finds the payload it declared. +// Name the program. mcpp looks for it in the runner's `requires`, then in +// the `bin/` of every payload the graph declares, then on PATH (docs/41); a +// program outside a package's `bin/` is `{pkg:}/`, its package +// among `requires` (`runner_spec`). A bare name that reached PATH alone +// resolved to a shim dispatching against its OWNER home (measured in CI as +// `xlings: 'qemu-system-riscv64' is not installed`), which the payload +// search answers first. // // Exactly one dependency may supply this. Two board-support packages both // claiming to know how to run the artifact is a configuration error, and mcpp @@ -178,6 +179,19 @@ inline void provides_pack_format(const char* name) { std::printf("mcpp:pack-format=%s\n", name); } +// The program of the package being packed that this provider assembles its +// distributable from (protocol 19). `mcpp pack --pkg-format ` with no +// program named packs this one, as `mcpp pack ` would; a named +// program outranks it. Declared unconditionally, as `provides_pack_format` +// is, for the same reason: the program is chosen from the build that precedes +// the packaging pass. +// +// mcpp::provides_pack_format("web"); +// mcpp::pack_program("app"); +inline void pack_program(const char* name) { + std::printf("mcpp:pack-program=%s\n", name); +} + // The memory layout for a freestanding link. Reaches the CONSUMER's link line // (like link_lib/link_search, unlike include_dir), because the package that // knows a board's layout is not the package being built. @@ -277,6 +291,102 @@ namespace roles { // unpacking an SDK. See `action::output_dir` below for its contract. inline constexpr const char* prepare = "prepare"; } +// THE JSON OF A DECLARATION (an action, a runner): a list field held already +// serialised, a string literal, and one entry of a list, by one rule. +namespace json_detail { +// One list field, held already serialised (`"a","b"`) so submit() prints +// it as it is. Owning and std-free, and both words are constraints this +// module carries: it may be compiled before a std BMI exists, so it must +// not `import std;`, and its exported interface must name no std type, so +// `std::string` may not appear in a signature. Neither forbids the heap: +// storage is `realloc` from the `` already in the global module +// fragment, and no exported signature mentions this type. +// +// An earlier revision held six fixed arrays (8192 bytes for `inputs` and +// `outputs`, chosen for a protoc command line) and a declaration that did +// not fit was refused. The bound was in bytes of serialised JSON, so a +// consumer's checkout depth decided whether a resource list of 44 files +// fit (HuxerUI#130 measured the margin at 45 bytes), and `outputs` is the +// one list an author cannot shorten: an output the program does not name +// cannot be built, and there is no depfile for outputs. See +// .agents/docs/2026-09-13-four-upstream-asks-from-a-ui-framework.md. +struct list { + char* p = nullptr; + unsigned long len = 0, cap = 0; + list() = default; + list(const list& o) { take(o); } + list& operator=(const list& o) { if (this != &o) { len = 0; take(o); } return *this; } + ~list() { std::free(p); } + const char* c_str() const { return p ? p : ""; } + // Grows by doubling. False only when the allocator refuses. + bool reserve(unsigned long need) { + if (need <= cap) return true; + unsigned long c = cap ? cap : 256; + while (c < need) c *= 2; + void* q = std::realloc(p, c); + if (!q) return false; + p = static_cast(q); + cap = c; + return true; + } + bool put(char c) { + if (!reserve(len + 2)) return false; + p[len++] = c; + p[len] = 0; + return true; + } + void take(const list& o) { + if (!o.len) { if (p) p[0] = 0; return; } + if (!reserve(o.len + 1)) return; + for (unsigned long i = 0; i <= o.len; ++i) p[i] = o.p[i]; + len = o.len; + } +}; +inline void esc(const char* s) { + std::putchar('"'); + for (const char* p = s; *p; ++p) { + unsigned char c = (unsigned char)*p; + if (c == '"' || c == '\\') { std::putchar('\\'); std::putchar(c); continue; } + // Any control character has to be escaped or the payload is not + // JSON at all. \n was handled before; \t and \r reach this code + // through ordinary Windows paths and log text. + if (c < 0x20) { std::printf("\\u%04x", c); continue; } + std::putchar(c); + } + std::putchar('"'); +} +// Appends one JSON string literal, with the escaping `esc` applies, so a +// list entry and a scalar field are encoded by one rule. A payload that +// decoded under the fixed-array revision is encoded to the same bytes +// here: that revision escaped `"` and `\\` and passed control characters +// through, and a control character passed through was not JSON, so no +// payload the engine accepted contained one. +// `value`, when given, is appended to `s` after an `=`, inside the same +// string literal: one `env` entry is one `NAME=value` string. +inline bool add(list& l, const char* s, const char* value = nullptr) { + bool ok = true; + if (l.len) ok = ok && l.put(','); + ok = ok && l.put('"'); + auto body = [&](const char* text) { + for (const char* p = text; ok && *p; ++p) { + unsigned char c = (unsigned char)*p; + if (c == '"' || c == '\\') { ok = l.put('\\') && l.put((char)c); continue; } + if (c < 0x20) { + static const char hex[] = "0123456789abcdef"; + ok = l.put('\\') && l.put('u') && l.put('0') && l.put('0') + && l.put(hex[c >> 4]) && l.put(hex[c & 0xf]); + continue; + } + ok = l.put((char)c); + } + }; + body(s); + if (value) { ok = ok && l.put('='); body(value); } + ok = ok && l.put('"'); + return ok; +} +} // namespace json_detail + // ── Build-graph nodes (mcpp 2026.8.5.1+) ──────────────────────────────── // Declare WORK instead of doing it. A build program is a good place to decide // what the build looks like and a bad place to perform it: work done here is @@ -287,6 +397,7 @@ namespace roles { // and the module graph during prepare, so an output whose name is unknown // cannot be built. Content may arrive later; names may not. struct action { + using list = json_detail::list; const char* id = ""; const char* role = roles::source; // one of `roles::{source, check, object, artifact, prepare}` const char* description = ""; @@ -372,105 +483,61 @@ struct action { std::printf("}\n"); } private: - // One list field, held already serialised (`"a","b"`) so submit() prints - // it as it is. Owning and std-free, and both words are constraints this - // module carries: it may be compiled before a std BMI exists, so it must - // not `import std;`, and its exported interface must name no std type, so - // `std::string` may not appear in a signature. Neither forbids the heap: - // storage is `realloc` from the `` already in the global module - // fragment, and no exported signature mentions this type. - // - // An earlier revision held six fixed arrays (8192 bytes for `inputs` and - // `outputs`, chosen for a protoc command line) and a declaration that did - // not fit was refused. The bound was in bytes of serialised JSON, so a - // consumer's checkout depth decided whether a resource list of 44 files - // fit (HuxerUI#130 measured the margin at 45 bytes), and `outputs` is the - // one list an author cannot shorten: an output the program does not name - // cannot be built, and there is no depfile for outputs. See - // .agents/docs/2026-09-13-four-upstream-asks-from-a-ui-framework.md. - struct list { - char* p = nullptr; - unsigned long len = 0, cap = 0; - list() = default; - list(const list& o) { take(o); } - list& operator=(const list& o) { if (this != &o) { len = 0; take(o); } return *this; } - ~list() { std::free(p); } - const char* c_str() const { return p ? p : ""; } - // Grows by doubling. False only when the allocator refuses. - bool reserve(unsigned long need) { - if (need <= cap) return true; - unsigned long c = cap ? cap : 256; - while (c < need) c *= 2; - void* q = std::realloc(p, c); - if (!q) return false; - p = static_cast(q); - cap = c; - return true; - } - bool put(char c) { - if (!reserve(len + 2)) return false; - p[len++] = c; - p[len] = 0; - return true; - } - void take(const list& o) { - if (!o.len) { if (p) p[0] = 0; return; } - if (!reserve(o.len + 1)) return; - for (unsigned long i = 0; i <= o.len; ++i) p[i] = o.p[i]; - len = o.len; - } - }; list inputs_, outputs_, command_, provides_, imports_, targets_, env_; // `prepare` only: see `output_dir()` above. A plain `const char*`, not a // `list`: it is one directory, never a JSON array. const char* outputDir_ = ""; const char* cwd_ = ""; mutable bool overflow_ = false; - static void esc(const char* s) { - std::putchar('"'); - for (const char* p = s; *p; ++p) { - unsigned char c = (unsigned char)*p; - if (c == '"' || c == '\\') { std::putchar('\\'); std::putchar(c); continue; } - // Any control character has to be escaped or the payload is not - // JSON at all. \n was handled before; \t and \r reach this code - // through ordinary Windows paths and log text. - if (c < 0x20) { std::printf("\\u%04x", c); continue; } - std::putchar(c); - } - std::putchar('"'); - } - // Appends one JSON string literal, with the escaping `esc` applies, so a - // list entry and a scalar field are encoded by one rule. A payload that - // decoded under the fixed-array revision is encoded to the same bytes - // here: that revision escaped `"` and `\\` and passed control characters - // through, and a control character passed through was not JSON, so no - // payload the engine accepted contained one. - // `value`, when given, is appended to `s` after an `=`, inside the same - // string literal: one `env` entry is one `NAME=value` string. + static void esc(const char* s) { json_detail::esc(s); } + // One entry of a list field; a list the allocator refused marks the + // declaration cut short (`overflow`). bool add(list& l, const char* s, const char* value = nullptr) { - bool ok = true; - if (l.len) ok = ok && l.put(','); - ok = ok && l.put('"'); - auto body = [&](const char* text) { - for (const char* p = text; ok && *p; ++p) { - unsigned char c = (unsigned char)*p; - if (c == '"' || c == '\\') { ok = l.put('\\') && l.put((char)c); continue; } - if (c < 0x20) { - static const char hex[] = "0123456789abcdef"; - ok = l.put('\\') && l.put('u') && l.put('0') && l.put('0') - && l.put(hex[c >> 4]) && l.put(hex[c & 0xf]); - continue; - } - ok = l.put((char)c); - } - }; - body(s); - if (value) { ok = ok && l.put('='); body(value); } - ok = ok && l.put('"'); + const bool ok = json_detail::add(l, s, value); if (!ok) overflow_ = true; return ok; } }; + +// A RUNNER AS ONE STATEMENT (protocol 19; design 2026-11-15 R2): the table a +// runner is in data (`runner = { cmd = [...], ... }`, SPEC-013 §10), field for +// field. `name` empty is the default runner; otherwise `mcpp run --runner +// ` reaches it. `{}` is the artifact, `{pkg:}` the root of a +// package of `require()` (a program outside the package's `bin/`, such as +// `{pkg:emsdk}/emscripten/emrun`), installed by the first run, never by a +// build. A name is stated by this type or by the token calls (`runner`, +// `runner_longlived`), not by both. +// +// mcpp::runner_spec web; +// web.name = "web"; +// web.longlived = true; +// web.arg("{pkg:emsdk}/emscripten/emrun").arg("{}").require("xim:emsdk"); +// web.submit(); +struct runner_spec { + const char* name = ""; + const char* host = ""; // "same-arch" | "any"; empty: unstated + bool longlived = false; // no natural end; `mcpp test` does not use it + bool exclusive = false; // runs do not overlap + bool enabled = true; // false: used once a project or this machine enables it + runner_spec& arg(const char* token) { put(cmd_, token); return *this; } + runner_spec& hosts(const char* pair) { put(hosts_, pair); return *this; } // `-` + // `requires` is a C++ keyword: the nearest spelling of the table's key. + runner_spec& require(const char* pkg) { put(requires_, pkg); return *this; } + void submit() const { + std::printf("mcpp:runner-spec={\"name\":"); json_detail::esc(name); + std::printf(",\"host\":"); json_detail::esc(host); + std::printf(",\"longlived\":%s,\"exclusive\":%s,\"enabled\":%s", + longlived ? "true" : "false", exclusive ? "true" : "false", enabled ? "true" : "false"); + if (overflow_) std::printf(",\"overflow\":true"); + std::printf(",\"cmd\":[%s]", cmd_.c_str()); + std::printf(",\"hosts\":[%s]", hosts_.c_str()); + std::printf(",\"requires\":[%s]}\n", requires_.c_str()); + } +private: + void put(json_detail::list& l, const char* s) { if (!json_detail::add(l, s)) overflow_ = true; } + json_detail::list cmd_, hosts_, requires_; + bool overflow_ = false; +}; inline void rerun_if_changed(const char* path) { std::printf("mcpp:rerun-if-changed=%s\n", path); } // mcpp#359: re-run when the SET of files matching `pattern` changes — a file // appearing or disappearing, not its contents (declare those with diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index a89032736..6940ae62d 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -40,6 +40,7 @@ import mcpp.toolchain.dialect; import mcpp.toolchain.provider; import mcpp.toolchain.registry; import mcpp.toolchain.triple; // shared_soname_flag decides by TARGET, not host +import mcpp.toolchain.target_rows; // the row's companions (N33) import mcpp.toolchain.fingerprint; // hash_string: a platform seal's digest import mcpp.toolchain.msvc_coroutines; import mcpp.provider; @@ -2386,22 +2387,47 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); return t ? (t->is_pe() || t->is_wasm()) : bool(mcpplibs::os::is_windows); }(); + // THE ROW'S COMPANIONS (design 2026-11-15 N33): removed before the link by + // `link-output`, so one present afterwards is this link's. An earlier + // link's `.wasm`, left beside a launcher linked with `-sSINGLE_FILE=1`, + // was staged as the program's module. + std::string companions; + if (auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple)) + if (auto const* row = mcpp::toolchain::target::find_row(t->str())) + for (auto c : row->companions) companions += (companions.empty() ? "" : ",") + std::string(c); + const std::string companionOption = companions.empty() ? std::string() : " --companions " + companions; auto publish_link = [&](std::string cmd) { if constexpr (mcpplibs::os::is_windows) { - return "$mcpp link-output --output $out -- " + cmd; + return "$mcpp link-output --output $out --aside" + companionOption + " -- " + cmd; } else { + if (!companions.empty()) return "$mcpp link-output --output $out" + companionOption + " -- " + cmd; if (inPlaceTarget) return cmd; - const auto pos = cmd.find("-o $out"); - if (pos == std::string::npos) return cmd; - cmd.replace(pos, 7, "-o $out.tmp"); - return cmd + " && mv -f $out.tmp $out"; + if (cmd.find("-o $out") == std::string::npos) return cmd; + // Through a temporary name, `/.` (cmd_link_output): one + // command, so a permit around it is held until the artifact is in + // place (2026.11.16.1; before it a shell sequence that renamed the + // file after the linker's command had ended). + return "$mcpp link-output --output $out --temporary -- " + cmd; } }; { constexpr bool useRsp = true; + // A LINK HOLDS A PERMIT AS A COMPILE DOES. Under the detached schedule + // ninja is given six slots per compiler and the generation's semaphore + // bounds the compilers, so every link a build reached at once ran at + // once: a change that every test program links relinked mcpp's 26 tests + // together, at 1 to 1.8 GB each, and a 16 GB machine ran out of memory. + // A link takes a permit of the same semaphore, so compiles and links + // together are what this machine runs at once; the graph still names no + // number (e2e 861). An archive copies its members and holds none. + const std::string linkPrefix = plan.scheduleCompilerCap > 0 + ? std::string("$mcpp __compiler-cap --sem .mcpp-sched --output $out -- ") + : std::string{}; auto link_rule = [&](std::string_view name, std::string cmd, std::string_view desc) { - if (desc != "AR") cmd = publish_link(std::move(cmd)); + // The permit wraps the publication: the attempt's output is the + // artifact in place when the command it holds exits. + if (desc != "AR") cmd = linkPrefix + publish_link(std::move(cmd)); append(std::format("rule {}\n", name)); if (useRsp) { if (auto pos = cmd.find("$in"); pos != std::string::npos) @@ -3432,17 +3458,8 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, // #622 A5: on `wasm32-emscripten` the link command is a single emcc // invocation that writes the named output (`.js`, this plan's - // artifact naming — see triple.cppm) AND `.wasm` beside it in one - // run. The `.wasm` is not a second edge (nothing would sequence it against - // the first) and not a naming convention applied after the fact — it is - // declared as an implicit output of the SAME edge, so `ninja -t clean`, - // the incremental check and `mcpp pack`'s graph walk all see it. - const auto planTargetTriple = - mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - const bool wasmTarget = - planTargetTriple - && planTargetTriple->object_format() - == mcpp::toolchain::triple::ObjectFormat::Wasm; + // artifact naming — see triple.cppm) AND, unless `-sSINGLE_FILE=1`, + // `.wasm` beside it in one run: a companion of the row (N33). // Link units for (auto& lu : plan.linkUnits) { @@ -3554,16 +3571,12 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, std::string implicitOut; if (!lu.importLibrary.empty()) implicitOut = " | " + escape_ninja_path(lu.importLibrary); - // The `.wasm` module emcc writes beside the JavaScript launcher for - // every executable link on this row (see the comment above the loop). - // Not the test binary's rule alone and not the `bin` rule alone: both - // ARE the executable form on this row, so both get the sibling. - if (wasmTarget - && (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary)) { - auto wasmSibling = lu.output; wasmSibling.replace_extension(".wasm"); - implicitOut += (implicitOut.empty() ? " | " : " ") - + escape_ninja_path(wasmSibling); - } + // THE ROW'S COMPANIONS ARE NOT DECLARED OUTPUTS (design 2026-11-15 + // N33): the `.wasm` Emscripten writes beside the launcher is absent + // under `-sSINGLE_FILE=1`, and a declared output the link did not + // write failed a clean pack and relinked every time. The link edge + // removes them first (`link-output --companions`), so a companion + // present after the link is this link's. // The link map, DECLARED rather than merely written. It is produced by // a flag on the link command, so it cannot be its own edge — a second // edge claiming to produce it would run the link twice. Declaring it diff --git a/src/build/plan.cppm b/src/build/plan.cppm index a78a550db..a348606bd 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -797,6 +797,14 @@ std::vector runtime_search_closure( bool path_is_under_any(const std::filesystem::path& p, const std::vector& roots); +// THE TEST FRAMEWORKS A PACKAGE STATES (docs/08): each entry of `m` that +// states its guide, its protocol, its session or capabilities, or that it +// begins a test, appended to `out`. `dir` is the package's root, which a +// guide's path is relative to. Read by the plan and by `mcpp test --guide`, +// which reads a resolved graph and plans nothing (#932). +void collect_test_frameworks(const mcpp::manifest::Manifest& m, const std::filesystem::path& dir, + std::vector& out); + // True if a source file defines a top-level `int main(`/`auto main(` entry, // ignoring comments and string/raw-string literals. Drives the archive-vs-inline // choice for kind="lib" dependencies (see plan.cppm). @@ -903,6 +911,20 @@ trim_map(const std::vector& packages, namespace mcpp::build { +void collect_test_frameworks(const mcpp::manifest::Manifest& m, const std::filesystem::path& dir, + std::vector& out) { + for (auto const& entry : m.targets) { + if (entry.kind != mcpp::manifest::Target::Entry) continue; + if (entry.testGuide.empty() && !entry.testProtocol && !entry.testMain + && entry.testCapabilities.empty() && entry.testSession.empty()) continue; + out.push_back({ + qualified_package_name(m), m.package.version, + std::format("{}/{}", m.package.name, entry.name), entry.testProtocol, + entry.testGuide.empty() ? std::string{} : (dir / entry.testGuide).lexically_normal().generic_string(), + m.package.repo, entry.testCapabilities, entry.testSession}); + } +} + std::string qualified_package_name(const mcpp::manifest::Manifest& manifest) { if (!manifest.package.namespace_.empty() && manifest.package.name.starts_with(manifest.package.namespace_ + ".")) { @@ -1976,7 +1998,16 @@ make_plan(const mcpp::manifest::Manifest& manifest, // The patterns were checked where they were read. // `ArtifactNaming` holds views; these are views into the platform's own // strings, which outlive the plan's construction. - if (platform && platform->active) { + // + // THE STATEMENT NAMES THEM ON EVERY ROW (design 2026-11-15 N9), with a + // platform or without: a build program, this machine or the project that + // names a program `{name}` is followed on a built-in row as on a + // platform's. The row's own suffix (`.exe` of a Windows row) was appended + // regardless where no platform was selected, and a program the project + // named `cmd.exe` was `cmd.exe.exe` on x86_64-windows-gnu and `cmd.exe` on + // x86_64-windows-musl. A platform's own suffix still stands against a + // build program (`keep_platform_naming`). + if (platform) { constexpr std::string_view kName = "{name}"; auto const& l = platform->fields.link; if (l.exe.set && mcpp::platform::split_name_pattern(l.exe.value)) { @@ -3597,20 +3628,9 @@ make_plan(const mcpp::manifest::Manifest& manifest, // The test frameworks of the graph: each package's entry that states its // guide, its protocol or that it begins a test (docs/08). - for (std::size_t i = 0; i < std::max(packages.size(), 1); ++i) { - auto const& m = i == 0 ? manifest : packages[i].manifest; - auto const& dir = i == 0 ? projectRoot : packages[i].root; - for (auto const& entry : m.targets) { - if (entry.kind != mcpp::manifest::Target::Entry) continue; - if (entry.testGuide.empty() && !entry.testProtocol && !entry.testMain - && entry.testCapabilities.empty() && entry.testSession.empty()) continue; - plan.testFrameworks.push_back({ - qualified_package_name(m), m.package.version, - std::format("{}/{}", m.package.name, entry.name), entry.testProtocol, - entry.testGuide.empty() ? std::string{} : (dir / entry.testGuide).lexically_normal().generic_string(), - m.package.repo, entry.testCapabilities, entry.testSession}); - } - } + for (std::size_t i = 0; i < std::max(packages.size(), 1); ++i) + collect_test_frameworks(i == 0 ? manifest : packages[i].manifest, + i == 0 ? projectRoot : packages[i].root, plan.testFrameworks); // 4. Link units (one per [targets.X]) // When any TestBinary target exists, skip Binary/Library/SharedLibrary diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d6022094d..a959b61e1 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -715,6 +715,12 @@ export struct BuildContext { // refines: the plan knows the set of formats and this knows who provides // each. std::map> packFormatProviders; + // By provider (a package in `packFormatProviders`): the programs it stated + // with `mcpp::pack_program`, and the warnings it stated in this pass, which + // a pack whose provider submitted nothing gives as the provider's reasons + // (design 2026-11-15 N30). + std::map> packPrograms; + std::map> packReasons; }; // The selected member a package acts for in a packaging pass: the package @@ -1134,6 +1140,14 @@ prepare_build(bool print_fingerprint, bool includeDevDeps = false, std::vector extraTargets = {}, BuildOverrides overrides = {}); +// THE RESOLVED GRAPH, AND NOTHING AFTER IT (#932): the packages `prepare_build` +// reads -- the manifest and its workspace, the toolchain request, the graph's +// resolution -- with the same phases, and no toolchain resolved after the +// graph, no build program, no scan and no plan. For a question the declared +// data answers (`mcpp test --guide`), which a plan would answer minutes later. +export std::expected, std::string> +resolve_graph(bool includeDevDeps, BuildOverrides overrides); + // ONE CONFIGURATION, PREPARED WITHOUT A BUILD (SPEC-013 §9): what `mcpp self // env --target` reports and what `mcpp toolchain install --target` fetches -- // the toolchain, the platform package, the packages it adds and the tools it diff --git a/src/build/prepare/build_programs.cpp b/src/build/prepare/build_programs.cpp index 8bbbe992b..d8ab7211d 100644 --- a/src/build/prepare/build_programs.cpp +++ b/src/build/prepare/build_programs.cpp @@ -615,6 +615,7 @@ std::expected step9_member_build_programs(PrepareState& state state.runnerProvider, pkg.manifest.package.name)); rootBc.runner.assign(bc.runner.begin() + static_cast(runnerN), bc.runner.end()); + rootBc.runnerFields = bc.runnerFields; state.runnerProvider = pkg.manifest.package.name; } for (auto const& [name, nr] : bc.namedRunners) { diff --git a/src/build/prepare/driver.cpp b/src/build/prepare/driver.cpp index 09fa5efe4..7e4456f5d 100644 --- a/src/build/prepare/driver.cpp +++ b/src/build/prepare/driver.cpp @@ -241,4 +241,26 @@ prepare_build_pass(bool print_fingerprint, } // namespace +std::expected, std::string> +resolve_graph(bool includeDevDeps, BuildOverrides overrides) { + read_build_programs_marker(overrides); + auto statePtr = std::make_unique(/*print_fingerprint=*/false, includeDevDeps, + std::vector{}, std::move(overrides)); + PrepareState& state = *statePtr; + // A restart a phase asks of `prepare_build` (a toolchain, platform or + // target statement) belongs to a build; the graph is read as it stands. + struct NoRestart { + ~NoRestart() { (void)take_toolchain_restart(); (void)take_platform_restart(); (void)take_target_restart(); } + } noRestart; + if (auto r = phase0_manifest_and_workspace(state); !r) return std::unexpected(r.error()); + if (auto r = check_engine_floors(state, /*rootOnly=*/true); !r) return std::unexpected(r.error()); + if (auto r = phase1_toolchain_spec_and_axes(state); !r) return std::unexpected(r.error()); + if (auto r = phase2_define_toolchain_resolver(state); !r) return std::unexpected(r.error()); + if (auto r = phase3_xlings_before_graph(state); !r) return std::unexpected(r.error()); + if (auto r = phase4a_graph_load(state); !r) return std::unexpected(r.error()); + if (auto r = phase4b_graph_worklist(state); !r) return std::unexpected(r.error()); + if (auto r = check_engine_floors(state, /*rootOnly=*/false); !r) return std::unexpected(r.error()); + return std::move(state.packages); +} + } // namespace mcpp::build diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 6649f451b..9d394ea77 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -2049,6 +2049,7 @@ static std::expected step6_dependency_build_programs(PrepareS state.runnerProvider, pkg.manifest.package.name)); } state.m->buildConfig.runner = std::move(supplied); + state.m->buildConfig.runnerFields = bcDep.runnerFields; state.runnerProvider = pkg.manifest.package.name; } for (auto const& [name, nr] : bcDep.namedRunners) { diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index ceb891cb9..be542b595 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -736,6 +736,32 @@ step4b_identity_version_merge(PrepareState& state, WorklistItemCtx& ctx, // * The secondary version must be a leaf (no own // transitive deps) — recursive mangling is // deferred to a follow-up. + // A BUILD DEPENDENCY IS NOT RENAMED (design 2026-11-15 + // N29). It is a module of a build program, which the + // rename does not rewrite: the program that asked keeps + // importing the authored name, and the stage of its + // package's sources -- a data package has none -- failed + // with a message about the package rather than about the + // two requests (a project asking for mcpp:plugins 0.21.0 + // over a platform that asks for 0.20.0). + if (item.fromBuildTable) { + return std::unexpected(std::format( + "build dependency '{}{}{}' has irreconcilable versions:\n" + " '{}' (constraint '{}') requested by '{}'\n" + " '{}' (constraint '{}') requested by '{}'\n" + "SemVer merge: {}\n" + "A build dependency is a module of a build program, and a build " + "loads one version of it. Align the requests on one version{}.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, effective_constraint(state, it->second), it->second.requestedBy, + spec.version, item.originalConstraint, item.requestedBy, + merged.error(), + state.platform.active + ? std::format(", or select the release of {} that carries the " + "version you need", state.platform.display()) + : std::string()) + + override_hint(key, it->second.version, spec.version)); + } if (item.consumerDepIndex == kMainConsumer) { return std::unexpected(std::format( "dependency '{}{}{}' has irreconcilable versions:\n" diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 1faa6ab73..1bf8e3b8a 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -1321,6 +1321,14 @@ static std::expected step13_build_graph_actions(PrepareState& auto& who = ctx.packFormatProviders[f]; if (std::ranges::find(who, owner) == who.end()) who.push_back(owner); } + // The program each provider assembles from, and what it said + // (N30): read by `mcpp pack` for the format's providers only. + if (!mm.buildConfig.packFormats.empty()) { + if (!mm.buildConfig.packPrograms.empty()) + ctx.packPrograms[owner] = mm.buildConfig.packPrograms; + if (!mm.buildConfig.statedReasons.empty()) + ctx.packReasons[owner] = mm.buildConfig.statedReasons; + } }; collect(*state.m, 0); for (std::size_t i = 1; i < state.packages.size(); ++i) diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 483d28cab..e7c603789 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -212,6 +212,8 @@ void define_platform(const mp::Statement& s, std::string_view shortName, std::st r.sharedSuffix = set.keep(d.sharedSuffix); r.loader = set.keep(d.loader); r.descriptorKeys = set.keep(d.descriptorKeys); + r.companions = set.keep(d.companions); + r.exceptions = set.keep(d.exceptions); r.execution = set.keep(d.execution.empty() ? seg[1] : d.execution); r.defined = true; set.rows.push_back(r); @@ -232,10 +234,14 @@ void define_platform(const mp::Statement& s, std::string_view shortName, std::st p.runner = set.keep(sp.runner); p.runnerHosts = set.keep(sp.runnerHosts); p.runnerRequires = set.keep(sp.runnerRequires); + p.runnerHost = set.keep(sp.runnerHost); + p.runnerEnabled = sp.runnerEnabled; + p.runnerExclusive = sp.runnerExclusive; set.providers.push_back(p); }; for (auto const& sys : s.systems) - set.systems.push_back({ set.keep(sys.name), set.keep(sys.uname) }); + set.systems.push_back({ set.keep(sys.name), set.keep(sys.uname), sys.resources, + set.keep(sys.commandScripts) }); // A platform the engine's own providers name (openkal-platform) is served by // those; its shorthands are not registered twice. const bool builtin = std::ranges::any_of(mcpp::toolchain::builtin::kProviders, [&](auto const& b) { @@ -1050,6 +1056,11 @@ mp::Statement platform_statement(const PrepareState& state, std::vectorrunner.begin(), p->runner.end()); s.runner.requires_.assign(p->runnerRequires.begin(), p->runnerRequires.end()); + s.runner.host = std::string(p->runnerHost); + s.runner.enabled = p->runnerEnabled; + s.runner.enabledSet = !p->runnerEnabled; + s.runner.exclusive = p->runnerExclusive; + s.runner.exclusiveSet = p->runnerExclusive; s.runner.origin = mp::Origin{ mp::Tier::BuiltIn, {}, 0, std::format("the provider '{}' of {}", p->name, state.targetChoice.identity) }; s.runner.set = true; @@ -1070,6 +1081,16 @@ mp::Statement platform_statement(const PrepareState& state, std::vectorbuildConfig.programPlatform); // A build program's runner and named runners (`mcpp::runner`), the BSP // spelling of the same fields. + // The rest of a runner's table (`runner_spec`, R2): the same fields as + // data, so `{pkg:}` and `enabled` mean in a build program what they + // mean in a platform. + auto table = [](mp::Runner& r, const mcpp::manifest::NamedRunner& f) { + r.requires_ = f.requires_; + if (!f.hosts.empty()) { r.hosts = f.hosts; r.hostsSet = true; } + if (!f.host.empty()) r.host = f.host; + if (!f.enabled) { r.enabled = false; r.enabledSet = true; } + if (f.exclusive) { r.exclusive = true; r.exclusiveSet = true; } + }; if (!state.m->buildConfig.runner.empty()) { program.runner.cmd = state.m->buildConfig.runner; program.runner.set = true; @@ -1079,11 +1100,19 @@ mp::Statement platform_statement(const PrepareState& state, std::vectorbuildConfig.runnerFields); } for (auto const& [name, nr] : state.m->buildConfig.namedRunners) { auto it = state.namedRunnerProvider.find(name); - program.runners[name] = mp::Words{ nr.argv, mp::Origin{ mp::Tier::Program, {}, 0, - it == state.namedRunnerProvider.end() ? state.m->package.name : it->second }, true }; + mp::Runner r; + r.cmd = nr.argv; + r.origin = mp::Origin{ mp::Tier::Program, {}, 0, + it == state.namedRunnerProvider.end() ? state.m->package.name : it->second }; + r.set = !nr.argv.empty(); + r.longlived = nr.longLived; + r.longlivedSet = nr.longLived; + table(r, nr); + program.runners[name] = std::move(r); } mcpp::platform::merge(s, program); std::vector config; @@ -1100,12 +1129,23 @@ mp::Statement platform_statement(const PrepareState& state, std::vector_RUNNER_ENABLED`, the + // spelling of the variables beside it, nearest of all. + const auto enabledKey = target_env_name(t, "RUNNER_ENABLED"); + if (auto e = mcpp::invocation::get(enabledKey); e && !e->empty()) { + if (*e == "true" || *e == "1") { s.runner.enabled = true; s.runner.enabledSet = true; } + else if (*e == "false" || *e == "0") { s.runner.enabled = false; s.runner.enabledSet = true; } + else mcpp::diag::warning("runner/enabled", std::format( + "{}={} is not true or false (ignored)", enabledKey, *e)); + } return s; } diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index a4e460ae5..866c157ee 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -246,6 +246,16 @@ static std::expected step11_scan_sources(PrepareState& state) } return mcpp::modgraph::scan_packages(scannedPackages, closures); }(); + // A `sources` entry that names a file which is not there (#938) stops a + // build of the tree it is written in, and is a warning in a package + // fetched into a store: its consumer cannot change it, and an engine that + // refused it would refuse a release that built before. + for (std::size_t i = 0; i < state.packages.size(); ++i) { + if (!state.compilesHere(i)) continue; + auto& into = state.editablePackage(i) ? state.scan.errors : state.scan.warnings; + for (auto& e : mcpp::modgraph::unnamed_sources(state.packages[i].root, state.packages[i].manifest)) + into.push_back(std::move(e)); + } if (!state.scan.errors.empty()) { std::string msg = "scanner errors:\n"; for (auto& e : state.scan.errors) msg += " " + e.format() + "\n"; diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 4d6328e96..781b2b993 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -1448,6 +1448,23 @@ static std::expected step1_device_axis_and_layer_merge(Prepar // `true` has meant since 2026.9.12.3. The flag is in the dialect set, so the // model is part of every cache key. const bool emscripten = state.cfgCtx().os == "emscripten"; + // THE ROW'S MODEL WHERE THE PROJECT STATES NONE (design 2026-11-15 N26): + // wasm32-emscripten's row states `exceptions = "wasm"`, and a program's + // `catch` was never reached under the SDK's default. Data, not a rule of + // the engine; `exceptions = false` states none. + if (emscripten && !state.m->buildConfig.abiExceptionsDeclared) + if (auto t = mcpp::toolchain::triple::parse(state.overrides.target_triple)) + if (auto const* row = mcpp::toolchain::target::find_row(t->str()); row && !row->exceptions.empty()) { + state.m->buildConfig.abiExceptions = true; + state.m->buildConfig.abiExceptionsModel = std::string(row->exceptions); + // The long jump follows WebAssembly exception handling (D6), + // stated for the C translation units too, which do not read + // the C++ flag; under another model it is the SDK's default. + if (!state.m->buildConfig.abiLongjmpDeclared && row->exceptions == "wasm") { + state.m->buildConfig.abiLongjmpDeclared = true; + state.m->buildConfig.abiLongjmpModel = "wasm"; + } + } const bool abiExceptionsRendered = state.m->buildConfig.abiExceptions && emscripten; if (abiExceptionsRendered) { const char* flag = state.m->buildConfig.abiExceptionsModel == "wasm" diff --git a/src/build/prepare/windows_resources.cpp b/src/build/prepare/windows_resources.cpp index 48c37ee63..c6ad7d931 100644 --- a/src/build/prepare/windows_resources.cpp +++ b/src/build/prepare/windows_resources.cpp @@ -264,7 +264,9 @@ std::expected step13_windows_resources(PrepareState& state, B // see a typo in `icon = …` at all and only the Windows job went red, // which is the same "find out late" failure the hard error exists to // remove. Existence is checked everywhere; only COMPILATION is PE-only. - // 3. On a non-PE target nothing is compiled — no units, no warning, + // 3. On a target whose programs Windows does not run (not PE, or a PE + // row whose execution system is another, UEFI's) nothing is + // compiled — no units, no warning, // byte-identical build. This is what makes `cfg(windows)` unnecessary // (and it could not be used anyway: the conditional channel carries // BuildInputs only). @@ -634,7 +636,13 @@ std::expected step13_windows_resources(PrepareState& state, B return {}; }; - if (trip.is_pe()) + // Resources are a property of a program whose system reads them, + // not of the PE container: a UEFI application is a PE image the + // firmware loads, and `[resources]` compiled into it failed the + // build (design 2026-11-15 N16). The system table states it for + // the row's execution system (SPEC-016 §2.3). + auto const* sys = mcpp::toolchain::target::find_system(trip.execution_system()); + if (trip.is_pe() && sys && sys->resources) if (auto r = plan_resources(); !r) return std::unexpected(r.error()); } } diff --git a/src/build/runner_lookup.cppm b/src/build/runner_lookup.cppm index 800784eaa..fc252dc31 100644 --- a/src/build/runner_lookup.cppm +++ b/src/build/runner_lookup.cppm @@ -79,30 +79,32 @@ inline Lookup locate(std::string_view argv0, const bool hasDir = a0.is_absolute() || argv0.find('/') != std::string_view::npos || argv0.find('\\') != std::string_view::npos; - if (hasDir) { - std::error_code ec; - if (detail::executable_file(a0)) out.program = std::filesystem::absolute(a0, ec); - out.searched.push_back(a0.parent_path()); - return out; - } - // A bare name on Windows is a program with the system's suffix, as the - // shell finds it: `okrun` is `okrun.exe`. The file without a suffix is not - // one: the system starts no file without one (`CreateProcess` appends - // `.exe`), and every file is "executable" there. A package for every - // system carries both -- dirun's `bin/dirun`, a POSIX launcher, beside - // `bin/dirun.exe` -- and the launcher was taken and refused - // (`executable format error`, 2026-11-14). - auto in_dir = [&](const std::filesystem::path& d) -> std::optional { + // A name on Windows is a program with the system's suffix, as the shell + // finds it: `okrun` is `okrun.exe`. The file without a suffix is not one: + // the system starts no file without one (`CreateProcess` appends `.exe`), + // and every file is "executable" there. A package for every system + // carries both -- dirun's `bin/dirun`, a POSIX launcher, beside + // `bin/dirun.exe` -- and the launcher was taken and refused (`executable + // format error`, 2026-11-14). A name with a directory follows the same + // rule (design 2026-11-15 R2, A3): `../tools/x` could not be started. + auto program_at = [&](const std::filesystem::path& c) -> std::optional { if constexpr (mcpplibs::os::is_windows) { - if (!a0.has_extension()) { + if (!c.has_extension()) { for (auto ext : { ".exe", ".cmd", ".bat" }) - if (auto c = d / (a0.string() + ext); detail::executable_file(c)) return c; + if (auto s = std::filesystem::path(c.string() + ext); detail::executable_file(s)) return s; return std::nullopt; } } - if (auto c = d / a0; detail::executable_file(c)) return c; + if (detail::executable_file(c)) return c; return std::nullopt; }; + if (hasDir) { + std::error_code ec; + if (auto c = program_at(a0)) out.program = std::filesystem::absolute(*c, ec); + out.searched.push_back(a0.parent_path()); + return out; + } + auto in_dir = [&](const std::filesystem::path& d) { return program_at(d / a0); }; for (auto const& d : depBinDirs) { out.searched.push_back(d); if (auto c = in_dir(d)) { out.program = *c; return out; } diff --git a/src/cli.cppm b/src/cli.cppm index 735d040c8..3d98b47b6 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -1378,8 +1378,11 @@ int run(int argc, char** argv) { .help("Place every `\\t` pair of FILE in one process (#734 E4); several lines naming one destination are its sources")) .action(cmd_stage)) .subcommand(cl::App("link-output") - .description("(internal: invoked by ninja on Windows) Move a linked artifact aside, then run the linker command after `--` (a running program keeps its file)") + .description("(internal: invoked by ninja) Remove the row's companions, move a linked artifact aside, then run the linker command after `--` (a running program keeps its file)") .option(cl::Option("output").takes_value().value_name("PATH").help("the artifact the command links")) + .option(cl::Option("companions").takes_value().value_name("SUFFIXES").help("the row's companions, removed before the link (`.wasm`)")) + .option(cl::Option("aside").help("move the previous artifact aside first (a Windows host)")) + .option(cl::Option("temporary").help("link under a temporary name in the artifact's directory, then rename it over the artifact (POSIX)")) .action([&passthrough](const cl::ParsedArgs& p) { return cmd_link_output(p, std::span(passthrough)); })) diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 6a4ff2562..7b09e87c6 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -1222,10 +1222,6 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, if (auto since = parsed.value("since")) to.since = *since; if (auto c = parsed.option("changed")) to.changed = c->get().values; to.guide = parsed.is_flag_set("guide"); - if (to.guide && (parsed.is_flag_set("workspace") || parsed.option_or_empty("package").values.size() > 1)) { - mcpp::ui::error("--guide describes one package's tests; name it with -p"); - return 2; - } if (auto sd = parsed.value("seed")) { std::uint64_t seed = 0; auto [p, ec] = std::from_chars(sd->data(), sd->data() + sd->size(), seed); @@ -1376,6 +1372,21 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, // selection design 2026-09-30, S4). auto selection = mcpp::cli::select_members(member_request(parsed)); if (!selection) { mcpp::ui::error(std::format("{}", selection.error())); return 2; } + // THE GUIDE (#932) is read from the resolved graph and nothing is planned + // or built, wherever it is asked: a package, a member, a workspace root. + // Several members are resolved by their configuration groups, as their + // tests are planned, and their frameworks united. + if (to.guide) { + std::vector> groups; + std::filesystem::path wsRoot; + if (*selection && ((*selection)->whole || (*selection)->members.size() > 1)) { + wsRoot = (*selection)->root; + if (auto g = workspace_groups(wsRoot, (*selection)->members, ov.target_triple)) groups = std::move(*g); + else for (auto const& mp : (*selection)->members) groups.push_back({mp}); + } + return mcpp::build::print_test_guide_of(ov, wsRoot, groups, + to.format == mcpp::build::TestMessageFormat::Json); + } if (*selection && ((*selection)->whole || (*selection)->members.size() > 1) && !to.archive.empty()) { mcpp::ui::error("--archive holds the tests of one package: select one member (-p)"); @@ -1829,8 +1840,13 @@ export int cmd_dyndep(const mcpplibs::cmdline::ParsedArgs& parsed) { // nothing holds any more are deleted here. A process started while the linker // writes fails to start (file not found, sharing violation) -- Windows never // starts an image that is being written -- rather than running half of one. -// (POSIX links write `.tmp` and rename it over the name in the -// edge's own shell command: a rename replaces a running program there.) +// On POSIX (`--temporary`) the linker writes `/.` -- the +// argument after its `-o`, renamed; the name keeps its extension, as a driver +// that names a file after it writes a second one otherwise (design 2026-11-15 +// N9) -- and the file is renamed over the artifact once the linker succeeds: a +// rename replaces a running program there, and a program started meanwhile +// never meets a partial file. The publication is one command, so an edge that +// holds a permit for the link holds it until the artifact is in place. export int cmd_link_output(const mcpplibs::cmdline::ParsedArgs& parsed, std::span command) { namespace fs = std::filesystem; @@ -1840,6 +1856,48 @@ export int cmd_link_output(const mcpplibs::cmdline::ParsedArgs& parsed, return 2; } std::error_code ec; + // The row's companions (`--companions .wasm`, design 2026-11-15 N33): the + // files named after the artifact that an earlier link wrote and this one + // may not, removed first. + if (auto c = parsed.option_or_empty("companions").value(); !c.empty()) + for (auto part : std::views::split(std::string_view(c), ',')) { + auto companion = out; + companion.replace_extension(std::string(std::string_view(part.begin(), part.end()))); + fs::remove(companion, ec); + } + // Moved aside when the edge asks (`--aside`, a Windows host, where a + // running program's file cannot be overwritten). An edge that reaches here + // for its companions alone does not: an aside file would be named like a + // companion (`app.js.old-1`). + if (parsed.is_flag_set("temporary")) { + std::vector argv(command.begin(), command.end()); + const auto temporary = out.parent_path() / ("." + out.filename().string()); + auto named = std::ranges::adjacent_find(argv, [&](const std::string& a, const std::string& b) { + return a == "-o" && fs::path(b) == out; + }); + if (named == argv.end()) { + std::println(libc::stderr(), "error: link-output --temporary: the linker command names no `-o {}`", + out.string()); + return 2; + } + *std::next(named) = temporary.string(); + const auto linked = mcpp::run::execute(argv, {.input = mcpp::run::Input::inherit, + .streams = mcpp::run::Streams::inherit}); + if (linked.exit_code != 0) { + fs::remove(temporary, ec); + return linked.exit_code; + } + fs::rename(temporary, out, ec); + if (ec) { + std::println(libc::stderr(), "error: link-output: cannot rename {} over {}: {}", + temporary.string(), out.string(), ec.message()); + fs::remove(temporary, ec); + return 1; + } + return 0; + } + if (!parsed.is_flag_set("aside")) + return mcpp::run::run_exec(std::vector(command.begin(), command.end())); const auto dir = out.parent_path().empty() ? fs::path(".") : out.parent_path(); const auto stem = out.filename().string() + ".old-"; for (auto it = fs::directory_iterator(dir, ec); !ec && it != fs::directory_iterator(); it.increment(ec)) diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 7a51226ea..df9ddef17 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -85,6 +85,14 @@ std::set package_source_files(const std::filesystem::path& root, const mcpp::manifest::Manifest& manifest); +// The entries of a package's `sources` that name one file by its path and +// find none (#938), each stated against the table it is written in. Exported +// rather than raised by the scan: whether such an entry stops the build is the +// caller's to decide by whose tree the package is (an editable tree is +// refused; a fetched package, which its consumer cannot change, is warned of). +std::vector unnamed_sources(const std::filesystem::path& root, + const mcpp::manifest::Manifest& manifest); + // Scan the entire package: collects all sources via manifest globs and returns a Graph. struct ScanResult { Graph graph; @@ -1164,6 +1172,58 @@ package_source_files(const std::filesystem::path& root, return all_files; } +// A SOURCE NAMED BY ITS PATH THAT IS NOT THERE (#938). An entry of a +// `sources` list without a pattern character names one file; when that file +// does not exist the entry selected nothing and the build went on, to fail at +// an importer ("failed to read compiled module", a module "imported but not +// provided") that names no manifest -- measured where an ignore rule left the +// file out of a fresh checkout. Each such entry of the lists this build reads +// is stated against the table that states it. A pattern that matches nothing +// is not: a selection may be empty on a target. A file an action of the build +// writes or a build program declares is the build's, not yet on disk, and an +// absolute entry is a build program's (`select_sources`). +std::vector unnamed_sources(const std::filesystem::path& root, + const mcpp::manifest::Manifest& manifest) { + namespace fs = std::filesystem; + std::set built; + auto key = [&](std::string_view p) { + auto n = native_path_from_generic(p); + return (n.is_absolute() ? n : root / n).lexically_normal().generic_string(); + }; + for (auto const& [path, _] : manifest.modules.declaredUnits) built.insert(key(path)); + for (auto const& a : manifest.buildConfig.actions) + for (auto const& o : a.outputs) built.insert(key(o)); + // The table an entry is stated in: the manifest's own `[build]` before the + // tables of the target were applied, then those tables, then a feature's. + auto const& stated = manifest.beforeConditionalMerge ? manifest.beforeConditionalMerge->buildConfig.sources + : manifest.buildConfig.sources; + auto table = [&](const std::string& entry) -> std::string { + if (std::ranges::find(stated, entry) != stated.end()) return "[build] sources"; + for (auto const& cc : manifest.conditionalConfigs) + if (std::ranges::find(cc.inputs.sources, entry) != cc.inputs.sources.end()) + return std::format("[target.'{}'.build] sources", cc.predicate); + for (auto const& [feature, globs] : manifest.buildConfig.featureSources) + if (std::ranges::find(globs, entry) != globs.end()) + return std::format("[features.{}] sources", feature); + return "sources"; + }; + std::vector out; + std::set seen; + for (auto const& entry : manifest.modules.sources) { + if (entry.empty() || entry.front() == '!' || entry.find_first_of("*?[{") != std::string::npos) continue; + if (!seen.insert(entry).second) continue; + if (native_path_from_generic(entry).is_absolute()) continue; + const auto at = key(entry); + if (built.contains(at)) continue; + std::error_code ec; + if (fs::exists(fs::path(at), ec)) continue; + out.push_back(ScanError{root / "mcpp.toml", 0, std::format( + "{}: \"{}\" names no file: a path without a pattern names one file, and {} does not exist " + "(a file a fresh checkout leaves out, an ignore rule, or a typo)", table(entry), entry, at)}); + } + return out; +} + namespace { // Phase 1: scan a single package, append units to result.graph.units; diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index db3475c4d..e2e95bc82 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -17,6 +17,7 @@ import mcpp.pack.compress; import mcpp.pack.zip; import mcpp.pack.polyglot; // a polyglot file is refused by name (§8) import mcpp.toolchain.triple; // the system of a package's leg (§8) +import mcpp.toolchain.target_rows; // the system table: `uname -s` words (N20) import mcpp.libs.toml; // a generated manifest quotes its keys (mcpp#911 B1) import mcpplibs.os; @@ -79,9 +80,11 @@ std::string posix_launcher(const Launch& l); // the launch facts as constants and builds it for `-windows-gnu` like // the single-file launcher: it finds the package from its own path // (`env::image`, not argv[0]), picks the image of this machine's architecture, -// looks for the runner in `libexec//windows-/` and then on PATH, -// and becomes it. A POSIX shell on Windows (Git Bash, MSYS, Cygwin) picks the -// script of the same name, which hands over to this program. +// runs the runner in `libexec//windows-/` when the package +// carries one (and otherwise refuses with 126, naming the systems it holds), +// or the runner on PATH when it carries none, and becomes it. A POSIX shell +// on Windows (Git Bash, MSYS, Cygwin) picks the script of the same name, which +// hands over to this program. std::string exe_launcher_manifest(std::string_view name, std::string_view version); std::string exe_launcher_main(const Launch& l); @@ -1012,6 +1015,26 @@ std::string posix_launcher(const Launch& l) { std::string cases; for (auto const& [arch, file] : l.entry) cases += std::format(" {}) entry={} ;;\n", sh_word(arch), sh_word(file)); + // THE SYSTEM BY THE SYSTEM TABLE (SPEC-016 §2.3): the word `uname -s` + // answers on each system the table knows, mapped to the system's name, + // which is how the package names its directories (`libexec// + // -`). A system the table does not know is its word in + // lower case. The word was used as it stood, so FreeBSD looked for + // `FreeBSD-x86_64` beside the `freebsd-x86_64` the package holds. + std::string systems; + for (auto const* s : mcpp::toolchain::target::systems()) + if (!s->uname.empty()) + systems += std::format(" {}) os={} ;;\n", sh_word(s->uname), sh_word(s->name)); + // AN ARCHITECTURE THIS MACHINE ALSO EXECUTES (the execution table, N23): a + // package without this machine's architecture takes one the machine runs + // besides its own, in the table's order -- an i686 package on x86_64 Linux. + std::string also; + for (auto const& e : mcpp::toolchain::target::execution_relations()) { + std::string words; + for (auto a : e.arches) words += (words.empty() ? "" : " ") + sh_word(a); + also += std::format(" {}) for a in {}; do case \" $arches \" in *\" $a \"*) arch=$a; break ;; esac; done ;;\n", + sh_word(e.host), words); + } // The runner: its program found in the package, then on PATH; none, the // image itself. std::string find, args = " \"$@\""; @@ -1020,11 +1043,19 @@ std::string posix_launcher(const Launch& l) { } else { args = runner_args(l.runner, { sh_word, "\"$image\"", "" }) + args; find = std::format("runner={}\n", sh_word(l.runner.front())); + // A PACKAGE THAT CARRIES ITS RUNNER RUNS IT, OR SAYS WHICH IT HOLDS. + // A runner of another release found on PATH is not this package's + // runner, so a machine the package holds none for is refused, naming + // the systems it holds. if (!l.loader.empty()) find = std::format( "loader={0}\n" "runner=\"$top/libexec/$loader/$os-$arch/$loader\"\n" - "[ -x \"$runner\" ] || runner={1}\n", sh_word(l.loader), sh_word(l.runner.front())); + "if [ ! -x \"$runner\" ]; then\n" + " held=$(ls \"$top/libexec/$loader\" 2>/dev/null | paste -s -d ' ' -)\n" + " printf '%s: this machine is %s-%s; the package holds %s for: %s\\n' {1} \"$os\" \"$arch\" \"$loader\" \"${{held:-none}}\" >&2\n" + " exit 126\n" + "fi\n", sh_word(l.loader), sh_word(l.name)); } return std::format( "#!/bin/sh\n" @@ -1050,10 +1081,15 @@ std::string posix_launcher(const Launch& l) { " *) arch=$(uname -m) ;;\n" "esac\n" "case \"$(uname -s)\" in\n" - " Linux) os=linux ;;\n" - " Darwin) os=macos ;;\n" + "{5}" " MINGW*|MSYS*|CYGWIN*) exec \"$top/bin/\"{0}.exe \"$@\" ;;\n" - " *) os=$(uname -s) ;;\n" + " *) os=$(uname -s | tr '[:upper:]' '[:lower:]') ;;\n" + "esac\n" + "arches={2}\n" + "case \" $arches \" in *\" $arch \"*) ;; *)\n" + " case \"$os-$arch\" in\n" + "{6}" + " esac ;;\n" "esac\n" "case \"$arch\" in\n" "{1}" @@ -1067,7 +1103,7 @@ std::string posix_launcher(const Launch& l) { " printf '%s: %s carries com.apple.quarantine (a browser downloaded it), and macOS may refuse to start it; remove the mark with: xattr -dr com.apple.quarantine %s\\n' {0} \"$runner\" \"$top\" >&2\n" "fi\n" "exec \"$runner\"{4}\n", - sh_word(l.name), cases, sh_word(arch_list(l)), find, args); + sh_word(l.name), cases, sh_word(arch_list(l)), find, args, systems, also); } std::string exe_launcher_manifest(std::string_view name, std::string_view version) { @@ -1093,6 +1129,9 @@ std::string exe_launcher_main(const Launch& l) { std::string words; const std::string entry = l.entry.empty() ? std::string() : l.entry.begin()->second; for (auto const& w : l.runner) words += std::format(" {},\n", cxx(w)); + std::string also; + for (auto const& e : mcpp::toolchain::target::execution_relations()) + for (auto a : e.arches) also += std::format(" {{{}, {}}},\n", cxx(e.host), cxx(a)); return std::format(R"MCPP(// The Windows launcher of {0} (design 2026-10-29 §4.3), written by mcpp pack. import std; import mcpplibs.os; @@ -1106,6 +1145,9 @@ constexpr std::string_view kEntry = {3}; constexpr std::string_view kRunner[] = {{ {4} std::string_view{{}}, }}; +constexpr std::pair kAlso[] = {{ +{5} {{}}, +}}; int main(int argc, char** argv) {{ // The package is the parent of the directory this program is in, found @@ -1120,9 +1162,14 @@ int main(int argc, char** argv) {{ std::string machine = os::env::get("PROCESSOR_ARCHITEW6432").value_or( os::env::get("PROCESSOR_ARCHITECTURE").value_or("")); for (auto& c : machine) c = static_cast(std::toupper(static_cast(c))); - const std::string arch = machine == "AMD64" || machine == "X86_64" ? "x86_64" + std::string arch = machine == "AMD64" || machine == "X86_64" ? "x86_64" : machine == "ARM64" || machine == "AARCH64" ? "aarch64" : machine; std::error_code dec; + // An architecture this machine also executes (the execution table, N23). + for (auto const& [host, also] : kAlso) + if (!fs::is_directory(top / "lib" / arch, dec) && host == "windows-" + arch + && fs::is_directory(top / "lib" / std::string(also), dec)) + arch = std::string(also); if (!fs::is_directory(top / "lib" / arch, dec)) {{ std::string all; for (auto const& d : fs::directory_iterator(top / "lib", dec)) @@ -1143,11 +1190,23 @@ int main(int argc, char** argv) {{ if (kRunner[0].empty()) {{ o.program = image; }} else {{ - fs::path carried; - if (!kLoader.empty()) - carried = top / "libexec" / std::string(kLoader) / ("windows-" + arch) / (std::string(kLoader) + ".exe"); - std::error_code ec; - o.program = !carried.empty() && fs::exists(carried, ec) ? carried : fs::path(std::string(kRunner[0])); + // A package that carries its runner runs it, or says which systems + // it holds one for: a runner of another release on PATH is not this + // package's runner (as the POSIX launcher). + o.program = fs::path(std::string(kRunner[0])); + if (!kLoader.empty()) {{ + const fs::path held = top / "libexec" / std::string(kLoader); + o.program = held / ("windows-" + arch) / (std::string(kLoader) + ".exe"); + std::error_code ec; + if (!fs::exists(o.program, ec)) {{ + std::string all; + for (auto const& d : fs::directory_iterator(held, ec)) + all += (all.empty() ? "" : " ") + d.path().filename().string(); + std::println(std::cerr, "{{}}: this machine is windows-{{}}; the package holds {{}} for: {{}}", + kName, arch, kLoader, all.empty() ? std::string("none") : all); + return 126; + }} + }} first = 1; }} for (std::size_t i = first; !kRunner[i].empty(); ++i) {{ @@ -1168,7 +1227,7 @@ int main(int argc, char** argv) {{ std::println(std::cerr, "{{}}: cannot start {{}}", kName, program); return 127; }} -)MCPP", l.name, cxx(l.name), cxx(l.loader), cxx(entry), words); +)MCPP", l.name, cxx(l.name), cxx(l.loader), cxx(entry), words, also); } // ─── mcpp-package.json ──────────────────────────────────────────────────── diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 553060b93..3fc913e59 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -256,6 +256,9 @@ struct Options { // `--all-programs` (mcpp#911 B3): every program of the package, the // chosen one and the others beside it. bool allPrograms = false; + // The others' file names, which are also in `runtimeFiles`: a tree of + // images gives each of them a launcher, as it gives the chosen one. + std::vector otherPrograms; }; // The strip decision for this run: `--strip`/`--no-strip` > `[pack] strip` > @@ -566,6 +569,16 @@ std::expected, Error> single_file(const Plan& std::string unbuilt_row(std::string_view system, std::string_view arch, const mcpp::platform::Resolved* P = nullptr); +// WHY NO SINGLE FILE OF AN IMAGE IS MADE FOR `system`-`arch`, as +// `: `: the row is one this mcpp builds no program for, or the +// platform's carrier, which names no hosts, is not a program where the +// platform's runner exists (`runner.hosts`). Empty when the file is made. The +// one reading of a named system before the build and of `--systems all` (a +// planned row the platform serves is built since N23, so the second condition +// is the one that answers for it). +std::string single_file_unmade(std::string_view system, std::string_view arch, + const mcpp::platform::Resolved& P); + // The systems `--systems` names for an architecture (design 2026-10-29 // §4.1): an ``, an `-` of that architecture, or every system // for `all`; this machine's system when it names none. @@ -2265,15 +2278,10 @@ run_wasm(const Plan& plan) "copy launcher failed: {}", ec.message())}); const auto builtDir = plan.builtBinary.parent_path(); + // The module beside the launcher is the row's companion (N33): absent when + // the link embedded it (`-sSINGLE_FILE=1`), and removed before every link, + // so a module present here is this launcher's. auto family = mcpp::pack::emscripten_stem_family(builtDir, plan.binaryName); - if (!family.hasWasm) { - const auto stem = plan.builtBinary.stem().string(); - return std::unexpected(Error{std::format( - "'{}.wasm' is missing beside the built launcher '{}' -- the " - "wasm32-emscripten link declares it as an implicit output, so its " - "absence means the build directory does not match the graph", - stem, plan.builtBinary.string())}); - } for (auto const& name : family.siblings) { std::error_code fec; std::filesystem::copy_file(builtDir / name, @@ -2564,14 +2572,25 @@ std::string_view launcher_platform() { // Whether this mcpp builds programs for `row`: a target its table does not // state (`aarch64-windows-gnu`), or states as planned (`x86_64-macos`, its // toolchain not published), is not built. -bool row_is_built(const ProgramRow& row) { +bool row_is_built(const ProgramRow& row, const mcpp::platform::Resolved* P = nullptr) { if (!row.served) return false; if (row.triple.empty()) return true; // this machine's own row auto t = mcpp::toolchain::triple::parse(row.triple); if (!t) return false; if (t->defined()) return !row.platform.empty(); // a platform's row, over that platform auto* k = mcpp::toolchain::triple::find_known_target(*t); - return k && k->tier != "planned"; + if (!k) return false; + // A PLANNED ROW THE PROGRAM'S PLATFORM SERVES IS BUILT (design 2026-11-15 + // N23), as a build for that row over that platform is (2026.11.14.1): the + // carrier of an i686 image is a program of i686-linux-musl, which the + // built-in table keeps planned and openkal-platform 0.28.0 lists with its + // toolchain. The carrier was refused as a row this mcpp builds no program + // for, while the project's own build of the row succeeded. + if (k->tier == "planned") + return P && P->active && !row.platform.empty() && row.platform == platform_ref_of(*P) + ? mcpp::manifest::platform::serves_with_toolchain(P->fields, P->package, t->str()) + : false; + return true; } } // namespace @@ -2596,7 +2615,7 @@ std::optional unbuilt_carrier(const mcpp::platform::Resolved& platf std::string_view system, std::string_view arch, bool single) { const auto row = program_row(system, arch, single, &platform); - if (!row_is_built(row)) + if (!row_is_built(row, &platform)) return std::format("{}'s carrier is built as a program of {}-{} ({}), and this mcpp builds " "no program for that row", platform.display(), system, arch, row.triple); // A CARRIER THAT NAMES NO HOSTS EXISTS WHERE THE PLATFORM'S RUNNER DOES: it is @@ -2821,54 +2840,67 @@ run_loaded(const Plan& plan) "cannot place the runner for {}-{}: {}", system, leg.arch, ec.message())}); } } - const auto posix = plan.stagingRoot / "bin" / launch.name; - { - std::ofstream out(posix, std::ios::binary | std::ios::trunc); - out << mcpp::pack::distribution::posix_launcher(launch); + // EVERY PROGRAM HAS A LAUNCHER (N11). Under `--all-programs` the others + // are images beside the chosen one in each `lib//`, and each is + // started as the chosen one is: by its name in `bin/`. + std::vector launches{ launch }; + for (auto const& other : plan.opts.otherPrograms) { + auto l = launch; + l.name = fs::path(other).stem().string(); + for (auto& [arch, file] : l.entry) file = other; + launches.push_back(std::move(l)); } - fs::permissions(posix, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec - | fs::perms::others_read | fs::perms::others_exec, - fs::perm_options::replace, ec); - // The Windows launcher, a program (design 2026-10-29 §4.3): built for - // x86_64 Windows, which an arm64 Windows machine also runs. A host - // that builds no program for that row leaves it out, says so, and - // the package's `runs_on` names no Windows pair. - // Over a platform's hosted arrangement where the build's platform - // serves the row, as the runners are: the same program whichever - // machine packs it. The launcher is mcpp's program and not the - // project's: its `mcpplibs.os` pins one openkal release, which only the - // platform mcpp's own openkal rows name agrees with, so it is built - // over that platform whichever release the project selected (a newer - // version group named another openkal, and the launcher's graph held - // two; openkal-platform 0.19.0's CI, 2026-10-30). - auto row = program_row("windows", "x86_64", true, &plan.platform); - if (!row.platform.empty()) row.platform = std::string(launcher_platform()); - if (!row_is_built(row)) { - mcpp::ui::warning(std::format( - "no Windows launcher in the package: this mcpp builds no program for {}; the package " - "does not run on Windows", row.triple)); - } else { - const auto dir = plan.projectRoot / "target" / "launcher-pack" / launch.name; - fs::create_directories(dir / "src", ec); - fs::remove(dir / "mcpp.lock", ec); - auto write = [&](const fs::path& p, std::string_view text) -> bool { - std::ofstream out(p, std::ios::binary | std::ios::trunc); - out.write(text.data(), static_cast(text.size())); - return bool(out); - }; - if (!write(dir / "mcpp.toml", mcpp::pack::distribution::exe_launcher_manifest( - launch.name, plan.packageVersion) - + static_row_pin(row, plan.platform.fields.toolchain.set - ? plan.platform.fields.toolchain.value : std::string())) - || !write(dir / "src" / "main.cpp", mcpp::pack::distribution::exe_launcher_main(launch))) - return std::unexpected(Error{std::format("cannot write the launcher's sources under {}", - dir.string())}); - auto packed = pack_generated_program(dir, row); - if (!packed) return std::unexpected(packed.error()); - fs::copy_file(*packed / (launch.name + ".exe"), plan.stagingRoot / "bin" / (launch.name + ".exe"), - fs::copy_options::overwrite_existing, ec); - if (ec) return std::unexpected(Error{std::format("cannot place the Windows launcher: {}", - ec.message())}); + for (auto const& launch : launches) { + const auto posix = plan.stagingRoot / "bin" / launch.name; + { + std::ofstream out(posix, std::ios::binary | std::ios::trunc); + out << mcpp::pack::distribution::posix_launcher(launch); + } + fs::permissions(posix, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec + | fs::perms::others_read | fs::perms::others_exec, + fs::perm_options::replace, ec); + // The Windows launcher, a program (design 2026-10-29 §4.3): built for + // x86_64 Windows, which an arm64 Windows machine also runs. A host + // that builds no program for that row leaves it out, says so, and + // the package's `runs_on` names no Windows pair. + // Over a platform's hosted arrangement where the build's platform + // serves the row, as the runners are: the same program whichever + // machine packs it. The launcher is mcpp's program and not the + // project's: its `mcpplibs.os` pins one openkal release, which only the + // platform mcpp's own openkal rows name agrees with, so it is built + // over that platform whichever release the project selected (a newer + // version group named another openkal, and the launcher's graph held + // two; openkal-platform 0.19.0's CI, 2026-10-30). + auto row = program_row("windows", "x86_64", true, &plan.platform); + if (!row.platform.empty()) row.platform = std::string(launcher_platform()); + if (!row_is_built(row)) { + if (&launch == &launches.front()) + mcpp::ui::warning(std::format( + "no Windows launcher in the package: this mcpp builds no program for {}; the package " + "does not run on Windows", row.triple)); + } else { + const auto dir = plan.projectRoot / "target" / "launcher-pack" / launch.name; + fs::create_directories(dir / "src", ec); + fs::remove(dir / "mcpp.lock", ec); + auto write = [&](const fs::path& p, std::string_view text) -> bool { + std::ofstream out(p, std::ios::binary | std::ios::trunc); + out.write(text.data(), static_cast(text.size())); + return bool(out); + }; + if (!write(dir / "mcpp.toml", mcpp::pack::distribution::exe_launcher_manifest( + launch.name, plan.packageVersion) + + static_row_pin(row, plan.platform.fields.toolchain.set + ? plan.platform.fields.toolchain.value : std::string())) + || !write(dir / "src" / "main.cpp", mcpp::pack::distribution::exe_launcher_main(launch))) + return std::unexpected(Error{std::format("cannot write the launcher's sources under {}", + dir.string())}); + auto packed = pack_generated_program(dir, row); + if (!packed) return std::unexpected(packed.error()); + fs::copy_file(*packed / (launch.name + ".exe"), plan.stagingRoot / "bin" / (launch.name + ".exe"), + fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("cannot place the Windows launcher: {}", + ec.message())}); + } } } @@ -3586,12 +3618,7 @@ std::vector single_file_rows(const Plan& plan) { for (auto const& arch : archs) for (auto const& system : systems_for(plan.opts.systems, arch)) { SingleFileRow r{ system, arch, {} }; - if (all) - if (auto row = unbuilt_row(system, arch, &plan.platform); !row.empty()) - r.skipped = std::format("no carrier row: this mcpp builds no program for {}", row); - if (all && r.skipped.empty() && plan.platform.active) - if (auto why = detail::unbuilt_carrier(plan.platform, system, arch, true)) - r.skipped = *why; + if (all) r.skipped = single_file_unmade(system, arch, plan.platform); // `all` in one polyglot file: the systems the file is started on. if (all && r.skipped.empty() && plan.opts.polyglot && !polyglot::holds(system)) r.skipped = std::format("a polyglot file is started by a shell or by the Windows loader, " @@ -3612,7 +3639,18 @@ bool output_names_directory(const std::filesystem::path& output) { std::string unbuilt_row(std::string_view system, std::string_view arch, const mcpp::platform::Resolved* P) { const auto row = detail::program_row(system, arch, true, P); - return detail::row_is_built(row) ? std::string() : row.triple; + return detail::row_is_built(row, P) ? std::string() : row.triple; +} + +std::string single_file_unmade(std::string_view system, std::string_view arch, + const mcpp::platform::Resolved& P) { + const auto row = detail::program_row(system, arch, true, &P); + if (!detail::row_is_built(row, &P)) + return std::format("{}: this mcpp builds no program for the row", row.triple); + if (P.active) + if (auto why = detail::unbuilt_carrier(P, system, arch, true)) + return std::format("{}: {}", row.triple, *why); + return {}; } std::vector carried_systems(const Plan& plan) { diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 48f27a308..854be48b0 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -515,19 +515,58 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { return t.kind == mcpp::manifest::Target::Application; return false; }; + // THE PROGRAM A FORMAT'S PROVIDER ASSEMBLES FROM (design 2026-11-15 N30). + // A provider that states `mcpp::pack_program` is followed when no program + // is named: the convention below knows nothing of the provider's own + // choice (a web directory configured for `app`), and packing another + // program left the provider nothing to wrap. A named program outranks it. + std::string want = run.wantTarget; + std::string wantFrom; + if (want.empty()) { + std::vector> stated; // provider, program + const auto& asked = run.opts.formats.empty() ? std::vector{ run.opts.formatName } + : run.opts.formats; + for (auto const& format : asked) { + auto it = ctx.packFormatProviders.find(format); + if (format.empty() || mcpp::pack::is_builtin_pack_format(format) + || it == ctx.packFormatProviders.end()) continue; + for (auto const& p : it->second) { + if (sharedPlan) { + auto reach = ctx.packReach.find(p); + if (reach == ctx.packReach.end() || mcpp::build::pack_owner(p, reach->second) != m.name) + continue; + } + auto pp = ctx.packPrograms.find(p); + if (pp == ctx.packPrograms.end()) continue; + for (auto const& name : pp->second) + if (std::ranges::none_of(stated, [&](auto const& e) { return e.second == name; })) + stated.emplace_back(p, name); + } + } + if (stated.size() > 1) { + std::string list; + for (auto const& [p, name] : stated) list += std::format("\n {} (stated by {})", name, p); + return Refusal{2, about(run, m, std::format( + "the providers of the requested formats state different programs:{}\n" + " name one: mcpp pack --pkg-format ", list))}; + } + if (!stated.empty()) { want = stated.front().second; wantFrom = stated.front().first; } + } std::filesystem::path mainBinary; const mcpp::build::LinkUnit* chosenLu = nullptr; - if (!run.wantTarget.empty()) { + if (!want.empty()) { for (auto& lu : ctx.plan.linkUnits) { - if (is_program_link_unit(lu) && lu.targetName == run.wantTarget) { + if (is_program_link_unit(lu) && lu.targetName == want) { mainBinary = ctx.outputDir / lu.output; chosenLu = &lu; break; } } if (mainBinary.empty()) - return Refusal{2, about(run, m, std::format( - "target '{}' is not a program in this build", run.wantTarget))}; + return Refusal{2, about(run, m, wantFrom.empty() + ? std::format("target '{}' is not a program in this build", want) + : std::format("{} states the program '{}' (mcpp::pack_program), which is not a program " + "in this build", wantFrom, want))}; } for (auto& lu : ctx.plan.linkUnits) { if (!mainBinary.empty()) break; @@ -551,7 +590,9 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { } if (mainBinary.empty()) return Refusal{1, about(run, m, "no binary target to pack")}; - if (programs.size() > 1 && !opts.allPrograms && run.wantTarget.empty()) { + if (programs.size() > 1 && !opts.allPrograms && !wantFrom.empty()) + mcpp::ui::note(std::format("packing {}, the program {} states for its format", want, wantFrom)); + if (programs.size() > 1 && !opts.allPrograms && want.empty()) { std::string others; for (auto const* lu : programs) if (lu != chosenLu) others += (others.empty() ? "" : ", ") + lu->targetName; @@ -648,10 +689,12 @@ std::optional member_plan(PackRun& run, GroupJob& g, MemberJob& m) { // `--all-programs`: the package's other programs, beside the chosen one. if (opts.allPrograms) for (auto const* u : programs) - if (u != chosenLu && u->kind == mcpp::build::LinkUnit::Binary) + if (u != chosenLu && u->kind == mcpp::build::LinkUnit::Binary) { opts.runtimeFiles.push_back(productDir == "bin" ? u->output.lexically_relative("bin") : std::filesystem::path(u->output.filename())); + opts.otherPrograms.push_back(u->output.filename().string()); + } // #634 A3: the Android row reads its closure against the directories // its link declared -- a prebuilt library named through `[runtime] // link_library_dirs` is a file the link used and the device does not @@ -957,18 +1000,21 @@ std::optional check_format(PackRun& run, GroupJob& g) { "--systems names {}, and no leg of this pack is of {}", e, e.substr(dash + 1))}; } // A single file of an image is made for each leg and system on this - // mcpp's row of it; a row it builds no program for, named, is refused - // before the build (`all` names the rows there are, and says the rest). + // mcpp's row of it; a system where none is made, named, is refused + // before the build, by the reading `all` applies to say the rest + // (`single_file_unmade`: a row this mcpp builds no program for, or a + // carrier where the platform's runner does not exist). The second was + // read only by `all`, so a named one was refused after the build. if (loaded && run.opts.format == mcpp::pack::Format::SingleFile) { std::size_t files = 0; std::vector unbuilt, built; const bool all = std::ranges::find(run.opts.systems, "all") != run.opts.systems.end(); for (auto const& arch : archs) for (auto const& system : mcpp::pack::systems_for(run.opts.systems, arch)) { - auto row = mcpp::pack::unbuilt_row(system, arch, &ctx.plan.platform); - if (row.empty()) built.push_back(std::format("{}-{}", system, arch)); - else if (!all) unbuilt.push_back(std::format("{}-{} ({})", system, arch, row)); - if (row.empty() || !all) ++files; + auto why = mcpp::pack::single_file_unmade(system, arch, ctx.plan.platform); + if (why.empty()) built.push_back(std::format("{}-{}", system, arch)); + else if (!all) unbuilt.push_back(std::format("{}-{} ({})", system, arch, why)); + if (why.empty() || !all) ++files; } if (files > 1 && !run.opts.output.empty() && !run.opts.polyglot) return Refusal{2, std::format( @@ -980,7 +1026,7 @@ std::optional check_format(PackRun& run, GroupJob& g) { for (auto const& u : unbuilt) say += (say.empty() ? "" : ", ") + u; for (auto const& b : built) use += (use.empty() ? "" : ",") + b; return Refusal{2, std::format( - "--systems: this mcpp builds no program for {}, so no single file is made there.\n" + "--systems: no single file is made for {}.\n" " use: {}", say, use.empty() ? std::string("--systems naming a system this mcpp builds for") : "--systems " + use)}; @@ -1487,7 +1533,22 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba if (m.result.rc != 0) continue; auto it = submitted.find(m.name); if (it != submitted.end() && !it->second.outputs.empty()) { any = true; continue; } - mcpp::ui::error(about(run, m, std::format( + // THE PROVIDER'S OWN REASON IS THE ANSWER (N30): what a provider + // of this format acting for the member stated in this pass, as a + // warning or a degradation, explains why it submitted nothing. + std::string said; + if (auto it = distCtx->packFormatProviders.find(run.opts.formatName); + it != distCtx->packFormatProviders.end()) + for (auto const& p : it->second) { + if (sharedPlan) { + auto reach = distCtx->packReach.find(p); + if (reach == distCtx->packReach.end() + || mcpp::build::pack_owner(p, reach->second) != m.name) continue; + } + if (auto r = distCtx->packReasons.find(p); r != distCtx->packReasons.end()) + for (auto const& reason : r->second) said += std::format("\n {} said: {}", p, reason); + } + mcpp::ui::error(about(run, m, said.empty() ? std::format( "no action claimed --pkg-format '{}'.\n" " A package declared it provides this format, and no build " "program submitted a new\n" @@ -1498,7 +1559,8 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba "// always\n" " if (std::string_view(mcpp::pack_format()) == \"{}\") ..." " // then submit", - run.opts.formatName, run.opts.formatName, run.opts.formatName))); + run.opts.formatName, run.opts.formatName, run.opts.formatName) + : std::format("no action claimed --pkg-format '{}'.{}", run.opts.formatName, said))); m.result.rc = 1; } if (!any) return std::nullopt; diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index 009abc113..183018e1c 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -23,6 +23,9 @@ import std; import mcpp.manifest; import mcpp.toolchain.triple; import mcpp.toolchain.target_schema; +import mcpp.toolchain.target_rows; // the execution relations (N23) +import mcpp.toolchain.target_select; // host_pair +import mcpp.toolchain.platform; import mcpp.invocation; export namespace mcpp::platform { @@ -143,7 +146,10 @@ std::vector expand_step(const std::vector& step, // runs the artifact on a machine of its architecture: here when the two // agree; on the execution host (`run_under`) otherwise, whose architecture is // the artifact's, so the program is that architecture's; refused when no -// execution host is named. `--no-runner` states the triple is native here. +// execution host is named. A machine the execution table states executes the +// artifact's architecture (`[[platform.executes]]`) is that execution host +// itself: the program is that architecture's, started here (design +// 2026-11-15 N23). `--no-runner` states the triple is native here. // Every other runner reaches the artifact from this machine. enum class RunnerReach { Here, TargetArchitecture, Refused }; RunnerReach runner_reach(const mcpp::manifest::platform::Runner& runner, std::string_view targetArch, @@ -349,6 +355,38 @@ std::expected select_host(std::string_view flagOrEnv, bo return none; } +// ONE RUNNER OVER ANOTHER (SPEC-013 §10; R1, R2): `cmd`, `requires` and +// `enabled` are one statement, replaced together; `host`, `hosts`, +// `exclusive`, `longlived` and `built-as` each stand until a nearer statement +// writes them. A table without `cmd` adjusts the runner below it. +void merge_runner(mp::Runner& b, const mp::Runner& o) { + if (o.set) { + b.cmd = o.cmd; + b.requires_ = o.requires_; + b.origin = o.origin; + b.set = true; + b.enabled = o.enabledSet ? o.enabled : true; + b.enabledSet = o.enabledSet; + } else if (o.enabledSet) { + b.enabled = o.enabled; + b.enabledSet = true; + } + if (!o.builtAs.empty()) b.builtAs = o.builtAs; + if (!o.host.empty()) b.host = o.host; + if (o.exclusiveSet) { + b.exclusive = o.exclusive; + b.exclusiveSet = true; + } + if (o.longlivedSet) { + b.longlived = o.longlived; + b.longlivedSet = true; + } + if (o.hostsSet) { + b.hosts = o.hosts; + b.hostsSet = true; + } +} + void merge(mp::Statement& b, const mp::Statement& o) { auto text = [](mp::Text& to, const mp::Text& from) { if (from.set) to = from; }; auto words = [](mp::Words& to, const mp::Words& from) { if (from.set) to = from; }; @@ -359,23 +397,8 @@ void merge(mp::Statement& b, const mp::Statement& o) { // `exclusive`; a table, or a program's `run-host`, states what it states. // `requires` are the packages of THAT command, so a statement that // replaces the command replaces them. - if (o.runner.set) { - b.runner.cmd = o.runner.cmd; - b.runner.requires_ = o.runner.requires_; - b.runner.origin = o.runner.origin; - b.runner.set = true; - } - if (!o.runner.builtAs.empty()) b.runner.builtAs = o.runner.builtAs; - if (!o.runner.host.empty()) b.runner.host = o.runner.host; - if (o.runner.exclusiveSet) { - b.runner.exclusive = o.runner.exclusive; - b.runner.exclusiveSet = true; - } - if (o.runner.hostsSet) { - b.runner.hosts = o.runner.hosts; - b.runner.hostsSet = true; - } - for (auto const& [k, v] : o.runners) b.runners[k] = v; + merge_runner(b.runner, o.runner); + for (auto const& [k, v] : o.runners) merge_runner(b.runners[k], v); if (!o.carriers.empty()) b.carriers = o.carriers; words(b.descriptorKeys, o.descriptorKeys); text(b.plugin, o.plugin); @@ -459,12 +482,12 @@ void take_provider_fields(mp::Statement& s, const mp::ProviderSpec& pr) { // THE PROGRAMS A STATEMENT'S CONVENTIONS RUN: its link's inputs, its seal's steps, // its runner and its named runners, each by its first word. std::set named_programs(const mp::Link& l, const mp::Runner& runner, - const std::map& runners) { + const std::map& runners) { std::set out; for (auto const& in : l.inputs) if (!in.from.empty()) out.insert(in.from.front()); for (auto const& step : l.seal) if (!step.empty()) out.insert(step.front()); if (!runner.cmd.empty()) out.insert(runner.cmd.front()); - for (auto const& [_, w] : runners) if (!w.value.empty()) out.insert(w.value.front()); + for (auto const& [_, w] : runners) if (!w.cmd.empty()) out.insert(w.cmd.front()); return out; } @@ -638,6 +661,14 @@ RunnerReach runner_reach(const mp::Runner& runner, std::string_view targetArch, std::string_view hostArch, bool executionHostNamed, bool noRunner) { if (noRunner || !runner.set || runner.host != "same-arch" || targetArch == hostArch) return RunnerReach::Here; + // A MACHINE THAT EXECUTES THE ARTIFACT'S ARCHITECTURE besides its own is + // its own execution host of that architecture (N23): the runner is the + // platform's runner built for that architecture, started here, as it is + // handed to a named execution host. An x86_64 Linux machine runs an i386 + // image through the i686 runner; its own runner refuses the image. + if (mcpp::toolchain::target::executes( + mcpp::toolchain::target::host_pair(mcpp::toolchain::platform::name, hostArch), targetArch)) + return RunnerReach::TargetArchitecture; return executionHostNamed ? RunnerReach::TargetArchitecture : RunnerReach::Refused; } @@ -705,7 +736,8 @@ std::vector> report_rows(const Resolved& r) rows.emplace_back("runner", std::format("{} ({})", f.runner.cmd.empty() ? std::string() : f.runner.cmd.front(), origin_text(f.runner.origin))); for (auto const& [name, w] : f.runners) - rows.emplace_back("runner " + name, std::format("{} ({})", argv_text(w.value), origin_text(w.origin))); + if (w.set) + rows.emplace_back("runner " + name, std::format("{} ({})", argv_text(w.cmd), origin_text(w.origin))); if (f.link.driver.set && f.link.driver.value != "compiler") rows.emplace_back("link", std::format("{}{} ({})", f.link.driver.value, f.link.start.set ? ", start " + f.link.start.value : "", @@ -735,7 +767,7 @@ std::string to_json(const Resolved& r) { }; if (f.toolchain.set) field("toolchain", q(f.toolchain.value), f.toolchain.origin); if (f.runner.set) field("runner", arr(f.runner.cmd), f.runner.origin); - for (auto const& [k, w] : f.runners) field("runners." + k, arr(w.value), w.origin); + for (auto const& [k, w] : f.runners) if (w.set) field("runners." + k, arr(w.cmd), w.origin); for (auto const& [layer, l] : r.layers) { mp::Origin origin{l.tier, {}, 0, {}}; if (auto it = f.layers.find(layer); it != f.layers.end() && l.tier != mp::Tier::Graph) diff --git a/src/run/run.cpp b/src/run/run.cpp index d67115b34..22e944ebb 100644 --- a/src/run/run.cpp +++ b/src/run/run.cpp @@ -13,6 +13,8 @@ import mcpplibs.os.interrupt; import mcpp.invocation; import mcpp.build.command_line; import mcpp.runtime.launch_env; +import mcpp.toolchain.platform; +import mcpp.toolchain.target_rows; namespace libc = mcpplibs::mwrap::libc::macros; @@ -207,6 +209,24 @@ std::optional program_of(const std::string& name) { return mcpp::invocation::which(name); } +// A SCRIPT THIS MACHINE STARTS THROUGH ITS COMMAND INTERPRETER (the system +// table's `command-scripts`), given an argument the interpreter would read as +// syntax: the refusal, or empty. Decided before the start, so the argument is +// neither rewritten nor run as a command (design 2026-11-15 R2, A3). +std::string command_script_refusal(const std::filesystem::path& program, std::span args) { + auto const* sys = mcpp::toolchain::target::find_system(mcpp::toolchain::platform::name); + if (!sys || sys->commandScripts.empty()) return {}; + auto ext = program.extension().string(); + for (auto& c : ext) c = static_cast(std::tolower(static_cast(c))); + if (std::ranges::find(sys->commandScripts, std::string_view(ext)) == sys->commandScripts.end()) return {}; + auto problem = mcpp::build::command_line::command_script_problem(args); + if (problem.empty()) return {}; + return std::format("cannot start '{}': {}; a {} file is started through the command interpreter, and the " + "argument is refused rather than rewritten. Start a program (.exe) instead, or pass the " + "value another way (a file, an environment variable).\n", + program.string(), problem, ext); +} + // Lines of `acc` up to its last line end, to `on_line`, without the ends. void deliver_lines(std::string& acc, const std::function& on_line, bool splitOnCr, bool all) { @@ -242,6 +262,11 @@ Outcome launch(const Spec& s) { o.program = *program; o.argv0 = s.argv.front(); o.args.assign(s.argv.begin() + 1, s.argv.end()); + if (auto why = command_script_refusal(*program, o.args); !why.empty()) { + out.spawn_error = static_cast(std::errc::invalid_argument); + out.spawn_message = std::move(why); + return out; + } o.cwd = s.cwd; o.env = child_env(s.env, s.hostTool, s.unset); o.in = s.nullInput ? os::process::stdio::null : os::process::stdio::inherit; @@ -489,6 +514,11 @@ int run_foreground(const std::vector& argv, const Env& extraEnv, in o.program = *program; o.argv0 = argv.front(); o.args.assign(argv.begin() + 1, argv.end()); + if (auto why = command_script_refusal(*program, o.args); !why.empty()) { + if (spawn_error) *spawn_error = static_cast(std::errc::invalid_argument); + else std::fputs(why.c_str(), libc::stderr()); + return 125; + } o.env = child_env(extraEnv, false); o.in = o.out = o.err = os::process::stdio::inherit; std::fflush(libc::stdout()); diff --git a/tests/e2e/653_exceptions_is_the_second_abi_member.sh b/tests/e2e/653_exceptions_is_the_second_abi_member.sh index 523ef25ab..9a9e22cf7 100755 --- a/tests/e2e/653_exceptions_is_the_second_abi_member.sh +++ b/tests/e2e/653_exceptions_is_the_second_abi_member.sh @@ -71,7 +71,7 @@ diff -u "$TMP/plain.ninja" "$with_ninja" > "$TMP/ninja.diff" \ || fail "a Linux build.ninja differs with the emscripten-only abi table present" "$TMP/ninja.diff" echo " ok: build.ninja is byte-identical" -echo "== 653/2: without exceptions, the build and link succeed but the run fails at the throw ==" +echo "== 653/2: the row's model catches; with exceptions off, the build and link succeed but the run fails at the throw ==" mkdir -p "$TMP/excdep/src" "$TMP/app/src" cat > "$TMP/excdep/mcpp.toml" <<'TOML' @@ -119,12 +119,18 @@ int main() { CPP cd "$TMP/app" +# No table: the row's exception model (`exceptions = "wasm"`, mcpp 2026.11.16.1, N26) catches. write_app '' +"$MCPP" run --target wasm32-emscripten > run-row.log 2>&1 \ + || fail "the row's exception model did not catch the throw" run-row.log +grep -q '^caught: boom$' run-row.log || fail "the row's model did not print the marker" run-row.log +echo " ok: with no table the row's model catches the throw" +write_app $'[target.\'cfg(os = "emscripten")\'.abi]\nexceptions = false\n' "$MCPP" build --target wasm32-emscripten > build-noexc.log 2>&1 \ - || fail "the build without the abi table failed (expected: it succeeds; only the run fails)" build-noexc.log + || fail "the build with exceptions off failed (expected: it succeeds; only the run fails)" build-noexc.log plain_dir=$(dirname "$(ls -t $(find target/wasm32-emscripten -name build.ninja -not -path '*/.build-mcpp/*') | head -1)") if "$MCPP" run --target wasm32-emscripten > run-noexc.log 2>&1; then - fail "mcpp run succeeded without -fexceptions -- the throw should have aborted" run-noexc.log + fail "mcpp run succeeded with exceptions off -- the throw should have aborted" run-noexc.log fi grep -q '^caught: boom$' run-noexc.log \ && fail "the marker printed without -fexceptions" run-noexc.log @@ -167,7 +173,7 @@ grep -qF -- '-fexceptions' "$stdmodule_json" \ || fail "-fexceptions is not recorded on the std module's cached prebuild command" "$stdmodule_json" echo " ok: -fexceptions reached the std module's prebuild command (recorded in the build cache build.ninja stages from)" -echo "== 653/4: a dependency requiring exceptions is refused on the Web build without the table ==" +echo "== 653/4: a dependency requiring exceptions is served by the row's model, and refused with exceptions off ==" mkdir -p "$TMP/excreq/src" "$TMP/app3/src" cat > "$TMP/excreq/mcpp.toml" <<'TOML' @@ -197,8 +203,11 @@ runner = ["node"] TOML printf 'import excreq;\nint main() { return v() == 1 ? 0 : 1; }\n' > "$TMP/app3/src/main.cpp" cd "$TMP/app3" +"$MCPP" build --target wasm32-emscripten > serve-exc.log 2>&1 \ + || fail "the row's exception model did not serve a dependency requiring exceptions" serve-exc.log +printf '%s' $'\n[target.\'cfg(os = "emscripten")\'.abi]\nexceptions = false\n' >> "$TMP/app3/mcpp.toml" if "$MCPP" build --target wasm32-emscripten > refuse-exc.log 2>&1; then - fail "a dependency requiring exceptions was built for the Web without the root's table" refuse-exc.log + fail "a dependency requiring exceptions was built for the Web with exceptions off" refuse-exc.log fi grep -qF "\`excreq\` requires the artefact's ABI to have exceptions (the package)" refuse-exc.log \ || fail "the refusal does not name 'exceptions'" refuse-exc.log diff --git a/tests/e2e/build/link.cpp b/tests/e2e/build/link.cpp index 6689defb8..df80b1645 100644 --- a/tests/e2e/build/link.cpp +++ b/tests/e2e/build/link.cpp @@ -279,7 +279,7 @@ ldflags = ["-Wl,-rpath='$$ORIGIN/../x'"] bool published = false; for (std::size_t i = 0; i + 1 < lines.size(); ++i) if (lines[i].starts_with("rule cxx_link")) - published = published || std::regex_search(lines[i + 1], std::regex{R"(\$out\.tmp.*&& mv -f \$out\.tmp \$out)"}); + published = published || lines[i + 1].contains("link-output --output $out --temporary -- "); expect(fatal(published)) << "A: the link rule does not publish through a temporary name"; const auto exe = path_ending(p.at("target"), "/bin/app"); expect(fatal(exe.has_value())) << "no bin/app"; diff --git a/tests/e2e/deps/graph.cpp b/tests/e2e/deps/graph.cpp index 3a43dfeac..f32d9e989 100644 --- a/tests/e2e/deps/graph.cpp +++ b/tests/e2e/deps/graph.cpp @@ -252,6 +252,26 @@ int main() { expect(fatal(e2e::succeeded(d))) << "D: the build failed"; expect(e2e::contains(s::text(d), "`override` on 'mcpplibs.os' is ignored")) << "D: a library's override is not reported"; }; + + "1088 two build dependencies that disagree are refused by their requesters"_test = [] { + // Two [build-dependencies] requests of one package that no merge reconciles are refused + // as such, naming both requesters and the override line: a build dependency is a module + // of a build program, which no rename separates (design 2026-11-15 N29). The rename was + // tried and failed on the requester's sources. + const auto tmp = s::case_dir(); + s::project tooling{tmp / "tooling", {{"mcpp.toml", "[package]\nname = \"tooling\"\nversion = \"0.1.0\"\n[targets.tooling]\nkind = \"lib\"\n[build-dependencies]\nmcpplibs.os = \"0.4.1\"\n"}, + {"build.mcpp", "import mcpp;\nint main() { return 0; }\n"}, + {"src/tooling.cppm", "export module tooling;\nexport int tooling_answer() { return 1; }\n"}}}; + s::project app{tmp / "app", {{"mcpp.toml", std::format("[package]\nname = \"app\"\nversion = \"0.1.0\"\n[build-dependencies]\nmcpplibs.os = \"^0.4.4\"\n[dependencies]\ntooling = {{ path = \"{}/tooling\" }}\n", tmp.generic_string())}, + {"build.mcpp", "import mcpp;\nint main() { return 0; }\n"}, + {"src/main.cpp", "import tooling;\nint main() { return tooling_answer() == 1 ? 0 : 1; }\n"}}}; + auto r = app.mcpp({"build", "--configure-only"}); + expect(fatal(failed(r))) << "two build dependencies no merge reconciles were accepted\n" << s::text(r); + expect(fatal(e2e::contains(s::text(r), "build dependency 'mcpplibs.os' has irreconcilable versions"))) << "the refusal does not name the conflict\n" << s::text(r); + expect(fatal(e2e::contains(s::text(r), "tooling") && e2e::contains(s::text(r), "app"))) << "the refusal does not name both requesters\n" << s::text(r); + expect(fatal(s::lacks(s::text(r), "mangl"))) << "the rename was tried\n" << s::text(r); + expect(s::has_line(s::text(r), R"(mcpplibs\.os = \{ version = "[0-9.]*", override = true \})")) << "the refusal does not name the override\n" << s::text(r); + }; }}; } // namespace diff --git a/tests/e2e/modules/scanner.cpp b/tests/e2e/modules/scanner.cpp index 86dd0671b..8a3a775fc 100644 --- a/tests/e2e/modules/scanner.cpp +++ b/tests/e2e/modules/scanner.cpp @@ -1,7 +1,7 @@ // What the module scanner reads: the walk pruned at `.mcpp`, defines that reach the scan, a // private module fragment, a byte-order mark, an implementation unit in a module file, comments -// and raw strings, scan overrides of files a build excludes (formerly tests/e2e/113, 167, 634, -// 635, 636, 639, 1064). +// and raw strings, scan overrides of files a build excludes, a source named by a path that is +// not there (formerly tests/e2e/113, 167, 634, 635, 636, 639, 1064; 1096, 1097). import std; import mcpplibs.mtest; import mcpp.test.support; @@ -255,6 +255,55 @@ path = "src/core.cppm" expect(fatal(failed(r))) << "a glob naming no file was accepted\n" << s::text(r); expect(fatal(e2e::contains(s::text(r), "scan_overrides glob 'src/nowhere.cppm' matches no file of the package"))) << "the refusal does not name the glob\n" << s::text(r); }; + + "1096 a source named by its path that is not there is refused"_test[s::gcc] = [] { + // #938: an entry without a pattern names one file; a missing one is refused with the table + // that states it, before an importer fails. A pattern that matches nothing is not. + s::project p({{"mcpp.toml", R"([package] +name = "s1096" +version = "0.1.0" +[targets.s1096] +kind = "lib" +[build] +sources = ["src/*/*.cppm", "src/none/*.cppm"] +[target.'cfg(unix)'.build] +sources = ["src/target/unix/target.cppm"] +)"}, + {"src/core/s1096.cppm", "export module s1096;\nexport import s1096.target;\n"}}); + auto r = p.mcpp({"build"}); + expect(fatal(failed(r))) << "a missing file named by its path was accepted\n" << s::text(r); + expect(e2e::contains(s::text(r), R"([target.'cfg(unix)'.build] sources: "src/target/unix/target.cppm" names no file)")) + << "the refusal does not name the table and the path\n" << s::text(r); + expect(!e2e::contains(s::text(r), "src/none/")) << "a pattern that matches nothing was refused"; + expect(!e2e::contains(s::text(r), "failed to read compiled module")) << "the build reached an importer"; + p.write("src/target/unix/target.cppm", "export module s1096.target;\nexport int t() { return 1; }\n"); + r = p.mcpp({"build"}); + expect(e2e::succeeded(r)) << "the build with the file in place failed\n" << s::text(r); + }; + + "1097 a fetched package's source named by its path that is not there is a warning"_test[s::gcc][s::fresh_sandbox] = [] { + // #938, the other side: the entry is in a package fetched into a store, which its consumer + // cannot change. The build is not refused; a warning names the entry. + s::project p({{"local-index/pkgs/a/acme.gone.lua", R"(package = { + spec = "1", namespace = "acme", name = "acme.gone", + description = "fixture", licenses = {"MIT"}, type = "package", + xpm = { + linux = { ["1.0.0"] = { url = "u", sha256 = "0" } }, + macosx = { ["1.0.0"] = { url = "u", sha256 = "0" } }, + windows = { ["1.0.0"] = { url = "u", sha256 = "0" } }, + }, + mcpp = { language = "c++23", import_std = false, sources = { "src/gone.cppm", "src/absent.cpp" }, + targets = { ["gone"] = { kind = "lib" } }, deps = {} }, +} +)"}, + {".mcpp/.xlings/data/xpkgs/acme.gone/1.0.0/src/gone.cppm", "export module gone;\nexport int gone_value() { return 3; }\n"}, + {"src/main.cpp", "import gone;\nint main() { return gone_value() == 3 ? 0 : 1; }\n"}, + {"mcpp.toml", "[package]\nname = \"s1097\"\nversion = \"0.1.0\"\n\n[indices]\nacme = { path = \"local-index\" }\n\n[dependencies.acme]\ngone = \"1.0.0\"\n\n[targets.s1097]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"}}); + auto r = p.mcpp({"build"}, {.where = s::home::inherited}); + expect(fatal(e2e::succeeded(r))) << "a fetched package's missing file refused the build\n" << s::text(r); + expect(e2e::contains(s::text(r), R"("src/absent.cpp" names no file)")) + << "no warning names the entry\n" << s::text(r); + }; }}; } // namespace diff --git a/tests/e2e/openkal/sets.cpp b/tests/e2e/openkal/sets.cpp index b41c795bc..594272dc7 100644 --- a/tests/e2e/openkal/sets.cpp +++ b/tests/e2e/openkal/sets.cpp @@ -419,7 +419,7 @@ E2E sets{"sets", timeout{std::chrono::minutes{30}}, [] { expect(fatal(e2e::succeeded(r))) << "C: --systems all: " << s::text(r); for (std::string_view p : {"linux-x86_64", "linux-aarch64", "windows-x86_64", "macos-aarch64", "freebsd-x86_64", "freebsd-aarch64"}) expect(fatal(!s::glob(app.at("all"), std::format("sf-0.1.0-{}*", p)).empty())) << "C: no file for " << p << ": " << s::text(r); - expect(fatal(e2e::contains(s::text(r), "Not covered macos-x86_64 (no carrier row"))) << "C: the pair without a row is not said: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "Not covered macos-x86_64 (x86_64-macos: "))) << "C: the pair without a carrier is not said: " << s::text(r); } // D r = app.mcpp({"pack", "--target", "openkal", "--pkg-format", "single-file", "-o", "one-file"}); diff --git a/tests/e2e/openkal/trees.cpp b/tests/e2e/openkal/trees.cpp index 198631606..5d3ab5e72 100644 --- a/tests/e2e/openkal/trees.cpp +++ b/tests/e2e/openkal/trees.cpp @@ -155,6 +155,12 @@ E2E trees{"trees", timeout{std::chrono::minutes{40}}, [] { d = s::json_file(sc / "mcpp-package.json"); const auto carried = names_in(sc / "libexec/okrun"); expect(fatal(d["mode"] == "self-contained" && strings(d["systems"]) == carried && std::ranges::contains(carried, std::format("{}-{}", os, host_arch)))) << "C: systems: " << d.dump(); + // C': a package that carries its runner runs no other: with this machine's runner moved + // away and one on PATH, the launcher exits 126 naming what the package holds (N20). + fs::rename(sc / std::format("libexec/okrun/{}-{}", os, host_arch), sc / "libexec/okrun/elsewhere"); + ran = s::run(sc / "bin/uni", {}, {}, {{"PATH", okrun.parent_path().string() + ":" + s::env("PATH").value_or("")}}); + expect(fatal(ran.status == 126 && e2e::contains(s::text(ran), "the package holds okrun for: elsewhere"))) << "C': a runner on PATH was used for a package that carries its own: " << ran.status << " " << s::text(ran); + fs::rename(sc / "libexec/okrun/elsewhere", sc / std::format("libexec/okrun/{}-{}", os, host_arch)); // D r = app.mcpp({"pack", "--target", t, "--pkg-format", "dir", "-o", "one"}); expect(fatal(e2e::succeeded(r))) << "D: one target: " << s::text(r); @@ -174,6 +180,24 @@ E2E trees{"trees", timeout{std::chrono::minutes{40}}, [] { } }; + "1087 every program of an image package has its launchers"_test[s::llvm][s::unix_shell] = [] { + // `--all-programs` of an ok image: each program has bin/ and bin/.exe, and + // each launcher runs its own image (N11). + needs_image_row(); + if (posix_system().empty()) skip("a POSIX launcher"); + const std::string t = std::string(host_arch) + "-openkal"; + s::project app({{"mcpp.toml", "[package]\nname = \"multi\"\nversion = \"0.1.0\"\n\n[targets.alpha]\nkind = \"bin\"\nmain = \"src/alpha.cpp\"\n\n[targets.beta]\nkind = \"bin\"\nmain = \"src/beta.cpp\"\n"}, + {"src/alpha.cpp", "int main() { return 4; }\n"}, {"src/beta.cpp", "int main() { return 5; }\n"}}); + auto r = app.mcpp({"pack", "alpha", "--all-programs", "--target", t, "--pkg-format", "dir"}); + expect(fatal(e2e::succeeded(r))) << "pack --all-programs: " << s::text(r); + const auto p = app.at("target/dist/multi-0.1.0-" + t); + for (std::string_view f : {"bin/alpha", "bin/beta", "bin/alpha.exe", "bin/beta.exe"}) + expect(fatal(fs::is_regular_file(p / f) || (f.ends_with(".exe") && e2e::contains(s::text(r), "no Windows launcher")))) << "the package has no " << f << ": " << s::text(r); + const auto okrun = runner_of(app, t); + const std::string path = okrun.parent_path().string() + ":" + s::env("PATH").value_or(""); + expect(fatal(s::run(p / "bin/alpha", {}, {}, {{"PATH", path}}).status == 4)) << "bin/alpha did not run alpha"; + expect(fatal(s::run(p / "bin/beta", {}, {}, {{"PATH", path}}).status == 5)) << "bin/beta did not run beta"; + }; "956 a single file runs without unpacking"_test[s::unix_shell][s::python3] = [] { // A a native single file runs with its arguments and exit status, extracting its tree // into the cache once; B (Linux) `--linkage static` makes a file with no interpreter; diff --git a/tests/e2e/pack/formats.cpp b/tests/e2e/pack/formats.cpp index 6dc83d5c3..2af090510 100644 --- a/tests/e2e/pack/formats.cpp +++ b/tests/e2e/pack/formats.cpp @@ -219,6 +219,48 @@ int main() { mcpp::provides_pack_format("zap"); return 0; } r = silent.mcpp({"pack", "--pkg-format", "zap"}); expect(fatal(failed(r))) << "a format nothing claimed reported success\n" << s::text(r); expect(fatal(e2e::contains(s::text(r), "no action claimed --pkg-format 'zap'"))) << "the refusal does not name the unclaimed format\n" << s::text(r); + // N30: what the provider stated in the pass is the refusal's reason. + auto declines = variant("declines", R"cpp(import mcpp; +#include +int main() { + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) == "zap") mcpp::warning("zap needs a program named tool"); + return 0; +} +)cpp"); + r = declines.mcpp({"pack", "--pkg-format", "zap"}); + expect(fatal(failed(r))) << "a declined format reported success\n" << s::text(r); + expect(fatal(e2e::contains(s::text(r), "said: zap needs a program named tool"))) << "the refusal does not carry the provider's reason\n" << s::text(r); + // N30: the program the provider states is the one packed when none is named, and a + // program it states that the build has not is refused by the provider's name. + auto chooses = variant("chooses", R"cpp(import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + mcpp::pack_program(MCPP_PACK_PROGRAM); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}/bin").arg(out.c_str()) + .input("${mcpp.target_file:tool}").output(out.c_str()).submit(); + return 0; +} +)cpp"); + chooses.write("mcpp.toml", chooses.read("mcpp.toml") + "\n[targets.tool]\nkind = \"bin\"\nmain = \"src/tool.cpp\"\n"); + chooses.write("src/tool.cpp", "int main() { return 0; }\n"); + const auto stated = chooses.read("build.mcpp"); + chooses.write("build.mcpp", std::string(stated).replace(stated.find("MCPP_PACK_PROGRAM"), 17, "\"tool\"")); + r = chooses.mcpp({"pack", "--pkg-format", "zap"}); + expect(fatal(e2e::succeeded(r))) << "the stated program was not packed\n" << s::text(r); + expect(fatal(e2e::contains(s::text(r), "packing tool, the program app states for its format"))) << "the choice is not said\n" << s::text(r); + const auto zc = s::find_file(chooses.at("target"), "app.zap"); + expect(fatal(zc.has_value() && has_exact_line(s::read_file(*zc), "tool"))) << "the staged tree's bin/ does not hold tool\n" << s::text(r); + chooses.write("build.mcpp", std::string(stated).replace(stated.find("MCPP_PACK_PROGRAM"), 17, "\"nope\"")); + r = chooses.mcpp({"pack", "--pkg-format", "zap"}); + expect(fatal(failed(r) && e2e::contains(s::text(r), "states the program 'nope' (mcpp::pack_program)"))) << "a stated program the build has not was not refused by name\n" << s::text(r); auto twoshapes = variant("twoshapes", R"cpp(import mcpp; #include #include diff --git a/tests/e2e/run/runners.cpp b/tests/e2e/run/runners.cpp index 7cc573326..88a87e7b1 100644 --- a/tests/e2e/run/runners.cpp +++ b/tests/e2e/run/runners.cpp @@ -370,6 +370,97 @@ int main() { expect(fatal(s::find_files(p.at("target"), R"(app\.blob)").empty())) << "a plain build regenerated the distributable"; }; + "1092 a runner that is not enabled is refused until it is enabled"_test[s::unix_shell] = [] { + // R1: a runner stated `enabled = false` is refused by `run` (exit 2) and leaves the tests + // not run, naming the token, what it would install and how to enable it; nothing is + // installed. The environment enables it once; naming a runner with `--runner` enables it; + // `--list-runners` marks it. + auto p = s::project::scaffold("app"); + for (const auto& t : s::glob(p.at("tests"), "*.cpp")) fs::remove(t); + p.write("src/main.cpp", "#include \nint main() { std::puts(\"ARTIFACT-RAN\"); return 0; }\n"); + p.write("tests/one.cpp", "int main() { return 0; }\n"); + auto r = p.mcpp({"build"}); + expect(fatal(e2e::succeeded(r))) << "initial build: " << s::text(r); + std::vector triples; + for (const auto& e : fs::directory_iterator(p.at("target"))) + if (!e.path().filename().string().starts_with(".")) triples.push_back(e.path().filename().string()); + std::ranges::sort(triples); + expect(fatal(!triples.empty())) << "could not determine the host triple from target/"; + const auto host = triples.front(); + std::string variable = "MCPP_TARGET_"; + for (char c : host) variable += std::isalnum(static_cast(c)) ? char(std::toupper(c)) : '_'; + variable += "_RUNNER_ENABLED"; + const auto runner = (s::case_dir() / "runner.sh").string(); + s::write_executable(runner, "#!/bin/sh\necho RUNNER-RAN\nexec \"$@\"\n"); + const auto base = p.read("mcpp.toml"); + p.write("mcpp.toml", base + std::format( + "\n[target.{}]\nrunner = {{ cmd = [\"{}\"], requires = [\"xim:mcpp-e2e-not-installed@1.0\"], enabled = false }}\n", + host, runner)); + r = p.mcpp({"run"}); + expect(fatal(r.started && r.status == 2)) << "A: a runner that is not enabled was not refused with 2: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "[runner-not-enabled]"))) << "A: no token: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "it needs: xim:mcpp-e2e-not-installed@1.0"))) << "A: the packages are not named: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), variable + "=true"))) << "A: the one-off variable is not named: " << s::text(r); + expect(fatal(s::lacks(s::text(r), "RUNNER-RAN") && s::lacks(s::text(r), "ARTIFACT-RAN"))) << "A: something ran: " << s::text(r); + r = p.mcpp({"test"}); + expect(fatal(r.started && r.status != 0)) << "B: the tests ran through a runner that is not enabled: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "[runner-not-enabled]"))) << "B: the tests' reason has no token: " << s::text(r); + + p.write("mcpp.toml", base + std::format( + "\n[target.{}]\nrunner = {{ cmd = [\"{}\"], enabled = false }}\n[target.{}.runners]\nprobe = {{ cmd = [\"{}\"], enabled = false }}\n", + host, runner, host, runner)); + r = p.mcpp({"run"}, s::options{.env = {{variable, "true"}}}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "RUNNER-RAN"))) << "C: the variable did not enable it: " << s::text(r); + r = p.mcpp({"run", "--runner", "probe"}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "RUNNER-RAN"))) << "D: naming the runner did not enable it: " << s::text(r); + r = p.mcpp({"run", "--list-runners"}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "[not enabled]"))) << "E: the list does not mark it: " << s::text(r); + p.write("mcpp.toml", base + std::format("\n[target.{}]\nrunner = [\"{}\"]\n", host, runner)); + r = p.mcpp({"run"}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "RUNNER-RAN"))) << "F: an argv runner is enabled: " << s::text(r); + }; + + "1093 a build program states a runner as its table"_test[s::unix_shell] = [] { + // R2: a dependency's `mcpp::runner_spec` states the default runner with `requires` and + // `enabled = false`, and a named long-lived one. The run is refused naming the package, + // enabling it installs from it, `--list-runners` marks both, and the named one runs. + const auto tmp = s::case_dir(); + const auto runner = (tmp / "runner.sh").string(); + s::write_executable(runner, "#!/bin/sh\necho RUNNER-RAN\nexec \"$@\"\n"); + s::project(tmp / "dep", {{"mcpp.toml", "[package]\nnamespace = \"demo\"\nname = \"dep\"\nversion = \"0.1.0\"\n[targets.dep]\nkind = \"lib\"\n"}, + {"src/dep.cppm", "export module dep;\nexport int dep_anchor() { return 0; }\n"}, + {"build.mcpp", std::format( + "import std;\nimport mcpp;\nint main() {{\n" + " mcpp::runner_spec run;\n run.enabled = false;\n" + " run.arg(\"{0}\").require(\"xim:mcpp-e2e-not-installed@1.0\");\n run.submit();\n" + " mcpp::runner_spec serve;\n serve.name = \"serve\";\n serve.longlived = true;\n" + " serve.arg(\"{0}\");\n serve.submit();\n return 0;\n}}\n", runner)}}); + s::project app(tmp / "app", {{"mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n[dependencies]\ndemo.dep = { path = \"../dep\" }\n"}, + {"src/main.cpp", "import dep;\nint main() { return dep_anchor(); }\n"}}); + auto r = app.mcpp({"build"}); + expect(fatal(e2e::succeeded(r))) << "the build failed: " << s::text(r); + r = app.mcpp({"run"}); + expect(fatal(r.started && r.status == 2 && e2e::contains(s::text(r), "[runner-not-enabled]"))) + << "A: the program's runner that is not enabled was not refused: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "it needs: xim:mcpp-e2e-not-installed@1.0"))) << "A: its requires did not arrive: " << s::text(r); + std::string variable; + for (const auto& e : fs::directory_iterator(app.at("target"))) + if (!e.path().filename().string().starts_with(".")) { + variable = "MCPP_TARGET_"; + for (char c : e.path().filename().string()) + variable += std::isalnum(static_cast(c)) ? char(std::toupper(c)) : '_'; + variable += "_RUNNER_ENABLED"; + } + r = app.mcpp({"run"}, s::options{.env = {{variable, "true"}}}); + expect(fatal(r.started && r.status != 0 && e2e::contains(s::text(r), "installed from xim:mcpp-e2e-not-installed@1.0"))) + << "B: enabling it did not install from its package: " << s::text(r); + r = app.mcpp({"run", "--list-runners"}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "[not enabled]") && e2e::contains(s::text(r), "[long-lived]"))) + << "C: the list does not show the table: " << s::text(r); + r = app.mcpp({"run", "--runner", "serve"}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "RUNNER-RAN"))) << "D: the named runner did not run: " << s::text(r); + }; + "684 a runner supplied twice is refused"_test[s::unix_shell] = [] { // A runner name a dependency's build program supplies alone is accepted (3); supplied // by the project's build program too, it is refused naming both (1); the manifest's diff --git a/tests/e2e/targets/abi.cpp b/tests/e2e/targets/abi.cpp index e6cd07781..529680283 100644 --- a/tests/e2e/targets/abi.cpp +++ b/tests/e2e/targets/abi.cpp @@ -111,9 +111,10 @@ E2E abi{"abi", [] { "653 exceptions is the second abi member"_test[s::elf] = [] { // `[target..abi] exceptions = true`: a Linux build is byte-identical with an - // emscripten-only table; on the Web row the program builds without it and aborts at the - // throw, and with it prints the marker, the flag reaching the dependency and the std - // module and the fingerprint moving; a dependency requiring it is refused without it. + // emscripten-only table; on the Web row the row's model catches with no table (N26), the + // program builds with `exceptions = false` and aborts at the throw, and with `true` prints + // the marker, the flag reaching the dependency and the std module and the fingerprint + // moving; a dependency requiring it is served by the row and refused with it off. bool have_emsdk = false; for (const auto& root : {s::shared_home() / "registry/data/xpkgs", fs::path(s::env("HOME").value_or("")) / ".xlings/data/xpkgs"}) for (const auto& em : s::glob(root, "xim-x-emsdk/*/emscripten/em++")) @@ -132,7 +133,8 @@ E2E abi{"abi", [] { expect(fatal(e2e::succeeded(r))) << "FAIL: the Linux build with the table failed: " << s::text(r); const auto with_ninja = s::read_file(first_ninja(lx.at("target")).value_or(fs::path())); expect(fatal(!plain_ninja.empty() && plain_ninja == with_ninja)) << "FAIL: a Linux build.ninja differs with the emscripten-only abi table present"; - // 2. Without exceptions the build and link succeed, and the run fails at the throw. + // 2. With no table the row's model (`exceptions = "wasm"`, N26) catches the throw; with + // `exceptions = false` the build and link succeed, and the run fails at the throw. s::project(tmp / "excdep", {{"mcpp.toml", "[package]\nname = \"excdep\"\nversion = \"0.1.0\"\n\n[targets.excdep]\nkind = \"lib\"\n"}, {"src/excdep.cppm", "export module excdep;\nimport std;\nexport void throwing() { throw std::runtime_error(\"boom\"); }\n"}}); s::project app(tmp / "app", {{"src/main.cpp", R"(import excdep; @@ -157,11 +159,15 @@ int main() { return args; }; write_app(""); + r = app.mcpp(with("run")); + expect(fatal(e2e::succeeded(r))) << "FAIL: the row's exception model did not catch the throw: " << s::text(r); + expect(fatal(s::has_line(s::text(r), "^caught: boom$"))) << "FAIL: the row's model did not print the marker"; + write_app("[target.'cfg(os = \"emscripten\")'.abi]\nexceptions = false\n"); r = app.mcpp(with("build")); - expect(fatal(e2e::succeeded(r))) << "FAIL: the build without the abi table failed (expected: it succeeds; only the run fails): " << s::text(r); + expect(fatal(e2e::succeeded(r))) << "FAIL: the build with exceptions off failed (expected: it succeeds; only the run fails): " << s::text(r); const auto plain_dir = newest_ninja(app.at("target/wasm32-emscripten")).value_or(fs::path()).parent_path(); r = app.mcpp(with("run")); - expect(fatal(!r)) << "FAIL: mcpp run succeeded without -fexceptions -- the throw should have aborted: " << s::text(r); + expect(fatal(!r)) << "FAIL: mcpp run succeeded with exceptions off -- the throw should have aborted: " << s::text(r); expect(fatal(s::no_line(s::text(r), "^caught: boom$"))) << "FAIL: the marker printed without -fexceptions"; expect(fatal(e2e::contains(s::text(r), "exception catching is not enabled"))) << "FAIL: the run did not fail with the expected Emscripten abort: " << s::text(r); // 3. With exceptions the marker prints, the flags reach both compiles, and the @@ -187,13 +193,17 @@ int main() { const auto stdmodule_json = fs::path(stdcache_pcm).parent_path().parent_path() / "std-module.json"; expect(fatal(fs::is_regular_file(stdmodule_json))) << "FAIL: no std-module.json beside the cached std.pcm build.ninja names: " << stdmodule_json.string(); expect(fatal(e2e::contains(s::read_file(stdmodule_json), "-fexceptions"))) << "FAIL: -fexceptions is not recorded on the std module's cached prebuild command"; - // 4. A dependency requiring exceptions is refused on the Web build without the table. + // 4. A dependency requiring exceptions is served by the row's model, and refused on the + // Web build whose root states `exceptions = false`. s::project(tmp / "excreq", {{"mcpp.toml", "[package]\nname = \"excreq\"\nversion = \"0.1.0\"\nrequires_abi = { exceptions = true }\n\n[targets.excreq]\nkind = \"lib\"\n"}, {"src/excreq.cppm", "export module excreq;\nexport int v() { return 1; }\n"}}); s::project app3(tmp / "app3", {{"mcpp.toml", "[package]\nname = \"app3\"\nversion = \"0.1.0\"\n\n[dependencies]\nexcreq = { path = \"../excreq\" }\n\n[targets.app3]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n[target.wasm32-emscripten]\nrunner = [\"node\"]\n"}, {"src/main.cpp", "import excreq;\nint main() { return v() == 1 ? 0 : 1; }\n"}}); r = app3.mcpp(with("build")); - expect(fatal(!r)) << "FAIL: a dependency requiring exceptions was built for the Web without the root's table: " << s::text(r); + expect(fatal(e2e::succeeded(r))) << "FAIL: the row's exception model did not serve a dependency requiring exceptions: " << s::text(r); + app3.write("mcpp.toml", app3.read("mcpp.toml") + "\n[target.'cfg(os = \"emscripten\")'.abi]\nexceptions = false\n"); + r = app3.mcpp(with("build")); + expect(fatal(!r)) << "FAIL: a dependency requiring exceptions was built for the Web with exceptions off: " << s::text(r); expect(fatal(e2e::contains(s::text(r), "`excreq` requires the artefact's ABI to have exceptions (the package)"))) << "FAIL: the refusal does not name 'exceptions': " << s::text(r); }; diff --git a/tests/e2e/test/workspace.cpp b/tests/e2e/test/workspace.cpp index 471021663..c6306ec1a 100644 --- a/tests/e2e/test/workspace.cpp +++ b/tests/e2e/test/workspace.cpp @@ -1,6 +1,6 @@ // `mcpp test` over a workspace's members: selection, one plan per configuration group, the report -// of a shared build, a failing member, a member's programs (formerly tests/e2e/90, 854, 855, 856 -// and 1034). +// of a shared build, a failing member, a member's programs, a shared module compiled once, the +// guide at a root (formerly tests/e2e/90, 854, 855, 856 and 1034; 1094, 1095). import std; import mcpplibs.mtest; import mcpp.test.support; @@ -374,6 +374,70 @@ int main() {{ expect(e2e::succeeded(ws.mcpp({"test", "-p", "a"}))); expect(e2e::contains(s::text(w), "finds ... ok")); }; + + "1094 a members test compiles a shared module once"_test[s::gcc] = [] { + // #936: the programs of a group are planned with the group, so a package two members share + // (here with a feature one of them asks for, and a module its build program writes) is + // compiled once in a run and not again in the next. + s::project ws{{"mcpp.toml", "[workspace]\nmembers = [\"core\", \"a\", \"app\"]\n"}, + {"core/mcpp.toml", "[package]\nname = \"core\"\nversion = \"0.1.0\"\n\n[features]\nextra = []\n\n[targets.core]\nkind = \"lib\"\n"}, + {"core/src/core.cppm", "export module s1094_core;\nexport import s1094_core.gen;\nexport int core_v() { return gen_v(); }\n"}, + {"core/build.mcpp", R"(import std; +import mcpp; +int main() { + const std::string p = std::string(mcpp::out_dir()) + "/gen.cppm"; + std::ofstream(p) << "export module s1094_core.gen;\nexport int gen_v() { return 1; }\n"; + std::printf("mcpp:generated=%s\n", p.c_str()); + return 0; +} +)"}, + {"a/mcpp.toml", "[package]\nname = \"a\"\nversion = \"0.1.0\"\n\n[dependencies]\ncore = { path = \"../core\", features = [\"extra\"] }\n\n[targets.a]\nkind = \"lib\"\n"}, + {"a/src/a.cppm", "export module s1094_a;\nimport s1094_core;\nexport int a_v() { return core_v(); }\n"}, + {"a/tests/main.cpp", "import s1094_a;\nint main() { return a_v() == 1 ? 0 : 1; }\n"}, + {"app/mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[dependencies]\ncore = { path = \"../core\" }\n\n[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"}, + {"app/src/main.cpp", "import s1094_core;\nint main() { return core_v() == 1 ? 0 : 1; }\n"}, + {"app/tests/main.cpp", "import std;\nint main() { return std::getenv(\"MCPP_BIN_APP\") ? 0 : 1; }\n"}}; + auto gen_edges = [&] { + std::ptrdiff_t n = 0; + for (const auto& d : graph_dirs(ws)) + n += std::ranges::count_if(s::ninja_log_outputs(d), [](const std::string& o) { + return std::regex_search(o, std::regex(R"(/core/gen\.m\.o$)")); + }); + return n; + }; + auto first = ws.mcpp({"test", "--workspace"}); + expect(fatal(e2e::succeeded(first))) << s::text(first); + expect(eq(gen_edges(), 1)) << "the shared module was compiled more than once in one run"; + expect(fatal(e2e::succeeded(ws.mcpp({"test", "--workspace"})))); + expect(eq(gen_edges(), 1)) << "an unchanged shared module was compiled again"; + }; + + "1095 the guide at a workspace root plans nothing"_test[s::gcc] = [] { + // #932: the guide reads the frameworks from the resolved graph -- every member's -- and + // plans, scans and builds nothing. + s::project ws{{"mcpp.toml", "[workspace]\nmembers = [\"fw\", \"a\", \"b\"]\n"}, + {"fw/mcpp.toml", "[package]\nname = \"fw\"\nversion = \"0.1.0\"\n\n[targets.main]\nkind = \"entry\"\nmain = \"src/main.cpp\"\ntest-protocol = 1\ntest-guide = \"GUIDE.md\"\n"}, + {"fw/GUIDE.md", "# writing tests\n"}, + {"fw/src/main.cpp", "int main() { return 0; }\n"}, + {"a/mcpp.toml", "[package]\nname = \"a\"\nversion = \"0.1.0\"\n\n[dev-dependencies]\nfw = { path = \"../fw\" }\n"}, + {"a/tests/t.cpp", "int main() { return 0; }\n"}, + {"b/mcpp.toml", "[package]\nname = \"b\"\nversion = \"0.1.0\"\n"}, + {"b/tests/t.cpp", "int main() { return 0; }\n"}}; + auto g = ws.mcpp({"test", "--guide"}); + expect(fatal(e2e::succeeded(g))) << s::text(g); + expect(e2e::contains(g.out, "Test frameworks of 3 members")); + expect(e2e::contains(g.out, "fw 0.1.0 (fw/main, test protocol 1)")); + expect(e2e::contains(g.out, "GUIDE.md")); + expect(!e2e::contains(s::text(g), "Compiling")) << "the guide built"; + expect(graph_dirs(ws).empty()) << "the guide planned a build"; + auto j = ws.mcpp({"test", "--guide", "--message-format", "json"}); + expect(fatal(e2e::succeeded(j))); + expect(e2e::contains(j.out, R"("entry":"fw/main")")); + expect(e2e::contains(j.out, R"("members":3)")); + auto one = ws.mcpp_in("b", {"test", "--guide"}); + expect(fatal(e2e::succeeded(one))); + expect(e2e::contains(one.out, "No test framework")) << "a member's guide named another member's framework"; + }; }}; } // namespace diff --git a/tests/e2e/web/emscripten.cpp b/tests/e2e/web/emscripten.cpp index 63c89311a..ec00ad63c 100644 --- a/tests/e2e/web/emscripten.cpp +++ b/tests/e2e/web/emscripten.cpp @@ -221,6 +221,33 @@ E2E emscripten{"emscripten", [] { expect(fatal(e2e::contains(s::text(r), "\"wasm\" or \"js\""))) << "the refusal does not name the values: " << s::text(r); }; + "1091 an emscripten build catches and packs without its module"_test[s::elf] = [] { + // The row's exception model applies where the project states none: the program catches + // (N26). A link with -sSINGLE_FILE=1 writes no module: the build and a clean pack succeed, + // and the module an earlier link wrote is removed, not packed (N33). + if (!have_emsdk() || !s::on_path("node")) skip("1091 -- no emsdk payload or node on this machine"); + s::project app(s::case_dir() / "app", {{"src/main.cpp", "import std;\nint main() {\n try { throw std::runtime_error(\"boom\"); }\n catch (std::exception const& e) { std::println(\"caught: {}\", e.what()); }\n}\n"}}); + const std::string base = "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n[target.wasm32-emscripten]\nrunner = [\"node\"]\n"; + app.write("mcpp.toml", base); + auto r = app.mcpp({"run", "--target", "wasm32-emscripten"}); + expect(fatal(e2e::succeeded(r) && s::has_line(s::text(r), "^caught: boom$"))) << "the row's model did not catch: " << s::text(r); + const auto bin = s::find_file(app.at("target/wasm32-emscripten"), "app.js"); + expect(fatal(bin.has_value())) << "no launcher"; + const auto module = bin->parent_path() / "app.wasm"; + expect(fatal(fs::exists(module))) << "the ordinary link wrote no module"; + app.write("mcpp.toml", base + "\n[target.wasm32-emscripten.build]\nldflags = [\"-sSINGLE_FILE=1\"]\n"); + r = app.mcpp({"build", "--target", "wasm32-emscripten"}); + expect(fatal(e2e::succeeded(r))) << "a single-file web link failed: " << s::text(r); + expect(fatal(!fs::exists(module))) << "the earlier link's module is still beside the launcher"; + const auto linked = fs::last_write_time(*bin); + r = app.mcpp({"build", "--target", "wasm32-emscripten"}); + expect(fatal(e2e::succeeded(r))) << "the second build failed: " << s::text(r); + expect(fatal(fs::last_write_time(*bin) == linked)) << "a link without its module is linked again"; + r = app.mcpp({"pack", "--target", "wasm32-emscripten", "--pkg-format", "dir"}); + expect(fatal(e2e::succeeded(r))) << "a single-file web link did not pack: " << s::text(r); + expect(fatal(!s::find_file(app.at("target/dist"), "app.wasm").has_value())) << "a module was packed"; + }; + "1079 an emscripten rows longjmp model round trips"_test[s::elf] = [] { // `longjmp = "wasm"` makes a non-local exit over two call levels round-trip; on a Linux // row it is refused naming that row's os. diff --git a/tests/e2e/windows/rows.cpp b/tests/e2e/windows/rows.cpp index bbf4fa055..61ee42c9c 100644 --- a/tests/e2e/windows/rows.cpp +++ b/tests/e2e/windows/rows.cpp @@ -73,6 +73,40 @@ E2E rows{"rows", [] { expect(fatal(e2e::succeeded(r))) << "[resources] did not build on x86_64-windows-musl: " << s::text(r); carries_versioninfo(p, "x86_64-windows-musl", "verinfo"); }; + + "1089 a program's stated name is followed on every windows row"_test[s::mingw_cross][s::llvm] = [] { + // A build program that names the program `{name}` is followed on the built-in GNU-ABI row + // as on the row a platform serves: a target named `cmd.exe` is `cmd.exe` on both, and + // no `cmd.exe.exe` (design 2026-11-15 N9). + s::project p{{"mcpp.toml", "[package]\nname = \"named\"\nversion = \"0.1.0\"\n\n[targets.\"cmd.exe\"]\nkind = \"bin\"\nmain = \"src/main.c\"\n"}, + {"src/main.c", "int main(void) { return 0; }\n"}, + {"build.mcpp", "import mcpp;\nint main() { mcpp::artifact_name(\"exe\", \"{name}\"); return 0; }\n"}}; + for (std::string_view row : {"x86_64-windows-gnu", "x86_64-windows-musl"}) { + auto r = p.mcpp({"build", "--target", std::string(row), "--toolchain", "llvm@" + s::llvm_version()}); + expect(fatal(e2e::succeeded(r))) << row << ": " << s::text(r); + expect(fatal(s::find_file(p.at("target") / std::string(row), "cmd.exe").has_value())) << row << ": no cmd.exe"; + expect(fatal(!s::find_file(p.at("target") / std::string(row), "cmd.exe.exe").has_value())) << row << ": the row's suffix was appended to the stated name"; + } + }; + + "1090 a uefi application compiles no windows resources"_test[s::llvm] = [] { + // `[resources]` belongs to a program Windows runs: the UEFI row, a PE image the firmware + // loads, builds without them and without a word (design 2026-11-15 N16). + // The row is the platform's: the project selects the engine's pin of it (builtin.toml). + const auto pin = [] { + std::ifstream in(s::repository() / "modules/toolchain-model/data/builtin.toml", std::ios::binary); + const std::string text{std::istreambuf_iterator{in}, {}}; + std::smatch m; + const std::regex line{R"rx(platform\s*=\s*"mcpplibs\.openkal-platform@([0-9.]+)")rx"}; + return std::regex_search(text, m, line) ? m[1].str() : std::string{}; + }(); + expect(fatal(!pin.empty())) << "builtin.toml names no openkal-platform release"; + s::project p{{"mcpp.toml", manifest("efiapp") + std::format("\n[target.x86_64-uefi]\nplatform = \"openkal-platform@{}\"\n", pin)}, + {"src/main.c", "int main(void) { return 0; }\n"}, {"app.rc", verinfo_rc}}; + const auto r = p.mcpp({"build", "--target", "x86_64-uefi"}); + expect(fatal(e2e::succeeded(r))) << "[resources] failed the UEFI build: " << s::text(r); + expect(fatal(s::lacks(s::text(r), "app.rc"))) << "the UEFI build spoke of the resource script: " << s::text(r); + }; }}; } // namespace diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index 468e8d64c..bc1aefed4 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -320,6 +320,65 @@ UTest a_pack_format_declaration_is_persisted{"BuildDirectives/APackFormatDeclara expect(eq(def->scope, dirs::Scope::Claim)); }}; +UTest a_pack_program_and_the_reasons_are_carried{"BuildDirectives/APackProgramAndTheReasonsAreCarried", [] { + // N30: the program a provider assembles from is carried as stated, and is + // persisted as `pack-format` is, for the same reason (the pass that reads + // it is never the first build); the warnings and degradations the program + // stated are kept as its reasons, notes are not. + auto d = parse("mcpp:pack-format=web\n" + "mcpp:pack-program=app\n" + "mcpp:warning=not the emscripten target\n" + "mcpp:diagnostic=note\tthe web directory\t\t\n" + "mcpp:diagnostic=degraded\tno launcher\t\t\n"); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + expect(eq(m.buildConfig.packPrograms, (std::vector{"app"}))); + expect(eq(m.buildConfig.statedReasons, (std::vector{"no launcher", "not the emscripten target"}))); + expect(m.buildConfig.ldflags.empty()); + const auto* def = dirs::find_by_wire("pack-program"); + expect(fatal(ne(def, nullptr))); + expect(!(def->tag.empty())) << "a tag-less row is not replayed on a cache hit"; + expect(eq(def->scope, dirs::Scope::Claim)); +}}; + +UTest a_runner_spec_is_the_runners_table{"BuildDirectives/ARunnerSpecIsTheRunnersTable", [] { + // R2: one payload states a runner as its table; the default's command is the runner, a + // named one's its argv; the rest of the table is carried. + auto d = parse("mcpp:runner-spec={\"name\":\"web\",\"host\":\"\",\"longlived\":true,\"exclusive\":false," + "\"enabled\":false,\"cmd\":[\"{pkg:emsdk}/emscripten/emrun\",\"{}\"],\"hosts\":[\"linux-x86_64\"]," + "\"requires\":[\"xim:emsdk\"]}\n" + "mcpp:runner-spec={\"name\":\"\",\"host\":\"same-arch\",\"longlived\":false,\"exclusive\":true," + "\"enabled\":true,\"cmd\":[\"okrun\",\"{}\"],\"hosts\":[],\"requires\":[]}\n"); + expect(eq(dirs::runner_spec_error(d), std::string())); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + auto const& web = m.buildConfig.namedRunners.at("web"); + expect(eq(web.argv, (std::vector{"{pkg:emsdk}/emscripten/emrun", "{}"}))); + expect(bool(web.longLived) && !web.enabled); + expect(eq(web.requires_, (std::vector{"xim:emsdk"}))); + expect(eq(web.hosts, (std::vector{"linux-x86_64"}))); + expect(eq(m.buildConfig.runner, (std::vector{"okrun", "{}"}))); + expect(eq(m.buildConfig.runnerFields.host, "same-arch")); + expect(bool(m.buildConfig.runExclusive)); +}}; + +UTest a_runner_is_stated_one_way{"BuildDirectives/ARunnerIsStatedOneWay", [] { + // R2: a name stated by runner_spec and by the token calls, twice, without a command, or + // with an unknown host, is refused before anything is applied. + auto spec = [](std::string name, std::string cmd, std::string host = "") { + return std::format("mcpp:runner-spec={{\"name\":\"{}\",\"host\":\"{}\",\"longlived\":false,\"exclusive\":false," + "\"enabled\":true,\"cmd\":[{}],\"hosts\":[],\"requires\":[]}}\n", name, host, cmd); + }; + expect(ne(dirs::runner_spec_error(parse(spec("web", "\"emrun\"") + "mcpp:runner-named=web:x\n")).find("one way"), + std::string::npos)); + expect(ne(dirs::runner_spec_error(parse(spec("", "\"okrun\"") + "mcpp:runner=okrun\n")).find("one way"), + std::string::npos)); + expect(ne(dirs::runner_spec_error(parse(spec("web", "\"a\"") + spec("web", "\"b\""))).find("twice"), + std::string::npos)); + expect(ne(dirs::runner_spec_error(parse(spec("web", ""))).find("no command"), std::string::npos)); + expect(ne(dirs::runner_spec_error(parse(spec("web", "\"a\"", "near"))).find("same-arch"), std::string::npos)); +}}; + UTest a_claim_reaches_neither_compile_nor_link{"BuildDirectives/AClaimReachesNeitherCompileNorLink", [] { auto d = parse("mcpp:fact=widget.driver=1.2\n" "mcpp:floor=widget.driver >= 2.0\n"); @@ -1045,13 +1104,13 @@ UTest deploy_row_is_protocol_eleven_with_link_global_scope_and_a_tag{"BuildDirec expect(eq(def->scope, dirs::Scope::LinkGlobal)); expect(eq(def->sinceProtocol, 11)); expect(!(def->tag.empty())); - expect(eq(dirs::kProtocolVersion, 18)); + expect(eq(dirs::kProtocolVersion, 19)); }}; -UTest protocol_eighteen_is_accepted_and_nineteen_is_not{"BuildDirectives/ProtocolEighteenIsAcceptedAndNineteenIsNot", [] { - auto ok = parse("mcpp:protocol=18\n"); +UTest protocol_nineteen_is_accepted_and_twenty_is_not{"BuildDirectives/ProtocolNineteenIsAcceptedAndTwentyIsNot", [] { + auto ok = parse("mcpp:protocol=19\n"); expect(!(dirs::protocol_error(ok).has_value())); - auto no = parse("mcpp:protocol=19\n"); + auto no = parse("mcpp:protocol=20\n"); expect(dirs::protocol_error(no).has_value()); }}; diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index dd94bae45..8f55084e6 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -169,6 +169,35 @@ UTest ordinary_compilers_share_the_detached_schedule_bound{"NinjaBackend/Ordinar << "a join must not wait for a second permit"; }}; +UTest links_hold_a_permit_of_the_schedule_bound{"NinjaBackend/LinksHoldAPermitOfTheScheduleBound", [] { + // A link takes a permit of the semaphore the compilers share, so links that + // become ready together do not all run at once; an archive holds none, and a + // graph without a cap wraps nothing. + auto plan = minimal_plan(); + plan.toolchain.version = "16.1.0"; + plan.scheduleTag = "detach-codegen"; + plan.scheduleCompilerCap = 2; + auto rule = [](const std::string& text, std::string_view name) { + const auto start = text.find("rule " + std::string(name) + "\n"); + return start == std::string::npos ? std::string{} : text.substr(start, text.find("\n\n", start) - start); + }; + const auto text = emit_ninja_string(plan); + const std::string permit = "$mcpp __compiler-cap --sem .mcpp-sched --output $out -- "; + for (const auto name : {"cxx_link", "cxx_shared"}) { + const auto r = rule(text, name); + expect(ne(r.find(permit), std::string::npos)) << name << "\n" << r; + // The permit wraps the publication, so the artifact is in place when + // the command it holds exits. + if constexpr (!mcpplibs::os::is_windows) + expect(ne(r.find(permit + "$mcpp link-output --output $out --temporary -- "), std::string::npos)) + << name << "\n" << r; + } + expect(eq(rule(text, "cxx_archive").find("__compiler-cap"), std::string::npos)) << rule(text, "cxx_archive"); + expect(eq(text.find("--cap"), std::string::npos)) << "the graph names no number of compilers"; + plan.scheduleCompilerCap = 0; + expect(eq(rule(emit_ninja_string(plan), "cxx_link").find("__compiler-cap"), std::string::npos)); +}}; + UTest objective_c_source_uses_c_object_rule_and_c_flags{"NinjaBackend/ObjectiveCSourceUsesCObjectRuleAndCFlags", [] { auto plan = minimal_plan(); plan.compileUnits.push_back({ diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp index 359534863..863bce5ae 100644 --- a/tests/unit/test_platform.cpp +++ b/tests/unit/test_platform.cpp @@ -5,6 +5,8 @@ import mcpp.manifest; import mcpp.platform; import mcpp.build.prepare_inputs; import mcpp.toolchain.triple; +import mcpp.toolchain.target_rows; +import mcpp.toolchain.platform; using namespace mcpplibs::mtest; @@ -359,6 +361,70 @@ UTest the_rules_of_section45{"PlatformMerge/TheRulesOfSection45", [] { expect(eq(base.link.start.value, "ok_main")); }}; +UTest enabled_belongs_to_the_command{"PlatformMerge/EnabledBelongsToTheCommand", [] { + // R1, R2: `cmd`, `requires` and `enabled` are one statement; a table without `cmd` + // adjusts the runner below it; `longlived` stands until a nearer statement writes it. + mp::Statement base; + base.runner.cmd = {"dirun", "{}"}; + base.runner.requires_ = {"xim:dirun@0.4.0"}; + base.runner.enabled = false; + base.runner.enabledSet = true; + base.runner.set = true; + base.runners["web"].cmd = {"emrun", "{}"}; + base.runners["web"].longlived = base.runners["web"].longlivedSet = true; + base.runners["web"].set = true; + + mp::Statement enable; // `runner = { enabled = true }` + enable.runner.enabled = true; + enable.runner.enabledSet = true; + enable.runners["web"].enabled = false; // `runners.web = { enabled = false }` + enable.runners["web"].enabledSet = true; + auto a = base; + pf::merge(a, enable); + expect(bool(a.runner.enabled)); + expect(eq(a.runner.cmd.front(), "dirun")); + expect(eq(a.runner.requires_.size(), 1u)); + expect(!a.runners.at("web").enabled); + expect(eq(a.runners.at("web").cmd.front(), "emrun")); + expect(bool(a.runners.at("web").longlived)); + + mp::Statement replace; // `runner = ["wine", "{}"]`: a new statement + replace.runner.cmd = {"wine", "{}"}; + replace.runner.set = true; + auto b = base; + pf::merge(b, replace); + expect(bool(b.runner.enabled)); + expect(b.runner.requires_.empty()); +}}; + +UTest a_runner_table_states_enabled_and_longlived{"PlatformManifest/ARunnerTableStatesEnabledAndLonglived", [] { + auto m = parse(R"( +[package] +name = "app" +version = "0.1.0" + +[target.x86_64-freebsd] +runner = { enabled = true } + +[target.wasm32-emscripten.runners] +web = { cmd = ["emrun", "{}"], longlived = true, enabled = false } +)"); + for (auto const& w : m.schemaWarnings) expect(false) << "unexpected warning: " << w; + expect(fatal(eq(m.platformOverrides.size(), 2u))); + auto of = [&](std::string_view selector) -> const mp::Statement& { + for (auto const& o : m.platformOverrides) + if (o.selector == selector) return o.fields; + expect(fatal(false)) << "no override for " << selector; + return m.platformOverrides.front().fields; + }; + auto const& f = of("x86_64-freebsd").runner; + expect(!f.set); + expect(bool(f.enabledSet) && bool(f.enabled)); + auto const& web = of("wasm32-emscripten").runners.at("web"); + expect(bool(web.set) && bool(web.longlived) && !web.enabled); + expect(eq(m.targetOverrides.at("wasm32-emscripten").namedRunners.at("web").front(), "emrun")); +}}; + UTest a_host_triple_takes_the_hosted_part_only{"PlatformMerge/AHostTripleTakesTheHostedPartOnly", [] { auto p = parse(kPlatform).platform; auto h = pf::hosted_view(p); @@ -457,6 +523,60 @@ UTest the_runner_reaches_the_artifacts_architecture{"PlatformRun/TheRunnerReache expect(eq(pf::runner_reach(*r.runner, r.target, "x86_64", r.executionHost, r.noRunner), r.want)) << r.runner->host << " " << r.target << " executionHost=" << r.executionHost << " noRunner=" << r.noRunner; + // N23: an x86_64 Linux machine executes i686 (the execution table), so a same-arch runner + // is built for i686 and started here; elsewhere the table states nothing and it is refused. + const bool linuxHost = std::string_view(mcpp::toolchain::platform::name) == "linux"; + expect(eq(pf::runner_reach(same, "i686", "x86_64", false, false), + linuxHost ? R::TargetArchitecture : R::Refused)); +}}; + +UTest the_execution_table_states_measured_relations{"PlatformRun/TheExecutionTableStatesMeasuredRelations", [] { + namespace tt = mcpp::toolchain::target; + expect(tt::executes("linux-x86_64", "i686")); + expect(tt::executes("linux-x86_64", "x86_64")); // its own architecture + expect(!tt::executes("linux-aarch64", "i686")); + expect(!tt::executes("macos-aarch64", "x86_64")); // not stated: not measured + expect(!tt::execution_relations().empty()); +}}; + +UTest the_system_table_states_which_systems_read_resources{"PlatformRun/TheSystemTableStatesWhichSystemsReadResources", [] { + // N16: `[resources]` compile for the programs of a system whose loader reads them, a key of + // the system table; a PE image another system loads (a UEFI application) has none. + namespace tt = mcpp::toolchain::target; + auto const* windows = tt::find_system("windows"); + expect(fatal(windows != nullptr)); + expect(windows->resources); + auto const* linux_ = tt::find_system("linux"); + expect(fatal(linux_ != nullptr)); + expect(!linux_->resources); + auto const* uefi = tt::find_system("uefi"); + expect(uefi == nullptr || !uefi->resources); +}}; + +UTest the_android_row_runs_in_the_emulator_on_request{"PlatformRun/TheAndroidRowRunsInTheEmulatorOnRequest", [] { + // G7, R1: an x86_64 Android program runs through dirun and the API 34 image's profile, on a + // machine of its architecture, one run at a time, once enabled. The aarch64 row states none: + // the emulator package serves no aarch64 machine. + namespace tt = mcpp::toolchain::target; + using provider = decltype(tt::providers())::value_type; + provider x86 = nullptr; + provider arm = nullptr; + for (auto const* p : tt::providers()) { + if (p->name != "android-ndk") continue; + for (auto sv : p->serves) { + if (sv == "x86_64-linux-android") x86 = p; + if (sv == "aarch64-linux-android") arm = p; + } + } + expect(fatal(x86 != nullptr && arm != nullptr)); + expect(fatal(!x86->runner.empty())); + expect(eq(x86->runner.front(), std::string_view("dirun"))); + expect(eq(x86->runnerHost, std::string_view("same-arch"))); + expect(!x86->runnerEnabled) << "a runner that boots an emulator is enabled by default"; + expect(x86->runnerExclusive) << "two runs would start two devices at one console port"; + expect(std::ranges::find(x86->runnerRequires, std::string_view("xim:android-system-image@34-default-x86_64")) + != x86->runnerRequires.end()); + expect(arm->runner.empty()) << "the aarch64 row names a runner no machine can install"; }}; // ── The report ────────────────────────────────────────────────────────────── diff --git a/tests/unit/test_runner_lookup.cpp b/tests/unit/test_runner_lookup.cpp index ab518d1f9..8a3d24226 100644 --- a/tests/unit/test_runner_lookup.cpp +++ b/tests/unit/test_runner_lookup.cpp @@ -186,3 +186,20 @@ UTest a_file_without_a_suffix_is_no_program{"RunnerLookup/AFileWithoutASuffixIsN }}; #endif + +#if defined(_WIN32) +// R2, A3: a name with a directory and no suffix takes the system's suffix, as a bare name does; +// the file without one is not a program on Windows. +UTest a_name_with_a_directory_takes_the_systems_suffix{"RunnerLookup/ANameWithADirectoryTakesTheSystemsSuffix", [] { + auto root = std::filesystem::temp_directory_path() / "mcpp-runner-lookup-dir-suffix"; + std::filesystem::remove_all(root); + std::filesystem::create_directories(root / "tools"); + { std::ofstream o(root / "tools" / "x.cmd"); o << "@echo off\r\n"; } + { std::ofstream o(root / "tools" / "y"); o << "#!/bin/sh\n"; } + auto l = locate((root / "tools" / "x").string(), {}, ""); + expect(fatal(l.program.has_value())); + expect(eq(l.program->filename().string(), std::string("x.cmd"))); + expect(!locate((root / "tools" / "y").string(), {}, "").program.has_value()); +}}; +#endif + diff --git a/tests/unit/test_windows_command_line.cpp b/tests/unit/test_windows_command_line.cpp index a1ec746bd..985ca174f 100644 --- a/tests/unit/test_windows_command_line.cpp +++ b/tests/unit/test_windows_command_line.cpp @@ -271,3 +271,17 @@ UTest an_argument_ending_in_a_backslash_does_not_swallow_the_next{"WindowsComman + mcpp::build::command_line::quote_windows("-o"); expect(eq(argv_of(line), (std::vector{"C:\\Kits\\10\\", "-o"}))) << line; }}; + +UTest a_script_argument_with_interpreter_syntax_is_named{"WindowsCommandLine/AScriptArgumentWithInterpreterSyntaxIsNamed", [] { + // R2, A3: an argument a command interpreter would read as syntax is named, the first one; + // ordinary paths, spaces, `=` and `!` pass. + namespace cl = mcpp::build::command_line; + const std::vector plain{"C:\\Program Files\\app.exe", "--level=3", "hello world!", "a,b;c"}; + expect(eq(cl::command_script_problem(plain), std::string())); + for (std::string_view bad : {"100%", "a^b", "x&calc", "a|b", "out", "say \"hi\"", "line\nfeed"}) { + const std::vector args{"ok", std::string(bad)}; + const auto why = cl::command_script_problem(args); + expect(ne(why.find("argument 2"), std::string::npos)) << bad << " => " << why; + } +}}; +