2026.11.15.1: CI time cost — main's test build reused, the tests at profile ci, a long case divided while it pays (mtest 0.9.0), the matrix scan side by side; the built-in platform openkal-platform 0.28.0

.agents/docs/2026-11-14-ci-and-release-time-cost-design.md

@@ -226,3 +226,79 @@ Windows 把每个 `\n` 写成 `\r\n`；`write_config_value` 只认 `\n` 结尾
 四个构建同时开始（最长为 Linux aarch64 交叉构建 39 分钟），`create-release` 在四者通过后创建 release，自分派至创建共 40 分钟；v2026.11.12.1 的构建依次进行，至最后一个构建结束约 104 分钟。
 封存与发布两个作业被跳过：未带条件的作业只在其之前的全部作业（含间接的）成功时运行，未请求时被跳过的 canaries 因此跳过了二者。
 二者改为各自声明所跟随的那一个作业（mcpp#933）；v2026.11.13.1 以 `tag: v2026.11.13.1` 再次分派完成，该分派重建该 tag 的提交。
+
+## 6. 第二轮：资源复用、编排与关键路径（负责人 2026-11-15 确认 W1–W4、子用例调度与 macOS 路径；W5 更大的 runner 不考虑）
+
+### 6.1 测量（PR 运行 43602794812，墙钟 97.6 分钟）
+
+75 个作业，runner 时间 606 分钟，排队合计 265 分钟。关键路径：
+
+| 区间（分钟） | 原因 |
+|---|---|
+| 0 → 9.1 | 上一次运行取消后，`target-matrix / coverage`（`if: always()`）仍排队 8.5 分钟等 runner，运行 6 s，期间并发组被占用 |
+| 9.1 → 20.7 | 首批作业排队 11.4 分钟：组织内同时有三次全量 CI（main push、另一个 PR、本次） |
+| 20.7 → 69.3 | archive：排队 9.3 分钟，构建 39.4 分钟（1900 条边，4 核上 36 分钟） |
+| 69.3 → 97.4 | 四个分片 28.0 / 27.9 / 12.4 / 9.8 分钟 |
+
+细粒度分解：
+
+- **archive 的编译。** 本包缺省 profile 为 release，测试连同库以 `-O2` 编译。本机 ninja 日志：库与单元测试的编译器
+  时间 8988 s（库 3423、测试 5565）；以 `-O0`、无调试信息为 4183 s（库 1709、测试 2471），减少 53%。4 核上约
+  36 分钟对 17 分钟，与 CI 的 36 分钟一致。
+- **分片。** 四个分片的工作量合计 198 分钟，16 个工作线程的理想值为 12.4 分钟。分片 1 与 3 的墙钟由两个套件决定：
+  `e2e/openkal/trees` 1373 s、`e2e/openkal/sets` 1375 s，利用率 43% 与 71%。两者的子用例在一个进程中依次运行，
+  其中最长者为 trees/955 564 s、sets/963 310 s。列表（P3.4）不给出子用例，`split = "auto"` 因而无法划分它们。
+- **macOS。** 单元测试在 macOS 作业中构建并运行 31 分钟；`target-matrix / scan` 依次扫描 59 格，1762 s。
+  `Toolchain: LLVM — build mcpp (self-host)` 是本提交的 mcpp 构建自身，不是重复构建，保留。
+- **缓存配额。** 仓库的 Actions 缓存已用 9.98 GB（上限 10 GB），测试构建目录放入缓存会逐出各宿主的 home 缓存。
+
+### 6.2 决定与实现
+
+| 编号 | 机制 | 实现 |
+|---|---|---|
+| W1 | main 的测试构建被复用 | main 的 archive 阶段把构建目录（含 `.source-blobs`：每个被跟踪文件的 blob）以 zstd 打包为 artifact `mcpp-unit-build-<host>`；PR 的 archive 阶段取 main 最近一次成功运行的该 artifact，`source_mtimes.py restore` 把 blob 未变的文件设为早于任何输出的时间，变化的文件保留检出时间。用 artifact 而不用缓存，见 6.1 的配额 |
+| W2 | CI 的测试 profile | `[profile.ci]`（`-O0`、无调试信息、strip）；archive 阶段与 macOS、Windows 的单元作业以 `--profile ci` 构建测试（`mcpp-unit` 的 `profile` 输入） |
+| W3 | 按声明的用例划分长用例 | mtest 0.9.0（P4.3）：选择行 `!<id>` 排除用例及其声明的用例，`?<id>` 在用例被声明时选中、未被声明时不报告 `missing`；`program` 事件陈述 `selection: ["except", "optional"]`。mcpp 的 O10：程序陈述两者时，长于份额且存储中测得两个以上声明用例的列出用例按最内层声明用例划分，前 k−1 组以 `?` 选中，第 k 组选中该用例并以 `!` 排除前几组；用例的估计不小于其声明用例之和；存储中用例的用时按程序与标识记取（此前只按标识，`layers`、`sets` 在两个程序中同名） |
+| W4 | 取消的运行立即释放并发组 | `coverage` 作业改为 `if: ${{ !cancelled() }}`；其余工作流的作业级条件已是此形式 |
+| M1 | 矩阵扫描并行 | `scan.sh` 每格在独立目录中运行（`scan_cell`），`MATRIX_JOBS`（缺省为处理器数）格同时进行，按表的次序输出 |
+
+**未采纳。** 「列出子用例」（运行器请求时在列表模式中执行用例体）：用例体本身就是叶子用例的测试，列出即运行测试。
+以历史与 `!`/`?` 选择达到同一目的，新增与改名的子用例都只运行一次。macOS 单元测试分片：macOS runner 的并发
+是排队的来源（6.1），增加作业会加重排队；以 W2 与 W3 缩短单个作业。
+
+本地核验：W1 恢复后无改动的重建 0.4 s，改动一个接口模块的重建 88 s（冷构建 248 s）；W3 演示工程四个子用例
+8.2 s → 3.0 s，改名的子用例只在余下组中运行，旧名在 `?` 组中不报告 `missing`；单元
+`TestSchedule/ACaseIsDividedByTheCasesItDeclared`、`TestTargets` 与 mtest `suites` 通过。
+
+### 6.3 CI 上的测量（PR 运行 43626158365 与 43641874830）
+
+两次运行均在 main 尚无 `mcpp-unit-build-<host>` artifact 时进行，W1 的复用在合入后的第一次 main 运行产生 artifact
+之后才生效。
+
+| 项 | 此前（43602794812） | 本 PR | 说明 |
+|---|---|---|---|
+| archive 构建（W2） | 39.4 分钟 | 14.3 分钟（855 s、862 s） | 冷构建，`--profile ci` |
+| `target-matrix / scan (macos-arm64)`（M1） | 1762 s | 912 s | 59 格并行 |
+| 整次运行墙钟 | 97.6 分钟 | 62.6 分钟 | 排队 89 分钟（此前 265 分钟），受组织内并发影响 |
+| 分片（W3） | 28.0 / 27.9 / 12.4 / 9.8 分钟 | 24.4 / 21.9 / 20.0 / 22.0 分钟 | 见下 |
+
+**W3 的测量否定了按声明的用例无条件划分。** `e2e/openkal/sets` 整体 1375 s，划分为三组后各为 909、1109、1306 s；
+`trees` 整体 1373 s，两组为 1070、1247 s。同一用例声明的用例共享第一个用例填充的状态（下载的工具链与包、构建
+缓存），每一组在各自的 runner 上重新付出，关键路径没有缩短，runner 时间约为两倍。而且划分后测得的内层用例
+用时偏大，用例的估计随之增大，下一次会划得更细。
+
+**决定。** 划分以测量判定：各组在程序进程一行写 `divides`；存储同时记取用例最近一次整体运行的用时与最近一次
+划分中最长一组的用时；从未被划分的用例划分一次以取得测量，此后仅当最长一组不超过整体的五分之四时继续划分；
+用例的估计取整体运行的用时（`test_schedule::division_pays`，单元 `TestSchedule/ADivisionIsKeptWhileItPays`）。
+main 的存储中尚无划分的记录：合入后的第一次 main 运行把这两个套件各划分一次并记录，此后整体运行，直到记录移出 30 次运行的窗口后再测一次。
+缩短这两个套件的途径是其共享状态的复用（预填充的 home），不是划分。
+
+**并行暴露的两个缺陷**（均已修复并有单元测试）：
+
+- 项目范围的安装同时写 home 的环境（`subos/default/.xlings.json`），状态锁却取在项目目录，Windows 上与另一项目的
+  安装冲突（`images/1033`，运行 43632995462）。锁改为每个 home 一个（`XlingsState/AProjectsInstallTakesTheHomesTurn`）。
+- 两次构建同时发布同一宿主工具时共用一个临时文件名，一方在另一方改名前删除了它（macOS 扫描，运行 43636471152）。
+  发布改用 `mcpp.build.stage::stage_file`，临时文件名带随机部分（`BuildStage/WritersOfOneDestinationDoNotCollide`）。
+
+Windows 的旧代码页检查此前以缺省 profile 运行两个用例，在 `--profile ci` 构建之后会重建全部测试并超时；现以同一
+profile 运行。

.github/actions/mcpp-unit/action.yml

@@ -23,6 +23,12 @@ inputs:
     description: the test archive to run (`mcpp test --archive`); empty to build the tests here
     required: false
     default: ''
+  profile:
+    description: >
+      the profile the tests are built with when they are built here (`mcpp test --profile`);
+      empty for the package's default
+    required: false
+    default: ''
   shard:
     description: I/N, this job's shard of the tests (`mcpp test --shard`)
     required: false
@@ -103,6 +109,7 @@ runs:
       env:
         ARCHIVE: ${{ inputs.archive }}
         SHARD: ${{ inputs.shard }}
+        PROFILE: ${{ inputs.profile }}
         SINCE: ${{ inputs.since }}
         PATTERNS: ${{ inputs.patterns }}
         TIMEOUT: ${{ inputs.timeout }}
@@ -117,7 +124,12 @@ runs:
         fresh="$MCPP_FRESH"
         if command -v cygpath >/dev/null 2>&1; then fresh="$(cygpath -w "$fresh")"; fi
         export MCPP_BIN_MCPP="${MCPP_BIN_MCPP:-$fresh}"
-        if [ -n "$ARCHIVE" ]; then args+=(--archive "$ARCHIVE"); else args+=(--no-programs); fi
+        if [ -n "$ARCHIVE" ]; then
+          args+=(--archive "$ARCHIVE")
+        else
+          args+=(--no-programs)
+          [ -z "$PROFILE" ] || args+=(--profile "$PROFILE")
+        fi
         if [ -n "${MCPP_UNIT_FAILED:-}" ]; then
           args+=(--failed)
         else

.github/tools/check_windows_home_legacy.py

@@ -14,9 +14,18 @@
 assert original.count(marker) == 1, 'identify the test-only code-page setting explicitly'
 
 
+# The profile the job built the tests with (MCPP_TEST_PROFILE; empty: the
+# manifest's default). Another profile builds every test again, which outlasted
+# the timeout on the Windows row (run 43636471152).
+profile = os.environ.get('MCPP_TEST_PROFILE', '')
+
+
 def run_case(passed_case):
     # The case by its identifier (mtest's): it runs in the process of its file.
-    result = subprocess.run([os.environ['MCPP'], '--offline', 'test', '--case', passed_case], cwd=root,
+    command = [os.environ['MCPP'], '--offline', 'test', '--case', passed_case]
+    if profile:
+        command += ['--profile', profile]
+    result = subprocess.run(command, cwd=root,
                             capture_output=True, text=True, encoding='utf-8', errors='replace',
                             timeout=600)
     sys.stdout.write(result.stdout)

.github/tools/source_mtimes.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""The sources' times for an incremental build restored from a cache.
+
+A checkout writes every file with the time of the checkout, so a build directory
+restored from a cache is older than every source and ninja rebuilds all of it.
+The cache records each tracked file's blob beside the build (`record`); after a
+checkout, a file whose blob is the one recorded is given a time older than any
+output (`restore`), and a changed or new file keeps the checkout's, so exactly
+what changed is rebuilt.
+
+    source_mtimes.py record  <build-dir>
+    source_mtimes.py restore <build-dir>
+"""
+import os
+import subprocess
+import sys
+
+STAMP = ".source-blobs"
+OLD = 946684800  # 2000-01-01: older than every output a build writes
+
+
+def blobs():
+    """Each tracked file's blob; a file the working tree changes has none."""
+    git = lambda *a: subprocess.run(["git", *a], check=True, capture_output=True).stdout
+    edited = {p.decode() for p in git("ls-files", "-m", "-z").split(b"\0") if p}
+    out = git("ls-files", "-s", "-z")
+    table = {}
+    for entry in out.split(b"\0"):
+        if not entry:
+            continue
+        meta, path = entry.split(b"\t", 1)
+        if path.decode() not in edited:
+            table[path.decode()] = meta.split()[1].decode()
+    return table
+
+
+def main(argv):
+    if len(argv) != 3 or argv[1] not in ("record", "restore"):
+        print(__doc__.strip(), file=sys.stderr)
+        return 2
+    stamp = os.path.join(argv[2], STAMP)
+    if argv[1] == "record":
+        os.makedirs(argv[2], exist_ok=True)
+        with open(stamp, "w", encoding="utf-8") as f:
+            for path, blob in sorted(blobs().items()):
+                f.write(f"{blob} {path}\n")
+        return 0
+    if not os.path.isfile(stamp):
+        print(f"no {stamp}: the build directory is rebuilt whole")
+        return 0
+    recorded = {}
+    with open(stamp, encoding="utf-8") as f:
+        for line in f:
+            blob, path = line.rstrip("\n").split(" ", 1)
+            recorded[path] = blob
+    same = changed = 0
+    current = blobs()
+    for path in subprocess.run(["git", "ls-files", "-z"], check=True,
+                               capture_output=True).stdout.decode().split("\0"):
+        if not path:
+            continue
+        blob = current.get(path)
+        if blob is not None and recorded.get(path) == blob and os.path.isfile(path):
+            os.utime(path, (OLD, OLD), follow_symlinks=False)
+            same += 1
+        else:
+            changed += 1
+    print(f"{same} file(s) as recorded, {changed} changed or new")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

.github/workflows/ci-macos.yml

@@ -321,6 +321,10 @@ jobs:
         uses: ./.github/actions/mcpp-unit
         with:
           name: ${{ matrix.image }}
+          # -O0 without debug information (mcpp.toml `[profile.ci]`): the
+          # package's default is release, and its tests took twice the compiler
+          # time with it (design 2026-11-14 §6).
+          profile: ci
           history: ${{ matrix.image }}
           since: ${{ env.MCPP_TEST_SINCE }}
           patterns: ${{ (matrix.image == 'xcode-27' && inputs.set == 'pr') && 'unit e2e/basics e2e/build e2e/macos e2e/modules e2e/runtime e2e/toolchain' || '' }}

.github/workflows/ci-target-matrix.yml

@@ -408,13 +408,16 @@ jobs:
     # workflow is green. This job is the one place that reads the expected table
     # as a whole and requires the run to have covered it.
     #
-    # It runs even when a scan failed (`always()`), because "which hosts were
-    # covered" is a different question from "did they pass" — and a run where a
-    # host is missing entirely should say so in those words rather than leaving
-    # a reader to infer it from a job list.
+    # It runs even when a scan failed, because "which hosts were covered" is a
+    # different question from "did they pass" — and a run where a host is
+    # missing entirely should say so in those words rather than leaving a
+    # reader to infer it from a job list. Not in a cancelled run: `always()`
+    # queued it for a runner after the cancellation, and the concurrency group
+    # was held until it ran (run 43599371222: 8.5 minutes, the next run's
+    # start delayed by 9.1).
     name: coverage (every host the table names was scanned)
     needs: scan
-    if: always()
+    if: ${{ !cancelled() }}
     runs-on: ubuntu-24.04
     steps:
       - uses: actions/checkout@v4

.github/workflows/ci-unit.yml

@@ -97,6 +97,30 @@ jobs:
             echo "the bootstrap $("$MCPP" --version 2>/dev/null | head -1) builds the package's program into the archive"
           fi
 
+      # MAIN'S TEST BUILD, REUSED (design 2026-11-14 §6, W1). The archive of
+      # main's last successful run carries its build directory and the blob of
+      # every tracked file it was built from. A file whose blob is the same is
+      # given a time older than every output; a changed one keeps the
+      # checkout's, so only what changed and what imports it is built again.
+      # The bootstrap mcpp is pinned, so the configuration is main's; a pin
+      # that moved builds whole, as a run without the artifact does.
+      - name: Main's test build
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+          HOST: ${{ inputs.host }}
+        run: |
+          set -uo pipefail
+          run=$(gh run list -R "$GITHUB_REPOSITORY" --workflow ci.yml --branch main --status success --limit 1 \
+                  --json databaseId -q '.[0].databaseId' 2>/dev/null || true)
+          [ -n "$run" ] || { echo "no successful run of main: the tests are built whole"; exit 0; }
+          if ! gh run download "$run" -R "$GITHUB_REPOSITORY" -n "mcpp-unit-build-$HOST" -D "$RUNNER_TEMP/build" 2>/dev/null; then
+            echo "main's run $run kept no test build for $HOST: the tests are built whole"; exit 0
+          fi
+          tar --zstd -xf "$RUNNER_TEMP/build/target.tar.zst" && rm -f "$RUNNER_TEMP/build/target.tar.zst"
+          python3 .github/tools/source_mtimes.py restore target
+          echo "main's run $run: its test build restored"
+
       # EVERY TEST, ON EVERY SET. The bootstrap mcpp that builds them selects
       # by change without the rule that a test running the package's program
       # is affected by the program's sources (2026.11.13.1), so it would leave
@@ -110,7 +134,10 @@ jobs:
           export MCPP_VENDORED_XLINGS="$XLINGS_BIN"
           mkdir -p "$RUNNER_TEMP/unit"
           # shellcheck disable=SC2086 -- the flag is one word or none
-          "$MCPP" test --no-run ${{ steps.programs.outputs.flag }} --archive "$RUNNER_TEMP/unit/unit.tar.gz" 2>&1 | tee "$RUNNER_TEMP/archive.log"
+          # The profile `ci` (mcpp.toml): -O0 and no debug information. The
+          # default profile of this package is release (-O2); the tests built
+          # with it took 8988 s of compiler time, and 4183 s with `ci`.
+          "$MCPP" test --profile ci --no-run ${{ steps.programs.outputs.flag }} --archive "$RUNNER_TEMP/unit/unit.tar.gz" 2>&1 | tee "$RUNNER_TEMP/archive.log"
           n=$(sed -n 's/.*Archived \([0-9][0-9]*\) test.*/\1/p' "$RUNNER_TEMP/archive.log" | tail -1)
           [ -f "$RUNNER_TEMP/unit/unit.tar.gz" ] || n=0
           echo "archived=${n:-0}" >> "$GITHUB_OUTPUT"
@@ -146,6 +173,27 @@ jobs:
           if-no-files-found: error
           retention-days: 3
 
+      # THE TEST BUILD FOR THE NEXT RUN: main's only, so a pull request
+      # starts from the build of the branch it merges into.
+      - name: Keep the test build
+        id: keep
+        if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.archive.outputs.archived != '0'
+        shell: bash
+        run: |
+          set -euo pipefail
+          python3 .github/tools/source_mtimes.py record target
+          tar --zstd -cf "$RUNNER_TEMP/target.tar.zst" target
+          du -h "$RUNNER_TEMP/target.tar.zst"
+      - name: Upload the test build
+        if: steps.keep.outcome == 'success'
+        timeout-minutes: 10
+        uses: actions/upload-artifact@v4
+        with:
+          name: mcpp-unit-build-${{ inputs.host }}
+          path: ${{ runner.temp }}/target.tar.zst
+          compression-level: 0
+          retention-days: 7
+
       # The tests a job that runs named cases takes (e2e-cases), built here
       # beside the others; the job names the program they test.
       - name: Write the tests other jobs run
@@ -155,7 +203,7 @@ jobs:
           set -euo pipefail
           export MCPP_VENDORED_XLINGS="$XLINGS_BIN"
           # shellcheck disable=SC2086 -- the input is a list of test paths
-          "$MCPP" test ${{ inputs.tests }} --no-run ${{ steps.programs.outputs.flag }} --archive "dist/mcpp-tests.tar.gz"
+          "$MCPP" test ${{ inputs.tests }} --profile ci --no-run ${{ steps.programs.outputs.flag }} --archive "dist/mcpp-tests.tar.gz"
 
       - name: Upload the tests other jobs run
         if: inputs.tests != ''

.github/workflows/ci-windows.yml

@@ -73,6 +73,10 @@ jobs:
         uses: ./.github/actions/mcpp-unit
         with:
           name: windows
+          # -O0 without debug information (mcpp.toml `[profile.ci]`): the
+          # package's default is release, and its tests took twice the compiler
+          # time with it (design 2026-11-14 §6).
+          profile: ci
           history: windows
           mcpp: ${{ env.MCPP_SELF }}
           since: ${{ env.MCPP_TEST_SINCE }}
@@ -81,7 +85,8 @@ jobs:
         shell: bash
         run: |
           export MCPP_VENDORED_XLINGS=$(cygpath -w "$USERPROFILE/.xlings/subos/default/bin/xlings.exe")
-          MCPP="$MCPP_SELF" python .github/tools/check_windows_home_legacy.py
+          # The tests as the step above built them (profile ci).
+          MCPP="$MCPP_SELF" MCPP_TEST_PROFILE=ci python .github/tools/check_windows_home_legacy.py
 
       # WS8: this row has Visual Studio, so its answer is the "Windows with
       # usable MSVC" row of docs/01 and docs/20.