Skip to content

Commit d979828

Browse files
Ecosystem review of 2026.10.28.1: a package's key covers every module it imports; --merge names its directory after the output; finish() reached again from a cleanup returns; upgrade-lane-logs/ leaves the tree
- cache key (plan.cpp): the provider of each module a package imports is an upstream of its key, a declared edge or not. The libraries of a build program import the engine interface (mcpp), which no manifest names, and were served entries compiled against another interface ("import 'mcpp' has CRC mismatch", measured with mcpp.plugins in an ok-image build). An import a declared path runs the other way (a module generated for a dependency below its consumer, e2e 849) is not one. e2e 959 fails before and passes after; ci-upgrade gains a programs variant. - pack: archive_directory() is the one rule a package's directory takes from its archive (pack -o, --merge -o, emit xpkg); --merge -o names the merged directory after its output; emit xpkg reads .tgz too. - run: a cleanup that ends mcpp again from inside finish() returns at once instead of waiting on its own call_once. - upgrade-lane-logs/, committed with 2026.10.17.1, is untracked and ignored. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
1 parent f96f068 commit d979828

29 files changed

Lines changed: 271 additions & 215 deletions

‎.github/tools/upgrade_lane.sh‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,13 @@
1717
# revisions (llvm revision 0, gcc 16.1.0 before -r1).
1818
# orphan every payload recorded at another revision, and `c++` active at
1919
# gcc's version whose registration is gone (a reported home).
20+
# modules a module dependency the previous release compiled is served from
21+
# its cache when the spelling of its BMI has not moved.
22+
# programs the previous release builds an ok image, whose platform's build
23+
# program imports mcpp.plugins over the engine interface (`mcpp`);
24+
# this commit's mcpp builds it again over that cache, and no
25+
# library compiled against another interface is served ("CRC
26+
# mismatch").
2027
#
2128
# Asserted, every variant: the first `mcpp new && mcpp run` of the new mcpp
2229
# succeeds and downloads nothing; a second run likewise; `mcpp self doctor`
@@ -158,6 +165,12 @@ if [ "$VARIANT" = modules ]; then
158165
fi
159166
new_project p-old "$OLD"
160167
run old-run "$W/p-old" "$OLD" run || fail "the previous release cannot run a new project"
168+
case "$PLAT" in linux-x86_64) OK_TARGET=x86_64-openkal ;; *) OK_TARGET=aarch64-openkal ;; esac
169+
if [ "$VARIANT" = programs ]; then
170+
new_project k-old "$OLD"
171+
run old-programs "$W/k-old" "$OLD" build --target "$OK_TARGET" \
172+
|| fail "the previous release cannot build an ok image"
173+
fi
161174

162175
if [ "$VARIANT" = orphan ]; then
163176
"$PY" - "$REG" <<'PY' || fail "could not shape the orphan home"
@@ -238,6 +251,12 @@ if [ "$VARIANT" = modules ]; then
238251
grep -q "STAGE gcm.cache/mcpplibs.openkal/" "$W/new-modules2.log" \
239252
|| fail "the rebuilt dependency was not served from the cache"
240253
fi
254+
if [ "$VARIANT" = programs ]; then
255+
run new-programs "$W/k-old" "$NEW" build --target "$OK_TARGET" \
256+
|| fail "the ok image the previous release built does not build over its cache"
257+
grep -q "CRC mismatch" "$W/new-programs.log" \
258+
&& fail "a build program's library compiled against the previous engine interface was served"
259+
fi
241260
if [ "$VARIANT" = orphan ]; then
242261
run new-provision "$W" "$NEW" toolchain install "$TC" || fail "toolchain install could not replace the payload"
243262
grep -q "installed asset registration missing" "$W/new-provision.log" \

‎.github/workflows/ci-upgrade.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ jobs:
3030
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: snapshot }
3131
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: orphan }
3232
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: modules }
33+
- { host: linux-x86_64, runs-on: ubuntu-24.04, variant: programs }
3334
- { host: linux-aarch64, runs-on: ubuntu-24.04-arm, variant: plain }
3435
- { host: macos-arm64, runs-on: macos-15, variant: plain }
3536
- { host: windows-x86_64, runs-on: windows-latest, variant: plain }

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,3 +56,7 @@ examples/*/target/
5656

5757
# Local review reports (not part of the design record).
5858
/.agents/reviews/
59+
60+
# The logs .github/tools/upgrade_lane.sh copies beside the workspace (CI uploads
61+
# them as an artifact; a local run leaves them here).
62+
/upgrade-lane-logs/

‎CHANGELOG.md‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,20 @@
88

99
### Fixed
1010

11+
- **A package's cache key covers every package whose module it imports.**
12+
A BMI is bound to the BMIs it read, and the key folded in only the
13+
dependencies a manifest states. A package can import a module its manifest
14+
does not name: the libraries of a build program import the engine
15+
interface (`mcpp`), which the engine adds to the build program's graph.
16+
Such a package was served an entry compiled against another BMI of that
17+
module and failed with "CRC mismatch" (measured: `mcpp.plugins` in a build
18+
of an ok image, over a cache earlier builds had filled). The provider of
19+
each imported module is now an upstream of the importer's key, a declared
20+
edge or not; a module this build generates for a dependency below its
21+
consumer's root is not one (e2e 849). e2e 959 serves a package an
22+
interface it does not declare at two versions; the upgrade lane gains a
23+
`programs` variant that builds an ok image over the previous release's
24+
cache.
1125
- **A build ended by SIGTERM exits 143 however the race falls.** The watcher
1226
that takes a request to end over ends the programs mcpp runs, writes the
1327
cancelled record, runs the cleanups and then ends mcpp as asked; the main
@@ -32,7 +46,10 @@
3246
archive's name without its extension, which is what `mcpp pack` names it
3347
after, so it holds for every system, architecture and later version; an
3448
archive renamed since it was packed, or a file xim does not unpack, is
35-
refused when the entry is written.
49+
refused when the entry is written. `mcpp pack --merge -o <archive>` names
50+
the merged package's directory after its output as `mcpp pack -o` does,
51+
where it kept the first input's, so a merged package is described as
52+
every other is.
3653
- **`mcpp emit xpkg` refuses a word that is not an option.** Each archive is
3754
its own `--package`; `--package a.zip b.zip` (a shell glob's expansion)
3855
described `a.zip` alone and left `b.zip` out without a word.
@@ -45,6 +62,9 @@
4562

4663
### Changed
4764

65+
- **`upgrade-lane-logs/` is no longer in the tree.** A local run's logs were
66+
committed with 2026.10.17.1; the directory is ignored, as the lane writes
67+
it beside the workspace for CI to upload.
4868
- **SPEC-014 1.2** states what 2026.10.27.1 already does: a package's name
4969
carries its mode's suffix, a single file's launcher exits 127 on its own
5070
failure, `--merge` takes its container from `[pack] format` before `-o`, and

‎docs/90-build-from-source.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ src/
8282
8383
tests/
8484
├── unit/ 156 C++ tests, discovered by `mcpp test`
85-
└── e2e/ 645 shell scripts against a real binary
85+
└── e2e/ 646 shell scripts against a real binary
8686
```
8787

8888
## Test Organization
@@ -93,7 +93,7 @@ Two layers, and they answer different questions.
9393
discovers. They exercise a module's contract with no binary and no filesystem
9494
state.
9595

96-
**End-to-end tests** (`tests/e2e/NN_<name>.sh`, 645 files) run a real `mcpp`
96+
**End-to-end tests** (`tests/e2e/NN_<name>.sh`, 646 files) run a real `mcpp`
9797
binary against a real project. `run_all.sh` is the CI entry point. `mcpp test`
9898
does **not** run them.
9999

‎docs/specs/packaging.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,8 @@ WebAssembly 的同名文件族,ok 映像的 `bin/`),规定见 docs/10。
230230
路径**必须**是包内的规范相对路径。输出**必须**是新的路径,或一个已有的、不是输入之一的本命令写出的归档;目录输出
231231
只写新目录。
232232
容器取 `--pkg-format`,否则取 `[pack] format`,否则取 `-o` 的扩展名,否则取第一个输入的容器;`-o` 缺省为当前目录下与树同名的文件。
233+
合并后的包的目录以输出命名,与 `mcpp pack -o` 相同:归档名去掉其扩展名(`-o app-1.0-openkal.zip` 含
234+
`app-1.0-openkal/`);不带 `-o` 时取第一个输入的目录名。
233235
不进行任何构建。
234236

235237
## 5. 容器
@@ -344,4 +346,4 @@ openkal 目标的单文件:载体携带映像本身(映像不引用其他映像
344346
|---|---|---|
345347
| 1.0 | 2026-10-28 | 首版(mcpp#847):三个维度与构建腿、组合的合法性、选项的全局用法与 `--pkg-format`、`[pack]` 的新键、管线、单腿与通用包的布局、启动器、`--systems`、`mcpp-package.json`、`--merge`、压缩、单文件、包格式提供者与平台插件、签名、分发描述、`mcpp.pack` 的新字段。 |
346348
| 1.1 | 2026-10-28 | 与实现对齐:zip 的条目以 DEFLATE 压缩;`[pack] universal`;压缩的拒绝与项目缺省值的搁置;`--systems` 也用于映像的单文件;启动器的根与退出状态;单文件的载荷格式、校验与执行;映像单文件的载体内容;`mcpp-package.json` 的 `linkage` 取值与角色判定;`--merge` 的输入、选项写法与容器;签名的范围;`mcpp emit xpkg --package`;判据对应 e2e 954–957。 |
347-
| 1.2 | 2026-10-28 | 包名的模式后缀;单文件启动器自身失败的退出状态 127;`--merge` 取 `[pack] format`;`--linkage static` 改为 musl 的行;`mcpp emit xpkg --package` 的每系统程序目录、带命名空间的运行器依赖、以归档名定位的顶层目录与多余词的拒绝。 |
349+
| 1.2 | 2026-10-28 | 包名的模式后缀;单文件启动器自身失败的退出状态 127;`--merge` 取 `[pack] format`,合并后的目录以输出命名;`--linkage static` 改为 musl 的行;`mcpp emit xpkg --package` 的每系统程序目录、带命名空间的运行器依赖、以归档名定位的顶层目录与多余词的拒绝。 |

‎docs/zh/90-build-from-source.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ src/
8484
8585
tests/
8686
├── unit/ 156 C++ tests, discovered by `mcpp test`
87-
└── e2e/ 645 shell scripts against a real binary
87+
└── e2e/ 646 shell scripts against a real binary
8888
```
8989

9090
## 测试组织
@@ -94,7 +94,7 @@ tests/
9494
**单元测试**(`tests/unit/`,156 个文件)是 `mcpp test` 发现的 C++
9595
程序。它们检验一个模块的契约,不需要二进制、也不需要文件系统状态。
9696

97-
**端到端测试**(`tests/e2e/NN_<name>.sh`,645 个文件)拿真实的 `mcpp`
97+
**端到端测试**(`tests/e2e/NN_<name>.sh`,646 个文件)拿真实的 `mcpp`
9898
二进制跑真实工程。`run_all.sh` 是 CI 的入口。`mcpp test` **不**运行它们。
9999

100100
```bash

‎src/build/cache_key.cppm‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,10 @@
2525
// D identity index name, package FQN, version
2626
// E own config features, cflags/cxxflags/asmflags/ldflags, per-glob flags,
2727
// defines, generated files, include dirs, source list
28-
// F upstream the KEY of each direct dependency, recursively (Merkle)
28+
// F upstream the KEY of each direct dependency and of each package whose
29+
// module the package imports, a declared edge or not (the
30+
// engine interface a build program's libraries import),
31+
// recursively (Merkle)
2932
// G epoch kCacheEpoch — cache-format compatibility, decoupled from the
3033
// mcpp release number
3134
//

‎src/build/prepare/plan.cpp‎

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1643,6 +1643,10 @@ static std::expected<void, std::string> step13_dependency_cache(PrepareState& st
16431643
// Sources belonging to each package, package-root-relative and sorted.
16441644
std::vector<std::vector<std::string>> pkgSources(state.packages.size());
16451645
std::vector<char> pkgProvidesModule(state.packages.size(), 0);
1646+
// The package each unit is filed under, and the package that provides
1647+
// each module: the import edges below read both.
1648+
std::vector<std::size_t> unitPackage;
1649+
std::map<std::string, std::size_t> moduleProvider;
16461650
for (auto& cu : ctx.plan.compileUnits) {
16471651
// Longest matching root wins. Package roots can nest — a workspace
16481652
// member lives under the workspace root — and taking the first match
@@ -1665,13 +1669,33 @@ static std::expected<void, std::string> step13_dependency_cache(PrepareState& st
16651669
best = p; bestLen = len; bestRel = std::move(rels);
16661670
}
16671671
}
1672+
unitPackage.push_back(best);
16681673
if (best != state.packages.size()) {
1669-
if (!cu.providesModule.empty()) pkgProvidesModule[best] = 1;
1674+
if (!cu.providesModule.empty()) {
1675+
pkgProvidesModule[best] = 1;
1676+
moduleProvider.emplace(cu.providesModule, best);
1677+
}
16701678
pkgSources[best].push_back(std::move(bestRel));
16711679
}
16721680
}
16731681
for (auto& v : pkgSources) std::ranges::sort(v);
16741682

1683+
// A package whose unit imports another package's module reads that
1684+
// package's BMI, whether a manifest states the edge or not: the engine
1685+
// interface (`mcpp`) the libraries of a build program import is named
1686+
// by none of theirs. Its key is an upstream as a declared
1687+
// dependency's is; without it an entry compiled against another mcpp
1688+
// version's interface was served and failed with "import 'mcpp' has
1689+
// CRC mismatch" (2026.10.27.1 -> 2026.10.28.1, mcpp.plugins).
1690+
std::vector<std::set<std::size_t>> importedFrom(state.packages.size());
1691+
for (std::size_t u = 0; u < unitPackage.size(); ++u) {
1692+
const auto p = unitPackage[u];
1693+
if (p == state.packages.size()) continue;
1694+
for (auto const& m : ctx.plan.compileUnits[u].imports)
1695+
if (auto it = moduleProvider.find(m); it != moduleProvider.end() && it->second != p)
1696+
importedFrom[p].insert(it->second);
1697+
}
1698+
16751699
std::vector<std::string> pkgKeys(state.packages.size());
16761700
std::vector<nlohmann::json> pkgInputs(state.packages.size(),
16771701
nlohmann::json::object());
@@ -1701,6 +1725,19 @@ static std::expected<void, std::string> step13_dependency_cache(PrepareState& st
17011725
for (auto& e : state.dependencyEdges)
17021726
if (e.consumerPackageIndex < pkgDeps.size())
17031727
pkgDeps[e.consumerPackageIndex].push_back(e.dependencyPackageIndex);
1728+
// An import a declared edge already covers adds nothing. One that a
1729+
// declared path runs the other way is a module this build generates
1730+
// for the importer below its consumer's root, filed under the
1731+
// consumer by its path (e2e 849): not an upstream, and left out
1732+
// rather than closing a cycle.
1733+
for (std::size_t p = 0; p < importedFrom.size(); ++p)
1734+
std::erase_if(importedFrom[p], [&](std::size_t q) {
1735+
if (std::ranges::find(pkgDeps[p], q) != pkgDeps[p].end()) return true;
1736+
const auto above = mcpp::graph::closure(pkgDeps, std::array{q}, /*include_roots=*/false);
1737+
return std::ranges::find(above, p) != above.end();
1738+
});
1739+
for (std::size_t p = 0; p < importedFrom.size(); ++p)
1740+
pkgDeps[p].insert(pkgDeps[p].end(), importedFrom[p].begin(), importedFrom[p].end());
17041741
auto keyOrder = mcpp::graph::topological_order(pkgDeps);
17051742
if (!keyOrder) {
17061743
keyCycleError = std::format(
@@ -1760,6 +1797,10 @@ static std::expected<void, std::string> step13_dependency_cache(PrepareState& st
17601797
if (localTaint[e.dependencyPackageIndex]) localTaint[idx] = 1;
17611798
for (auto& f : e.requestedFeatures) pa.features.push_back(f);
17621799
}
1800+
for (auto q : importedFrom[idx]) {
1801+
if (!pkgKeys[q].empty()) pa.upstreamKeys.push_back(pkgKeys[q]);
1802+
if (localTaint[q]) localTaint[idx] = 1;
1803+
}
17631804
std::ranges::sort(pa.upstreamKeys);
17641805
pa.upstreamKeys.erase(std::unique(pa.upstreamKeys.begin(),
17651806
pa.upstreamKeys.end()),

‎src/pack/distribution.cppm‎

Lines changed: 32 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,18 @@ std::string_view launcher_main();
140140

141141
// ─── the distribution description (SPEC-014 §8) ───────────────────────────
142142

143+
// The suffixes of the archives a package is written as, the longer of two
144+
// that end alike first.
145+
inline constexpr std::array<std::string_view, 6> kArchiveSuffixes{ ".tar.gz", ".tgz", ".tar.zst", ".tar.xz",
146+
".tar", ".zip" };
147+
148+
// The name of a package's one directory: its archive's name without the
149+
// suffix (`app-1.0-openkal.zip` holds `app-1.0-openkal/`), and a name with no
150+
// archive's suffix as written (`-o app-1.0-openkal` with `--pkg-format dir`;
151+
// a stem would cut at the version's last dot). `mcpp pack -o` and `--merge -o`
152+
// name the directory so, and an index entry finds it so.
153+
std::string archive_directory(const std::filesystem::path& archive);
154+
143155
// A package written by `mcpp pack`, as an index entry reads it.
144156
struct Asset {
145157
std::filesystem::path file;
@@ -1462,6 +1474,13 @@ std::string lua_list(const std::vector<std::string>& v) {
14621474

14631475
} // namespace
14641476

1477+
std::string archive_directory(const fs::path& archive) {
1478+
const auto name = archive.filename().string();
1479+
for (auto suffix : kArchiveSuffixes)
1480+
if (name.ends_with(suffix)) return name.substr(0, name.size() - suffix.size());
1481+
return name;
1482+
}
1483+
14651484
std::expected<std::string, std::string> xpkg_descriptor(const std::vector<Asset>& assets,
14661485
std::string_view urlPrefix) {
14671486
if (assets.empty()) return std::unexpected(std::string("no package to describe"));
@@ -1578,23 +1597,22 @@ std::expected<std::string, std::string> xpkg_descriptor(const std::vector<Asset>
15781597
// names the archive's one top-level directory after the archive; the
15791598
// install takes the directory from that file's name, which holds for
15801599
// every system, architecture and later version of the entry.
1581-
constexpr std::array<std::string_view, 5> kArchives{ ".tar.gz", ".tar.zst", ".tar.xz", ".tar", ".zip" };
1582-
std::array<bool, kArchives.size()> used{};
1600+
std::array<bool, kArchiveSuffixes.size()> used{};
15831601
for (auto const& a : assets) {
15841602
const auto file = a.file.filename().string();
1585-
const auto ext = std::ranges::find_if(kArchives, [&](std::string_view e) { return file.ends_with(e); });
1586-
if (ext == kArchives.end())
1587-
return std::unexpected(std::format("{} is not an archive xim unpacks (.tar.gz, .tar.zst, .tar.xz, "
1588-
".tar, .zip)", file));
1589-
if (file.substr(0, file.size() - ext->size()) != a.top)
1603+
const auto ext = std::ranges::find_if(kArchiveSuffixes, [&](std::string_view e) { return file.ends_with(e); });
1604+
if (ext == kArchiveSuffixes.end())
1605+
return std::unexpected(std::format("{} is not an archive xim unpacks (.tar.gz, .tgz, .tar.zst, "
1606+
".tar.xz, .tar, .zip)", file));
1607+
if (archive_directory(a.file) != a.top)
15901608
return std::unexpected(std::format(
15911609
"{} holds the directory {}/: an entry installs the directory named after the archive, as "
15921610
"mcpp pack writes it (name the archive {}{})", file, a.top, a.top, *ext));
1593-
used[static_cast<std::size_t>(ext - kArchives.begin())] = true;
1611+
used[static_cast<std::size_t>(ext - kArchiveSuffixes.begin())] = true;
15941612
}
15951613
std::string exts;
1596-
for (std::size_t i = 0; i < kArchives.size(); ++i)
1597-
if (used[i]) exts += std::format("{}\"{}\"", exts.empty() ? "" : ", ", kArchives[i]);
1614+
for (std::size_t i = 0; i < kArchiveSuffixes.size(); ++i)
1615+
if (used[i]) exts += std::format("{}\"{}\"", exts.empty() ? "" : ", ", kArchiveSuffixes[i]);
15981616
return std::format(
15991617
"-- Generated by `mcpp emit xpkg --package` from the packages' mcpp-package.json\n"
16001618
"-- (SPEC-014 §8).\n"
@@ -1702,7 +1720,10 @@ std::expected<fs::path, std::string> merge(const MergeRequest& req) {
17021720
differ("linkage", first.linkage, p.linkage), differ("mode", first.mode, p.mode) })
17031721
if (r) return std::unexpected(*r);
17041722
}
1705-
const auto wrapper = trees.front().first.filename();
1723+
// The merged package's directory: named after the output, as `mcpp pack
1724+
// -o` names it, else the first input's.
1725+
const fs::path wrapper = req.output.empty() ? trees.front().first.filename()
1726+
: fs::path(archive_directory(req.output));
17061727

17071728
// The container: as asked, else as the output's name says, else as the
17081729
// first input was written.

0 commit comments

Comments
 (0)