0.6.0: the mcpp test spec's session, case output and checks; a program only a runner starts is listed through it

.github/versions.env

@@ -1,4 +1,4 @@
 # The versions CI builds and tests with; .github/actions/setup-mcpp exports them.
-MCPP_VERSION=2026.11.8.1
+MCPP_VERSION=2026.11.10.1
 # The xlings that mcpp release bundles.
 XLINGS_VERSION=v2026.10.26.1

CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## [0.6.0] - 2026-11-10
+
+The mtest protocol's normative text is the mcpp test spec (mcpp `docs/specs/test.md`, SPEC-017 v2.0),
+which keeps its clause numbers; `spec/protocol.md` lists what each part of mtest implements of it.
+
+- **A test session** (spec O9): `mtest serve` answers `test/plan`, `test/run`, `test/add`,
+  `test/cancel` and `test/status`, and states `test/build`, `test/case`, `test/program` and
+  `test/run` as they happen. A run is answered at once and its programs run on the session's
+  workers beside other runs; a case added joins the run in progress; cancelling ends the process and
+  everything it started, and the cases that had not begun run in another process. The session builds
+  nothing: it starts each program from the launch mcpp gives (`mcpp test --no-run`, spec B5).
+- **A case's own output** (spec O8). Under a runner the library keeps what each case writes, on each
+  stream, states it as `output` events and writes it on to the terminal; a case declared inside
+  another keeps its own.
+- **A case's checks** (spec O3): its `end` states the checks that held and failed (`checks`); a
+  failure states its statement (`source`) and whether it ended the case (`fatal`).
+- **A failing case reads as a person reads it.** Under its line, each failure: its place, the
+  statement, the two sides or that it did not hold, the message the check added, and whether the case
+  went on; the line counts the checks that failed, and a case that declared cases counts them.
+- The library's entry states the capabilities it implements and its session (`test-capabilities`,
+  `test-session`; mcpp 2026.11.10.1 reads them, an earlier mcpp ignores them).
+- The engine and the conformance suite ask for mcpplibs.os `^0.4.1`, as the library does, and build
+  `<arch>-openkal` over openkal-platform 0.25.0.
+- **A program only a runner starts is listed through that runner.** The listing pass (P3) starts a
+  program as its run does: under `--run-under` (and a `[trace.*]` tool's `run_under`), with the
+  run's `--env` variables. A program of another system -- started on a virtual machine of its system
+  or under an emulator -- was started directly to list its cases and failed with "did not start";
+  its cases could not be run through mtest from the host that built it.
+
 ## [0.5.2] - 2026-11-09
 
 - **A copy of a test program that does something else is an object** (`child`). A test that starts

README.md

@@ -38,7 +38,7 @@ int main() {
 
 ```toml
 [dev-dependencies]
-mcpplibs.mtest = "0.5.2"
+mcpplibs.mtest = "0.6.0"
 ```
 
 A failure names its place and both sides (`expected 2, actual 3`). Under a runner the same program

README.zh-CN.md

@@ -37,7 +37,7 @@ int main() {
 
 ```toml
 [dev-dependencies]
-mcpplibs.mtest = "0.5.2"
+mcpplibs.mtest = "0.6.0"
 ```
 
 失败时给出位置与两侧的值（`expected 2, actual 3`）。在运行器之下，同一程序按协议报告；单独运行时每个用例输出一行。

adapters/gtest/mcpp.toml

@@ -13,7 +13,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "gtest"
-version     = "0.5.2"
+version     = "0.6.0"
 description = "GoogleTest speaking the mtest protocol: a test main that lists, selects and reports GoogleTest's cases"
 license     = "Apache-2.0"
 

adapters/gtest/src/protocol.cpp

@@ -128,7 +128,7 @@ int mcpplibs_mtest_gtest_main(int argc, char** argv) {
                   {"protocol", 1},
                   {"level", 1},
                   {"framework", "googletest"},
-                  {"version", "0.5.2"},
+                  {"version", "0.6.0"},
                   {"isolation", {"process", "case"}},
                   {"guide", "https://github.com/mcpp-community/mtest/blob/main/adapters/gtest/docs/"
                             "writing-tests.md"}});

capi/mcpp.toml

@@ -4,7 +4,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "capi"
-version     = "0.5.2"
+version     = "0.6.0"
 description = "mtest's C ABI: the protocol's events for any language with a C FFI"
 license     = "Apache-2.0"
 

capi/src/mtest.cpp

@@ -116,7 +116,7 @@ void mtest_init(const char* framework, int level) {
              {"protocol", 1},
              {"level", std::clamp(level, 1, 3)},
              {"framework", text(framework)},
-             {"version", "0.5.2"},
+             {"version", "0.6.0"},
              {"isolation", {"process", "case"}}});
 }
 

conformance/mcpp.toml

@@ -8,7 +8,7 @@
 [package]
 namespace   = "mcpplibs.mtest"
 name        = "conformance"
-version     = "0.5.2"
+version     = "0.6.0"
 description = "The conformance suite of the mtest protocol, for every implementation of its program side"
 license     = "Apache-2.0"
 
@@ -33,5 +33,5 @@ main = "src/fixture.cpp"
 
 [dependencies]
 mcpplibs.mtest = { path = "../lib" }
-mcpplibs.os    = "0.4.1"
+mcpplibs.os    = "^0.4.1"
 nlohmann.json  = "3.12.0"

docs/commands.md

@@ -36,7 +36,7 @@ nothing; a built program's own listing is the authority (`manifest/src/manifest.
 
 | Command | Does |
 |---|---|
-| `run` | runs the selection; `--jobs N`, `--seed N`, `--fail-fast`, `--retries N`, `--isolate` (a process per case), `--timeout 5m`, `--env K=V`, `--run-under CMD`, `--monitor LEVEL` (P10), `--group G` (runs reported together); `--junit`, `--html`, `--report-json FILE`, `--md FILE`; `--coverage` measures the coverage map. A program's cases are divided among workers that would otherwise idle (`[run] split`), and programs run together within the machine's memory (`[run] memory`) |
+| `run` | runs the selection; `--jobs N`, `--seed N`, `--fail-fast`, `--retries N`, `--isolate` (a process per case), `--timeout 5m`, `--env K=V`, `--run-under CMD` (every start of a program, its listing included: a runner such as an emulator or a machine of the program's system), `--monitor LEVEL` (P10), `--group G` (runs reported together); `--junit`, `--html`, `--report-json FILE`, `--md FILE`; `--coverage` measures the coverage map. A program's cases are divided among workers that would otherwise idle (`[run] split`), and programs run together within the machine's memory (`[run] memory`) |
 | `rerun` | the cases that failed in the last run |
 | `smoke` | the `smoke` set |
 | `golden` | a run with `MTEST_GOLDEN=update`: golden files are written, not compared |

docs/service.md

@@ -12,3 +12,35 @@
 
 Each request runs on its own thread, so `status` and `cancel` are answered while a run runs. An MCP
 client is configured with the command `mtest serve` in the project's directory.
+
+## The test session (0.6.0)
+
+`mtest serve` is also the test session of the mcpp test spec (mcpp `docs/specs/test.md`, O9): a client
+that holds it runs an mcpp project's cases as a person asks for them, adds cases to a run in progress
+and cancels a run, a program or a case. The library's entry states it (`test-session = ["mtest",
+"serve"]`), and `mcpp test --guide` reports it.
+
+| Method | Params | Result |
+|---|---|---|
+| `test/plan` | `project`, optional `filter`, `options` | `programs`: the records of `mcpp test --cases` |
+| `test/run` | `project`, `selection` (`[{program, cases}]`; empty: every program), `options` | `run`, at once |
+| `test/add` | `run`, `selection` | `run`: the cases join the run in progress |
+| `test/cancel` | `run`, optional `program` and `case` | what was cancelled |
+| `test/status` | optional `run` | each run's status, counts and cases |
+| `shutdown` | | every run cancelled |
+
+`options`: `mcpp` (a program, or a program and its first arguments; else `MTEST_MCPP`, else `mcpp`),
+`args` (more arguments of `mcpp test`: `--profile`, `--target`), `isolation` (`case`: a process per
+case).
+
+- **Notifications.** `test/build` (`begin`, then `end` with the programs built), `test/case` (`begin`,
+  `output`, `failure`, `end`, with the fields of the program's events), `test/program` (a process's
+  status, exit and time) and `test/run` (the run's status and counts).
+- **Builds are mcpp's.** The session asks mcpp for the programs it runs (`mcpp test --no-run
+  --message-format json`, mcpp 2026.11.10.1 or later), which builds them when they need it, and starts
+  each from the launch the record carries. One build runs at a time; the processes share the
+  session's workers.
+- **Cancelling** ends the process and everything it started. A case that had begun ends `cancelled`;
+  the cases its process was to run and that had not begun run in another process, unless the run or
+  the program was cancelled. A run of cancelled cases and no failure ends `cancelled`.
+- **The end of the input** lets the runs in progress end; `shutdown` cancels them.

engine/mcpp.toml

@@ -10,7 +10,7 @@
 [package]
 namespace   = "mtest"
 name        = "mtest"
-version     = "0.5.2"
+version     = "0.6.0"
 description = "The mtest engine: run, schedule, store, explain and analyse tests of any language"
 license     = "Apache-2.0"
 authors     = ["mcpp-community"]
@@ -30,7 +30,7 @@ main = "src/main.cpp"
 
 [dependencies]
 mcpplibs.mtest.manifest = { path = "../manifest" }   # the static case manifest
-mcpplibs.os         = "0.4.1"
+mcpplibs.os         = "^0.4.1"
 mcpplibs.cmdline    = "0.0.2"
 nlohmann.json       = "3.12.0"
 marzer.tomlplusplus = "3.4.0"

engine/src/commands.cpp

@@ -121,6 +121,11 @@ std::expected<selection_request, answer> selection_of(const context& ctx, const
     r.exclude_tags = strings(p, "exclude_tag");
     r.exclude_programs = strings(p, "exclude_program");
     r.failed_only = p.value("failed", false);
+    r.run_under = strings(p, "run_under");
+    if (auto e = p.find("env"); e != p.end() && e->is_object())
+        for (auto& item_ : e->items())
+            if (item_.value().is_string())
+                r.env.emplace_back(item_.key(), item_.value().get<std::string>());
     if (auto since = text_of(p, "since"); !since.empty()) {
         auto paths = changed_since(ctx.cfg.root, since);
         if (!paths)
@@ -1829,7 +1834,7 @@ json catalog() {
         plain({{"out", {{"type", "string"}}},
                {"argv", {{"type", "array"}, {"items", {{"type", "string"}}}}}}),
         true);
-    return {{"schema", "mtest.catalog/1"}, {"version", "0.5.2"}, {"tools", tools}};
+    return {{"schema", "mtest.catalog/1"}, {"version", "0.6.0"}, {"tools", tools}};
 }
 
 }  // namespace mcpplibs::mtest::engine

engine/src/exec.cpp

@@ -189,7 +189,9 @@ report_reading read_report(std::string_view text, std::string_view program) {
 
 std::expected<std::vector<case_info>, std::string>
 list_program(const program_spec& spec, const std::filesystem::path& work,
-             const std::filesystem::path& root, std::vector<component_info>* components) {
+             const std::filesystem::path& root, std::vector<component_info>* components,
+             const std::vector<std::string>& run_under,
+             const std::vector<std::pair<std::string, std::string>>& env) {
     if (detect_level(spec) < 1) {
         case_info c;
         c.id = spec.name;
@@ -203,14 +205,19 @@ list_program(const program_spec& spec, const std::filesystem::path& work,
     const auto report = work / "list.ndjson";
     std::error_code ec;
     std::filesystem::remove(report, ec);
-    const auto argv = with_loader(spec.argv);
+    // Started as its run starts it: under the run's prefix, with the run's variables.
+    auto argv = run_under;
+    for (auto& a : with_loader(spec.argv))
+        argv.push_back(std::move(a));
     os::process::spawn_options o;
     o.program = resolve_program(argv.front());
     o.args.assign(argv.begin() + 1, argv.end());
     o.cwd = spec.cwd.empty() ? root : (root / os::fs::from_utf8(spec.cwd));
     o.in = os::process::stdio::null;
     for (const auto& [k, v] : spec.env)
         o.env.emplace_back(k, v);
+    for (const auto& [k, v] : env)
+        o.env.emplace_back(k, v);
     o.env.emplace_back("MTEST_PROTOCOL", "1");
     o.env.emplace_back("MTEST_MODE", "list");
     o.env.emplace_back("MTEST_REPORT", os::fs::utf8(report));
@@ -220,7 +227,7 @@ list_program(const program_spec& spec, const std::filesystem::path& work,
     auto done = os::process::run(std::move(o), limits);
     if (!done)
         return std::unexpected(
-            std::format("{} did not start: {}", spec.argv.front(), done.error().message()));
+            std::format("{} did not start: {}", argv.front(), done.error().message()));
     auto text = os::fs::read(report);
     auto r = read_report(text ? std::string_view(*text) : std::string_view{}, spec.name);
     if (r.level == 0) {

engine/src/exec.cppm

@@ -34,7 +34,9 @@ struct run_outcome {
 // Lists the cases of a program at level 1 or more (P3); an L0 program has one case, its own name.
 std::expected<std::vector<case_info>, std::string>
 list_program(const program_spec& spec, const std::filesystem::path& work,
-             const std::filesystem::path& root, std::vector<component_info>* components = nullptr);
+             const std::filesystem::path& root, std::vector<component_info>* components = nullptr,
+             const std::vector<std::string>& run_under = {},
+             const std::vector<std::pair<std::string, std::string>>& env = {});
 
 // `mtest measure --out FILE -- ARGV`: runs ARGV with this process's streams, writes what it used
 // (its children's account) to FILE as JSON, and ends with its status.

engine/src/main.cpp

@@ -98,7 +98,7 @@ const json* tool_named(const json& tools, std::string_view name) {
 
 cmdline::App application(const json& tools) {
     auto app = cmdline::App("mtest")
-                   .version("0.5.2")
+                   .version("0.6.0")
                    .description("run tests of any language, and ask about their runs")
                    .option(cmdline::Option("json").global().help("print the answer's data"));
     for (const auto& t : tools)
@@ -253,7 +253,7 @@ int main(int argc, char** argv) {
     const std::string name(parsed->subcommand_name());
     const auto& sub = parsed->subcommand()->get();
     if (name == "version") {
-        std::println("mtest 0.5.2");
+        std::println("mtest 0.6.0");
         return 0;
     }
     const auto cwd = std::filesystem::current_path();

engine/src/select.cpp

@@ -561,7 +561,7 @@ std::expected<plan, std::string> make_plan(const config& c, const store& s,
         auto listed =
             list_program(p,
                          s.dir / "work" / "list" / std::to_string(std::hash<std::string>{}(p.name)),
-                         c.root, &components);
+                         c.root, &components, r.run_under, r.env);
         if (!listed)
             return std::unexpected(listed.error());
         for (const auto& ci : components) {

engine/src/select.cppm

@@ -32,6 +32,11 @@ struct selection_request {
     // The change could not be read (a shallow clone without the base): a set that selects by
     // change selects every case, and says why. Empty: nothing went wrong.
     std::string unknown_change;
+    // How a program is started (`--run-under`, `--env`): its listing (P3) starts it as its run
+    // does, so that a program only a runner can start (an emulator, a machine of its system) is
+    // listed through that runner.
+    std::vector<std::string> run_under;
+    std::vector<std::pair<std::string, std::string>> env;
 };
 
 struct unit_case {

engine/src/service.cpp

@@ -3,6 +3,7 @@ module mcpplibs.mtest.engine.service;
 import std;
 import mcpplibs.mtest.engine.model;
 import mcpplibs.mtest.engine.commands;
+import mcpplibs.mtest.engine.session;
 
 namespace mcpplibs::mtest::engine {
 
@@ -25,6 +26,17 @@ json error_reply(const json& id, int code, std::string message) {
 }  // namespace
 
 int serve(const std::filesystem::path& cwd) {
+    // The test session (mcpp test spec O9), made on its first method.
+    std::unique_ptr<session> tests;
+    std::mutex tests_lock;
+    auto the_session = [&]() -> session& {
+        std::lock_guard g(tests_lock);
+        if (!tests)
+            tests = std::make_unique<session>([](std::string_view method, const json& params) {
+                send({{"jsonrpc", "2.0"}, {"method", std::string(method)}, {"params", params}});
+            });
+        return *tests;
+    };
     std::vector<std::jthread> requests;
     for (std::string line; std::getline(std::cin, line);) {
         if (line.empty() || line == "\r")
@@ -44,19 +56,45 @@ int serve(const std::filesystem::path& cwd) {
                   {"result",
                    {{"protocolVersion", params.value("protocolVersion", std::string("2025-06-18"))},
                     {"capabilities", {{"tools", {{"listChanged", false}}}}},
-                    {"serverInfo", {{"name", "mtest"}, {"version", "0.5.2"}}},
+                    {"serverInfo", {{"name", "mtest"}, {"version", "0.6.0"}}},
                     {"instructions",
                      "Run tests and ask about runs: run, plan, explain, history, analyze, graph, "
-                     "gate, ci. "
+                     "gate, ci. A test session (test/plan, test/run, test/add, test/cancel, "
+                     "test/status) runs an mcpp project's cases as they are asked for. "
                      "Each result carries a `conclusion`; a failure's `again` repeats it."}}}});
             continue;
         }
         if (method == "notifications/initialized" || method == "initialized")
             continue;
         if (method == "shutdown") {
+            {
+                std::lock_guard g(tests_lock);
+                if (tests)
+                    tests->shutdown();
+            }
             send({{"jsonrpc", "2.0"}, {"id", id}, {"result", nullptr}});
             continue;
         }
+        if (method.starts_with("test/")) {
+            const auto verb = method.substr(5);
+            requests.emplace_back([&, id, verb, params, notification] {
+                auto& t = the_session();
+                json r = verb == "plan"     ? t.plan(params)
+                         : verb == "run"    ? t.run(params)
+                         : verb == "add"    ? t.add(params)
+                         : verb == "cancel" ? t.cancel(params)
+                         : verb == "status"
+                             ? t.status(params)
+                             : json{{"error", std::format("no method test/{}", verb)}};
+                if (notification)
+                    return;
+                if (r.contains("error"))
+                    send(error_reply(id, -32602, r["error"].get<std::string>()));
+                else
+                    send({{"jsonrpc", "2.0"}, {"id", id}, {"result", r}});
+            });
+            continue;
+        }
         if (method == "exit")
             break;
         if (method == "tools/list") {
@@ -96,6 +134,15 @@ int serve(const std::filesystem::path& cwd) {
             }
         });
     }
+    // The input ended: what was asked for runs to its end (`shutdown` is how a client cancels).
+    requests.clear();
+    {
+        std::lock_guard g(tests_lock);
+        if (tests) {
+            tests->drain();
+            tests->shutdown();
+        }
+    }
     return 0;
 }