You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 899013f
Browse filesBrowse the repository at this point in the historyBrowse files
1.5.0: tests by the mcpp test spec, test sessions, tests beside each other (#11)
* Tests by the mcpp test spec: no framework is known, tests run beside each other
- The backend knows no test framework (mcpp test spec, docs/specs/test.md): a program's cases are what it lists (B3), one listing of the project kept until it changes; cases run by identifier (P4) whatever framework wrote them, and a program of no protocol runs whole. spec/test-frameworks.json, the framework table, its static scanning and report readers, the setting mcpp.test.macros and the dependency on mtest's static manifest are removed; scan reads where a source starts and nothing else.
- A case's outcome carries its checks (held, failed), each failure's source and fatal (O3) and its own output (O8); VS Code shows one message per failed check, a diff where it states both sides, the statement and whether the case went on, the case's own output, and the failed checks on the case's line.
- A test runs at once in a lane of its own: two tests of a project, or a test beside a build, do not wait for each other, and cancelling one leaves the other (mcpp releases its build directory before tests run, mcpp#915). mcpp.runtime.concurrency holds only clean and pack.
- One set of arguments for every test command (profile, target, mcpp.task.testArgs), so a listing and a run plan one configuration; discovery after events is debounced.
- Debugging some cases of a protocol program states the protocol's variables and the selection.
* Cases run through the test session a framework states (mcpp test spec O9)
- Where a project's test framework states a test session (test-session, B6) and its mcpp gives launches (test-launch-records, B5), test/runCases runs the cases through it: the session is started once per project and spoken to in JSON-RPC on its standard streams; a run is a job of its own whose cases arrive as test/caseEvent with their checks, failures and output, and whose end is job/finished. Cancelling the job cancels the run.
- A run is known before the session's first notification of it: its answer is taken on the session's reading thread.
- The session ends with its input, so it ends with the backend; a lifetime bound by the system is not asked, as on Linux it follows the thread that started it.
- The setting mcpp.test.session (auto, off) runs every case through mcpp test when off.
- docs/architecture.md states a test run by the mcpp test spec.
* Cases stay as their program last listed them; CLion and Neovim by the test spec
- test/cases answers the project's last listing without `list`, and lists again with it only when the project changed since: the cases shown stay until a listing replaces them, and writing a file builds nothing.
- CLion runs some cases of a test by their identifiers whatever framework wrote them: the run configuration names a test and its cases (its framework field is removed), and the gutter places a case's run on the first element of the line its program lists it at, in the file it names. Cases are listed when first shown and after a build or a test, one listing at a time per program.
- Neovim lists the cases when a file is read, when the session starts and after a run; writing a file shows the last listing.
- protocol.schema.json states test/cases, test/runCases, test/caseResult and test/caseEvent by the mcpp test spec: a case's checks (O3), its output (O8), a failure's source and fatal; source/entries and launch parameters name no framework.
- mcpp.lock no longer holds mcpplibs.mtest.manifest, which no target depends on.
* 1.5.0: tests by the mcpp test spec, test sessions, tests beside each other
* docs/settings.md: mcpp.test.session in place of mcpp.test.macros
Copy file name to clipboardExpand all lines: .agents/docs/2026-11-10-tests-from-mcpp.md
+30-1Lines changed: 30 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Tests as mcpp states them: programs, parts, cases, live results, metrics and history
2
2
3
-
Status, 2026-11-10: implemented in mcpp-ide 1.3.0 (mcpp-ide#8), with the client side of mcpp#917; the backend's own tests are mtest's.
3
+
Status, 2026-11-10: implemented in mcpp-ide 1.3.0 (mcpp-ide#8), with the client side of mcpp#917; the backend's own tests are mtest's. mcpp-ide 1.5.0 reads tests by the mcpp test spec (last section): the static reading of cases with mtest's manifest is withdrawn.
4
4
5
5
## Scope and principle
6
6
@@ -183,3 +183,32 @@ insights are empty and say so.
183
183
- Neovim: the smoke test covers live results, the virtual text and `:Mcpp test.insights`.
184
184
- CLion: the comparison failure and the nested suites, in the plugin's unit tests.
185
185
- The insights page is rendered by a Chromium test, as the graph page is.
186
+
187
+
## 1.5.0: the mcpp test spec
188
+
189
+
mcpp 2026.11.10.1 states what an editor may rely on of any test framework as the mcpp test spec
190
+
(mcpp docs/specs/test.md, SPEC-017 v2.0): a base every framework implements (B1–B6) and optional
191
+
capabilities a framework states of itself (O1–O9). From 1.5.0 the backend reads tests by that spec and
192
+
knows no framework by name.
193
+
194
+
-**Cases.** A program's cases are its listing (B3, `mcpp test --cases`), one listing per project. The
195
+
last listing is kept: `test/cases` with `list` lists again when the project changed since, and
196
+
without it answers the kept listing. Writing a file builds nothing; the cases shown stay until a
197
+
listing after a build or a test replaces them. No case is read from a source.
198
+
-**Selection.** Cases run by identifier (P4, `--case`). A configuration names a test and its cases;
199
+
no framework is named.
200
+
-**Outcomes.** A case carries its checks (O3: held, failed; each failure's statement and whether the
201
+
case went on) and its own output (O8).
202
+
-**Frameworks.** What a framework offers beyond the base is what `mcpp test --guide` states of it
203
+
(B6): its name, shown beside a program, and its capabilities.
204
+
-**Sessions.** A framework that states a test session (`test-session`, O9), with an mcpp that gives
205
+
launches (`test-launch-records`, B5), runs a selection of cases without a new `mcpp test`: the backend
206
+
starts the session once per project and speaks JSON-RPC 2.0 to it on its standard streams. A run is
207
+
answered at once; its build, its cases and its end arrive as notifications and become the job's
208
+
`test/caseEvent` and `job/finished`. A run is registered when its answer is read, before any
209
+
notification of it. The session ends at the end of its input, hence with the backend; a lifetime
210
+
bound by the system is not used, since on Linux it follows the thread that started the process.
211
+
`mcpp.test.session = "off"` runs every case through `mcpp test`.
212
+
-**Concurrency.** A test runs in a lane of its own; two tests, or a test and a build, do not wait for
213
+
each other. `mcpp.runtime.concurrency` holds only clean and pack.
0 commit comments