diff --git a/.agents/docs/2026-11-15-tests-in-mtest-layout-design.md b/.agents/docs/2026-11-15-tests-in-mtest-layout-design.md new file mode 100644 index 0000000..44c593d --- /dev/null +++ b/.agents/docs/2026-11-15-tests-in-mtest-layout-design.md @@ -0,0 +1,34 @@ +# 规范仓库的测试：mtest 标准布局（0.25.2，负责人 2026-11-15 确认） + +## 结论 + +| 目录 | 类型 | 内容 | +|---|---|---| +| `tests/unit/` | UTest | 版本：`KAL_VERSION_*` 由编译器读取，与清单版本一致 | +| `tests/e2e/headers.cpp` | E2E | 宏的模块拼写、C 形式在 `-nostdinc` 下编译（cc、clang）、无后端类型及其反证 | +| `tests/e2e/abi.cpp` | E2E | 冻结布局在 riscv32 与 riscv64 上编译（`tests/e2e/data/abi`） | +| `tests/e2e/substitution.cpp` | E2E | 一份源码对两个实现、表面检查拒绝新增名字、接口列表取自产物 | +| `tests/cts/` | CTS、Bench | 一致性套件（原 `conformance/`），独立成员包 | +| `kit/tests/` | UTest | openkal-kit 的用例，随包走 | + +## 约束与取舍 + +- **cts 不使用 mtest 的 C++ 库。** 库依赖 `import std`，而套件要在没有 C 库的实现上运行（固件、裸机、DOS）。 + 套件自己实现协议（L2：cts 用例、bench 用例、选择、列出）。bench 留在套件内，与各节共享状态，所有平台都有成本数据。 +- **检查用 C++ 驱动工具。** `tools/` 下的脚本同时供实现仓库使用，保持不变。用例在自己的临时目录里操作副本 + （头文件副本、示例副本、生成到临时文件的 `macros.cppm`），不写检出目录，因此无需串行。 + 缺少的编译器作为能力（capability）声明，相应用例跳过并给出原因。 +- **CI 收敛。** `tests` 作业在 Linux（gcc、llvm）与 macOS（llvm）上执行 `mtest run --set full`； + `declarations` 作业在五行上只构建模块形式。这些用例的对象是文本、头文件和工具，与消费者使用的工具链无关， + 所以不必在每一行上运行。 +- **兼容。** `tools/run-conformance.sh` 的参数不变，实现仓库的调用方式不变。套件的存储变为 + `tests/cts/.mtest`；读取该存储的实现仓库 CI（linux、macos、windows、posix、emscripten）和 freebsd 的 + 独立运行脚本在本版合入后同步路径。 + +## 生态侧发现 + +- mcpp：默认的 `tests/**/*.cpp` 会把成员包 `tests/cts/src/*.cpp` 当作根包测试。mcpp 2026.11.14.1 规定， + 包根之下含 `mcpp.toml` 的目录是另一个包，不参与发现。本仓库显式声明 `[test] discover = ["tests/*/*.cpp"]`， + 不依赖该版本。CI 的 mcpp 固定为 2026.11.14.1。 +- mtest 0.8.5：库在 macOS 的 LLVM 20 libc++（`import std`）下无法编译（`lib/src/mtest.cpp` 中 + `directory_iterator` 的比较未经 std 模块导出）。`tests` 作业的 macOS 行因此使用 LLVM 22。 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b9a71a4..762837e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -3,17 +3,23 @@ name: CI # What this workflow asserts about the specification. # # A specification is a claim about programs, so the claims are tested by -# building and running programs rather than by reading the text. There are four: +# building and running programs rather than by reading the text. The tests are +# written with mtest in its layout (mtest.toml) and each job runs a set of them: # -# declarations the two forms compile, under three compiler families and on -# three systems, and declare the same entities -# substitution a program's source is invariant under a change of -# implementation, and the check that says so fails when it should -# conformance the suite in this repository runs against the implementation -# for each system and every observation holds +# declarations the module form compiles, under three compiler families and +# on three systems +# tests tests/unit and tests/e2e: the version, the C form and its +# entities, no backend type, the layout at 32 and 64 bits, and +# that a program's source is invariant under a change of +# implementation, with the surface checker failing when it should +# conformance the suite (tests/cts) runs against the implementation for each +# system: one `cts' case per interface and one `bench' case per +# measured cost; the kit's cases over the same implementation # composability an implementation that provides three interfaces is examined # for three, rather than failing to link # +# The `report' job takes every job's store into one report of the run. +# # The three compiler families are covered because the specification is a # contract and a contract that holds only under the compiler its author used is # a description of that compiler. @@ -33,12 +39,12 @@ env: # A version verified to build these packages, not a measured minimum. The pin # exists for reproducibility rather than because an older mcpp is known to # fail. - MCPP_VERSION: 2026.11.5.1 + MCPP_VERSION: 2026.11.14.1 XLINGS_VERSION: v2026.10.26.1 # The mtest engine the conformance suite reports to (tools/run-conformance.sh # with MTEST set): an xlings package, else the release bundle with this hash. - MTEST_VERSION: 0.5.2 - MTEST_SHA256: be443f8c546b3f4628dcd3f1eb6f45b4d698a7f805232f966349faf8250863c6 + MTEST_VERSION: 0.8.5 + MTEST_SHA256: 4b890455ee86eada349b9831db6c0b3101861ad53a64effa548c9749f65f5355 XLINGS_NON_INTERACTIVE: '1' jobs: @@ -187,237 +193,26 @@ jobs: # The C++ form. The modules are the artefact a C++ consumer imports, and # building them is what proves the compiler accepts them. - # A 32-BIT TARGET, AND NOTHING ELSE IN THIS REPOSITORY IS ONE. - # - # Every row of every job here is hosted, and every hosted target this - # specification serves is 64-bit. That is a structural blind spot rather - # than an omission: the C ABI this repository freezes is instantiated at - # two pointer widths, and only one of them was ever compiled. - # - # Measured 2026-08-20: `kal_node_info`'s frozen-layout assertion read - # `offsetof(modified_ns) == sizeof(kal_uintptr)`, which is true at 64 bits - # and false at 32 — the timestamp is a naturally-aligned `kal_u64`, so a - # 32-bit `size` is followed by four bytes of padding and the offset stays - # at eight. The failure surfaced in a CONSUMER (`mcpplibs/riscv-virt-rt`, - # whose rv32 leg activates the openkal feature), which is the only place - # the 32-bit layout existed at all. - # - # Compile-only: there is no 32-bit implementation to run against, and the - # claim under test is about layout rather than behaviour. - # A 32-BIT TARGET, AND NOTHING ELSE IN THIS REPOSITORY IS ONE. - # - # Every other row here is hosted, and every hosted target this - # specification serves is 64-bit. That is a structural blind spot: the C - # ABI this repository freezes is instantiated at two pointer widths, and - # only one of them was ever compiled. - # - # Measured 2026-08-20: `kal_node_info`'s frozen-layout assertion read - # `offsetof(modified_ns) == sizeof(kal_uintptr)`, true at 64 bits and - # false at 32 — the timestamp is a naturally-aligned `kal_u64`, so a - # 32-bit `size` is followed by four bytes of padding and the offset stays - # at eight. It surfaced in a CONSUMER (`mcpplibs/riscv-virt-rt`'s rv32 - # leg), the only place the 32-bit layout existed at all. - # - # THROUGH `mcpp`, NOT THROUGH A BORROWED clang. The first version of - # this step globbed the LLVM payload for `clang++` — which is absent on - # the gcc and msvc rows, and is named `clang++.exe` on Windows, so the - # command expanded to nothing and the step died with `: command not - # found` on four rows. mcpp resolves the toolchain for a target itself; - # asking it is the only spelling that holds on every row. - # A 32-BIT TARGET, AND NOTHING ELSE IN THIS REPOSITORY IS ONE. - # - # Every other row here is hosted, and every hosted target this - # specification serves is 64-bit. That is a structural blind spot: the C - # ABI this repository freezes is instantiated at two pointer widths, and - # only one of them was ever compiled. - # - # Measured 2026-08-20: `kal_node_info`'s frozen-layout assertion read - # `offsetof(modified_ns) == sizeof(kal_uintptr)`, true at 64 bits and - # false at 32 — the timestamp is a naturally-aligned `kal_u64`, so a - # 32-bit `size` is followed by four bytes of padding and the offset stays - # at eight. It surfaced in a CONSUMER (`mcpplibs/riscv-virt-rt`'s rv32 - # leg), the only place the 32-bit layout existed at all. - # - # THE PROBE LIVES INSIDE THE CHECKOUT AND USES A RELATIVE PATH. Two - # earlier spellings failed for reasons that had nothing to do with the - # ABI: globbing the LLVM payload for `clang++` died with `: command not - # found` on the rows whose toolchain is gcc or msvc, and interpolating - # `$GITHUB_WORKSPACE` into a C++ string literal produced - # `"D:\a\openkal\openkal/include"`, whose backslashes the compiler read - # as escape sequences. A relative directory has neither problem, and - # `include_dir` resolves it against the package root. - - name: The C ABI holds at thirty-two bits as well as sixty-four - run: | - set -euo pipefail - rm -rf .abi-probe && mkdir -p .abi-probe/src - cat > .abi-probe/mcpp.toml <<'EOF' - [package] - name = "abi" - version = "0.1.0" - [build] - include_dirs = ["../include"] - EOF - cat > .abi-probe/src/main.cpp <<'EOF' - #include - #include - // `self_size' IS FIRST, AND THAT IS THE WHOLE OF THE GROWTH RULE. - // An implementation built at a later version reads this field before - // it touches anything else, which is how it learns how much of the - // structure the caller has. A layout in which it were not first would - // make the field unreadable by exactly the implementation it exists - // to inform. - static_assert(__builtin_offsetof(kal_node_info, self_size) == 0); - static_assert(__builtin_offsetof(kal_node_info, present) == 4); - // The fields that existed at 0.9.0, at the offsets they had. Growth - // is permitted only past the end; these do not move. - static_assert(__builtin_offsetof(kal_node_info, size) == 8); - static_assert(__builtin_offsetof(kal_node_info, modified_ns) == 16); - static_assert(__builtin_offsetof(kal_node_info, identity) == 24); - static_assert(__builtin_offsetof(kal_node_info, kind) == 40); - static_assert(__builtin_offsetof(kal_node_info, writable) == 44); - // Appended in 0.13, past the end of what existed. - static_assert(__builtin_offsetof(kal_node_info, executable) == 48); - static_assert(__builtin_offsetof(kal_node_info, reserved) == 52); - // The same at both pointer widths, which is the property this - // 32-bit row exists to observe: no field of it is pointer-sized. - static_assert(sizeof(kal_node_info) == 56); - static_assert(sizeof(kal_dir) == sizeof(kal_uintptr)); - static_assert(sizeof(kal_file) == sizeof(kal_uintptr)); - // A transfer is one signed word, and a signed word is a pointer's - // width. An implementation returning `int' here would truncate a - // count on a 64-bit machine and would be caught by nothing else. - static_assert(sizeof(kal_intptr) == sizeof(kal_uintptr)); - static_assert((kal_intptr)-1 < 0); - extern "C" void _start() { for (;;) {} } - EOF - for t in riscv64-none-elf riscv32-none-elf; do - ( cd .abi-probe && rm -rf target && mcpp build --target "$t" ) \ - || { echo "the frozen layout does not hold for $t"; exit 1; } - echo "$t: layout holds" - done - rm -rf .abi-probe - - # EVERY MACRO HAS A MODULE SPELLING, AND THIS IS CHECKED RATHER THAN - # REMEMBERED --- because twice it was not. `KAL_LOCK_*' in 0.10 and - # `KAL_SPAWN_*' in 0.11 were each added to a header, used from a module - # consumer, and failed to compile there, because a macro does not cross a - # module boundary. Both were found by a consumer's build in another - # repository rather than here. - # - # ⇒ src/macros.cppm is GENERATED. Regenerating and diffing turns "somebody - # forgot" into a red build in the repository that made the change. - - name: Every macro has a module spelling - if: runner.os != 'Windows' - run: | - bash tools/gen-macros.sh - git diff --exit-code src/macros.cppm || { - echo "::error::src/macros.cppm is out of date --- run tools/gen-macros.sh and commit it" - exit 1 - } - - # WHAT THE DECLARATIONS SAY THE VERSION IS, AND WHAT THE PACKAGE IS. - # - # `kal_version' is `return KAL_VERSION;' in every implementation there is, - # so the three constants in version.h are what the ecosystem states about - # itself and the only thing a consumer bound at load can ask. They had - # drifted through two releases, and the comparison clause 6.2 gives that - # consumer was between two equal numbers the whole time. - - name: The declarations state the version this package is - if: runner.os != 'Windows' - run: bash tools/check-version.sh - - # AND THE CHECK DETECTS WHAT IT EXISTS TO DETECT, for the reason the type - # check below is proved the same way: a check that reports success by - # finding nothing reports it identically when it has read nothing. - - name: The version check detects a drift - if: runner.os != 'Windows' - run: | - # `sed -i` IS NOT PORTABLE AND THIS MATRIX REACHES THE SYSTEM WHERE - # IT IS NOT: one of the three reads the next argument as the suffix - # for a backup file, so the edit is attempted upon a file named by - # the expression. Measured on this row, on the first run of this - # step. The edit is therefore written beside and moved over. - cp include/openkal/version.h /tmp/version.h.orig - sed 's/^#define KAL_VERSION_MINOR .*/#define KAL_VERSION_MINOR 11u/' \ - /tmp/version.h.orig > include/openkal/version.h - if bash tools/check-version.sh; then - cp /tmp/version.h.orig include/openkal/version.h - echo "the version check accepted a number the package is not" >&2 - exit 1 - fi - cp /tmp/version.h.orig include/openkal/version.h - echo "the version check rejects a drifted number, as it must" - - name: The module form compiles run: mcpp build - # The C form, with the environment's own headers excluded --- because the - # consumer this form exists for, a C library being ported onto openkal, is - # compiled that way. The tool needs a driver it can pass -nostdinc to; the - # toolchain that has no such spelling compiles the same declarations in - # the translation unit the conformance suite carries, which every row of - # the conformance job below builds. - - name: The C form compiles without the environment's headers - if: runner.os != 'Windows' - run: | - CC=cc bash tools/check-declarations.sh - command -v clang >/dev/null && CC=clang bash tools/check-declarations.sh || true - - # AND NO DECLARATION NAMES A BACKEND TYPE. - # - # The job above asserts the declarations COMPILE. This asserts what they - # are written WITH, which is a different freedom and the one that decides - # whether a `kal_*` signature means the same thing on four object formats. - # `sizeof(long)` is 8 on x86_64 Linux and 4 on x86_64 Windows; a `long` in - # an interface would be one signature with two meanings. - # - # It runs on every row of this matrix rather than once, because a type - # is a claim about a TARGET and this matrix is where the targets are. - # - # Needs a clang-family driver (`-Xclang -ast-print`); rows whose - # toolchain has no such spelling are covered by the other rows, and the - # step says so rather than passing silently. - - name: No declaration names a backend type - if: runner.os != 'Windows' - run: | - cc="$(command -v clang || true)" - if [ -z "$cc" ]; then - echo "no clang on this row; the type check runs on the rows that have one" - exit 0 - fi - bash tools/check-types.sh "$cc" include - - # AND THE CHECK DETECTS WHAT IT EXISTS TO DETECT. - # - # A check that succeeds by finding nothing succeeds identically when it - # reads nothing, and every false green in this ecosystem has had that - # shape. `check-types.sh` proves it parsed the headers before reporting; - # this proves it would have failed. - - name: The type check detects a leak - if: runner.os != 'Windows' - run: | - cc="$(command -v clang || true)" - [ -n "$cc" ] || exit 0 - cp include/openkal/env.h /tmp/env.h.orig - printf '\nint kal_env_probe_leak(unsigned long n);\n' >> include/openkal/env.h - if bash tools/check-types.sh "$cc" include; then - cp /tmp/env.h.orig include/openkal/env.h - echo "the type check accepted 'unsigned long' in an interface" >&2 - exit 1 - fi - cp /tmp/env.h.orig include/openkal/env.h - echo "the type check rejects a backend type, as it must" - # --------------------------------------------------------------------------- - # The property the specification exists for. - substitution: - name: substitution holds (${{ matrix.toolchain }}) - runs-on: ubuntu-24.04 + # The tests of tests/unit and tests/e2e (mtest.toml), among them the property + # the specification exists for. Their subjects are the text, the headers + # through the tools, the layout at both widths and substitution; none depends + # on the toolchain a consumer builds with, so they run where the tools run + # (POSIX systems), under two compiler families on Linux and on macOS. + tests: + name: tests (${{ matrix.os }}, ${{ matrix.toolchain }}) + runs-on: ${{ matrix.os }} timeout-minutes: 40 strategy: fail-fast: false matrix: - toolchain: ['gcc@16.1.0', 'llvm@22.1.8'] + include: + - { os: ubuntu-24.04, toolchain: 'gcc@16.1.0' } + - { os: ubuntu-24.04, toolchain: 'llvm@22.1.8' } + - { os: macos-15, toolchain: 'llvm@22.1.8' } steps: - uses: actions/checkout@v4 @@ -532,113 +327,23 @@ jobs: esac mcpp toolchain list - - name: Substitution holds - working-directory: examples/substitution/app - run: | - # The property under test is that the source is invariant. The - # checksum is taken before the first build and compared after the - # second, so that a change made by either build would be detected. - before="$(sha256sum src/main.cpp | cut -d' ' -f1)" - - mcpp build > /dev/null - ./target/*/*/bin/app > with-fd.log 2>&1 - grep -q 'the application produced this line' with-fd.log - - sed -i 's|openkal-fd = { path = "../impl-fd" }|openkal-discard = { path = "../impl-discard" }|' mcpp.toml - rm -rf target - mcpp build > /dev/null - ./target/*/*/bin/app > with-discard.log 2>&1 - if grep -q 'the application produced this line' with-discard.log; then - echo "the discarding implementation produced output"; exit 1 - fi - - after="$(sha256sum src/main.cpp | cut -d' ' -f1)" - [ "$before" = "$after" ] || { echo "the source changed between builds"; exit 1; } - - - name: The surface checker detects an addition - working-directory: examples/substitution/impl-fd - run: | - # The checker is only useful if it fails when it should. An - # unspecified name is introduced, the checker is required to reject - # it, and the name is then removed. - mcpp build > /dev/null - bash ../../../tools/check-surface.sh ../../../SURFACE.txt \ - $(find target -name '*.o' | tr '\n' ' ') - - printf 'extern "C" void kal_vendor_extension(void) {}\n' > src/extra.cpp - mcpp build > /dev/null - if bash ../../../tools/check-surface.sh ../../../SURFACE.txt \ - $(find target -name '*.o' | tr '\n' ' '); then - echo "the checker accepted an unspecified name"; exit 1 - fi - rm -f src/extra.cpp - - - name: The interface list is derived from the artefact - working-directory: examples/substitution/impl-fd + # Every case: the version, the macros' module spellings, the C form + # without the environment's headers (each C compiler present), no backend + # type in a declaration, the frozen layout at 32 and 64 bits, one source + # over two implementations, and the surface checker. A case whose + # compiler this machine lacks is skipped and says why. + - uses: ./.github/actions/mtest + - name: Every test of the specification run: | - set -euo pipefail - # `provides-interfaces` in a package manifest is a declaration, and a - # declaration is checked rather than trusted. The check available to - # it is clause 9's surface comparison stopped one step earlier: the - # walk that decides whether each group is exported whole already - # answers which interfaces an implementation provides. - # - # THE SUBJECT IS A SYNTHETIC OBJECT BUILT FROM SURFACE.txt, AND NOT - # THIS EXAMPLE. `impl-fd` exists to show substitution, not - # conformance: it defines six of openkal.stream's seven names and - # none of the other groups, so no interface is exported whole and - # every leg below would be measuring an implementation that claims - # nothing. The object here is constructed so that the answer is known - # in advance, which is what lets the check fail for the one reason it - # exists to catch. - python3 - ../../../SURFACE.txt > fake.c <<'PY' - import sys - groups, cur = {}, None - for line in open(sys.argv[1], encoding='utf-8'): - line = line.rstrip('\n') - if line.startswith('# openkal.'): - cur = line[2:].split(' ')[0]; groups[cur] = [] - elif line.startswith('#') or not line.strip(): - continue - elif cur: - groups[cur].append(line.strip()) - # Three groups whole, and two names of a fourth: an interface is - # provided in whole or not at all, so the fourth must not be listed. - for g in ('openkal.version', 'openkal.abort', 'openkal.stream', 'openkal.memory'): - for n in groups.get(g, []): - print(f'int {n}(void);\nint {n}(void) {{ return 0; }}') - for n in groups.get('openkal.fs', [])[:2]: - print(f'int {n}(void);\nint {n}(void) {{ return 0; }}') - PY - cc -c fake.c -o fake.o - bash ../../../tools/check-surface.sh --interfaces \ - ../../../SURFACE.txt fake.o > provided.txt - cat provided.txt - - for want in openkal.abort openkal.stream openkal.memory; do - grep -qx "$want" provided.txt || { - echo "::error::$want is exported whole and was not listed"; exit 1; } - done - # `openkal.version` is exported by every conforming implementation - # and is not an interface --- SURFACE.txt says so in its own header. - # Listing it would put a name that answers nothing in every package. - if grep -qx 'openkal.version' provided.txt; then - echo "::error::openkal.version is not an interface and must not be listed"; exit 1 - fi - # HALF AN INTERFACE IS NOT A SMALLER CLAIM. Without this leg the - # checks above pass against a script that prints every group in - # SURFACE.txt and reads no artefact at all. - if grep -qx 'openkal.fs' provided.txt; then - echo "::error::a group exported in part was reported as provided"; exit 1 - fi - # And the --toml spelling is the same answer in the shape a manifest - # carries, because a package's declaration is generated from it. - bash ../../../tools/check-surface.sh --toml \ - ../../../SURFACE.txt fake.o > provided.toml - grep -qx 'provides-interfaces = \[' provided.toml - grep -qx ' "openkal.stream",' provided.toml - rm -f fake.c fake.o provided.txt provided.toml - echo " ok the list is the artefact's own answer" + "$MTEST" run --set full --group "$GITHUB_RUN_ID" + "$MTEST" report --format md --group "$GITHUB_RUN_ID" >> "$GITHUB_STEP_SUMMARY" + - uses: actions/upload-artifact@v4 + if: always() + with: + name: mtest-tests-${{ strategy.job-index }} + path: .mtest + include-hidden-files: true + if-no-files-found: ignore # --------------------------------------------------------------------------- # The suite, against the implementation for each system. @@ -878,24 +583,24 @@ jobs: # THE SUITE REPORTS BY THE TEST PROTOCOL HERE (MTEST is set): each # interface a case of kind `cts', each observation a case within it, kept - # in the engine's store under conformance/.mtest. A failure is explained + # in the engine's store under tests/cts/.mtest. A failure is explained # in the log, since the engine prints a summary and not the suite's text. - name: Every interface, every kind of examination run: | bash tools/run-conformance.sh '${{ matrix.implementation }}' .impl '${{ matrix.features }}' \ - || { ( cd conformance && "$MTEST" explain ) || true; exit 1; } + || { ( cd tests/cts && "$MTEST" explain ) || true; exit 1; } - name: The run, reported if: always() run: | - [ -f conformance/.mtest/index.ndjson ] || { echo "the suite kept no run"; exit 0; } - ( cd conformance && "$MTEST" report --format md --group "$GITHUB_RUN_ID" ) >> "$GITHUB_STEP_SUMMARY" + [ -f tests/cts/.mtest/index.ndjson ] || { echo "the suite kept no run"; exit 0; } + ( cd tests/cts && "$MTEST" report --format md --group "$GITHUB_RUN_ID" ) >> "$GITHUB_STEP_SUMMARY" - uses: actions/upload-artifact@v4 if: always() with: name: mtest-cts-${{ matrix.implementation }}-${{ strategy.job-index }} - path: conformance/.mtest + path: tests/cts/.mtest include-hidden-files: true if-no-files-found: ignore retention-days: 7 @@ -955,6 +660,13 @@ jobs: if: matrix.implementation == 'openkal-linux' run: | bash tools/run-kit-tests.sh '${{ matrix.implementation }}' .impl + - uses: actions/upload-artifact@v4 + if: always() && matrix.implementation == 'openkal-linux' + with: + name: mtest-kit-${{ strategy.job-index }} + path: kit/target/mtest + include-hidden-files: true + if-no-files-found: ignore # THE EXAMPLE A CONSUMER READS, BUILT AND RUN. # @@ -1022,7 +734,7 @@ jobs: - name: A selection of three interfaces is examined, not refused run: | - rm -rf conformance/target + rm -rf tests/cts/target # The suite's own text is what this step reads, so no runner is named. MTEST= bash tools/run-conformance.sh '${{ matrix.implementation }}' .impl core,fs,task \ | tee selected.log @@ -1040,13 +752,14 @@ jobs: grep -qE 'held +\[behaviour\].*' selected.log # --------------------------------------------------------------------------- - # EVERY IMPLEMENTATION'S RUN AS ONE REPORT: each conformance job's store, - # taken into one, and reported together as the workflow run's group - # (`mtest ci report`): the cases of each interface on each implementation, - # the failures, and the measurements of cost. - conformance-report: - name: conformance (report) - needs: conformance + # EVERY RUN OF THE WORKFLOW AS ONE REPORT: the stores of the tests (`unit', + # `e2e'), of the kit's tests and of each + # implementation's conformance run (`cts', with the interfaces' costs as + # `bench' cases), taken into one and reported together as the workflow run's + # group (`mtest ci report'). + report: + name: the run, reported (unit, e2e, kit, cts, bench) + needs: [tests, conformance] if: always() runs-on: ubuntu-24.04 timeout-minutes: 10 @@ -1055,9 +768,9 @@ jobs: - uses: ./.github/actions/mtest - uses: actions/download-artifact@v4 with: - pattern: mtest-cts-* + pattern: mtest-* path: ${{ runner.temp }}/stores - - name: Every implementation's run, reported together + - name: Every store of the run, reported together run: | set -euo pipefail found=0 @@ -1066,5 +779,5 @@ jobs: "$MTEST" store import --from "$store" found=1 done - if [ "$found" = 0 ]; then echo "no conformance job kept a run" >> "$GITHUB_STEP_SUMMARY"; exit 0; fi + if [ "$found" = 0 ]; then echo "no job kept a run" >> "$GITHUB_STEP_SUMMARY"; exit 0; fi "$MTEST" ci report --out "$RUNNER_TEMP/conformance.md" diff --git a/README.md b/README.md index 0b811f0..4574805 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ words become types that cannot be mixed. | --- | --- | | `tools/check-declarations.sh` | compiles a translation unit naming every entity, with `-nostdinc` — because the consumer the header exists for is compiled that way | | a test in each implementation | the same list, reached through `import openkal.*` | -| `conformance/src/declarations.c` | the same list again, compiled by every toolchain that builds the suite | +| `tests/cts/src/declarations.c` | the same list again, compiled by every toolchain that builds the suite | The header includes nothing. openkal must be usable on a freestanding target, and a consumer compiled with `-nostdinc` has nothing to include. @@ -88,6 +88,15 @@ observed (entry 60). The conformance suite reports to a test runner by the mtest protocol, and checks that an implementation's manifest, `kal_interfaces()` and its exported names state the same interfaces. +0.25.2 changes no clause. The suite reaches every name of the surface without +requiring it, so a link that discards unreached definitions keeps the surface +it compares (entry 63), and reports each interface's measured cost as a `bench` +case beside its `cts` case. The tests of this repository are written with mtest +0.8.5 in its layout (`mtest.toml`): `tests/unit` and `tests/e2e` hold the +version, the headers through the tools, the 32-bit layout and substitution +(`mtest run --set checks`, `--set substitution`), `tests/cts` is the suite, and +openkal-kit's tests use the mtest library; each CI run reports them together. + ## How a program uses openkal A program declares two dependencies. The first fixes the version of the contract @@ -96,7 +105,7 @@ conditional on the target. ```toml [dependencies] -openkal = "0.25.1" +openkal = "0.25.2" [target.'cfg(os = "linux")'.dependencies] openkal-linux = "0.26.1" @@ -235,7 +244,7 @@ clause 6.2 tabulates, and the earliest at which the question can be answered — source that asks the same question with `#ifdef` asks it during preprocessing, which is earlier than any answer exists. -**The behaviour.** [`conformance/`](conformance/) is a program an implementation +**The behaviour.** [`tests/cts/`](tests/cts/) is a program an implementation runs against itself — 193 observations across fifteen interfaces, in four kinds: behaviour, ABI, stability and cost. diff --git a/SPEC.md b/SPEC.md index 84a584d..361ea31 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,4 +1,4 @@ -# openkal Specification, version 0.25.1 +# openkal Specification, version 0.25.2 ## 1. Scope @@ -2224,6 +2224,13 @@ The following are recorded so that they are not mistaken for oversights. what an image of such an architecture is, were not written down. No declaration changes. +63. **The surface the conformance suite links.** **Changed in 0.25.2**; no + clause and no declaration changes. A link that discards definitions nothing + reaches kept of an implementation only what the suite calls, and the + comparison of clause 9 then reported an interface absent whose definitions + the implementation holds. The suite now reaches every name without + requiring it, so a defined name is kept and an absent one is null. + ## 12. Images A program written against this specification is built for an environment: its diff --git a/conformance/src/declarations.c b/conformance/src/declarations.c deleted file mode 100644 index 8e4d190..0000000 --- a/conformance/src/declarations.c +++ /dev/null @@ -1,188 +0,0 @@ -/* The C declarations, named once, so that every toolchain that builds this - * suite compiles them. - * - * Clause 4.3 requires the two forms of the declarations to declare the same - * entities, and tools/check-declarations.sh performs that comparison against - * SURFACE.txt --- which is normative --- with the environment's own headers - * excluded. That tool needs a compiler driver it can pass -nostdinc to, and one - * of the three toolchains this suite is built with has no such spelling. So the - * comparison is performed in one place and the compiling is performed here, in - * a translation unit the suite carries: a header that fails to compile as C - * under a toolchain then fails that toolchain's build rather than going - * unnoticed until a C consumer meets it. - * - * No name is referenced in an evaluated context. The suite is composable --- an - * implementation provides an interface in whole or not at all --- so a file that - * required all fifty-one definitions would fail to link against a conforming - * implementation of three interfaces, which is the arrangement the suite exists - * to accommodate. sizeof is unevaluated, so this file declares a dependency on - * nothing while still rejecting a name the header fails to declare. - * - * Regenerated by tools/check-declarations.sh --write. It is committed rather - * than generated during the build so that the suite compiles in a checkout that - * has nothing but a compiler. - */ -#include - -void okc_declarations_c(void); -void okc_declarations_c(void) -{ - (void)sizeof(&kal_abort); - (void)sizeof(&kal_alloc); - (void)sizeof(&kal_datagram_close); - (void)sizeof(&kal_datagram_local); - (void)sizeof(&kal_datagram_open); - (void)sizeof(&kal_datagram_props); - (void)sizeof(&kal_datagram_recv_from); - (void)sizeof(&kal_datagram_send_to); - (void)sizeof(&kal_debug_props); - (void)sizeof(&kal_debug_symbolize); - (void)sizeof(&kal_env_arg); - (void)sizeof(&kal_env_arg_count); - (void)sizeof(&kal_env_image); - (void)sizeof(&kal_env_loader); - (void)sizeof(&kal_env_resources); - (void)sizeof(&kal_env_system); - (void)sizeof(&kal_env_var); - (void)sizeof(&kal_env_var_at); - (void)sizeof(&kal_env_var_count); - (void)sizeof(&kal_exec_alloc); - (void)sizeof(&kal_exec_alloc_at); - (void)sizeof(&kal_exec_free); - (void)sizeof(&kal_exec_granularity); - (void)sizeof(&kal_exec_map_part); - (void)sizeof(&kal_exec_props); - (void)sizeof(&kal_exec_publish); - (void)sizeof(&kal_exec_publish_part); - (void)sizeof(&kal_exec_seal_part); - (void)sizeof(&kal_exit); - (void)sizeof(&kal_free); - (void)sizeof(&kal_fs_capacity); - (void)sizeof(&kal_fs_close_dir); - (void)sizeof(&kal_fs_close_file); - (void)sizeof(&kal_fs_dir_attenuate); - (void)sizeof(&kal_fs_file_info); - (void)sizeof(&kal_fs_hard_link); - (void)sizeof(&kal_fs_info); - (void)sizeof(&kal_fs_link_create); - (void)sizeof(&kal_fs_link_read); - (void)sizeof(&kal_fs_list_begin); - (void)sizeof(&kal_fs_list_next); - (void)sizeof(&kal_fs_list_next_info); - (void)sizeof(&kal_fs_lock); - (void)sizeof(&kal_fs_max_name); - (void)sizeof(&kal_fs_mkdir); - (void)sizeof(&kal_fs_open); - (void)sizeof(&kal_fs_open_dir); - (void)sizeof(&kal_fs_preopen); - (void)sizeof(&kal_fs_preopen_count); - (void)sizeof(&kal_fs_props); - (void)sizeof(&kal_fs_read_at); - (void)sizeof(&kal_fs_remove); - (void)sizeof(&kal_fs_rename); - (void)sizeof(&kal_fs_rename_flags); - (void)sizeof(&kal_fs_resolved_name); - (void)sizeof(&kal_fs_seek); - (void)sizeof(&kal_fs_set_executable_at); - (void)sizeof(&kal_fs_set_modified); - (void)sizeof(&kal_fs_set_modified_at); - (void)sizeof(&kal_fs_stream); - (void)sizeof(&kal_fs_truncate); - (void)sizeof(&kal_fs_unlock); - (void)sizeof(&kal_fs_write_at); - (void)sizeof(&kal_interfaces); - (void)sizeof(&kal_map_file); - (void)sizeof(&kal_map_release); - (void)sizeof(&kal_map_sync); - (void)sizeof(&kal_memory_granularity); - (void)sizeof(&kal_memory_info); - (void)sizeof(&kal_net_accept); - (void)sizeof(&kal_net_close); - (void)sizeof(&kal_net_close_listener); - (void)sizeof(&kal_net_connect); - (void)sizeof(&kal_net_connect_at); - (void)sizeof(&kal_net_listen); - (void)sizeof(&kal_net_listen_at); - (void)sizeof(&kal_net_listener_local); - (void)sizeof(&kal_net_local); - (void)sizeof(&kal_net_peer); - (void)sizeof(&kal_net_props); - (void)sizeof(&kal_net_shutdown); - (void)sizeof(&kal_net_stream); - (void)sizeof(&kal_process_channel); - (void)sizeof(&kal_process_channel_close); - (void)sizeof(&kal_process_close); - (void)sizeof(&kal_process_end_as_requested); - (void)sizeof(&kal_process_info); - (void)sizeof(&kal_process_job_close); - (void)sizeof(&kal_process_job_create); - (void)sizeof(&kal_process_job_enter); - (void)sizeof(&kal_process_job_limit); - (void)sizeof(&kal_process_job_observe); - (void)sizeof(&kal_process_job_open); - (void)sizeof(&kal_process_job_retire); - (void)sizeof(&kal_process_job_terminate); - (void)sizeof(&kal_process_job_usage); - (void)sizeof(&kal_process_open_id); - (void)sizeof(&kal_process_props); - (void)sizeof(&kal_process_self); - (void)sizeof(&kal_process_spawn); - (void)sizeof(&kal_process_stop_requested); - (void)sizeof(&kal_process_terminate); - (void)sizeof(&kal_process_wait); - (void)sizeof(&kal_prof_active); - (void)sizeof(&kal_prof_counter); - (void)sizeof(&kal_prof_mark); - (void)sizeof(&kal_prof_props); - (void)sizeof(&kal_prof_sample_begin); - (void)sizeof(&kal_prof_sample_end); - (void)sizeof(&kal_prof_sample_take); - (void)sizeof(&kal_prof_touch_begin); - (void)sizeof(&kal_prof_touch_end); - (void)sizeof(&kal_prof_touch_take); - (void)sizeof(&kal_random_fill); - (void)sizeof(&kal_random_props); - (void)sizeof(&kal_region_commit); - (void)sizeof(&kal_region_decommit); - (void)sizeof(&kal_region_discard); - (void)sizeof(&kal_region_props); - (void)sizeof(&kal_region_release); - (void)sizeof(&kal_region_reserve); - (void)sizeof(&kal_space_props); - (void)sizeof(&kal_space_start); - (void)sizeof(&kal_stderr); - (void)sizeof(&kal_stdin); - (void)sizeof(&kal_stdout); - (void)sizeof(&kal_stream_flush); - (void)sizeof(&kal_stream_props); - (void)sizeof(&kal_stream_read); - (void)sizeof(&kal_stream_write); - (void)sizeof(&kal_task_cores); - (void)sizeof(&kal_task_current); - (void)sizeof(&kal_task_join); - (void)sizeof(&kal_task_parallelism); - (void)sizeof(&kal_task_props); - (void)sizeof(&kal_task_set_priority); - (void)sizeof(&kal_task_stack); - (void)sizeof(&kal_task_start); - (void)sizeof(&kal_task_start_sized); - (void)sizeof(&kal_task_wait); - (void)sizeof(&kal_task_wake); - (void)sizeof(&kal_task_yield); - (void)sizeof(&kal_terminal_get_mode); - (void)sizeof(&kal_terminal_props); - (void)sizeof(&kal_terminal_set_mode); - (void)sizeof(&kal_terminal_size); - (void)sizeof(&kal_time_monotonic); - (void)sizeof(&kal_time_monotonic_granularity); - (void)sizeof(&kal_time_props); - (void)sizeof(&kal_time_sleep); - (void)sizeof(&kal_time_wall); - (void)sizeof(&kal_timeout_accept); - (void)sizeof(&kal_timeout_granularity); - (void)sizeof(&kal_timeout_read); - (void)sizeof(&kal_timeout_recv_from); - (void)sizeof(&kal_timeout_wait_process); - (void)sizeof(&kal_timeout_write); - (void)sizeof(&kal_version); -} diff --git a/conformance/src/suite.cpp b/conformance/src/suite.cpp deleted file mode 100644 index 072bcea..0000000 --- a/conformance/src/suite.cpp +++ /dev/null @@ -1,73 +0,0 @@ -module okc.suite; - -import okc.spec; -import okc.report; -import okc.protocol; -import okc.version; -import okc.abort; -import okc.stream; -import okc.memory; -import okc.env; -import okc.time; -import okc.random; -import okc.fs; -import okc.process; -import okc.task; -import okc.exec; -import okc.terminal; -import okc.net; -import okc.datagram; -import okc.space; -import okc.timeout; -import okc.map; -import okc.region; -import okc.prof; -import okc.debug; - -namespace okc { - -namespace { - -// The cases a runner lists and selects (the mtest protocol): one per section, -// in the order the sections run. A section's case begins at its heading. -struct section_entry { const char* id; void (*run)(); }; -constexpr section_entry sections[] = { - { "openkal.version", version::run }, - { "openkal.stream", stream::run }, - { "openkal.memory", memory::run }, - { "openkal.env", env::run }, - { "openkal.time", time::run }, - { "openkal.random", random::run }, - { "openkal.fs", fs::run }, - { "openkal.process", process::run }, - { "openkal.task", task::run }, - { "openkal.exec", exec::run }, - { "openkal.terminal", terminal::run }, - { "openkal.net", net::run }, - { "openkal.datagram", datagram::run }, - { "openkal.space", space::run }, - { "openkal.timeout", timeout::run }, - { "openkal.map", map::run }, - { "openkal.region", region::run }, - { "openkal.prof", prof::run }, - { "openkal.debug", debug::run }, - { "openkal.abort", termination::run }, -}; - -} // namespace - -int run_all() { - protocol::begin(); - if (protocol::listing()) { - for (const section_entry& e : sections) protocol::list_case(e.id); - return protocol::finish(0, 0, 0); - } - - write_inventory(); - for (const section_entry& e : sections) - if (protocol::wanted(e.id)) e.run(); - - return summarise(); -} - -} // namespace okc diff --git a/examples/portable/mcpp.toml b/examples/portable/mcpp.toml index 2167baf..ecd77ae 100644 --- a/examples/portable/mcpp.toml +++ b/examples/portable/mcpp.toml @@ -9,7 +9,7 @@ name = "portable" version = "0.1.0" [dependencies] -openkal = "0.25.1" +openkal = "0.25.2" # AND THE VERSIONS HAVE TO FOLLOW THE SOURCE, which is the reason this example # is a criterion rather than an assumption. diff --git a/include/openkal/version.h b/include/openkal/version.h index d8c7a59..aff167c 100644 --- a/include/openkal/version.h +++ b/include/openkal/version.h @@ -43,7 +43,7 @@ * consumer actually asks for was checked by nobody. */ #define KAL_VERSION_MAJOR 0u #define KAL_VERSION_MINOR 25u -#define KAL_VERSION_PATCH 1u +#define KAL_VERSION_PATCH 2u #define KAL_VERSION_MAKE(major, minor, patch) \ (((kal_u64)(major) << 32) | ((kal_u64)(minor) << 16) | \ diff --git a/kit/mcpp.toml b/kit/mcpp.toml index df896e5..238cfb2 100644 --- a/kit/mcpp.toml +++ b/kit/mcpp.toml @@ -97,5 +97,10 @@ openkal = "0.25.0" # tools/run-conformance.sh already does for the conformance suite --- and for the # same reason: a change spanning the specification and an implementation must be # tested against both halves as written, not against whichever half is published. +# The tests are written with mtest, so that a runner lists, selects and +# reports each observation as a case. +[dev-dependencies] +mcpplibs.mtest = "0.8.5" + [target.'cfg(os = "linux")'.dev-dependencies] -openkal-linux = { version = "0.26.0", features = ["standalone"] } +openkal-linux = { version = "0.26.0" } diff --git a/kit/tests/kit.cpp b/kit/tests/kit.cpp index 5cc2f9c..ea40026 100644 --- a/kit/tests/kit.cpp +++ b/kit/tests/kit.cpp @@ -3,19 +3,18 @@ // The observations are of behaviour. A test that named each entity and did // nothing with it would compile against a package whose every operation did // nothing, and would report that as working. -#include #include +import mcpplibs.mtest; import openkal.types; import openkal.task; import openkal.kit.endpoint; import openkal.kit.channel; +using namespace mcpplibs::mtest; + namespace { -int failures = 0; -void check(bool held, const char* what) { - if (!held) { std::printf("FAIL: %s\n", what); ++failures; } -} +void check(bool held, const char* what) { expect(held) << what; } void endpoint_section() { using namespace kal::kit; @@ -152,10 +151,7 @@ void channel_section() { } kal_task_join(t); - check(got == kPayload, "every byte written is read"); - if (got != kPayload) - std::printf(" read %llu of %llu\n", - (unsigned long long)got, (unsigned long long)kPayload); + expect(eq(got, kPayload)) << "every byte written is read"; check(all_a5, "the bytes read are the bytes written"); // A read after the close reports end of input rather than waiting. @@ -168,9 +164,5 @@ void channel_section() { } // namespace -int main() { - endpoint_section(); - channel_section(); - if (failures == 0) std::printf("openkal-kit: every observation held\n"); - return failures == 0 ? 0 : 1; -} +UTest endpoints{"endpoint", [] { endpoint_section(); }}; +UTest channels{"channel", [] { channel_section(); }}; diff --git a/mcpp.toml b/mcpp.toml index 932edb9..44b8b01 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,7 +1,7 @@ [package] namespace = "mcpplibs" name = "openkal" -version = "0.25.1" +version = "0.25.2" description = "openkal: a portable kernel ABI specification. This package carries the normative declarations; implementations are separate packages." license = "Apache-2.0" authors = ["mcpplibs"] @@ -30,3 +30,14 @@ kind = "lib" # built against. [build] include_dirs = ["include"] + +# The tests of the specification (tests/, mtest's layout; mtest.toml). A +# dependency for development is not resolved for a consumer. +[dev-dependencies] +mcpplibs.mtest = "0.8.5" + +# One program per file of tests/unit and tests/e2e. tests/cts is a package of +# its own, run against an implementation (tools/run-conformance.sh), and +# tests/e2e/data holds what the cases read. +[test] +discover = ["tests/*/*.cpp"] diff --git a/mtest.toml b/mtest.toml new file mode 100644 index 0000000..e59c41f --- /dev/null +++ b/mtest.toml @@ -0,0 +1,22 @@ +# The tests of the specification, run by mtest (https://github.com/mcpp-community/mtest), +# in its layout: +# +# tests/unit/ the declarations by themselves UTest +# tests/e2e/ the headers through the tools, the 32-bit layout, +# substitution and the surface checker E2E +# tests/cts/ the suite, a package run against an implementation +# (tools/run-conformance.sh): a `cts' case per interface, +# a `bench' case per measured cost +# kit/tests/ openkal-kit's cases (tools/run-kit-tests.sh) +# +# Here `mtest run' builds and runs the first two through `mcpp test' (CI: the +# job `tests', `--set full'). The suite and the kit need an implementation and +# keep their runs in their own stores; `mtest report' reads them together. +[project] +store = ".mtest" + +[host] +kind = "mcpp" + +[report] +sections = ["summary", "kinds", "failures", "slowest", "metrics"] diff --git a/conformance/.gitignore b/tests/cts/.gitignore similarity index 100% rename from conformance/.gitignore rename to tests/cts/.gitignore diff --git a/conformance/README.md b/tests/cts/README.md similarity index 92% rename from conformance/README.md rename to tests/cts/README.md index 5fc1ec5..eb0c989 100644 --- a/conformance/README.md +++ b/tests/cts/README.md @@ -11,7 +11,7 @@ mcpp build --features full The implementation under examination is not named in this package's manifest. It is supplied by whoever runs the suite, exactly as any consumer supplies one — -and `tools/run-conformance.sh` in the parent directory is where that is done, so +and `tools/run-conformance.sh` of the specification is where that is done, so that a change to how it is done is one change: ```bash @@ -24,7 +24,10 @@ This package is not published to the index separately. It is a program rather than a library — a dependency that contributed a binary target to its dependant would be a surprising thing for `[dependencies]` to name — and it is reached by checking out the specification, which is the package it is the conformance -procedure for. +procedure for. It is the specification's `tests/cts`, a package of its own in +mtest's layout, because the implementation it needs comes from outside its +build. It is written without the C++ library, so that it runs where an +implementation has no C library, and speaks the protocol itself. ## A runner's view: the mtest test protocol @@ -32,9 +35,12 @@ The suite speaks the [mtest test protocol](https://github.com/mcpp-community/mte version 1, at level 2, written without a library as the rest of the suite is. Each interface is a case of kind `cts` that names the interface as the clause it covers; each observation is a case within it, reported as it runs (protocol P3.4): `pass`, -`fail`, or `skip` with the reason an observation was not made; each measurement of -cost is a metric in nanoseconds. A selection may name an interface or an -observation within one; the second runs its interface. Every case holds the +`fail`, or `skip` with the reason an observation was not made. The measurements +of an interface's cost (`--features cost`) are a case of their own, of kind +`bench`, named `.cost` and reported after the interface's case, each +measurement a metric in nanoseconds (0.10.0). A selection may name an interface, +an observation within one, or an interface's cost; each runs its interface, and +reports only what it named. Every case holds the resource `serial:openkal-conformance`, because the sections create fixed names in the working directory. @@ -45,7 +51,7 @@ and the run is kept in `.mtest`: ```bash MTEST=mtest bash .spec/tools/run-conformance.sh openkal-linux . full,optional -(cd .spec/conformance && mtest report --format md) +(cd .spec/tests/cts && mtest report --format md) ``` The protocol's variables are not passed to the copies the suite starts: a copy is diff --git a/conformance/abi/interposer.cpp b/tests/cts/abi/interposer.cpp similarity index 100% rename from conformance/abi/interposer.cpp rename to tests/cts/abi/interposer.cpp diff --git a/conformance/abi/probe.c b/tests/cts/abi/probe.c similarity index 100% rename from conformance/abi/probe.c rename to tests/cts/abi/probe.c diff --git a/conformance/mcpp.toml b/tests/cts/mcpp.toml similarity index 99% rename from conformance/mcpp.toml rename to tests/cts/mcpp.toml index b173c7b..ea1924b 100644 --- a/conformance/mcpp.toml +++ b/tests/cts/mcpp.toml @@ -1,7 +1,7 @@ [package] namespace = "mcpplibs" name = "openkal-conformance" -version = "0.9.0" +version = "0.10.0" description = "The behavioural half of clause 9: a suite an implementation of openkal runs against itself, selectable to the interfaces it provides." license = "Apache-2.0" authors = ["mcpplibs"] @@ -18,7 +18,7 @@ repo = "https://github.com/mcpplibs/openkal" # What it depends upon is therefore openkal and the language. The formatting in # okc.report is the price of that, and it is sixty lines. [dependencies] -openkal = "0.25.1" +openkal = "0.25.2" # The implementation under examination is not named here. # diff --git a/conformance/src/atomic.cppm b/tests/cts/src/atomic.cppm similarity index 100% rename from conformance/src/atomic.cppm rename to tests/cts/src/atomic.cppm diff --git a/tests/cts/src/declarations.c b/tests/cts/src/declarations.c new file mode 100644 index 0000000..c3dc13a --- /dev/null +++ b/tests/cts/src/declarations.c @@ -0,0 +1,522 @@ +/* The C declarations, named once, so that every toolchain that builds this + * suite compiles them. + * + * Clause 4.3 requires the two forms of the declarations to declare the same + * entities, and tools/check-declarations.sh performs that comparison against + * SURFACE.txt --- which is normative --- with the environment's own headers + * excluded. That tool needs a compiler driver it can pass -nostdinc to, and one + * of the three toolchains this suite is built with has no such spelling. So the + * comparison is performed in one place and the compiling is performed here, in + * a translation unit the suite carries: a header that fails to compile as C + * under a toolchain then fails that toolchain's build rather than going + * unnoticed until a C consumer meets it. + * + * No name is referenced strongly. The suite is composable --- an implementation + * provides an interface in whole or not at all --- so a file that required every + * definition would fail to link against a conforming implementation of three + * interfaces, which is the arrangement the suite exists to accommodate. sizeof is + * unevaluated, so the first function declares a dependency on nothing while still + * rejecting a name the header fails to declare; the second reaches every name + * weakly, on ELF, so that a link which discards unreached sections keeps what the + * implementation defines (below). + * + * Regenerated by tools/check-declarations.sh --write. It is committed rather + * than generated during the build so that the suite compiles in a checkout that + * has nothing but a compiler. + */ +#include + +void okc_declarations_c(void); +void okc_declarations_c(void) +{ + (void)sizeof(&kal_abort); + (void)sizeof(&kal_alloc); + (void)sizeof(&kal_datagram_close); + (void)sizeof(&kal_datagram_local); + (void)sizeof(&kal_datagram_open); + (void)sizeof(&kal_datagram_props); + (void)sizeof(&kal_datagram_recv_from); + (void)sizeof(&kal_datagram_send_to); + (void)sizeof(&kal_debug_props); + (void)sizeof(&kal_debug_symbolize); + (void)sizeof(&kal_env_arg); + (void)sizeof(&kal_env_arg_count); + (void)sizeof(&kal_env_image); + (void)sizeof(&kal_env_loader); + (void)sizeof(&kal_env_resources); + (void)sizeof(&kal_env_system); + (void)sizeof(&kal_env_var); + (void)sizeof(&kal_env_var_at); + (void)sizeof(&kal_env_var_count); + (void)sizeof(&kal_exec_alloc); + (void)sizeof(&kal_exec_alloc_at); + (void)sizeof(&kal_exec_free); + (void)sizeof(&kal_exec_granularity); + (void)sizeof(&kal_exec_map_part); + (void)sizeof(&kal_exec_props); + (void)sizeof(&kal_exec_publish); + (void)sizeof(&kal_exec_publish_part); + (void)sizeof(&kal_exec_seal_part); + (void)sizeof(&kal_exit); + (void)sizeof(&kal_free); + (void)sizeof(&kal_fs_capacity); + (void)sizeof(&kal_fs_close_dir); + (void)sizeof(&kal_fs_close_file); + (void)sizeof(&kal_fs_dir_attenuate); + (void)sizeof(&kal_fs_file_info); + (void)sizeof(&kal_fs_hard_link); + (void)sizeof(&kal_fs_info); + (void)sizeof(&kal_fs_link_create); + (void)sizeof(&kal_fs_link_read); + (void)sizeof(&kal_fs_list_begin); + (void)sizeof(&kal_fs_list_next); + (void)sizeof(&kal_fs_list_next_info); + (void)sizeof(&kal_fs_lock); + (void)sizeof(&kal_fs_max_name); + (void)sizeof(&kal_fs_mkdir); + (void)sizeof(&kal_fs_open); + (void)sizeof(&kal_fs_open_dir); + (void)sizeof(&kal_fs_preopen); + (void)sizeof(&kal_fs_preopen_count); + (void)sizeof(&kal_fs_props); + (void)sizeof(&kal_fs_read_at); + (void)sizeof(&kal_fs_remove); + (void)sizeof(&kal_fs_rename); + (void)sizeof(&kal_fs_rename_flags); + (void)sizeof(&kal_fs_resolved_name); + (void)sizeof(&kal_fs_seek); + (void)sizeof(&kal_fs_set_executable_at); + (void)sizeof(&kal_fs_set_modified); + (void)sizeof(&kal_fs_set_modified_at); + (void)sizeof(&kal_fs_stream); + (void)sizeof(&kal_fs_truncate); + (void)sizeof(&kal_fs_unlock); + (void)sizeof(&kal_fs_write_at); + (void)sizeof(&kal_interfaces); + (void)sizeof(&kal_map_file); + (void)sizeof(&kal_map_release); + (void)sizeof(&kal_map_sync); + (void)sizeof(&kal_memory_granularity); + (void)sizeof(&kal_memory_info); + (void)sizeof(&kal_net_accept); + (void)sizeof(&kal_net_close); + (void)sizeof(&kal_net_close_listener); + (void)sizeof(&kal_net_connect); + (void)sizeof(&kal_net_connect_at); + (void)sizeof(&kal_net_listen); + (void)sizeof(&kal_net_listen_at); + (void)sizeof(&kal_net_listener_local); + (void)sizeof(&kal_net_local); + (void)sizeof(&kal_net_peer); + (void)sizeof(&kal_net_props); + (void)sizeof(&kal_net_shutdown); + (void)sizeof(&kal_net_stream); + (void)sizeof(&kal_process_channel); + (void)sizeof(&kal_process_channel_close); + (void)sizeof(&kal_process_close); + (void)sizeof(&kal_process_end_as_requested); + (void)sizeof(&kal_process_info); + (void)sizeof(&kal_process_job_close); + (void)sizeof(&kal_process_job_create); + (void)sizeof(&kal_process_job_enter); + (void)sizeof(&kal_process_job_limit); + (void)sizeof(&kal_process_job_observe); + (void)sizeof(&kal_process_job_open); + (void)sizeof(&kal_process_job_retire); + (void)sizeof(&kal_process_job_terminate); + (void)sizeof(&kal_process_job_usage); + (void)sizeof(&kal_process_open_id); + (void)sizeof(&kal_process_props); + (void)sizeof(&kal_process_self); + (void)sizeof(&kal_process_spawn); + (void)sizeof(&kal_process_stop_requested); + (void)sizeof(&kal_process_terminate); + (void)sizeof(&kal_process_wait); + (void)sizeof(&kal_prof_active); + (void)sizeof(&kal_prof_counter); + (void)sizeof(&kal_prof_mark); + (void)sizeof(&kal_prof_props); + (void)sizeof(&kal_prof_sample_begin); + (void)sizeof(&kal_prof_sample_end); + (void)sizeof(&kal_prof_sample_take); + (void)sizeof(&kal_prof_touch_begin); + (void)sizeof(&kal_prof_touch_end); + (void)sizeof(&kal_prof_touch_take); + (void)sizeof(&kal_random_fill); + (void)sizeof(&kal_random_props); + (void)sizeof(&kal_region_commit); + (void)sizeof(&kal_region_decommit); + (void)sizeof(&kal_region_discard); + (void)sizeof(&kal_region_props); + (void)sizeof(&kal_region_release); + (void)sizeof(&kal_region_reserve); + (void)sizeof(&kal_space_props); + (void)sizeof(&kal_space_start); + (void)sizeof(&kal_stderr); + (void)sizeof(&kal_stdin); + (void)sizeof(&kal_stdout); + (void)sizeof(&kal_stream_flush); + (void)sizeof(&kal_stream_props); + (void)sizeof(&kal_stream_read); + (void)sizeof(&kal_stream_write); + (void)sizeof(&kal_task_cores); + (void)sizeof(&kal_task_current); + (void)sizeof(&kal_task_join); + (void)sizeof(&kal_task_parallelism); + (void)sizeof(&kal_task_props); + (void)sizeof(&kal_task_set_priority); + (void)sizeof(&kal_task_stack); + (void)sizeof(&kal_task_start); + (void)sizeof(&kal_task_start_sized); + (void)sizeof(&kal_task_wait); + (void)sizeof(&kal_task_wake); + (void)sizeof(&kal_task_yield); + (void)sizeof(&kal_terminal_get_mode); + (void)sizeof(&kal_terminal_props); + (void)sizeof(&kal_terminal_set_mode); + (void)sizeof(&kal_terminal_size); + (void)sizeof(&kal_time_monotonic); + (void)sizeof(&kal_time_monotonic_granularity); + (void)sizeof(&kal_timeout_accept); + (void)sizeof(&kal_timeout_granularity); + (void)sizeof(&kal_timeout_read); + (void)sizeof(&kal_timeout_recv_from); + (void)sizeof(&kal_timeout_wait_process); + (void)sizeof(&kal_timeout_write); + (void)sizeof(&kal_time_props); + (void)sizeof(&kal_time_sleep); + (void)sizeof(&kal_time_wall); + (void)sizeof(&kal_version); +} + +/* Every name, reached without being required (0.25.2): a link that discards + * what nothing reaches keeps the names the implementation defines, for the + * comparison of clause 9, and an absent name is null. A weak reference to a + * name defined nowhere is accepted by the static link of ELF only. */ +#if defined(__ELF__) +#pragma weak kal_abort +#pragma weak kal_alloc +#pragma weak kal_datagram_close +#pragma weak kal_datagram_local +#pragma weak kal_datagram_open +#pragma weak kal_datagram_props +#pragma weak kal_datagram_recv_from +#pragma weak kal_datagram_send_to +#pragma weak kal_debug_props +#pragma weak kal_debug_symbolize +#pragma weak kal_env_arg +#pragma weak kal_env_arg_count +#pragma weak kal_env_image +#pragma weak kal_env_loader +#pragma weak kal_env_resources +#pragma weak kal_env_system +#pragma weak kal_env_var +#pragma weak kal_env_var_at +#pragma weak kal_env_var_count +#pragma weak kal_exec_alloc +#pragma weak kal_exec_alloc_at +#pragma weak kal_exec_free +#pragma weak kal_exec_granularity +#pragma weak kal_exec_map_part +#pragma weak kal_exec_props +#pragma weak kal_exec_publish +#pragma weak kal_exec_publish_part +#pragma weak kal_exec_seal_part +#pragma weak kal_exit +#pragma weak kal_free +#pragma weak kal_fs_capacity +#pragma weak kal_fs_close_dir +#pragma weak kal_fs_close_file +#pragma weak kal_fs_dir_attenuate +#pragma weak kal_fs_file_info +#pragma weak kal_fs_hard_link +#pragma weak kal_fs_info +#pragma weak kal_fs_link_create +#pragma weak kal_fs_link_read +#pragma weak kal_fs_list_begin +#pragma weak kal_fs_list_next +#pragma weak kal_fs_list_next_info +#pragma weak kal_fs_lock +#pragma weak kal_fs_max_name +#pragma weak kal_fs_mkdir +#pragma weak kal_fs_open +#pragma weak kal_fs_open_dir +#pragma weak kal_fs_preopen +#pragma weak kal_fs_preopen_count +#pragma weak kal_fs_props +#pragma weak kal_fs_read_at +#pragma weak kal_fs_remove +#pragma weak kal_fs_rename +#pragma weak kal_fs_rename_flags +#pragma weak kal_fs_resolved_name +#pragma weak kal_fs_seek +#pragma weak kal_fs_set_executable_at +#pragma weak kal_fs_set_modified +#pragma weak kal_fs_set_modified_at +#pragma weak kal_fs_stream +#pragma weak kal_fs_truncate +#pragma weak kal_fs_unlock +#pragma weak kal_fs_write_at +#pragma weak kal_interfaces +#pragma weak kal_map_file +#pragma weak kal_map_release +#pragma weak kal_map_sync +#pragma weak kal_memory_granularity +#pragma weak kal_memory_info +#pragma weak kal_net_accept +#pragma weak kal_net_close +#pragma weak kal_net_close_listener +#pragma weak kal_net_connect +#pragma weak kal_net_connect_at +#pragma weak kal_net_listen +#pragma weak kal_net_listen_at +#pragma weak kal_net_listener_local +#pragma weak kal_net_local +#pragma weak kal_net_peer +#pragma weak kal_net_props +#pragma weak kal_net_shutdown +#pragma weak kal_net_stream +#pragma weak kal_process_channel +#pragma weak kal_process_channel_close +#pragma weak kal_process_close +#pragma weak kal_process_end_as_requested +#pragma weak kal_process_info +#pragma weak kal_process_job_close +#pragma weak kal_process_job_create +#pragma weak kal_process_job_enter +#pragma weak kal_process_job_limit +#pragma weak kal_process_job_observe +#pragma weak kal_process_job_open +#pragma weak kal_process_job_retire +#pragma weak kal_process_job_terminate +#pragma weak kal_process_job_usage +#pragma weak kal_process_open_id +#pragma weak kal_process_props +#pragma weak kal_process_self +#pragma weak kal_process_spawn +#pragma weak kal_process_stop_requested +#pragma weak kal_process_terminate +#pragma weak kal_process_wait +#pragma weak kal_prof_active +#pragma weak kal_prof_counter +#pragma weak kal_prof_mark +#pragma weak kal_prof_props +#pragma weak kal_prof_sample_begin +#pragma weak kal_prof_sample_end +#pragma weak kal_prof_sample_take +#pragma weak kal_prof_touch_begin +#pragma weak kal_prof_touch_end +#pragma weak kal_prof_touch_take +#pragma weak kal_random_fill +#pragma weak kal_random_props +#pragma weak kal_region_commit +#pragma weak kal_region_decommit +#pragma weak kal_region_discard +#pragma weak kal_region_props +#pragma weak kal_region_release +#pragma weak kal_region_reserve +#pragma weak kal_space_props +#pragma weak kal_space_start +#pragma weak kal_stderr +#pragma weak kal_stdin +#pragma weak kal_stdout +#pragma weak kal_stream_flush +#pragma weak kal_stream_props +#pragma weak kal_stream_read +#pragma weak kal_stream_write +#pragma weak kal_task_cores +#pragma weak kal_task_current +#pragma weak kal_task_join +#pragma weak kal_task_parallelism +#pragma weak kal_task_props +#pragma weak kal_task_set_priority +#pragma weak kal_task_stack +#pragma weak kal_task_start +#pragma weak kal_task_start_sized +#pragma weak kal_task_wait +#pragma weak kal_task_wake +#pragma weak kal_task_yield +#pragma weak kal_terminal_get_mode +#pragma weak kal_terminal_props +#pragma weak kal_terminal_set_mode +#pragma weak kal_terminal_size +#pragma weak kal_time_monotonic +#pragma weak kal_time_monotonic_granularity +#pragma weak kal_timeout_accept +#pragma weak kal_timeout_granularity +#pragma weak kal_timeout_read +#pragma weak kal_timeout_recv_from +#pragma weak kal_timeout_wait_process +#pragma weak kal_timeout_write +#pragma weak kal_time_props +#pragma weak kal_time_sleep +#pragma weak kal_time_wall +#pragma weak kal_version +#endif + +unsigned long okc_names_reached(void); +unsigned long okc_names_reached(void) +{ + unsigned long n = 0; +#if defined(__ELF__) + n += (const void*)&kal_abort != 0; + n += (const void*)&kal_alloc != 0; + n += (const void*)&kal_datagram_close != 0; + n += (const void*)&kal_datagram_local != 0; + n += (const void*)&kal_datagram_open != 0; + n += (const void*)&kal_datagram_props != 0; + n += (const void*)&kal_datagram_recv_from != 0; + n += (const void*)&kal_datagram_send_to != 0; + n += (const void*)&kal_debug_props != 0; + n += (const void*)&kal_debug_symbolize != 0; + n += (const void*)&kal_env_arg != 0; + n += (const void*)&kal_env_arg_count != 0; + n += (const void*)&kal_env_image != 0; + n += (const void*)&kal_env_loader != 0; + n += (const void*)&kal_env_resources != 0; + n += (const void*)&kal_env_system != 0; + n += (const void*)&kal_env_var != 0; + n += (const void*)&kal_env_var_at != 0; + n += (const void*)&kal_env_var_count != 0; + n += (const void*)&kal_exec_alloc != 0; + n += (const void*)&kal_exec_alloc_at != 0; + n += (const void*)&kal_exec_free != 0; + n += (const void*)&kal_exec_granularity != 0; + n += (const void*)&kal_exec_map_part != 0; + n += (const void*)&kal_exec_props != 0; + n += (const void*)&kal_exec_publish != 0; + n += (const void*)&kal_exec_publish_part != 0; + n += (const void*)&kal_exec_seal_part != 0; + n += (const void*)&kal_exit != 0; + n += (const void*)&kal_free != 0; + n += (const void*)&kal_fs_capacity != 0; + n += (const void*)&kal_fs_close_dir != 0; + n += (const void*)&kal_fs_close_file != 0; + n += (const void*)&kal_fs_dir_attenuate != 0; + n += (const void*)&kal_fs_file_info != 0; + n += (const void*)&kal_fs_hard_link != 0; + n += (const void*)&kal_fs_info != 0; + n += (const void*)&kal_fs_link_create != 0; + n += (const void*)&kal_fs_link_read != 0; + n += (const void*)&kal_fs_list_begin != 0; + n += (const void*)&kal_fs_list_next != 0; + n += (const void*)&kal_fs_list_next_info != 0; + n += (const void*)&kal_fs_lock != 0; + n += (const void*)&kal_fs_max_name != 0; + n += (const void*)&kal_fs_mkdir != 0; + n += (const void*)&kal_fs_open != 0; + n += (const void*)&kal_fs_open_dir != 0; + n += (const void*)&kal_fs_preopen != 0; + n += (const void*)&kal_fs_preopen_count != 0; + n += (const void*)&kal_fs_props != 0; + n += (const void*)&kal_fs_read_at != 0; + n += (const void*)&kal_fs_remove != 0; + n += (const void*)&kal_fs_rename != 0; + n += (const void*)&kal_fs_rename_flags != 0; + n += (const void*)&kal_fs_resolved_name != 0; + n += (const void*)&kal_fs_seek != 0; + n += (const void*)&kal_fs_set_executable_at != 0; + n += (const void*)&kal_fs_set_modified != 0; + n += (const void*)&kal_fs_set_modified_at != 0; + n += (const void*)&kal_fs_stream != 0; + n += (const void*)&kal_fs_truncate != 0; + n += (const void*)&kal_fs_unlock != 0; + n += (const void*)&kal_fs_write_at != 0; + n += (const void*)&kal_interfaces != 0; + n += (const void*)&kal_map_file != 0; + n += (const void*)&kal_map_release != 0; + n += (const void*)&kal_map_sync != 0; + n += (const void*)&kal_memory_granularity != 0; + n += (const void*)&kal_memory_info != 0; + n += (const void*)&kal_net_accept != 0; + n += (const void*)&kal_net_close != 0; + n += (const void*)&kal_net_close_listener != 0; + n += (const void*)&kal_net_connect != 0; + n += (const void*)&kal_net_connect_at != 0; + n += (const void*)&kal_net_listen != 0; + n += (const void*)&kal_net_listen_at != 0; + n += (const void*)&kal_net_listener_local != 0; + n += (const void*)&kal_net_local != 0; + n += (const void*)&kal_net_peer != 0; + n += (const void*)&kal_net_props != 0; + n += (const void*)&kal_net_shutdown != 0; + n += (const void*)&kal_net_stream != 0; + n += (const void*)&kal_process_channel != 0; + n += (const void*)&kal_process_channel_close != 0; + n += (const void*)&kal_process_close != 0; + n += (const void*)&kal_process_end_as_requested != 0; + n += (const void*)&kal_process_info != 0; + n += (const void*)&kal_process_job_close != 0; + n += (const void*)&kal_process_job_create != 0; + n += (const void*)&kal_process_job_enter != 0; + n += (const void*)&kal_process_job_limit != 0; + n += (const void*)&kal_process_job_observe != 0; + n += (const void*)&kal_process_job_open != 0; + n += (const void*)&kal_process_job_retire != 0; + n += (const void*)&kal_process_job_terminate != 0; + n += (const void*)&kal_process_job_usage != 0; + n += (const void*)&kal_process_open_id != 0; + n += (const void*)&kal_process_props != 0; + n += (const void*)&kal_process_self != 0; + n += (const void*)&kal_process_spawn != 0; + n += (const void*)&kal_process_stop_requested != 0; + n += (const void*)&kal_process_terminate != 0; + n += (const void*)&kal_process_wait != 0; + n += (const void*)&kal_prof_active != 0; + n += (const void*)&kal_prof_counter != 0; + n += (const void*)&kal_prof_mark != 0; + n += (const void*)&kal_prof_props != 0; + n += (const void*)&kal_prof_sample_begin != 0; + n += (const void*)&kal_prof_sample_end != 0; + n += (const void*)&kal_prof_sample_take != 0; + n += (const void*)&kal_prof_touch_begin != 0; + n += (const void*)&kal_prof_touch_end != 0; + n += (const void*)&kal_prof_touch_take != 0; + n += (const void*)&kal_random_fill != 0; + n += (const void*)&kal_random_props != 0; + n += (const void*)&kal_region_commit != 0; + n += (const void*)&kal_region_decommit != 0; + n += (const void*)&kal_region_discard != 0; + n += (const void*)&kal_region_props != 0; + n += (const void*)&kal_region_release != 0; + n += (const void*)&kal_region_reserve != 0; + n += (const void*)&kal_space_props != 0; + n += (const void*)&kal_space_start != 0; + n += (const void*)&kal_stderr != 0; + n += (const void*)&kal_stdin != 0; + n += (const void*)&kal_stdout != 0; + n += (const void*)&kal_stream_flush != 0; + n += (const void*)&kal_stream_props != 0; + n += (const void*)&kal_stream_read != 0; + n += (const void*)&kal_stream_write != 0; + n += (const void*)&kal_task_cores != 0; + n += (const void*)&kal_task_current != 0; + n += (const void*)&kal_task_join != 0; + n += (const void*)&kal_task_parallelism != 0; + n += (const void*)&kal_task_props != 0; + n += (const void*)&kal_task_set_priority != 0; + n += (const void*)&kal_task_stack != 0; + n += (const void*)&kal_task_start != 0; + n += (const void*)&kal_task_start_sized != 0; + n += (const void*)&kal_task_wait != 0; + n += (const void*)&kal_task_wake != 0; + n += (const void*)&kal_task_yield != 0; + n += (const void*)&kal_terminal_get_mode != 0; + n += (const void*)&kal_terminal_props != 0; + n += (const void*)&kal_terminal_set_mode != 0; + n += (const void*)&kal_terminal_size != 0; + n += (const void*)&kal_time_monotonic != 0; + n += (const void*)&kal_time_monotonic_granularity != 0; + n += (const void*)&kal_timeout_accept != 0; + n += (const void*)&kal_timeout_granularity != 0; + n += (const void*)&kal_timeout_read != 0; + n += (const void*)&kal_timeout_recv_from != 0; + n += (const void*)&kal_timeout_wait_process != 0; + n += (const void*)&kal_timeout_write != 0; + n += (const void*)&kal_time_props != 0; + n += (const void*)&kal_time_sleep != 0; + n += (const void*)&kal_time_wall != 0; + n += (const void*)&kal_version != 0; +#endif + return n; +} diff --git a/conformance/src/macros_reach.cpp b/tests/cts/src/macros_reach.cpp similarity index 100% rename from conformance/src/macros_reach.cpp rename to tests/cts/src/macros_reach.cpp diff --git a/conformance/src/main.cpp b/tests/cts/src/main.cpp similarity index 93% rename from conformance/src/main.cpp rename to tests/cts/src/main.cpp index bb3d8ae..bde76f6 100644 --- a/conformance/src/main.cpp +++ b/tests/cts/src/main.cpp @@ -45,7 +45,13 @@ import okc.child; // two spellings are the same symbol; and clause 9's behavioural half is // required of every implementation, including one of a machine with no // operating system, so this program has to be linkable in both arrangements. +// Every name of the surface, reached without being required (declarations.c). +extern "C" unsigned long okc_names_reached(void); +volatile unsigned long okc_names_present; + extern "C" int main() { + okc_names_present = okc_names_reached(); + // Two of the specification's requirements end the program that satisfies // them, so they are observed in a copy this program starts. A copy is told // what to do through its argument vector and answers through its status. diff --git a/conformance/src/protocol.cpp b/tests/cts/src/protocol.cpp similarity index 82% rename from conformance/src/protocol.cpp rename to tests/cts/src/protocol.cpp index b465669..5ab3859 100644 --- a/conformance/src/protocol.cpp +++ b/tests/cts/src/protocol.cpp @@ -39,6 +39,16 @@ bool g_matched[kMaxSelected]; // that two observations of one text are two identifiers (P3.2). char g_case[64]; bool g_case_open = false; +// THE COST OF AN INTERFACE IS ITS OWN CASE (0.10.0): kind `bench', id +// `.cost', reported after the interface's `cts' case with the +// measurements made in it, so that a runner reads, selects and gates the two +// apart. Found only by running, as the observations are, and listed by none. +bool g_cts_wanted = true; +bool g_bench_wanted = true; +struct measurement { char name[128]; kal_u64 ns; }; +constexpr int kMaxMeasurements = 32; +measurement g_measured[kMaxMeasurements]; +int g_measured_n = 0; kal_u64 g_case_began = 0; bool g_case_failed = false; int g_case_parts = 0; @@ -130,9 +140,29 @@ void end_event(const char* id, const char* status, const char* reason, kal_u64 m e.raw("}"); e.send(); } +void bench_id(const char* id, char* out, kal_uintptr cap) { + const char suffix[] = ".cost"; + kal_uintptr n = 0; + for (; id[n] && n + sizeof suffix < cap; ++n) out[n] = id[n]; + for (kal_uintptr i = 0; i < sizeof suffix; ++i) out[n + i] = suffix[i]; +} + void close_case() { if (!g_case_open) return; - end_event(g_case, g_case_failed ? "fail" : "pass", nullptr, elapsed_ms() - g_case_began); + if (g_cts_wanted) + end_event(g_case, g_case_failed ? "fail" : "pass", nullptr, elapsed_ms() - g_case_began); + if (g_bench_wanted && g_measured_n > 0) { + char id[80]; bench_id(g_case, id, sizeof id); + line b; b.raw("{\"e\":\"begin\",\"id\":"); b.str(id); b.raw(",\"t\":"); b.num(elapsed_ms()); + b.raw(",\"kind\":\"bench\",\"clauses\":["); b.str(g_case); + b.raw("],\"resources\":[\"serial:openkal-conformance\"]}"); b.send(); + for (int i = 0; i < g_measured_n; ++i) { + line e; e.raw("{\"e\":\"metric\",\"id\":"); e.str(id); e.raw(",\"name\":"); e.str(g_measured[i].name); + e.raw(",\"value\":"); e.num(g_measured[i].ns); e.raw(",\"unit\":\"ns\",\"better\":\"lower\"}"); e.send(); + } + end_event(id, "pass", nullptr); + } + g_measured_n = 0; g_case_open = false; } @@ -213,7 +243,7 @@ bool begin() { line e; e.raw("{\"e\":\"program\",\"protocol\":1,\"level\":2,\"framework\":\"openkal-conformance\"," - "\"version\":\"0.9.0\",\"isolation\":[\"process\"]}"); + "\"version\":\"0.10.0\",\"isolation\":[\"process\"]}"); e.send(); return true; #else @@ -224,7 +254,7 @@ bool begin() { bool active() { return g_active; } bool listing() { return g_listing; } -bool wanted(const char* id) { +bool wanted_one(const char* id) { if (!g_selecting) return true; // A line names the section's case, or a part within it (`/'): // selecting a part runs the case that declares it (protocol P3.4). @@ -255,24 +285,39 @@ bool wanted(const char* id) { // and remove fixed names in the working directory they were given, so two // copies of this program running at once would observe each other's files; a // runner that divides the cases among workers runs them one at a time. -void list_case(const char* id) { +bool wanted(const char* id) { + char b[80]; bench_id(id, b, sizeof b); + const bool cts = wanted_one(id); + const bool bench = wanted_one(b); + return cts || bench; +} + +void list_case(const char* id, bool bench) { line e; e.raw("{\"e\":\"case\",\"id\":"); e.str(id); e.raw(",\"name\":"); e.str(id); e.raw(",\"kind\":\"cts\",\"clauses\":["); e.str(id); e.raw("],\"resources\":[\"serial:openkal-conformance\"]}"); e.send(); + if (!bench) return; + char b[80]; bench_id(id, b, sizeof b); + line c; c.raw("{\"e\":\"case\",\"id\":"); c.str(b); c.raw(",\"name\":"); c.str(b); + c.raw(",\"kind\":\"bench\",\"clauses\":["); c.str(id); c.raw("],\"resources\":[\"serial:openkal-conformance\"]}"); c.send(); } void section(const char* heading_text) { if (!g_active) return; close_case(); id_of(heading_text, g_case, sizeof g_case); + char b[80]; bench_id(g_case, b, sizeof b); + g_cts_wanted = wanted_one(g_case); + g_bench_wanted = wanted_one(b); g_case_open = true; g_case_failed = false; g_case_parts = 0; + g_measured_n = 0; g_case_began = elapsed_ms(); - begin_event(g_case); + if (g_cts_wanted) begin_event(g_case); } void observation(kind k, const char* what, bool held, const char* because) { - if (!g_active || !g_case_open) return; + if (!g_active || !g_case_open || !g_cts_wanted) return; // The part's identifier: the case's, the observation's text, and its // ordinal where the text recurs in the case. char id[1024]; kal_uintptr n = 0; @@ -308,9 +353,12 @@ void observation(kind k, const char* what, bool held, const char* because) { } void metric(const char* what, kal_u64 ns) { - if (!g_active || !g_case_open) return; - line e; e.raw("{\"e\":\"metric\",\"id\":"); e.str(g_case); e.raw(",\"name\":"); e.str(what); - e.raw(",\"value\":"); e.num(ns); e.raw(",\"unit\":\"ns\",\"better\":\"lower\"}"); e.send(); + if (!g_active || !g_case_open || !g_bench_wanted || g_measured_n >= kMaxMeasurements) return; + measurement& m = g_measured[g_measured_n++]; + kal_uintptr n = 0; + for (; what && what[n] && n + 1 < sizeof m.name; ++n) m.name[n] = what[n]; + m.name[n] = 0; + m.ns = ns; } int finish(int held, int failed, int unobserved) { diff --git a/conformance/src/protocol.cppm b/tests/cts/src/protocol.cppm similarity index 94% rename from conformance/src/protocol.cppm rename to tests/cts/src/protocol.cppm index d578ac0..fa6a15a 100644 --- a/conformance/src/protocol.cppm +++ b/tests/cts/src/protocol.cppm @@ -31,7 +31,8 @@ bool listing(); // MTEST_MODE=list: list and run nothing // no selection was given or the protocol is not spoken. bool wanted(const char* id); -void list_case(const char* id); +// The interface's `cts' case, and its `bench' case `.cost' when `bench'. +void list_case(const char* id, bool bench); void section(const char* heading_text); // ends the previous case, begins the next void observation(kind k, const char* what, bool held, const char* because); void metric(const char* what, kal_u64 ns); diff --git a/conformance/src/report.cpp b/tests/cts/src/report.cpp similarity index 100% rename from conformance/src/report.cpp rename to tests/cts/src/report.cpp diff --git a/conformance/src/report.cppm b/tests/cts/src/report.cppm similarity index 100% rename from conformance/src/report.cppm rename to tests/cts/src/report.cppm diff --git a/conformance/src/sections/abort.cpp b/tests/cts/src/sections/abort.cpp similarity index 100% rename from conformance/src/sections/abort.cpp rename to tests/cts/src/sections/abort.cpp diff --git a/conformance/src/sections/abort.cppm b/tests/cts/src/sections/abort.cppm similarity index 100% rename from conformance/src/sections/abort.cppm rename to tests/cts/src/sections/abort.cppm diff --git a/conformance/src/sections/child.cpp b/tests/cts/src/sections/child.cpp similarity index 100% rename from conformance/src/sections/child.cpp rename to tests/cts/src/sections/child.cpp diff --git a/conformance/src/sections/child.cppm b/tests/cts/src/sections/child.cppm similarity index 100% rename from conformance/src/sections/child.cppm rename to tests/cts/src/sections/child.cppm diff --git a/conformance/src/sections/datagram.cpp b/tests/cts/src/sections/datagram.cpp similarity index 100% rename from conformance/src/sections/datagram.cpp rename to tests/cts/src/sections/datagram.cpp diff --git a/conformance/src/sections/datagram.cppm b/tests/cts/src/sections/datagram.cppm similarity index 100% rename from conformance/src/sections/datagram.cppm rename to tests/cts/src/sections/datagram.cppm diff --git a/conformance/src/sections/debug.cpp b/tests/cts/src/sections/debug.cpp similarity index 100% rename from conformance/src/sections/debug.cpp rename to tests/cts/src/sections/debug.cpp diff --git a/conformance/src/sections/debug.cppm b/tests/cts/src/sections/debug.cppm similarity index 100% rename from conformance/src/sections/debug.cppm rename to tests/cts/src/sections/debug.cppm diff --git a/conformance/src/sections/env.cpp b/tests/cts/src/sections/env.cpp similarity index 100% rename from conformance/src/sections/env.cpp rename to tests/cts/src/sections/env.cpp diff --git a/conformance/src/sections/env.cppm b/tests/cts/src/sections/env.cppm similarity index 100% rename from conformance/src/sections/env.cppm rename to tests/cts/src/sections/env.cppm diff --git a/conformance/src/sections/exec.cpp b/tests/cts/src/sections/exec.cpp similarity index 100% rename from conformance/src/sections/exec.cpp rename to tests/cts/src/sections/exec.cpp diff --git a/conformance/src/sections/exec.cppm b/tests/cts/src/sections/exec.cppm similarity index 100% rename from conformance/src/sections/exec.cppm rename to tests/cts/src/sections/exec.cppm diff --git a/conformance/src/sections/fs.cpp b/tests/cts/src/sections/fs.cpp similarity index 100% rename from conformance/src/sections/fs.cpp rename to tests/cts/src/sections/fs.cpp diff --git a/conformance/src/sections/fs.cppm b/tests/cts/src/sections/fs.cppm similarity index 100% rename from conformance/src/sections/fs.cppm rename to tests/cts/src/sections/fs.cppm diff --git a/conformance/src/sections/map.cpp b/tests/cts/src/sections/map.cpp similarity index 100% rename from conformance/src/sections/map.cpp rename to tests/cts/src/sections/map.cpp diff --git a/conformance/src/sections/map.cppm b/tests/cts/src/sections/map.cppm similarity index 100% rename from conformance/src/sections/map.cppm rename to tests/cts/src/sections/map.cppm diff --git a/conformance/src/sections/memory.cpp b/tests/cts/src/sections/memory.cpp similarity index 100% rename from conformance/src/sections/memory.cpp rename to tests/cts/src/sections/memory.cpp diff --git a/conformance/src/sections/memory.cppm b/tests/cts/src/sections/memory.cppm similarity index 100% rename from conformance/src/sections/memory.cppm rename to tests/cts/src/sections/memory.cppm diff --git a/conformance/src/sections/net.cpp b/tests/cts/src/sections/net.cpp similarity index 100% rename from conformance/src/sections/net.cpp rename to tests/cts/src/sections/net.cpp diff --git a/conformance/src/sections/net.cppm b/tests/cts/src/sections/net.cppm similarity index 100% rename from conformance/src/sections/net.cppm rename to tests/cts/src/sections/net.cppm diff --git a/conformance/src/sections/process.cpp b/tests/cts/src/sections/process.cpp similarity index 100% rename from conformance/src/sections/process.cpp rename to tests/cts/src/sections/process.cpp diff --git a/conformance/src/sections/process.cppm b/tests/cts/src/sections/process.cppm similarity index 100% rename from conformance/src/sections/process.cppm rename to tests/cts/src/sections/process.cppm diff --git a/conformance/src/sections/prof.cpp b/tests/cts/src/sections/prof.cpp similarity index 100% rename from conformance/src/sections/prof.cpp rename to tests/cts/src/sections/prof.cpp diff --git a/conformance/src/sections/prof.cppm b/tests/cts/src/sections/prof.cppm similarity index 100% rename from conformance/src/sections/prof.cppm rename to tests/cts/src/sections/prof.cppm diff --git a/conformance/src/sections/random.cpp b/tests/cts/src/sections/random.cpp similarity index 100% rename from conformance/src/sections/random.cpp rename to tests/cts/src/sections/random.cpp diff --git a/conformance/src/sections/random.cppm b/tests/cts/src/sections/random.cppm similarity index 100% rename from conformance/src/sections/random.cppm rename to tests/cts/src/sections/random.cppm diff --git a/conformance/src/sections/region.cpp b/tests/cts/src/sections/region.cpp similarity index 100% rename from conformance/src/sections/region.cpp rename to tests/cts/src/sections/region.cpp diff --git a/conformance/src/sections/region.cppm b/tests/cts/src/sections/region.cppm similarity index 100% rename from conformance/src/sections/region.cppm rename to tests/cts/src/sections/region.cppm diff --git a/conformance/src/sections/space.cpp b/tests/cts/src/sections/space.cpp similarity index 100% rename from conformance/src/sections/space.cpp rename to tests/cts/src/sections/space.cpp diff --git a/conformance/src/sections/space.cppm b/tests/cts/src/sections/space.cppm similarity index 100% rename from conformance/src/sections/space.cppm rename to tests/cts/src/sections/space.cppm diff --git a/conformance/src/sections/stream.cpp b/tests/cts/src/sections/stream.cpp similarity index 100% rename from conformance/src/sections/stream.cpp rename to tests/cts/src/sections/stream.cpp diff --git a/conformance/src/sections/stream.cppm b/tests/cts/src/sections/stream.cppm similarity index 100% rename from conformance/src/sections/stream.cppm rename to tests/cts/src/sections/stream.cppm diff --git a/conformance/src/sections/task.cpp b/tests/cts/src/sections/task.cpp similarity index 100% rename from conformance/src/sections/task.cpp rename to tests/cts/src/sections/task.cpp diff --git a/conformance/src/sections/task.cppm b/tests/cts/src/sections/task.cppm similarity index 100% rename from conformance/src/sections/task.cppm rename to tests/cts/src/sections/task.cppm diff --git a/conformance/src/sections/terminal.cpp b/tests/cts/src/sections/terminal.cpp similarity index 100% rename from conformance/src/sections/terminal.cpp rename to tests/cts/src/sections/terminal.cpp diff --git a/conformance/src/sections/terminal.cppm b/tests/cts/src/sections/terminal.cppm similarity index 100% rename from conformance/src/sections/terminal.cppm rename to tests/cts/src/sections/terminal.cppm diff --git a/conformance/src/sections/time.cpp b/tests/cts/src/sections/time.cpp similarity index 100% rename from conformance/src/sections/time.cpp rename to tests/cts/src/sections/time.cpp diff --git a/conformance/src/sections/time.cppm b/tests/cts/src/sections/time.cppm similarity index 100% rename from conformance/src/sections/time.cppm rename to tests/cts/src/sections/time.cppm diff --git a/conformance/src/sections/timeout.cpp b/tests/cts/src/sections/timeout.cpp similarity index 100% rename from conformance/src/sections/timeout.cpp rename to tests/cts/src/sections/timeout.cpp diff --git a/conformance/src/sections/timeout.cppm b/tests/cts/src/sections/timeout.cppm similarity index 100% rename from conformance/src/sections/timeout.cppm rename to tests/cts/src/sections/timeout.cppm diff --git a/conformance/src/sections/version.cpp b/tests/cts/src/sections/version.cpp similarity index 100% rename from conformance/src/sections/version.cpp rename to tests/cts/src/sections/version.cpp diff --git a/conformance/src/sections/version.cppm b/tests/cts/src/sections/version.cppm similarity index 100% rename from conformance/src/sections/version.cppm rename to tests/cts/src/sections/version.cppm diff --git a/conformance/src/spec.cppm b/tests/cts/src/spec.cppm similarity index 100% rename from conformance/src/spec.cppm rename to tests/cts/src/spec.cppm diff --git a/tests/cts/src/suite.cpp b/tests/cts/src/suite.cpp new file mode 100644 index 0000000..98ee44b --- /dev/null +++ b/tests/cts/src/suite.cpp @@ -0,0 +1,75 @@ +module okc.suite; + +import okc.spec; +import okc.report; +import okc.protocol; +import okc.version; +import okc.abort; +import okc.stream; +import okc.memory; +import okc.env; +import okc.time; +import okc.random; +import okc.fs; +import okc.process; +import okc.task; +import okc.exec; +import okc.terminal; +import okc.net; +import okc.datagram; +import okc.space; +import okc.timeout; +import okc.map; +import okc.region; +import okc.prof; +import okc.debug; + +namespace okc { + +namespace { + +// The cases a runner lists and selects (the mtest protocol): one per section, +// in the order the sections run. A section's case begins at its heading. +// `measures': the section reports its cost (kind::cost), which a runner selects +// as the case `.cost' and therefore has to find listed. +struct section_entry { const char* id; void (*run)(); bool measures; }; +constexpr section_entry sections[] = { + { "openkal.version", version::run, false }, + { "openkal.stream", stream::run, true }, + { "openkal.memory", memory::run, true }, + { "openkal.env", env::run, false }, + { "openkal.time", time::run, true }, + { "openkal.random", random::run, false }, + { "openkal.fs", fs::run, true }, + { "openkal.process", process::run, true }, + { "openkal.task", task::run, true }, + { "openkal.exec", exec::run, false }, + { "openkal.terminal", terminal::run, false }, + { "openkal.net", net::run, false }, + { "openkal.datagram", datagram::run, false }, + { "openkal.space", space::run, false }, + { "openkal.timeout", timeout::run, false }, + { "openkal.map", map::run, false }, + { "openkal.region", region::run, true }, + { "openkal.prof", prof::run, true }, + { "openkal.debug", debug::run, false }, + { "openkal.abort", termination::run, false }, +}; + +} // namespace + +int run_all() { + protocol::begin(); + if (protocol::listing()) { + for (const section_entry& e : sections) protocol::list_case(e.id, e.measures && performs(kind::cost)); + return protocol::finish(0, 0, 0); + } + + write_inventory(); + for (const section_entry& e : sections) + if (protocol::wanted(e.id)) e.run(); + + return summarise(); +} + +} // namespace okc diff --git a/conformance/src/suite.cppm b/tests/cts/src/suite.cppm similarity index 100% rename from conformance/src/suite.cppm rename to tests/cts/src/suite.cppm diff --git a/tests/e2e/abi.cpp b/tests/e2e/abi.cpp new file mode 100644 index 0000000..c9f5cd2 --- /dev/null +++ b/tests/e2e/abi.cpp @@ -0,0 +1,41 @@ +// THE C ABI HOLDS AT THIRTY-TWO BITS AS WELL AS SIXTY-FOUR. +// +// The ABI this repository freezes is instantiated at two pointer widths, and +// every hosted target it serves is 64-bit. `kal_node_info's assertion once read +// `offsetof(modified_ns) == sizeof(kal_uintptr)', true at 64 bits and false at +// 32, and it surfaced first in a consumer's rv32 build. The probe +// (tests/e2e/data/abi/main.cpp) is compiled for a target of each width; the +// claim is about layout, so it is compiled and not run. mcpp resolves the +// toolchain of each target itself, which holds on every row. +import std; +import mcpplibs.mtest; +import mcpplibs.mtest.e2e; + +using namespace mcpplibs::mtest; +namespace fs = std::filesystem; + +namespace { + +const capability mcpp{"mcpp", {.program = {"mcpp"}}}; + +std::string read(const fs::path& file) { + std::ifstream in{file, std::ios::binary}; + return {std::istreambuf_iterator{in}, {}}; +} + +E2E abi{"abi", mcpp, timeout{std::chrono::minutes{15}}, [] { + const auto include = fs::absolute("include").generic_string(); + const auto probe = read("tests/e2e/data/abi/main.cpp"); + "the frozen layout holds"_test = values{"riscv64-none-elf", "riscv32-none-elf"} | + [&](const std::string& target) { + const auto manifest = std::format( + "[package]\nname = \"abi\"\nversion = \"0.1.0\"\n\n[build]\ninclude_dirs = [\"{}\"]\n", + include); + const auto root = e2e::tree({{"mcpp.toml", manifest}, {"src/main.cpp", probe}}); + expect(e2e::succeeded(e2e::run({.argv = {"mcpp", "build", "--target", target}, + .cwd = root, + .timeout = std::chrono::minutes{10}}))); + }; +}}; + +} // namespace diff --git a/tests/e2e/data/abi/main.cpp b/tests/e2e/data/abi/main.cpp new file mode 100644 index 0000000..8067184 --- /dev/null +++ b/tests/e2e/data/abi/main.cpp @@ -0,0 +1,25 @@ +// The frozen layout, compiled for a target of each pointer width (tests/e2e/abi.cpp). +#include +#include +// `self_size' is first, and that is the whole of the growth rule: an +// implementation built at a later version reads it before anything else, to +// learn how much of the structure the caller has. +static_assert(__builtin_offsetof(kal_node_info, self_size) == 0); +static_assert(__builtin_offsetof(kal_node_info, present) == 4); +// The fields of 0.9.0, at the offsets they had. Growth is past the end only. +static_assert(__builtin_offsetof(kal_node_info, size) == 8); +static_assert(__builtin_offsetof(kal_node_info, modified_ns) == 16); +static_assert(__builtin_offsetof(kal_node_info, identity) == 24); +static_assert(__builtin_offsetof(kal_node_info, kind) == 40); +static_assert(__builtin_offsetof(kal_node_info, writable) == 44); +// Appended in 0.13. +static_assert(__builtin_offsetof(kal_node_info, executable) == 48); +static_assert(__builtin_offsetof(kal_node_info, reserved) == 52); +// No field is pointer-sized, so the size is one at both widths. +static_assert(sizeof(kal_node_info) == 56); +static_assert(sizeof(kal_dir) == sizeof(kal_uintptr)); +static_assert(sizeof(kal_file) == sizeof(kal_uintptr)); +// A transfer is one signed word of a pointer's width. +static_assert(sizeof(kal_intptr) == sizeof(kal_uintptr)); +static_assert((kal_intptr)-1 < 0); +extern "C" void _start() { for (;;) {} } diff --git a/tests/e2e/headers.cpp b/tests/e2e/headers.cpp new file mode 100644 index 0000000..d4bf223 --- /dev/null +++ b/tests/e2e/headers.cpp @@ -0,0 +1,73 @@ +// The headers, examined by the tools of this repository (tools/), as a +// consumer's compiler reads them. Each case states why it exists; none changes +// the checkout, so they run beside each other. +import std; +import mcpplibs.mtest; +import mcpplibs.mtest.e2e; + +using namespace mcpplibs::mtest; +namespace fs = std::filesystem; + +namespace { + +const capability cc{"cc", {.command = {"cc", "--version"}}}; +const capability clang{"clang", {.command = {"clang", "--version"}}}; + +std::string read(const fs::path& file) { + std::ifstream in{file, std::ios::binary}; + return {std::istreambuf_iterator{in}, {}}; +} + +e2e::outcome tool(std::vector argv, + std::vector>> env = {}) { + argv.insert(argv.begin(), "bash"); + return e2e::run({.argv = std::move(argv), .env = std::move(env), + .timeout = std::chrono::minutes{5}}); +} + +// The tools are POSIX shell scripts. +E2E headers{"headers", except{platform::windows}, [] { + // EVERY MACRO HAS A MODULE SPELLING, and it is checked rather than + // remembered: `KAL_LOCK_*' in 0.10 and `KAL_SPAWN_*' in 0.11 were each added + // to a header, used from a module consumer, and failed to compile there, + // because a macro does not cross a module boundary. src/macros.cppm is + // generated, and generating it again must give the file committed. + "every macro has its module spelling"_test = [] { + const auto out = temp_dir() / "macros.cppm"; + expect(fatal(e2e::succeeded(tool({"tools/gen-macros.sh", out.string()})))); + expect(eq(read(out), read("src/macros.cppm"))) + << "src/macros.cppm is out of date: run tools/gen-macros.sh and commit it"; + }; + + // The C form with the environment's own headers excluded, because the + // consumer it exists for (a C library ported onto openkal) is compiled that + // way; and the translation unit the suite carries names the same entities. + "the C form compiles alone under cc"_test[cc] = [] { + expect(e2e::succeeded(tool({"tools/check-declarations.sh"}, {{"CC", "cc"}}))); + }; + "the C form compiles alone under clang"_test[clang] = [] { + expect(e2e::succeeded(tool({"tools/check-declarations.sh"}, {{"CC", "clang"}}))); + }; + + // NO DECLARATION NAMES A BACKEND TYPE: `sizeof(long)' is 8 on x86_64 Linux + // and 4 on x86_64 Windows, so a `long' in an interface is one signature with + // two meanings. + "no declaration names a backend type"_test[clang] = [] { + expect(e2e::succeeded(tool({"tools/check-types.sh", "clang", "include"}))); + }; + + // And the type check fails when it should: a check that succeeds by finding + // nothing succeeds identically when it reads nothing. A copy of the headers + // is given a declaration in `unsigned long'. + "a backend type in a declaration is reported"_test[clang] = [] { + const auto copy = temp_dir() / "include"; + fs::copy("include", copy, fs::copy_options::recursive); + std::ofstream{copy / "openkal" / "env.h", std::ios::app} + << "\nint kal_env_probe_leak(unsigned long n);\n"; + const auto r = tool({"tools/check-types.sh", "clang", copy.string()}); + expect(r.started && !r.timed_out && r.status != 0) + << "the type check accepted 'unsigned long' in an interface"; + }; +}}; + +} // namespace diff --git a/tests/e2e/substitution.cpp b/tests/e2e/substitution.cpp new file mode 100644 index 0000000..b47c60d --- /dev/null +++ b/tests/e2e/substitution.cpp @@ -0,0 +1,175 @@ +// THE PROPERTY THE SPECIFICATION EXISTS FOR: one source, built against two +// implementations, behaves as each of them does, and the source does not change +// (examples/substitution). And the surface checker of clause 9 (tools/ +// check-surface.sh) fails when it should and answers what an artefact provides. +// +// Each case works on a copy of the example in its temporary directory, whose +// manifests name this working tree by its absolute path, so the checkout is +// never written. +import std; +import mcpplibs.mtest; +import mcpplibs.mtest.e2e; + +using namespace mcpplibs::mtest; +namespace fs = std::filesystem; + +namespace { + +const capability mcpp{"mcpp", {.program = {"mcpp"}}}; +const capability cc{"cc", {.command = {"cc", "--version"}}}; + +std::string read(const fs::path& file) { + std::ifstream in{file, std::ios::binary}; + return {std::istreambuf_iterator{in}, {}}; +} + +void write(const fs::path& file, std::string_view text) { + std::ofstream{file, std::ios::binary} << text; +} + +// The line of `file' that begins with `key', and the words after it, replaced by `to'. +void replace(const fs::path& file, std::string_view key, std::string_view to) { + auto text = read(file); + const std::regex line{std::format("(^|\n){}\\s*=[^\n]*", key)}; + expect(fatal(std::regex_search(text, line))) << file.generic_string() << " states " << key; + write(file, std::regex_replace(text, line, std::format("$1{}", to))); +} + +// examples/substitution without what a build left there, naming this tree. +fs::path example() { + const auto to = temp_dir() / "substitution"; + fs::create_directories(to); + for (auto it = fs::recursive_directory_iterator{"examples/substitution"}; + it != fs::recursive_directory_iterator{}; ++it) { + const auto leaf = it->path().filename(); + if (leaf == "target" || leaf == ".mcpp" || leaf == "mcpp.lock") { + it.disable_recursion_pending(); + continue; + } + const auto at = to / it->path().lexically_relative("examples/substitution"); + if (it->is_directory()) fs::create_directories(at); + else fs::copy_file(it->path(), at); + } + const auto spec = std::format("openkal = {{ path = \"{}\" }}", fs::absolute(".").generic_string()); + for (auto p : {"app", "impl-fd", "impl-discard"}) + replace(to / p / "mcpp.toml", "openkal", spec); + return to; +} + +e2e::outcome mcpp_in(const fs::path& dir, std::vector args) { + args.insert(args.begin(), "mcpp"); + return e2e::run({.argv = std::move(args), .cwd = dir, .timeout = std::chrono::minutes{10}}); +} + +// Every object file a build of `dir` produced. +std::vector objects(const fs::path& dir) { + std::vector out; + for (auto& e : fs::recursive_directory_iterator{dir / "target"}) + if (e.is_regular_file() && e.path().extension() == ".o") out.push_back(e.path().string()); + return out; +} + +e2e::outcome surface(std::vector args) { + args.insert(args.begin(), {"bash", fs::absolute("tools/check-surface.sh").string()}); + return e2e::run({.argv = std::move(args), .timeout = std::chrono::minutes{2}}); +} + +std::set lines(std::string_view text) { + std::set out; + for (auto part : std::views::split(text, '\n')) out.emplace(std::string_view{part}); + return out; +} + +E2E substitution{"substitution", mcpp, except{platform::windows}, timeout{std::chrono::minutes{20}}, [] { + const auto produced = std::string_view{"the application produced this line"}; + + "one source over two implementations"_test = [&] { + const auto root = example(); + const auto app = root / "app"; + const auto before = read(app / "src" / "main.cpp"); + + const auto with_fd = mcpp_in(app, {"run"}); + expect(fatal(e2e::succeeded(with_fd))); + expect(e2e::contains(with_fd.out + with_fd.err, produced)); + + replace(app / "mcpp.toml", "openkal-fd", "openkal-discard = { path = \"../impl-discard\" }"); + fs::remove_all(app / "target"); + const auto with_discard = mcpp_in(app, {"run"}); + expect(with_discard.started && !with_discard.timed_out); + expect(!(with_discard.out + with_discard.err).contains(produced)) + << "the discarding implementation produced output"; + + expect(eq(read(app / "src" / "main.cpp"), before)) << "the source changed between builds"; + }; + + // The checker is only useful if it fails when it should: an unspecified + // name is introduced and must be rejected. + "the surface checker rejects an addition"_test = [&] { + const auto impl = example() / "impl-fd"; + const auto list = fs::absolute("SURFACE.txt").string(); + expect(fatal(e2e::succeeded(mcpp_in(impl, {"build"})))); + auto args = objects(impl); + args.insert(args.begin(), list); + expect(fatal(e2e::succeeded(surface(args)))); + + write(impl / "src" / "extra.cpp", "extern \"C\" void kal_vendor_extension(void) {}\n"); + expect(fatal(e2e::succeeded(mcpp_in(impl, {"build"})))); + args = objects(impl); + args.insert(args.begin(), list); + const auto r = surface(args); + expect(r.started && !r.timed_out && r.status != 0) << "the checker accepted an unspecified name"; + }; + + // `provides-interfaces' in a package manifest is a declaration, checked + // rather than trusted: the walk that decides whether each group is exported + // whole answers which interfaces an artefact provides. The subject is an + // object built from SURFACE.txt whose answer is known in advance: three + // groups whole, and two names of a fourth. + "the interface list is the artefact's own answer"_test[cc] = [&] { + std::map> groups; + std::string current; + for (auto part : std::views::split(read("SURFACE.txt"), '\n')) { + std::string line{std::string_view{part}}; + if (line.starts_with("# openkal.")) { + current = line.substr(2, line.find(' ', 2) - 2); + groups[current]; + } else if (!line.empty() && !line.starts_with('#') && !current.empty()) { + line.erase(0, line.find_first_not_of(" \t")); + line.erase(line.find_last_not_of(" \t\r") + 1); + if (!line.empty()) groups[current].push_back(line); + } + } + std::string fake; + auto define = [&](const std::string& n) { + fake += std::format("int {0}(void);\nint {0}(void) {{ return 0; }}\n", n); + }; + for (auto g : {"openkal.version", "openkal.abort", "openkal.stream", "openkal.memory"}) + for (auto& n : groups[g]) define(n); + for (auto& n : groups["openkal.fs"] | std::views::take(2)) define(n); + + const auto dir = e2e::tree({{"fake.c", fake}}); + expect(fatal(e2e::succeeded( + e2e::run({.argv = {"cc", "-c", "fake.c", "-o", "fake.o"}, .cwd = dir})))); + const auto object = (dir / "fake.o").string(); + const auto list = fs::absolute("SURFACE.txt").string(); + + const auto r = surface({"--interfaces", list, object}); + expect(fatal(e2e::succeeded(r))); + const auto provided = lines(r.out); + for (auto want : {"openkal.abort", "openkal.stream", "openkal.memory"}) + expect(provided.contains(want)) << want << " is exported whole and was not listed"; + // openkal.version is exported by every implementation and is not an interface. + expect(!provided.contains("openkal.version")) << "openkal.version is not an interface"; + // Half an interface is not a smaller claim. + expect(!provided.contains("openkal.fs")) << "a group exported in part was reported as provided"; + + // The --toml spelling is the same answer in the shape a manifest carries. + const auto t = surface({"--toml", list, object}); + expect(fatal(e2e::succeeded(t))); + const auto toml = lines(t.out); + expect(toml.contains("provides-interfaces = [")); + expect(toml.contains(" \"openkal.stream\",")); + }; +}}; + +} // namespace diff --git a/tests/unit/version.cpp b/tests/unit/version.cpp new file mode 100644 index 0000000..29a5931 --- /dev/null +++ b/tests/unit/version.cpp @@ -0,0 +1,54 @@ +// The version the declarations state is the version the package is. +// +// `kal_version' is `return KAL_VERSION;' in every implementation, so the three +// constants of version.h are what a consumer bound at load can ask, and clause +// 6.2's comparison is made against them. They drifted through 0.12.0 and 0.13.0, +// when the comparison was between two equal numbers on every implementation. +// The manifest is the authority: it is what the index publishes and what a +// consumer resolves. +// +// The constants are read by the compiler, not by a pattern over the header's +// text, so a renamed constant is a build failure rather than a comparison that +// read nothing. +#include + +import std; +import mcpplibs.mtest; + +using namespace mcpplibs::mtest; + +namespace { + +// The `version = "..."` of the [package] table. +std::optional packaged(std::string_view manifest) { + static const std::regex line{R"re(^version\s*=\s*"([^"]*)")re", std::regex::multiline}; + std::match_results m; + if (!std::regex_search(manifest.begin(), manifest.end(), m, line)) return std::nullopt; + return m[1].str(); +} + +std::string read(const std::filesystem::path& file) { + std::ifstream in{file, std::ios::binary}; + return {std::istreambuf_iterator{in}, {}}; +} + +UTest version{"version", [] { + const auto declared = std::format("{}.{}.{}", KAL_VERSION_MAJOR, KAL_VERSION_MINOR, + KAL_VERSION_PATCH); + + "the declarations state the package's version"_test = [&] { + const auto text = read("mcpp.toml"); + expect(fatal(!text.empty())) << "mcpp.toml is read from the package root"; + const auto v = packaged(text); + expect(fatal(v.has_value())) << "the manifest declares a version"; + expect(eq(declared, *v)) << "every implementation answers kal_version with the " + "first; edit include/openkal/version.h to the second"; + }; + + "a manifest without a version is not read as one"_test = [] { + expect(!packaged("[package]\nname = \"openkal\"\n").has_value()); + expect(eq(*packaged("[package]\nversion = \"9.8.7\"\n"), std::string{"9.8.7"})); + }; +}}; + +} // namespace diff --git a/tools/check-declarations.sh b/tools/check-declarations.sh index e25a3db..8e76f57 100755 --- a/tools/check-declarations.sh +++ b/tools/check-declarations.sh @@ -59,7 +59,7 @@ echo "the C declarations are complete and compile without the environment's head # carries a file naming the same entities, every toolchain compiles it, and this # is where the file is written --- and where a file that has fallen behind the # list is detected, since a stale copy would be compiled happily by all three. -carried="$here/conformance/src/declarations.c" +carried="$here/tests/cts/src/declarations.c" if [ "$write" -eq 1 ]; then { @@ -73,6 +73,28 @@ if [ "$write" -eq 1 ]; then [ -n "$n" ] && printf ' (void)sizeof(&%s);\n' "$n" done <<< "$names" echo '}' + echo + echo '/* Every name, reached without being required (0.25.2): a link that discards' + echo ' * what nothing reaches keeps the names the implementation defines, for the' + echo ' * comparison of clause 9, and an absent name is null. A weak reference to a' + echo ' * name defined nowhere is accepted by the static link of ELF only. */' + echo '#if defined(__ELF__)' + while read -r n; do + [ -n "$n" ] && printf '#pragma weak %s\n' "$n" + done <<< "$names" + echo '#endif' + echo + echo 'unsigned long okc_names_reached(void);' + echo 'unsigned long okc_names_reached(void)' + echo '{' + echo ' unsigned long n = 0;' + echo '#if defined(__ELF__)' + while read -r n; do + [ -n "$n" ] && printf ' n += (const void*)&%s != 0;\n' "$n" + done <<< "$names" + echo '#endif' + echo ' return n;' + echo '}' } > "$carried.next" mv "$carried.next" "$carried" echo "wrote $carried: $count name(s)" @@ -81,9 +103,9 @@ fi [ -f "$carried" ] || exit 0 -carried_names="$(grep -oE '&kal_[a-z_0-9]+' "$carried" | cut -c2- | sort -u)" +carried_names="$(grep -oE 'sizeof\(&kal_[a-z_0-9]+' "$carried" | cut -c9- | sort -u)" if [ "$carried_names" != "$names" ]; then - echo "conformance/src/declarations.c does not name the surface list." >&2 + echo "tests/cts/src/declarations.c does not name the surface list." >&2 echo "only in SURFACE.txt:" >&2 comm -23 <(printf '%s\n' "$names") <(printf '%s\n' "$carried_names") >&2 echo "only in declarations.c:" >&2 diff --git a/tools/gen-macros.sh b/tools/gen-macros.sh index 1ca48b1..cc2d501 100755 --- a/tools/gen-macros.sh +++ b/tools/gen-macros.sh @@ -1,6 +1,8 @@ #!/usr/bin/env bash # Generate src/macros.cppm from the headers. # +# gen-macros.sh [] the file written; src/macros.cppm by default +# # THIS EXISTS BECAUSE THE SAME DEFECT HAPPENED TWICE. `KAL_LOCK_*' in 0.10 # and `KAL_SPAWN_*' in 0.11 were both added to a header, both used from a module # consumer, and both failed to compile there --- because a macro DOES NOT CROSS A @@ -19,8 +21,9 @@ set -euo pipefail # KAL_TERM_ECHO before KAL_TERMINATED_LIMIT here and after it in CI. export LC_ALL=C +out="${1:-src/macros.cppm}" +case "$out" in /*) ;; src/macros.cppm) ;; *) out="$PWD/$out" ;; esac cd "$(dirname "$0")/.." -out=src/macros.cppm names=$(grep -rhoE '^#define +KAL_[A-Z0-9_]+' include/openkal/*.h \ | awk '{print $2}' | sort -u) diff --git a/tools/run-abi-test.sh b/tools/run-abi-test.sh index 95c8e3c..586a95f 100755 --- a/tools/run-abi-test.sh +++ b/tools/run-abi-test.sh @@ -17,7 +17,7 @@ # was no artifact of the shape that could test it. This produces one. # # The second implementation is built FROM the first: the same objects, with four -# names renamed out of the way and answered by `conformance/abi/interposer.cpp' +# names renamed out of the way and answered by `tests/cts/abi/interposer.cpp' # instead. It is a second implementation in the only sense that matters to a # consumer --- a different shared object, exporting the same surface, answering # differently --- and it is four hundred lines lighter than a second port. @@ -66,13 +66,13 @@ for o in "$work/obj"/*.o; do "$o" "$work/obj2/$(basename "$o")" done $cxx -std=c++23 -O1 -fPIC -fno-exceptions -fno-rtti -I"$here/include" \ - -c "$here/conformance/abi/interposer.cpp" -o "$work/obj2/interposer.o" + -c "$here/tests/cts/abi/interposer.cpp" -o "$work/obj2/interposer.o" $cxx -shared -o "$work/two/libopenkal.so.0" "$work/obj2"/*.o -lpthread \ -Wl,-soname,libopenkal.so.0 ln -sf libopenkal.so.0 "$work/two/libopenkal.so" echo "--- one probe, built once, against neither by name ---" -$cc -std=c11 -O1 -I"$here/include" "$here/conformance/abi/probe.c" \ +$cc -std=c11 -O1 -I"$here/include" "$here/tests/cts/abi/probe.c" \ -o "$work/probe" -L"$work/one" -lopenkal -Wl,-rpath,'$ORIGIN' # THE ONE OBSERVATION THAT MAKES THIS A TEST OF DISTRIBUTION. The binary is diff --git a/tools/run-conformance.sh b/tools/run-conformance.sh index 457796a..a073e55 100755 --- a/tools/run-conformance.sh +++ b/tools/run-conformance.sh @@ -31,7 +31,7 @@ native() { } here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -suite="$here/conformance" +suite="$here/tests/cts" [ -f "$suite/mcpp.toml" ] || { echo "no suite at $suite" >&2; exit 2; } implementation="$(cd "$implementation" && pwd)"