Skip to content

90-build-from-source.md

Latest commit

 

History

History
222 lines (171 loc) · 10.8 KB

90-build-from-source.md

File metadata and controls

222 lines (171 loc) · 10.8 KB
Raw

90 — Building from Source and Contributing

Reader: a contributor who will build and change mcpp itself.

The question this chapter answers: how is mcpp built from source, how are its own tests organised, and what does a contribution have to satisfy.

Not here: how a user builds their own project, which is 01 — Getting Started, and how a release is cut, which is 92 — Releasing mcpp.

mcpp is self-hosting — mcpp builds mcpp from source using mcpp itself. Any environment that already has a working mcpp binary can build from source.

Prerequisites

Follow 01 — Getting Started to install a working copy of mcpp, then clone the repository:

git clone https://github.com/mcpp-community/mcpp
cd mcpp

Building and Testing

mcpp build              # compile the current source with the existing mcpp → ./target/.../bin/mcpp
mcpp run -- --version   # run the artifact you just built
mcpp test               # build and run C++ tests discovered under tests/**/*.cpp (including tests/unit)

mcpp test does not run the shell end-to-end suite under tests/e2e/; run those separately against the freshly built binary.

The first build automatically fetches the default toolchain; see 20 — Toolchain Management for details.

To produce a fully static binary identical to a release (the path taken by release.yml):

mcpp build --target x86_64-linux-musl
# → target/x86_64-linux-musl/.../bin/mcpp is a fully static ELF

Source Layout

mcpp is a workspace. modules/ holds ten packages that the engine imports; src/ is the engine itself. The operating system is reached through mcpplibs.os; the one primitive mcpp keeps per system, the lock every release of mcpp takes, is in src/native/.

modules/                  workspace members, imported by src/
├── invocation/           the environment of one invocation: statements over the started one
├── manifest/             manifest and descriptor parsing
├── toolchain-model/      triples, dialects, fingerprints, the link model
├── buildmcpp/            the build.mcpp contract: protocol, directives, provisions
├── source-kind/          source-file role classification
├── versioning/           this binary's version, and SemVer requirements
├── dyndep/               ninja dyndep emission
├── graph/                the generic dependency graph
├── libs/                 vendored text-format parsers
└── log/                  leveled logging

src/
├── main.cpp              entry point
├── native/               the lock, per system (flock, LockFileEx)
├── run.cppm  run/        every program mcpp starts, and how mcpp ends at a request
├── effects.cppm          the constraints of a run (offline, denied effects)
├── home.cppm  home/      the home, its lock and its trust check
├── ui.cppm  ui/          output, the terminal, key input
├── cli.cppm  cli/        command dispatch and the commands
├── build/                build orchestration and the ninja backend
├── modgraph/             P1689 module scanning and the dependency graph
├── pm/                   the resolver and the package-management commands
├── toolchain/            detection, fingerprinting, the std module
├── pack/  publish/       mcpp pack, mcpp publish and xpkg generation
├── fetcher/  fallback/   download, installation, fallback resolution
├── bmi_cache/            the cross-project BMI cache
├── runtime/  provider/  the runtime contract, launch environment and the xlings bridge
├── freestanding/         bare-metal targets, link line and runner
└── scaffold/             `mcpp new` and templates

tests/
├── unit/                 166 files of mtest cases, one test program (`[test] programs = "directory"`)
└── e2e/                  711 shell scripts against a real binary

Test Organization

Two layers, and they answer different questions.

Unit tests (tests/unit/, 166 files, and each member's modules/<name>/tests/) are C++ programs mcpp test discovers. They exercise a module's contract with no binary and no filesystem state. Each is a file of mtest cases: an object UTest name{"Suite/Case", [] { ... }}; is one case, with no main (mcpp test --guide). mcpp test --case 'TargetSelect/*' runs the cases a pattern names; mcpp test -p <member> runs a member's. The files are one program, linked once and run as one process per file.

End-to-end tests (tests/e2e/NN_<name>.sh, 721 files) run a real mcpp binary against a real project. run_all.sh is the CI entry point. mcpp test does not run them.

MCPP=<fresh-mcpp-binary> bash tests/e2e/02_new_build_run.sh

A capability gate decides which run. The first lines of an e2e script declare what it needs, and a runner without that capability skips it:

#!/usr/bin/env bash
# requires: elf gcc

gcc (80 scripts), elf, unix-shell, llvm, jq and fresh-sandbox are the ones in use. A script that requires a capability no CI job provides never runs anywhere, and its greenness means nothing — check that some job supplies what a new script asks for.

CI runs them by mtest (2026.11.8.1+): tests/e2e/mtest.toml makes each script a program, its # requires: line its tags and a # resources: line what it holds while it runs (machine: nothing runs beside it; a script that changes the shared home's configuration says so). run_all.sh with E2E_MTEST=<mtest> runs the suite through it: E2E_SHARD divides by the measured history of the store (tests/e2e/.mtest, which CI takes from main's last run), E2E_JOBS runs that many scripts at once, and E2E_SET=pr runs the scripts a change affects by the nightly coverage map (.github/workflows/e2e-coverage.yml); a script the set leaves out is reported unaffected, which the check that every script runs somewhere (rule R5) accepts for that run, main's full runs being its coverage. mtest explain in tests/e2e says where a script failed and how to run it again.

Writing a check that measures something

The most transferable rule in this repository, and the one that fails silently when it is skipped:

After writing a check, remove the fix and run it once. A check that has never been seen to fail is not known to measure anything.

Three shapes it catches, all of which have shipped here at least once:

shape form
the criterion never runs a test gated on a capability no job provides
the criterion cannot fail a substring search satisfied by any wording
the criterion measures the wrong object a fixture whose directory accumulates across runs, so the search answers about an earlier build

State the denominator too. "Every host in the table was scanned" is a check; "the scan found nothing" is not, because an empty enumeration also finds nothing.

The checks CI runs beside the tests

.github/tools/ holds eighteen scripts. Four are worth knowing before a first PR:

script the condition it refuses
check_docs_style.sh question headings, second person in a reference chapter, a 简体中文 page whose heading structure has fallen behind
check_docs_structure.sh a chapter citing a design record, a docs/NN-*.md path that does not resolve, a translation missing a table
check_version_pins.sh a version written in one place and not the others
check_modules_wiring.sh a workspace member wired into some of the three places that must know about it and not the others

Adding a manifest field: the admission criteria

Closed syntax, open vocabulary: whoever owns the parsing semantics defines the keys; whoever owns the domain knowledge defines the values.

  • mcpp only defines mechanisms (feature union/closure, capability require/provide/override, profile→compiler flags, platform→triple); the keys and shapes are fixed. Domain vocabulary such as feature names, capability names, and backend names appears only in values, never in mcpp's code.
  • Package-custom toml keys are not supported: key legitimacy must not depend on "first parsing the target package," otherwise the manifest loses static parseability (a prerequisite for lockfiles/LSP/auditing). A package's extension point = open value domains within fixed mechanisms.
  • Package-level knobs all converge into features; for sugar keys (such as backend=) to enter the core syntax, they must satisfy: ① domain-neutral (a cross-ecosystem general pattern) ② 1:1 desugaring with zero new parsing semantics.
  • A key that duplicates an answer another section already gives is not admitted. Two places to state one fact is two places that can disagree, and the failure is silent — whichever reader loses the race is simply wrong. Library packaging (12) is the worked example: it added zero manifest keys, because what to pack is [targets.<n>].kind, which interface to publish is [lib] plus the module graph, which headers are public is [build].include_dirs, and the per-artifact evidence is [[runtime.artifacts]].
  • A field that describes what a generated package IS (rather than what a build should DO) belongs on [[runtime.artifacts]] — see §2.11. provenance beginning with mcpp-pack is what marks a directory as one, and mcpp refuses to build inside it.

Issue and PR Guidelines

Issues

File issues at github.com/mcpp-community/mcpp/issues, ideally including the following:

  • The full output of mcpp self env
  • The full output of the failing command (MCPP_LOG_LEVEL=debug gives more detail)
  • Your operating system, distribution, and glibc version (check with ldd --version)

Pull Requests

mcpp is in early iteration and its interfaces may change. Before submitting a PR, please note:

  1. For changes touching the CLI or the mcpp.toml schema, open an issue first to align on direction.
  2. Keep each PR focused on a single change; write commit titles in English imperative form (fix: ... / feat: ...).
  3. For behavior changes or test documentation, run mcpp test and the relevant E2E scripts against a fresh binary before submitting. For documentation-only changes, recheck the examples and links; use gh pr checks <pr-number> for the PR's actual required checks.

Community Resources