Repository navigation
ci(e2e): build the tmachine suite bundles once per run - #5432
Conversation
The features and drivers rows each built testkit-ready and their family's tests before booting the VM, about three minutes per row. Branch E2E now builds the bundles once, and the rows run the downloaded bundle. openshell-test-runner gains build and run subcommands. build writes a family's bundle to a directory; run --bundle runs it, and --package selects the tests to run from it. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
/ok to test c0263e4 |
|
Label |
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. cli — No explicit risk acceptance found
Review details
Advisory review; publication gates are unchanged. |
build-e2e-bundles restores a Rust target cache, and Rust Cache saves it on main, where Branch E2E never runs. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
/ok to test 1cecef4 |
|
Note that the compatibility concern is non-blocking since this tooling is internal only and there are no external consumers. |
Summary
Every features and drivers row of the tmachine E2E suites built
testkit-readyand its family's tests before it booted the VM. That took about 167s per row. Branch E2E now builds those test bundles once per run, and the rows download them and run them.Related Issue
Closes #5429
Changes
openshell-test-runnernow has two subcommands:build --family F --out DIRwrites a family's bundle to a directory. The bundle holds the nextest archive,testkit-ready, the workspace manifests and abundle.json.run ... [--bundle DIR | --program PATH]runs it. With--bundle,--packagebecomes a nextest filter on the prebuilt bundle.build-e2e-bundlesjob inbranch-e2e.yml. It runs off the critical path, afterpr_metadataonly. It builds thefeaturesanddrivers/podmanbundles forx86_64-unknown-linux-muslwith line-table debuginfo: about 108 MB per bundle instead of 348 MB, and backtraces keep their line numbers.build-e2e-bundlesrestores a Rust target cache. A newe2e-bundlesjob inrust-cache.ymlsaves it on main, because Branch E2E never runs there.e2e-suites.yml,prebuilt-conformanceis renamed toprebuilt. Conformance rows still runopenshell-conformance, and the other rows run their family's bundle. Release workflows don't passprebuilt, so they still build inline.run, and describebuildand--bundle.Timings
Before: run 42707934133. After: run 42720815711. Each cell shows the job time and the "Run the suites" step time.
Phases of the
features/host-gateway, fedora-podman-rootful step:testkit-readyand the testsThis PR edits a comment in
tests/tmachine/config.nix, which the tmachine disk cache key hashes. So every row in the after run missed that cache once and rebuilt its base layers (~40s). That miss also makes the conformance rows slightly slower in this run, although this PR doesn't change them. Later runs restore the cache, so a Podman features row should take about 3m10s.build-e2e-bundlestook 2m40s with no Rust cache and finished 12 minutes before the rows started. It isn't on the critical path.The Nix download (~50s per row) is handled in #5433.
Testing
tests/runner:cargo test(17 pass),clippy -D warnings,fmt --checkopenshell-test-runner build --family features(aarch64 musl)actionlinton the changed workflows.zizmorreports one existing finding ine2e-suites.ymlfrom main.Checklist