Repository navigation
Open
Labels
Description
User Story
I'm moving e2e/rust tests onto testkit, and I want driver authors and operators to run openshell-conformance against their own gateway as is. Today that needs OPENSHELL_TEST_PYTHON_IMAGE, because the default test image isn't published. I need one test image that any gateway can pull, with the tools the ported tests use, so conformance runs against any gateway without an override.
Description
Sandbox tests use two images today:
openshell/e2e-python:dev, built frome2e/python/Dockerfile.workload, is the default fortestkit::python_image(). It has system Python, a uv Python and virtualenv, curl, git, openssl,requests,google-auth,cloudpickle, and a non-rootsandboxuser. It isn't published, so it exists only where a lane builds it (the Docker and Podmane2e/rustscripts,mise run e2e:workload:build) or loads it (the tmachine installers). Its name has no registry, so a gateway that doesn't have it pulls from Docker Hub and fails.ghcr.io/astral-sh/uv:0.12.17-python3.12-trixie-slim@sha256:…, a public image pinned by digest. The Kubernetes and VM runners pass it throughOPENSHELL_TEST_PYTHON_IMAGE. It has the Python standard library only: no curl, git or openssl, and no non-root user.
#5318 tried making the public uv image the default. It passed CI on every lane, but it was reverted in review because:
- Nothing pre-pulls it, so each host or node pulls it during its first sandbox test, inside that test's timeout.
- Every lane then needs outbound registry access.
- It drops curl, git and openssl, which many tests still to be ported (test(e2e): decide which e2e/rust tests to port and in what order #5297) use.
- It can't satisfy a policy that names the
sandboxuser, which the base test policy does. Podman rejects a named user the image doesn't have.
Options
- A trusted public image with more tools. It should come from a known publisher, be pinned by digest, and include Python, curl, git, openssl and a non-root user. That leaves us nothing to publish or maintain.
- Publish our own test image. CI publishes
e2e/python/Dockerfile.workload(or a slimmer version) to ghcr, and testkit pins it by digest. We control its contents, at the cost of a publish step and keeping its packages up to date.
Either way, the follow-up also has to decide:
- Pulling during tests: whether the tmachine installers and lane scripts pre-load the pinned image, so tests don't pull it inside their timeouts.
- Gateways that can't pull it: whether conformance checks up front that the image can be pulled, and fails or skips with a clear message instead of a sandbox timeout.
Context
tests/testkit/testkit/src/image.rs:python_image(),PYTHON_BINARIES.tests/testkit/testkit/src/base-policy.yaml: namesrun_as_user: sandbox.e2e/support/conformance.sh(E2E_PULLABLE_PYTHON_IMAGE),e2e/rust/e2e-{docker,podman,kubernetes,vm}.sh.- The legacy harness has the same split:
E2E_WORKLOAD_IMAGEine2e/rust/src/harness/sandbox.rs. - tmachine builds and loads
e2e-python:dev(tests/tmachine/artifacts.nix,config.nix, theopenshell_gatewayrole, and theopenshell-k3s.yamlplaybook). The Podman driver e2e playbook (drivers/podman/e2e.yaml) uses it directly.
Definition of Done
- One image is chosen, with the reasoning recorded here.
-
testkit::python_image()defaults to it, pinned by registry and digest, andPYTHON_BINARIEScovers its Python paths. - Lanes don't pull it inside a test's timeout, or the pull time is measured and shown to be acceptable.
-
openshell-conformancepasses against every conformance row and host lane with noOPENSHELL_TEST_PYTHON_IMAGEset. - The lane scripts and tmachine no longer build the workload image or override the image for testkit suites, or a remaining use is documented.
- If we publish our own image, CI publishes it and something keeps its base and packages updated.
-
tests/testkit/README.md,tests/suites/conformance/README.mdandTESTING.mddescribe the image and the override.
Activity
[-]test(testkit): use one pullable test image with the tools ported tests need[/-][+]test(testkit): run conformance against any gateway with one pullable test image[/+]