refactor: specify the boundary and supervisor-registration contracts compute drivers use #5456

Part of #5200.

User Story

As an OpenShell Maintainer I want the files and values a compute driver hands to the supervisor to be specified as language-neutral formats, so that drivers can live in other repositories, and be written in other languages, without importing Rust types from core crates.

Problem Statement

A compute driver has two contracts with OpenShell. The first is the ComputeDriver gRPC service in proto/compute_driver.proto, which is already language-neutral. The second is what the driver places inside or beside the sandbox for the supervisor: the runtime descriptor, TLS material, gateway verification keys, bootstrap and authentication files, the listener and transport settings, and the resource claims and paths that go with them.

That second contract is defined only by Rust types. The Docker, Podman, and VM drivers imported these from openshell-sandbox-backend (boundary_protocol: BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeDescriptor, SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTlsMaterial, SandboxTransport, generate_sandbox_tls_material), from openshell-isolation-interface (ResolvedWorkloadIdentity, ProvisioningEvidence, BackendError), and as constants from the openshell-sandbox-backend root (BACKEND_NAME, GPU_RESOURCE_CLAIM, ALLOW_EXTRA_SUPPLEMENTARY_GROUPS_RESOURCE_CLAIM, SUPERVISOR_CA_RUNTIME_ROOT, SUPERVISOR_CA_RUNTIME_DIR). RFC 0012 defines the descriptor envelope (backend name, version, opaque payload) but leaves the payload and the file formats unspecified.

openshell-sandbox-backend depends on openshell-core, so a driver that imports these types depends on core even after the SDK carve-out in #5383 and #5405.

A third contract has no specification either: how a driver registers each sandbox with the gateway's shared supervisor. In-process drivers receive the gateway's SharedSupervision coordinator directly. An out-of-tree driver has no way to reach it, and openshell-supervisor-control types such as SandboxInstanceRegistration and ProxyCaMaterial are Rust-only. The desktop drivers do not use this path; only the in-tree Kubernetes driver does.

Impact / Why This Matters

• A driver in another repository must either depend on these crates, which ties it to their versions, or copy the formats from the source, which drifts without a compile error.
• A driver written in another language cannot be built from the repository's documented contracts.
• An out-of-tree compute driver built against a pinned checkout of this repository shows the cost. It depends on five crates (openshell-core, openshell-crypto, openshell-isolation-interface, openshell-sandbox-backend, openshell-supervisor-control) and carries a patch set on top of upstream, including a new gateway-initiated bidirectional stream and a capability, openshell.compute.shared-supervision, to reach the shared supervisor.
• refactor: carve out the driver SDK and container-runtime crate #5383 deliberately left isolation_interface, isolation_registration, and boundary_protocol where they were, because moving them into the driver SDK would have turned Rust types into the driver-facing API before the formats were agreed.

Proposed Design

The work splits into three parts. The first is done in #5488. The other two are open and need decisions.

1. Move the existing Rust types into the SDK, marked as legacy (#5488). The documents are JSON written by the drivers' serde types and read by the supervisor, the sandbox boundary, and, for Docker, the driver itself. Those types move into an openshell-driver-sdk module behind a legacy-contract feature and are re-exported from their old paths. The module is documented as the format that exists today: it accepts no new fields, and it is removed once a replacement has shipped and no supported release reads it. Golden JSON examples and a round-trip test pin the encoding. This is enough for the Rust desktop drivers to stop depending on openshell-core and openshell-sandbox-backend. Whether the module should instead be its own crate is open; the effort is about the same.

2. Specify a language-neutral format for the documents (open). The documents that need one are the launch authentication the gateway passes to the driver, the supervisor auth bundle, the runtime descriptor, and the boundary config. Decide the form:

• JSON Schema describes today's bytes exactly, including the kind-tagged transport and listener variants and the readers' rejection of unknown fields, and it states constraints such as nonzero values. It is a second description of concepts that compute_driver.proto and sandbox_protocol.proto already define (ResolvedWorkloadIdentity, the evidence digest, the resource claims).
• Protobuf messages written as proto3 JSON avoid that duplication and can import the existing messages, but proto3 JSON is not the encoding in use (tags, key casing, oneof shape, string-encoded 64-bit integers), so it is a change of encoding for the files.

A migration that avoids reading two formats: a driver writes the new format under a new file name beside the legacy one, and the supervisor and sandbox prefer the new file when present. Every document is regenerated at each create and restart, so there is no stored data to migrate. The launch authentication travels over gRPC as opaque bytes in CreateSandbox and StartSandbox, so it can gain a typed message field beside them. Docker rewrites its own boundary config and descriptor in place at restart, so it would first need to keep its restart state privately, as Podman does. The window is roughly two to three minor releases under RFC 0014.

The built-in openshell-sandbox runtime is already an IsolationBackend (the supervisor calls Describe and OpenBoundary for every backend), so the format should treat it as the reserved backend name openshell-sandbox, not as a special case. An external backend must receive the gateway identity, verification keys, session rotation, and auth epoch; the rest of BoundaryConfig is the built-in runtime's bootstrap, which a backend such as MXC may reuse by wrapping it.

Validate the format against more than one kind of driver: a container driver (Docker or Podman), a microVM driver, and the native Windows MXC driver on the windows branch, whose boundary runs as a separate process, connects outward to the host, and uses its own isolation backend name and evidence. The format should not assume a Linux container, POSIX paths, or a particular connection direction.

3. Specify the shared-supervision registration (open, separate scope). A driver registers each sandbox with the gateway's shared supervisor: launch, withdraw, recovery, readiness, and supervisor token delivery, under a capability the driver and gateway negotiate, with the gateway keeping assignment authority. A design has been drafted as a gateway-initiated bidirectional gRPC stream with request and response pairs and a generic error response. It serves an out-of-tree shared-supervisor driver, not the desktop drivers, so it is not on the critical path for #5200 and can follow as its own change.

Acceptance Criteria

• The Docker, Podman, and VM drivers build without depending on openshell-sandbox-backend or openshell-core. (Done in refactor(driver-sdk): move the driver contract types into the SDK #5488 for normal builds; open until merged.)
• A test checks the Rust implementation against golden documents. (In refactor(driver-sdk): move the driver contract types into the SDK #5488: golden examples and a round-trip test.)
• The location of isolation_interface, isolation_registration, and boundary_protocol is decided and recorded. (refactor(driver-sdk): move the driver contract types into the SDK #5488 puts the document types in a legacy SDK module; a separate crate remains open, and isolation_registration stays in core.)
• Existing Docker, Podman, and VM e2e suites pass without behavior change. (Not yet run.)
• The files and values a driver gives the supervisor are specified in a versioned, language-neutral form, or a decision is recorded to keep the legacy format for the Rust drivers only.
• The shared-supervision registration is specified, and an out-of-tree driver can use it without patching this repository.

Alternatives Considered

• Export the Rust types from the driver SDK as the long-term contract. Fastest, but turns Rust types into the public contract and ties every driver to Rust. refactor(driver-sdk): move the driver contract types into the SDK #5488 does this only for the legacy format and says so in the module.
• Leave the formats Rust-only and require drivers to depend on the crates. No new specification work, but blocks drivers in other repositories and languages from being independent of core.
• Specify the legacy files as JSON Schema now. Describes the existing bytes, but duplicates definitions that protos already hold and would be thrown away once the format is replaced.

Agent Investigation

• The documents and who handles them today:
  • SandboxLaunchAuthentication is produced by the gateway as JSON and read by every driver.
  • SupervisorAuthBundle is written by the driver from it and read by the supervisor at startup.
  • SandboxRuntimeDescriptor and BoundaryConfig are written by the driver; the supervisor and the sandbox boundary read them.
  • Docker re-reads and rewrites its own boundary config and descriptor at restart; Kubernetes creates new Secrets per generation.
• None of the documents is a gRPC message; each travels as a file or as opaque bytes.
• The desktop drivers import about a dozen items from openshell_sandbox_backend::boundary_protocol and openshell_isolation_interface::contract, none from runtime, runtime_routed, protobuf, sandbox_auth, or stream_adapter.
• openshell-sandbox-backend depended on openshell-core for jwt, provider_credentials, net, SandboxSessionId, and generated protos.
• refactor(driver-sdk): carve out the driver SDK and container-runtime crate #5405 already moves the launch authentication, auth bundle, and session types into the SDK; refactor(driver-sdk): move the driver contract types into the SDK #5488 moves the rest.
• The out-of-tree driver imports, from core, the compute proto, extension_protocol, the launch-authentication and session types in jwt, SandboxSessionId, resource_admission, sandbox_env, and driver_utils, which refactor(driver-sdk): carve out the driver SDK and container-runtime crate #5405 moves into the SDK. From openshell-sandbox-backend it imports the boundary_protocol types above and runtime_routed::new_route_handle, and from openshell-supervisor-control it imports SandboxInstanceRegistration, ProxyCaMaterial, and SecretPem. It builds one set of inputs into both the guest's BoundaryConfig and the shared supervisor's SandboxRuntimeDescriptor so their claims and evidence digest cannot drift.