Repository navigation
Conversation
Add selectable container workspaces alongside the default VM backend, using shared BOM/provider/tool-gate configuration and the stock OpenShell Kubernetes driver. Unify RHDH, static GitOps and Make composition, enrollment, lifecycle, credential isolation, owner browser access and retained audit handling. Include offline contracts, build workflows, installation and architecture guides, and the bounded OpenAI/data-science qualification matrix. Live checks covered container lifecycle and fresh-workspace replay plus VM inference, chat, audited tools, model update/restart persistence and RHDH deletion. Developed with assistance from OpenAI Codex.
aaadb1a to
49e545b
Compare
Hadar301
left a comment
Hadar301
left a comment
There was a problem hiding this comment.
I found one high-severity issue in the Kubernetes admission policy. runtime-validation.yaml:36-39 sets runtimePod only for Pods with the sandbox service account or a workload/supervisor label. The validations at :73-85 then allow any Pod that has none of those markers. The gateway service account can create Pods (rbac.yaml:21-23), so a compromised gateway can create an unmarked Pod whose PodSpec bypasses these checks. That Pod can mount namespace Secrets even though its service account cannot read them through the API. Please deny every Pod that is not an approved gateway Pod or a Sandbox-owned runtime Pod, while keeping the creator and owner checks for each allowed class.
dc8da2c to
c607880
Compare
c607880 to
f647f8a
Compare
|
@Hadar301, I reproduced the bypass you reported. A server-side dry-run as the gateway service account admitted an unmarked Pod referencing a Secret the gateway could not read directly. Fixed in On OpenShift, the installed policy rejected all six probes, including the original unmarked Pod, a forged workload owner, an unrelated supervisor Secret, a malicious workload submitted by the legitimate Sandbox controller, role removal and image replacement. Legitimate workload creation and scheduling-gate removal passed. Final CI and owner chat on both recreated RHDH sandboxes also passed. Could you re-review the Pod admission change? Admission policy · Reproducible admission probes |
yuvalk
left a comment
yuvalk
left a comment
There was a problem hiding this comment.
Review summary
I checked each finding below against the branch head (f647f8a); details are in the inline comments. The diff is about 20k lines across 253 files. The review covered the core Python closely (RHDH portal.py/workspace.py, the standalone projector, and kubernetes-runtime controller/reconcile/intent/boundary/adapters/serve), plus the saw-users templates, enroll.py and the apply_bom.py move. Only skimmed: the Rust driver patch, the vendored agent-sandbox manifest and most Helm templates.
Suggested order
| # | Priority | Where | Issue |
|---|---|---|---|
| 1 | P1, fix before merge | kubernetes-runtime/saw_kubernetes/reconcile.py:397 |
Stopping a workspace before its sandbox ID is recorded leaves it Blocked for good |
| 2 | P1, fix before merge | charts/openshell-rhdh/files/portal.py:708 |
A failed or crashed credential write hides the workspace from the catalog and the generator |
| 3 | P2 | kubernetes-runtime/saw_kubernetes/intent.py:74 |
Intent CLI rejects restartGeneration, so it can't be used with lifecycle workspaces |
| 4 | P2 | charts/openshell-rhdh/files/workspace.py:106 |
Bad site config → KeyError escapes the RHDH generator (dropped connection instead of a 500) |
| 5 | P3 | kubernetes-runtime/saw_kubernetes/serve.py:36 |
Boundary verification runs twice per Running pass |
| 6 | P3 | charts/saw-users/files/projector.py:64 |
One missing publication returns 503 for every standalone workspace |
| 7 | P3 | charts/openshell-rhdh/files/workspace.py:145 + charts/saw-users/templates/_helpers.tpl:473 |
Dead if not site check; misleading "not implemented" error for a missing container site |
| 8 | P3 | charts/openshell-rhdh/files/portal.py:1081 |
Catalog /entities probes every route one after another (minutes with many workspaces) |
| 9 | nit | charts/openshell-rhdh/files/portal.py:459 |
Hard-coded web-search exclusion belongs in the catalog |
Recommendations
- Fix the two P1s before merging. Both leave a workspace stuck with no way for the owner to recover from the UI. Each should get a regression test: create → immediate stop for #1, and a Vault write that fails after
reserve_credentialsfor #2. - Make RHDH and the standalone projector agree. Several findings (#4, #6, #7) come from the two entry points into the shared
workspace.pyhandling errors differently: which exceptions get converted, what a 404 publication means, and whether a missing site is rejected. Choosing one behaviour and covering it with shared tests would remove this whole class of bug. - Make the CLI accept the same intent schema as the controller (#3), ideally with a test that runs a portal-emitted lifecycle intent through
_previous. - Load and latency (#5, #8) can follow in a later PR, but are cheap enough to do now.
- Consider splitting future work of this size. At 253 files, the Rust driver patch and the vendored manifests didn't get a real review here. A dedicated pass on the driver patch is worth it before release.
Checked and not raised: the RuntimeClass read in serve.py's scoped reader can't be reached, because compiler.py:248 rejects any RuntimeClass in container mode.
🤖 Generated with Claude Code
| # A stopped new intent never starts a workload to stop it again. | ||
| pending.append(ref) | ||
| return False | ||
| self._mutate("CreateSandbox", {**scope, **sandbox}, plan, generation) |
There was a problem hiding this comment.
[P1] Stopping a new workspace before its sandbox ID is recorded leaves it Blocked for good
This line ignores the CreateSandbox reply, so the sandbox's objectIDs entry is only written by _identity on a later Running pass (line 400). If the owner switches to Stopped before that pass, _stop_references finds the sandbox through GetSandbox but ref not in ids, so it raises ContractError("shutdown refuses unrecorded or replaced workspace/sandbox identity") (line 288). The controller reports Blocked/CoreRejected. Every retry gets the same answer, so the workspace stays stuck until the owner starts it again.
Steps: register a container workspace → first pass sends CreateSandbox and returns Pending → owner submits stop within the ~5s retry window.
Suggestion: record the identity from the create reply, as _apply_provider already does for credentialed providers (lines 356-359):
reply = self._mutate("CreateSandbox", {**scope, **sandbox}, plan, generation)
if reply and reply.get("sandbox"):
self._identity("sandbox", ws["name"], sname, reply["sandbox"], plan, state)Also worth deciding whether the stop path should adopt an unrecorded sandbox when its owner/managed labels match this installation, instead of refusing. A regression test for create → immediate stop would cover both.
| log(f"ERROR: registry entry {name} is malformed (no valid user.json); skipped") | ||
| bad.append(name) | ||
| continue | ||
| if cm["metadata"].get("annotations", {}).get(CREDENTIAL_WRITER): |
There was a problem hiding this comment.
[P1] A failed or crashed create/update hides the workspace until someone re-runs create
reserve_credentials (line 636) puts saw.redhat.com/credential-writer on the entry, and only the final put_configmap (around line 1455) writes the entry back without it. If vault.write raises, or the TaskRun pod is OOM-killed or evicted between the two, the annotation stays. This check then skips the entry without looking at the writer, unlike check_credential_writer (line 621), which does check whether the PipelineRun has finished.
Effect: the workspace disappears from the RHDH catalog (list_workspaces(..., deleting=True) at line 1686) and from the ApplicationSet generator (line 1765). Updates and revision pins stop reaching it, and the owner has no catalog entry to act on. Cleanup still works, because finish-delete doesn't go through list_workspaces.
Suggestion: only skip while the writer is actually running, e.g. reuse the check_credential_writer logic (a cached PipelineRun lookup is fine). Or keep showing the entry as Failed / "credential publication incomplete" so the owner or operator can retry.
| except (ValueError, TypeError): | ||
| raise IntentError("previous intent contains invalid JSON") from None | ||
| if (not isinstance(intent, dict) or set(intent) != { | ||
| "schemaVersion", "generation", "desiredState", "profileRef", "bindings"} |
There was a problem hiding this comment.
[P2] _previous rejects intents that include restartGeneration
The controller accepts restartGeneration (controller.py:269, set(intent) - {"restartGeneration"}). The portal emits it for lifecycle registrations (portal.py:1440), and enrolled-runtime.yaml:16 puts it in the published intent. This exact-set check makes python -m saw_kubernetes.intent --previous-intent current.json ... fail with "previous intent has an invalid schema" for any model-lifecycle workspace. build_intent also can't emit the field.
Suggestion: allow restartGeneration as an optional non-negative int here, matching the controller, and let build_intent take it so the CLI stays a faithful generator and rollback check.
| if not data: | ||
| return fresh | ||
| bound=json.loads(data["binding.json"]) | ||
| if set(bound)!={"registration","intent","approved","gatewayCA"} or bound["registration"]!=reg or set(bound["approved"])!=set(a["reference"]): |
There was a problem hiding this comment.
[P2] Bad site config fails with KeyError instead of WorkspaceError
enrolled_site reads a["reference"] directly here, and below a["relay"] and a["revision"]/a["image"]/a["buildNamespace"]. json.loads(data["binding.json"]) can also raise. reference is only injected when profileBoundaries[selected] exists (project_workspace, around line 168). So automaticEnrollment without a boundary for the selected profile raises KeyError as soon as the enrollment is published.
The standalone projector catches KeyError/TypeError/ValueError (projector.py:96). portal.generator_params only converts WorkspaceError (portal.py:1636), and the serve handler only catches PortalError. The ApplicationSet plugin call then fails with a dropped connection and a traceback instead of a clear 500 naming the bad site.
Suggestion: check the required automaticEnrollment keys up front and raise WorkspaceError("site … missing <key>"), or also catch KeyError/TypeError/ValueError in generator_params the way the projector does. Ideally the chart would also reject this config at render time.
|
|
||
| def boundary(plan): | ||
| verify(config, plan, get) | ||
| resources = KubernetesResourceVerifier(read, boundary) |
There was a problem hiding this comment.
[P3] The full boundary check runs twice on every Running pass
boundary is passed to both Controller(..., boundary) (called at controller.py:352) and KubernetesResourceVerifier(read, boundary), whose __call__ calls verify_boundary(plan) again from Reconciler.reconcile. boundary.verify makes about 22 API reads, including AdminNetworkPolicy and Node lists, so every pass does them twice. At the 5s Pending / 30s resync cadence, that's twice the API-server load and pass latency per workspace, with no added safety.
Suggestion: keep one of the two. The controller-level check maps the failure to BoundaryRejected, which is the better status, so passing a no-op to the verifier (or adding a flag to skip it there) is probably the smaller change. Note adapters.py documents verify_boundary as mandatory, so update that docstring too.
| status = missing.code | ||
| missing.close() | ||
| if status == 404: | ||
| return {} # first projection, before foundation creation |
There was a problem hiding this comment.
[P3] One missing publication blocks projection for every workspace
When the saw-ctl-<reg> namespace exists but saw-portal-enrollment (or a subject map) returns 404/403, this raises ProjectionError. parameters() is all-or-nothing, so /getparams returns 503 for every standalone workspace. The RHDH path treats the same 404 as "not yet published" (ok404=True plus 403 → {} in generator_params).
In practice the namespace is sync-wave -2 and the publication ConfigMaps/Role are wave 0 (saw-portal-container/templates/publications.yaml), so this usually lasts a few seconds. But if that sync stalls (a failed wave -1 resource, a hook failure), every standalone workspace stops updating.
Suggestion: either treat 404 like RHDH does (return {}, i.e. "not bound yet", which enrolled_site already handles), or limit the failure to the one workspace so the other workspaces still get projected. Erroring is the safer default only if a missing publication can mean tampering. If that's the intent, a comment saying so would help.
| values = {**{k:v for k,v in defaults.items() if k != "workspaceRevisions"}, "users": [ws]} | ||
| if ws.get("name") in revisions: | ||
| values["global"] = {**defaults.get("global", {}), "targetRevision": revisions[ws["name"]]} | ||
| if mode != "container" or not site: |
There was a problem hiding this comment.
[P3] The if not site check below never runs
This line already returns for mode != "container" or not site, so the next line's if not site is dead. For a container workspace with no containerSites entry (removed or mistyped), RHDH's generator_params passes site=None/{} and gets plain values back with no registration or installationID.
The chart does stop it (_helpers.tpl:473), but with a misleading message; see the comment there. Suggestion: return early only for mode != "container", and let a container workspace without a site raise WorkspaceError("container site is missing for <name>"). That matches the standalone projector, which already does this at projector.py:92.
| {{- if or (not $u.ownerSubject) (ne $u.ownerSubject $site.subject) (not (regexMatch "^[0-9a-f]{16}$" (default "" $u.registration))) (ne (len ($u.profiles | default list)) 1) (not (hasKey $catalog $profile)) (not $approved) (not $u.pruneOnRemove) -}} | ||
| {{- fail "container requires verified owner subject, scoped registration, approved BOM profile and foreground cleanup" -}} | ||
| {{- end -}} | ||
| {{- else if ne $mode "vm" -}} |
There was a problem hiding this comment.
[P3] A container user with no site gets a misleading error
When deployment.mode=container and containerSites has no entry for the user, the if above is false and we land here with: "deployment.mode=container is not implemented in this revision; no VM fallback". Container mode is implemented; the real problem is the missing site.
Suggestion: add an explicit branch before this one, e.g.
{{- else if eq $mode "container" -}}
{{- fail (printf "user %q: deployment.mode=container requires a containerSites entry" .user.name) -}}
| "text": f"Deleting: {detail}"} | ||
| mode = entry_deployment_mode(entry) | ||
| if mode == "container" and container_site(user): | ||
| return container_status(k8s, user, entry, argo_ns, domain, ping or owner_route_available, catalog) |
There was a problem hiding this comment.
[P3] Catalog fetches probe every UI route one after another
/entities calls workspace_status for each workspace in turn (around line 1690). For container workspaces that now probes each UI link with owner_route_available: up to 3 HTTPS hops with a 3s timeout each. An unreachable route fails on the first hop, so about 3s per link. A route that answers slowly at each redirect can take longer. With 20 container workspaces × 2 UI links that aren't up yet, one catalog refresh takes about 2 minutes, which risks RHDH catalog-provider timeouts and repeats outbound probes on every refresh.
The VM path already pinged one link at a time (ping_url), so the pattern is older than this PR, but the multi-hop probe makes it worse. Suggestion: probe concurrently (e.g. a small ThreadPoolExecutor), or cache readiness for ~30s, or both.
| dedicated_endpoint = (request_deployment_mode(data) == "container" and | ||
| any(sb.get("endpointInput") for _, sb in enabled_catalog_sandboxes(catalog[profile]))) | ||
| for name, spec in sorted(catalog[profile].get("secrets", {}).items()): | ||
| if dedicated_endpoint and name == "web-search": |
There was a problem hiding this comment.
[nit] web-search is excluded by a hard-coded name
For container profiles with a dedicated endpointInput, the request parser drops the secret named web-search, and handle() separately forces webSearch.enabled=False (around line 1446). Both catalog profiles use that exact name today, so nothing is broken. But a renamed search secret, or another tool secret that can't get past the dedicated broker host, would still be required and written to Vault.
Suggestion: declare this in the catalog, e.g. a per-secret "dedicatedEndpoint": "exclude" flag or listing the excluded secrets on the sandbox entry, and have parse_request and handle() read it. Then the profile engine owns the rule.
|
Yuval, I addressed your nine findings in For the create/stop race, the reconciler now journals the create request and records the verified sandbox ID from the gateway reply before returning For interrupted Vault writes, the catalog now keeps the registration visible while GitOps uses the last committed configuration. We verified this with a real Vault denial, a terminal Tekton writer and an owner retry. The follow-up also aligns CLI and controller handling of CI passed, and both freshly recreated RHDH workspaces answered through their authenticated browser chats. Could you re-review the follow-up, particularly the sandbox receipt and credential-publication paths? |
yuvalk
left a comment
yuvalk
left a comment
There was a problem hiding this comment.
Re-review after acadb64
All nine findings from the first review are addressed. 304 tests pass.
| # | Finding | Status | How it's fixed |
|---|---|---|---|
| 1 | P1 Stopping before sandbox ID recorded | ✅ Fixed | Sandbox create now persists a pendingSandboxCreates entry, requires a receipt, records the identity from the reply. Stop path raises MissingSandboxReceipt → StartToRecoverSandboxReceipt if the receipt is missing. Two new tests. |
| 2 | P1 Failed credential write hides workspace | ✅ Fixed | reserve_credentials preserves committed.json (last good state). list_workspaces uses it to keep showing the entry. workspace_status shows the writer state with actionable messages. Runtime actions blocked while a writer is pending. |
| 3 | P2 Intent CLI rejects restartGeneration |
✅ Fixed | valid_lifecycle_intent() shared between controller and intent CLI. build_intent accepts --restart-generation and preserves it on update. Two new tests including a full Helm → CLI → controller round-trip. |
| 4 | P2 KeyError escapes generator_params |
✅ Fixed | project_workspace wraps _project_workspace and catches KeyError/TypeError/ValueError/AttributeError → WorkspaceError with safe diagnostics. enrolled_site validates the five required keys up front. |
| 5 | P3 Boundary verification runs twice | ✅ Fixed | KubernetesResourceVerifier split into __call__ (boundary + attachments) and verify_attachments (identity + PVCs). serve.py passes the split methods to the Reconciler; Controller runs boundary once. |
| 6 | P3 One missing publication blocks all workspaces | ✅ Fixed | Projector returns {} for 404/403 (matches RHDH). New previous_child mechanism reads the existing Application to retain last good values during bootstrap or a lost publication. RBAC changed from ClusterRole on namespaces to namespaced Role on Application names. |
| 7 | P3 Dead if not site / misleading error |
✅ Fixed | _project_workspace raises WorkspaceError("container site is missing for <name>"). _helpers.tpl message now says "requires a containerSites entry". Projector template adds a matching early fail. |
| 8 | P3 Slow catalog route probes | ✅ Fixed | New CatalogRouteCache with ThreadPoolExecutor(4), 30s TTL, capacity-bounded. probe() returns False immediately if the check is in flight, starts a background check. Scope-keyed to invalidate on registration changes. |
| 9 | nit Hard-coded web-search exclusion |
✅ Fixed | Catalog now carries dedicatedEndpointSecrets (generated from BOM). parse_request and handle() use it instead of a magic string. |
No new issues found in the fix commit. The test coverage for each fix looks solid — the runtime tests cover the create → stop race, lost reply, and replacement detection; the portal tests cover interrupted updates, committed entry projection, cache concurrency, and the catalog-driven secret exclusion; the intent test does a full Helm render → CLI → controller round-trip for lifecycle intents.
🤖 Generated with Claude Code
|
summary.md |
on it. i started with shared RHDH installation already present -/ thanks, Hadar! |
e054670 to
a535ebb
Compare
a535ebb to
df3919b
Compare
…dering Order and verify RHDH startup, preserve bounded reviewed enrollment, and provide stock Kubernetes image preparation for slow cold pulls. Drain owned consumers before webhook removal and verify OpenShell replay by request and sandbox identity.
df3919b to
aa3cb54
Compare
|
Hadar, thanks for the report. We fixed the RHDH bootstrap ordering, teardown with a blocked Argo hook, sandbox creation recovery after a lost response, and the Pod admission bypass you identified. |
|
Round2: The full local test suite also did not pass. Static GitOps and Make entry points did not complete. VM lifecycle and live inference coverage remain incomplete. |
|
@Hadar301 thanks! checking |
This PR adds container workspaces as a second SAW deployment option while keeping VM as the default. A workspace selects the same SAW-BOM profile, provider inputs, harnesses and policies through either backend. RHDH, static Validated Pattern configuration and the Make entry point converge on the same container composition and readiness contract.
This PR was prepared with assistance from OpenAI Codex (GPT-6).
Why this change
SAW's existing profile and self-service model is useful independently of VM provisioning. Running that model on Kubernetes should not introduce another BOM format, duplicate profile-specific behavior, or make Developer Hub a requirement for creating a container workspace. The implementation separates common desired-state logic from the infrastructure adapters that realize it.
The VM adapter continues to install and operate the VM-local OpenShell/Podman stack. The container adapter calls the pinned stock OpenShell gateway's Kubernetes driver; it does not create application Pods independently or require changes to OpenShell gateway, supervisor or driver code.
Architecture and ownership
Each container registration owns its runtime namespace and gateway. Control-plane helpers, provider Secrets and the controller ledger reside in a separate trusted namespace. Audit history is isolated from the application and retained independently. Browser access uses a separate relay namespace and a workspace-scoped nonadmin identity. Keycloak, Vault/ESO, governance, Argo and Agent Sandbox are shared platform dependencies; deleting a workspace does not uninstall those dependencies.
There is no new SAW workspace CRD. The public registration, BOM and intent remain ConfigMap/chart inputs. Agent Sandbox is the stock workload API used by the OpenShell Kubernetes driver.
Shared code and backend adapters
saw_core.pycontains profile parsing and validation, provider selection, dependency ordering, typed inference inputs and tool-gate bundle construction. The VM and Kubernetes adapters load the same canonical source. The VM embeds it in its existing signedapply_bom.py, preserving the verifier's manifest format.workspace.pyprojects a registration and catalog intosaw-usersvalues, including sandbox topology, model inputs, approved profile digest and enrollment publications. It also implements common container readiness checks. RHDH packages an exact generated copy; the standalone ApplicationSet plugin imports the same implementation.Policy and credential distribution
The operator approves profile digests, images, destinations, storage and the deployment boundary. An owner chooses an allowed profile and public model inputs; forms do not accept arbitrary PVCs, executable images or driver configuration. Enrollment records resource UIDs and approved hashes in public immutable publications, which the projector incorporates into the runtime intent. Discovery alone does not approve a boundary.
Provider credentials follow the existing RHDH request-to-Vault flow and ESO projections. A static installation supplies the corresponding Vault references. The credential proxy substitutes real credentials only for approved provider destinations; applications receive opaque markers. Shared provider/profile policy governs both backends.
Native OpenShift admission and network policy enforce the Kubernetes resource and network boundaries. The shared OpenClaw tool gate retains its policy decision semantics. For containers, a restricted forwarder sends audit records from an exchange PVC to a writer that owns the retained history and signing key. A valid signed append receipt is required for a gated action to continue.
Browser traffic follows owner OIDC login → scoped relay → OpenShell forwarding → application. The relay's dedicated Keycloak client receives only the intended workspace's user membership. It does not hold gateway administration credentials.
Security follow-up
The runtime Pod policy defaults to denial. The admission review exposed three ways to submit an unauthorized Pod. The fix binds each of the five supported Pod classes to its producer, owner, executable and approved inputs. It rejects a malicious workload template even when the legitimate Agent Sandbox controller submits it. Argo owns the two stock network fences. The gateway can manage its own boundary Services and bootstrap Secrets under admission. The site enrolls the gateway Role, the admission policies and their bindings.
The portal checks the enrolled Keycloak subject and reserves the registration before writing credentials to Vault. Browser forwarding requires a per-slot proof. Relay registration uses a realm-scoped provisioner and a per-client deletion receipt. The signed tool gate terminates the executor if it cannot record an approval. The runtime and governance image workflows both give PR builds read-only access. Only their main-branch publication jobs can write packages.
Administrator identity lifecycle
Both backends use the same
saw_admin.pyper-operation identity code. It creates a one-day P-256 CA/client pair in tmpfs, destroys the CA signing key immediately, and removes the client files when the operation ends. Containers do not retain an administrator key or require certificate-renewal inputs.The VM adapter updates its local client-trust file. The Kubernetes adapter updates only the named public client-CA Secret, using
resourceVersioncompare-and-swap so the reconciler and relay membership Job can operate concurrently. Each temporary CA is bound to a Pod UID and expiry. The caller waits for an authenticated RPC after the stock gateway reloads projected trust, then withdraws its CA and closes its connection. Subsequent updates remove expired or abandoned sessions. A per-Pod lock protects recovery after a container restart: the helper withdraws its prior entries and removes stale private files from the surviving memory volume before issuing a new identity.These trust-write permissions remain administrative and belong only to trusted helpers. They are not granted to the application or browser service accounts. Gateway server TLS, JWT signing keys, credential-encryption key and workspace identities remain unchanged by an admin session.
Installation and lifecycle
saw-usersentries enable the read-only ApplicationSet projector; Argo owns component applications. VM entries retain their existing path.DEPLOYMENT_MODE=containerdispatches create/status/delete to the samesaw-userscomposition. Helm owns the root release and Argo owns its children. RHDH is optional.The standalone projector has named public ConfigMap read access and no Secret-read or resource-write permissions. Its plugin authentication material is generated once and scoped to the owning installation. Both portal and standalone paths consume the same enrollment and relay publications.
Image/input changes create new revision-named preparation, enrollment and membership Jobs with immutable script inputs. Public intent generations drive model changes and stop/start behavior; foreground cleanup removes owned compute and access while retaining designated audit history. Existing registrations cannot silently change backend or adopt replaced resource identities.
The runtime image packages the reconciler, pinned protobuf SDK, helpers and shared source. Its publication workflow builds both amd64 and arm64; deployment values consume reviewed digest pins. Documentation covers component roles, configuration and policy distribution, inference/browser/audit data paths, installation inputs and lifecycle ownership.
Boundaries and next steps
Validation
OpenAI/data-science qualification passed on both container sandboxes: actual inference and owner browser chat; shared tool approval/denial and durable audit; model update, stop/start/restart persistence; RHDH create/update/delete/recreate; owner and credential-routing isolation; and fresh static GitOps and RHDH workspace installation. The final fresh installations used normal automatic Job retries, with no manual workload repair or operator retry.
A separate VM regression passed both sandboxes' inference/chat, audited tool decisions, isolation and temporary administrator cleanup. RHDH model update plus the documented targeted restart preserved the VM/root PVC and original workspace/audit data while creating a fresh VMI. RHDH deletion removed compute, root storage, registration and scoped Vault keys. Existing workspaces were unchanged. VM root-disk audit deletion and container audit-history retention remain distinct contracts.
PR history contains one feature commit and one security commit. The security revision was exercised with API-server admission probes and fresh static and RHDH OpenAI installations. The support matrix records provider and hardware scope. Site inputs, credentials and raw evidence remain outside the PR. Support matrix.
Ready for review. Review should settle the supported platform/storage matrix and publication pins before merge. Other providers, Kata/confidential, a fresh entire-platform installation and continuity of an already-open WebSocket are not qualified. An earlier intermittent stock-exec transport failure remains recorded separately; later passes do not erase it.