bug(kubernetes): sandboxes cannot start under the EKS Nitro Sandbox RuntimeClass because the workspace PVC is mounted at two paths #5476

User Story

I run agent sandboxes on Amazon EKS and want each OpenShell sandbox to have a VM boundary, so I deployed OpenShell 0.1.8 on EKS Auto Mode with AWS Nitro Sandboxes, using the isolated-sandbox RuntimeClass that EKS provides as server.defaultRuntimeClassName. Every sandbox fails to start: the Nitro runtime refuses the workload pod because the workspace PVC is mounted at two different paths.

Problem Statement

The Kubernetes driver seeds the workspace with the workspace-init init container, which mounts the workspace PVC at /mnt/openshell-workspace (WORKSPACE_INIT_MOUNT_PATH), while the agent container mounts the same PVC at /sandbox. The Nitro Sandbox runtime does not allow one PVC at different paths across the containers of a sandbox pod, so the agent container fails to start:

failed to create containerd task: failed to create shim task: Others("Other: shared filesystem PVC (/dev/nvme3n1) is already mounted at /mnt/openshell-workspace; container agent requests /sandbox. Mounting the same PVC at different paths across containers in one sandbox pod is not supported.")

The CLI shows only stream closed after the images are pulled, and the gateway logs rolled back stale fail-closed sandbox-runtime bootstrap and deletes the sandbox.

Impact / Why This Matters

The runtime-class support (server.defaultRuntimeClassName, pod.runtime_class_name) is how an operator adds a VM boundary on Kubernetes, and on EKS the isolated-sandbox RuntimeClass is how a pod gets its own Nitro micro-VM. With the current workspace layout no sandbox starts under it. I found no driver setting that changes the built-in workspace mounts, so I have no workaround.

Other parts I checked work inside the Nitro micro-VM. I ran openshell-sandbox capability-probe from ghcr.io/nvidia/openshell/sandbox:0.1.8 in an isolated-sandbox pod with the workload pod's security settings (UID 65532, no capabilities, no privilege escalation, RuntimeDefault seccomp, net.ipv4.ip_unprivileged_port_start=0, writable /tmp) and it reported qualified: true with Landlock ABI 7, the same as on the host runtime. NetworkPolicy egress denial was enforced for pods in the micro-VM. Admission, scheduling and image pulls worked, and the init containers started. The workspace mount is the blocker I hit; I cannot rule out others behind it.

Acceptance Criteria

• With server.defaultRuntimeClassName: isolated-sandbox on an EKS Auto Mode cluster with Nitro Sandboxes, openshell sandbox create -- <command> starts the sandbox and runs the command.
• The workspace is still seeded from the image's /sandbox on first start and persists across pod restarts.
• No container in the sandbox pod mounts the workspace PVC at a path other than /sandbox.
• Behavior on the default runtime (runc) is unchanged.

Reproduction Steps

1. Create an EKS 1.37 Auto Mode cluster with a NodePool that requires eks.amazonaws.com/nitro-sandbox=true (I used m7i.xlarge), a default StorageClass, and the amazon-vpc-cni ConfigMap with enable-network-policy-controller: "true".
2. Install Agent Sandbox v1.0.10 and the OpenShell chart 0.1.8 with server.defaultRuntimeClassName: isolated-sandbox.
3. Label the RuntimeClass so OpenShell admits it: kubectl label runtimeclass isolated-sandbox openshell.ai/sandbox-attachable=true openshell.ai/sandbox-attachable-workspace=default.
4. Run openshell sandbox create --name nitro1 --no-tty -- uname -r.
5. The CLI reports stream closed after the images are pulled. kubectl -n openshell get events shows the failed to create shim task error above on the workload pod.

Suggested UX (if applicable)

No user-facing change is needed; the fix is in how the driver builds the pod. Two constraints from testing, for whoever picks it up:

• One PVC at the same path in an init container and the main container of a Nitro sandbox pod works.
• Kubernetes image volumes do not work in Nitro sandbox pods (the runtime fails to open containerd's image-volumes path), so seeding from an image volume while mounting the PVC at /sandbox is not an option there.

One layout within those constraints, which I have not tested: a first init container copies the image's /sandbox into an emptyDir, a second mounts that emptyDir and the PVC at /sandbox and copies it across, and the agent mounts the PVC at /sandbox as now. The emptyDir-backed workspaces proposed in #2787 would also avoid the PVC, without persistence.

Environment

• OpenShell 0.1.8 (CLI and Helm chart 0.1.8, with the chart's default images), Kubernetes compute driver. For the test the gateway ran with server.disableTls=true and unauthenticated users, reached only through kubectl port-forward.
• Agent Sandbox v1.0.10.
• EKS 1.37 Auto Mode, us-west-2, m7i.xlarge nodes with eks.amazonaws.com/nitro-sandbox=true, kubelet v1.37.0-eks-29d7970.
• Workload pods on RuntimeClass isolated-sandbox (guest kernel 6.18.54); the supervisor pod on the default runtime.

Logs

Warning  Failed  pod/default--nitro1  Error: failed to create containerd task: failed to create shim task: Others("Other: shared filesystem PVC (/dev/nvme3n1) is already mounted at /mnt/openshell-workspace; container agent requests /sandbox. Mounting the same PVC at different paths across containers in one sandbox pod is not supported.")
WARN openshell_driver_kubernetes::driver: rolled back stale fail-closed sandbox-runtime bootstrap sandbox="default--nitro1"