feat(helm): separate metrics enablement from listener port

.agents/skills/helm-dev-environment/SKILL.md

@@ -57,6 +57,16 @@ Override with env vars before running `helm:k3s:create`:
   `mcr.microsoft.com/dotnet/aspire-dashboard:latest`)
 - `HELM_K3S_COLLECTOR_HEALTH_TIMEOUT` (default: `120` seconds)
 
+### Gateway metrics
+
+The chart enables its dedicated Prometheus endpoint by default. Set
+`metrics.enabled=false` to omit the listener and its container and Service
+ports without changing `service.metricsPort` (default `9090`). The port must
+remain between 1 and 65535 even when disabled. Legacy `metricsPort: 0` or
+`false` values fail rendering; migrate them to `metrics.enabled=false` and a
+valid port in the same upgrade. `metrics.tls.enabled=true` requires
+`metrics.enabled=true`. This setting does not disable OpenTelemetry export.
+
 ### Shared supervisor plumbing
 
 Set `gatewayConfig.openshell.drivers.kubernetes.supervisor_topology`

deploy/helm/openshell/README.md

@@ -464,10 +464,17 @@ memory-based scaling does not guarantee spare capacity for handoff. See the
 
 ## Secure metrics scraping
 
+Metrics are enabled by default on `service.metricsPort: 9090`. Set
+`metrics.enabled=false` to disable the dedicated listener and its container
+and Service ports while retaining the configured port. This does not disable
+OpenTelemetry export. `service.metricsPort` must remain an integer from 1 to
+65535, even when metrics are disabled.
+
 The dedicated metrics endpoint is plaintext by default for compatibility with
 existing Prometheus deployments. Enable `metrics.tls.enabled` to serve it over
-HTTPS. The initial implementation uses operator-provided Kubernetes Secrets;
-it does not generate metrics PKI or create a `ServiceMonitor`.
+HTTPS; this requires `metrics.enabled=true`. The initial implementation uses
+operator-provided Kubernetes Secrets; it does not generate metrics PKI or
+create a `ServiceMonitor`.
 
 A metrics TLS Secret that exists with malformed contents leaves the metrics
 listener disabled at startup while the gateway keeps serving; it never falls
@@ -590,6 +597,7 @@ discovery endpoint or its TLS CA.
 | grpcRoute.hostnames | list | `[]` | Hostnames the GRPCRoute matches on. Leave empty to match all hosts. |
 | grpcRoute.replicaRouting.enabled | bool | `false` | Route requests carrying an `x-openshell-replica` header straight to that gateway replica. The CLI sets the header on long-lived sandbox connections (SSH, port forwards, exec) using the owner the gateway reports, so they skip the relay through a peer replica. Renders one Service per replica. Requires workload.kind=statefulset, workload.allowMultiReplicaStatefulSet=true, and at most 15 replicas. |
 | imagePullSecrets | list | `[]` | Image pull secrets attached to gateway and helper pods. |
+| metrics.enabled | bool | `true` | Enable the dedicated Prometheus metrics listener and its container and Service ports. |
 | metrics.tls.certSecretName | string | `""` | Kubernetes TLS Secret with tls.crt and tls.key. Empty reuses server.tls.certSecretName. |
 | metrics.tls.clientCaSecretName | string | `""` | Secret with ca.crt for verifying metrics scraper client certificates. It must not contain the gateway client CA. The chart rejects reusing the gateway client CA and client TLS Secret names; the check is by name only and cannot detect a different Secret holding the same CA. Required when requireClientCert is true. |
 | metrics.tls.enabled | bool | `false` | Enable TLS for the dedicated metrics listener. |
@@ -731,7 +739,7 @@ discovery endpoint or its TLS CA.
 | server.workspaceDefaultStorageSize | string | `""` | Default storage size for the workspace PVC in sandbox pods. Uses Kubernetes quantity syntax (e.g. "2Gi", "10Gi", "500Mi"). Empty = built-in default (2Gi). |
 | server.workspaceStorageClass | string | `""` | Kubernetes StorageClass for the workspace PVC in sandbox pods. Empty (default) = omit storageClassName, using the cluster's default StorageClass. Set this on clusters with no default StorageClass, otherwise the workspace PVC stays Pending and the sandbox never starts. |
 | service.healthPort | int | `8081` | Gateway health service port. |
-| service.metricsPort | int | `9090` | Gateway metrics service port. |
+| service.metricsPort | int | `9090` | Gateway metrics service port (1-65535). Disable the endpoint with metrics.enabled, not port 0 or false. |
 | service.port | int | `8080` | Gateway gRPC/HTTP service port. |
 | service.type | string | `"ClusterIP"` | Kubernetes Service type for the gateway. |
 | serviceAccount.annotations | object | `{}` | Annotations to add to the generated service account. |

deploy/helm/openshell/README.md.gotmpl

@@ -465,10 +465,17 @@ memory-based scaling does not guarantee spare capacity for handoff. See the
 
 ## Secure metrics scraping
 
+Metrics are enabled by default on `service.metricsPort: 9090`. Set
+`metrics.enabled=false` to disable the dedicated listener and its container
+and Service ports while retaining the configured port. This does not disable
+OpenTelemetry export. `service.metricsPort` must remain an integer from 1 to
+65535, even when metrics are disabled.
+
 The dedicated metrics endpoint is plaintext by default for compatibility with
 existing Prometheus deployments. Enable `metrics.tls.enabled` to serve it over
-HTTPS. The initial implementation uses operator-provided Kubernetes Secrets;
-it does not generate metrics PKI or create a `ServiceMonitor`.
+HTTPS; this requires `metrics.enabled=true`. The initial implementation uses
+operator-provided Kubernetes Secrets; it does not generate metrics PKI or
+create a `ServiceMonitor`.
 
 A metrics TLS Secret that exists with malformed contents leaves the metrics
 listener disabled at startup while the gateway keeps serving; it never falls

deploy/helm/openshell/templates/_gateway-workload.tpl

@@ -234,7 +234,7 @@ spec:
         - name: health
           containerPort: {{ .Values.service.healthPort }}
           protocol: TCP
-        {{- if .Values.service.metricsPort }}
+        {{- if include "openshell.metricsEnabled" . }}
         - name: metrics
           containerPort: {{ .Values.service.metricsPort }}
           protocol: TCP

deploy/helm/openshell/templates/_helpers.tpl

@@ -580,6 +580,22 @@ Booleans are rejected because they would otherwise convert to 1 or 0.
 {{- end -}}
 {{- end }}
 
+{{/*
+Whether the dedicated metrics listener is enabled. Missing legacy values
+default to enabled; an explicit false must not be replaced by that default.
+*/}}
+{{- define "openshell.metricsEnabled" -}}
+{{- $metrics := .Values.metrics | default dict -}}
+{{- $enabled := true -}}
+{{- if hasKey $metrics "enabled" -}}
+{{- if not (kindIs "bool" $metrics.enabled) -}}
+{{- fail "metrics.enabled must be a boolean" -}}
+{{- end -}}
+{{- $enabled = $metrics.enabled -}}
+{{- end -}}
+{{- if $enabled -}}true{{- end -}}
+{{- end }}
+
 {{/*
 Validate a non-empty, user-provided Kubernetes Secret name. Secret data never
 passes through Helm values into gateway.toml; only this reference is rendered.
@@ -723,6 +739,7 @@ like user-supplied schema-v2 fields during compatibility resolution. */}}
 Validate chart values that Helm would otherwise accept silently.
 */}}
 {{- define "openshell.validateValues" -}}
+{{- $metricsEnabled := include "openshell.metricsEnabled" . -}}
 {{- $kubernetes := include "openshell.effectiveKubernetesConfig" . | fromYaml -}}
 {{- if hasKey $kubernetes "supervisor_topology" -}}
 {{- $topology := get $kubernetes "supervisor_topology" -}}
@@ -783,19 +800,13 @@ Validate chart values that Helm would otherwise accept silently.
 {{- end -}}
 {{- $servicePortValue := toString .Values.service.port -}}
 {{- $healthPortValue := toString .Values.service.healthPort -}}
-{{- $metricsPortValue := "0" -}}
-{{- if hasKey .Values.service "metricsPort" -}}
-{{- $metricsPortValue = toString (get .Values.service "metricsPort") -}}
-{{- end -}}
+{{- $metricsPortValue := toString .Values.service.metricsPort -}}
 {{- if not (regexMatch `^-?[0-9]+$` $servicePortValue) -}}
 {{- fail "service.port must be an integer." -}}
 {{- end -}}
 {{- if not (regexMatch `^-?[0-9]+$` $healthPortValue) -}}
 {{- fail "service.healthPort must be an integer." -}}
 {{- end -}}
-{{- if not (regexMatch `^-?[0-9]+$` $metricsPortValue) -}}
-{{- fail "service.metricsPort must be an integer." -}}
-{{- end -}}
 {{- $servicePort := int $servicePortValue -}}
 {{- $healthPort := int $healthPortValue -}}
 {{- $metricsPort := int $metricsPortValue -}}
@@ -808,11 +819,11 @@ Validate chart values that Helm would otherwise accept silently.
 {{- if eq $servicePort $healthPort -}}
 {{- fail "service.port and service.healthPort must be different." -}}
 {{- end -}}
-{{- if and (ne $metricsPort 0) (or (lt $metricsPort 1) (gt $metricsPort 65535)) -}}
-{{- fail "service.metricsPort must be 0 (disabled) or between 1 and 65535." -}}
+{{- if or (not (regexMatch `^[0-9]+$` $metricsPortValue)) (lt $metricsPort 1) (gt $metricsPort 65535) -}}
+{{- fail "service.metricsPort must be an integer between 1 and 65535; to disable metrics, set metrics.enabled=false and keep a valid service.metricsPort (default 9090)" -}}
 {{- end -}}
-{{- if and (ne $metricsPort 0) (or (eq $metricsPort $servicePort) (eq $metricsPort $healthPort)) -}}
-{{- fail "service.metricsPort must differ from service.port and service.healthPort." -}}
+{{- if and $metricsEnabled (or (eq $metricsPort $servicePort) (eq $metricsPort $healthPort)) -}}
+{{- fail "service.metricsPort must differ from service.port and service.healthPort when metrics.enabled=true" -}}
 {{- end -}}
 {{- if and .Values.grpcRoute.enabled (dig "replicaRouting" "enabled" false .Values.grpcRoute) -}}
 {{- if ne $workloadKind "statefulset" -}}

deploy/helm/openshell/templates/_toml.tpl

@@ -106,7 +106,7 @@ Top-level lists represent TOML arrays of tables and preserve their YAML order. *
 {{/* The Service and probes own listener ports, so keep runtime listeners aligned. */}}
 {{- $_ := set $gateway "bind_address" (printf "0.0.0.0:%v" .Values.service.port) -}}
 {{- if .Values.service.healthPort -}}{{- $_ := set $gateway "health_bind_address" (printf "0.0.0.0:%v" .Values.service.healthPort) -}}{{- else -}}{{- $_ := unset $gateway "health_bind_address" -}}{{- end -}}
-{{- if .Values.service.metricsPort -}}{{- $_ := set $gateway "metrics_bind_address" (printf "0.0.0.0:%v" .Values.service.metricsPort) -}}{{- else -}}{{- $_ := unset $gateway "metrics_bind_address" -}}{{- end -}}
+{{- if include "openshell.metricsEnabled" . -}}{{- $_ := set $gateway "metrics_bind_address" (printf "0.0.0.0:%v" .Values.service.metricsPort) -}}{{- else -}}{{- $_ := unset $gateway "metrics_bind_address" -}}{{- end -}}
 {{- range $legacyKey, $runtimeKey := dict "logLevel" "log_level" "enableLoopbackServiceHttp" "enable_loopback_service_http" "enableWebsocketTunnel" "enable_websocket_tunnel" "policyValidationFailureMode" "policy_validation_failure_mode" -}}
 {{- if not (hasKey $gateway $runtimeKey) -}}{{- $_ := set $gateway $runtimeKey (get $legacyServer $legacyKey) -}}{{- end -}}
 {{- end -}}
@@ -333,8 +333,8 @@ boundary from the gateway listener so its client CA can stay independent. */}}
 {{- $metrics := .Values.metrics | default dict -}}
 {{- $metricsTls := get $metrics "tls" | default dict -}}
 {{- if get $metricsTls "enabled" -}}
-{{- if not .Values.service.metricsPort -}}
-{{- fail "metrics.tls.enabled requires service.metricsPort to be non-zero" -}}
+{{- if not (include "openshell.metricsEnabled" .) -}}
+{{- fail "metrics.tls.enabled requires metrics.enabled=true" -}}
 {{- end -}}
 {{- if and (get $metricsTls "requireClientCert") (not (get $metricsTls "clientCaSecretName")) -}}
 {{- fail "metrics.tls.requireClientCert requires metrics.tls.clientCaSecretName" -}}

deploy/helm/openshell/templates/service.yaml

@@ -18,7 +18,7 @@ spec:
       {{- if and (eq .Values.service.type "NodePort") .Values.service.nodePort }}
       nodePort: {{ .Values.service.nodePort }}
       {{- end }}
-    {{- if .Values.service.metricsPort }}
+    {{- if include "openshell.metricsEnabled" . }}
     - port: {{ .Values.service.metricsPort }}
       targetPort: metrics
       protocol: TCP

deploy/helm/openshell/tests/gateway_config_test.yaml

@@ -193,14 +193,14 @@ tests:
       - failedTemplate:
           errorMessage: 'metrics.tls.clientCaSecretName must not reuse the gateway client TLS Secret (server.tls.clientTlsSecretName)'
 
-  - it: rejects metrics TLS when the metrics port is disabled
+  - it: rejects metrics TLS when metrics are disabled
     template: templates/statefulset.yaml
     set:
-      service.metricsPort: 0
+      metrics.enabled: false
       metrics.tls.enabled: true
     asserts:
       - failedTemplate:
-          errorMessage: 'metrics.tls.enabled requires service.metricsPort to be non-zero'
+          errorMessage: 'metrics.tls.enabled requires metrics.enabled=true'
 
   - it: defaults to label admission with caller driver config disabled
     template: templates/gateway-config.yaml

deploy/helm/openshell/tests/gateway_ports_test.yaml

@@ -69,7 +69,7 @@ tests:
       service.metricsPort: -1
     asserts:
       - failedTemplate:
-          errorPattern: "service.metricsPort must be 0 \\(disabled\\) or between 1 and 65535"
+          errorPattern: "service.metricsPort must be an integer between 1 and 65535"
 
   - it: rejects a boolean primary port
     template: templates/statefulset.yaml
@@ -117,14 +117,14 @@ tests:
       service.metricsPort: 65536
     asserts:
       - failedTemplate:
-          errorPattern: "service.metricsPort must be 0 \\(disabled\\) or between 1 and 65535"
+          errorPattern: "service.metricsPort must be an integer between 1 and 65535"
 
   - it: accepts boundary ports with metrics disabled
     template: templates/gateway-config.yaml
     set:
       service.port: 1
       service.healthPort: 65535
-      service.metricsPort: 0
+      metrics.enabled: false
     asserts:
       - matchRegex:
           path: data["gateway.toml"]

deploy/helm/openshell/tests/metrics_test.yaml

@@ -0,0 +1,162 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+suite: metrics endpoint enablement
+templates:
+  - templates/gateway-config.yaml
+  - templates/service.yaml
+  - templates/statefulset.yaml
+  - templates/deployment.yaml
+
+tests:
+  - it: enables the default listener and exposes port 9090
+    asserts:
+      - template: templates/gateway-config.yaml
+        matchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^metrics_bind_address = "0\.0\.0\.0:9090"$'
+      - template: templates/service.yaml
+        contains:
+          path: spec.ports
+          content:
+            name: metrics
+            port: 9090
+            targetPort: metrics
+            protocol: TCP
+      - template: templates/statefulset.yaml
+        contains:
+          path: spec.template.spec.containers[0].ports
+          content:
+            name: metrics
+            containerPort: 9090
+            protocol: TCP
+
+  - it: disables metrics without changing gateway and health listeners
+    set:
+      metrics.enabled: false
+      service.metricsPort: 19090
+      gatewayConfig:
+        openshell.gateway:
+          metrics_bind_address: 0.0.0.0:29090
+    asserts:
+      - template: templates/gateway-config.yaml
+        notMatchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^metrics_bind_address'
+      - template: templates/gateway-config.yaml
+        matchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^bind_address = "0\.0\.0\.0:8080"$'
+      - template: templates/gateway-config.yaml
+        matchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^health_bind_address = "0\.0\.0\.0:8081"$'
+      - template: templates/service.yaml
+        lengthEqual:
+          path: spec.ports
+          count: 1
+      - template: templates/statefulset.yaml
+        lengthEqual:
+          path: spec.template.spec.containers[0].ports
+          count: 2
+      - template: templates/statefulset.yaml
+        equal:
+          path: spec.template.spec.containers[0].readinessProbe.httpGet.port
+          value: health
+
+  - it: enables metrics on the retained custom port
+    set:
+      metrics.enabled: true
+      service.metricsPort: 19090
+    asserts:
+      - template: templates/gateway-config.yaml
+        matchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^metrics_bind_address = "0\.0\.0\.0:19090"$'
+      - template: templates/service.yaml
+        contains:
+          path: spec.ports
+          content:
+            name: metrics
+            port: 19090
+            targetPort: metrics
+            protocol: TCP
+      - template: templates/statefulset.yaml
+        contains:
+          path: spec.template.spec.containers[0].ports
+          content:
+            name: metrics
+            containerPort: 19090
+            protocol: TCP
+
+  - it: disables the Deployment metrics port as well
+    template: templates/deployment.yaml
+    set:
+      workload.kind: deployment
+      server.externalDbSecret: external-postgres
+      metrics.enabled: false
+    asserts:
+      - lengthEqual:
+          path: spec.template.spec.containers[0].ports
+          count: 2
+
+  - it: keeps metrics enabled when legacy values omit the enablement setting
+    template: templates/gateway-config.yaml
+    set:
+      metrics.enabled: null
+      service.metricsPort: 19090
+    asserts:
+      - matchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^metrics_bind_address = "0\.0\.0\.0:19090"$'
+
+  - it: rejects a string instead of a boolean enablement setting
+    template: templates/statefulset.yaml
+    set:
+      metrics.enabled: "false"
+    asserts:
+      - failedTemplate:
+          errorMessage: metrics.enabled must be a boolean
+
+  - it: rejects a numeric enablement setting
+    template: templates/statefulset.yaml
+    set:
+      metrics.enabled: 0
+    asserts:
+      - failedTemplate:
+          errorMessage: metrics.enabled must be a boolean
+
+  - it: rejects the legacy zero disable sentinel with migration guidance
+    template: templates/statefulset.yaml
+    set:
+      service.metricsPort: 0
+    asserts:
+      - failedTemplate:
+          errorPattern: 'to disable metrics, set metrics.enabled=false and keep a valid service.metricsPort'
+
+  - it: rejects the legacy boolean disable sentinel with migration guidance
+    template: templates/statefulset.yaml
+    set:
+      service.metricsPort: false
+    asserts:
+      - failedTemplate:
+          errorPattern: 'to disable metrics, set metrics.enabled=false and keep a valid service.metricsPort'
+
+  - it: requires migration of the old port even with explicit disablement
+    template: templates/statefulset.yaml
+    set:
+      metrics.enabled: false
+      service.metricsPort: 0
+    asserts:
+      - failedTemplate:
+          errorPattern: 'service.metricsPort must be an integer between 1 and 65535'
+
+  - it: permits a retained port collision while metrics are disabled
+    template: templates/gateway-config.yaml
+    set:
+      metrics.enabled: false
+      service.metricsPort: 8080
+    asserts:
+      - notMatchRegex:
+          path: data["gateway.toml"]
+          pattern: '(?m)^metrics_bind_address'