feat(supervisor): accept OTLP/HTTP JSON in the agent trace relay

crates/openshell-supervisor/Cargo.toml

@@ -40,7 +40,8 @@ hyper-util = { workspace = true, features = ["tokio", "http1", "server", "server
 miette = { workspace = true }
 nix = { workspace = true }
 opentelemetry = { workspace = true }
-opentelemetry-proto = { workspace = true }
+# `with-serde` is the JSON half of the OTLP/HTTP relay's transcoding.
+opentelemetry-proto = { workspace = true, features = ["with-serde"] }
 prost = { workspace = true }
 prost-types = { workspace = true }
 rustls = { workspace = true }

crates/openshell-supervisor/README.md

@@ -125,9 +125,9 @@ Console JSON is installed independently of optional file appenders and uses an I
 
 ## OTLP agent trace relay
 
-The `otlp_relay` module forwards the workload's OTLP/HTTP protobuf trace exports to the collector named by `OPENSHELL_OTLP_AGENT_ENDPOINT`, over the transport named by `OPENSHELL_OTLP_AGENT_PROTOCOL` (`grpc` when unset, or `http/protobuf`). Each export carries the collector headers from the launch bundle's `agent_otlp_headers` and, for an `https` collector, verifies it and authenticates with `agent_otlp_tls`; the supervisor's own span and OCSF exporters use the `traces_` and `ocsf_` fields the same way. The gateway sets those variables only when `[openshell.gateway.otlp.agent]` is enabled, so trace export alone leaves the relay inactive. Metrics and logs exports to the relay are acknowledged and discarded. The workload exports to the reserved address `192.0.0.8:4318`; the sandbox broker stages that connection to the supervisor, and the network proxy hands the stream to the relay's HTTP/1.1 receiver through the generic reserved-destination hook instead of evaluating policy. The receiver replaces the `openshell.sandbox.id` and `openshell.telemetry.source=agent` resource attributes, pushes the re-encoded batch into a bounded buffer, and a single export task forwards the encoded batches to the collector with a lazy connection.
+The `otlp_relay` module forwards the workload's OTLP/HTTP trace exports, as protobuf or JSON, to the collector named by `OPENSHELL_OTLP_AGENT_ENDPOINT`, over the transport named by `OPENSHELL_OTLP_AGENT_PROTOCOL` (`grpc` when unset, or `http/protobuf`). Each export carries the collector headers from the launch bundle's `agent_otlp_headers` and, for an `https` collector, verifies it and authenticates with `agent_otlp_tls`; the supervisor's own span and OCSF exporters use the `traces_` and `ocsf_` fields the same way. The gateway sets those variables only when `[openshell.gateway.otlp.agent]` is enabled, so trace export alone leaves the relay inactive. Metrics and logs exports to the relay are acknowledged and discarded. The workload exports to the reserved address `192.0.0.8:4318`; the sandbox broker stages that connection to the supervisor, and the network proxy hands the stream to the relay's HTTP/1.1 receiver through the generic reserved-destination hook instead of evaluating policy. The receiver transcodes a JSON body to protobuf, replaces the `openshell.sandbox.id` and `openshell.telemetry.source=agent` resource attributes, pushes the re-encoded batch into a bounded buffer, and a single export task forwards the encoded batches to the collector with a lazy connection.
 
-Limits are fixed: 2 MiB per request body, at most 512 `ResourceSpans` entries, 256 KiB of encoded `Resource` fields, and 8192 resource attributes per request (each of these answers 413), a 10 second header-read timeout, 16 concurrent connections, 32 buffered batches, and a 5 second deadline per export attempt. Only the resources are decoded; spans are copied byte for byte, at most two requests at a time. Worst-case memory attributable to the agent is about 67 MiB of buffered batches, plus 32 MiB of bodies awaiting enrichment across the 16 connections, plus about 12 MiB for the two enrichments in progress, plus about 4 MiB for the batch being exported and its encoded copy. A full buffer answers 503 with `Retry-After`. Without an agent endpoint nothing starts and the proxy refuses the relay address. After the agent exits, `run_sandbox_with_backend` flushes buffered batches for up to 5 seconds before it reports the exit to the gateway, then logs a final summary of rejected and failed counts. Flushes skip the wait while the most recent export failed, and batches still pending when the relay is dropped without a flush count as failed. The supervisor tells the sandbox runtime through `AgentSpec::otlp_relay_active` whether the relay runs, and the runtime injects the generic `OTEL_EXPORTER_OTLP_*` defaults only then.
+Limits are fixed: 2 MiB per request body, at most 512 `ResourceSpans` entries, 256 KiB of encoded `Resource` fields, and 8192 resource attributes per request (each of these answers 413), a 10 second header-read timeout, 16 concurrent connections, 32 buffered batches, and a 5 second deadline per export attempt. A JSON body is also refused with 413 when it nests more than 32 levels or holds more than 100,000 values (every `{`, `[` and `,` outside strings), checked before it is decoded, and a JSON error message returned to the agent is cut to 1 KiB. Two requests are enriched at a time, on the blocking pool. For a protobuf body only the resources are decoded and spans are copied byte for byte; a JSON body is decoded in full, so `json.rs` bounds it by depth and value count instead of size. The decoder cannot read an `arrayValue` or `kvlistValue` without `values` or a `doubleValue` given as `"NaN"`, which answer 400. Worst-case memory attributable to the agent is about 67 MiB of buffered batches, plus 32 MiB of bodies awaiting enrichment across the 16 connections, plus about 12 MiB for the two enrichments in progress (about 88 MiB when both are JSON, from a worst case of about 44 MiB measured within the limits), plus about 4 MiB for the batch being exported and its encoded copy. A full buffer answers 503 with `Retry-After`. Without an agent endpoint nothing starts and the proxy refuses the relay address. After the agent exits, `run_sandbox_with_backend` flushes buffered batches for up to 5 seconds before it reports the exit to the gateway, then logs a final summary of rejected and failed counts. Flushes skip the wait while the most recent export failed, and batches still pending when the relay is dropped without a flush count as failed. The supervisor tells the sandbox runtime through `AgentSpec::otlp_relay_active` whether the relay runs, and the runtime injects the generic `OTEL_EXPORTER_OTLP_*` defaults only then.
 
 ## Safe runtime and parent liveness
 

crates/openshell-supervisor/src/otlp_relay/enrichment.rs

@@ -100,14 +100,17 @@ const BUFFER_UNDERFLOW: &str = "buffer underflow";
 /// The request could not be enriched.
 #[derive(Debug)]
 pub enum EnrichError {
-    /// The bytes are not a well-formed protobuf message for this schema.
+    /// The bytes are not a well-formed protobuf message for this schema, or a
+    /// JSON body is not a well-formed OTLP/JSON request.
     Malformed(String),
     /// More than [`MAX_RESOURCE_SPANS`] entries.
     TooManyResourceSpans,
     /// More than [`MAX_RESOURCE_ATTRIBUTES`] resource attributes in total.
     TooManyResourceAttributes,
     /// The `resource` fields of the request exceed [`MAX_RESOURCE_BYTES`].
     ResourceTooLarge,
+    /// A JSON body exceeds the shape limits in [`super::json`].
+    JsonTooComplex,
 }
 
 impl std::fmt::Display for EnrichError {
@@ -126,6 +129,12 @@ impl std::fmt::Display for EnrichError {
                     "resource fields exceed {MAX_RESOURCE_BYTES} bytes in total"
                 )
             }
+            Self::JsonTooComplex => write!(
+                f,
+                "JSON body nests more than {} levels or holds more than {} values",
+                super::json::MAX_DEPTH,
+                super::json::MAX_VALUES
+            ),
         }
     }
 }