feat(examples): score the smoke run's behavior into a pinned Trust Index

.github/workflows/openshell-smoke.yml

@@ -1,7 +1,9 @@
 # Runs the extension against the pinned OpenShell release with a real gateway,
-# supervisor and sandbox on the Docker driver. Slower than the unit suite and
-# dependent on GitHub's Docker daemon and ghcr.io, so it runs on pushes to main,
-# on pull requests that touch what it exercises, and on demand.
+# supervisor and sandbox on the Docker driver, once for the identity checks and
+# once with the run's behavior scored into a pinned agent-trust-discovery.
+# Slower than the unit suite and dependent on GitHub's Docker daemon and
+# ghcr.io, so it runs on pushes to main, on pull requests that touch what it
+# exercises, and on demand. Neither job is a required status check.
 name: OpenShell smoke
 
 on:
@@ -12,7 +14,10 @@ on:
     paths:
       - "cmd/ans-openshell-middleware/**"
       - "cmd/ans-reference-callee/**"
+      - "cmd/ans-ocsf-hydrator/**"
       - "internal/**"
+      - "pkg/**"
+      - "deploy/examples/trust-index-score-signals.yaml"
       - "gen/**"
       - "third_party/openshell/**"
       - "examples/openshell-smoke/**"
@@ -27,34 +32,49 @@ permissions:
 
 jobs:
   smoke:
-    name: Gateway, supervisor and sandbox on Docker
+    name: ${{ matrix.target }}
     runs-on: ubuntu-latest
     timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        target: [openshell-smoke, openshell-smoke-trust-index]
+        include:
+          # The gateway runs at debug so a middleware_failed verdict on CI
+          # explains itself; smoke.sh refuses debug under --with-trust-index
+          # (see its usage text), so that run stays at info.
+          - target: openshell-smoke
+            log_level: debug
+          - target: openshell-smoke-trust-index
+            log_level: info
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #v7.0.1
       - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e #v7.0.0
         with:
           go-version: "1.26.9"
-      - name: Cache OpenShell release assets
+      - name: Cache release assets
+        # Keyed per target: the two jobs cache different asset sets, and a shared
+        # key would be saved by whichever job finished first and never refilled.
         uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 #v4.3.0
         with:
           path: ~/.cache/ans-openshell-smoke
-          key: openshell-${{ hashFiles('examples/openshell-smoke/openshell-release.env') }}-${{ runner.os }}-${{ runner.arch }}
+          key: ${{ matrix.target }}-${{ hashFiles('examples/openshell-smoke/*-release.env') }}-${{ runner.os }}-${{ runner.arch }}
       - name: Run the smoke test
         # A fixed TMPDIR makes the retained log directory addressable by the
-        # upload step below; the gateway runs at debug so a middleware_failed
-        # verdict on CI explains itself.
+        # upload step below.
         env:
           TMPDIR: ${{ runner.temp }}/smoke
-          ANS_SMOKE_LOG_LEVEL: debug
+          ANS_SMOKE_LOG_LEVEL: ${{ matrix.log_level }}
         run: |
           mkdir -p "$TMPDIR"
-          make openshell-smoke
+          make ${{ matrix.target }}
       - name: Upload retained logs
-        if: failure()
+        # Also on cancellation, so a run cut off by timeout-minutes still
+        # leaves its logs behind.
+        if: failure() || cancelled()
         uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 #v7.0.2
         with:
-          name: openshell-smoke-logs
+          name: ${{ matrix.target }}-logs
           path: ${{ runner.temp }}/smoke/ans-openshell-smoke.*/logs/
           if-no-files-found: ignore
           retention-days: 7

Makefile

@@ -5,7 +5,7 @@ LINT_GOTOOLCHAIN ?= local
 BIN := bin
 CMDS := ans-openshell-middleware ans-openshell-interceptor ans-ocsf-hydrator ans-reference-callee
 
-.PHONY: all build proto test cover lint vet demo openshell-smoke clean
+.PHONY: all build proto test cover lint vet demo openshell-smoke openshell-smoke-trust-index clean
 
 all: build
 
@@ -48,3 +48,8 @@ clean:
 # End-to-end check against a real OpenShell gateway (Docker). See examples/openshell-smoke/README.md.
 openshell-smoke:
 	examples/openshell-smoke/smoke.sh
+
+# The same run, then its OCSF records scored by the hydrator and imported into a pinned
+# agent-trust-discovery. See "Scoring the run's behavior" in the same README.
+openshell-smoke-trust-index:
+	examples/openshell-smoke/smoke.sh --with-trust-index

README.md

@@ -62,6 +62,11 @@ down. See [examples/openshell-smoke](examples/openshell-smoke/README.md):
 make openshell-smoke
 ```
 
+`make openshell-smoke-trust-index` runs the same test and then scores the run's behavior: it
+enables OpenShell's OCSF export, feeds the supervisor's records through the hydrator into a pinned
+`agent-trust-discovery` release it downloads and runs, and checks the score on the agent's trust
+evaluation.
+
 ## Running the services
 
 Every service takes one YAML file. Flags: `--config` (required), `--listen` to override the
@@ -75,7 +80,9 @@ configured address, `--log-level` (`debug`, `info`, `warn`, `error`), `--log-for
 ```
 
 The hydrator is a batch job over the sandbox supervisors' OCSF records. Enable OpenShell's JSON
-export with `openshell settings set --global --key ocsf_json_enabled --value true`; each
+export with `openshell settings set --global --key ocsf_json_enabled --value true` (at a terminal
+the CLI asks you to confirm that a global value disables per-sandbox control of the key; pass
+`--yes` only where no terminal is attached); each
 supervisor then writes one `OCSF-JSON` line per event to its stderr and, where its `/var/log` is
 writable, to `/var/log/openshell-ocsf.<date>.jsonl` inside the supervisor container. Collect
 either with your log pipeline into a file the hydrator can read, one bare line per record: it

docs/hydrator-activity.md

@@ -178,7 +178,12 @@ the example leaves it unset.
   The evidence output keeps the refusal counts.
 - The score is rounded, so a single violation among 200 or more decisions still
   reports 100. The evidence output keeps the exact counts.
-- Rollout behavior (weight 0 has no effect on trust decisions, retained scores,
-  whole-batch rejection) belongs to the consumer and is not tested in this
-  repository. Verify it against the deployed `agent-trust-discovery` before
-  changing the weight.
+- Rollout behavior belongs to the consumer. `make openshell-smoke-trust-index`
+  exercises the registration from
+  [trust-index-score-signals.yaml](../deploy/examples/trust-index-score-signals.yaml),
+  the import, and the weight-0 state (the score is listed on the agent and the
+  behavior dimension stays 0) against the pinned `agent-trust-discovery` release
+  named in [trust-index-release.env](../examples/openshell-smoke/trust-index-release.env).
+  Retention of stale scores and whole-batch rejection are not exercised there;
+  verify both against the deployed `agent-trust-discovery` before changing the
+  weight.

examples/openshell-smoke/README.md

@@ -7,7 +7,9 @@ that OpenShell invokes the service, applies its header mutations before the requ
 and blocks the request when the service is down.
 
 > OpenShell's supervisor-middleware contract is a research preview upstream and may change.
-> The test pins one release; see [Updating the OpenShell pin](#updating-the-openshell-pin).
+> The test pins one OpenShell release, and one agent-trust-discovery release for the behavior
+> path; see [Updating the OpenShell pin](#updating-the-openshell-pin) and
+> [Updating the Trust Index pin](#updating-the-trust-index-pin).
 
 ## What it does
 
@@ -46,6 +48,41 @@ make openshell-smoke
 Replay and body tampering are covered by the verifier's own tests and by `make demo`, and are
 reported there, not here.
 
+## Scoring the run's behavior
+
+```bash
+make openshell-smoke-trust-index
+```
+
+The same run with `--with-trust-index`, which adds the behavior path: the policy decisions
+OpenShell makes for the sandbox become a Trust Index score. Before the callee starts, the harness
+starts a pinned [agent-trust-discovery](https://github.com/agentnameservice/agent-trust-discovery)
+release on a loopback port, registers the fixture agent with it, and, once the gateway is up,
+turns on OpenShell's OCSF JSON export for the run's sandboxes. The Trust Index loads the signal
+registration this repository ships for operators,
+[deploy/examples/trust-index-score-signals.yaml](../../deploy/examples/trust-index-score-signals.yaml),
+and a profile that names only that signal, at weight 0. Right after the sandbox is created the
+run proves the export is live, so that check prints before the identity checks; after them it
+makes one request the policy does not allow, collects the supervisor's OCSF records, scores them
+with the hydrator built from this checkout and reads the score back. Five more checks:
+
+| Check | Expected observation |
+|-------|----------------------|
+| Export live | Before the first request, the approved sandbox's supervisor log already carries `OCSF-JSON` records; the supervisor applies `ocsf_json_enabled` on its first settings poll. |
+| Policy-denied request | A `DELETE /v1/tasks`, which the policy does not allow, gets `403 policy_denied` from the supervisor, and the callee's log does not grow. |
+| Attributable records | Every collected record names the approved sandbox by its gateway-issued id, and the records hold exactly the run's L7 decisions: one `allow` per request the sandbox made, including the ones answered `middleware_failed`, and the one `deny`. |
+| Hydrator score | The hydrator's evidence shows those counts and a score that is not withheld; the import to the Trust Index succeeds. The score's formula belongs to the hydrator and its unit tests, so the harness reads the computed value rather than restating the formula. |
+| Trust Index read-back | The agent's trust evaluation lists `openshell.behavior.activity` under `behavior` with the imported score at weight 0, and the behavior dimension stays 0. |
+
+Only the approved sandbox is mapped in the hydrator configuration, by its gateway-issued id, and
+only its supervisor's records are collected; the unapproved sandbox never enters the behavior
+path. The import route runs with a per-run
+admin key that reaches the server through its environment and the hydrator and `curl` through
+files, so no command line and no retained log carries it. The Trust Index listens on
+`127.0.0.1` only, which is the one control on its unauthenticated read routes, and the hydrator's
+hop to it is plaintext on that interface. A run under the flag downloads and executes a second
+pinned third-party binary, verified the same way as the OpenShell assets.
+
 ## Requirements
 
 - Linux (x86_64 or arm64) or macOS on Apple silicon, with Bash, `curl`, `jq`, `tar`,
@@ -73,11 +110,13 @@ the next run removes everything under a label whose pid no longer runs.
 
 | Flag or variable | Effect |
 |------------------|--------|
-| `--keep-running` | After the assertions pass, keep the gateway, middleware, callee and sandbox up until Ctrl-C and print the commands to use them. |
-| `--print-config` | Print the rendered `gateway.toml` and `middleware.yaml` and exit. |
+| `--keep-running` | After the assertions pass, keep the gateway, middleware, callee and sandbox up until Ctrl-C and print the commands to use them. Under `--with-trust-index` the Trust Index stays up too and its detail query is printed. |
+| `--print-config` | Print the rendered `gateway.toml` and `middleware.yaml` and exit. Under `--with-trust-index` the Trust Index and hydrator configuration follow. |
+| `--with-trust-index` | Add the behavior checks: OCSF export, the hydrator, and a pinned `agent-trust-discovery` (see [Scoring the run's behavior](#scoring-the-runs-behavior)). |
 | `ANS_SMOKE_HOST` | Address the gateway and sandboxes use to reach the middleware and callee. |
 | `ANS_SMOKE_CACHE` | Where release assets are cached. |
-| `ANS_SMOKE_LOG_LEVEL` | Gateway log level, default `info`. |
+| `ANS_SMOKE_LOG_LEVEL` | Gateway log level, default `info`. The Docker driver forwards it to the supervisors, whose console queue is lossy and carries the OCSF records the behavior path counts, so `--with-trust-index` refuses `debug` and `trace`. |
+| `ANS_SMOKE_TRUST_INDEX_LOG_LEVEL` | Log level of the Trust Index and the hydrator under `--with-trust-index`, default `info`. |
 
 ## Fixture trust versus production trust
 
@@ -102,6 +141,7 @@ fixtures minted so the middleware's JWKS matches.
 | Vendored extension protos | `OPENSHELL_COMMIT` in the same file | CI's proto-drift job diffs them against that commit |
 | Supervisor and sandbox runtime images | manifest-list digests in the same file, passed to the gateway as `supervisor_image` and `sandbox_runtime_image` | the container engine, by digest |
 | Workload base image | manifest-list digest in [workload/Dockerfile](workload/Dockerfile) | the container engine, by digest |
+| agent-trust-discovery tarball (`--with-trust-index`) | SHA-256 per platform in [trust-index-release.env](trust-index-release.env), copied from the release's `checksums.txt` (the release publishes no signature or attestation, so the digest and the tarball come from the same page); `TRUST_INDEX_COMMIT` is printed in the run summary | `smoke.sh`, every run, before extraction into the run directory |
 
 The `apt-get install curl` layer in the workload image is the one unpinned input; the identity
 assertions do not rest on it, because the callee's own log corroborates the two denial checks and
@@ -120,8 +160,8 @@ its rejection logging is proven live first.
 
 ## Ownership
 
-The harness, its OpenShell pin and the `OpenShell smoke` workflow are maintained by this
-repository's maintainers under its normal [contribution process](../../CONTRIBUTING.md); a
+The harness, its OpenShell and agent-trust-discovery pins and the `OpenShell smoke` workflow are
+maintained by this repository's maintainers under its normal [contribution process](../../CONTRIBUTING.md); a
 maintainer roster is proposed in the governance change under review. File problems as issues
 on this repository. The link from OpenShell's supervisor-middleware documentation belongs to
 the upstream contribution and is not part of this repository.
@@ -134,3 +174,15 @@ the upstream contribution and is not part of this repository.
 2. Re-pin the vendored contract (`third_party/openshell`, then `make proto`) if the proto-drift
    job reports a difference at the new commit.
 3. Run `make openshell-smoke`.
+
+## Updating the Trust Index pin
+
+1. Edit [trust-index-release.env](trust-index-release.env): the version, the tag's commit, and
+   the SHA-256 of each platform's tarball from the release's `checksums.txt`.
+2. The release tarballs ship no configuration, so [trust-index.sh](trust-index.sh) carries the
+   contract the harness relies on: the `runtime.yaml` and profile keys it writes, the agent
+   import body it posts, and the trust-evaluation fields it reads back. Compare them with the
+   new release's `config/runtime.yaml`, `config/default-profile.yaml`, agent import handler and
+   agent detail response, and adjust the script where they differ. The signal registration stays
+   `deploy/examples/trust-index-score-signals.yaml`.
+3. Run `make openshell-smoke-trust-index`.