Repository navigation
feat(examples): score the smoke run's behavior into a pinned Trust Index - #27
Open
kperry-godaddy wants to merge 1 commit into
Open
kperry-godaddy wants to merge 1 commit into
kperry-godaddy wants to merge 1 commit into
Conversation
Add make openshell-smoke-trust-index, the OpenShell smoke run plus five checks that take the run's policy decisions to a Trust Index score: the gateway's OCSF JSON export is switched on, the approved sandbox's supervisor records are collected, ans-ocsf-hydrator scores them and imports the score into a pinned agent-trust-discovery release the harness starts on a loopback port, and the score is read back from the agent's trust evaluation at weight 0. make openshell-smoke is unchanged. The behavior path lives in trust-index.sh, sourced by smoke.sh only under --with-trust-index. The Trust Index loads the signal registration this repository ships for operators and a profile naming only that signal. The per-run admin key travels through the server's environment and files for the hydrator and curl, never a command line or a retained log. The decision count the assertions expect comes from sandbox_curl, so it cannot drift from the checks that run. Release downloads use a per-run partial file, which also closes a latent race between two cold-cache OpenShell runs. CI runs both targets as matrix legs with per-leg cache keys; the trust-index leg runs the gateway at info because the supervisor's lossy console queue is shared with the records it counts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: kperry <kperry@godaddy.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
make openshell-smoke-trust-index: the existing OpenShell smoke run plus five checks that take the run's policy decisions all the way to a Trust Index score. The gateway's OCSF JSON export is switched on, the approved sandbox's supervisor records are collected,ans-ocsf-hydratorscores them and imports the score into a pinnedagent-trust-discoveryrelease the harness starts on a loopback port, and the score is read back from the agent's trust evaluation at weight 0.make openshell-smokeis unchanged.Until now the behavior signal
openshell.behavior.activityhad no automated real-runtime proof: the hydrator was covered by unit tests over synthetic records and by hand-run checks on OpenShell v0.1.5, and nothing exercised the import contract against a running Trust Index.Changes
examples/openshell-smoke/trust-index.sh(new): the behavior path, sourced bysmoke.shonly under--with-trust-index. Starts the pinned Trust Index with the signal registration fromdeploy/examples/trust-index-score-signals.yamland a minimal profile, registers the fixture agent, enablesocsf_json_enabledon the run's gateway, and runs checks 6 to 10: export live before the first request, a policy-deniedDELETEthat never reaches the callee, every collected record attributed to the gateway-issued sandbox id with exactly the run's decisions, the hydrator's evidence and score, and the Trust Index read-back.examples/openshell-smoke/trust-index-release.env(new): agent-trust-discovery v0.1.4 pinned by SHA-256 per platform, copied from the release'schecksums.txt.examples/openshell-smoke/smoke.sh: the flag (which refusesANS_SMOKE_LOG_LEVEL=debug, because the Docker driver forwards the level to the supervisor whose console queue is lossy and carries the records the run counts), hook calls, a per-run partial file in the generalizedfetch_release_asset(closes a latent race between two cold-cache runs), a request journal written bysandbox_curl,failtailing.jsonlevidence too, and a failure-path cleanup that keeps onlylogs/. With the flag off the rendered configuration and the five checks are the same as before.Makefile: the new target..github/workflows/openshell-smoke.yml: a two-leg matrix with per-leg cache keys and log levels (the trust-index leg runs atinfobecause the supervisor's console queue is lossy and shared with the records the leg counts), and the path filter gains the hydrator command,pkg/, and the signals file.--yeson thesettings setexample, which the CLI requires off a TTY); the hydrator doc's rollout bullet now says what the target exercises and what it still does not (stale-score retention, whole-batch rejection).The per-run admin key reaches the Trust Index through its environment and the hydrator and
curlthrough files, so it is never on a command line or in a retained log; on failure the key, header file and database are removed with the other secrets.Testing
make openshell-smokeon macOS (Rancher Desktop):ALL PASS 5 checks against OpenShell v0.1.7 (5d25211);--print-configoutput identical to main apart from per-run ports and paths.make openshell-smoke-trust-indexon the same daemon, three times:ALL PASS 10 checks against OpenShell v0.1.7 (5d25211) and agent-trust-discovery v0.1.4 (27a30a1); the hydrator scored 75 from 3 allowed and 1 denied decision; no containers, volumes, processes or temp directories left.hydrator.log, onlylogs/remains under the run directory, and no retained file contains either key; export left disabled fails check 6 after 60 s with the settings dump, toggle count and log line count;--with-trust-indexwithANS_SMOKE_LOG_LEVEL=debugis refused before anything starts.shellcheck -xon both shell files,actionlinton the workflow,go build,go vet,go test -race ./...,golangci-lint(0 issues). No Go code changed.AI assistance
Assisted-by: Claude Code (claude-fable-5-1)