Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .claude/rules/bff-go.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,12 +191,13 @@ answers a request without a bearer with `unauthorized` (`pkg/auth/proxy.go`).
The SDK is pinned to the commit of an OpenShell **release tag** and moves
together with the gateway the branch pins, in one commit (ADR 0005, decision 7;
hard facts 17 and 20 in `openshell-api.md`). The Follow upstream workflow makes
that commit, on the `next` branch: never make it by hand, and never
`go get ...@latest` or a branch. A person does the work the move makes
necessary, in a pull request into `next`: the call sites and the test doubles
in `mock_sdk_test.go` when `check-backend` is red on the `next` pull request,
and the dashboard itself when `compat` is red there or the issue labelled
`compat-migrate` is open. How the workflow works is in
that commit, on a `bump/openshell-X.Y.Z` branch with a pull request into
`main`: never make it by hand, and never `go get ...@latest` or a branch. The
bot creates the branch and never touches it again. A person does the work the
move makes necessary, pushed to that bump branch: the call sites and the test
doubles in `mock_sdk_test.go` when `check-backend` is red on the bump pull
request, and the dashboard itself when `compat` is red there or the issue
labelled `compat-migrate` is open. How the workflow works is in
[deploy/ci/upstream/README.md](../../deploy/ci/upstream/README.md).

To inspect an RPC or a type, read the SDK package (`openshell/v1`, `types/*`)
Expand Down
18 changes: 9 additions & 9 deletions .claude/rules/openshell-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ alwaysApply: false

# OpenShell API Reference

**THE OPENSHELL GO SDK IS THE SOURCE OF TRUTH.** The BFF calls `github.com/NVIDIA/OpenShell/sdk/go` directly, and this repository holds no copy of the gateway's proto files and no generated stubs. Before writing any handler or TypeScript type, read the actual RPC/message definitions in `$(go env GOMODCACHE)/github.com/!n!v!i!d!i!a/!open!shell/sdk/go@<version>/openshell/v1/` (hand-written client wrappers with doc comments; `types/*.go` for message shapes) or via `go doc`. Never invent RPCs, fields, or lifecycle states. If a UI idea has no backing RPC, flag it — do not fabricate an endpoint. The SDK pin moves only together with the gateway the branch pins, to the commit of an upstream *release tag*, and the Follow upstream workflow makes that move on the `next` branch (hard facts 17 and 20). Never run `go get github.com/NVIDIA/OpenShell/sdk/go@latest`.
**THE OPENSHELL GO SDK IS THE SOURCE OF TRUTH.** The BFF calls `github.com/NVIDIA/OpenShell/sdk/go` directly, and this repository holds no copy of the gateway's proto files and no generated stubs. Before writing any handler or TypeScript type, read the actual RPC/message definitions in `$(go env GOMODCACHE)/github.com/!n!v!i!d!i!a/!open!shell/sdk/go@<version>/openshell/v1/` (hand-written client wrappers with doc comments; `types/*.go` for message shapes) or via `go doc`. Never invent RPCs, fields, or lifecycle states. If a UI idea has no backing RPC, flag it — do not fabricate an endpoint. The SDK pin moves only together with the gateway the branch pins, to the commit of an upstream *release tag*, and the Follow upstream workflow makes that move on a `bump/openshell-X.Y.Z` branch (hard facts 17 and 20). Never run `go get github.com/NVIDIA/OpenShell/sdk/go@latest`.

**There are three direct gRPC calls**, all in `backend/pkg/clients`, each for a named gap in the SDK and all on the proto client the SDK ships. `rawexec.go` does binary-safe file uploads, because the public `Exec().Run(...)` API takes no stdin and `Exec().Interactive(...)` always asks for a PTY (hard fact 22). `rawprovider.go` reads the keys of the credentials a provider holds, because the SDK's `ProviderFromProto` drops the map the gateway returns them in (hard fact 8); `rawprovider_allworkspaces.go` is the same read for the all-workspaces list. `rawprofile.go` reads and writes provider profiles, because the SDK's profile endpoint type keeps a host, a port and a protocol and drops everything else the gateway stores (hard fact 24). Do not add another unless the SDK has a gap you can name, and delete each one when its gap closes.

Expand Down Expand Up @@ -44,14 +44,14 @@ Skip `GatewayInterceptor`, `SupervisorMiddleware`, `ComputeDriver` — internal/
15. **`Delete` returns `(*DeletionResult, error)`, and a nil error does not mean the resource is gone.** `DeletionOutcome` is one of Unspecified / Completed / Accepted / AlreadyAbsent. Only **Completed** and **AlreadyAbsent** establish completion; **Accepted** means the gateway queued asynchronous cleanup, and unrecognized numeric values must not be treated as completion. Convert through `models.FromSDKDeletion` rather than hardcoding `{"deleted": true}`.
16. **Network policy enum fields are proto enums, not free strings.** `access`, `enforcement` and `tls` on a policy network endpoint serialize through protojson as `NETWORK_ACCESS_PRESET_*`, `NETWORK_ENFORCEMENT_MODE_*`, `NETWORK_TLS_MODE_*`. The gateway does not accept the lowercase spellings (`read-only`, `enforce`, `verify`).

17. **Three links, each proven separately — and the gateway and the SDK move together, on `next` (ADR 0005, decisions 6 to 8).** The chain is gateway → SDK → BFF → UI.
17. **Three links, each proven separately — and the gateway and the SDK move together, on a bump branch (ADR 0005, decisions 6 to 8 and the amendment of 2026-11-06).** The chain is gateway → SDK → BFF → UI.
- **Wire** (gateway ↔ SDK) is proven only by running `backend/test/compat` against a real gateway; it is never inferred from version numbers. **Source** (SDK ↔ BFF) is proven by the compiler, `go vet` and the unit tests. BFF and UI ship together.
- **Say which link failed.** A compile error after an SDK move is a source migration, never "the gateway is incompatible".
- **A branch pins ONE upstream release** in `deploy/ci/gateway-pins.json` — the gateway and supervisor images and the SDK at that release's tag — and every change is tested against that gateway.
- **Moving to another release is one commit that moves all of it:** the pins file, `backend/go.mod`, `backend/go.sum`, the built-in release line when the minor changes, and the README's generated block. On `main` and `next` nobody makes that commit by hand. `follow-upstream.yml` reads upstream's tags every hour and makes it on the `next` branch, as the first commit after `main`, titled `fix: move to OpenShell X.Y.Z`. `next` is the bump pull request opened early, and merging it moves `main` and cuts a release: the next patch, or `X.Y.0` when the move starts a new gateway minor.
- **Upstream pre-releases are tested ahead of the stable release in two ways:** *check A*, the BFF as `main` builds it against the target gateway (a failure is the one issue labelled `compat-migrate`), and *check B*, CI on the `next` pull request, the dashboard on the target's SDK against the target's gateway.
- **A change that needs the upcoming release** — an SDK method or an RPC the released gateway does not have — goes to `next`; everything else goes to `main`.
- **An agent never moves the pins, never pushes to `next` and never opens its pull request.** If a move is needed and the workflow has not made it, stop and say so.
- **Moving to another release is one commit that moves all of it:** the pins file, `backend/go.mod`, `backend/go.sum`, the built-in release line when the minor changes, and the README's generated block. Nobody makes that commit by hand. `follow-upstream.yml` reads upstream's tags once a day and, for a new stable release, makes it on `bump/openshell-X.Y.Z`: `main` plus that one commit, titled `fix: move to OpenShell X.Y.Z`, with a pull request into `main`. A person can run it with a `version` to bump to a given release, a pre-release included. The bot creates a bump branch once and never rebases, updates or force-pushes it; after that it is an ordinary branch. Merging the pull request moves `main` and releases nothing: a person cuts the release (`publish.yml`), the next patch, or `X.Y.0` when the move starts a new gateway minor.
- **Upstream pre-releases are tested ahead of the stable release in two ways:** *check A*, the BFF as `main` builds it against the target gateway (a failure is the one issue labelled `compat-migrate`), and CI on the bump pull request, the dashboard on the target's SDK against the target's gateway.
- **A change that needs the upcoming release** — an SDK method or an RPC the released gateway does not have — goes on that release's bump branch; before upstream ships it, on a prep branch made by running Follow upstream with the pre-release as `version`. Everything else goes to `main`.
- **An agent never moves the pins by hand.** The Follow upstream bump does. An agent may push fixes to a bump branch when asked, like any other branch. If a move is needed and no bump branch exists, stop and say so: a person runs the workflow.
- **The compat suite proves only what it varies.** A request field the gateway silently drops fails nothing (fact 18), so new compat tests should vary what the dashboard sends, starting with a non-default workspace.
18. **The wire link really breaks — loudly or silently — so what a build supports is measured, not assumed.** The example is the change from `string workspace` to a `WorkspaceSelector workspace_scope` message on every request, between gateway 0.0.116 and 0.1.0 (measured 2026-10-05).
- **Upstream made the change in two steps.** An SDK in between put the selector at *new* field numbers and reserved the old ones (`GetSandboxRequest` 2 → 3, `ListSandboxesRequest` 4 → 6, `CreateSandboxRequest` 5 → 8). Release 0.1.0 then moved it onto the *old* `workspace` numbers (`GetSandboxRequest` 2, `ListSandboxesRequest` 4, `ListProvidersRequest` 3; 7 on `CreateSandboxRequest`).
Expand All @@ -62,12 +62,12 @@ Skip `GatewayInterceptor`, `SupervisorMiddleware`, `ComputeDriver` — internal/
- **What a build supports is one gateway minor release line, not a range of versions (ADR 0005).** The line is the major.minor of the release `deploy/ci/gateway-pins.json` names (`node scripts/gateway-range.mjs` prints it), and that release is the one gateway the compat suite runs against for every change to the branch. Every gateway of the line is `supported` — any patch, pre-release, dev build or rebuild — and the BFF reports any other line as `unsupported`. The rest of the line is claimed because upstream promises no Stable break inside a minor, not because it was run here.
- **Every compat run passes `-count=1`,** because Go's test cache cannot see the gateway and replays a pass against one gateway as `ok (cached)` for another.
19. **The gateway's TOML config is versioned and the schemas are mutually exclusive.** Releases up to `0.0.116` need `version = 1`; `0.1.0` and newer need `version = 2`, which renames `compute_drivers` (array) to `compute_driver` (string), switches `image_pull_policy` to snake_case (`if_not_present`), and drops `sandbox_namespace`. A v1 config on a v2 gateway fails to start outright. `deploy/ci/gateway.e2e.v1.toml.tmpl` / `.v2.` hold both; select with `OPENSHELL_CONFIG_SCHEMA` (`auto` tries v2, then v1). `gateway-pins.json` names the schema of the pinned gateway (`config_schema`); a pin move carries it over, so a release that needs a new schema fails the compat suite until a person adds the template.
20. **A branch pins one upstream release — never `latest`, `dev` or `@latest`, and a pre-release only on `next`.**
20. **A branch pins one upstream release — never `latest`, `dev` or `@latest`, and a pre-release never on `main`.**
- **Tags.** On ghcr.io the gateway tag `latest` is the newest *release* and `dev` is upstream HEAD; both move. Upstream tags a stable release `vX.Y.Z` and a pre-release `vX.Y.Z-pre.N`. Those two shapes are the only tags followed; anything else (`dev`, `vm-runtime`) is ignored. Upstream HEAD is not watched: pre-releases are, and a stable release is cut from one.
- **The pins file.** `deploy/ci/gateway-pins.json` holds `release` (without the `v`), that release's gateway and supervisor images pinned by digest (`tag@sha256:…`, the tag being the release), `sdk`, `sandbox_image` and `config_schema`.
- **Which branch pins what.** `main` and `release/**` pin a stable release. `next` pins the upcoming one, a pre-release until upstream cuts it, and `ci.yml`'s `pins a stable release` check refuses a pre-release in a pull request into any other branch. That check is deliberately not `compat`: it means "not released upstream yet", not "does not work".
- **Which branch pins what.** `main` and `release/**` pin a stable release. A bump branch pins the release in its name; only a prep branch a person asked for pins a pre-release, and `ci.yml`'s `pins a stable release` check refuses a pre-release in every pull request, so a prep branch cannot merge until upstream ships. That check is deliberately not `compat`: it means "not released upstream yet", not "does not work".
- **Never pin a `dev` build, with or without a digest.** A dev gateway pulls `ghcr.io/nvidia/openshell/sandbox:dev` at runtime, so it drifts with upstream HEAD anyway.
- **The SDK pin.** The SDK module has no tags of its own, so the pin is the pseudo-version of the commit the release tag points at (`v0.1.3` → `sdk/go@v0.0.0-20261009050449-e1f3c82caa3e`), written identically in `backend/go.mod` and in the `sdk` field. CI fails when they differ, when the pins file is malformed, or when it is not in canonical form (`python3 deploy/ci/upstream/follow.py format-pins` rewrites it). CI cannot tell a release tag's commit from any other commit; the Follow upstream workflow can, and its run summary says so when `main`'s SDK is not at the tag of the release it pins.
- **The SDK pin.** The pin is the SDK at the commit the release tag points at, written identically in `backend/go.mod` and in the `sdk` field. Up to 0.1.6 the SDK module had no tags of its own, so the pin is a pseudo-version of that commit (`v0.1.3` → `sdk/go@v0.0.0-20261009050449-e1f3c82caa3e`). From 0.1.7 upstream tags the module itself (`sdk/go/v0.1.7`), `go get` of the release commit resolves to that tag, and the pin is `v0.1.7`; the bump checks with `go list -m -json` that the tag is the release commit. CI fails when they differ, when the pins file is malformed, or when it is not in canonical form (`python3 deploy/ci/upstream/follow.py format-pins` rewrites it). CI cannot tell a release tag's commit from any other commit; the Follow upstream workflow can, and its run summary says so when `main`'s SDK is not at the tag of the release it pins.
- **The line in the BFF.** One more thing restates the pins: the gateway release line compiled into the BFF (`BuiltInGatewayReleaseLine` in `backend/pkg/models/gateway_release_line.go`), so that an image knows its line with no build args and no environment. It must be the major.minor of the pinned release, and `node scripts/gateway-range.mjs --check` fails in CI when it is not; the pin move changes the constant when the minor changes.

21. **Gateway settings are typed and type-checked.** `SettingValue` is a oneof of string, bool, int and bytes. Each registered key takes exactly one kind and the gateway refuses any other (`setting 'ocsf_json_enabled' expects bool value`). Gateways 0.1.0 to 0.1.2 register the same four keys: `ocsf_json_enabled` and `agent_policy_proposals_enabled` are bools, `ocsf_schema_version` and `proposal_approval_mode` are strings with a fixed set of values. `GetGatewayConfig` lists every registered key, and one that was never set comes back with an *empty* value: no kind. The API does not expose the registry (the CLI and TUI link it in), so the type of an unset setting cannot be read. The BFF mirrors all of this: a setting is a JSON string, boolean or integer in both directions, absent when unset, and nothing is coerced — the Settings page edits a value in the type it has and asks for the type when it has none.
Expand Down
9 changes: 6 additions & 3 deletions .claude/skills/add-rpc/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,12 @@ Start with `openshell/v1/` and `types/` in the pinned module version. Note:
If the capability exists upstream but not in the **pinned** SDK, do not run
`go get ...@latest` to reach it. The SDK moves with the gateway, to the commit
of an upstream release tag, and the Follow upstream workflow makes that move
on the `next` branch (ADR 0005, decision 7; see "SDK updates" in
`.claude/rules/bff-go.md`). If the capability is in the upcoming release, the
endpoint is work for `next`: branch from it and open the pull request into it.
on a `bump/openshell-X.Y.Z` branch (ADR 0005, decision 7 and the amendment of
2026-11-06; see "SDK updates" in `.claude/rules/bff-go.md`). If the capability
is in the upcoming release, the endpoint is work for that release's bump
branch, or, before upstream ships it, for a prep branch made by running Follow
upstream with the pre-release as `version`. Do not start that run yourself:
say what is needed and stop.
Also check that the gateway this branch pins (`node scripts/gateway-range.mjs`
prints its release) actually serves the RPC: a gateway answers gRPC
`UNIMPLEMENTED` for an RPC it does not have.
Expand Down
Loading