Skip to content

fix(test-controller): sandbox gate returns PERMISSION_DENIED, but the mode-gate storyboard requires FORBIDDEN #1011

Closed

Description

@numarasSigmaSoftware

adcp/server/test_controller.py's sandbox-authority gate denies live-mode callers with PERMISSION_DENIED:

# adcp/server/test_controller.py — _apply_sandbox_gate
        return _controller_error(
            "PERMISSION_DENIED",
            ...

The AdCP 3.1.1 conformance storyboard that grades this exact path requires FORBIDDEN:

# dist/compliance/3.1.1/universal/comply-controller-mode-gate.yaml
# phase live_account_denial → step deny_live_caller
        validations:
          - check: field_value
            path: 'error'
            allowed_values:
              - 'FORBIDDEN'
            description: 'Error code is FORBIDDEN — seller detected live-mode account and denied dispatch before scenario execution'

The storyboard's own narrative is explicit that this is the graded behavior: "the seller MUST refuse with error: FORBIDDEN before dispatching any scenario," and it describes itself as "the keystone for the (Sandbox) AAO Verified tier."

The prose spec agrees on the code and scopes it precisely — dist/docs/3.1.1/building/by-layer/L3/comply-test-controller.mdx, §Sandbox gating: "FORBIDDEN is reserved for the in-sandbox case where the caller is authorized to call the controller but params reference a non-sandbox account."

Effect: a seller wiring dispatch(...) with an account_resolver and returning its result verbatim fails comply_controller_mode_gate, because the graded field is an exact-match allowlist of one value.

Note the adjacent semantic distinction, in case it affects the fix: the same spec section requires that a live-mode caller probing the tool by name receive the transport's standard unknown-tool error (indistinguishable from a seller that never implemented the controller) rather than any authorization error — FORBIDDEN is only for the in-sandbox caller referencing a non-sandbox account. If the current gate is intended to cover the probing case rather than the referenced-account case, then the divergence may be one of layering rather than a simple string.

Verified against adcp==6.6.0; a2a-sdk==1.0.1.

Related: #1007, #1008, #1009, #1010 (other A2A/server seam reports from the same audit).

Activity

  1. added
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Aug 4, 2026
  2. bokelley commented on Aug 4, 2026

    @bokelley
    Contributor

    Triage

    Classification: Bug
    Bucket(s): handlers
    Status: ready-to-implement

    What the experts said:

    • ad-tech-protocol-expert: storyboard requirement is unambiguous — case 3 (resolver returns live account) requires FORBIDDEN; cases 1 (no resolver wired) and 2 (resolver raised) correctly stay PERMISSION_DENIED as infrastructure/misconfiguration failures, not authorization decisions against a known account.
    • code-reviewer: the split-branch logic is correct (resolved_account is not None and not account_is_sandbox precisely identifies the live-account case); three test assertions need updating; edge case: duck-typed non-Account resolver returns also get FORBIDDEN under the split, which is defensible but worth calling out in the PR description.

    My take: The current if not allowed: at line 692 of _apply_sandbox_gate is over-broad — it merges two semantically distinct denial paths (live account explicitly resolved vs. no account resolved, no sandbox signal) under one PERMISSION_DENIED. The minimal fix is to split that branch: FORBIDDEN when a live account was explicitly resolved, PERMISSION_DENIED for the no-signal / unresolved case. This aligns with both the storyboard's graded assertion and the spec prose distinction.

    Implementation scope (non-breaking, spec-conformance fix):

    • src/adcp/server/test_controller.py — split line 692 if not allowed: into:
      if resolved_account is not None and not account_is_sandbox:
          return _controller_error(
              "FORBIDDEN",
              "comply_test_controller requires a sandbox or mock account; "
              "resolved account is in live mode.",
          )
      if not allowed:
          return _controller_error(
              "PERMISSION_DENIED",
              "comply_test_controller requires a sandbox or mock account; "
              "no account resolved and no sandbox signal present.",
          )
    • tests/test_account_mode_gate.py — update three test assertions from "PERMISSION_DENIED" to "FORBIDDEN":
      • test_resolver_live_denied (line 99)
      • test_spoofed_wire_sandbox_ignored_when_resolver_returns_live (line 177)
      • test_resolved_live_with_context_sandbox_denied (line 249)
    • Required checks: pytest tests/test_account_mode_gate.py, mypy src/, ruff check .
    • Non-breaking rationale: comply_test_controller is a conformance/testing tool; the prior code returned a non-spec-conformant error code, so code correctly expecting FORBIDDEN was already broken.

    Triaged by Claude Code. Session: https://claude.ai/code/cse_01HBBfZWZrUXDvqf6vM77qHQ


    Generated by Claude Code

  3. added
    bugSomething isn't working
    and removed
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions