Repository navigation
Description
A storyboard whose phases are all optional can never grade as passing, and billing_gate_dispatch became one in #7581. Example: a seller that advertises supported_billing: ["operator"] passes the two sync_accounts rejection steps and the capability read with every validation green, and still gets billing_gate_dispatch/storyboard: storyboard-level verdict failed.
Summary
#7581 (fixes #7573) marked capability_discovery optional: true so a signals agent without an account block skips it. That was the last required phase. universal/billing-gate-dispatch.yaml 1.1.1 now has six phases and all six are optional.
The storyboard runner computes overall_passed from the required phases only (src/lib/testing/storyboard/runner.ts, same on main at 15.6.0):
const requiredPhaseHasExecutedPass = phaseResults.some((p, idx) => {
const phaseDef = storyboard.phases[idx];
if (!phaseDef || phaseDef.optional || !p.passed) return false; // optional never counts
return p.steps.some(s => !s.skipped && s.passed);
});
const requiredPhaseDefs = storyboard.phases.filter(phaseDef => !phaseDef.optional); // []
const requiredPhasesNonFailingSkipped = requiredPhaseDefs.length > 0 && …; // false
const requiredPhasesCoveredByCapabilityGates = … && requiredPhaseDefs.length > 0 && …; // false
const requiredPhasesPassed =
!hasExecutableSteps ||
(requiredPhaseDefs.every(…) && (requiredPhaseHasExecutedPass || requiredPhasesNonFailingSkipped || …)); // falseThe comment on that block says the guard exists so "a storyboard where every phase is optional … would [not] pass vacuously". With every phase optional it is unpassable instead.
Why it matters
Hosted grader, bundle 3.2.3, @adcp/sdk 14.2.0, https://dev-demo-mcp.gotom.io/mcp: billing_gate_dispatch 3/4 with one failure and no failing step, which keeps the error_handling track at partial (17/18). Local run against the same bundle with 14.0.0:
| phase | step | result |
|---|---|---|
capability_discovery |
Read supported_billing | passed, 6/6 validations |
capability_gate_operator |
— | skipped not_applicable (seller advertises operator) |
capability_gate_agent |
Submit omitted agent billing | passed, 8/8 (BILLING_NOT_SUPPORTED, recovery: correctable, details.scope: capability, context echoed) |
capability_gate_advertiser |
Submit omitted advertiser billing | passed, 8/8 |
per_agent_gate_reject |
— | skipped not_applicable (test kit is not passthrough_only) |
per_agent_gate_recover |
— | skipped not_applicable |
storyboard |
— | failed: "storyboard-level verdict failed" |
Steps: 3 passed, 0 failed, 3 skipped. The hosted 3/4 is those three steps plus the synthetic storyboard scenario. Any seller that omits a BillingParty value from supported_billing lands here; a seller that advertises all three skips every gate phase and lands here too.
Ask
Make capability_discovery required again and keep its requires_capability: { path: account, present: true } gate. The runner already passes a required phase that a capability gate skipped (requiredPhasesCoveredByCapabilityGates), so the #7573 case still skips cleanly, and the executed sync_accounts phases count as the storyboard's pass. One-line change plus a patch changeset, 3.1.x backport alongside #7406.
Alternatively, in adcp-client: when requiredPhaseDefs.length === 0, accept an executed pass in any phase instead of failing. Happy to open either PR.
Activity
bokelley commentedon Oct 9, 2026
Thanks for filing!
Triage
Classification: Conformance failure
Bucket(s): compliance-suite
Status: ready-to-implement
Milestone: P0 Bugs
What the experts said:
optional: truefromcapability_discoveryrestores scoring without changing wire behavior or step semantics; the existingrequires_capabilitygate handles agents withoutaccountblock viarequiredPhasesCoveredByCapabilityGates; Option B (runner fix) is unsafe and would silently pass future misconfigured all-optional storyboards; patch-eligible for 3.1.x.requiredPhasesCoveredByCapabilityGatesinadcp-client(unverifiable from this repo but supported by the schema contract); Ready-to-implement not Execute — this reverses a deliberate change from fix(compliance): gate billing capability discovery #7581 which requires explicit judgment.My take: Bug confirmed: all 6 phases in
billing-gate-dispatch.yaml1.1.1 haveoptional: trueafter #7581, making the storyboard always-fail. Fix is narrow and patch-eligible for both main and 3.1.x; confirm therequiredPhasesCoveredByCapabilityGatespath inadcp-clientbefore merging.Implementation:
static/compliance/source/universal/billing-gate-dispatch.yamlline 142optional: truefromcapability_discovery; leaverequires_capability: { path: account, present: true }intactrequiredPhasesCoveredByCapabilityGatesrunner path covers agents without anaccountblock — billing_gate_dispatch: capability_discovery has no required_tools guard, failing signals agents on an account capability they cannot declare #7573 does not re-emergeadcontextprotocol: patchchangeset; land onmain, cherry-pick to3.1.xrequiredPhasesCoveredByCapabilityGatesinadcp-clienthandles a required phase skipped by capability gateTriaged by Claude Code. Session: https://claude.ai/code/session_01EQpUrgG1RRn8U5SP2L5EQX
Generated by Claude Code
bokelley commentedon Oct 16, 2026
Verified the released v3.2.4 artifact: universal/billing-gate-dispatch.yaml is storyboard version 1.1.2 and capability_discovery is required (the optional flag is absent), with the account capability gate preserved. The SDK runner retains requiredPhasesCoveredByCapabilityGates for agents without an account block. This ships the fix from #8129, so closing the all-optional storyboard defect. Evidence: https://github.com/adcontextprotocol/adcp/blob/v3.2.4/dist/compliance/3.2.4/universal/billing-gate-dispatch.yaml and #8129.