Skip to content

3.2.x storyboards call optional tools (list_products, control_media_buy, sync_event_sources) with no gate, so hosted grading fails sellers that omit them #8253

Closed

Description

@EvgenyAndroid

Summary

Several 3.2.x storyboards call a tool the spec lets a seller leave out, with no requires_tool on the step and no storyboard gate that implies the tool. The SDK runs a storyboard when the agent advertises any one of its required_tools, then skips each step whose tool isn't advertised as missing_tool. The hosted grader counts that skip as a failed step unless the pinned step's requires_tool names the missing tool, and it counts the stateful steps that cascade from it as dependency_failed. A seller that leaves out an optional surface therefore gets a failing or partial storyboard for it.

When #6753 made wholesale discovery optional, its grading audit found nothing that graded a seller on a wholesale path it hadn't opted into, relying on capability gates and requires_tool. Since then, #6619 (sales-dooh) and #7444 (product_coverage_filters) added list_products steps with neither, and the hosted classification from #7454 counts an ungated missing_tool skip as a failure. change_rights_state_projection (#6794) and revenue_share_pricing (#6095) have the same shape for control_media_buy and sync_event_sources.

Three of these fail on our own seller in a local run that we re-graded with the registry's policy: product_coverage_filters, revenue_share_pricing and change_rights_state_projection. Restricted to ten tools the spec makes optional, a scan of the 3.2.4 bundle finds 20 such steps in 9 storyboards. The nine files are identical in 3.2.5 (released 2026-10-16), in static/compliance/source on main, and in the 3.3.0-beta.0 snapshot that the release-bot draft #7897 prepares. #7586 reported the same pattern for the provenance storyboards, and #7599 / #7646 fixed it there.

Our stake: our sales agent (https://sell.nofluffadvisory.com/mcp) is graded on 3.1.24 and stays on 3.1 until 3.3. Its 3.2 build is behind a flag that is off, so nothing here changes our current card. It matters for any seller that moves to 3.2 without the wholesale feed, control_media_buy or conversion tracking. Eight of the nine storyboards declare requires: [controller], so in practice that means sellers that expose comply_test_controller. No hosted card shows it yet. Of the 35 agents in the public registry, 32 answered on 2026-10-16 at 19:54Z, two of them opted out of grading. The only one graded on 3.2.x is our signals agent (adcp.signal-stack.io, 3.2.4), which doesn't run these storyboards.

Evidence

Links are pinned to adcp main 68c90b6 and to @adcp/sdk 15.9.2 (90a40c3), which the registry has pinned since #8231.

Runner.

  • required_tools is the "Lenient any-of" gate (storyboard-schema.yaml L54-L56). hasAnyRequiredTool ("Storyboard applicability is any-of; it does not authorize individual steps") runs the storyboard if the agent advertises any listed tool (runner.ts L3353-L3371).
  • A step whose task the agent doesn't advertise returns skipped: true, skip_reason: "missing_tool" (L6127-L6139). A step whose requires_tool isn't advertised returns the same reason (L6041-L6046).
  • A missing-tool skip on a stateful step trips the phase cascade, unless it is the phase's only stateful step or a same-phase peer declares provides_state_for for it (L4190-L4241).

Grader.

Spec.

Observed on our seller

The run.

  • Local adcp storyboard run on 2026-10-14 at 12:50Z, @adcp/sdk 15.8.1, bundle 3.2.3, against a local instance of our sales agent with 3.2 serving on.
  • The agent advertises 15 tools. None of them is list_products, control_media_buy or sync_event_sources.
  • The CLI reports missing_tool=6. Those are the six steps in the table.

How we graded it.

  • We re-graded the run with a reimplementation of classifyRunStep / deriveStoryboardStatuses from the registry source above, using the 3.2.4 bundle as the pinned metadata.
  • We then graded it again with requires_tool added to those six steps in a scratch copy of the bundle.

Why a 3.2.3 run stands in for 3.2.4 / 15.9.2.

  • product_coverage_filters and revenue_share_pricing differ between 3.2.3 and 3.2.4 only by requires: [controller], and we advertise the controller. change_rights_state_projection is identical.
  • runner.js, agent-routing.js, compliance.js and comply.js are byte-identical in 15.8.1 and 15.9.2.
Storyboard Steps without requires_tool Hosted policy, as published Hosted policy, requires_tool added to those steps
product_coverage_filters compact_coverage/compact_country_and_metro (list_products) failing 0/2 failing 0/1: the list_products step is N/A
revenue_share_pricing setup/sync_event_sources partial 1/7 (1 failed, 5 dependency_failed) partial 1/6: the step is N/A, the 5 later steps stay dependency_failed
change_rights_state_projection 4 control_media_buy steps: L196, L226, L312, L348 failing 0/6 failing 0/2: the 4 steps are N/A, both get_media_buys reads still fail

product_coverage_filters. The compact step is non-stateful and comes last, so requires_tool: list_products on the step is enough. The failure that remains, legacy_country_and_metro, is our bug: we don't apply coverage filters on get_products.

revenue_share_pricing. sync_event_sources is stateful and shares its phase with sync_accounts. The runner therefore cascades prerequisite_failed into all five later steps, and the grader keeps those as dependency failures. A step-level requires_tool doesn't fix this storyboard. The documented revenue-share settlement flow also starts with the buyer registering the event source through sync_event_sources (pricing-models.mdx L414-L416).

change_rights_state_projection. This storyboard has no gate besides requires: [controller], and its required_tools include get_media_buys, so every seller with that tool and the test controller runs it.

  • The first read already asserts an available_actions entry with task: control_media_buy (L156-L163), so the whole storyboard assumes the tool.
  • In our run, the paused-state read ran after the pause step was skipped. It failed with Expected "paused", got "active".
  • If the active read passes, the runner source says the skipped pause instead cascades prerequisite_failed into the paused read. We read that in the source; we didn't run it.
  • Our agent also fails the active read because we don't implement accepted-proposal change rights. That part is our gap.

Scan of the 3.2.4 bundle

Method.

  • Load all 213 storyboards in dist/compliance/3.2.4 with the SDK's listAllComplianceStoryboards.
  • Keep steps whose task is a tool listed in dist/schemas/3.2.4/index.json, that have no requires_tool, and whose storyboard can run without that tool (its required_tools is empty or names another tool).
  • Drop comply_test_controller steps in requires: [controller] storyboards.

Result. That leaves 1,077 steps. We triaged only the steps whose task the spec text above makes optional: list_products, request_proposals, refine_proposals, decline_proposals, buy_products, accept_proposal, control_media_buy, sync_event_sources, log_event and sync_audiences. That is 84 steps in 26 storyboards.

64 of the 84 sit behind a gate that implies the tool:

  • a media_buy.conversion_tracking or media_buy.audience_targeting gate;
  • an opt-in compact-lifecycle probe scenario in compliance_testing.scenarios;
  • proposal_refinement, which is valid only with refine_proposals (L645);
  • a specialism defined around the tool.

The other 20 fall into two groups.

No gate related to the tool:

Storyboard Step Task Stateful Cascade
media_buy_seller/product_coverage_filters compact_coverage/compact_country_and_metro list_products no none
media_buy_seller/revenue_share_pricing setup/sync_event_sources sync_event_sources yes all 5 later steps
media_buy_seller/change_rights_state_projection the 4 steps above control_media_buy yes get_paused_buy and the later control steps
sales_dooh (stable specialism; runs only for sellers that claim sales-dooh) product_discovery/get_dooh_products list_products yes none by cascade (only stateful step in its phase), but later steps read $context.dooh_product_id from it (L379, L444, L549)

Gated on a capability that doesn't imply the tool. These only catch a seller that declares the capability without the tool, and some may be intended:

Storyboard Gate Steps
frequency_cap_capability_updates media_buy.frequency_capping present ("retains its legacy meaning", L682) control_media_buy L211, L236, L262, L287
media_buy_frequency_cap_updates media_buy.aggregate_frequency_capping present control_media_buy L271 (cascades into clear_root_cap_and_add_package), L413
availability_windows media_buy.availability_horizon equals true; required_tools: [list_products, buy_products] list_products L143, L202, L309; buy_products L360, L404. Only affects a seller with one of the two tools. #6753's audit accepted this gate as the opt-in.
typed_proposal_negotiation lifecycle_tools contains request_proposals, plus proposal_refinement accept_proposal L611
refine_frequency_cap_negotiation proposal_refinement (implies refine_proposals only), plus aggregate capping request_proposals L207

Not triaged: the deprecated facades. In 3.2, get_products, create_media_buy and update_media_buy are compatibility facades that a server advertises when it supports callers that use them (schemas.mdx L289-L292), and get_products is itself in the L623 subset. 351 of the 1,077 candidate steps call one of the three, so a compact-only seller may hit the same grading there. #8201 raises this for sales-dooh. We haven't triaged it.

None of the 117 open PRs here changes these nine files, step-disposition.ts or classifyRunStep. #7897 only copies the nine files unchanged into the 3.3.0-beta.0 snapshot.

Ask

  1. Gate these steps so that a seller without an optional tool grades not applicable, on main and on 3.2.x (as fix(compliance): backport provenance requires_tool gates to 3.1.x #7646 did for 3.1.x).
    • For non-stateful steps that nothing depends on, add requires_tool: <tool>. product_coverage_filters/compact_country_and_metro is the clear case, and it is what L623 already promises for the wholesale path.
    • Where later steps need the optional step's state, requires_tool on that one step isn't enough under the current grader, because the dependents stay dependency failures. Gate the storyboard instead.
      • change_rights_state_projection: requires_capability: {path: media_buy.lifecycle_tools, contains: control_media_buy}. typed_proposal_negotiation already gates on media_buy.lifecycle_tools containing request_proposals, with the same rationale in its comment (L15-L21).
      • revenue_share_pricing: requires_capability: {path: media_buy.conversion_tracking, present: true}. That is the existing declaration that makes sync_event_sources mandatory.
      • sales_dooh: requires_tool on get_dooh_products alone would leave the later steps without a product ID. Either give the specialism a discovery path that works without list_products, or say that sales-dooh requires list_products, which list_products.mdx L10 currently rules out. compliance: opt the 3.2 compact proposal storyboards and sales-dooh into fixture_resolution (discover), and allow list_products discovery at 3.2 #8201 asks the related surface question for this specialism.
    • The second table is your call, storyboard by storyboard.
  2. If you'd rather fix this in the grader, grade missing_tool as not applicable when the seller's own declarations make the tool optional, and carry that to the steps that cascade from it. Examples: a compact tool missing from media_buy.lifecycle_tools, or sync_event_sources / log_event without media_buy.conversion_tracking. That narrows a rule fix(compliance): classify hosted skips and expose redacted run provenance #7454 made deliberately and tests, so the storyboard change looks smaller to us. It could also cover the facade case above without editing each storyboard.
  3. A build check that flags a step calling one of these tools without requires_tool or a related gate would catch the next one. docs: wholesale discovery is optional and never a capability signal #6753's audit was a one-off, and feat(compliance): add DOOH sales profile #6619 and fix(media-buy): preserve legacy discovery coverage and request semantics #7444 landed after it.

Out of scope: whether a seller that has the tool but doesn't sell the feature (revenue share, change rights) should run these storyboards. That's a separate question. For revenue share, media_buy.supported_pricing_models already exists in 3.2.4 and could be combined with the gate above under requires_all_capabilities.

What isn't verified:

  • No hosted run. Our 3.2 build isn't serving publicly, and no sales agent that answered the registry has a 3.2 card.
  • The grading above is our reimplementation of the registry code. It matches steps by title and task, because CLI JSON has no step_id, and it skips the observations pass.
  • The run used 15.8.1 and bundle 3.2.3. That it carries over to 15.9.2 and 3.2.4 rests on the byte-identical runner files and the bundle diff above.
  • We haven't run the six storyboards outside our run against any agent. Their behavior comes from the storyboard text and the runner source, and so do the cascade counts.
  • We didn't emulate the storyboard-level gates in the Ask. We expect them to grade not_applicable, since the runner reports an unmet storyboard gate as capability_unsupported and the grader maps that to not_applicable (step-disposition.ts L16-L22).
  • We didn't check whether the tools in the other 993 candidate steps are optional, beyond the facade note above.

Related: #7586, #7599, #7646 (same fix for the provenance steps); #6729, #6753 (wholesale is optional; the grading audit); #6619, #7444, #6794, #6095 (where these storyboards came from); #7454 (hosted skip classification); #8201 (sales-dooh discovery surface); #7723, #7728, #6907, #7824 (other applicability leaks); #8231; #7897; adcp-client#2890, adcp-client#2401, adcp-client#2652.

Activity

  1. EvgenyAndroid commented on Oct 16, 2026

    @EvgenyAndroid
    ContributorAuthor

    Related 3.2 applicability findings from the same sweep, filed separately because each needs a different fix: #8251 (prerelease 3.2-beta.6 pins), #8252 (refine phase not gated on buying_modes), #8254 (optional features with no capability gate). Also #8199, the provenance truth-of-claim verifier, with fix PR #8218 open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions