Skip to content

product_filter_behavior: refine phase isn't gated on buying_modes, so sellers that don't declare refine fail it #8252

Closed

Description

@EvgenyAndroid

Summary

media_buy_seller/product_filter_behavior checks that get_products filters apply in three buying modes: brief, wholesale and refine. Its refine phase (refine_filters, step filter_refined_product_out) sends buying_mode: "refine", but neither the storyboard nor the phase has a capability gate.

Refine is optional. media_buy.buying_modes defaults to ["brief"], and the refinement guide says refinement is not required. refine_products is already gated on media_buy.buying_modes containing refine (#6702).

The storyboard runs for any seller that exposes comply_test_controller. Take a seller that declares ["brief", "wholesale"] but not refine. For that seller, the refine step can't pass:

The only way for the seller to clear the step is to declare refine. That would be a false capability claim, and it would also select refine_products.

The storyboard file is byte-identical in three places:

  • on main
  • in the 3.2.4 bundle, the latest in published-versions.json and the bundle hosted 3.2 grading uses
  • in 3.2.5, released 2026-10-16

Among published bundles it ships only in 3.2.x, not in 3.0.x or 3.1.x.

Evidence

Main is at 68c90b6. adcp-client links are at the @adcp/sdk@15.9.2 tag, 90a40c3.

The step

  • The refine_filters phase has no requires_capability: L176–L206.
  • The storyboard header has requires: [controller] and required_tools: [get_products, comply_test_controller], and no capability gate: L1–L12.
  • The refine request reuses the product id that the wholesale phase captured.

Refine is optional

What the runner does (15.9.2)

  • getProducts throws UnsupportedBuyingModeError before the wire call when a seller's declared modes don't include the requested mode. The check applies when the seller serves 3.1 or later, or declares buying_modes.
  • runStep catches the throw and returns passed: false with the message and no skip reason: client.ts L558–L571.
  • Hosted runs refuse the call the same way.
    • The storyboard runner turns request validation off only for expect_error steps: runner.ts L6405.
    • validateFeatures appears nowhere in server/ on main.

How the hosted grader scores it

Runs

  1. Our agent, local, SDK 15.8.1. Our 3.2 build is behind a flag that is off in production. It declares buying_modes: ["brief", "wholesale"].
    • Full local run on 2026-10-14 against bundle 3.2.3.
    • Brief and wholesale passed.
    • filter_refined_product_out failed in 2 ms with "Seller does not support refine buying mode. Declared modes: brief, wholesale. …".
    • Between 3.2.3 and 3.2.4, the only change to this file is that it gains requires: [controller].
  2. Stub seller, SDK 15.9.2.
    • Run with runAgainstLocalAgent over real MCP HTTP, using the 3.2.4 compliance cache that ships in the package.
    • The stub logs every get_products call it receives.
    • The hosted status column applies the classifyComplianceStep and deriveStoryboardStatuses rules above.
Storyboard Seller declares Modes the stub received Refine step Hosted status
as shipped brief, wholesale brief, wholesale failed: "Seller does not support refine buying mode…" partial 2/3
as shipped brief, wholesale, refine brief, wholesale, refine passed passing 3/3
with the phase gate below brief, wholesale brief, wholesale skipped, not_applicable passing 2/2
with the phase gate below brief, wholesale, refine brief, wholesale, refine passed passing 3/3

For comparison, refine_products as shipped behaves the way the table's gated rows do:

  • A seller declaring [brief, wholesale] skips it as capability_unsupported, which the hosted policy treats as not applicable.
  • A seller declaring [brief, wholesale, refine] selects it and sends a refine call.

For rows 3 and 4, I added the gate to the loaded storyboard in memory rather than editing the YAML. 15.9.2 accepted the shape and applied the gate.

What isn't verified

  • No hosted card shows this failure yet.
    • I read the public registry's compliance cards twice on 2026-10-16, between 19:38 and 19:57Z. 35 agents are listed, and the API returned HTTP 500 for some cards.
    • Of the cards that came back, one seller card is graded on a 3.2 bundle: 3.2.4 with SDK 15.8.1, graded 2026-10-13.
    • On that card, product_filter_behavior is untested (0/0) and refine_products passes 3/3, so that seller declares refine.
    • The hosted result above comes from the grader source at 68c90b6 and the pinned SDK version, not from a card.
  • Our own agent was run on 15.8.1, not 15.9.2. The 15.9.2 result comes from the stub. The refusal code path is the same in both versions.
  • The stub returns a fixed product. It shows what the runner sends and how the step is scored, not how a real seller filters.

Our stake. We run https://sell.nofluffadvisory.com/mcp. It serves 3.1 and will stay on 3.1 until 3.3, so this doesn't affect our card today. Our 3.2 build doesn't iterate on prior products and doesn't declare refine, so this step would be one of its failures if we moved. The same is true for any seller that exposes the test controller, declares wholesale but not refine, and moves to 3.2.

Ask

  1. Gate the refine_filters phase on the same capability as refine_products:

      - id: refine_filters
        title: "Apply replacement filters during refinement"
        requires_capability:
          path: media_buy.buying_modes
          contains: refine
        steps:
  2. Backport the fix to 3.2.x for the next patch.

Same file, not our case: wholesale. wholesale_filters has no gate either, and the schema says wholesale is opt-in.

  • With 15.9.2 and a seller that declares only ["brief"], the wholesale step fails the same way. The refine step is then skipped as prerequisite_failed, and the storyboard reads partial 1/3.
  • If wholesale gets a gate too, the refine phase needs to capture its product id from the brief phase instead. Otherwise a seller that declares [brief, refine] loses it.
  • I haven't checked the other storyboards that send buying_mode: "wholesale".

Related:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions