Repository navigation
Description
I co-lead the Signals and Measurement working group. Filed as an operator of two registered agents reading the grader's output; no WG position attaches.
Two registered agents, both graded on line 3.1.20, both showing "degraded" for reasons that do not reproduce with the published runner against the same production endpoints. Registry feeds (public, no auth):
https://agenticadvertising.org/api/registry/agents/https%3A%2F%2Fsell.nofluffadvisory.com%2Fmcp/compliance— media-buy seller, heartbeat 2026-09-10T00:38:51Z, 159/193 scenarios, 13/35 storyboardshttps://agenticadvertising.org/api/registry/agents/https%3A%2F%2Fadcp.signal-stack.io%2Fmcp/compliance— signals agent, heartbeat 2026-09-10T00:35:07Z, 66/68 scenarios, 9/35 storyboards, verified
Summing failure_count over storyboard_statuses: 25 failing steps on the sales card, 37 on the signals card, 62 total. 53 of them fall into the two classes below. None of the 53 can be made to pass by any change to either agent.
1. required_tools is an OR gate in the per-storyboard runner and an AND gate in the suite selector
universal/canonical-format-validate-input.yaml (3.1.20 and 3.2.0-rc.1 alike):
required_tools:
- validate_input
- comply_test_controller
requires:
- controllerNeither agent exposes validate_input (anonymous tools/list returns 11 tools on each; the runner's own discovery observation on both cards lists none). Both expose comply_test_controller. Result on the cards: signals 0/17 failing, sales 1/18 with 17 failing steps. Every one of those steps calls a tool that does not exist.
The published runner disagrees with itself about what required_tools means. @adcp/sdk@13.0.2, dist/lib/testing/compliance/comply.js:896 (suite selection, what storyboard run / comply use):
const missing = (sb.required_tools ?? []).filter((t) => !discoveredToolNames.has(t));
if (missing.length > 0) {
missingToolStoryboards.push({ ..., reason: `missing required_tools: ${missing.join(", ")}` });Any missing tool excludes the storyboard. Locally that is what happens: the storyboard is absent from the 61-storyboard selection for the sales agent and listed under storyboards.missing_tools for the signals agent.
dist/lib/testing/storyboard/runner.js:1421 (per-storyboard gate):
const hasAnyRequired = storyboard.required_tools.some((t) => options.agentTools.includes(t));Any present tool admits it. The hosted grader evidently runs the second path: it selected 75 storyboards for the sales agent (its own budget message says so, see 3) against the selector's 61, and canonical_format_validate_input is among the extra 14.
This is the #6773 shape. PR #6774 (a1e72ec, 3.2.0-beta.6) fixed it for get_adcp_capabilities by removing that tool from 17 storyboards' required_tools. It did not touch this storyboard, whose gate is still satisfiable by comply_test_controller alone, and it is not on 3.1.x: billing-gate-dispatch.yaml there still lists get_adcp_capabilities, which is why that storyboard reads 1/2 on the signals card (its sync_accounts step fails on an agent with no accounts protocol).
2. Steps the SDK marks not_applicable are graded as failures on the hosted runner
Signals card: deterministic_testing 0/7 failing, idempotency 0/11 failing. Neither is a gate case. deterministic-testing.yaml requires only comply_test_controller, which the agent has; idempotency.yaml has no required_tools at all. Both are legitimately selected. Their media-buy steps (accounts, media buys, creatives, delivery, budget) cannot apply to a signals-only agent.
Locally, @adcp/sdk@13.0.2 storyboard run against the same production endpoint on 2026-09-10 marks those steps not_applicable and the storyboards pass: 57/57 scenarios, 32/108 steps passed, 0 failed, 76 skipped, both storyboards in the executed set. The hosted runner fails every step in both.
I can see the difference in outcome but not the cause. Candidates: a different runner build on the hosted side, or a configuration that disables step-level applicability. Which one it is decides whether this is a bug or a policy.
3. The 600 s budget truncated a run and the card published the truncated result
Sales card, heartbeat 2026-09-09T15:46:21Z: total_duration_ms 638171 against the neighbouring 3.1.20 heartbeats at 150.5, 221.1, 182.0 and 219.7 s. The run's observations carried:
Compliance timeout budget of 600000ms was reached. Stopped starting new storyboards after 50/75 selected storyboard(s).
For that cycle the card read 2/13 storyboards and 130 scenarios: every storyboard the run never reached dropped out of the denominator, and 11 that had been passing at 01:41Z vanished with them. The next heartbeat (00:38Z) restored 13/35. Nothing changed on the agent between those runs. Normal runs sit at 25–37% of the budget, so this is one slow cycle, but the card presented it as a grade.
Asks
- Make the two gates agree. Either the per-storyboard runner requires every listed tool, as the selector does, or
required_toolsis documented as OR and storyboards that need a specific tool declare it another way. Whichever way,canonical_format_validate_inputshould not be selectable for an agent withoutvalidate_input. - Backport, or say that 3.1.x will not get, fix(compliance): drop get_adcp_capabilities from required_tools across 17 storyboards (OR-gate bug) #6774 and whatever fixes 1. Both cards grade on 3.1.20 today.
- State what the hosted runner does with step-level
not_applicable, and if it differs from the published SDK, which build or setting produces the difference. - When a run hits the budget, keep the previous complete result on the card (or mark the run partial) instead of publishing the truncated denominator.
Reproduce
# published runner, same line as the cards
npx @adcp/sdk@13.0.2 storyboard run https://sell.nofluffadvisory.com/mcp --compliance-version 3.1.20 --timeout 300
npx @adcp/sdk@13.0.2 storyboard run https://adcp.signal-stack.io/mcp --compliance-version 3.1.20 --timeout 300
Sales: 228 passed / 1 failed / 76 skipped, the 1 being a deliberate provenance decline; canonical_format_validate_input not selected. Signals: 57/57 scenarios, 0 failed; deterministic_testing and idempotency pass with their media-buy steps not_applicable. Not @adcp/sdk@13.0.3: it fails every comply_test_controller response-schema check under storyboard run (adcp-client#2867), which would mask the second class.
The 9 failing steps on the two cards that are not in these classes (8 on sales across deterministic_testing, idempotency, get_media_buys_pagination_integrity, stale_response_advisory; 1 on signals in error_compliance_signals) also do not reproduce locally, but I have no step text for them: the feed returns first_failed_step_id and first_failure_message as null on all 35 storyboards of both cards. Separate question; happy to take it elsewhere if the feed is not meant to expose them.
Activity
bokelley commentedon Sep 10, 2026
Triage
Classification: Conformance failure
Bucket(s): compliance-suite
Status: ready-for-human
What the experts said:
not_applicableis defined harness behavior and the published SDK 13.0.2 is the reference. P0: 53/62 failing steps on public registry cards reflect harness failures that appear as agent defects, directly undermining compliance program trust.static/compliance/source/universal/canonical-format-validate-input.yaml), patch changeset, backport to 3.1.x. Also confirmsbilling_gate_dispatchsource is already at v1.1.0 (correct:required_tools: [sync_accounts]only) but no 3.1.x dist carries it — both fixes ship in the same next 3.1.x patch cut. Bugs 2–3 are service-level defects outside storyboard YAML scope.My take: Two surfaces — storyboard content (Bug 1, ready to implement from source) and hosted grader service (Bugs 2–3, need someone with runner access to investigate). Bug 4 (
first_failed_step_id: null) is best filed separately once the grader investigation starts; its root cause (grader never writing the field vs. feed publisher not reading it) is independent.Bug 1 — ready to implement
canonical_format_validate_inputis selected by the hosted grader's OR gate for any agent withcomply_test_controller(universal), then fails all 17 steps because the agent has novalidate_input. Fix is the same pattern as #6774:static/compliance/source/universal/canonical-format-validate-input.yamlpatchonadcontextprotocolmain, then cherry-pick tobackport/3.1.x-required-tools-gate-fixesbilling_gate_dispatch— source v1.1.0 is already correct but nodist/compliance/3.1.x/artifact carries it yet; no additional source edit needed, just include in the backportvalidate_inputrun identically; agents without it receive a cleancoverage_gapskip instead of 17 spurious failuresBug 2 — hosted runner grades
not_applicablesteps as failures (flag for human)deterministic_testingandidempotencycorrectly select on the signals agent; the SDK marks their media-buy stepsnot_applicable(0 failed, 76 skipped). The hosted grader counts every one as a failure.not_applicableis defined harness behavior — no protocol decision required, but someone with grader access needs to confirm whether the service runs a stale build or a config flag disables step-level applicability (COMPLIANCE_HARNESS_SKIP_NOT_APPLICABLEor equivalent).Bug 3 — budget-truncated run overwrites complete grade (flag for human)
A 638 s run hit the 600 s budget and published 2/13 storyboards, dropping 11 previously-passing storyboards from the denominator. The grader should gate on a
run_completesignal (orbudget_exhaustedfield in runner output) before committing results. If runner output lacks this field, adding it is patch-eligible. The card should retain the prior complete result until a new complete run is available.Triaged by Claude Code. Session: https://claude.ai/code/session_01PsYtFTvHPQZesBaaswFhzW
Generated by Claude Code
EvgenyAndroid commentedon Sep 10, 2026
Bug 1 PRs: #7405 (main) and #7406 (3.1.x backport, bundling the four #6774 storyboards that exist on that line — the other 13 are 3.2-only).
EvgenyAndroid commentedon Sep 10, 2026
Bug 3 recurred, and there is new data on both bug 3 and the "slow agent" reading.
Second truncation, 15 hours after the first. Sales card, heartbeat 2026-09-10T15:56:03Z:
total_duration_ms702418, observation verbatim:The card again published the truncated result — 13
storyboard_statusesentries, 2 passing, andmedia_buy52/65 where the untruncated 00:38:51Z run read 62/78. Recent durations on the same line and the same agent: 219.7 s (00:38Z today), 638.2 s (15:46Z yesterday, truncated), 702.4 s (15:56Z today, truncated). Two of the last three scheduled runs published a partial grade.The two steps that run flagged are not slow at the agent. Same observation block:
Measured against the same production endpoint just now, from an ordinary client, sandbox principal, one call each:
list_creative_formatsget_products(static)update_media_buy(pause)update_media_buy(cancel)create_media_buysync_creatives(live image, bound to package)get_products(brief → curation, the heaviest path)Nothing on the agent is within an order of magnitude of 10 s. One thing I can rule out as a cause: the storyboard fixture host
test-assets.adcontextprotocol.orgis NXDOMAIN from every resolver I tried (Google DNS returns Status 3;adcontextprotocol.orgitself resolves fine). Our creative gate HEAD-probes image URLs with a 4 s budget, so a hanging host would have shown up — but DNS failure returns immediately, andsync_creativeswith three such creatives measured 248 ms total. If the hosted runner resolves that host and gets a slow answer rather than NXDOMAIN, its per-step latency would be its own network, not ours.That leaves the budget interacting with something on the runner side. Whatever the cause, the card-facing half of bug 3 stands on its own: a run that stops at 48 of 75 storyboards should not overwrite a complete grade.
EvgenyAndroid commentedon Sep 11, 2026
Three additions to the hosted-grader-service half (bugs 2 and 3), which hasn't moved since triage. One is new, one puts numbers on an existing bug, one is a question that decides whether bug 4 is worth filing.
1. New: the hosted results move without an agent change
sell.nofluffadvisory.comdeployed at 2026-09-09T22:24Z and not again until 2026-09-11T10:54Z. Two heartbeats fall inside that window, so both graded the identical Worker build, on the same line, with the same storyboard selection:One passing scenario left
corebetween them with nothing changed underneath it. Running that storyboard against the same production endpoint on the same line and SDK (@adcp/sdk@13.0.2,--compliance-version 3.1.20) reads21 passed, 0 failedboth before and after, so I can see that a step flipped but not which one.This is a different claim from "these failures don't reproduce locally". Two runs of the same grader against the same build disagreeing with each other is a property of the grader, not of the agent, and it is measurable from the public feed without any runner access.
2. Bug 3: the truncation publishes a shrunken denominator, not a partial result
Both slow runs carried
Compliance timeout budget of 600000ms was reached. Stopped starting new storyboards after 48/75 selected storyboard(s)(50/75 on the earlier one). Every storyboard the run never reached dropped out of the denominator rather than being reported as unrun, which is why the card read2/13for a cycle andmedia_buyfell from 62/78 to 52/65. It has self-recovered twice, so this is intermittent rather than a trend.3. Question that decides bug 4
Nine failing steps across the two cards are outside the
required_toolsclass and don't reproduce locally: eight on the sales card (deterministic_testing,idempotency,get_media_buys_pagination_integrity,stale_response_advisory) and one on the signals card (error_compliance_signals6/7). The public feed returnsfirst_failed_step_idandfirst_failure_messageasnullon all 35 storyboards of both agents.Does the grader write those fields at all? If it does and the feed isn't publishing them, that's bug 4 as triaged. If it never writes them, the fields are decorative and I'll stop asking for step detail through that route. Either answer is useful; I can't tell which from outside.
Unrelated, for the record
Chasing item 1 surfaced a real conformance defect of my own, so this isn't all pointing outward.
core/brand-ref.jsonisadditionalProperties: false, and mylist_accountswas echoing buyer-supplied keys fromaccount.brandstraight into the response — a buyer sendingbrand.namemade the entire response schema-invalid, not just that item. Fixed and deployed 2026-09-11T10:54Z. It surfaced as an advisory-levelSchema validation failed for list_accounts (non-blocking)line in a local storyboard run, never on the card.fix(compliance): drop comply_test_controller from canonical_format_va…
fix(compliance): drop comply_test_controller from canonical_format_va…
fix(compliance): correct 3.1 backport changeset scope
fix(compliance): preserve discovered-agent schedules during migration
fix(compliance): classify hosted skips and expose redacted run proven…
3 remaining items
fix(compliance): tolerate omitted observations during normalization
fix(compliance): classify hosted skips and expose redacted run proven…
EvgenyAndroid commentedon Sep 13, 2026
Update from the operator side, with one structural question at the end.
I spent today fixing everything on my two agents that was genuinely mine, so the cards would stop conflating my defects with this issue's. On the signals agent that work is finished, and what remains is a clean measurement.
A card where these two bugs are the only thing left
https://adcp.signal-stack.io/mcp, heartbeat2026-09-13T14:26:11.417Z, 67/68 scenarios, 10/35 storyboards. Every failing step, classified against each storyboard's ownrequired_toolsin the 3.1.20 cache:canonical_format_validate_inputvalidate_inputbilling_gate_dispatchsync_accountsdeterministic_testingnot_applicableidempotency36 failing steps. 18 from bug 1, 18 from bug 2, nothing else. The residue is zero, not small.
The one real defect that was on this card is gone:
activate_signalreturnedINTERNAL_ERRORwhereerror_compliance_signalsassertsINVALID_REQUESTorVALIDATION_ERROR. Fixed this morning, and the grader confirmed it — that storyboard went 6/7 to 7/7 and the card 9 to 10. So the agent side of this card is now empty, and the 36 are not mine to fix.For bug 2 it is worth quoting what the runner says while failing the step, because it states the correct grade itself:
It then counts all 18 of those steps as failures across the two storyboards. A signals agent has no products or media buys to seed;
seed_productis not a gap it can close.The media-buy card corroborates it
Same exercise on
https://sell.nofluffadvisory.com/mcp, where I shipped five conformance fixes today (total_budgetonget_media_buysitems, commitment timestamps on forced and seeded buys, thecompliance_testing.scenariosclosed enum, and the per-package pricing trio onget_media_buy_delivery). Run2026-09-13T14:46:01.285Zagainst2026-09-13T04:26:43.320Zthis morning:Twelve scenarios and two storyboards, entirely from fixing my own schema defects. What is left on that card is 20 failing steps: 17 are
canonical_format_validate_input— bug 1 again, novalidate_input— and the other 3 are two separately-known upstream items, a runner test-kit contract that is not configured and an advisory-severity step. None of the 20 is an agent defect.I mention the movement only to make the point that the numbers on both cards now move when a real defect is fixed, and the remainder does not.
billing_gate_dispatch confirms the class rather than extending it
billing_gate_dispatchdeclaresrequired_tools: [get_adcp_capabilities, sync_accounts]. The agent has the first and not the second, so the OR gate admits it and the capability-discovery step then fails on anaccountblock a signals agent has no reason to carry. Its own prerequisites say the quiet part: "The agent under test exposessync_accountsand advertises anaccountcapability block."I am not filing this separately, because #7406 already patches that exact file. I raise it only as a second independent instance of the same gate behaviour, on a storyboard the original report did not cover.
The structural question
The remediation in this repository is data, not logic. #7406's changeset says so plainly:
That works, and it is the right immediate move — stripping
get_adcp_capabilitiesfromrequired_toolsdenies the OR gate the one tool every agent has. But the gate still reads.some(...)while the suite selector requires all, so the property being relied on is "no storyboard lists two capability-specific tools where an agent might hold exactly one". That has to be maintained by hand on every future storyboard, forever, and nothing tests it.Which raises the question I cannot answer from outside: where does the gate itself get reconciled?
comply.jsandrunner.jsship in@adcp/sdk, whoserepository.urlisgit+https://github.com/adcontextprotocol/adcp-client.git, so a fix to the divergence cannot land in this repository at all. I looked for runner source here and found none.If there is an
adcp-clientissue tracking the.some(...)versus all-of divergence, I would like to follow it and will stop restating the point here. If there is not, I am happy to file one there with the same reproduction, on a word from whoever owns that split.Separately, and with no expectation attached: #7406 is approved and mergeable, #7405 is currently blocked. Both would retire the 18 mis-selection steps above.
bokelley commentedon Sep 13, 2026
Structural question — cross-repo escalation
The
.some()vs all-of gate divergence (runner.js:1421vscomply.js:896) lives inadcontextprotocol/adcp-client, not in this repository. There is no adcp-client issue on record here for that split. Filing one there is the right call.Sibling-repo-fix-needed:adcp-client —dist/lib/testing/storyboard/runner.js:1421uses.some()(OR) wheredist/lib/testing/compliance/comply.js:896uses.filter()(AND). The per-storyboard gate therefore admits storyboards the suite selector would skip. The storyboard-YAML fix in #7406 is the correct near-term patch; reconciling the two paths in the SDK eliminates the invariant you described ("no storyboard lists two capability-specific tools where an agent might hold exactly one") so it no longer has to be maintained by hand on every future storyboard.For the adcp-client issue: the reproduction steps from the original report (
runner.js:1421exact line, the two affected call paths, the@adcp/sdk@13.0.2test command) are sufficient. The fix candidate is changing.some(...)to.every(...)or aligning it with the.filter()shape — but that's for the SDK maintainer to decide on the correct semantics.Triaged by Claude Code. Session: https://claude.ai/code/session_011Jt9GMuxWo5VknTzFUQNoD
Generated by Claude Code
EvgenyAndroid commentedon Sep 13, 2026
Filed as adcontextprotocol/adcp-client#2882.
It leads with the recurrence rather than the divergence, because that is the part that is new. adcp-client#2652 reported this exact
billing_gate_dispatchmis-selection and was closed 2026-08-25 on the strength of #6774, with the explicit note that the loader/helper work was "no longer required to prevent this failure." That was correct for the line #6774 landed on;billing_gate_dispatchthen mis-selected again on 3.1.20 last week, because the maintenance backport is #7406 and it is still open. So the issue asks for the divergence to be tracked rather than patched a particular way — and says plainly that if the any-of gate turns out to be deliberate and the suite selector is the side that should move, that answers it equally well.I offered to test a candidate against both registered agents.
Nothing outstanding for you here. Both
.some(...)instances on my cards are covered by #7405 and #7406, and I would still like to see #7406 land — it retires 18 of the 36 failing steps on the signals card on its own.fix(compliance): drop comply_test_controller from canonical_format_va…
fix(compliance): drop comply_test_controller from canonical_format_va…
fix(compliance): backport required_tools OR-gate fixes to 3.1.x (#6774 …
EvgenyAndroid commentedon Sep 15, 2026
Two observations on bug 2, both from the public feed only. I tried to build a stronger local-vs-hosted reproduction and could not, so I am posting what survived rather than what I hoped to show.
failure_countin the public feed is a derived residual. Across all 35 storyboards onsell.nofluffadvisory.com/mcpand all 38 onadcp.signal-stack.io/mcp,failure_countis exactlysteps_total - steps_passed— every storyboard, no exception — andskipped_countis 0 on all 73.first_failed_step_id,first_failure_messageandfirst_failure_validationsare null or empty on all 73 as well.So in the public projection there is no representation of a skipped step at all: anything that is not passed is reported as a failure, and an owner cannot tell which class it was. That is worth separating from the grading question in this issue. Even where the grader internally classifies a step as
not_applicable, a reader of the card sees a failure — so part of what bug 2 looks like from outside may be a reporting artifact rather than a grading one. Those need different fixes, and the reporting half looks cheap.Concretely on my sales card (run 2026-09-13T14:46:01.285Z):
sum(failure_count)is 20, of whichcanonical_format_validate_inputcontributes 17. 17 of the 35 storyboards areuntestedat 0/0 steps and contribute nothing either way.The 3.1.20 cache line is still what runs post-fix. The
required_toolsOR-gate fix merged in #7405/#7406 on 2026-09-14 (12:15:41Z and 12:16:01Z) and shipped in v3.1.23. My signals card re-ran at 2026-09-14T20:03:40.370Z — 7h48m after that merge — and still reportsadcp_version: 3.1.20. I am not claiming the grader should have moved by then; only recording that a post-merge run is still on the 3.1.20 line, so cards graded on it will keep carrying the pre-fixcanonical_format_validate_inputnumbers until the line advances. (My sales card is useless as evidence here: it last ran 21.5 hours before the merge.)What I could not establish, in case someone else tries the same route. I tried to show that steps the SDK skips locally are the same steps the hosted grader counts as failures, using
idempotency(hosted 9/11,failure_count2) against a local run on@adcp/sdk13.0.4 with--compliance-version 3.1.20(STORYBOARD-OK, 9 passed / 0 failed / 2 skipped). The arithmetic lines up, but the inference does not hold up:requires_contractgates inuniversal/idempotency.yaml—parallel_dispatch_runner(line 590) andrate_limit_trip_runner(line 769). Those kits are runner capability contracts (harness_mode: black_box,burst_step_contract), not credentials, so the local skip means the local CLI does not implement that harness. Loading the kits explicitly changes nothing. Whether the hosted runner implements them I cannot see, so its 2 failures may be real.rate_limit_trip_runnerdeclaresexhausted_outcome: not_applicableafter up to 500 attempts. This agent implements no rate limiting at all, so a runner that does drive the burst would exhaust it and producenot_applicable— which, per the residual above, would surface on the card as a failure regardless.stale_response_advisory(hosted 3/4,failure_count1) is not a candidate either: this agent does not implementforce_upstream_unavailable, and the local run is alsoSTORYBOARD-PARTIALwith 4 advisory validation failures. Local and hosted agree it is not clean.One housekeeping note: this issue's title still says "53 of 62 failing steps on two cards". Across the two cards today that is 21 (20 + 1), mostly because #7405/#7406 fixed the selector half. Might be worth a retitle so the remaining scope reads as bug 2 only.
fix(compliance): correct two validate_input gates and coverage floors
EvgenyAndroid commentedon Sep 16, 2026
Correction to my 2026-09-15 comment. The numbers in it were right for the two runs I sampled. The conclusions I drew from them were wrong, and #7454, linked in this thread on 2026-09-13, already answered most of it.
1. The feed does represent skips, and bug 2 was never a reporting artifact. Before #7454 the serializer already passed
skipped_countthrough, but onlyprerequisite_failedcascades were counted there. Neutral skips and their cascades were dropped fromsteps_total. Every other skip counted as a failure. My 73 rows had no counted cascade, sofailure_countmatched the residual everywhere. That is grading, as filed.first_failed_*is also null because the public feed is redacted ("public failure observations/headlines remain redacted", #7454). So my line that an owner cannot tell which class a step was is wrong too.2. Bug 2 moved, and I missed it. #7454 (merged 2026-09-13 14:16 UTC) classifies
fixture_seed_unsupportedasnot_applicable. That is the reason I quoted for these steps on 2026-09-13; before #7454 it fell through to a failure. On the signals card,deterministic_testing(0/7) andidempotency(0/11) have readuntested0/0 on every run since 2026-09-14 20:03 UTC, the first run to carry #7454's provenance fields and the run I cited. Those 18 steps are no longer counted as failures, though the card doesn't show them as passing either, as the local SDK does. Across the same line, the whole signals card went from the 36 failing steps I counted on 2026-09-13 to 1 (billing_gate_dispatch). All of those runs also reportsdk_version14.0.0-rc.35, so I can't attribute the whole change to #7454.3. My 3.1.20 point was contradicted by its own evidence. That same 09-14 run already showed
canonical_format_validate_inputasuntested0/0.For the retitle note: the two cards now sum to 11 failing steps (sales 10, signals 1), not 21. On sales,
canonical_format_validate_inputpasses 16/16 because this agent now implementsvalidate_input, not because of a runner change. All four runs dated 2026-09-16 still reportcompliance_bundle_version: 3.1.20.[-]Hosted grader (3.1.20): the storyboard gate admits on any one required tool, and steps the SDK marks not_applicable are graded as failures — 53 of 62 failing steps on two cards are ungradable[/-][+]Hosted grader: 11 failing steps remain on two cards after #7454/#7405/#7406 (sales 10, signals 1); budget-truncated runs still published as complete grades[/+]chore(release): forward-merge 3.1.x into main (#7562)
fix(compliance): correct two validate_input gates and coverage floors (…