Repository navigation
Description
Summary
The vendored AdCP request-signing conformance vectors under tests/conformance/vectors/request-signing/ are an incomplete copy of the spec set, and the tests that grade against them are structurally incapable of noticing.
Both graders load vectors by globbing and then assert only that the result is non-empty:
tests/conformance/signing/test_verifier_vectors.py:77-78—_load()doesglob("*.json")thenassert result, f"no vectors under ..."tests/conformance/signing/test_canonicalization.py:24-31— same shape for vectors carryingexpected_signature_base
Any non-empty subset satisfies that assertion. A missing vector is indistinguishable from a complete set, so the suite reports green on a partial copy — which is exactly what has been happening.
The gap
Measured against AdCP 3.1.1, the version src/adcp/ADCP_VERSION already targets (spec repo dist/compliance/3.1.1/test-vectors/request-signing/):
| spec 3.1.1 | vendored | missing | |
|---|---|---|---|
positive/ |
12 | 8 | 009-percent-encoded-unreserved-decoded, 010-percent-encoded-slash-preserved, 011-ipv6-authority, 012-ipv6-authority-default-port-stripped |
negative/ |
28 | 20 | 021-duplicate-signature-input-label, 022-multi-valued-content-type, 023-multi-valued-content-digest, 024-unquoted-string-param, 025-jwk-alg-crv-mismatch, 026-non-ascii-host, 027-webhook-registration-authentication-unsigned, 028-unsigned-protocol-method-required |
canonicalization.json |
ships | absent | the standalone signing-base fixture |
canonicalization.json is the conspicuous one: the vendored README.md documents it in the file-layout tree and devotes a full section to it, describing it as "the fastest way to surface cross-implementation divergence" and recommending SDKs "run canonicalization.json on every commit". The file was never copied in.
Three further vectors had silently drifted from their upstream contents: 016-replayed-nonce, 017-key-revoked, 020-rate-abuse. 017 is the consequential one — it was vendored at a revision that revokes test-ed25519-2026, the primary signing key, rather than the dedicated test-revoked-2026 keypair the spec now uses. keys.json had never been refreshed to carry that key either.
Why it matters
Signature-base construction is the known-hard part of RFC 9421 interop, and the missing vectors sit exactly on its edges: 009/010 are percent-encoding, 011/012 are IPv6 authority, 026 is a non-ASCII host, 028 is the protocol_methods_required_for enforcement path.
A verifier can pass every currently-vendored vector and still reject validly-signed requests from a conformant counterparty. And because the SDK owns canonicalization on behalf of its consumers, these gaps are untested in every downstream implementation too, not just in this suite — this was found while adopting adcp.signing in a seller implementation that delegates canonicalization entirely to the SDK.
Fix
PR #980 vendors the complete 3.1.1 set and, more importantly, replaces the non-empty assertion with a completeness pin so the set cannot silently drift again at the next spec bump. Adding the 12 files without that guard would only reset the clock.
The pin is over content hashes, not filenames — a filename-only manifest would not have caught the 016/017/020 drift. It is tied to get_adcp_spec_version(), so bumping the targeted spec version without re-vendoring fails loudly.
Grading the new vectors turns up 12 real conformance failures
They are filed separately and marked strict-xfail in the PR rather than fixed there, since they are production behavior changes in a security-critical verifier:
- The verifier has no step-1 strict-parse stage, so malformed/ambiguous input flows through to a later check and is rejected under the wrong error code (negative
021,022,023,026). Conformance requires byte-for-byte error codes, so "rejected anyway" is not a pass. @target-uricanonicalization applies no UTS-46 A-label conversion, drops a trailing empty query, and accepts five malformed authority shapes the spec requires be rejected.
Activity
bokelley commentedon Jul 28, 2026
Triage
Classification: Bug
Bucket(s): signing, validation (conformance test infrastructure)
Status: ready-to-implement
What the experts said:
assert resultdefect is real in both loaders; the 017 drift is confirmed (test-ed25519-2026is in the revocation list but notest-revoked-2026keypair exists inkeys.json); content-hash pin tied toget_adcp_spec_version()is the right circuit-breaker;strict-xfailis correct tradeoff; non-breaking.{version: {subdir: expected_count}}) over a full content-hash dict — same missing-file coverage with lower maintenance cost on legitimate vector refreshes;strict-xfailis honest but should be treated as strictly temporary with linked tracking issues and a milestone.My take: The diagnosis is airtight and independently verified against the filesystem. The implementation is non-breaking (test-only changes; new parametrized IDs are additive;
strict-xfailkeeps CI green). The one open implementation question is content-hash vs. filename-count for the completeness guard — the code reviewer makes a reasonable point that a count manifest survives content refreshes with less ceremony, while the protocol expert notes that content hashes would have caught the silent 016/017/020 drift that the issue describes. I'd give the implementer discretion here, but flag the tradeoff explicitly.Implementation scope:
009-percent-encoded-unreserved-decoded,010-percent-encoded-slash-preserved,011-ipv6-authority,012-ipv6-authority-default-port-stripped), 8 negative (021–028), andcanonicalization.jsonfromdist/compliance/3.1.1/test-vectors/request-signing/in the spec repo.016-replayed-nonce,017-key-revoked,020-rate-abuseto upstream 3.1.1 contents; addtest-revoked-2026tokeys.json.test_verifier_vectors.py:_load()andtest_canonicalization.py:_vectors_with_expected_base()with a completeness guard keyed onget_adcp_spec_version(). A content-hash dict catches silent drift (what happened to 016/017/020); a filename-count manifest is lighter to maintain on refreshes. Pick one and document the tradeoff in a comment.canonicalization.jsonloader — the file isn't inpositive/ornegative/, sotest_canonicalization.pyneeds an explicit path for it. Wire it into the parametrize call or a separate test function.strict-xfail— per the linked issues (Verifier does not reject malformed structured-field input at step 1 (wrong error code; covered-component smuggling) #976, @target-uri canonicalization applies no IDNA A-label conversion, and the verifier does not reject raw U-labels #977).strict=Trueensures these are revisited when production conformance improves; they must not be markedxfailwithoutstrict.pytest tests/conformance/signing/(no slow integration tier unlesstest_revocation_e2e.pyortest_pg_replay_store_e2e.pyare touched),ruff check .,black --check .. Nosrc/edits expected, somypy src/can be skipped unless the completeness guard introduces new imports.tests/; no public API surface is touched; existing test IDs are unchanged;strict-xfailprevents new failures from breaking CI until the production fixes land.Note: the issue body references "PR #<PR>" as a companion PR, but no open PR was found targeting this issue. The companion PR has not yet been filed.
Triaged by Claude Code. Session: https://claude.ai/code/${CLAUDE_CODE_REMOTE_SESSION_ID}
Generated by Claude Code
test(signing): re-pin the vendored vectors at AdCP 3.1.8
fix(signing): preserve a trailing empty query in @target-uri
fix(signing): preserve a trailing empty query in @target-uri
fix(signing): preserve a trailing empty query in @target-uri
test(signing): vendor the complete AdCP 3.1.8 request-signing vector …
fix(signing): preserve a trailing empty query in @target-uri
fix(signing): preserve a trailing empty query in @target-uri (#987)