Skip to content

feat(signatures): publish RFC 9421 test vectors and sign every example in the spec - #1008

Open
sakinaroufid wants to merge 3 commits into
Universal-Commerce-Protocol:mainfrom
sakinaroufid:feat/signature-test-vectors
Open

sakinaroufid wants to merge 3 commits into
Universal-Commerce-Protocol:mainfrom
sakinaroufid:feat/signature-test-vectors

Conversation

@sakinaroufid

Copy link
Copy Markdown
Contributor

Description

Summary: the specification’s signed examples carried placeholder digest and signature values that could not be verified. This change publishes RFC 9421 test vectors for the UCP signing profile, replaces every inline example with a message signed over the exact bytes shown using the RFC’s published test keys, and adds a CI check that rebuilds and verifies each example on every change. An example that is edited without being re-signed fails the build.

1. Test vectors. signatures.md profiles RFC 9421 for UCP but every signature on it was a placeholder, so there was nothing to run a signer or verifier against. This adds scripts/signature_vectors.py, a small RFC 9421 implementation scoped to what the UCP profile uses (derived components @method @authority @path @query @status, fields, ;key dictionary member selection, created/expires/nonce/keyid/tag), proven against the RFC's own Appendix B vectors. It generates docs/specification/signatures/test-vectors.json: six vectors covering each signing shape the spec defines (REST request, GET, response, WBA-shape request with Ed25519, order webhook, MCP request), signed with the RFC 9421 B.1.3 and B.1.4 test keys so a suite that already has those keys needs nothing new. ECDSA uses RFC 6979 nonces, so a deterministic ES256 signer reproduces the values byte for byte. A new Test Vectors section on signatures.md quotes the corpus and test_signature_vectors.py checks the corpus verifies, matches the generator, and is quoted verbatim.

2. The inline examples. The checkout, order, booking and signatures pages show 15 signed requests/responses. All of them were placeholders: 6G4i8TS6oUkGrx8KnCFUpsSPwd74... pasted eleven times, X48E9q... digests, and the four Lodging ones start with MEUC/MFQC, i.e. DER, which the page itself forbids. That's how #342, #370, #590, #659, #795 and #963 happened: an example and the rule next to it could disagree and nothing noticed.

signature_vectors.py verify-docs now finds every fenced block with a complete Signature-Input, rebuilds the message from the block (request/status line, headers in wire order, body = exact bytes after the blank line, no trailing newline), checks Content-Digest against the body, checks the covered components against the coverage gate, and verifies the signature with the test keys. sign-docs writes real digest/keyid/signature values into a block and leaves everything else alone, so what an example claims to cover never changes under it. Blocks with a truncated Signature-Input (... in the component list) are fragments and are skipped; ... in a digest or signature value is a placeholder and fails.

All 15 are now signed. Elided bodies became short complete payloads (a digest over {...} isn't reproducible). Two content changes beyond that, both things the coverage gate requires: the GET example on signatures.md gains UCP-Agent and covers it, and the Lodging REST request now covers ucp-agent like every other request does. The GET example ends up byte-identical to the rest-get-es256 vector.

verify-docs runs in the docs lint job and from pre-commit on changed spec pages. Edit a signed example and the hook tells you to re-sign it.

Not in this PR: the webhook delivery contract and the ready_for_complete definition that were on the same branch earlier. Those are normative changes and will come separately.

Category (Required)

  • Core Protocol
  • Governance/Contributing
  • Capability
  • Documentation: Updates to README, or documentations regarding schema or capabilities. (Requires Maintainer approval)
  • Infrastructure: CI/CD, Linters, or build scripts. (Requires DevOps Maintainer approval)
  • Maintenance
  • SDK
  • Samples / Conformance
  • UCP Schema
  • Community Health (.github)

Related Issues

#342 #370 #590 #659 #795 #963

Checklist

  • I have followed the Contributing Guide (including Conventional Commits title requirements and ! for breaking changes).
  • I have updated the documentation (if applicable).
  • My changes pass all local linting and formatting checks.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • (For Core/Capability) I have included/updated the relevant JSON schemas.
  • I have regenerated Python Pydantic models by running generate_models.sh under python_sdk.

Screenshots / Logs (if applicable)

verify-docs against the pages on main, two of the fourteen placeholder blocks (the key check comes first, so that is what a placeholder fails on):

  FAIL  docs/specification/lodging/booking/rest.md:1582
          keyid 'platform-2025' is not a published test key; run `signature_vectors.py sign-docs` to sign this example
  FAIL  docs/specification/shopping/checkout/rest.md:1452
          keyid 'merchant-2026' is not a published test key; run `signature_vectors.py sign-docs` to sign this example

The one example on main that already used a test key id (the WBA example on signatures.md) fails on its content instead:

  FAIL  docs/specification/signatures.md:262
          content-digest ['sha-256=:X48E9q...:'] != sha-256=:XmgkaOke8y0x8ktvJd2/H7ygjWV7ZJI3yHho1XefrAI=:
          malformed Signature: Incorrect padding

this branch:

$ uv run python scripts/signature_vectors.py verify-docs | tail -1
20 passed, 0 failed
$ uv run python scripts/signature_vectors.py sign-docs | tail -1
0 example(s) rewritten
$ uv run python scripts/test_signature_vectors.py | tail -1
100 passed, 0 failed
$ uv run python scripts/validate_examples.py --schema-base source/schemas/ | tail -1
407 passed, 0 failed, 0 errors, 55 skipped

Before/after on the Lodging REST page, complete request:

-Content-Digest: sha-256=:X48E9qOokqqrvdts8nOJRJN3OWDUoyWxBf7kbu9DBPE=:
-Signature-Input: sig1=("@method" "@authority" "@path" "idempotency-key" "content-digest" "content-type");keyid="platform-2025"
-Signature: sig1=:MEUCIQDTxNq8h7LGHpvVZQp1iHkFp9+3N8Mxk2zH1wK4YuVN8w...:
+Content-Digest: sha-256=:<sha-256 of the body shown>:
+Signature-Input: sig1=("@method" "@authority" "@path" "ucp-agent" "idempotency-key" "content-digest" "content-type");keyid="test-key-ecc-p256"
+Signature: sig1=:<64-byte raw r||s, verifies with test-key-ecc-p256>:

…rpus

Every example signature in the Message Signatures specification is a
placeholder, so implementers have had nothing to run a signer or
verifier against. Add worked, reproducible test vectors for each signing
shape the spec defines: REST request with and without a body, REST
response, Web Bot Auth-shaped request (Ed25519 with Dictionary-member
selection, thumbprint keyid, created/expires/nonce/tag), order webhook
delivery, and an MCP tools/call over streamable HTTP.

The vectors sign with the RFC 9421 Appendix B.1.3 and B.1.4 test keys,
whose Ed25519 thumbprint is the kid the spec already uses. ECDSA
signatures use RFC 6979 deterministic nonces and raw r||s encoding so a
deterministic ES256 signer reproduces them byte for byte.

The corpus is published next to the spec as
specification/signatures/test-vectors.json and quoted verbatim in a new
non-normative Test Vectors section. scripts/signature_vectors.py
generates it; scripts/test_signature_vectors.py proves the builder
against RFC 9421's own Appendix B vectors, re-verifies every published
vector from first principles, rejects tampered bodies, headers,
signatures, keys and DER-encoded ECDSA, and fails if the page or the
corpus drifts from the generator. The check runs in pre-commit and in
the docs CI lint job. cryptography is added to the dev dependency group.
… example

The spec's GET Request Example carries no UCP-Agent header, so a default
UCP verifier cannot resolve its key. The rest-get-es256 vector adds the
header and covers it; say so instead of claiming it mirrors the example.
The capability pages and the signatures page show 15 signed requests
and responses, and every one carried placeholder values: the same
`6G4i8TS6oUkGrx8KnCFUpsSPwd74...` pasted eleven times, digests like
`X48E9q...`, and four Lodging examples whose signatures start with
`MEUC`/`MFQC`, which is DER, the encoding the page forbids. Nothing
could verify them, so an example and the rule it illustrates could
disagree until someone read both (Universal-Commerce-Protocol#342, Universal-Commerce-Protocol#370, Universal-Commerce-Protocol#590, Universal-Commerce-Protocol#659, Universal-Commerce-Protocol#795, Universal-Commerce-Protocol#963).

Add `verify-docs` and `sign-docs` to scripts/signature_vectors.py.
verify-docs finds every fenced block with a complete Signature-Input,
rebuilds the message from the block (request or status line, headers
in wire order, body as the exact bytes after the blank line, no
trailing newline), checks Content-Digest against the body, checks the
covered components against the spec's coverage gate, and verifies the
signature with the published test keys. sign-docs writes the digest,
keyid and signature into the block and leaves everything else as
written, so what an example says it covers never changes under it. A
block whose Signature-Input is truncated with "..." is a fragment and
is skipped; a "..." in a digest or signature is a placeholder and
fails.

Sign the 15 examples. Elided bodies become short complete payloads,
since a digest over `{...}` is not something an implementer can
reproduce. The GET example gains the UCP-Agent header the coverage
gate requires, and the Lodging REST request now covers ucp-agent like
every other request. The GET example ends up byte-identical to the
rest-get-es256 vector.

verify-docs runs in the docs lint job and from pre-commit on changed
spec pages. test_signature_vectors.py covers the page examples: all
verify, sign-docs is a no-op on the committed pages, a placeholder is
rejected until signed, a tampered body or covered header is reported,
a missing required component is reported, and a DER value is named as
such.

Verified: test_signature_vectors 100 passed; verify-docs 20 passed;
validate_examples 407 passed; full draft build and check_links clean.
@sakinaroufid

Copy link
Copy Markdown
Contributor Author

Review guide, since this is a large PR:

  • scripts/signature_vectors.py (1,118): RFC 9421 implementation scoped to the UCP profile. self_test() reproduces Appendix B.2.4 and B.2.6 byte for byte; the structured-field parser is the bulk. Worth a read: signature_base, verify_example, sign_example.
  • scripts/test_signature_vectors.py (655): fixtures and assertions, 100 tests.
  • docs/specification/signatures/test-vectors.json (237) and the Test Vectors section of signatures.md (257): generated and quoted output. A test asserts they match the generator verbatim, so no need to read them by hand.
  • The 15 signed examples on the checkout, order, booking and signatures pages (110): the part that changes what readers see. Each is now verified in CI.
  • Lockfile, cspell, CI wiring (90): mechanical.

The three commits are independent and each builds and passes tests on its own, so they can be reviewed one at a time from the Commits tab.

@sakinaroufid

Copy link
Copy Markdown
Contributor Author

I'm anticipating this qn from tech council (since this is quite a large PR by LOC) ; Why this PR has its own signing code instead of a library. so answering it below;

scripts/signature_vectors.py implements the parts of RFC 9421 that UCP uses, and only those:

  • the five derived components: @method, @authority, @path, @query, @status
  • HTTP header components, including ;key="sig1" member selection for Signature-Agent
  • the five signature parameters: created, expires, nonce, keyid, tag
  • ECDSA output as raw r||s (64 bytes), which the spec requires and most libraries do not produce

It is checked against the RFC's own test vectors every time it runs: self_test() reproduces Appendix B.2.4 (ES256 response) and B.2.6 (Ed25519 request) byte for byte, and fails the build if it cannot.

No Python library I found covers both ;key selection and raw r||s output, which is why the code is here. If the Tech Council prefers a dependency once one does, the change is limited to three functions (sign, verify, signature_base). The corpus, the signed page examples and the CI check do not depend on how the bytes are produced.

@damaz91 damaz91 added the status:needs-triage Signal that the PR is ready for human triage label Oct 29, 2026
@gsmith85 gsmith85 added this to the 2026-12-08 milestone Oct 29, 2026
@gsmith85 gsmith85 removed this from the 2026-12-08 milestone Nov 9, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status:needs-triage Signal that the PR is ready for human triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants