Skip to content

feat: extract core/ and gate the suites on advertised capabilities (RFC #520, 3/3) - #122

Merged
carolinerg1 merged 5 commits into
Universal-Commerce-Protocol:mainfrom
segiodongo:split/3-core-common-suites
Nov 13, 2026
Merged

carolinerg1 merged 5 commits into
Universal-Commerce-Protocol:mainfrom
segiodongo:split/3-core-common-suites

Conversation

@segiodongo

@segiodongo segiodongo commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Last of three PRs replacing #107 (after #120 and #121). All test-content changes live here, after the moves have landed:

  • core/ — protocol_test.py (discovery document: required ucp wrapper, version, capability and handler names, URLs) and binding_test.py (transport headers / JSON content type). shopping/checkout/protocol_test.py keeps the service-advertisement checks and the version negotiation test, which asserts the 422 on POST /checkout-sessions.
  • Capability gating — every suite declares what it exercises with @requires_capability; common/payments/ gates on dev.ucp.common.payment.ap2_mandate (the pre-2026-08-25 name is recognised via framework.discovery.FORMER_NAMES) and on the checkout endpoint for card credentials and token binding.
  • Imports — tests use shopping.base directly; integration_test_utils becomes a deprecated facade for external users. The readiness test from test: assert ready_for_complete when nothing is outstanding #117, which landed in the moved file, uses self.get_headers() / get_valid_payment_payload() here for the same reason.
  • Discount tests grouped under shopping/discount/ (moved from the business-logic and validation modules, bodies unchanged).

Review feedback from #107 addressed here: capability names that no server advertises, TokenBindingTest placement, shopping-dependent tests kept out of core/, the ucp wrapper assertion, and the two core/ tests that could not observe what they claimed (GET-based negotiation, Content-Type on a GET-only route) are not reintroduced.

Verification (sample server, 2026-04-08): 80 tests, 78 passed, 2 skipped (test_discovery_urls; AP2, which the sample server does not advertise), 0 failed — #107's final result plus #117's test, whose content this branch reproduces.

@niranjanmanjunath niranjanmanjunath left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to rebase

@segiodongo

Copy link
Copy Markdown
Collaborator Author

Rebased onto main now that #121 has merged — this is now just the four core/ + capability-gating commits (80 tests, 78 passed, 2 skipped against the sample server; the extra test is #117's readiness check, which now lives in the moved file).

Ready for review @damaz91 @carolinerg1 @niranjanmanjunath — thanks!

@damaz91 damaz91 added status:needs-triage Signal that the PR is ready for human triage status:under-review and removed status:needs-triage Signal that the PR is ready for human triage labels Nov 11, 2026

@damaz91 damaz91 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for completing the 3-part RFC #520 refactor and addressing all the review feedback from #107, @segiodongo! Verified that all 80 tests resolve cleanly without deprecation warnings across ucp-sdk 0.4.6 and 0.5.0 and pass against the sample server.

Approving with a couple of non-blocking inline suggestions (two method-level @requires_capability annotations for optional shopping extensions and two minor doc/help-text nits) that can be addressed here or in a quick follow-up.

@@ -380,12 +394,12 @@ def test_complete_without_fulfillment(self) -> None:
response_json = self.create_checkout_session(select_fulfillment=False)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking suggestion: consider decorating test_complete_without_fulfillment (line 385) with @requires_capability("dev.ucp.shopping.fulfillment") (similar to BusinessLogicTest.test_buyer_consent). On a checkout server that does not advertise dev.ucp.shopping.fulfillment (e.g., digital goods or services), completing without fulfillment legitimately returns 200 OK (status: "completed"), which would fail assert_business_error instead of skipping.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — test_complete_without_fulfillment now carries @requires_capability("dev.ucp.shopping.fulfillment"), so it skips rather than fails on a server without the extension.



class TotalsTest(integration_test_utils.IntegrationTestBase):
@requires_capability("dev.ucp.shopping.checkout")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking suggestion: test_discount_entry_is_negative (line 180) calls _discounted_checkout_totals(), which sends discounts={"codes": [discount_code]} on PUT /checkout-sessions/{id} and asserts 200 OK before reaching the skipTest guard. Decorating test_discount_entry_is_negative with @requires_capability("dev.ucp.shopping.discount") will avoid sending an unnegotiated discounts extension field to servers that do not advertise dev.ucp.shopping.discount and validate strictly against their composed capability schema.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — test_discount_entry_is_negative now carries @requires_capability("dev.ucp.shopping.discount"), so no unnegotiated discounts field is sent to a server that doesn't advertise it.

Comment thread core/__init__.py Outdated

"""Core UCP protocol conformance tests.

Covers transport, discovery, idempotency, and security.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: idempotency_test.py and simulation_url_security_test.py now live under shopping/.

Suggested change
Covers transport, discovery, idempotency, and security.
Covers transport binding and discovery.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — applied your wording.

Comment thread README.md
`--suite` accepts a suite name, a test module or class path, or a
comma-separated combination. Use `-k/--filter` to run a subset of tests by
name, and `--dry-run` to list what would execute without contacting the server.
`--suite` accepts `core`, `common`, `shopping`, `all`, or a comma-separated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (also in runner.py --suite help text): _resolve_suites() still accepts module, class, or file paths (e.g., --suite=shopping/discount/discount_test.py), which may still be worth mentioning here now that the direct uv run shopping/checkout/lifecycle_test.py example below was removed.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — the README paragraph and the --suite help text now mention the module / class / file-path forms, with two examples (verified both resolve: 5 and 12 tests).

Third step of the RFC #520 split of Universal-Commerce-Protocol#107. Every shopping test now imports
`ShoppingIntegrationTestBase` from `shopping.base` instead of the
`integration_test_utils` facade, calls `self.get_headers()`, tolerates the
SDK's older and newer locations for the payment models, and declares the
capability it exercises with `@requires_capability`, so a server that does
not advertise, say, `dev.ucp.shopping.discount` skips those tests instead
of failing them. Test bodies are unchanged.

`conftest.py` takes `FLAGS` from the framework for the same reason.
The three payment credential tests under `common/payments/` were gated on
capability names no server advertises, so they always skipped. Now:

- `Ap2MandateTest` requires `dev.ucp.common.payment.ap2_mandate`, the name
  the schema defines. Releases before 2026-08-25 advertised it as
  `dev.ucp.shopping.ap2_mandate`; `framework.discovery` records that rename
  (`FORMER_NAMES`) so the test runs against either generation of server.
- `CardCredentialTest` and `TokenBindingTest` are gated on the checkout
  endpoint they drive: card credentials and token binding are payment data
  types, not advertised capabilities.

They import `shopping.base` directly, which leaves `integration_test_utils`
without in-tree users; it now emits a DeprecationWarning for external ones.
`core/` holds the tests any UCP server must pass regardless of the
verticals it implements, which is everything observable from the
discovery document alone:

- `core/protocol_test.py`: the discovery checks from the former shopping
  protocol test (required `ucp` wrapper, date-based version matching the
  conformance input, reverse-DNS capability and handler names, payment
  handler structure, published spec/schema URLs).
- `core/binding_test.py`: the standard transport headers are accepted and
  discovery answers with `application/json`.

`shopping/checkout/protocol_test.py` keeps what needs the shopping vertical:
the service advertisement checks and version negotiation, which can only
be observed on `POST /checkout-sessions` (422 for an unsupported version).

The runner gains the `core` suite and the README documents the tiers.
The discount flow tests (`test_discount_flow`, `test_multiple_discounts_*`,
`test_fixed_amount_discount`) move from the checkout business-logic module
to `shopping/discount/discount_test.py`, and `test_unknown_discount_code`
joins them from the validation suite, so every discount test lives with
the capability it exercises and is gated on `dev.ucp.shopping.discount`.
`shopping/discount/semantics_test.py` (the former `discount_test.py`) is
unchanged. Test bodies are unchanged; only their module and gate move.
…view)

Review feedback on Universal-Commerce-Protocol#122:

- test_complete_without_fulfillment requires dev.ucp.shopping.fulfillment:
  on a server without the extension, completing without fulfillment is a
  legitimate 200, so the test now skips there instead of failing.
- test_discount_entry_is_negative requires dev.ucp.shopping.discount, so it
  no longer sends an unnegotiated discounts field to a server that does not
  advertise the extension and validates strictly.
- core/__init__.py: the idempotency and simulation-URL tests live under
  shopping/, so the docstring names only transport binding and discovery.
- README and --suite help: a test module, class or file path is also
  accepted, with examples.
@segiodongo
segiodongo force-pushed the split/3-core-common-suites branch from f6a8d35 to f87ae19 Compare November 12, 2026 18:50
@segiodongo

Copy link
Copy Markdown
Collaborator Author

Thanks for the review @damaz91! All four suggestions are in as a separate commit (f87ae19), replies inline. Also rebased onto main for #112: field_ownership_test.py now imports from shopping.base and declares dev.ucp.shopping.checkout, like its siblings, and is registered alongside the discount module.

Sample server: 83 tests, 81 passed, 2 skipped, 0 failed (80 + #112's three).

@carolinerg1
carolinerg1 merged commit 0642adf into Universal-Commerce-Protocol:main Nov 13, 2026
19 checks passed
aeoess added a commit to aeoess/conformance that referenced this pull request Nov 14, 2026
Rebased onto the tiered layout (Universal-Commerce-Protocol#121, Universal-Commerce-Protocol#122). The AP2 helpers and the mock
agent profile server that declares AP2 Mandates move out of the deprecated
integration_test_utils facade into common/payments/ap2_support.py, next to
the test. Ap2MandateTest keeps the requires_capability gate that main added
and still skips unless AP2 is in the negotiated capability intersection.
Test bodies and the AP2 helpers are unchanged from afcc958.

Signed-off-by: Tymofii Pidlisnyi <tima@aeoess.com>
vishkaty added a commit to vishkaty/conformance that referenced this pull request Nov 17, 2026
Four cases over cart conversion: conversion uses cart contents over the
payload, a second conversion returns the existing checkout, cancel returns the
cart state before deletion, and cart discount codes carry forward.

Placed as `shopping/cart/cart_test.py` in a new `shopping/cart/` package,
following the layout Universal-Commerce-Protocol#121 landed, and registered as `shopping.cart.cart_test`.

What makes the module execute: without the registry entry `--suite=all`
resolves 83 cases and selects none of these four, and with it 87.

Rebased onto main past Universal-Commerce-Protocol#122. The only conflict was the registry list, where
Universal-Commerce-Protocol#112 and Universal-Commerce-Protocol#122 had each added an entry. The cart entry sits at the head of the
shopping list, which keeps that list grouped by subpackage in alphabetical
order.

Obligations are cited by section name, in the module docstring and at each
case, rather than by line number. The line numbers the module carried were
correct when it was written and had since drifted, one of them onto an
unrelated paragraph, which is the hazard of citing a moving document by line.

Two cases were judging a conformant business wrongly, and both are corrected
here. The discount case used the presence of the code in the response as its
evidence that the code had applied, but a business may return a code it
received without applying it, as the reference does for an unknown code. With
a configured code that does not apply at the business under test, the case
therefore ran with nothing applied, and it failed a business that carried its
applied codes forward and dropped the unapplied one with a
`discount_code_invalid` warning, which discount.md allows. Application now
needs an applied entry naming the code or a discount in totals, so such a
code skips the case instead. Carry-forward is still satisfied by naming the
code, and a discount standing in for it must now be at least as large as the
one on the cart, so an unrelated discount no longer counts.

The contents comparison took line items as a bag of id and quantity pairs,
which failed a business expressing two units as two single-unit lines. It now
sums quantity per item id and leaves out lines naming a parent, which still
catches a business that takes its quantities from the request payload.

Verified against the flower shop reference: the module alone gives 4 passed
and 0 skipped; --suite=all gives 87 executed, 85 passed, 2 skipped, 0
failures; the pinned pre-commit, ruff, ruff format and cspell are clean. Nine
servers each breaking one obligation redden their own case and nothing else,
and the two corrections above are each held by a server that fails the old
code and passes the new one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants