Repository navigation
feat: extract core/ and gate the suites on advertised capabilities (RFC #520, 3/3) - #122
Conversation
8d84e15 to
55a9994
Compare
niranjanmanjunath
left a comment
There was a problem hiding this comment.
I think you need to rebase
55a9994 to
f6a8d35
Compare
|
Rebased onto Ready for review @damaz91 @carolinerg1 @niranjanmanjunath — thanks! |
damaz91
left a comment
There was a problem hiding this comment.
Thank you for completing the 3-part RFC #520 refactor and addressing all the review feedback from #107, @segiodongo! Verified that all 80 tests resolve cleanly without deprecation warnings across ucp-sdk 0.4.6 and 0.5.0 and pass against the sample server.
Approving with a couple of non-blocking inline suggestions (two method-level @requires_capability annotations for optional shopping extensions and two minor doc/help-text nits) that can be addressed here or in a quick follow-up.
| @@ -380,12 +394,12 @@ def test_complete_without_fulfillment(self) -> None: | |||
| response_json = self.create_checkout_session(select_fulfillment=False) | |||
There was a problem hiding this comment.
Non-blocking suggestion: consider decorating test_complete_without_fulfillment (line 385) with @requires_capability("dev.ucp.shopping.fulfillment") (similar to BusinessLogicTest.test_buyer_consent). On a checkout server that does not advertise dev.ucp.shopping.fulfillment (e.g., digital goods or services), completing without fulfillment legitimately returns 200 OK (status: "completed"), which would fail assert_business_error instead of skipping.
There was a problem hiding this comment.
Done — test_complete_without_fulfillment now carries @requires_capability("dev.ucp.shopping.fulfillment"), so it skips rather than fails on a server without the extension.
|
|
||
|
|
||
| class TotalsTest(integration_test_utils.IntegrationTestBase): | ||
| @requires_capability("dev.ucp.shopping.checkout") |
There was a problem hiding this comment.
Non-blocking suggestion: test_discount_entry_is_negative (line 180) calls _discounted_checkout_totals(), which sends discounts={"codes": [discount_code]} on PUT /checkout-sessions/{id} and asserts 200 OK before reaching the skipTest guard. Decorating test_discount_entry_is_negative with @requires_capability("dev.ucp.shopping.discount") will avoid sending an unnegotiated discounts extension field to servers that do not advertise dev.ucp.shopping.discount and validate strictly against their composed capability schema.
There was a problem hiding this comment.
Done — test_discount_entry_is_negative now carries @requires_capability("dev.ucp.shopping.discount"), so no unnegotiated discounts field is sent to a server that doesn't advertise it.
|
|
||
| """Core UCP protocol conformance tests. | ||
|
|
||
| Covers transport, discovery, idempotency, and security. |
There was a problem hiding this comment.
Nit: idempotency_test.py and simulation_url_security_test.py now live under shopping/.
| Covers transport, discovery, idempotency, and security. | |
| Covers transport binding and discovery. |
There was a problem hiding this comment.
Done — applied your wording.
| `--suite` accepts a suite name, a test module or class path, or a | ||
| comma-separated combination. Use `-k/--filter` to run a subset of tests by | ||
| name, and `--dry-run` to list what would execute without contacting the server. | ||
| `--suite` accepts `core`, `common`, `shopping`, `all`, or a comma-separated |
There was a problem hiding this comment.
Nit (also in runner.py --suite help text): _resolve_suites() still accepts module, class, or file paths (e.g., --suite=shopping/discount/discount_test.py), which may still be worth mentioning here now that the direct uv run shopping/checkout/lifecycle_test.py example below was removed.
There was a problem hiding this comment.
Done — the README paragraph and the --suite help text now mention the module / class / file-path forms, with two examples (verified both resolve: 5 and 12 tests).
Third step of the RFC #520 split of Universal-Commerce-Protocol#107. Every shopping test now imports `ShoppingIntegrationTestBase` from `shopping.base` instead of the `integration_test_utils` facade, calls `self.get_headers()`, tolerates the SDK's older and newer locations for the payment models, and declares the capability it exercises with `@requires_capability`, so a server that does not advertise, say, `dev.ucp.shopping.discount` skips those tests instead of failing them. Test bodies are unchanged. `conftest.py` takes `FLAGS` from the framework for the same reason.
The three payment credential tests under `common/payments/` were gated on capability names no server advertises, so they always skipped. Now: - `Ap2MandateTest` requires `dev.ucp.common.payment.ap2_mandate`, the name the schema defines. Releases before 2026-08-25 advertised it as `dev.ucp.shopping.ap2_mandate`; `framework.discovery` records that rename (`FORMER_NAMES`) so the test runs against either generation of server. - `CardCredentialTest` and `TokenBindingTest` are gated on the checkout endpoint they drive: card credentials and token binding are payment data types, not advertised capabilities. They import `shopping.base` directly, which leaves `integration_test_utils` without in-tree users; it now emits a DeprecationWarning for external ones.
`core/` holds the tests any UCP server must pass regardless of the verticals it implements, which is everything observable from the discovery document alone: - `core/protocol_test.py`: the discovery checks from the former shopping protocol test (required `ucp` wrapper, date-based version matching the conformance input, reverse-DNS capability and handler names, payment handler structure, published spec/schema URLs). - `core/binding_test.py`: the standard transport headers are accepted and discovery answers with `application/json`. `shopping/checkout/protocol_test.py` keeps what needs the shopping vertical: the service advertisement checks and version negotiation, which can only be observed on `POST /checkout-sessions` (422 for an unsupported version). The runner gains the `core` suite and the README documents the tiers.
The discount flow tests (`test_discount_flow`, `test_multiple_discounts_*`, `test_fixed_amount_discount`) move from the checkout business-logic module to `shopping/discount/discount_test.py`, and `test_unknown_discount_code` joins them from the validation suite, so every discount test lives with the capability it exercises and is gated on `dev.ucp.shopping.discount`. `shopping/discount/semantics_test.py` (the former `discount_test.py`) is unchanged. Test bodies are unchanged; only their module and gate move.
…view) Review feedback on Universal-Commerce-Protocol#122: - test_complete_without_fulfillment requires dev.ucp.shopping.fulfillment: on a server without the extension, completing without fulfillment is a legitimate 200, so the test now skips there instead of failing. - test_discount_entry_is_negative requires dev.ucp.shopping.discount, so it no longer sends an unnegotiated discounts field to a server that does not advertise the extension and validates strictly. - core/__init__.py: the idempotency and simulation-URL tests live under shopping/, so the docstring names only transport binding and discovery. - README and --suite help: a test module, class or file path is also accepted, with examples.
f6a8d35 to
f87ae19
Compare
|
Thanks for the review @damaz91! All four suggestions are in as a separate commit ( Sample server: 83 tests, 81 passed, 2 skipped, 0 failed (80 + #112's three). |
0642adf
into
Universal-Commerce-Protocol:main
Rebased onto the tiered layout (Universal-Commerce-Protocol#121, Universal-Commerce-Protocol#122). The AP2 helpers and the mock agent profile server that declares AP2 Mandates move out of the deprecated integration_test_utils facade into common/payments/ap2_support.py, next to the test. Ap2MandateTest keeps the requires_capability gate that main added and still skips unless AP2 is in the negotiated capability intersection. Test bodies and the AP2 helpers are unchanged from afcc958. Signed-off-by: Tymofii Pidlisnyi <tima@aeoess.com>
Four cases over cart conversion: conversion uses cart contents over the payload, a second conversion returns the existing checkout, cancel returns the cart state before deletion, and cart discount codes carry forward. Placed as `shopping/cart/cart_test.py` in a new `shopping/cart/` package, following the layout Universal-Commerce-Protocol#121 landed, and registered as `shopping.cart.cart_test`. What makes the module execute: without the registry entry `--suite=all` resolves 83 cases and selects none of these four, and with it 87. Rebased onto main past Universal-Commerce-Protocol#122. The only conflict was the registry list, where Universal-Commerce-Protocol#112 and Universal-Commerce-Protocol#122 had each added an entry. The cart entry sits at the head of the shopping list, which keeps that list grouped by subpackage in alphabetical order. Obligations are cited by section name, in the module docstring and at each case, rather than by line number. The line numbers the module carried were correct when it was written and had since drifted, one of them onto an unrelated paragraph, which is the hazard of citing a moving document by line. Two cases were judging a conformant business wrongly, and both are corrected here. The discount case used the presence of the code in the response as its evidence that the code had applied, but a business may return a code it received without applying it, as the reference does for an unknown code. With a configured code that does not apply at the business under test, the case therefore ran with nothing applied, and it failed a business that carried its applied codes forward and dropped the unapplied one with a `discount_code_invalid` warning, which discount.md allows. Application now needs an applied entry naming the code or a discount in totals, so such a code skips the case instead. Carry-forward is still satisfied by naming the code, and a discount standing in for it must now be at least as large as the one on the cart, so an unrelated discount no longer counts. The contents comparison took line items as a bag of id and quantity pairs, which failed a business expressing two units as two single-unit lines. It now sums quantity per item id and leaves out lines naming a parent, which still catches a business that takes its quantities from the request payload. Verified against the flower shop reference: the module alone gives 4 passed and 0 skipped; --suite=all gives 87 executed, 85 passed, 2 skipped, 0 failures; the pinned pre-commit, ruff, ruff format and cspell are clean. Nine servers each breaking one obligation redden their own case and nothing else, and the two corrections above are each held by a server that fails the old code and passes the new one.
Last of three PRs replacing #107 (after #120 and #121). All test-content changes live here, after the moves have landed:
core/—protocol_test.py(discovery document: requireducpwrapper, version, capability and handler names, URLs) andbinding_test.py(transport headers / JSON content type).shopping/checkout/protocol_test.pykeeps the service-advertisement checks and the version negotiation test, which asserts the 422 onPOST /checkout-sessions.@requires_capability;common/payments/gates ondev.ucp.common.payment.ap2_mandate(the pre-2026-08-25 name is recognised viaframework.discovery.FORMER_NAMES) and on the checkout endpoint for card credentials and token binding.shopping.basedirectly;integration_test_utilsbecomes a deprecated facade for external users. The readiness test from test: assert ready_for_complete when nothing is outstanding #117, which landed in the moved file, usesself.get_headers()/get_valid_payment_payload()here for the same reason.shopping/discount/(moved from the business-logic and validation modules, bodies unchanged).Review feedback from #107 addressed here: capability names that no server advertises,
TokenBindingTestplacement, shopping-dependent tests kept out ofcore/, theucpwrapper assertion, and the twocore/tests that could not observe what they claimed (GET-based negotiation, Content-Type on a GET-only route) are not reintroduced.Verification (sample server, 2026-04-08): 80 tests, 78 passed, 2 skipped (
test_discovery_urls; AP2, which the sample server does not advertise), 0 failed — #107's final result plus #117's test, whose content this branch reproduces.