Skip to content

feat(models): remove postprocess_models.py and emit pure OOTB Pydantic DTOs - #103

Merged
gsmith85 merged 3 commits into
mainfrom
feat/pure-ootb-pydantic-dtos
Nov 18, 2026
Merged

gsmith85 merged 3 commits into
mainfrom
feat/pure-ootb-pydantic-dtos

Conversation

@gsmith85

@gsmith85 gsmith85 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Removes the custom Python AST/regex constraint post-processor (postprocess_models.py) and its test suite (tests/test_codegen_pipeline.py), emitting pure out-of-the-box Pydantic v2 wire DTOs directly from ucp-schema generate-types and datamodel-code-generator (Universal-Commerce-Protocol/ucp#817).

  • Delete postprocess_models.py and tests/test_codegen_pipeline.py, and remove obsolete postprocess_models.py rules from pyproject.toml.
  • Update generate_models.sh to keep src/ucp_sdk/models/schemas/__init__.py intact during codegen (mkdir -p "$OUTPUT_DIR" && rm -f "$OUTPUT_DIR/_generated.py") and emit the PEP 562 lazy src/ucp_sdk/models/schemas/__init__.py index directly via stdlib ast without mutating _generated.py.
  • Update .github/workflows/tests.yml to rename the unit test job label to Unit tests and install the SHA256-verified ucp-schema-linux-x86_64 binary from Universal-Commerce-Protocol/ucp-schema release v2.0.0.
  • Regenerate src/ucp_sdk/models/schemas/_generated.py and src/ucp_sdk/models/schemas/__init__.py against 2026-08-25, port pure structural and LocationFilter AwareDatetime checks to tests/test_models.py, and add DTO openness tests for ErrorResponse, RequestConstraints, and ValueConstraint.
  • Update README.md to document Tier 1 structural typing vs pre-deserialization payload verification via ucp-schema validate.

@damaz91 damaz91 added the status:needs-triage Signal that the PR is ready for human triage label Oct 28, 2026
Base automatically changed from feat/ucp-schema-generate-types to main October 28, 2026 19:15
@damaz91 damaz91 added status:under-review and removed status:needs-triage Signal that the PR is ready for human triage labels Oct 29, 2026
@damaz91
damaz91 force-pushed the feat/pure-ootb-pydantic-dtos branch from bc881f4 to 24f0fb4 Compare October 29, 2026 08:35

@damaz91 damaz91 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together! Removing the 2,900+ line AST/regex post-processor (postprocess_models.py) in favor of pure out-of-the-box Pydantic v2 DTOs aligns cleanly with Universal-Commerce-Protocol/ucp#817, and the JsonrpcRequest / JsonrpcSuccessResponse flattening from Universal-Commerce-Protocol/ucp-schema#97 also fixes the previously widened result / method fields on McpToolCall*, A2aMessage*, and EmbeddedMessage* to be properly required.

Everything is green after the rebase except Generated models match the pinned UCP spec (model-drift), which is waiting on Universal-Commerce-Protocol/ucp-schema#96 and Universal-Commerce-Protocol/ucp-schema#97 to merge and v2.0.0 to be tagged/published. Left a few inline comments and minor suggestions below.

(cd "$tmp_dir" && sha256sum -c --ignore-missing SHA256SUMS)
mkdir -p "$HOME/.local/bin"
install -m 0755 "$tmp_dir/ucp-schema-linux-x86_64" "$HOME/.local/bin/ucp-schema"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two notes here:

  1. Upstream release dependency: model-drift currently fails with HTTP 404 because Universal-Commerce-Protocol/ucp-schema v2.0.0 hasn't been published yet (waiting on Universal-Commerce-Protocol/ucp-schema#96 and Universal-Commerce-Protocol/ucp-schema#97 to merge and v2.0.0 to be tagged).
  2. Workflow script hygiene (nit): To keep .github/workflows/*.yml declarative with single-line run: steps, consider moving the ucp-schema release binary download + SHA256SUMS verification into a dedicated script (or generate_models.sh), and removing the duplicate Normalize file endings inline Python step below (lines 78–88), since generate_models.sh already normalizes file endings on src/ucp_sdk/models.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 61b6301: removed the duplicate Normalize file endings step from tests.yml, and ucp-schema v2.0.0 is now published so model-drift runs live. Kept the SHA256-verified binary install step in CI so local generate_models.sh runs never mutate a developer's $HOME/.local/bin.

Comment thread generate_models.sh
imports = ", ".join(exports)
all_items = ", ".join(f'"{n}"' for n in exports)
(schemas_dir / "__init__.py").write_text(
'# generated by generate_models.sh\n\n'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Including the Apache 2.0 license header at the top of generated src/ucp_sdk/models/schemas/__init__.py (as _write_schemas_reexport_init previously did) would keep license header consistency across src/ucp_sdk/.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 61b6301: generate_models.sh now emits the Apache 2.0 header at the top of generated src/ucp_sdk/models/schemas/__init__.py.

Comment thread tests/test_models.py
# must NOT silently fall back to FulfillmentDestinationBase.
with self.assertRaises(ValidationError):
self.destination_adapter.validate_python(
ShippingDestination.model_validate(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Observation on open discriminated union fallback behavior: Because ucp-schema generate-types omits the OpenAPI discriminator keyword (OU-3) and expresses known-tag exclusion on <Union>Base via JSON Schema not.enum (OU-2), datamodel-code-generator emits an untagged Python union (VariantA | VariantB | UnionBase) where UnionBase.type is a plain str.

Without _enforce_not_enum, validating a payload with a known discriminator ("type": "shipping_address" or "type": "oauth2") that fails a variant-specific field constraint through the union (TypeAdapter(FulfillmentDestination) / TypeAdapter(Provider) or a parent DTO like Checkout) will now fall back to FulfillmentDestinationBase / ProviderBase instead of raising ValidationError. While delegating not.enum enforcement to Tier 2 (ucp-schema validate) matches Universal-Commerce-Protocol/ucp#817, it may be worth calling out this open-union fallback caveat in the README.md Validation section so callers know why pre-validation with ucp-schema validate matters for union variants.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! Added a note in README.md (### Validation) in 61b6301 calling out the open discriminated union *Base fallback behavior and why pre-validation against the compiled JSON Schema matters for union variants.

@gsmith85
gsmith85 force-pushed the feat/pure-ootb-pydantic-dtos branch from 24f0fb4 to 69f84d0 Compare November 2, 2026 04:17

@damaz91 damaz91 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Verified locally that regenerating against 2026-08-25 with Universal-Commerce-Protocol/ucp-schema#97 produces zero model drift, and all unit tests pass. Left two non-blocking notes on the schemas/__init__.py license header and README.md open-union fallback behavior.

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# generated by generate_models.sh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (non-blocking): Please retain the Copyright 2026 UCP Authors Apache 2.0 license header at the top of generated src/ucp_sdk/models/schemas/__init__.py (via generate_models.sh) to stay consistent with the rest of src/ucp_sdk/.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 61b6301: restored the Copyright 2026 UCP Authors Apache 2.0 header via generate_models.sh.

Comment thread README.md Outdated
`dependentRequired`, `if`/`then`, `uniqueItems`, `contains`, `propertyNames`,
`not.enum`, and closed-schema rules), verify received JSON payloads with
[`ucp-schema validate`](https://crates.io/crates/ucp-schema) prior to
deserializing into SDK DTOs:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (non-blocking): Worth adding a brief note here that open discriminated unions (e.g., FulfillmentDestination, FulfillmentMethod, Provider) are emitted as untagged unions with an open *Base fallback (type: str), so a payload with a known type that fails a variant-specific field check will deserialize into *Base unless validated upfront with ucp-schema validate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 61b6301 right under ### Validation in README.md.

@gsmith85
gsmith85 requested a review from jingyli November 5, 2026 02:51
@vishkaty

Copy link
Copy Markdown
Contributor

Small note on the new in-memory validation section, from running both of its examples against
a bundle generated by ucp-schema generate-types.

The js-sdk twin of this section, in js-sdk#95 at f0a60d63, calls addFormats(ajv), so it
asserts format. The Python example here constructs the validator without a format checker,
so it does not. Measured on jsonschema 4.26.0 with a payload carrying continue_url of
not a url and expires_at of 2026-04-08, both of which the bundle marks format: uri
and format: date-time:

validator as constructed errors
Draft202012Validator(schema), as the example reads 0
plus format_checker=FormatChecker() and the format-nongpl extra 2, naming both
control, expires_at as the number 5 1 in both forms

A bare FormatChecker() is not enough. Without the extra installed it registers only
date, email, idn-email, ipv4, ipv6, regex, time, uuid, so uri and date-time stay
unasserted and the measured result is still 0 errors. The extra has to be named.

The extra to name is format-nongpl. jsonschema[format] pulls rfc3987, which is GPLv3+;
jsonschema[format-nongpl] uses rfc3986-validator and rfc3339-validator, both MIT, and
registers uri and date-time.

So the suggestion is:

from jsonschema import Draft202012Validator, FormatChecker

checkout_validator = Draft202012Validator(
    {**bundle["$defs"]["Checkout"], "$defs": bundle["$defs"]},
    format_checker=FormatChecker(),
)

with pip install "jsonschema[format-nongpl]" alongside it.

Worth saying how little this matters in the example as written: the next line is
Checkout.model_validate(raw_payload), and the DTO refuses both values on its own
(url_parsing and timezone_aware on the #103 head), so a reader who follows the snippet
end to end is covered. It matters on the validator only path the same section names, an OpenAPI
ASGI middleware validating inbound payloads before anything builds a DTO, where the js and
Python paths otherwise behave differently on the same bundle.

@damaz91

damaz91 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Thanks for testing this and sharing the breakdown, @vishkaty — great catch!

Agreed on updating the README.md snippet to recommend pip install "jsonschema[format-nongpl]" and enable format checking so the Python example matches Universal-Commerce-Protocol/js-sdk#95 (addFormats(ajv)) and avoids the GPLv3+ rfc3987 dependency. That also covers formats like duration (which generates as a plain str in Pydantic) and temporal fields where ucp-schema generate-types strips sibling pattern regexes during bundle normalization.

Using Draft202012Validator.FORMAT_CHECKER keeps the Draft 2020-12 format set scoped without an extra import:

checkout_validator = Draft202012Validator(
    {**bundle["$defs"]["Checkout"], "$defs": bundle["$defs"]},
    format_checker=Draft202012Validator.FORMAT_CHECKER,
)

@gsmith85 mind folding pip install "jsonschema[format-nongpl]" and format_checker=Draft202012Validator.FORMAT_CHECKER into the README.md snippet?

@gsmith85

Copy link
Copy Markdown
Contributor Author

Thanks @vishkaty and @damaz91! Updated README.md in 61b6301 to recommend pip install "jsonschema[format-nongpl]" (MIT format validators without GPL dependencies) and pass format_checker=Draft202012Validator.FORMAT_CHECKER to Draft202012Validator.

@gsmith85
gsmith85 merged commit 91abfda into main Nov 18, 2026
20 checks passed
@gsmith85
gsmith85 deleted the feat/pure-ootb-pydantic-dtos branch November 18, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants