Skip to content

fix(runner): log bodies with --verbose_http, show the last exchange on failure, count per test - #140

Open
segiodongo wants to merge 1 commit into
Universal-Commerce-Protocol:mainfrom
segiodongo:upstream/u4-http-logging-and-counts
Open

segiodongo wants to merge 1 commit into
Universal-Commerce-Protocol:mainfrom
segiodongo:upstream/u4-http-logging-and-counts

Conversation

@segiodongo

Copy link
Copy Markdown
Collaborator

Description

Improves HTTP diagnostics and console summary accuracy in the conformance runner:

  • Log request and response bodies under --verbose_http and attach the last exchange on failure: Previously, --verbose_http only raised the httpx logger to INFO (logging a single request/status line without bodies), and an unexpected response shape (for example, an in-band HTTP 200 refusal followed by a KeyError) showed none of the payload that triggered it.
    • BaseIntegrationTest now attaches httpx request and response event hooks. Under --verbose_http, they log each request (method, URL, identifying UCP headers, body) and response (status, Content-Type, body), pretty-printing JSON, clipping bodies at 4 KB, and redacting Authorization, Signature, and Signature-Input.
    • With or without --verbose_http, the response hook retains the test's last response, and RecordingResult appends "Last HTTP exchange:" to every failure, error, and failing subtest output.
  • Count console summary outcomes per test rather than per subtest: unittest's result.failures list contains one entry per failing subTest, so a single test method failing two subtests previously incremented Failures twice and subtracted from Passed. runner.py now runs under RecordingResult (framework/outcomes.py) and counts one consolidated outcome per test ID.
  • Documentation & unit tests: Documents --verbose_http and automatic failure-exchange output in README.md, adds tests/http_and_summary_test.py, and wires uv run pytest tests into CI.

Verification

  • uv run pytest tests: 8 passed
  • uv run ucp-conformance --suite=all against the Flower Shop sample server: 83 executed, 81 passed, 2 skipped, 0 failures, 0 errors

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@damaz91 damaz91 added the status:needs-triage Signal that the PR is ready for human triage label Nov 18, 2026
…n failure, count per test

Attach httpx request and response hooks to BaseIntegrationTest so --verbose_http logs identifying headers and pretty-printed bodies (clipped at 4 KB, credentials redacted) and failures/errors append the test's last HTTP exchange via RecordingResult. Count console summary outcomes once per test ID rather than once per failing subtest, document --verbose_http in README.md, and add unit tests.
@segiodongo
segiodongo force-pushed the upstream/u4-http-logging-and-counts branch from 3850ea6 to f120adf Compare November 18, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status:needs-triage Signal that the PR is ready for human triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants