Repository navigation
Conversation
|
/ug-review |
|
Local validation completed: 298 combined integration-contract and CUJ-evidence cases passed. Ruff check/format passed for tests and scripts/run_integration.py. The new negative-control case still rejects HTTP 400 and confirms prompt, credential/header, response-body and extra-field contents do not enter the routing artifact. Live 422 reproduction and artifact delivery remain for PR CI. This comment was generated with GitHub MCP. |
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s). |
UG review recommendationRECOMMENDATION: APPROVE Reasons:
Complete review of 2 files; no findings. Advisory only: this recommendation never approves or merges the pull request. The |
Polly AI ReviewBlocking issuesNone found. Security vulnerabilitiesNone found. The added artifact uses explicit field allowlisting and the existing redacted writer; prompts, arbitrary headers and response bodies are excluded. Non-blocking notes
ApproachThe approach is sound: reuse the existing request/response recorder and redacted artifact writer without changing routing, retries, success assertions or timeouts. No new abstraction is needed. SummaryDiff size (computed): 3 files; +55 / -1 lines (56 changed text lines); 0 binary files.
Shares use additions + deletions, including generated text and lockfiles. Binary files have no line count. Tests include fixtures/helpers under test paths and colocated test/spec files; categories are path-based, not coverage measurements. ChangesRetains routing metadata before saved-provider response validation, adds a focused offline regression case, and documents the artifact. The changes are proportional to the diagnostic goal. No concrete code consolidation is needed; the documentation qualification above would align its promise with the capture lifecycle. TestsTest-by-test assessment
Existing happy-path journeys and the new failure case cover the scoped change without another live test. Live reproduction, actual gateway request-ID correlation and uploaded failure-artifact delivery remain unverified. The focused offline test could not reach collection because ScopeAll changes support retaining saved-provider routing evidence. No unrelated changes were found. This adds diagnostics, not a fix for the backend routing rejection. Automated review by Polly · workflow run |
|
Live CUJ7 validation passed all 10 tests on c86b046. I downloaded artifact14636088792 and verified both saved-provider Claude and Codex journeys delivered routing-request JSON containing only router_name, model/harness pairs, status_code and a gateway request_id; both responses were HTTP200. The formerly failing Codex case selected gpt-6-luna through recipe m2-r315-quality-20260929. Failure-path artifact preservation remains covered by the offline negative control. Broad CI currently has an unrelated Windows Codex desktop Get-AppxPackage timeout before inference; it will be rerun after the workflow finishes. This comment was generated with GitHub MCP. |
|
Addressed Polly's documentation note in 32ab7bb: the CUJ guide now states that capture occurs after task completion, as responses reach validation, and that earlier failures can prevent later artifacts. No executable code changed after the reviewed c86b046 head. That head passed all 10 live CUJ7 cases and delivered the verified metadata artifact; the documentation push starts a fresh CI run, including the previously timed-out Windows package query. This comment was generated with GitHub MCP. |
|
Final-head CI completed with five direct failures, all confirmed upstream HTTP429 rate limits: Anthropic relay Cowork on macOS/Windows, Claude relay full journeys on Linux/Windows, and native Codex managed routing. The CUJ7 routing change is unchanged from its live-validated executable head. I requested a failed-only rerun of run44755740715 on the same 32ab7bb commit; passing jobs are preserved. No tests, timeouts, or success conditions were weakened. This comment was generated with GitHub MCP. |
What did you change, and why?
CUJ7's saved-provider Codex routing test failed with a gateway 400 wrapping
Smart Router returned HTTP 422(job, request ID85cf5768-2dbd-4bc6-946d-64da672a3514), while the same test passed on the previous PR head and recent main. Its request recorder held the router inputs only in memory, so the uploaded artifacts could not establish which recipe and offered models produced the rejection.Before checking each matched routing response, retain its router name, offered model/harness pairs, status and gateway request ID through the existing redacted artifact writer. Exclude prompts, extra option/selector fields, arbitrary headers and response bodies. Preserve every success assertion and all timeouts; this captures evidence for backend correlation rather than retrying or concealing the rejection. Both existing Claude and Codex saved-provider journeys share the check.
How do you know it works?
tests/README.mdis unchanged.Checklist before requesting review
/ug-reviewand address the comments. Completed UG review returned APPROVE with zero findings (run 44753980691). Polly found no blockers/security issues; its documentation note was addressed in 32ab7bb and answered in the discussion./integration-testwill not be posted, as instructed.