Repository navigation
ci: run the Envoy ext_proc path against a real Envoy, in every body mode - #113
Open
CharmingGroot wants to merge 3 commits into
Open
CharmingGroot wants to merge 3 commits into
CharmingGroot wants to merge 3 commits into
Conversation
The ext_proc front-end was verified against Envoy 1.37 by hand. Those runs are now a harness (tests/e2e/envoy/run.sh) and a CI job: a real Envoy between curl and a stdlib mock OpenAI provider, pasu-proxy judging with the production ruleset (deploy/rules.yaml), 13 cases each asserting PASS/FAIL: passthrough, text and streamed text byte-equal to what the provider sent, identity upstream for a gzip client, shell tool calls (streamed or not) refused 403 with no chunk delivered, /v1/embeddings and a Korean RRN refused before the provider, redaction with a matching content-length, response_body_mode NONE refused with 500, and a stopped pasu answered 500. The mock records every request before replying, so "never reached the provider" is asserted rather than assumed. Readiness waits for pasu's gRPC port and for Envoy's admin to report LIVE with the listener bound; a trap removes the container and processes. On Linux Envoy runs with --network host; elsewhere it reaches the host through host.docker.internal. Envoy is pinned to v1.37.6 rather than v1.37-latest, so a red job means pasu changed, not Envoy. With misconfigured() forced to return None the NONE case fails with the shell call delivered as a 200. Signed-off-by: Ho Geun Choi <ohyes9711@gmail.com>
AGENTS.md said CI was four jobs; it is six in ci.yml (MSRV was missing, and the Envoy E2E is new), with pii.yml alongside. It now says how to run the Envoy harness locally. integrations/README.md notes that its Envoy claims are checked by that harness; CHANGELOG records it. Signed-off-by: Ho Geun Choi <ohyes9711@gmail.com>
The harness only ran BUFFERED, and its mock wrote small streams in one go and recorded a request only once it had read the whole body. Under those conditions disabling pasu's piece clearing in STREAMED still passed: Envoy coalesced every body into one piece, and a prompt reset mid-body left no record at the provider. The mock now paces its SSE events and records a request cut off mid-body; a small client sends a prompt in two pieces with the resident number in the first. With clearing disabled the run now fails three ways: a streamed text completion duplicated, a streamed shell tool call delivered with a 200, and the number reaching the provider. The misconfiguration case checks the new pasu_misconfigured type. 25 cases, all passing on Envoy 1.38. Signed-off-by: Ho Geun Choi <ohyes9711@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #112.
The Envoy behaviour in #110 and #112 was checked by hand. This PR asserts it on every push against a real Envoy, using the production ruleset (
deploy/rules.yaml, deliberately not overridable).tests/e2e/envoy/run.sh <pasu-proxy> <envoy-image>builds nothing./ready= LIVE, then the listener.--network hoston Linux,host.docker.internalelsewhere.mock_provider.pyis a stdlib OpenAI-compatible provider.slow_post.pysends a prompt as a chunked request in two pieces, with the resident number in the first.envoy.yaml.tmplrenders the request/response body and trailer modes. Its filter block matchesintegrations/envoy/envoy.yaml.25 cases
BUFFEREDidentityupstream · shell tool call (streamed or not) 403 with no chunk ·/v1/embeddings403, never at the providerBUFFERED+--block-pii-kr--redactdelivers[REDACTED:ko-rrn]with a matchingcontent-lengthresponse_body_mode: NONEpasu_misconfigurednaming the settingfailure_mode_allow: false)STREAMED/FULL_DUPLEX_STREAMEDMutation-checked on Envoy 1.38:
misconfigured()returningNone, theNONEcase fails: the shell call gets a 200.Before the mock paced its events and recorded truncated requests, that second mutant passed: Envoy coalesced every body into one piece.
CI job
Envoy ext_proc E2E (linux)pinsenvoyproxy/envoy:v1.37.6. A red job should mean pasu changed, not Envoy.AGENTS.mdnow lists the jobs as they actually are.Run locally on Envoy 1.38 (macOS, bridge network): 25 passed. Not run locally: the Linux
--network hostpath and the 1.37.6 image. This PR's CI run is the first check of both.🤖 Generated with Claude Code