Add BBR end-to-end tests with multi-pool routing verification#2511
Add BBR end-to-end tests with multi-pool routing verification#2511asaadbalum wants to merge 1 commit intokubernetes-sigs:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: asaadbalum The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @asaadbalum. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
✅ Deploy Preview for gateway-api-inference-extension ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Create a comprehensive Ginkgo/Gomega E2E test suite for the Body Based Router (BBR) that validates BBR's core production behavior: request body parsing, model name extraction, base model lookup via ConfigMaps, header mutation (X-Gateway-Base-Model-Name with ClearRouteCache), and header-based routing to distinct backend clusters. Test infrastructure: - Two model server simulators (Llama + DeepSeek) with LoRA adapters - Envoy proxy with ext_proc filter routing to BBR - Header-based routing with x-bbr-routed-pool response headers for explicit verification of which cluster served each request - Makefile target (test-e2e-bbr) via unified hack/test-e2e.sh script parameterized by COMPONENT env var (epp|bbr) Test cases: - Base model routing to correct pools (both /completions and /chat/completions API paths) - LoRA adapter routing via base model lookup (food-review-1 -> llama, ski-resorts + movie-critique -> deepseek) - Streaming requests with SSE output and correct pool routing - BBR Prometheus metrics exposure (bbr_success_total, bbr_info) Signed-off-by: asaadbalum <asaad.balum@gmail.com>
886fbb7 to
bcf34c8
Compare
What type of PR is this?
/kind test
What this PR does / why we need it:
Adds a comprehensive Ginkgo/Gomega E2E test suite for the Body Based Router (BBR) that validates its core production behavior: request body parsing, model name extraction, base model lookup via ConfigMaps, header mutation (
X-Gateway-Base-Model-NamewithClearRouteCache), and header-based routing to distinct backend clusters.The test infrastructure deploys two model server simulators (Llama + DeepSeek) with LoRA adapters, an Envoy proxy with
ext_procfilter routing to BBR, and header-based routing withx-bbr-routed-poolresponse headers for explicit pool verification.Test cases:
x-bbr-routed-poolheader, covering both/completionsand/chat/completionsAPIs.data:chunks arrive at the client.bbr_success_totalandbbr_infometrics.Run with
make test-e2e-bbr. Follows the same pattern as the existing EPP E2E tests.Which issue(s) this PR fixes:
Relates to #2493
Does this PR introduce a user-facing change?:
Note
Reviewer note —
hack/test-e2e.shunificationThis PR modifies the existing
hack/test-e2e.shrather than adding a separatehack/test-e2e-bbr.shscript. The two scripts would have been ~95% identical (Kind setup, cluster detection, image loading), differing only in the image variable, make target, and test path.The unified script uses a
COMPONENTenvironment variable (eppby default, set by the Makefile targets), somake test-e2econtinues to work identically with zero behavioral change for EPP.Note: the existing
test-e2eMakefile target still addresses EPP (not renamed totest-e2e-epp), since renaming it would affect the EPP README, conformance docs, and developer workflows — out of scope for this PR.If reviewers prefer keeping separate scripts or renaming the targets, happy to adjust.