Skip to content

Add BBR end-to-end tests with multi-pool routing verification#2511

Open
asaadbalum wants to merge 1 commit intokubernetes-sigs:mainfrom
asaadbalum:issue-2493-bbr-e2e-tests
Open

Add BBR end-to-end tests with multi-pool routing verification#2511
asaadbalum wants to merge 1 commit intokubernetes-sigs:mainfrom
asaadbalum:issue-2493-bbr-e2e-tests

Conversation

@asaadbalum
Copy link
Contributor

@asaadbalum asaadbalum commented Mar 7, 2026

What type of PR is this?
/kind test

What this PR does / why we need it:

Adds a comprehensive Ginkgo/Gomega E2E test suite for the Body Based Router (BBR) that validates its core production behavior: request body parsing, model name extraction, base model lookup via ConfigMaps, header mutation (X-Gateway-Base-Model-Name with ClearRouteCache), and header-based routing to distinct backend clusters.

The test infrastructure deploys two model server simulators (Llama + DeepSeek) with LoRA adapters, an Envoy proxy with ext_proc filter routing to BBR, and header-based routing with x-bbr-routed-pool response headers for explicit pool verification.

Test cases:

  1. Base model routing — verifies base models route to the correct pool via x-bbr-routed-pool header, covering both /completions and /chat/completions APIs.
  2. LoRA adapter routing — verifies BBR resolves LoRA adapter names to the correct base model via ConfigMap lookup, routing to the correct pool.
  3. Streaming routing — verifies streaming requests route correctly and SSE data: chunks arrive at the client.
  4. BBR metrics — scrapes BBR's Prometheus endpoint and verifies bbr_success_total and bbr_info metrics.

Run with make test-e2e-bbr. Follows the same pattern as the existing EPP E2E tests.

Which issue(s) this PR fixes:

Relates to #2493

Does this PR introduce a user-facing change?:

NONE

Note

Reviewer note — hack/test-e2e.sh unification

This PR modifies the existing hack/test-e2e.sh rather than adding a separate hack/test-e2e-bbr.sh script. The two scripts would have been ~95% identical (Kind setup, cluster detection, image loading), differing only in the image variable, make target, and test path.

The unified script uses a COMPONENT environment variable (epp by default, set by the Makefile targets), so make test-e2e continues to work identically with zero behavioral change for EPP.

Note: the existing test-e2e Makefile target still addresses EPP (not renamed to test-e2e-epp), since renaming it would affect the EPP README, conformance docs, and developer workflows — out of scope for this PR.

If reviewers prefer keeping separate scripts or renaming the targets, happy to adjust.

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 7, 2026
@k8s-ci-robot k8s-ci-robot requested review from liu-cong and shmuelk March 7, 2026 09:25
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: asaadbalum
Once this PR has been reviewed and has the lgtm label, please assign danehans for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Mar 7, 2026
@k8s-ci-robot
Copy link
Contributor

Hi @asaadbalum. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Mar 7, 2026
@netlify
Copy link

netlify bot commented Mar 7, 2026

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit bcf34c8
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/69ad71faf89de70008737812
😎 Deploy Preview https://deploy-preview-2511--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@asaadbalum asaadbalum marked this pull request as ready for review March 8, 2026 08:50
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 8, 2026
@k8s-ci-robot k8s-ci-robot requested review from elevran and kfswain March 8, 2026 08:50
Create a comprehensive Ginkgo/Gomega E2E test suite for the Body Based
Router (BBR) that validates BBR's core production behavior: request body
parsing, model name extraction, base model lookup via ConfigMaps, header
mutation (X-Gateway-Base-Model-Name with ClearRouteCache), and
header-based routing to distinct backend clusters.

Test infrastructure:
- Two model server simulators (Llama + DeepSeek) with LoRA adapters
- Envoy proxy with ext_proc filter routing to BBR
- Header-based routing with x-bbr-routed-pool response headers for
  explicit verification of which cluster served each request
- Makefile target (test-e2e-bbr) via unified hack/test-e2e.sh script
  parameterized by COMPONENT env var (epp|bbr)

Test cases:
- Base model routing to correct pools (both /completions and
  /chat/completions API paths)
- LoRA adapter routing via base model lookup (food-review-1 -> llama,
  ski-resorts + movie-critique -> deepseek)
- Streaming requests with SSE output and correct pool routing
- BBR Prometheus metrics exposure (bbr_success_total, bbr_info)

Signed-off-by: asaadbalum <asaad.balum@gmail.com>
@asaadbalum asaadbalum force-pushed the issue-2493-bbr-e2e-tests branch from 886fbb7 to bcf34c8 Compare March 8, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants