Send context signals in routing predict request#4377
Open
aashna wants to merge 1 commit intomicrosoft:mainfrom
Open
Send context signals in routing predict request#4377aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna wants to merge 1 commit intomicrosoft:mainfrom
Conversation
4d042c1 to
88cd833
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the auto-mode model routing request payload with additional context signals derived from the current ChatRequest, enabling the router service to make more informed decisions.
Changes:
- Added a
RoutingContextSignalsinterface and included its fields in the router prediction request body. - Populated routing context signals in
AutomodeService(location, session id, references, prompt length, previous model, turn number). - Added a helper to serialize
ChatLocationto a string value for routing.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| src/platform/endpoint/node/routerDecisionFetcher.ts | Adds RoutingContextSignals and spreads them into the router request body; extends getRouterDecision signature. |
| src/platform/endpoint/node/automodeService.ts | Builds and passes RoutingContextSignals from ChatRequest data; adds chatRequestLocationToString helper. |
88cd833 to
f31446c
Compare
a69c43c to
e48fd0d
Compare
e48fd0d to
6a594be
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/platform/endpoint/node/automodeService.ts:300
- The cache’s
turnCountis incremented unconditionally on every call, even when routing is skipped due to unchanged prompt or router-disabled cases. IfturnCountis meant to represent user turns for routing signals, it should be updated only when advancing to a new turn; otherwise it will drift over time (and affect futureturn_numbervalues).
this._autoModelCache.set(conversationId, { endpoint: autoEndpoint, tokenBank, lastSessionToken: token.session_token, lastRoutedPrompt, turnCount: (entry?.turnCount ?? 0) + 1 });
return autoEndpoint;
Add RoutingContextSignals interface with optional fields (has_image, chat_location, turn_number, session_id, previous_model, reference_count, prompt_char_count) and spread them into the router request body. AutomodeService populates signals from the ChatRequest context before each routing call.
6a594be to
382d84c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add RoutingContextSignals interface with optional fields (has_image, chat_location, turn_number, session_id, previous_model, reference_count, prompt_char_count) and spread them into the router request body.
AutomodeService populates signals from the ChatRequest context before each routing call.