Skip to content

Send context signals in routing predict request#4377

Open
aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna:aashnagarg/routing-context-signals
Open

Send context signals in routing predict request#4377
aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna:aashnagarg/routing-context-signals

Conversation

@aashna
Copy link
Contributor

@aashna aashna commented Mar 12, 2026

Add RoutingContextSignals interface with optional fields (has_image, chat_location, turn_number, session_id, previous_model, reference_count, prompt_char_count) and spread them into the router request body.

AutomodeService populates signals from the ChatRequest context before each routing call.

Copilot AI review requested due to automatic review settings March 12, 2026 19:07
@aashna aashna force-pushed the aashnagarg/routing-context-signals branch from 4d042c1 to 88cd833 Compare March 12, 2026 19:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the auto-mode model routing request payload with additional context signals derived from the current ChatRequest, enabling the router service to make more informed decisions.

Changes:

  • Added a RoutingContextSignals interface and included its fields in the router prediction request body.
  • Populated routing context signals in AutomodeService (location, session id, references, prompt length, previous model, turn number).
  • Added a helper to serialize ChatLocation to a string value for routing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
src/platform/endpoint/node/routerDecisionFetcher.ts Adds RoutingContextSignals and spreads them into the router request body; extends getRouterDecision signature.
src/platform/endpoint/node/automodeService.ts Builds and passes RoutingContextSignals from ChatRequest data; adds chatRequestLocationToString helper.

@aashna aashna force-pushed the aashnagarg/routing-context-signals branch from 88cd833 to f31446c Compare March 12, 2026 19:14
@aashna aashna force-pushed the aashnagarg/routing-context-signals branch 2 times, most recently from a69c43c to e48fd0d Compare March 12, 2026 19:35
@aashna aashna requested a review from Copilot March 12, 2026 19:43
@aashna aashna force-pushed the aashnagarg/routing-context-signals branch from e48fd0d to 6a594be Compare March 12, 2026 19:48
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/platform/endpoint/node/automodeService.ts:300

  • The cache’s turnCount is incremented unconditionally on every call, even when routing is skipped due to unchanged prompt or router-disabled cases. If turnCount is meant to represent user turns for routing signals, it should be updated only when advancing to a new turn; otherwise it will drift over time (and affect future turn_number values).
		this._autoModelCache.set(conversationId, { endpoint: autoEndpoint, tokenBank, lastSessionToken: token.session_token, lastRoutedPrompt, turnCount: (entry?.turnCount ?? 0) + 1 });
		return autoEndpoint;

Add RoutingContextSignals interface with optional fields (has_image,
chat_location, turn_number, session_id, previous_model, reference_count,
prompt_char_count) and spread them into the router request body.

AutomodeService populates signals from the ChatRequest context before
each routing call.
@aashna aashna force-pushed the aashnagarg/routing-context-signals branch from 6a594be to 382d84c Compare March 12, 2026 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants