diff --git a/docs-site/src/content/docs/fr/reference/configuration/advisor.md b/docs-site/src/content/docs/fr/reference/configuration/advisor.md index 206cbde2454..de932d06d6b 100644 --- a/docs-site/src/content/docs/fr/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/fr/reference/configuration/advisor.md @@ -87,6 +87,8 @@ OpenCodex n'insère pas dans ce prompt de clés d'API de fournisseur, d'en-tête Le conseil manuel est le résultat d'outil de l'appel `advisor` que le worker a lui-même émis. Ce résultat est un objet JSON. Le champ `advice` est le texte du modèle conseiller. Le champ `status` est écrit par le runtime. +Lors des tours de recherche web basés sur fetch, la boucle de recherche n'expose l'outil synthétique `advisor` et n'intercepte ses appels de consultation manuelle que si le `tool_choice` effectif l'autorise. Un choix `allowed_tools` limité à `web_search` exclut les consultations manuelles du conseiller. Activer la recherche hébergée n'élargit pas cette autorisation. + Le conseil automatique est un objet JSON cité dans un message consultatif de rôle user distinct. Seule l’instruction fixe du runtime reste dans le message developer ; le texte généré par le conseiller ne passe jamais dans developer/system. Cela fonctionne avec OpenAI Chat et Anthropic sans inventer un appel d’outil. Les guillemets empêchent la rupture structurelle et les champs falsifiés, pas toute injection en langage naturel. Un protocole dédié pourrait mieux distinguer le conseil d’une demande utilisateur. Chaque requête permet au plus trois consultations et quatre continuations du worker. À la limite, l’outil advisor est retiré ; un nouvel appel reçoit une dernière continuation avec un résultat de limite, puis un autre appel termine avec 502 advisor_continuation_limit sans nouvel envoi. La limite est partagée avec les reprises de complétion vide. La suppression ne lit pas les chaînes du conseiller. Le dédoublonnage automatique appartient au registre du serveur. Un message developer, même s'il recopie le texte de transport, ne supprime pas le preflight. diff --git a/docs-site/src/content/docs/ja/reference/configuration/advisor.md b/docs-site/src/content/docs/ja/reference/configuration/advisor.md index 4a5615b8c10..59b4614e1ed 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/ja/reference/configuration/advisor.md @@ -70,6 +70,8 @@ OpenCodex はプロバイダー API キー、Authorization ヘッダー、OAuth 手動の助言は、ワーカー自身が行った `advisor` 呼び出しに対するツール結果です。結果は JSON オブジェクトです。`advice` はアドバイザーモデルのテキストです。`status` はランタイムが書きます。 +fetch ベースの Web 検索ターンでは、実際に適用される `tool_choice` が合成 `advisor` ツールを許可する場合に限り、検索ループがそのツールを公開し、手動相談の呼び出しを処理します。`allowed_tools` を `web_search` だけに制限すると、アドバイザーへの手動相談は許可されません。ホスト型検索を有効にしても、このツールの許可範囲は広がりません。 + 自動助言の引用済み JSON は、別の user ロールの助言メッセージで渡します。developer メッセージには固定のランタイム指示だけを残し、アドバイザーの生成文は developer/system に入りません。OpenAI Chat と Anthropic の両方で、偽のツール呼び出しなしに送れます。引用は構造の破壊やフィールドの偽造を防ぎますが、自然言語によるプロンプトインジェクションを完全には防ぎません。専用プロトコルなら通常のユーザー入力と区別しやすくなります。1 リクエストにつき相談は最大 3 回、ワーカー継続は最大 4 回です。上限で advisor ツールを除去し、再呼び出しには上限結果付きの最終継続を 1 回だけ許可します。さらに呼ばれた場合は 502 advisor_continuation_limit で終了し、追加送信しません。空の完了の再試行も同じ上限を共有します。 抑制はアドバイザーの文字列を読みません。自動の重複排除はサーバー所有の台帳だけです。転送文を写した developer メッセージも preflight を抑制しません。 diff --git a/docs-site/src/content/docs/ko/reference/configuration/advisor.md b/docs-site/src/content/docs/ko/reference/configuration/advisor.md index c49656a19fb..dd7c22c8114 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/ko/reference/configuration/advisor.md @@ -70,6 +70,8 @@ OpenCodex는 프로바이더 API 키, Authorization 헤더, OAuth 토큰, 백엔 수동 조언은 워커가 직접 호출한 `advisor`에 대한 도구 결과입니다. 결과는 JSON 객체입니다. `advice`는 어드바이저 모델의 텍스트입니다. `status`는 런타임이 씁니다. +fetch 기반 웹 검색 턴에서는 실제 적용되는 `tool_choice`가 합성 `advisor` 도구를 허용할 때만 해당 도구를 노출하고 수동 상담 호출을 가로챕니다. `allowed_tools`가 `web_search`로 제한되어 있으면 어드바이저 수동 상담은 허용되지 않습니다. 호스팅 검색을 활성화해도 이 도구 허용 범위가 넓어지지는 않습니다. + 자동 조언의 인용된 JSON은 별도의 user 역할 자문 메시지로 전달합니다. developer 메시지에는 고정된 런타임 지시만 남고, Advisor가 생성한 텍스트는 developer/system에 들어가지 않습니다. OpenAI Chat과 Anthropic 모두 가짜 도구 호출 없이 이를 지원합니다. JSON 인용은 구조 탈출과 필드 위조를 막지만 자연어 프롬프트 주입을 완전히 차단하지는 않습니다. 전용 프로토콜은 일반 사용자 입력과 조언을 더 명확히 구분할 수 있습니다. 요청당 상담은 최대 3회, 워커 이어가기는 최대 4회입니다. 상담 한도에 도달하면 advisor 도구를 제거하고 반복 호출에 한도 결과를 전달하는 마지막 이어가기를 한 번만 허용합니다. 다시 호출하면 추가 전송 없이 502 advisor_continuation_limit로 끝납니다. 빈 완료 재시도도 같은 한도를 공유합니다. 억제는 어드바이저 문자열을 읽지 않습니다. 자동 중복 제거는 서버가 소유한 원장뿐입니다. 전송 문장을 복사한 developer 메시지도 preflight를 억제하지 않습니다. diff --git a/docs-site/src/content/docs/reference/configuration/advisor.md b/docs-site/src/content/docs/reference/configuration/advisor.md index aacd2ff6d7e..36c557abf5d 100644 --- a/docs-site/src/content/docs/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/reference/configuration/advisor.md @@ -157,10 +157,15 @@ After consultation exhaustion the `advisor` tool is removed. A repeated call rec paired limit result; if the worker calls it again, typed 502 `advisor_continuation_limit` ends the request without another hidden worker call. The bound is shared with empty-completion retries. -Fetch-based web-search turns support the same manual consultation path. The search loop intercepts -`web_search` and `advisor` calls in worker order, so enabling hosted search does not hide the -advisor tool. Search and Advisor iterations have separate allowances, and a real client tool call -still returns control to the client. Run-turn and image/video sidecar turns remain subject to their existing limitations. +Fetch-based web-search turns support the same manual consultation path when the effective +`tool_choice` permits the synthetic `advisor` tool. The search loop exposes and intercepts +`advisor` calls only within that scope; an `allowed_tools` choice restricted to `web_search` +excludes manual Advisor consultations. Permitted `web_search` and `advisor` calls are handled in +worker order. Search and Advisor iterations have separate allowances, +and a real client tool call still returns control to the client. Run-turn and image/video sidecar turns remain subject to their existing limitations. +If the worker emits an excluded `advisor` call, the turn fails with `undeclared_tool_call` +before that call reaches the client. A normal client tool named `advisor` remains available +only while the current iteration declares and permits it; a tools-disabled recovery cannot reuse an earlier declaration. Usage accounting includes each worker turn once, including the leg rejected by the Advisor continuation limit. Suppression does not read Advisor strings. Automatic dedup is the server-owned ledger. A diff --git a/docs-site/src/content/docs/ru/reference/configuration/advisor.md b/docs-site/src/content/docs/ru/reference/configuration/advisor.md index bc0f09bd583..30b9a518175 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/ru/reference/configuration/advisor.md @@ -70,6 +70,8 @@ OpenCodex не вставляет в этот запрос ключи API про Ручной совет — это результат инструмента для вызова `advisor`, который сделал сам воркер. Результат — объект JSON. Поле `advice` — текст модели консультанта. Поле `status` записывает runtime. +В шагах веб-поиска через fetch цикл поиска предоставляет синтетический инструмент `advisor` и перехватывает вызовы ручной консультации только тогда, когда это разрешает применяемый `tool_choice`. Если `allowed_tools` ограничен `web_search`, ручные консультации Advisor не разрешены. Включение встроенного поиска не расширяет эти разрешения. + Автоматический совет передаётся как экранированный JSON в отдельном консультативном сообщении роли user. В developer остаётся только фиксированная инструкция runtime; сгенерированный консультантом текст никогда не попадает в developer/system. OpenAI Chat и Anthropic поддерживают это без выдуманного вызова инструмента. Экранирование предотвращает структурный выход и подделку полей, но не гарантирует защиту от инструкций на естественном языке. Отдельный протокол мог бы лучше отличать совет от запроса пользователя. На запрос разрешено не более трёх консультаций и четырёх продолжений воркера. При исчерпании консультаций инструмент advisor удаляется; повторный вызов получает одно последнее продолжение с результатом о лимите. Ещё один вызов завершает запрос с 502 advisor_continuation_limit без новой отправки. Повторы пустого завершения используют тот же лимит. Подавление не читает строки консультанта. Автоматическая дедупликация принадлежит серверному журналу. Сообщение developer, даже скопировавшее текст транспорта, не подавляет preflight. diff --git a/docs-site/src/content/docs/tr/reference/configuration/advisor.md b/docs-site/src/content/docs/tr/reference/configuration/advisor.md index 4d20a4c9b6e..b371679afc1 100644 --- a/docs-site/src/content/docs/tr/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/tr/reference/configuration/advisor.md @@ -70,6 +70,8 @@ OpenCodex bu isteme sağlayıcı API anahtarlarını, Authorization başlıklar Elle danışma, worker'ın kendisinin yaptığı `advisor` çağrısının araç sonucudur. Sonuç bir JSON nesnesidir. `advice` alanı danışman modelinin metnidir. `status` alanını çalışma zamanı yazar. +fetch tabanlı web arama turlarında arama döngüsü, yalnızca geçerli `tool_choice` izin veriyorsa sentetik `advisor` aracını sunar ve elle danışma çağrılarını yakalar. `allowed_tools` seçimini `web_search` ile sınırlamak, elle danışmaya izin vermez. Barındırılan aramayı etkinleştirmek bu araç izinlerini genişletmez. + Otomatik tavsiyenin alıntılanmış JSON içeriği ayrı bir user rolü danışma mesajında taşınır. Developer mesajında yalnızca sabit çalışma zamanı yönergesi kalır; danışmanın ürettiği metin developer/system içeriğine girmez. OpenAI Chat ve Anthropic bunu sahte araç çağrısı olmadan destekler. JSON alıntılama yapısal kaçışı ve alan sahteciliğini önler; doğal dildeki saldırılara karşı kusursuz yalıtım sağlamaz. Özel bir protokol tavsiyeyi kullanıcı isteğinden daha açık ayırabilir. Her istekte en fazla üç danışma ve dört worker sürdürmesi vardır. Danışma sınırında advisor aracı kaldırılır; yinelenen çağrıya sınır sonucu ile yalnızca bir son sürdürme verilir. Sonraki çağrı yeni gönderim yapılmadan 502 advisor_continuation_limit ile biter. Boş tamamlama tekrarları da aynı sınırı paylaşır. Bastırma, danışmanın dizelerini okumaz. Otomatik yineleme ayıklama sunucunun defterine aittir. Taşıma metnini kopyalayan bir developer iletisi de preflight'ı bastırmaz. diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/advisor.md b/docs-site/src/content/docs/zh-cn/reference/configuration/advisor.md index 41f38dcafea..da488791a18 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/advisor.md @@ -70,6 +70,8 @@ OpenCodex 不会把 provider API key、Authorization 头、OAuth token、仅后 手动建议是 Worker 自己发出的 `advisor` 调用所对应的工具结果。结果是一个 JSON 对象。`advice` 是顾问模型的文本。`status` 由运行时写入。 +基于 fetch 的 Web 搜索轮次中，只有实际生效的 `tool_choice` 允许合成 `advisor` 工具时，搜索循环才会提供该工具并拦截手动咨询调用。将 `allowed_tools` 限制为 `web_search` 不允许手动咨询顾问。启用托管搜索不会扩大此工具权限范围。 + 自动建议的 JSON 内容放在独立的 user-role 建议消息中。developer 消息只保留固定的运行时传输说明，顾问生成的文本不会进入 developer/system 内容；OpenAI Chat 和 Anthropic 均可传递它，无需伪造工具调用。JSON 转义能防止结构突破和字段伪造，但不能保证模型忽略自然语言中的恶意指令。专门的咨询结果协议可进一步区分建议与用户请求。每个请求最多进行 3 次咨询和 4 次 Advisor worker 续写。咨询额度耗尽后移除 advisor 工具，重复调用只允许一次携带上限结果的最终续写；再次调用会以 502 advisor_continuation_limit 结束，不再发送隐藏的 worker 请求。空完成重试共享此上限。 抑制不读取顾问字符串。自动去重只看服务端账本。复制了传输文本的 developer 消息也不能抑制 preflight。 diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/advisor.md b/docs-site/src/content/docs/zh-tw/reference/configuration/advisor.md index 2bcaa37e209..917cb4b8bc6 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/advisor.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/advisor.md @@ -70,6 +70,8 @@ OpenCodex 不會把 provider API key、Authorization 標頭、OAuth token、僅 手動建議是 Worker 自己發出的 `advisor` 呼叫所對應的工具結果。結果是一個 JSON 物件。`advice` 是顧問模型的文字。`status` 由執行期寫入。 +基於 fetch 的 Web 搜尋回合中，只有實際生效的 `tool_choice` 允許合成 `advisor` 工具時，搜尋迴圈才會提供該工具並攔截手動諮詢呼叫。將 `allowed_tools` 限制為 `web_search` 不允許手動諮詢顧問。啟用託管搜尋不會擴大此工具權限範圍。 + 自動建議的 JSON 內容放在獨立的 user-role 建議訊息中。developer 訊息只保留固定的執行期傳輸說明，顧問產生的文字不會進入 developer/system 內容；OpenAI Chat 與 Anthropic 都能傳遞它，不需偽造工具呼叫。JSON 跳脫能防止結構突破與欄位偽造，但不能保證模型忽略自然語言中的惡意指令。專門的諮詢結果協定可進一步區分建議與使用者請求。每個請求最多進行 3 次諮詢與 4 次 Advisor worker 續寫。諮詢額度耗盡後移除 advisor 工具，重複呼叫只允許一次攜帶上限結果的最終續寫；再次呼叫會以 502 advisor_continuation_limit 結束，不再傳送隱藏的 worker 請求。空完成重試共用此上限。 抑制不讀取顧問字串。自動去重只看伺服器端帳本。複製了傳輸文字的 developer 訊息也不能抑制 preflight。 diff --git a/src/server/responses/sidecar-execution.ts b/src/server/responses/sidecar-execution.ts index cc1990bd336..4ae3f142c6c 100644 --- a/src/server/responses/sidecar-execution.ts +++ b/src/server/responses/sidecar-execution.ts @@ -48,7 +48,7 @@ import { } from "../../oauth/anthropic-routing"; import { resolveWireProtocolOverride } from "../adapter-resolve"; import { bindRouteReasoningReplayScope, adapterNeedsForcedContinuation, adapterNeedsToolCallContinuation } from "./core-replay"; -import { namespacedToolName } from "../../types"; +import { namespacedToolName, toolChoiceToolPredicate } from "../../types"; import { providerFetch } from "./fetch-helpers"; import type { AttemptRecoveryKind } from "../../usage/log"; import { recordAdapterReasoning, recordAdapterTier } from "../request-log"; @@ -657,12 +657,15 @@ export async function executeResponsesSidecars( // Placed BEFORE the runTurn early-return for non-runTurn adapters so dual-tool turns dispatch // through web-search instead of being swallowed. runTurn adapters never enter this branch. if (canRunWebSearch && wsPlan) { - const webSearchAdvisor = advisorPlan && !routedCompaction + const requestedTools = parsed.context.tools ?? []; + const advisorAllowed = advisorPlan + && toolChoiceToolPredicate(parsed.options.toolChoice, [...requestedTools, advisorPlan.tool])(advisorPlan.tool); + const webSearchAdvisor = advisorPlan && advisorAllowed && !routedCompaction && typeof advisorPlan.consult === "function" && typeof advisorPlan.formatUnavailable === "function" ? advisorPlan : undefined; parsed.context.tools = [ - ...(parsed.context.tools ?? []).filter(tool => !webSearchAdvisor || (!tool.advisor && tool.name !== ADVISOR_TOOL_NAME)), + ...requestedTools.filter(tool => !webSearchAdvisor || (!tool.advisor && tool.name !== ADVISOR_TOOL_NAME)), buildWebSearchTool(), ...(webSearchAdvisor ? [webSearchAdvisor.tool] : []), ]; diff --git a/src/web-search/loop.ts b/src/web-search/loop.ts index c53796453b0..19954b78f95 100644 --- a/src/web-search/loop.ts +++ b/src/web-search/loop.ts @@ -14,7 +14,7 @@ import { bridgeToResponsesSSE } from "../bridge"; import { guardEmptyToolCatalogStream, type ClientToolCatalogRestriction } from "../lib/empty-tool-catalog"; import { mergeUsage } from "../server/responses/empty-completion-guard"; import { markSidecarContextOverflow, sidecarLoopTerminalEvent, upstreamErrorCode, type SidecarLoopFailure } from "../server/responses/context-overflow"; -import { markVerifiedModelNotInPlanResponse, MODEL_NOT_IN_PLAN_CODE, verifiedPlanRefusalMessage } from "../lib/errors"; +import { markVerifiedModelNotInPlanResponse, MODEL_NOT_IN_PLAN_CODE, UNDECLARED_TOOL_CALL_ERROR_CODE, verifiedPlanRefusalMessage } from "../lib/errors"; import { runWebSearch, type SidecarOutcome, type SidecarOutcomeRecorder, type SidecarSettings } from "./executor"; import { runAnthropicWebSearch } from "./anthropic-executor"; import { runXaiWebSearch, type XaiSearchOptions } from "./xai-executor"; @@ -90,7 +90,7 @@ function parseQueries(argsBuf: string): string[] { * (Codex never sees the synthetic tool); every other event — text, thinking, real tool calls, done — * is preserved in order. */ -export function scanEventsForWebSearch(events: AdapterEvent[], options: { advisor?: boolean } = {}): { +export function scanEventsForWebSearch(events: AdapterEvent[], options: { advisor?: boolean; clientAdvisorAllowed?: boolean } = {}): { calls: WebSearchCall[]; passthrough: AdapterEvent[]; hasRealToolCall: boolean; @@ -100,6 +100,7 @@ export function scanEventsForWebSearch(events: AdapterEvent[], options: { adviso const passthrough: AdapterEvent[] = []; let hasRealToolCall = false; let hasMalformedToolCall = false; + let refusal: Extract | undefined; let pending: { name: string; id: string; argsBuf: string; closed: boolean; events: AdapterEvent[]; providerMetadata?: OcxProviderOpaqueToolCallMetadata } | null = null; const isBlank = (value: string): boolean => value.trim().length === 0; const flushPending = (): void => { @@ -113,8 +114,18 @@ export function scanEventsForWebSearch(events: AdapterEvent[], options: { adviso pending = null; }; for (const e of events) { + if (refusal) { + if ((e.type === "done" || e.type === "incomplete" || e.type === "error") && e.usage) refusal.usage = e.usage; + continue; + } if (e.type === "tool_call_start") { flushPending(); + if (e.name === ADVISOR_TOOL_NAME && !options.advisor && !options.clientAdvisorAllowed) { + refusal = { type: "error", status: 502, errorType: "upstream_error", code: UNDECLARED_TOOL_CALL_ERROR_CODE, + message: "routed provider emitted advisor outside the current iteration's permitted tool catalog" }; + passthrough.push(refusal); + continue; + } if (isBlank(e.id) || isBlank(e.name)) hasMalformedToolCall = true; pending = { name: e.name, id: e.id, argsBuf: "", closed: false, events: [e], providerMetadata: e.providerMetadata }; } else if (e.type === "tool_call_delta") { @@ -513,6 +524,8 @@ export async function runWithWebSearch(deps: WebSearchLoopDeps): Promise Promise; } type IterationSplit = ReturnType & { @@ -692,7 +705,19 @@ export async function runWithWebSearch(deps: WebSearchLoopDeps): Promise !tool.advisor && !tool.namespace + && tool.name === ADVISOR_TOOL_NAME && toolAllowed(tool)); + const declaredAdvisor = iterTools.find(tool => tool.advisor && tool.name === ADVISOR_TOOL_NAME); + // Exhaustion removes the declaration, but its paired limit result still belongs + // to the sidecar. A tools-disabled recovery revokes that permission too. + const advisorCandidate = declaredAdvisor ?? { name: ADVISOR_TOOL_NAME }; + const syntheticAdvisorAllowed = !!deps.advisor && !recoveringEmptyAnswer + && (!!declaredAdvisor || deps.advisor.exhausted()) + && toolChoiceToolPredicate(iterParsed.options.toolChoice, + declaredAdvisor ? iterTools : [...iterTools, advisorCandidate])(advisorCandidate); + return { response, responseAdapter: requestAdapter, clientAdvisorAllowed, syntheticAdvisorAllowed }; }; const recover = async (error: WebSearchAdapterEventError): Promise => { @@ -908,7 +933,7 @@ export async function runWithWebSearch(deps: WebSearchLoopDeps): Promise { expect(body).not.toContain("call_adv_1"); }); + test("web-search tool choice cannot be widened to Advisor", async () => { + releaseSpendHome = acquireOwnedSpendHome(); + const workerBodies: string[] = []; + const chatRequests: string[] = []; + const config = advisorConfig( + { enabled: true, model: "expert/gpt-6-astra", contextSharingConsent: "v1", policy: "manual" }, + workerProviderFetch([advisorCallFrames], workerBodies), + ); + config.webSearchSidecar = { backend: "exa", exaApiKey: "fixture-search-key" }; + loopbackInterceptor(config, { chatRequests }); + + const response = await handleResponses(workerRequest( + "Search without consulting the expert", undefined, true, [{ type: "web_search" }], + { type: "allowed_tools", mode: "required", tools: [{ type: "web_search" }] }, + ), config, logCtx); + const body = await response.text(); + + expect(workerBodies).toHaveLength(1); + const dispatched = JSON.parse(workerBodies[0]!) as { tools?: { function?: { name?: string } }[] }; + expect(dispatched.tools?.map(tool => tool.function?.name)).toEqual(["web_search"]); + expect(chatRequests).toHaveLength(0); + expect(body).toContain("undeclared_tool_call"); + expect(body).not.toContain("call_adv_1"); + expect(body).not.toContain("response.completed"); + }); + + test.each([true, false])("client advisor passthrough respects the effective choice: allowed=%s", async allowed => { + releaseSpendHome = acquireOwnedSpendHome(); + const workerBodies: string[] = []; + const config = advisorConfig(undefined, workerProviderFetch([advisorCallFrames], workerBodies)); + config.webSearchSidecar = { backend: "exa", exaApiKey: "fixture-search-key" }; + const response = await handleResponses(workerRequest("Use available tools", undefined, true, [ + { type: "web_search" }, { type: "function", name: "advisor", parameters: { type: "object" } }, + ], allowed ? "auto" : { type: "allowed_tools", mode: "required", tools: [{ type: "web_search" }] }), config, logCtx); + const body = await response.text(); + expect(workerBodies).toHaveLength(1); + if (allowed) { + expect(body).toContain("call_adv_1"); + expect(body).not.toContain("undeclared_tool_call"); + } else { + expect(body).toContain("undeclared_tool_call"); + expect(body).not.toContain("call_adv_1"); + expect(body).not.toContain("response.completed"); + } + }); + + test.each(["complete", "incomplete", "unterminated"])("scanner refuses an unauthorized advisor atomically: %s", shape => { + const usage = { inputTokens: 11, outputTokens: 3 }; + const events: AdapterEvent[] = [ + { type: "tool_call_start", id: "private-advisor-id", name: "advisor" }, + { type: "tool_call_delta", arguments: '{"question":"private-question"}' }, + ...(shape === "unterminated" ? [] : [{ type: "tool_call_end" as const, ...(shape === "incomplete" ? { incomplete: true } : {}) }]), + { type: "done", stopReason: "tool_calls", usage }, + ]; + const split = scanEventsForWebSearch(events); + expect(split.calls).toEqual([]); + expect(split.hasRealToolCall).toBe(false); + expect(split.hasMalformedToolCall).toBe(false); + expect(split.passthrough).toHaveLength(1); + expect(split.passthrough[0]).toMatchObject({ type: "error", code: "undeclared_tool_call", usage }); + expect(JSON.stringify(split.passthrough)).not.toContain("private-advisor-id"); + expect(JSON.stringify(split.passthrough)).not.toContain("private-question"); + }); + test("web_search and advisor calls are fulfilled in worker order without leaking advisor", async () => { releaseSpendHome = acquireOwnedSpendHome(); const workerBodies: string[] = []; @@ -440,7 +508,7 @@ describe("advisor responses wiring (end-to-end)", () => { expect(body).not.toContain("call_adv_mixed"); }); - test("advisor remains bounded while web-search owns repeated continuations", async () => { + test.each([false, true])("advisor remains bounded while web-search owns repeated continuations (allowlist=%s)", async allowlist => { releaseSpendHome = acquireOwnedSpendHome(); const workerBodies: string[] = []; const chatRequests: string[] = []; @@ -451,11 +519,20 @@ describe("advisor responses wiring (end-to-end)", () => { config.webSearchSidecar = { backend: "exa", exaApiKey: "fixture-search-key" }; loopbackInterceptor(config, { chatRequests }); - const response = await handleResponses(workerRequest("Repeat advisor with search enabled", undefined, true, [{ type: "web_search" }]), config, logCtx); + const response = await handleResponses(workerRequest( + "Repeat advisor with search enabled", undefined, true, [{ type: "web_search" }], + allowlist ? { type: "allowed_tools", mode: "required", tools: [ + { type: "function", name: "advisor" }, { type: "web_search" }, + ] } : undefined, + ), config, logCtx); const body = await response.text(); + const dispatched = JSON.parse(workerBodies[0]!) as { tools?: { function?: { name?: string } }[] }; + expect(dispatched.tools?.map(tool => tool.function?.name)).toContain("web_search"); + expect(dispatched.tools?.map(tool => tool.function?.name)).toContain("advisor"); expect(body).toContain("advisor_continuation_limit"); expect(workerBodies).toHaveLength(5); - expect(chatRequests.length).toBeLessThanOrEqual(3); + // The repeated fixture question is deduplicated by the real Advisor runtime. + expect(chatRequests).toHaveLength(1); }); test("scanner drops incomplete and unterminated advisor calls", () => { diff --git a/tests/advisor/advisor-web-search.test.ts b/tests/advisor/advisor-web-search.test.ts index 7995e74b30c..d333e232ea5 100644 --- a/tests/advisor/advisor-web-search.test.ts +++ b/tests/advisor/advisor-web-search.test.ts @@ -15,7 +15,7 @@ const call = (name: string, id: string, incomplete = false): AdapterEvent[] => [ ]; const answer: AdapterEvent[] = [{ type: "text_delta", text: "Final answer" }, { type: "done", usage: usage(30) }]; -async function drive(passes: AdapterEvent[][], maxSearches = 1) { +async function drive(passes: AdapterEvent[][], maxSearches = 1, clientAdvisor = false, toolChoice?: OcxParsedRequest["options"]["toolChoice"]) { const consultations: string[] = []; const searches: string[] = []; const requests: OcxParsedRequest[] = []; @@ -43,14 +43,14 @@ async function drive(passes: AdapterEvent[][], maxSearches = 1) { async *parseStream() { yield* events; }, }; const response = await runWithWebSearch({ - parsed: { modelId: "fixture", stream: true, options: {}, context: { + parsed: { modelId: "fixture", stream: true, options: { toolChoice }, context: { messages: [], tools: [ { name: "web_search", description: "Search", parameters: {} }, - { name: "advisor", description: "Advisor", parameters: {}, advisor: true }, + { name: "advisor", description: "Advisor", parameters: {}, ...(clientAdvisor ? {} : { advisor: true }) }, { name: "write", description: "Write", parameters: {} }, ], } }, - adapter, advisor, backend: "exa", exaApiKey: "fixture", hostedTool: { type: "web_search" }, + adapter, ...(clientAdvisor ? {} : { advisor }), backend: "exa", exaApiKey: "fixture", hostedTool: { type: "web_search" }, settings: { model: "fixture", reasoning: "low", timeoutMs: 1000 }, maxSearches, incomingMeta: { headers: new Headers(), translatorBudget: createTestTranslatorBudget() }, onUsage: value => reported.push(value), @@ -61,10 +61,16 @@ async function drive(passes: AdapterEvent[][], maxSearches = 1) { return { consultations, searches, requests, reported, text, terminal }; } -test("Advisor limit reports every billed worker leg once", async () => { +const advisorLimitChoices: OcxParsedRequest["options"]["toolChoice"][] = [ + undefined, "required", + { allowedTools: ["advisor", "web_search"], mode: "required" }, + // Direct-loop controls only: the web-search planner rejects Advisor-only choices. + { name: "advisor" }, { allowedTools: ["advisor"], mode: "required" }, +]; +test.each(advisorLimitChoices)("Advisor limit reports every billed worker leg once (choice=%j)", async toolChoice => { const result = await drive(Array.from({ length: 5 }, (_, i) => [ ...call("advisor", `advisor-${i}`), { type: "done", usage: usage(10) }, - ])); + ]), 1, false, toolChoice); expect(result.consultations).toHaveLength(3); expect(result.requests).toHaveLength(5); expect(result.text).toContain("advisor_continuation_limit"); @@ -111,3 +117,21 @@ test("incomplete Advisor calls suppress consultation and search execution", asyn expect(result.text).not.toContain("private-incomplete"); expect(result.reported).toEqual([expect.objectContaining({ inputTokens: 10, outputTokens: 1 })]); }); + +test.each([true, false])("empty-answer recovery revokes earlier advisor permission (client=%s)", async clientAdvisor => { + const result = await drive([ + [...call("web_search", "search-1"), { type: "done", usage: usage(10) }], + [{ type: "done", usage: usage(20) }], + [...call("advisor", "recovery-private-call"), { type: "done", usage: usage(30) }], + ], 1, clientAdvisor); + expect(result.requests).toHaveLength(3); + expect(result.requests[0]!.context.tools!.some(tool => tool.name === "advisor")).toBe(true); + expect(result.requests[2]!.context.tools).toEqual([]); + expect(result.requests[2]!.options.toolChoice).toBe("none"); + expect(result.text).toContain("undeclared_tool_call"); + expect(result.text).not.toContain("recovery-private-call"); + expect(result.text).not.toContain("response.completed"); + expect(result.consultations).toEqual([]); + expect(result.searches).toHaveLength(1); + expect(result.reported).toEqual([expect.objectContaining({ inputTokens: 60, outputTokens: 3 })]); +});