diff --git a/SECURITY.md b/SECURITY.md index d09dd397..e4e0f75f 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -154,7 +154,7 @@ The token is a lock on the **write** surface, not a wall around the whole API. - **The real boundary is loopback + the DSH host listener.** The host-side routes (`/api/dsh-mneme/*`) are mounted on the DSH port, which binds `127.0.0.1`. Every one of them asks the host for a verdict before doing anything else — `ctx.connection.requestRejection(req)` — so the host's own fence against DNS rebinding and cross-site browser requests applies to this plugin's routes too: a page open in your browser cannot call them, and gets `403` (fence) or `401` (not signed in to the host) instead of memory data. That fence is not identity: any process already running locally, and any page in the host's own origin, is inside the perimeter. When the host provides no `connection` service (headless, an older host, tests), the check is skipped and behaviour falls back to what it was before this gate existed — a missing host service must not 401 the panel. - **Authentication is opt-in and fail-open.** `apiToken` defaults to empty; while it is empty, `isAuthorized()` returns true for every request — reads *and* writes. Setting a token gates the write routes and the sensitive reads; it does not gate plain reads. - **Open reads (no token even when one is configured)**: `GET /`, `/info`, `/version-check`, `/list`, `/recall-stats`, `/search`, `/semantic`, `/semantic/llm-audit`, `/semantic/llm-audit/stats`, `/semantic/consolidation-stats`, `/semantic/graph/*`, `/entities`, `/memories/entities`, `/dream-status`, `/inject-status`, `/inject-preview`, `/pipeline`, `/candidates`, `/diary`, `/diary/entry`, `/diary/library`, `/diary/library/entry`, `/llm-providers`, `/config-keys`, plus the `GET`/`HEAD` branches of `/profile`, `/rules`, `/persona`, `/persona/session`, `/profile-facts`, `/mode`, `/features`, `/commands`, `/nudge`, `/diary/compose`, `/diary/export`, of the two share-receive routes `/receive` and `/receive/undo`, of `/profile-facts/apply`, of the six chorography read routes `/choro/repos`, `/choro/modules`, `/choro/page`, `/choro/search`, `/choro/claims`, `/choro/run` (M1/finale: they read repo-local wiki pages and `choro_*` index tables — not memory rows; with `chorographyEnabled` off they uniformly answer `409 disabled`, the #497 mount-anyway-gate-in-logic pattern), of the two T1 job routes `/choro/t1` and `/choro/t1/status` (same `token-write` shape as `/choro/run`: the only token-exempt branch is a `405`, and the POST itself answers `409 disabled` unless both `chorographyEnabled` and `chorographyT1FromPanel` are on), and of the three `token-write` knowledge-base routes `/kb/collections`, `/kb/ingest` and `/kb/search` (again only a `405` is genuinely open on them, because `needsToken()` exempts `GET`/`HEAD` under `token-write`; the two plain-`token` reads under `/kb/*` stay gated even on `GET`). `/choro/run` is the same shape as `/receive`: a `token-write` route whose GET branch is token-exempt, so the only genuinely open thing on that path is a `405` — the POST write itself stays gated. Treat them as readable by anything local. `/receive` and `/receive/undo` belong on this list even though they write: under the `token-write` policy, `needsToken()` exempts `GET`/`HEAD` from the token, so the only thing genuinely open on those two paths is a `405` — no read branch exists. `/profile-facts/apply` is POST-only for the same reason (its only open branch is a `405`). The `GET` branch of `/profile-facts` stays open even while `profileFactsEnabled` is off: facts survive a switch-off and the panel must be able to show them (same reasoning as `GET /profile`). This section mirrors the policy declarations rather than anyone's intent, which is why a write route shows up under "open reads"; `dsh-mneme/test/security-doc.test.js` compares both lists against the routes actually mounted, in both directions. -- **Gated routes (when a token is configured)**: every write — `/create`, `/update`, `/delete`, `/import`, `/bootstrap`, `/vector-config`, `/vector-reindex`, `/runtime/provision`, `/maintenance/reclaim`, `/test-model`, `/conflicts/resolve`, `/profile/extract` (POST-only: one host-LLM call that returns a profile **draft**; other methods get 405 before any side effect), `/profile-facts/apply` (POST-only: per-fact manual corrections from the panel; writes the `profile_facts` tables, never `user_settings`), `/semantic/graph/affective/apply` (POST-only: per-group manual corrections on `affective_context` edges — rename/archive/restore/promote/demote; writes edge metadata plus the `affective_journal` ledger, never `user_settings`; its GET branch is open under the wildcard above — no gated read exists there), `/profile-facts/run` (POST-only, `token` policy like `/profile/extract`: one host-LLM maintenance pass with no read branch; 409 while `profileFactsEnabled` is off), `/scope/promote` (POST-only, plain `token`: re-stamps the `workspace_scope` of selected rows, one at a time, with a `scope_changes` audit row per success — it moves memories out of a hard-walled import area, so an unauthenticated caller must never reach it), `/choro/run` (POST-only: the panel "rebuild skeleton" write path, re-running T0 into the repo's `.chorography/**`; with `chorographyEnabled` off it uniformly answers 409 disabled), `/choro/t1` (POST-only: spawns a real T1 run through the choro CLI — a host-LLM pass that costs money, so it carries its own `chorographyT1FromPanel` switch on top of `chorographyEnabled`), `/choro/t1/status` (the job poll: its `logTail` carries CLI/model output, so it is not a public-repo read like the rest of `/choro/*`), `/kb/collections`, `/kb/search` (POST-only writes; the only token-exempt branch is the `405`, same shape as `/choro/run`), `/kb/ingest` (POST-only: pulls a configured local source into the knowledge base; source-level failures still answer `200` with `ok:false`, which is a result, not a permission), `/kb/docs`, `/kb/doc` (plain `token`: they hand back whatever collection content the user ingested — that is private material, unlike the repo-documentation `/choro/*` reads), `/receive`, `/receive/undo` (the two share-receive routes write into *this* machine's library, which is why they are `token-write`), `/diary/compose` (POST-only: one host-LLM call that writes `/diaries//YYYY-MM-DD.md`; 409 while `diaryEnabled` is off), `/diary/export` (POST-only: copies one entry into the *user's* bound project directory, which is outside every path the plugin otherwise owns — 409 on an existing target unless `overwrite` is explicit), and the non-`GET` branches of `/profile`, `/rules`, `/persona`, `/persona/session`, `/profile-facts`, `/mode`, `/features`, `/commands`, `/nudge` — plus seven reads: `/export`, `/external-api`, `/conflicts`, `/conflicts/ledger` (read-only triage ledger — tier/status/detection timestamps and title-level info, same sensitivity as `/conflicts`), `/health`, `/share`, `/memory/provenance`. `/share` is gated because the share pack is a whole-library read delivered over POST, and it answers 405 to GET — no open read path exists. `/memory/provenance` is gated because it returns the **verbatim raw text** of distilled conversation windows (opt-in via `distillRecords.enabled`); chained with the open list route it would otherwise let any local process enumerate ids and then pull that text back, which reaches the same result as `/export` in a different shape. +- **Gated routes (when a token is configured)**: every write — `/create`, `/update`, `/delete`, `/import`, `/bootstrap`, `/vector-config`, `/vector-reindex`, `/runtime/provision`, `/maintenance/reclaim`, `/maintenance/backfill` (POST-only: the batch pass over imported DeepSeek-export sessions; it has no read branch — any other method is a `404` — and `dryRun` defaults to true, so the first call spends nothing; only a run with `"dryRun": false` reaches the host LLM), `/test-model`, `/conflicts/resolve`, `/profile/extract` (POST-only: one host-LLM call that returns a profile **draft**; other methods get 405 before any side effect), `/profile-facts/apply` (POST-only: per-fact manual corrections from the panel; writes the `profile_facts` tables, never `user_settings`), `/semantic/graph/affective/apply` (POST-only: per-group manual corrections on `affective_context` edges — rename/archive/restore/promote/demote; writes edge metadata plus the `affective_journal` ledger, never `user_settings`; its GET branch is open under the wildcard above — no gated read exists there), `/profile-facts/run` (POST-only, `token` policy like `/profile/extract`: one host-LLM maintenance pass with no read branch; 409 while `profileFactsEnabled` is off), `/scope/promote` (POST-only, plain `token`: re-stamps the `workspace_scope` of selected rows, one at a time, with a `scope_changes` audit row per success — it moves memories out of a hard-walled import area, so an unauthenticated caller must never reach it), `/choro/run` (POST-only: the panel "rebuild skeleton" write path, re-running T0 into the repo's `.chorography/**`; with `chorographyEnabled` off it uniformly answers 409 disabled), `/choro/t1` (POST-only: spawns a real T1 run through the choro CLI — a host-LLM pass that costs money, so it carries its own `chorographyT1FromPanel` switch on top of `chorographyEnabled`), `/choro/t1/status` (the job poll: its `logTail` carries CLI/model output, so it is not a public-repo read like the rest of `/choro/*`), `/kb/collections`, `/kb/search` (POST-only writes; the only token-exempt branch is the `405`, same shape as `/choro/run`), `/kb/ingest` (POST-only: pulls a configured local source into the knowledge base; source-level failures still answer `200` with `ok:false`, which is a result, not a permission), `/kb/docs`, `/kb/doc` (plain `token`: they hand back whatever collection content the user ingested — that is private material, unlike the repo-documentation `/choro/*` reads), `/receive`, `/receive/undo` (the two share-receive routes write into *this* machine's library, which is why they are `token-write`), `/diary/compose` (POST-only: one host-LLM call that writes `/diaries//YYYY-MM-DD.md`; 409 while `diaryEnabled` is off), `/diary/export` (POST-only: copies one entry into the *user's* bound project directory, which is outside every path the plugin otherwise owns — 409 on an existing target unless `overwrite` is explicit), and the non-`GET` branches of `/profile`, `/rules`, `/persona`, `/persona/session`, `/profile-facts`, `/mode`, `/features`, `/commands`, `/nudge` — plus seven reads: `/export`, `/external-api`, `/conflicts`, `/conflicts/ledger` (read-only triage ledger — tier/status/detection timestamps and title-level info, same sensitivity as `/conflicts`), `/health`, `/share`, `/memory/provenance`. `/share` is gated because the share pack is a whole-library read delivered over POST, and it answers 405 to GET — no open read path exists. `/memory/provenance` is gated because it returns the **verbatim raw text** of distilled conversation windows (opt-in via `distillRecords.enabled`); chained with the open list route it would otherwise let any local process enumerate ids and then pull that text back, which reaches the same result as `/export` in a different shape. - **The standalone data plane** (`127.0.0.1:8790`) is the opposite shape: every route except `GET /health` requires its own Bearer token. Pointing `externalApiHost` at a non-loopback address therefore turns it into a plaintext, token-only HTTP surface — an explicit operator decision. - **Consequence**: if you expose the plugin beyond loopback (reverse proxy, non-loopback bind), set `apiToken` *and* check that the plain reads above are acceptable to expose — they are not covered by it. - **How this list stays true**: every route declares its own `policy` (`open` / `token` / `token-write`) at registration, and `createRouteRegistrar` throws when a route has none or reuses a path. A new route that forgets its classification **fails to boot** instead of silently opening. The two lists above are compared against the routes actually mounted by `dsh-mneme/test/security-doc.test.js` in both directions, so adding a route without updating this section fails CI rather than quietly drifting — the matrix is derived from code, not from anyone's memory. @@ -552,7 +552,7 @@ Token 是**写**面上的锁,不是把整个 API 围起来的墙。 - **真正的边界是回环 + 宿主监听器。** 宿主侧路由(`/api/dsh-mneme/*`)挂在 DSH 端口上,该端口绑 `127.0.0.1`。**每一条路由在动手之前都先问宿主一句 `ctx.connection.requestRejection(req)`**,所以宿主挡 DNS rebinding 与跨站浏览器请求的那道围栏,对插件的路由同样生效:浏览器里开着的网页调不动它们,拿到的是 `403`(围栏不过)或 `401`(没登录宿主),而不是记忆数据。这道围栏**不建立身份**:任何本地进程、以及任何跑在宿主同源页面里的脚本,都在这道围栏之内。宿主没提供 `connection` 服务时(headless、旧宿主、测试)这道检查跳过,行为退回加它之前的样子——缺一个宿主服务不能让整个面板 401。 - **鉴权是 opt-in 且 fail-open。** `apiToken` 默认为空;为空期间 `isAuthorized()` 对每个请求都返回 true——读和写都放行。配了 token 之后,它守的是写路由与敏感读,不守普通读。 - **开放读(即使配了 token 也不需要)**:`GET /`、`/info`、`/version-check`、`/list`、`/recall-stats`、`/search`、`/semantic`、`/semantic/llm-audit`、`/semantic/llm-audit/stats`、`/semantic/consolidation-stats`、`/semantic/graph/*`、`/entities`、`/memories/entities`、`/dream-status`、`/inject-status`、`/inject-preview`、`/pipeline`、`/candidates`、`/diary`、`/diary/entry`、`/diary/library`、`/diary/library/entry`、`/llm-providers`、`/config-keys`,以及 `/profile`、`/rules`、`/persona`、`/persona/session`、`/profile-facts`、`/mode`、`/features`、`/commands`、`/nudge`、`/diary/compose`、`/diary/export`、`/receive`、`/receive/undo`、`/profile-facts/apply`、`/choro/repos`、`/choro/modules`、`/choro/page`、`/choro/search`、`/choro/claims`、`/choro/run`、`/choro/t1`、`/choro/t1/status`、`/kb/collections`、`/kb/ingest`、`/kb/search` 的 `GET`/`HEAD` 分支。`/choro/run` 与 `/receive` 同款:token-write 的 GET 分支免 token 后真正开放的只有一个 405(它只收 POST),出现在这里是 policy 声明的忠实投影。把它们当作「本地任何东西都能读」。`/receive`、`/receive/undo` 明明是写路由也出现在这一串里,这不是笔误:`token-write` 下 `needsToken()` 对 `GET`/`HEAD` 免 token,所以这两条路径上真正「开放」的只有一个 `405`,压根没有可读分支;`/profile-facts/apply` 同理只收 POST,开放的唯一分支是一个 `405`。`/profile-facts` 的 `GET` 分支在 `profileFactsEnabled` 关着时同样开放:开关关掉后存量事实还在库里,面板要能展示(与 `GET /profile` 同一理由)。`/choro/*` 六条(chorography 仓库地志读侧,M1 + 收尾档)读的是**仓库里的 MD 页与 choro_* 索引表**,不是记忆行——内容是代码仓库的公开文档面,`chorographyEnabled` 关着时统一回 409 disabled(路由挂着、逻辑层挡,#497 先例),不因为开放读而绕过这个闸。这一节照 policy 声明抄、不照意图写,`dsh-mneme/test/security-doc.test.js` 会拿真实装配出来的路由双向比对这两行。 -- **受门控的路由(配了 token 之后)**:全部写操作——`/create`、`/update`、`/delete`、`/import`、`/bootstrap`、`/vector-config`、`/vector-reindex`、`/runtime/provision`、`/maintenance/reclaim`、`/test-model`、`/conflicts/resolve`、`/profile/extract`(只收 POST:花一次宿主 LLM 产一段画像**草稿**,其它方法在产生任何副作用之前就 405)、`/profile-facts/apply`(只收 POST:面板的逐条人工修正;写的是 profile_facts 三表,绝不碰 user_settings)、`/semantic/graph/affective/apply`(只收 POST:情绪的逐组人工修正——改词/归档/恢复/升格/降格,作用于 affective_context 边;写的是边的 metadata 加 affective_journal 账本,绝不碰 user_settings;其 GET 分支按上方通配开放,那里没有受门控的读分支)、`/profile-facts/run`(只收 POST,`token` 政策与 `/profile/extract` 同款:一次宿主 LLM 维护轮,无读分支;`profileFactsEnabled` 关着时 409)、`/scope/promote`(只收 POST,`token` 政策:把选中条目的 `workspace_scope` 逐条改写出去,每条成功都落一行 `scope_changes` 审计——它负责把记忆搬出硬墙导入区,绝不能给未鉴权的调用方)、`/choro/run`(只收 POST:面板「重建骨架」的写入口,重跑 T0 落仓库磁盘的 `.chorography/**`——chorographyEnabled 关着时统一 409 disabled)、`/choro/t1`(只收 POST:经 choro CLI 真起一次 T1——一次要花模型额度的宿主 LLM 作业,所以在 `chorographyEnabled` 之上另有一枚 `chorographyT1FromPanel` 闸)、`/choro/t1/status`(任务轮询:`logTail` 里带 CLI 与模型的原始输出,不属 `/choro/*` 其余几条那种「仓库公开文档面」)、`/kb/collections`、`/kb/search`(只收 POST 的写入口,token-write 下真正开放的只有一个 405,与 `/choro/run` 同款)、`/kb/ingest`(只收 POST:把用户配置的本地来源抽进知识库;来源级失败仍回 `200` + `ok:false`,那是结果不是权限)、`/kb/docs`、`/kb/doc`(纯 `token`:回的是用户导进来的集合内容,属私料,与 `/choro/*` 那类仓库文档读侧不同量级)、`/receive`、`/receive/undo`(这两条写的是**本机自己的库**,所以归 `token-write`)、`/diary/compose`(只收 POST:花一次宿主 LLM 写 `/diaries/<项目键>/YYYY-MM-DD.md`;`diaryEnabled` 关着时 409)、`/diary/export`(只收 POST:把某一篇**复制**进用户绑定的项目目录——那是插件本来不拥有的路径,目标已存在时默认 409,要覆盖必须显式传 `overwrite`),以及 `/profile`、`/rules`、`/persona`、`/persona/session`、`/profile-facts`、`/mode`、`/features`、`/commands`、`/nudge` 的非 `GET` 分支——外加七条读:`/export`、`/external-api`、`/conflicts`、`/conflicts/ledger`(矛盾分诊账本的只读面:tier/status/检出时间与标题级信息,与 `/conflicts` 同级——处置由 enforce 档自动完成,行不是待办)、`/health`、`/share`(分享包是「用 POST 递出来的全库读取」,GET 一律 405,没有开放的读分支)、`/memory/provenance`(它回的是蒸馏窗口的**逐字会话原文**,并自 #512 起按窗口带出 `distill_run_id` / `run_type` / `trigger_source` / `run_status` 的运行归因;`distillRecords.enabled` 打开才有;否则把它和上面那个开放的列表路由串起来,任何本地进程都能先枚举 id、再按 id 把那些原文捞回去——可达结果与 `/export`(一次拿全库)同量级,只是换了形状)。 +- **受门控的路由(配了 token 之后)**:全部写操作——`/create`、`/update`、`/delete`、`/import`、`/bootstrap`、`/vector-config`、`/vector-reindex`、`/runtime/provision`、`/maintenance/reclaim`、`/maintenance/backfill`(只收 POST:把导入的 DeepSeek 导出会话按批过一遍蒸馏;没有读分支,其它方法一律 `404`;`dryRun` 默认 true,第一次调用一个 token 都不花,只有显式 `"dryRun": false` 才真的走到宿主 LLM)、`/test-model`、`/conflicts/resolve`、`/profile/extract`(只收 POST:花一次宿主 LLM 产一段画像**草稿**,其它方法在产生任何副作用之前就 405)、`/profile-facts/apply`(只收 POST:面板的逐条人工修正;写的是 profile_facts 三表,绝不碰 user_settings)、`/semantic/graph/affective/apply`(只收 POST:情绪的逐组人工修正——改词/归档/恢复/升格/降格,作用于 affective_context 边;写的是边的 metadata 加 affective_journal 账本,绝不碰 user_settings;其 GET 分支按上方通配开放,那里没有受门控的读分支)、`/profile-facts/run`(只收 POST,`token` 政策与 `/profile/extract` 同款:一次宿主 LLM 维护轮,无读分支;`profileFactsEnabled` 关着时 409)、`/scope/promote`(只收 POST,`token` 政策:把选中条目的 `workspace_scope` 逐条改写出去,每条成功都落一行 `scope_changes` 审计——它负责把记忆搬出硬墙导入区,绝不能给未鉴权的调用方)、`/choro/run`(只收 POST:面板「重建骨架」的写入口,重跑 T0 落仓库磁盘的 `.chorography/**`——chorographyEnabled 关着时统一 409 disabled)、`/choro/t1`(只收 POST:经 choro CLI 真起一次 T1——一次要花模型额度的宿主 LLM 作业,所以在 `chorographyEnabled` 之上另有一枚 `chorographyT1FromPanel` 闸)、`/choro/t1/status`(任务轮询:`logTail` 里带 CLI 与模型的原始输出,不属 `/choro/*` 其余几条那种「仓库公开文档面」)、`/kb/collections`、`/kb/search`(只收 POST 的写入口,token-write 下真正开放的只有一个 405,与 `/choro/run` 同款)、`/kb/ingest`(只收 POST:把用户配置的本地来源抽进知识库;来源级失败仍回 `200` + `ok:false`,那是结果不是权限)、`/kb/docs`、`/kb/doc`(纯 `token`:回的是用户导进来的集合内容,属私料,与 `/choro/*` 那类仓库文档读侧不同量级)、`/receive`、`/receive/undo`(这两条写的是**本机自己的库**,所以归 `token-write`)、`/diary/compose`(只收 POST:花一次宿主 LLM 写 `/diaries/<项目键>/YYYY-MM-DD.md`;`diaryEnabled` 关着时 409)、`/diary/export`(只收 POST:把某一篇**复制**进用户绑定的项目目录——那是插件本来不拥有的路径,目标已存在时默认 409,要覆盖必须显式传 `overwrite`),以及 `/profile`、`/rules`、`/persona`、`/persona/session`、`/profile-facts`、`/mode`、`/features`、`/commands`、`/nudge` 的非 `GET` 分支——外加七条读:`/export`、`/external-api`、`/conflicts`、`/conflicts/ledger`(矛盾分诊账本的只读面:tier/status/检出时间与标题级信息,与 `/conflicts` 同级——处置由 enforce 档自动完成,行不是待办)、`/health`、`/share`(分享包是「用 POST 递出来的全库读取」,GET 一律 405,没有开放的读分支)、`/memory/provenance`(它回的是蒸馏窗口的**逐字会话原文**,并自 #512 起按窗口带出 `distill_run_id` / `run_type` / `trigger_source` / `run_status` 的运行归因;`distillRecords.enabled` 打开才有;否则把它和上面那个开放的列表路由串起来,任何本地进程都能先枚举 id、再按 id 把那些原文捞回去——可达结果与 `/export`(一次拿全库)同量级,只是换了形状)。 - **独立数据面**(`127.0.0.1:8790`)是反过来的形状:除 `GET /health` 外每条路由都要自己的 Bearer token。因此把 `externalApiHost` 指向非回环地址,就等于把它变成一个「明文、只靠 token」的 HTTP 面——这是运营者的显式决定。 - **推论**:如果你把插件暴露到回环之外(反代、绑非回环地址),请设 `apiToken`,**并**核对上面那串开放读是否也能被看到——它们不受 token 保护。 - **这张表怎么保持为真**:每条路由在注册时自己声明 `policy`(`open` / `token` / `token-write`),`createRouteRegistrar` 在路由没声明或复用同一路径时直接抛错。新路由忘了归类会**启动失败**,而不是静默开门。上面两张清单还会被 `dsh-mneme/test/security-doc.test.js` 拿真实装配出来的路由做双向比对——加了路由却不更新这一节,CI 会红,而不是悄悄漂走。这张矩阵来自代码,不来自谁的记忆。 diff --git a/dsh-mneme/CHANGELOG.md b/dsh-mneme/CHANGELOG.md index 650072c1..dd83addc 100644 --- a/dsh-mneme/CHANGELOG.md +++ b/dsh-mneme/CHANGELOG.md @@ -2,6 +2,31 @@ ## [Unreleased] +- **保护层复审:七处真问题,三处会让功能实际失效(#755 复审)**。CodeRabbit 在 PR #755 上提了 7 条,逐条核实**全部属实**,其中最要紧的三条不是风格问题—— + ① `distillExcludeProjects` 是字符串数组,却被我登记进了 `FEATURE_FLAG_BOOLEANS`。`validateFlag` 对这张表的成员只收 boolean,于是**写名单抛 TypeError、写 `false` 却通过并存下来**:用户以为自己关掉了「导入隔离」,重启后名单变成一个假的开关。核查既有 5 个数组型 schema 键(`codingKeywords`、`diaryMemoryExcludeTypes`、`externalApiAllowedOrigins`、`scopeHardWallWorkspaces`、`sessionGateSessions`)在 `settings.js` 里出现次数**全为 0**——数组键一律不进白名单、走 `fileOnly` 桶,这条口径原先就立着,是我自己撞上去的。现在把它移出白名单,只留 `distillSkipImportedSessions`(真布尔)。 + ② `createSummarizer` 第一行 `if (!config.autoSummarize) return { dispose }` —— 关掉自动蒸馏时**连 `summarizeNow` 一起消失**,而回填正是靠它手动跑,`runBackfill` 会直接返回 `error: "summarizer-unavailable"`。等于「关掉自动蒸馏」悄悄搭上了「导入会话永远补不上记忆」。改为只跳过 `session/event` 订阅,`summarizeNow` 照常提供。 + ③ `backfill.js` 调 `summarizeNow` 时只传 `{ id, events }`,**没传 header**,而项目排除门卫读的是 `session.header.cwd` —— 回填路径上 `distillExcludeProjects` 永远不生效,恰恰是「这些导入会话不该进库」最该守的那一条。现在把 `reading.header` 一起给过去。 + 另四条较轻但都修了:`positive()` 对 `limit: 0.5` 先判 `> 0` 再 `Math.floor` 得 0,而 0 在 `planBackfill` 里是「不限制」——一次手滑把 dry-run 变成全量跑(改为先 floor 再判,下界钉 1,小数回退默认值);`normalizeProjectPath` 把 `/` 剥成空串再 `|| null`,名单写 `["/"]` 本意「全挡」却整条失效(根目录保留,任何绝对路径都算其后代);`runBackfill` 返回 `no-distill-route` 时路由仍回 200,调用方记成功而一条都没跑(`no-distill-route` 与 `summarizer-unavailable` 一律 503);导入门在 `collectMessages` **之后**才判,而被挡的会话已经逐图调过一轮视觉模型(门卫前先花了一轮描述钱)——改为把同一条件透给 `collectMessages`,就地复用刚读好的 events 做判定,不额外快照。 + **测试** 3,678 → **3,691**(+13):新 `test/distill-guard.test.js` 7 条(路径归一含反斜杠/大小写/连续分隔符折叠、根目录条目挡全部、兄弟目录 `private-public` 不被 `private/` 吃掉、名单夹无效条目时照常判其余、无 cwd 或空名单一律放行);`test/summarize.test.js` +2(被挡下的导入会话**一个视觉调用都不发**,以及反向对照——正常会话的图确实描述一次,否则前一条在「描述档压根没生效」时也会绿);`test/api.test.js` +4(小数 limit 不许 floor 成 0、limit 传 1 就用 1、`no-distill-route`/`summarizer-unavailable` 回 503、真跑了仍是 200);`test/backfill.test.js` 加一条 header 断言;`test/lib-smoke.test.js` 断言 `autoSummarize:false` 时 `summarizeNow` 仍在。`test/api.test.js` 的计数锁 140 → **141**(只 +1:`distillExcludeProjects` 不在白名单里,这笔账如实记)。 + +- **蒸馏保护层接线完成:导入会话与排除项目不再烧蒸馏(#688 续)**。上一条给了导入会话一个显式回填入口,但那条线反过来提出一个没回答的问题:**在用户点回填之前,这些会话若是继续说话,会发生什么?** 答案是坏的——导入会话没有 `distill_cursors`,`snapshotSessionEvents(session, afterSeq, throughSeq)` 在 `afterSeq` 为空时只按 `throughSeq` 截断(`dsh-mneme/src/summarize.js:518`),于是「导入进去、以后再说」的每一轮 `turn/end` 都会把整段历史当新窗口(尾部 `distillMaxChars` 24,000 字)烧一次蒸馏。修法是两道**确定性**门卫,默认开、面板可关: + + ① **导入会话隔离**(`distillSkipImportedSessions`,默认 `true`)。判定读事件不判 header——导入器不留 header 标记,唯一信号是 `assistant/message` 的 `source.provider === "deepseek-chat-export"`(判据唯一定义在 `dsh-mneme/src/imported-session.js`,backfill 复用同一处,不另抄一份)。判定放在窗口读之后(`collectMessages` 把读好的事件原样带回 `collected.events`),**复用本轮已读的快照**,不给每轮 turn/end 增加第二次全量读;游标刻意不消费(与 peak / max-runs 同口径:「这个会话先不再蒸馏了」,窗口留在原地)。`opts.bypassImported` 是回填的显式旁路(`backfill.js` 与 `bypassPeak` 同形透传)——回填喂的恰恰是导入会话,保护层不能挡自己的合法调用方。 + + ② **项目排除**(`distillExcludeProjects`,默认 `[]`)。cwd 前缀命中(含同一目录)即整条项目线挡在库外,含之后新建的会话,不用逐条加 id。cwd 只读 `session.header`(创建时冻结的那份,`requestHeader` 会遮蔽——scope.js 踩过的坑)。两键都进 `FEATURE_FLAG_BOOLEANS` 白名单(计数锁自动收),与 `sessionGateEnabled`(逐会话无痕,连注入都不给)分层:这里只管「蒸馏进不进库」。 + + 被挡的调用写 `status:"skipped"` 审计(`error_message` = `"imported-session"` / `"excluded-project"`),「这一轮为什么没蒸馏」保持可观测。 + + **测试** 3,534 → **3,541**(+7):`test/summarize.test.js` 覆盖默认挡(零调用零落库,审计 `imported-session`)、`distillSkipImportedSessions:false` 放行、`summarizeNow + bypassImported:true` 放行(回填调用形状)、正常会话不受影响、项目排除(审计 `excluded-project`,cwd 混合分隔符归一)、兄弟目录放行(`private-public` 不是 `private/` 的子路径)、两门独立(排除在前先记账)。附带一次真回归拦截:门卫初版在门卫区多打一次全量快照,破坏了「无新事件不调 LLM」用例的快照间谍(`ranges[1][0] === 2`)——这也是把判定挪到窗口读之后的原因之一。`test/backfill.test.js` 14 条全绿(`bypassImported` 透传路径),`config-keys-routes` + `settings` 计数锁 36 条全绿(两键入白名单、`undefaulted` 桶不变仍是 dreamModel/dreamProvider)。 + +- **导入的 DeepSeek 会话没有「离线蒸馏」入口:蒸馏只挂在 `turn/end` 上,导入进来的几百条会话永远等不到下一步(#606 之后的新场景)**。DSH 0.2.2-rc.3 的实验性导入插件(`@deepseek-ai/dsh-deepseek-chat-import`,默认关)把 DeepSeek APP 的导出会话原样铺成普通 DSH 会话——不做压缩、不做摘要;而 mneme 的蒸馏是 `ctx.on("session/event", …)` 里对 `turn/end` 的反应(`dsh-mneme/src/summarize.js:1620-1627`),导入的会话不会再产生 `turn/end`,于是「几百条未压缩会话」对记忆系统等于不存在。修法是补一个手动入口 + 一个薄方法,**不复制任何蒸馏逻辑**:`createSummarizer` 的返回对象多一个 `summarizeNow(session, triggerEvent, opts)`(`dsh-mneme/src/summarize.js`,方法体只有一行 `return summarize(...)`),新增 `dsh-mneme/src/backfill.js` 负责扫描、判别、跳过已消费、按时间倒序取批、循环调用。落库口径全部沿用现有链路:每窗条数走 `summarizeMaxEntriesPerRun` 的**这一次性覆盖**(`opts.maxEntries`,不传时逐字等于旧行为),去重、质量判定与游标都不变。 + + 判别是纯函数 `isImportedSession(events)`:主判据是 `assistant/message` 的 `data.message.source.provider === "deepseek-chat-export"`(DSH 自己从不产生该字符串,不会误判正常会话),`turn/end` 的 `DEEPSEEK_EXPORT_*` 错误码只兜「整条会话的回复都被排除、一条 `assistant/message` 都没有」的那种;刻意**不认** header 与 `session/title`(header 只写 version/id/createdAt/cwd/isSeeded,正常会话也有 title)。窗口取 `turn/start` 切出的最近 20 轮、一次调用;断点续跑不新建表——`distill_cursors` 的单调 UPSERT 加 `distill_records` 的 `(session_id, seq)` 主键已经让重跑幂等,中途取消、断电、插件重装都能接着来。 + + **默认 `dryRun`**:`POST /api/dsh-mneme/maintenance/backfill` 只收 POST(`policy: "token"`,没有读分支,其它方法一律 404),body 缺省即 `{"dryRun": true}`,一个 token 都不花,只回「扫到多少文件、认出几条导入会话、多少超 90 天窗、已消费多少、估算几次调用」;`"dryRun": false` 才真跑。没配蒸馏路由时真跑**直接拒绝**(`error: "no-distill-route"`,一条都不调)——否则 50 条会话会变成 50 条 failed 加 50 行日志,读起来像批量任务坏了,其实只是没配模型。本机实测(`D:/DSH/.dsh/sessions`,limit 50 / days 90):**836 个会话文件、识别到 0 条导入会话、0 次调用**——插件默认关、还没导入,这一格是空的而不是坏的。 + + **测试** 3,520 → **3,534**(+14):`test/backfill.test.js` 覆盖 provider 主判据、整条无 assistant 消息时错误码兜底(并确认普通的 `STREAM_CLOSED` 不被误判成导入)、不认 header/标题、`recentTurns` 25→20 且首条落回第 6 轮、磁盘事件缺 `seq` 时按行号补、`planBackfill` 的六格计数与倒序、`limit` 只切候选不改账目、未配蒸馏路由时 `callsPlanned` 归零且真跑零调用、dry-run 一次 summarizer 都不调、真跑确实传 `bypassPeak` 与 `maxEntries`、一条失败不影响后续,以及一条**按插件真实写盘形状**造的端到端用例(首行 header 无 `seq`、其后事件都带 `seq`、第二轮只有 `turn/end` 带 `DEEPSEEK_EXPORT_*`):用真 store/service 跑完,断言游标推进到末尾、`distill_records` 落一行、第二次 `planBackfill` 判为已消费且零调用。`SECURITY.md` 两条清单同步登记该路由,`test/security-doc.test.js` 的双向比对通过。 + ## [0.16.1] - 2026-11-18 - **宿主 HTTP 路由从没问过宿主「这个请求该不该放行」:浏览器里打开的任意网页都能调记忆接口(awesome 市场的条目因此被 maintainer 补了一句安全说明)**。`createRouteRegistrar`(`dsh-mneme/src/api.js:166`)只做 apiToken 判定,而 `open` 策略的路由——`/api/dsh-mneme`、`/info`、`/version-check`、`/list`、`/recall-stats`、`/search`、`/profile` 等——按 `needsToken` 的定义**永不鉴权**,于是这两层之外再没有任何一处问过宿主「这个请求该不该放行」:同机任何进程、以及浏览器里打开的任意网页,都能直接读出这些路由返回的记忆数据。修法是在 handler 最开头、在 apiToken 判定**之前**先问一次 `ctx.connection.requestRejection(req)`——`403`(Host/Origin 围栏不过)或 `401`(未登录宿主)直接结束响应、一个字节记忆内容都不回,只有 `undefined` 才继续走原有的 policy 路径。门控收在 `createRouteRegistrar` 里一次(`gate(req, res)`),新增的每条路由自动带上,不靠各路由自己记得。 diff --git a/dsh-mneme/docs/CONFIGURATION.md b/dsh-mneme/docs/CONFIGURATION.md index 0406a4d8..eb9d10cb 100644 --- a/dsh-mneme/docs/CONFIGURATION.md +++ b/dsh-mneme/docs/CONFIGURATION.md @@ -135,6 +135,8 @@ | `summarizeParseFailureMaxRetries` | `0` | 连续解析失败几次后放弃当前窗口(0–10,0 = 关,#493) | 达到阈值强制消费游标跳过该窗口;失败行 metadata 恒带 `raw_head`/`raw_has_bracket`/`streak` | | `summarizePeakHours` | `""` | 高峰时段串(`"09:00-18:00"` / `"mon-fri 08:00-12:00,14:00-18:00"`,支持跨零点;全角逗号/冒号/数字自动归一,#424) | 命中高峰不调 LLM、顺延补跑;任一写法非法整串按未配置处理。**非法不静默(#622)**:非空且解析失败时启动日志 warn 一条(带键名与原串)、面板该键输入行下出红色提示(`/features` 的 `invalidPeakHours`);空串是合法的未配置,不告警 | | `summarizePeakMaxDeferMinutes` | `120` | 高峰顺延上限(分钟,≤1440) | 0 = 不设上限;到点仍处高峰照跑(bypassPeak) | +| `distillSkipImportedSessions` | `true` | 导入的 DeepSeek 会话(`source.provider === "deepseek-chat-export"`)不进自动蒸馏 | 导入会话没有蒸馏游标,继续说话会把整段历史当新窗口烧一次蒸馏——默认挡掉;判定读事件([backfill](../dsh-mneme/) 同源的 `isImportedSession`),窗口保留、游标不消费、留 `skipped/imported-session` 审计。要进库走显式回填 `POST /api/dsh-mneme/maintenance/backfill`(回填自带 `bypassImported` 旁路) | +| `distillExcludeProjects` | `[]` | 按项目目录排除自动蒸馏:会话 `cwd` 前缀命中(含同一目录与子路径)即不进库 | 覆盖该目录之后新建的会话,不用逐条加 id;留 `skipped/excluded-project` 审计。与 `sessionGateEnabled`(逐会话无痕,连注入都不给)分层:这里只管「蒸馏进不进库」 | | `summarizeDedupeMode` | `off` | 落库前去重档位:`off` / `title`(拦完全同名)/ `vector`(同会话语义近邻) | `vector` 复用已有 embedding 列,无 LLM 调用 | | `summarizeDedupeMinSim` | `0.92` | `vector` 档并入阈值(0.5–0.99) | 仅 `vector` 档生效 | | `summarizeDedupeWindowHours` | `24` | `vector` 档回看窗口(小时,≤168) | 仅 `vector` 档生效 | diff --git a/dsh-mneme/lib/api.js b/dsh-mneme/lib/api.js index 22ed2189..ab97f270 100644 --- a/dsh-mneme/lib/api.js +++ b/dsh-mneme/lib/api.js @@ -255,7 +255,7 @@ function parseBody(text) { // 不传时日记路由一律 503 diary-unavailable(装配缺件),而不是伪装成「用户没开」。 // 与 choro 同一套 503/409 分叉:#497 定下「503 = 环境缺件,409 = 用户没开」。 // 顺序是刻意的:kb 先占第 13 位(它先合进 main),diary 追加在其后,谁先合谁靠前。 -export function createApi(ctx, service, settings, commands, embedder, semantic = null, apiToken = "", config = null, maintenance = null, profileExtract = null, choro = null, facts = null, kb = null, diary = null) { +export function createApi(ctx, service, settings, commands, embedder, semantic = null, apiToken = "", config = null, maintenance = null, profileExtract = null, choro = null, facts = null, kb = null, diary = null, backfill = null) { const disposers = []; // #182 注入状态探测:面板状态页的「极简模式下注入按宿主设计关闭」提示需要 @@ -1487,6 +1487,66 @@ export function createApi(ctx, service, settings, commands, embedder, semantic = } }); + // --- maintenance/backfill(导入会话批量回填,第一期只有手动入口)---------- + // 为什么是手动入口:一次几百条会话、每条一次 LLM 调用,绝不能在插件加载或面板 + // 挂载时自己跑起来。为什么 **默认 dryRun=true**:dry-run 一个 token 都不花,只回 + // 「扫到多少条导入会话、多少条超窗、要跑几次调用」——先看数字是这类批量任务唯一 + // 安全的开场(与 #275 的回收同一口径:手动 + dry-run + receipt)。 + // + // 真跑也走同一条路由(body `{"dryRun": false}`):写入口径(每窗最多 5 条、同一套 + // 去重、同一个游标)与 dry-run 的筛选是同一份代码,不会出现「预估 50 实跑 500」。 + register({ + kind: "exact", + path: "/api/dsh-mneme/maintenance/backfill", + // `token` 而不是 `token-write`:这条路径没有读分支(非 POST 直接 404), + // 而 `token-write` 会让 GET/HEAD 免 token——那在这里只是给文档制造一句 + // 「唯一开放的是 405/404」的废话。与 `/profile-facts/run` 同一类:一次 + // 花模型额度的维护轮,无读可开放。 + policy: "token", + handler(req, res) { + try { + if (req.method !== "POST") { + sendJson(res, 404, { error: "not-found" }); + return; + } + return readBody(req, res).then(async (text) => { + if (text === null) return; + const body = parseBody(text); + if (!backfill || typeof backfill.run !== "function") { + sendJson(res, 503, { error: "backfill-unavailable" }); + return; + } + // 先 floor 再判:0.5 这类小数 floor 成 0,而 0 在 planBackfill 里是 + // 「不限制」(slice(0, 0) 的语义相反),一次手滑就能把 dry-run 变成全量跑。 + // 下界因此钉在 1,宁可多跑一条也不放行 0。 + const positive = (value, fallback) => { + if (!Number.isFinite(value)) return fallback; + const floored = Math.floor(value); + return floored >= 1 ? floored : fallback; + }; + try { + const result = await backfill.run({ + dryRun: body.dryRun !== false, + limit: positive(body.limit, 50), + days: positive(body.days, 90), + rounds: positive(body.rounds, 20), + maxEntries: positive(body.maxEntries, 5) + }); + // 「一条都没跑」不该是 200:no-distill-route / summarizer-unavailable + // 都是「服务端现在干不了这活」,调用方按 200 记成功会以为回填过。 + const refused = result?.error === "no-distill-route" || result?.error === "summarizer-unavailable"; + sendJson(res, refused ? 503 : 200, result); + } catch (error) { + ctx.logger?.warn?.(`dsh-mneme: backfill failed: ${String(error)}`); + sendJson(res, 500, { error: "internal" }); + } + }); + } catch { + sendJson(res, 500, { error: "internal" }); + } + } + }); + // --- rules --- register({ kind: "exact", diff --git a/dsh-mneme/lib/backfill.js b/dsh-mneme/lib/backfill.js new file mode 100644 index 00000000..78750287 --- /dev/null +++ b/dsh-mneme/lib/backfill.js @@ -0,0 +1,262 @@ +// 导入会话的批量回填(DeepSeek APP 导出 → DSH 会话,一次几百条)。 +// +// 为什么需要它:DSH 0.2.2-rc.3 的 @deepseek-ai/dsh-deepseek-chat-import 只把导出的 +// 会话写成**普通 DSH 会话**——不做压缩、不做摘要、header 里连一个「导入」标记都不写 +// (插件 lib/index.js:1355-1377 的 writeSession 只有 version/id/createdAt/cwd/ +// isSeeded)。蒸馏链只在新的 `turn/end` 事件上被唤醒(src/summarize.js:1620-1627), +// 历史事件永远不会再触发它,于是这些会话永远不会进记忆库,几百条会话就是几百条 +// 空白的 turn——这正是维护者 2026-11-14 说的「又没压缩又要做记忆整理,很麻烦」。 +// +// 边界(维护者 2026-11-14 拍板): +// - 不碰注入:注入是读侧,回填只写库。 +// - 不占 dream 预算:dream 与蒸馏各有一套 run 表与节流键,这里不碰 dream 的任何计数。 +// - 不新建表:`distill_cursors`(按 session.id 记消费到哪)、`distill_records` +// (主键 (session_id, seq) 吃重跑)、`distill_runs`(run_type 已留 manual 槽) +// 三张现成的就够断点续跑与去重——重跑安全这条比什么都重要。 +// - 不加第二套抽取语义:沿用同一条 summarize 链、同一份提示词、同一套去重, +// 只把「每窗条数」压到 5 条(`summarizeMaxEntriesPerRun` 的同一条闸门)。 +// +// 入口形态(第一期):手动触发 + dry-run 优先。dry-run 一个 token 都不花,只报数字: +// 扫到多少条导入会话、多少条超出窗口、估算要跑多少次调用。面板按钮第二期再说。 +import { readFile } from "node:fs/promises"; + +import { findSessionFiles, forEachFrame, resolveSessionDir } from "./session-usage.js"; +// 判据本身住在 imported-session.js:蒸馏(src/summarize.js)也要用它,而那条路径 +// 每轮 turn/end 都会走一次,不该为了一个纯函数把本文件的 fs 扫描链拖进热路径。 +// 这里保留 export 是为了不打断既有调用方与测试的 import 路径。 +import { IMPORT_ERROR_PREFIX, IMPORT_PROVIDER, isImportedSession } from "./imported-session.js"; + +export { IMPORT_ERROR_PREFIX, IMPORT_PROVIDER, isImportedSession }; + +export const DEFAULT_LIMIT = 50; +export const DEFAULT_DAYS = 90; +export const DEFAULT_ROUNDS = 20; +export const DEFAULT_MAX_ENTRIES = 5; + +/** + * 只保留最近 `maxTurns` 轮(`turn/start` 起算),更早的事件整段丢掉。 + * + * 为什么按轮不按条:一轮的边界是 `turn/start`…`turn/end`,按条切会把工具调用和 + * 它的结果劈开——那比少截一点更糟。`maxTurns <= 0` 表示不截。 + */ +export function recentTurns(events, maxTurns = DEFAULT_ROUNDS) { + const list = Array.isArray(events) ? events : []; + if (!(maxTurns > 0)) return list; + const starts = []; + for (let index = 0; index < list.length; index += 1) { + if (list[index]?.type === "turn/start") starts.push(index); + } + if (starts.length <= maxTurns) return list; + return list.slice(starts[starts.length - maxTurns]); +} + +/** + * 把一帧帧解码出来的 JSONL 文本拼成事件数组。 + * + * 磁盘上的事件**带 seq**(实测 19/20 条有;`session` header 那一条没有),所以优先 + * 用文件里的 seq,缺了就按行号补——蒸馏链只认 `Number.isSafeInteger(event.seq)` + * (src/summarize.js:380-382),没有 seq 的事件会被当「无法证明是新增」而丢掉。 + */ +export function parseSessionTexts(texts) { + const events = []; + let index = 0; + for (const text of texts) { + for (const raw of String(text).split("\n")) { + const line = raw.trim(); + if (!line) continue; + let event; + try { + event = JSON.parse(line); + } catch { + // 末帧半行是常态(进程被杀/断电),跳过它就是「宁可少一条,不要错一条」。 + continue; + } + if (!event || typeof event !== "object") continue; + if (!Number.isSafeInteger(event.seq)) event.seq = index; + events.push(event); + index += 1; + } + } + return events; +} + +/** + * 读一条会话日志:事件、header、以及这一条最后的事件序。 + * + * `frames`/`failed` 要带回去——解不出来的帧数不为 0 时,这条会话的「已消费」判断 + * 本来就不可靠,上层得能看见。 + */ +export async function readSessionFile(file) { + const buffer = await readFile(file); + const texts = []; + const { frames, failed } = forEachFrame(buffer, (text) => texts.push(text)); + const events = parseSessionTexts(texts); + const header = events.find((event) => event?.type === "session") ?? null; + const lastSeq = events.length > 0 ? events[events.length - 1].seq : -1; + return { file, header, events, frames, failed, lastSeq }; +} + +/** + * 扫一遍会话目录,给出「要跑哪些、为什么跳过」的账。 + * + * 不调模型、不写库:`dryRun` 的每一条数字都来自这里,所以 dry-run 与真跑的 + * **筛选口径必须是同一份代码**——两套筛选是这类批量任务最常见的谎。 + * + * @returns {Promise} 见下方 return 里的字段名。 + */ +export async function planBackfill({ + service = null, + dshHome = undefined, + sessionDir = undefined, + now = Date.now(), + days = DEFAULT_DAYS, + limit = DEFAULT_LIMIT, + rounds = DEFAULT_ROUNDS, + config = null +} = {}) { + const dir = sessionDir ?? resolveSessionDir({ dshHome }); + const scanned = findSessionFiles(dir); + const cutoff = now - days * 86400000; + const plan = { + sessionDir: dir, + present: scanned.present, + truncated: scanned.truncated, + unreadable: scanned.unreadable.length, + filesScanned: scanned.files.length, + importedSessions: 0, + outsideWindow: 0, + noCreatedAt: 0, + alreadyDistilled: 0, + unreadableFiles: 0, + pending: 0, + candidates: [], + limit, + days, + rounds + }; + const candidates = []; + for (const file of scanned.files) { + let reading; + try { + reading = await readSessionFile(file); + } catch { + plan.unreadableFiles += 1; + continue; + } + if (!isImportedSession(reading.events)) continue; + plan.importedSessions += 1; + const createdAt = Number.isFinite(reading.header?.createdAt) ? reading.header.createdAt : null; + if (createdAt === null) { + plan.noCreatedAt += 1; + continue; + } + if (createdAt < cutoff) { + plan.outsideWindow += 1; + continue; + } + const sessionId = typeof reading.header?.id === "string" && reading.header.id + ? reading.header.id + : null; + if (!sessionId) { + plan.unreadableFiles += 1; + continue; + } + // 断点续跑:游标追上最后一条事件就不再跑(重跑安全靠这张表,不靠内存)。 + const cursor = typeof service?.getDistillCursor === "function" + ? service.getDistillCursor(sessionId)?.last_seq + : undefined; + if (Number.isSafeInteger(cursor) && cursor >= reading.lastSeq) { + plan.alreadyDistilled += 1; + continue; + } + candidates.push({ + file, + sessionId, + createdAt, + lastSeq: reading.lastSeq, + cursor: Number.isSafeInteger(cursor) ? cursor : null, + frames: reading.frames, + failed: reading.failed + }); + } + // 时间倒序:先处理最近的,预算砍掉时损失最小。 + candidates.sort((left, right) => right.createdAt - left.createdAt); + plan.pending = candidates.length; + plan.candidates = candidates.slice(0, limit > 0 ? limit : candidates.length); + // 一次会话一次调用(窗口都压在一轮里),所以调用数就是候选数。 + const routeConfigured = Boolean(config?.summarizeProvider && config?.summarizeModel); + plan.routeConfigured = routeConfigured; + plan.callsPlanned = routeConfigured ? plan.candidates.length : 0; + return plan; +} + +/** + * 真跑(或只跑 dry-run)。 + * + * `summarizeNow` 由 `createSummarizer` 暴露,方法体是薄薄一层:拿 `snapshotSessionEvents` + * 组 `{id, events}` 喂进现有链路——**这里不复制任何蒸馏逻辑**。 + * + * @param {object} options `{ service, summarizer, config, logger, dryRun, limit, days, rounds, sessionDir, dshHome, now, maxEntries }` + * @returns {Promise} dry-run 回 `{ dryRun: true, plan }`;真跑回逐会话结果。 + */ +export async function runBackfill({ + service = null, + summarizer = null, + config = null, + logger = null, + dryRun = true, + sessionDir = undefined, + dshHome = undefined, + now = Date.now(), + days = DEFAULT_DAYS, + limit = DEFAULT_LIMIT, + rounds = DEFAULT_ROUNDS, + maxEntries = DEFAULT_MAX_ENTRIES +} = {}) { + const plan = await planBackfill({ service, dshHome, sessionDir, now, days, limit, rounds, config }); + if (dryRun) return { dryRun: true, plan }; + // 没配蒸馏路由时真跑没有意义:plan 已经把 `callsPlanned` 报成 0,真跑却仍会逐条 + // 调 summarizeNow,每条都因缺 provider/model 失败——50 条会话就是 50 条 failed 加 + // 50 行日志,读起来像「批量任务坏了」而不是「你没配模型」。按 plan 的同一判据直接 + // 拒绝,一条都不跑(dry-run 不受影响,它本来就不调模型)。 + if (!plan.routeConfigured) { + return { dryRun: false, plan, error: "no-distill-route", ran: 0, failed: 0, results: [] }; + } + if (typeof summarizer?.summarizeNow !== "function") { + return { dryRun: false, plan, error: "summarizer-unavailable", results: [] }; + } + const results = []; + for (const candidate of plan.candidates) { + let reading; + try { + reading = await readSessionFile(candidate.file); + } catch (error) { + results.push({ sessionId: candidate.sessionId, status: "unreadable", error: String(error?.message ?? error) }); + continue; + } + const windowed = recentTurns(reading.events, rounds); + const trigger = windowed[windowed.length - 1]; + if (!trigger) { + results.push({ sessionId: candidate.sessionId, status: "empty" }); + continue; + } + try { + await summarizer.summarizeNow( + // header 必须一起给:项目排除门卫读的是 session.header.cwd(创建时冻结 + // 的那份),只带 id + events 的话回填路径上 distillExcludeProjects 永远 + // 不生效——那正是「导入会话不该进库」最该守的一条。 + { id: candidate.sessionId, header: reading.header, events: windowed }, + trigger, + // bypassImported:回填喂的正是导入会话,必须显式旁路保护层 + // (src/summarize.js 的 imported-session 门卫),否则被自己的墙挡死。 + { bypassPeak: true, bypassImported: true, maxEntries } + ); + results.push({ sessionId: candidate.sessionId, status: "ran", events: windowed.length }); + } catch (error) { + logger?.warn?.(`dsh-mneme: backfill ${candidate.sessionId} failed: ${String(error)}`); + results.push({ sessionId: candidate.sessionId, status: "failed", error: String(error?.message ?? error) }); + } + } + const ran = results.filter((item) => item.status === "ran").length; + return { dryRun: false, plan, ran, failed: results.length - ran, results }; +} \ No newline at end of file diff --git a/dsh-mneme/lib/config.js b/dsh-mneme/lib/config.js index 4cb7dcca..187fe7b8 100644 --- a/dsh-mneme/lib/config.js +++ b/dsh-mneme/lib/config.js @@ -31,6 +31,25 @@ export const Config = z.object({ // 蒸馏把完整对话上下文交给 LLM 提炼,不硬裁到 8000 字就截断语义;默认 // 24000 字符(约覆盖一整轮中等对话),需要更完整可调大。 distillMaxChars: z.natural().min(1000).max(200000).default(24000), + // 隔离(不入库):默认不让**导入器写的会话**进记忆库。 + // + // 为什么需要它:导入器(@deepseek-ai/dsh-deepseek-chat-import)把 APP 导出的 + // 会话写成普通 DSH 会话,不压缩、不摘要、header 也不留标记,只在每条 assistant + // 消息上盖 provider 戳。蒸馏只在新 `turn/end` 唤醒,所以「导进来」本身不进库—— + // 但用户哪天在那种会话里继续说话,那个会话没有蒸馏游标(previousSeq 为空), + // snapshotSessionEvents 在 afterSeq 为空时只按 throughSeq 截断,**整段历史会被 + // 当成新窗口**(尾部 distillMaxChars)拿去蒸馏。「先导进去、以后再说」这个中间 + // 态因此并不安全,默认挡掉正是为了它。 + // + // 要进库走回填(src/backfill.js,显式点名会话,不经过这条路径)。 + distillSkipImportedSessions: z.boolean().default(true), + // 按项目目录排除蒸馏:cwd 前缀命中即不进库,整条项目线(含之后新建的会话)一次 + // 挡在外面,不用逐条会话加 id。前缀按 `/` 归一后比较,命中含等号(同一目录)与 + // 目录下所有子路径。空数组 = 不排除任何项目(默认行为不变)。 + // + // 与上面的导入排除、与 sessionGateEnabled(逐会话无痕)分层:无痕是「这条会话对 + // 记忆系统整个不可见」(注入也不给),这里只管「蒸馏进不进库」。 + distillExcludeProjects: z.array(z.string()).default([]), // 图像描述档(docs/DISTILL.md:346):蒸馏前先用一个**多模态模型**给窗口里的 // 每张图写一段描述,并把描述并进转录——图本身仍然只是 `[图: …]` 占位,原图 // 不落库、不读字节(#611 的口径不变,这里只补「图里是什么」这段文字)。 diff --git a/dsh-mneme/lib/distill-guard.js b/dsh-mneme/lib/distill-guard.js new file mode 100644 index 00000000..fa50af8b --- /dev/null +++ b/dsh-mneme/lib/distill-guard.js @@ -0,0 +1,52 @@ +// 2026-11-18 保护层接线(#688 续):导入会话与排除项目的蒸馏门卫。 +// 设计要点: +// 1. 项目排除放在门卫区(route 解析之后、间隔门之前)——被挡的会话连路由都不该解析。 +// 2. backfill(src/backfill.js)走 summarizeNow 且必须透传 bypassImported: true, +// 否则回填被自己的保护层挡死。opts 显式旁路,与 bypassPeak 同形。 +// 3. 判据只读会话上已有的事件/header:导入判定由 src/imported-session.js 的纯函数 +// 负责(summarize.js 复用 collectMessages 已读好的窗口,不额外快照)。 +// 4. cwd 读 session.header?.cwd(创建时冻结的那份,不能读 requestHeader—— +// scope.js 踩过的坑,pipeline-runner.js:57-62 projectRootOf 同口径)。 +// 5. 前缀比较按平台分隔符归一后 startsWith;大小写不敏感(Windows 盘符 +// C:/c、D:/d 都会出现)。 + +/** + * 把一段路径归一成可比较形态:统一 / 与 \、剥尾部斜杠、小写。 + * 非字符串或纯空白返回 null(调用方跳过)。 + * + * **文件系统根是有效条目,不当作空值丢掉**:名单里写 `/` 的意思是「全挡」, + * 剥尾后不能变成 null 让整条规则失效(那是 CodeRabbit 在 #755 抓到的洞)。 + * 盘符根(`C:\`)剥尾后是 `c:`,它作为前缀本身就能命中 `c:/...`,无需特判。 + */ +function normalizeProjectPath(raw) { + if (typeof raw !== "string") return null; + // 连续分隔符顺手折叠:手写配置里 `/work//private` 与 `/work/private` 是同一个目录, + // 折叠后比较才不会因为一个多打的斜杠漏挡。 + const unified = raw.trim().replace(/[\\/]/g, "/").replace(/\/{2,}/g, "/"); + if (!unified) return null; + const stripped = unified.replace(/\/+$/, ""); + // 剥干净了说明它本身就是根("/")——保留,不要掉成 null。 + if (!stripped) return "/"; + return stripped.toLowerCase(); +} + +/** + * 排除名单命中判定:名单项归一后,cwd 必须等于它或是它的子路径。 + * 名单项里带通配符/正则一律按字面路径处理(不展开语义)。 + * + * 根目录(`/`)是「整机全挡」,任何绝对路径都算它的后代;否则按 + * `target + "/"` 做前缀比较,`private-public` 不会被 `private/` 误伤。 + */ +function isExcludedProject(sessionCwd, excludeList) { + const cwd = normalizeProjectPath(sessionCwd); + if (!cwd || !Array.isArray(excludeList) || excludeList.length === 0) return false; + for (const item of excludeList) { + const target = normalizeProjectPath(item); + if (!target) continue; + if (target === "/") return true; // 根目录:全挡 + if (cwd === target || cwd.startsWith(target + "/")) return true; + } + return false; +} + +export { isExcludedProject, normalizeProjectPath }; diff --git a/dsh-mneme/lib/imported-session.js b/dsh-mneme/lib/imported-session.js new file mode 100644 index 00000000..d3d1426c --- /dev/null +++ b/dsh-mneme/lib/imported-session.js @@ -0,0 +1,40 @@ +// 「这条会话是导入器写的」判据的唯一定义处。 +// +// 单独成文件、且不 import 任何别的东西:两个消费方的冷热差别很大——回填 +// (src/backfill.js)本来就要拉 fs 与 session-usage,而蒸馏(src/summarize.js) +// 在每一轮 turn/end 都被调用。判定函数不该把整条扫描链带进热路径。 + +/** 导入器给每条 assistant 消息盖的 provider 戳(插件 lib/index.js:181-185、621-633)。 */ +export const IMPORT_PROVIDER = "deepseek-chat-export"; +/** 导入器给「这条回复进不了模型历史」的会话留的错误码前缀(插件 lib/index.js:191-195)。 */ +export const IMPORT_ERROR_PREFIX = "DEEPSEEK_EXPORT_"; + +/** + * 这一组事件是不是导入器写的会话。 + * + * **纯函数**:输入事件数组,输出 bool,不读盘、不看状态。以后 DSH 改了戳法, + * 只改这一个函数(与它的一条测试),不用翻扫描循环。 + * + * 两条信号取并集,主判据是消息 source(DSH 自己从不产生这个 provider 字符串); + * `turn/end` 的错误码只用来兜「整条会话的回复全被排除、一条 assistant/message + * 都没有」的那种会话(TEMPLATE_RESPONSE / EMPTY_REPLY / NO_REPLY / WIP / + * FAILED / TIMEOUT / CONTEXT_LENGTH_EXCEEDED / INCOMPLETE)。 + * + * 不用 header、不用 session/title:前者没有标记,后者正常会话同样有。 + */ +export function isImportedSession(events) { + if (!Array.isArray(events)) return false; + for (const event of events) { + if (!event || typeof event !== "object") continue; + if (event.type === "assistant/message") { + const provider = event.data?.message?.source?.provider; + if (provider === IMPORT_PROVIDER) return true; + continue; + } + if (event.type === "turn/end") { + const code = event.data?.reason?.error?.code; + if (typeof code === "string" && code.startsWith(IMPORT_ERROR_PREFIX)) return true; + } + } + return false; +} \ No newline at end of file diff --git a/dsh-mneme/lib/index.js b/dsh-mneme/lib/index.js index bcb62375..d21eee63 100644 --- a/dsh-mneme/lib/index.js +++ b/dsh-mneme/lib/index.js @@ -28,6 +28,7 @@ import { createStandaloneApi } from "./api-standalone.js"; import { registerStandaloneApi, clearStandaloneApi } from "./standalone-registry.js"; // #275 存储生命周期第一批:无损回收(手动入口,不挂启动路径)。 import { createMaintenance } from "./maintenance.js"; +import { runBackfill } from "./backfill.js"; import { createChoroQueries } from "./wiki/queries.js"; // chorography M3:开局注入块与 /choro 只读回执。两者都只吃 queries 读侧,不写任何东西。 import { createChoroInject } from "./wiki/inject.js"; @@ -738,6 +739,20 @@ export const apply = (ctx, config) => { const summarizer = createSummarizer(ctx, service, cfg, {}, sessionGate); disposers.push(summarizer.dispose); + // 导入会话批量回填(DSH 0.2.2-rc.3 的 DeepSeek 导出导入,一次几百条会话)。 + // 复用**同一个** summarizer:不另建一套蒸馏链、不碰 dream 的预算与节流键, + // 只借它的 `summarizeNow`(薄薄一层,把会话样对象喂进现有链路)。构造期零副作用, + // 只有手动入口(面板 API 的 POST /maintenance/backfill)会调 `run`。 + const backfill = { + run: (options = {}) => runBackfill({ + service, + summarizer, + config: cfg, + logger: ctx.logger, + ...options + }) + }; + // QQ 日记本(v0.13,diaryEnabled):构造期零副作用(只闭包住依赖,不建目录、不读盘、 // 不调模型),所以位置可以放在这里——它必须早于下面的 createApi,面板的 // 「写一篇」按钮要转调同一个 composer;而自动触发侧在 webServer 块之后挂(同 @@ -800,7 +815,7 @@ export const apply = (ctx, config) => { add: () => { throw new Error("commands unavailable"); }, remove: () => false, list: () => [] - }, embedder, { ...semantic, vectorIndex, reranker }, cfg.apiToken, { ...cfg, memoryDir }, maintenance, extractProfileDraft, choro, profileFacts, kb, diary); + }, embedder, { ...semantic, vectorIndex, reranker }, cfg.apiToken, { ...cfg, memoryDir }, maintenance, extractProfileDraft, choro, profileFacts, kb, diary, backfill); disposers.push(api.dispose); // dsh-go 原生面板:只有宿主提供 dshGo 服务(dsh-go 手机 App)时才挂载。 diff --git a/dsh-mneme/lib/settings.js b/dsh-mneme/lib/settings.js index edde6891..cfee8a0f 100644 --- a/dsh-mneme/lib/settings.js +++ b/dsh-mneme/lib/settings.js @@ -123,6 +123,17 @@ function serializePersonaStore(store) { // 3. 写入是逐键校验的合并写:未知键/类型/范围不符抛 TypeError(消息含键名, // 供 API 透传给前端定位),校验不通过不落库,坏值永远进不了 kv。 const FEATURE_FLAG_BOOLEANS = [ + // #688 保护层:导入会话的蒸馏门卫。面板可关、默认挡——导入会话无游标, + // 继续说话会把整段历史当新窗口烧一次蒸馏;要进库走 backfill 显式路径 + // (POST /maintenance/backfill,bypassImported 旁路)。 + // + // 同一层的 `distillExcludeProjects`(项目排除名单)**刻意不在这张表**: + // 它是字符串数组,而本表只收 boolean(validateFlag 见到非布尔直接抛)。 + // 硬塞进来会被读成「开关」——写 ["/x"] 抛 TypeError、写 false 却存得下, + // 重启后名单变成一个假的关闭状态。它走「仅配置文件」那条路(config.js + // 声明、面板改不到),与 codingKeywords / sessionGateSessions 同口径。 + "distillSkipImportedSessions", + "autoInject", "autoSummarize", "hotMemoryEnabled", diff --git a/dsh-mneme/lib/summarize.js b/dsh-mneme/lib/summarize.js index e0c66168..da81e8c9 100644 --- a/dsh-mneme/lib/summarize.js +++ b/dsh-mneme/lib/summarize.js @@ -9,6 +9,8 @@ import { injectChildEnabled } from "./config.js"; import { withEffortFallback, describeStreamFailure, EFFORT_REJECT_RE } from "./dream.js"; // v0.9.0(M.O. 项目记忆):蒸馏写入复用工具路径那把作用域解析器,见 createSummarizer。 import { createScopeResolver, isHardWallWorkspace, resolveScopeStamp } from "./scope.js"; +import { isImportedSession } from "./imported-session.js"; +import { isExcludedProject } from "./distill-guard.js"; // #493 补充 §③:断链与巩固侧同一份实现(src/fence.js)。转录里出现闭合标记时 // 原先能提前闭合围栏——那是把用户消息升格成指令最短的一条路。 import { breakFenceMarkers } from "./fence.js"; @@ -560,8 +562,13 @@ function describeAuditPatch(stats) { return out; } -async function collectMessages(session, maxChars = 8000, language = "zh", afterSeq, throughSeq, describePart) { +async function collectMessages(session, maxChars = 8000, language = "zh", afterSeq, throughSeq, describePart, mutesDescribe = false) { const events = snapshotSessionEvents(session, afterSeq, throughSeq); + // #755 复审(CodeRabbit 第 7 条):图像描述是逐 part 真的调一次模型。被导入门挡下 + // 的会话在门卫之前就要走完整个 collectMessages,于是「挡掉蒸馏」反而先付了一轮 + // 视觉调用。判定就地复用刚读好的 events(不再 snapshot 第二次),命中就让 + // describePart 这一段不跑,转录退回纯文本——与「describePart 缺席」逐字节一致。 + const describeMuted = mutesDescribe === true && isImportedSession(events); const lines = []; const seenSubagentMessages = new Set(); // Issue #611:这一窗里被折成占位的图片张数。占位本身不丢信息,但「折了几张」 @@ -624,7 +631,7 @@ async function collectMessages(session, maxChars = 8000, language = "zh", afterS // 图像描述档(docs/DISTILL.md:346):开关开着且这一 part 真的带图时,才 // 去出描述。describePart 由调用方注入(默认 undefined)——本函数不认识 // 任何路由配置,只负责「文本最后长什么样」,这样单测可以只塞一个桩进去。 - const described = typeof describePart === "function" && part.blocks?.length + const described = !describeMuted && typeof describePart === "function" && part.blocks?.length ? await describePart(part) : part.text; if (part.role === "user") { @@ -692,6 +699,9 @@ async function collectMessages(session, maxChars = 8000, language = "zh", afterS : [], hasEvents: events.length > 0, lastSeq, + // #688 保护层复用:门卫要判「这是不是导入会话」,直接拿这份已读好的窗口, + // 不再单独 snapshot 一遍(每轮 turn/end 多一次全量读)。 + events, // Issue #611:这一窗折成占位的图片张数。调用方把它写进审计 metadata, // 让「图进来了几张」成为可对账的事实而不是推断。 imageCount, @@ -797,7 +807,6 @@ function enqueueDistill(task, intervalMs = 0) { import { parsePeakSpec, isInPeakWindow, nextOffPeakAt } from "./peak-hours.js"; export { parsePeakSpec, isInPeakWindow, nextOffPeakAt }; export function createSummarizer(ctx, service, config, deps = {}, sessionGate = null) { - if (!config.autoSummarize) return { dispose: () => {} }; // Issue #239(第 4 项):时钟与定时器可注入——排程不绑死真实时钟,测试才能确定性 // 覆盖「高峰顺延 → 非高峰补跑」。房型同 dream/sleep.js 的注入参数。 @@ -949,6 +958,27 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = ? { provider: header.provider, model: header.model } : undefined; + // 2026-11-18(#688 续,保护层):两道确定性门卫,放在间隔门之前——被挡的会话 + // 连「刚刚跑过」的间隔点都不该占,也不该进入后面任何一条链。 + // + // ① 项目排除(distillExcludeProjects,默认 []):cwd 只读 session.header + // (创建时冻结的那份)。前缀命中即整条项目线挡在库外。 + // + // ② 导入会话隔离(distillSkipImportedSessions,默认 true)。为什么判事件不判 + // header:导入器不留 header 标记,唯一信号在消息 source 上(src/imported- + // session.js)。判定成本是一次快照读取,比一次 LLM 调用便宜三个数量级。 + // opts.bypassImported 由回填(backfill.js)显式传:喂给 summarizeNow 的恰恰 + // 是导入会话,保护层不能挡自己的合法调用方(与 bypassPeak 同形)。 + if (Array.isArray(config.distillExcludeProjects) && config.distillExcludeProjects.length > 0 + && isExcludedProject(session.header?.cwd, config.distillExcludeProjects)) { + writeAudit({ + timestamp: new Date(now()).toISOString(), + model_id: route ? route.provider + ":" + route.model : "unknown", + status: "skipped", + error_message: "excluded-project" + }); + return; + } // Issue #127 间隔门:同一会话两次蒸馏之间的最小间隔(0 = 现状,不限)。进入 // 即打点——失败/degraded 的 run 也占用间隔,与 dreamMinIntervalMinutes 语义 // 一致,防止失败连发。被挡下的 turn/end 静默跳过,但留一行 status='skipped' @@ -1077,12 +1107,33 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = langOf(config), previousSeq, triggerSeq, - describePart + describePart, + // 与下面那道导入门同一个条件:门会挡下的会话,这里就不该先花一轮描述。 + config.distillSkipImportedSessions !== false && opts.bypassImported !== true ); const hasNewEvents = collected.hasEvents || (triggerSeq !== undefined && (previousSeq === undefined || triggerSeq > previousSeq)); if (!hasNewEvents) return; + // #688 保护层②:导入会话隔离(distillSkipImportedSessions,默认 true)。 + // 放在窗口读之后而不是门卫区:判定直接复用 collectMessages 刚读好的那份 + // 事件(collected.events),不给每轮 turn/end 增加第二次全量快照。判定读 + // 事件不判 header——导入器不留 header 标记,唯一信号在消息 source 上 + // (src/imported-session.js)。游标刻意不消费:与 peak/max-runs 同口径, + // 「这个会话先不再蒸馏了」,窗口留在原地;要进库走回填(backfill.js 显式 + // 点名会话并透传 opts.bypassImported,与 bypassPeak 同形)。 + if (config.distillSkipImportedSessions !== false + && opts.bypassImported !== true + && isImportedSession(collected.events)) { + writeAudit({ + timestamp: new Date(now()).toISOString(), + model_id: route ? route.provider + ":" + route.model : "unknown", + status: "skipped", + error_message: "imported-session" + }); + return; + } + const nextSeq = Math.max(collected.lastSeq ?? Number.NEGATIVE_INFINITY, triggerSeq ?? Number.NEGATIVE_INFINITY); if (!collected.messages.length) { // 没有可蒸馏的公开文本也算成功消费当前事件窗口,避免每个 turn/end @@ -1354,7 +1405,10 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = const parsed = parsedResult.entries; // Issue #127 条数上限:0 = 不限(现状)。被截断的条数与去重命中数一并进 // 审计 metadata——此前产出条数完全由模型输出决定,用户无从自查。 - const cap = config.summarizeMaxEntriesPerRun ?? 0; + // `opts.maxEntries` 是给「导入会话批量回填」用的单窗覆盖(src/backfill.js): + // 回填要的是「一个会话最多留几笔」,不该改全局配置、更不该造第二套抽取语义, + // 所以复用同一条上限闸门、只换这一次的数值。不传时逐字等于旧行为。 + const cap = opts.maxEntries ?? config.summarizeMaxEntriesPerRun ?? 0; const entries = cap > 0 ? parsed.slice(0, cap) : parsed; const capped = parsed.length - entries.length; const dedupeMode = config.summarizeDedupeMode ?? "off"; @@ -1679,7 +1733,11 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = } } - const unsubscribe = ctx.on("session/event", (session, event) => { + // CodeRabbit(#755):autoSummarize 关掉时只关掉**自动**订阅,不再整个函数早退。 + // 早退会让 summarizeNow 一起消失,而回填(backfill.js)正是靠它手动跑—— + // 「关掉自动蒸馏」不该等于「导入会话永远补不上记忆」。 + const autoEnabled = config.autoSummarize !== false; + const unsubscribe = !autoEnabled ? null : ctx.on("session/event", (session, event) => { if (disposed || event.type !== "turn/end") return; // Return the summarization promise so awaiters observe the writes; the // catch keeps listener dispatch from rejecting. Dispose-initiated aborts @@ -1691,6 +1749,18 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = }); return { + /** + * 手动跑一次蒸馏——导入会话批量回填的唯一入口(src/backfill.js 调它)。 + * + * 方法体刻意只有一行:**这里不复制任何蒸馏逻辑**,就是把一个「有 id 和 events + * 的会话样对象」喂进同一条 summarize。session 只需要 `id` 与带 `seq` 的 `events` + * (`snapshotSessionEvents` 对没有 `snapshotEvents` 的对象直接读 `.events`), + * `triggerEvent` 也只被用来取 `seq` 当窗口右界;`opts` 透传(`bypassPeak`、 + * `maxEntries`)。 + */ + summarizeNow(session, triggerEvent, opts = {}) { + return summarize(session, triggerEvent, opts); + }, dispose() { if (disposed) return; disposed = true; diff --git a/dsh-mneme/src/api.js b/dsh-mneme/src/api.js index 22ed2189..ab97f270 100644 --- a/dsh-mneme/src/api.js +++ b/dsh-mneme/src/api.js @@ -255,7 +255,7 @@ function parseBody(text) { // 不传时日记路由一律 503 diary-unavailable(装配缺件),而不是伪装成「用户没开」。 // 与 choro 同一套 503/409 分叉:#497 定下「503 = 环境缺件,409 = 用户没开」。 // 顺序是刻意的:kb 先占第 13 位(它先合进 main),diary 追加在其后,谁先合谁靠前。 -export function createApi(ctx, service, settings, commands, embedder, semantic = null, apiToken = "", config = null, maintenance = null, profileExtract = null, choro = null, facts = null, kb = null, diary = null) { +export function createApi(ctx, service, settings, commands, embedder, semantic = null, apiToken = "", config = null, maintenance = null, profileExtract = null, choro = null, facts = null, kb = null, diary = null, backfill = null) { const disposers = []; // #182 注入状态探测:面板状态页的「极简模式下注入按宿主设计关闭」提示需要 @@ -1487,6 +1487,66 @@ export function createApi(ctx, service, settings, commands, embedder, semantic = } }); + // --- maintenance/backfill(导入会话批量回填,第一期只有手动入口)---------- + // 为什么是手动入口:一次几百条会话、每条一次 LLM 调用,绝不能在插件加载或面板 + // 挂载时自己跑起来。为什么 **默认 dryRun=true**:dry-run 一个 token 都不花,只回 + // 「扫到多少条导入会话、多少条超窗、要跑几次调用」——先看数字是这类批量任务唯一 + // 安全的开场(与 #275 的回收同一口径:手动 + dry-run + receipt)。 + // + // 真跑也走同一条路由(body `{"dryRun": false}`):写入口径(每窗最多 5 条、同一套 + // 去重、同一个游标)与 dry-run 的筛选是同一份代码,不会出现「预估 50 实跑 500」。 + register({ + kind: "exact", + path: "/api/dsh-mneme/maintenance/backfill", + // `token` 而不是 `token-write`:这条路径没有读分支(非 POST 直接 404), + // 而 `token-write` 会让 GET/HEAD 免 token——那在这里只是给文档制造一句 + // 「唯一开放的是 405/404」的废话。与 `/profile-facts/run` 同一类:一次 + // 花模型额度的维护轮,无读可开放。 + policy: "token", + handler(req, res) { + try { + if (req.method !== "POST") { + sendJson(res, 404, { error: "not-found" }); + return; + } + return readBody(req, res).then(async (text) => { + if (text === null) return; + const body = parseBody(text); + if (!backfill || typeof backfill.run !== "function") { + sendJson(res, 503, { error: "backfill-unavailable" }); + return; + } + // 先 floor 再判:0.5 这类小数 floor 成 0,而 0 在 planBackfill 里是 + // 「不限制」(slice(0, 0) 的语义相反),一次手滑就能把 dry-run 变成全量跑。 + // 下界因此钉在 1,宁可多跑一条也不放行 0。 + const positive = (value, fallback) => { + if (!Number.isFinite(value)) return fallback; + const floored = Math.floor(value); + return floored >= 1 ? floored : fallback; + }; + try { + const result = await backfill.run({ + dryRun: body.dryRun !== false, + limit: positive(body.limit, 50), + days: positive(body.days, 90), + rounds: positive(body.rounds, 20), + maxEntries: positive(body.maxEntries, 5) + }); + // 「一条都没跑」不该是 200:no-distill-route / summarizer-unavailable + // 都是「服务端现在干不了这活」,调用方按 200 记成功会以为回填过。 + const refused = result?.error === "no-distill-route" || result?.error === "summarizer-unavailable"; + sendJson(res, refused ? 503 : 200, result); + } catch (error) { + ctx.logger?.warn?.(`dsh-mneme: backfill failed: ${String(error)}`); + sendJson(res, 500, { error: "internal" }); + } + }); + } catch { + sendJson(res, 500, { error: "internal" }); + } + } + }); + // --- rules --- register({ kind: "exact", diff --git a/dsh-mneme/src/backfill.js b/dsh-mneme/src/backfill.js new file mode 100644 index 00000000..78750287 --- /dev/null +++ b/dsh-mneme/src/backfill.js @@ -0,0 +1,262 @@ +// 导入会话的批量回填(DeepSeek APP 导出 → DSH 会话,一次几百条)。 +// +// 为什么需要它:DSH 0.2.2-rc.3 的 @deepseek-ai/dsh-deepseek-chat-import 只把导出的 +// 会话写成**普通 DSH 会话**——不做压缩、不做摘要、header 里连一个「导入」标记都不写 +// (插件 lib/index.js:1355-1377 的 writeSession 只有 version/id/createdAt/cwd/ +// isSeeded)。蒸馏链只在新的 `turn/end` 事件上被唤醒(src/summarize.js:1620-1627), +// 历史事件永远不会再触发它,于是这些会话永远不会进记忆库,几百条会话就是几百条 +// 空白的 turn——这正是维护者 2026-11-14 说的「又没压缩又要做记忆整理,很麻烦」。 +// +// 边界(维护者 2026-11-14 拍板): +// - 不碰注入:注入是读侧,回填只写库。 +// - 不占 dream 预算:dream 与蒸馏各有一套 run 表与节流键,这里不碰 dream 的任何计数。 +// - 不新建表:`distill_cursors`(按 session.id 记消费到哪)、`distill_records` +// (主键 (session_id, seq) 吃重跑)、`distill_runs`(run_type 已留 manual 槽) +// 三张现成的就够断点续跑与去重——重跑安全这条比什么都重要。 +// - 不加第二套抽取语义:沿用同一条 summarize 链、同一份提示词、同一套去重, +// 只把「每窗条数」压到 5 条(`summarizeMaxEntriesPerRun` 的同一条闸门)。 +// +// 入口形态(第一期):手动触发 + dry-run 优先。dry-run 一个 token 都不花,只报数字: +// 扫到多少条导入会话、多少条超出窗口、估算要跑多少次调用。面板按钮第二期再说。 +import { readFile } from "node:fs/promises"; + +import { findSessionFiles, forEachFrame, resolveSessionDir } from "./session-usage.js"; +// 判据本身住在 imported-session.js:蒸馏(src/summarize.js)也要用它,而那条路径 +// 每轮 turn/end 都会走一次,不该为了一个纯函数把本文件的 fs 扫描链拖进热路径。 +// 这里保留 export 是为了不打断既有调用方与测试的 import 路径。 +import { IMPORT_ERROR_PREFIX, IMPORT_PROVIDER, isImportedSession } from "./imported-session.js"; + +export { IMPORT_ERROR_PREFIX, IMPORT_PROVIDER, isImportedSession }; + +export const DEFAULT_LIMIT = 50; +export const DEFAULT_DAYS = 90; +export const DEFAULT_ROUNDS = 20; +export const DEFAULT_MAX_ENTRIES = 5; + +/** + * 只保留最近 `maxTurns` 轮(`turn/start` 起算),更早的事件整段丢掉。 + * + * 为什么按轮不按条:一轮的边界是 `turn/start`…`turn/end`,按条切会把工具调用和 + * 它的结果劈开——那比少截一点更糟。`maxTurns <= 0` 表示不截。 + */ +export function recentTurns(events, maxTurns = DEFAULT_ROUNDS) { + const list = Array.isArray(events) ? events : []; + if (!(maxTurns > 0)) return list; + const starts = []; + for (let index = 0; index < list.length; index += 1) { + if (list[index]?.type === "turn/start") starts.push(index); + } + if (starts.length <= maxTurns) return list; + return list.slice(starts[starts.length - maxTurns]); +} + +/** + * 把一帧帧解码出来的 JSONL 文本拼成事件数组。 + * + * 磁盘上的事件**带 seq**(实测 19/20 条有;`session` header 那一条没有),所以优先 + * 用文件里的 seq,缺了就按行号补——蒸馏链只认 `Number.isSafeInteger(event.seq)` + * (src/summarize.js:380-382),没有 seq 的事件会被当「无法证明是新增」而丢掉。 + */ +export function parseSessionTexts(texts) { + const events = []; + let index = 0; + for (const text of texts) { + for (const raw of String(text).split("\n")) { + const line = raw.trim(); + if (!line) continue; + let event; + try { + event = JSON.parse(line); + } catch { + // 末帧半行是常态(进程被杀/断电),跳过它就是「宁可少一条,不要错一条」。 + continue; + } + if (!event || typeof event !== "object") continue; + if (!Number.isSafeInteger(event.seq)) event.seq = index; + events.push(event); + index += 1; + } + } + return events; +} + +/** + * 读一条会话日志:事件、header、以及这一条最后的事件序。 + * + * `frames`/`failed` 要带回去——解不出来的帧数不为 0 时,这条会话的「已消费」判断 + * 本来就不可靠,上层得能看见。 + */ +export async function readSessionFile(file) { + const buffer = await readFile(file); + const texts = []; + const { frames, failed } = forEachFrame(buffer, (text) => texts.push(text)); + const events = parseSessionTexts(texts); + const header = events.find((event) => event?.type === "session") ?? null; + const lastSeq = events.length > 0 ? events[events.length - 1].seq : -1; + return { file, header, events, frames, failed, lastSeq }; +} + +/** + * 扫一遍会话目录,给出「要跑哪些、为什么跳过」的账。 + * + * 不调模型、不写库:`dryRun` 的每一条数字都来自这里,所以 dry-run 与真跑的 + * **筛选口径必须是同一份代码**——两套筛选是这类批量任务最常见的谎。 + * + * @returns {Promise} 见下方 return 里的字段名。 + */ +export async function planBackfill({ + service = null, + dshHome = undefined, + sessionDir = undefined, + now = Date.now(), + days = DEFAULT_DAYS, + limit = DEFAULT_LIMIT, + rounds = DEFAULT_ROUNDS, + config = null +} = {}) { + const dir = sessionDir ?? resolveSessionDir({ dshHome }); + const scanned = findSessionFiles(dir); + const cutoff = now - days * 86400000; + const plan = { + sessionDir: dir, + present: scanned.present, + truncated: scanned.truncated, + unreadable: scanned.unreadable.length, + filesScanned: scanned.files.length, + importedSessions: 0, + outsideWindow: 0, + noCreatedAt: 0, + alreadyDistilled: 0, + unreadableFiles: 0, + pending: 0, + candidates: [], + limit, + days, + rounds + }; + const candidates = []; + for (const file of scanned.files) { + let reading; + try { + reading = await readSessionFile(file); + } catch { + plan.unreadableFiles += 1; + continue; + } + if (!isImportedSession(reading.events)) continue; + plan.importedSessions += 1; + const createdAt = Number.isFinite(reading.header?.createdAt) ? reading.header.createdAt : null; + if (createdAt === null) { + plan.noCreatedAt += 1; + continue; + } + if (createdAt < cutoff) { + plan.outsideWindow += 1; + continue; + } + const sessionId = typeof reading.header?.id === "string" && reading.header.id + ? reading.header.id + : null; + if (!sessionId) { + plan.unreadableFiles += 1; + continue; + } + // 断点续跑:游标追上最后一条事件就不再跑(重跑安全靠这张表,不靠内存)。 + const cursor = typeof service?.getDistillCursor === "function" + ? service.getDistillCursor(sessionId)?.last_seq + : undefined; + if (Number.isSafeInteger(cursor) && cursor >= reading.lastSeq) { + plan.alreadyDistilled += 1; + continue; + } + candidates.push({ + file, + sessionId, + createdAt, + lastSeq: reading.lastSeq, + cursor: Number.isSafeInteger(cursor) ? cursor : null, + frames: reading.frames, + failed: reading.failed + }); + } + // 时间倒序:先处理最近的,预算砍掉时损失最小。 + candidates.sort((left, right) => right.createdAt - left.createdAt); + plan.pending = candidates.length; + plan.candidates = candidates.slice(0, limit > 0 ? limit : candidates.length); + // 一次会话一次调用(窗口都压在一轮里),所以调用数就是候选数。 + const routeConfigured = Boolean(config?.summarizeProvider && config?.summarizeModel); + plan.routeConfigured = routeConfigured; + plan.callsPlanned = routeConfigured ? plan.candidates.length : 0; + return plan; +} + +/** + * 真跑(或只跑 dry-run)。 + * + * `summarizeNow` 由 `createSummarizer` 暴露,方法体是薄薄一层:拿 `snapshotSessionEvents` + * 组 `{id, events}` 喂进现有链路——**这里不复制任何蒸馏逻辑**。 + * + * @param {object} options `{ service, summarizer, config, logger, dryRun, limit, days, rounds, sessionDir, dshHome, now, maxEntries }` + * @returns {Promise} dry-run 回 `{ dryRun: true, plan }`;真跑回逐会话结果。 + */ +export async function runBackfill({ + service = null, + summarizer = null, + config = null, + logger = null, + dryRun = true, + sessionDir = undefined, + dshHome = undefined, + now = Date.now(), + days = DEFAULT_DAYS, + limit = DEFAULT_LIMIT, + rounds = DEFAULT_ROUNDS, + maxEntries = DEFAULT_MAX_ENTRIES +} = {}) { + const plan = await planBackfill({ service, dshHome, sessionDir, now, days, limit, rounds, config }); + if (dryRun) return { dryRun: true, plan }; + // 没配蒸馏路由时真跑没有意义:plan 已经把 `callsPlanned` 报成 0,真跑却仍会逐条 + // 调 summarizeNow,每条都因缺 provider/model 失败——50 条会话就是 50 条 failed 加 + // 50 行日志,读起来像「批量任务坏了」而不是「你没配模型」。按 plan 的同一判据直接 + // 拒绝,一条都不跑(dry-run 不受影响,它本来就不调模型)。 + if (!plan.routeConfigured) { + return { dryRun: false, plan, error: "no-distill-route", ran: 0, failed: 0, results: [] }; + } + if (typeof summarizer?.summarizeNow !== "function") { + return { dryRun: false, plan, error: "summarizer-unavailable", results: [] }; + } + const results = []; + for (const candidate of plan.candidates) { + let reading; + try { + reading = await readSessionFile(candidate.file); + } catch (error) { + results.push({ sessionId: candidate.sessionId, status: "unreadable", error: String(error?.message ?? error) }); + continue; + } + const windowed = recentTurns(reading.events, rounds); + const trigger = windowed[windowed.length - 1]; + if (!trigger) { + results.push({ sessionId: candidate.sessionId, status: "empty" }); + continue; + } + try { + await summarizer.summarizeNow( + // header 必须一起给:项目排除门卫读的是 session.header.cwd(创建时冻结 + // 的那份),只带 id + events 的话回填路径上 distillExcludeProjects 永远 + // 不生效——那正是「导入会话不该进库」最该守的一条。 + { id: candidate.sessionId, header: reading.header, events: windowed }, + trigger, + // bypassImported:回填喂的正是导入会话,必须显式旁路保护层 + // (src/summarize.js 的 imported-session 门卫),否则被自己的墙挡死。 + { bypassPeak: true, bypassImported: true, maxEntries } + ); + results.push({ sessionId: candidate.sessionId, status: "ran", events: windowed.length }); + } catch (error) { + logger?.warn?.(`dsh-mneme: backfill ${candidate.sessionId} failed: ${String(error)}`); + results.push({ sessionId: candidate.sessionId, status: "failed", error: String(error?.message ?? error) }); + } + } + const ran = results.filter((item) => item.status === "ran").length; + return { dryRun: false, plan, ran, failed: results.length - ran, results }; +} \ No newline at end of file diff --git a/dsh-mneme/src/config.js b/dsh-mneme/src/config.js index 4cb7dcca..187fe7b8 100644 --- a/dsh-mneme/src/config.js +++ b/dsh-mneme/src/config.js @@ -31,6 +31,25 @@ export const Config = z.object({ // 蒸馏把完整对话上下文交给 LLM 提炼,不硬裁到 8000 字就截断语义;默认 // 24000 字符(约覆盖一整轮中等对话),需要更完整可调大。 distillMaxChars: z.natural().min(1000).max(200000).default(24000), + // 隔离(不入库):默认不让**导入器写的会话**进记忆库。 + // + // 为什么需要它:导入器(@deepseek-ai/dsh-deepseek-chat-import)把 APP 导出的 + // 会话写成普通 DSH 会话,不压缩、不摘要、header 也不留标记,只在每条 assistant + // 消息上盖 provider 戳。蒸馏只在新 `turn/end` 唤醒,所以「导进来」本身不进库—— + // 但用户哪天在那种会话里继续说话,那个会话没有蒸馏游标(previousSeq 为空), + // snapshotSessionEvents 在 afterSeq 为空时只按 throughSeq 截断,**整段历史会被 + // 当成新窗口**(尾部 distillMaxChars)拿去蒸馏。「先导进去、以后再说」这个中间 + // 态因此并不安全,默认挡掉正是为了它。 + // + // 要进库走回填(src/backfill.js,显式点名会话,不经过这条路径)。 + distillSkipImportedSessions: z.boolean().default(true), + // 按项目目录排除蒸馏:cwd 前缀命中即不进库,整条项目线(含之后新建的会话)一次 + // 挡在外面,不用逐条会话加 id。前缀按 `/` 归一后比较,命中含等号(同一目录)与 + // 目录下所有子路径。空数组 = 不排除任何项目(默认行为不变)。 + // + // 与上面的导入排除、与 sessionGateEnabled(逐会话无痕)分层:无痕是「这条会话对 + // 记忆系统整个不可见」(注入也不给),这里只管「蒸馏进不进库」。 + distillExcludeProjects: z.array(z.string()).default([]), // 图像描述档(docs/DISTILL.md:346):蒸馏前先用一个**多模态模型**给窗口里的 // 每张图写一段描述,并把描述并进转录——图本身仍然只是 `[图: …]` 占位,原图 // 不落库、不读字节(#611 的口径不变,这里只补「图里是什么」这段文字)。 diff --git a/dsh-mneme/src/distill-guard.js b/dsh-mneme/src/distill-guard.js new file mode 100644 index 00000000..fa50af8b --- /dev/null +++ b/dsh-mneme/src/distill-guard.js @@ -0,0 +1,52 @@ +// 2026-11-18 保护层接线(#688 续):导入会话与排除项目的蒸馏门卫。 +// 设计要点: +// 1. 项目排除放在门卫区(route 解析之后、间隔门之前)——被挡的会话连路由都不该解析。 +// 2. backfill(src/backfill.js)走 summarizeNow 且必须透传 bypassImported: true, +// 否则回填被自己的保护层挡死。opts 显式旁路,与 bypassPeak 同形。 +// 3. 判据只读会话上已有的事件/header:导入判定由 src/imported-session.js 的纯函数 +// 负责(summarize.js 复用 collectMessages 已读好的窗口,不额外快照)。 +// 4. cwd 读 session.header?.cwd(创建时冻结的那份,不能读 requestHeader—— +// scope.js 踩过的坑,pipeline-runner.js:57-62 projectRootOf 同口径)。 +// 5. 前缀比较按平台分隔符归一后 startsWith;大小写不敏感(Windows 盘符 +// C:/c、D:/d 都会出现)。 + +/** + * 把一段路径归一成可比较形态:统一 / 与 \、剥尾部斜杠、小写。 + * 非字符串或纯空白返回 null(调用方跳过)。 + * + * **文件系统根是有效条目,不当作空值丢掉**:名单里写 `/` 的意思是「全挡」, + * 剥尾后不能变成 null 让整条规则失效(那是 CodeRabbit 在 #755 抓到的洞)。 + * 盘符根(`C:\`)剥尾后是 `c:`,它作为前缀本身就能命中 `c:/...`,无需特判。 + */ +function normalizeProjectPath(raw) { + if (typeof raw !== "string") return null; + // 连续分隔符顺手折叠:手写配置里 `/work//private` 与 `/work/private` 是同一个目录, + // 折叠后比较才不会因为一个多打的斜杠漏挡。 + const unified = raw.trim().replace(/[\\/]/g, "/").replace(/\/{2,}/g, "/"); + if (!unified) return null; + const stripped = unified.replace(/\/+$/, ""); + // 剥干净了说明它本身就是根("/")——保留,不要掉成 null。 + if (!stripped) return "/"; + return stripped.toLowerCase(); +} + +/** + * 排除名单命中判定:名单项归一后,cwd 必须等于它或是它的子路径。 + * 名单项里带通配符/正则一律按字面路径处理(不展开语义)。 + * + * 根目录(`/`)是「整机全挡」,任何绝对路径都算它的后代;否则按 + * `target + "/"` 做前缀比较,`private-public` 不会被 `private/` 误伤。 + */ +function isExcludedProject(sessionCwd, excludeList) { + const cwd = normalizeProjectPath(sessionCwd); + if (!cwd || !Array.isArray(excludeList) || excludeList.length === 0) return false; + for (const item of excludeList) { + const target = normalizeProjectPath(item); + if (!target) continue; + if (target === "/") return true; // 根目录:全挡 + if (cwd === target || cwd.startsWith(target + "/")) return true; + } + return false; +} + +export { isExcludedProject, normalizeProjectPath }; diff --git a/dsh-mneme/src/imported-session.js b/dsh-mneme/src/imported-session.js new file mode 100644 index 00000000..d3d1426c --- /dev/null +++ b/dsh-mneme/src/imported-session.js @@ -0,0 +1,40 @@ +// 「这条会话是导入器写的」判据的唯一定义处。 +// +// 单独成文件、且不 import 任何别的东西:两个消费方的冷热差别很大——回填 +// (src/backfill.js)本来就要拉 fs 与 session-usage,而蒸馏(src/summarize.js) +// 在每一轮 turn/end 都被调用。判定函数不该把整条扫描链带进热路径。 + +/** 导入器给每条 assistant 消息盖的 provider 戳(插件 lib/index.js:181-185、621-633)。 */ +export const IMPORT_PROVIDER = "deepseek-chat-export"; +/** 导入器给「这条回复进不了模型历史」的会话留的错误码前缀(插件 lib/index.js:191-195)。 */ +export const IMPORT_ERROR_PREFIX = "DEEPSEEK_EXPORT_"; + +/** + * 这一组事件是不是导入器写的会话。 + * + * **纯函数**:输入事件数组,输出 bool,不读盘、不看状态。以后 DSH 改了戳法, + * 只改这一个函数(与它的一条测试),不用翻扫描循环。 + * + * 两条信号取并集,主判据是消息 source(DSH 自己从不产生这个 provider 字符串); + * `turn/end` 的错误码只用来兜「整条会话的回复全被排除、一条 assistant/message + * 都没有」的那种会话(TEMPLATE_RESPONSE / EMPTY_REPLY / NO_REPLY / WIP / + * FAILED / TIMEOUT / CONTEXT_LENGTH_EXCEEDED / INCOMPLETE)。 + * + * 不用 header、不用 session/title:前者没有标记,后者正常会话同样有。 + */ +export function isImportedSession(events) { + if (!Array.isArray(events)) return false; + for (const event of events) { + if (!event || typeof event !== "object") continue; + if (event.type === "assistant/message") { + const provider = event.data?.message?.source?.provider; + if (provider === IMPORT_PROVIDER) return true; + continue; + } + if (event.type === "turn/end") { + const code = event.data?.reason?.error?.code; + if (typeof code === "string" && code.startsWith(IMPORT_ERROR_PREFIX)) return true; + } + } + return false; +} \ No newline at end of file diff --git a/dsh-mneme/src/index.js b/dsh-mneme/src/index.js index bcb62375..d21eee63 100644 --- a/dsh-mneme/src/index.js +++ b/dsh-mneme/src/index.js @@ -28,6 +28,7 @@ import { createStandaloneApi } from "./api-standalone.js"; import { registerStandaloneApi, clearStandaloneApi } from "./standalone-registry.js"; // #275 存储生命周期第一批:无损回收(手动入口,不挂启动路径)。 import { createMaintenance } from "./maintenance.js"; +import { runBackfill } from "./backfill.js"; import { createChoroQueries } from "./wiki/queries.js"; // chorography M3:开局注入块与 /choro 只读回执。两者都只吃 queries 读侧,不写任何东西。 import { createChoroInject } from "./wiki/inject.js"; @@ -738,6 +739,20 @@ export const apply = (ctx, config) => { const summarizer = createSummarizer(ctx, service, cfg, {}, sessionGate); disposers.push(summarizer.dispose); + // 导入会话批量回填(DSH 0.2.2-rc.3 的 DeepSeek 导出导入,一次几百条会话)。 + // 复用**同一个** summarizer:不另建一套蒸馏链、不碰 dream 的预算与节流键, + // 只借它的 `summarizeNow`(薄薄一层,把会话样对象喂进现有链路)。构造期零副作用, + // 只有手动入口(面板 API 的 POST /maintenance/backfill)会调 `run`。 + const backfill = { + run: (options = {}) => runBackfill({ + service, + summarizer, + config: cfg, + logger: ctx.logger, + ...options + }) + }; + // QQ 日记本(v0.13,diaryEnabled):构造期零副作用(只闭包住依赖,不建目录、不读盘、 // 不调模型),所以位置可以放在这里——它必须早于下面的 createApi,面板的 // 「写一篇」按钮要转调同一个 composer;而自动触发侧在 webServer 块之后挂(同 @@ -800,7 +815,7 @@ export const apply = (ctx, config) => { add: () => { throw new Error("commands unavailable"); }, remove: () => false, list: () => [] - }, embedder, { ...semantic, vectorIndex, reranker }, cfg.apiToken, { ...cfg, memoryDir }, maintenance, extractProfileDraft, choro, profileFacts, kb, diary); + }, embedder, { ...semantic, vectorIndex, reranker }, cfg.apiToken, { ...cfg, memoryDir }, maintenance, extractProfileDraft, choro, profileFacts, kb, diary, backfill); disposers.push(api.dispose); // dsh-go 原生面板:只有宿主提供 dshGo 服务(dsh-go 手机 App)时才挂载。 diff --git a/dsh-mneme/src/settings.js b/dsh-mneme/src/settings.js index edde6891..cfee8a0f 100644 --- a/dsh-mneme/src/settings.js +++ b/dsh-mneme/src/settings.js @@ -123,6 +123,17 @@ function serializePersonaStore(store) { // 3. 写入是逐键校验的合并写:未知键/类型/范围不符抛 TypeError(消息含键名, // 供 API 透传给前端定位),校验不通过不落库,坏值永远进不了 kv。 const FEATURE_FLAG_BOOLEANS = [ + // #688 保护层:导入会话的蒸馏门卫。面板可关、默认挡——导入会话无游标, + // 继续说话会把整段历史当新窗口烧一次蒸馏;要进库走 backfill 显式路径 + // (POST /maintenance/backfill,bypassImported 旁路)。 + // + // 同一层的 `distillExcludeProjects`(项目排除名单)**刻意不在这张表**: + // 它是字符串数组,而本表只收 boolean(validateFlag 见到非布尔直接抛)。 + // 硬塞进来会被读成「开关」——写 ["/x"] 抛 TypeError、写 false 却存得下, + // 重启后名单变成一个假的关闭状态。它走「仅配置文件」那条路(config.js + // 声明、面板改不到),与 codingKeywords / sessionGateSessions 同口径。 + "distillSkipImportedSessions", + "autoInject", "autoSummarize", "hotMemoryEnabled", diff --git a/dsh-mneme/src/summarize.js b/dsh-mneme/src/summarize.js index e0c66168..da81e8c9 100644 --- a/dsh-mneme/src/summarize.js +++ b/dsh-mneme/src/summarize.js @@ -9,6 +9,8 @@ import { injectChildEnabled } from "./config.js"; import { withEffortFallback, describeStreamFailure, EFFORT_REJECT_RE } from "./dream.js"; // v0.9.0(M.O. 项目记忆):蒸馏写入复用工具路径那把作用域解析器,见 createSummarizer。 import { createScopeResolver, isHardWallWorkspace, resolveScopeStamp } from "./scope.js"; +import { isImportedSession } from "./imported-session.js"; +import { isExcludedProject } from "./distill-guard.js"; // #493 补充 §③:断链与巩固侧同一份实现(src/fence.js)。转录里出现闭合标记时 // 原先能提前闭合围栏——那是把用户消息升格成指令最短的一条路。 import { breakFenceMarkers } from "./fence.js"; @@ -560,8 +562,13 @@ function describeAuditPatch(stats) { return out; } -async function collectMessages(session, maxChars = 8000, language = "zh", afterSeq, throughSeq, describePart) { +async function collectMessages(session, maxChars = 8000, language = "zh", afterSeq, throughSeq, describePart, mutesDescribe = false) { const events = snapshotSessionEvents(session, afterSeq, throughSeq); + // #755 复审(CodeRabbit 第 7 条):图像描述是逐 part 真的调一次模型。被导入门挡下 + // 的会话在门卫之前就要走完整个 collectMessages,于是「挡掉蒸馏」反而先付了一轮 + // 视觉调用。判定就地复用刚读好的 events(不再 snapshot 第二次),命中就让 + // describePart 这一段不跑,转录退回纯文本——与「describePart 缺席」逐字节一致。 + const describeMuted = mutesDescribe === true && isImportedSession(events); const lines = []; const seenSubagentMessages = new Set(); // Issue #611:这一窗里被折成占位的图片张数。占位本身不丢信息,但「折了几张」 @@ -624,7 +631,7 @@ async function collectMessages(session, maxChars = 8000, language = "zh", afterS // 图像描述档(docs/DISTILL.md:346):开关开着且这一 part 真的带图时,才 // 去出描述。describePart 由调用方注入(默认 undefined)——本函数不认识 // 任何路由配置,只负责「文本最后长什么样」,这样单测可以只塞一个桩进去。 - const described = typeof describePart === "function" && part.blocks?.length + const described = !describeMuted && typeof describePart === "function" && part.blocks?.length ? await describePart(part) : part.text; if (part.role === "user") { @@ -692,6 +699,9 @@ async function collectMessages(session, maxChars = 8000, language = "zh", afterS : [], hasEvents: events.length > 0, lastSeq, + // #688 保护层复用:门卫要判「这是不是导入会话」,直接拿这份已读好的窗口, + // 不再单独 snapshot 一遍(每轮 turn/end 多一次全量读)。 + events, // Issue #611:这一窗折成占位的图片张数。调用方把它写进审计 metadata, // 让「图进来了几张」成为可对账的事实而不是推断。 imageCount, @@ -797,7 +807,6 @@ function enqueueDistill(task, intervalMs = 0) { import { parsePeakSpec, isInPeakWindow, nextOffPeakAt } from "./peak-hours.js"; export { parsePeakSpec, isInPeakWindow, nextOffPeakAt }; export function createSummarizer(ctx, service, config, deps = {}, sessionGate = null) { - if (!config.autoSummarize) return { dispose: () => {} }; // Issue #239(第 4 项):时钟与定时器可注入——排程不绑死真实时钟,测试才能确定性 // 覆盖「高峰顺延 → 非高峰补跑」。房型同 dream/sleep.js 的注入参数。 @@ -949,6 +958,27 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = ? { provider: header.provider, model: header.model } : undefined; + // 2026-11-18(#688 续,保护层):两道确定性门卫,放在间隔门之前——被挡的会话 + // 连「刚刚跑过」的间隔点都不该占,也不该进入后面任何一条链。 + // + // ① 项目排除(distillExcludeProjects,默认 []):cwd 只读 session.header + // (创建时冻结的那份)。前缀命中即整条项目线挡在库外。 + // + // ② 导入会话隔离(distillSkipImportedSessions,默认 true)。为什么判事件不判 + // header:导入器不留 header 标记,唯一信号在消息 source 上(src/imported- + // session.js)。判定成本是一次快照读取,比一次 LLM 调用便宜三个数量级。 + // opts.bypassImported 由回填(backfill.js)显式传:喂给 summarizeNow 的恰恰 + // 是导入会话,保护层不能挡自己的合法调用方(与 bypassPeak 同形)。 + if (Array.isArray(config.distillExcludeProjects) && config.distillExcludeProjects.length > 0 + && isExcludedProject(session.header?.cwd, config.distillExcludeProjects)) { + writeAudit({ + timestamp: new Date(now()).toISOString(), + model_id: route ? route.provider + ":" + route.model : "unknown", + status: "skipped", + error_message: "excluded-project" + }); + return; + } // Issue #127 间隔门:同一会话两次蒸馏之间的最小间隔(0 = 现状,不限)。进入 // 即打点——失败/degraded 的 run 也占用间隔,与 dreamMinIntervalMinutes 语义 // 一致,防止失败连发。被挡下的 turn/end 静默跳过,但留一行 status='skipped' @@ -1077,12 +1107,33 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = langOf(config), previousSeq, triggerSeq, - describePart + describePart, + // 与下面那道导入门同一个条件:门会挡下的会话,这里就不该先花一轮描述。 + config.distillSkipImportedSessions !== false && opts.bypassImported !== true ); const hasNewEvents = collected.hasEvents || (triggerSeq !== undefined && (previousSeq === undefined || triggerSeq > previousSeq)); if (!hasNewEvents) return; + // #688 保护层②:导入会话隔离(distillSkipImportedSessions,默认 true)。 + // 放在窗口读之后而不是门卫区:判定直接复用 collectMessages 刚读好的那份 + // 事件(collected.events),不给每轮 turn/end 增加第二次全量快照。判定读 + // 事件不判 header——导入器不留 header 标记,唯一信号在消息 source 上 + // (src/imported-session.js)。游标刻意不消费:与 peak/max-runs 同口径, + // 「这个会话先不再蒸馏了」,窗口留在原地;要进库走回填(backfill.js 显式 + // 点名会话并透传 opts.bypassImported,与 bypassPeak 同形)。 + if (config.distillSkipImportedSessions !== false + && opts.bypassImported !== true + && isImportedSession(collected.events)) { + writeAudit({ + timestamp: new Date(now()).toISOString(), + model_id: route ? route.provider + ":" + route.model : "unknown", + status: "skipped", + error_message: "imported-session" + }); + return; + } + const nextSeq = Math.max(collected.lastSeq ?? Number.NEGATIVE_INFINITY, triggerSeq ?? Number.NEGATIVE_INFINITY); if (!collected.messages.length) { // 没有可蒸馏的公开文本也算成功消费当前事件窗口,避免每个 turn/end @@ -1354,7 +1405,10 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = const parsed = parsedResult.entries; // Issue #127 条数上限:0 = 不限(现状)。被截断的条数与去重命中数一并进 // 审计 metadata——此前产出条数完全由模型输出决定,用户无从自查。 - const cap = config.summarizeMaxEntriesPerRun ?? 0; + // `opts.maxEntries` 是给「导入会话批量回填」用的单窗覆盖(src/backfill.js): + // 回填要的是「一个会话最多留几笔」,不该改全局配置、更不该造第二套抽取语义, + // 所以复用同一条上限闸门、只换这一次的数值。不传时逐字等于旧行为。 + const cap = opts.maxEntries ?? config.summarizeMaxEntriesPerRun ?? 0; const entries = cap > 0 ? parsed.slice(0, cap) : parsed; const capped = parsed.length - entries.length; const dedupeMode = config.summarizeDedupeMode ?? "off"; @@ -1679,7 +1733,11 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = } } - const unsubscribe = ctx.on("session/event", (session, event) => { + // CodeRabbit(#755):autoSummarize 关掉时只关掉**自动**订阅,不再整个函数早退。 + // 早退会让 summarizeNow 一起消失,而回填(backfill.js)正是靠它手动跑—— + // 「关掉自动蒸馏」不该等于「导入会话永远补不上记忆」。 + const autoEnabled = config.autoSummarize !== false; + const unsubscribe = !autoEnabled ? null : ctx.on("session/event", (session, event) => { if (disposed || event.type !== "turn/end") return; // Return the summarization promise so awaiters observe the writes; the // catch keeps listener dispatch from rejecting. Dispose-initiated aborts @@ -1691,6 +1749,18 @@ export function createSummarizer(ctx, service, config, deps = {}, sessionGate = }); return { + /** + * 手动跑一次蒸馏——导入会话批量回填的唯一入口(src/backfill.js 调它)。 + * + * 方法体刻意只有一行:**这里不复制任何蒸馏逻辑**,就是把一个「有 id 和 events + * 的会话样对象」喂进同一条 summarize。session 只需要 `id` 与带 `seq` 的 `events` + * (`snapshotSessionEvents` 对没有 `snapshotEvents` 的对象直接读 `.events`), + * `triggerEvent` 也只被用来取 `seq` 当窗口右界;`opts` 透传(`bypassPeak`、 + * `maxEntries`)。 + */ + summarizeNow(session, triggerEvent, opts = {}) { + return summarize(session, triggerEvent, opts); + }, dispose() { if (disposed) return; disposed = true; diff --git a/dsh-mneme/test/api.test.js b/dsh-mneme/test/api.test.js index a51086ea..1692dd3a 100644 --- a/dsh-mneme/test/api.test.js +++ b/dsh-mneme/test/api.test.js @@ -882,7 +882,11 @@ test("GET /api/dsh-mneme/features returns empty overrides and effective config d // #718 注入前判定判官专用路由新增两个键:preInjectGateProvider / // preInjectGateModel(顶层字符串键,走 FEATURE_FLAG_STRINGS 表,成对生效) // = +2,故 138 + 2 = 140。 - assert.equal(Object.keys(data.effective).length, 140); + // #688 续(自动蒸馏保护层)新增 distillSkipImportedSessions(顶层布尔键,走 + // BOOLEANS 表)= +1,故 140 + 1 = 141。同批的 distillExcludeProjects 是字符串 + // 数组,按既有口径**不进**白名单(数组键一律走 fileOnly,与 codingKeywords / + // sessionGateSessions 同表),所以它不在这笔账里。 + assert.equal(Object.keys(data.effective).length, 141); // #718 判官路由默认值单独钉:默认空串 = 回落 agentDefaultModel(entity // 同款成对语义,计数锁钉不住空串缺省被翻转)。 assert.equal(data.effective.preInjectGateProvider, ""); @@ -3167,3 +3171,68 @@ test("读 connection 抛错时按未提供处理,不让面板 500", async () = await findHandler(routes, "/api/dsh-mneme/info").handler(req("/api/dsh-mneme/info"), res); assert.equal(res.statusCode, 200, "宿主卸载中的抛错不能变成面板 500"); }); + +// --- #755 复审:回填路由的入参下限与「一条都没跑」的状态码 ------------------- +function backfillRouteFor(run) { + const store = createStore(":memory:"); + const service = createService({ store, mirror: null, config: testConfig({}) }); + const settings = createSettings(store.db); + const routes = []; + const ctx = { webServer: { register(route) { routes.push(route); return () => {}; } } }; + createApi( + ctx, service, settings, { add() {}, remove() {}, list() { return []; } }, + undefined, undefined, FAKE_API_TOKEN, null, + null, null, null, null, null, null, + { run } + ); + return routes.find((r) => r.path === "/api/dsh-mneme/maintenance/backfill"); +} + +// policy=token:请求要带 Bearer,否则先撞 401、测不到入参校验。 +function authedReq(path, method = "POST", body = null) { + const r = req(path, method, body); + r.headers = { authorization: `Bearer ${FAKE_API_TOKEN}` }; + return r; +} + +test("#755 复审:小数 limit 不许被 floor 成 0 变成「不限制」", async () => { + const calls = []; + const route = backfillRouteFor((options) => { + calls.push(options); + return { dryRun: true, plan: { candidates: [] } }; + }); + const res = new FakeRes(); + await route.handler(authedReq("/api/dsh-mneme/maintenance/backfill", "POST", { limit: 0.5 }), res); + assert.equal(res.statusCode, 200); + assert.equal(calls.length, 1); + // planBackfill 里 limit=0 的语义是「不限制」,一次手滑(0.5)就能把 dry-run + // 变成全量跑。下界钉在 1:宁可取默认 50,也不放行 0。 + assert.equal(calls[0].limit, 50, "0.5 不能 floor 成 0,回退到默认值"); +}); + +test("#755 复审:limit 传 1 就真的用 1(下界不误伤合法输入)", async () => { + const calls = []; + const route = backfillRouteFor((options) => { + calls.push(options); + return { dryRun: true, plan: { candidates: [] } }; + }); + await route.handler(authedReq("/api/dsh-mneme/maintenance/backfill", "POST", { limit: 1 }), new FakeRes()); + assert.equal(calls[0].limit, 1); +}); + +test("#755 复审:一条都没跑时回 503,不伪装成 200", async () => { + for (const error of ["no-distill-route", "summarizer-unavailable"]) { + const route = backfillRouteFor(() => ({ dryRun: false, plan: {}, error, ran: 0, results: [] })); + const res = new FakeRes(); + await route.handler(authedReq("/api/dsh-mneme/maintenance/backfill", "POST", {}), res); + assert.equal(res.statusCode, 503, `${error} 必须报 503`); + assert.equal(JSON.parse(res.body).error, error); + } +}); + +test("#755 复审:真跑了若干条仍是 200", async () => { + const route = backfillRouteFor(() => ({ dryRun: false, plan: {}, ran: 2, failed: 0, results: [] })); + const res = new FakeRes(); + await route.handler(authedReq("/api/dsh-mneme/maintenance/backfill", "POST", {}), res); + assert.equal(res.statusCode, 200); +}); diff --git a/dsh-mneme/test/backfill.test.js b/dsh-mneme/test/backfill.test.js new file mode 100644 index 00000000..1e62041d --- /dev/null +++ b/dsh-mneme/test/backfill.test.js @@ -0,0 +1,441 @@ +// 导入会话批量回填的判据与计划(#606 之后的 DSH 0.2.2-rc.3 场景)。 +// +// 这里锁三件事:① 判别是纯函数、只认两条信号;② dry-run 的数字与真跑用同一份 +// 筛选;③ 真跑真的把「最近 N 轮」和「每窗 5 条」传下去了。 +import assert from "node:assert/strict"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { zstdCompressSync } from "node:zlib"; +import test from "node:test"; + +import { + DEFAULT_DAYS, + DEFAULT_MAX_ENTRIES, + DEFAULT_ROUNDS, + IMPORT_PROVIDER, + isImportedSession, + parseSessionTexts, + planBackfill, + readSessionFile, + recentTurns, + runBackfill +} from "../src/backfill.js"; +import { createService } from "../src/service.js"; +import { createStore } from "../src/store.js"; +import { testConfig } from "./helpers/test-config.js"; + +const importedMessage = (seq) => ({ + type: "assistant/message", + seq, + data: { + turn: 1, + step: 1, + message: { + id: `m${seq}`, + role: "assistant", + content: [{ type: "text", text: "答" }], + source: { kind: "model", provider: IMPORT_PROVIDER, model: IMPORT_PROVIDER } + }, + stream: [] + } +}); + +const normalMessage = (seq) => ({ + type: "assistant/message", + seq, + data: { + turn: 1, + step: 1, + message: { + id: `m${seq}`, + role: "assistant", + content: [{ type: "text", text: "答" }], + source: { kind: "model", provider: "deepseek-chat", model: "deepseek-chat" } + }, + stream: [] + } +}); + +const turnStart = (turn, seq) => ({ type: "turn/start", seq, data: { turn } }); +const turnEnd = (turn, seq, code = null) => ({ + type: "turn/end", + seq, + data: { turn, reason: code === null ? { kind: "completed" } : { kind: "error", error: { code, message: code } } } +}); + +function writeSessionFile(dir, id, { createdAt, events, name = "session.v4.jsonl.zstd" }) { + const target = join(dir, "workspace-slug", id); + mkdirSync(target, { recursive: true }); + const lines = [ + JSON.stringify({ type: "session", version: 4, id, createdAt, cwd: "/workspace", isSeeded: false }), + ...events.map((event) => JSON.stringify(event)) + ]; + const buffer = zstdCompressSync(Buffer.from(`${lines.join("\n")}\n`, "utf8")); + writeFileSync(join(target, name), buffer); + return join(target, name); +} + +function makeRoot() { + return mkdtempSync(join(tmpdir(), "mneme-backfill-")); +} + +test("isImportedSession: provider 戳是主判据", () => { + assert.equal(isImportedSession([turnStart(1, 0), importedMessage(1), turnEnd(1, 2)]), true); + assert.equal(isImportedSession([turnStart(1, 0), normalMessage(1), turnEnd(1, 2)]), false); + assert.equal(isImportedSession([]), false); + assert.equal(isImportedSession(null), false); + assert.equal(isImportedSession([{ type: "assistant/message" }]), false); +}); + +test("isImportedSession: 整条会话没有 assistant/message 时靠错误码兜底", () => { + const events = [turnStart(1, 0), turnEnd(1, 1, "DEEPSEEK_EXPORT_TEMPLATE_RESPONSE")]; + assert.equal(isImportedSession(events), true); + // 别把普通会话的失败也认成导入(错误码前缀必须吻合)。 + assert.equal(isImportedSession([turnStart(1, 0), turnEnd(1, 1, "STREAM_CLOSED")]), false); +}); + +test("isImportedSession: 不认 header 与 session/title", () => { + const events = [ + { type: "session", version: 4, id: "s1", createdAt: 1, cwd: "/w", isSeeded: false }, + { type: "session/title", seq: 0, data: { title: "导入的标题" } } + ]; + assert.equal(isImportedSession(events), false); +}); + +test("recentTurns: 只留最近 N 轮,且按 turn/start 切", () => { + const events = []; + for (let turn = 1; turn <= 25; turn += 1) { + events.push(turnStart(turn, events.length)); + events.push({ type: "user/message", seq: events.length, data: { turn } }); + events.push(importedMessage(events.length)); + events.push(turnEnd(turn, events.length)); + } + const trimmed = recentTurns(events, 20); + assert.equal(trimmed[0].type, "turn/start"); + assert.equal(trimmed[0].data.turn, 6); + assert.equal(trimmed[trimmed.length - 1].type, "turn/end"); + // 不截、以及没有 turn/start 的会话原样返回。 + assert.equal(recentTurns(events, 0).length, events.length); + assert.equal(recentTurns(events, 100).length, events.length); + const flat = [{ type: "user/message", seq: 0 }]; + assert.deepEqual(recentTurns(flat, 20), flat); +}); + +test("parseSessionTexts: 保留文件里的 seq,缺了按行号补", () => { + const parsed = parseSessionTexts([ + `${JSON.stringify({ type: "session", id: "s1" })}\n${JSON.stringify({ type: "turn/start", seq: 7, data: { turn: 1 } })}`, + `${JSON.stringify({ type: "turn/end", data: { turn: 1 } })}\nnot-json\n` + ]); + assert.equal(parsed.length, 3); + assert.equal(parsed[0].seq, 0); // header 没有 seq,按行号补 + assert.equal(parsed[1].seq, 7); // 文件里的 seq 原样保留 + assert.equal(parsed[2].seq, 2); +}); + +test("readSessionFile: 解出 header、事件与最后一条 seq", async () => { + const root = makeRoot(); + try { + const file = writeSessionFile(root, "session-a", { + createdAt: 1_700_000_000_000, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + const reading = await readSessionFile(file); + assert.equal(reading.header.id, "session-a"); + assert.equal(reading.header.createdAt, 1_700_000_000_000); + assert.equal(reading.events.length, 4); + assert.equal(reading.lastSeq, 2); + assert.equal(reading.failed, 0); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("planBackfill: 计数、窗口、已消费与倒序都按同一份筛选算", async () => { + const root = makeRoot(); + const now = Date.UTC(2026, 10, 14, 12, 0, 0); + try { + const day = 86400000; + // 近 3 天内的导入会话(要跑) + writeSessionFile(root, "imported-recent", { + createdAt: now - day, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + // 更近的一条(用来验证倒序) + writeSessionFile(root, "imported-newest", { + createdAt: now - 1000, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + // 超出 90 天窗口 + writeSessionFile(root, "imported-old", { + createdAt: now - 200 * day, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + // 普通会话:不计入导入 + writeSessionFile(root, "normal", { + createdAt: now - day, + events: [turnStart(1, 0), normalMessage(1), turnEnd(1, 2)] + }); + // 导入且已被蒸馏完(游标追平) + writeSessionFile(root, "imported-done", { + createdAt: now - day, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + + const cursors = new Map([["imported-done", 2]]); + const service = { getDistillCursor: (id) => (cursors.has(id) ? { last_seq: cursors.get(id) } : undefined) }; + const plan = await planBackfill({ + service, + sessionDir: root, + now, + config: { summarizeProvider: "deepseek-flash", summarizeModel: "deepseek-flash" } + }); + + assert.equal(plan.present, true); + assert.equal(plan.filesScanned, 5); + assert.equal(plan.importedSessions, 4); + assert.equal(plan.outsideWindow, 1); + assert.equal(plan.alreadyDistilled, 1); + assert.equal(plan.pending, 2); + assert.deepEqual(plan.candidates.map((item) => item.sessionId), ["imported-newest", "imported-recent"]); + assert.equal(plan.routeConfigured, true); + assert.equal(plan.callsPlanned, 2); + assert.equal(plan.limit, 50); + assert.equal(plan.days, DEFAULT_DAYS); + assert.equal(plan.rounds, DEFAULT_ROUNDS); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("planBackfill: 没配蒸馏路由时 callsPlanned 为 0(dry-run 要看得见)", async () => { + const root = makeRoot(); + try { + writeSessionFile(root, "imported", { + createdAt: Date.now(), + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + const plan = await planBackfill({ sessionDir: root, config: {} }); + assert.equal(plan.pending, 1); + assert.equal(plan.routeConfigured, false); + assert.equal(plan.callsPlanned, 0); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("planBackfill: limit 只切候选,不改变账目", async () => { + const root = makeRoot(); + try { + const now = Date.now(); + for (let index = 0; index < 3; index += 1) { + writeSessionFile(root, `imported-${index}`, { + createdAt: now - index * 1000, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + } + const plan = await planBackfill({ sessionDir: root, now, limit: 2 }); + assert.equal(plan.pending, 3); + assert.equal(plan.candidates.length, 2); + assert.equal(plan.candidates[0].sessionId, "imported-0"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("runBackfill: dry-run 不调 summarizer,一个 token 都不花", async () => { + const root = makeRoot(); + try { + writeSessionFile(root, "imported", { + createdAt: Date.now(), + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + let called = 0; + const result = await runBackfill({ + sessionDir: root, + config: { summarizeProvider: "p", summarizeModel: "m" }, + summarizer: { summarizeNow: () => { called += 1; } } + }); + assert.equal(result.dryRun, true); + assert.equal(result.plan.callsPlanned, 1); + assert.equal(called, 0); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("runBackfill: 真跑把最近 20 轮与 maxEntries 传下去,并逐条记账", async () => { + const root = makeRoot(); + try { + const events = []; + for (let turn = 1; turn <= 25; turn += 1) { + events.push(turnStart(turn, events.length)); + events.push({ type: "user/message", seq: events.length, data: { turn } }); + events.push(importedMessage(events.length)); + events.push(turnEnd(turn, events.length)); + } + writeSessionFile(root, "imported", { createdAt: Date.now(), events }); + const calls = []; + const result = await runBackfill({ + sessionDir: root, + dryRun: false, + config: { summarizeProvider: "p", summarizeModel: "m" }, + summarizer: { + summarizeNow: (session, trigger, opts) => { + calls.push({ session, trigger, opts }); + } + } + }); + assert.equal(result.dryRun, false); + assert.equal(result.ran, 1); + assert.equal(calls.length, 1); + // 25 轮只送最近 20 轮:第一条是第 6 轮的 turn/start + assert.equal(calls[0].session.events[0].data.turn, 6); + assert.equal(calls[0].session.events.length, 80); + assert.equal(calls[0].trigger.type, "turn/end"); + // 项目排除门卫读的是 session.header.cwd:只带 id + events 的话,回填路径上 + // distillExcludeProjects 永远不生效(CodeRabbit #755 第 3 条)。 + assert.equal(calls[0].session.header?.cwd, "/workspace"); + assert.equal(calls[0].opts.bypassPeak, true); + assert.equal(calls[0].opts.maxEntries, DEFAULT_MAX_ENTRIES); + assert.equal(result.results[0].status, "ran"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("runBackfill: summarizer 自己的失败不影响后面的会话", async () => { + const root = makeRoot(); + try { + const now = Date.now(); + writeSessionFile(root, "a-bad", { + createdAt: now - 1000, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + writeSessionFile(root, "b-good", { + createdAt: now - 2000, + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + const seen = []; + const result = await runBackfill({ + sessionDir: root, + dryRun: false, + config: { summarizeProvider: "p", summarizeModel: "m" }, + summarizer: { + summarizeNow: (session) => { + seen.push(session.id); + if (session.id === "a-bad") throw new Error("boom"); + } + } + }); + assert.deepEqual(seen, ["a-bad", "b-good"]); + assert.equal(result.ran, 1); + assert.equal(result.failed, 1); + assert.equal(result.results[0].status, "failed"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); +test("runBackfill: 没配蒸馏路由时真跑直接拒绝,一条都不调", async () => { + const root = makeRoot(); + try { + writeSessionFile(root, "imported", { + createdAt: Date.now(), + events: [turnStart(1, 0), importedMessage(1), turnEnd(1, 2)] + }); + let called = 0; + const result = await runBackfill({ + sessionDir: root, + dryRun: false, + config: {}, + summarizer: { summarizeNow: () => { called += 1; } } + }); + // 不判这一下:50 条会话会变成 50 条 failed 加 50 行日志,读起来像批量任务坏了, + // 实际只是没配模型。dry-run 不受影响(它本来就不调模型)。 + assert.equal(result.error, "no-distill-route"); + assert.equal(result.ran, 0); + assert.equal(result.failed, 0); + assert.deepEqual(result.results, []); + assert.equal(called, 0); + assert.equal(result.plan.callsPlanned, 0); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +// 上面那些 fixture 的形状是照着设计推的;这条按插件真实写盘的样子造,并且用真 +// store/service 跑完整一遍——「0/836」之后最该补的就是这一步:真数据通路有没有被 +// 真数据验证过。形状要点:首行 session header 没有 seq、header 只有 +// version/id/createdAt/cwd/isSeeded、其后事件都带 seq、第二轮是被排除的回答 +// (没有 assistant/message,只有一条带 DEEPSEEK_EXPORT_* 的 turn/end)。 +test("真数据形状:认得出、跑得动、游标推进、第二次判断为已消费", async () => { + const root = makeRoot(); + const store = createStore(":memory:"); + try { + const service = createService({ store, mirror: null, config: testConfig({}) }); + const id = "imported-real-shape"; + const events = [ + { type: "permission/preset", seq: 1, data: { preset: "default" } }, + { type: "sandbox/mode", seq: 2, data: { mode: "workspace-write" } }, + turnStart(1, 3), + { + type: "user/message", + seq: 4, + data: { turn: 1, message: { id: "u1", role: "user", content: [{ type: "text", text: "问" }] } } + }, + importedMessage(5), + turnEnd(1, 6), + turnStart(2, 7), + { + type: "user/message", + seq: 8, + data: { turn: 2, message: { id: "u2", role: "user", content: [{ type: "text", text: "再问" }] } } + }, + turnEnd(2, 9, "DEEPSEEK_EXPORT_TEMPLATE_RESPONSE") + ]; + writeSessionFile(root, id, { createdAt: Date.now(), events }); + const config = { summarizeProvider: "p", summarizeModel: "m" }; + + const plan = await planBackfill({ sessionDir: root, config, service }); + assert.equal(plan.filesScanned, 1); + assert.equal(plan.importedSessions, 1); + assert.equal(plan.pending, 1); + assert.equal(plan.callsPlanned, 1); + + const calls = []; + const summarizer = { + summarizeNow: (session, trigger, opts) => { + calls.push({ session, trigger, opts }); + // stub 只承担真 summarizer 会做的两件持久化:推进游标 + 落原文凭证。 + const lastSeq = Math.max(...session.events.map((event) => event.seq)); + service.setDistillCursor(session.id, lastSeq); + service.saveDistillRecords([ + { sessionId: session.id, seq: 5, role: "assistant", text: "答", windowId: "w1", distilledAt: Date.now() } + ]); + } + }; + + const first = await runBackfill({ sessionDir: root, dryRun: false, config, summarizer, service }); + assert.equal(first.ran, 1); + assert.equal(calls.length, 1); + assert.equal(calls[0].session.id, id); + assert.equal(calls[0].opts.bypassPeak, true); + assert.equal(calls[0].opts.maxEntries, DEFAULT_MAX_ENTRIES); + assert.equal(service.getDistillCursor(id)?.last_seq, 9); + assert.equal(service.distillRecordsFor(id, 0, 999).length, 1); + + // 第二次:游标已经吃到末尾,不该再跑,凭证也不该多出来(幂等靠主键)。 + const planAgain = await planBackfill({ sessionDir: root, config, service }); + assert.equal(planAgain.alreadyDistilled, 1); + assert.equal(planAgain.pending, 0); + assert.equal(planAgain.callsPlanned, 0); + + const second = await runBackfill({ sessionDir: root, dryRun: false, config, summarizer, service }); + assert.equal(second.ran, 0); + assert.equal(calls.length, 1); + assert.equal(service.distillRecordsFor(id, 0, 999).length, 1); + } finally { + store.close?.(); + rmSync(root, { recursive: true, force: true }); + } +}); diff --git a/dsh-mneme/test/distill-guard.test.js b/dsh-mneme/test/distill-guard.test.js new file mode 100644 index 00000000..a3a56288 --- /dev/null +++ b/dsh-mneme/test/distill-guard.test.js @@ -0,0 +1,55 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { isExcludedProject, normalizeProjectPath } from "../src/distill-guard.js"; + +test("#688 normalizeProjectPath: 尾部分隔符、反斜杠、大小写都归一", () => { + assert.equal(normalizeProjectPath("/work/private"), "/work/private"); + assert.equal(normalizeProjectPath("/work/private/"), "/work/private"); + assert.equal(normalizeProjectPath("C:\\Work\\Private\\"), "c:/work/private"); + assert.equal(normalizeProjectPath(" /work/private "), "/work/private"); + assert.equal(normalizeProjectPath("/work//private/"), "/work/private"); +}); + +test("#688 normalizeProjectPath: 空输入返回 null,根目录不掉成 null", () => { + assert.equal(normalizeProjectPath(""), null); + assert.equal(normalizeProjectPath(" "), null); + assert.equal(normalizeProjectPath(null), null); + assert.equal(normalizeProjectPath(undefined), null); + assert.equal(normalizeProjectPath(42), null); + // CodeRabbit #755 第 5 条:"/" 剥尾后是空串,但它的语义是「根目录=全挡」, + // 掉成 null 会被跳过,名单写 ["/"] 就形同虚设。它必须是有效条目。 + assert.equal(normalizeProjectPath("/"), "/"); + assert.equal(normalizeProjectPath("//"), "/"); +}); + +test("#688 isExcludedProject: 等值与前缀命中,兄弟目录不误伤", () => { + const list = ["/work/private"]; + assert.equal(isExcludedProject("/work/private", list), true); + assert.equal(isExcludedProject("/work/private/sub/dir", list), true); + assert.equal(isExcludedProject("/work/private-other", list), false); + assert.equal(isExcludedProject("/work", list), false); + assert.equal(isExcludedProject("/work/priv", list), false); +}); + +test("#688 isExcludedProject: 形如前缀的兄弟目录名不被吃掉", () => { + assert.equal(isExcludedProject("/work/private-public", ["/work/private"]), false); +}); + +test("#688 isExcludedProject: 根目录条目挡全部", () => { + assert.equal(isExcludedProject("/work/a", ["/"]), true); + assert.equal(isExcludedProject("/", ["/"]), true); +}); + +test("#688 isExcludedProject: 名单里夹着无效条目时照常判其余条目", () => { + assert.equal(isExcludedProject("/work/private", ["", " ", null, "/work/private"]), true); + assert.equal(isExcludedProject("/work/public", ["", null]), false); +}); + +test("#688 isExcludedProject: 会话没有 cwd 或名单为空时一律放行", () => { + assert.equal(isExcludedProject(undefined, ["/work/private"]), false); + assert.equal(isExcludedProject(null, ["/work/private"]), false); + assert.equal(isExcludedProject("", ["/work/private"]), false); + assert.equal(isExcludedProject("/work/private", []), false); + assert.equal(isExcludedProject("/work/private", undefined), false); + assert.equal(isExcludedProject("/work/private", "/work/private"), false); +}); diff --git a/dsh-mneme/test/lib-smoke.test.js b/dsh-mneme/test/lib-smoke.test.js index 8baa3817..2cbea66a 100644 --- a/dsh-mneme/test/lib-smoke.test.js +++ b/dsh-mneme/test/lib-smoke.test.js @@ -94,6 +94,11 @@ test("lib summarize constructs with autoSummarize disabled", async () => { const { createSummarizer } = await import("../lib/summarize.js"); const summarizer = createSummarizer({}, {}, { autoSummarize: false }); assert.equal(typeof summarizer.dispose, "function"); + // #755 复审:关掉自动蒸馏只该关掉**自动**订阅,summarizeNow 必须还在—— + // 回填(backfill)靠它手动跑,否则「关掉自动蒸馏」等于「导入会话永远补不上 + // 记忆」(CodeRabbit #755 第 2 条)。 + assert.equal(typeof summarizer.summarizeNow, "function"); + assert.equal(summarizer.summarizeNow.length >= 1, true); summarizer.dispose(); }); diff --git a/dsh-mneme/test/summarize.test.js b/dsh-mneme/test/summarize.test.js index 9c4b24be..9f95acce 100644 --- a/dsh-mneme/test/summarize.test.js +++ b/dsh-mneme/test/summarize.test.js @@ -2411,3 +2411,234 @@ test("#687 batch dedupe supersedes the weaker twin without rewriting its text", summarizer.dispose(); store.close(); }); + + +// --- #688 保护层:导入会话隔离 + 项目排除(蒸馏门卫) --------------------------- +// 背景:导入会话没有蒸馏游标,继续说话时 snapshotSessionEvents(afterSeq=undefined) +// 会把整段历史当新窗口拿去蒸馏(尾部 distillMaxChars 24000 字)——烧一次 API。 +// 「先导进去、以后再说」这个中间态并不安全,所以默认挡掉(distillSkipImportedSessions +// 默认 true);要进库走回填(backfill.js 显式点名 + bypassImported 旁路)。 +// 项目排除(distillExcludeProjects)按 session.header.cwd 前缀整线挡在外面。 + +// 导入会话的典型事件形状:assistant 消息带 source.provider 戳(lib/index.js:181-185)。 +function importedUserMessage(text, seq) { + return { seq, type: "user/message", data: { source: { kind: "user" }, content: [{ type: "text", text }] } }; +} +function importedAssistantMessage(text, seq) { + return { + seq, + type: "assistant/message", + data: { message: { source: { kind: "assistant", provider: "deepseek-chat-export" }, content: [{ type: "text", text }] } } + }; +} + +test("#688 默认挡:导入会话不蒸馏,skip 审计写明原因,游标不动", async () => { + const { events, calls, service, store } = setup(); + const handler = eventHandler(events); + const session = { + id: "s-import", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + events: [ + importedUserMessage("导入来的第一问", 1), + importedAssistantMessage("导入来的回复", 2), + { seq: 3, type: "turn/end" } + ] + }; + await handler(session, { seq: 3, type: "turn/end" }); + assert.equal(calls.length, 0, "导入会话不得发起任何 LLM 调用"); + assert.equal(store.count(), 0, "不得写入记忆"); + const rows = service.listLlmAudits(); + assert.equal(rows[0].status, "skipped"); + assert.equal(rows[0].error_message, "imported-session", "skip 原因必须可观测"); +}); + +test("#688 关掉放行:distillSkipImportedSessions=false 恢复老行为", async () => { + const { events, calls, store } = setup({ distillSkipImportedSessions: false }); + const handler = eventHandler(events); + const session = { + id: "s-import-off", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + events: [ + importedUserMessage("导入来的第一问", 1), + importedAssistantMessage("导入来的回复", 2), + { seq: 3, type: "turn/end" } + ] + }; + await handler(session, { seq: 3, type: "turn/end" }); + assert.equal(calls.length, 1, "关掉保护层后照常蒸馏"); + assert.ok(store.count() > 0, "照常写入记忆"); +}); + +test("#688 旁路:bypassImported=true 只给回填走,正常会话不受影响", async () => { + const { events, calls, store, summarizer } = setup(); + // 直接打 summarizeNow(回填的调用形状,backfill.js:244-247 透传 bypassImported)。 + const session = { + id: "s-import-bypass", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + events: [ + importedUserMessage("导入来的第一问", 1), + importedAssistantMessage("导入来的回复", 2), + { seq: 3, type: "turn/end" } + ] + }; + await summarizer.summarizeNow(session, { seq: 3, type: "turn/end" }, { bypassImported: true }); + assert.equal(calls.length, 1, "显式旁路必须放行"); + assert.ok(store.count() > 0, "回填路径照常写入"); + summarizer.dispose(); +}); + +test("#688 正常会话不受导入门影响:无 provider 戳照常蒸馏", async () => { + const { events, calls, store } = setup(); + const handler = eventHandler(events); + const session = { + id: "s-normal", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + events: [userMessage("正常会话的提问", 1), { seq: 2, type: "turn/end" }] + }; + await handler(session, { seq: 2, type: "turn/end" }); + assert.equal(calls.length, 1, "正常会话一次调用照旧"); + assert.ok(store.count() > 0); +}); + +test("#688 项目排除:cwd 前缀命中不蒸馏,审计写 excluded-project", async () => { + const { events, calls, service, store } = setup({ distillExcludeProjects: ["C:/work/private"] }); + const handler = eventHandler(events); + const session = { + id: "s-excluded", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + header: { cwd: "C:\\work\\private\\sub\\app" }, + events: [userMessage("私有项目的提问", 1), { seq: 2, type: "turn/end" }] + }; + await handler(session, { seq: 2, type: "turn/end" }); + assert.equal(calls.length, 0, "排除项目不得调模型"); + assert.equal(store.count(), 0); + const rows = service.listLlmAudits(); + assert.equal(rows[0].status, "skipped"); + assert.equal(rows[0].error_message, "excluded-project"); +}); + +test("#688 项目排除只挡子路径:兄弟目录照常蒸馏", async () => { + const { events, calls, store } = setup({ distillExcludeProjects: ["C:/work/private"] }); + const handler = eventHandler(events); + const session = { + id: "s-sibling", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + header: { cwd: "C:\\work\\private-public" }, + events: [userMessage("兄弟目录的提问", 1), { seq: 2, type: "turn/end" }] + }; + await handler(session, { seq: 2, type: "turn/end" }); + // "private-public" 以 "private-" 开头但不是 "private/" 的子路径,必须放行。 + assert.equal(calls.length, 1, "前缀必须按目录边界比较,不能按纯字符串"); + assert.ok(store.count() > 0); +}); + +test("#688 排除名单不挡导入门:两个门卫独立生效", async () => { + // 名单为空时导入门仍挡;名单命中时即使关掉导入也挡——两道门互不替代。 + const { events, calls, service } = setup({ distillExcludeProjects: ["C:/work/private"] }); + const handler = eventHandler(events); + const session = { + id: "s-both", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + header: { cwd: "C:/work/private" }, + events: [ + importedUserMessage("导入且被排除", 1), + importedAssistantMessage("两道门都该拦", 2), + { seq: 3, type: "turn/end" } + ] + }; + await handler(session, { seq: 3, type: "turn/end" }); + assert.equal(calls.length, 0); + const rows = service.listLlmAudits(); + assert.equal(rows[0].error_message, "excluded-project", "项目门在前,先记它的账"); +}); + +test("#755 复审:被导入门挡下的会话不白烧一轮图片描述", async () => { + // 描述档开着、路由也配了,模型声称能看图——只要窗口里真有图,就会逐图调一次 + // 视觉模型。导入门在 collectMessages 之后才判,这条测试钉住「挡下 = 一个视觉 + // 调用都不发」(CodeRabbit #755 第 7 条)。 + const describeCalls = []; + const { events, calls } = setup( + { + imageDescribeEnabled: true, + imageDescribeProvider: "vp", + imageDescribeModel: "vm" + }, + { + stream(options) { + // 描述调用与蒸馏调用走同一个 ctx.llm.stream,靠 purpose 分开记账。 + if (options?.purpose === "image-description") { + describeCalls.push(options); + return (async function* () { + yield { type: "text-delta", delta: "一张截图" }; + yield { type: "finish", kind: "ok" }; + })(); + } + calls.push(options); + return (async function* () { + yield { type: "finish", kind: "ok" }; + })(); + } + } + ); + const handler = eventHandler(events); + const withImage = importedUserMessage("导入来的带图提问", 1); + withImage.data.content.push({ + type: "image", + attachment: { attachmentId: "att-1", mediaType: "image/png", name: "a.png" } + }); + const session = { + id: "s-import-image", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + events: [withImage, importedAssistantMessage("导入来的回复", 2), { seq: 3, type: "turn/end" }] + }; + await handler(session, { seq: 3, type: "turn/end" }); + assert.equal(calls.length, 0, "导入会话本来就不蒸馏"); + assert.equal(describeCalls.length, 0, "被挡下的会话不该先付一轮视觉调用"); +}); + +test("#755 复审:描述档开着时,正常会话照常出图描述", async () => { + // 反向对照:同一条路径,非导入会话的图必须真的被描述——否则上一条测试 + // 可能在「描述档压根没生效」的情况下也绿。 + const describeCalls = []; + const { events, calls } = setup( + { + imageDescribeEnabled: true, + imageDescribeProvider: "vp", + imageDescribeModel: "vm" + }, + { + stream(options) { + if (options?.purpose === "image-description") { + describeCalls.push(options); + return (async function* () { + yield { type: "text-delta", delta: "一张截图" }; + yield { type: "finish", kind: "ok" }; + })(); + } + calls.push(options); + const json = JSON.stringify([ + { type: "decision", title: "看图", content: "用户贴了一张截图", importance: 3 }, + { type: "preference", title: "偏好", content: "用户喜欢中文交流", importance: 4 } + ]); + return (async function* () { + yield { type: "text-delta", delta: json }; + yield { type: "finish", kind: "ok" }; + })(); + } + } + ); + const handler = eventHandler(events); + const prompt = userMessage("看看这张图", 1); + prompt.data.content.push({ + type: "image", + attachment: { attachmentId: "att-2", mediaType: "image/png", name: "b.png" } + }); + const session = { + id: "s-normal-image", + requestHeader: () => ({ config: testConfig({ provider: "deepseek", model: "deepseek-chat" }) }), + events: [prompt, { seq: 2, type: "turn/end" }] + }; + await handler(session, { seq: 2, type: "turn/end" }); + assert.equal(describeCalls.length, 1, "正常会话的图要真的描述一次"); + assert.ok(calls.length >= 1, "然后照常蒸馏"); +});