diff --git a/devlog/_plan/261111_windows_service_stop/000_plan.md b/devlog/_plan/261111_windows_service_stop/000_plan.md index 58ade97dacd..82070aa0a1e 100644 --- a/devlog/_plan/261111_windows_service_stop/000_plan.md +++ b/devlog/_plan/261111_windows_service_stop/000_plan.md @@ -1,6 +1,6 @@ # 000 — Windows service stop that frees the port (#7771 Option B): design -Status: approved (030, bounded ARM64 fallback). Implementation is split into three pull requests (060); the primitives and the two operations land first, unwired. +Status: approved (030). Implementation is split into three pull requests (060): PR1 (primitives and the two operations, #8564) and PR3 (Desktop, #8560) have landed; PR2 wires every caller. ARM64 ships the strict variant until the 040 ARM64 device case runs (050, 060). ## Problem diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index b517be8c912..a4a4179cdea 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -5,6 +5,10 @@ The proxy child is stopped by each caller AFTER the service stop, and `/api/stop` runs inside the very proxy that holds the port. A port check inside the service operation would therefore always time out. The design is split: - Part A, `stopWindowsServiceChecked()` (async; the current `stopServiceIfInstalledDetailed` and the guarded `stopManager` slot become async on Windows): stop the manager, terminate bound wrappers, then wait until the manager is inactive and no bound wrapper is alive. Returns `absent` / `stopped` / `survived` / `denied` / `unknown` for the manager and wrappers only. `absent` means no OpenCodex task or WinSW service is registered for this home and no wrapper is bound: it maps to `ServiceStopOutcome` `absent`, prints no "Service manager stopped.", lets plain `ocx stop` send its POST, and skips part B, so a Windows install without a service keeps today's timing. With nothing registered, a process list that cannot be read is also `absent` (the wrapper sweep is skipped, as today's best-effort kill silently skips it), never `unknown`, so such a machine is never refused a stop it gets today; the residual is a stray wrapper of a deleted task that cannot be seen, which today's sweep also misses. `unknown` is reserved for evidence that cannot be read about a registered manager or a bound candidate, and for a registration probe that itself cannot answer: an unreadable `schtasks` query (today's `state-unknown`) stays `unknown`, it never becomes `absent`. + +The updater runs the authorizer-gated Part A for both WinSW and Task Scheduler; only `stopped` or `absent` permits restart. +When `ocx service stop` finds Part A absent, every candidate endpoint must be definitively dead before shared restoration. +Service stop, removal and install resolve the tracked or discovered target once and use its exact endpoint for both the receipt and stop; no target means no receipt or POST. - Part B, `confirmWindowsStopFreed()`, run by the caller after its graceful child stop: the port is free and stays free for the hold, and a remaining listener is classified as ours or foreign. Part B replaces `proxyStillLiveAfterStop` in `ocx service stop`, the `schedulerCanRespawn` re-poll in `ocx stop`, and the respawn-window check in `ocx uninstall` (src/cli/index.ts, uninstall-plan.ts), which today run only on `stopped-respawnable`; on Windows they run part B unconditionally after a part A `stopped`. On the guarded path part B is added after the existing `settle`, which keeps its approved-PID exit check (`waitExit`) and the probe of the approved endpoint; there "the port" is the approved endpoint. - A part A outcome other than `stopped` aborts before anything destructive: `ocx service uninstall` / `remove` and `prepareServiceInstall` refuse with the outcome and reason before deleting or replacing the task (today uninstall only warns and continues), `ocx uninstall` keeps client configuration, and the updater aborts before replacing files. - Plain `ocx stop` (no consent snapshot) treats a part A outcome other than `stopped` or `absent` exactly as the guarded path treats `manager-still-active`: it does not stop the child (no `/api/stop` POST), sets `stopFailed` and `ownershipBlocked`, and skips the shared teardown, so client configuration is never restored under a wrapper that can relaunch the proxy (#3008). Today this path still sends the receipt-backed POST after a failed manager stop and relies on the route re-running the service stop and answering 409; once the route skips part A for a receipt holder (below), that safety net would be gone, so the parent must not send the POST at all. The record uses existing values only: `service: failed` (or `state-unknown`), `proxy: unknown` (the child stop was not attempted), `sharedTeardown: skipped`, `outcome: failed`. @@ -18,8 +22,8 @@ Part A replaces, on Windows only, the "manager stop + best-effort wrapper kill" - `ocx service stop` calls part A instead of `stopWindows`, then claims a teardown receipt (as `ocx stop` does) before stopping the tracked proxy, so the route defers the shared teardown; it completes and clears the receipt only after part B passes and keeps it for recovery otherwise. If the claim fails while this home owns the shared teardown, it does not send an undeferred child stop at all: it reports the failure and leaves the proxy running, because part A only covers wrappers it bound and a wrapper that appears later could relaunch the proxy after an undeferred teardown. The existing refusals and the not-owned teardown path are unchanged. - `ocx service uninstall` / `remove` call part A before deleting the task (today they call `stopWindows` and kill no wrappers, so a looping wrapper can outlive its task, #764), then part B. - `prepareServiceInstall` (service install and backend switch) calls part A instead of its manager stop + wrapper kill, then its existing tracked-proxy stop, then part B. A part B result other than a free port (our identity back, or `portHeldByForeign`) stops install and switch before they publish assets or start the service, with the port owner named; the service outcome stays `stopped` and no enum changes. -- `/api/stop`: when the caller presents a valid receipt, the parent `ocx stop` has already run part A, so the route skips part A (no second enumeration, `/end` or wait inside the proxy) and only performs its graceful shutdown; part B belongs to the parent, since the route cannot observe its own post-exit window. Only `ocx stop` and `ocx service stop` write the receipt, and each sends the POST only after its own part A returned `stopped` or `absent` (see "Two parts"). The receipt alone is not the proof: after part A the parent records its outcome in the receipt (`partA: "stopped" | "absent"`, with the bound manager and the wrapper set it ended as pid + creation time). The route skips part A only when that record is present for this attempt and the manager it names is still the registered one; a receipt without the record (an older CLI, or a receipt written before part A) gets today's route behavior, which re-runs the service stop and answers 409 on failure. Without a receipt the existing `respawnable_service` refusal stays for a respawnable Task Scheduler service. Where the respawn risk is `none` the route splits by case. For a registered WinSW service it keeps today's in-process service stop unchanged; part A is not run there, because WinSW's own stop ends this very process and cannot be capped from inside it. For a deleted task with a stray wrapper it runs only part A's wrapper sweep (bind, terminate, handle wait) with a 4 s cap, and then, before it answers, its own shared teardown (`performStopTeardown`, which a caller without a receipt still gets); the drain runs after the answer. Every parent POST gets the full 10 s abort, never a shortened one, and admission always counts it at 10 s; the implementation must measure the teardown's worst case and keep sweep plus teardown under those 10 s, and 040 carries that as a test. A sweep overrun answers `service_state_unknown` and changes nothing further. -- The updater's `stopWindowsServiceWrappersBestEffort` moves to part A in the same change (Task Scheduler backend; for the native WinSW backend it keeps today's behavior of stopping WinSW without a wrapper kill, because WinSW has no `:loop` wrapper), through an authorizer-gated variant (`guardedStopWindowsServiceChecked(authorize)`) that runs the authorizer once at entry and again before each terminate, as the existing `mutationGate` does, so an update cannot race a respawned runtime either. The updater keeps its own reclaim checks. Its three call sites in src/update/job.ts discard the result today, and one runs inside the synchronous `preparePort` callback; each gets a return path (the callback becomes async, or part A runs before it) so a non-`stopped` outcome really aborts the update before files are replaced. +- `/api/stop`: when the caller presents a valid receipt, the parent `ocx stop` has already run part A, so the route skips part A (no second enumeration, `/end` or wait inside the proxy) and only performs its graceful shutdown; part B belongs to the parent, since the route cannot observe its own post-exit window. `ocx stop`, `ocx service stop`, `ocx service uninstall`/`remove`, `ocx uninstall` and service install/switch write the receipt before the child stop, naming the endpoint that stop contacts (the tracked pid's own runtime record or the discovered live proxy), and on Windows a receipt that cannot be recorded means no child stop. Each of them except `ocx uninstall` sends the POST only after its own part A returned `stopped` or `absent` (see "Two parts"); `ocx uninstall` still sends the graceful POST after an unproven part A, but with the forced fallback forbidden and the failure recorded, so shared teardown stays blocked. The receipt alone is not the proof: after part A the parent records its outcome in the receipt (`partA: "stopped" | "absent"`, with the bound manager and the wrapper set it ended as pid + creation time). The route skips part A only when that record is present for this attempt and the manager it names is still the registered one; a receipt without the record (an older CLI, or a receipt written before part A) gets today's route behavior, which re-runs the service stop and answers 409 on failure. Without a receipt the existing `respawnable_service` refusal stays for a respawnable Task Scheduler service. Where the respawn risk is `none` the route splits by case. For a registered WinSW service it keeps today's in-process service stop unchanged; part A is not run there, because WinSW's own stop ends this very process and cannot be capped from inside it. For a deleted task with a stray wrapper it runs only part A's wrapper sweep (bind, terminate, handle wait) with a 4 s cap, and then, before it answers, its own shared teardown (`performStopTeardown`, which a caller without a receipt still gets); the drain runs after the answer. Every parent POST gets the full 10 s abort, never a shortened one, and admission always counts it at 10 s; the implementation must measure the teardown's worst case and keep sweep plus teardown under those 10 s, and 040 carries that as a test. A sweep overrun answers `service_state_unknown` and changes nothing further. +- The updater's `stopWindowsServiceWrappersBestEffort` moves to part A in the same change (Task Scheduler backend; for the native WinSW backend it keeps today's behavior of stopping WinSW without a wrapper kill, because WinSW has no `:loop` wrapper), through an authorizer-gated variant (`guardedStopWindowsServiceChecked(authorize)`) that runs the authorizer once at entry and again before each manager stop and each terminate, as the existing `mutationGate` does, so an update cannot race a respawned runtime either. The updater keeps its own reclaim checks. Its three call sites in src/update/job.ts discard the result today, and one runs inside the synchronous `preparePort` callback; each gets a return path (the callback becomes async, or part A runs before it) so a non-`stopped` outcome really aborts the update before files are replaced. Desktop reaches it through two `ocx stop` children, both ordinary CLI processes run as the user: the consented takeover at startup (`--expect-*`, under the startup sequence's deadline, which `resolve_consent` extends by the consent wait plus 60 s on a Windows approval) and the exit/restart drain (plain `ocx stop --json` under `runtime_stop::DEADLINE`, 30 s, desktop/src-tauri/src/exit.rs). The #5443 boundary is unchanged and Desktop never elevates. @@ -50,17 +54,17 @@ New work, named explicitly (x64): the creation-time reader, the handle terminate Part A: -1. Probe registration: `probeWindowsSchedulerTask` and, for WinSW, the service status (both capped at 2 s; an overrun or unreadable answer is `unknown`, nothing changed). With nothing registered, the process enumeration that looks for a stray wrapper of a deleted task (capped at 5 s) also runs here, before admission. Nothing registered and nothing bound → `absent`, return. +1. Probe registration: `probeWindowsSchedulerTask` and, for WinSW, the service status (both capped at 2 s; an overrun or unreadable answer is `unknown`, nothing changed). A running WinSW service is only this home's manager when its registered `BINARY_PATH_NAME` (`sc qc`) names this installation's WinSW executable: the service ID is fixed, so a successful `sc query` alone would let Part A stop another installation's service. An unreadable registration is `unknown`; a foreign executable is treated as not registered for this home. With nothing registered, the process enumeration that looks for a stray wrapper of a deleted task (capped at 5 s) also runs here, before admission. Nothing registered and nothing bound → `absent`, return. 2. Bind (rule above) and record (pid, startTime, image, command-line digest). 3. Ask the manager. Task Scheduler: `schtasks /end` (cap 2 s). WinSW: a non-waiting stop (the stop command capped at 2 s instead of today's 15 s `SERVICE_COMMAND_TIMEOUT_MS`) followed by a capped SCM state poll (`sc query`) instead of today's `stopwait`; if WinSW reports stopped but the proxy is still alive, the outcome is `survived`; the poll's cap is the service's `` (20 s in the generated definition) plus 2 s, because WinSW itself stops the proxy within its stop timeout. Access denied → `denied`, stop there. A cap overrun → `unknown`, with nothing further changed (no terminate). -4. Terminate bound wrapper survivors through the handle primitive (`cmd.exe` running the loop first, then the launcher). WinSW has no wrapper, so this step is empty there. The proxy child is NOT terminated here: it stops through the existing graceful path (`stopProxy` / `stopWithDeferral` / `signalApproved`) with its drain, the #7423 teardown-deferral receipt, and the rule that a received refusal (409, `ProxyOwnershipRefusedError`) forbids a forced stop. It is force-killed only where today's code already forces after a graceful timeout, never after a refusal. This operation gives no new kill authority over the child. +4. Terminate bound wrapper survivors through the handle primitive (`cmd.exe` running the loop first, then the launcher). WinSW-only registration also checks for wrappers left by a previous backend. The proxy child is NOT terminated here: it stops through the existing graceful path (`stopProxy` / `stopWithDeferral` / `signalApproved`) with its drain, the #7423 teardown-deferral receipt, and the rule that a received refusal (409, `ProxyOwnershipRefusedError`) forbids a forced stop. It is force-killed only where today's code already forces after a graceful timeout, never after a refusal. This operation gives no new kill authority over the child. 5. Wait (cap 3 s) until every bound wrapper has exited, by waiting on the process handles opened at bind time (no enumeration, no PowerShell during the wait), then read the manager state once with `schtasks /query` / `sc query` only (cap 1 s). The CIM enumeration that would find a wrapper started after bind is NOT repeated here; it runs once at the end of part B, where a late relaunch is what it has to catch. ## Time budget (one deadline, checked before anything changes) Windows only. The async `stopManager` change reaches every platform, but macOS and Linux keep today's timing and ignore the deadline. -`ocx stop` takes the deadline as an input, `--deadline-ms ` or the environment variable `OCX_STOP_DEADLINE_MS` (decimal milliseconds), measured from its own process start. Desktop passes it through the environment variable only, because an older CLI rejects an unknown flag and ignores an unknown variable, and does not set a value that would not be positive. It passes it on both paths: for the consented takeover, the remaining startup deadline minus a 5 s margin for the silence probe and `claim::run` that Desktop still runs under that deadline after the stop (desktop/src-tauri/src/startup.rs); for the exit drain, its deadline minus a 3 s spawn/parse margin. Without either (a terminal, a script, an older Desktop) the deadline is 60 s. The other callers (`ocx service stop`, `ocx uninstall`, service uninstall/remove, install/switch) are interactive and use 60 s; the updater uses its job budget. +`ocx stop` takes the deadline as an input, `--deadline-ms ` or the environment variable `OCX_STOP_DEADLINE_MS` (decimal milliseconds), measured from its own process start — that is the operator's budget, and it is the only caller for which process start and command start are the same instant. Every other caller starts the default 60 s when its stop phase begins (PR2, 060): it reaches the stop after work it cannot charge to the stop, so a budget measured from process start can already be spent. Desktop passes it through the environment variable only, because an older CLI rejects an unknown flag and ignores an unknown variable, and does not set a value that would not be positive. It passes it on both paths: for the consented takeover, the remaining startup deadline minus a 5 s margin for the silence probe and `claim::run` that Desktop still runs under that deadline after the stop (desktop/src-tauri/src/startup.rs); for the exit drain, its deadline minus a 3 s spawn/parse margin. Without either (a terminal, a script, an older Desktop) the deadline is 60 s. The other callers (`ocx service stop`, `ocx uninstall`, service uninstall/remove, install/switch) are interactive and use 60 s; the updater uses its job budget. Admission. Part A's registration probe (step 1) runs first, so a machine with nothing registered returns `absent` and is never refused for budget. Then, before step 2 and after every pre-step has already run (on the consented path: `runResolve`, `describeTrackedRuntimeTarget` and the first `inspectGuardedManagerTarget` inside `runApprovedStop`, then `revalidateManager`, each with its existing CIM 5 s timeout and `schtasks`; their actual elapsed time is what counts), the stop computes the worst case of everything still to come from the table below. If the remaining time cannot hold it, the stop changes NOTHING (no `/end`, no WinSW stop, no terminate, no POST) and reports `serviceStopReason: "unverified-budget"` with `manager-still-active` (guarded) or `failed` (snapshotless). Every step after admission has a cap, and a step without a natural bound today gets one (below); the caps sum to at most the remaining time, so the graceful exit wait is never shortened and the stop does not overrun. @@ -69,21 +73,21 @@ Worst case still to come, at admission: | Step | Worst case | Notes | |---|---|---| | part A, Task Scheduler | guarded 6 s (`/end` 2 s, handle wait 3 s, manager state 1 s; the bind reuses `revalidateManager`'s enumeration); snapshotless 11 s (its own enumeration, 5 s, plus the same 6 s) | the registration probe already ran before admission | -| part A, WinSW | 25 s: stop command 2 s + SCM poll `` + 2 s = 22 s + manager state 1 s | WinSW stops the proxy itself, so the POST and exit-wait rows below do not apply to it. The stray enumeration of step 1 does not apply either: it runs only when nothing is registered | -| orphan discovery (snapshotless, no PID file; also after WinSW stopped the proxy) | 3 s | `findLiveProxy` and `proveLiveProxyOwnedByHome`, capped at 3 s together; an overrun is treated as "no proven orphan" and refuses, as an unproven owner does today | +| part A, WinSW | 33 s: enumeration 5 s + wrapper termination/wait 3 s + stop command 2 s + SCM poll `` + 2 s = 22 s + manager state 1 s | WinSW stops the proxy itself, so the POST and exit-wait rows below do not apply to it. Enumeration also checks ancestry and stray wrappers when WinSW is registered but stopped. | +| orphan discovery (snapshotless, no PID file; also after WinSW stopped the proxy) | 3 s | `findLiveProxy` and `proveLiveProxyOwnedByHome`, capped at 3 s together; an overrun is treated as "no proven orphan" and refuses, as an unproven owner does today. The cap is threaded to the helper (`orphanDiscoveryCapMs`) so a checked Windows caller passes exactly the 3 s admission priced; the helper's own default stays the legacy 7 s for macOS and Linux | | POST answer | 10 s, always | never shortened. The receipt and its part A record are written after admission (the snapshotless receipt inside `stopWithDeferral`), so admission cannot know whether they will exist and always counts the full abort. With the record the route skips part A and answers at once; without it (the write failed, or an older CLI) the route keeps today's behavior inside the same 10 s | | child exit: graceful exit wait, then forced | `shutdownTimeoutMs` + 3 s + 10 s = 18 s by default | today's ladder, kept in sequence: the graceful exit wait (never shortened) runs after an answered POST; if the child has not exited, or the POST aborted, `stopProxy` forces (never after a refusal): `taskkill` with a new 5 s timeout (it has none today) plus today's `waitForExit` 5 s | | inherited-receipt probes | 3 s × N | N abandoned receipts, known at admission; each probe tries both loopback hosts at today's 1.5 s per host (`probeEndpointLiveness`) | -| part B | 6 s floor, up to `min(12 s, remaining − 5 s − reserve)` | `waitForStoppedPort` (today a separate 15 s wait inside `stopProxy`) is the same port-free predicate and on Windows becomes part B's first phase; `settle`'s `waitExit` and endpoint probe are its first sample; the final CIM enumeration (`managerState` and `guardFinalStopSummary` share it) runs after the hold, capped at 5 s, and is counted: part B's worst case is 6 s + 5 s = 11 s. An unreadable result or a wrapper found that was not there at bind is non-success. Not run after a guarded `absent` manager (where `stopManager` is not called, as today). When admission lands exactly at the minimum, part B has no slack beyond its floor, so a slow port release reports failure instead of overrunning. Part B keeps today's `reclaimListenPort` clearing of a listener left by the exited child, and a listener whose owner has exited is not classified as `portHeldByForeign` | +| part B | 6 s floor, up to `min(12 s, remaining − 5 s − reserve)` | `waitForStoppedPort` (today a separate 15 s wait inside `stopProxy`) is the same port-free predicate and on Windows becomes part B's first phase; `settle`'s `waitExit` and endpoint probe are its first sample; the final CIM enumeration (`managerState` and `guardFinalStopSummary` share it) runs after the hold, capped at 5 s, and is counted: part B's worst case is 6 s + 5 s + the 0.5 s closing sample = 11.5 s. An unreadable result or a wrapper found that was not there at bind is non-success. Not run after a guarded `absent` manager (where `stopManager` is not called, as today). When admission lands exactly at the minimum, part B has no slack beyond its floor, so a slow port release reports failure instead of overrunning. Part B keeps today's `reclaimListenPort` clearing of a listener left by the exited child, and a listener whose owner has exited is not classified as `portHeldByForeign` | | reserve | 3 s | shared restore (`restoreSharedClientStateAfterStop`; its worst case is measured in 040 and the reserve grows if it does not fit), `revertSystemEnv` (a real 1 s cap on what is a synchronous call today; an overrun is logged and left for the next start), and the JSON summary | -Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 + 18 + 11 + 3 = 48 s; snapshotless Task Scheduler 11 + 3 + 10 + 18 + 11 + 3 = 56 s; WinSW snapshotless 25 + 3 + 11 + 3 = 42 s. The stray enumeration of a deleted task (5 s) runs in step 1, before admission, and so comes out of the time left at admission. +Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 + 18 + 11.5 + 3 = 48.5 s; snapshotless Task Scheduler 11 + 3 + 10 + 18 + 11.5 + 3 = 56.5 s; WinSW snapshotless 33 + 3 + 11.5 + 3 = 50.5 s. Part B is counted at 11.5 s: the 6 s hold, the 5 s final enumeration, and the 0.5 s closing port sample that can start at the very end of the window (the hold is measured between sample starts; 060 "Where PR1 differs from 030"). The stray enumeration of a deleted task (5 s) runs in step 1, before admission, and so comes out of the time left at admission. | Caller | Deadline | Fits? | |---|---|---| | consented takeover (`--expect-*`) | remaining startup deadline − 5 s (a Windows approval adds 60 s) | yes in practice; otherwise refused before anything changes | -| Desktop exit/restart drain (plain `ocx stop --json`) | today `runtime_stop::DEADLINE` − 3 s = 27 s | NO for a registered service. The same change raises Desktop's exit-drain deadline on Windows to 75 s (72 s passed as `OCX_STOP_DEADLINE_MS`). That holds the largest case (56 s) plus an 8 s allowance for what runs inside the deadline before admission (CLI start, the 2 s registration probe, and in the stray case the 5 s step-1 enumeration) plus two inherited receipts (3 × 2 = 6 s): 56 + 8 + 6 = 70 s ≤ 72 s; beyond that the stop is refused up front and the operator stops from a terminal. An older Desktop passes no flag, the CLI uses 60 s and can outlive Desktop's 30 s wait, which ends in `DrainVerdict::Failed`; today's stop can already outlive it (10 s POST + 8 s exit wait + 15 s port wait), so this is not new, and it is listed in 050 | -| terminal, scripts, other interactive callers | 60 s | yes | +| Desktop exit/restart drain (plain `ocx stop --json`) | today `runtime_stop::DEADLINE` − 3 s = 27 s | NO for a registered service. The same change raises Desktop's exit-drain deadline on Windows to 75 s (72 s passed as `OCX_STOP_DEADLINE_MS`). That holds the largest case (56.5 s) plus an 8 s allowance for what runs inside the deadline before admission (CLI start, the 2 s registration probe, and in the stray case the 5 s step-1 enumeration) plus two inherited receipts (3 × 2 = 6 s): 56.5 + 8 + 6 = 70.5 s ≤ 72 s; beyond that the stop is refused up front and the operator stops from a terminal. An older Desktop passes no flag, the CLI uses 60 s and can outlive Desktop's 30 s wait, which ends in `DrainVerdict::Failed`; today's stop can already outlive it (10 s POST + 8 s exit wait + 15 s port wait), so this is not new, and it is listed in 050 | +| terminal, scripts, other interactive callers | 60 s, from the start of the stop phase (PR2; `ocx stop` measures from command start) | yes | | `/api/stop` route | receipt with a part A record: no part A; receipt without the record: today's service stop, now the async part A, capped at the route's 6 s (an overrun answers `service_state_unknown`); WinSW: today's in-process stop; stray of a deleted task, no receipt: 4 s sweep (its enumeration capped at 4 s instead of today's 5 s; an overrun answers `service_state_unknown`, which fails safe) + teardown before the answer | the parent's 10 s POST abort, never shortened | | updater | job budget | yes | diff --git a/devlog/_plan/261111_windows_service_stop/040_verification.md b/devlog/_plan/261111_windows_service_stop/040_verification.md index 7bc683508b9..a0b2dd6d17a 100644 --- a/devlog/_plan/261111_windows_service_stop/040_verification.md +++ b/devlog/_plan/261111_windows_service_stop/040_verification.md @@ -19,7 +19,7 @@ - `ocx uninstall` runs part B after a part A `stopped` and keeps client configuration on any other outcome; service uninstall/install refuse before deleting/replacing the task on a non-stopped outcome. - `/api/stop` with a receipt skips part A; the whole consented stop fits the stated budget with injected clocks. - Guarded path: a stray wrapper bound by the rule is `bound-stray` and reaches part A; an unbindable one stays approval-changed. -- Output mapping per path: `ServiceStopOutcome`, `ocx-stop/1` (no new outcome/proxy enum values; additive fields only), `/api/stop` codes, Desktop parser unit tests with old and new documents; updater variant refuses without authorization at entry and before each terminate. +- Output mapping per path: `ServiceStopOutcome`, `ocx-stop/1` (no new outcome/proxy enum values; additive fields only), `/api/stop` codes, Desktop parser unit tests with old and new documents; updater variant refuses without authorization at entry, before each manager stop and before each terminate. - Plain `ocx stop`: part A `denied`, `survived` and `unknown` each send no `/api/stop` POST, set `ownershipBlocked`, skip the shared restore, and record `outcome: failed`, `proxy: unknown`, `sharedTeardown: skipped`. - `absent`: no registered task or WinSW service and no bound wrapper → `ServiceStopOutcome` `absent`, no "Service manager stopped.", the POST is sent, part B does not run (no 6 s hold); the same with an unreadable process list → still `absent`, and plain `ocx stop` still stops the proxy. - Budget, with injected clocks: admission refuses with `unverified-budget` and changes nothing (no `/end`, terminate or POST) when the remaining time is below the worst case still to come; once admitted, every step at its cap ends before the deadline; the graceful exit wait is never shortened; part B never gets less than 6 s; `--deadline-ms` from both Desktop paths is honored, and a missing flag means 60 s; macOS/Linux ignore it. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 85e153ad61d..8bf7916cbb3 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -37,14 +37,14 @@ Contract doc: `structure/ops/windows-checked-stop.md`. ## Where PR1 differs from 030 -- **Part B's closing sample.** The 6 s hold is measured between port sample starts, and the window floor is also 6 s, so the last sample starts at the window's end and can run up to its 0.5 s cap past it. The budget counts this, which raises the admission minimums to 48.5 / 56.5 / 42.5 s. The Desktop drain case becomes 70.5 s, which still fits in 72 s. -- **WinSW binds nothing.** WinSW-only part A neither enumerates nor binds. This follows 030's step 4 and its 25 s row: a stray scheduler wrapper left behind by an earlier backend is caught only when nothing is registered. "Proxy still alive after WinSW stopped" is an injected `proxyAlive` check inside the 1 s manager-state step. An answer of "cannot tell" leaves the decision to part B. +- **Part B's closing sample.** The 6 s hold is measured between port sample starts, and the window floor is also 6 s, so the last sample starts at the window's end and can run up to its 0.5 s cap past it. The budget counts this; the current admission minimums are 48.5 / 56.5 / 50.5 s, including the later WinSW wrapper check. The Desktop drain case becomes 70.5 s, which still fits in 72 s. +- **WinSW wrapper evidence.** WinSW-only Part A enumerates and binds wrappers, including with the service stopped, so the own-wrapper guard and stray evidence apply. "Proxy still alive after WinSW stopped" is an injected `proxyAlive` check inside the 1 s manager-state step. An answer of "cannot tell" leaves the decision to Part B. - **Terminates share the wait cap.** 030's tables do not price the terminate calls themselves. They share the 3 s handle-wait window, which matters for the ARM64 script (about 1 s each). - **Unbound wrappers with nothing registered.** Binding finds no bound wrapper but an unbound one is alive. The result is `survived` / `unbound-wrapper`, not `absent`, matching 040's bullet; the 030 "absent" definition reads as "nothing bound" alone. Nothing is changed either way. - **Unreadable candidates with nothing registered.** An unreadable wrapper image above the child counts only while a manager is registered. A hand-started proxy under an elevated shell would otherwise lose the stop it gets today. A matching wrapper without a creation time stays `unknown` even with nothing registered. - **Snapshot type.** The snapshot type is `WindowsStopProcessEntry`, which is `WindowsProcessEntry` plus `startTime`, in the evidence module. `windowsProcessList` and the guarded inspection stay untouched until PR2 switches them. - **The Schedule PID is read before admission.** Part A reads the Task Scheduler host PID from the SCM (cap 2 s) right after the registration probe, before admission, whenever a task is registered or a stray sweep runs. 030's tables do not list this read. Admission measures the time left on the caller's deadline, so the read's actual elapsed time is already taken out of what admission sees, the same way the step 1 stray enumeration is. -- **Not every effect is capped.** The registration probes, enumeration, Schedule PID, `/end`, WinSW stop and poll, terminates, manager state and `proxyAlive` run under caps. `openWatch` and the authorizer do not. `openWatch` is one `OpenProcess` per wrapper with no wait in it. The authorizer runs before each terminate, inside the shared 3 s terminate-and-wait window, so a slow authorizer eats that window: the next terminate gets less time, and an authorizer that never returns hangs Part A. PR2's authorizer (the updater's `mutationGate` check) has to be bounded by its caller. +- **Not every effect is capped.** The registration probes, enumeration, Schedule PID, `/end`, WinSW stop and poll, terminates, manager state and `proxyAlive` run under caps. `openWatch` and the authorizer do not. `openWatch` is one `OpenProcess` per wrapper with no wait in it. The authorizer runs before each manager stop and before each terminate, inside the shared 3 s terminate-and-wait window, so a slow authorizer eats that window: the next terminate gets less time, and an authorizer that never returns hangs Part A. PR2's authorizer (the updater's `mutationGate` check) has to be bounded by its caller. - **Fractional timeouts.** A cap derived from the deadline (`windowEnd - now`) is fractional on the real clock. The spawned terminate fallback floors it to whole milliseconds before building the script and calling `execFile`, and spawns nothing when it floors to 0, because `execFile` reads a 0 timeout as "no timeout". ## ARM64 decision for PR2 @@ -55,8 +55,52 @@ Contract doc: `structure/ops/windows-checked-stop.md`. - **The route's caps.** 030 caps the `/api/stop` route's stray sweep at 4 s and its receipt-less Part A at 6 s. `stopWindowsServiceChecked` has no overall cap: its own caps add up to more (the stray path alone has a 5 s enumeration), and it never returns before its steps do. The route needs an outer race on its deadline that answers `service_state_unknown` on overrun, or cap overrides passed in, such as a 4 s enumeration for the sweep. +## Where PR2 differs from 030 + +PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("Callers") is the contract. Points where the wiring makes a decision 030 leaves open, or departs from it: + +- **Route caps.** The outer race lives in `runWindowsServiceStopPartA({ capMs })`: an overrun answers `unknown` at once, and from then on the live `/end` and WinSW stop are gated off and the authorizer refuses every terminate, so nothing changes after the answer. A step that already started (a `/end` in flight) still completes. +- **The receipt record is checked against registration only.** The route skips Part A when the record's managers match a fresh registration probe (2 s each). It does not re-enumerate the recorded wrapper set; that would be the second enumeration 030 rules out. +- **The ancestry guard (r2 review).** The binding rule excludes the stopper but not its ancestors, so interactive callers refuse a Part A whose stopper descends from this home's wrapper. `/api/stop` and the updater opt out because both run beneath that wrapper by design. The check reads the bind snapshot (or the guarded snapshot), so it costs no extra enumeration. +- **Updater WinSW stop.** The updater now runs the same authorized checked Part A for both backends and aborts unless the result is stopped or absent; a WinSW error cannot be treated as success. +- **Absent service stop.** Before restoring shared config, `ocx service stop` proves every candidate endpoint dead; live or unknown evidence keeps the receipt. +- **Receipt target.** Service stop, removal and install resolve the proxy once, claim only for that target's exact endpoint, and send the same endpoint to the child stop. No target means no receipt or POST. +- **The guarded evidence snapshot (r1 review).** The guarded inspection reads `windowsProcessSnapshot` and the Schedule PID once per inspection instead of the CIM-only `windowsProcessList`, and `stopManager` binds on the revalidation's copy. A CIM-only table would make the x64 terminate answer `reused` and Part A report a live wrapper as exited. +- **Absent counts as a pass for the updater and for install.** 040 says call sites abort on a non-`stopped` outcome. `absent` (nothing registered, nothing bound) has nothing that could relaunch the proxy, so the updater, install and switch continue on it, as `ocx stop` does. Refusing it would block every update on a machine whose recorded service was deleted. +- **`ocx service stop` on Part A failure** refuses before the receipt and the child stop, the same rule as plain `ocx stop`, so it never restores under a wrapper that can relaunch the proxy. +- **Part B's foreign result** fails install and switch (030) but not `ocx stop`, `ocx service stop` or uninstall, which only need our processes gone. +- **Not capped here.** The guarded path's own pre-steps (`runResolve`, the tracked-record check, the two inspections and their 5 s snapshot) run before admission and are measured, as 030 says. `proveLiveProxyOwnedByHome` is not capped separately; orphan discovery's `findLiveProxy` is capped at 3 s, and a missed orphan is caught by Part B. `revertSystemEnv` returns at once off macOS, so its 1 s cap holds on Windows without a change. `restoreSharedClientStateAfterStop` and `performStopTeardown` were not measured on a device; 040 keeps those measurements. +- **The manager-state read.** `schtasks /query /fo CSV` first, `Get-ScheduledTask` within the same 1 s cap when the status word is localized. On a slow or non-English machine that can overrun and report `unknown`; the 040 device run decides whether the cap needs to grow. +- **ARM64 ships strict** as recorded above; `windowsStopTerminate` enforces it whenever the native API is missing. +- **The deadline's origin depends on the caller.** 030 prices `ocx stop` from process start, which is right for an operator's command and wrong for every other caller: `installFreshWindowsSchedulerSafely` reaches the stop only after registration and the UAC prompt, so a 60 s budget measured from process start can already be spent and admission refuses `unverified-budget` a stop that has its whole budget left. `windows-stop-deadline.ts` therefore keeps an explicit command deadline (set by the dispatcher from `--deadline-ms`/`OCX_STOP_DEADLINE_MS`) separate from a phase deadline that the first read in a stop phase arms. One deadline governs a phase; a later read does not slide it forward, and setting or clearing the command deadline re-arms. +- **The port wait is skipped on both Windows paths.** `windowsPartBDue` is snapshotless-only because it also gates the legacy respawn re-poll, so keying `skipPortWait` to it left the guarded path and `ocx uninstall`'s two `stopProxy` calls running the legacy 15 s `waitForStoppedPort` on top of Part B. `windowsPartBOwnsPortWait` (any Windows Part A `stopped`) now drives `skipPortWait`; the respawn semantics stay snapshotless. +- **Service removal claims its receipt first.** Part A leaves the scheduler task registered, so `stopTrackedProxyForServiceCommand`'s receipt-less POST answered `respawnable_service` (409) and the removal refused. `windowsServiceRemovalStop` now claims a receipt carrying Part A's record before the child stop, keeps it until the shared restore succeeds, and sends no child stop when the claim fails — the same rule `ocx service stop` already had. +- **Orphan discovery takes the cap its caller admitted.** `stopTrackedProxyIfRunning` hardcoded 7 s, which is more than the 3 s admission priced, so discovery could overrun the budget admission had just proven and Part B's window with it. The helper takes `orphanDiscoveryCapMs`; the checked Windows callers pass `WINDOWS_STOP_CAPS.orphanDiscovery` and the default keeps macOS and Linux unchanged. +- **Listener identity is tri-state.** `verifyPidIdentity`'s null covers both "not an ocx process" and "the probe could not answer", and mapping it to `foreign` reported a listener we merely could not read as another program holding the port — `portHeldByForeign` then let the caller restore shared config with the proxy possibly alive. Only `inspectOcxStartProcessIdentity`'s `mismatch` is foreign; `unverifiable` is unknown. +- **The Part B port sample honors its timeout.** `sampleWindowsStopPort` called the synchronous `scanListenPids` and ignored `timeoutMs`, so the closing sample could stall the stop past its cap. It is now async over `netstat.exe` through `execFile` with the caller's timeout and `SIGKILL`; a timeout or an unreadable scan samples `unknown`, and the timeout is floored because `execFile` reads 0 as unlimited. +- **A not-running task needs its registration proof before `bound-stray`.** The state word says the task is inert, not that it is ours; promoting on the word alone let a same-named foreign task make this home's hand-started wrappers terminable. The promotion now follows `winTaskXml` and `winRegistrationOurs`, and anything else keeps the previous verdict. + +- **Uninstall's proxy stop claims its receipt before the child stop.** `handleUninstall` sent `/api/stop` without a teardown receipt, so the route finished the shared teardown while the wrapper respawn window had not been checked yet; a failed `verifyUninstallStopWindow` then left shared config restored under a wrapper that could relaunch the proxy. On Windows the uninstall proxy stop is now `src/cli/uninstall-stop.ts`: it claims a receipt carrying Part A's record before the child stop, sends its nonce, and discharges it only after Part B and the shared restore. A failed claim sends no child stop; a failed later step keeps the receipt for recovery. +- **Uninstall never forces a stop its Part A could not prove.** When Part A did not end `stopped` or `absent`, the uninstall proxy stop still sends the graceful POST but passes `allowForcedFallback: false` to `stopProxy`, so a POST timeout, transport error or non-409 answer cannot fall through to `taskkill /T /F`. The uninstall failure stays recorded. A 409 refusal already forbids the forced path; this closes the remaining ones. +- **An absent Part A is not proof the proxy is down.** `windowsServiceRemovalStop` returned `ok` on Part A `absent` without checking the endpoint, and `findLiveProxy` answers null when its 3 s discovery cap runs out even while the endpoint is live. Removal now probes every candidate endpoint (the runtime record and the configured endpoint, the same set `ocx uninstall` proves, plus the resolved stop target) with the tri-state contract (`endpointLiveness`, the same one `endpointProvenDown` uses in `uninstall-stop` and `proxyEndpointProvenDown` in `src/cli/index.ts`); `proxyStillLiveAfterStop` looked down on a null, which its own probe produces on a timeout as well. Only `"dead"` on every candidate permits deleting the task — `"live"` refuses and `"unknown"` (a timeout or transport failure) refuses too, and both keep the receipt. No respawn hold is needed: a wait would price removal at Part B's 6 s for a manager that is not registered. +- **Uninstall's receipt names the endpoint its stop contacts.** `stopUninstallProxy` claimed the receipt first and resolved its target afterwards, and `defaultReceiptEndpoint()` marks any runtime record "exact" — so a stale record plus a proxy discovered on another port left a receipt for a port the stop never contacted, and a failed Part A with no proxy left a "guessed" obligation outstanding that recovery refuses. It now resolves the target first (the tracked pid with its own runtime endpoint, or the discovered live proxy's endpoint), claims only when a stop will be attempted, and hands the receipt and `stopProxy` the same endpoint. No target, no receipt. +- **An unconfirmed stop removes no tracked state.** The tracked-pid path called `removeTracked(pid)` regardless of the stop's result, so a failed stop deleted the record that a rerun needed. It runs only when the stop was proven, and the step reports `changed` only when state actually changed. +- **A forbidden forced fallback waits without killing.** With `allowForcedFallback === false`, `waitForStoppedPort` still called `reclaimListenPort` with `killOcxHolders`/`onlyKillPids: [pid]`, so the wait could kill through `killProxy` — the path the flag exists to close. The wait now runs observationally (the reclaim dependency is injected, and the forbidden case passes a mode that samples without kill eligibility); `stopProxy` keeps its literal `killProxy(pid)` call, which `tests/providers/xai/grok-lifecycle.test.ts` reads as order evidence. +- **A foreign WinSW registration survives `ocx uninstall`.** Two gaps: on Windows the "service removed" step called `uninstallServiceDetailed()` even when Part A failed to stop the manager, and the native branch stopped and deleted whatever carried the fixed `opencodex-proxy-native` ID — including another installation's service. Removal is now gated on a Part A that ended stopped or absent (`serviceRemovalAllowed`, Windows only; other platforms unchanged), recording `failed` otherwise so the shared teardown stays blocked as `ocx service uninstall` already refuses. Immediately before the native removal, `uninstallServiceDetailed` re-reads `sc qc` `BINARY_PATH_NAME` and compares it with this home's WinSW executable: foreign leaves the registration untouched, unreadable refuses and keeps install state. +- **Install preparation resolves the stop port before Part A.** `windowsServiceStopPort()` ran after the tracked cleanup had already removed the runtime record, so Part B confirmed the wrapper's baked port (10100) while the proxy's runtime port (12000) was never checked. `prepareServiceInstall` resolves the port once before Part A (`windowsStopPort` is the test seam) and carries it into Part B. +- **Part A proves the WinSW registration is this home's.** `sc query` answering "running" says a service exists under the fixed `opencodex-proxy-native` ID, not that it is this installation's. The live probe now reads `BINARY_PATH_NAME` through `sc qc` and requires this home's `winswExePath()`, the same proof `inspectWindowsWinswManager` uses; an unreadable registration is `unknown` (`evidence-unreadable`, nothing changed) and a foreign executable is reported as `nonexistent`, so Part A does not stop it and reports `absent`. +- **Install preparation claims its receipt before the stopped-Part-A stop.** Part A leaves the scheduler task registered, so the tracked stop's receipt-less POST answered `respawnable_service` (409), `stopProxy` threw, and the install aborted while the old proxy kept serving. `prepareServiceInstall` now claims a receipt carrying Part A's record before the stop and forwards its nonce. An install replaces the proxy, so shared config stays pointed at OpenCodex and `installServiceSafely` (and the fresh-scheduler path) clears the receipt only after the new service is installed and started; an install that fails after the stop keeps it for the next `ocx stop`. A failed claim sends no child stop. +- **An absent Part A still proves every endpoint down before install.** With no manager registered, Part B is skipped and the tracked stop can return `"none"` after its orphan-discovery cap while an unresponsive proxy still holds the port. `prepareServiceInstall` resolves `endpointsToProve(readRuntimePort(), loadConfig())` before cleanup and requires every candidate to be definitively dead; live or unknown throws like a Part B failure. The candidate and tri-state helpers live in `windows-service-stop.ts` so removal and install share them. +- **Windows removal probes before it deletes.** `uninstallServiceDetailed` deleted the scheduler task before reading the WinSW status and `sc qc` ownership, so an unreadable probe left a half-removed install with no task to retry from. Both probes now run first; unknown refuses with the existing error and removes nothing. +- **An unconfirmed uninstall stop asks the endpoint.** `stopProxy` answers false without contacting the endpoint when the tracked pid is already dead, so `confirmed(false)` reported `provenDown` with no endpoint check and a different proxy on the configured port was missed. The step now requires the tri-state `endpointProvenDown()` unless the forced fallback is forbidden, and still never forces. +- **Part B clears only the exited owner's TCP rows.** After a listener PID was found exited, the sampler called `dropWindowsTcpRowsForLocalPort(port)`, which resets every IPv4 row on the port — including rows of a program that took the port between scans. `parseTcpQuadsForLocalPort` and `dropWindowsTcpRowsForLocalPort` take an owner PID filter and the sampler passes the exited PID(s); the hard-kill path still clears the whole port. + +## Follow-up slices + +- **`ocx service repair` still stops through `stopWindows()`** (`src/service/repair.ts:259`), the unguarded `schtasks /end` path. Repair does not restore shared client state, so the #3008 relaunch window the checked stop closes does not apply and it is not wired here. Its own slice: decide whether repair needs Part A (it re-registers the task immediately after, so a surviving wrapper can be rebound by the new definition) and, if so, which Part B applies when repair then starts the task again. + ## Open questions - Is the closing-sample accounting acceptable, or should the hold be measured differently, for example by shortening the final enumeration to 4.5 s to keep 030's totals? -- Should a WinSW-only stop also sweep a stray scheduler wrapper (that would add an enumeration and 3 s to the WinSW row)? +- WinSW-only stops enumerate and bind wrappers as well; the WinSW row includes the enumeration and termination window. - `GetLastError` after a failed `OpenProcess` through Bun FFI separates "gone" from "denied". If the runtime clobbers it, the answer degrades to "unavailable" (`unknown`). The live x64 test in `tests/windows/windows-process-evidence.test.ts` checks this on the Windows CI lane. diff --git a/docs-site/src/content/docs/reference/cli/lifecycle.md b/docs-site/src/content/docs/reference/cli/lifecycle.md index f287581c217..bda02308126 100644 --- a/docs-site/src/content/docs/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/reference/cli/lifecycle.md @@ -80,7 +80,7 @@ ocx start --port 10100 --socks5 ocx start --socks5-off ``` -### `ocx stop [--json]` +### `ocx stop [--json] [--deadline-ms ]` An unsupervised Link Child handling a shutdown signal restores native Codex and managed Grok before releasing its listener, rechecking the served connection before each write. Service/Desktop supervision, sibling ownership, restart/recycle handoff and pending parent stop receipts preserve integrations. Shutdown rejects bound restart with `409 runtime_stopping`. Forced termination can bypass signal handlers; parent `ocx stop` restoration remains necessary. To route Codex through Home again after such a stop, start the Child (`ocx start`, or `ocx service start` for an installed service) and then run `ocx sync --restart-codex`. Starting alone does not reapply routing. A sync run while the Child is stopped still reapplies routing, which points Codex at a listener that is not running, so start the Child first. @@ -141,6 +141,46 @@ and its catalog to the restarted proxy instead of restoring native Codex. With 8 down, and the restarted proxy releases the kept teardown after its first sync; if it never starts, the next `ocx stop` restores native Codex. +#### Windows service stop + +On Windows, `ocx stop` stops a registered Task Scheduler task or native WinSW service with a +checked stop. It ends the task, then ends only the `wscript.exe` and `cmd.exe` wrapper processes +it can tie to this installation (this home's launcher or service script, a parent chain that leads +to the proxy or to the Task Scheduler service, and a recorded start time), and waits for them to +exit. Only then does it stop the proxy gracefully. Afterwards it confirms that the port stays free +for about six seconds and that no wrapper started again, so a wrapper's restart loop cannot bring +the proxy back behind a successful stop. + +When the service cannot be proven stopped, `ocx stop` sends no stop request to the proxy, leaves +your client configuration alone, and fails. With `--json` the summary keeps `outcome: "failed"` +and `proxy: "unknown"`, and adds `serviceStopReason`: + +| `serviceStopReason` | Meaning | What to do | +|---|---|---| +| `access-denied` | Ending the task or a wrapper was denied, usually an elevated task stopped from an unelevated terminal | Run `ocx stop` from an elevated terminal | +| `unbound-wrapper` | A wrapper running this home's service script could not be tied to the service, for example one started by hand | End that `wscript.exe`/`cmd.exe`, then retry | +| `evidence-unreadable` | The service or process state could not be read, or the port could not be confirmed free | Check `ocx service status`, then retry | +| `unverified-budget` | The time left could not hold a verified stop, so nothing was changed | Retry from a terminal | + +If another program is listening on the port after this installation's processes are gone, the +stop still succeeds and `--json` reports `portHeldByForeign: true`. Both fields are additive and +no new `outcome` value exists. `serviceStopReason` is omitted when there is nothing to report; +`portHeldByForeign` appears (`true` or `false`) whenever the port check ran, and is omitted otherwise. + +`--deadline-ms ` bounds the whole stop, measured from the moment the command started. Before +changing anything the stop checks that the time left can hold its worst case (about 57 seconds for +a registered Task Scheduler service with default settings) and otherwise refuses with +`unverified-budget`. Without the flag the stop reads `OCX_STOP_DEADLINE_MS` (the desktop app sets +it) and otherwise uses 60 seconds. macOS and Linux ignore both. + +The stop ends the running instance only. The task's logon and unlock triggers stay enabled, so the +proxy comes back at the next logon or unlock; use `ocx service uninstall` to remove the service. +`ocx service stop`, `ocx service uninstall`, `ocx uninstall` and `ocx service install` use the +same checked stop: uninstall and install refuse before deleting or replacing the task when the +service cannot be proven stopped, and install also stops when another program holds the port. On +Windows on Arm the wrappers cannot be ended yet, so a stop that would need to end one reports +`evidence-unreadable`; end the wrapper by hand, then retry. + For a desktop-approved takeover, `ocx stop --json` also accepts the complete set of `--expect-*` values supplied by `ocx resolve --json`. On Windows, this guarded stop can stop a Task Scheduler task only when its registered definition, running state, and wrapper diff --git a/docs-site/src/content/docs/reference/management-api.md b/docs-site/src/content/docs/reference/management-api.md index 245f81315e2..1188048a14a 100644 --- a/docs-site/src/content/docs/reference/management-api.md +++ b/docs-site/src/content/docs/reference/management-api.md @@ -872,7 +872,7 @@ the 503 without reporting model capacity; provider overload errors retain their | --- | --- | --- | | `GET /api/system/memory` | Return scalar process, heap, stream, response-state, watchdog, and active-turn metrics. Response-state diagnostics include spill-write status, consecutive failures, fixed privacy-safe failure class, and last failure/success timestamps. `spillLastWriteFailureOrigin` is `retry_returned_timeout`, `timeout_memo_refusal`, or null; cumulative `spillAclRetryReturnedTimeouts` and `spillAclTimeoutMemoRefusals` count terminal failed publications; cumulative `spillCapacityRefusals` and `spillHeadroomEvictions` count continuations refused at the spill ceiling and continuations evicted to make room for a newer one. See [Windows spill diagnostics](/troubleshooting/windows-memory/) for process-local semantics. Raw errors and paths are never returned. | — | | `POST /api/system/restart` | Begin a drain-aware process restart without removing client injection; optional JSON `drainGraceMs` explicitly shortens the default 60s grace | Returns 202; repeated calls report the existing grace; 400 invalid body or grace | -| `POST /api/stop` | Stop the service, restore native Codex, remove managed Grok injection, and drain the proxy | 409 service ownership conflict; 409 `respawnable_service` when a Windows Task Scheduler wrapper could respawn the proxy and the caller is not `ocx stop` (nothing is changed); 409 `self_unload_service` when this proxy is running as the installed launchd/systemd service, because stopping the manager from inside it would end the process before native Codex is restored — run `ocx stop` instead (nothing is changed); 409 when the installed manager refuses to stop; 409 `service_state_unknown` when the Task Scheduler state cannot be read (nothing is changed; repair the query and retry) | +| `POST /api/stop` | Stop the service, restore native Codex, remove managed Grok injection, and drain the proxy | 409 service ownership conflict; 409 `respawnable_service` when a Windows Task Scheduler wrapper could respawn the proxy and the caller is not `ocx stop` (nothing is changed); 409 `self_unload_service` when this proxy is running as the installed launchd/systemd service, because stopping the manager from inside it would end the process before native Codex is restored — run `ocx stop` instead (nothing is changed); 409 when the installed manager refuses to stop; 409 `service_state_unknown` when the Task Scheduler state cannot be read (nothing is changed; repair the query and retry), or when the Windows checked service stop does not finish within the route's cap (6 s; 4 s for a leftover wrapper of a deleted task). On Windows a request from `ocx stop` whose receipt records that the CLI already ran the checked service stop does not repeat it | | `GET /api/system/codex-app-server` | Report whether running Codex app-servers predate the current model catalog | — | | `POST /api/system/codex-restart` | Refresh the catalog, then restart Codex app-servers. A running Codex desktop app is given a restart attempt so the model picker can reload; the `desktopApp` response fields report a refusal, surviving targets, or a failed relaunch. A shell newer than the catalog is still reported as `desktopApp.pickerState: "unknown"`, because its start time cannot prove which `model_catalog_json` binding it loaded; an earlier sync may have rebound an already-existing catalog. A shell no newer than the catalog, or one whose binding this request changed, is `stale`. A closed desktop returns `desktopApp.reason: "no_targets"`; its `pickerState` may be `not_running`, or `stale` when this request changed the catalog binding. When the proxy itself is running inside the Codex app, the desktop restart is refused rather than handed off. | Returns 200 with `code: partially_stopped` when a target survives | diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 980ea7769da..b3fe1002f26 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -1925,8 +1925,8 @@ "windows-deploy-close-regressions.test.ts": "windows", "windows-elevation-spawn.test.ts": "windows", "windows-elevation.test.ts": "windows", - "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", - "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", + "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", "windows-checked-stop-live.test.ts": "windows", "windows-service-stop-wiring.test.ts": "windows", + "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", "windows-stop-callers.test.ts": "windows", "windows-stop-ocx-stop.test.ts": "windows", "windows-process-priority.test.ts": "windows", "windows-scheduler-errors.test.ts": "windows", "windows-scheduler-install-verification.test.ts": "windows", "windows-lifecycle-root-acl.test.ts": "windows", diff --git a/src/cli/dispatch.ts b/src/cli/dispatch.ts index b9150d3b7e0..5c132eb427d 100644 --- a/src/cli/dispatch.ts +++ b/src/cli/dispatch.ts @@ -35,6 +35,7 @@ import { normalizeUpdateChannel, runGuiUpdateWorker } from "../update/job"; import { isJsonOption, takeFlag } from "./runtime-api"; import { printStopSummary, type StopOutcome } from "./stop-report"; import { parseStopApproval, type StopApproval } from "./stop-approval"; +import { resolveStopDeadlineMs, setStopDeadlineMs } from "../service/windows-stop-deadline"; import type { ResolveArgs } from "./resolve"; import type { ClientConnectionState } from "../client/state"; import { OCX_NATIVE_REPLAY_RECOVERY_NOTE } from "../responses/compaction"; @@ -111,9 +112,11 @@ const commandRunners: Record = { stop: async deps => { const parsed = parseStopApproval(deps.args.slice(1)); if (!parsed.ok) { - console.error("Usage: ocx stop [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); + console.error("Usage: ocx stop [--deadline-ms ] [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); return 64; } + // #7771: the Windows checked stop's deadline, from this process's start (flag, then env, then 60 s). + setStopDeadlineMs(resolveStopDeadlineMs(parsed.deadlineMs)); // Downtime warning lives HERE, not in handleStop: `restart`/tray-restart callers // re-start the proxy immediately, so warning there would contradict the next line. const warning = "⚠️ Codex/Claude requests through the proxy will fail until it is restarted ('ocx start' or 'ocx service start')."; diff --git a/src/cli/index.ts b/src/cli/index.ts index 870000f41dd..03cee08f0e1 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -79,7 +79,8 @@ import { quarantinePendingTeardown, } from "../config/pending-teardown"; import { collectStatus, detectMissingCodexCatalogPath, hubStatusLines, missingCodexCatalogLines, remoteHubBannerLine, remoteHubStatusLines, runtimeSupervisorLine, statusDeadProxyLines, statusProxyUp, statusUnusedProxyLines } from "./status"; -import { endpointsToProve, everyEndpointProvenDownAsync, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; +import { stopUninstallProxy } from "./uninstall-stop"; +import { captureUninstallEndpointProof, endpointsToProve, respawnWindowRequired, sharedTeardownAuthorized, uninstallServiceRemoval, type UninstallObservation } from "./uninstall-plan"; import { takeFlag } from "./runtime-api"; import { parseStartOptions, StartArgsError } from "./start-args"; @@ -124,12 +125,13 @@ import { type GuardedStopSnapshot, type StopApproval, } from "./stop-approval"; -import { inspectGuardedManagerTarget, observeGuardedManagerStopped } from "../service/guarded-manager-target"; import { summarizeStopRun, type StopOutcome, type StopRunRecord } from "./stop-report"; +import { confirmGuardedFreed, inspectGuardedTarget, noServiceStop, observeGuardedStopped, orphanDiscoveryCap, stopGuardedManager, stopServiceForOcxStop, stopServiceForUninstall, stopWindowFailed, verifyUninstallStopWindow, windowsChildStopHeld, windowsPartAHolds, windowsPartAReceiptRecord, windowsPartBDue, windowsPartBOwnsPortWait, windowsStopRequiresReceipt } from "./stop-windows-checked"; import { runCli } from "./root"; import { isProcessAlive, ProxyOwnershipRefusedError, refusalNextStep, stopProxy } from "../lib/process-control"; import { startupDataPlaneToken } from "../lib/service-secrets"; -import { assertNotAdminToken, diagnoseService, isServiceOwnershipError, proxyStillLiveAfterStop, serviceCommand, serviceEnvironmentOwnedHere, serviceStartableFromTray, serviceStatePaths, serviceStatusSummary, stopServiceIfInstalledDetailed, uninstallServiceIfInstalled, uninstallServiceDetailed } from "../service"; +import { assertNotAdminToken, diagnoseService, isServiceOwnershipError, serviceCommand, serviceEnvironmentOwnedHere, serviceStartableFromTray, serviceStatePaths, serviceStatusSummary, uninstallServiceIfInstalled, uninstallServiceDetailed } from "../service"; +import { completeWindowsServiceStopReceipt } from "../service/cli-windows-stop"; import { acquireOwnershipMutationLease } from "../service/ownership-mutation-lease.mjs"; import { formatStartupRoutingDetail, startupHealthSummary } from "../codex/autostart-health"; import { injectSystemEnv, reconcileShellHook, revertSystemEnv, uninstallShellHook } from "../server/system-env"; @@ -1026,7 +1028,7 @@ async function handleStop(approval?: StopApproval) { catch { return null; } }, describeTrackedRuntimeTarget, - () => inspectGuardedManagerTarget(approval.pid, approval.port), + () => inspectGuardedTarget(approval.pid, approval.port), snapshot => handleStopUnlocked(snapshot), ); } @@ -1146,11 +1148,10 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { if (teardownNonce || stoppingSibling) return; if (connectedClientSkip) return; try { - teardownNonce = claimPendingTeardown(endpoint, endpointSource, process.pid, handoverRequested ? "update-restart" : undefined).nonce; + teardownNonce = claimPendingTeardown(endpoint, endpointSource, process.pid, handoverRequested ? "update-restart" : undefined, windowsPartAReceiptRecord()).nonce; } catch (err) { - // Without a receipt the proxy performs its own teardown, which is the pre-#3008 - // behaviour: correct for every backend that cannot respawn, and merely early for - // Task Scheduler. Losing the deferral is far better than losing the stop. + // Windows wrappers can relaunch the proxy, so an undeferred stop is unsafe. + if (windowsStopRequiresReceipt()) { ownershipBlocked = true; stopFailed = true; throw err; } console.warn(`⚠️ Could not record the deferred-teardown receipt: ${err instanceof Error ? err.message : String(err)}`); } }; @@ -1173,6 +1174,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // Only an exact endpoint may direct the request; the configured fallback is a guess // good enough to record an obligation against, not to POST a stop to. runtimeEndpoint: exact ?? undefined, + skipPortWait: windowsPartBOwnsPortWait(), // #7771: Part B owns the Windows port wait on both paths. }); // A valid receipt means the proxy deferred shared teardown to this process. If the // receipt could not be written, the proxy restores it itself and the caller must not @@ -1190,22 +1192,23 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { try { const serviceStop = snapshot ? (guardedStep = await runGuardedManagerStep(snapshot, { - revalidateManager: () => inspectGuardedManagerTarget(snapshot.approval.pid, snapshot.approval.port), + revalidateManager: () => inspectGuardedTarget(snapshot.approval.pid, snapshot.approval.port), stopManager: () => { if (approvedEndpoint) claimTeardown(approvedEndpoint, "exact"); - return stopServiceIfInstalledDetailed(); + return stopGuardedManager({ approvedPort: snapshot.approval.port, receiptNonce: () => teardownNonce, inheritedReceipts: inheritedTeardowns.length }); }, signalApproved: () => stopWithDeferral(snapshot.approval.pid, approvedEndpoint), settle: () => settleApprovedTarget(snapshot.approval), - managerState: () => observeGuardedManagerStopped(snapshot.manager), + confirmFreed: () => confirmGuardedFreed(record), + managerState: () => observeGuardedStopped(snapshot.manager), })).service // A sibling never runs under a service manager: an installed one belongs to the live owner, // and its ownership check would fail this stop and skip the stale-record purge below. // #7423: a manager stop takes the proxy down itself, so a handover claims its receipt first. - : stoppingSibling || connectedClientAlreadyStopped ? "absent" : (claimHandoverBeforeManager(), stopServiceIfInstalledDetailed()); + : stoppingSibling || connectedClientAlreadyStopped ? noServiceStop() : (claimHandoverBeforeManager(), await stopServiceForOcxStop({ receiptNonce: () => teardownNonce, inheritedReceipts: inheritedTeardowns.length })); record.service = serviceStop; stoppedService = serviceStop === "stopped" || serviceStop === "stopped-respawnable"; - schedulerCanRespawn = serviceStop === "stopped-respawnable"; + schedulerCanRespawn = serviceStop === "stopped-respawnable" || windowsPartBDue(); // No "won't respawn" claim here: a stopped Task Scheduler can still respawn through // its wrapper, which the verification below is what actually settles. if (stoppedService) console.log("🛑 Service manager stopped."); @@ -1223,12 +1226,13 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { console.error("❌ The Windows Task Scheduler state could not be read, so this stop cannot tell whether a wrapper would respawn the proxy."); console.error(" Run 'ocx service status' to see the query error, repair Task Scheduler access, then retry."); } + ownershipBlocked = windowsPartAHolds(record) || ownershipBlocked; // #7771: no POST and no shared teardown. } catch (err) { record.service = "error"; - if (isServiceOwnershipError(err)) { + if (isServiceOwnershipError(err) || windowsChildStopHeld()) { ownershipBlocked = true; stopFailed = true; - console.error(`❌ ${err.message}`); + console.error(`❌ ${err instanceof Error ? err.message : String(err)}`); console.error(" Skipping shared teardown (native Codex restore, Grok config): the installed service is still running."); } else { console.error(`⚠️ Service manager stop failed: ${err instanceof Error ? err.message : String(err)}`); @@ -1237,7 +1241,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { if (snapshot) { if (guardedStep?.effect === "approval-changed") { - return approvalChanged(guardedStep.detail); + return approvalChanged(guardedStep.detail, guardedStep.stopReason); } if (guardedStep?.effect === "manager-still-active") { return managerStillActive(record.service, record); @@ -1253,7 +1257,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { record.proxy = "unknown"; console.error("The approved runtime could not be verified after the guarded stop step."); } - } else { + } else if (windowsChildStopHeld()) { /* #7771: Part A did not prove the manager stopped, so no POST. */ } else { const pid = readPid(); if (pid) { try { @@ -1261,8 +1265,8 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // the proxy's shutdown handlers actually run; taskkill /F is the fallback inside. // Shared teardown is deferred to this process: it happens after the respawn // verification below, so a survivor does not get its client config pulled first. - // The receipt goes down first — the proxy honours the deferral only when it can - // see one, so an unrecordable claim degrades to the child doing its own teardown. + // The receipt goes down first; Windows refuses if it cannot be recorded. + // Other platforms retain the existing undeferred fallback. nativeRestoreHandledByProxy = await stopWithDeferral(pid); console.log(`✅ Proxy (PID ${pid}) stopped.`); removePid(pid); @@ -1294,7 +1298,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { const staleRuntimePid = readRuntimePort()?.pid ?? null; // Orphan recovery: a live proxy can outlive its pid file (crash, manual delete, // corrupt file). Identity-checked liveness still finds it via the runtime record. - const live = await findLiveProxy({ acceptPackageTreeFenced: true }); + const live = await findLiveProxy({ acceptPackageTreeFenced: true, ...orphanDiscoveryCap() }); if (siblingStopFoundOwner(siblingOfPort, live)) { record.proxy = "not-running"; console.log(`The sibling instance is already gone; the proxy on port ${siblingOfPort} was left running.`); @@ -1355,21 +1359,13 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // Environment ownership is independent from service ownership. Roll back current-home // variables (the helper refuses foreign markers on its own) unless a sibling never set them. if (!stoppingSibling) { try { revertSystemEnv(); } catch { /* best-effort */ } } - // A stopped Windows scheduler is not a proven-down proxy. `killWindowsSchedulerWrappers` - // is explicitly best-effort and the `:loop` wrapper respawns its child after ~5s, so an - // immediate probe can see a dead interval and an update can start replacing files right - // before the proxy comes back. Poll across the restart window before this stop is allowed - // to report anything but failure (#3008) — and ONLY for that backend, since making every - // launchd and systemd stop wait seven seconds would be a regression in ordinary use. + // A stopped Windows service is not a proven-down proxy: the `:loop` wrapper relaunches its + // child about 5 s later, so an immediate probe can see a dead interval (#764, #3008). On + // Windows Part B of the checked stop (#7771) holds the port free across that window and + // re-enumerates the wrappers; only then may the stop report anything but failure. launchd + // and systemd are down when they say so and pay no wait. if (schedulerCanRespawn && !ownershipBlocked) { - const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); - if (survivor) { - stopFailed = true; - record.proxy = "respawned"; - console.error(`❌ A proxy is still listening on port ${survivor.port} after the service stop; it is being respawned.`); - console.error(" Skipping shared teardown: restoring client config while the proxy runs leaves both pointing at each other."); - ownershipBlocked = true; - } + if (await stopWindowFailed(record)) { stopFailed = true; ownershipBlocked = true; } } // Recovering somebody else's abandoned obligation is not the same act as finishing this // run's own. This run stopped a proxy and verified the result; the inherited case has no @@ -1521,7 +1517,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { }; const summary = summarizeStopRun(record, signals); if (snapshot && !stopFailed) { - return guardFinalStopSummary(record, () => observeGuardedManagerStopped(snapshot.manager), + return guardFinalStopSummary(record, () => observeGuardedStopped(snapshot.manager), () => ({ ok: !stopFailed, summary })); } return { ok: !stopFailed, summary }; @@ -1531,17 +1527,8 @@ async function handleUninstall() { // Before every teardown step: an elevated run must not act on a home another account can change. const privilegeRefusal = uninstallPrivilegeRefusal(getConfigDir()); if (privilegeRefusal) { console.error(`❌ ${privilegeRefusal}`); process.exit(1); } - /** Definitive "nothing is answering" on the endpoint this home would serve. */ - const proxyEndpointProvenDown = async (): Promise => { - try { - // Every candidate, not just the preferred one: a stale runtime record pointing at a - // closed port would otherwise "prove" a live proxy on the configured port is gone. - const endpoints = endpointsToProve(readRuntimePort(), loadConfig()); - return await everyEndpointProvenDownAsync(endpoints, probeEndpointLiveness); - } catch { - return false; - } - }; + /** Definitive "nothing is answering" on every endpoint this home could serve, captured before the stop. */ + const proxyEndpointProvenDown = captureUninstallEndpointProof(() => endpointsToProve(readRuntimePort(), loadConfig()), probeEndpointLiveness); const failures: string[] = []; const runStep = async (label: string, step: () => void | boolean | Promise) => { @@ -1567,8 +1554,11 @@ async function handleUninstall() { serviceRemoval: null, respawnWindowVerified: false, }; - await runStep("service stopped", () => { - const outcome = stopServiceIfInstalledDetailed(); + // The receipt an uninstall's Windows proxy stop claimed; discharged only after Part B + // and the shared restore succeeded, kept for recovery otherwise (#7771, #3008). + let uninstallStopNonce: string | undefined; + await runStep("service stopped", async () => { + const outcome = await stopServiceForUninstall(); observed.serviceStop = outcome; if (outcome === "absent") return false; if (outcome === "failed") { @@ -1581,39 +1571,35 @@ async function handleUninstall() { }); await runStep("proxy stopped", async () => { - const pid = readPid(); - if (!pid) { - // A missing pid file is not proof that nothing is serving: a proxy can outlive its - // record (crash, manual delete, corrupt file), which is exactly why `ocx stop` falls - // back to identity-checked discovery. Without this, uninstall restored shared config - // and reported success while that proxy kept running (#3008). - const live = await findLiveProxy(); - if (!live) { - // A miss is not proof: `findLiveProxy` collapses a timeout and a transport failure - // into the same null as a dead endpoint. Ask the tri-state probe, which only says - // "dead" for a refused connection or a definitive non-OpenCodex answer (#3008). - observed.proxyProvenDown = await proxyEndpointProvenDown(); - if (!observed.proxyProvenDown) { - throw new Error("no proxy could be found, but its endpoint could not be confirmed down either; confirm nothing is serving, then rerun"); - } - return false; - } - if (!live.pid) { - throw new Error(`a proxy is answering on port ${live.port} but no process id could be resolved for it; stop it from the home that started it, then rerun`); - } - await stopProxy(live.pid); - observed.proxyProvenDown = true; - return true; - } - await stopProxy(pid); - removePid(pid); - removeRuntimePort(pid); - observed.proxyProvenDown = true; - return true; + // #7771 / #3008: on Windows this claims the teardown receipt before the child stop and + // forbids the forced fallback when Part A did not prove the manager stopped, so shared + // client config is never restored under a wrapper that can relaunch the proxy. + const result = await stopUninstallProxy({ + platform: process.platform, + serviceStop: observed.serviceStop, + readPid, + findLiveProxy, + endpointProvenDown: proxyEndpointProvenDown, + partARecord: windowsPartAReceiptRecord, + stop: (target, io) => stopProxy(target, io), + removeTracked: pid => { + removePid(pid); + removeRuntimePort(pid); + }, + portWaitOwned: windowsPartBOwnsPortWait, + }); + uninstallStopNonce = result.nonce; + observed.proxyProvenDown = result.provenDown; + return result.changed; }); await runStep("service removed", () => { - const outcome = uninstallServiceDetailed(); + // Windows only: a Part A that failed (or could not read the state, or left a wrapper + // that can still respawn) leaves a manager this uninstall cannot account for. Deleting + // its registration would remove the evidence without stopping the wrapper, and + // `ocx service uninstall` refuses in the same states. The failure stays recorded so the + // shared teardown below stays blocked. + const outcome = uninstallServiceRemoval(observed, uninstallServiceDetailed); observed.serviceRemoval = outcome; // "absent" and "removed" are both fine; a failure is not, and it used to look like // absence on darwin and linux. @@ -1624,10 +1610,9 @@ async function handleUninstall() { // Only Task Scheduler can respawn through a surviving wrapper, and removing the // registration does not prove the running one died. Poll the same window `ocx stop` does // before shared config is allowed down (#764, #3008). - if (observed.serviceStop === "stopped-respawnable") { + if (respawnWindowRequired(observed)) { await runStep("respawn window verified", async () => { - const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); - if (survivor) throw new Error(`a proxy is still listening on port ${survivor.port} after the service was removed; it is being respawned`); + await verifyUninstallStopWindow(); // Windows: Part B (#7771); otherwise the legacy re-poll. // A null from that poll is not proof either: its identity probe returns null on a // timeout, so a respawned-but-unresponsive proxy looks the same as none. Require the // tri-state probe to say dead on every candidate before calling the window verified. @@ -1672,6 +1657,12 @@ async function handleUninstall() { console.error(" 'ocx restore' is an interim step if you need native routing back before then."); } + // The obligation the proxy stop deferred is discharged only now: Part B and the shared + // restore both succeeded. Otherwise the receipt stays for the next stop to finish. + if (uninstallStopNonce && failures.length === 0) { + completeWindowsServiceStopReceipt(uninstallStopNonce); + } + await runStep("system env vars reverted", () => { const r = revertSystemEnv(); if (!r.reverted && r.reason !== "no tracking file" && r.reason !== "not macOS") throw new Error(r.reason ?? "revert failed"); diff --git a/src/cli/registry.ts b/src/cli/registry.ts index c94cb1dfe3e..237fe0610d2 100644 --- a/src/cli/registry.ts +++ b/src/cli/registry.ts @@ -51,11 +51,12 @@ export const CLI_COMMANDS: CliCommandEntry[] = [ }, { name: "stop", - usage: "ocx stop [--json]", + usage: "ocx stop [--json] [--deadline-ms ]", summary: "Stop the proxy; preserve connected-client integrations, restore native Codex config for standalone owners.", details: [ "--json keeps the stop path unchanged and prints one structured summary document on stdout; human output moves to stderr.", "Exit codes are identical with and without --json: 0, 1, 79 (history cleanup incomplete), 80 (teardown deferred).", + "--deadline-ms Windows: finish the service stop within of this command's start, or refuse up front with nothing changed (default 60000; OCX_STOP_DEADLINE_MS is read when the flag is absent).", ], }, { diff --git a/src/cli/root.ts b/src/cli/root.ts index 9628af27afc..81f7d8ca433 100644 --- a/src/cli/root.ts +++ b/src/cli/root.ts @@ -141,7 +141,7 @@ export async function runCli(argv: string[]): Promise { process.exit(1); } if (head.command === "stop" && !parseStopApproval(head.args.slice(1)).ok) { - console.error("Usage: ocx stop [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); + console.error("Usage: ocx stop [--deadline-ms ] [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); process.exit(64); } const uninstallError = uninstallArgsError(head.command, head.args); diff --git a/src/cli/stop-approval.ts b/src/cli/stop-approval.ts index 0e45cca200f..770c65c6eef 100644 --- a/src/cli/stop-approval.ts +++ b/src/cli/stop-approval.ts @@ -4,7 +4,9 @@ import { waitForPortAvailable } from "../server/ports"; import { probeEndpointLiveness, probeHostname } from "../server/proxy-liveness"; import type { GuardedManagerTarget } from "../service/guarded-manager-target"; import type { ResolveJson } from "./resolve"; -import { STOP_SUMMARY_SCHEMA, type StopOutcome, type StopRunRecord, type StopServiceOutcome } from "./stop-report"; +import { STOP_SUMMARY_SCHEMA, stopReasonFields, type StopOutcome, type StopRunRecord, type StopServiceOutcome, type StopServiceReason } from "./stop-report"; + +type MaybePromise = T | Promise; export interface StopApproval { pid: number; @@ -36,15 +38,43 @@ function sameGuardedManager( current: GuardedManagerTarget, ): boolean { if (expected.kind === "absent") return current.kind === "absent"; + if (expected.kind === "bound-stray") { + // The bound wrapper set is the identity (#7771): same PIDs with the same creation times. + if (current.kind !== "bound-stray" || current.pid !== expected.pid) return false; + const key = (set: { pid: number; startTime: number }[]) => + set.map(wrapper => `${wrapper.pid}@${wrapper.startTime}`).sort().join(","); + return key(current.wrappers) === key(expected.wrappers); + } return current.kind === "bound" && current.pid === expected.pid && current.managerPid === expected.managerPid && current.backend === expected.backend; } -export type StopApprovalParse = { ok: true; json: boolean; approval: StopApproval | null } | { ok: false }; +export type StopApprovalParse = + | { ok: true; json: boolean; approval: StopApproval | null; deadlineMs: number | null } + | { ok: false }; -export function parseStopApproval(argv: string[]): StopApprovalParse { - if (argv.length === 0) return { ok: true, json: false, approval: null }; - if (argv.every(arg => arg === "--json")) return { ok: true, json: true, approval: null }; +/** + * --deadline-ms (#7771): a positive decimal number of milliseconds, measured from this + * process's start; at most once, anywhere in the argument list. Removed here so the + * approval grammar below stays exactly what it was. + */ +function takeStopDeadline(argv: string[]): { argv: string[]; deadlineMs: number | null } | null { + const at = argv.indexOf("--deadline-ms"); + if (at < 0) return { argv, deadlineMs: null }; + if (argv.indexOf("--deadline-ms", at + 1) >= 0) return null; + const raw = argv[at + 1]; + if (raw === undefined || !/^\d+$/.test(raw)) return null; + const value = Number(raw); + if (!Number.isSafeInteger(value) || value <= 0) return null; + return { argv: [...argv.slice(0, at), ...argv.slice(at + 2)], deadlineMs: value }; +} + +export function parseStopApproval(input: string[]): StopApprovalParse { + const taken = takeStopDeadline(input); + if (!taken) return { ok: false }; + const { argv, deadlineMs } = taken; + if (argv.length === 0) return { ok: true, json: false, approval: null, deadlineMs }; + if (argv.every(arg => arg === "--json")) return { ok: true, json: true, approval: null, deadlineMs }; const names = ["--expect-pid", "--expect-port", "--expect-hostname", "--expect-config-home", "--expect-cli-version", "--expect-compatibility-token"]; if (argv.filter(arg => arg === "--json").length !== 1) return { ok: false }; @@ -68,7 +98,7 @@ export function parseStopApproval(argv: string[]): StopApprovalParse { const cliVersion = values.get("--expect-cli-version") ?? ""; if (pid === null || port === null || !/^[a-f0-9]{64}$/.test(compatibilityToken) || !configHome || !cliVersion) return { ok: false }; - return { ok: true, json: true, approval: { + return { ok: true, json: true, deadlineMs, approval: { pid, port, hostname: values.get("--expect-hostname") ?? "", configHome, cliVersion, compatibilityToken, } }; @@ -104,23 +134,25 @@ export function stopApprovalMismatch(expected: StopApproval, now: ResolveJson): return null; } -export function approvalChanged(detail?: string): StopOutcome { +export function approvalChanged(detail?: string, serviceStopReason?: StopServiceReason): StopOutcome { return { ok: false, summary: { schema: STOP_SUMMARY_SCHEMA, ok: false, outcome: "approval-changed", exitCode: 1, runtimeDown: false, service: "absent", proxy: "unknown", sharedTeardown: "skipped", ...(detail ? { detail } : {}), + ...stopReasonFields({ serviceStopReason }), message: `The approved runtime or managing CLI changed${detail ? `: ${detail}` : ""}; no stop was attempted.`, } }; } export function managerStillActive( service: StopServiceOutcome, - record?: Pick, + record?: Pick, ): StopOutcome { return { ok: false, summary: { schema: STOP_SUMMARY_SCHEMA, ok: false, outcome: "manager-still-active", exitCode: 1, runtimeDown: false, service, proxy: record?.proxy ?? "unknown", sharedTeardown: record?.sharedTeardown ?? "skipped", + ...stopReasonFields(record ?? {}), message: "The approved runtime and service manager could not both be confirmed stopped.", } }; } @@ -129,14 +161,14 @@ export async function runApprovedStop( expected: StopApproval, read: () => Promise, tracked: () => TrackedRuntimeCheck, - managerTarget: () => GuardedManagerTarget, + managerTarget: () => MaybePromise, stop: (snapshot: GuardedStopSnapshot) => Promise, ): Promise { let snapshot: GuardedStopSnapshot | null = null; try { const now = await read(); const target = tracked(); - const manager = managerTarget(); + const manager = await managerTarget(); if (now === null) return approvalChanged("the runtime could not be re-resolved"); const mismatch = stopApprovalMismatch(expected, now); if (mismatch !== null) return approvalChanged(mismatch); @@ -149,9 +181,9 @@ export async function runApprovedStop( return approvalChanged("the tracked runtime record changed"); } if (manager.kind === "unknown") { - return approvalChanged(`the service manager could not be re-verified (${manager.reason})`); + return approvalChanged(`the service manager could not be re-verified (${manager.reason})`, manager.stopReason); } - if (manager.kind === "bound" && manager.pid !== expected.pid) { + if ((manager.kind === "bound" || manager.kind === "bound-stray") && manager.pid !== expected.pid) { return approvalChanged("a different manager owns the approved runtime"); } snapshot = { approval: expected, manager }; @@ -188,30 +220,40 @@ export async function settleApprovedTarget( export async function runGuardedManagerStep( snapshot: GuardedStopSnapshot, io: { - revalidateManager: () => GuardedManagerTarget; - stopManager: () => StopServiceOutcome; + revalidateManager: () => MaybePromise; + /** Async on every platform; on Windows it is Part A of the checked stop (#7771). */ + stopManager: () => MaybePromise; signalApproved: () => Promise; settle: () => Promise; + /** + * Windows Part B after settle (#7771): the approved endpoint stays free for the hold + * and no wrapper of this home came back. Runs only where stopManager ran. + */ + confirmFreed?: () => Promise; managerState: () => Promise<"inactive" | "active" | "unknown">; }, ): Promise<{ service: StopServiceOutcome; effect: "stopped" | "approval-changed" | "manager-still-active" | "failed"; - proxy: "stopped" | "unknown"; handledByProxy: boolean; detail?: string }> { + proxy: "stopped" | "unknown"; handledByProxy: boolean; detail?: string; stopReason?: StopServiceReason }> { try { - const current = io.revalidateManager(); + const current = await io.revalidateManager(); if (!sameGuardedManager(snapshot.manager, current)) { // An unprovable manager is not a proven change: name the observation failure, // not a swap, or the stop log loses why the revalidation could not answer. const detail = current.kind === "unknown" ? `the service manager could not be re-verified (${current.reason})` : "the service manager changed between approval and stop"; - return { service: "absent", effect: "approval-changed", proxy: "unknown", handledByProxy: false, detail }; + return { service: "absent", effect: "approval-changed", proxy: "unknown", handledByProxy: false, detail, + ...(current.kind === "unknown" && current.stopReason ? { stopReason: current.stopReason } : {}) }; } } catch { return { service: "absent", effect: "approval-changed", proxy: "unknown", handledByProxy: false, detail: "the service manager could not be re-validated" }; } - const service = snapshot.manager.kind === "absent" ? "absent" : io.stopManager(); - if (snapshot.manager.kind === "bound" + // The order is the #7771 contract: the manager and its wrappers end first (nothing can + // relaunch the child), then the approved PID gets its graceful signal and receipt. + const managed = snapshot.manager.kind !== "absent"; + const service = managed ? await io.stopManager() : "absent"; + if (managed && service !== "stopped" && service !== "stopped-respawnable") { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy: false }; } @@ -231,6 +273,11 @@ export async function runGuardedManagerStep( if (!await io.settle()) { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; } + if (managed && io.confirmFreed) { + try { + if (!await io.confirmFreed()) return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; + } catch { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; } + } try { if (await io.managerState() !== "inactive") { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; diff --git a/src/cli/stop-report.ts b/src/cli/stop-report.ts index 744c77e6d6f..831a81073c3 100644 --- a/src/cli/stop-report.ts +++ b/src/cli/stop-report.ts @@ -41,6 +41,12 @@ export type StopSharedTeardownOutcome = | "failed" | "skipped"; +/** + * Why the checked Windows service stop did not succeed (#7771). Additive on ocx-stop/1: + * absent when the CLI has no reason to name, so older documents read unchanged. + */ +export type StopServiceReason = "access-denied" | "evidence-unreadable" | "unbound-wrapper" | "unverified-budget"; + /** Facts recorded where the stop path already decides them. */ export interface StopRunRecord { /** What stopServiceIfInstalledDetailed returned, or "error" when it threw. */ @@ -53,6 +59,10 @@ export interface StopRunRecord { inheritedTeardownBlocks: boolean; /** A discharged receipt could not be removed from disk. */ receiptClearFailed: boolean; + /** Windows checked stop: the reason a Part A or Part B result was not a success. */ + serviceStopReason?: StopServiceReason; + /** Windows checked stop: Part B found another program on the port after ours were gone. */ + portHeldByForeign?: boolean; } /** The internal booleans that already pick the process exit code. */ @@ -81,6 +91,10 @@ export interface StopSummaryJson { * run can name one. Additive on `ocx-stop/1`; absent on every other outcome. */ detail?: string; + /** Additive (#7771): see StopRunRecord.serviceStopReason. Omitted when there is none. */ + serviceStopReason?: StopServiceReason; + /** Additive (#7771): true or false once Part B ran, omitted otherwise; never null. */ + portHeldByForeign?: boolean; } /** What handleStop returns: the pre-existing boolean plus the structured twin. */ @@ -126,6 +140,14 @@ function stopMessage(record: StopRunRecord, signals: StopRunSignals, outcome: St return "The stop failed."; } +/** The additive #7771 fields of a record, omitted when unset (never null). */ +export function stopReasonFields(record: Partial>): Pick { + return { + ...(record.serviceStopReason ? { serviceStopReason: record.serviceStopReason } : {}), + ...(typeof record.portHeldByForeign === "boolean" ? { portHeldByForeign: record.portHeldByForeign } : {}), + }; +} + /** Pure mapper from the recorded run to the wire document. */ export function summarizeStopRun(record: StopRunRecord, signals: StopRunSignals): StopSummaryJson { const outcome = stopOutcome(record, signals); @@ -139,6 +161,7 @@ export function summarizeStopRun(record: StopRunRecord, signals: StopRunSignals) proxy: record.proxy, sharedTeardown: record.sharedTeardown, message: stopMessage(record, signals, outcome), + ...stopReasonFields(record), }; } diff --git a/src/cli/stop-windows-checked.ts b/src/cli/stop-windows-checked.ts new file mode 100644 index 00000000000..35299e01a11 --- /dev/null +++ b/src/cli/stop-windows-checked.ts @@ -0,0 +1,304 @@ +/** + * ocx stop's side of the checked Windows service stop (#7771, 030 "Two parts"). + * + * handleStopUnlocked in src/cli/index.ts keeps its structure; the Windows-specific + * decisions live here so that file does not grow: + * + * - snapshotless: Part A runs as the service stop, with admission against the stop + * deadline. A result other than stopped/absent holds the child stop (no POST), and a + * stopped result makes Part B replace the old respawn re-poll; + * - guarded: the manager inspection reads the evidence snapshot (windowsProcessSnapshot, + * native creation times on x64) once per inspection, and Part A binds against the + * revalidation's snapshot; Part B runs after settle on the approved endpoint, and its + * final enumeration is the one the manager-state checks share; + * - the teardown receipt carries the Part A record, so /api/stop skips a second Part A. + * + * Off Windows every function keeps today's behavior and timing. + */ +import { recordPendingTeardownPartA, type PendingTeardownPartA } from "../config/pending-teardown"; +import { readPid } from "../config/process-state"; +import { observeGuardedManagerStopped, inspectGuardedManagerTarget, type GuardedManagerTarget } from "../service/guarded-manager-target"; +// Through the barrel, like every other stop-path call, so the parent-stop fixtures can see it. +import { proxyStillLiveAfterStop, stopServiceIfInstalledDetailed, stopServiceIfInstalledReport } from "../service"; +import type { ServiceStopOutcome } from "../service/orchestration"; +import type { WindowsCheckedStopResult } from "../service/windows-checked-stop"; +import { readScheduleServicePid, windowsProcessSnapshot, type WindowsStopProcessEntry } from "../service/windows-process-evidence"; +import { + confirmWindowsServiceStop, + describeWindowsPartA, + describeWindowsPartB, + windowsPartARecord, + windowsServiceStopHooks, + windowsServiceStopPort, + windowsStopAdmission, + windowsStopReason, + windowsStopRemainingMs, +} from "../service/windows-service-stop"; +import { WINDOWS_STOP_CAPS } from "../service/windows-stop-budget"; +import type { StopRunRecord } from "./stop-report"; + +const EVIDENCE_SNAPSHOT_TIMEOUT_MS = 5_000; +const SCHEDULE_PID_TIMEOUT_MS = 2_000; +const ORPHAN_DISCOVERY_CAP_MS = WINDOWS_STOP_CAPS.orphanDiscovery; + +interface WindowsStopState { + platform: NodeJS.Platform; + result: WindowsCheckedStopResult | null; + snapshotless: boolean; + port: number | null; + /** The evidence the last guarded inspection read; the bind reuses it. */ + evidence: { snapshot: readonly WindowsStopProcessEntry[] | null; schedulePid: number | null } | null; + /** Part B's final enumeration, which the guarded manager-state checks share. */ + partBSnapshot: readonly WindowsStopProcessEntry[] | null; +} + +const state: WindowsStopState = { + platform: process.platform, result: null, snapshotless: false, port: null, evidence: null, partBSnapshot: null, +}; + +/** Test-only: simulate the platform the stop wiring acts for, and start from a clean run. */ +export function resetOcxWindowsStopForTests(platform: NodeJS.Platform | null = null): void { + state.platform = platform ?? process.platform; + state.result = null; + state.snapshotless = false; + state.port = null; + state.evidence = null; + state.partBSnapshot = null; +} + +function windows(): boolean { + return state.platform === "win32"; +} + +export function windowsStopRequiresReceipt(): boolean { + return windows(); +} + +function begin(snapshotless: boolean): void { + state.result = null; + state.snapshotless = snapshotless; + state.port = null; + state.partBSnapshot = null; +} + +function annotate(nonce: string | undefined): void { + const record = state.result ? windowsPartARecord(state.result) : null; + if (!nonce || !record) return; + try { + recordPendingTeardownPartA(nonce, record); + } catch { + // Without the record the route runs its own capped Part A, which is today's behavior. + } +} + +/** A stop that never reaches the service manager (a sibling, an already-stopped client). */ +export function noServiceStop(): ServiceStopOutcome { + begin(false); + return "absent"; +} + +/** The snapshotless service stop: Part A on Windows, today's manager stop elsewhere. */ +export async function stopServiceForOcxStop(io: { + receiptNonce: () => string | undefined; + inheritedReceipts: number; +}): Promise { + begin(true); + if (!windows()) return stopServiceIfInstalledDetailed(); + state.port = windowsServiceStopPort(); + const report = await stopServiceIfInstalledReport({ + admission: windowsStopAdmission({ orphanDiscovery: readPid() === null, inheritedReceipts: io.inheritedReceipts }), + }, state.platform); + state.result = report.windows; + annotate(io.receiptNonce()); + return report.outcome; +} + +/** The record a receipt claimed after Part A carries (null before Part A or off Windows). */ +export function windowsPartAReceiptRecord(): PendingTeardownPartA | null { + return state.result ? windowsPartARecord(state.result) : null; +} + +/** Part A held the child stop; handleStopUnlocked skips the POST branch entirely. */ +export function windowsChildStopHeld(): boolean { + const outcome = state.result?.outcome; + return windows() && state.snapshotless && outcome !== "stopped" && outcome !== "absent"; +} + +/** ocx uninstall's service stop: Part A on Windows (60 s deadline), today's stop elsewhere. */ +export async function stopServiceForUninstall(): Promise { + begin(true); + if (!windows()) return stopServiceIfInstalledDetailed(); + state.port = windowsServiceStopPort(); + const report = await stopServiceIfInstalledReport({ + admission: windowsStopAdmission({ orphanDiscovery: readPid() === null }), + }, state.platform); + state.result = report.windows; + return report.outcome; +} + +/** + * ocx uninstall's restart-window check after the service was removed: Part B on Windows + * after a Part A stopped, the legacy re-poll otherwise. Throws when the window fails; a + * port held by another program does not stop an uninstall. + */ +export async function verifyUninstallStopWindow(): Promise { + if (!windowsPartBDue()) { + const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); + if (survivor) throw new Error(`a proxy is still listening on port ${survivor.port} after the service was removed; it is being respawned`); + return; + } + const result = await confirmWindowsServiceStop({ port: state.port ?? windowsServiceStopPort(), remainingMs: windowsStopRemainingMs() }); + if (result.outcome === "freed") return; + if (result.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(result)); + return; + } + throw new Error(describeWindowsPartB(result)); +} + +/** Snapshotless Windows only: Part A stopped, so Part B confirms the port afterwards. */ +export function windowsPartBDue(): boolean { + return windows() && state.snapshotless && state.result?.outcome === "stopped"; +} + +/** + * Part B owns the port wait after a stopped Part A on BOTH Windows paths (#7771). + * + * `windowsPartBDue` stays snapshotless-only because it also gates the legacy respawn + * re-poll (`schedulerCanRespawn`). This predicate is only about the port wait: on the + * guarded path Part B runs after `settle`, so `stopProxy`'s legacy 15 s + * `waitForStoppedPort` must not run on top of it. + */ +export function windowsPartBOwnsPortWait(): boolean { + return windows() && state.result?.outcome === "stopped"; +} + +/** + * A Part A result other than stopped/absent holds the child stop (030 "Two parts"): no + * POST, the shared teardown skipped, proxy unknown. Records the additive reason. + */ +export function windowsPartAHolds(record: StopRunRecord): boolean { + const result = state.result; + if (!windows() || !result || result.outcome === "stopped" || result.outcome === "absent") return false; + const reason = windowsStopReason(result); + if (reason) record.serviceStopReason = reason; + console.error(" " + describeWindowsPartA(result)); + if (!state.snapshotless) return true; + record.proxy = "unknown"; + console.error(" The proxy was left running and the shared teardown skipped: the service could relaunch it."); + return true; +} + +/** Orphan discovery after a checked Windows stop is capped at 3 s (030 "Time budget"). */ +export function orphanDiscoveryCap(): { deadlineAt?: number } { + return windowsPartBDue() ? { deadlineAt: Date.now() + ORPHAN_DISCOVERY_CAP_MS } : {}; +} + +/** + * After the child stop: true when the stop has to fail and hold the shared teardown. + * Windows runs Part B; a legacy respawnable stop keeps the old re-poll. + */ +export async function stopWindowFailed(record: StopRunRecord): Promise { + if (!windowsPartBDue()) { + const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); + if (!survivor) return false; + record.proxy = "respawned"; + console.error("❌ A proxy is still listening on port " + survivor.port + " after the service stop; it is being respawned."); + console.error(" Skipping shared teardown: restoring client config while the proxy runs leaves both pointing at each other."); + return true; + } + const result = await confirmWindowsServiceStop({ port: state.port ?? windowsServiceStopPort(), remainingMs: windowsStopRemainingMs() }); + record.portHeldByForeign = result.portHeldByForeign; + if (result.outcome === "freed") return false; + if (result.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(result)); + return false; + } + if (result.outcome === "survived") record.proxy = "respawned"; + else { + record.proxy = "unknown"; + record.serviceStopReason = "evidence-unreadable"; + } + console.error("❌ " + describeWindowsPartB(result)); + console.error(" Skipping shared teardown: restoring client config while the proxy runs leaves both pointing at each other."); + return true; +} + +async function readEvidence(): Promise> { + const hooks = windowsServiceStopHooks(); + const [snapshot, schedulePid] = await Promise.all([ + hooks?.snapshot ? hooks.snapshot() : windowsProcessSnapshot({ timeoutMs: EVIDENCE_SNAPSHOT_TIMEOUT_MS }), + hooks?.schedulePid ? hooks.schedulePid() : readScheduleServicePid({ timeoutMs: SCHEDULE_PID_TIMEOUT_MS }), + ]); + return { snapshot, schedulePid }; +} + +/** + * The guarded manager inspection. On Windows the process table is the evidence snapshot + * with creation times, read once here and kept: Part A binds against exactly this table. + */ +export async function inspectGuardedTarget( + approvedPid: number, + approvedPort: number, + inspect: typeof inspectGuardedManagerTarget = inspectGuardedManagerTarget, +): Promise { + if (!windows()) return inspect(approvedPid, approvedPort); + const evidence = await readEvidence(); + state.evidence = evidence; + return inspect(approvedPid, approvedPort, { + win: { + winProcs: () => (evidence.snapshot ? [...evidence.snapshot] : null), + winSchedulePid: () => evidence.schedulePid, + }, + }); +} + +/** The guarded stopManager: Part A on Windows, binding on the revalidation's snapshot. */ +export async function stopGuardedManager(io: { + approvedPort: number; + receiptNonce: () => string | undefined; + inheritedReceipts: number; +}): Promise { + begin(false); + if (!windows()) return stopServiceIfInstalledDetailed(); + state.port = io.approvedPort; + const report = await stopServiceIfInstalledReport({ + admission: windowsStopAdmission({ orphanDiscovery: false, inheritedReceipts: io.inheritedReceipts }), + ...(state.evidence?.snapshot ? { bindSnapshot: state.evidence.snapshot } : {}), + }, state.platform); + state.result = report.windows; + annotate(io.receiptNonce()); + return report.outcome; +} + +/** The guarded Part B, after settle, on the approved endpoint. Off Windows: nothing to do. */ +export async function confirmGuardedFreed(record: StopRunRecord): Promise { + if (!windows() || state.snapshotless || state.result?.outcome !== "stopped" || state.port === null) return true; + const result = await confirmWindowsServiceStop({ + port: state.port, + remainingMs: windowsStopRemainingMs(), + snapshot: async timeoutMs => { + const snapshot = await windowsProcessSnapshot({ timeoutMs }); + state.partBSnapshot = snapshot; + return snapshot; + }, + }); + record.portHeldByForeign = result.portHeldByForeign; + if (result.outcome === "freed") return true; + if (result.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(result)); + return true; + } + if (result.outcome !== "survived") record.serviceStopReason = "evidence-unreadable"; + console.error("❌ " + describeWindowsPartB(result)); + return false; +} + +/** The guarded manager-state checks; on Windows they share Part B's final enumeration. */ +export function observeGuardedStopped( + manager: Exclude, + observe: typeof observeGuardedManagerStopped = observeGuardedManagerStopped, +): ReturnType { + const snapshot = windows() ? state.partBSnapshot : null; + return snapshot ? observe(manager, { win: { winProcs: () => [...snapshot] } }) : observe(manager); +} diff --git a/src/cli/uninstall-plan.ts b/src/cli/uninstall-plan.ts index 4bfb9d4a7a6..a1502b7a794 100644 --- a/src/cli/uninstall-plan.ts +++ b/src/cli/uninstall-plan.ts @@ -24,26 +24,75 @@ export type UninstallObservation = { /** Service removal outcome, or null when the step threw. */ serviceRemoval: "absent" | "removed" | "failed" | null; /** - * For a Task Scheduler backend: was the restart window verified AFTER removal? + * Was the restart window verified AFTER removal? Needed after an older CLI's + * `stopped-respawnable` and, on Windows, after every `stopped`: there it is Part B of the + * checked stop (#7771), which proves the port stays free and no wrapper came back. * * Deleting the registration does not prove an already-running `:loop` wrapper died — - * killing it is best-effort (#764). `ocx stop` polls across the window; uninstall has - * to do the same before it may take shared config down. + * `ocx stop` confirms across the window; uninstall has to do the same before it may take + * shared config down (#764). */ respawnWindowVerified: boolean; }; -export function sharedTeardownAuthorized(o: UninstallObservation): boolean { +/** Whether the restart window must be verified before shared config may come down. */ +export function respawnWindowRequired( + o: Pick, + platform: NodeJS.Platform = process.platform, +): boolean { + return o.serviceStop === "stopped-respawnable" || (platform === "win32" && o.serviceStop === "stopped"); +} + +/** + * Whether the uninstall may remove the installed manager. + * + * On Windows the removal deletes a Task Scheduler registration and stops/deletes a WinSW + * service. Both are only safe once Part A proved the manager stopped or that none is + * registered: `ocx service uninstall` refuses in the same states, and removing the + * registration of a manager that could not be stopped hides the reason the proxy may + * come back instead of fixing it. Other platforms remove after their own stop attempt, as + * before. + */ +export function serviceRemovalAllowed( + o: Pick, + platform: NodeJS.Platform = process.platform, +): boolean { + if (platform !== "win32") return true; + return o.serviceStop === "stopped" || o.serviceStop === "absent"; +} + +export function sharedTeardownAuthorized(o: UninstallObservation, platform: NodeJS.Platform = process.platform): boolean { if (o.serviceStop === null) return false; // "absent" and a clean stop are the only service states that prove nothing is managing // the proxy. if (o.serviceStop === "failed" || o.serviceStop === "state-unknown") return false; // Removing the registration is not the same as proving the running wrapper is gone. - if (o.serviceStop === "stopped-respawnable" && !o.respawnWindowVerified) return false; + if (respawnWindowRequired(o, platform) && !o.respawnWindowVerified) return false; if (o.serviceRemoval === null || o.serviceRemoval === "failed") return false; return o.proxyProvenDown; } +/** + * The "service removed" step: gate, then remove. + * + * Kept beside the rule so the command body stays a call, and so the refusal is testable + * without reading `handleUninstall`. A refused gate throws, which the step records as a + * failure and which keeps the shared teardown blocked. + */ +export function uninstallServiceRemoval( + o: Pick, + remove: () => "absent" | "removed" | "failed", + platform: NodeJS.Platform = process.platform, +): "absent" | "removed" | "failed" { + if (!serviceRemovalAllowed(o, platform)) { + throw new Error( + "the service manager could not be proven stopped, so its registration was left in place; " + + "resolve the 'service stopped' failure, then rerun 'ocx uninstall'", + ); + } + return remove(); +} + /** An endpoint an uninstall must account for before shared config comes down. */ export type ProbeEndpoint = { hostname: string; port: number }; @@ -93,3 +142,26 @@ export async function everyEndpointProvenDownAsync( const results = await Promise.all(endpoints.map(e => probe(e))); return results.every(result => result === "dead"); } + +/** + * The uninstall's "nothing answers on any endpoint this home could serve" proof, with the + * candidate set captured NOW. The proxy stop removes the runtime record (and a stopped proxy + * may remove its own), so a set read after the stop would silently drop the stopped proxy's + * port. Every candidate, not just the preferred one: a stale runtime record pointing at a + * closed port would otherwise "prove" a live proxy on the configured port is gone. Any + * failure to read or probe is not proof. + */ +export function captureUninstallEndpointProof( + readCandidates: () => ProbeEndpoint[], + probe: (e: ProbeEndpoint) => Promise<"live" | "dead" | "unknown"> | "live" | "dead" | "unknown", +): () => Promise { + let candidates: ProbeEndpoint[] | null = null; + try { candidates = readCandidates(); } catch { candidates = null; } + return async () => { + try { + return candidates !== null && await everyEndpointProvenDownAsync(candidates, probe); + } catch { + return false; + } + }; +} diff --git a/src/cli/uninstall-stop.ts b/src/cli/uninstall-stop.ts new file mode 100644 index 00000000000..d2aa2a31e0b --- /dev/null +++ b/src/cli/uninstall-stop.ts @@ -0,0 +1,152 @@ +/** + * `ocx uninstall`'s proxy stop, extracted from `handleUninstall` so the decision is + * callable rather than only readable (#7771, #3008). + * + * The step has two jobs that used to be implicit in the command body: find the proxy + * (tracked pid, or an identity-checked orphan), and stop it without letting a wrapper + * relaunch it after shared client config has already been restored. + * + * On Windows the stop is deferred exactly as `ocx stop` and `ocx service stop` do it: a + * teardown receipt is claimed BEFORE the child stop and its nonce travels on the POST, so + * the proxy does not restore shared client config while a Task Scheduler wrapper can still + * relaunch it. The caller discharges the receipt only after Part B and its own restore + * succeeded; otherwise it stays on disk for recovery. + * + * A Part A that did not end stopped/absent still gets the graceful POST — the proxy may be + * running and a graceful stop is always preferable — but never the forced fallback: a + * `taskkill /T /F` under a manager that would not stop is the one outcome the checked stop + * exists to prevent. + */ +import { claimPendingTeardown, type PendingTeardownPartA } from "../config/pending-teardown"; +import { readRuntimePort } from "../config/process-state"; +import type { GracefulStopIo } from "../lib/process-control"; + +export interface UninstallProxyStopIo { + platform: NodeJS.Platform; + /** Detailed Part A outcome, or null when the service-stop step threw. */ + serviceStop: "absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown" | null; + readPid: () => number | null; + findLiveProxy: () => Promise<{ pid?: number | null; port: number; hostname?: string } | null>; + /** Tri-state proof that nothing answers on any endpoint this home could serve. */ + endpointProvenDown: () => Promise; + /** The Part A record the receipt carries, so /api/stop skips a second Part A. */ + partARecord?: () => PendingTeardownPartA | null; + claim?: ( + endpoint: { hostname: string; port: number }, + source: "exact" | "guessed", + ownerPid: number, + handover: "update-restart" | undefined, + partA: PendingTeardownPartA | null, + ) => { nonce: string }; + /** The tracked target's own runtime record; null when there is none for that pid. */ + runtimeEndpointFor?: (pid: number) => { hostname: string; port: number } | null; + stop: (pid: number, io: GracefulStopIo) => Promise; + removeTracked: (pid: number) => void; + /** Whether Part B owns the Windows port wait (#7771). */ + portWaitOwned: () => boolean; +} + +export interface UninstallProxyStopResult { + changed: boolean; + provenDown: boolean; + /** The deferred-teardown receipt this step claimed, for the caller to discharge. */ + nonce?: string; +} + +/** The runtime record a tracked pid wrote, which is the only exact endpoint it can be reached on. */ +function trackedRuntimeEndpoint(pid: number): { hostname: string; port: number } | null { + const runtime = readRuntimePort(pid); + return runtime?.port ? { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port } : null; +} + +export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise { + const windows = io.platform === "win32"; + // Part A not stopped/absent means a manager may still be running and could respawn the + // proxy. The graceful POST is still worth sending; the forced fallback is not. + const partAProven = io.serviceStop === "stopped" || io.serviceStop === "stopped-respawnable" || io.serviceStop === "absent"; + const forbidForcedFallback = windows && !partAProven; + // An unconfirmed stop normally still leaves this step's proof to the respawn window that + // follows. With the forced ladder forbidden it must not: the process may be alive, and the + // Part A failure the caller already recorded is what keeps shared config up. + // + // An unconfirmed stop is also NOT evidence that this home's proxy is down. `stopProxy` + // answers false without contacting the endpoint when the tracked pid is already dead, so + // "the tracked proxy is gone" and "nothing else is serving on this home's ports" are two + // different claims; Part A saying `absent`/`stopped` covers only the manager it saw. A + // different proxy on the configured port would otherwise be missed, and shared config + // restored under it (#3008). The tri-state probe the discovery path already requires is + // what settles it, and only a definitive "nothing answers anywhere" keeps the proof. + const confirmed = async (stopped: boolean): Promise => { + // A confirmed stop proves the tracked proxy down, nothing more. After an "absent" Part A on + // Windows no Part B follows, so a second proxy on another candidate endpoint would go + // unseen and shared config would be restored under it (#3008): every candidate has to + // answer definitively dead. After "stopped", Part B confirms the port instead. + if (stopped) return !(windows && io.serviceStop === "absent") || await io.endpointProvenDown(); + if (forbidForcedFallback) return false; + return await io.endpointProvenDown(); + }; + + // Resolve the target BEFORE anything is recorded. The receipt is an obligation on a + // specific address, so claiming it from a runtime record the stop never contacts — or + // before knowing whether a stop will happen at all — writes a promise nobody can + // discharge: recovery probes exactly the endpoint the receipt names. + let target: { pid: number; tracked: boolean; endpoint: { hostname: string; port: number } | null }; + const pid = io.readPid(); + if (pid) { + target = { pid, tracked: true, endpoint: (io.runtimeEndpointFor ?? trackedRuntimeEndpoint)(pid) }; + } else { + // A missing pid file is not proof that nothing is serving: a proxy can outlive its + // record (crash, manual delete, corrupt file), which is exactly why `ocx stop` falls + // back to identity-checked discovery. Without this, uninstall restored shared config + // and reported success while that proxy kept running (#3008). + const live = await io.findLiveProxy(); + if (!live) { + // A miss is not proof: `findLiveProxy` collapses a timeout and a transport failure + // into the same null as a dead endpoint. Ask the tri-state probe, which only says + // "dead" for a refused connection or a definitive non-OpenCodex answer (#3008). + const provenDown = await io.endpointProvenDown(); + if (!provenDown) { + throw new Error("no proxy could be found, but its endpoint could not be confirmed down either; confirm nothing is serving, then rerun"); + } + // Nothing will be stopped, so nothing may be deferred. + return { changed: false, provenDown }; + } + if (!live.pid) { + throw new Error(`a proxy is answering on port ${live.port} but no process id could be resolved for it; stop it from the home that started it, then rerun`); + } + // A discovered proxy answers where discovery found it, which is the only address a stop + // can reach it on — not whatever a stale runtime record happens to say. + target = { pid: live.pid, tracked: false, endpoint: { hostname: live.hostname ?? "127.0.0.1", port: live.port } }; + } + // A receipt exists to carry a deferred teardown across the POST. Without an exact endpoint + // there is no POST to defer: `stopProxy` falls to its own path, and a "guessed" receipt + // would name an address no stop contacted while recovery then refuses to finish it. + let nonce: string | undefined; + if (windows && target.endpoint) { + try { + nonce = (io.claim ?? claimPendingTeardown)(target.endpoint, "exact", process.pid, undefined, (io.partARecord ?? (() => null))()).nonce; + } catch (error) { + // Without the receipt the proxy performs its own teardown during the stop, which is + // the pre-#3008 behaviour — and on Windows a wrapper can relaunch it afterwards, so + // client config would already be restored under a live proxy. Refuse instead. + throw new Error( + "the deferred-teardown receipt could not be recorded, so the proxy was left running: " + + (error instanceof Error ? error.message : String(error)), + ); + } + } + const stopIo: GracefulStopIo = { + skipPortWait: io.portWaitOwned(), + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + ...(forbidForcedFallback ? { allowForcedFallback: false } : {}), + // The same endpoint the receipt names, so the POST and the obligation cannot diverge. + ...(target.endpoint ? { runtimeEndpoint: target.endpoint } : {}), + }; + const provenDown = await confirmed(await io.stop(target.pid, stopIo)); + // Only a proven stop cleared the record it names. Removing it after a POST that timed out + // drops the only pointer to a process that may still be running, and reporting the step as + // changed would claim a state that was never observed. `provenDown` is how the caller + // learns it must record the failure (it keeps the shared teardown up). + if (target.tracked && provenDown) io.removeTracked(target.pid); + return { changed: target.tracked ? provenDown : true, provenDown, ...(nonce ? { nonce } : {}) }; +} diff --git a/src/config/pending-teardown.ts b/src/config/pending-teardown.ts index 5632d0739e5..ea29289bd5c 100644 --- a/src/config/pending-teardown.ts +++ b/src/config/pending-teardown.ts @@ -63,6 +63,20 @@ export type PendingTeardownReceipt = { * replacement ever comes up, the next `ocx stop` finishes it like any abandoned receipt. */ handover?: "update-restart"; + /** + * Windows checked stop (#7771): what the owner's Part A did before it sent the stop. + * /api/stop skips its own Part A only for a receipt carrying this record whose manager + * is still the registered one; a receipt without it (an older CLI, or one written + * before Part A) gets the route's own capped Part A. Readers that predate it ignore it. + */ + partA?: PendingTeardownPartA; +}; + +/** The Part A record of a teardown receipt: the outcome, the manager, the ended wrappers. */ +export type PendingTeardownPartA = { + partA: "stopped" | "absent"; + manager: { scheduler: boolean; winsw: boolean }; + wrappers: { pid: number; startTime: number }[]; }; /** @@ -130,6 +144,7 @@ export function claimPendingTeardown( endpointSource: "exact" | "guessed", ownerPid: number = process.pid, handover?: "update-restart", + partA?: PendingTeardownPartA | null, ): PendingTeardownReceipt { const dir = getConfigDir(); assertNotRealHomeUnderTest(dir); @@ -137,11 +152,44 @@ export function claimPendingTeardown( const receipt: PendingTeardownReceipt = { ownerPid, nonce, createdAt: new Date().toISOString(), endpoint, endpointSource, ...(handover ? { handover } : {}), + ...(partA ? { partA } : {}), }; atomicWriteFile(pendingTeardownPathFor(nonce), JSON.stringify(receipt, null, 2) + "\n"); return receipt; } +/** + * Add the Part A record to a receipt this process claimed before Part A ran (the guarded + * stop and an update handover claim first). Only the owner's own valid receipt is + * rewritten; anything else is left alone and reported as false. + */ +export function recordPendingTeardownPartA(nonce: string, partA: PendingTeardownPartA, ownerPid: number = process.pid): boolean { + const read = readPendingTeardown(nonce); + if (read.state !== "valid" || read.receipt.ownerPid !== ownerPid) return false; + assertNotRealHomeUnderTest(getConfigDir()); + atomicWriteFile(pendingTeardownPathFor(nonce), JSON.stringify({ ...read.receipt, partA }, null, 2) + "\n"); + return true; +} + +/** The Part A record of the receipt a request names, when it is present and well formed. */ +export function pendingTeardownPartA(nonce: string | null): PendingTeardownPartA | null { + if (!nonce || !NONCE_RE.test(nonce)) return null; + const read = readPendingTeardown(nonce); + if (read.state !== "valid") return null; + const record = read.receipt.partA as unknown; + if (!record || typeof record !== "object") return null; + const value = record as Record; + const manager = value.manager as Record | undefined; + const ok = (value.partA === "stopped" || value.partA === "absent") + && !!manager && typeof manager === "object" + && typeof manager.scheduler === "boolean" && typeof manager.winsw === "boolean" + && Array.isArray(value.wrappers) + && value.wrappers.every(entry => !!entry && typeof entry === "object" + && Number.isSafeInteger((entry as { pid?: unknown }).pid) + && Number.isFinite((entry as { startTime?: unknown }).startTime)); + return ok ? read.receipt.partA! : null; +} + export function readPendingTeardown(nonce: string): PendingTeardownRead { if (!NONCE_RE.test(nonce)) return { state: "missing" }; let raw: string; diff --git a/src/lib/process-control.ts b/src/lib/process-control.ts index d2fe5b7471f..ad79b814bd2 100644 --- a/src/lib/process-control.ts +++ b/src/lib/process-control.ts @@ -61,6 +61,34 @@ export interface GracefulStopIo { * endpoint the stop never contacted — and recovery probes exactly that endpoint. */ runtimeEndpoint?: { hostname: string; port: number }; + /** + * Windows checked stop (#7771): Part B confirms the port after this stop returns and + * owns the wait, so the 15 s waitForStoppedPort here is skipped (030 "Time budget"). + */ + skipPortWait?: boolean; + /** + * Whether a graceful stop that did not confirm may fall back to the platform kill ladder. + * + * `ocx uninstall` sets this false when its Part A did not end stopped/absent: a manager + * that would not stop can respawn the proxy, and `taskkill /T /F` under it is exactly the + * forced stop the checked Windows stop exists to prevent. A 409 refusal never reaches the + * fallback either way. Defaults to true, which is every other caller's behaviour. + */ + allowForcedFallback?: boolean; + /** + * The forced ladder, injectable so the rule above can be proven without signalling a + * process. Production callers omit it and get {@link killProxy}. + */ + forceKill?: (pid: number) => void; + /** + * The post-stop port wait, injectable so a test can present a verified listener without + * starting one. Production callers omit it and get `reclaimListenPort`. + */ + reclaimPort?: ( + port: number, + hostname: string, + opts: { timeoutMs?: number; intervalMs?: number; scanIntervalMs?: number; killOcxHolders?: boolean; onlyKillPids?: number[] }, + ) => Promise; } /** @@ -319,7 +347,7 @@ function drainDeadlineMs(): number { /** Graceful-first stop: management-API drain, then the platform kill ladder. */ export async function stopProxy(pid: number, io: GracefulStopIo = {}): Promise { if (!isProcessAlive(pid)) return false; - const runtime = io.runtimeEndpoint ?? readRuntimePort(pid); + const runtime = io.skipPortWait ? null : io.runtimeEndpoint ?? readRuntimePort(pid); const { result: graceful, refusal } = await stopProxyGracefullyDetailed(pid, io); if (graceful === "refused") { // The proxy refused on purpose. Forcing would strip shared config while whatever owns @@ -334,39 +362,62 @@ export async function stopProxy(pid: number, io: GracefulStopIo = {}): Promise { if (!runtime?.port) return; + const mayKill = io.allowForcedFallback !== false; try { - const { reclaimListenPort } = await import("../server/port-reclaim"); - await reclaimListenPort(runtime.port, runtime.hostname ?? "127.0.0.1", { + const reclaim = io.reclaimPort ?? (await import("../server/port-reclaim")).reclaimListenPort; + await reclaim(runtime.port, runtime.hostname ?? "127.0.0.1", { timeoutMs: 15_000, intervalMs: 100, scanIntervalMs: 500, // Only the process we just stopped — never kill a newly started twin proxy. - killOcxHolders: !!(stoppedPid && stoppedPid > 0), - onlyKillPids: stoppedPid && stoppedPid > 0 ? [stoppedPid] : [], + killOcxHolders: mayKill && !!(stoppedPid && stoppedPid > 0), + onlyKillPids: mayKill && stoppedPid && stoppedPid > 0 ? [stoppedPid] : [], }); } catch { /* best-effort — callers that need a hard guarantee reclaim again before bind */ } } +/** taskkill in the forced fallback; it had no timeout before #7771. */ +export const FORCED_TASKKILL_TIMEOUT_MS = 5_000; + export function killProxy(pid: number, mayTerminate?: () => boolean): void { if (!isProcessAlive(pid)) return; const authorize = () => { @@ -381,7 +432,8 @@ export function killProxy(pid: number, mayTerminate?: () => boolean): void { const args = ["/PID", String(pid), "/T", "/F"]; authorize(); try { - execFileSync(taskkill, args, { stdio: "pipe", windowsHide: true }); + // Capped (#7771): the checked stop's budget counts the forced kill at 5 s. + execFileSync(taskkill, args, { stdio: "pipe", windowsHide: true, timeout: FORCED_TASKKILL_TIMEOUT_MS }); } catch (err) { if (isProcessAlive(pid)) throw err; } diff --git a/src/server/management-api.ts b/src/server/management-api.ts index b397ab53934..4b319e7c92b 100644 --- a/src/server/management-api.ts +++ b/src/server/management-api.ts @@ -495,7 +495,9 @@ export async function handleManagementAPI( } let serviceStop: import("../service").ServiceStopOutcome; try { - serviceStop = sibling ? "absent" : stopServiceIfInstalledDetailed(); + // #7771: on Windows the route skips Part A for a receipt that records it, caps it at + // 6 s for one that does not, and sweeps a deleted task's stray wrapper in 4 s. + serviceStop = sibling ? "absent" : await (await import("./stop-service-route")).routeServiceStop(url, holdsReceipt); } catch (err) { if (isServiceOwnershipError(err)) { // The installed service belongs to another CODEX_HOME/OPENCODEX_HOME: it would respawn diff --git a/src/server/stop-service-route.ts b/src/server/stop-service-route.ts new file mode 100644 index 00000000000..4945940fa54 --- /dev/null +++ b/src/server/stop-service-route.ts @@ -0,0 +1,85 @@ +/** + * The service stop inside POST /api/stop (#7771, 030 "/api/stop"). + * + * The route runs inside the proxy that holds the port, so it can never confirm the port + * itself; Part B belongs to the parent ocx stop. On Windows the route splits by case: + * + * - a receipt whose Part A record names the manager that is still registered: the parent + * already ran Part A, so the route skips it (no second enumeration, /end or wait); + * - a receipt without that record (an older CLI, or one claimed before Part A): today's + * behavior, the service stop, now Part A capped at 6 s; an overrun is + * service_state_unknown; + * - no receipt (the caller already refused a respawnable task, so nothing is registered + * with the Task Scheduler): a registered WinSW service keeps today's in-process stop, + * because WinSW's own stop ends this very process and cannot be capped from inside it; + * otherwise Part A's stray sweep for a deleted task's wrapper, capped at 4 s, so the + * route still answers inside the parent's 10 s POST abort. + * + * Off Windows it is today's stopServiceIfInstalledDetailed. + */ +import { pendingTeardownPartA, type PendingTeardownPartA } from "../config/pending-teardown"; +import type { ServiceStopOutcome } from "../service/orchestration"; + +export const ROUTE_PART_A_CAP_MS = 6_000; +export const ROUTE_STRAY_SWEEP_CAP_MS = 4_000; +const REGISTRATION_PROBE_MS = 2_000; + +export interface RouteServiceStopIo { + platform?: NodeJS.Platform; + stop?: (request: { capMs?: number; allowUnderOwnWrapper?: boolean }) => Promise; + assertEnvironment?: () => void; + readRecord?: (nonce: string | null) => PendingTeardownPartA | null; + registration?: () => Promise<{ scheduler: "present" | "absent" | "unknown"; winsw: "started" | "stopped" | "nonexistent" | "unknown" }>; + winswRegistered?: () => boolean; + stopWinsw?: () => void; +} + +async function liveRegistration(): Promise>>> { + const { windowsServiceStopHooks } = await import("../service/windows-service-stop"); + const hooked = windowsServiceStopHooks()?.registration; + if (hooked) return hooked(); + const { windowsCheckedStopLiveDeps } = await import("../service/windows-checked-stop-live"); + const deps = windowsCheckedStopLiveDeps(); + const [scheduler, winsw] = await Promise.all([ + Promise.resolve(deps.probeScheduler(REGISTRATION_PROBE_MS)).then(probe => probe.status, () => "unknown" as const), + Promise.resolve(deps.probeWinsw(REGISTRATION_PROBE_MS)).catch(() => "unknown" as const), + ]); + return { scheduler, winsw }; +} + +/** The record still describes this machine: the same managers are registered. */ +export function partARecordMatchesRegistration( + record: PendingTeardownPartA, + current: Awaited>>, +): boolean { + if (current.scheduler === "unknown" || current.winsw === "unknown") return false; + return record.manager.scheduler === (current.scheduler === "present") + && record.manager.winsw === (current.winsw !== "nonexistent"); +} + +export async function routeServiceStop(url: URL, holdsReceipt: boolean, io: RouteServiceStopIo = {}): Promise { + const service = await import("../service"); + const stop = io.stop ?? (request => service.stopServiceIfInstalledDetailed(request)); + if ((io.platform ?? process.platform) !== "win32") return stop({}); + // The ownership check every service stop ran before still runs first. + (io.assertEnvironment ?? service.assertServiceEnvironmentMatchesInstall)(); + if (holdsReceipt) { + const record = (io.readRecord ?? pendingTeardownPartA)(url.searchParams.get("teardownNonce")); + if (record && partARecordMatchesRegistration(record, await (io.registration ?? liveRegistration)())) { + return record.partA; + } + // The route runs inside the proxy its wrapper supervises: ending that wrapper is the point. + return stop({ capMs: ROUTE_PART_A_CAP_MS, allowUnderOwnWrapper: true }); + } + const winsw = await import("../lib/winsw"); + const winswRegistered = io.winswRegistered ?? (() => winsw.statusWinswRaw() !== "nonexistent"); + if (winswRegistered()) { + try { + (io.stopWinsw ?? (() => winsw.stopWinswService()))(); + return "stopped"; + } catch { + return "failed"; + } + } + return stop({ capMs: ROUTE_STRAY_SWEEP_CAP_MS, allowUnderOwnWrapper: true }); +} diff --git a/src/server/windows-tcp-drop.ts b/src/server/windows-tcp-drop.ts index 518aae24904..0b646abbb20 100644 --- a/src/server/windows-tcp-drop.ts +++ b/src/server/windows-tcp-drop.ts @@ -13,8 +13,14 @@ export type TcpQuad = { state: string; }; -/** Parse netstat -ano rows whose local address uses `port`. Exported for tests. */ -export function parseTcpQuadsForLocalPort(output: string, port: number): TcpQuad[] { +/** + * Parse netstat -ano rows whose local address uses `port`. Exported for tests. + * + * `ownerPids` restricts the rows to those the listed processes own. A row left by a process + * that has exited is that process's to clear; a program that took the port between scans owns + * its own rows, and resetting them would break a connection this stop does not own. + */ +export function parseTcpQuadsForLocalPort(output: string, port: number, ownerPids?: readonly number[]): TcpQuad[] { const rows: TcpQuad[] = []; const portSuffix = `:${port}`; for (const rawLine of output.split(/\r?\n/)) { @@ -26,6 +32,10 @@ export function parseTcpQuadsForLocalPort(output: string, port: number): TcpQuad const remote = parts[2]!; const state = parts[3]!; if (!local.endsWith(portSuffix) && !local.endsWith(`]:${port}`)) continue; + if (ownerPids !== undefined) { + const rowPid = Number(parts[4]); + if (!Number.isSafeInteger(rowPid) || !ownerPids.includes(rowPid)) continue; + } const localAddr = stripPort(local, port); const remoteParsed = splitHostPort(remote); if (!localAddr || !remoteParsed) continue; @@ -134,10 +144,17 @@ function readNetstatAno(): string { /** * Force-delete IPv4 TCP rows bound to `localPort` via SetTcpEntry(DELETE_TCB). * Does not kill foreign processes — only resets sockets so the listen port can bind again. + * `ownerPids` restricts the deletion to rows those processes own (see + * parseTcpQuadsForLocalPort); omitted, every row on the port is cleared, which is what the + * hard-kill path wants and what a stale-port sweep must not do. * Bare IPv6 rows (`::1`, `::`, etc.) are skipped (not coerced into IPv4 wildcards); * IPv6 TCB reclamation is unsupported on this path. */ -export function dropWindowsTcpRowsForLocalPort(port: number, mayMutate?: () => boolean): WindowsTcpDropResult { +export function dropWindowsTcpRowsForLocalPort( + port: number, + mayMutate?: () => boolean, + ownerPids?: readonly number[], +): WindowsTcpDropResult { if (process.platform !== "win32" || !Number.isFinite(port) || port <= 0) { return { dropped: 0, skippedIpv6: 0, accessDenied: 0 }; } @@ -151,7 +168,7 @@ export function dropWindowsTcpRowsForLocalPort(port: number, mayMutate?: () => b return { dropped: 0, skippedIpv6: 0, accessDenied: 0 }; } - const rows = parseTcpQuadsForLocalPort(output, Math.trunc(port)); + const rows = parseTcpQuadsForLocalPort(output, Math.trunc(port), ownerPids); let dropped = 0; let skippedIpv6 = 0; let accessDenied = 0; diff --git a/src/service.ts b/src/service.ts index ceb10e010de..da13da0eb5a 100644 --- a/src/service.ts +++ b/src/service.ts @@ -25,13 +25,13 @@ export { decodeSchtasksOutput, setQuerySchtasksForTests, formatWindowsSchedulerS export type { WindowsSchedulerXmlState } from "./service/windows-taskxml"; export { buildWindowsServiceScript, buildWindowsSchtasksCreateArgs, buildWindowsSchtasksCreateArgsForXml, buildWindowsLauncherVbs, buildWindowsTaskXml, buildWindowsTaskXmlDocument, windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy, readWindowsSchedulerXmlState } from "./service/windows-taskxml"; export type { WindowsSchedulerRegistrationStageDeps, FreshWindowsSchedulerRegistrationDeps, RemoveNativeWindowsServiceDeps } from "./service/windows-ops"; -export { windowsListenPort, winswListenPort, writeServiceDefinitionFile, definitionCarriesCredential, stageWindowsSchedulerRegistrationXml, stageElevatedSchedulerRegistration, describeElevatedRegistrationFailure, registerFreshWindowsSchedulerTask, removeNativeWindowsServiceForScheduler, assertWindowsNativeServiceAccountSupported, isWindowsSchedulerEndBenign, stopWindows, stopWindowsChecked, classifyWindowsServiceStop } from "./service/windows-ops"; +export { windowsListenPort, winswListenPort, writeServiceDefinitionFile, definitionCarriesCredential, stageWindowsSchedulerRegistrationXml, stageElevatedSchedulerRegistration, describeElevatedRegistrationFailure, registerFreshWindowsSchedulerTask, removeNativeWindowsServiceForScheduler, assertWindowsNativeServiceAccountSupported, isWindowsSchedulerEndBenign, stopWindows } from "./service/windows-ops"; export type { ServiceRepairVerb, RepairServiceDeps } from "./service/repair"; export { repairService, foreignServiceOwnerRefusal, unknownServiceOwnerRefusal } from "./service/repair"; export type { ServiceChildOwnershipDecision } from "./service/service-child-ownership"; export { serviceChildOwnershipDecision, serviceChildStayOutExitCode } from "./service/service-child-ownership"; export type { ServiceInstallPreparationDeps, FreshWindowsSchedulerInstallDeps, ServiceStopOutcome, ServiceUninstallOutcome } from "./service/orchestration"; -export { proxyStillLiveAfterStop, prepareServiceInstall, installServiceSafely, installFreshWindowsSchedulerSafely, installedServiceRespawnRisk, stopServiceIfInstalledDetailed, setUninstallServiceHooksForTests, uninstallServiceDetailed, uninstallServiceIfInstalled, isServiceInstalled, isServiceViable } from "./service/orchestration"; +export { proxyStillLiveAfterStop, prepareServiceInstall, installServiceSafely, installFreshWindowsSchedulerSafely, installedServiceRespawnRisk, stopServiceIfInstalledDetailed, stopServiceIfInstalledReport, setUninstallServiceHooksForTests, uninstallServiceDetailed, uninstallServiceIfInstalled, isServiceInstalled, isServiceViable } from "./service/orchestration"; export type { ServiceDiagnostic, WindowsTaskDiagnosticIdentityDeps, WindowsServiceDiagnosticInputs, LaunchdServiceDiagnosticInputs } from "./service/diagnostics"; export { bakedServicePathsDiagnostic, serviceStartableFromTray, resolveWindowsTaskDiagnosticUserId, deriveWindowsServiceDiagnostic, deriveWindowsServiceDiagnosticForCurrentUser, deriveLaunchdServiceDiagnostic, diagnoseService, serviceStatusSummary, serviceStatusReport } from "./service/diagnostics"; export type { ParsedServiceArgs, ServiceInstallationState, ServiceInstallationProbe, ServiceInstallationProbeHooks, ServiceCommandPlan } from "./service/cli"; diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts new file mode 100644 index 00000000000..dac0c2c097d --- /dev/null +++ b/src/service/cli-windows-stop.ts @@ -0,0 +1,189 @@ +/** + * ocx service stop / uninstall / remove on Windows (#7771, 030 "Where it sits"). + * + * - stop: Part A instead of stopWindows; a result other than stopped/absent refuses and + * leaves the proxy alone. Then a teardown receipt is claimed before the tracked proxy is + * stopped, so the proxy defers the shared teardown to this command. When the claim fails + * while this home owns the shared teardown, no undeferred child stop is sent: a wrapper + * appearing later could relaunch the proxy after its own teardown ran. Part B replaces + * the legacy survivor check after a Part A stopped. With an absent Part A every candidate + * endpoint must be proven dead before restore; the receipt is kept otherwise. + * - uninstall / remove: Part A before the task is deleted (a looping wrapper used to + * outlive its task, #764), refusing on anything but stopped/absent, then a receipt that + * carries Part A's record before the tracked proxy stop (Part A leaves the task + * registered, so a receipt-less POST answers 409 respawnable_service), then Part B. The + * receipt is kept until the caller's restore succeeds. + */ +import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-teardown"; +import { readClientConnectionState } from "../client/state"; +import type { ProbeEndpoint } from "../cli/uninstall-plan"; +import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; +import { resolveTrackedProxyTarget, stopTrackedProxyForServiceCommand } from "./orchestration"; +import { + confirmWindowsServiceStop, + describeWindowsPartA, + describeWindowsPartB, + endpointsAllDead, + runWindowsServiceStopPartA, + windowsPartARecord, + windowsRemovalEndpoints, + windowsServiceStopPort, + windowsStopAdmission, + windowsStopRemainingMs, +} from "./windows-service-stop"; +import { WINDOWS_STOP_CAPS } from "./windows-stop-budget"; + +export interface WindowsServiceStopCommandIo { + ownsSharedTeardown?: () => boolean; + claim?: typeof claimPendingTeardown; + stopTracked?: typeof stopTrackedProxyForServiceCommand; + resolveTarget?: typeof resolveTrackedProxyTarget; + /** + * Tri-state probe of an endpoint. Only a definitive "dead" on EVERY candidate permits + * deleting the task; "live" and "unknown" refuse and keep the receipt. + */ + endpointLiveness?: (endpoint: { hostname: string; port: number }) => Promise; + /** + * Every endpoint this home could be serving on, for the absent-Part-A proof. Defaults to + * `endpointsToProve(readRuntimePort(), loadConfig())` — the set `ocx uninstall` proves. + */ + endpoints?: () => ProbeEndpoint[]; +} + +/** Shared teardown belongs to this home unless a client connection owns the routing. */ +function ownsSharedTeardownByDefault(): boolean { + try { + return readClientConnectionState().kind !== "connected"; + } catch { + return true; + } +} + +function withNonce(nonce: string | undefined): { ok: true; nonce?: string } { + return nonce ? { ok: true, nonce } : { ok: true }; +} + +/** + * Everything ocx service stop does before its restore. ok=false means the command fails + * and restores nothing; nonce is the receipt to complete once the restore succeeded. + */ +export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopCommandIo = {}): Promise<{ ok: boolean; nonce?: string }> { + const port = windowsServiceStopPort(); + const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); + if (partA.outcome !== "stopped" && partA.outcome !== "absent") { + console.error("❌ service stop did not take effect: the Windows service could not be proven stopped (" + partA.outcome + ")."); + console.error(" " + describeWindowsPartA(partA)); + console.error(" The proxy was left running and native Codex was NOT restored."); + return { ok: false }; + } + let nonce: string | undefined; + const target = await (io.resolveTarget ?? resolveTrackedProxyTarget)(); + if (target && (io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + if (!target.endpoint) { console.error("❌ service stop cannot defer teardown without the target's endpoint."); return { ok: false }; } + try { + nonce = (io.claim ?? claimPendingTeardown)(target.endpoint, "exact", process.pid, undefined, windowsPartARecord(partA)).nonce; + } catch (error) { + console.error("❌ service stop could not record its deferred-teardown receipt: " + (error instanceof Error ? error.message : String(error))); + console.error(" The proxy was left running: stopping it without the receipt would let it tear down client config while a wrapper can still relaunch it."); + return { ok: false }; + } + } + const endpoints = partA.outcome === "absent" ? (io.endpoints ?? windowsRemovalEndpoints)() : []; + if (target?.endpoint && partA.outcome === "absent" && !endpoints.some(e => e.hostname === target.endpoint!.hostname && e.port === target.endpoint!.port)) endpoints.push(target.endpoint); + await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ + target, + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, + }); + if (partA.outcome === "absent") { + const proof = await endpointsAllDead(endpoints, io.endpointLiveness ?? probeEndpointLiveness); + if (proof.dead) return withNonce(nonce); + console.error(proof.live.length ? "❌ service stop did not take effect: a proxy is still listening on port " + proof.live.join(", ") + "." + : "❌ service stop did not take effect: endpoint liveness could not be confirmed on port " + proof.unknown.join(", ") + "."); + console.error(" Native Codex was NOT restored; the teardown receipt is kept for a later stop."); + return { ok: false }; + } + const partB = await confirmWindowsServiceStop({ port: target?.endpoint?.port ?? port, remainingMs: windowsStopRemainingMs() }); + if (partB.outcome === "freed") return withNonce(nonce); + if (partB.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(partB)); + return withNonce(nonce); + } + console.error("❌ service stop did not take effect: " + describeWindowsPartB(partB)); + console.error(" Native Codex was NOT restored; the teardown receipt is kept so a later stop can finish it."); + return { ok: false }; +} + +/** After a successful restore: the obligation the receipt recorded is discharged. */ +export function completeWindowsServiceStopReceipt(nonce: string | undefined, clear: (nonce: string) => boolean = clearPendingTeardown): void { + if (!nonce) return; + if (!clear(nonce)) console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); +} + +/** + * ocx service uninstall / remove: stop and confirm before the task is deleted. + * + * Part A leaves the scheduler task registered, so a receipt-less POST still answers + * `respawnable_service` (409) and the uninstall then refuses with the proxy left running. + * The receipt is therefore claimed with Part A's record before the child stop, exactly as + * `ocx service stop` does, and kept until the caller's restore succeeds. + */ +export async function windowsServiceRemovalStop( + io: Pick = {}, +): Promise<{ ok: boolean; nonce?: string }> { + const port = windowsServiceStopPort(); + const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); + if (partA.outcome !== "stopped" && partA.outcome !== "absent") { + console.error("❌ Service uninstall refused: the Windows service could not be proven stopped (" + partA.outcome + "), so the task was not deleted."); + console.error(" " + describeWindowsPartA(partA)); + return { ok: false }; + } + let nonce: string | undefined; + const target = await (io.resolveTarget ?? resolveTrackedProxyTarget)(); + const endpoints = partA.outcome === "absent" ? (io.endpoints ?? windowsRemovalEndpoints)() : []; + if (target?.endpoint && partA.outcome === "absent" && !endpoints.some(e => e.hostname === target.endpoint!.hostname && e.port === target.endpoint!.port)) endpoints.push(target.endpoint); + if (target && (io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + if (!target.endpoint) { console.error("❌ Service uninstall cannot defer teardown without the target's endpoint."); return { ok: false }; } + try { + nonce = (io.claim ?? claimPendingTeardown)(target.endpoint, "exact", process.pid, undefined, windowsPartARecord(partA)).nonce; + } catch (error) { + console.error("❌ Service uninstall could not record its deferred-teardown receipt: " + (error instanceof Error ? error.message : String(error))); + console.error(" The proxy was left running: stopping it without the receipt would let it tear down client config while the task can still relaunch it."); + return { ok: false }; + } + } + await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ + target, + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, + }); + if (partA.outcome === "absent") { + // "absent" says no manager is registered, not that nothing is serving. `findLiveProxy` + // inside the tracked stop can return null when its discovery cap runs out while the + // endpoint is live, and `proxyStillLiveAfterStop` returns that same null for a probe + // that timed out or failed — which the old branch read as success. Ask the endpoints + // themselves, tri-state, and delete the task only when EVERY candidate is definitively + // "dead": the receipt endpoint is only one of the addresses this home can serve on, and + // a stale runtime record pointing at a closed port used to prove removal while the + // configured port had a live listener. No respawn hold: with no manager registered + // there is nothing to relaunch the proxy. + const proof = await endpointsAllDead(endpoints, io.endpointLiveness ?? probeEndpointLiveness); + if (proof.dead) return withNonce(nonce); + console.error(proof.live.length > 0 + ? "❌ Service uninstall refused: a proxy is still listening on port " + proof.live.join(", ") + "." + : "❌ Service uninstall refused: the endpoint on port " + (proof.unknown.join(", ") || port) + " could not be confirmed down."); + console.error(" The task was not deleted. Stop that proxy, then re-run 'ocx service uninstall'."); + return { ok: false }; + } + const partB = await confirmWindowsServiceStop({ port: target?.endpoint?.port ?? port, remainingMs: windowsStopRemainingMs() }); + if (partB.outcome === "freed") return withNonce(nonce); + if (partB.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(partB)); + return withNonce(nonce); + } + console.error("❌ Service uninstall refused: " + describeWindowsPartB(partB) + " The task was not deleted."); + console.error(" The teardown receipt is kept so a later stop can finish it."); + return { ok: false }; +} diff --git a/src/service/cli.ts b/src/service/cli.ts index 2d7f8bd5d6c..a891f052655 100644 --- a/src/service/cli.ts +++ b/src/service/cli.ts @@ -32,6 +32,7 @@ import { runServiceClaim } from "./claim"; import { createSupervisionLatch, desktopServiceCommandRefusal } from "./desktop-command-guard"; import { linuxServiceBootHint } from "./linux-linger"; import { diagnoseDesktopStartup, desktopLoginStatusSummary } from "./desktop-startup"; +import { completeWindowsServiceStopReceipt, windowsServiceRemovalStop, windowsServiceStopBeforeRestore } from "./cli-windows-stop"; /** * `restart` is NO LONGER folded into `repair`. @@ -398,17 +399,26 @@ export async function serviceCommand(...args: (string | undefined)[]): Promise { +/** + * How long orphan discovery may take when the caller does not name a cap. + * + * Seven seconds is the legacy multi-candidate window. It stays the default so macOS and + * Linux stops behave exactly as before; a Windows caller that admitted a budget + * (030 "Time budget") passes its own 3 s cap instead. + */ +export const DEFAULT_ORPHAN_DISCOVERY_MS = 7_000; + +/** How the tracked proxy is stopped: the deferral receipt and the Windows port wait. */ +export interface TrackedProxyStopIo { + /** Pre-resolved Windows target; null means there is no child to stop. */ + target?: TrackedProxyTarget | null; + /** ocx service stop on Windows: the claimed receipt, so the proxy defers its teardown. */ + deferSharedTeardownNonce?: string; + /** Part B follows and owns the port wait (030), so stopProxy skips its own. */ + skipPortWait?: boolean; + /** + * Cap on the identity-checked orphan discovery that follows the tracked stop. + * + * The checked Windows stop admits `WINDOWS_STOP_CAPS.orphanDiscovery` (3 s) for this step + * (030 "Time budget"), and Part B runs inside the same deadline: discovery that spent the + * legacy 7 s would overrun the budget admission had just proven. Omitted keeps 7 s. + */ + orphanDiscoveryCapMs?: number; +} + +export interface TrackedProxyTarget { + pid: number; + endpoint: { hostname: string; port: number } | null; +} + +/** Resolve once before claiming a receipt, so its endpoint is the stop's endpoint. */ +export async function resolveTrackedProxyTarget(): Promise { + const pid = verifiedKillTarget(readPid()); + if (pid !== null && isProcessAlive(pid)) { + const runtime = readRuntimePort(pid); + return { pid, endpoint: runtime?.port ? { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port } : null }; + } + const live = await findLiveProxy({ ...SERVICE_STOP_LIVENESS, deadlineAt: Date.now() + WINDOWS_STOP_CAPS.orphanDiscovery, acceptPackageTreeFenced: true }); + if (!live) return null; + const livePid = verifiedKillTarget(live.pid); + if (livePid === null) throw new Error("A live proxy was found but its process identity could not be verified."); + return { pid: livePid, endpoint: { hostname: live.hostname ?? "127.0.0.1", port: live.port } }; +} + +async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise { + if (io.target !== undefined) { + if (!io.target) return "none"; + await stopProxy(io.target.pid, { ...io, ...(io.target.endpoint ? { runtimeEndpoint: io.target.endpoint } : {}) }); + removePid(io.target.pid); + removeRuntimePort(io.target.pid); + return "stopped"; + } let stopped = false; const pid = readPid(); const trackedKillPid = verifiedKillTarget(pid); if (trackedKillPid !== null && isProcessAlive(trackedKillPid)) { - await stopProxy(trackedKillPid); + await stopProxy(trackedKillPid, io); removePid(trackedKillPid); removeRuntimePort(trackedKillPid); stopped = true; @@ -206,15 +266,16 @@ async function stopTrackedProxyIfRunning(): Promise { } // Orphan recovery: the pid file can be missing/stale while the service wrapper keeps // a live proxy running — mirror `ocx stop`'s identity-checked findLiveProxy fallback. - // Cap multi-candidate discovery so stop cleanup cannot hang for three full retry budgets. + // Cap multi-candidate discovery so stop cleanup cannot hang for three full retry budgets, + // and let a caller that admitted a budget name its own cap (#7771). const live = await findLiveProxy({ ...SERVICE_STOP_LIVENESS, - deadlineAt: Date.now() + 7000, + deadlineAt: Date.now() + (io.orphanDiscoveryCapMs ?? DEFAULT_ORPHAN_DISCOVERY_MS), acceptPackageTreeFenced: true, }); const liveKillPid = verifiedKillTarget(live?.pid); if (liveKillPid !== null) { - await stopProxy(liveKillPid); + await stopProxy(liveKillPid, io); removePid(liveKillPid); removeRuntimePort(liveKillPid); stopped = true; @@ -224,9 +285,9 @@ async function stopTrackedProxyIfRunning(): Promise { return "none"; } -export async function stopTrackedProxyForServiceCommand(): Promise { +export async function stopTrackedProxyForServiceCommand(io: TrackedProxyStopIo = {}): Promise { try { - return await stopTrackedProxyIfRunning(); + return await stopTrackedProxyIfRunning(io); } catch (err) { console.error(`⚠️ Failed to stop proxy: ${err instanceof Error ? err.message : String(err)}`); return "none"; @@ -236,17 +297,68 @@ export async function stopTrackedProxyForServiceCommand(): Promise ServiceDiagnostic; managerOps?: (backend: ServiceBackend) => ServiceInstallCleanupOps | null; /** - * Task Scheduler only: ends surviving launcher/wrapper processes of THIS - * installation. Scoped to canonical script/launcher tokens, never a bare PID. + * Windows: Part A of the checked stop (#7771) for every registered manager and the + * bound wrappers of this home. Replaces the manager stops plus the best-effort sweep. */ - killServiceWrappers?: () => void; - stopTrackedProxy?: () => Promise; + windowsPartA?: (request: WindowsPartARequest) => Promise; + /** Windows: Part B after the tracked-proxy stop; the port must be free and stay free. */ + windowsPartB?: (request: { port: number; remainingMs: number }) => Promise; + /** + * Windows: the port the stopped proxy was serving, resolved ONCE before Part A (#7771). + * + * Resolving it after the tracked cleanup reads a runtime record the cleanup just deleted, + * so Part B confirmed the wrapper's port while the proxy's own port was never checked. + */ + windowsStopPort?: () => number; + /** Receives the deferral nonce so the wrapper's POST does not run the teardown. */ + stopTrackedProxy?: (io?: TrackedProxyStopIo) => Promise; + resolveTarget?: typeof resolveTrackedProxyTarget; + claimTeardown?: typeof claimPendingTeardown; + clearTeardown?: (nonce: string) => boolean; + /** Every endpoint an absent-manager install must prove down, resolved once. */ + installEndpoints?: () => ProbeEndpoint[]; + endpointLiveness?: (endpoint: ProbeEndpoint) => Promise; platform?: NodeJS.Platform; } +/** + * What an install preparation left behind. + * + * `deferredTeardownNonce` exists on Windows when the install stopped a target: it stopped + * the old proxy with the shared teardown deferred, so it owes the receipt. The install that + * replaces the proxy keeps shared config pointed at OpenCodex and completes the receipt + * after the new service is installed and started; a failed install leaves it for the next + * `ocx stop` to finish. + */ +export interface ServiceInstallPreparationResult { + deferredTeardownNonce?: string; +} + +/** + * Claim the deferred-teardown receipt the install's tracked stop will name. + * + * Part A leaves the scheduler task registered, so a receipt-less POST answers 409 + * `respawnable_service` and the stop throws while the old proxy keeps serving. The + * receipt is claimed with Part A's record before the POST, exactly as `ocx service stop` + * does; without it no undeferred stop is sent. + */ +function claimInstallTeardownReceipt( + deps: ServiceInstallPreparationDeps, + partA: WindowsCheckedStopResult, + endpoint: { hostname: string; port: number }, +): PendingTeardownReceipt { + try { + return (deps.claimTeardown ?? claimPendingTeardown)(endpoint, "exact", process.pid, undefined, windowsPartARecord(partA)); + } catch (error) { + throw new Error( + "The deferred-teardown receipt could not be recorded, so the proxy was left running and nothing was replaced: " + + (error instanceof Error ? error.message : String(error)), + ); + } +} + /** * Stop every manager that could own the install port, then stop the tracked * standalone listener. Any unknown status or cleanup failure rejects, so callers @@ -255,48 +367,86 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { export async function prepareServiceInstall( requestedBackend: ServiceBackend, deps: ServiceInstallPreparationDeps = {}, -): Promise { +): Promise { const latch = deps.supervisionLatch ?? createSupervisionLatch(); assertNoDesktopSupervision(deps.inspectSupervision, latch); - const diagnostic = (deps.diagnose ?? diagnoseService)(); const platform = deps.platform ?? process.platform; + if (platform === "win32") { + // Resolve the stopped proxy's port BEFORE anything removes its runtime record: Part B + // has to confirm that port, and the wrapper's port is only a fallback (#7771). + const stopPort = (deps.windowsStopPort ?? windowsServiceStopPort)(); + // Part A stops whichever managers are registered (both on a conflicting install) and + // ends the bound wrappers, or refuses with nothing deleted or replaced (#7771). It + // also covers a registration deleted mid-uninstall that left a detached wrapper + // looping: with nothing registered it binds no orphan, and an unbound survivor + // refuses instead of being killed by path alone. + const partA = await (deps.windowsPartA ?? runWindowsServiceStopPartA)({ + admission: windowsStopAdmission({ orphanDiscovery: true }), + }); + if (partA.outcome !== "stopped" && partA.outcome !== "absent") { + throw new Error(`The installed Windows service could not be proven stopped (${partA.outcome}); nothing was replaced. ${describeWindowsPartA(partA)}`); + } + // With no manager registered, Part B is skipped and the tracked stop can come back + // "none" after its discovery cap while an unresponsive proxy still holds the port. + // Resolve the candidates now: the cleanup below deletes the runtime record that is one + // of them. Nothing may be published over a candidate that is not definitively dead. + const absentProofEndpoints = partA.outcome === "absent" + ? (deps.installEndpoints ?? windowsRemovalEndpoints)() + : []; + const target = await (deps.resolveTarget ?? resolveTrackedProxyTarget)(); + if (target?.endpoint && partA.outcome === "absent" && !absentProofEndpoints.some(e => e.hostname === target.endpoint!.hostname && e.port === target.endpoint!.port)) absentProofEndpoints.push(target.endpoint); + let nonce: string | undefined; + if (target) { + // Before the tracked stop: a receipt-less POST would answer 409 and abort the install + // while the old proxy kept serving. + if (!target.endpoint) throw new Error("The target endpoint is unavailable, so deferred teardown cannot be recorded."); + nonce = claimInstallTeardownReceipt(deps, partA, target.endpoint).nonce; + } + // Part B owns the port wait after a stopped manager (030 "Time budget"). + await (deps.stopTrackedProxy ?? ((io?: TrackedProxyStopIo) => stopTrackedProxyIfRunning(io ?? { + skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, + })))({ + target, + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, + }); + if (partA.outcome === "stopped") { + const partB = await (deps.windowsPartB ?? confirmWindowsServiceStop)({ + port: target?.endpoint?.port ?? stopPort, + remainingMs: windowsStopRemainingMs(), + }); + if (partB.outcome !== "freed") { + throw new Error(`The service port is not free after the stop; nothing was published or started. ${describeWindowsPartB(partB)}`); + } + return { deferredTeardownNonce: nonce }; + } + if (partA.outcome === "absent") { + const proof = await endpointsAllDead(absentProofEndpoints, deps.endpointLiveness ?? probeEndpointLiveness); + if (!proof.dead) { + throw new Error(proof.live.length > 0 + ? `A proxy is still listening on port ${proof.live.join(", ")}; nothing was published or started. Check for a manually started proxy, then retry.` + : `The endpoint on port ${(proof.unknown.join(", ") || stopPort)} could not be confirmed down; nothing was published or started. Check for a manually started proxy, then retry.`); + } + } + return nonce ? { deferredTeardownNonce: nonce } : {}; + } const resolveOps = deps.managerOps ?? platformServiceInstallCleanupOps; const backends: ServiceBackend[] = []; const addBackend = (backend: ServiceBackend) => { if (!backends.includes(backend)) backends.push(backend); }; - - if (platform === "win32") { - // The recorded backend owns the old installation and must be stopped first. - // A conflicting diagnostic means both managers exist, so stop both even when - // the requested backend happens to match the recorded one. - if (diagnostic.backend === "scheduler" || diagnostic.backend === "native") { - addBackend(diagnostic.backend); - if (diagnostic.conflict) addBackend(diagnostic.backend === "scheduler" ? "native" : "scheduler"); - } - addBackend(requestedBackend); - } else { - addBackend(requestedBackend); - } + addBackend(requestedBackend); for (const backend of backends) { const manager = resolveOps(backend); if (!manager) throw new Error(`Background service manager is unavailable for ${backend}.`); if (manager.status() !== null) manager.stop(); } - if (platform === "win32") { - // `schtasks /end` ends the task instance but leaves the cmd `:loop` wrapper - // alive to respawn the proxy mid-install — the same survivor - // `ocx stop` kills via killWindowsServiceWrapperProcesses. Run it - // after every manager has been asked to stop and before the tracked-proxy - // kill so a child respawned in that gap is still caught below. It also runs - // when no manager stop happened: a registration deleted mid-uninstall can - // leave the detached wrapper looping. A manager stop that refused propagates - // above before this runs, so the respawner of a still-running service is - // never killed. - (deps.killServiceWrappers ?? killWindowsServiceWrapperProcesses)(); - } await (deps.stopTrackedProxy ?? stopTrackedProxyIfRunning)(); + // No manager deferral off Windows: the launchd/systemd stop is down when it reports so. + return {}; } export async function installServiceSafely( @@ -309,9 +459,16 @@ export async function installServiceSafely( const runtime = Object.freeze({ ...(deps.selectRuntime ?? durableBunRuntime)() }); const configDir = (deps.configDir ?? getConfigDir)(); (deps.assertRuntimeWritable ?? assertSelectedRuntimeWritable)(runtime, configDir, { platform: deps.platform, rootWasAbsent: !existsSync(configDir) }); - await prepareServiceInstall(requestedBackend, { ...deps, supervisionLatch: latch }); + const prepared = await prepareServiceInstall(requestedBackend, { ...deps, supervisionLatch: latch }); assertNoDesktopSupervision(deps.inspectSupervision, latch); await install(runtime); + // The new service is installed and started, so shared config stays pointed at OpenCodex + // and the deferral is discharged. An install failure above throws before this line and + // keeps the receipt for the next stop. + const nonce = prepared.deferredTeardownNonce; + if (nonce && !(deps.clearTeardown ?? clearPendingTeardown)(nonce)) { + console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); + } } export interface FreshWindowsSchedulerInstallDeps extends RuntimePreflightDeps { @@ -320,7 +477,7 @@ export interface FreshWindowsSchedulerInstallDeps extends RuntimePreflightDeps { stageRegistrationXml?: (attemptNonce: string) => string; register?: (xmlPath: string, attemptNonce: string) => Promise; recordOwnership?: () => boolean; - prepare?: () => Promise; + prepare?: () => Promise; removeNativeService?: () => void; publishAssets?: (runtime: DurableBunRuntime) => void; verifyBeforeRun?: (attemptNonce: string) => void | Promise; @@ -402,7 +559,7 @@ export async function installFreshWindowsSchedulerSafely( ); } (deps.assertRuntimeWritable ?? assertSelectedRuntimeWritable)(runtime, configDir, { platform: deps.platform, rootWasAbsent: false }); - await prepare(); + const prepared = await prepare(); assertNoDesktopSupervision(deps.inspectSupervision, latch); removeNativeService(); publishAssets(runtime); @@ -411,6 +568,13 @@ export async function installFreshWindowsSchedulerSafely( runTask(); started = true; writeState(runtime); + // A prepare that stopped an old manager claimed the deferral receipt; the replacement + // is running now, so shared config stays pointed at OpenCodex and the obligation is + // discharged. A failure above keeps it for the next stop. + const nonce = prepared && "deferredTeardownNonce" in prepared ? prepared.deferredTeardownNonce : undefined; + if (nonce && !clearPendingTeardown(nonce)) { + console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); + } } catch (error) { const detail = error instanceof Error ? error.message : String(error); if (registered && !started) { @@ -514,11 +678,12 @@ export function installedServiceRespawnRisk( * install (#3008). */ /** - * `stopped-respawnable` is Task Scheduler specifically: `schtasks /end` ends the task + * `stopped-respawnable` was Task Scheduler specifically: `schtasks /end` ends the task * instance while the `cmd :loop` wrapper survives and respawns its child seconds later - * (#764). Only that backend needs the restart-window wait — launchd, systemd and WinSW - * are down when they report stopped, and making them pay a seven-second poll would be a - * regression in every ordinary `ocx stop`. + * (#764). Since the checked Windows stop (#7771) no current backend returns it: Windows + * reports `stopped` only after Part A ended the manager and every bound wrapper, and its + * callers then run Part B unconditionally. The value stays for documents and receipts + * written by older CLIs, which callers still read. */ /** * `state-unknown` is kept apart from `failed` because the remedies differ. A manager that @@ -528,46 +693,41 @@ export function installedServiceRespawnRisk( */ export type ServiceStopOutcome = "absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown"; -export function stopServiceIfInstalledDetailed(): ServiceStopOutcome { +/** The outcome plus, on Windows, the Part A result behind it (reason, bound wrappers). */ +export interface ServiceStopReport { + outcome: ServiceStopOutcome; + windows: WindowsCheckedStopResult | null; +} + +/** + * Stop whatever manager is installed. On Windows this is Part A of the checked stop + * (#7771): both backends are probed, the registered managers stopped and the bound + * wrappers ended and waited for, with admission against the caller's deadline. The proxy + * child is left to the caller's graceful stop, and the caller runs Part B after it. + */ +export async function stopServiceIfInstalledReport( + request: WindowsPartARequest = {}, + platform: NodeJS.Platform = process.platform, +): Promise { assertServiceEnvironmentMatchesInstall(); - if (process.platform === "darwin") { + if (platform === "darwin") { if (existsSync(plistPath())) { - try { stopLaunchd(); return "stopped"; } catch { return "failed"; } - } - } else if (process.platform === "win32") { - // Query BOTH backends regardless of state: a failed switch or stale state can leave - // two managers installed, and either one would respawn the proxy after `ocx stop`. - let stopped = false; - let failed = false; - let schedulerStopped = false; - let stateUnknown = false; - // `probeWindowsSchedulerTask` is tri-state on purpose: a query that THROWS is not the - // same as a task that is absent, and treating it as absent lets a live scheduler - // survive a "successful" stop. - const probe = probeWindowsSchedulerTask(); - if (probe.status === "present") { - if (stopWindowsChecked()) { stopped = true; schedulerStopped = true; } - else failed = true; - } else if (probe.status === "unknown") { - // Not "failed": nothing refused to stop. The query itself could not answer, which is - // a different problem with a different fix. - stateUnknown = true; + try { stopLaunchd(); return { outcome: "stopped", windows: null }; } catch { return { outcome: "failed", windows: null }; } } - if (statusWinswRaw() !== "nonexistent") { - try { stopWinswService(); stopped = true; } catch { failed = true; } - } - // `schtasks /end` ends the task instance but the cmd `:loop` wrapper survives and - // respawns its child seconds later (issue #764), resurrecting the proxy during a - // stop or a tray restart. Kill the launcher/wrapper processes outright. - killWindowsServiceWrapperProcesses(); - // A failure on either backend wins: the other one stopping does not make the live one - // safe to update over. - const outcome = classifyWindowsServiceStop({ stopped, failed, schedulerStopped, stateUnknown }); - if (outcome !== "absent") return outcome; - } else if (process.platform === "linux" && isSystemd() && existsSync(unitPath())) { - try { stopSystemd(); return "stopped"; } catch { return "failed"; } + } else if (platform === "win32") { + // Both backends are probed regardless of state (a failed switch can leave two), an + // unreadable probe is state-unknown and never absence, and a wrapper that survives + // `schtasks /end` is ended through the binding rule instead of a path-only sweep. + const windows = await runWindowsServiceStopPartA(request); + return { outcome: serviceStopOutcomeFromWindows(windows), windows }; + } else if (platform === "linux" && isSystemd() && existsSync(unitPath())) { + try { stopSystemd(); return { outcome: "stopped", windows: null }; } catch { return { outcome: "failed", windows: null }; } } - return "absent"; + return { outcome: "absent", windows: null }; +} + +export async function stopServiceIfInstalledDetailed(request: WindowsPartARequest = {}): Promise { + return (await stopServiceIfInstalledReport(request)).outcome; } /** Delete install-state files; stale state would make `ocx update` "reinstall" a service that no longer exists. */ @@ -581,6 +741,13 @@ type UninstallServiceHooksForTests = { probeWindowsTask: () => WindowsSchedulerTaskProbe; uninstallWindowsTask: () => void; nativeStatus: () => WinswStatus; + /** + * Is the registered `opencodex-proxy-native` service THIS home's? `sc stop/delete` name a + * fixed ID, so a registration is not evidence of ownership — another installation's + * WinSW service under the same ID would be stopped and deleted here. Production reads + * `sc qc` BINARY_PATH_NAME and compares it with this home's executable. + */ + nativeOwnership?: () => "owned" | "foreign" | "unknown"; uninstallNative: () => void; removeInstallState: () => void; launchdPlistPath?: () => string; @@ -609,6 +776,26 @@ export function setUninstallServiceHooksForTests(hooks: UninstallServiceHooksFor */ export type ServiceUninstallOutcome = "absent" | "removed" | "failed"; +/** + * Does the registered WinSW service run this home's executable? + * + * The same proof the guarded manager and Part A use (`sc qc` `BINARY_PATH_NAME`), reduced to + * the three answers removal needs: this home's (`owned`), another's (`foreign`), or one that + * could not be read (`unknown`, which refuses). + */ +function winswOwnershipOfThisHome( + query: () => string | null = queryWinswBinaryPathName, +): "owned" | "foreign" | "unknown" { + let binaryPathName: string | null; + try { + binaryPathName = query(); + } catch { + return "unknown"; + } + if (binaryPathName === null) return "unknown"; + return scBinaryPathNamesExecutable(binaryPathName, winswExePath()) ? "owned" : "foreign"; +} + /** * Remove installed manager assets, including orphaned launchd recovery files. * Keep install state on removal failure; Windows refusal and unknown probes may throw. @@ -632,11 +819,30 @@ export function uninstallServiceDetailed(): ServiceUninstallOutcome { if (scheduler.status === "unknown") { throw new Error(`Could not determine Task Scheduler state: ${scheduler.detail}`); } + // Every probe runs before the first destructive step. The task used to be deleted first, + // so an unreadable `sc qc` left a half-removed install: no scheduler task to retry from, + // native registration still on disk, and no way back (#7771 round 9). + const nativeStatus = (hooks?.nativeStatus ?? statusWinswRaw)(); + if (nativeStatus === "unknown") { + throw new Error("Native service status could not be verified."); + } + let ownership: "owned" | "foreign" | "unknown" | null = null; + if (nativeStatus !== "nonexistent") { + // Re-prove ownership right before the destructive step: `sc qc` may have changed + // since the install-state read, and a foreign registration must survive an uninstall + // of this home. Unreadable is not a licence to delete — fail closed and keep state. + ownership = (hooks?.nativeOwnership ?? winswOwnershipOfThisHome)(); + if (ownership === "unknown") { + throw new Error( + `The native service registration's ownership could not be verified (sc qc ${WINSW_SERVICE_ID}); refusing to remove it. Check 'sc query ${WINSW_SERVICE_ID}' and retry.`, + ); + } + } if (scheduler.status === "present") { (hooks?.uninstallWindowsTask ?? uninstallWindows)(); removed = true; } - if ((hooks?.nativeStatus ?? statusWinswRaw)() !== "nonexistent") { + if (ownership === "owned") { (hooks?.uninstallNative ?? uninstallWinswService)(); removed = true; } diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts new file mode 100644 index 00000000000..1580c109834 --- /dev/null +++ b/src/service/windows-checked-stop-live.ts @@ -0,0 +1,467 @@ +/** + * Live dependencies of the checked Windows service stop (#7771, 060 "PR2: caller wiring"). + * + * Part A (src/service/windows-checked-stop.ts) and Part B (src/service/windows-stop-confirm.ts) + * take every effect injected. This module is the production side of those seams: capped + * asynchronous schtasks and sc.exe calls, the evidence snapshot with native creation times + * (src/service/windows-process-evidence.ts), the checked terminate, the bind-time exit watch, + * the listening-port sampler and the listener identity check. + * + * The bind snapshot MUST be windowsProcessSnapshot: on x64 it overlays the native creation + * time, which is the exact value the handle terminate compares. A CIM-only snapshot carries + * a millisecond-truncated CIM time, the terminate then answers "reused" and Part A treats a + * live wrapper as exited. + * + * Windows ARM64 ships the STRICT variant (050 "Rollout", step 3): no ARM64 device run has + * happened, so a terminate there is "unavailable" and Part A reports unknown with + * evidence-unreadable instead of killing through the bounded PowerShell fallback. The + * fallback stays in the evidence module, unreachable until the constant below flips after + * the 040 ARM64 device case is run. + */ +import { execFile } from "node:child_process"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; + +import { inspectOcxStartProcessIdentity, readPid, verifyPidIdentity, type OcxStartIdentityObservation } from "../config/process-state"; +import { isProcessAlive } from "../lib/process-control"; +import { isTestHomeGuardArmed } from "../lib/test-home-guard"; +import { resolveTrustedWindowsPowerShellExe, resolveTrustedWindowsSchtasksExe, resolveTrustedWindowsSystemDirectory, toWindowsSchtasksError } from "../lib/windows-elevation"; +import { WINSW_SERVICE_ID, winswExePath, type WinswStatus } from "../lib/winsw"; +import { decodeWindowsTextBytes } from "../lib/windows-text"; +import { parseScQcBinaryPathName, scBinaryPathNamesExecutable } from "../service-manager-probe"; +import { parseListenPidsFromNetstat, type ListenPidScan } from "../server/port-reclaim"; +import { dropWindowsTcpRowsForLocalPort } from "../server/windows-tcp-drop"; +import { TASK, windowsLauncherVbsPath, windowsServiceScriptPath } from "./state"; +import { ancestorWrapperPids } from "./windows-guarded-manager"; +import type { WindowsCheckedStopDeps, WindowsTaskEndResult, WindowsWinswScmState, WindowsWinswStopResult } from "./windows-checked-stop"; +import { + openWindowsProcessWatch, + readScheduleServicePid, + terminateWindowsProcessBound, + windowsProcessSnapshot, + type WindowsStopProcessEntry, +} from "./windows-process-evidence"; +import { windowsProcessNative, type WindowsProcessNativeApi, type WindowsTerminateResult } from "./windows-process-native"; +import type { WindowsListenerIdentity, WindowsPortSample, WindowsStopConfirmDeps } from "./windows-stop-confirm"; +import { windowsSchedulerCsvIncludesTask } from "./windows-scheduler"; + +/** + * 050: without an ARM64 device run, ARM64 ships strict. Flip only after the 040 ARM64 device + * case passes; the bounded PowerShell terminate in windows-process-evidence.ts then runs. + */ +export const WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED = false; + +export interface WindowsStopCommandResult { + status: number | null; + stdout: string; + stderr: string; + timedOut: boolean; +} + +/** One spawned helper; tests inject it, production runs execFile with a hard timeout. */ +export type WindowsStopCommandRunner = ( + program: "schtasks" | "sc" | "powershell", + args: string[], + timeoutMs: number, +) => Promise; + +function programPath(program: "schtasks" | "sc" | "powershell"): string { + if (program === "schtasks") return resolveTrustedWindowsSchtasksExe(); + if (program === "powershell") return resolveTrustedWindowsPowerShellExe(); + const file = join(resolveTrustedWindowsSystemDirectory(), "sc.exe"); + if (!existsSync(file)) throw new Error("sc.exe was not found in the trusted system directory."); + return file; +} + +export const liveWindowsStopCommandRunner: WindowsStopCommandRunner = (program, args, timeoutMs) => new Promise(resolve => { + // execFile reads a 0 timeout as "no timeout": a deadline-derived fraction is floored, and a + // timeout that floors to 0 spawns nothing (the same rule as terminateWindowsProcessBound). + const timeout = Math.floor(timeoutMs); + if (!(timeout > 0)) { + resolve({ status: null, stdout: "", stderr: "", timedOut: true }); + return; + } + // Task Scheduler and the SCM are machine-global: an armed test process may only observe. + // `sc query` and `sc qc` only read the SCM row and its registration. + const mutating = program === "schtasks" + ? args[0]?.toLowerCase() !== "/query" + : program === "sc" && !["query", "qc"].includes((args[0] ?? "").toLowerCase()); + if (isTestHomeGuardArmed() && mutating) { + resolve({ status: null, stdout: "", stderr: "refusing a machine-global service mutation from an armed test process", timedOut: false }); + return; + } + let file: string; + try { + file = programPath(program); + } catch (error) { + resolve({ status: null, stdout: "", stderr: error instanceof Error ? error.message : String(error), timedOut: false }); + return; + } + execFile(file, args, { encoding: "buffer", timeout, windowsHide: true, maxBuffer: 1024 * 1024 }, + (error, stdout, stderr) => { + const text = (value: Buffer | string | undefined): string => + value === undefined ? "" : typeof value === "string" ? value : decodeWindowsTextBytes(value); + const failure = error as (NodeJS.ErrnoException & { killed?: boolean; signal?: string | null; code?: number | string }) | null; + resolve({ + status: failure ? (typeof failure.code === "number" ? failure.code : null) : 0, + stdout: text(stdout), + stderr: text(stderr), + timedOut: !!failure?.killed && failure.signal !== null, + }); + }); +}); + +/** schtasks /end, mapped: already stopped is not-running; access denied is denied. */ +export function classifyTaskEnd(result: WindowsStopCommandResult): WindowsTaskEndResult { + if (result.timedOut) return "failed"; + if (result.status === 0) return "ended"; + const error = Object.assign(new Error(result.stderr || result.stdout), { status: result.status, stderr: result.stderr, stdout: result.stdout }); + const detail = (result.stderr + "\n" + result.stdout).toLowerCase(); + if (detail.includes("no running instance") || detail.includes("not currently running") || detail.includes("0x41330")) return "not-running"; + return toWindowsSchtasksError(error, ["/end", "/tn", TASK]).reason === "access-denied" ? "denied" : "failed"; +} + +/** schtasks /query /fo CSV /nh status column. Localized values fall back to PowerShell. */ +export function parseTaskCsvState(csv: string): "running" | "not-running" | "unknown" { + for (const line of csv.split(/\r?\n/)) { + if (!windowsSchedulerCsvIncludesTask(line, TASK)) continue; + const fields = [...line.matchAll(/"([^"]*)"/g)].map(match => match[1]!.trim()); + const status = fields[fields.length - 1] ?? ""; + if (/^running$/i.test(status)) return "running"; + if (/^(ready|disabled|queued)$/i.test(status)) return "not-running"; + return "unknown"; + } + return "unknown"; +} + +/** sc.exe query: the numeric STATE and the 1060 "does not exist" code are not localized. */ +export function parseScQueryState(result: WindowsStopCommandResult): WindowsWinswScmState { + const text = result.stdout + "\n" + result.stderr; + if (/\b1060\b/.test(text)) return "nonexistent"; + if (result.timedOut || result.status !== 0) return "unknown"; + const state = /^\s*STATE\s*:\s*(\d+)/m.exec(text); + if (!state) return "unknown"; + const value = Number(state[1]); + if (value === 1) return "stopped"; + if (value === 4) return "running"; + if (value >= 2 && value <= 7) return "pending"; + return "unknown"; +} + +/** sc.exe stop without waiting: 1062 (not started) is fine, 5 is denied. */ +export function classifyScStop(result: WindowsStopCommandResult): WindowsWinswStopResult { + if (result.timedOut) return "failed"; + const text = result.stdout + "\n" + result.stderr; + if (result.status === 0 || /\b1062\b/.test(text) || /\b1060\b/.test(text)) return "sent"; + if (/\bFAILED 5\b/.test(text) || result.status === 5) return "denied"; + return "failed"; +} + +export function winswStatusFromScState(state: WindowsWinswScmState): WinswStatus { + if (state === "nonexistent") return "nonexistent"; + if (state === "stopped") return "stopped"; + // A pending service is still an active manager: Part A stops it and polls. + if (state === "running" || state === "pending") return "started"; + return "unknown"; +} + +export function windowsStopCanonicalPaths(): string[] { + return [windowsLauncherVbsPath(), windowsServiceScriptPath()]; +} + +/** + * The terminate Part A uses. terminateWindowsProcessBound runs the PowerShell fallback by + * itself whenever the native API is missing (ARM64, or an x64 whose DLL load failed), so the + * strict variant is enforced HERE (060 "ARM64 decision for PR2"): without the native + * primitive the answer is "unavailable" and the fallback is never reached. + */ +export async function windowsStopTerminate( + pid: number, + startTime: number, + timeoutMs: number, + io: { native?: () => WindowsProcessNativeApi | null; terminate?: typeof terminateWindowsProcessBound } = {}, +): Promise { + if ((io.native ?? windowsProcessNative)() === null && !WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED) return { status: "unavailable" }; + return (io.terminate ?? terminateWindowsProcessBound)(pid, startTime, { timeoutMs }); +} + +/** Floor a deadline-derived timeout; null when it floors to 0 (nothing may be spawned). */ +export function wholeTimeoutMs(timeoutMs: number): number | null { + const floored = Math.floor(timeoutMs); + return floored > 0 ? floored : null; +} + +/** The evidence snapshot with a floored timeout; 0 ms reads nothing (null, unreadable). */ +export function windowsStopSnapshot(timeoutMs: number): Promise { + const whole = wholeTimeoutMs(timeoutMs); + return whole === null ? Promise.resolve(null) : windowsProcessSnapshot({ timeoutMs: whole }); +} + +/** + * Does the stopping process descend from a wrapper carrying this home's launcher or + * service-script path? The binding rule excludes the stopper itself but not its ancestors, + * so such a stop would end its own supervision chain (structure/ops/windows-checked-stop.md, + * "Who may run Part A"). + */ +export function stopperUnderHomeWrapper( + processes: readonly WindowsStopProcessEntry[], + canonicalPaths: readonly string[], + selfPid: number = process.pid, +): boolean { + return ancestorWrapperPids(selfPid, processes, canonicalPaths).length > 0; +} + +export interface WindowsCheckedStopLiveOptions { + run?: WindowsStopCommandRunner; + /** + * The `sc.exe qc` query that proves the registered WinSW service runs THIS installation's + * executable. Defaults to the capped runner; tests inject it. + */ + scQcRun?: (timeoutMs: number) => Promise; + /** Checked before every manager command (the route's cap, the updater's authorizer). */ + gate?: () => boolean; + snapshot?: (timeoutMs: number) => Promise; +} + +/** + * Is the registered WinSW service this installation's? + * + * `sc stop opencodex-proxy-native` names a fixed ID, so a successful `sc query` is evidence + * that SOMETHING is registered, not that it is ours: another installation's WinSW service + * under the same ID would be stopped by this home's Part A. The registered + * `BINARY_PATH_NAME` must name this home's WinSW executable, the same proof the guarded + * manager path uses (`windowsWinswServiceInfo` + `commandLineHasPathToken`). Unreadable is + * unknown; a foreign executable is `nonexistent`, which keeps Part A from stopping it. + */ +export function classifyWinswOwnership( + qc: WindowsStopCommandResult | null, + winswExe: string = winswExePath(), +): WinswStatus | "foreign" { + if (qc === null || qc.timedOut || qc.status !== 0) return "unknown"; + const binaryPathName = parseScQcBinaryPathName(qc.stdout); + if (binaryPathName === null) return "unknown"; + return scBinaryPathNamesExecutable(binaryPathName, winswExe) ? "started" : "foreign"; +} + +/** The identity-checked proxy child, or null. */ +export function windowsStopChildPid(): number | null { + try { + const pid = readPid(); + return pid !== null && verifyPidIdentity(pid) === pid ? pid : null; + } catch { + return null; + } +} + +export function windowsCheckedStopLiveDeps(options: WindowsCheckedStopLiveOptions = {}): WindowsCheckedStopDeps { + const run = options.run ?? liveWindowsStopCommandRunner; + const gate = options.gate ?? (() => true); + const scQuery = async (timeoutMs: number) => parseScQueryState(await run("sc", ["query", WINSW_SERVICE_ID], timeoutMs)); + const scQcRun = options.scQcRun ?? (async (timeoutMs: number) => run("sc", ["qc", WINSW_SERVICE_ID], timeoutMs)); + return { + canonicalPaths: windowsStopCanonicalPaths(), + childPid: windowsStopChildPid(), + async probeScheduler(timeoutMs) { + const started = Date.now(); + const direct = await run("schtasks", ["/query", "/tn", TASK], timeoutMs); + if (direct.status === 0 && direct.stdout.includes(TASK)) return { status: "present" }; + const listing = await run("schtasks", ["/query", "/fo", "CSV"], Math.max(1, timeoutMs - (Date.now() - started))); + if (listing.status !== 0 || listing.timedOut) return { status: "unknown", detail: "the Task Scheduler query did not answer" }; + return windowsSchedulerCsvIncludesTask(listing.stdout, TASK) ? { status: "present" } : { status: "absent" }; + }, + async probeWinsw(timeoutMs) { + // One cap covers both reads: the ownership query gets only what the state query left. + const started = Date.now(); + const state = await scQuery(timeoutMs); + const status = winswStatusFromScState(state); + // Absence, and anything already unreadable, needs no ownership read. + if (status !== "started") return status; + const remaining = Math.floor(timeoutMs - (Date.now() - started)); + if (remaining < 1) return "unknown"; + const qc = await scQcRun(remaining); + const ownership = classifyWinswOwnership(qc); + // Foreign is `nonexistent` for Part A: a manager this home must not stop. + if (ownership === "foreign") return "nonexistent"; + return ownership === "unknown" ? "unknown" : status; + }, + enumerate: timeoutMs => (options.snapshot ?? windowsStopSnapshot)(timeoutMs), + scheduleServicePid: timeoutMs => { + const whole = wholeTimeoutMs(timeoutMs); + return whole === null ? null : readScheduleServicePid({ timeoutMs: whole }); + }, + async endTask(timeoutMs) { + if (!gate()) return "failed"; + return classifyTaskEnd(await run("schtasks", ["/end", "/tn", TASK], timeoutMs)); + }, + async taskState(timeoutMs) { + const started = Date.now(); + const csv = await run("schtasks", ["/query", "/tn", TASK, "/fo", "CSV", "/nh"], timeoutMs); + const parsed = csv.status === 0 ? parseTaskCsvState(csv.stdout) : "unknown"; + if (parsed !== "unknown") return parsed; + // A localized status word: ask Get-ScheduledTask, whose State is an enum name. + const left = timeoutMs - (Date.now() - started); + if (left <= 0) return "unknown"; + const ps = await run("powershell", ["-NoProfile", "-NoLogo", "-NonInteractive", "-Command", + "(Get-ScheduledTask -TaskName '" + TASK + "' -TaskPath '\\' -ErrorAction Stop).State"], left); + const state = ps.stdout.trim(); + if (/^running$/i.test(state)) return "running"; + if (/^(ready|disabled|queued)$/i.test(state)) return "not-running"; + return "unknown"; + }, + async winswStop(timeoutMs) { + if (!gate()) return "failed"; + return classifyScStop(await run("sc", ["stop", WINSW_SERVICE_ID], timeoutMs)); + }, + winswState: scQuery, + proxyAlive: () => { + const pid = windowsStopChildPid(); + return pid === null ? false : isProcessAlive(pid); + }, + openWatch: (pid, startTime) => openWindowsProcessWatch(pid, startTime), + terminate: (pid, startTime, timeoutMs) => windowsStopTerminate(pid, startTime, timeoutMs), + }; +} + +/** One listener scan, bounded by the caller's timeout; a scan that misses it is unreadable. */ +export type WindowsStopPortScan = (port: number, timeoutMs: number) => ListenPidScan | Promise; + +const WINDOWS_STOP_SCAN_MAX_BUFFER = 4 * 1024 * 1024; + +/** + * The production listener scan: netstat -ano through execFile with the caller's timeout. + * + * `netstat -ano` is asked directly rather than through `cmd /c chcp 437`: the locale-safe + * wildcard-foreign rule in parseListenEntriesFromNetstat already recognizes a listen row, and + * running the program itself means the timeout terminates the only helper spawned. The table + * is read from the trusted System32 path — SystemRoot is caller-controlled. + */ +export function scanWindowsStopListenPids(port: number, timeoutMs: number): Promise { + const whole = wholeTimeoutMs(timeoutMs); + if (whole === null) return Promise.resolve({ ok: false, error: "no positive timeout to sample with" }); + let netstat: string; + try { + netstat = join(resolveTrustedWindowsSystemDirectory(), "netstat.exe"); + } catch (error) { + return Promise.resolve({ ok: false, error: error instanceof Error ? error.message : String(error) }); + } + return new Promise(resolve => { + // execFile's timeout terminates the helper; the callback then answers unreadable. + execFile(netstat, ["-ano", "-p", "tcp"], { + encoding: "buffer", + timeout: whole, + killSignal: "SIGKILL", + windowsHide: true, + maxBuffer: WINDOWS_STOP_SCAN_MAX_BUFFER, + }, (error, stdout) => { + if (error) { + resolve({ ok: false, error: error.message }); + return; + } + const text = typeof stdout === "string" ? stdout : decodeWindowsTextBytes(stdout); + resolve({ ok: true, pids: parseListenPidsFromNetstat(text, port) }); + }); + }); +} + +/** + * One port sample, capped by `timeoutMs`. A listener whose owner has exited is cleared like + * reclaimListenPort does. + * + * A scan that does not answer within the budget is "unknown": Part B's closing sample is + * bounded (030 "Time budget"), and a synchronous scan both ignored that cap and could stall + * the stop for the helper's own 5 s. The timeout is floored and a value below 1 ms samples + * nothing, because execFile reads 0 as unlimited — the same rule as the command runner. + */ +export async function sampleWindowsStopPort( + port: number, + timeoutMs: number, + io: { + scan?: WindowsStopPortScan; + alive?: (pid: number) => boolean; + /** Clears the ghost rows of the exited owners only; never another program's rows. */ + drop?: (port: number, ownerPids: readonly number[]) => unknown; + selfPid?: number; + } = {}, +): Promise { + const whole = wholeTimeoutMs(timeoutMs); + if (whole === null) return { state: "unknown" }; + const scan = await runBoundedScan(port, whole, io.scan ?? scanWindowsStopListenPids); + if (scan === null || !scan.ok) return { state: "unknown" }; + const pids = scan.pids.filter(pid => pid !== (io.selfPid ?? process.pid)); + if (pids.length === 0) return { state: "free" }; + const alive = io.alive ?? isProcessAlive; + const live = pids.filter(pid => alive(pid)); + if (live.length === 0) { + // Clear only the ghost rows the exited PID(s) left: a program that took the port after + // the scan owns its rows, and resetting them would break a connection this stop does not own. + try { + (io.drop ?? ((target: number, owners: readonly number[]) => dropWindowsTcpRowsForLocalPort(target, undefined, owners)))(port, pids); + } catch { /* the next sample decides */ } + } + return { state: "listening", ownerPid: live[0] ?? pids[0] ?? null }; +} + +/** Resolve null once the budget is spent; a scan that never answers cannot hold the stop. */ +function runBoundedScan(port: number, timeoutMs: number, scan: WindowsStopPortScan): Promise { + return new Promise(resolve => { + let settled = false; + const finish = (result: ListenPidScan | null): void => { + if (settled) return; + settled = true; + clearTimeout(timer); + resolve(result); + }; + const timer = setTimeout(() => finish(null), timeoutMs); + timer.unref?.(); + let pending: ListenPidScan | Promise; + try { + pending = scan(port, timeoutMs); + } catch { + finish(null); + return; + } + Promise.resolve(pending).then(finish, () => finish(null)); + }); +} + +/** + * ours = an opencodex process (the respawned proxy); exited = its owner is gone. + * + * Tri-state (#7771): only a positive "mismatch" is foreign. `verifyPidIdentity` collapses + * "not an ocx process" and "the probe could not answer" into the same null, and a probe that + * cannot read another account's or an elevated process's command line is not evidence that + * another program holds the port — Part B would then report portHeldByForeign and let the + * caller restore shared config with the proxy possibly still alive. Unreadable stays unknown. + */ +export function identifyWindowsStopListener( + ownerPid: number | null, + io: { + alive?: (pid: number) => boolean; + identity?: (pid: number) => OcxStartIdentityObservation; + } = {}, +): WindowsListenerIdentity { + if (ownerPid === null) return "unknown"; + if (!(io.alive ?? isProcessAlive)(ownerPid)) return "exited"; + try { + const observed = (io.identity ?? inspectOcxStartProcessIdentity)(ownerPid); + if (observed === "match") return "ours"; + return observed === "mismatch" ? "foreign" : "unknown"; + } catch { + return "unknown"; + } +} + +export function windowsStopConfirmLiveDeps( + port: number, + remainingMs: number, + options: { snapshot?: (timeoutMs: number) => Promise } = {}, +): WindowsStopConfirmDeps { + return { + port, + remainingMs, + canonicalPaths: windowsStopCanonicalPaths(), + samplePort: (samplePort, timeoutMs) => sampleWindowsStopPort(samplePort, timeoutMs), + identifyListener: (_port, ownerPid) => identifyWindowsStopListener(ownerPid), + enumerate: timeoutMs => (options.snapshot ?? windowsStopSnapshot)(timeoutMs), + }; +} diff --git a/src/service/windows-checked-stop.ts b/src/service/windows-checked-stop.ts index f81eb759eec..950c5f57d54 100644 --- a/src/service/windows-checked-stop.ts +++ b/src/service/windows-checked-stop.ts @@ -98,7 +98,7 @@ export interface WindowsCheckedStopOptions { /** A snapshot that already ran (guarded: revalidateManager's), reused for the bind. */ bindSnapshot?: readonly WindowsStopProcessEntry[]; admission?: WindowsStopAdmission; - /** Throws to refuse. Runs at entry and before each terminate. */ + /** Throws to refuse. Runs at entry, before each manager stop (/end, the WinSW stop) and before each terminate. */ authorize?: () => MaybePromise; } @@ -257,7 +257,7 @@ export async function stopWindowsServiceChecked( } // Step 2: bind. - if (taskRegistered) { + if (taskRegistered || winswRegistered) { let processes = options.bindSnapshot ?? null; if (processes === null) { const snapshot = await runCapped(clock, C.enumeration, deps.enumerate); @@ -267,7 +267,7 @@ export async function stopWindowsServiceChecked( processes = snapshot.value; } binding = bindWindowsStopWrappers({ - ...bindContext, processes, taskRegistered: true, managerRegistered: true, + ...bindContext, processes, taskRegistered, managerRegistered: true, scheduleServicePid: schedulePid, }); } @@ -298,6 +298,8 @@ export async function stopWindowsServiceChecked( // Step 3: ask the manager. if (taskRegistered) { + // The updater re-checks its authority right before the manager stop, as before each terminate. + await authorize(); result.changed = true; const ended = await runCapped(clock, C.taskEnd, deps.endTask); if (!ended.ok) return finish("unknown", "evidence-unreadable", "the task end command did not answer within its cap"); @@ -305,6 +307,7 @@ export async function stopWindowsServiceChecked( if (ended.value === "failed") return finish("unknown", "evidence-unreadable", "the task end command failed"); } if (winswStarted) { + await authorize(); result.changed = true; const stop = await runCapped(clock, C.winswStopCommand, deps.winswStop); if (!stop.ok) return finish("unknown", "evidence-unreadable", "the native service stop did not answer within its cap"); @@ -422,7 +425,7 @@ async function unboundOnlyVerdict( return unboundStillAlive(binding, watches); } -/** The updater's variant: the authorizer runs at entry and before every terminate. */ +/** The updater's variant: the authorizer runs at entry, before each manager stop and before every terminate. */ export function guardedStopWindowsServiceChecked( authorize: () => MaybePromise, deps: WindowsCheckedStopDeps, diff --git a/src/service/windows-command-line-token.ts b/src/service/windows-command-line-token.ts new file mode 100644 index 00000000000..54f51b8073c --- /dev/null +++ b/src/service/windows-command-line-token.ts @@ -0,0 +1,23 @@ +// Leaf module: whole-token path matching on a process command line, shared by the guarded +// manager inspection, the checked-stop binding rule and Part B without an import cycle. + +/** + * True when path appears in commandLine as a COMPLETE token — the same boundary + * rule the windows-service-wrappers kill list uses, ported so the read-only proof + * cannot accept a substring hit. A quoted path inside an argument string counts; + * a suffix or prefix glued to it does not. + */ +export function commandLineHasPathToken(commandLine: string | null | undefined, path: string): boolean { + if (!commandLine || !path) return false; + const haystack = commandLine.toLowerCase(); + const needle = path.toLowerCase(); + const boundary = (ch: string | undefined): boolean => ch === undefined || /[\s"']/.test(ch); + let index = 0; + for (;;) { + index = haystack.indexOf(needle, index); + if (index < 0) return false; + const end = index + needle.length; + if (boundary(haystack[index - 1]) && boundary(haystack[end])) return true; + index = end; + } +} diff --git a/src/service/windows-guarded-manager.ts b/src/service/windows-guarded-manager.ts index f74c056e161..91f60447f7b 100644 --- a/src/service/windows-guarded-manager.ts +++ b/src/service/windows-guarded-manager.ts @@ -29,9 +29,15 @@ import { cachedCurrentWindowsIdentity, resolveCurrentWindowsPrincipal, WINDOWS_P import { statusWinswRaw, winswExePath, WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; import type { GuardedManagerStopped, GuardedManagerTarget } from "./guarded-manager-target"; import { TASK, windowsLauncherVbsPath, windowsServiceScriptPath } from "./state"; +import { bindWindowsStopWrappers } from "./windows-stop-binding"; import { probeWindowsSchedulerTask, querySchtasks, type WindowsSchedulerTaskProbe } from "./windows-scheduler"; import { windowsWscript } from "./windows-wscript"; import { windowsTaskRegistrationHealthy, windowsTaskRegistrationRefreshableLegacy } from "./windows-taskxml"; +import { commandLineHasPathToken } from "./windows-command-line-token"; + +// Re-exported for existing importers; the function lives in a leaf module to keep the binding rule +// (windows-stop-binding.ts) free of an import cycle with this inspection. +export { commandLineHasPathToken }; /** Runtime table of one Win32 process; a null parent means the chain is unreadable. */ export interface WindowsProcessEntry { @@ -57,7 +63,14 @@ export interface WindowsWinswServiceInfo { * default implementation exits early off win32, mirroring probeWindowsSchedulerTask. */ export interface WindowsGuardedManagerDeps { + /** + * The table may carry creation times (windowsProcessSnapshot, #7771): only then can a + * surviving wrapper of a not-running task be bound (bound-stray). A table without them + * leaves every such wrapper unreadable, which stays unknown. + */ winProcs?: () => WindowsProcessEntry[] | null; + /** The Task Scheduler service host PID from the SCM, for the binding rule's lineage. */ + winSchedulePid?: () => number | null; winTaskXml?: () => string; winTaskState?: () => WindowsTaskState; /** @@ -216,26 +229,6 @@ export function currentWindowsTaskUserIds(): readonly string[] | null { } } -/** - * True when path appears in commandLine as a COMPLETE token — the same boundary - * rule the windows-service-wrappers kill list uses, ported so the read-only proof - * cannot accept a substring hit. A quoted path inside an argument string counts; - * a suffix or prefix glued to it does not. - */ -export function commandLineHasPathToken(commandLine: string | null | undefined, path: string): boolean { - if (!commandLine || !path) return false; - const haystack = commandLine.toLowerCase(); - const needle = path.toLowerCase(); - const boundary = (ch: string | undefined): boolean => ch === undefined || /[\s"']/.test(ch); - let index = 0; - for (;;) { - index = haystack.indexOf(needle, index); - if (index < 0) return false; - const end = index + needle.length; - if (boundary(haystack[index - 1]) && boundary(haystack[end])) return true; - index = end; - } -} function resolveWindowsDeps(deps: WindowsGuardedManagerDeps, live: boolean): ResolvedWindowsDeps { // "live" is false whenever the caller simulated platform win32 (tests or a future @@ -257,6 +250,7 @@ function resolveWindowsDeps(deps: WindowsGuardedManagerDeps, live: boolean): Res winScriptPath: deps.winScriptPath ?? windowsServiceScriptPath, winLauncherPath: deps.winLauncherPath ?? windowsLauncherVbsPath, winWinswExePath: deps.winWinswExePath ?? winswExePath, + winSchedulePid: deps.winSchedulePid ?? (() => null), }; } @@ -380,8 +374,42 @@ export function unreadableWrappersOfManager(managerPid: number, processes: reado }).map(entry => entry.pid); } -function unknown(reason: string): GuardedManagerTarget { - return { kind: "unknown", reason }; +function unknown(reason: string, stopReason?: "unbound-wrapper" | "evidence-unreadable"): GuardedManagerTarget { + return stopReason ? { kind: "unknown", reason, stopReason } : { kind: "unknown", reason }; +} + +/** + * A registered task that is not running, with wrappers of this home still alive (#7771, + * 030 "Guarded (consent) path"). The checked stop's binding rule decides: when every + * surviving wrapper is bound (wrapper image, this home's path token, lineage, a creation + * time) the state is bound-stray and reaches Part A; anything it cannot bind stays + * unknown, naming unbound-wrapper or evidence-unreadable. + */ +function bindStrayWrappers( + approvedPid: number, + processes: readonly WindowsProcessEntry[], + strays: readonly number[], + io: ResolvedWindowsDeps, +): GuardedManagerTarget { + const binding = bindWindowsStopWrappers({ + processes: processes.map(entry => ({ ...entry, startTime: (entry as { startTime?: number | null }).startTime ?? null })), + canonicalPaths: wrapperPaths(io), + childPid: approvedPid, + taskRegistered: true, + managerRegistered: true, + scheduleServicePid: io.winSchedulePid(), + }); + if (binding.unreadable.length > 0) { + return unknown("a surviving scheduler wrapper has unreadable evidence", "evidence-unreadable"); + } + const bound = new Set(binding.bound.map(wrapper => wrapper.pid)); + if (binding.unbound.length > 0 || strays.some(pid => !bound.has(pid))) { + return unknown("a surviving scheduler wrapper could not be tied to the registered task", "unbound-wrapper"); + } + return { + kind: "bound-stray", pid: approvedPid, backend: "scheduler", childNeedsSeparateStop: true, + wrappers: binding.bound.map(wrapper => ({ pid: wrapper.pid, startTime: wrapper.startTime })), + }; } function inspectWindowsSchedulerManager( @@ -394,16 +422,14 @@ function inspectWindowsSchedulerManager( const ancestors = ancestorWrapperPids(approvedPid, processes, paths); const strays = wrapperProcessesAlive(processes, paths); if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { - return unknown("a possible scheduler wrapper has an unreadable command line"); + return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } - // A registration that cannot run anything owes only the stray-wrapper check. - // That covers BOTH a not-running task and a task proven deleted between the - // caller's presence probe and this inspection: in either case nothing - // registered can respawn the proxy, while a surviving wrapper still can — - // schtasks /end would report success on the inert task and leave it alive. + // A registration that cannot run anything owes only the stray-wrapper check: nothing + // registered can respawn the proxy, while a surviving wrapper still can — schtasks /end + // would report success on the inert task and leave it alive. const inertVerdict = (): GuardedManagerTarget => strays.length === 0 ? { kind: "absent" } - : unknown("a surviving scheduler wrapper could not be tied to the registered task"); + : unknown("a surviving scheduler wrapper could not be tied to the registered task", "unbound-wrapper"); const state = io.winTaskState(); if (state === "unknown") { // Missing vs unreadable: the state query also fails for a task deleted @@ -413,7 +439,24 @@ function inspectWindowsSchedulerManager( ? inertVerdict() : unknown("the registered task's running state could not be proven"); } - if (state === "not-running") return inertVerdict(); + if (state === "not-running") { + // Only a registration that is provably this installation's definition may promote its + // survivors to bound-stray (#7771). The state word alone says the task is inert; it does + // not say the task is ours — a same-named foreign task would otherwise make this home's + // hand-started wrappers terminable. Reading the definition costs nothing when nothing + // survived, so a clean inert task keeps the absent verdict without a read. + if (strays.length === 0) return { kind: "absent" }; + let xml: string; + try { + xml = io.winTaskXml(); + } catch { + return unknown("the Task Scheduler registration of the surviving wrappers could not be read", "evidence-unreadable"); + } + if (!io.winRegistrationOurs(xml, io.winTaskUserIds())) { + return unknown("the registered task is not a recognized OpenCodex definition"); + } + return bindStrayWrappers(approvedPid, processes, strays, io); + } // Only the running path needs the registered definition: an unreadable // registration cannot prove the task is ours, but it must not hold an // otherwise-provable inert verdict unknown. @@ -502,21 +545,21 @@ export function inspectWindowsGuardedManager( const processes = io.winProcs(); if (processes === null) return unknown("the Windows process list could not be read"); if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { - return unknown("a possible scheduler wrapper has an unreadable command line"); + return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } return wrapperProcessesAlive(processes, wrapperPaths(io)).length === 0 ? { kind: "absent" } - : unknown("a surviving scheduler wrapper could not be tied to an installed manager"); + : unknown("a surviving scheduler wrapper could not be tied to an installed manager", "unbound-wrapper"); } // scheduler absent + winsw nonexistent — still have to rule out a detached wrapper. const processes = io.winProcs(); if (processes === null) return unknown("the Windows process list could not be read"); if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { - return unknown("a possible scheduler wrapper has an unreadable command line"); + return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } return wrapperProcessesAlive(processes, wrapperPaths(io)).length === 0 ? { kind: "absent" } - : unknown("a surviving scheduler wrapper could not be tied to an installed manager"); + : unknown("a surviving scheduler wrapper could not be tied to an installed manager", "unbound-wrapper"); } export interface WindowsGuardedStopDeps { diff --git a/src/service/windows-ops.ts b/src/service/windows-ops.ts index 440b267726b..2a92405b39e 100644 --- a/src/service/windows-ops.ts +++ b/src/service/windows-ops.ts @@ -16,10 +16,8 @@ import { OCX_ELEVATED_STAGING_UNREADABLE, runWindowsElevatedScheduledTaskRegistr import { defaultWinswEntry, installWinswService, statusWinswRaw, uninstallWinswService, WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; import { forgetEphemeralSecretDir, forgetEphemeralSecretPath, hardenElevatedStageDir, hardenElevatedStagePath, hardenSecretDir } from "../lib/windows-secret-acl"; import { recordOwnedConfigPath } from "../lib/config-ownership"; -import { killWindowsSchedulerWrappers } from "../lib/windows-service-wrappers"; import { isTestHomeGuardArmed } from "../lib/test-home-guard"; import { writeServiceApiTokenFile } from "./guards"; -import type { ServiceStopOutcome } from "./orchestration"; import { TASK, windowsLauncherVbsPath, windowsTaskXmlPath, serviceStatePath, writeServiceInstallState, serviceSourceDir, cliEntry } from "./state"; import { decodeSchtasksOutput, querySchtasks, schtasks, type WindowsSchedulerTaskProbe, schtasksErrorDetail, probeWindowsSchedulerTask, rollbackWindowsSchedulerTaskOwnedByAttempt } from "./windows-scheduler"; import { buildWindowsServiceScript, buildWindowsSchtasksCreateArgs, buildWindowsSchtasksCreateArgsForXml, buildWindowsLauncherVbs, resolvedWindowsTaskSid, buildWindowsTaskXmlDocument, windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy } from "./windows-taskxml"; @@ -851,42 +849,10 @@ export function stopWindows(beforeMutation?: () => void): void { } } -/** - * `stopWindows` for callers that need to know whether it worked. - * - * The void form swallows a non-benign `/end` failure, which is right for best-effort - * teardown and wrong for deciding whether an update may replace files: a scheduler that - * refused to stop can respawn the proxy on top of a half-written install (#3008). - */ -export function stopWindowsChecked(): boolean { - try { - schtasks(["/end", "/tn", TASK]); - return true; - } catch (error) { - return isWindowsSchedulerEndBenign(error); - } -} - export function statusWindows(): string { try { return schtasks(["/query", "/tn", TASK]); } catch { return ""; } } export function statusWindowsXml(): string { try { return schtasks(["/query", "/tn", TASK, "/xml"]); } catch { return ""; } } -/** - * Best-effort termination of surviving Windows scheduler launcher/wrapper processes. - * `schtasks /end` ends the task instance but often leaves wscript/cmd running the - * `:loop` batch, which brings the proxy back during a stop or restart. - * - * The matching rule — canonical paths of THIS installation, as complete - * command-line tokens — lives in lib/windows-service-wrappers so the update job - * cannot drift away from it again. - */ -export function killWindowsServiceWrapperProcesses(): void { - killWindowsSchedulerWrappers({ - scriptPath: windowsServiceScriptPath(), - launcherPath: windowsLauncherVbsPath(), - }); -} - export function uninstallWindows(): void { const probe = probeWindowsSchedulerTask(TASK); if (probe.status === "present") { @@ -909,22 +875,3 @@ export function uninstallWindows(): void { if (existsSync(windowsLauncherVbsPath())) unlinkSync(windowsLauncherVbsPath()); if (existsSync(windowsTaskXmlPath())) unlinkSync(windowsTaskXmlPath()); } - -/** - * Collapse the Windows backend observations into one outcome. - * - * Extracted so the precedence is testable by calling it. The rule that matters: a readable - * failure outranks an unreadable state, and an unreadable state outranks success — a - * scheduler we cannot see may still respawn the proxy. - */ -export function classifyWindowsServiceStop(o: { - stopped: boolean; - failed: boolean; - schedulerStopped: boolean; - stateUnknown: boolean; -}): ServiceStopOutcome { - if (o.failed) return "failed"; - if (o.stateUnknown) return "state-unknown"; - if (o.stopped) return o.schedulerStopped ? "stopped-respawnable" : "stopped"; - return "absent"; -} diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts new file mode 100644 index 00000000000..aac593c73b0 --- /dev/null +++ b/src/service/windows-service-stop.ts @@ -0,0 +1,321 @@ +/** + * Caller side of the checked Windows service stop (#7771, 030 "Where it sits"). + * + * Every Windows stop path reaches Part A and Part B through this module: ocx stop with and + * without a consent snapshot, ocx uninstall, /api/stop, ocx service stop / uninstall / + * remove, prepareServiceInstall and the updater. It owns: + * + * - the stop deadline (030 "Time budget"): --deadline-ms or OCX_STOP_DEADLINE_MS for + * ocx stop, measured from the command's start (performance.now() counts from + * performance.timeOrigin, the process start, so the deadline is an absolute value on the + * Part A clock); every other caller starts the default 60 s when its stop phase begins, + * because registration and the UAC prompt can come first; + * - the admission inputs and the mapping of Part A onto ServiceStopOutcome; + * - the Part A record a teardown receipt carries, so /api/stop can skip a second Part A; + * - a hook seam so caller tests run hermetically (no schtasks, PowerShell or FFI). + */ +import { loadConfig } from "../config"; +import { endpointsToProve, type ProbeEndpoint } from "../cli/uninstall-plan"; +import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; +import type { PendingTeardownPartA } from "../config/pending-teardown"; +import { readRuntimePort } from "../config/process-state"; +import type { WinswStatus } from "../lib/winsw"; +import type { ServiceStopOutcome } from "./orchestration"; +import { guardedStopWindowsServiceChecked, stopWindowsServiceChecked, windowsStopClock, type WindowsCheckedStopDeps, type WindowsCheckedStopOptions, type WindowsCheckedStopResult, type WindowsStopAdmission, type WindowsStopReason } from "./windows-checked-stop"; +import { stopperUnderHomeWrapper, windowsCheckedStopLiveDeps, windowsStopConfirmLiveDeps, type WindowsCheckedStopLiveOptions } from "./windows-checked-stop-live"; +import { windowsListenPort } from "./windows-ops"; +import type { WindowsStopProcessEntry } from "./windows-process-evidence"; +import { confirmWindowsStopFreed, resolveWindowsStopPort, type WindowsStopConfirmResult } from "./windows-stop-confirm"; +import { windowsStopDeadlineAt } from "./windows-stop-deadline"; + +export { parseStopDeadlineMs, resolveStopDeadlineMs, setStopDeadlineMs, STOP_DEADLINE_ENV, windowsStopDeadlineAt } from "./windows-stop-deadline"; + +/** Time left on the stop deadline, measured on the Part A clock (one clock read). */ +export function windowsStopRemainingMs(now: () => number = windowsStopClock.now): number { + const at = now(); + return windowsStopDeadlineAt(at) - at; +} + +export function configuredShutdownTimeoutMs(): number | undefined { + try { + const value = loadConfig().shutdownTimeoutMs; + return typeof value === "number" && value > 0 ? value : undefined; + } catch { + return undefined; + } +} + +export function windowsStopAdmission(input: { + orphanDiscovery: boolean; + inheritedReceipts?: number; + partB?: boolean; +}): WindowsStopAdmission { + return { + deadlineAt: windowsStopDeadlineAt(), + budget: { + orphanDiscovery: input.orphanDiscovery, + inheritedReceipts: input.inheritedReceipts ?? 0, + shutdownTimeoutMs: configuredShutdownTimeoutMs(), + partB: input.partB ?? true, + }, + }; +} + +export interface WindowsPartARequest { + admission?: WindowsStopAdmission; + /** The evidence snapshot (windowsProcessSnapshot) a guarded revalidation already read. */ + bindSnapshot?: readonly WindowsStopProcessEntry[]; + /** The updater's authorizer: throws to refuse, at entry, before each manager stop and before each terminate. */ + authorize?: () => void | Promise; + /** + * A hard cap (the route: 6 s with a receipt, 4 s for a stray sweep). An overrun answers + * unknown, and nothing further is changed: the manager commands and every terminate are + * gated off once the cap has passed. + */ + capMs?: number; + /** + * The stopping process may descend from this home's wrapper. Only the /api/stop route + * (it IS the wrapped proxy) and the updater (its worker descends from the service-run + * proxy) pass true; every interactive caller refuses such a stop with nothing changed. + */ + allowUnderOwnWrapper?: boolean; +} + +export interface WindowsPartBRequest { + port: number; + remainingMs: number; + /** The final enumeration; the guarded path wraps it to share the table it read. */ + snapshot?: (timeoutMs: number) => Promise; +} + +export interface WindowsServiceStopHooks { + partA?: (request: WindowsPartARequest) => Promise; + /** Run the real Part A against these deps instead of the live ones. */ + liveDeps?: (options: WindowsCheckedStopLiveOptions) => WindowsCheckedStopDeps; + partB?: (request: WindowsPartBRequest) => Promise; + /** The evidence snapshot the guarded inspection and the bind share. */ + snapshot?: () => Promise; + schedulePid?: () => Promise; + /** The route's registration re-check against a receipt's Part A record. */ + registration?: () => Promise<{ scheduler: "present" | "absent" | "unknown"; winsw: WinswStatus }>; +} + +let hooks: WindowsServiceStopHooks | null = null; + +/** Test-only: replace Part A / Part B / the evidence reads with hermetic fakes. */ +export function setWindowsServiceStopHooksForTests(next: WindowsServiceStopHooks | null): void { + hooks = next; +} + +export function windowsServiceStopHooks(): WindowsServiceStopHooks | null { + return hooks; +} + +function unknownResult(detail: string): WindowsCheckedStopResult { + return { + outcome: "unknown", reason: "evidence-unreadable", detail, + registration: { scheduler: "unknown", winsw: "unknown" }, + bound: [], unbound: [], terminated: [], changed: false, + }; +} + +export const UNDER_OWN_WRAPPER_DETAIL = "this stop runs under this home's service wrapper; run it from a separate terminal"; + +/** Part A with the live dependencies (or the test hook), optionally capped. */ +export async function runWindowsServiceStopPartA(request: WindowsPartARequest = {}): Promise { + let expired = false; + let underOwnWrapper = false; + const options: WindowsCheckedStopOptions = { + ...(request.admission ? { admission: request.admission } : {}), + ...(request.bindSnapshot ? { bindSnapshot: request.bindSnapshot } : {}), + }; + const run = async (): Promise => { + if (hooks?.partA) return hooks.partA(request); + const live: WindowsCheckedStopLiveOptions = { gate: () => !expired }; + const base = (hooks?.liveDeps ?? windowsCheckedStopLiveDeps)(live); + const selfPid = base.selfPid ?? process.pid; + const guard = (table: readonly WindowsStopProcessEntry[] | null): boolean => + !request.allowUnderOwnWrapper && table !== null && stopperUnderHomeWrapper(table, base.canonicalPaths, selfPid); + // The guarded path binds on a snapshot it already read: check it before anything runs. + if (request.bindSnapshot && guard(request.bindSnapshot)) return unknownResult(UNDER_OWN_WRAPPER_DETAIL); + // Otherwise the bind enumeration is checked as it arrives; a hit reads as an unreadable + // table, which Part A answers before /end when a manager is registered. + const deps = { + ...base, + enumerate: async (timeoutMs: number) => { + const table = await base.enumerate(timeoutMs); + if (!guard(table)) return table; + underOwnWrapper = true; + return null; + }, + }; + const authorize = async (): Promise => { + if (expired) throw new Error("the checked stop's cap has passed"); + await request.authorize?.(); + }; + const result = await (request.authorize || request.capMs !== undefined + ? guardedStopWindowsServiceChecked(authorize, deps, options) + : stopWindowsServiceChecked(deps, options)); + // The refusal is the answer whatever the stray path would have concluded: with nothing + // registered an unreadable enumeration reads as absent ("nothing to end"), which would + // let the caller stop the child and restore shared config beneath this home's own looping + // wrapper. Nothing has been changed at this point in either path, so it is unknown. + if (underOwnWrapper) { + result.outcome = "unknown"; + result.reason = "evidence-unreadable"; + result.detail = UNDER_OWN_WRAPPER_DETAIL; + result.changed = false; + } + return result; + }; + if (request.capMs === undefined) return run(); + const work = run().catch(error => { + if (error && typeof error === "object" && "partial" in error) return (error as { partial: WindowsCheckedStopResult }).partial; + throw error; + }); + let timer: ReturnType | undefined; + const cap = new Promise(resolve => { timer = setTimeout(() => resolve(null), request.capMs); }); + try { + const result = await Promise.race([work, cap]); + if (result === null) { + expired = true; + work.catch(() => undefined); + return unknownResult("the checked stop did not finish within its cap"); + } + return result; + } finally { + if (timer) clearTimeout(timer); + } +} + +/** Part A onto the existing ServiceStopOutcome (030 "Outcomes"). */ +export function serviceStopOutcomeFromWindows(result: WindowsCheckedStopResult): ServiceStopOutcome { + switch (result.outcome) { + case "absent": return "absent"; + case "stopped": return "stopped"; + case "unknown": return "state-unknown"; + default: return "failed"; + } +} + +/** The additive ocx-stop/1 serviceStopReason for a Part A result, if it has one. */ +export function windowsStopReason(result: WindowsCheckedStopResult | null | undefined): WindowsStopReason | undefined { + if (!result || result.outcome === "stopped" || result.outcome === "absent") return undefined; + return result.reason ?? undefined; +} + +/** One operator-facing line for a Part A result that was not stopped/absent. */ +export function describeWindowsPartA(result: WindowsCheckedStopResult): string { + switch (result.reason) { + case "access-denied": + return "The Windows service stop was denied for this account; run the stop from an elevated terminal."; + case "unbound-wrapper": + return "A service wrapper of this home is running but could not be tied to the service; end the wscript.exe/cmd.exe running the opencodex service script, then retry."; + case "unverified-budget": + return "The remaining time cannot hold a verified Windows service stop, so nothing was changed; retry from a terminal."; + case "evidence-unreadable": + return "The Windows service or process state could not be read (" + result.detail + "); nothing that could not be verified was stopped."; + default: + return "The Windows service manager or its wrapper is still running (" + result.detail + ")."; + } +} + +/** What a teardown receipt records after Part A (030 "/api/stop"). */ +export function windowsPartARecord(result: WindowsCheckedStopResult): PendingTeardownPartA | null { + if (result.outcome !== "stopped" && result.outcome !== "absent") return null; + return { + partA: result.outcome, + manager: { + scheduler: result.registration.scheduler === "present", + winsw: result.registration.winsw !== "nonexistent" && result.registration.winsw !== "unknown", + }, + wrappers: result.bound.map(wrapper => ({ pid: wrapper.pid, startTime: wrapper.startTime })), + }; +} + +/** + * Every endpoint this home could be serving on, exactly the set `ocx uninstall` proves. + * + * The receipt endpoint is ONE candidate; a stale runtime record naming a closed port must + * not stand in for the configured port where a live listener can sit (#3008, #7771). + */ +export function windowsRemovalEndpoints(): ProbeEndpoint[] { + let config: { port?: number; hostname?: string } = {}; + try { config = loadConfig(); } catch { /* defaults below */ } + return endpointsToProve(readRuntimePort(), config); +} + +/** + * Definitive absence on EVERY candidate. "unknown" is not absence: a listener that + * accepts connections but withholds /healthz, or one that times out, is exactly the state + * where deleting a registration or installing over it is most harmful. + */ +export async function endpointsAllDead( + endpoints: readonly ProbeEndpoint[], + probe: (endpoint: ProbeEndpoint) => Promise = probeEndpointLiveness, +): Promise<{ dead: boolean; live: number[]; unknown: number[] }> { + const results = await Promise.all(endpoints.map(async endpoint => ( + await probe(endpoint).catch((): EndpointLiveness => "unknown") + ))); + const live: number[] = []; + const unknown: number[] = []; + endpoints.forEach((endpoint, index) => { + const answer = results[index]; + if (answer === "live") live.push(endpoint.port); + else if (answer !== "dead") unknown.push(endpoint.port); + }); + // No candidate is not proof of anything. + return { dead: endpoints.length > 0 && live.length === 0 && unknown.length === 0, live, unknown }; +} + +/** + * The endpoint a teardown receipt should name: the runtime record when it has one, the + * configured address otherwise. `exact` is the only source a stop may defer against. + */ +export function windowsReceiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { + const runtime = readRuntimePort(); + if (runtime?.port) return { endpoint: { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port }, source: "exact" }; + try { + const config = loadConfig(); + return { endpoint: { hostname: config.hostname ?? "127.0.0.1", port: typeof config.port === "number" && config.port > 0 ? config.port : 10100 }, source: "guessed" }; + } catch { + return { endpoint: { hostname: "127.0.0.1", port: 10100 }, source: "guessed" }; + } +} + +/** The port Part B watches: runtime record, then the wrapper's baked port, then the config. */ +export function windowsServiceStopPort(): number { + return resolveWindowsStopPort({ + runtimePort: () => readRuntimePort()?.port ?? null, + wrapperPort: () => windowsListenPort(), + configuredPort: () => { + try { + const port = loadConfig().port; + return typeof port === "number" && port > 0 ? port : 10100; + } catch { + return 10100; + } + }, + }).port; +} + +/** Part B with the live dependencies (or the test hook). */ +export async function confirmWindowsServiceStop(request: WindowsPartBRequest): Promise { + if (hooks?.partB) return hooks.partB(request); + return confirmWindowsStopFreed(windowsStopConfirmLiveDeps(request.port, request.remainingMs, + request.snapshot ? { snapshot: request.snapshot } : {})); +} + +/** One operator-facing line for a Part B result other than freed. */ +export function describeWindowsPartB(result: WindowsStopConfirmResult): string { + if (result.outcome === "port-held-by-foreign") { + const owner = result.listener?.ownerPid ? " (PID " + result.listener.ownerPid + ")" : ""; + return "Another program" + owner + " holds port " + result.port + " after the stop."; + } + if (result.outcome === "survived") { + return result.lateWrappers.length > 0 + ? "A service wrapper of this home started again after the stop; it can relaunch the proxy." + : "The proxy answers on port " + result.port + " again after the stop; it is being respawned."; + } + return "Port " + result.port + " could not be confirmed free after the stop (" + result.detail + ")."; +} diff --git a/src/service/windows-stop-binding.ts b/src/service/windows-stop-binding.ts index b1c45dd6280..a2f7c2f0a18 100644 --- a/src/service/windows-stop-binding.ts +++ b/src/service/windows-stop-binding.ts @@ -26,7 +26,7 @@ */ import { createHash } from "node:crypto"; -import { commandLineHasPathToken } from "./windows-guarded-manager"; +import { commandLineHasPathToken } from "./windows-command-line-token"; import type { WindowsStopProcessEntry } from "./windows-process-evidence"; const WRAPPER_IMAGES = new Set(["wscript.exe", "cscript.exe", "cmd.exe"]); diff --git a/src/service/windows-stop-budget.ts b/src/service/windows-stop-budget.ts index ca937c6c851..344696ffb32 100644 --- a/src/service/windows-stop-budget.ts +++ b/src/service/windows-stop-budget.ts @@ -79,10 +79,10 @@ export function windowsPartAWorstCaseMs(input: Pick, so the SCM poll is capped at that plus a margin. + // The service stop itself is capped at plus a margin. ms += C.winswStopCommand + stopTimeout + C.winswPollMargin + C.managerState; } // Nothing registered: the stray enumeration already ran in the registration step. diff --git a/src/service/windows-stop-confirm.ts b/src/service/windows-stop-confirm.ts index 38d2c9baa6b..20d519c332e 100644 --- a/src/service/windows-stop-confirm.ts +++ b/src/service/windows-stop-confirm.ts @@ -18,7 +18,7 @@ import type { WindowsStopProcessEntry } from "./windows-process-evidence"; import { windowsHomeWrapperPids } from "./windows-stop-binding"; import { runCapped, windowsStopClock, type WindowsStopClock } from "./windows-checked-stop"; -import { commandLineHasPathToken } from "./windows-guarded-manager"; +import { commandLineHasPathToken } from "./windows-command-line-token"; import { WINDOWS_STOP_CAPS, windowsPartBWindowMs } from "./windows-stop-budget"; type MaybePromise = T | Promise; diff --git a/src/service/windows-stop-deadline.ts b/src/service/windows-stop-deadline.ts new file mode 100644 index 00000000000..8ab7c367b46 --- /dev/null +++ b/src/service/windows-stop-deadline.ts @@ -0,0 +1,56 @@ +/** + * The checked Windows stop's deadline (#7771, 030 "Time budget"). Kept free of the stop + * machinery because the CLI dispatcher, which loads on every command, records it. + * + * ocx stop reads --deadline-ms or OCX_STOP_DEADLINE_MS (the flag wins; an empty or + * invalid variable counts as absent) and records the resolved budget as a command-start + * deadline: performance.now() counts from performance.timeOrigin, so N ms of budget at + * command start is the Part A clock value N. Every other caller starts the default 60 s + * when its stop phase begins, not when the process did -- installFreshWindowsSchedulerSafely + * reaches the stop only after registration and the UAC prompt, so measuring from process + * start would refuse `unverified-budget` a stop that had its whole budget left. + */ +import { DEFAULT_WINDOWS_STOP_DEADLINE_MS } from "./windows-stop-budget"; + +export const STOP_DEADLINE_ENV = "OCX_STOP_DEADLINE_MS"; + +/** A positive decimal integer of milliseconds, or null (absent, empty or invalid). */ +export function parseStopDeadlineMs(raw: string | undefined | null): number | null { + if (raw === undefined || raw === null) return null; + const trimmed = raw.trim(); + if (!/^\d+$/.test(trimmed)) return null; + const value = Number(trimmed); + return Number.isSafeInteger(value) && value > 0 ? value : null; +} + +/** The flag wins; an empty or invalid variable is absent; neither means 60 s. */ +export function resolveStopDeadlineMs(flagMs: number | null | undefined, env: NodeJS.ProcessEnv = process.env): number { + if (typeof flagMs === "number" && Number.isSafeInteger(flagMs) && flagMs > 0) return flagMs; + return parseStopDeadlineMs(env[STOP_DEADLINE_ENV]) ?? DEFAULT_WINDOWS_STOP_DEADLINE_MS; +} + +/** ocx stop's command-start deadline, or null while no command recorded one. */ +let commandDeadlineMs: number | null = null; +/** The unconfigured caller's deadline, armed once when its stop phase first asks. */ +let phaseDeadlineAt: number | null = null; + +/** + * ocx stop records its resolved deadline here once its arguments are parsed. A new value + * is an absolute command-start deadline; null clears it and re-arms the stop phase, so the + * next unconfigured caller measures its own 60 s from when it gets there. + */ +export function setStopDeadlineMs(ms: number | null): void { + commandDeadlineMs = ms; + phaseDeadlineAt = null; +} + +/** + * The absolute deadline on the Part A clock (ms since process start). An explicit command + * deadline wins; otherwise the first call in a stop phase arms the default 60 s and every + * later call inside that phase keeps it -- one deadline governs the whole stop. + */ +export function windowsStopDeadlineAt(now: number = performance.now()): number { + if (commandDeadlineMs !== null) return commandDeadlineMs; + if (phaseDeadlineAt === null) phaseDeadlineAt = now + DEFAULT_WINDOWS_STOP_DEADLINE_MS; + return phaseDeadlineAt; +} diff --git a/src/update/guarded-runtime-ops.ts b/src/update/guarded-runtime-ops.ts index 289f801e17e..dd2eec192b0 100644 --- a/src/update/guarded-runtime-ops.ts +++ b/src/update/guarded-runtime-ops.ts @@ -4,8 +4,7 @@ import { atomicWriteFile } from "../config/atomic-write"; import { removePid, removeRuntimePort } from "../config/process-state"; import { dropWindowsTcpRowsForLocalPort } from "../server/windows-tcp-drop"; import { stopWinswService } from "../lib/winsw"; -import { stopWindows } from "../service"; -import { killWindowsSchedulerWrappers } from "../lib/windows-service-wrappers"; +import { runWindowsServiceStopPartA } from "../service/windows-service-stop"; import { guardedRun, guardedKill, guardedExecFileSync, materializeMutationData, UpdateMutationRefusedError, type UpdateMutationAuthorizer } from "./mutation-guard.mjs"; export type KillProxy = typeof killProxy; @@ -50,9 +49,8 @@ export function guardedDropWindowsTcp(authorize: (kind: "runtime") => boolean, p export function guardedStopWinsw(authorize: UpdateMutationAuthorizer) { return stopWinswService(mutationGate(authorize, "service")); } -export function guardedStopWindows(authorize: UpdateMutationAuthorizer) { - return stopWindows(mutationGate(authorize, "service")); -} -export function guardedKillWindowsWrappers(authorize: UpdateMutationAuthorizer, paths: Parameters[0]) { - return killWindowsSchedulerWrappers(materializeMutationData(paths), mutationGate(authorize, "service")); +/** Part A of the checked Windows stop (#7771), the service gate at entry and before each terminate. */ +export async function guardedStopWindowsChecked(authorize: UpdateMutationAuthorizer) { + // The update worker descends from the service-run proxy, so its own wrapper is fair game. + return (await runWindowsServiceStopPartA({ authorize: mutationGate(authorize, "service"), allowUnderOwnWrapper: true })).outcome; } diff --git a/src/update/job-windows-restart.ts b/src/update/job-windows-restart.ts index 89f4f45839d..5f7bda07647 100644 --- a/src/update/job-windows-restart.ts +++ b/src/update/job-windows-restart.ts @@ -1,39 +1,26 @@ -import { join } from "node:path"; -import { getConfigDir } from "../config"; -import { guardedStopWinsw, guardedStopWindows, guardedKillWindowsWrappers } from "./guarded-runtime-ops"; -import { readServiceBackend } from "../service"; +import { guardedStopWindowsChecked } from "./guarded-runtime-ops"; import type { UpdateMutationAuthorizer } from "./mutation-guard.mjs"; -/** Stop the installed Windows backend and best-effort kill surviving :loop wrappers. */ -export function stopWindowsServiceWrappersBestEffort(authorizeMutation: UpdateMutationAuthorizer): void { - if (process.platform !== "win32") return; - try { - if (readServiceBackend() === "native") { - if (!authorizeMutation("service")) return; - guardedStopWinsw(authorizeMutation); - return; - } - if (!authorizeMutation("service")) return; - guardedStopWindows(authorizeMutation); - } catch { /* already stopped */ } - if (!authorizeMutation("service")) return; - killWindowsServiceWrapperProcesses(authorizeMutation); -} +/** Why an automatic update restart stopped instead of starting a runtime over a live service. */ +export const WINDOWS_SERVICE_STOP_ABORT = "Automatic update restart stopped: the Windows service could not be proven stopped, " + + "so no runtime was started over it. Run 'ocx stop' from a terminal, then 'ocx start'."; /** - * Best-effort termination of surviving Windows scheduler launcher/wrapper processes. - * `schtasks /end` ends the task instance but often leaves wscript/cmd running the - * `:loop` batch, which brings the proxy back during post-update reclaim. + * Stop the installed Windows backend before the updater replaces or restarts the runtime. + * Returns false when the restart must abort; true when nothing can relaunch the proxy. * - * This used to match the bare filenames with -like '*name*', which could stop a - * wrapper belonging to a DIFFERENT OpenCodex home under the same account. The - * shared killer scopes to this home's canonical paths as complete tokens. + * Both backends use Part A of the checked stop (#7771) through the authorizer-gated + * variant: the service authorization runs at entry and before every terminate, and only + * stopped or absent lets the restart go on. */ -function killWindowsServiceWrapperProcesses(authorizeMutation: UpdateMutationAuthorizer): void { - const paths = { - scriptPath: join(getConfigDir(), "opencodex-service.cmd"), - launcherPath: join(getConfigDir(), "opencodex-service-launcher.vbs"), - }; - if (!authorizeMutation("service")) return; - guardedKillWindowsWrappers(authorizeMutation, paths); +export async function stopWindowsServiceForRestart(authorizeMutation: UpdateMutationAuthorizer): Promise { + if (process.platform !== "win32") return true; + if (!authorizeMutation("service")) return false; + try { + const outcome = await guardedStopWindowsChecked(authorizeMutation); + return outcome === "stopped" || outcome === "absent"; + } catch { + // A refusal by the service authorizer (at entry or before a terminate) aborts too. + return false; + } } diff --git a/src/update/job.ts b/src/update/job.ts index b5926fceb40..8d84ad2ad65 100644 --- a/src/update/job.ts +++ b/src/update/job.ts @@ -22,7 +22,7 @@ import { type HealthzIdentity, } from "../server/proxy-liveness"; import { isServiceInstalled, isServiceViable } from "../service"; -import { stopWindowsServiceWrappersBestEffort } from "./job-windows-restart"; +import { stopWindowsServiceForRestart, WINDOWS_SERVICE_STOP_ABORT } from "./job-windows-restart"; import { createUpdateServiceHomePolicy, inspectUpdateServiceHome, type UpdateServiceHomeInspector } from "./service-home-authority"; import { runUpdateRestartWithOwnershipLease, type ServiceOwnershipResolution } from "./restart-ownership"; import { restartServiceManagerAfterUpdate, unsettledManagerRestartFailure, unsupervisedRestartFailure, type ManagerRestartFn } from "./job-supervised-restart"; @@ -396,6 +396,10 @@ export function readUpdateJob(jobId?: string | null): UpdateJobState | null { function updateJob(job: UpdateJobState, patch: Partial, logLine?: string): UpdateJobState { const next = prepareUpdateJob(job, patch, logLine); writeJob(next); return next; } +/** A Windows service that could not be proven stopped aborts the restart (#7771). */ +function windowsServiceStopAborted(job: UpdateJobState): false { + updateJob(job, { status: "failed", restarted: false, error: WINDOWS_SERVICE_STOP_ABORT }, WINDOWS_SERVICE_STOP_ABORT); return false; +} function prepareUpdateJob(job: UpdateJobState, patch: Partial, logLine?: string): UpdateJobState { const current = readUpdateJob(job.id) ?? job; const next = { @@ -954,7 +958,8 @@ export interface RestartIo { serviceInstalledFn?: () => boolean; serviceHomeAuthorityFn?: UpdateServiceHomeInspector; serviceHomeBaseline?: ReturnType; // Worker delegation pins its original authority. - stopServiceWrappersFn?: (authorizeMutation: UpdateMutationAuthorizer) => void; + /** Windows service stop before a restart; an explicit false aborts the restart (#7771). */ + stopServiceWrappersFn?: (authorizeMutation: UpdateMutationAuthorizer) => boolean | void | Promise; readPidFn?: typeof readPid; /** * After a service reinstall exits 0, only trust the service path when this is true. @@ -1121,7 +1126,7 @@ async function restartAfterUpdate( if (!homePolicy.authorizeMutation("runtime")) return false; // a home change during the service-args import refuses before stop/reclaim // schtasks /end often leaves the hidden wrapper alive; its :loop respawns // `ocx start` and races reclaim, so end wrappers before the socket. - if (homePolicy.authorizeMutation("service")) (io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation); + if (homePolicy.authorizeMutation("service") && await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job); const preServiceAllow = reclaimKillAllowlist(); const freed = await waitFn(port, hostname, reclaimOptsFor(preServiceAllow)); if (!freed && !reclaimGate()) return false; // a latched denial already refused the job @@ -1249,7 +1254,7 @@ async function restartAfterUpdate( // below are the path that repairs stuck Windows listeners. } } - if (serviceInstalled && homePolicy.authorizeMutation("service")) (io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation); + if (serviceInstalled && homePolicy.authorizeMutation("service") && await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job); // Reclaim before the pinned start: a busy old socket used to hop ports or fail health. const directAllow = reclaimKillAllowlist(); const freed = await waitFn(port, hostname, reclaimOptsFor(directAllow)); @@ -1348,7 +1353,9 @@ async function restartAfterUpdate( lastChild = null; } if (!homePolicy.authorizeMutation("runtime")) return false; - if (preparePort(job, port, listPids, aliveFn, verifyOcx, () => { if (serviceInstalled && homePolicy.authorizeMutation("service")) (io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation); }, reclaimGate) === false) return false; + // #7771: the service stop runs before the synchronous preparation so its outcome can abort. + if (serviceInstalled && homePolicy.authorizeMutation("service") && await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job); + if (preparePort(job, port, listPids, aliveFn, verifyOcx, undefined, reclaimGate) === false) return false; const ready = await waitForGhost( port, hostname, diff --git a/structure/INDEX.md b/structure/INDEX.md index 85b0a275bbc..520647991a0 100644 --- a/structure/INDEX.md +++ b/structure/INDEX.md @@ -165,7 +165,7 @@ Background service, docs, release, and design discipline. | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | Durable Desktop ownership read, takeover-consent handling, live-supervision evidence, the Windows probe's trusted system-directory acquisition, and the supervision latch blocking contract consumed by service and update guards. | | [`ops/service-logging.md`](ops/service-logging.md) | Service-only console timestamps and bounded copy-truncate log retention. Producer bounds for combo positions, target providers and effort diagnostics. | | [`ops/service-health-recovery.md`](ops/service-health-recovery.md) | Opt-in sentinel that turns a hung but listening managed service proxy into an exit its manager restarts. | -| [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | Windows process evidence, the wrapper binding rule, and the checked manager stop and port confirmation that are not yet wired to a stop path. | +| [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | Windows process evidence, the wrapper binding rule, the checked manager stop and port confirmation, and their wiring into every Windows stop path (ocx stop, uninstall, /api/stop, service commands, install/switch, updater). | | [`ops/update-supervised-recovery.md`](ops/update-supervised-recovery.md) | Manager-first recovery after a failed post-update service refresh, and the non-zero unsupervised fallback. | | [`ops/plugins.md`](ops/plugins.md) | Plugin loading from OPENCODEX_HOME/plugins and the upstream rewrite slot plugins attach to. | | [`ops/docs-and-release.md`](ops/docs-and-release.md) | Docs site, workflow map, branch policy, release flow, and cross-platform CI. | @@ -267,6 +267,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/cli/models-runtime.ts` | [`pricing.md`](pricing.md) | | `src/cli/status-codex-settings.ts` | [`remote-link.md`](remote-link.md) | | `src/cli/status-link.ts` | [`remote-link.md`](remote-link.md) | +| `src/cli/stop-windows-checked.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/cli/telemetry.ts` | [`telemetry.md`](telemetry.md) | | `src/cli/update-restart.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/client/` | [`runtime.md`](runtime.md)
[`fallback-policy.md`](fallback-policy.md)
[`clients/claude-desktop.md`](clients/claude-desktop.md) | @@ -515,10 +516,12 @@ A source area can be described by more than one doc, because these docs are orga | `src/server/safe-response-headers.ts` | [`data-planes/error-detail.md`](data-planes/error-detail.md) | | `src/server/search.ts` | [`providers/codex-installation-identity.md`](providers/codex-installation-identity.md) | | `src/server/sse-payload-rewrite.ts` | [`data-planes/error-detail.md`](data-planes/error-detail.md) | +| `src/server/stop-service-route.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/server/ws-bridge.ts` | [`providers/codex-installation-identity.md`](providers/codex-installation-identity.md) | | `src/service-manager-probe.ts` | [`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/service.ts` | [`runtime.md`](runtime.md)
[`ops/docs-and-release.md`](ops/docs-and-release.md) | | `src/service/` | [`runtime.md`](runtime.md) | +| `src/service/cli-windows-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/cli.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/desktop-command-guard.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/desktop-recovery-process.ts` | [`desktop-recovery.md`](desktop-recovery.md) | @@ -535,12 +538,15 @@ A source area can be described by more than one doc, because these docs are orga | `src/service/logging.ts` | [`ops/service-logging.md`](ops/service-logging.md) | | `src/service/orchestration.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/repair.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | +| `src/service/windows-checked-stop-live.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-checked-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-process-evidence.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-process-native.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-service-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-stop-binding.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-stop-budget.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-stop-confirm.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-stop-deadline.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/sidecar/` | [`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/sidecar/anthropic-binding.ts` | [`providers/anthropic-account-pool.md`](providers/anthropic-account-pool.md) | | `src/stall-timeout.ts` | [`transports/timeouts.md`](transports/timeouts.md)
[`runtime.md`](runtime.md) | diff --git a/structure/cli-management.md b/structure/cli-management.md index 9625521dc16..5e0e83a6ffe 100644 --- a/structure/cli-management.md +++ b/structure/cli-management.md @@ -63,7 +63,7 @@ The agents/routing leaf declares local `message sessions` and `message send`, th ## Generated operating reference -The stop declaration in `src/cli/capabilities-observe-system.ts` covers the read-only preservation preflight and shutdown methods at `/api/stop`, both declared in `src/server/management/route-registry.ts`. Discovery and generated help describe the existing `ocx stop [--json]` envelope and ownership exceptions; the declaration grants no shutdown authority. [Runtime](runtime.md) owns refusal, absence and teardown semantics. +The stop declaration in `src/cli/capabilities-observe-system.ts` covers the read-only preservation preflight and shutdown methods at `/api/stop`, both declared in `src/server/management/route-registry.ts`. Discovery and generated help describe the existing `ocx stop [--json] [--deadline-ms ]` envelope and ownership exceptions; the declaration grants no shutdown authority. [Runtime](runtime.md) owns refusal, absence and teardown semantics. `ocx gui pair --desktop --origin ` creates a Desktop-purpose one-use grant through the signed Desktop path and a private-home intent. It accepts the actual standalone diff --git a/structure/manifest.json b/structure/manifest.json index 2204c2368db..9788432dbd2 100644 --- a/structure/manifest.json +++ b/structure/manifest.json @@ -1307,14 +1307,20 @@ "path": "ops/windows-checked-stop.md", "tier": 6, "title": "Windows Checked Service Stop", - "scope": "Windows process evidence, the wrapper binding rule, and the checked manager stop and port confirmation that are not yet wired to a stop path.", + "scope": "Windows process evidence, the wrapper binding rule, the checked manager stop and port confirmation, and their wiring into every Windows stop path (ocx stop, uninstall, /api/stop, service commands, install/switch, updater).", "documents": [ + "src/cli/stop-windows-checked.ts", + "src/server/stop-service-route.ts", + "src/service/cli-windows-stop.ts", "src/service/windows-checked-stop.ts", + "src/service/windows-checked-stop-live.ts", "src/service/windows-process-evidence.ts", "src/service/windows-process-native.ts", + "src/service/windows-service-stop.ts", "src/service/windows-stop-binding.ts", "src/service/windows-stop-budget.ts", - "src/service/windows-stop-confirm.ts" + "src/service/windows-stop-confirm.ts", + "src/service/windows-stop-deadline.ts" ] }, { diff --git a/structure/ops/update-supervised-recovery.md b/structure/ops/update-supervised-recovery.md index 070a71428b7..9e01dc3d8b2 100644 --- a/structure/ops/update-supervised-recovery.md +++ b/structure/ops/update-supervised-recovery.md @@ -103,9 +103,14 @@ is a home failure and an actually started unsupervised fallback remains a failur `src/update/job-pinned-start.ts` performs pinned-start I/O and checks after log setup, just before spawning. Its caller passes the same latched authorizer. `src/update/job-windows-restart.ts` receives that same authorizer at all worker call -sites. It rechecks service permission after backend observation, before each backend -stop and after the blocking stop or its catch path before wrapper termination. Path -setup precedes a final service check at the scoped scheduler-wrapper killer. +sites. It checks service permission after backend observation and before the backend stop. +WinSW keeps its stop; Task Scheduler runs Part A of the checked Windows stop +(`guardedStopWindowsChecked`, [Windows checked service stop](windows-checked-stop.md)), whose +authorizer is the service mutation gate at entry and again before every bound-wrapper +terminate. A refusal, or any outcome other than stopped or absent, returns false, and each of +the three call sites in `src/update/job.ts` then fails the restart before reclaim or a start; +the one in front of the synchronous pinned-start preparation runs before it so its outcome can +abort. The Node launcher, Bun updater and dashboard worker use the same transition policy in `src/update/service-home-policy.mjs`. Their direct recovery helpers reacquire the ownership @@ -212,7 +217,7 @@ restart refusal, including after a transient re-match; void-returning preparatio The manager helper checks independently after lease release, immediately before contacting the manager, and returns distinct `refused` for foreign or unknown homes; callers cannot treat this as an ordinary failed restart. `src/update/job-windows-restart.ts` owns scoped Windows -wrapper cleanup under the same latched service guard. Pinned direct recovery receives global service-stop authority only for a matching +service stops under the same latched service guard. Pinned direct recovery receives global service-stop authority only for a matching managed home; foreground recovery does not acquire that authority. The preparation callback tests the current `serviceAllowed` after preparation's fresh mutation gate: a matching registration disappearing still allows foreground PID/port cleanup, while diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index c6dc76270c9..4271a9fadbe 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -2,11 +2,16 @@ The checked Windows service stop (#7771) is a library of primitives and two operations that stop a Task Scheduler or WinSW manager, end the wrappers that would relaunch the proxy, and -then confirm the port is free. **No stop path calls it yet.** `ocx stop`, `ocx service stop`, -`ocx uninstall`, service uninstall and install, `/api/stop` and the updater still use the -best-effort sweep in `src/lib/windows-service-wrappers.ts` and the guarded inspection in -`src/service/windows-guarded-manager.ts`, unchanged. The plan for wiring the callers lives in -`devlog/_plan/261111_windows_service_stop/`. +then confirm the port is free. Every Windows stop path that also restores shared client state +uses it: `ocx stop` with and without a consent snapshot, `ocx uninstall`, `/api/stop`, +`ocx service stop` / `uninstall` / `remove`, service install and backend switch, and the updater. +The best-effort path-token sweep in `src/lib/windows-service-wrappers.ts` is no longer called by +any stop path. `ocx service repair` is the one remaining exception: it still stops the manager +through `stopWindows()` (`src/service/repair.ts:259`) because it does not restore shared client +state, so the #3008 relaunch window this stop exists to close does not apply to it. Wiring repair +is its own follow-up slice (`devlog/_plan/261111_windows_service_stop/060_implementation.md`, +"Follow-up slices"). macOS and Linux keep their manager stops and timing; only the signatures +turned async. The design and its slices live in `devlog/_plan/261111_windows_service_stop/`. ## Process evidence @@ -60,7 +65,9 @@ deepest first and carry pid, creation time, image and a SHA-256 digest of the co external effect is injected. It probes registration (2 s each). An unreadable answer is `unknown` with nothing changed. With nothing registered it enumerates once for a stray wrapper of a deleted task; an unreadable table or no wrapper is `absent`. With a task registered or a stray sweep -running, it reads the Schedule host PID (2 s). Admission then compares the time left on the +running, it reads the Schedule host PID (2 s). A WinSW-only registration also binds against a +process enumeration, including when WinSW is stopped, so ancestry and stray-wrapper evidence +are checked before a stop can be confirmed. Admission then compares the time left on the caller's deadline, which already excludes these first steps, with the worst case still to come, and returns `unverified-budget` without changing anything when it does not fit. It binds, ends the task (`schtasks /end`, 2 s) or stops WinSW (a @@ -69,7 +76,7 @@ deepest first, waits on the bind-time watches (terminates and wait share 3 s), a manager state once (1 s). Denied stops every further step. A cap overrun is `unknown`. A wrapper or manager still running is `survived` (`unbound-wrapper` when the survivor was never bound). WinSW is `survived` while the proxy is still alive after the service reports stopped. The authorizer -variant `guardedStopWindowsServiceChecked` runs its authorizer at entry and before each terminate and +variant `guardedStopWindowsServiceChecked` runs its authorizer at entry, before each manager stop (/end, the WinSW stop) and before each terminate and reports a refusal as `WindowsCheckedStopRefusedError` with what had already happened. Each step that asks the machine runs under a cap that races a timer and rechecks the elapsed time @@ -90,9 +97,166 @@ configured port. `src/service/windows-stop-budget.ts` holds the caps and the admission arithmetic. With default settings the minimum is 48.5 s for a guarded Task Scheduler stop, 56.5 s for a snapshotless one -and 42.5 s for WinSW. Each includes the 0.5 s closing port sample that Part B can take past its +and 50.5 s for WinSW. Each includes the 0.5 s closing port sample that Part B can take past its window. +## Callers + +`src/service/windows-checked-stop-live.ts` is the production side of every seam: capped +asynchronous `schtasks` and `sc.exe` calls through `execFile` (a deadline-derived timeout is +floored to whole milliseconds and one that floors to 0 spawns nothing, because `execFile` reads 0 +as no timeout), the evidence snapshot `windowsProcessSnapshot` for every bind, the checked +terminate, the exit watch, the Part B port sampler and the listener identity. The task's running +state is read from `schtasks /query /fo CSV`, falling back to `Get-ScheduledTask` inside the same +1 s cap when the status word is localized. A running WinSW service is only this home's manager +when `sc qc` shows a `BINARY_PATH_NAME` naming this installation's WinSW executable, the same +proof the guarded manager path uses: the service ID is fixed, so a successful `sc query` alone +would let Part A stop another installation's service. An unreadable registration is `unknown` +(`evidence-unreadable`, nothing changed); a foreign executable is reported as `nonexistent`, so +Part A stops nothing and answers `absent`. + +The port sampler is `sampleWindowsStopPort(port, timeoutMs)`, async and bounded by the caller's +timeout: it runs `netstat.exe` from the trusted System32 directory through `execFile` with that +timeout and `SIGKILL`, and a scan that misses the timeout, or cannot be read, samples `unknown`. +A synchronous scan would ignore the cap and could stall the stop for the helper's own 5 s. A row +whose owner has exited is cleared like `reclaimListenPort` does, but only the rows that exited +PID owns: `dropWindowsTcpRowsForLocalPort` takes an owner filter, so a program that took the port +between scans keeps its connections. The listener identity is +tri-state over `inspectOcxStartProcessIdentity`: only a positive `mismatch` is `foreign`, and an +`unverifiable` probe (another account's or an elevated process's command line) stays `unknown`. +`verifyPidIdentity`'s null collapses those two, which would report a listener we merely could not +read as another program holding the port. + +`src/service/windows-service-stop.ts` is what callers call: `runWindowsServiceStopPartA` (optional +admission, a reused bind snapshot, the updater's authorizer, a hard cap for the route), +`confirmWindowsServiceStop` (Part B), the mapping onto `ServiceStopOutcome` (absent, stopped; +survived, denied and unverified-budget are failed; unknown is state-unknown), the additive +`serviceStopReason` and the receipt record. The deadline lives in +`src/service/windows-stop-deadline.ts`, and its origin depends on who is stopping. `ocx stop` +takes `--deadline-ms `, else `OCX_STOP_DEADLINE_MS` (empty or invalid counts as absent), else +60 s, measured from the command's own start: that is the operator's budget, and +`performance.now()` counting from `performance.timeOrigin` makes it an absolute value on the Part +A clock. Every other caller starts the default 60 s when its stop phase begins, because it +reaches the stop after work it cannot charge to the stop — the scheduler registration and the UAC +prompt — and a budget measured from process start could already be spent, which made admission +refuse `unverified-budget` a stop that had its whole budget left. The first deadline read in a +stop phase arms it and later reads inside that phase keep it, so one deadline governs the whole +stop instead of sliding forward. + +- `stopServiceIfInstalledReport` / `stopServiceIfInstalledDetailed` (`src/service/orchestration.ts`) + are async and run Part A on Windows. On Windows Task Scheduler never yields `stopped-respawnable`. +- Plain `ocx stop` (`src/cli/stop-windows-checked.ts`) admits against the deadline with orphan + discovery and the inherited receipts counted. A Part A other than stopped or absent holds the + child: no `/api/stop` POST, the shared teardown skipped, `service` failed or state-unknown, + `proxy` unknown. A thrown Part A also holds the child and reports a service error. After a stopped Part A, the POST skips `stopProxy`'s legacy 15 s + `waitForStoppedPort`, orphan discovery is capped at 3 s, and Part B replaces the respawn re-poll: + our identity back on the port is `proxy: respawned`, an unconfirmed port is + `evidence-unreadable`, and another program on the port sets `portHeldByForeign` without failing + the stop. The port wait is skipped on BOTH Windows paths, guarded included, through + `windowsPartBOwnsPortWait`; `windowsPartBDue` stays snapshotless-only because it also gates the + legacy respawn re-poll. A failed owner receipt claim refuses the child stop and shared teardown + on both paths. `ocx uninstall`'s two `stopProxy` calls pass the same predicate. +- The guarded path reads the evidence snapshot and the Schedule host PID once per inspection; + `stopManager` binds on the revalidation's snapshot (so its admission does not count a second + enumeration), Part B runs after `settle` on the approved endpoint, and the manager-state checks + reuse Part B's final enumeration. +- A registered task whose state is `not-running` promotes its surviving wrappers to `bound-stray` + only after `winTaskXml` and `winRegistrationOurs` pass (`src/service/windows-guarded-manager.ts`). + The state word says the task is inert, not that it is ours; promoting on the word alone let a + same-named foreign task make this home's hand-started wrappers terminable. Nothing surviving + keeps the `absent` verdict without the read; an unreadable definition is `evidence-unreadable` + and a definition that is not this installation's is `unknown`, both leaving the previous + verdict. +- The teardown receipt carries the Part A record (`partA` stopped or absent, the registered + managers, the ended wrappers as pid + creation time): written at the claim, or added by the owner + for a receipt claimed before Part A. `/api/stop` (`src/server/stop-service-route.ts`) skips its + own Part A only for a record whose managers are still the registered ones; a receipt without it + runs Part A capped at 6 s; without a receipt a registered WinSW keeps its in-process stop and + anything else gets the stray sweep capped at 4 s. A cap overrun is `service_state_unknown`, and + the manager commands and terminates are gated off once the cap has passed. +- `ocx service stop` (`src/service/cli-windows-stop.ts`) refuses on a Part A other than stopped or + absent, claims a teardown receipt before the deferred tracked-proxy stop when this home owns the + shared teardown (a failed claim sends no child stop), runs Part B, and clears the receipt only + after the restore. With Part A absent it requires every candidate endpoint to be definitively + dead; live or unknown keeps the receipt and blocks restoration. `ocx service uninstall` / `remove` refuse before deleting the task unless + Part A stopped (or found nothing) and Part B did not find our identity or an unreadable port. + Removal claims the same receipt, carrying Part A's record, before its child stop: Part A leaves + the scheduler task registered, so a receipt-less POST answers `respawnable_service` (409) and + the removal refuses with the proxy left running. The receipt is discharged only after the + shared restore, and a failed claim sends no child stop. Part A `absent` says no manager is + registered, not that nothing serves, so removal then proves every candidate endpoint + (`endpointsToProve(readRuntimePort(), loadConfig())`, the set `ocx uninstall` proves) tri-state: + only a definitive dead on all of them permits deleting the task, while live or unknown refuses + and keeps the receipt. The receipt endpoint alone is not enough — a stale runtime record naming + a closed port must not stand in for the configured port where a live listener can sit. +- Every tracked-proxy stop takes its orphan-discovery cap from the caller + (`TrackedProxyStopIo.orphanDiscoveryCapMs`). The checked Windows callers pass the admitted 3 s + (`WINDOWS_STOP_CAPS.orphanDiscovery`), which is what admission priced; the omitted default stays + the legacy 7 s so macOS and Linux timing is unchanged. +- Windows service stop, removal and install resolve one tracked or discovered proxy target before + claiming its receipt. The receipt and POST use that target's exact endpoint; Part B checks its + port. No target means no receipt or POST. An owning caller refuses a target without an endpoint. +- `ocx uninstall` (`src/cli/uninstall-stop.ts`) resolves its stop target before it claims + anything: the tracked pid with its own runtime endpoint, or the discovered live proxy with the + endpoint discovery found it on. The receipt and `stopProxy` then name the same endpoint, so a + stale runtime record cannot produce a receipt for a port the stop never contacted, and no target + means no receipt at all (a guessed obligation recovery cannot resolve is worse than none). The + tracked record is removed only for a stop that was proven, and the step reports a change only + when state changed. When Part A did not end stopped or absent the graceful POST still goes out + with `allowForcedFallback: false`, and that unconfirmed wait is observational: it samples the + port without `reclaimListenPort`'s kill eligibility, so a forbidden fallback cannot kill through + the reclaim path either. An unconfirmed stop is not proof the proxy is down: `stopProxy` answers + false without contacting the endpoint when the tracked pid is already dead, so the step then + requires the tri-state `endpointProvenDown()` over every candidate before it reports + `provenDown`, exactly as the discovery path does. A different proxy on the configured port + would otherwise be missed and shared config restored under it (#3008). +- `ocx uninstall` removes the Windows manager only after a Part A that ended stopped or absent + (`serviceRemovalAllowed` in `src/cli/uninstall-plan.ts`); otherwise the step records `failed` and + leaves the registration, as `ocx service uninstall` refuses. `uninstallServiceDetailed` + (`src/service/orchestration.ts`) reads the scheduler probe, the native status and `sc qc` + `BINARY_PATH_NAME` before the first destructive step: a registration naming another + installation's WinSW executable is left untouched, and an unreadable status or ownership + refuses and keeps install state. The task used to be deleted first, so an unreadable `sc qc` + left a half-removed install with no task to retry from. Non-Windows removal is unchanged. +- `prepareServiceInstall` refuses before replacing anything on a Part A other than stopped or + absent and stops install and switch on any Part B result other than a free port, naming the + owner. `ocx uninstall` runs Part B as its respawn window after a stopped Part A. +- Install and backend switch claim a teardown receipt before their tracked stop when Part A + ended `stopped`: the task is still registered, so a receipt-less POST answers 409 + `respawnable_service` and the install aborted while the old proxy kept serving. The receipt + carries Part A's record and its nonce travels on the POST. An install replaces the proxy, so + shared config stays pointed at OpenCodex and the receipt is cleared only after the new service + is installed and started; an install that fails after the stop keeps it for the next + `ocx stop`. A failed claim sends no child stop. +- With Part A `absent`, install preparation skips Part B, and the tracked stop can return + `"none"` after its orphan-discovery cap while an unresponsive proxy still holds the port. The + candidate set is resolved before cleanup and every endpoint must be definitively dead before + the install callback runs; live or unknown throws like a Part B failure. +- The updater (`src/update/job-windows-restart.ts`) uses `guardedStopWindowsChecked` for both + backends, with the service mutation gate as the authorizer. Each of the + three call sites in `src/update/job.ts` aborts the restart unless the outcome is stopped or absent. + +## Who may run Part A + +The binding rule excludes the stopping process but not its ancestors. A stop that runs beneath a +wrapper carrying this home's launcher or service-script path would end its own supervision chain, +so `runWindowsServiceStopPartA` refuses such a stop with nothing changed (`unknown`, "runs under +this home's service wrapper") by checking the bind snapshot's ancestry of the current PID. Two +callers opt out on purpose: `/api/stop` runs inside the wrapped proxy, and the updater's worker +descends from the service-run proxy; both exist to end that wrapper, and ending an ancestor never +ends the stopper. Every interactive caller (`ocx stop`, `ocx uninstall`, `ocx service` +commands, install and switch) refuses. + +## ARM64 + +No ARM64 device run exists, so ARM64 ships the strict variant (050). +`terminateWindowsProcessBound` falls back to the PowerShell script whenever the native API is +missing, so the strict gate sits in the caller's adapter: `windowsStopTerminate` answers +`unavailable` whenever `windowsProcessNative()` is null (ARM64, or an x64 whose DLL load failed) +and never reaches the fallback, which leaves Part A `unknown` / `evidence-unreadable` wherever a +terminate would be needed. `WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED` turns the fallback on once +the 040 ARM64 device case has run. + ## Tests `tests/windows/windows-stop-binding.test.ts`, `tests/windows/windows-checked-stop.test.ts`, @@ -100,3 +264,14 @@ window. hermetic: injected deps, a virtual clock and simulated platforms, and no real `schtasks`, PowerShell or FFI call. The FFI primitive itself is exercised only by the cases gated on real Windows x64. + +The callers are covered by `tests/windows/windows-checked-stop-live.test.ts` (adapters, ARM64 +strict, whole-millisecond timeouts, the ancestry check, the bounded port sampler and the tri-state +listener identity), `tests/windows/windows-service-stop-wiring.test.ts` +(the real Part A through the wiring: the own-wrapper refusal, the cap, admission, the deadline's +two origins and the receipt record), `tests/windows/windows-stop-callers.test.ts` (the route, the +service commands, the guarded order, `bound-stray`, the orphan-discovery cap and the bind on the +evidence snapshot), `tests/windows/windows-guarded-manager.test.ts` (the not-running promotion +behind its registration proof) and `tests/windows/windows-stop-ocx-stop.test.ts` (plain `ocx stop` +through the real CLI with the platform simulated). None of them reaches `schtasks`, PowerShell or +FFI. diff --git a/structure/runtime.md b/structure/runtime.md index 98b1126ef9f..0b5b2c7c5e4 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -59,7 +59,7 @@ The [CLI management contract](cli-management.md#cli-readiness-diagnostics) owns `ocx resolve` (`src/cli/resolve.ts`) is the machine surface a desktop shell asks instead of resolving the config home, the port, and liveness itself: the home comes from `src/config/paths.ts`, the effective port is the live listener's when the identity-checked `findLiveProxy` answers and the configured `config.port ?? 10100` otherwise, and the liveness verdict is that same module's output (pid, runtime-versus-config provenance, version, role). Config reads go through `readConfigDiagnostics`, not `loadConfig`: a missing file is defaults, but an invalid file exits 1 instead of being repaired to defaults, because a defaulted port is a guess the caller must refuse. Liveness is three-valued: when `findLiveProxy` returns null, resolve re-asks the endpoints with `endpointsToProve` and `everyEndpointProvenDown`, using the in-process `probeEndpointLiveness` from `src/server/proxy-liveness.ts`. Only unanimous definitive "dead" becomes `absent-proven`; a timeout or reset stays unknown, exits 1 and never authorises a start. The transient exclusive-bind fallback in `src/update/proxy-liveness-probe.mjs` belongs only to the npm/Bun updater. Resolve stays in-process because a compiled standalone binary cannot run a JavaScript probe with `execPath -e`. Discovery borrows `START_OWNERSHIP_LIVENESS`, the start path's ownership budget — the verdict feeds the shell's launch decision, so the cost of a false "nobody listening" is the duplicate proxy (#5004). The verb is in `skipsCodexShimAutoRestore`, so a read-only lookup never triggers a shim repair. Arguments are pre-parsed in `src/cli/root.ts` and exit 64 before any preflight side effect, the same ordering `ocx ready` obeys. `ocx-resolve/1` also reports `ownership` (`none`, `owned`, or `unknown`) and `takeover` (`supported` with protocol version, minimum CLI version and compatibility token, or `blocked` with a reason). A known liveness verdict still exits 0 when ownership is unknown, but takeover is blocked. An unreadable second service-state read blocks takeover instead of becoming an absent registration. `src/service/managing-cli.ts` observes the registered and selected PATH CLIs, including Windows PATHEXT order and file validation, before compatibility is offered. A `--version` probe is retried once on Windows — one transient failure must not read as an unobservable CLI — and a Windows `.CMD`/`.BAT` shim is invoked through a single verbatim-quoted `cmd /c` line, so a spaced executable path and spaced arguments both survive cmd's re-parse. `src/cli/route-policy.ts` distinguishes a missing profile (exit 4) from malformed `show` arguments (exit 2); an upstream failure propagates through the management error mapping. -`ocx stop --json` is a reporting layer over the unchanged stop path. `src/cli/index.ts` threads a `StopRunRecord` through the existing receipt, drain, respawn-verification and restore flow, and `src/cli/stop-report.ts` maps the recorded facts plus the signals that already pick the exit code into one versioned document (`schema: "ocx-stop/1"`). With `--json` the human lines print on stderr and stdout carries only that document; exit codes 0/1/79/80/81 cross the process boundary unchanged. The desktop's opt-in guarded stop adds exact approved PID, endpoint, home, CLI version and compatibility-token expectations; plain `ocx stop` retains its ordinary behavior. Under the ownership mutation lease, `src/cli/stop-approval.ts` re-resolves them and `src/service/guarded-manager-target.ts` binds any installed manager's process tree to the approved PID before a stop action. The same guard re-reads `ocx.pid` and `runtime-port.json` through `src/config/tracked-runtime-inspection.ts`, a single-pass read-only probe that walks the same checks `readPid()` and `readRuntimePort(pid)` perform and reports which one failed, answering the recorded process's start-identity question through the same cache `readPid()` fills so it cannot disagree with the admission it diagnoses. A pre-action mismatch returns `approval-changed` without stopping, and the summary carries `detail` naming the single guard fact that failed — live PID/port/hostname/CLI drift, an unverifiable ownership claim, a blocked takeover, the compatibility fingerprint, or the tracked-record cause — an absent, denied, unreadable or invalid record, a dead, foreign or unverifiable recorded process, or a record naming a different PID — so a transient re-probe can be told from genuine drift in the caller's log. The tracked-record causes are fixed strings: denied is reported only on an observed EACCES/EPERM, an unreadable command line stays unverifiable rather than attributed to an account or elevation, and no probed path, command line, stderr or token reaches the detail. Immediately before any manager stop or direct PID signal, the CLI rechecks the same manager identity or proven absence; a change also returns `approval-changed`. A non-OpenCodex process replacing the OS job in the remaining instant before the manager command is a residual same-user risk outside the ownership lease. The guarded path stops the bound manager, or the approved PID directly when managers are proven absent, then uses a five-second shared deadline to verify PID exit, port availability and definitive endpoint absence. Only after settlement does a read-only manager probe require launchd `not-loaded` in both domains or systemd `inactive` with `MainPID=0`; the state is checked again before a success summary. Timeout, active or unreadable manager state returns terminal `manager-still-active`, even if the endpoint briefly refuses. Windows managers bind through `src/service/windows-guarded-manager.ts`: a Task Scheduler backend must prove a recognized OpenCodex registration against the bounded current-user SID/name lookup, a running task, and a wrapper ancestor carrying this home's canonical paths — presence and running state are read root-folder-scoped because every `/tn` operation resolves a bare name there and nowhere else, so a same-named task inside a task folder is a different registration neither the probe nor the stop can reach; the registered definition itself is read only on the running path, and a state or registration read that cannot answer re-probes presence so a task deleted between probes — which can respawn nothing — reads absent while a merely unreadable one stays unknown; an unreadable current identity leaves the manager unknown; a WinSW backend must prove this installation's binary and SCM service PID ancestry; either marks `childNeedsSeparateStop`, so the approved PID receives its own graceful signal only after a successful manager stop because `schtasks /end` does not reliably cascade to the wrapper's child (#764); a failed or unknown manager stop leaves the child untouched. Post-stop Windows inactivity means the task not running, no surviving wrapper process, and no started WinSW — never mere registration presence; a running-state read that cannot answer re-probes presence the same way, so a task deleted between probes reports inactive while a persisting one stays unknown. Every Windows manager command a verdict or stop depends on — schtasks, sc.exe, WinSW verbs — runs under a bounded `execFileSync` timeout so a wedged Task Scheduler or Service Control Manager cannot hang the guarded stop; a killed command classifies through the same fail-closed paths a nonzero exit would (the interactive `install /p` password prompt is the one deliberate exception). A `wscript.exe`, `cscript.exe` or `cmd.exe` whose command line CIM cannot read makes the state unprovable only when it is tied to the proxy: pre-stop, as an ancestor of the approved PID; post-stop, as the former bound manager process or a descendant of it. Unrelated unreadable shells, which a non-admin query returns for other users' and elevated processes, are not evidence either way. An unprovable Windows manager still blocks guarded takeover. The token checks snapshot consistency, not whether a person approved the desktop prompt. `src/cli/stop-restore.ts` owns the shared Codex/Grok restoration and its history-only, history-deferred, and other failure categories. The checked Windows stop that will replace the best-effort wrapper sweep and bind a stray wrapper by lineage and creation time is described in [Windows checked service stop](ops/windows-checked-stop.md); no stop path calls it yet. +`ocx stop --json` is a reporting layer over the unchanged stop path. `src/cli/index.ts` threads a `StopRunRecord` through the existing receipt, drain, respawn-verification and restore flow, and `src/cli/stop-report.ts` maps the recorded facts plus the signals that already pick the exit code into one versioned document (`schema: "ocx-stop/1"`). With `--json` the human lines print on stderr and stdout carries only that document; exit codes 0/1/79/80/81 cross the process boundary unchanged. The desktop's opt-in guarded stop adds exact approved PID, endpoint, home, CLI version and compatibility-token expectations; plain `ocx stop` retains its ordinary behavior. Under the ownership mutation lease, `src/cli/stop-approval.ts` re-resolves them and `src/service/guarded-manager-target.ts` binds any installed manager's process tree to the approved PID before a stop action. The same guard re-reads `ocx.pid` and `runtime-port.json` through `src/config/tracked-runtime-inspection.ts`, a single-pass read-only probe that walks the same checks `readPid()` and `readRuntimePort(pid)` perform and reports which one failed, answering the recorded process's start-identity question through the same cache `readPid()` fills so it cannot disagree with the admission it diagnoses. A pre-action mismatch returns `approval-changed` without stopping, and the summary carries `detail` naming the single guard fact that failed — live PID/port/hostname/CLI drift, an unverifiable ownership claim, a blocked takeover, the compatibility fingerprint, or the tracked-record cause — an absent, denied, unreadable or invalid record, a dead, foreign or unverifiable recorded process, or a record naming a different PID — so a transient re-probe can be told from genuine drift in the caller's log. The tracked-record causes are fixed strings: denied is reported only on an observed EACCES/EPERM, an unreadable command line stays unverifiable rather than attributed to an account or elevation, and no probed path, command line, stderr or token reaches the detail. Immediately before any manager stop or direct PID signal, the CLI rechecks the same manager identity or proven absence; a change also returns `approval-changed`. A non-OpenCodex process replacing the OS job in the remaining instant before the manager command is a residual same-user risk outside the ownership lease. The guarded path stops the bound manager, or the approved PID directly when managers are proven absent, then uses a five-second shared deadline to verify PID exit, port availability and definitive endpoint absence. Only after settlement does a read-only manager probe require launchd `not-loaded` in both domains or systemd `inactive` with `MainPID=0`; the state is checked again before a success summary. Timeout, active or unreadable manager state returns terminal `manager-still-active`, even if the endpoint briefly refuses. Windows managers bind through `src/service/windows-guarded-manager.ts`: a Task Scheduler backend must prove a recognized OpenCodex registration against the bounded current-user SID/name lookup, a running task, and a wrapper ancestor carrying this home's canonical paths — presence and running state are read root-folder-scoped because every `/tn` operation resolves a bare name there and nowhere else, so a same-named task inside a task folder is a different registration neither the probe nor the stop can reach; the registered definition itself is read only on the running path, and a state or registration read that cannot answer re-probes presence so a task deleted between probes — which can respawn nothing — reads absent while a merely unreadable one stays unknown; an unreadable current identity leaves the manager unknown; a WinSW backend must prove this installation's binary and SCM service PID ancestry; either marks `childNeedsSeparateStop`, so the approved PID receives its own graceful signal only after a successful manager stop because `schtasks /end` does not reliably cascade to the wrapper's child (#764); a failed or unknown manager stop leaves the child untouched. Post-stop Windows inactivity means the task not running, no surviving wrapper process, and no started WinSW — never mere registration presence; a running-state read that cannot answer re-probes presence the same way, so a task deleted between probes reports inactive while a persisting one stays unknown. Every Windows manager command a verdict or stop depends on — schtasks, sc.exe, WinSW verbs — runs under a bounded `execFileSync` timeout so a wedged Task Scheduler or Service Control Manager cannot hang the guarded stop; a killed command classifies through the same fail-closed paths a nonzero exit would (the interactive `install /p` password prompt is the one deliberate exception). A `wscript.exe`, `cscript.exe` or `cmd.exe` whose command line CIM cannot read makes the state unprovable only when it is tied to the proxy: pre-stop, as an ancestor of the approved PID; post-stop, as the former bound manager process or a descendant of it. Unrelated unreadable shells, which a non-admin query returns for other users' and elevated processes, are not evidence either way. An unprovable Windows manager still blocks guarded takeover. The token checks snapshot consistency, not whether a person approved the desktop prompt. `src/cli/stop-restore.ts` owns the shared Codex/Grok restoration and its history-only, history-deferred, and other failure categories. Every Windows stop path now runs the checked stop described in [Windows checked service stop](ops/windows-checked-stop.md) (#7771): on the guarded path Part A ends the bound manager and the wrappers bound by lineage and creation time before the approved PID gets its graceful signal, a registered task that is not running but left wrappers the binding rule ties to the approved PID is the provable `bound-stray` manager (revalidation compares its wrapper set; one the rule cannot bind stays unknown and the `approval-changed` summary carries `serviceStopReason` `unbound-wrapper` or `evidence-unreadable`), and Part B confirms the approved endpoint after settle. Plain `ocx stop` sends no `/api/stop` POST while Part A cannot prove the manager stopped, and runs Part B instead of the respawn re-poll after it did. Both paths honour `--deadline-ms` / `OCX_STOP_DEADLINE_MS` on Windows and add the optional `serviceStopReason` and `portHeldByForeign` fields to `ocx-stop/1` without a new outcome or proxy value. `ocx service status --json` (`src/service/status-json.ts`) is the passive machine form of the service report, read by peers such as GOAT's device inventory. `src/service/cli.ts` accepts `--json` for `status` only (`planServiceCommand` refuses it on every other verb before any installation probe) and answers it before `platformOps()` resolves, so no install, start, stop, repair or migration path is reachable; `skipsCodexShimAutoRestore` exempts `service status --json` so the CLI preflight repairs no Codex shim first, the collection runs inside `runPassiveRead` so the serving probe's config load neither hardens modes nor backs up a malformed file, and `--json` on any other verb is refused before the Desktop check and the Windows mutation lock; an unsupported platform or Docker gets a document instead of the text path's exit. The document reuses exactly the reads the text report makes — `diagnoseService()` (launchd `print`, systemd observation verbs, Task Scheduler `/query`, SCM/WinSW status) and, only when a registration exists, the 1.5-second loopback `confirmServiceServing` probe — plus the ownership-lease and Desktop login lines. `running` is the manager's verdict (the value `startup.serviceRunning` carries in `ocx status --json`); `serving` reports separately whether a proxy answers on the installed port. `backend` is `launchd`, `systemd`, `schtasks`, `winsw`, or `none` when nothing is registered. The systemd branch of `diagnoseService` derives through the pure `deriveSystemdServiceDiagnostic`, matching the launchd and Windows derivations. `tests/service/service-status-json.test.ts` snapshots each platform branch and records every subprocess entry point on darwin, linux and win32 against a read-only allowlist. `ocx status --json` carries an additive `install` verdict (`src/cli/status-install.ts`) mapped from the updater's own `detectInstallOwnership()`: `method` is `bun`, `npm`, `pnpm`, `mise`, `source`, `standalone` or `unknown` (detector failure), and `evidence.basis` names the package layout, the mise backend record (tool alias and backend only) or the standalone binary without printing a path (`tests/cli/status-install-method.test.ts`). `ocx service restart` reads the pid of the proxy answering on the installed port before it restarts (`servingRuntimePid`, `src/service/health.ts`) and passes it to the serving check as `replacing`. While that pid is set, only a proxy reporting a different pid counts as serving: a Windows wrapper that survived `schtasks /end` keeps its old child answering healthz, and the restart used to report "restarted and serving" over it (#7771). If the replaced pid is still the one answering at the deadline, the command prints that the previous runtime is still serving and the replacement was not confirmed, prints no checkmark, and exits 1; a restart that threw names either the surviving pid (kept, not replaced) or the new pid. An answer that reports no pid cannot be shown to be the old runtime and counts as serving. `ocx service repair` keeps the plain any-proxy check, because a repair may preserve a healthy runtime by design. `tests/service/service-restart-replacement.test.ts` covers each case. diff --git a/tests/cli/cli-dispatch.test.ts b/tests/cli/cli-dispatch.test.ts index 87aadde1220..f0a822910ae 100644 --- a/tests/cli/cli-dispatch.test.ts +++ b/tests/cli/cli-dispatch.test.ts @@ -643,7 +643,7 @@ describe("a sibling start leaves shared client routing to the live owner", () => const readAt = stop.indexOf("const siblingOfPort = readRuntimePort()?.siblingOfPort;"); expect(readAt).toBeGreaterThan(-1); // Read before the first stop can remove the record. - expect(readAt).toBeLessThan(stop.indexOf("stopServiceIfInstalledDetailed()")); + expect(readAt).toBeLessThan(stop.indexOf("stopServiceForOcxStop(")); expect(stop).toContain("if (teardownNonce || stoppingSibling) return;"); // #7563 adds a connected client to the same gate; the sibling term is unchanged. expect(stop).toContain("const restoreBlocked = ownershipBlocked || inheritedBlocks || nativeRestoreHandledByProxy || stoppingSibling || connectedClientSkip !== null;"); @@ -656,10 +656,12 @@ describe("a sibling start leaves shared client routing to the live owner", () => // it failed the ownership check from the sibling's home: exit 1, and after a hard kill the // stale sibling records were never purged. // #7423: the non-sibling branch claims an update-handover receipt first; the sibling term is unchanged. - expect(stop).toContain(': stoppingSibling || connectedClientAlreadyStopped ? "absent" : (claimHandoverBeforeManager(), stopServiceIfInstalledDetailed());'); + expect(stop).toContain(': stoppingSibling || connectedClientAlreadyStopped ? noServiceStop() : (claimHandoverBeforeManager(), await stopServiceForOcxStop({ receiptNonce: () => teardownNonce, inheritedReceipts: inheritedTeardowns.length }));'); // The only other caller is the guarded desktop step, which reaches the manager only when it - // provably owns the approved pid; a sibling's pid never is. - expect(stop.split("stopServiceIfInstalledDetailed()").length - 1).toBe(2); + // provably owns the approved pid; a sibling's pid never is. Both go through the #7771 wiring. + expect(stop).not.toContain("stopServiceIfInstalledDetailed("); + expect(stop.split("stopServiceForOcxStop(").length - 1).toBe(1); + expect(stop.split("stopGuardedManager(").length - 1).toBe(1); }); test("ocx stop of a hard-killed sibling never stops the live owner discovery falls back to", () => { @@ -676,7 +678,7 @@ describe("a sibling start leaves shared client routing to the live owner", () => // Wiring: the orphan path asks before it may stop anything it found, and reports success. const stop = slice("async function handleStopUnlocked(", "async function handleUninstall("); - const findAt = stop.indexOf("const live = await findLiveProxy({ acceptPackageTreeFenced: true });"); + const findAt = stop.indexOf("const live = await findLiveProxy({ acceptPackageTreeFenced: true, ...orphanDiscoveryCap() });"); const askAt = stop.indexOf("if (siblingStopFoundOwner(siblingOfPort, live)) {"); const attestAt = stop.indexOf('} else if (live?.pid && (await proveLiveProxyOwnedByHome(live)) !== "proven") {'); expect(findAt).toBeGreaterThan(-1); diff --git a/tests/cli/cli-ready.test.ts b/tests/cli/cli-ready.test.ts index 0c739b47020..92c2fb84466 100644 --- a/tests/cli/cli-ready.test.ts +++ b/tests/cli/cli-ready.test.ts @@ -872,13 +872,15 @@ describe("handleStart OCX_SERVICE exit guard (source-level)", () => { expect(nonService, "non-service refusal keeps the exit 1 conflict error").not.toBeNull(); }); - test("service.ts teardown kills surviving wrapper processes on stop", () => { + test("service.ts teardown ends surviving wrapper processes on stop through the checked stop", () => { const serviceSource = readFileSync(repoPath("src/service/orchestration.ts"), "utf8"); - expect(serviceSource).toMatch(/killWindowsServiceWrapperProcesses/); // The boolean `stopServiceIfInstalled` is gone — it collapsed a live manager into the // same false as "not installed" (#3008). The stop itself is the detailed function. - const callSite = serviceSource.match(/stopServiceIfInstalledDetailed[\s\S]{0,1600}?killWindowsServiceWrapperProcesses\(\)/); - expect(callSite, "wrapper kill must run during stopServiceIfInstalledDetailed").not.toBeNull(); + // #7771: on Windows it is Part A of the checked stop, which ends the bound wrappers and + // waits for them instead of the best-effort path-token sweep. + const callSite = serviceSource.match(/stopServiceIfInstalledReport[\s\S]{0,2400}?runWindowsServiceStopPartA\(request\)/); + expect(callSite, "Part A must run during the service stop on Windows").not.toBeNull(); + expect(serviceSource).not.toMatch(/killWindowsServiceWrapperProcesses/); }); test("wrapper kill matches the canonical paths of THIS installation, not bare filenames", () => { @@ -886,15 +888,15 @@ describe("handleStart OCX_SERVICE exit guard (source-level)", () => { // wrapper from another OpenCodex home (or any process whose command line // merely contains the name). The kill must target the exact canonical // paths windowsServiceScriptPath()/windowsLauncherVbsPath() produce. - const serviceSource = readFileSync(repoPath("src/service/windows-ops.ts"), "utf8"); - expect(serviceSource).toMatch(/windowsServiceScriptPath\(\)/); - expect(serviceSource).toMatch(/windowsLauncherVbsPath\(\)/); - const killBody = serviceSource.match(/function killWindowsServiceWrapperProcesses\(\)[\s\S]*?\n}/); - expect(killBody, "killWindowsServiceWrapperProcesses body must exist").not.toBeNull(); - expect(killBody![0]).toContain("windowsServiceScriptPath()"); - expect(killBody![0]).toContain("windowsLauncherVbsPath()"); + // Since #7771 the stop binds wrappers through the checked stop's binding rule, whose + // path set is exactly these canonical paths (tests/windows/windows-stop-binding.test.ts + // covers other homes, substrings and image names). + const live = readFileSync(repoPath("src/service/windows-checked-stop-live.ts"), "utf8"); + const paths = live.match(/export function windowsStopCanonicalPaths\(\)[\s\S]*?\n}/); + expect(paths, "windowsStopCanonicalPaths body must exist").not.toBeNull(); + expect(paths![0]).toContain("return [windowsLauncherVbsPath(), windowsServiceScriptPath()];"); // Bare wrapper filenames must NOT be the match target. - expect(killBody![0]).not.toMatch(/\$pats = @\('opencodex-service\.cmd'\)/); + expect(live).not.toMatch(/\$pats = @\('opencodex-service\.cmd'\)/); }); test("wrapper kill requires the canonical path as a complete command-line token", () => { diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 8cea81ac4cf..5f3b18db9e6 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -1,6 +1,7 @@ import { afterEach, describe, expect, test } from "bun:test"; import { setUninstallServiceHooksForTests, + uninstallServiceDetailed, uninstallServiceIfInstalled, } from "../../src/service"; import { pathToFileURL } from "node:url"; @@ -11,13 +12,14 @@ import { dirname, join } from "node:path"; import { removeOwnedConfigAfterDesktopCleanup, type UninstallClientStateDeps } from "../../src/cli/uninstall-client-state"; import { cleanupOwnedIntegrationsBeforeUninstall } from "../../src/cli/uninstall-integrations"; import { assertClientLifecycleHeld, withClientLifecycle, withClientLifecycleSync, type ClientLifecycleHeld } from "../../src/client/lifecycle-lock"; -import type { UninstallObservation } from "../../src/cli/uninstall-plan"; +import { captureUninstallEndpointProof, type UninstallObservation } from "../../src/cli/uninstall-plan"; import type { DesktopDisconnectReceipt } from "../../src/claude/desktop-remote-store"; import type { ExportModel } from "../../src/clients/config-export"; import { INTEGRATION_CLIENTS } from "../../src/integrations/registry"; import { createIntegrationStateStore } from "../../src/integrations/store"; import { applyIntegration, disableIntegrationCoordinated } from "../../src/integrations/writer"; import type { OcxConfig } from "../../src/types"; +import { stopUninstallProxy } from "../../src/cli/uninstall-stop"; const root = pathToFileURL(repoRoot() + "/"); @@ -51,7 +53,7 @@ describe("full uninstall command", () => { expect(exit).toBeGreaterThan(refusal); // Nothing before the check: every step that stops, removes or rewrites anything comes after the exit. expect(body.slice(0, refusal).split("\n").filter(line => line.trim() && !line.trim().startsWith("//"))).toEqual(["async function handleUninstall() {"]); - for (const step of ["await runStep(", "stopServiceIfInstalledDetailed", "uninstallServiceDetailed", "restoreNativeCodexAsync", "revertSystemEnv", "uninstallCodexShimAsync", "removeOwnedConfigAfterDesktopCleanup"]) { + for (const step of ["await runStep(", "stopServiceForUninstall", "uninstallServiceDetailed", "restoreNativeCodexAsync", "revertSystemEnv", "uninstallCodexShimAsync", "removeOwnedConfigAfterDesktopCleanup"]) { expect(body.indexOf(step)).toBeGreaterThan(exit); } }); @@ -89,6 +91,7 @@ describe("full uninstall command", () => { probeWindowsTask: () => ({ status: "present" }), uninstallWindowsTask: () => { calls.push("scheduler"); }, nativeStatus: () => "started", + nativeOwnership: () => "owned", uninstallNative: () => { calls.push("native"); throw new Error("native removal failed"); @@ -101,6 +104,91 @@ describe("full uninstall command", () => { expect(stateRemovals).toBe(0); }); + test("a foreign WinSW registration is left alone and an unreadable one refuses removal", () => { + const calls: string[] = []; + let stateRemovals = 0; + const hooks = (ownership: "owned" | "foreign" | "unknown") => ({ + platform: "win32" as const, + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "absent" }) as never, + uninstallWindowsTask: () => { calls.push("scheduler"); }, + nativeStatus: () => "started" as const, + nativeOwnership: () => ownership, + uninstallNative: () => { calls.push("native"); }, + removeInstallState: () => { stateRemovals++; }, + }); + + // `sc stop/delete opencodex-proxy-native` names a fixed ID, so a registration under it + // is not proof it is this home's. Another installation's WinSW service must survive. + setUninstallServiceHooksForTests(hooks("foreign")); + expect(uninstallServiceDetailed()).toBe("absent"); + expect(calls).toEqual([]); + expect(stateRemovals).toBe(0); + + // Unreadable ownership is not absence: refuse and keep install state for a retry. + setUninstallServiceHooksForTests(hooks("unknown")); + expect(() => uninstallServiceDetailed()).toThrow("could not be verified"); + expect(calls).toEqual([]); + expect(stateRemovals).toBe(0); + + setUninstallServiceHooksForTests(hooks("owned")); + expect(uninstallServiceDetailed()).toBe("removed"); + expect(calls).toEqual(["native"]); + expect(stateRemovals).toBe(1); + }); + + test("an unreadable WinSW ownership removes nothing, scheduler task included", () => { + // #7771 round 9: the scheduler task was deleted before the ownership probe ran, so an + // `sc qc` that could not be read left a half-removed install: task gone, native + // registration still there, no retry that can put the task back. Every probe now runs + // before the first destructive step. + const calls: string[] = []; + let stateRemovals = 0; + setUninstallServiceHooksForTests({ + platform: "win32", + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "present" }), + uninstallWindowsTask: () => { calls.push("uninstall:task"); }, + nativeStatus: () => "stopped", + nativeOwnership: () => "unknown", + uninstallNative: () => { calls.push("uninstall:native"); }, + removeInstallState: () => { stateRemovals++; }, + }); + expect(() => uninstallServiceDetailed()).toThrow("could not be verified"); + expect(calls).toEqual([]); + expect(stateRemovals).toBe(0); + + // An unreadable native status is the same refusal, and it also precedes the task delete. + calls.length = 0; + setUninstallServiceHooksForTests({ + platform: "win32", + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "present" }), + uninstallWindowsTask: () => { calls.push("uninstall:task"); }, + nativeStatus: () => "unknown", + nativeOwnership: () => "owned", + uninstallNative: () => { calls.push("uninstall:native"); }, + removeInstallState: () => {}, + }); + expect(() => uninstallServiceDetailed()).toThrow("could not be verified"); + expect(calls).toEqual([]); + + // Both probes answer: the task and the owned service are removed. + calls.length = 0; + setUninstallServiceHooksForTests({ + platform: "win32", + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "present" }), + uninstallWindowsTask: () => { calls.push("uninstall:task"); }, + nativeStatus: () => "started", + nativeOwnership: () => "owned", + uninstallNative: () => { calls.push("uninstall:native"); }, + removeInstallState: () => { calls.push("state"); }, + }); + expect(uninstallServiceDetailed()).toBe("removed"); + expect(calls).toEqual(["uninstall:task", "uninstall:native", "state"]); + }); + test("scheduler removal failure propagates without deleting install state", () => { let stateRemovals = 0; setUninstallServiceHooksForTests({ @@ -124,11 +212,36 @@ describe("full uninstall command", () => { expect(uninstallBody).toContain('runStep("service stopped"'); expect(uninstallBody).toContain('runStep("proxy stopped"'); expect(uninstallBody).toContain('runStep("service removed"'); - expect(uninstallBody).toContain("await stopProxy(pid);"); - expect(uninstallBody).toContain("uninstallServiceDetailed()"); + // #7771 round 7: the stop itself lives in `uninstall-stop`, which the step calls. + expect(uninstallBody).toContain("await stopUninstallProxy({"); + // The removal runs through the gate that keeps a Part A failure from deleting the + // registration; the gate's rule is exercised in the removal test below. + expect(uninstallBody).toContain("uninstallServiceRemoval(observed, uninstallServiceDetailed)"); expect(uninstallBody.indexOf('runStep("service stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("proxy stopped"')); expect(uninstallBody.indexOf('runStep("proxy stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("service removed"')); - expect(uninstallBody.indexOf("await stopProxy(pid);")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); + expect(uninstallBody.indexOf("await stopUninstallProxy({")).toBeLessThan(uninstallBody.indexOf("uninstallServiceRemoval(observed, uninstallServiceDetailed)")); + }); + + test("Windows uninstall leaves the port wait to Part B instead of stopProxy's legacy 15 s", async () => { + const cli = await readText("src/cli/index.ts"); + const uninstallBody = cli.slice(cli.indexOf("async function handleUninstall()"), cli.indexOf("type HealthCheck")); + // #7771: after a stopped Part A on Windows the respawn-window step IS Part B, which + // owns the port wait; running stopProxy's 15 s waitForStoppedPort on top of it spends + // the stop budget twice. Both tracked and orphan stops pass the port-wait predicate + // itself (not `windowsPartBDue`, which additionally gates the legacy re-poll): the + // uninstall step must keep asking the question it means, so a later change to the + // snapshotless rule cannot silently bring the 15 s wait back. + expect(uninstallBody).toContain("portWaitOwned: windowsPartBOwnsPortWait,"); + const stopHelper = await readText("src/cli/uninstall-stop.ts"); + // The helper asks the port-wait question itself rather than re-deciding it. + expect(stopHelper).toContain("skipPortWait: io.portWaitOwned(),"); + expect(stopHelper).not.toContain("windowsPartBDue"); + const checked = await readText("src/cli/stop-windows-checked.ts"); + // The predicate is about the port wait only; the legacy re-poll key stays snapshotless. + const portWait = checked.slice(checked.indexOf("export function windowsPartBOwnsPortWait")); + expect(portWait).toContain('return windows() && state.result?.outcome === "stopped";'); + const partBDue = checked.slice(checked.indexOf("export function windowsPartBDue"), checked.indexOf("export function windowsPartBOwnsPortWait")); + expect(partBDue).toContain("state.snapshotless"); }); test("restore forwards the explicit provider-table removal flag and warns before mutation", async () => { @@ -150,24 +263,44 @@ describe("uninstall gates shared teardown on a proven service stop", () => { serviceRemoval: "removed" as const, respawnWindowVerified: false, }; - expect(sharedTeardownAuthorized(base)).toBe(true); - expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent" })).toBe(true); - expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "absent" })).toBe(true); - // Removing the registration does not prove an already-running wrapper died; killing it - // is best-effort (#764), so the restart window has to be polled first. - expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable" })).toBe(false); - expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable", respawnWindowVerified: true })).toBe(true); - // A manager that refused to stop, or one we could not read, may still be running. - expect(sharedTeardownAuthorized({ ...base, serviceStop: "failed" })).toBe(false); - expect(sharedTeardownAuthorized({ ...base, serviceStop: "state-unknown" })).toBe(false); - // The step itself threw: we know nothing. - expect(sharedTeardownAuthorized({ ...base, serviceStop: null })).toBe(false); - // A proxy that could not be PROVEN down — a live orphan with no pid, or an endpoint - // that would not answer — blocks it. A findLiveProxy miss is not proof. - expect(sharedTeardownAuthorized({ ...base, proxyProvenDown: false })).toBe(false); - // A removal that failed used to look like absence on darwin and linux. - expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "failed" })).toBe(false); - expect(sharedTeardownAuthorized({ ...base, serviceRemoval: null })).toBe(false); + // The platform is explicit in every expectation below. It is an input to the rule, not + // an ambient fact: a clean Windows stop is Part B of the checked stop (#7771) and owes a + // verified restart window, while a clean stop elsewhere owes nothing more. Asserting on + // the default platform made this test's verdict depend on the machine it ran on. + for (const platform of ["linux", "darwin"] as const) { + expect(sharedTeardownAuthorized(base, platform)).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent" }, platform)).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "absent" }, platform)).toBe(true); + // Removing the registration does not prove an already-running wrapper died; killing it + // is best-effort (#764), so the restart window has to be polled first. + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable" }, platform)).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable", respawnWindowVerified: true }, platform)).toBe(true); + // A manager that refused to stop, or one we could not read, may still be running. + expect(sharedTeardownAuthorized({ ...base, serviceStop: "failed" }, platform)).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "state-unknown" }, platform)).toBe(false); + // The step itself threw: we know nothing. + expect(sharedTeardownAuthorized({ ...base, serviceStop: null }, platform)).toBe(false); + // A proxy that could not be PROVEN down — a live orphan with no pid, or an endpoint + // that would not answer — blocks it. A findLiveProxy miss is not proof. + expect(sharedTeardownAuthorized({ ...base, proxyProvenDown: false }, platform)).toBe(false); + // A removal that failed used to look like absence on darwin and linux. + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "failed" }, platform)).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: null }, platform)).toBe(false); + } + // On Windows a clean stop is only half the proof: Part B has to have shown the port free + // and no wrapper back. Without that window the same input is refused. + expect(sharedTeardownAuthorized(base, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, respawnWindowVerified: true }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent" }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent", respawnWindowVerified: true }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable" }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable", respawnWindowVerified: true }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "failed" }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "state-unknown" }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: null }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, proxyProvenDown: false, respawnWindowVerified: true }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "failed", respawnWindowVerified: true }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: null, respawnWindowVerified: true }, "win32")).toBe(false); }); test("a removal failure is distinguishable from nothing being installed", async () => { @@ -191,17 +324,16 @@ describe("uninstall gates shared teardown on a proven service stop", () => { }); test("a live orphan with no pid file blocks the teardown", async () => { - const cli = await readText("src/cli/index.ts"); - const at = cli.indexOf("async function handleUninstall("); - const fn = cli.slice(at, at + 9000); + const helper = await readText("src/cli/uninstall-stop.ts"); + const fn = helper.slice(helper.indexOf("export async function stopUninstallProxy(")); // A missing pid file is not proof that nothing is serving — the same discovery // `ocx stop` performs. Without it, uninstall restored shared config under a live proxy. - expect(fn).toContain("const live = await findLiveProxy();"); - expect(fn).toContain("observed.proxyProvenDown = await proxyEndpointProvenDown();"); + expect(fn).toContain("const live = await io.findLiveProxy();"); + expect(fn).toContain("const provenDown = await io.endpointProvenDown();"); expect(fn).toContain("no process id could be resolved for it"); // The orphan-with-no-pid branch THROWS, so `proxyProvenDown` stays false and the // authorization rule refuses the shared teardown. - const orphanBranch = fn.slice(fn.indexOf("const live = await findLiveProxy();"), fn.indexOf("const live = await findLiveProxy();") + 600); + const orphanBranch = fn.slice(fn.indexOf("const live = await io.findLiveProxy();"), fn.indexOf("const live = await io.findLiveProxy();") + 600); expect(orphanBranch).toContain("throw new Error("); // A findLiveProxy miss is not proof either: it goes through the tri-state probe first. expect(orphanBranch).toContain("could not be confirmed down either"); @@ -214,12 +346,45 @@ describe("uninstall gates shared teardown on a proven service stop", () => { return cli.slice(at, at + 9000); } + test("Windows removal is gated on a Part A that ended stopped or absent", async () => { + const { serviceRemovalAllowed } = await import("../../src/cli/uninstall-plan"); + for (const outcome of ["stopped", "absent"] as const) { + expect(serviceRemovalAllowed({ serviceStop: outcome }, "win32")).toBe(true); + } + // A Part A that failed, could not read the state, or left a respawnable wrapper still + // has a manager that can relaunch the proxy; `ocx service uninstall` refuses there, and + // deleting the registration would remove the evidence without stopping the wrapper. + for (const outcome of ["failed", "state-unknown", "stopped-respawnable", null] as const) { + expect(serviceRemovalAllowed({ serviceStop: outcome }, "win32")).toBe(false); + } + // Non-Windows removal is unchanged. + for (const platform of ["linux", "darwin"] as const) { + for (const outcome of ["stopped", "absent", "stopped-respawnable", "failed", "state-unknown", null] as const) { + expect(serviceRemovalAllowed({ serviceStop: outcome }, platform)).toBe(true); + } + } + const fn = await uninstallFn(); + const step = fn.slice(fn.indexOf('runStep("service removed"'), fn.indexOf('runStep("service removed"') + 800); + // The command body calls the gate; the rule itself is exercised above. + expect(step).toContain("uninstallServiceRemoval(observed, uninstallServiceDetailed)"); + // A refused gate throws before the removal runs, and the step records the failure. + const { uninstallServiceRemoval } = await import("../../src/cli/uninstall-plan"); + let removed = 0; + expect(() => uninstallServiceRemoval({ serviceStop: "failed" }, () => { removed++; return "removed"; }, "win32")) + .toThrow("could not be proven stopped"); + expect(removed).toBe(0); + expect(uninstallServiceRemoval({ serviceStop: "stopped" }, () => { removed++; return "removed"; }, "win32")).toBe("removed"); + expect(removed).toBe(1); + }); + test("the detailed outcome is consumed, not the boolean collapse", async () => { const fn = await uninstallFn(); // stopServiceIfInstalled returns false for "not installed", "refused to stop" and // "state could not be read" alike, so this step reported "not installed" for a manager // that might still be running (#3008). - expect(fn).toContain("stopServiceIfInstalledDetailed()"); + // #7771: through the stop wiring, which is stopServiceIfInstalledDetailed off Windows and + // Part A of the checked stop on Windows; both return the detailed outcome. + expect(fn).toContain("const outcome = await stopServiceForUninstall();"); expect(fn).not.toContain("stopServiceIfInstalled()"); expect(fn).toContain('if (outcome === "absent") return false;'); expect(fn).toContain('if (outcome === "failed")'); @@ -238,7 +403,9 @@ describe("uninstall gates shared teardown on a proven service stop", () => { expect(fn).toContain("respawnWindowVerified: false,"); expect(fn).toContain("observed.serviceStop = outcome;"); expect(fn).toContain("observed.serviceRemoval = outcome;"); - expect(fn).toContain('if (observed.serviceStop === "stopped-respawnable")'); + // A legacy respawnable stop and, on Windows, every stopped one (Part B, #7771). + expect(fn).toContain("if (respawnWindowRequired(observed)) {"); + expect(fn).toContain("await verifyUninstallStopWindow();"); expect(fn).toContain("observed.respawnWindowVerified = true;"); const gateAt = fn.indexOf("if (sharedTeardownAuthorized(observed)) {"); expect(gateAt).toBeLessThan(fn.indexOf("native Codex restored", gateAt)); @@ -303,9 +470,13 @@ describe("uninstall gates shared teardown on a proven service stop", () => { expect(windowStep).toContain("could not be confirmed down either"); expect(windowStep.indexOf("if (!await proxyEndpointProvenDown())")) .toBeLessThan(windowStep.indexOf("observed.respawnWindowVerified = true;")); - // And the proof itself asks every candidate. - expect(fn).toContain("endpointsToProve(readRuntimePort(), loadConfig())"); - expect(fn).toContain("everyEndpointProvenDownAsync(endpoints, probeEndpointLiveness)"); + // And the proof itself asks every candidate, captured before the proxy stop removes the + // runtime record; the helper requires every captured candidate to answer dead. + expect(fn).toContain("captureUninstallEndpointProof(() => endpointsToProve(readRuntimePort(), loadConfig()), probeEndpointLiveness)"); + expect(fn.indexOf("captureUninstallEndpointProof(")).toBeLessThan(fn.indexOf('runStep("proxy stopped"')); + const plan = await readText("src/cli/uninstall-plan.ts"); + const helper = plan.slice(plan.indexOf("export function captureUninstallEndpointProof(")); + expect(helper).toContain("everyEndpointProvenDownAsync(candidates, probe)"); }); const safeTeardown: UninstallObservation = { @@ -798,3 +969,272 @@ describe("uninstall restores recorded third-party integrations before deleting r } finally { rmSync(f.root, { recursive: true, force: true }); } }); }); + +describe("uninstall proxy stop defers the shared teardown through a receipt (#7771)", () => { + // Round 7 (A/B): the uninstall POST used to carry no deferral, so the proxy performed the + // shared client teardown itself BEFORE the Windows respawn window ran. If that window then + // failed, client config had already been restored under a wrapper that could relaunch the + // proxy. And when the service stop itself failed, the step could still fall through to + // taskkill /T /F. Both are decided here so they can be exercised without a machine. + const NONCE = "c".repeat(32); + const io = (over: Partial[0]> = {}): Parameters[0] => ({ + platform: "win32", + serviceStop: "stopped", + readPid: () => 4242, + runtimeEndpointFor: pid => (pid === 4242 ? { hostname: "127.0.0.1", port: 10100 } : null), + findLiveProxy: async () => null, + endpointProvenDown: async () => true, + claim: () => ({ nonce: NONCE }), + stop: async () => true, + removeTracked: () => {}, + portWaitOwned: () => true, + ...over, + }); + + test("Windows: the receipt is claimed before the child stop and its nonce travels on the POST", async () => { + const steps: string[] = []; + const seen: unknown[] = []; + const result = await stopUninstallProxy(io({ + claim: () => { steps.push("claim"); return { nonce: NONCE }; }, + stop: async (pid, stopIo) => { steps.push("stop:" + pid); seen.push(stopIo); return true; }, + removeTracked: pid => { steps.push("remove:" + pid); }, + })); + expect(steps).toEqual(["claim", "stop:4242", "remove:4242"]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }]); + expect(result).toEqual({ changed: true, provenDown: true, nonce: NONCE }); + }); + + test("an orphan stop carries the same nonce and leaves the tracked records alone", async () => { + const seen: unknown[] = []; + const removed: number[] = []; + const result = await stopUninstallProxy(io({ + readPid: () => null, + findLiveProxy: async () => ({ pid: 777, port: 12000 }), + stop: async (pid, stopIo) => { seen.push([pid, stopIo]); return true; }, + removeTracked: pid => { removed.push(pid); }, + })); + expect(seen).toEqual([[777, { skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 12000 } }]]); + expect(removed).toEqual([]); + expect(result.nonce).toBe(NONCE); + }); + + test("Part A not stopped: the graceful POST still goes out, but the forced fallback is forbidden", async () => { + for (const serviceStop of ["failed", "state-unknown", null] as const) { + const seen: unknown[] = []; + await stopUninstallProxy(io({ serviceStop, stop: async (_pid, stopIo) => { seen.push(stopIo); return false; } })); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, allowForcedFallback: false, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }]); + } + // A Part A that ended stopped (or found nothing) keeps today's fallback available. + const allowed: unknown[] = []; + for (const serviceStop of ["stopped", "absent"] as const) { + await stopUninstallProxy(io({ serviceStop, stop: async (_pid, stopIo) => { allowed.push(stopIo); return false; } })); + } + expect(allowed).toEqual([ + { skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }, + { skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }, + ]); + }); + + test("a forbidden fallback that does not confirm withholds the proof of a stopped proxy", async () => { + // The graceful POST may stop the proxy, but when it does not and the ladder was + // forbidden there is no proof left: the caller's recorded Part A failure is what keeps + // shared config up. Off Windows an unconfirmed stop keeps today's proof (the respawn + // window follows). + const windows = await stopUninstallProxy(io({ serviceStop: "failed", stop: async () => false })); + expect(windows.provenDown).toBe(false); + const stopped = await stopUninstallProxy(io({ serviceStop: "failed", stop: async () => true })); + expect(stopped.provenDown).toBe(true); + const elsewhere = await stopUninstallProxy(io({ platform: "linux", serviceStop: "failed", stop: async () => false })); + expect(elsewhere.provenDown).toBe(true); + }); + + test("an unconfirmed tracked stop asks the endpoint before claiming the proxy is down", async () => { + // #7771 round 9: `stopProxy` returns false without contacting anything when the tracked + // pid is already dead. `confirmed(false)` then reported provenDown:true on the strength of + // Part A alone, so a DIFFERENT proxy on the configured port was never probed and shared + // config could be restored under it. The tri-state probe is what the discovery path + // already requires; the tracked path has to ask the same question. + let probed = 0; + const dead = await stopUninstallProxy(io({ + serviceStop: "absent", + stop: async () => false, + endpointProvenDown: async () => { probed++; return true; }, + })); + expect(probed).toBe(1); + expect(dead.provenDown).toBe(true); + + // A live or unreadable endpoint is not proof, and the record stays for the next run. + for (const answer of [false]) { + const removed: number[] = []; + const result = await stopUninstallProxy(io({ + serviceStop: "stopped", + stop: async () => false, + endpointProvenDown: async () => answer, + removeTracked: pid => { removed.push(pid); }, + })); + expect(result.provenDown).toBe(false); + expect(result.changed).toBe(false); + expect(removed).toEqual([]); + } + + // A confirmed stop needs no second opinion. + let asked = 0; + const stopped = await stopUninstallProxy(io({ + stop: async () => true, + endpointProvenDown: async () => { asked++; return false; }, + })); + expect(stopped.provenDown).toBe(true); + expect(asked).toBe(0); + }); + + test("a receipt that cannot be recorded refuses the stop instead of sending an undeferred POST", async () => { + let stopped = 0; + await expect(stopUninstallProxy(io({ + claim: () => { throw new Error("disk full"); }, + stop: async () => { stopped++; return true; }, + }))).rejects.toThrow(/deferred-teardown receipt could not be recorded[\s\S]*disk full/); + expect(stopped).toBe(0); + }); + + test("off Windows nothing changes: no receipt, no fallback narrowing, today's port wait", async () => { + const seen: unknown[] = []; + let claimed = 0; + const result = await stopUninstallProxy(io({ + platform: "linux", serviceStop: "failed", + claim: () => { claimed++; return { nonce: NONCE }; }, + portWaitOwned: () => false, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + expect(claimed).toBe(0); + expect(seen).toEqual([{ skipPortWait: false, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }]); + expect(result).toEqual({ changed: true, provenDown: true }); + }); + + test("the receipt names the endpoint the stop actually contacts (tracked pid)", async () => { + // #7771 round 8: the receipt used to be claimed from `readRuntimePort()` with no + // expectation, so a stale record could name an endpoint the stop never contacted, and + // `stopProxy` re-read the record on its own. The receipt and the POST now carry the one + // endpoint resolved for the target. + const claimed: unknown[] = []; + const seen: unknown[] = []; + await stopUninstallProxy(io({ + runtimeEndpointFor: pid => (pid === 4242 ? { hostname: "127.0.0.1", port: 12000 } : null), + claim: (endpoint, source) => { claimed.push([endpoint, source]); return { nonce: NONCE }; }, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + expect(claimed).toEqual([[{ hostname: "127.0.0.1", port: 12000 }, "exact"]]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 12000 } }]); + }); + + test("a stale runtime record does not become the receipt for a proxy found on another port", async () => { + // The pid file is gone but a stale runtime record survives; discovery finds the live + // proxy elsewhere. Claiming the stale record marked it "exact" while the stop went to + // the discovered endpoint, so the obligation named an address nobody contacted. + const claimed: unknown[] = []; + const seen: unknown[] = []; + await stopUninstallProxy(io({ + readPid: () => null, + runtimeEndpointFor: () => ({ hostname: "127.0.0.1", port: 10999 }), + findLiveProxy: async () => ({ pid: 777, port: 12000, hostname: "127.0.0.1" }), + claim: (endpoint, source) => { claimed.push([endpoint, source]); return { nonce: NONCE }; }, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + expect(claimed).toEqual([[{ hostname: "127.0.0.1", port: 12000 }, "exact"]]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 12000 } }]); + }); + + test("no target means no receipt: a guessed obligation recovery cannot resolve is not claimed", async () => { + // A missing pid file, no live proxy, and a definitively dead endpoint: nothing will be + // stopped, so nothing may be deferred. The old order claimed a "guessed" receipt first + // and left it outstanding, which recovery then refused to finish. + let claimed = 0; + const result = await stopUninstallProxy(io({ + readPid: () => null, + findLiveProxy: async () => null, + endpointProvenDown: async () => true, + claim: () => { claimed++; return { nonce: NONCE }; }, + })); + expect(claimed).toBe(0); + expect(result).toEqual({ changed: false, provenDown: true }); + }); + + test("a tracked target with no runtime record claims nothing rather than a guessed endpoint", async () => { + let claimed = 0; + const seen: unknown[] = []; + await stopUninstallProxy(io({ + runtimeEndpointFor: () => null, + claim: () => { claimed++; return { nonce: NONCE }; }, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + // With no endpoint there is no graceful POST to defer, so no receipt may name one. + expect(claimed).toBe(0); + expect(seen).toEqual([{ skipPortWait: true }]); + }); + + test("the tracked record is removed only for a stop that was proven", async () => { + // Keeping a record the stop did not clear leaves a live pid for the next command to + // chase; removing it on a failed POST claims a state change that did not happen. + const removed: number[] = []; + // Part A failed and the forced ladder was forbidden: nothing proves the proxy exited, so + // the record it names must stay for the next command to chase, and the step reports no + // change (the caller records the failure and keeps the shared teardown up). + const failed = await stopUninstallProxy(io({ + serviceStop: "failed", + stop: async () => false, + removeTracked: pid => { removed.push(pid); }, + })); + expect(removed).toEqual([]); + expect(failed.changed).toBe(false); + expect(failed.provenDown).toBe(false); + + const stopped = await stopUninstallProxy(io({ + stop: async () => true, + removeTracked: pid => { removed.push(pid); }, + })); + expect(removed).toEqual([4242]); + expect(stopped.changed).toBe(true); + expect(stopped.provenDown).toBe(true); + }); + + test("the receipt is finished only after the shared restore ran", async () => { + const { completeWindowsServiceStopReceipt } = await import("../../src/service/cli-windows-stop"); + const cleared: string[] = []; + completeWindowsServiceStopReceipt(NONCE, nonce => { cleared.push(nonce); return true; }); + expect(cleared).toEqual([NONCE]); + }); + + test("after an absent Part A a confirmed tracked stop still proves every candidate endpoint dead", async () => { + // A confirmed stop proves the tracked proxy down, not that nothing else serves on this + // home's ports; with no manager registered no Part B follows, so a second proxy on the + // configured endpoint would otherwise go unseen and shared config be restored under it. + let probes = 0; + const live = await stopUninstallProxy(io({ + serviceStop: "absent", + stop: async () => true, + endpointProvenDown: async () => { probes++; return false; }, + })); + expect(probes).toBe(1); + expect(live.provenDown).toBe(false); + const dead = await stopUninstallProxy(io({ serviceStop: "absent", stop: async () => true, endpointProvenDown: async () => true })); + expect(dead.provenDown).toBe(true); + // After a stopped Part A, Part B confirms the port, so the stop's own proof stands. + probes = 0; + const stopped = await stopUninstallProxy(io({ serviceStop: "stopped", stop: async () => true, endpointProvenDown: async () => { probes++; return false; } })); + expect(probes).toBe(0); + expect(stopped.provenDown).toBe(true); + }); + + test("the uninstall endpoint proof reads its candidates once, before the stop removes the runtime record", async () => { + let reads = 0; + const probed: number[] = []; + let runtime: { port: number } | null = { port: 12000 }; + const proof = captureUninstallEndpointProof(() => { reads++; return runtime ? [{ hostname: "127.0.0.1", port: runtime.port }, { hostname: "127.0.0.1", port: 10100 }] : [{ hostname: "127.0.0.1", port: 10100 }]; }, + endpoint => { probed.push(endpoint.port); return endpoint.port === 12000 ? "dead" : "dead"; }); + runtime = null; // the tracked stop removed the runtime record + expect(await proof()).toBe(true); + expect(reads).toBe(1); + expect(probed).toEqual([12000, 10100]); + const unreadable = captureUninstallEndpointProof(() => { throw new Error("config unreadable"); }, () => "dead"); + expect(await unreadable()).toBe(false); + }); +}); diff --git a/tests/clients/desktop-supervised-restart.test.ts b/tests/clients/desktop-supervised-restart.test.ts index d3ced166fd4..96120e043c6 100644 --- a/tests/clients/desktop-supervised-restart.test.ts +++ b/tests/clients/desktop-supervised-restart.test.ts @@ -260,7 +260,7 @@ describe("the dashboard's Stop under the desktop app", () => { const refusal = handler.indexOf("desktopSupervisedStopRefusal(principal)"); expect(refusal).toBeGreaterThan(-1); expect(handler.slice(refusal, refusal + 200)).toContain("return jsonResponse(desktopRefusal, 409, req, config)"); - for (const later of ["installedServiceRespawnRisk()", "stopServiceIfInstalledDetailed()", "noteExplicitShutdownRequested()", "performStopTeardown(url"]) { + for (const later of ["installedServiceRespawnRisk()", "routeServiceStop(url, holdsReceipt)", "noteExplicitShutdownRequested()", "performStopTeardown(url"]) { expect(handler.indexOf(later)).toBeGreaterThan(refusal); } }); diff --git a/tests/fixtures/parent-stop-runner.ts b/tests/fixtures/parent-stop-runner.ts index db621f4b751..6ae4191848e 100644 --- a/tests/fixtures/parent-stop-runner.ts +++ b/tests/fixtures/parent-stop-runner.ts @@ -21,6 +21,8 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { clientServer?: "legacy" | "unknown" | "wrong-pid" | "malformed" | "oversized" | "unauthorized" | "redirect" | "body-timeout"; managerCascades?: boolean; guarded?: boolean; + /** #7771: the guarded path on a simulated win32, with a bound scheduler manager (separate child stop). */ + windowsGuarded?: boolean; orphan?: boolean; exitCleanup?: boolean; nativeSignal?: boolean; @@ -51,6 +53,16 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { noRuntimeRecord?: boolean; /** Like the real readPid, report no PID once the process is gone (a manager stop exited it). */ deadPidReadsNull?: boolean; + /** #7771: simulate win32 and answer Part A (and Part B) of the checked Windows stop. */ + windowsPartA?: "stopped" | "absent" | "survived" | "denied" | "unknown" | "unverified-budget"; + windowsPartB?: "freed" | "survived" | "port-held-by-foreign" | "unconfirmed" | "unknown"; + /** + * #7771: run the REAL Part A while this stop runs under this home's wrapper, with no task and + * no WinSW service registered, so the own-wrapper refusal has to answer for itself. + */ + windowsPartAUnderOwnWrapper?: boolean; + windowsWinswStopped?: boolean; + windowsPartAReject?: boolean; }; if (options.proxyProbePort) { // Deliberately bypass every fixture mock for the real proxy-environment regression. @@ -97,7 +109,11 @@ mock.module("../../src/client/lifecycle-lock", () => ({ ...lifecycle, })); const endpoint = { hostname: "127.0.0.1", port: 10100 }; const fakePid = 4242; -const calls = { killed: 0, native: 0, grok: 0, cleared: 0, exited: 0, service: 0, lifecycle: 0 }; +const calls = { killed: 0, native: 0, grok: 0, cleared: 0, exited: 0, service: 0, lifecycle: 0, partB: 0, reclaim: 0 }; +/** The timeout each reclaimListenPort call asked for; #7771 makes a skipped 15 s wait visible. */ +const reclaimTimeouts: number[] = []; +/** The Part A record each stop POST's receipt carried when the POST was sent (#7771). */ +const postPartA: unknown[] = []; let receiptBlocker: import("bun:sqlite").Database | undefined; const { Database } = await import("bun:sqlite"); const { prepareConfigMutationDatabasePathForWrite } = await import("../../src/config"); @@ -137,7 +153,7 @@ const receipts = await import("../../src/config/pending-teardown"); process.on("exit", () => { receiptBlocker?.close(); writeFileSync(join(home, "parent-stop-result.json"), JSON.stringify({ - calls, urls, probes, transports, unexpectedIo, nonce, + calls, reclaimTimeouts, urls, probes, transports, unexpectedIo, nonce, postPartA, receiptExists: nonce !== undefined && existsSync(receipts.pendingTeardownPathFor(nonce)), })); }); @@ -170,7 +186,9 @@ mock.module("../../src/service", () => ({ ...service, stopServiceIfInstalledDeta } })); if (options.guarded) { const approval = { pid: fakePid, ...endpoint, configHome: home, cliVersion: "2.93.0", compatibilityToken: "fixture" }; - const manager = { kind: "bound", pid: fakePid, managerPid: fakePid + 1, backend: "launchd" }; + const manager = options.windowsGuarded + ? { kind: "bound", pid: fakePid, managerPid: fakePid + 1, backend: "scheduler", childNeedsSeparateStop: true } + : { kind: "bound", pid: fakePid, managerPid: fakePid + 1, backend: "launchd" }; const stopApproval = await import("../../src/cli/stop-approval"); mock.module("../../src/cli/stop-approval", () => ({ ...stopApproval, parseStopApproval: () => ({ ok: true, json: true, approval }), @@ -179,7 +197,8 @@ if (options.guarded) { })); const guarded = await import("../../src/service/guarded-manager-target"); mock.module("../../src/service/guarded-manager-target", () => ({ ...guarded, - inspectGuardedManagerTarget: () => manager, observeGuardedManagerStopped: () => "stopped", + inspectGuardedManagerTarget: () => manager, + observeGuardedManagerStopped: () => (options.windowsGuarded ? "inactive" : "stopped"), })); } if (options.orphan || options.alreadyStopped) { @@ -225,7 +244,12 @@ mock.module("../../src/grok/inject", () => ({ const systemEnv = await import("../../src/server/system-env"); mock.module("../../src/server/system-env", () => ({ ...systemEnv, revertSystemEnv() {} })); const portReclaim = await import("../../src/server/port-reclaim"); -mock.module("../../src/server/port-reclaim", () => ({ ...portReclaim, reclaimListenPort: async () => {} })); +mock.module("../../src/server/port-reclaim", () => ({ ...portReclaim, + reclaimListenPort: async (_port: number, _host: string, options?: { timeoutMs?: number }) => { + calls.reclaim += 1; + reclaimTimeouts.push(options?.timeoutMs ?? 0); + }, +})); // Only shim preflight is unrelated to this stop contract; parsing and dispatch stay real. mock.module("../../src/cli/codex-shim-autorestore", () => ({ async maybeAutoRestoreCodexShimAsync() {} })); @@ -261,6 +285,7 @@ globalThis.fetch = (async (input: string | URL | Request, init?: RequestInit) => } if (!url.startsWith("http://127.0.0.1:10100/api/stop")) return unexpected(`fetch(${url})`); urls.push(url); + postPartA.push(receipts.pendingTeardownPartA(new URL(url).searchParams.get("teardownNonce"))); transports.push(viaDirect ? "direct" : "global"); if (options.connectDuringPost) writeHubConnection(); const body = options.serverTeardown && !options.apiFailure @@ -290,6 +315,36 @@ mock.module("../../src/server/direct-local-http", () => ({ ...direct, })); process.argv = [process.execPath, "ocx", "stop", ...(options.json ? ["--json"] : [])]; +if (options.windowsPartA || options.windowsPartAReject) { + // The stop wiring acts for win32 while the host stays untouched: Part A and Part B answer here. + (await import("../../src/cli/stop-windows-checked")).resetOcxWindowsStopForTests("win32"); + const { partAResult, partBResult } = await import("../helpers/windows-stop-caller-fixtures"); + (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ + partA: async () => { calls.service++; if (options.windowsPartAReject) throw new Error("wrapper watch rejected"); return partAResult(options.windowsPartA!); }, + partB: async () => { calls.partB++; return partBResult(options.windowsPartB ?? "freed"); }, + // The guarded inspection reads evidence before it asks the mocked manager; nothing may + // reach the host's process table. + ...(options.guarded ? { snapshot: async () => [], schedulePid: async () => null } : {}), + }); +} +if (options.windowsPartAUnderOwnWrapper) { + // This stop runs under this home's wrapper: ocx stop <- cmd.exe (.cmd) <- wscript.exe (.vbs), + // with nothing registered. The real Part A answers the own-wrapper refusal. + (await import("../../src/cli/stop-windows-checked")).resetOcxWindowsStopForTests("win32"); + const { PATHS, launcher, loop } = await import("../helpers/windows-checked-stop-fixtures"); + const self = { pid: process.pid, parentPid: 30, name: "ocx.exe", commandLine: "ocx stop", startTime: 6_000 }; + (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ + liveDeps: () => ({ + canonicalPaths: PATHS, childPid: null, selfPid: process.pid, + probeScheduler: () => { calls.service++; return { status: "absent" }; }, probeWinsw: () => options.windowsWinswStopped ? "stopped" : "nonexistent", + enumerate: () => [launcher(), loop(), self], + scheduleServicePid: () => null, endTask: () => unexpected("schtasks /end"), taskState: () => "not-running", + winswStop: () => "sent", winswState: () => "stopped", proxyAlive: () => false, + openWatch: () => ({ status: "gone" }), + terminate: () => unexpected("terminate"), + }), + }); +} if (options.handover) process.env.OCX_UPDATE_TEARDOWN_HANDOVER = "1"; if (options.exitCleanup) { delete process.env.OCX_SERVICE; diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 2c0c5bcc8dc..7313f7a2b74 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1539,8 +1539,8 @@ "windows-deploy-close-regressions.test.ts": "windows", "windows-elevation-spawn.test.ts": "windows", "windows-elevation.test.ts": "windows", - "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", - "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", + "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", "windows-checked-stop-live.test.ts": "windows", "windows-service-stop-wiring.test.ts": "windows", + "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", "windows-stop-callers.test.ts": "windows", "windows-stop-ocx-stop.test.ts": "windows", "windows-process-priority.test.ts": "windows", "windows-scheduler-errors.test.ts": "windows", "windows-scheduler-install-verification.test.ts": "windows", "windows-lifecycle-root-acl.test.ts": "windows", diff --git a/tests/helpers/update-guarded-io.mjs b/tests/helpers/update-guarded-io.mjs index f9cf329a9b9..a7586d62e27 100644 --- a/tests/helpers/update-guarded-io.mjs +++ b/tests/helpers/update-guarded-io.mjs @@ -23,8 +23,10 @@ export function guardedUpdateIo(deps) { guardedRemoveRuntimePort: auth => guardedRun(auth, "runtime", native("removeRuntimePort"), []), guardedDropWindowsTcp: (auth, ...args) => guardedRun(auth, "runtime", native("dropWindowsTcpRowsForLocalPort"), args), guardedStopWinsw: auth => guardedRun(auth, "service", native("stopWinswService"), []), - guardedStopWindows: auth => guardedRun(auth, "service", native("stopWindows"), []), - guardedKillWindowsWrappers: (auth, ...args) => guardedRun(auth, "service", native("killWindowsSchedulerWrappers"), args), + // #7771: Part A takes the service gate as its authorizer (entry and before each terminate). + guardedStopWindowsChecked: async auth => native("checkedStop")(() => { + if (auth("service") !== true) throw new UpdateMutationRefusedError(); + }), }; } diff --git a/tests/helpers/update-helper-control-contracts.ts b/tests/helpers/update-helper-control-contracts.ts index c94203ab44a..8578cd95da7 100644 --- a/tests/helpers/update-helper-control-contracts.ts +++ b/tests/helpers/update-helper-control-contracts.ts @@ -106,8 +106,7 @@ const baseHelperControlContracts: readonly Contract[] = [ ["src/update/guarded-runtime-ops.ts","guardedRemoveRuntimePort","removeRuntimePort(undefined,",1,"same runtime gate immediately before record and registry unlink"], ["src/update/guarded-runtime-ops.ts","guardedDropWindowsTcp","dropWindowsTcpRowsForLocalPort(target,",1,"same runtime gate immediately before each prepared SetTcpEntry"], ["src/update/guarded-runtime-ops.ts","guardedStopWinsw","stopWinswService(mutationGate(",1,"same service gate at prepared native command"], - ["src/update/guarded-runtime-ops.ts","guardedStopWindows","stopWindows(mutationGate(",1,"same service gate at prepared scheduler command"], - ["src/update/guarded-runtime-ops.ts","guardedKillWindowsWrappers","killWindowsSchedulerWrappers(materializeMutationData(paths),",1,"same service gate after path/script preparation"], + ["src/update/guarded-runtime-ops.ts","guardedStopWindowsChecked","runWindowsServiceStopPartA({ authorize: mutationGate(authorize, \"service\"),",1,"same service gate at Part A entry and again before each bound-wrapper terminate (#7771)"], ["src/update/guarded-tray-ops.ts","guardedStartWindowsTray","guardedRun(authorize, \"own-tray\", startWindowsTray, [])",1,"materialize plain data then synchronous authorization and primitive dispatch"], ["src/update/guarded-tray-ops.ts","guardedStopWindowsTray","stopWindowsTray(()",1,"same tray gate after heartbeat and command preparation"], ["src/update/mutation-guard.mjs","guardedSpawn","guardedRun(authorize, kind, spawn, [file, args, options])",1,"materialize plain data then synchronous authorization and primitive dispatch"], diff --git a/tests/helpers/update-helper-inventory.ts b/tests/helpers/update-helper-inventory.ts index 86749aaf8c6..30d2236d9b4 100644 --- a/tests/helpers/update-helper-inventory.ts +++ b/tests/helpers/update-helper-inventory.ts @@ -56,7 +56,7 @@ export function helperControlInventory(parser?: API) { const visit = (node: ts.Node) => { if (ts.isCallExpression(node) && !["spawnFailure", "registrySpawnTarget"].includes(compact(node.expression)) && !(ts.isPropertyAccessExpression(node.expression) && node.expression.name.text === "trim") - && (capabilityCalls.has(node) || /[sS]pawn|execFile|guardedRun|guardedRename|guardedKill|guardedExecFile|ownedArtifact|^native$|^operation$|(?:^|\.)kill$|^run$|^runPnpm$|^io\.(?:stop|start)$|^(?:removePid|removeRuntimePort|dropWindowsTcpRowsForLocalPort|stopWinswService|stopWindows|killWindowsSchedulerWrappers|stopWindowsTray)$|(?:^|\.)(?:rename(?:WithRetry)?|rm|unlink|writeFile|copyFile)(?:Sync)?$/.test(compact(node.expression)))) { + && (capabilityCalls.has(node) || /[sS]pawn|execFile|guardedRun|guardedRename|guardedKill|guardedExecFile|ownedArtifact|^native$|^operation$|(?:^|\.)kill$|^run$|^runPnpm$|^io\.(?:stop|start)$|^(?:removePid|removeRuntimePort|dropWindowsTcpRowsForLocalPort|stopWinswService|stopWindows|killWindowsSchedulerWrappers|runWindowsServiceStopPartA|stopWindowsTray)$|(?:^|\.)(?:rename(?:WithRetry)?|rm|unlink|writeFile|copyFile)(?:Sync)?$/.test(compact(node.expression)))) { let context = ""; for (let p: ts.Node | undefined = node.parent; p; p = p.parent) { if (ts.isFunctionDeclaration(p) && p.name) { context = p.name.text; break; } diff --git a/tests/helpers/update-import-policy.ts b/tests/helpers/update-import-policy.ts index 43c0fa1646e..c563854b541 100644 --- a/tests/helpers/update-import-policy.ts +++ b/tests/helpers/update-import-policy.ts @@ -87,8 +87,8 @@ const baseStaticImports: Readonly> = { "import { removePid, removeRuntimePort } from \"../config/process-state\";", "import { dropWindowsTcpRowsForLocalPort } from \"../server/windows-tcp-drop\";", "import { stopWinswService } from \"../lib/winsw\";", - "import { stopWindows } from \"../service\";", - "import { killWindowsSchedulerWrappers } from \"../lib/windows-service-wrappers\";", + // #7771: Part A of the checked Windows stop; its authorizer is the service mutation gate. + "import { runWindowsServiceStopPartA } from \"../service/windows-service-stop\";", "import { guardedRun, guardedKill, guardedExecFileSync, materializeMutationData, UpdateMutationRefusedError, type UpdateMutationAuthorizer } from \"./mutation-guard.mjs\";" ], "src/update/guarded-tray-ops.ts": [ @@ -166,9 +166,7 @@ const baseStaticImports: Readonly> = { "import { handoffWindowsTrayForUpdate, planWindowsTrayUpdate } from \"./tray-update-plan.mjs\";" ], "src/update/job-windows-restart.ts": [ - "import { join } from \"node:path\";", - "import { getConfigDir } from \"../config\";", - "import { guardedStopWinsw, guardedStopWindows, guardedKillWindowsWrappers } from \"./guarded-runtime-ops\";", + "import { guardedStopWinsw, guardedStopWindowsChecked } from \"./guarded-runtime-ops\";", "import { readServiceBackend } from \"../service\";" ], "src/update/job.ts": [ @@ -187,7 +185,7 @@ const baseStaticImports: Readonly> = { "import { guardedKillProxy, guardedRemovePid, guardedRemoveRuntimePort, guardedDropWindowsTcp, guardedPublishJob, type KillProxy } from \"./guarded-runtime-ops\";", "import { isHealthzVersion, isOpencodexHealthz, probeHostname, proxyIdentityAt, type HealthzIdentity, } from \"../server/proxy-liveness\";", "import { isServiceInstalled, isServiceViable } from \"../service\";", - "import { stopWindowsServiceWrappersBestEffort } from \"./job-windows-restart\";", + "import { stopWindowsServiceForRestart, WINDOWS_SERVICE_STOP_ABORT } from \"./job-windows-restart\";", "import { createUpdateServiceHomePolicy, inspectUpdateServiceHome, type UpdateServiceHomeInspector } from \"./service-home-authority\";", "import { runUpdateRestartWithOwnershipLease, type ServiceOwnershipResolution } from \"./restart-ownership\";", "import { restartServiceManagerAfterUpdate, unsettledManagerRestartFailure, unsupervisedRestartFailure, type ManagerRestartFn } from \"./job-supervised-restart\";", diff --git a/tests/helpers/windows-stop-caller-fixtures.ts b/tests/helpers/windows-stop-caller-fixtures.ts new file mode 100644 index 00000000000..cc59f458b8b --- /dev/null +++ b/tests/helpers/windows-stop-caller-fixtures.ts @@ -0,0 +1,42 @@ +/** + * Result builders for the caller-level tests of the checked Windows stop (#7771): what Part A + * and Part B return, so a caller can be driven through every outcome without the machine. + */ +import type { WindowsCheckedStopResult } from "../../src/service/windows-checked-stop"; +import type { WindowsStopConfirmResult } from "../../src/service/windows-stop-confirm"; + +export function partAResult( + outcome: WindowsCheckedStopResult["outcome"], + extra: Partial = {}, +): WindowsCheckedStopResult { + const reason: WindowsCheckedStopResult["reason"] = outcome === "denied" ? "access-denied" + : outcome === "unknown" ? "evidence-unreadable" + : outcome === "unverified-budget" ? "unverified-budget" : null; + return { + outcome, reason, detail: "fixture " + outcome, + registration: { scheduler: outcome === "absent" ? "absent" : "present", winsw: "nonexistent" }, + bound: outcome === "stopped" ? [{ pid: 30, startTime: 5_100, image: "cmd.exe", parentPid: 20, commandLineDigest: "d" }] : [], + unbound: [], terminated: outcome === "stopped" ? [30] : [], changed: outcome !== "absent" && outcome !== "unverified-budget", + ...extra, + }; +} + +export function partBResult( + outcome: WindowsStopConfirmResult["outcome"], + extra: Partial = {}, +): WindowsStopConfirmResult { + return { + outcome, + portHeldByForeign: outcome === "port-held-by-foreign", + reason: outcome === "unknown" ? "evidence-unreadable" : null, + detail: "fixture " + outcome, + port: 10100, + windowMs: 6_000, + listener: outcome === "port-held-by-foreign" ? { ownerPid: 777, identity: "foreign" } + : outcome === "survived" ? { ownerPid: 43, identity: "ours" } : null, + lateWrappers: [], + ...extra, + }; +} + +export const ALL_PART_A_FAILURES = ["survived", "denied", "unknown", "unverified-budget"] as const; diff --git a/tests/lib/process-control-graceful.test.ts b/tests/lib/process-control-graceful.test.ts index 9f20eb386c4..9bf89fe055a 100644 --- a/tests/lib/process-control-graceful.test.ts +++ b/tests/lib/process-control-graceful.test.ts @@ -5,6 +5,24 @@ function okResponse(): Response { return new Response(JSON.stringify({ success: true, sharedTeardown: "performed" }), { status: 200 }); } +/** + * A live process this test owns, for the pid-taking stop paths. + * + * `spawn("sleep", ...)` depends on a POSIX `sleep` on PATH and, worse, waits forever when + * the spawn itself fails: the `spawn` event never fires and the `error` event is unhandled. + * The runtime's own binary is always present, and a failed start now rejects instead of + * hanging the suite. + */ +async function spawnIdleChild(): Promise { + const { spawn } = await import("node:child_process"); + const child = spawn(process.execPath, ["-e", "setTimeout(() => {}, 60_000)"], { stdio: "ignore" }); + await new Promise((resolve, reject) => { + child.once("spawn", () => resolve()); + child.once("error", reject); + }); + return child; +} + describe("gracefulStopHost", () => { test("loopback aliases and wildcard binds answer on IPv4 loopback", () => { for (const host of [undefined, "", " ", "localhost", "LOCALHOST", "127.0.0.1", "0.0.0.0", "::", "[::]"]) { @@ -353,6 +371,97 @@ describe("409 refusal reporting", () => { expect((thrown as ProxyOwnershipRefusedError).message).toBe("wrapper owns it"); }); + test("a caller that forbids the forced fallback never reaches the kill ladder", async () => { + // #7771 round 7 (B): uninstall's proxy step runs after a Part A that did not prove the + // manager stopped. The graceful POST is still worth sending; taskkill /T /F under a + // wrapper that could respawn the proxy is exactly what the checked stop prevents. + // + // The pid is a real live process this test owns, and the ladder is injected: the rule + // is "was the ladder entered", which a mock of the whole stop could not show. + const child = await spawnIdleChild(); + const failing = (async () => new Response("nope", { status: 500 })) as typeof fetch; + try { + const forbidden: number[] = []; + const unconfirmed = await stopProxy(child.pid!, { + readRuntime: () => null, fetchFn: failing, waitExit: () => true, env: {}, + allowForcedFallback: false, forceKill: pid => { forbidden.push(pid); }, + }); + expect(unconfirmed).toBe(false); + expect(forbidden).toEqual([]); + + // Default behaviour is unchanged: an unconfirmed graceful stop still falls back. + const fallback: number[] = []; + await stopProxy(child.pid!, { + readRuntime: () => null, fetchFn: failing, waitExit: () => true, env: {}, + forceKill: pid => { fallback.push(pid); }, + }); + expect(fallback).toEqual([child.pid!]); + + // A 409 refusal is never forced, with or without the option. + for (const allowForcedFallback of [true, false]) { + const refused: number[] = []; + await expect(stopProxy(child.pid!, { + readRuntime: () => ({ port: 10100 }), + fetchFn: (async () => new Response( + JSON.stringify({ success: false, code: "respawnable_service", message: "wrapper owns it" }), + { status: 409, headers: { "content-type": "application/json" } }, + )) as typeof fetch, + waitExit: () => true, env: {}, allowForcedFallback, + forceKill: pid => { refused.push(pid); }, + })).rejects.toBeInstanceOf(ProxyOwnershipRefusedError); + expect(refused).toEqual([]); + } + } finally { + child.kill("SIGKILL"); + } + }); + + test("a forbidden fallback's port wait never kills a verified listener", async () => { + // #7771 round 8: `waitForStoppedPort` reclaimed the port with `killOcxHolders` and + // `onlyKillPids: [pid]` even when the caller had forbidden the forced ladder. A + // verified-but-still-live listener was then killed through `killProxy` — the forced + // stop the caller had just refused. The wait is observational in that mode: it runs, + // it just may not terminate anything. + const { reclaimListenPort } = await import("../../src/server/port-reclaim"); + const child = await spawnIdleChild(); + const kills: number[] = []; + const pid = child.pid!; + const reclaimed: unknown[] = []; + const unconfirmed = await stopProxy(pid, { + runtimeEndpoint: { hostname: "127.0.0.1", port: 45999 }, + fetchFn: (async () => new Response("nope", { status: 500 })) as typeof fetch, + waitExit: () => true, + env: {}, + allowForcedFallback: false, + forceKill: target => { kills.push(target); }, + reclaimPort: async (port, hostname, opts) => { + reclaimed.push({ port, hostname, killOcxHolders: opts.killOcxHolders, onlyKillPids: opts.onlyKillPids }); + return reclaimListenPort(port, hostname, { + ...opts, + timeoutMs: 30, + intervalMs: 5, + scanIntervalMs: 0, + sleepMs: async () => {}, + isAvailableFn: async () => false, + listListenPidsFn: () => ({ ok: true, pids: [pid] }), + isAliveFn: () => true, + verifyOcxFn: candidate => candidate, + killFn: target => { kills.push(target); }, + dropTcpFn: () => {}, + }); + }, + }); + try { + expect(unconfirmed).toBe(false); + // The wait ran against the resolved endpoint (it is not silently skipped)... + expect(reclaimed).toEqual([{ port: 45999, hostname: "127.0.0.1", killOcxHolders: false, onlyKillPids: [] }]); + // ...and a verified listener that was still alive survived it. + expect(kills).toEqual([]); + } finally { + child.kill("SIGKILL"); + } + }); + test("no next step sends the operator back to the command that just refused", () => { // #4169's loop: POST /api/stop answers "the stop must be run by `ocx stop`", and // `ocx stop` reprints it. Whatever the cause, the next step has to be something else, diff --git a/tests/providers/xai/grok-lifecycle.test.ts b/tests/providers/xai/grok-lifecycle.test.ts index b3e5634357a..2c70dac7727 100644 --- a/tests/providers/xai/grok-lifecycle.test.ts +++ b/tests/providers/xai/grok-lifecycle.test.ts @@ -1,7 +1,9 @@ import { describe, expect, test } from "bun:test"; import { readFileSync } from "node:fs"; import { join } from "node:path"; -import { classifyWindowsServiceStop, installedServiceRespawnRisk, isServiceOwnershipError, ServiceOwnershipError } from "../../../src/service"; +import { installedServiceRespawnRisk, isServiceOwnershipError, ServiceOwnershipError } from "../../../src/service"; +import { serviceStopOutcomeFromWindows } from "../../../src/service/windows-service-stop"; +import type { WindowsCheckedStopResult } from "../../../src/service/windows-checked-stop"; import { decideStartExitTeardown } from "../../../src/cli/dispatch"; import { repoPath } from "../../helpers/repo-root"; @@ -193,26 +195,30 @@ describe("Grok fence lifecycle wiring", () => { test("a stopped scheduler is verified across the respawn window before stop succeeds", () => { const stopFn = sliceFn(CLI_SOURCE, "async function handleStop(", "async function handleUninstall("); - // killWindowsSchedulerWrappers is best-effort and the `:loop` wrapper respawns after - // ~5s, so "stopped" alone is not a proven-down proxy. An update that trusts it can - // start replacing files during the dead interval (#3008). - expect(stopFn).toContain("proxyStillLiveAfterStop({ canRespawn: true })"); + // The `:loop` wrapper respawns after ~5s, so "stopped" alone is not a proven-down + // proxy. An update that trusts it can start replacing files during the dead interval + // (#3008). On Windows the window is Part B of the checked stop (#7771); its behavior is + // covered in tests/windows/windows-stop-callers.test.ts and the parent-stop runner. + expect(stopFn).toContain("if (await stopWindowFailed(record)) { stopFailed = true; ownershipBlocked = true; }"); + const wiring = readFileSync(repoPath("src", "cli", "stop-windows-checked.ts"), "utf8"); + expect(wiring).toContain("proxyStillLiveAfterStop({ canRespawn: true })"); + expect(wiring).toContain("confirmWindowsServiceStop("); // A survivor is an ordinary failure AND blocks shared teardown: restoring client // config while the proxy runs leaves both pointing at each other. expect(stopFn).toContain("stopFailed = true;"); expect(stopFn).toContain("ownershipBlocked = true;"); }); - test("only Task Scheduler earns the respawn wait", () => { + test("only a stop that can be respawned earns the respawn wait", () => { const stopFn = sliceFn(CLI_SOURCE, "async function handleStop(", "async function handleUninstall("); const serviceSource = readFileSync(repoPath("src", "service", "orchestration.ts"), "utf8"); // schtasks /end leaves the `cmd :loop` wrapper alive to respawn its child (#764). // launchd, systemd and WinSW are down when they report stopped, so charging them a // seven-second poll on every ocx stop would be a regression in ordinary use. expect(serviceSource).toContain('"absent" | "stopped" | "stopped-respawnable" | "failed"'); - const windowsOps = readFileSync(repoPath("src", "service", "windows-ops.ts"), "utf8"); - expect(windowsOps).toContain('schedulerStopped ? "stopped-respawnable" : "stopped"'); expect(stopFn).toContain("if (schedulerCanRespawn && !ownershipBlocked)"); + // #7771: a legacy respawnable outcome or a Windows Part A that stopped (Part B follows). + expect(stopFn).toContain('schedulerCanRespawn = serviceStop === "stopped-respawnable" || windowsPartBDue();'); // The wait is gated on the scheduler flag, not on "a service stopped". expect(stopFn).not.toContain("if (stoppedService && !ownershipBlocked)"); }); @@ -438,7 +444,7 @@ describe("POST /api/stop teardown", () => { const handler = sliceFn(MANAGEMENT_SOURCE, '"/api/stop"', "/api/codex-auth/"); expect(handler).toContain("const sibling = siblingOfLivePort() !== null;"); expect(handler).toContain('const respawnRisk = holdsReceipt || sibling ? "none" : installedServiceRespawnRisk();'); - expect(handler).toContain('serviceStop = sibling ? "absent" : stopServiceIfInstalledDetailed();'); + expect(handler).toContain('serviceStop = sibling ? "absent" : await (await import("./stop-service-route")).routeServiceStop(url, holdsReceipt);'); }); test("strips the Grok fence on an accepted stop", () => { @@ -461,21 +467,19 @@ describe("POST /api/stop teardown", () => { // wrong thing, and `ocx stop` was the command the API told them to run (#3008). expect(serviceSource).toContain('"absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown"'); // Behavioural, because a source-text assertion cannot tell whether an unreadable probe - // is still being folded into the generic failure. - expect(classifyWindowsServiceStop({ stopped: false, failed: false, schedulerStopped: false, stateUnknown: true })) - .toBe("state-unknown"); - // A readable failure outranks it — something actually refused to stop. - expect(classifyWindowsServiceStop({ stopped: false, failed: true, schedulerStopped: false, stateUnknown: true })) - .toBe("failed"); - // And an unreadable state outranks success: a scheduler we cannot see may respawn. - expect(classifyWindowsServiceStop({ stopped: true, failed: false, schedulerStopped: true, stateUnknown: true })) - .toBe("state-unknown"); - expect(classifyWindowsServiceStop({ stopped: true, failed: false, schedulerStopped: true, stateUnknown: false })) - .toBe("stopped-respawnable"); - expect(classifyWindowsServiceStop({ stopped: true, failed: false, schedulerStopped: false, stateUnknown: false })) - .toBe("stopped"); - expect(classifyWindowsServiceStop({ stopped: false, failed: false, schedulerStopped: false, stateUnknown: false })) - .toBe("absent"); + // is still being folded into the generic failure. Since #7771 the Windows outcome is + // Part A's: unreadable evidence is state-unknown, a refusal or survivor is failed, and + // Task Scheduler no longer reports stopped-respawnable (Part B covers the window). + const partA = (outcome: WindowsCheckedStopResult["outcome"]) => serviceStopOutcomeFromWindows({ + outcome, reason: null, detail: "", registration: { scheduler: "present", winsw: "nonexistent" }, + bound: [], unbound: [], terminated: [], changed: false, + }); + expect(partA("unknown")).toBe("state-unknown"); + expect(partA("denied")).toBe("failed"); + expect(partA("survived")).toBe("failed"); + expect(partA("unverified-budget")).toBe("failed"); + expect(partA("stopped")).toBe("stopped"); + expect(partA("absent")).toBe("absent"); const stopFn = sliceFn(CLI_SOURCE, "async function handleStop(", "async function handleUninstall("); expect(stopFn).toContain('if (serviceStop === "state-unknown")'); const unknownBranch = stopFn.slice(stopFn.indexOf('if (serviceStop === "state-unknown")'), stopFn.indexOf('if (serviceStop === "state-unknown")') + 700); @@ -499,7 +503,7 @@ describe("POST /api/stop teardown", () => { // stopServiceIfInstalled collapses "failed" into the same false as "not installed", so // this route used to tear down shared config while a manager that refused to stop was // still there to respawn the proxy (#3008). - expect(handler).toContain("stopServiceIfInstalledDetailed()"); + expect(handler).toContain("routeServiceStop(url, holdsReceipt)"); expect(handler).not.toContain("stopServiceIfInstalled();"); expect(handler).toContain('if (serviceStop === "failed")'); expect(handler.indexOf('if (serviceStop === "failed")')).toBeLessThan(handler.indexOf("await performStopTeardown")); @@ -512,7 +516,7 @@ describe("POST /api/stop teardown", () => { // sends a bare request on every backend. expect(handler).toContain('const respawnRisk = holdsReceipt || sibling ? "none" : installedServiceRespawnRisk();'); expect(handler).toContain('code: "respawnable_service"'); - expect(handler.indexOf("installedServiceRespawnRisk()")).toBeLessThan(handler.indexOf("stopServiceIfInstalledDetailed()")); + expect(handler.indexOf("installedServiceRespawnRisk()")).toBeLessThan(handler.indexOf("routeServiceStop(url, holdsReceipt)")); // The refusal must say nothing was changed, because nothing was. expect(handler).toContain("Nothing was changed."); // An unreadable scheduler state is its own answer: sending that operator to `ocx stop` diff --git a/tests/server/port-reclaim.test.ts b/tests/server/port-reclaim.test.ts index 21a26984d40..4b02092eaba 100644 --- a/tests/server/port-reclaim.test.ts +++ b/tests/server/port-reclaim.test.ts @@ -467,6 +467,20 @@ describe("parseTcpQuadsForLocalPort / IPv6", () => { ]); }); + test("an owner filter keeps only the rows the named processes own", () => { + const output = [ + " TCP 127.0.0.1:10100 0.0.0.0:0 LISTENING 18268", + " TCP 127.0.0.1:10100 127.0.0.1:60001 ESTABLISHED 18268", + " TCP 127.0.0.1:10100 127.0.0.1:60002 ESTABLISHED 20001", + ].join("\n"); + // The exited proxy's rows are cleared; the program that took the port keeps its own. + expect(parseTcpQuadsForLocalPort(output, 10100, [18268]).map(row => row.remotePort)).toEqual([0, 60001]); + expect(parseTcpQuadsForLocalPort(output, 10100, [20001]).map(row => row.remotePort)).toEqual([60002]); + expect(parseTcpQuadsForLocalPort(output, 10100, [])).toEqual([]); + // No filter keeps every row, which is what the hard-kill path wants. + expect(parseTcpQuadsForLocalPort(output, 10100)).toHaveLength(3); + }); + test("keeps IPv6 local rows parseable without coercing them into IPv4 wildcards", () => { const output = [ " TCP [::1]:10100 [::]:0 LISTENING 18268", diff --git a/tests/service/service-desktop-runtime-preflight.test.ts b/tests/service/service-desktop-runtime-preflight.test.ts index e60e6f75c2a..340f01fc344 100644 --- a/tests/service/service-desktop-runtime-preflight.test.ts +++ b/tests/service/service-desktop-runtime-preflight.test.ts @@ -11,6 +11,7 @@ import { installFreshWindowsSchedulerSafely, installServiceSafely } from "../../ import { repairService } from "../../src/service/repair"; import { buildWindowsTaskXml } from "../../src/service/windows-taskxml"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; const roots: string[] = []; afterEach(() => { for (const root of roots.splice(0)) removeTreeWithRetry(root); }); @@ -57,9 +58,8 @@ function fixture(supervised: boolean, passes: boolean) { const run = (action: string) => { if (action === "install") return installServiceSafely("scheduler", consume("install"), { ...admission, inspectSupervision, - diagnose: () => { order.push("diagnose"); return diagnostic; }, - managerOps: () => ({ status: () => "present", stop: () => { order.push("stop"); } }), - killServiceWrappers: () => { order.push("wrappers"); }, + windowsPartA: async () => { order.push("stop"); return partAResult("stopped"); }, + windowsPartB: async () => { order.push("confirm"); return partBResult("freed"); }, stopTrackedProxy: async () => { order.push("proxy"); }, }); if (action === "fresh install") return installFreshWindowsSchedulerSafely({ diff --git a/tests/service/service-install-wrapper-cleanup.test.ts b/tests/service/service-install-wrapper-cleanup.test.ts index 0b4dab028fa..50c3292b598 100644 --- a/tests/service/service-install-wrapper-cleanup.test.ts +++ b/tests/service/service-install-wrapper-cleanup.test.ts @@ -1,92 +1,259 @@ import { describe, expect, test } from "bun:test"; -import { prepareServiceInstall } from "../../src/service"; +import { installServiceSafely, prepareServiceInstall } from "../../src/service"; +import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; +// #7771: on Windows, install and backend switch stop the old installation through the checked +// stop. Part A ends the managers and the bound wrappers (or refuses with nothing replaced), +// the tracked proxy stops next, and Part B must find the port free before anything is +// published or started. describe("service install preparation wrapper cleanup", () => { - test("install preparation kills surviving scheduler wrappers before the tracked proxy stop", async () => { - // The defect: `schtasks /end` ends the task instance but the detached cmd - // `:loop` wrapper survives and respawns its child seconds later — landing - // after the one-shot tracked-proxy kill and holding the install port. Model - // the respawner: while the wrapper lives, a stopped proxy comes back. + test("install claims the discovered orphan endpoint and carries it to the stop", async () => { + const endpoint = { hostname: "127.0.0.1", port: 12001 }; + let claimed: unknown; + let target: unknown; + let port = 0; + await prepareServiceInstall("scheduler", { + platform: "win32", windowsStopPort: () => 10100, + windowsPartA: async () => partAResult("stopped"), + resolveTarget: async () => ({ pid: 42, endpoint }), + claimTeardown: value => { claimed = value; return { nonce: "n" }; }, + stopTrackedProxy: async io => { target = io?.target; }, + windowsPartB: async request => { port = request.port; return partBResult("freed"); }, + }); + expect(claimed).toEqual(endpoint); + expect(target).toEqual({ pid: 42, endpoint }); + expect(port).toBe(12001); + }); + test("install preparation ends the respawning wrapper (Part A) before the tracked proxy stop", async () => { + // The defect: schtasks /end ends the task instance but the detached cmd :loop wrapper + // survives and respawns its child seconds later, landing after the one-shot tracked-proxy + // kill and holding the install port. Model the respawner: while it lives, a stopped proxy + // comes back. const calls: string[] = []; let wrapperAlive = true; let respawned = false; await prepareServiceInstall("scheduler", { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps: () => ({ - status: () => "present", - // `schtasks /end` ends the task instance; it cannot reach the detached wrapper. - stop: () => { calls.push("stop:scheduler"); }, - }), - killServiceWrappers: () => { calls.push("kill:wrappers"); wrapperAlive = false; }, + windowsPartA: async () => { calls.push("partA"); wrapperAlive = false; return partAResult("stopped"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); if (wrapperAlive) respawned = true; }, }); - expect(calls).toEqual(["stop:scheduler", "kill:wrappers", "stop:standalone"]); + expect(calls).toEqual(["partA", "stop:standalone", "partB"]); expect(respawned).toBe(false); }); - test("install preparation kills stray scheduler wrappers even when no manager stop runs", async () => { - // A registration deleted mid-uninstall leaves no task for `schtasks /end`, but - // the detached wrapper may still be looping: it must not respawn the old proxy - // onto the port the fresh install is about to claim. - let killed = false; + test("a stray wrapper of a deleted task that cannot be bound refuses instead of being killed by path", async () => { + // With nothing registered an orphaned wrapper is never bound (unbound-wrapper): the install + // stops and names the manual step rather than replacing assets under a live loop. + const calls: string[] = []; + await expect(prepareServiceInstall("scheduler", { + platform: "win32", + windowsPartA: async () => partAResult("survived", { reason: "unbound-wrapper", registration: { scheduler: "absent", winsw: "nonexistent" } }), + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + })).rejects.toThrow(/could not be proven stopped \(survived\)[\s\S]*could not be tied to the service/); + expect(calls).toEqual([]); + }); + + test("a manager that refuses to stop aborts install preparation before anything else", async () => { + const calls: string[] = []; + await expect(prepareServiceInstall("scheduler", { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("denied"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + })).rejects.toThrow("could not be proven stopped (denied)"); + expect(calls).toEqual(["partA"]); + }); + + test("install and backend switch stop when another program holds the port, naming it", async () => { + for (const backend of ["scheduler", "native"] as const) { + let installed = false; + await expect(installServiceSafely(backend, () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => partAResult("stopped"), + windowsPartB: async () => partBResult("port-held-by-foreign"), + stopTrackedProxy: async () => {}, + })).rejects.toThrow(/nothing was published or started\. Another program \(PID 777\) holds port 10100/); + expect(installed).toBe(false); + } + await expect(prepareServiceInstall("scheduler", { + platform: "win32", windowsPartA: async () => partAResult("stopped"), + windowsPartB: async () => partBResult("survived"), stopTrackedProxy: async () => {}, + })).rejects.toThrow("nothing was published or started"); + }); + + test("Part B checks the stopped proxy's runtime port, resolved before the tracked cleanup", async () => { + // The defect: `windowsServiceStopPort()` was resolved AFTER the tracked cleanup removed + // the runtime record, so it fell through to the wrapper's port (10100) and Part B + // confirmed the wrong port while the proxy's own 12000 was never checked. + const calls: string[] = []; + let runtimePort: number | null = 12000; + const ports: number[] = []; + await prepareServiceInstall("scheduler", { + platform: "win32", + windowsStopPort: () => { + calls.push("port"); + // The runtime record is what makes this the stopped proxy's port; the wrapper's + // 10100 is only a fallback once that record is gone. + return runtimePort ?? 10100; + }, + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), + claimTeardown: () => ({ nonce: "fixture" }), + stopTrackedProxy: async () => { calls.push("stop:standalone"); runtimePort = null; }, + windowsPartB: async request => { calls.push("partB"); ports.push(request.port); return partBResult("freed"); }, + }); + expect(calls).toEqual(["port", "partA", "stop:standalone", "partB"]); + expect(ports).toEqual([12000]); + }); + + test("nothing registered: no Part B and no hold", async () => { const calls: string[] = []; await prepareServiceInstall("scheduler", { platform: "win32", - diagnose: () => ({ supported: true, installed: false, enabled: false, running: false, viable: false, startable: false, stale: false, conflict: false, backend: null, summary: "test" }), - managerOps: () => ({ - status: () => null, - stop: () => { calls.push("stop:scheduler"); throw new Error("no task to stop"); }, - }), - killServiceWrappers: () => { calls.push("kill:wrappers"); killed = true; }, + windowsPartA: async () => { calls.push("partA"); return partAResult("absent"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async () => "dead", stopTrackedProxy: async () => { calls.push("stop:standalone"); }, }); - expect(killed).toBe(true); - expect(calls).toEqual(["kill:wrappers", "stop:standalone"]); + expect(calls).toEqual(["partA", "stop:standalone"]); }); - test("a manager that refuses to stop aborts install preparation before the wrapper kill", async () => { - // Propagating the refusal is the fail-closed contract — and killing the wrapper - // of a service whose task is still running would break that surviving service. + test("a stopped Part A claims a receipt before the tracked stop and forwards its nonce", async () => { + // #7771 round 9: Part A leaves the scheduler task registered, so the tracked stop's + // /api/stop answers 409 respawnable_service without a receipt. stopProxy then throws and + // the install aborts while the old proxy keeps serving. The install claims the receipt + // with Part A's record before the stop, exactly as `ocx service stop` does. + const calls: string[] = []; + const claimed: Array<{ endpoint: { hostname: string; port: number }; partA: unknown }> = []; + const stopIo: Array<{ deferSharedTeardownNonce?: string }> = []; + const cleared: string[] = []; + let installed = false; + await installServiceSafely("scheduler", () => { calls.push("install"); installed = true; }, { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, + windowsStopPort: () => 12000, + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), + claimTeardown: (endpoint, _source, _pid, _handover, partA) => { + calls.push("claim"); + claimed.push({ endpoint, partA }); + return { nonce: "nonce-1" }; + }, + clearTeardown: nonce => { calls.push("clear"); cleared.push(nonce); return true; }, + stopTrackedProxy: async io => { calls.push("stop:standalone"); stopIo.push(io ?? {}); }, + }); + expect(installed).toBe(true); + expect(calls).toEqual(["partA", "claim", "stop:standalone", "partB", "install", "clear"]); + expect(claimed).toEqual([{ + endpoint: { hostname: "127.0.0.1", port: 12000 }, + partA: { partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }, + }]); + expect(stopIo).toEqual([{ target: { pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }, deferSharedTeardownNonce: "nonce-1", skipPortWait: true, orphanDiscoveryCapMs: expect.any(Number) }]); + // The install replaces the proxy, so shared config stays pointed at OpenCodex and the + // obligation is discharged only once the new service is installed and started. + expect(cleared).toEqual(["nonce-1"]); + }); + + test("an install that fails after the stop keeps the receipt for the next stop", async () => { + const cleared: string[] = []; + await expect(installServiceSafely("scheduler", () => { throw new Error("install failed"); }, { + platform: "win32", + windowsPartA: async () => partAResult("stopped"), + windowsPartB: async () => partBResult("freed"), + windowsStopPort: () => 12000, + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), + claimTeardown: () => ({ nonce: "nonce-2" }), + clearTeardown: nonce => { cleared.push(nonce); return true; }, + stopTrackedProxy: async () => {}, + })).rejects.toThrow("install failed"); + expect(cleared).toEqual([]); + }); + + test("a receipt that cannot be claimed refuses before the tracked stop", async () => { const calls: string[] = []; await expect(prepareServiceInstall("scheduler", { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps: () => ({ - status: () => "present", - stop: () => { calls.push("stop:scheduler"); throw new Error("end refused"); }, - }), - killServiceWrappers: () => { calls.push("kill:wrappers"); }, + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + windowsStopPort: () => 12000, + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), + claimTeardown: () => { throw new Error("disk full"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, - })).rejects.toThrow("end refused"); - expect(calls).toEqual(["stop:scheduler"]); + })).rejects.toThrow(/deferred-teardown receipt could not be recorded[\s\S]*disk full/); + expect(calls).toEqual(["partA"]); + }); + + test("an absent manager still proves every endpoint down before the install callback", async () => { + // #7771 round 9: with Part A absent, Part B is skipped and the tracked stop can return + // "none" after its orphan-discovery cap while an unresponsive proxy still holds the port. + // The install must not publish assets over it. + const calls: string[] = []; + let installed = false; + await expect(installServiceSafely("scheduler", () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("absent"); }, + windowsStopPort: () => 12000, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async endpoint => (endpoint.port === 10100 ? "live" : "dead"), + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + })).rejects.toThrow(/port 10100/); + expect(installed).toBe(false); + expect(calls).toEqual(["partA", "stop:standalone"]); + }); + + test("an absent manager with an unreadable endpoint refuses too", async () => { + let installed = false; + await expect(installServiceSafely("scheduler", () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => partAResult("absent"), + windowsStopPort: () => 12000, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }], + endpointLiveness: async () => "unknown", + stopTrackedProxy: async () => {}, + })).rejects.toThrow(/could not be confirmed down/); + expect(installed).toBe(false); + }); + + test("an absent manager with every endpoint dead proceeds without a receipt", async () => { + const calls: string[] = []; + let installed = false; + await installServiceSafely("scheduler", () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => partAResult("absent"), + windowsStopPort: () => 12000, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }], + endpointLiveness: async () => "dead", + resolveTarget: async () => null, + claimTeardown: () => { calls.push("claim"); return { nonce: "unused" }; }, + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + }); + expect(installed).toBe(true); + expect(calls).toEqual(["stop:standalone"]); }); - test("non-Windows install preparation never invokes the scheduler wrapper kill", async () => { - let killed = 0; + test("non-Windows install preparation never reaches the checked Windows stop", async () => { + let partA = 0; await prepareServiceInstall("scheduler", { platform: "linux", managerOps: () => ({ status: () => "present", stop: () => {} }), - killServiceWrappers: () => { killed++; }, + windowsPartA: async () => { partA++; return partAResult("stopped"); }, stopTrackedProxy: async () => {}, }); - expect(killed).toBe(0); + expect(partA).toBe(0); }); - test("live Desktop supervision refuses install preparation before the wrapper kill", async () => { - // The pre-mutation supervision refusal verified here stays ahead of every - // cleanup step: a desktop-owned runtime keeps its respawner until the - // operator quits Desktop and retries. + test("live Desktop supervision refuses install preparation before the checked stop", async () => { + // The pre-mutation supervision refusal stays ahead of every cleanup step: a desktop-owned + // runtime keeps its respawner until the operator quits Desktop and retries. const calls: string[] = []; await expect(prepareServiceInstall("scheduler", { platform: "win32", inspectSupervision: () => ({ kind: "desktop", runtimePid: 4242, supervisorPid: 4200, app: "opencodex-desktop", proxy: "ocx" }), - diagnose: () => { calls.push("diagnose"); return { supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }; }, - managerOps: () => { calls.push("manager"); return { status: () => "present", stop: () => {} }; }, - killServiceWrappers: () => { calls.push("kill:wrappers"); }, + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, })).rejects.toThrow(/OpenCodex Desktop supervises the running proxy/); expect(calls).toEqual([]); diff --git a/tests/service/service-runtime-preflight.test.ts b/tests/service/service-runtime-preflight.test.ts index c6a2aa4a9ad..211f79ec869 100644 --- a/tests/service/service-runtime-preflight.test.ts +++ b/tests/service/service-runtime-preflight.test.ts @@ -14,6 +14,7 @@ import { recordOwnedConfigPath, CONFIG_OWNER_FILE, CONFIG_UNINSTALL_MANIFEST } f import type { ServiceDiagnostic } from "../../src/service/diagnostics"; import { windowsEnvIndirectBatchValue } from "../../src/lib/win-paths"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; import { repoPath } from "../helpers/repo-root"; const runtime: DurableBunRuntime = { path: process.execPath, source: "override", overrideEnv: "OPENCODEX_BUN_PATH" }; @@ -43,9 +44,8 @@ describe("service selected-runtime admission", () => { const deny = () => { throw new RuntimePreflightError(reason); }; const deps = { platform: "win32" as const, selectRuntime: () => runtime, configDir: () => temporary(), assertRuntimeWritable: deny }; await expect(installServiceSafely("scheduler", () => { effects.push("install"); }, { - ...deps, diagnose: () => { effects.push("diagnose"); return diagnostic; }, - managerOps: () => ({ status: () => "present", stop: () => { effects.push("stop"); } }), - killServiceWrappers: () => {}, + ...deps, + windowsPartA: async () => { effects.push("stop"); return partAResult("stopped"); }, stopTrackedProxy: async () => { effects.push("proxy"); }, })).rejects.toMatchObject({ code: "OCX_RUNTIME_PREFLIGHT_FAILED", reason }); const home = join(temporary(), "absent-home"); @@ -89,13 +89,12 @@ describe("service selected-runtime admission", () => { }, { platform: "win32", configDir: temporary, selectRuntime: () => { selections++; return selected; }, assertRuntimeWritable: admitted => { order.push("preflight"); expect(admitted).toEqual(runtime); selected.path = "unprobed.exe"; }, - diagnose: () => diagnostic, - managerOps: () => ({ status: () => "present", stop: () => { order.push("stop"); } }), - killServiceWrappers: () => { order.push("wrappers"); }, + windowsPartA: async () => { order.push("stop"); return partAResult("stopped"); }, + windowsPartB: async () => { order.push("confirm"); return partBResult("freed"); }, stopTrackedProxy: async () => { order.push("proxy"); }, }); expect(selections).toBe(1); - expect(order).toEqual(["preflight", "stop", "wrappers", "proxy", "install"]); + expect(order).toEqual(["preflight", "stop", "proxy", "confirm", "install"]); }); for (const backend of ["scheduler", "native"] as const) { @@ -138,9 +137,9 @@ describe("service selected-runtime admission", () => { }; const ops = platformOps(backend, { platform: "win32", windowsInstall: effects })!; const result = installServiceSafely(backend, ops.install, { - ...admission, diagnose: () => ({ ...diagnostic, backend }), - managerOps: () => ({ status: () => "present", stop: () => { order.push("stop"); } }), - killServiceWrappers: () => { order.push("wrappers"); }, + ...admission, + windowsPartA: async () => { order.push("stop"); return partAResult("stopped"); }, + windowsPartB: async () => { order.push("confirm"); return partBResult("freed"); }, stopTrackedProxy: async () => { order.push("proxy"); }, }); if (refuses) { @@ -149,8 +148,8 @@ describe("service selected-runtime admission", () => { } else { await result; expect(order).toEqual(backend === "scheduler" - ? ["preflight", "stop", "wrappers", "proxy", "ownership", "remove-native", "installer-stop", "assets", "/create", "/run", "state"] - : ["preflight", "stop", "wrappers", "proxy", "account", "native-config", "/query", "winsw", "state"]); + ? ["preflight", "stop", "proxy", "confirm", "ownership", "remove-native", "installer-stop", "assets", "/create", "/run", "state"] + : ["preflight", "stop", "proxy", "confirm", "account", "native-config", "/query", "winsw", "state"]); expect(JSON.parse(readFileSync(join(home, "state.json"), "utf8"))).toMatchObject({ backend, bunPath: runtime.path }); } expect(probes).toBe(1); expect(selections).toBe(1); diff --git a/tests/service/service-stop-verification.test.ts b/tests/service/service-stop-verification.test.ts index 5f228b9e2ad..1c37a17287f 100644 --- a/tests/service/service-stop-verification.test.ts +++ b/tests/service/service-stop-verification.test.ts @@ -1,5 +1,7 @@ -import { describe, expect, test } from "bun:test"; +import { describe, expect, spyOn, test } from "bun:test"; +import * as proxyLiveness from "../../src/server/proxy-liveness"; import { proxyStillLiveAfterStop } from "../../src/service"; +import { WINDOWS_STOP_CAPS } from "../../src/service/windows-stop-budget"; /** * #764: `ocx service stop` reported success while the proxy kept running, and native Codex was @@ -126,3 +128,69 @@ describe("service stop verification (#764)", () => { } }); }); + +/** + * #7771: the tracked-proxy stop's orphan discovery is part of the checked Windows stop's + * admitted budget (030 "Time budget"), which reserves 3 s for it. The helper used to + * hard-code 7 s for every platform and every caller, so a Windows stop that had already + * paid for Part A and Part B could still spend 7 s on discovery — past the deadline its + * admission had just proven — and a non-Windows stop must keep today's 7 s. + */ +describe("orphan discovery is capped by the caller, not hard-coded (#7771)", () => { + // The helper calls findLiveProxy directly, so the spy is the seam: it records the + // deadline the helper asked for and answers "nothing live" so no process is touched. + const discoveredBudget = async (io: Record): Promise => { + const { stopTrackedProxyForServiceCommand } = await import("../../src/service/orchestration"); + let seen = -1; + const started = Date.now(); + const spy = spyOn(proxyLiveness, "findLiveProxy").mockImplementation((async (options: { deadlineAt?: number }) => { + seen = (options.deadlineAt ?? 0) - started; + return null; + }) as never); + try { + await stopTrackedProxyForServiceCommand(io as never); + } finally { + spy.mockRestore(); + } + return seen; + }; + + test("the default keeps today's 7 s window", async () => { + const budget = await discoveredBudget({}); + expect(budget).toBeGreaterThanOrEqual(6_900); + expect(budget).toBeLessThanOrEqual(7_100); + }); + + test("a Windows checked caller's admitted 3 s is what discovery gets", async () => { + const budget = await discoveredBudget({ orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery }); + expect(WINDOWS_STOP_CAPS.orphanDiscovery).toBe(3_000); + expect(budget).toBeGreaterThanOrEqual(2_900); + expect(budget).toBeLessThanOrEqual(3_100); + }); + + test("an absent Part A still names the cap: discovery is the only port proof left", async () => { + const { windowsServiceRemovalStop, windowsServiceStopBeforeRestore } = await import("../../src/service/cli-windows-stop"); + const { setWindowsServiceStopHooksForTests } = await import("../../src/service/windows-service-stop"); + const { partAResult } = await import("../helpers/windows-stop-caller-fixtures"); + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + try { + // The endpoint probes are injected: both callers would otherwise fall through to + // the real ones, which on Windows spend their full window before answering, and this + // test is about the discovery budget, not about that wait. + const probed: unknown[] = []; + for (const run of [windowsServiceStopBeforeRestore, windowsServiceRemovalStop]) { + const seen: unknown[] = []; + await run({ + ownsSharedTeardown: () => false, + resolveTarget: async () => null, + stopTracked: async io => { seen.push(io); return "stopped"; }, + endpointLiveness: async endpoint => { probed.push(endpoint); return "dead"; }, + }); + expect(seen).toEqual([{ target: null, skipPortWait: false, orphanDiscoveryCapMs: 3_000 }]); + } + expect(probed.length).toBe(2); + } finally { + setWindowsServiceStopHooksForTests(null); + } + }); +}); diff --git a/tests/service/service.test.ts b/tests/service/service.test.ts index 03c5f7bcbec..1dc9d854562 100644 --- a/tests/service/service.test.ts +++ b/tests/service/service.test.ts @@ -18,6 +18,7 @@ import { resolveCurrentWindowsPrincipal, setWindowsPrincipalRunnerForTests } fro import { setAsyncIcaclsRunnerForTests, setIcaclsRunnerForTests } from "../../src/lib/windows-secret-acl"; import type { OcxConfig } from "../../src/types"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { ALL_PART_A_FAILURES, partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; import { repoRoot } from "../helpers/repo-root"; const TEST_WINDOWS_TASK_SID = "S-1-5-21-111-222-333-1001"; @@ -2269,24 +2270,16 @@ describe("service lifecycle cleanup ordering", () => { ]); }); - test("service install stops the recorded backend, requested backend, and standalone before loading assets", async () => { + test("Windows service install runs the checked stop, the standalone stop and Part B before loading assets", async () => { const calls: string[] = []; - const managerOps = (backend: "scheduler" | "native") => ({ - status: () => { calls.push(`status:${backend}`); return "present"; }, - stop: () => { calls.push(`stop:${backend}`); }, - }); await installServiceSafely("native", () => { calls.push("install:native"); }, { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps, - killServiceWrappers: () => { calls.push("kill:wrappers"); }, + // #7771: Part A stops every registered manager (both on a conflicting install) and ends the bound wrappers. + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, }); - expect(calls).toEqual([ - "status:scheduler", "stop:scheduler", - "status:native", "stop:native", - "kill:wrappers", "stop:standalone", "install:native", - ]); + expect(calls).toEqual(["partA", "stop:standalone", "partB", "install:native"]); }); test("legacy systemd reports an absent unit without stopping cleanup or blocking install", async () => { @@ -2379,21 +2372,12 @@ describe("service lifecycle cleanup ordering", () => { }); test("service install fails closed before install on manager or standalone cleanup errors", async () => { - for (const failure of ["status", "stop", "standalone"] as const) { + for (const failure of ["partA", "partB", "standalone"] as const) { let installed = false; const run = installServiceSafely("scheduler", () => { installed = true; }, { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps: () => ({ - status: () => { - if (failure === "status") throw new Error("status failed"); - return "present"; - }, - stop: () => { - if (failure === "stop") throw new Error("stop failed"); - }, - }), - killServiceWrappers: () => {}, + windowsPartA: async () => { if (failure === "partA") throw new Error("partA failed"); return partAResult("stopped"); }, + windowsPartB: async () => { if (failure === "partB") throw new Error("partB failed"); return partBResult("freed"); }, stopTrackedProxy: async () => { if (failure === "standalone") throw new Error("standalone failed"); }, @@ -2403,16 +2387,14 @@ describe("service lifecycle cleanup ordering", () => { } }); - test("conflicting Windows install preparation stops both managers", async () => { - const stopped: string[] = []; - await prepareServiceInstall("scheduler", { - platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: false, startable: false, stale: false, conflict: true, backend: "scheduler", summary: "test" }), - managerOps: backend => ({ status: () => "present", stop: () => { stopped.push(backend); } }), - killServiceWrappers: () => { stopped.push("wrappers"); }, - stopTrackedProxy: async () => {}, - }); - expect(stopped).toEqual(["scheduler", "native", "wrappers"]); + test("Windows install preparation refuses before replacing anything unless Part A stopped or found nothing", async () => { + for (const outcome of ALL_PART_A_FAILURES) { + const stopped: string[] = []; + await expect(prepareServiceInstall("scheduler", { platform: "win32", windowsPartA: async () => partAResult(outcome), + windowsPartB: async () => { stopped.push("partB"); return partBResult("freed"); }, + stopTrackedProxy: async () => { stopped.push("standalone"); } })).rejects.toThrow(`could not be proven stopped (${outcome})`); + expect(stopped).toEqual([]); + } }); test("direct service stop kills the tracked proxy before restoring native Codex", async () => { @@ -2474,7 +2456,7 @@ describe("service lifecycle cleanup ordering", () => { test("Windows service uninstall verifies task deletion before removing assets", async () => { const service = await readText("src/service/windows-ops.ts"); - const uninstallWindows = service.slice(service.indexOf("function uninstallWindows()"), service.indexOf("function classifyWindowsServiceStop(")); + const uninstallWindows = service.slice(service.indexOf("function uninstallWindows()")); expect(uninstallWindows).toContain("probeWindowsSchedulerTask(TASK)"); expect(uninstallWindows).toContain("windowsServiceScriptPath()"); @@ -2493,12 +2475,12 @@ describe("service lifecycle cleanup ordering", () => { expect(service).toContain('type TrackedProxyCleanupResult = "none" | "stale" | "stopped";'); const health = await readText("src/service/health.ts"); expect(health).toContain('import { proxyIdentityAt } from "../server/proxy-liveness";'); - expect(service).toContain("async function stopTrackedProxyIfRunning(): Promise"); + expect(service).toContain("async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise"); expect(service).toContain("...SERVICE_STOP_LIVENESS"); expect(service).toContain("deadlineAt:"); expect(service).toContain("SERVICE_STOP_LIVENESS"); - expect(service).toContain("await stopProxy(trackedKillPid);"); - expect(service).toContain("await stopProxy(liveKillPid);"); + expect(service).toContain("await stopProxy(trackedKillPid, io);"); + expect(service).toContain("await stopProxy(liveKillPid, io);"); expect(service).toContain("removePid(pid);"); expect(service).toContain('return "stopped";'); }); @@ -2540,7 +2522,7 @@ describe("service lifecycle cleanup ordering", () => { test("service command cleanup logs kill failures without skipping restore/delete", async () => { const service = await readText("src/service/orchestration.ts"); - expect(service).toContain("async function stopTrackedProxyForServiceCommand(): Promise"); + expect(service).toContain("async function stopTrackedProxyForServiceCommand(io: TrackedProxyStopIo = {}): Promise"); expect(service).toContain("catch (err)"); expect(service).toContain("Failed to stop proxy"); expect(service).toContain('return "none";'); diff --git a/tests/service/stop-deferred-teardown.test.ts b/tests/service/stop-deferred-teardown.test.ts index 86d418b9ac6..5d6bff20d4b 100644 --- a/tests/service/stop-deferred-teardown.test.ts +++ b/tests/service/stop-deferred-teardown.test.ts @@ -1284,7 +1284,7 @@ test("the route refuses a self-unload before the manager is touched", () => { expect(handler).toContain('code: "self_unload_service"'); // Same invariant the Windows guard carries: refuse BEFORE acting, and say so. expect(handler.indexOf('code: "self_unload_service"')) - .toBeLessThan(handler.indexOf("stopServiceIfInstalledDetailed()")); + .toBeLessThan(handler.indexOf("routeServiceStop(url, holdsReceipt)")); const branch = handler.slice(handler.indexOf('code: "self_unload_service"'), handler.indexOf('code: "self_unload_service"') + 600); expect(branch).toContain("Nothing was changed."); expect(branch).toContain("ocx stop"); diff --git a/tests/service/stop-direct-fetch.test.ts b/tests/service/stop-direct-fetch.test.ts index 9ff9dfc3297..4d2cffde617 100644 --- a/tests/service/stop-direct-fetch.test.ts +++ b/tests/service/stop-direct-fetch.test.ts @@ -4,6 +4,28 @@ import { createServer, type Socket } from "node:net"; import { stopProxyGracefully } from "../../src/lib/process-control"; import { repoPath } from "../helpers/repo-root"; +/** Argument text of each `await (...)`, balanced so nested calls stay separate. */ +function callArguments(body: string, call: string): string[] { + const marker = `await ${call}(`; + const found: string[] = []; + let from = 0; + for (;;) { + const at = body.indexOf(marker, from); + if (at < 0) return found; + let depth = 0; + let index = at + marker.length - 1; + for (; index < body.length; index += 1) { + if (body[index] === "(") depth += 1; + else if (body[index] === ")") { + depth -= 1; + if (depth === 0) break; + } + } + found.push(body.slice(at + marker.length, index)); + from = index; + } +} + const stopRunner = ` const { stopProxyGracefully } = await import("./src/lib/process-control.ts"); const port = Number(process.env.OCX_TEST_STOP_PORT); @@ -108,10 +130,14 @@ describe("local stop transport", () => { // These paths mutate service/process state; exercise their shared real transport above // and verify each caller delegates without overriding it with the global fetch. - for (const [file, functionName, count] of [ - ["src/cli/index.ts", "handleStopUnlocked", 1], - ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 2], - ["src/cli/index.ts", "handleUninstall", 2], + // `handleUninstall` delegates its two stops to `stopUninstallProxy` (#7771), which owns + // the receipt and the no-forced-fallback rule; the transport default is still not overridden. + for (const [file, functionName, count, call] of [ + ["src/cli/index.ts", "handleStopUnlocked", 1, "stopProxy"], + ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 3, "stopProxy"], + // One site: the target is resolved before the receipt, so both the tracked and the + // discovered path reach the stop through the same call (#7771 round 8). + ["src/cli/uninstall-stop.ts", "stopUninstallProxy", 1, "io.stop"], ] as const) { test(`${functionName} retains the direct stop default`, () => { const source = readFileSync(repoPath(file), "utf8"); @@ -119,12 +145,12 @@ describe("local stop transport", () => { expect(start).toBeGreaterThanOrEqual(0); const end = source.indexOf("\n}", start); expect(end).toBeGreaterThan(start); - const calls = [...source.slice(start, end).matchAll(/await stopProxy\(([\s\S]*?)\);/g)]; + const calls = callArguments(source.slice(start, end), call); expect(calls).toHaveLength(count); - for (const call of calls) { + for (const args of calls) { // Connected stops also retain the shared default's refusal-status handling. - expect(call[1]!).not.toMatch(/\bfetchFn\s*:/); - expect(call[1]!.replace(/as typeof fetch\b/g, "")).not.toMatch(/\bfetch\b/); + expect(args).not.toMatch(/\bfetchFn\s*:/); + expect(args.replace(/as typeof fetch\b/g, "")).not.toMatch(/\bfetch\b/); } }); } diff --git a/tests/update/update-effect-authority.test.ts b/tests/update/update-effect-authority.test.ts index 0845f47f76f..4f07e4cc837 100644 --- a/tests/update/update-effect-authority.test.ts +++ b/tests/update/update-effect-authority.test.ts @@ -50,7 +50,7 @@ for (const phase of ["identity", "liveness", "kill-liveness", "escalation", "sta if (phase !== "stable") { home = "same"; expect(policy.authorizeMutation("publish")).toBe(false); } }); -for (const operation of ["wrapper", "winsw", "scheduler", "tray", "pid", "runtime", "registry"]) +for (const operation of ["winsw", "tray", "pid", "runtime", "registry"]) for (const changed of [true, false]) test(`leaf ${operation} gates after preparation (${changed})`, () => { let home: "same" | "unknown" = "same", effects = 0; const policy = createUpdateServiceHomePolicy(() => ({ kind: home }), () => {}); @@ -59,26 +59,12 @@ for (const operation of ["wrapper", "winsw", "scheduler", "tray", "pid", "runtim throw Object.assign(new Error("refused"), { code: "EUPDATEAUTHORITY" }); }; const effect = () => { effects++; return "stopped"; }; let invoke: () => unknown; - if (operation === "wrapper") { - const fn = load("src/lib/windows-service-wrappers.ts", "killWindowsSchedulerWrappers", { - process: { platform: "win32" }, resolveTrustedWindowsPowerShellExe: prepare, - windowsWrapperKillScript: () => "fixture", spawnSync: effect, - }); - const wrapped = adapter("guardedKillWindowsWrappers", { killWindowsSchedulerWrappers: fn }); - invoke = () => wrapped(policy.authorizeMutation, { scriptPath: "script", launcherPath: "launcher" }); - } else if (operation === "winsw") { + if (operation === "winsw") { const run = load("src/lib/winsw.ts", "runWinsw", { winswExePath: prepare, execFileSync: effect, SERVICE_STOPWAIT_TIMEOUT_MS: 1, SERVICE_COMMAND_TIMEOUT_MS: 1, }); const fn = load("src/lib/winsw.ts", "stopWinswService", { runWinsw: run, statusWinswRaw: () => "stopped" }); invoke = () => adapter("guardedStopWinsw", { stopWinswService: fn })(policy.authorizeMutation); - } else if (operation === "scheduler") { - const query = load("src/service/windows-scheduler.ts", "querySchtasks", { - isTestHomeGuardArmed: () => false, querySchtasksForTests: null, windowsSchtasks: prepare, runFile: effect, - }); - const schtasks = load("src/service/windows-scheduler.ts", "schtasks", { querySchtasks: query, toWindowsSchtasksError: (e: unknown) => e }); - const stop = load("src/service/windows-ops.ts", "stopWindows", { schtasks, TASK: "fixture", isWindowsSchedulerEndBenign: () => false }); - invoke = () => adapter("guardedStopWindows", { stopWindows: stop })(policy.authorizeMutation); } else if (operation === "tray") { const signal = load("src/tray/windows.ts", "signalTrayStop", { readHeartbeat: () => ({}), windowsPowerShellPath: prepare, currentEntry: () => ({}), @@ -107,6 +93,24 @@ for (const operation of ["wrapper", "winsw", "scheduler", "tray", "pid", "runtim if (changed) { home = "same"; expect(policy.authorizeMutation("publish")).toBe(false); } }); +// #7771: the scheduler stop and the wrapper sweep are one checked operation (Part A). Its service +// gate is the authorizer Part A runs at entry and before every terminate. +for (const changed of [true, false]) test(`leaf checked scheduler stop gates after preparation (${changed})`, async () => { + let home: "same" | "unknown" = "same", effects = 0; + const policy = createUpdateServiceHomePolicy(() => ({ kind: home }), () => {}); + const runWindowsServiceStopPartA = async (request: { authorize: () => void; allowUnderOwnWrapper?: boolean }) => { + expect(request.allowUnderOwnWrapper).toBe(true); + if (changed) home = "unknown"; // preparation: the registration probe and the bind + await request.authorize(); + effects++; + return { outcome: "stopped" }; + }; + const invoke = adapter("guardedStopWindowsChecked", { runWindowsServiceStopPartA })(policy.authorizeMutation); + if (changed) await expect(invoke).rejects.toThrow(); else expect(await invoke).toBe("stopped"); + expect(effects).toBe(changed ? 0 : 1); + if (changed) { home = "same"; expect(policy.authorizeMutation("publish")).toBe(false); } +}); + test("Windows kill refusal propagates even when a later liveness check would report exit", () => { let reads = 0; const fn = adapter("guardedKillProxy", { diff --git a/tests/update/update-recovery-boundaries.test.ts b/tests/update/update-recovery-boundaries.test.ts index bde387661c9..e3d4f41ac34 100644 --- a/tests/update/update-recovery-boundaries.test.ts +++ b/tests/update/update-recovery-boundaries.test.ts @@ -136,10 +136,12 @@ for (const outcome of ["false", "throw", "promise"] as const) test(`async transa test("production pinned caller checks current service permission inside its wrapper callback", async () => { const prepare = section(worker, "function preparePortForPinnedStart(", "function spawnDetachedStart("); - const caller = section(worker, ' if (!homePolicy.authorizeMutation("runtime")) return false;\n if (preparePort(', " const ready = await waitForGhost("); + // #7771: the service stop now runs right before the synchronous preparation, so its outcome can + // abort; the service permission is still read immediately before it. + const caller = section(worker, ' if (!homePolicy.authorizeMutation("runtime")) return false;\n // #7771', " const ready = await waitForGhost("); for (const changed of ["same", "absent", "unknown"] as const) { let reads = 0, wrappers = 0, cleanup = 0; - const homePolicy = createUpdateServiceHomePolicy(() => ({ kind: ++reads < 3 ? "same" : changed }), () => {}); + const homePolicy = createUpdateServiceHomePolicy(() => ({ kind: ++reads < 2 ? "same" : changed }), () => {}); const result = await execute(prepare + "const preparePort = preparePortForPinnedStart;" + caller, { homePolicy, reclaimGate: () => homePolicy.authorizeMutation("runtime"), job: {}, port: 19001, listPids: () => [], aliveFn: () => false, verifyOcx: () => null, serviceInstalled: true, @@ -241,7 +243,7 @@ for (const caller of callers) for (const outcome of ["vetoed", "refused", true] test("production Node full stop refuses lost service permission before Windows wrapper cleanup", async () => { const stop = section(node, " const stopNeeded =", " const replacementOwnership ="); const childStop = section(readFileSync(repoPath("src/service/orchestration.ts"), "utf8"), - "export function stopServiceIfInstalledDetailed()", "/** Delete install-state files").replace("export ", ""); + "export async function stopServiceIfInstalledReport(", "/** Delete install-state files").replaceAll("export ", ""); const guard = section(readFileSync(repoPath("src/service/guards.ts"), "utf8"), "export function assertServiceEnvironmentMatchesInstall(", "/**\n * Refuse a management").replace("export ", ""); for (const [captured, changed] of [[true, "same"], [true, "absent"], [true, "unknown"], [false, "absent"]] as const) { @@ -260,8 +262,10 @@ test("production Node full stop refuses lost service permission before Windows w resolveServiceState: () => ({ kind: changed === "same" ? "state" : "none", state: {} }), currentCodexHome: () => "", serviceCodexHomeMatchesInstall: () => true, currentOpenCodexHome: () => "", servicePathMatchesInstall: () => true, - probeWindowsSchedulerTask: () => ({ status: "absent" }), statusWinswRaw: () => "nonexistent", - killWindowsServiceWrapperProcesses: () => { wrappers++; }, classifyWindowsServiceStop: () => "absent", + // #7771: Part A of the checked stop is the Windows wrapper cleanup now; it is called + // synchronously at the first await, so a lost permission must refuse before it. + runWindowsServiceStopPartA: async () => { wrappers++; return { outcome: "absent" }; }, + serviceStopOutcomeFromWindows: () => "absent", spawnSync: () => { stops++; const js = new Bun.Transpiler({ loader: "ts" }).transformSync(guard + childStop); diff --git a/tests/update/update-windows-wrapper-authority.test.ts b/tests/update/update-windows-wrapper-authority.test.ts index 1b6456d71ba..aff72dcb9a2 100644 --- a/tests/update/update-windows-wrapper-authority.test.ts +++ b/tests/update/update-windows-wrapper-authority.test.ts @@ -7,7 +7,7 @@ import { createUpdateServiceHomePolicy, type UpdateServiceHomeAuthority } from " const helper = readFileSync(repoPath("src/update/job-windows-restart.ts"), "utf8"); const worker = readFileSync(repoPath("src/update/job.ts"), "utf8"); -const helperBody = helper.slice(helper.indexOf("export function stopWindowsServiceWrappersBestEffort")).replace("export ", ""); +const helperBody = helper.slice(helper.indexOf("export const WINDOWS_SERVICE_STOP_ABORT")).replaceAll("export ", ""); function execute(body: string, deps: object): any { // Complete production bodies; module loading and external OS/store observations are injected. const js = new Bun.Transpiler({ loader: "ts" }).transformSync(body.replaceAll('await import("../service")', "await moduleIo.service()")); @@ -16,28 +16,50 @@ function execute(body: string, deps: object): any { const authority = (kind: "same" | "absent" | "unknown" | "foreign"): UpdateServiceHomeAuthority => kind === "foreign" ? { kind, subject: "a".repeat(64) } : { kind }; -for (const phase of ["stop", "throwing-stop", "backend-native", "backend-throw"] as const) { +for (const scenario of ["refused stop", "unreadable WinSW state", "conflicting scheduler task"]) { + test(`native backend aborts for ${scenario}`, async () => { + const homePolicy = createUpdateServiceHomePolicy(() => authority("same"), () => {}); + let checked = 0; + const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { + homePolicy, process: { platform: "win32" }, readServiceBackend: () => "native", + stopWinswService: () => { throw new Error("refused"); }, + checkedStop: async () => { checked++; return scenario === "refused stop" ? "denied" : "unknown"; }, + }); + expect(result).toBe(false); + expect(checked).toBe(1); + }); +} + +/** + * Part A of the checked stop (#7771) as the updater sees it: the authorizer at entry, the + * manager stop, then the authorizer again before the wrapper terminate. + */ +function checkedStop(events: string[], home: () => UpdateServiceHomeAuthority, change: () => void) { + return async (authorize: () => void) => { + authorize(); + events.push(`stop:${home().kind}`); + change(); + authorize(); + events.push(`terminate:${home().kind}`); + return "stopped"; + }; +} + +for (const phase of ["checked"] as const) { for (const changed of ["same", "absent", "unknown", "foreign"] as const) { - test(`Windows wrapper helper checks ${changed} service authority after ${phase}`, () => { + test(`Windows service stop checks ${changed} service authority after ${phase}`, async () => { let home = authority("same"); const events: string[] = []; const homePolicy = createUpdateServiceHomePolicy(() => home, () => {}); - execute(helperBody + ';stopWindowsServiceWrappersBestEffort(homePolicy.authorizeMutation);', { - homePolicy, process: { platform: "win32" }, join, getConfigDir: () => "/fixture/own", - readServiceBackend: () => { - if (phase.startsWith("backend")) home = authority(changed); - if (phase === "backend-throw") throw new Error("fixture backend read"); - return phase === "backend-native" ? "native" : "scheduler"; - }, + const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { + homePolicy, process: { platform: "win32" }, + readServiceBackend: () => "scheduler", stopWinswService: () => events.push(`native:${home.kind}`), - stopWindows: () => { - events.push(`stop:${home.kind}`); home = authority(changed); - if (phase === "throwing-stop") throw new Error("fixture scheduler stop"); - }, - killWindowsSchedulerWrappers: () => events.push(`kill:${home.kind}`), + checkedStop: checkedStop(events, () => home, () => { if (phase === "checked") home = authority(changed); }), }); - const stable = phase === "backend-native" ? ["native:same"] : phase === "backend-throw" ? ["kill:same"] : ["stop:same", "kill:same"]; - expect(events).toEqual(changed === "same" ? stable : phase.startsWith("backend") ? [] : ["stop:same"]); + expect(events).toEqual(changed === "same" ? ["stop:same", "terminate:same"] : ["stop:same"]); + // Anything but a proven stop aborts the restart. + expect(result).toBe(changed === "same"); // The same policy must retain unknown/foreign refusal even if registration re-matches. home = authority("same"); expect(homePolicy.authorizeMutation("service")).toBe(changed === "same" || changed === "absent"); @@ -45,10 +67,22 @@ for (const phase of ["stop", "throwing-stop", "backend-native", "backend-throw"] } } +for (const outcome of ["survived", "denied", "unknown", "unverified-budget", "absent", "stopped"] as const) { + test(`Windows service stop aborts the restart unless Part A is stopped or absent: ${outcome}`, async () => { + const homePolicy = createUpdateServiceHomePolicy(() => authority("same"), () => {}); + const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { + homePolicy, process: { platform: "win32" }, readServiceBackend: () => "scheduler", + checkedStop: async (authorize: () => void) => { authorize(); return outcome; }, + }); + expect(result).toBe(outcome === "stopped" || outcome === "absent"); + }); +} + for (const changed of ["same", "absent", "unknown", "foreign"] as const) { - test(`complete worker passes its latched authorizer through Windows wrapper recovery: ${changed}`, async () => { + test(`complete worker passes its latched authorizer through the Windows service stop: ${changed}`, async () => { let home = authority("same"); const events: string[] = []; + let aborted = 0; const from = worker.indexOf("async function restartAfterUpdate("), to = worker.indexOf("/** Compact listen-holder summary", from); expect(from).toBeGreaterThan(0); expect(to).toBeGreaterThan(from); const result = await execute(helperBody + worker.slice(from, to) @@ -56,18 +90,22 @@ for (const changed of ["same", "absent", "unknown", "foreign"] as const) { createUpdateServiceHomePolicy, join, getConfigDir: () => "/fixture/own", process: { platform: "win32", env: {}, pid: 99 }, loadConfig: () => ({}), restartCommand: () => ({ bin: "fixture", args: [], display: "fixture" }), updateJob() {}, + windowsServiceStopAborted: () => { aborted++; return false; }, RESTART_PORT_RECLAIM_MS: 30000, WINDOWS_SERVICE_REPAIR_TIMEOUT_MS: 1000, RESTART_TIMEOUT_MS: 1000, moduleIo: { service: async () => ({ serviceReinstallArgs: () => ["service", "repair"] }) }, readServiceBackend: () => "scheduler", stopWinswService() {}, - stopWindows: () => { events.push(`stop:${home.kind}`); home = authority(changed); }, - killWindowsSchedulerWrappers: () => events.push(`kill:${home.kind}`), + checkedStop: checkedStop(events, () => home, () => { home = authority(changed); }), io: { serviceHomeAuthorityFn: () => home, serviceInstalledFn: () => true, packageLauncherPathFn: () => "fixture", waitForPort: async (_p: unknown, _h: unknown, options: any) => options.mayMutateFn(), listListenPidsFn: () => [], verifyOcxFn: () => null, isAliveFn: () => false, serviceViableFn: () => false, runService: () => ({ status: 1 }), restartServiceManagerFn: async () => "failed", reacquireForDirectStartFn: () => ({ notice: "fixture ownership veto", failed: false }), readPidFn: () => null, spawnStart: () => events.push("spawn") }, }); - expect(events).toEqual(changed === "same" ? ["stop:same", "kill:same"] : ["stop:same"]); - if (changed === "unknown" || changed === "foreign") expect(result).toBe(false); + expect(events).toEqual(changed === "same" ? ["stop:same", "terminate:same"] : ["stop:same"]); + // A refused or unproven service stop aborts before reclaim or any start (#7771). + if (changed !== "same") { + expect(result).toBe(false); + expect(aborted).toBe(1); + } }); } diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts new file mode 100644 index 00000000000..9f6b05ab491 --- /dev/null +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -0,0 +1,242 @@ +/** + * The live adapters of the checked Windows stop (#7771): parsing of the capped schtasks and + * sc.exe answers, the ARM64 strict gate on the terminate, the whole-millisecond timeouts, the + * Part B port sampler and listener identity, and the stopper-ancestry check. Hermetic: no + * command is spawned and no FFI is loaded. + */ +import { describe, expect, test } from "bun:test"; + +import { + classifyScStop, + classifyTaskEnd, + identifyWindowsStopListener, + liveWindowsStopCommandRunner, + parseScQueryState, + parseTaskCsvState, + sampleWindowsStopPort, + stopperUnderHomeWrapper, + wholeTimeoutMs, + windowsStopSnapshot, + windowsStopTerminate, + winswStatusFromScState, + windowsCheckedStopLiveDeps, + WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED, +} from "../../src/service/windows-checked-stop-live"; +import type { WindowsProcessNativeApi } from "../../src/service/windows-process-native"; +import { winswExePath } from "../../src/lib/winsw"; +import { PATHS, entry, launcher, loop, proxy, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; + +const result = (status: number | null, stdout = "", stderr = "", timedOut = false) => ({ status, stdout, stderr, timedOut }); + +describe("ARM64 strict variant (050, 060 'ARM64 decision for PR2')", () => { + test("ships strict: the bounded PowerShell fallback is disabled", () => { + expect(WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED).toBe(false); + }); + + test("without the native primitive the terminate is unavailable and the fallback is never called", async () => { + let fallbackCalls = 0; + const terminate = async () => { fallbackCalls++; return { status: "terminated" as const, exited: true }; }; + // ARM64 has no native API; an x64 whose DLL load failed is the same case. + expect(await windowsStopTerminate(30, 5_100, 3_000, { native: () => null, terminate })).toEqual({ status: "unavailable" }); + expect(fallbackCalls).toBe(0); + }); + + test("with the native primitive the checked terminate runs", async () => { + const calls: unknown[] = []; + const native = {} as WindowsProcessNativeApi; + const terminate = async (pid: number, start: number, options: { timeoutMs: number }) => { + calls.push([pid, start, options.timeoutMs]); + return { status: "terminated" as const, exited: true }; + }; + expect(await windowsStopTerminate(30, 5_100, 2_500, { native: () => native, terminate })) + .toEqual({ status: "terminated", exited: true }); + expect(calls).toEqual([[30, 5_100, 2_500]]); + }); +}); + +describe("whole-millisecond timeouts (execFile reads 0 as unlimited)", () => { + test("a fraction is floored and a value below 1 ms spawns nothing", async () => { + expect(wholeTimeoutMs(1999.9)).toBe(1999); + expect(wholeTimeoutMs(0.9)).toBeNull(); + expect(wholeTimeoutMs(0)).toBeNull(); + expect(wholeTimeoutMs(-5)).toBeNull(); + // The runner answers a sub-millisecond cap as timed out without resolving any program. + expect(await liveWindowsStopCommandRunner("sc", ["query", "x"], 0.4)).toEqual(result(null, "", "", true)); + // The snapshot reads nothing (unreadable) rather than running with no timeout. + expect(await windowsStopSnapshot(0.7)).toBeNull(); + }); +}); + +describe("capped command answers", () => { + test("schtasks /end", () => { + expect(classifyTaskEnd(result(0))).toBe("ended"); + expect(classifyTaskEnd(result(1, "", "ERROR: There is no running instance of the task."))).toBe("not-running"); + expect(classifyTaskEnd(result(1, "", "ERROR: Access is denied."))).toBe("denied"); + expect(classifyTaskEnd(result(1, "", "ERROR: something else"))).toBe("failed"); + expect(classifyTaskEnd(result(null, "", "", true))).toBe("failed"); + }); + + test("schtasks CSV state reads English values and leaves a localized word unknown", () => { + expect(parseTaskCsvState('"\\opencodex-proxy","N/A","Running"')).toBe("running"); + expect(parseTaskCsvState('"\\opencodex-proxy","N/A","Ready"')).toBe("not-running"); + expect(parseTaskCsvState('"\\opencodex-proxy","N/A","Wird ausgeführt"')).toBe("unknown"); + expect(parseTaskCsvState('"\\Tools\\opencodex-proxy","N/A","Running"')).toBe("unknown"); + }); + + test("sc.exe query and stop", () => { + expect(parseScQueryState(result(1060, "[SC] EnumQueryServicesStatus:OpenService FAILED 1060:"))).toBe("nonexistent"); + expect(parseScQueryState(result(0, " STATE : 4 RUNNING"))).toBe("running"); + expect(parseScQueryState(result(0, " STATE : 1 STOPPED"))).toBe("stopped"); + expect(parseScQueryState(result(0, " STATE : 3 STOP_PENDING"))).toBe("pending"); + expect(parseScQueryState(result(5, "[SC] OpenService FAILED 5:"))).toBe("unknown"); + expect(winswStatusFromScState("pending")).toBe("started"); + expect(winswStatusFromScState("unknown")).toBe("unknown"); + expect(classifyScStop(result(0))).toBe("sent"); + expect(classifyScStop(result(1062, "[SC] ControlService FAILED 1062:"))).toBe("sent"); + expect(classifyScStop(result(5, "[SC] OpenService FAILED 5:"))).toBe("denied"); + expect(classifyScStop(result(null, "", "", true))).toBe("failed"); + }); +}); + +describe("Part B port sampler and listener identity", () => { + test("free, listening, unreadable, and an exited owner's row is cleared", async () => { + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [] }) })).toEqual({ state: "free" }); + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: false }) })).toEqual({ state: "unknown" }); + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => true })) + .toEqual({ state: "listening", ownerPid: 77 }); + const drops: Array<[number, readonly number[]]> = []; + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => false, drop: (port, pids) => drops.push([port, pids]) })) + .toEqual({ state: "listening", ownerPid: 77 }); + // Only the exited owner's rows may be cleared: a program that took the port between + // scans owns its own rows, and resetting them would break a connection this stop does + // not own. + expect(drops).toEqual([[10100, [77]]]); + // The stopping process's own socket is not a listener on the stopped port. + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [5] }), selfPid: 5 })).toEqual({ state: "free" }); + }); + + test("the sample honors its timeout: an unfinished scan is unknown, never a hung part", async () => { + const seen: number[] = []; + const started = Date.now(); + // The scan never answers; the sample must give up on the timeout. + const sample = await sampleWindowsStopPort(10100, 60, { + scan: (_port, timeoutMs) => { + seen.push(timeoutMs); + return new Promise(() => undefined); + }, + }); + expect(sample).toEqual({ state: "unknown" }); + expect(seen).toEqual([60]); + expect(Date.now() - started).toBeLessThan(2_000); + }); + + test("a sub-millisecond timeout samples nothing: execFile reads 0 as unlimited", async () => { + let scans = 0; + expect(await sampleWindowsStopPort(10100, 0.4, { scan: () => { scans++; return { ok: true, pids: [] }; } })) + .toEqual({ state: "unknown" }); + expect(await sampleWindowsStopPort(10100, 0, { scan: () => { scans++; return { ok: true, pids: [] }; } })) + .toEqual({ state: "unknown" }); + expect(scans).toBe(0); + }); + + test("the scan sees a floored whole-millisecond budget", async () => { + let seen = -1; + expect(await sampleWindowsStopPort(10100, 499.9, { + scan: (_port, timeoutMs) => { seen = timeoutMs; return { ok: true, pids: [] }; }, + })).toEqual({ state: "free" }); + expect(seen).toBe(499); + }); + + test("ours, foreign, exited, unknown", () => { + expect(identifyWindowsStopListener(42, { alive: () => true, identity: () => "match" })).toBe("ours"); + expect(identifyWindowsStopListener(42, { alive: () => true, identity: () => "mismatch" })).toBe("foreign"); + // Unreadable is not foreign (#7771): a probe that cannot answer must not let the caller + // conclude that another program holds the port. + expect(identifyWindowsStopListener(42, { alive: () => true, identity: () => "unverifiable" })).toBe("unknown"); + expect(identifyWindowsStopListener(42, { alive: () => false, identity: () => "match" })).toBe("exited"); + expect(identifyWindowsStopListener(null)).toBe("unknown"); + // A throwing probe is unreadable too, never foreign. + expect(identifyWindowsStopListener(42, { + alive: () => true, + identity: () => { throw new Error("denied"); }, + })).toBe("unknown"); + }); +}); + +describe("who may run Part A: the stopper's ancestry", () => { + test("a stopper under this home's wrapper is detected; one under a plain shell is not", () => { + const tree = supervisedTree(); + // ocx launched as a child of the looping cmd.exe (or of the proxy it supervises). + expect(stopperUnderHomeWrapper([...tree, entry(70, 30, "ocx.exe", "ocx stop", 6_000)], PATHS, 70)).toBe(true); + expect(stopperUnderHomeWrapper([...tree, entry(71, 42, "bun.exe", "bun worker", 6_000)], PATHS, 71)).toBe(true); + // ocx launched from a terminal. + const shell = [launcher(), loop(), proxy(), entry(60, 1, "pwsh.exe", "pwsh", 100), entry(61, 60, "ocx.exe", "ocx stop", 6_000)]; + expect(stopperUnderHomeWrapper(shell, PATHS, 61)).toBe(false); + }); +}); + +describe("WinSW ownership before Part A stops it (#7771 follow-up)", () => { + // `sc stop opencodex-proxy-native` names a fixed ID. A registration that runs another + // installation's WinSW executable must never be stopped by this home's Part A. + const qc = (binaryPathName: string) => result(0, [ + "SERVICE_NAME: opencodex-proxy-native", + " TYPE : 10 WIN32_OWN_PROCESS", + " BINARY_PATH_NAME : " + binaryPathName, + ].join("\r\n")); + + const probe = (answers: { query: ReturnType; qc?: ReturnType }) => + windowsCheckedStopLiveDeps({ + run: async program => { + if (program !== "sc") throw new Error("unexpected program " + program); + return answers.query; + }, + scQcRun: async () => answers.qc ?? result(1), + }).probeWinsw(2_000); + + test("a service registered against another installation is not this home's manager", async () => { + const foreign = "C:\\Users\\other\\.opencodex\\winsw\\opencodex-proxy-native.exe"; + expect(await probe({ query: result(0, " STATE : 4 RUNNING"), qc: qc(foreign) })) + .toBe("nonexistent"); + }); + + test("this home's registered executable is the manager Part A may stop", async () => { + expect(await probe({ query: result(0, " STATE : 4 RUNNING"), qc: qc(`"${winswExePath()}"`) })) + .toBe("started"); + }); + + test("an unreadable registration fails closed instead of claiming or stopping it", async () => { + expect(await probe({ query: result(0, " STATE : 4 RUNNING"), qc: result(5, "", "Access is denied.") })) + .toBe("unknown"); + expect(await probe({ query: result(0, " STATE : 4 RUNNING") })).toBe("unknown"); + }); + + test("the ownership read gets only the time the state query left of one cap", async () => { + const running = result(0, " STATE : 4 RUNNING"); + let qcTimeout = -1; + const status = await windowsCheckedStopLiveDeps({ + run: async () => { await Bun.sleep(120); return running; }, + scQcRun: async timeoutMs => { qcTimeout = timeoutMs; return qc(`"${winswExePath()}"`); }, + }).probeWinsw(400); + expect(status).toBe("started"); + expect(qcTimeout).toBeGreaterThan(0); + expect(qcTimeout).toBeLessThanOrEqual(400 - 100); + // A state query that spent the whole cap leaves nothing: unknown, and no ownership read. + let qcCalls = 0; + const spent = await windowsCheckedStopLiveDeps({ + run: async () => { await Bun.sleep(60); return running; }, + scQcRun: async () => { qcCalls++; return qc(`"${winswExePath()}"`); }, + }).probeWinsw(40); + expect(spent).toBe("unknown"); + expect(qcCalls).toBe(0); + }); + + test("a service that does not exist needs no ownership read", async () => { + let qcCalls = 0; + const status = await windowsCheckedStopLiveDeps({ + run: async () => result(1060, "[SC] EnumQueryServicesStatus:OpenService FAILED 1060:"), + scQcRun: async () => { qcCalls++; return result(0); }, + }).probeWinsw(2_000); + expect(status).toBe("nonexistent"); + expect(qcCalls).toBe(0); + }); +}); diff --git a/tests/windows/windows-checked-stop.test.ts b/tests/windows/windows-checked-stop.test.ts index 9580c9a7d84..06b58b24b3c 100644 --- a/tests/windows/windows-checked-stop.test.ts +++ b/tests/windows/windows-checked-stop.test.ts @@ -272,8 +272,8 @@ describe("checked Windows stop, part A", () => { }); const result = await stopWindowsServiceChecked(h.deps); expect(result.outcome).toBe("survived"); - expect(mutations(h.calls)).toEqual(["winsw-stop"]); - expect(h.calls).not.toContain("enumerate"); + expect(mutations(h.calls)).toEqual(["winsw-stop", "terminate:30", "terminate:20"]); + expect(h.calls).toContain("enumerate"); }); test("WinSW: an SCM state that never reports stopped is unknown after + 2 s", async () => { @@ -383,7 +383,8 @@ describe("checked Windows stop, part A", () => { let checks = 0; const error = await guardedStopWindowsServiceChecked(() => { checks += 1; - if (checks === 3) throw new Error("respawned runtime"); + // Entry, before /end, before terminate:30, before terminate:20. + if (checks === 4) throw new Error("respawned runtime"); }, h.deps).catch(caught => caught); expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); expect((error as WindowsCheckedStopRefusedError).partial.terminated).toEqual([30]); @@ -391,9 +392,35 @@ describe("checked Windows stop, part A", () => { }); }); +describe("the updater's authorizer guards each manager stop", () => { + test("a refusal right before /end sends no /end and changes nothing", async () => { + const h = harness(); + let checks = 0; + const error = await guardedStopWindowsServiceChecked(() => { + checks += 1; + if (checks === 2) throw new Error("service home changed"); + }, h.deps).catch(caught => caught); + expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); + expect((error as WindowsCheckedStopRefusedError).partial.changed).toBe(false); + expect(mutations(h.calls)).toEqual([]); + }); + + test("a refusal right before the WinSW stop sends no stop and changes nothing", async () => { + const h = harness({ probeScheduler: () => ({ status: "absent" }), probeWinsw: () => "started" as WinswStatus }); + let checks = 0; + const error = await guardedStopWindowsServiceChecked(() => { + checks += 1; + if (checks === 2) throw new Error("service home changed"); + }, h.deps).catch(caught => caught); + expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); + expect((error as WindowsCheckedStopRefusedError).partial.changed).toBe(false); + expect(mutations(h.calls)).toEqual([]); + }); +}); + describe("checked Windows stop budget", () => { test("admission minimums with default settings", () => { - // 030's 48 / 56 / 42 s plus the 0.5 s closing sample part B can take past its window. + // The closing sample and WinSW's wrapper enumeration and termination are included. expect(windowsStopWorstCase({ managers: { scheduler: true, winsw: false, stray: false }, enumerationReused: true, orphanDiscovery: false, }).totalMs).toBe(48_500); @@ -402,7 +429,7 @@ describe("checked Windows stop budget", () => { }).totalMs).toBe(56_500); expect(windowsStopWorstCase({ managers: { scheduler: false, winsw: true, stray: false }, enumerationReused: false, orphanDiscovery: true, - }).totalMs).toBe(42_500); + }).totalMs).toBe(50_500); }); test("receipts, a longer shutdown and a longer WinSW stop timeout grow the worst case", () => { @@ -412,7 +439,7 @@ describe("checked Windows stop budget", () => { expect(windowsStopWorstCase({ managers: { scheduler: false, winsw: true, stray: false }, enumerationReused: false, orphanDiscovery: false, winswStopTimeoutMs: 30_000, - }).rows.find(row => row.step === "part-a")!.ms).toBe(35_000); + }).rows.find(row => row.step === "part-a")!.ms).toBe(43_000); expect(windowsStopWorstCase({ ...base, partB: false }).rows.map(row => row.step)).not.toContain("part-b"); }); diff --git a/tests/windows/windows-deploy-close-regressions.test.ts b/tests/windows/windows-deploy-close-regressions.test.ts index d6ea97d0b92..8e039a511d7 100644 --- a/tests/windows/windows-deploy-close-regressions.test.ts +++ b/tests/windows/windows-deploy-close-regressions.test.ts @@ -50,21 +50,19 @@ describe("update-job restart avoids the shell-less .cmd EINVAL (Windows, bun/sou // update — the common Windows path — with a stale service it could have refreshed. expect(src).toContain('process.env.OCX_SERVICE === "1" && refreshRegisters'); expect(src).toContain('const refreshRegisters = (svcArgs ?? []).includes("install")'); - // Follow the extracted teardown helper through its import and restart call site. - expect(src).toContain('import { stopWindowsServiceWrappersBestEffort } from "./job-windows-restart"'); - expect(src).toContain("(io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation);"); + // Follow the extracted teardown helper through its import and restart call sites. Since + // #7771 every call site gets a return path: a service that is not proven stopped aborts. + expect(src).toContain('import { stopWindowsServiceForRestart, WINDOWS_SERVICE_STOP_ABORT } from "./job-windows-restart"'); + expect(src.split("await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job);").length - 1).toBe(3); const restart = read("src/update/job-windows-restart.ts"); - expect(restart).toContain("export function stopWindowsServiceWrappersBestEffort(authorizeMutation: UpdateMutationAuthorizer)"); - expect(restart).toContain('if (process.platform !== "win32") return;'); - // Native WinSW installs stop and return before Task Scheduler /end or wrapper cleanup. - expect(restart).toMatch(/if \(readServiceBackend\(\) === "native"\) \{\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedStopWinsw\(authorizeMutation\);\s*return;\s*\}\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedStopWindows\(authorizeMutation\);/); - expect(restart.indexOf("killWindowsServiceWrapperProcesses(authorizeMutation);")).toBeGreaterThan(restart.indexOf("guardedStopWindows(authorizeMutation);")); - // The wrapper-killer script (and its self-exclusion) moved to the shared - // lib/windows-service-wrappers so the updater and the service teardown path - // cannot drift apart again. Follow the invariant to where it lives; the - // matching rules themselves are covered by windows-service-wrappers.test.ts. - expect(restart).toMatch(/const paths = \{\s*scriptPath: join\(getConfigDir\(\), "opencodex-service.cmd"\),\s*launcherPath: join\(getConfigDir\(\), "opencodex-service-launcher.vbs"\),\s*\};\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedKillWindowsWrappers\(authorizeMutation, paths\);/); - expect(read("src/lib/windows-service-wrappers.ts")).toContain("$_.ProcessId -eq $PID"); + expect(restart).toContain("export async function stopWindowsServiceForRestart(authorizeMutation: UpdateMutationAuthorizer): Promise"); + expect(restart).toContain('if (process.platform !== "win32") return true;'); + // Both manager registrations are probed by the checked stop. + expect(restart).not.toContain("guardedStopWinsw"); + // Task Scheduler: Part A ends the manager and the bound wrappers (the binding rule in + // windows-stop-binding.ts excludes the stopper itself); only stopped or absent continues. + expect(restart).toContain("const outcome = await guardedStopWindowsChecked(authorizeMutation);"); + expect(restart).toContain('return outcome === "stopped" || outcome === "absent";'); // Pinned-child cleanup is exercised through the retry loop in update/update-job.test.ts, // including exited children, live retry/final cleanup, and successful health probes. }); diff --git a/tests/windows/windows-guarded-manager.test.ts b/tests/windows/windows-guarded-manager.test.ts index 982746b5bba..8093257a7eb 100644 --- a/tests/windows/windows-guarded-manager.test.ts +++ b/tests/windows/windows-guarded-manager.test.ts @@ -242,6 +242,24 @@ describe("windows guarded manager target", () => { expect(target.kind).toBe("unknown"); }); + test("a not-running task only promotes its survivors to bound-stray when the registration is provably ours", () => { + // #7771: the not-running state alone does not make the registration this + // installation's. The definition has to be read and recognised before its + // surviving wrappers may be bound — otherwise any foreign task of the same + // name would turn this home's stray wrappers into terminable ones. + const inertTask = { winTaskState: () => "not-running" as const, winProcs: () => SUPERVISED }; + expect(inspectGuardedManagerTarget(42, 10100, schedulerDeps({ + win: { ...inertTask, winRegistrationOurs: () => false }, + }))).toMatchObject({ kind: "unknown" }); + expect(inspectGuardedManagerTarget(42, 10100, schedulerDeps({ + win: { ...inertTask, winTaskXml: () => { throw new Error("denied"); } }, + }))).toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + // Provably ours: the binding rule decides as before. These fixtures carry no creation + // times, so it stops at the unreadable-evidence refusal rather than the definition one. + expect(inspectGuardedManagerTarget(42, 10100, schedulerDeps({ win: inertTask }))) + .toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + }); + test("two registered managers stay unknown", () => { const target = inspectGuardedManagerTarget(42, 10100, schedulerDeps({ winsw: winswStarted })); diff --git a/tests/windows/windows-service-stop-wiring.test.ts b/tests/windows/windows-service-stop-wiring.test.ts new file mode 100644 index 00000000000..66eae6f9b5c --- /dev/null +++ b/tests/windows/windows-service-stop-wiring.test.ts @@ -0,0 +1,297 @@ +/** + * Caller side of the checked Windows stop (#7771): the real Part A run through the wiring in + * src/service/windows-service-stop.ts with every machine effect injected. Covers the ARM64 + * strict variant end to end, the own-wrapper refusal, the route's cap, admission against the + * stop deadline, the deadline flag and variable, the receipt's Part A record and the additive + * ocx-stop/1 fields. No schtasks, PowerShell or FFI call is made. + */ +import { afterEach, describe, expect, test } from "bun:test"; + +import { approvalChanged, managerStillActive, parseStopApproval } from "../../src/cli/stop-approval"; +import { summarizeStopRun, type StopRunRecord } from "../../src/cli/stop-report"; +import { claimPendingTeardown, clearPendingTeardown, pendingTeardownPartA, recordPendingTeardownPartA } from "../../src/config/pending-teardown"; +import type { WindowsCheckedStopDeps } from "../../src/service/windows-checked-stop"; +import { windowsStopTerminate } from "../../src/service/windows-checked-stop-live"; +import type { WindowsProcessWatch } from "../../src/service/windows-process-native"; +import { + runWindowsServiceStopPartA, + setStopDeadlineMs, + setWindowsServiceStopHooksForTests, + UNDER_OWN_WRAPPER_DETAIL, + windowsPartARecord, + windowsStopAdmission, + windowsStopRemainingMs, + resolveStopDeadlineMs, + parseStopDeadlineMs, + serviceStopOutcomeFromWindows, + windowsStopDeadlineAt, +} from "../../src/service/windows-service-stop"; +import { DEFAULT_WINDOWS_STOP_DEADLINE_MS } from "../../src/service/windows-stop-budget"; +import { PATHS, SCHEDULE_PID, entry, launcher, loop, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; +import { partAResult } from "../helpers/windows-stop-caller-fixtures"; + +afterEach(() => { + setWindowsServiceStopHooksForTests(null); + setStopDeadlineMs(null); +}); + +interface FakeMachine { + deps: WindowsCheckedStopDeps; + calls: string[]; + terminated: number[]; +} + +/** A registered, running task with the supervised tree; terminate kills through the watch. */ +function machine(overrides: Partial & { gate?: () => boolean; extra?: ReturnType[] } = {}): FakeMachine { + const calls: string[] = []; + const terminated: number[] = []; + const dead = new Set(); + const deps: WindowsCheckedStopDeps = { + canonicalPaths: PATHS, + childPid: 42, + selfPid: 1, + probeScheduler: () => ({ status: "present" }), + probeWinsw: () => "nonexistent", + enumerate: () => { calls.push("enumerate"); return [...supervisedTree(), ...(overrides.extra ?? [])]; }, + scheduleServicePid: () => SCHEDULE_PID, + endTask: () => { calls.push("end"); return overrides.gate && !overrides.gate() ? "failed" : "ended"; }, + taskState: () => "not-running", + winswStop: () => "sent", + winswState: () => "stopped", + proxyAlive: () => false, + openWatch: (pid): WindowsProcessWatch => ({ status: "open", exited: () => dead.has(pid), close() {} }), + terminate: (pid) => { calls.push("terminate:" + pid); terminated.push(pid); dead.add(pid); return { status: "terminated", exited: true }; }, + ...overrides, + }; + return { deps, calls, terminated }; +} + +describe("Part A through the caller wiring", () => { + test("WinSW registered and stopped still checks the caller's wrapper ancestry", async () => { + const under = entry(70, 30, "ocx.exe", "ocx stop", 6_000); + const fake = machine({ selfPid: 70, probeScheduler: () => ({ status: "absent" }), + probeWinsw: () => "stopped", extra: [under] }); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(result.changed).toBe(false); + }); + test("ends the bound wrappers with the live-style deps", async () => { + const fake = machine(); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("stopped"); + expect(fake.terminated).toEqual([30, 20]); + }); + + test("ARM64 strict: nothing is terminated and the stop is unknown / evidence-unreadable", async () => { + let fallback = 0; + const fake = machine({ + // No native API: no exit watch, and the strict gate answers the terminate. + openWatch: () => ({ status: "unavailable" }), + terminate: (pid, start, timeoutMs) => windowsStopTerminate(pid, start, timeoutMs, { + native: () => null, + terminate: async () => { fallback++; return { status: "terminated", exited: true }; }, + }), + }); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.reason).toBe("evidence-unreadable"); + expect(result.terminated).toEqual([]); + expect(fallback).toBe(0); + }); + + test("a stop launched under this home's wrapper refuses before /end, unless the caller is the wrapped proxy", async () => { + const under = entry(70, 30, "ocx.exe", "ocx stop", 6_000); + const refused = machine({ selfPid: 70, extra: [under] }); + setWindowsServiceStopHooksForTests({ liveDeps: () => refused.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(refused.calls).toEqual(["enumerate"]); + + // The guarded path binds on the snapshot it already read: refused before any probe. + const guarded = await runWindowsServiceStopPartA({ bindSnapshot: [...supervisedTree(), under] }); + expect(guarded.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(refused.calls).toEqual(["enumerate"]); + + const allowed = machine({ selfPid: 70, extra: [under] }); + setWindowsServiceStopHooksForTests({ liveDeps: () => allowed.deps }); + expect((await runWindowsServiceStopPartA({ allowUnderOwnWrapper: true })).outcome).toBe("stopped"); + expect(allowed.calls).toContain("end"); + }); + + test("under this home's wrapper with nothing registered: unknown, never absent, and nothing is ended", async () => { + // self 70 (ocx stop) <- cmd 30 (.cmd of this home) <- wscript 20 (.vbs of this home), with + // no task and no WinSW service. The own-wrapper guard makes the enumeration unreadable, and + // the stray path would otherwise read that as absent: no service registered, nothing to do. + const tree = () => [launcher(), loop(), entry(70, 30, "ocx.exe", "ocx stop", 6_000)]; + const calls: string[] = []; + const refused = machine({ + selfPid: 70, + probeScheduler: () => ({ status: "absent" }), + enumerate: () => { calls.push("enumerate"); return tree(); }, + }); + setWindowsServiceStopHooksForTests({ liveDeps: () => refused.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.reason).toBe("evidence-unreadable"); + expect(result.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(result.changed).toBe(false); + expect(result.terminated).toEqual([]); + expect(calls).toEqual(["enumerate"]); + expect(serviceStopOutcomeFromWindows(result)).toBe("state-unknown"); + expect(windowsPartARecord(result)).toBeNull(); + + // The same machine where the refusal is allowed (the route and the updater) still sweeps. + const allowed = machine({ selfPid: 70, probeScheduler: () => ({ status: "absent" }), enumerate: tree }); + setWindowsServiceStopHooksForTests({ liveDeps: () => allowed.deps }); + expect((await runWindowsServiceStopPartA({ allowUnderOwnWrapper: true })).outcome).not.toBe("unknown"); + }); + + test("a cap overrun is unknown at once, and nothing is ended or terminated after it", async () => { + let gate: () => boolean = () => true; + const fake = machine({ + endTask: async () => { + await Bun.sleep(120); + return gate() ? "ended" : "failed"; + }, + }); + setWindowsServiceStopHooksForTests({ liveDeps: options => { gate = options.gate ?? gate; return fake.deps; } }); + const started = performance.now(); + const result = await runWindowsServiceStopPartA({ capMs: 40 }); + expect(performance.now() - started).toBeLessThan(110); + expect(result.outcome).toBe("unknown"); + expect(result.detail).toBe("the checked stop did not finish within its cap"); + await Bun.sleep(250); + expect(gate()).toBe(false); + expect(fake.terminated).toEqual([]); + }); + + test("admission refuses with unverified-budget and changes nothing when the deadline cannot hold the stop", async () => { + const fake = machine(); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + setStopDeadlineMs(Math.ceil(performance.now()) + 20_000); + const result = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); + expect(result.outcome).toBe("unverified-budget"); + expect(result.reason).toBe("unverified-budget"); + expect(result.budget?.requiredMs).toBe(56_500); + expect(fake.calls.filter(call => call === "end" || call.startsWith("terminate"))).toEqual([]); + // The same stop with the default 60 s from process start is admitted early in a process. + setStopDeadlineMs(Math.ceil(performance.now()) + 60_000); + expect((await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) })).outcome).toBe("stopped"); + }); +}); + +describe("the stop deadline", () => { + test("--deadline-ms wins over OCX_STOP_DEADLINE_MS; empty or invalid is absent; neither is 60 s", () => { + expect(resolveStopDeadlineMs(72_000, { OCX_STOP_DEADLINE_MS: "30000" })).toBe(72_000); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "30000" })).toBe(30_000); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "" })).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "-5" })).toBe(60_000); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "12.5" })).toBe(60_000); + expect(resolveStopDeadlineMs(null, {})).toBe(60_000); + expect(parseStopDeadlineMs(" 45000 ")).toBe(45_000); + expect(parseStopDeadlineMs("0")).toBeNull(); + }); + + test("ocx stop's deadline is the command-start clock value it recorded", () => { + setStopDeadlineMs(72_000); + expect(windowsStopDeadlineAt(0)).toBe(72_000); + expect(windowsStopDeadlineAt(42_000)).toBe(72_000); + // Command start, not stop phase: 30 s of the budget are already spent at t=30_000. + expect(windowsStopRemainingMs(() => 30_000)).toBe(42_000); + }); + + test("every other caller starts its 60 s when the stop begins, not when the process did", () => { + // installFreshWindowsSchedulerSafely reaches the stop only after registration and the UAC + // prompt, so by then this process can be minutes old. Measuring from process start made + // admission refuse `unverified-budget` for a stop that had its whole budget left. + setStopDeadlineMs(null); + const phaseStart = 600_000; + expect(windowsStopRemainingMs(() => phaseStart)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + // One deadline governs the whole stop: a later read inside it does not slide forward. + expect(windowsStopDeadlineAt(phaseStart + 20_000)).toBe(phaseStart + DEFAULT_WINDOWS_STOP_DEADLINE_MS); + expect(windowsStopRemainingMs(() => phaseStart + 20_000)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS - 20_000); + }); + + test("recording a new command deadline clears an armed stop phase", () => { + setStopDeadlineMs(null); + expect(windowsStopRemainingMs(() => 5_000)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + setStopDeadlineMs(9_000); + expect(windowsStopDeadlineAt(5_000)).toBe(9_000); + // And un-setting it again re-arms at the next stop phase rather than keeping 9_000. + setStopDeadlineMs(null); + expect(windowsStopRemainingMs(() => 5_000)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + }); + + test("ocx stop parses --deadline-ms anywhere and keeps the approval grammar", () => { + expect(parseStopApproval(["--deadline-ms", "72000"])).toEqual({ ok: true, json: false, approval: null, deadlineMs: 72_000 }); + expect(parseStopApproval(["--json", "--deadline-ms", "5000"])).toEqual({ ok: true, json: true, approval: null, deadlineMs: 5_000 }); + expect(parseStopApproval(["--json"])).toEqual({ ok: true, json: true, approval: null, deadlineMs: null }); + for (const bad of [["--deadline-ms"], ["--deadline-ms", "0"], ["--deadline-ms", "1e4"], ["--deadline-ms", "-1"], + ["--deadline-ms", "5", "--deadline-ms", "6"], ["--json", "--deadline-ms", "abc"]]) { + expect(parseStopApproval(bad).ok).toBe(false); + } + const token = "a".repeat(64); + const approval = ["--json", "--expect-pid", "42", "--expect-port", "10100", "--expect-hostname", "127.0.0.1", + "--expect-config-home", "/h", "--expect-cli-version", "2.0.0", "--expect-compatibility-token", token]; + const parsed = parseStopApproval([...approval.slice(0, 3), "--deadline-ms", "65000", ...approval.slice(3)]); + expect(parsed.ok && parsed.approval?.pid === 42 && parsed.deadlineMs === 65_000).toBe(true); + }); +}); + +describe("the receipt's Part A record", () => { + test("only stopped or absent produce a record, with the bound wrapper set", () => { + expect(windowsPartARecord(partAResult("stopped"))).toEqual({ + partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }], + }); + expect(windowsPartARecord(partAResult("absent"))).toEqual({ partA: "absent", manager: { scheduler: false, winsw: false }, wrappers: [] }); + for (const outcome of ["survived", "denied", "unknown", "unverified-budget"] as const) { + expect(windowsPartARecord(partAResult(outcome))).toBeNull(); + } + }); + + test("written at claim, or added by the owner afterwards; read back only when well formed", () => { + const endpoint = { hostname: "127.0.0.1", port: 10100 }; + const record = windowsPartARecord(partAResult("stopped"))!; + const atClaim = claimPendingTeardown(endpoint, "exact", process.pid, undefined, record); + expect(pendingTeardownPartA(atClaim.nonce)).toEqual(record); + const before = claimPendingTeardown(endpoint, "exact", process.pid); + expect(pendingTeardownPartA(before.nonce)).toBeNull(); + expect(recordPendingTeardownPartA(before.nonce, record, process.pid + 1)).toBe(false); + expect(recordPendingTeardownPartA(before.nonce, record)).toBe(true); + expect(pendingTeardownPartA(before.nonce)).toEqual(record); + const malformed = claimPendingTeardown(endpoint, "exact", process.pid, undefined, { partA: "stopped" } as never); + expect(pendingTeardownPartA(malformed.nonce)).toBeNull(); + for (const nonce of [atClaim.nonce, before.nonce, malformed.nonce]) clearPendingTeardown(nonce); + }); +}); + +describe("ocx-stop/1 additive fields", () => { + const record = (extra: Partial): StopRunRecord => ({ + service: "failed", proxy: "unknown", sharedTeardown: "skipped", inheritedTeardownBlocks: false, receiptClearFailed: false, ...extra, + }); + const signals = { failed: true, historyOnly: false, historyDeferred: false, exitCode: 1 }; + + test("serviceStopReason and portHeldByForeign are omitted until set, and never null", () => { + const plain = summarizeStopRun(record({}), signals); + expect("serviceStopReason" in plain).toBe(false); + expect("portHeldByForeign" in plain).toBe(false); + const denied = summarizeStopRun(record({ serviceStopReason: "access-denied" }), signals); + expect(denied).toMatchObject({ outcome: "failed", service: "failed", proxy: "unknown", sharedTeardown: "skipped", serviceStopReason: "access-denied" }); + const foreign = summarizeStopRun(record({ service: "stopped", proxy: "stopped", portHeldByForeign: true }), + { failed: false, historyOnly: false, historyDeferred: false, exitCode: 0 }); + expect(foreign).toMatchObject({ outcome: "stopped", service: "stopped", portHeldByForeign: true }); + expect(summarizeStopRun(record({ portHeldByForeign: false }), signals).portHeldByForeign).toBe(false); + }); + + test("the guarded refusals carry them; no new outcome or proxy value appears", () => { + expect(managerStillActive("failed", record({ serviceStopReason: "unverified-budget" })).summary) + .toMatchObject({ outcome: "manager-still-active", serviceStopReason: "unverified-budget" }); + expect(approvalChanged("x", "unbound-wrapper").summary).toMatchObject({ outcome: "approval-changed", serviceStopReason: "unbound-wrapper" }); + expect("serviceStopReason" in approvalChanged("x").summary).toBe(false); + }); +}); diff --git a/tests/windows/windows-service-wrappers.test.ts b/tests/windows/windows-service-wrappers.test.ts index 3e91d5d0a61..e01bc6bf07e 100644 --- a/tests/windows/windows-service-wrappers.test.ts +++ b/tests/windows/windows-service-wrappers.test.ts @@ -129,17 +129,18 @@ describe("the generated script still implements that rule", () => { }); }); -describe("both teardown paths use the shared killer", () => { - test("neither file keeps a private matcher", () => { - for (const rel of ["src/service/windows-ops.ts", "src/update/job-windows-restart.ts"]) { +describe("every teardown path ends wrappers through the checked stop (#7771)", () => { + test("no stop path keeps a private matcher or the best-effort sweep", () => { + for (const rel of ["src/service/windows-ops.ts", "src/service/orchestration.ts", "src/update/job-windows-restart.ts", "src/update/guarded-runtime-ops.ts"]) { const src = read(rel); - if (rel === "src/update/job-windows-restart.ts") { - expect(src).toContain("killWindowsServiceWrapperProcesses(authorizeMutation);"); - expect(src).toMatch(/const paths = \{[\s\S]*?scriptPath: join\(getConfigDir\(\), "opencodex-service.cmd"\),[\s\S]*?launcherPath: join\(getConfigDir\(\), "opencodex-service-launcher.vbs"\),\s*\};\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedKillWindowsWrappers\(authorizeMutation, paths\);/); - } else expect(src).toContain("killWindowsSchedulerWrappers({"); + expect(src).not.toContain("killWindowsSchedulerWrappers"); + expect(src).not.toContain("killWindowsServiceWrapperProcesses"); expect(src).not.toContain("-like ('*' + $p + '*')"); expect(src).not.toContain("$pats = @('opencodex-service.cmd'"); } + // The one matcher is the binding rule; the service stop and the updater reach it through Part A. + expect(read("src/service/orchestration.ts")).toContain("runWindowsServiceStopPartA(request)"); + expect(read("src/update/guarded-runtime-ops.ts")).toContain("runWindowsServiceStopPartA({ authorize: mutationGate(authorize, \"service\"), allowUnderOwnWrapper: true })"); }); }); diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts new file mode 100644 index 00000000000..19ef54be61c --- /dev/null +++ b/tests/windows/windows-stop-callers.test.ts @@ -0,0 +1,567 @@ +/** + * Caller-level wiring of the checked Windows stop (#7771, 030 "Where it sits"): the + * /api/stop route, ocx service stop / uninstall, the guarded (consent) path with its + * bound-stray state, and the evidence snapshot the guarded bind must use. Hermetic: Part A and + * Part B are answered by fixtures or run against injected deps; nothing reaches the machine. + */ +import { afterEach, describe, expect, test } from "bun:test"; + +import { inspectGuardedTarget, resetOcxWindowsStopForTests, stopGuardedManager } from "../../src/cli/stop-windows-checked"; +import { runGuardedManagerStep, type GuardedStopSnapshot } from "../../src/cli/stop-approval"; +import { ROUTE_PART_A_CAP_MS, ROUTE_STRAY_SWEEP_CAP_MS, routeServiceStop } from "../../src/server/stop-service-route"; +import { completeWindowsServiceStopReceipt, windowsServiceRemovalStop as removalStop, windowsServiceStopBeforeRestore as stopBeforeRestore } from "../../src/service/cli-windows-stop"; +import { inspectGuardedManagerTarget, type GuardedManagerTarget } from "../../src/service/guarded-manager-target"; +import type { WindowsCheckedStopDeps } from "../../src/service/windows-checked-stop"; +import type { WindowsProcessWatch } from "../../src/service/windows-process-native"; + +const fixtureTarget = { pid: 4321, endpoint: { hostname: "127.0.0.1", port: 10100 } }; +const windowsServiceStopBeforeRestore: typeof stopBeforeRestore = io => stopBeforeRestore({ resolveTarget: async () => fixtureTarget, ...io }); +const windowsServiceRemovalStop: typeof removalStop = io => removalStop({ resolveTarget: async () => fixtureTarget, ...io }); +import type { WindowsStopProcessEntry } from "../../src/service/windows-process-evidence"; +import { runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, setStopDeadlineMs, setWindowsServiceStopHooksForTests } from "../../src/service/windows-service-stop"; +import { CMD, PATHS, SCHEDULE_PID, VBS, entry, launcher, loop, proxy, scheduleHost } from "../helpers/windows-checked-stop-fixtures"; +import { ALL_PART_A_FAILURES, partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; + +afterEach(() => { + setWindowsServiceStopHooksForTests(null); + setStopDeadlineMs(null); + resetOcxWindowsStopForTests(null); +}); + +const RECORD = { partA: "stopped" as const, manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }; +const url = (nonce = "a".repeat(32)) => new URL("http://127.0.0.1:10100/api/stop?deferSharedTeardown=1&teardownNonce=" + nonce); + +describe("/api/stop service stop on Windows", () => { + const route = (holdsReceipt: boolean, io: Parameters[2] = {}) => { + const stops: unknown[] = []; + const run = routeServiceStop(url(), holdsReceipt, { + platform: "win32", assertEnvironment: () => {}, winswRegistered: () => false, + stop: async request => { stops.push(request); return "stopped"; }, + ...io, + }); + return { run, stops }; + }; + + test("a receipt whose Part A record names the registered manager skips Part A", async () => { + const { run, stops } = route(true, { readRecord: () => RECORD, registration: async () => ({ scheduler: "present", winsw: "nonexistent" }) }); + expect(await run).toBe("stopped"); + expect(stops).toEqual([]); + }); + + test("a receipt without the record, or naming another manager, re-runs Part A capped at 6 s", async () => { + for (const io of [ + { readRecord: () => null }, + { readRecord: () => RECORD, registration: async () => ({ scheduler: "absent" as const, winsw: "nonexistent" as const }) }, + { readRecord: () => RECORD, registration: async () => ({ scheduler: "unknown" as const, winsw: "nonexistent" as const }) }, + ]) { + const { run, stops } = route(true, io); + await run; + expect(stops).toEqual([{ capMs: ROUTE_PART_A_CAP_MS, allowUnderOwnWrapper: true }]); + } + expect(ROUTE_PART_A_CAP_MS).toBe(6_000); + }); + + test("no receipt: WinSW keeps today's in-process stop; otherwise the 4 s stray sweep", async () => { + let winsw = 0; + const native = route(false, { winswRegistered: () => true, stopWinsw: () => { winsw++; } }); + expect(await native.run).toBe("stopped"); + expect(native.stops).toEqual([]); + expect(winsw).toBe(1); + expect(await route(false, { winswRegistered: () => true, stopWinsw: () => { throw new Error("x"); } }).run).toBe("failed"); + const stray = route(false); + await stray.run; + expect(stray.stops).toEqual([{ capMs: ROUTE_STRAY_SWEEP_CAP_MS, allowUnderOwnWrapper: true }]); + expect(ROUTE_STRAY_SWEEP_CAP_MS).toBe(4_000); + }); + + test("off Windows the route keeps today's service stop", async () => { + const { run, stops } = route(true, { platform: "darwin", readRecord: () => RECORD }); + await run; + expect(stops).toEqual([{}]); + }); + + test("a capped Part A that overruns answers service_state_unknown's outcome without waiting for it", async () => { + setWindowsServiceStopHooksForTests({ partA: () => new Promise(() => {}) }); + const started = performance.now(); + const result = await runWindowsServiceStopPartA({ capMs: 30, allowUnderOwnWrapper: true }); + expect(performance.now() - started).toBeLessThan(500); + expect(serviceStopOutcomeFromWindows(result)).toBe("state-unknown"); + }); +}); + +describe("ocx service stop on Windows", () => { + test("a discovered orphan's endpoint is used for both receipt and stop", async () => { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + const endpoint = { hostname: "127.0.0.1", port: 12001 }; + let claimed: unknown; + let stopped: unknown; + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + resolveTarget: async () => ({ pid: 4321, endpoint }), + claim: (address) => { claimed = address; return { nonce: "n" } as never; }, + stopTracked: async io => { stopped = io?.target; return "stopped"; }, + }); + expect(result.ok).toBe(true); + expect(claimed).toEqual(endpoint); + expect(stopped).toEqual({ pid: 4321, endpoint }); + }); + + test("no resolved target claims no receipt", async () => { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + let claimed = false; + let target: unknown = "unset"; + const result = await windowsServiceStopBeforeRestore({ resolveTarget: async () => null, + ownsSharedTeardown: () => true, claim: () => { claimed = true; return { nonce: "n" } as never; }, + stopTracked: async io => { target = io?.target; return "none"; } }); + expect(result.ok).toBe(true); + expect(claimed).toBe(false); + expect(target).toBeNull(); + }); + test("a Part A other than stopped/absent refuses before the receipt and the child stop", async () => { + for (const outcome of ALL_PART_A_FAILURES) { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult(outcome) }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + claim: () => { steps.push("claim"); return { nonce: "n" } as never; }, + stopTracked: async () => { steps.push("child"); return "stopped"; }, + }); + expect(result.ok).toBe(false); + expect(steps).toEqual([]); + } + }); + + test("claims the receipt before the deferred child stop, and completes it only after Part B and the restore", async () => { + const steps: string[] = []; + let claimed: unknown; + setWindowsServiceStopHooksForTests({ + partA: async () => { steps.push("partA"); return partAResult("stopped"); }, + partB: async () => { steps.push("partB"); return partBResult("freed"); }, + }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + claim: (_endpoint, _source, _owner, _handover, partA) => { steps.push("claim"); claimed = partA; return { nonce: "n1" } as never; }, + stopTracked: async io => { steps.push("child:" + io?.deferSharedTeardownNonce + ":" + io?.skipPortWait); return "stopped"; }, + }); + expect(steps).toEqual(["partA", "claim", "child:n1:true", "partB"]); + expect(claimed).toEqual(RECORD); + expect(result).toEqual({ ok: true, nonce: "n1" }); + const cleared: string[] = []; + completeWindowsServiceStopReceipt(result.nonce, nonce => { cleared.push(nonce); return true; }); + expect(cleared).toEqual(["n1"]); + }); + + test("a claim failure while this home owns the teardown sends no undeferred child stop", async () => { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped") }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + claim: () => { throw new Error("disk full"); }, + stopTracked: async () => { steps.push("child"); return "stopped"; }, + }); + expect(result.ok).toBe(false); + expect(steps).toEqual([]); + }); + + test("the not-owned teardown path is unchanged: no receipt, an ordinary child stop", async () => { + const children: unknown[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => false, + claim: () => { throw new Error("must not claim"); }, + stopTracked: async io => { children.push(io); return "stopped"; }, + }); + expect(result).toEqual({ ok: true }); + // #7771: the stop also names the admitted discovery cap, so the helper cannot spend 7 s + // of a budget whose 3 s for this step was already proven at admission. + expect(children).toEqual([{ target: fixtureTarget, skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); + }); + + test("Part B failure keeps the receipt (no restore); a foreign listener does not fail the stop", async () => { + for (const [outcome, ok] of [["survived", false], ["unknown", false], ["unconfirmed", false], ["port-held-by-foreign", true]] as const) { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult(outcome) }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, claim: () => ({ nonce: "n2" }) as never, stopTracked: async () => "stopped", + }); + expect(result.ok).toBe(ok); + } + }); + + test("absent refuses an unknown endpoint and keeps the receipt", async () => { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, claim: () => ({ nonce: "kept" }) as never, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10100 }], endpointLiveness: async () => "unknown", + }); + expect(result.ok).toBe(false); + }); + + test("absent checks endpoints instead of Part B", async () => { + let partB = 0; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent"), partB: async () => { partB++; return partBResult("freed"); } }); + const live = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => false, stopTracked: async () => "none", endpoints: () => [{ hostname: "127.0.0.1", port: 10100 }], endpointLiveness: async () => "live", + }); + expect(live.ok).toBe(false); + expect(partB).toBe(0); + }); +}); + +describe("ocx service uninstall / remove on Windows", () => { + test("refuses before deleting the task unless Part A stopped (or found nothing) and Part B freed the port", async () => { + for (const outcome of ALL_PART_A_FAILURES) { + let child = 0; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult(outcome) }); + expect((await windowsServiceRemovalStop({ stopTracked: async () => { child++; return "stopped"; } })).ok).toBe(false); + expect(child).toBe(0); + } + for (const [outcome, ok] of [["freed", true], ["port-held-by-foreign", true], ["survived", false], ["unknown", false]] as const) { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult(outcome) }); + expect((await windowsServiceRemovalStop({ stopTracked: async () => "stopped" })).ok).toBe(ok); + } + }); + + test("an absent Part A asks the endpoint itself: only a definitive down permits removal", async () => { + // Part A "absent" only says no manager is registered. `findLiveProxy` can return null + // when its discovery cap runs out while the endpoint is live, so removal must ask the + // endpoint itself before deleting the task — and a probe that could not answer + // (timeout, transport failure) is not the answer "nothing is serving". Only the + // tri-state "dead" admits removal; "live" and "unknown" refuse and keep the receipt. + for (const [answer, ok] of [["dead", true], ["live", false], ["unknown", false]] as const) { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const probed: { port: number }[] = []; + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpointLiveness: async endpoint => { probed.push(endpoint); return answer; }, + }); + expect(result.ok).toBe(ok); + expect(probed.length).toBe(1); + expect(probed[0]!.port).toBeGreaterThan(0); + } + }); + + test("an absent Part A proves EVERY candidate endpoint dead, not just the receipt's", async () => { + // #7771 round 9: with Part A absent only the receipt endpoint was probed. A stale runtime + // record naming a closed port then "proved" removal while the configured port had a live + // listener, and the task was deleted under it. The candidate set is the one `ocx uninstall` + // proves (`endpointsToProve(readRuntimePort(), loadConfig())`), and every candidate has to + // answer definitively dead. + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const probed: { hostname: string; port: number }[] = []; + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10999 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async endpoint => { + probed.push(endpoint); + return endpoint.port === 10999 ? "dead" : "live"; + }, + }); + expect(result.ok).toBe(false); + expect(probed.map(e => e.port)).toEqual([10999, 10100]); + + // One unreadable candidate refuses too, and nothing is deleted. + probed.length = 0; + const unknown = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10999 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async endpoint => { probed.push(endpoint); return endpoint.port === 10999 ? "dead" : "unknown"; }, + }); + expect(unknown.ok).toBe(false); + expect(probed).toHaveLength(2); + + // Every candidate dead is the only proof. + const dead = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10999 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async () => "dead", + }); + expect(dead.ok).toBe(true); + }); + + test("an unreadable absent-Part-A probe refuses and keeps the receipt", async () => { + // The old branch read `proxyStillLiveAfterStop`'s null — its answer for a timed-out or + // failed probe — as success and deleted the task, and a claimed receipt was reported + // as discharged by the caller. + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + let claimed = 0; + let probed = 0; + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, + claim: () => { claimed++; return { nonce: "r3" } as never; }, + stopTracked: async () => "none", + endpointLiveness: async () => { probed++; throw new Error("probe timed out"); }, + }); + expect(claimed).toBe(1); + // The endpoint is what has to answer; a survivor probe's null never stood in for it. + expect(probed).toBe(1); + expect(result).toEqual({ ok: false }); + }); + + test("claims a receipt carrying the Part A record before the POST, so the route does not answer 409", async () => { + // Deleting the registration leaves the scheduler task's receipt-less POST answering + // respawnable_service, which fails the uninstall. `ocx service stop` already claims; + // removal has to do the same, and keep the receipt until the restore succeeds. + const steps: string[] = []; + let claimed: unknown; + setWindowsServiceStopHooksForTests({ + partA: async () => { steps.push("partA"); return partAResult("stopped"); }, + partB: async () => { steps.push("partB"); return partBResult("freed"); }, + }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, + claim: (_endpoint, _source, _owner, _handover, partA) => { steps.push("claim"); claimed = partA; return { nonce: "r1" } as never; }, + stopTracked: async io => { steps.push("child:" + io?.deferSharedTeardownNonce + ":" + io?.skipPortWait); return "stopped"; }, + }); + expect(steps).toEqual(["partA", "claim", "child:r1:true", "partB"]); + expect(claimed).toEqual(RECORD); + expect(result).toEqual({ ok: true, nonce: "r1" }); + // Keep until the restore runs: only the caller's completion clears it. + const cleared: string[] = []; + completeWindowsServiceStopReceipt(result.nonce, nonce => { cleared.push(nonce); return true; }); + expect(cleared).toEqual(["r1"]); + }); + + test("a claim failure sends no undeferred child stop", async () => { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped") }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, + claim: () => { throw new Error("disk full"); }, + stopTracked: async () => { steps.push("child"); return "stopped"; }, + }); + expect(result.ok).toBe(false); + expect(steps).toEqual([]); + }); + + test("a failed Part B keeps the receipt for recovery", async () => { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("survived") }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, claim: () => ({ nonce: "r2" }) as never, + stopTracked: async io => { steps.push("child:" + io?.deferSharedTeardownNonce); return "stopped"; }, + }); + expect(result).toEqual({ ok: false }); + expect(steps).toEqual(["child:r2"]); + }); + + test("the not-owned teardown path is unchanged: no receipt, an ordinary child stop", async () => { + const children: unknown[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + claim: () => { throw new Error("must not claim"); }, + stopTracked: async io => { children.push(io); return "stopped"; }, + }); + expect(result).toEqual({ ok: true }); + // #7771: the stop also names the admitted discovery cap, so the helper cannot spend 7 s + // of a budget whose 3 s for this step was already proven at admission. + expect(children).toEqual([{ target: fixtureTarget, skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); + }); +}); + +describe("guarded (consent) path", () => { + const strayManager: Extract = { + kind: "bound-stray", pid: 42, backend: "scheduler", childNeedsSeparateStop: true, + wrappers: [{ pid: 30, startTime: 5_100 }, { pid: 20, startTime: 5_000 }], + }; + const snapshot = (manager: GuardedStopSnapshot["manager"]): GuardedStopSnapshot => ({ + approval: { pid: 42, port: 10100, hostname: "127.0.0.1", configHome: "/h", cliVersion: "2.0.0", compatibilityToken: "a".repeat(64) }, + manager, + }); + + test("order: the wrappers end before signalApproved; the approved PID still gets its graceful signal; Part B after settle", async () => { + const order: string[] = []; + const step = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: async () => { order.push("revalidate"); return strayManager; }, + stopManager: async () => { order.push("partA"); return "stopped"; }, + signalApproved: async () => { order.push("signal"); return false; }, + settle: async () => { order.push("settle"); return true; }, + confirmFreed: async () => { order.push("partB"); return true; }, + managerState: async () => { order.push("state"); return "inactive"; }, + }); + expect(order).toEqual(["revalidate", "partA", "signal", "settle", "partB", "state"]); + expect(step.effect).toBe("stopped"); + }); + + test("Part A not stopped is manager-still-active and the approved PID is never signalled (a refusal is never forced)", async () => { + for (const service of ["failed", "state-unknown"] as const) { + const order: string[] = []; + const step = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: () => strayManager, + stopManager: async () => service, + signalApproved: async () => { order.push("signal"); return true; }, + settle: async () => { order.push("settle"); return true; }, + confirmFreed: async () => { order.push("partB"); return true; }, + managerState: async () => "inactive", + }); + expect(step.effect).toBe("manager-still-active"); + expect(order).toEqual([]); + } + // A graceful signal that is refused (409) fails the step without settle or Part B. + const order: string[] = []; + const refused = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: () => strayManager, stopManager: async () => "stopped", + signalApproved: async () => { throw new Error("409"); }, + settle: async () => { order.push("settle"); return true; }, + confirmFreed: async () => { order.push("partB"); return true; }, + managerState: async () => "inactive", + }); + expect(refused.effect).toBe("failed"); + expect(order).toEqual([]); + }); + + test("Part B failure is manager-still-active; an absent manager runs neither Part A nor Part B", async () => { + const failed = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: () => strayManager, stopManager: async () => "stopped", signalApproved: async () => false, + settle: async () => true, confirmFreed: async () => false, managerState: async () => "inactive", + }); + expect(failed.effect).toBe("manager-still-active"); + const order: string[] = []; + const absent = await runGuardedManagerStep(snapshot({ kind: "absent" }), { + revalidateManager: () => ({ kind: "absent" }), stopManager: async () => { order.push("partA"); return "stopped"; }, + signalApproved: async () => false, settle: async () => true, + confirmFreed: async () => { order.push("partB"); return true; }, managerState: async () => "inactive", + }); + expect(absent.effect).toBe("stopped"); + expect(order).toEqual([]); + }); + + test("revalidation compares the bound wrapper set; an unbindable survivor carries its reason", async () => { + const io = (current: GuardedManagerTarget) => ({ + revalidateManager: () => current, stopManager: async () => "stopped" as const, signalApproved: async () => false, + settle: async () => true, managerState: async () => "inactive" as const, + }); + const changed = await runGuardedManagerStep(snapshot(strayManager), io({ ...strayManager, wrappers: [{ pid: 30, startTime: 9_999 }, { pid: 20, startTime: 5_000 }] })); + expect(changed.effect).toBe("approval-changed"); + const reordered = await runGuardedManagerStep(snapshot(strayManager), io({ ...strayManager, wrappers: [...strayManager.wrappers].reverse() })); + expect(reordered.effect).toBe("stopped"); + const unbound = await runGuardedManagerStep(snapshot(strayManager), io({ kind: "unknown", reason: "x", stopReason: "unbound-wrapper" })); + expect(unbound).toMatchObject({ effect: "approval-changed", stopReason: "unbound-wrapper" }); + }); +}); + +describe("guarded inspection: the binding rule decides bound-stray", () => { + const inspect = (processes: WindowsStopProcessEntry[] | Omit[]) => + inspectGuardedManagerTarget(42, 10100, { + platform: "win32", verifyPid: pid => pid, + scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", + win: { + winProcs: () => processes as WindowsStopProcessEntry[], winTaskState: () => "not-running", + // #7771: a not-running task promotes its survivors only after the definition is read + // and recognised as this installation's. + winTaskXml: () => "xml", winRegistrationOurs: () => true, + winSchedulePid: () => SCHEDULE_PID, winScriptPath: () => CMD, winLauncherPath: () => VBS, + }, + }); + // The task is registered but not running; its launcher and loop survive with the child. + const tree = () => [scheduleHost, launcher(), loop(), proxy()]; + + test("surviving wrappers bound by the rule are bound-stray with their pid + creation time", () => { + expect(inspect(tree())).toEqual({ + kind: "bound-stray", pid: 42, backend: "scheduler", childNeedsSeparateStop: true, + wrappers: [{ pid: 30, startTime: 5_100 }, { pid: 20, startTime: 5_000 }], + }); + }); + + test("a CIM-only table (no creation times) cannot bind: unknown / evidence-unreadable", () => { + const cimOnly = tree().map(({ startTime: _startTime, ...rest }) => rest); + expect(inspect(cimOnly)).toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + }); + + test("the same bindable survivors stay unknown when the not-running registration is not provably ours", () => { + // #7771: this tree binds to bound-stray when the definition is ours (test above). A + // same-named foreign task, or a definition that cannot be read, must not turn this + // home's surviving wrappers into terminable ones. + const withDefinition = (win: { winTaskXml?: () => string; winRegistrationOurs?: () => boolean }) => + inspectGuardedManagerTarget(42, 10100, { + platform: "win32", verifyPid: pid => pid, + scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", + win: { + winProcs: () => tree(), winTaskState: () => "not-running", + winTaskXml: () => "xml", winRegistrationOurs: () => true, + winSchedulePid: () => SCHEDULE_PID, winScriptPath: () => CMD, winLauncherPath: () => VBS, + ...win, + }, + }); + expect(withDefinition({}).kind).toBe("bound-stray"); + expect(withDefinition({ winRegistrationOurs: () => false }).kind).toBe("unknown"); + expect(withDefinition({ winTaskXml: () => { throw new Error("access denied"); } })) + .toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + }); + + test("a hand-started launcher under a live shell stays unknown / unbound-wrapper", () => { + const shell = entry(60, 1, "pwsh.exe", "pwsh", 100); + // The task's own wrappers bind; a second launcher started by hand from a live shell is not + // an ancestor of the approved child, its parent is neither the Schedule host nor gone, and + // its loop descends only from it, so neither may be ended. + const handStarted = [...tree(), shell, launcher(50, 60, 7_000), loop(51, 50, 7_100)]; + const target = inspect(handStarted); + expect(target.kind).toBe("unknown"); + expect(target).toMatchObject({ stopReason: "unbound-wrapper" }); + }); +}); + +describe("the guarded bind uses the evidence snapshot (native creation times), never the CIM-only list", () => { + // On x64 the evidence snapshot carries the native creation time, which is exactly what the + // handle terminate compares. The CIM-only time differs below the millisecond, and a terminate + // checked against it answers "reused": Part A would then treat a live wrapper as exited. + const NATIVE = { 20: 5_000, 30: 5_100 } as Record; + const evidence = (): WindowsStopProcessEntry[] => [scheduleHost, launcher(20, SCHEDULE_PID, NATIVE[20]), loop(30, 20, NATIVE[30]), proxy()]; + const cimOnly = (): WindowsStopProcessEntry[] => [scheduleHost, launcher(20, SCHEDULE_PID, NATIVE[20]! - 1), loop(30, 20, NATIVE[30]! - 1), proxy()]; + + function machine() { + const terminated: number[] = []; + const reused: number[] = []; + const enumerations: string[] = []; + const dead = new Set(); + const deps: WindowsCheckedStopDeps = { + canonicalPaths: PATHS, childPid: 42, selfPid: 1, + probeScheduler: () => ({ status: "present" }), probeWinsw: () => "nonexistent", + // What a CIM-only enumeration would return; the guarded bind must not use it. + enumerate: () => { enumerations.push("cim"); return cimOnly(); }, + scheduleServicePid: () => SCHEDULE_PID, endTask: () => "not-running", taskState: () => "not-running", + winswStop: () => "sent", winswState: () => "stopped", proxyAlive: () => false, + openWatch: (pid): WindowsProcessWatch => ({ status: "open", exited: () => dead.has(pid), close() {} }), + terminate: (pid, startTime) => { + if (NATIVE[pid] !== startTime) { reused.push(pid); return { status: "reused" }; } + terminated.push(pid); dead.add(pid); return { status: "terminated", exited: true }; + }, + }; + return { deps, terminated, reused, enumerations }; + } + + test("inspectGuardedTarget reads the evidence snapshot once and stopGuardedManager binds on it", async () => { + resetOcxWindowsStopForTests("win32"); + setStopDeadlineMs(Math.ceil(performance.now()) + 120_000); + const fake = machine(); + setWindowsServiceStopHooksForTests({ snapshot: async () => evidence(), schedulePid: async () => SCHEDULE_PID, liveDeps: () => fake.deps }); + const target = await inspectGuardedTarget(42, 10100, (pid, port, deps) => inspectGuardedManagerTarget(pid, port, { + ...deps, platform: "win32", verifyPid: p => p, scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", + win: { + ...deps?.win, winTaskState: () => "not-running", winScriptPath: () => CMD, winLauncherPath: () => VBS, + winTaskXml: () => "xml", winRegistrationOurs: () => true, + }, + })); + expect(target.kind).toBe("bound-stray"); + const outcome = await stopGuardedManager({ approvedPort: 10100, receiptNonce: () => undefined, inheritedReceipts: 0 }); + expect(outcome).toBe("stopped"); + expect(fake.enumerations).toEqual([]); + expect(fake.reused).toEqual([]); + expect(fake.terminated).toEqual([30, 20]); + }); + + test("the hazard it prevents: binding on the CIM-only list ends nothing yet reports stopped", async () => { + const fake = machine(); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(fake.enumerations).toEqual(["cim"]); + expect(fake.terminated).toEqual([]); + expect(fake.reused).toEqual([30, 20]); + expect(result.outcome).toBe("stopped"); + }); +}); diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts new file mode 100644 index 00000000000..5d8eaf29d15 --- /dev/null +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -0,0 +1,181 @@ +/** + * Plain ocx stop (no consent snapshot) on Windows (#7771, 030 "Two parts"), through the real + * CLI module graph in tests/fixtures/parent-stop-runner.ts with the platform simulated and + * Part A / Part B answered by fixtures. Nothing reaches schtasks, PowerShell or FFI. + * + * A Part A outcome other than stopped/absent holds the child stop: no /api/stop POST, + * ownershipBlocked, the shared teardown skipped, and the record says service failed (or + * state-unknown), proxy unknown, outcome failed. After a stopped Part A, Part B replaces the + * respawn re-poll; the receipt the POST carries already records Part A. + */ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { spawnSync } from "node:child_process"; +import { existsSync, mkdtempSync, readFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { fixturePath, repoPath } from "../helpers/repo-root"; + +let home: string; +beforeEach(() => { home = mkdtempSync(join(tmpdir(), "ocx-windows-plain-stop-")); }); +afterEach(() => { removeTreeWithRetry(home); }); + +const restored = { success: true, message: "native Codex restored", + artifacts: { config: { state: "restored" }, catalog: { state: "restored" }, history: { state: "restored" } } }; + +function runStop(windowsPartA: string | undefined, windowsPartB?: string, extra: Record = {}) { + const child = spawnSync(process.execPath, [fixturePath("parent-stop-runner.ts")], { + cwd: repoPath(), env: { ...process.env, OPENCODEX_HOME: home }, + input: JSON.stringify({ receipt: true, json: true, response: { success: true, sharedTeardown: "deferred" }, + restore: restored, ...(windowsPartA ? { windowsPartA } : {}), ...(windowsPartB ? { windowsPartB } : {}), ...extra }), + encoding: "utf8", timeout: 20_000, windowsHide: true, + }); + expect(child.error, child.stderr).toBeUndefined(); + const reportPath = join(home, "parent-stop-result.json"); + expect(existsSync(reportPath), child.stderr).toBe(true); + const report = JSON.parse(readFileSync(reportPath, "utf8")) as { + calls: { native: number; grok: number; service: number; partB: number }; + /** The timeout each reclaimListenPort call asked for; #7771 makes a skipped legacy wait visible. */ + reclaimTimeouts: number[]; + urls: string[]; unexpectedIo: string[]; nonce?: string; receiptExists: boolean; postPartA: unknown[]; + }; + expect(report.unexpectedIo, child.stderr).toEqual([]); + const lines = child.stdout.trim().split("\n"); + const summary = JSON.parse(lines[lines.length - 1]!) as Record; + return { ...report, summary, exitCode: child.status, stderr: child.stderr }; +} + +describe("plain ocx stop on Windows: Part A holds the child stop", () => { + for (const guarded of [false, true]) { + test(`receipt failure holds the ${guarded ? "guarded" : "plain"} child stop`, () => { + const run = runStop("stopped", "freed", { receipt: false, guarded, windowsGuarded: guarded }); + expect(run.urls).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.exitCode).toBe(1); + }); + } + test("a rejected Part A holds the child and shared teardown", () => { + const run = runStop(undefined, undefined, { windowsPartAReject: true }); + expect(run.urls).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", service: "error", proxy: "unknown", sharedTeardown: "skipped" }); + }); + for (const [outcome, service, reason] of [ + ["denied", "failed", "access-denied"], + ["survived", "failed", undefined], + ["unknown", "state-unknown", "evidence-unreadable"], + ["unverified-budget", "failed", "unverified-budget"], + ] as const) { + test(`${outcome}: no POST, ownership blocked, shared teardown skipped`, () => { + const run = runStop(outcome); + expect(run.calls.service, run.stderr).toBe(1); + expect(run.urls, run.stderr).toEqual([]); + expect(run.calls.partB).toBe(0); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.nonce).toBeUndefined(); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ schema: "ocx-stop/1", outcome: "failed", service, proxy: "unknown", sharedTeardown: "skipped" }); + if (reason) expect(run.summary.serviceStopReason).toBe(reason); + else expect("serviceStopReason" in run.summary).toBe(false); + expect(run.stderr).toContain("The proxy was left running and the shared teardown skipped"); + }); + } +}); + +describe("plain ocx stop on Windows: Part B after a stopped Part A", () => { + + test("stopped WinSW and own wrapper: unknown and no POST", () => { + const run = runStop(undefined, undefined, { windowsPartAUnderOwnWrapper: true, windowsWinswStopped: true }); + expect(run.urls).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ service: "state-unknown", proxy: "unknown", sharedTeardown: "skipped" }); + }); + + test("under this home's wrapper with nothing registered: unknown, no POST, no shared teardown", () => { + // The real Part A refuses: this stop runs under this home's .cmd/.vbs wrapper and no + // service is registered. Reading that as absent would stop the child and restore shared + // config beneath a wrapper that can relaunch the proxy. + const run = runStop(undefined, undefined, { windowsPartAUnderOwnWrapper: true }); + expect(run.calls.service, run.stderr).toBe(1); + expect(run.urls, run.stderr).toEqual([]); + expect(run.calls.partB).toBe(0); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.nonce).toBeUndefined(); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", service: "state-unknown", proxy: "unknown", sharedTeardown: "skipped", serviceStopReason: "evidence-unreadable" }); + expect(run.stderr).toContain("this stop runs under this home's service wrapper"); + }); + + test("freed: the POST's receipt records Part A, then the restore runs and the receipt clears", () => { + const run = runStop("stopped", "freed"); + // Part B owns the port wait: stopProxy's legacy 15 s wait must not run on top of it. + expect(run.reclaimTimeouts, run.stderr).toEqual([]); + expect(run.urls, run.stderr).toHaveLength(1); + expect(run.postPartA).toEqual([{ partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }]); + expect(run.calls.partB).toBe(1); + expect(run.calls.native).toBe(1); + expect(run.receiptExists).toBe(false); + expect(run.exitCode, run.stderr).toBe(0); + expect(run.summary).toMatchObject({ outcome: "stopped", service: "stopped", proxy: "stopped", portHeldByForeign: false }); + }); + + test("our identity back on the port: respawned, shared teardown skipped, the receipt kept", () => { + const run = runStop("stopped", "survived"); + expect(run.urls).toHaveLength(1); + expect(run.calls.native).toBe(0); + expect(run.receiptExists).toBe(true); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", proxy: "respawned", sharedTeardown: "skipped", portHeldByForeign: false }); + }); + + test("a foreign listener: the stop succeeds and says so", () => { + const run = runStop("stopped", "port-held-by-foreign"); + expect(run.calls.native).toBe(1); + expect(run.exitCode, run.stderr).toBe(0); + expect(run.summary).toMatchObject({ outcome: "stopped", portHeldByForeign: true }); + }); + + test("an unconfirmed port: failed with evidence-unreadable, nothing restored", () => { + const run = runStop("stopped", "unknown"); + expect(run.calls.native).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", proxy: "unknown", serviceStopReason: "evidence-unreadable" }); + }); + + test("guarded: Part B owns the port wait, so the legacy 15 s wait never runs", () => { + // The guarded path's Part B runs after settle; keying the skip to the snapshotless + // flag alone left the approved endpoint's 15 s waitForStoppedPort on top of it. + const run = runStop("stopped", "freed", { guarded: true, windowsGuarded: true }); + expect(run.reclaimTimeouts, run.stderr).toEqual([]); + expect(run.calls.partB).toBe(1); + expect(run.urls).toHaveLength(1); + expect(run.exitCode, run.stderr).toBe(0); + }); + + test("guarded: our identity back on the port still holds the shared teardown", () => { + // A surviving listener never reaches the forced-kill fallback either: the guarded step + // reports manager-still-active and nothing is restored. + const run = runStop("stopped", "survived", { guarded: true, windowsGuarded: true }); + expect(run.reclaimTimeouts, run.stderr).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.calls.killed).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "manager-still-active", proxy: "unknown", sharedTeardown: "skipped" }); + }); + + test("absent: today's stop, the POST without Part B and without the 6 s hold", () => { + const run = runStop("absent"); + expect(run.urls).toHaveLength(1); + expect(run.postPartA).toEqual([{ partA: "absent", manager: { scheduler: false, winsw: false }, wrappers: [] }]); + expect(run.calls.partB).toBe(0); + expect(run.exitCode, run.stderr).toBe(0); + expect(run.summary).toMatchObject({ outcome: "stopped", service: "absent" }); + expect("portHeldByForeign" in run.summary).toBe(false); + }); +});