From c4456bda7f0d29ef3eb9d7edf610abce1d937367 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 01:14:14 +0900 Subject: [PATCH 01/53] feat(service): add checked Windows stop primitives and operations, unwired Adds the library half of #7771 option B as tested code that nothing calls yet, so behavior is unchanged on every platform: - windows-process-native: x64 FFI from the trusted system directory for the process creation time (GetProcessTimes), a handle terminate that compares the creation time on the same handle it terminates, a bind-time exit watch, and the Task Scheduler service PID (QueryServiceStatusEx). - windows-process-evidence: the CIM snapshot with creation times (native times kept only when they agree with CIM to the ms), an sc.exe queryex fallback, and the bounded ARM64 PowerShell terminate (StartTime check within 1 s, Kill() on the same object, exit read back). - windows-stop-binding: the 030 binding rule as a pure function. - windows-stop-budget: the caps and the admission arithmetic, including the 0.5 s closing port sample part B can take past its window (48.5 / 56.5 / 42.5 s). - windows-checked-stop: part A with every effect injected, admission after the registration probe, and the authorizer-gated variant for the updater. - windows-stop-confirm: part B (6 s hold, final enumeration, ours vs foreign vs exited-owner listener) and the port resolution order. Contract in structure/ops/windows-checked-stop.md; slice plan and the points where this differs from 030 in devlog/_plan/261111_windows_service_stop/060. Refs #7771 --- .../261111_windows_service_stop/000_plan.md | 3 +- .../060_implementation.md | 52 +++ scripts/test-layout/layout.json | 4 +- src/service/windows-checked-stop.ts | 431 ++++++++++++++++++ src/service/windows-process-evidence.ts | 288 ++++++++++++ src/service/windows-process-native.ts | 262 +++++++++++ src/service/windows-stop-binding.ts | 187 ++++++++ src/service/windows-stop-budget.ts | 122 +++++ src/service/windows-stop-confirm.ts | 158 +++++++ structure/INDEX.md | 7 + structure/manifest.json | 14 + structure/ops/windows-checked-stop.md | 88 ++++ structure/runtime.md | 2 +- tests/fixtures/test-layout-expected.json | 4 +- .../helpers/windows-checked-stop-fixtures.ts | 56 +++ tests/windows/windows-checked-stop.test.ts | 333 ++++++++++++++ .../windows/windows-process-evidence.test.ts | 214 +++++++++ tests/windows/windows-stop-binding.test.ts | 155 +++++++ tests/windows/windows-stop-confirm.test.ts | 108 +++++ 19 files changed, 2482 insertions(+), 6 deletions(-) create mode 100644 devlog/_plan/261111_windows_service_stop/060_implementation.md create mode 100644 src/service/windows-checked-stop.ts create mode 100644 src/service/windows-process-evidence.ts create mode 100644 src/service/windows-process-native.ts create mode 100644 src/service/windows-stop-binding.ts create mode 100644 src/service/windows-stop-budget.ts create mode 100644 src/service/windows-stop-confirm.ts create mode 100644 structure/ops/windows-checked-stop.md create mode 100644 tests/helpers/windows-checked-stop-fixtures.ts create mode 100644 tests/windows/windows-checked-stop.test.ts create mode 100644 tests/windows/windows-process-evidence.test.ts create mode 100644 tests/windows/windows-stop-binding.test.ts create mode 100644 tests/windows/windows-stop-confirm.test.ts diff --git a/devlog/_plan/261111_windows_service_stop/000_plan.md b/devlog/_plan/261111_windows_service_stop/000_plan.md index 783cd00b0d5..58ade97dacd 100644 --- a/devlog/_plan/261111_windows_service_stop/000_plan.md +++ b/devlog/_plan/261111_windows_service_stop/000_plan.md @@ -1,6 +1,6 @@ # 000 — Windows service stop that frees the port (#7771 Option B): design -Status: design only, awaiting owner approval. Nothing here is implemented. +Status: approved (030, bounded ARM64 fallback). Implementation is split into three pull requests (060); the primitives and the two operations land first, unwired. ## Problem @@ -17,6 +17,7 @@ What has shipped since: refusal diagnostics (#7973, option A), in-process comman | 030 | Recommended design: one checked Windows stop | | 040 | Verification: unit, hermetic integration, and real Windows device | | 050 | Risks, compatibility and rollout | +| 060 | Implementation slices and where the code differs from 030 | ## Decision requested diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md new file mode 100644 index 00000000000..9bd281b5bd4 --- /dev/null +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -0,0 +1,52 @@ +# 060 — Implementation slices + +The owner approved 030 with the bounded ARM64 fallback. The work lands in three pull requests so each one is reviewable on its own, and only the second changes behavior. + +## PR1: primitives and the two operations (this slice) + +Library code with hermetic tests. Nothing calls it yet, so no platform's behavior changes. + +| Module | Content | +|---|---| +| `src/service/windows-process-native.ts` | x64 FFI from the trusted system directory: `GetProcessTimes` creation time, the handle terminate (open, compare creation time on that handle, `TerminateProcess`, close), the bind-time exit watch, `QueryServiceStatusEx` for the Schedule host PID | +| `src/service/windows-process-evidence.ts` | facades: the CIM snapshot with `CreationDate` (native times on x64 kept only when they agree with CIM to the ms), `sc.exe queryex Schedule` fallback, the ARM64 PowerShell terminate, the watch | +| `src/service/windows-stop-binding.ts` | the 030 binding rule as a pure function: bound (deepest first, pid + creation time + image + command-line digest), unbound, unreadable | +| `src/service/windows-stop-budget.ts` | the caps and the admission arithmetic of the 030 tables, plus the part B window | +| `src/service/windows-checked-stop.ts` | part A, `stopWindowsServiceChecked(deps, options)` with every effect injected, admission after the registration probe, and the authorizer variant `guardedStopWindowsServiceChecked` | +| `src/service/windows-stop-confirm.ts` | part B, `confirmWindowsStopFreed(deps)`, and `resolveWindowsStopPort` | + +Contract doc: `structure/ops/windows-checked-stop.md`. + +## PR2: caller wiring + +- Live deps adapters for part A and part B: capped `schtasks /query` and `/end` (2 s, through the test-home guard), WinSW non-waiting `stop` (2 s) and `sc query` polling, a port sampler, the listener identity check (`findLiveProxy` identity), and the clock. +- `stopServiceIfInstalledDetailed` and the guarded `stopManager` slot become async on every platform. On Windows they call part A instead of `stopWindowsChecked` plus `killWindowsServiceWrapperProcesses`, and all their callers change in the same PR. +- Part B replaces `proxyStillLiveAfterStop`, the `schedulerCanRespawn` re-poll and the uninstall respawn window on Windows, and is added after the guarded `settle`. +- `ocx stop --deadline-ms `, also read from `OCX_STOP_DEADLINE_MS` so a Desktop can pass a deadline to an older CLI without that CLI rejecting an unknown flag. The flag wins over the variable, and 60 s applies when neither is set. +- `ocx-stop/1` additive fields `serviceStopReason` and `portHeldByForeign`, with no new `outcome` or `proxy` value. +- The receipt's part A record (`partA`, the bound manager, and the ended wrapper set as pid + creation time), and `/api/stop` skipping part A for a receipt that carries it. +- The guarded inspection's `bound-stray` state through the binding rule, `sameGuardedManager` comparing the bound wrapper set. +- Service uninstall/remove and `prepareServiceInstall` refuse on a part A outcome other than `stopped`, and on a part B result other than a free port. +- The updater's three call sites move to the authorizer variant and abort on a non-`stopped` outcome (`preparePort` becomes async or part A runs before it). +- The new caps from 030: the forced `taskkill` at 5 s, orphan discovery at 3 s, and `revertSystemEnv` at 1 s. + +## PR3: Desktop + +- Optional `serviceStopReason` / `portHeldByForeign` fields on `StopSummary` (`serde(default)`), with the elevation hint and the warning text. +- The exit/restart drain deadline goes to 75 s on Windows, and the deadline passes through `OCX_STOP_DEADLINE_MS` (and `--deadline-ms` once the CLI floor allows it) on both the takeover and the drain. + +## Where PR1 differs from 030 + +- **Part B's closing sample.** The 6 s hold is measured between port sample starts, and the window floor is also 6 s, so the last sample starts at the window's end and can run up to its 0.5 s cap past it. The budget counts this, which raises the admission minimums to 48.5 / 56.5 / 42.5 s. The Desktop drain case becomes 70.5 s, which still fits in 72 s. +- **WinSW binds nothing.** WinSW-only part A neither enumerates nor binds. This follows 030's step 4 and its 25 s row: a stray scheduler wrapper left behind by an earlier backend is caught only when nothing is registered. "Proxy still alive after WinSW stopped" is an injected `proxyAlive` check inside the 1 s manager-state step. An answer of "cannot tell" leaves the decision to part B. +- **Terminates share the wait cap.** 030's tables do not price the terminate calls themselves. They share the 3 s handle-wait window, which matters for the ARM64 script (about 1 s each). +- **Unbound wrappers with nothing registered.** Binding finds no bound wrapper but an unbound one is alive. The result is `survived` / `unbound-wrapper`, not `absent`, matching 040's bullet; the 030 "absent" definition reads as "nothing bound" alone. Nothing is changed either way. +- **Unreadable candidates with nothing registered.** An unreadable wrapper image above the child counts only while a manager is registered. A hand-started proxy under an elevated shell would otherwise lose the stop it gets today. A matching wrapper without a creation time stays `unknown` even with nothing registered. +- **Snapshot type.** The snapshot type is `WindowsStopProcessEntry`, which is `WindowsProcessEntry` plus `startTime`, in the evidence module. `windowsProcessList` and the guarded inspection stay untouched until PR2 switches them. + +## Open questions + +- Is the closing-sample accounting acceptable, or should the hold be measured differently, for example by shortening the final enumeration to 4.5 s to keep 030's totals? +- Should a WinSW-only stop also sweep a stray scheduler wrapper (that would add an enumeration and 3 s to the WinSW row)? +- ARM64 ships the bounded fallback here. 050 requires an ARM64 device run before release, or else the strict variant. PR2 needs that decision before the fallback is reachable. +- `GetLastError` after a failed `OpenProcess` through Bun FFI separates "gone" from "denied". If the runtime clobbers it, the answer degrades to "unavailable" (`unknown`). The live x64 test in `tests/windows/windows-process-evidence.test.ts` checks this on the Windows CI lane. diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index e413e2845ac..c2ad2913d08 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -1932,8 +1932,8 @@ "windows-deploy-close-regressions.test.ts": "windows", "windows-elevation-spawn.test.ts": "windows", "windows-elevation.test.ts": "windows", - "windows-guarded-manager.test.ts": "windows", - "windows-popup-fix.test.ts": "windows", + "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", + "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", "windows-process-priority.test.ts": "windows", "windows-scheduler-errors.test.ts": "windows", "windows-scheduler-install-verification.test.ts": "windows", "windows-lifecycle-root-acl.test.ts": "windows", diff --git a/src/service/windows-checked-stop.ts b/src/service/windows-checked-stop.ts new file mode 100644 index 00000000000..8af35fda589 --- /dev/null +++ b/src/service/windows-checked-stop.ts @@ -0,0 +1,431 @@ +/** + * Part A of the checked Windows service stop (#7771, 030 "Steps"): stop the manager, end + * the bound wrappers, then prove the manager inactive and every bound wrapper exited. + * + * Part A covers the manager and the wrappers only. The proxy child is stopped afterwards + * by the caller's graceful path (it may refuse, and a refusal forbids a forced stop), and + * the port is confirmed by Part B (src/service/windows-stop-confirm.ts). + * + * 1. Probe registration (task, WinSW; 2 s caps). An unreadable answer is unknown with + * nothing changed. With nothing registered, one enumeration (5 s) looks for a stray + * wrapper of a deleted task; unreadable, or nothing found, is absent. + * Admission runs here, after the probe and before anything changes. + * 2. Bind (src/service/windows-stop-binding.ts) and open an exit watch per wrapper. + * 3. Ask the manager: schtasks /end (2 s); WinSW a non-waiting stop (2 s) and an SCM poll + * capped at + 2 s. Denied is denied; an overrun is unknown; neither goes on. + * 4. Terminate the bound survivors through the checked primitive, deepest first. + * 5. Wait on the bind-time watches (steps 4 and 5 share 3 s), then read the manager state + * once (1 s). No enumeration and no PowerShell run in the wait. + * + * Every external effect is injected; nothing here touches the machine by itself, and + * nothing calls it yet. + */ +import type { WinswStatus } from "../lib/winsw"; +import type { WindowsProcessWatch, WindowsTerminateResult } from "./windows-process-native"; +import type { WindowsStopProcessEntry } from "./windows-process-evidence"; +import type { WindowsSchedulerTaskProbe } from "./windows-scheduler"; +import { bindWindowsStopWrappers, type WindowsBoundWrapper, type WindowsStopBinding } from "./windows-stop-binding"; +import { + DEFAULT_WINSW_STOP_TIMEOUT_MS, + WINDOWS_STOP_CAPS, + windowsStopWorstCase, + type WindowsStopBudgetInput, +} from "./windows-stop-budget"; + +type MaybePromise = T | Promise; + +/** Monotonic clock and cancellable sleep; tests inject a virtual one. */ +export interface WindowsStopClock { + now(): number; + sleep(ms: number, signal?: AbortSignal): Promise; +} + +export const windowsStopClock: WindowsStopClock = { + now: () => performance.now(), + sleep: (ms, signal) => new Promise(resolve => { + if (signal?.aborted) { + resolve(); + return; + } + const done = (): void => { + clearTimeout(timer); + signal?.removeEventListener("abort", done); + resolve(); + }; + const timer = setTimeout(done, Math.max(0, ms)); + signal?.addEventListener("abort", done, { once: true }); + }), +}; + +export type WindowsStopReason = "access-denied" | "evidence-unreadable" | "unbound-wrapper" | "unverified-budget"; +export type WindowsCheckedStopOutcome = "absent" | "stopped" | "survived" | "denied" | "unknown" | "unverified-budget"; +export type WindowsTaskEndResult = "ended" | "not-running" | "denied" | "failed"; +export type WindowsWinswStopResult = "sent" | "denied" | "failed"; +export type WindowsWinswScmState = "stopped" | "running" | "pending" | "nonexistent" | "unknown"; + +export interface WindowsCheckedStopDeps { + clock?: WindowsStopClock; + /** This home's canonical launcher VBS and service script paths. */ + canonicalPaths: readonly string[]; + /** The identity-checked proxy child (readPid), when known. */ + childPid: number | null; + selfPid?: number; + probeScheduler(timeoutMs: number): MaybePromise; + probeWinsw(timeoutMs: number): MaybePromise; + enumerate(timeoutMs: number): MaybePromise; + scheduleServicePid(timeoutMs: number): MaybePromise; + endTask(timeoutMs: number): MaybePromise; + /** Running state of the task (schtasks /query); the final manager-state read. */ + taskState(timeoutMs: number): MaybePromise<"running" | "not-running" | "unknown">; + winswStop(timeoutMs: number): MaybePromise; + winswState(timeoutMs: number): MaybePromise; + /** The WinSW definition's ; 20 s for the generated one. */ + winswStopTimeoutMs?: number; + /** After WinSW reports stopped: is the proxy still alive? null = cannot tell. */ + proxyAlive(timeoutMs: number): MaybePromise; + openWatch(pid: number, startTime: number): MaybePromise; + terminate(pid: number, startTime: number, timeoutMs: number): MaybePromise; +} + +export interface WindowsStopAdmission { + /** Absolute deadline on the deps clock. */ + deadlineAt: number; + budget: Omit; +} + +export interface WindowsCheckedStopOptions { + /** A snapshot that already ran (guarded: revalidateManager's), reused for the bind. */ + bindSnapshot?: readonly WindowsStopProcessEntry[]; + admission?: WindowsStopAdmission; + /** Throws to refuse. Runs at entry and before each terminate. */ + authorize?: () => MaybePromise; +} + +export interface WindowsCheckedStopResult { + outcome: WindowsCheckedStopOutcome; + reason: WindowsStopReason | null; + /** Fixed text for logs and output; it never carries a path or a command line. */ + detail: string; + registration: { scheduler: "present" | "absent" | "unknown"; winsw: WinswStatus }; + /** The bound set (pid + creation time) for the later receipt record. */ + bound: WindowsBoundWrapper[]; + unbound: number[]; + terminated: number[]; + /** True once a manager stop or a terminate was attempted. */ + changed: boolean; + budget?: { requiredMs: number; remainingMs: number }; +} + +/** The authorizer refused; partial says what had already happened. */ +export class WindowsCheckedStopRefusedError extends Error { + constructor(readonly partial: WindowsCheckedStopResult, options: { cause: unknown }) { + super("The checked Windows service stop was refused by its authorizer.", options); + this.name = "WindowsCheckedStopRefusedError"; + } +} + +const C = WINDOWS_STOP_CAPS; +const WINSW_POLL_INTERVAL_MS = 250; +const WATCH_POLL_INTERVAL_MS = 100; +const TIMED_OUT = Symbol("timed-out"); + +type Capped = { ok: true; value: T } | { ok: false; timedOut: boolean }; + +/** + * Run one step under a cap. The race ends the wait for an asynchronous dependency; the + * elapsed check catches one that blocked past the cap anyway. Either is an overrun. + */ +export async function runCapped( + clock: WindowsStopClock, + capMs: number, + run: (timeoutMs: number) => MaybePromise, +): Promise> { + if (capMs <= 0) return { ok: false, timedOut: true }; + const started = clock.now(); + const abort = new AbortController(); + const work = Promise.resolve().then(() => run(capMs)); + work.catch(() => undefined); + try { + const value = await Promise.race([work, clock.sleep(capMs, abort.signal).then(() => TIMED_OUT)]); + if (value === TIMED_OUT || clock.now() - started > capMs) return { ok: false, timedOut: true }; + return { ok: true, value: value as T }; + } catch { + return { ok: false, timedOut: false }; + } finally { + abort.abort(); + } +} + +export async function stopWindowsServiceChecked( + deps: WindowsCheckedStopDeps, + options: WindowsCheckedStopOptions = {}, +): Promise { + const clock = deps.clock ?? windowsStopClock; + const watches = new Map>(); + const result: WindowsCheckedStopResult = { + outcome: "unknown", reason: null, detail: "", + registration: { scheduler: "unknown", winsw: "unknown" }, + bound: [], unbound: [], terminated: [], changed: false, + }; + const finish = (outcome: WindowsCheckedStopOutcome, reason: WindowsStopReason | null, detail: string) => { + result.outcome = outcome; + result.reason = reason; + result.detail = detail; + return result; + }; + const authorize = async (): Promise => { + if (!options.authorize) return; + try { + await options.authorize(); + } catch (cause) { + throw new WindowsCheckedStopRefusedError({ ...result, detail: "the authorizer refused the stop" }, { cause }); + } + }; + try { + await authorize(); + + // Step 1: registration. + const [scheduler, winsw] = await Promise.all([ + runCapped(clock, C.registrationProbe, deps.probeScheduler), + runCapped(clock, C.registrationProbe, deps.probeWinsw), + ]); + result.registration = { + scheduler: scheduler.ok && scheduler.value.status !== "unknown" ? scheduler.value.status : "unknown", + winsw: winsw.ok ? winsw.value : "unknown", + }; + if (result.registration.scheduler === "unknown" || result.registration.winsw === "unknown") { + return finish("unknown", "evidence-unreadable", "the service registration could not be read"); + } + const taskRegistered = result.registration.scheduler === "present"; + const winswRegistered = result.registration.winsw !== "nonexistent"; + const winswStarted = result.registration.winsw === "started"; + const readSchedulePid = async (): Promise => { + const pid = await runCapped(clock, C.scheduleServicePid, deps.scheduleServicePid); + return pid.ok ? pid.value : null; + }; + const bindContext = { + canonicalPaths: deps.canonicalPaths, + childPid: deps.childPid, + selfPid: deps.selfPid ?? process.pid, + }; + + let binding: WindowsStopBinding | null = null; + let schedulePid: number | null = null; + const stray = !taskRegistered && !winswRegistered; + if (stray) { + const snapshot = await runCapped(clock, C.enumeration, deps.enumerate); + // Nothing registered and nothing readable: today's best-effort sweep skips too. + if (!snapshot.ok || snapshot.value === null) { + return finish("absent", null, "no service is registered; the process list could not be read"); + } + schedulePid = await readSchedulePid(); + binding = bindWindowsStopWrappers({ + ...bindContext, processes: snapshot.value, taskRegistered: false, managerRegistered: false, + scheduleServicePid: schedulePid, + }); + if (binding.bound.length === 0 && binding.unbound.length === 0 && binding.unreadable.length === 0) { + return finish("absent", null, "no service is registered and no wrapper of this home is running"); + } + // Neither of these changes anything, so neither waits for admission. + result.unbound = binding.unbound.map(entry => entry.pid); + if (binding.unreadable.length > 0) { + return finish("unknown", "evidence-unreadable", "a possible wrapper of this home has unreadable evidence"); + } + if (binding.bound.length === 0) { + return await unboundOnlyVerdict(binding, deps, watches) + ? finish("survived", "unbound-wrapper", "a wrapper of this home is running but cannot be tied to a manager") + : finish("absent", null, "no service is registered and no wrapper of this home is running"); + } + } else if (taskRegistered) { + schedulePid = await readSchedulePid(); + } + + // Admission: everything still to come must fit before anything changes. + if (options.admission) { + const budget = windowsStopWorstCase({ + ...options.admission.budget, + managers: { scheduler: taskRegistered, winsw: winswStarted, stray }, + enumerationReused: stray || options.bindSnapshot !== undefined, + winswStopTimeoutMs: options.admission.budget.winswStopTimeoutMs ?? deps.winswStopTimeoutMs, + }); + const remainingMs = options.admission.deadlineAt - clock.now(); + result.budget = { requiredMs: budget.totalMs, remainingMs }; + if (remainingMs < budget.totalMs) { + return finish("unverified-budget", "unverified-budget", "the remaining time cannot hold the worst case of the stop"); + } + } + + // Step 2: bind. + if (taskRegistered) { + let processes = options.bindSnapshot ?? null; + if (processes === null) { + const snapshot = await runCapped(clock, C.enumeration, deps.enumerate); + if (!snapshot.ok || snapshot.value === null) { + return finish("unknown", "evidence-unreadable", "the process list could not be read"); + } + processes = snapshot.value; + } + binding = bindWindowsStopWrappers({ + ...bindContext, processes, taskRegistered: true, managerRegistered: true, + scheduleServicePid: schedulePid, + }); + } + if (binding) { + result.unbound = binding.unbound.map(entry => entry.pid); + if (binding.unreadable.length > 0) { + return finish("unknown", "evidence-unreadable", "a possible wrapper of this home has unreadable evidence"); + } + } + const targets: WindowsBoundWrapper[] = []; + for (const wrapper of binding?.bound ?? []) { + const watch = await deps.openWatch(wrapper.pid, wrapper.startTime); + if (watch.status === "gone" || watch.status === "reused") continue; + if (watch.status === "denied") { + return finish("denied", "access-denied", "a bound wrapper cannot be opened by this account"); + } + if (watch.status === "open") watches.set(wrapper.pid, watch); + targets.push(wrapper); + } + result.bound = targets; + const unboundWatches = new Map(); + for (const entry of binding?.unbound ?? []) { + if (entry.startTime === null) continue; + const watch = await deps.openWatch(entry.pid, entry.startTime); + unboundWatches.set(entry.pid, watch); + if (watch.status === "open") watches.set(entry.pid, watch); + } + + // Step 3: ask the manager. + if (taskRegistered) { + result.changed = true; + const ended = await runCapped(clock, C.taskEnd, deps.endTask); + if (!ended.ok) return finish("unknown", "evidence-unreadable", "the task end command did not answer within its cap"); + if (ended.value === "denied") return finish("denied", "access-denied", "ending the task was denied"); + if (ended.value === "failed") return finish("unknown", "evidence-unreadable", "the task end command failed"); + } + if (winswStarted) { + result.changed = true; + const stop = await runCapped(clock, C.winswStopCommand, deps.winswStop); + if (!stop.ok) return finish("unknown", "evidence-unreadable", "the native service stop did not answer within its cap"); + if (stop.value === "denied") return finish("denied", "access-denied", "stopping the native service was denied"); + if (stop.value === "failed") return finish("unknown", "evidence-unreadable", "the native service stop failed"); + const pollEnd = clock.now() + (deps.winswStopTimeoutMs ?? DEFAULT_WINSW_STOP_TIMEOUT_MS) + C.winswPollMargin; + for (;;) { + const left = pollEnd - clock.now(); + const state = await runCapped(clock, Math.min(C.managerState, left), deps.winswState); + if (state.ok && (state.value === "stopped" || state.value === "nonexistent")) break; + if (pollEnd - clock.now() <= 0) { + return finish("unknown", "evidence-unreadable", "the native service did not report stopped within its stop timeout"); + } + await clock.sleep(Math.min(WINSW_POLL_INTERVAL_MS, pollEnd - clock.now())); + } + } + + // Steps 4 and 5: terminate the bound survivors, then wait on the bind-time watches. + const windowEnd = clock.now() + C.terminateAndWait; + const exited = new Map(); + for (const wrapper of targets) { + if (watches.get(wrapper.pid)?.exited() === true) { + exited.set(wrapper.pid, true); + continue; + } + await authorize(); + result.changed = true; + const outcome = await runCapped(clock, windowEnd - clock.now(), + timeoutMs => deps.terminate(wrapper.pid, wrapper.startTime, timeoutMs)); + if (!outcome.ok) return finish("unknown", "evidence-unreadable", "terminating a bound wrapper did not answer within its cap"); + const status = outcome.value.status; + if (status === "denied") return finish("denied", "access-denied", "terminating a bound wrapper was denied"); + if (status === "unavailable") return finish("unknown", "evidence-unreadable", "a bound wrapper could not be terminated"); + if (status === "terminated") { + result.terminated.push(wrapper.pid); + if (outcome.value.exited === true) exited.set(wrapper.pid, true); + } else { + // gone, or reused: the bound process no longer exists either way, and a reused PID + // is a different process that is left untouched. + exited.set(wrapper.pid, true); + } + } + const pending = (): WindowsBoundWrapper[] => targets.filter(wrapper => { + if (exited.get(wrapper.pid)) return false; + if (watches.get(wrapper.pid)?.exited() === true) { + exited.set(wrapper.pid, true); + return false; + } + return true; + }); + while (pending().length > 0 && clock.now() < windowEnd) { + await clock.sleep(Math.min(WATCH_POLL_INTERVAL_MS, windowEnd - clock.now())); + } + let boundAlive = false; + let unreadable = false; + for (const wrapper of pending()) { + const state = watches.get(wrapper.pid)?.exited(); + if (state === false) boundAlive = true; + else unreadable = true; + } + + // Step 5: the manager state, read once. + let managerActive = false; + if (taskRegistered) { + const state = await runCapped(clock, C.managerState, deps.taskState); + if (state.ok && state.value === "running") managerActive = true; + else if (!state.ok || state.value === "unknown") unreadable = true; + } + if (winswStarted) { + const stepEnd = clock.now() + C.managerState; + const state = await runCapped(clock, C.managerState, deps.winswState); + if (state.ok && (state.value === "running" || state.value === "pending")) managerActive = true; + else if (!state.ok || state.value === "unknown") unreadable = true; + const alive = await runCapped(clock, stepEnd - clock.now(), deps.proxyAlive); + if (alive.ok && alive.value === true) managerActive = true; + } + + const unboundAlive = binding ? unboundStillAlive(binding, unboundWatches) : false; + if (unboundAlive) { + return finish("survived", "unbound-wrapper", "a wrapper of this home is running but cannot be tied to a manager"); + } + if (boundAlive || managerActive) return finish("survived", null, "the manager or a bound wrapper is still running"); + if (unreadable) return finish("unknown", "evidence-unreadable", "the stop could not be verified"); + return finish("stopped", null, "the manager is stopped and every bound wrapper has exited"); + } finally { + for (const watch of watches.values()) { + try { watch.close(); } catch { /* closing a handle cannot change the verdict */ } + } + } +} + +/** An unbound wrapper counts as alive unless its watch proves it exited. */ +function unboundStillAlive(binding: WindowsStopBinding, watches: ReadonlyMap): boolean { + return binding.unbound.some(entry => { + const watch = watches.get(entry.pid); + if (!watch) return true; + if (watch.status === "gone" || watch.status === "reused") return false; + return watch.status !== "open" || watch.exited() !== true; + }); +} + +/** Stray path with nothing bound: are the unbound wrappers still there? Read-only. */ +async function unboundOnlyVerdict( + binding: WindowsStopBinding, + deps: WindowsCheckedStopDeps, + opened: Map>, +): Promise { + const watches = new Map(); + for (const entry of binding.unbound) { + if (entry.startTime === null) continue; + const watch = await deps.openWatch(entry.pid, entry.startTime); + watches.set(entry.pid, watch); + if (watch.status === "open") opened.set(entry.pid, watch); + } + return unboundStillAlive(binding, watches); +} + +/** The updater's variant: the authorizer runs at entry and before every terminate. */ +export function guardedStopWindowsServiceChecked( + authorize: () => MaybePromise, + deps: WindowsCheckedStopDeps, + options: Omit = {}, +): Promise { + return stopWindowsServiceChecked(deps, { ...options, authorize }); +} diff --git a/src/service/windows-process-evidence.ts b/src/service/windows-process-evidence.ts new file mode 100644 index 00000000000..a70ce60185d --- /dev/null +++ b/src/service/windows-process-evidence.ts @@ -0,0 +1,288 @@ +/** + * Process evidence for the checked Windows service stop (#7771): a process snapshot that + * carries creation times, the Task Scheduler service PID, a checked terminate, and a + * bind-time exit watch. Each facade uses the in-process primitive from + * src/service/windows-process-native.ts where Bun's FFI runs (Windows x64) and a bounded + * spawned fallback elsewhere on Windows (ARM64): + * + * - snapshot: one Get-CimInstance Win32_Process query with CreationDate. On x64 every + * creation time is re-read through GetProcessTimes and kept only when it agrees with + * CIM to the millisecond, so the bound value is the exact one the handle terminate + * compares later, and a PID reused between the CIM query and the native read is null + * (unreadable) instead of silently taking the new process's identity; + * - Schedule PID: QueryServiceStatusEx on x64, else sc.exe queryex Schedule run from the + * trusted system directory with a cap; unreadable is null; + * - terminate: the handle primitive on x64. ARM64 runs ONE trusted PowerShell script that + * re-reads Get-Process -Id StartTime, compares it with the bound CIM creation time + * (1 s tolerance, UTC), calls Kill() on the same Process object and reads the exit back. + * Kill() opens its own handle by PID, so a window between the in-script check and the + * kill remains; that, and the 1 s tolerance, are the accepted residuals of the fallback; + * - watch: x64 only; elsewhere "unavailable", and the checked stop takes the exit from + * the terminate read-back. + * + * Only integers ever reach a script; paths and command lines are compared in TypeScript. + * Nothing calls this module yet. + */ +import { execFile } from "node:child_process"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; + +import { resolveTrustedWindowsPowerShellExe, resolveTrustedWindowsSystemDirectory } from "../lib/windows-elevation"; +import type { WindowsProcessEntry } from "./windows-guarded-manager"; +import { + windowsProcessNative, + windowsProcessPlatform, + windowsProcessPlatformSimulated, + type WindowsProcessWatch, + type WindowsTerminateResult, +} from "./windows-process-native"; + +/** A spawned helper. "powershell" and "sc" resolve inside the trusted system directory. */ +export interface WindowsEvidenceCommand { + program: "powershell" | "sc"; + args: string[]; + timeoutMs: number; + maxBufferBytes: number; +} + +/** Runs a command and resolves stdout; rejects on a nonzero exit, a timeout or overflow. */ +export type WindowsEvidenceRunner = (command: WindowsEvidenceCommand) => Promise; + +export interface WindowsEvidenceOptions { + timeoutMs: number; + /** Injected in tests. Unset on a simulated platform means no spawn at all. */ + run?: WindowsEvidenceRunner; +} + +/** A process-table row with the creation time the binding rule requires (null = unreadable). */ +export type WindowsStopProcessEntry = WindowsProcessEntry & { startTime: number | null }; + +const SNAPSHOT_MAX_BUFFER = 16 * 1024 * 1024; +const SMALL_MAX_BUFFER = 64 * 1024; +const NATIVE_CIM_TOLERANCE_MS = 1; +/** Kill() read-back inside the ARM64 script; the checked stop's wait covers the rest. */ +export const FALLBACK_TERMINATE_EXIT_WAIT_MS = 1_000; + +function trustedProgram(program: WindowsEvidenceCommand["program"]): string { + if (program === "powershell") return resolveTrustedWindowsPowerShellExe(); + const sc = join(resolveTrustedWindowsSystemDirectory(), "sc.exe"); + if (!existsSync(sc)) throw new Error("sc.exe was not found in the trusted system directory."); + return sc; +} + +const defaultRunner: WindowsEvidenceRunner = command => new Promise((resolve, reject) => { + let file: string; + try { + file = trustedProgram(command.program); + } catch (error) { + reject(error); + return; + } + execFile(file, command.args, { + encoding: "utf8", + timeout: command.timeoutMs, + maxBuffer: command.maxBufferBytes, + windowsHide: true, + }, (error, stdout) => { + if (error) reject(error); + else resolve(stdout); + }); +}); + +function runnerFor(options: WindowsEvidenceOptions): WindowsEvidenceRunner | null { + if (options.run) return options.run; + if (windowsProcessPlatformSimulated() || windowsProcessPlatform() !== "win32") return null; + return defaultRunner; +} + +function powershellArgs(script: string): string[] { + return ["-NoProfile", "-NoLogo", "-NonInteractive", "-Command", script]; +} + +/** The snapshot query. CreationDate is emitted as an ISO UTC string, never as a JSON date. */ +export const WINDOWS_PROCESS_SNAPSHOT_SCRIPT = "Get-CimInstance Win32_Process | Select-Object " + + "ProcessId,ParentProcessId,Name,CommandLine," + + "@{Name='CreationDate';Expression={ if ($_.CreationDate) { $_.CreationDate.ToUniversalTime().ToString('o') } else { $null } }} " + + "| ConvertTo-Json -Compress"; + +/** Parse the snapshot JSON. Unparseable output is null: an unread table proves nothing. */ +export function parseWindowsProcessSnapshot(output: string): WindowsStopProcessEntry[] | null { + const trimmed = output.trim(); + if (!trimmed) return []; + let parsed: unknown; + try { + parsed = JSON.parse(trimmed); + } catch { + return null; + } + const rows = Array.isArray(parsed) ? parsed : [parsed]; + const entries: WindowsStopProcessEntry[] = []; + for (const row of rows) { + if (typeof row !== "object" || row === null) continue; + const record = row as Record; + const pid = Number(record.ProcessId); + const parent = Number(record.ParentProcessId); + if (!Number.isSafeInteger(pid) || pid <= 0) continue; + const created = typeof record.CreationDate === "string" ? Date.parse(record.CreationDate) : Number.NaN; + entries.push({ + pid, + parentPid: Number.isSafeInteger(parent) && parent > 0 ? parent : null, + name: typeof record.Name === "string" ? record.Name : null, + commandLine: typeof record.CommandLine === "string" ? record.CommandLine : null, + startTime: Number.isFinite(created) ? created : null, + }); + } + return entries; +} + +/** + * Replace each CIM creation time with the native one when both agree to the millisecond. + * A disagreement (or a missing side) is null: the binding rule then treats the process as + * unreadable rather than binding an identity the terminate check would not reproduce. + */ +export function overlayNativeCreationTimes( + entries: readonly WindowsStopProcessEntry[], + creationTime: (pid: number) => number | null, +): WindowsStopProcessEntry[] { + return entries.map(entry => { + if (entry.startTime === null) return entry; + let native: number | null; + try { + native = creationTime(entry.pid); + } catch { + native = null; + } + const agrees = native !== null && Math.abs(native - entry.startTime) <= NATIVE_CIM_TOLERANCE_MS; + return { ...entry, startTime: agrees ? native : null }; + }); +} + +/** Every process with its creation time, or null when the table cannot be read. */ +export async function windowsProcessSnapshot(options: WindowsEvidenceOptions): Promise { + const run = runnerFor(options); + if (!run) return null; + let entries: WindowsStopProcessEntry[] | null; + try { + entries = parseWindowsProcessSnapshot(await run({ + program: "powershell", + args: powershellArgs(WINDOWS_PROCESS_SNAPSHOT_SCRIPT), + timeoutMs: options.timeoutMs, + maxBufferBytes: SNAPSHOT_MAX_BUFFER, + })); + } catch { + return null; + } + if (entries === null) return null; + const native = windowsProcessNative(); + return native ? overlayNativeCreationTimes(entries, pid => native.creationTime(pid)) : entries; +} + +/** + * sc.exe queryex Schedule: the PID only while STATE is 4 (RUNNING). The field names and + * the numeric state are not localized; the trailing state word is ignored. + */ +export function parseScQueryexPid(output: string): number | null { + const state = /^\s*STATE\s*:\s*(\d+)/m.exec(output); + if (!state || Number(state[1]) !== 4) return null; + const pid = /^\s*PID\s*:\s*(\d+)\s*$/m.exec(output); + if (!pid) return null; + const value = Number(pid[1]); + return Number.isSafeInteger(value) && value > 0 ? value : null; +} + +/** The Task Scheduler service host's PID, or null when it cannot be proven. */ +export async function readScheduleServicePid(options: WindowsEvidenceOptions): Promise { + const native = options.run ? null : windowsProcessNative(); + if (native) { + try { + return native.scheduleServicePid(); + } catch { + return null; + } + } + const run = runnerFor(options); + if (!run) return null; + try { + return parseScQueryexPid(await run({ + program: "sc", + args: ["queryex", "Schedule"], + timeoutMs: options.timeoutMs, + maxBufferBytes: SMALL_MAX_BUFFER, + })); + } catch { + return null; + } +} + +/** + * The ARM64 terminate script. pid and the bound time must be safe integers; nothing else + * is interpolated. One status word is printed on the last line. + */ +export function windowsFallbackTerminateScript(pid: number, boundStartTime: number, exitWaitMs: number): string { + for (const value of [pid, boundStartTime, exitWaitMs]) { + if (!Number.isSafeInteger(value) || value < 0) throw new Error("terminate script inputs must be non-negative integers"); + } + return [ + "$ErrorActionPreference = 'Stop';", + "try { $p = Get-Process -Id " + pid + " -ErrorAction Stop } catch { 'gone'; exit 0 };", + "try { $start = $p.StartTime.ToUniversalTime() } catch {", + " try { if ($p.HasExited) { 'gone' } else { 'denied' } } catch { 'denied' }; exit 0 };", + "$bound = [DateTimeOffset]::FromUnixTimeMilliseconds(" + boundStartTime + ").UtcDateTime;", + "if ([Math]::Abs(($start - $bound).TotalMilliseconds) -gt 1000) { 'reused'; exit 0 };", + "try { $p.Kill() }", + "catch [System.InvalidOperationException] { 'gone'; exit 0 }", + "catch [System.ComponentModel.Win32Exception] { 'denied'; exit 0 }", + "catch { 'unavailable'; exit 0 };", + "if ($p.WaitForExit(" + exitWaitMs + ")) { 'terminated-exited' } else { 'terminated' }", + ].join(" "); +} + +/** Map the script's last line; anything unexpected is "unavailable", never success. */ +export function parseFallbackTerminateOutput(output: string): WindowsTerminateResult { + const lines = output.split(/\r?\n/).map(line => line.trim()).filter(Boolean); + const word = lines[lines.length - 1] ?? ""; + if (word === "terminated-exited") return { status: "terminated", exited: true }; + if (word === "terminated") return { status: "terminated", exited: false }; + if (word === "gone" || word === "reused" || word === "denied") return { status: word }; + return { status: "unavailable" }; +} + +/** Terminate pid only if it is still the process created at boundStartTime. */ +export async function terminateWindowsProcessBound( + pid: number, + boundStartTime: number, + options: WindowsEvidenceOptions, +): Promise { + const native = options.run ? null : windowsProcessNative(); + if (native) { + try { + return native.terminate(pid, boundStartTime); + } catch { + return { status: "unavailable" }; + } + } + const run = runnerFor(options); + if (!run) return { status: "unavailable" }; + const exitWaitMs = Math.max(0, Math.min(FALLBACK_TERMINATE_EXIT_WAIT_MS, options.timeoutMs - 500)); + try { + return parseFallbackTerminateOutput(await run({ + program: "powershell", + args: powershellArgs(windowsFallbackTerminateScript(pid, boundStartTime, exitWaitMs)), + timeoutMs: options.timeoutMs, + maxBufferBytes: SMALL_MAX_BUFFER, + })); + } catch { + return { status: "unavailable" }; + } +} + +/** A bind-time exit watch; "unavailable" wherever the handle primitive cannot run. */ +export function openWindowsProcessWatch(pid: number, boundStartTime: number): WindowsProcessWatch { + const native = windowsProcessNative(); + if (!native) return { status: "unavailable" }; + try { + return native.openWatch(pid, boundStartTime); + } catch { + return { status: "unavailable" }; + } +} diff --git a/src/service/windows-process-native.ts b/src/service/windows-process-native.ts new file mode 100644 index 00000000000..0c5b1bff635 --- /dev/null +++ b/src/service/windows-process-native.ts @@ -0,0 +1,262 @@ +/** + * In-process Windows process primitives for the checked service stop (#7771). + * + * Four questions the checked stop asks of the kernel and the SCM: + * + * - when was this process created (GetProcessTimes), so a PID that was reused between + * bind and terminate is told apart from the process that was bound; + * - terminate exactly the bound process: OpenProcess, read the creation time on THAT + * handle, compare, TerminateProcess on the same handle, close. There is no gap between + * the identity check and the kill in which the PID could be reused; + * - has a bound process exited (WaitForSingleObject with a zero timeout on a handle + * opened at bind time), so the wait needs neither an enumeration nor PowerShell; + * - which process hosts the Task Scheduler service (QueryServiceStatusEx on "Schedule"), + * the parent the binding rule accepts for a launcher the scheduler started. + * + * Loading follows src/config/windows-process-command-line.ts: bun:ffi is required lazily + * (every importer loads this module on every platform), the DLLs load from the + * GetSystemDirectoryW path so the search order cannot substitute them, a failed load is + * cached as null, and every handle is closed in a finally. Windows ARM64 and every other + * platform get null here; src/service/windows-process-evidence.ts then uses the spawned + * fallbacks. A test that simulates a platform never reaches the real kernel. + * + * Nothing calls this module yet; the stop paths move to it in a later change. + */ +import { join } from "node:path"; + +import { resolveTrustedWindowsSystemDirectory } from "../lib/windows-elevation"; + +const PROCESS_TERMINATE = 0x0001; +const PROCESS_QUERY_LIMITED_INFORMATION = 0x1000; +const SYNCHRONIZE = 0x0010_0000; +const WAIT_OBJECT_0 = 0; +const WAIT_TIMEOUT = 0x102; +const ERROR_ACCESS_DENIED = 5; +const ERROR_INVALID_PARAMETER = 87; +const SC_MANAGER_CONNECT = 0x0001; +const SERVICE_QUERY_STATUS = 0x0004; +const SC_STATUS_PROCESS_INFO = 0; +const SERVICE_RUNNING = 4; +/** SERVICE_STATUS_PROCESS is nine DWORDs; dwCurrentState is the second, dwProcessId the eighth. */ +const SERVICE_STATUS_PROCESS_BYTES = 36; +const SERVICE_STATUS_STATE_OFFSET = 4; +const SERVICE_STATUS_PID_OFFSET = 28; +/** 100 ns intervals between 1601-01-01 and 1970-01-01. */ +const FILETIME_UNIX_EPOCH = 116_444_736_000_000_000n; +const TERMINATE_EXIT_CODE = 1; + +/** Result of one checked terminate. Only "terminated" means this call ended the process. */ +export type WindowsTerminateStatus = "terminated" | "gone" | "reused" | "denied" | "unavailable"; + +export interface WindowsTerminateResult { + status: WindowsTerminateStatus; + /** + * True when the primitive itself observed the process exit after terminating it. The + * x64 primitive does not wait (the checked stop waits on its bind-time handles); the + * ARM64 fallback reads the exit back inside its script and reports it here. + */ + exited?: boolean; +} + +/** A bind-time handle the checked stop polls instead of enumerating processes again. */ +export type WindowsProcessWatch = + | { status: "open"; exited(): boolean | null; close(): void } + | { status: "gone" | "reused" | "denied" | "unavailable" }; + +export interface WindowsProcessNativeApi { + /** Creation time in epoch ms (FILETIME truncated to ms), or null when unreadable. */ + creationTime(pid: number): number | null; + terminate(pid: number, boundStartTime: number): WindowsTerminateResult; + openWatch(pid: number, boundStartTime: number): WindowsProcessWatch; + /** PID of the running Task Scheduler service host, or null when it cannot be read. */ + scheduleServicePid(): number | null; +} + +/** FILETIME (100 ns since 1601) to epoch ms, truncated. Zero or pre-1970 is unreadable. */ +export function fileTimeToEpochMs(fileTime: bigint): number | null { + if (fileTime <= FILETIME_UNIX_EPOCH) return null; + const ms = (fileTime - FILETIME_UNIX_EPOCH) / 10_000n; + return ms > BigInt(Number.MAX_SAFE_INTEGER) ? null : Number(ms); +} + +/** Decode SERVICE_STATUS_PROCESS: the PID only while the service reports RUNNING. */ +export function decodeServiceStatusProcess(buffer: Uint8Array): number | null { + if (buffer.byteLength < SERVICE_STATUS_PROCESS_BYTES) return null; + const view = new DataView(buffer.buffer, buffer.byteOffset, buffer.byteLength); + if (view.getUint32(SERVICE_STATUS_STATE_OFFSET, true) !== SERVICE_RUNNING) return null; + const pid = view.getUint32(SERVICE_STATUS_PID_OFFSET, true); + return pid > 0 ? pid : null; +} + +let nativeApi: WindowsProcessNativeApi | null | undefined; +let nativeApiForTests: WindowsProcessNativeApi | null | undefined; +let platformForTests: NodeJS.Platform | null = null; + +/** Replace the native API in tests; null forces "unavailable", undefined restores the real one. */ +export function setWindowsProcessNativeForTests(next: WindowsProcessNativeApi | null | undefined): void { + nativeApiForTests = next; +} + +/** + * Simulate a platform in tests. While set, the real kernel is never called: the native + * API reads as unavailable unless a test API is installed as well. + */ +export function setWindowsProcessPlatformForTests(next: NodeJS.Platform | null): void { + platformForTests = next; +} + +/** The platform the Windows stop primitives act for (the simulated one in tests). */ +export function windowsProcessPlatform(): NodeJS.Platform { + return platformForTests ?? process.platform; +} + +/** True while a test simulates the platform; nothing may then reach the real machine. */ +export function windowsProcessPlatformSimulated(): boolean { + return platformForTests !== null; +} + +type Handle = number | bigint; + +function loadNativeApi(): WindowsProcessNativeApi | null { + if (nativeApi !== undefined) return nativeApi; + nativeApi = null; + if (process.platform !== "win32" || process.arch !== "x64") return null; + let ptr: typeof import("bun:ffi").ptr; + let kernel32; + let advapi32; + try { + const ffi = require("bun:ffi") as typeof import("bun:ffi"); + ptr = ffi.ptr; + const systemDirectory = resolveTrustedWindowsSystemDirectory(); + kernel32 = ffi.dlopen(join(systemDirectory, "kernel32.dll"), { + OpenProcess: { args: ["u32", "i32", "u32"], returns: "u64" }, + CloseHandle: { args: ["u64"], returns: "i32" }, + GetProcessTimes: { args: ["u64", "ptr", "ptr", "ptr", "ptr"], returns: "i32" }, + TerminateProcess: { args: ["u64", "u32"], returns: "i32" }, + WaitForSingleObject: { args: ["u64", "u32"], returns: "u32" }, + GetLastError: { args: [], returns: "u32" }, + }).symbols; + advapi32 = ffi.dlopen(join(systemDirectory, "advapi32.dll"), { + OpenSCManagerW: { args: ["ptr", "ptr", "u32"], returns: "u64" }, + OpenServiceW: { args: ["u64", "ptr", "u32"], returns: "u64" }, + QueryServiceStatusEx: { args: ["u64", "i32", "ptr", "u32", "ptr"], returns: "i32" }, + CloseServiceHandle: { args: ["u64"], returns: "i32" }, + }).symbols; + } catch { + return null; + } + const k = kernel32; + const a = advapi32; + + const handleCreationTime = (handle: Handle): number | null => { + const times = new BigUint64Array(4); + const base = ptr(times); + const ok = k.GetProcessTimes(handle, base, base + 8, base + 16, base + 24); + return ok ? fileTimeToEpochMs(times[0]!) : null; + }; + // true = exited, false = running, null = unreadable. + const handleExited = (handle: Handle): boolean | null => { + const wait = Number(k.WaitForSingleObject(handle, 0)) >>> 0; + if (wait === WAIT_OBJECT_0) return true; + if (wait === WAIT_TIMEOUT) return false; + return null; + }; + // GetLastError is read immediately after the failed call. If the runtime ever clobbers + // it, the answer degrades to "unavailable", which the checked stop reports as unknown. + const openFailure = (): "gone" | "denied" | "unavailable" => { + const error = Number(k.GetLastError()) >>> 0; + if (error === ERROR_INVALID_PARAMETER) return "gone"; + if (error === ERROR_ACCESS_DENIED) return "denied"; + return "unavailable"; + }; + + nativeApi = { + creationTime(pid) { + const handle = k.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, 0, pid); + if (!handle) return null; + try { + return handleCreationTime(handle); + } finally { + k.CloseHandle(handle); + } + }, + terminate(pid, boundStartTime) { + const handle = k.OpenProcess(PROCESS_TERMINATE | PROCESS_QUERY_LIMITED_INFORMATION | SYNCHRONIZE, 0, pid); + if (!handle) return { status: openFailure() }; + try { + const created = handleCreationTime(handle); + if (created === null) return { status: "unavailable" }; + if (created !== boundStartTime) return { status: "reused" }; + const before = handleExited(handle); + if (before === true) return { status: "gone" }; + if (before === null) return { status: "unavailable" }; + if (k.TerminateProcess(handle, TERMINATE_EXIT_CODE)) return { status: "terminated" }; + const error = Number(k.GetLastError()) >>> 0; + // TerminateProcess on a process that exited meanwhile also fails with access denied. + if (handleExited(handle) === true) return { status: "gone" }; + return { status: error === ERROR_ACCESS_DENIED ? "denied" : "unavailable" }; + } finally { + k.CloseHandle(handle); + } + }, + openWatch(pid, boundStartTime) { + const handle = k.OpenProcess(SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION, 0, pid); + if (!handle) return { status: openFailure() }; + let keep = false; + try { + const created = handleCreationTime(handle); + if (created === null) return { status: "unavailable" }; + if (created !== boundStartTime) return { status: "reused" }; + keep = true; + let closed = false; + return { + status: "open", + exited: () => (closed ? null : handleExited(handle)), + close: () => { + if (closed) return; + closed = true; + k.CloseHandle(handle); + }, + }; + } finally { + if (!keep) k.CloseHandle(handle); + } + }, + scheduleServicePid() { + const manager = a.OpenSCManagerW(null, null, SC_MANAGER_CONNECT); + if (!manager) return null; + try { + const name = Buffer.from("Schedule\0", "utf16le"); + const service = a.OpenServiceW(manager, ptr(name), SERVICE_QUERY_STATUS); + if (!service) return null; + try { + const buffer = new Uint8Array(SERVICE_STATUS_PROCESS_BYTES); + const needed = new Uint32Array(1); + const ok = a.QueryServiceStatusEx( + service, SC_STATUS_PROCESS_INFO, ptr(buffer), SERVICE_STATUS_PROCESS_BYTES, ptr(needed), + ); + return ok ? decodeServiceStatusProcess(buffer) : null; + } finally { + a.CloseServiceHandle(service); + } + } finally { + a.CloseServiceHandle(manager); + } + }, + }; + return nativeApi; +} + +/** + * The native API, or null where it cannot run: non-Windows, Windows ARM64, a failed DLL + * load, or a test that simulates a platform without installing a test API. + */ +export function windowsProcessNative(): WindowsProcessNativeApi | null { + if (nativeApiForTests !== undefined) return nativeApiForTests; + if (platformForTests !== null) return null; + try { + return loadNativeApi(); + } catch { + return null; + } +} diff --git a/src/service/windows-stop-binding.ts b/src/service/windows-stop-binding.ts new file mode 100644 index 00000000000..b1c45dd6280 --- /dev/null +++ b/src/service/windows-stop-binding.ts @@ -0,0 +1,187 @@ +/** + * Which processes the checked Windows stop may terminate (#7771, 030 "Binding rule"). + * + * A process is bound, and only a bound process is ever terminated, when ALL hold: + * + * 1. its image is wscript.exe, cscript.exe or cmd.exe. The proxy child is observed for + * lineage only and is never a target; + * 2. its command line carries this home's canonical launcher VBS or service script path + * as a complete token (commandLineHasPathToken); + * 3. lineage: it is an ancestor of the identity-checked child, or a descendant of a bound + * wrapper (which covers "its parent is a bound wrapper"), or its parent is the Task + * Scheduler service host proven by the SCM PID, or, only while this home's task is + * registered, its parent is gone. A parent PID that is not in the table, or whose + * creation time is later than the wrapper's (a reused PID), counts as gone; + * 4. a creation time was read for it. + * + * A wrapper that passes 1-2 but fails lineage (a launcher started by hand from a live + * shell, an orphan while no task is registered) is reported as unbound and never touched. + * A candidate whose evidence cannot be read is reported as unreadable, never skipped and + * never killed: a matching wrapper without a creation time or image name, a matching + * wrapper whose parent PID is unreadable while the task is registered, and a wrapper-image + * process with no readable command line that descends from a bound wrapper or, while a + * manager is registered, is an ancestor of the child. + * + * Pure: the caller supplies the snapshot and every fact; nothing here reads the machine. + */ +import { createHash } from "node:crypto"; + +import { commandLineHasPathToken } from "./windows-guarded-manager"; +import type { WindowsStopProcessEntry } from "./windows-process-evidence"; + +const WRAPPER_IMAGES = new Set(["wscript.exe", "cscript.exe", "cmd.exe"]); +const DEPTH_LIMIT = 32; + +export interface WindowsStopBindingContext { + processes: readonly WindowsStopProcessEntry[]; + /** This home's canonical launcher VBS and service script paths. */ + canonicalPaths: readonly string[]; + /** The identity-checked proxy child (readPid), when there is one. */ + childPid: number | null; + /** This home's OpenCodex Task Scheduler task is registered (enables "parent gone"). */ + taskRegistered: boolean; + /** Any manager (task or WinSW) is registered: unreadable ancestors of the child count. */ + managerRegistered: boolean; + /** PID of the Task Scheduler service host from the SCM; null disables that clause. */ + scheduleServicePid: number | null; + /** The stopping process itself, never a candidate. */ + selfPid?: number; +} + +/** What a later receipt records about one bound wrapper. The command line is only digested. */ +export interface WindowsBoundWrapper { + pid: number; + startTime: number; + image: string; + parentPid: number | null; + commandLineDigest: string; +} + +export interface WindowsStopBinding { + /** Terminate order: the deepest bound wrapper first (the looping cmd.exe before its launcher). */ + bound: WindowsBoundWrapper[]; + /** Match image and path but fail lineage. Never terminated; alive at the end = unbound-wrapper. */ + unbound: { pid: number; startTime: number | null }[]; + /** Candidates whose evidence cannot be read; any entry makes the stop unknown. */ + unreadable: number[]; + /** The child as observed in the snapshot, for lineage and a WinSW survival check. */ + child: { pid: number; startTime: number | null } | null; +} + +/** The lowercased image name when it is one of the three wrapper images, else null. */ +export function windowsWrapperImage(name: string | null | undefined): string | null { + if (!name) return null; + const base = name.trim().toLowerCase().split(/[\\/]/).pop() ?? ""; + return WRAPPER_IMAGES.has(base) ? base : null; +} + +function digest(commandLine: string): string { + return createHash("sha256").update(commandLine, "utf8").digest("hex"); +} + +type ParentLink = WindowsStopProcessEntry | "gone" | "unreadable"; + +export function bindWindowsStopWrappers(context: WindowsStopBindingContext): WindowsStopBinding { + const processes = context.processes.filter(entry => entry.pid !== context.selfPid); + const byPid = new Map(processes.map(entry => [entry.pid, entry])); + + const parentLink = (entry: WindowsStopProcessEntry): ParentLink => { + if (entry.parentPid === null) return "unreadable"; + const parent = byPid.get(entry.parentPid); + if (!parent) return "gone"; + if (entry.startTime !== null && parent.startTime !== null && parent.startTime > entry.startTime) return "gone"; + return parent; + }; + const ancestors = (entry: WindowsStopProcessEntry): WindowsStopProcessEntry[] => { + const chain: WindowsStopProcessEntry[] = []; + const seen = new Set([entry.pid]); + let current = entry; + for (let depth = 0; depth < DEPTH_LIMIT; depth += 1) { + const link = parentLink(current); + if (typeof link === "string" || seen.has(link.pid)) break; + seen.add(link.pid); + chain.push(link); + current = link; + } + return chain; + }; + + const childEntry = context.childPid === null ? undefined : byPid.get(context.childPid); + const childAncestors = new Set(childEntry ? ancestors(childEntry).map(entry => entry.pid) : []); + const matchesPath = (entry: WindowsStopProcessEntry): boolean => + context.canonicalPaths.some(path => commandLineHasPathToken(entry.commandLine, path)); + + const unreadable = new Set(); + const candidates: WindowsStopProcessEntry[] = []; + for (const entry of processes) { + if (entry.pid === context.childPid || !matchesPath(entry)) continue; + if (entry.name === null) unreadable.add(entry.pid); + else if (windowsWrapperImage(entry.name)) candidates.push(entry); + } + + const bound = new Set(); + const descendsFromBound = (entry: WindowsStopProcessEntry): boolean => + ancestors(entry).some(ancestor => bound.has(ancestor.pid)); + const lineage = (entry: WindowsStopProcessEntry): boolean => { + if (childAncestors.has(entry.pid) || descendsFromBound(entry)) return true; + const link = parentLink(entry); + if (link === "gone") return context.taskRegistered; + if (link === "unreadable") return false; + return context.scheduleServicePid !== null && link.pid === context.scheduleServicePid; + }; + for (let changed = true; changed;) { + changed = false; + for (const entry of candidates) { + if (bound.has(entry.pid) || entry.startTime === null) continue; + if (lineage(entry)) { + bound.add(entry.pid); + changed = true; + } + } + } + + const unbound: WindowsStopBinding["unbound"] = []; + for (const entry of candidates) { + if (bound.has(entry.pid)) continue; + if (entry.startTime === null) unreadable.add(entry.pid); + else if (context.taskRegistered && parentLink(entry) === "unreadable") unreadable.add(entry.pid); + else unbound.push({ pid: entry.pid, startTime: entry.startTime }); + } + + for (const entry of processes) { + if (entry.pid === context.childPid || entry.commandLine?.trim() || !windowsWrapperImage(entry.name)) continue; + if (descendsFromBound(entry) || (context.managerRegistered && childAncestors.has(entry.pid))) { + unreadable.add(entry.pid); + } + } + + const boundEntries = candidates.filter(entry => bound.has(entry.pid)); + const depth = (entry: WindowsStopProcessEntry): number => + ancestors(entry).filter(ancestor => bound.has(ancestor.pid)).length; + boundEntries.sort((a, b) => depth(b) - depth(a) || a.pid - b.pid); + + return { + bound: boundEntries.map(entry => ({ + pid: entry.pid, + startTime: entry.startTime!, + image: windowsWrapperImage(entry.name)!, + parentPid: entry.parentPid, + commandLineDigest: digest(entry.commandLine ?? ""), + })), + unbound, + unreadable: [...unreadable].sort((a, b) => a - b), + child: childEntry ? { pid: childEntry.pid, startTime: childEntry.startTime } : null, + }; +} + +/** Wrapper-image processes still carrying this home's paths (Part B's final enumeration). */ +export function windowsHomeWrapperPids( + processes: readonly WindowsStopProcessEntry[], + canonicalPaths: readonly string[], + selfPid?: number, +): number[] { + return processes + .filter(entry => entry.pid !== selfPid && windowsWrapperImage(entry.name) + && canonicalPaths.some(path => commandLineHasPathToken(entry.commandLine, path))) + .map(entry => entry.pid); +} diff --git a/src/service/windows-stop-budget.ts b/src/service/windows-stop-budget.ts new file mode 100644 index 00000000000..bc82fa239d6 --- /dev/null +++ b/src/service/windows-stop-budget.ts @@ -0,0 +1,122 @@ +/** + * Time budget of the checked Windows stop (#7771, 030 "Time budget"). + * + * One deadline governs the whole stop. Admission runs after Part A's registration probe + * and before anything changes: it adds up the worst case of every step still to come and + * refuses (unverified-budget, nothing changed) when the time left cannot hold it. Every + * step after admission has a cap, and these constants are those caps, so an admitted stop + * cannot overrun its deadline and the graceful child exit wait is never shortened. + * + * Pure arithmetic; the callers own the clock. + */ + +/** Caps of the individual steps, in ms. Part A and Part B enforce exactly these. */ +export const WINDOWS_STOP_CAPS = { + registrationProbe: 2_000, + enumeration: 5_000, + scheduleServicePid: 2_000, + taskEnd: 2_000, + winswStopCommand: 2_000, + winswPollMargin: 2_000, + /** Terminate calls and the bind-time handle wait share this window. */ + terminateAndWait: 3_000, + managerState: 1_000, + orphanDiscovery: 3_000, + postAnswer: 10_000, + forcedKill: 5_000, + forcedExitWait: 5_000, + childExitMargin: 3_000, + inheritedReceiptProbe: 3_000, + partBHoldFloor: 6_000, + partBWindowCeiling: 12_000, + /** + * A port sample may start at the very end of the window (the hold is measured between + * sample starts), so the closing sample can run this long past it. 030 did not count it. + */ + partBClosingSample: 500, + partBFinalEnumeration: 5_000, + reserve: 3_000, +} as const; + +/** The of the generated WinSW definition. */ +export const DEFAULT_WINSW_STOP_TIMEOUT_MS = 20_000; +/** shutdownTimeoutMs default the child exit row is computed from. */ +export const DEFAULT_SHUTDOWN_TIMEOUT_MS = 5_000; +/** Deadline of a caller that passes none (a terminal, a script, an older Desktop). */ +export const DEFAULT_WINDOWS_STOP_DEADLINE_MS = 60_000; + +export interface WindowsStopBudgetInput { + /** What Part A's registration probe found. */ + managers: { scheduler: boolean; winsw: boolean; stray: boolean }; + /** The bind reuses an enumeration that already ran (guarded: revalidateManager's). */ + enumerationReused: boolean; + /** Snapshotless stop without a PID file, or after WinSW stopped the proxy. */ + orphanDiscovery: boolean; + shutdownTimeoutMs?: number; + /** Abandoned teardown receipts whose endpoints are probed (two hosts each). */ + inheritedReceipts?: number; + winswStopTimeoutMs?: number; + /** Part B runs after Part A (false only where 030 skips it, such as the route). */ + partB?: boolean; +} + +export interface WindowsStopBudgetRow { + step: "part-a" | "orphan-discovery" | "post-answer" | "child-exit" | "inherited-receipts" | "part-b" | "reserve"; + ms: number; +} + +export interface WindowsStopBudget { + rows: WindowsStopBudgetRow[]; + totalMs: number; +} + +const C = WINDOWS_STOP_CAPS; + +/** Worst case of Part A still to come after its registration probe. */ +export function windowsPartAWorstCaseMs(input: Pick): number { + const { scheduler, winsw, stray } = input.managers; + const enumeration = input.enumerationReused ? 0 : C.enumeration; + let ms = 0; + if (scheduler) ms += enumeration + C.taskEnd + C.terminateAndWait + C.managerState; + if (winsw) { + const stopTimeout = input.winswStopTimeoutMs ?? DEFAULT_WINSW_STOP_TIMEOUT_MS; + // WinSW has no wrapper: no bind, no terminate. Its own stop ends the proxy within + // , so the SCM poll is capped at that plus a margin. + ms += C.winswStopCommand + stopTimeout + C.winswPollMargin + C.managerState; + } + // Nothing registered: the stray enumeration already ran in the registration step. + if (!scheduler && !winsw && stray) ms += C.terminateAndWait + C.managerState; + return ms; +} + +/** + * Part B's sampling window: min(12 s, remaining - closing sample - final enumeration - + * reserve), floor 6 s. + */ +export function windowsPartBWindowMs(remainingMs: number, reserveMs: number = C.reserve): number { + const room = remainingMs - C.partBClosingSample - C.partBFinalEnumeration - reserveMs; + return Math.max(C.partBHoldFloor, Math.min(C.partBWindowCeiling, room)); +} + +/** The worst case of everything still to come at admission, row by row. */ +export function windowsStopWorstCase(input: WindowsStopBudgetInput): WindowsStopBudget { + const { scheduler, winsw, stray } = input.managers; + // WinSW stops the proxy itself, so the POST and the child exit ladder do not apply to it. + const winswOnly = winsw && !scheduler && !stray; + const shutdown = input.shutdownTimeoutMs ?? DEFAULT_SHUTDOWN_TIMEOUT_MS; + const rows: WindowsStopBudgetRow[] = [ + { step: "part-a", ms: windowsPartAWorstCaseMs(input) }, + ]; + if (input.orphanDiscovery) rows.push({ step: "orphan-discovery", ms: C.orphanDiscovery }); + if (!winswOnly) { + rows.push({ step: "post-answer", ms: C.postAnswer }); + rows.push({ step: "child-exit", ms: shutdown + C.childExitMargin + C.forcedKill + C.forcedExitWait }); + } + const receipts = input.inheritedReceipts ?? 0; + if (receipts > 0) rows.push({ step: "inherited-receipts", ms: receipts * C.inheritedReceiptProbe }); + if (input.partB ?? true) { + rows.push({ step: "part-b", ms: C.partBHoldFloor + C.partBClosingSample + C.partBFinalEnumeration }); + } + rows.push({ step: "reserve", ms: C.reserve }); + return { rows, totalMs: rows.reduce((sum, row) => sum + row.ms, 0) }; +} diff --git a/src/service/windows-stop-confirm.ts b/src/service/windows-stop-confirm.ts new file mode 100644 index 00000000000..38d2c9baa6b --- /dev/null +++ b/src/service/windows-stop-confirm.ts @@ -0,0 +1,158 @@ +/** + * Part B of the checked Windows service stop (#7771, 030 "Part B"), run by the caller + * after its graceful child stop: the port is free and stays free for one relaunch period, + * no wrapper of this home is running, and a remaining listener is classified. + * + * 1. Sample the port until it has been free for the 6 s hold, measured between sample + * starts, within the window from windowsPartBWindowMs (6-12 s, starting now). A loop that + * relaunches the proxy late breaks the streak. + * 2. Then, under one 5 s cap, run the final enumeration and, if the port is still held, + * identify its listener. The enumeration must show no wrapper-image process carrying + * this home's path token; an unreadable table is non-success, and a wrapper started + * after bind makes the stop survived even with the port free. + * 3. A remaining listener is ours (survived), foreign (portHeldByForeign) or left by an + * owner that has exited, which is not foreign: the caller's reclaim clears that one. + * + * Every effect is injected; nothing calls this yet. + */ +import type { WindowsStopProcessEntry } from "./windows-process-evidence"; +import { windowsHomeWrapperPids } from "./windows-stop-binding"; +import { runCapped, windowsStopClock, type WindowsStopClock } from "./windows-checked-stop"; +import { commandLineHasPathToken } from "./windows-guarded-manager"; +import { WINDOWS_STOP_CAPS, windowsPartBWindowMs } from "./windows-stop-budget"; + +type MaybePromise = T | Promise; + +export type WindowsPortSample = + | { state: "free" } + | { state: "listening"; ownerPid: number | null } + | { state: "unknown" }; + +/** ours = this installation's identity answers; exited = the owning process is gone. */ +export type WindowsListenerIdentity = "ours" | "foreign" | "exited" | "unknown"; + +export interface WindowsStopConfirmDeps { + clock?: WindowsStopClock; + port: number; + /** Time left in the caller's deadline when Part B starts. */ + remainingMs: number; + reserveMs?: number; + canonicalPaths: readonly string[]; + selfPid?: number; + samplePort(port: number, timeoutMs: number): MaybePromise; + identifyListener(port: number, ownerPid: number | null, timeoutMs: number): MaybePromise; + enumerate(timeoutMs: number): MaybePromise; +} + +export type WindowsStopConfirmOutcome = "freed" | "survived" | "port-held-by-foreign" | "unconfirmed" | "unknown"; + +export interface WindowsStopConfirmResult { + outcome: WindowsStopConfirmOutcome; + portHeldByForeign: boolean; + reason: "evidence-unreadable" | null; + detail: string; + port: number; + windowMs: number; + listener: { ownerPid: number | null; identity: WindowsListenerIdentity } | null; + /** Wrapper-image processes of this home found by the final enumeration. */ + lateWrappers: number[]; +} + +const C = WINDOWS_STOP_CAPS; +const SAMPLE_INTERVAL_MS = 250; + +export async function confirmWindowsStopFreed(deps: WindowsStopConfirmDeps): Promise { + const clock = deps.clock ?? windowsStopClock; + const windowMs = windowsPartBWindowMs(deps.remainingMs, deps.reserveMs ?? C.reserve); + const windowEnd = clock.now() + windowMs; + const result: WindowsStopConfirmResult = { + outcome: "unknown", portHeldByForeign: false, reason: null, detail: "", + port: deps.port, windowMs, listener: null, lateWrappers: [], + }; + const finish = (outcome: WindowsStopConfirmOutcome, detail: string): WindowsStopConfirmResult => { + result.outcome = outcome; + result.portHeldByForeign = outcome === "port-held-by-foreign"; + result.reason = outcome === "unknown" ? "evidence-unreadable" : null; + result.detail = detail; + return result; + }; + + let freeSince: number | null = null; + let held = false; + let last: WindowsPortSample = { state: "unknown" }; + for (;;) { + const startedAt = clock.now(); + const sample = await runCapped(clock, C.partBClosingSample, timeoutMs => deps.samplePort(deps.port, timeoutMs)); + last = sample.ok ? sample.value : { state: "unknown" }; + if (last.state === "free") { + freeSince ??= startedAt; + if (startedAt - freeSince >= C.partBHoldFloor) { + held = true; + break; + } + } else { + freeSince = null; + } + const left = windowEnd - clock.now(); + if (left <= 0) break; + await clock.sleep(Math.min(SAMPLE_INTERVAL_MS, left)); + } + + const listening = last.state === "listening" ? last : null; + const [snapshot, identity] = await Promise.all([ + runCapped(clock, C.partBFinalEnumeration, deps.enumerate), + listening + ? runCapped(clock, C.partBFinalEnumeration, + timeoutMs => deps.identifyListener(deps.port, listening.ownerPid, timeoutMs)) + : Promise.resolve(null), + ]); + if (listening) { + result.listener = { + ownerPid: listening.ownerPid, + identity: identity && identity.ok ? identity.value : "unknown", + }; + } + + const processes = snapshot.ok ? snapshot.value : null; + if (processes) { + result.lateWrappers = windowsHomeWrapperPids(processes, deps.canonicalPaths, deps.selfPid ?? process.pid); + } + if (result.lateWrappers.length > 0) return finish("survived", "a wrapper of this home is running after the stop"); + if (result.listener?.identity === "ours") return finish("survived", "this installation answers on the port again"); + if (processes === null) return finish("unknown", "the final process list could not be read"); + const unnamed = processes.some(entry => entry.name === null && entry.pid !== (deps.selfPid ?? process.pid) + && deps.canonicalPaths.some(path => commandLineHasPathToken(entry.commandLine, path))); + if (unnamed) return finish("unknown", "a process carrying this home's paths has an unreadable image name"); + if (held) return finish("freed", "the port stayed free for the hold and no wrapper of this home is running"); + if (result.listener?.identity === "foreign") return finish("port-held-by-foreign", "another program holds the port"); + if (result.listener?.identity === "exited") { + return finish("unconfirmed", "the port is held by a listener whose owner has exited"); + } + if (last.state === "free") return finish("unconfirmed", "the port was not free for the whole hold within the window"); + return finish("unknown", "the port state could not be read"); +} + +/** + * Which port Part B watches: the runtime-port record when present, else the port baked + * into the wrapper script, else the configured port. + */ +export function resolveWindowsStopPort(sources: { + runtimePort: () => number | null; + wrapperPort: () => number | null; + configuredPort: () => number; +}): { port: number; source: "runtime" | "wrapper" | "config" } { + const valid = (value: number | null): value is number => + value !== null && Number.isSafeInteger(value) && value > 0 && value <= 65_535; + const read = (get: () => number | null): number | null => { + try { + return get(); + } catch { + return null; + } + }; + const runtime = read(sources.runtimePort); + if (valid(runtime)) return { port: runtime, source: "runtime" }; + const wrapper = read(sources.wrapperPort); + if (valid(wrapper)) return { port: wrapper, source: "wrapper" }; + return { port: sources.configuredPort(), source: "config" }; +} diff --git a/structure/INDEX.md b/structure/INDEX.md index 13dbfcdec61..f7a9d3f6912 100644 --- a/structure/INDEX.md +++ b/structure/INDEX.md @@ -165,6 +165,7 @@ Background service, docs, release, and design discipline. | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | Durable Desktop ownership read, takeover-consent handling, live-supervision evidence, the Windows probe's trusted system-directory acquisition, and the supervision latch blocking contract consumed by service and update guards. | | [`ops/service-logging.md`](ops/service-logging.md) | Service-only console timestamps and bounded copy-truncate log retention. Producer bounds for combo positions, target providers and effort diagnostics. | | [`ops/service-health-recovery.md`](ops/service-health-recovery.md) | Opt-in sentinel that turns a hung but listening managed service proxy into an exit its manager restarts. | +| [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | Windows process evidence, the wrapper binding rule, and the checked manager stop and port confirmation that are not yet wired to a stop path. | | [`ops/update-supervised-recovery.md`](ops/update-supervised-recovery.md) | Manager-first recovery after a failed post-update service refresh, and the non-zero unsupervised fallback. | | [`ops/plugins.md`](ops/plugins.md) | Plugin loading from OPENCODEX_HOME/plugins and the upstream rewrite slot plugins attach to. | | [`ops/docs-and-release.md`](ops/docs-and-release.md) | Docs site, workflow map, branch policy, release flow, and cross-platform CI. | @@ -529,6 +530,12 @@ A source area can be described by more than one doc, because these docs are orga | `src/service/logging.ts` | [`ops/service-logging.md`](ops/service-logging.md) | | `src/service/orchestration.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/repair.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | +| `src/service/windows-checked-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-process-evidence.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-process-native.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-stop-binding.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-stop-budget.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-stop-confirm.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/sidecar/` | [`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/sidecar/anthropic-binding.ts` | [`providers/anthropic-account-pool.md`](providers/anthropic-account-pool.md) | | `src/stall-timeout.ts` | [`transports/timeouts.md`](transports/timeouts.md)
[`runtime.md`](runtime.md) | diff --git a/structure/manifest.json b/structure/manifest.json index 4e7b9b0b34e..05dd11805cc 100644 --- a/structure/manifest.json +++ b/structure/manifest.json @@ -1303,6 +1303,20 @@ "src/service/health-sentinel-windows.ts" ] }, + { + "path": "ops/windows-checked-stop.md", + "tier": 6, + "title": "Windows Checked Service Stop", + "scope": "Windows process evidence, the wrapper binding rule, and the checked manager stop and port confirmation that are not yet wired to a stop path.", + "documents": [ + "src/service/windows-checked-stop.ts", + "src/service/windows-process-evidence.ts", + "src/service/windows-process-native.ts", + "src/service/windows-stop-binding.ts", + "src/service/windows-stop-budget.ts", + "src/service/windows-stop-confirm.ts" + ] + }, { "path": "ops/update-supervised-recovery.md", "tier": 6, diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md new file mode 100644 index 00000000000..92209f543af --- /dev/null +++ b/structure/ops/windows-checked-stop.md @@ -0,0 +1,88 @@ +# Windows Checked Service Stop + +The checked Windows service stop (#7771) is a library of primitives and two operations that +stop a Task Scheduler or WinSW manager, end the wrappers that would relaunch the proxy, and +then confirm the port is free. **No stop path calls it yet.** `ocx stop`, `ocx service stop`, +`ocx uninstall`, service uninstall and install, `/api/stop` and the updater still use the +best-effort sweep in `src/lib/windows-service-wrappers.ts` and the guarded inspection in +`src/service/windows-guarded-manager.ts`, unchanged. The plan for wiring the callers lives in +`devlog/_plan/261111_windows_service_stop/`. + +## Process evidence + +`src/service/windows-process-native.ts` holds the in-process primitives for Windows x64. It loads +`kernel32.dll` and `advapi32.dll` from `resolveTrustedWindowsSystemDirectory()`, requires `bun:ffi` +lazily, caches a failed load as unavailable, and closes every handle in a `finally`: + +- creation time through `GetProcessTimes`, as epoch milliseconds truncated from the FILETIME; +- a checked terminate: `OpenProcess(PROCESS_TERMINATE | PROCESS_QUERY_LIMITED_INFORMATION | SYNCHRONIZE)`, + the creation time read on that handle and compared with the bound value, `TerminateProcess` on + the same handle. The result is `terminated`, `gone`, `reused` (identity mismatch, untouched), + `denied` or `unavailable`; +- an exit watch: a handle opened at bind time and polled with `WaitForSingleObject(handle, 0)`; +- the Task Scheduler service host PID through `QueryServiceStatusEx` on `Schedule`, only while it runs. + +Windows ARM64, other platforms and any test that simulates a platform get no native API. +`src/service/windows-process-evidence.ts` is the facade the operations use. Its process snapshot is +one capped `Get-CimInstance Win32_Process` query that emits `CreationDate` as an ISO UTC string. On +x64 each creation time is re-read natively and kept only when it agrees with CIM to the +millisecond; a disagreement (a PID reused between the two reads) is unreadable. Elsewhere the +CIM time is the bound value. The Schedule PID falls back to `sc.exe queryex Schedule` from the +trusted system directory with a cap. The ARM64 terminate is one trusted PowerShell script that +re-reads `Get-Process -Id` StartTime, compares it with the bound time within 1 s (UTC), calls +`Kill()` on the same `Process` object and reads the exit back. `Kill()` opens its own handle by PID, so +a window between the in-script check and the kill remains; that window and the 1 s tolerance are +the accepted residuals of the fallback. Only integers reach a script. + +## Binding rule + +`src/service/windows-stop-binding.ts` decides, as a pure function over a snapshot, which processes +may be terminated. A process is bound only when its image is `wscript.exe`, `cscript.exe` or +`cmd.exe`; its command line carries this home's launcher VBS or service script path as a complete +token; it is an ancestor of the identity-checked child, a descendant of a bound wrapper, a child +of the Schedule host (SCM PID), or, only while this home's task is registered, an orphan (a parent +that is missing or was created after it); and its creation time was read. The proxy child is +never a candidate. A wrapper that matches image and path but fails lineage is reported unbound and +left alone. Unreadable evidence about a candidate is reported unreadable: a matching wrapper +without a creation time or image name, and a wrapper image with no readable command line under a +bound wrapper or, while a manager is registered, above the child. Bound wrappers are ordered +deepest first and carry pid, creation time, image and a SHA-256 digest of the command line. + +## Part A and Part B + +`src/service/windows-checked-stop.ts` is Part A, `stopWindowsServiceChecked(deps, options)`. Every +external effect is injected, and every step runs under a cap that races a timer and rechecks the +elapsed time on the injected clock. It probes registration (2 s each). An unreadable answer is +`unknown` with nothing changed. With nothing registered it enumerates once for a stray wrapper of a +deleted task; an unreadable table or no wrapper is `absent`. Admission then compares the caller's +deadline with the worst case still to come and returns `unverified-budget` without changing +anything when it does not fit. It binds, ends the task (`schtasks /end`, 2 s) or stops WinSW (a +2 s stop command and an SCM poll capped at `` + 2 s), terminates bound survivors +deepest first, waits on the bind-time watches (terminates and wait share 3 s), and reads the +manager state once (1 s). Denied stops every further step. A cap overrun is `unknown`. A wrapper or +manager still running is `survived` (`unbound-wrapper` when the survivor was never bound). WinSW +is `survived` while the proxy is still alive after the service reports stopped. The authorizer +variant `guardedStopWindowsServiceChecked` runs its authorizer at entry and before each terminate and +reports a refusal as `WindowsCheckedStopRefusedError` with what had already happened. + +`src/service/windows-stop-confirm.ts` is Part B, `confirmWindowsStopFreed(deps)`, run after the caller's +graceful child stop. The port must stay free for a 6 s hold, measured between sample starts, +inside a window of `min(12 s, remaining - 0.5 s - 5 s - reserve)` with a 6 s floor. A final +enumeration (5 s, sharing its cap with the listener identification) must show no wrapper image +with this home's path token. An unreadable table is not success. A listener is ours +(`survived`), foreign (`portHeldByForeign`) or left by an exited owner, which is not foreign. +`resolveWindowsStopPort` picks the runtime-port record, then the wrapper's baked port, then the +configured port. + +`src/service/windows-stop-budget.ts` holds the caps and the admission arithmetic. With default +settings the minimum is 48.5 s for a guarded Task Scheduler stop, 56.5 s for a snapshotless one +and 42.5 s for WinSW. Each includes the 0.5 s closing port sample that Part B can take past its +window. + +## Tests + +`tests/windows/windows-stop-binding.test.ts`, `tests/windows/windows-checked-stop.test.ts`, +`tests/windows/windows-stop-confirm.test.ts` and `tests/windows/windows-process-evidence.test.ts` are +hermetic: injected deps, a virtual clock and simulated platforms, and no real `schtasks`, +PowerShell or FFI call. The FFI primitive itself is exercised only by the cases gated on real +Windows x64. diff --git a/structure/runtime.md b/structure/runtime.md index 0a047af9a84..4c3f52164c9 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -59,7 +59,7 @@ The [CLI management contract](cli-management.md#cli-readiness-diagnostics) owns `ocx resolve` (`src/cli/resolve.ts`) is the machine surface a desktop shell asks instead of resolving the config home, the port, and liveness itself: the home comes from `src/config/paths.ts`, the effective port is the live listener's when the identity-checked `findLiveProxy` answers and the configured `config.port ?? 10100` otherwise, and the liveness verdict is that same module's output (pid, runtime-versus-config provenance, version, role). Config reads go through `readConfigDiagnostics`, not `loadConfig`: a missing file is defaults, but an invalid file exits 1 instead of being repaired to defaults, because a defaulted port is a guess the caller must refuse. Liveness is three-valued: when `findLiveProxy` returns null, resolve re-asks the endpoints with `endpointsToProve` and `everyEndpointProvenDown`, using the in-process `probeEndpointLiveness` from `src/server/proxy-liveness.ts`. Only unanimous definitive "dead" becomes `absent-proven`; a timeout or reset stays unknown, exits 1 and never authorises a start. The transient exclusive-bind fallback in `src/update/proxy-liveness-probe.mjs` belongs only to the npm/Bun updater. Resolve stays in-process because a compiled standalone binary cannot run a JavaScript probe with `execPath -e`. Discovery borrows `START_OWNERSHIP_LIVENESS`, the start path's ownership budget — the verdict feeds the shell's launch decision, so the cost of a false "nobody listening" is the duplicate proxy (#5004). The verb is in `skipsCodexShimAutoRestore`, so a read-only lookup never triggers a shim repair. Arguments are pre-parsed in `src/cli/root.ts` and exit 64 before any preflight side effect, the same ordering `ocx ready` obeys. `ocx-resolve/1` also reports `ownership` (`none`, `owned`, or `unknown`) and `takeover` (`supported` with protocol version, minimum CLI version and compatibility token, or `blocked` with a reason). A known liveness verdict still exits 0 when ownership is unknown, but takeover is blocked. An unreadable second service-state read blocks takeover instead of becoming an absent registration. `src/service/managing-cli.ts` observes the registered and selected PATH CLIs, including Windows PATHEXT order and file validation, before compatibility is offered. A `--version` probe is retried once on Windows — one transient failure must not read as an unobservable CLI — and a Windows `.CMD`/`.BAT` shim is invoked through a single verbatim-quoted `cmd /c` line, so a spaced executable path and spaced arguments both survive cmd's re-parse. `src/cli/route-policy.ts` distinguishes a missing profile (exit 4) from malformed `show` arguments (exit 2); an upstream failure propagates through the management error mapping. -`ocx stop --json` is a reporting layer over the unchanged stop path. `src/cli/index.ts` threads a `StopRunRecord` through the existing receipt, drain, respawn-verification and restore flow, and `src/cli/stop-report.ts` maps the recorded facts plus the signals that already pick the exit code into one versioned document (`schema: "ocx-stop/1"`). With `--json` the human lines print on stderr and stdout carries only that document; exit codes 0/1/79/80/81 cross the process boundary unchanged. The desktop's opt-in guarded stop adds exact approved PID, endpoint, home, CLI version and compatibility-token expectations; plain `ocx stop` retains its ordinary behavior. Under the ownership mutation lease, `src/cli/stop-approval.ts` re-resolves them and `src/service/guarded-manager-target.ts` binds any installed manager's process tree to the approved PID before a stop action. The same guard re-reads `ocx.pid` and `runtime-port.json` through `src/config/tracked-runtime-inspection.ts`, a single-pass read-only probe that walks the same checks `readPid()` and `readRuntimePort(pid)` perform and reports which one failed, answering the recorded process's start-identity question through the same cache `readPid()` fills so it cannot disagree with the admission it diagnoses. A pre-action mismatch returns `approval-changed` without stopping, and the summary carries `detail` naming the single guard fact that failed — live PID/port/hostname/CLI drift, an unverifiable ownership claim, a blocked takeover, the compatibility fingerprint, or the tracked-record cause — an absent, denied, unreadable or invalid record, a dead, foreign or unverifiable recorded process, or a record naming a different PID — so a transient re-probe can be told from genuine drift in the caller's log. The tracked-record causes are fixed strings: denied is reported only on an observed EACCES/EPERM, an unreadable command line stays unverifiable rather than attributed to an account or elevation, and no probed path, command line, stderr or token reaches the detail. Immediately before any manager stop or direct PID signal, the CLI rechecks the same manager identity or proven absence; a change also returns `approval-changed`. A non-OpenCodex process replacing the OS job in the remaining instant before the manager command is a residual same-user risk outside the ownership lease. The guarded path stops the bound manager, or the approved PID directly when managers are proven absent, then uses a five-second shared deadline to verify PID exit, port availability and definitive endpoint absence. Only after settlement does a read-only manager probe require launchd `not-loaded` in both domains or systemd `inactive` with `MainPID=0`; the state is checked again before a success summary. Timeout, active or unreadable manager state returns terminal `manager-still-active`, even if the endpoint briefly refuses. Windows managers bind through `src/service/windows-guarded-manager.ts`: a Task Scheduler backend must prove a recognized OpenCodex registration against the bounded current-user SID/name lookup, a running task, and a wrapper ancestor carrying this home's canonical paths — presence and running state are read root-folder-scoped because every `/tn` operation resolves a bare name there and nowhere else, so a same-named task inside a task folder is a different registration neither the probe nor the stop can reach; the registered definition itself is read only on the running path, and a state or registration read that cannot answer re-probes presence so a task deleted between probes — which can respawn nothing — reads absent while a merely unreadable one stays unknown; an unreadable current identity leaves the manager unknown; a WinSW backend must prove this installation's binary and SCM service PID ancestry; either marks `childNeedsSeparateStop`, so the approved PID receives its own graceful signal only after a successful manager stop because `schtasks /end` does not reliably cascade to the wrapper's child (#764); a failed or unknown manager stop leaves the child untouched. Post-stop Windows inactivity means the task not running, no surviving wrapper process, and no started WinSW — never mere registration presence; a running-state read that cannot answer re-probes presence the same way, so a task deleted between probes reports inactive while a persisting one stays unknown. Every Windows manager command a verdict or stop depends on — schtasks, sc.exe, WinSW verbs — runs under a bounded `execFileSync` timeout so a wedged Task Scheduler or Service Control Manager cannot hang the guarded stop; a killed command classifies through the same fail-closed paths a nonzero exit would (the interactive `install /p` password prompt is the one deliberate exception). A `wscript.exe`, `cscript.exe` or `cmd.exe` whose command line CIM cannot read makes the state unprovable only when it is tied to the proxy: pre-stop, as an ancestor of the approved PID; post-stop, as the former bound manager process or a descendant of it. Unrelated unreadable shells, which a non-admin query returns for other users' and elevated processes, are not evidence either way. An unprovable Windows manager still blocks guarded takeover. The token checks snapshot consistency, not whether a person approved the desktop prompt. `src/cli/stop-restore.ts` owns the shared Codex/Grok restoration and its history-only, history-deferred, and other failure categories. +`ocx stop --json` is a reporting layer over the unchanged stop path. `src/cli/index.ts` threads a `StopRunRecord` through the existing receipt, drain, respawn-verification and restore flow, and `src/cli/stop-report.ts` maps the recorded facts plus the signals that already pick the exit code into one versioned document (`schema: "ocx-stop/1"`). With `--json` the human lines print on stderr and stdout carries only that document; exit codes 0/1/79/80/81 cross the process boundary unchanged. The desktop's opt-in guarded stop adds exact approved PID, endpoint, home, CLI version and compatibility-token expectations; plain `ocx stop` retains its ordinary behavior. Under the ownership mutation lease, `src/cli/stop-approval.ts` re-resolves them and `src/service/guarded-manager-target.ts` binds any installed manager's process tree to the approved PID before a stop action. The same guard re-reads `ocx.pid` and `runtime-port.json` through `src/config/tracked-runtime-inspection.ts`, a single-pass read-only probe that walks the same checks `readPid()` and `readRuntimePort(pid)` perform and reports which one failed, answering the recorded process's start-identity question through the same cache `readPid()` fills so it cannot disagree with the admission it diagnoses. A pre-action mismatch returns `approval-changed` without stopping, and the summary carries `detail` naming the single guard fact that failed — live PID/port/hostname/CLI drift, an unverifiable ownership claim, a blocked takeover, the compatibility fingerprint, or the tracked-record cause — an absent, denied, unreadable or invalid record, a dead, foreign or unverifiable recorded process, or a record naming a different PID — so a transient re-probe can be told from genuine drift in the caller's log. The tracked-record causes are fixed strings: denied is reported only on an observed EACCES/EPERM, an unreadable command line stays unverifiable rather than attributed to an account or elevation, and no probed path, command line, stderr or token reaches the detail. Immediately before any manager stop or direct PID signal, the CLI rechecks the same manager identity or proven absence; a change also returns `approval-changed`. A non-OpenCodex process replacing the OS job in the remaining instant before the manager command is a residual same-user risk outside the ownership lease. The guarded path stops the bound manager, or the approved PID directly when managers are proven absent, then uses a five-second shared deadline to verify PID exit, port availability and definitive endpoint absence. Only after settlement does a read-only manager probe require launchd `not-loaded` in both domains or systemd `inactive` with `MainPID=0`; the state is checked again before a success summary. Timeout, active or unreadable manager state returns terminal `manager-still-active`, even if the endpoint briefly refuses. Windows managers bind through `src/service/windows-guarded-manager.ts`: a Task Scheduler backend must prove a recognized OpenCodex registration against the bounded current-user SID/name lookup, a running task, and a wrapper ancestor carrying this home's canonical paths — presence and running state are read root-folder-scoped because every `/tn` operation resolves a bare name there and nowhere else, so a same-named task inside a task folder is a different registration neither the probe nor the stop can reach; the registered definition itself is read only on the running path, and a state or registration read that cannot answer re-probes presence so a task deleted between probes — which can respawn nothing — reads absent while a merely unreadable one stays unknown; an unreadable current identity leaves the manager unknown; a WinSW backend must prove this installation's binary and SCM service PID ancestry; either marks `childNeedsSeparateStop`, so the approved PID receives its own graceful signal only after a successful manager stop because `schtasks /end` does not reliably cascade to the wrapper's child (#764); a failed or unknown manager stop leaves the child untouched. Post-stop Windows inactivity means the task not running, no surviving wrapper process, and no started WinSW — never mere registration presence; a running-state read that cannot answer re-probes presence the same way, so a task deleted between probes reports inactive while a persisting one stays unknown. Every Windows manager command a verdict or stop depends on — schtasks, sc.exe, WinSW verbs — runs under a bounded `execFileSync` timeout so a wedged Task Scheduler or Service Control Manager cannot hang the guarded stop; a killed command classifies through the same fail-closed paths a nonzero exit would (the interactive `install /p` password prompt is the one deliberate exception). A `wscript.exe`, `cscript.exe` or `cmd.exe` whose command line CIM cannot read makes the state unprovable only when it is tied to the proxy: pre-stop, as an ancestor of the approved PID; post-stop, as the former bound manager process or a descendant of it. Unrelated unreadable shells, which a non-admin query returns for other users' and elevated processes, are not evidence either way. An unprovable Windows manager still blocks guarded takeover. The token checks snapshot consistency, not whether a person approved the desktop prompt. `src/cli/stop-restore.ts` owns the shared Codex/Grok restoration and its history-only, history-deferred, and other failure categories. The checked Windows stop that will replace the best-effort wrapper sweep and bind a stray wrapper by lineage and creation time is described in [Windows checked service stop](ops/windows-checked-stop.md); no stop path calls it yet. `ocx service status --json` (`src/service/status-json.ts`) is the passive machine form of the service report, read by peers such as GOAT's device inventory. `src/service/cli.ts` accepts `--json` for `status` only (`planServiceCommand` refuses it on every other verb before any installation probe) and answers it before `platformOps()` resolves, so no install, start, stop, repair or migration path is reachable; `skipsCodexShimAutoRestore` exempts `service status --json` so the CLI preflight repairs no Codex shim first, the collection runs inside `runPassiveRead` so the serving probe's config load neither hardens modes nor backs up a malformed file, and `--json` on any other verb is refused before the Desktop check and the Windows mutation lock; an unsupported platform or Docker gets a document instead of the text path's exit. The document reuses exactly the reads the text report makes — `diagnoseService()` (launchd `print`, systemd observation verbs, Task Scheduler `/query`, SCM/WinSW status) and, only when a registration exists, the 1.5-second loopback `confirmServiceServing` probe — plus the ownership-lease and Desktop login lines. `running` is the manager's verdict (the value `startup.serviceRunning` carries in `ocx status --json`); `serving` reports separately whether a proxy answers on the installed port. `backend` is `launchd`, `systemd`, `schtasks`, `winsw`, or `none` when nothing is registered. The systemd branch of `diagnoseService` derives through the pure `deriveSystemdServiceDiagnostic`, matching the launchd and Windows derivations. `tests/service/service-status-json.test.ts` snapshots each platform branch and records every subprocess entry point on darwin, linux and win32 against a read-only allowlist. `ocx status --json` carries an additive `install` verdict (`src/cli/status-install.ts`) mapped from the updater's own `detectInstallOwnership()`: `method` is `bun`, `npm`, `pnpm`, `mise`, `source`, `standalone` or `unknown` (detector failure), and `evidence.basis` names the package layout, the mise backend record (tool alias and backend only) or the standalone binary without printing a path (`tests/cli/status-install-method.test.ts`). `ocx service restart` reads the pid of the proxy answering on the installed port before it restarts (`servingRuntimePid`, `src/service/health.ts`) and passes it to the serving check as `replacing`. While that pid is set, only a proxy reporting a different pid counts as serving: a Windows wrapper that survived `schtasks /end` keeps its old child answering healthz, and the restart used to report "restarted and serving" over it (#7771). If the replaced pid is still the one answering at the deadline, the command prints that the previous runtime is still serving and the replacement was not confirmed, prints no checkmark, and exits 1; a restart that threw names either the surviving pid (kept, not replaced) or the new pid. An answer that reports no pid cannot be shown to be the old runtime and counts as serving. `ocx service repair` keeps the plain any-proxy check, because a repair may preserve a healthy runtime by design. `tests/service/service-restart-replacement.test.ts` covers each case. diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 8b3b5f6cc4d..e63f42563bd 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1528,8 +1528,8 @@ "windows-deploy-close-regressions.test.ts": "windows", "windows-elevation-spawn.test.ts": "windows", "windows-elevation.test.ts": "windows", - "windows-guarded-manager.test.ts": "windows", - "windows-popup-fix.test.ts": "windows", + "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", + "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", "windows-process-priority.test.ts": "windows", "windows-scheduler-errors.test.ts": "windows", "windows-scheduler-install-verification.test.ts": "windows", "windows-lifecycle-root-acl.test.ts": "windows", diff --git a/tests/helpers/windows-checked-stop-fixtures.ts b/tests/helpers/windows-checked-stop-fixtures.ts new file mode 100644 index 00000000000..a78a248ccc6 --- /dev/null +++ b/tests/helpers/windows-checked-stop-fixtures.ts @@ -0,0 +1,56 @@ +/** + * Shared fixtures for the checked Windows stop tests: a supervised process tree and a + * virtual clock. The clock advances only when a sleep is not aborted, so a capped step + * that answers in time costs no virtual time and a hung one costs exactly its cap. + */ +import type { WindowsStopProcessEntry } from "../../src/service/windows-process-evidence"; +import type { WindowsStopClock } from "../../src/service/windows-checked-stop"; + +export const HOME = "C:\\Users\\me\\.opencodex"; +export const VBS = HOME + "\\opencodex-service-launcher.vbs"; +export const CMD = HOME + "\\opencodex-service.cmd"; +export const OTHER_HOME_VBS = "C:\\Users\\other\\.opencodex\\opencodex-service-launcher.vbs"; +export const PATHS = [VBS, CMD] as const; +export const SCHEDULE_PID = 900; + +export const entry = ( + pid: number, + parentPid: number | null, + name: string | null, + commandLine: string | null, + startTime: number | null, +): WindowsStopProcessEntry => ({ pid, parentPid, name, commandLine, startTime }); + +export const scheduleHost = entry(SCHEDULE_PID, 600, "svchost.exe", "C:\\Windows\\system32\\svchost.exe -k netsvcs -s Schedule", 1_000); +export const launcher = (pid = 20, parent = SCHEDULE_PID, start: number | null = 5_000) => + entry(pid, parent, "wscript.exe", 'C:\\Windows\\System32\\wscript.exe //B //NoLogo "' + VBS + '"', start); +export const loop = (pid = 30, parent = 20, start: number | null = 5_100) => + entry(pid, parent, "cmd.exe", 'C:\\Windows\\System32\\cmd.exe /c "' + CMD + '"', start); +export const proxy = (pid = 42, parent = 30, start = 5_200) => + entry(pid, parent, "bun.exe", "C:\\bun\\bun.exe C:\\pkg\\src\\cli\\index.ts start --port 10100", start); + +/** svchost(Schedule) 900 -> wscript 20 (.vbs) -> cmd 30 (.cmd) -> proxy 42. */ +export const supervisedTree = (): WindowsStopProcessEntry[] => [scheduleHost, launcher(), loop(), proxy()]; + +export interface VirtualClock { + clock: WindowsStopClock; + advance(ms: number): void; + readonly now: number; +} + +export function virtualClock(start = 0): VirtualClock { + let now = start; + return { + clock: { + now: () => now, + sleep: (ms, signal) => new Promise(resolve => { + setTimeout(() => { + if (!signal?.aborted) now += Math.max(0, ms); + resolve(); + }, 0); + }), + }, + advance(ms) { now += ms; }, + get now() { return now; }, + }; +} diff --git a/tests/windows/windows-checked-stop.test.ts b/tests/windows/windows-checked-stop.test.ts new file mode 100644 index 00000000000..f59cb8c844f --- /dev/null +++ b/tests/windows/windows-checked-stop.test.ts @@ -0,0 +1,333 @@ +// Part A of the checked Windows stop (#7771, 030 "Steps" and "Time budget"). Hermetic: +// every probe, command, watch and terminate is injected; nothing reaches the machine. +import { describe, expect, test } from "bun:test"; + +import type { WinswStatus } from "../../src/lib/winsw"; +import { + guardedStopWindowsServiceChecked, + stopWindowsServiceChecked, + WindowsCheckedStopRefusedError, + type WindowsCheckedStopDeps, + type WindowsWinswScmState, +} from "../../src/service/windows-checked-stop"; +import type { WindowsProcessWatch, WindowsTerminateResult } from "../../src/service/windows-process-native"; +import type { WindowsStopProcessEntry } from "../../src/service/windows-process-evidence"; +import { + DEFAULT_WINSW_STOP_TIMEOUT_MS, + windowsPartBWindowMs, + windowsStopWorstCase, +} from "../../src/service/windows-stop-budget"; +import { + PATHS, SCHEDULE_PID, entry, launcher, loop, scheduleHost, supervisedTree, virtualClock, +} from "../helpers/windows-checked-stop-fixtures"; + +const never = (): Promise => new Promise(() => undefined); + +interface Harness { + deps: WindowsCheckedStopDeps; + calls: string[]; + alive: Set; + clock: ReturnType; +} + +function harness(overrides: Partial = {}, snapshot: WindowsStopProcessEntry[] = supervisedTree()): Harness { + const calls: string[] = []; + const alive = new Set(snapshot.map(process => process.pid)); + const clock = virtualClock(1_000); + const watch = (pid: number): WindowsProcessWatch => alive.has(pid) + ? { status: "open", exited: () => !alive.has(pid), close: () => calls.push("close:" + pid) } + : { status: "gone" }; + const deps: WindowsCheckedStopDeps = { + clock: clock.clock, + canonicalPaths: PATHS, + childPid: 42, + selfPid: 1, + probeScheduler: () => { calls.push("probe-scheduler"); return { status: "present" }; }, + probeWinsw: () => { calls.push("probe-winsw"); return "nonexistent"; }, + enumerate: () => { calls.push("enumerate"); return snapshot; }, + scheduleServicePid: () => SCHEDULE_PID, + endTask: () => { calls.push("end"); return "ended"; }, + taskState: () => "not-running", + winswStop: () => { calls.push("winsw-stop"); return "sent"; }, + winswState: () => "stopped", + proxyAlive: () => false, + openWatch: pid => watch(pid), + terminate: (pid): WindowsTerminateResult => { + calls.push("terminate:" + pid); + alive.delete(pid); + return { status: "terminated" }; + }, + ...overrides, + }; + return { deps, calls, alive, clock }; +} + +const mutations = (calls: string[]) => calls.filter(call => call === "end" || call === "winsw-stop" || call.startsWith("terminate:")); + +describe("checked Windows stop, part A", () => { + test("/end leaves the loop alive: the wrappers are terminated deepest first and the child is left alone", async () => { + const h = harness(); + const result = await stopWindowsServiceChecked(h.deps); + expect(result.outcome).toBe("stopped"); + expect(result.reason).toBeNull(); + expect(mutations(h.calls)).toEqual(["end", "terminate:30", "terminate:20"]); + expect(h.calls).not.toContain("terminate:42"); + expect(h.alive.has(42)).toBe(true); + expect(result.bound.map(wrapper => [wrapper.pid, wrapper.startTime])).toEqual([[30, 5_100], [20, 5_000]]); + expect(result.terminated).toEqual([30, 20]); + expect(result.changed).toBe(true); + // Every bind-time handle is closed. + expect(h.calls.filter(call => call.startsWith("close:")).sort()).toEqual(["close:20", "close:30"]); + }); + + test("a PID reused between bind and terminate is left untouched", async () => { + const h = harness(); + h.deps.terminate = pid => { + h.calls.push("terminate:" + pid); + if (pid === 30) return { status: "reused" }; + h.alive.delete(pid); + return { status: "terminated" }; + }; + // The bound cmd.exe is gone; PID 30 now names a different process, still running. + h.deps.openWatch = (pid): WindowsProcessWatch => ({ + status: "open", exited: () => pid === 30 ? false : !h.alive.has(pid), close: () => undefined, + }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result.terminated).toEqual([20]); + expect(h.alive.has(30)).toBe(true); + expect(result.outcome).toBe("stopped"); + }); + + test("access denied on /end is denied and nothing is terminated", async () => { + const h = harness({ endTask: () => "denied" }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "denied", reason: "access-denied" }); + expect(h.calls.filter(call => call.startsWith("terminate:"))).toEqual([]); + }); + + test("access denied on a terminate stops every further kill", async () => { + const h = harness(); + h.deps.terminate = pid => { h.calls.push("terminate:" + pid); return { status: "denied" }; }; + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "denied", reason: "access-denied" }); + expect(h.calls.filter(call => call.startsWith("terminate:"))).toEqual(["terminate:30"]); + }); + + test("an unreadable candidate under the bound manager is unknown and nothing changes", async () => { + const hidden = entry(35, 20, "cmd.exe", null, 5_300); + const h = harness({}, [...supervisedTree(), hidden]); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: false }); + expect(mutations(h.calls)).toEqual([]); + }); + + test("an unreadable creation time on a wrapper is unknown and nothing changes", async () => { + const h = harness({}, [scheduleHost, launcher(), loop(30, 20, null), entry(42, 30, "bun.exe", "bun", 5_200)]); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: false }); + expect(mutations(h.calls)).toEqual([]); + }); + + test("an unreadable process list under a registered task is unknown, never absent", async () => { + const h = harness({ enumerate: () => null }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: false }); + expect(mutations(h.calls)).toEqual([]); + }); + + test("a stray wrapper with the task not running is bound through the Schedule host and ended", async () => { + const h = harness({ childPid: null }, [scheduleHost, launcher(), loop()]); + const result = await stopWindowsServiceChecked(h.deps); + expect(result.outcome).toBe("stopped"); + expect(mutations(h.calls)).toEqual(["end", "terminate:30", "terminate:20"]); + }); + + test("an orphan is ended while the task is registered and left alone once it is not", async () => { + const orphans = [launcher(20, 777), loop()]; + const registered = harness({ childPid: null }, orphans); + expect((await stopWindowsServiceChecked(registered.deps)).outcome).toBe("stopped"); + expect(registered.calls).toContain("terminate:20"); + + const deleted = harness({ childPid: null, probeScheduler: () => ({ status: "absent" }) }, orphans); + const result = await stopWindowsServiceChecked(deleted.deps); + expect(result).toMatchObject({ outcome: "survived", reason: "unbound-wrapper", changed: false }); + expect(result.unbound.sort()).toEqual([20, 30]); + expect(mutations(deleted.calls)).toEqual([]); + }); + + test("a hand-started launcher that fails lineage is not terminated: survived, unbound-wrapper", async () => { + const shell = entry(500, 4, "explorer.exe", "explorer.exe", 2_000); + const tree = [...supervisedTree(), shell, launcher(60, 500, 7_000), loop(61, 60, 7_100)]; + const h = harness({}, tree); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "survived", reason: "unbound-wrapper" }); + expect(h.calls).not.toContain("terminate:60"); + expect(h.calls).not.toContain("terminate:61"); + expect(result.unbound.sort()).toEqual([60, 61]); + }); + + test("a bound wrapper that outlives its terminate is survived", async () => { + const h = harness(); + h.deps.terminate = pid => { h.calls.push("terminate:" + pid); return { status: "terminated" }; }; + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "survived", reason: null }); + // The wait is capped: 3 s of virtual time, then the verdict. + expect(h.clock.now - 1_000).toBeLessThanOrEqual(3_000 + 100); + }); + + test("a task still running after the stop is survived", async () => { + const h = harness({ taskState: () => "running" }); + expect((await stopWindowsServiceChecked(h.deps)).outcome).toBe("survived"); + }); + + test("WinSW: a non-waiting stop, then survived when the proxy is still alive", async () => { + const h = harness({ + probeScheduler: () => ({ status: "absent" }), + probeWinsw: () => "started" as WinswStatus, + proxyAlive: () => true, + }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result.outcome).toBe("survived"); + expect(mutations(h.calls)).toEqual(["winsw-stop"]); + expect(h.calls).not.toContain("enumerate"); + }); + + test("WinSW: an SCM state that never reports stopped is unknown after + 2 s", async () => { + const states: WindowsWinswScmState[] = []; + const h = harness({ + probeScheduler: () => ({ status: "absent" }), + probeWinsw: () => "started" as WinswStatus, + winswState: () => { states.push("pending"); return "pending"; }, + }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: true }); + const elapsed = h.clock.now - 1_000; + expect(elapsed).toBeGreaterThanOrEqual(DEFAULT_WINSW_STOP_TIMEOUT_MS + 2_000); + expect(elapsed).toBeLessThanOrEqual(DEFAULT_WINSW_STOP_TIMEOUT_MS + 2_000 + 250); + expect(states.length).toBeGreaterThan(1); + }); + + test("a registration probe that never answers is unknown and changes nothing", async () => { + const h = harness({ probeScheduler: () => never() }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: false }); + expect(result.registration.scheduler).toBe("unknown"); + expect(mutations(h.calls)).toEqual([]); + }); + + test("an unreadable schtasks answer stays unknown and never becomes absent", async () => { + const h = harness({ probeScheduler: () => ({ status: "unknown", detail: "query failed" }) }); + expect((await stopWindowsServiceChecked(h.deps)).outcome).toBe("unknown"); + }); + + test("/end that overruns its cap is unknown and nothing is terminated", async () => { + const h = harness({ endTask: () => { h.calls.push("end"); return never(); } }); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", changed: true }); + expect(h.calls.filter(call => call.startsWith("terminate:"))).toEqual([]); + }); + + test("a dependency that blocks past its cap is an overrun even though it answered", async () => { + const h = harness(); + h.deps.endTask = () => { h.clock.advance(2_500); return "ended"; }; + const result = await stopWindowsServiceChecked(h.deps); + expect(result.outcome).toBe("unknown"); + expect(h.calls.filter(call => call.startsWith("terminate:"))).toEqual([]); + }); + + test("admission refuses before anything changes when the worst case does not fit", async () => { + const budget = { orphanDiscovery: false }; + const guardedMinimum = windowsStopWorstCase({ + managers: { scheduler: true, winsw: false, stray: false }, enumerationReused: true, orphanDiscovery: false, + }).totalMs; + const refused = harness(); + const result = await stopWindowsServiceChecked(refused.deps, { + bindSnapshot: supervisedTree(), + admission: { deadlineAt: refused.clock.now + guardedMinimum - 1, budget }, + }); + expect(result).toMatchObject({ outcome: "unverified-budget", reason: "unverified-budget", changed: false }); + expect(result.budget).toEqual({ requiredMs: guardedMinimum, remainingMs: guardedMinimum - 1 }); + expect(mutations(refused.calls)).toEqual([]); + expect(refused.calls).not.toContain("enumerate"); + + const admitted = harness(); + const ok = await stopWindowsServiceChecked(admitted.deps, { + bindSnapshot: supervisedTree(), + admission: { deadlineAt: admitted.clock.now + guardedMinimum, budget }, + }); + expect(ok.outcome).toBe("stopped"); + expect(admitted.calls).not.toContain("enumerate"); + }); + + test("nothing registered is absent and is never refused for budget", async () => { + const nothing = harness({ probeScheduler: () => ({ status: "absent" }) }, [scheduleHost, entry(42, 10, "bun.exe", "bun", 5_200)]); + const result = await stopWindowsServiceChecked(nothing.deps, { + admission: { deadlineAt: 0, budget: { orphanDiscovery: true } }, + }); + expect(result).toMatchObject({ outcome: "absent", changed: false }); + + const unreadable = harness({ probeScheduler: () => ({ status: "absent" }), enumerate: () => null }); + expect((await stopWindowsServiceChecked(unreadable.deps)).outcome).toBe("absent"); + }); + + test("a stray wrapper of a deleted task is bound through the Schedule host and swept", async () => { + const h = harness({ childPid: null, probeScheduler: () => ({ status: "absent" }) }, [scheduleHost, launcher(), loop()]); + const result = await stopWindowsServiceChecked(h.deps); + expect(result.outcome).toBe("stopped"); + expect(mutations(h.calls)).toEqual(["terminate:30", "terminate:20"]); + }); + + test("the updater's authorizer refuses at entry before any probe", async () => { + const h = harness(); + const refusal = new Error("supervision blocked"); + const error = await guardedStopWindowsServiceChecked(() => { throw refusal; }, h.deps).catch(caught => caught); + expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); + expect((error as WindowsCheckedStopRefusedError).cause).toBe(refusal); + expect((error as WindowsCheckedStopRefusedError).partial.changed).toBe(false); + expect(h.calls).toEqual([]); + }); + + test("the updater's authorizer runs again before each terminate and its refusal ends the stop", async () => { + const h = harness(); + let checks = 0; + const error = await guardedStopWindowsServiceChecked(() => { + checks += 1; + if (checks === 3) throw new Error("respawned runtime"); + }, h.deps).catch(caught => caught); + expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); + expect((error as WindowsCheckedStopRefusedError).partial.terminated).toEqual([30]); + expect(h.calls.filter(call => call.startsWith("terminate:"))).toEqual(["terminate:30"]); + }); +}); + +describe("checked Windows stop budget", () => { + test("admission minimums with default settings", () => { + // 030's 48 / 56 / 42 s plus the 0.5 s closing sample part B can take past its window. + expect(windowsStopWorstCase({ + managers: { scheduler: true, winsw: false, stray: false }, enumerationReused: true, orphanDiscovery: false, + }).totalMs).toBe(48_500); + expect(windowsStopWorstCase({ + managers: { scheduler: true, winsw: false, stray: false }, enumerationReused: false, orphanDiscovery: true, + }).totalMs).toBe(56_500); + expect(windowsStopWorstCase({ + managers: { scheduler: false, winsw: true, stray: false }, enumerationReused: false, orphanDiscovery: true, + }).totalMs).toBe(42_500); + }); + + test("receipts, a longer shutdown and a longer WinSW stop timeout grow the worst case", () => { + const base = { managers: { scheduler: true, winsw: false, stray: false }, enumerationReused: true, orphanDiscovery: false }; + expect(windowsStopWorstCase({ ...base, inheritedReceipts: 2 }).totalMs).toBe(48_500 + 6_000); + expect(windowsStopWorstCase({ ...base, shutdownTimeoutMs: 10_000 }).totalMs).toBe(48_500 + 5_000); + expect(windowsStopWorstCase({ + managers: { scheduler: false, winsw: true, stray: false }, enumerationReused: false, orphanDiscovery: false, + winswStopTimeoutMs: 30_000, + }).rows.find(row => row.step === "part-a")!.ms).toBe(35_000); + expect(windowsStopWorstCase({ ...base, partB: false }).rows.map(row => row.step)).not.toContain("part-b"); + }); + + test("part B's window is min(12 s, remaining minus its tail and the reserve), never under 6 s", () => { + expect(windowsPartBWindowMs(60_000)).toBe(12_000); + expect(windowsPartBWindowMs(20_000)).toBe(11_500); + expect(windowsPartBWindowMs(10_000)).toBe(6_000); + }); +}); diff --git a/tests/windows/windows-process-evidence.test.ts b/tests/windows/windows-process-evidence.test.ts new file mode 100644 index 00000000000..c7425bf204f --- /dev/null +++ b/tests/windows/windows-process-evidence.test.ts @@ -0,0 +1,214 @@ +// Process evidence for the checked Windows stop (#7771): creation times, the Schedule +// service PID, the checked terminate and the exit watch. Hermetic everywhere except the +// cases gated on real Windows x64, which exercise the FFI primitive on a child of this test. +import { afterEach, describe, expect, test } from "bun:test"; + +import { + decodeServiceStatusProcess, + fileTimeToEpochMs, + setWindowsProcessNativeForTests, + setWindowsProcessPlatformForTests, + windowsProcessNative, + type WindowsProcessNativeApi, +} from "../../src/service/windows-process-native"; +import { + WINDOWS_PROCESS_SNAPSHOT_SCRIPT, + openWindowsProcessWatch, + overlayNativeCreationTimes, + parseFallbackTerminateOutput, + parseScQueryexPid, + parseWindowsProcessSnapshot, + readScheduleServicePid, + terminateWindowsProcessBound, + windowsFallbackTerminateScript, + windowsProcessSnapshot, + type WindowsEvidenceCommand, +} from "../../src/service/windows-process-evidence"; + +afterEach(() => { + setWindowsProcessNativeForTests(undefined); + setWindowsProcessPlatformForTests(null); +}); + +const fakeNative = (overrides: Partial = {}): WindowsProcessNativeApi => ({ + creationTime: () => null, + terminate: () => ({ status: "unavailable" }), + openWatch: () => ({ status: "unavailable" }), + scheduleServicePid: () => null, + ...overrides, +}); + +const SNAPSHOT_JSON = JSON.stringify([ + { ProcessId: 20, ParentProcessId: 900, Name: "wscript.exe", CommandLine: "wscript.exe x.vbs", CreationDate: "2026-11-12T01:02:03.1234567Z" }, + { ProcessId: 4, ParentProcessId: 0, Name: "System", CommandLine: null, CreationDate: null }, +]); + +describe("Windows process evidence: pure decoding", () => { + test("FILETIME converts to epoch ms by truncation; zero is unreadable", () => { + expect(fileTimeToEpochMs(116_444_736_000_000_000n + 12_345_678n)).toBe(1_234); + expect(fileTimeToEpochMs(0n)).toBeNull(); + expect(fileTimeToEpochMs(116_444_736_000_000_000n)).toBeNull(); + }); + + test("SERVICE_STATUS_PROCESS yields the PID only while the service runs", () => { + const buffer = new Uint8Array(36); + const view = new DataView(buffer.buffer); + view.setUint32(4, 4, true); + view.setUint32(28, 1_234, true); + expect(decodeServiceStatusProcess(buffer)).toBe(1_234); + view.setUint32(4, 1, true); + expect(decodeServiceStatusProcess(buffer)).toBeNull(); + expect(decodeServiceStatusProcess(new Uint8Array(20))).toBeNull(); + }); + + test("the snapshot carries ISO creation times as epoch ms; malformed output is unread", () => { + expect(parseWindowsProcessSnapshot(SNAPSHOT_JSON)).toEqual([ + { pid: 20, parentPid: 900, name: "wscript.exe", commandLine: "wscript.exe x.vbs", startTime: Date.parse("2026-11-12T01:02:03.123Z") }, + { pid: 4, parentPid: null, name: "System", commandLine: null, startTime: null }, + ]); + expect(parseWindowsProcessSnapshot(JSON.stringify({ ProcessId: 7, ParentProcessId: 1, Name: "a", CommandLine: "a", CreationDate: "x" }))) + .toEqual([{ pid: 7, parentPid: 1, name: "a", commandLine: "a", startTime: null }]); + expect(parseWindowsProcessSnapshot("")).toEqual([]); + expect(parseWindowsProcessSnapshot("{not json")).toBeNull(); + }); + + test("a native creation time replaces CIM's only when both agree to the millisecond", () => { + const rows = parseWindowsProcessSnapshot(SNAPSHOT_JSON)!; + const cim = rows[0]!.startTime!; + expect(overlayNativeCreationTimes(rows, () => cim + 1)[0]!.startTime).toBe(cim + 1); + // A PID reused between the CIM query and the native read: the identities disagree. + expect(overlayNativeCreationTimes(rows, () => cim + 2_000)[0]!.startTime).toBeNull(); + expect(overlayNativeCreationTimes(rows, () => null)[0]!.startTime).toBeNull(); + expect(overlayNativeCreationTimes(rows, () => { throw new Error("ffi"); })[0]!.startTime).toBeNull(); + }); + + test("sc.exe queryex yields the PID only for a RUNNING service", () => { + const running = "SERVICE_NAME: Schedule\r\n TYPE : 30 WIN32\r\n STATE : 4 RUNNING\r\n" + + " PID : 1488\r\n FLAGS :\r\n"; + expect(parseScQueryexPid(running)).toBe(1_488); + expect(parseScQueryexPid(running.replace(": 4 RUNNING", ": 1 STOPPED"))).toBeNull(); + expect(parseScQueryexPid(running.replace(": 1488", ": 0"))).toBeNull(); + expect(parseScQueryexPid("garbage")).toBeNull(); + }); + + test("the ARM64 terminate script compares StartTime and kills the same Process object", () => { + const script = windowsFallbackTerminateScript(4_242, 1_794_445_323_123, 1_000); + expect(script).toContain("Get-Process -Id 4242"); + expect(script).toContain("FromUnixTimeMilliseconds(1794445323123)"); + expect(script).toContain("-gt 1000) { 'reused'"); + expect(script.indexOf("$p.Kill()")).toBeGreaterThan(script.indexOf("'reused'")); + expect(script).toContain("$p.WaitForExit(1000)"); + expect(() => windowsFallbackTerminateScript(1.5, 1, 1)).toThrow(); + expect(() => windowsFallbackTerminateScript(-1, 1, 1)).toThrow(); + expect(parseFallbackTerminateOutput("terminated-exited\r\n")).toEqual({ status: "terminated", exited: true }); + expect(parseFallbackTerminateOutput("terminated")).toEqual({ status: "terminated", exited: false }); + expect(parseFallbackTerminateOutput("noise\nreused\n")).toEqual({ status: "reused" }); + expect(parseFallbackTerminateOutput("denied")).toEqual({ status: "denied" }); + expect(parseFallbackTerminateOutput("gone")).toEqual({ status: "gone" }); + expect(parseFallbackTerminateOutput("")).toEqual({ status: "unavailable" }); + expect(parseFallbackTerminateOutput("Killed")).toEqual({ status: "unavailable" }); + }); +}); + +describe("Windows process evidence: facades", () => { + test("a simulated platform reaches neither the kernel nor a spawn", async () => { + setWindowsProcessPlatformForTests("win32"); + expect(windowsProcessNative()).toBeNull(); + expect(await windowsProcessSnapshot({ timeoutMs: 5_000 })).toBeNull(); + expect(await readScheduleServicePid({ timeoutMs: 2_000 })).toBeNull(); + expect(await terminateWindowsProcessBound(20, 5_000, { timeoutMs: 1_000 })).toEqual({ status: "unavailable" }); + expect(openWindowsProcessWatch(20, 5_000)).toEqual({ status: "unavailable" }); + }); + + test("the snapshot runs one capped CIM query and overlays native creation times", async () => { + setWindowsProcessPlatformForTests("win32"); + const cim = Date.parse("2026-11-12T01:02:03.123Z"); + setWindowsProcessNativeForTests(fakeNative({ creationTime: pid => (pid === 20 ? cim : null) })); + const commands: WindowsEvidenceCommand[] = []; + const snapshot = await windowsProcessSnapshot({ + timeoutMs: 5_000, + run: async command => { commands.push(command); return SNAPSHOT_JSON; }, + }); + expect(commands).toHaveLength(1); + expect(commands[0]).toMatchObject({ program: "powershell", timeoutMs: 5_000 }); + expect(commands[0]!.args.at(-1)).toBe(WINDOWS_PROCESS_SNAPSHOT_SCRIPT); + expect(snapshot?.map(row => [row.pid, row.startTime])).toEqual([[20, cim], [4, null]]); + expect(await windowsProcessSnapshot({ timeoutMs: 5_000, run: async () => { throw new Error("timeout"); } })).toBeNull(); + }); + + test("the Schedule PID prefers the SCM read and falls back to sc.exe queryex", async () => { + setWindowsProcessPlatformForTests("win32"); + setWindowsProcessNativeForTests(fakeNative({ scheduleServicePid: () => 1_234 })); + expect(await readScheduleServicePid({ timeoutMs: 2_000 })).toBe(1_234); + setWindowsProcessNativeForTests(null); + const commands: WindowsEvidenceCommand[] = []; + const pid = await readScheduleServicePid({ + timeoutMs: 2_000, + run: async command => { commands.push(command); return "STATE : 4 RUNNING\nPID : 77\n"; }, + }); + expect(pid).toBe(77); + expect(commands).toEqual([{ program: "sc", args: ["queryex", "Schedule"], timeoutMs: 2_000, maxBufferBytes: 64 * 1024 }]); + expect(await readScheduleServicePid({ timeoutMs: 2_000, run: async () => { throw new Error("1060"); } })).toBeNull(); + }); + + test("terminate prefers the handle primitive and otherwise runs the bounded fallback script", async () => { + setWindowsProcessPlatformForTests("win32"); + const seen: Array<[number, number]> = []; + setWindowsProcessNativeForTests(fakeNative({ terminate: (pid, start) => { seen.push([pid, start]); return { status: "reused" }; } })); + expect(await terminateWindowsProcessBound(20, 5_000, { timeoutMs: 3_000 })).toEqual({ status: "reused" }); + expect(seen).toEqual([[20, 5_000]]); + + setWindowsProcessNativeForTests(null); + const scripts: string[] = []; + const result = await terminateWindowsProcessBound(20, 5_000, { + timeoutMs: 3_000, + run: async command => { scripts.push(command.args.at(-1)!); return "terminated-exited\n"; }, + }); + expect(result).toEqual({ status: "terminated", exited: true }); + expect(scripts[0]).toContain("Get-Process -Id 20"); + expect(await terminateWindowsProcessBound(20, 5_000, { timeoutMs: 3_000, run: async () => { throw new Error("x"); } })) + .toEqual({ status: "unavailable" }); + }); +}); + +const liveX64 = process.platform !== "win32" || process.arch !== "x64"; + +describe("Windows process evidence: live x64 primitive", () => { + test.skipIf(liveX64)("reads a child's creation time, refuses a mismatched identity and ends the bound one", async () => { + const child = Bun.spawn([process.execPath, "-e", "setInterval(() => {}, 1000)"], { + stdio: ["ignore", "ignore", "ignore"], windowsHide: true, + }); + let created: number | null = null; + try { + const native = windowsProcessNative(); + expect(native).not.toBeNull(); + created = native!.creationTime(child.pid); + expect(typeof created).toBe("number"); + expect(Math.abs(created! - Date.now())).toBeLessThan(60_000); + + expect(native!.terminate(child.pid, created! - 5_000)).toEqual({ status: "reused" }); + const watch = native!.openWatch(child.pid, created!); + expect(watch.status).toBe("open"); + if (watch.status !== "open") return; + try { + expect(watch.exited()).toBe(false); + expect(native!.terminate(child.pid, created!)).toEqual({ status: "terminated" }); + await child.exited; + expect(watch.exited()).toBe(true); + } finally { + watch.close(); + } + } finally { + child.kill(); + await child.exited; + } + // Exited: a zombie still held open reads as exited, a released PID fails to open + // (ERROR_INVALID_PARAMETER), and a PID already reused by another process is refused. + expect(["gone", "reused"]).toContain(windowsProcessNative()!.terminate(child.pid, created!).status); + }); + + test.skipIf(liveX64)("reads the Task Scheduler service PID from the SCM", () => { + const pid = windowsProcessNative()!.scheduleServicePid(); + expect(pid === null || (Number.isSafeInteger(pid) && pid > 0)).toBe(true); + }); +}); diff --git a/tests/windows/windows-stop-binding.test.ts b/tests/windows/windows-stop-binding.test.ts new file mode 100644 index 00000000000..0f391fe22d2 --- /dev/null +++ b/tests/windows/windows-stop-binding.test.ts @@ -0,0 +1,155 @@ +// Binding rule of the checked Windows stop (#7771, 030 "Binding rule"). +import { describe, expect, test } from "bun:test"; +import { createHash } from "node:crypto"; + +import { + bindWindowsStopWrappers, + windowsHomeWrapperPids, + windowsWrapperImage, + type WindowsStopBindingContext, +} from "../../src/service/windows-stop-binding"; +import { + CMD, OTHER_HOME_VBS, PATHS, SCHEDULE_PID, VBS, + entry, launcher, loop, proxy, scheduleHost, supervisedTree, +} from "../helpers/windows-checked-stop-fixtures"; + +const context = (overrides: Partial = {}): WindowsStopBindingContext => ({ + processes: supervisedTree(), + canonicalPaths: PATHS, + childPid: 42, + taskRegistered: true, + managerRegistered: true, + scheduleServicePid: SCHEDULE_PID, + selfPid: 1, + ...overrides, +}); + +describe("checked Windows stop binding", () => { + test("binds the launcher and the looping cmd.exe above the child, deepest first", () => { + const binding = bindWindowsStopWrappers(context()); + expect(binding.bound.map(wrapper => [wrapper.pid, wrapper.image, wrapper.startTime])).toEqual([ + [30, "cmd.exe", 5_100], + [20, "wscript.exe", 5_000], + ]); + expect(binding.unbound).toEqual([]); + expect(binding.unreadable).toEqual([]); + expect(binding.child).toEqual({ pid: 42, startTime: 5_200 }); + // The receipt keeps a digest; the command line itself never leaves the snapshot. + expect(binding.bound[0]!.commandLineDigest) + .toBe(createHash("sha256").update(loop().commandLine!, "utf8").digest("hex")); + expect(JSON.stringify(binding)).not.toContain("opencodex-service"); + }); + + test("the proxy child is never a candidate, whatever its image or command line", () => { + const child = entry(42, 30, "cmd.exe", 'cmd.exe /c "' + CMD + '"', 5_200); + const binding = bindWindowsStopWrappers(context({ processes: [scheduleHost, launcher(), loop(), child] })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + }); + + test("the path token under a non-wrapper image is neither bound nor reported", () => { + const editor = entry(70, 500, "notepad.exe", 'notepad.exe "' + VBS + '"', 6_000); + const binding = bindWindowsStopWrappers(context({ processes: [...supervisedTree(), editor] })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + expect(binding.unbound).toEqual([]); + }); + + test("another home's wrapper and a substring-only match are not this home's wrappers", () => { + const other = entry(80, SCHEDULE_PID, "wscript.exe", 'wscript.exe //B "' + OTHER_HOME_VBS + '"', 6_000); + const suffixed = entry(81, SCHEDULE_PID, "cmd.exe", 'cmd.exe /c "' + CMD + '.bak"', 6_000); + const prefixed = entry(82, SCHEDULE_PID, "cmd.exe", "cmd.exe /c X" + CMD, 6_000); + const binding = bindWindowsStopWrappers(context({ processes: [...supervisedTree(), other, suffixed, prefixed] })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + expect(binding.unbound).toEqual([]); + }); + + test("a stray launcher of a stopped task binds through the Schedule host, with its loop", () => { + // The task is no longer running and the proxy is gone, but the wrapper relaunches it. + const binding = bindWindowsStopWrappers(context({ + processes: [scheduleHost, launcher(), loop()], childPid: null, + })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + }); + + test("an unreadable Schedule host PID disables that clause", () => { + const binding = bindWindowsStopWrappers(context({ + processes: [scheduleHost, launcher(), loop()], childPid: null, scheduleServicePid: null, + })); + expect(binding.bound).toEqual([]); + expect(binding.unbound.map(wrapper => wrapper.pid).sort()).toEqual([20, 30]); + }); + + test("an orphan binds only while this home's task is registered", () => { + const orphanTree = [launcher(20, 777), loop()]; + const registered = bindWindowsStopWrappers(context({ processes: orphanTree, childPid: null })); + expect(registered.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + const unregistered = bindWindowsStopWrappers(context({ + processes: orphanTree, childPid: null, taskRegistered: false, managerRegistered: false, + })); + expect(unregistered.bound).toEqual([]); + expect(unregistered.unbound.map(wrapper => wrapper.pid).sort()).toEqual([20, 30]); + }); + + test("a parent PID created after the wrapper is a reused PID, so the parent is gone", () => { + const reusedParent = entry(777, 4, "explorer.exe", "explorer.exe", 9_000); + const binding = bindWindowsStopWrappers(context({ + processes: [reusedParent, launcher(20, 777), loop()], childPid: null, + })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + }); + + test("a launcher started by hand from a live shell fails lineage, and so does its loop", () => { + const shell = entry(500, 4, "explorer.exe", "explorer.exe", 2_000); + const binding = bindWindowsStopWrappers(context({ + processes: [scheduleHost, shell, launcher(20, 500), loop()], childPid: null, + })); + expect(binding.bound).toEqual([]); + expect(binding.unbound.map(wrapper => wrapper.pid).sort()).toEqual([20, 30]); + }); + + test("a matching wrapper without a creation time is unreadable, never bound", () => { + const binding = bindWindowsStopWrappers(context({ + processes: [scheduleHost, launcher(), loop(30, 20, null), proxy()], + })); + expect(binding.unreadable).toEqual([30]); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([20]); + }); + + test("a matching process whose image name is unreadable is unreadable", () => { + const unnamed = entry(31, 20, null, 'cmd.exe /c "' + CMD + '"', 5_150); + const binding = bindWindowsStopWrappers(context({ processes: [...supervisedTree(), unnamed] })); + expect(binding.unreadable).toEqual([31]); + }); + + test("an unreadable wrapper image under a bound wrapper is unreadable", () => { + const hidden = entry(35, 20, "cmd.exe", null, 5_300); + const binding = bindWindowsStopWrappers(context({ processes: [...supervisedTree(), hidden] })); + expect(binding.unreadable).toEqual([35]); + }); + + test("an unreadable ancestor of the child counts only while a manager is registered", () => { + const tree = [entry(10, 4, "cmd.exe", null, 1_500), proxy(42, 10)]; + expect(bindWindowsStopWrappers(context({ processes: tree })).unreadable).toEqual([10]); + expect(bindWindowsStopWrappers(context({ + processes: tree, taskRegistered: false, managerRegistered: false, + })).unreadable).toEqual([]); + }); + + test("an unrelated unreadable shell is not evidence", () => { + const unrelated = entry(60, 4, "cmd.exe", null, 1_500); + expect(bindWindowsStopWrappers(context({ processes: [...supervisedTree(), unrelated] })).unreadable).toEqual([]); + }); + + test("the stopping process is never a candidate", () => { + const binding = bindWindowsStopWrappers(context({ selfPid: 30 })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([20]); + }); + + test("wrapper images and the final-enumeration filter", () => { + expect(windowsWrapperImage("WScript.EXE")).toBe("wscript.exe"); + expect(windowsWrapperImage("C:\\Windows\\System32\\cscript.exe")).toBe("cscript.exe"); + expect(windowsWrapperImage("bun.exe")).toBeNull(); + expect(windowsWrapperImage(null)).toBeNull(); + const editor = entry(70, 500, "notepad.exe", 'notepad.exe "' + VBS + '"', 6_000); + expect(windowsHomeWrapperPids([...supervisedTree(), editor], PATHS, 1)).toEqual([20, 30]); + }); +}); diff --git a/tests/windows/windows-stop-confirm.test.ts b/tests/windows/windows-stop-confirm.test.ts new file mode 100644 index 00000000000..061dda51d33 --- /dev/null +++ b/tests/windows/windows-stop-confirm.test.ts @@ -0,0 +1,108 @@ +// Part B of the checked Windows stop (#7771, 030 "Part B"): the port stays free for the +// hold, the final enumeration finds no wrapper of this home, and a listener is classified. +import { describe, expect, test } from "bun:test"; + +import type { WindowsStopProcessEntry } from "../../src/service/windows-process-evidence"; +import { + confirmWindowsStopFreed, + resolveWindowsStopPort, + type WindowsListenerIdentity, + type WindowsPortSample, + type WindowsStopConfirmDeps, +} from "../../src/service/windows-stop-confirm"; +import { PATHS, entry, launcher, scheduleHost, virtualClock } from "../helpers/windows-checked-stop-fixtures"; + +const quiet = (): WindowsStopProcessEntry[] => [scheduleHost, entry(70, 4, "explorer.exe", "explorer.exe", 1_000)]; + +function confirmDeps(options: { + sample?: (elapsed: number) => WindowsPortSample; + identity?: WindowsListenerIdentity; + processes?: WindowsStopProcessEntry[] | null; + remainingMs?: number; +} = {}) { + const clock = virtualClock(10_000); + const calls: string[] = []; + const deps: WindowsStopConfirmDeps = { + clock: clock.clock, + port: 10_100, + remainingMs: options.remainingMs ?? 30_000, + canonicalPaths: PATHS, + selfPid: 1, + samplePort: () => (options.sample ?? (() => ({ state: "free" })))(clock.now - 10_000), + identifyListener: (_port, ownerPid) => { calls.push("identify:" + ownerPid); return options.identity ?? "unknown"; }, + enumerate: () => { calls.push("enumerate@" + (clock.now - 10_000)); return options.processes === undefined ? quiet() : options.processes; }, + }; + return { deps, clock, calls }; +} + +describe("checked Windows stop, part B", () => { + test("a port that stays free for the hold with no wrapper left is freed", async () => { + const { deps, clock, calls } = confirmDeps(); + const result = await confirmWindowsStopFreed(deps); + expect(result).toMatchObject({ outcome: "freed", portHeldByForeign: false, reason: null, port: 10_100 }); + // The hold is 6 s between sample starts; the final enumeration runs after it. + expect(calls).toEqual(["enumerate@6000"]); + expect(clock.now - 10_000).toBeLessThanOrEqual(result.windowMs); + }); + + test("a wrapper relaunched after bind is caught by the final enumeration even with the port free", async () => { + const { deps } = confirmDeps({ processes: [...quiet(), launcher(90, 4, 12_000)] }); + const result = await confirmWindowsStopFreed(deps); + expect(result).toMatchObject({ outcome: "survived", lateWrappers: [90] }); + }); + + test("this installation answering on the port again is survived", async () => { + const { deps, calls } = confirmDeps({ + sample: elapsed => (elapsed < 3_000 ? { state: "free" } : { state: "listening", ownerPid: 4_242 }), + identity: "ours", + }); + const result = await confirmWindowsStopFreed(deps); + expect(result).toMatchObject({ outcome: "survived", listener: { ownerPid: 4_242, identity: "ours" } }); + expect(calls).toContain("identify:4242"); + }); + + test("a foreign listener after our processes are gone is portHeldByForeign", async () => { + const { deps } = confirmDeps({ sample: () => ({ state: "listening", ownerPid: 777 }), identity: "foreign" }); + const result = await confirmWindowsStopFreed(deps); + expect(result).toMatchObject({ outcome: "port-held-by-foreign", portHeldByForeign: true }); + }); + + test("a listener whose owner has exited is not foreign", async () => { + const { deps } = confirmDeps({ sample: () => ({ state: "listening", ownerPid: 555 }), identity: "exited" }); + const result = await confirmWindowsStopFreed(deps); + expect(result).toMatchObject({ outcome: "unconfirmed", portHeldByForeign: false, listener: { identity: "exited" } }); + }); + + test("an unreadable final enumeration is not success, even with the port free", async () => { + const { deps } = confirmDeps({ processes: null }); + expect(await confirmWindowsStopFreed(deps)).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable" }); + }); + + test("a final enumeration that overruns its cap is not success", async () => { + const { deps } = confirmDeps(); + deps.enumerate = () => new Promise(() => undefined); + expect((await confirmWindowsStopFreed(deps)).outcome).toBe("unknown"); + }); + + test("a slow port release reports failure instead of overrunning the window", async () => { + const { deps, clock } = confirmDeps({ + remainingMs: 14_500, + sample: elapsed => (elapsed < 2_000 ? { state: "listening", ownerPid: 4_242 } : { state: "free" }), + identity: "ours", + }); + const result = await confirmWindowsStopFreed(deps); + expect(result.windowMs).toBe(6_000); + expect(result.outcome).toBe("unconfirmed"); + expect(clock.now - 10_000).toBeLessThanOrEqual(6_000 + 500 + 5_000); + }); + + test("the port comes from the runtime record, then the wrapper script, then the config", () => { + expect(resolveWindowsStopPort({ runtimePort: () => 10_200, wrapperPort: () => 10_300, configuredPort: () => 10_100 })) + .toEqual({ port: 10_200, source: "runtime" }); + expect(resolveWindowsStopPort({ runtimePort: () => null, wrapperPort: () => 10_300, configuredPort: () => 10_100 })) + .toEqual({ port: 10_300, source: "wrapper" }); + expect(resolveWindowsStopPort({ + runtimePort: () => { throw new Error("unreadable"); }, wrapperPort: () => 0, configuredPort: () => 10_100, + })).toEqual({ port: 10_100, source: "config" }); + }); +}); From 727915223a7d1938ff7585acf8999791bd26cd0c Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 01:29:15 +0900 Subject: [PATCH 02/53] fix(service): floor fractional timeouts in the terminate fallback A cap derived from the deadline (windowEnd - now) is fractional on the real clock. execFile and windowsFallbackTerminateScript take whole milliseconds, so the spawned fallback answered unavailable. Floor the timeout and the derived exit wait, and spawn nothing when it floors to 0, which execFile would read as no timeout. Refs #7771 --- src/service/windows-process-evidence.ts | 8 ++++++-- tests/windows/windows-process-evidence.test.ts | 14 ++++++++++++++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/src/service/windows-process-evidence.ts b/src/service/windows-process-evidence.ts index a70ce60185d..e5c51ec3df1 100644 --- a/src/service/windows-process-evidence.ts +++ b/src/service/windows-process-evidence.ts @@ -263,12 +263,16 @@ export async function terminateWindowsProcessBound( } const run = runnerFor(options); if (!run) return { status: "unavailable" }; - const exitWaitMs = Math.max(0, Math.min(FALLBACK_TERMINATE_EXIT_WAIT_MS, options.timeoutMs - 500)); + // A timeout derived from a deadline can be fractional, and both execFile and the script + // take whole milliseconds. A timeout that floors to 0 would mean no timeout at all. + const timeoutMs = Math.floor(options.timeoutMs); + if (!(timeoutMs > 0)) return { status: "unavailable" }; + const exitWaitMs = Math.max(0, Math.min(FALLBACK_TERMINATE_EXIT_WAIT_MS, timeoutMs - 500)); try { return parseFallbackTerminateOutput(await run({ program: "powershell", args: powershellArgs(windowsFallbackTerminateScript(pid, boundStartTime, exitWaitMs)), - timeoutMs: options.timeoutMs, + timeoutMs, maxBufferBytes: SMALL_MAX_BUFFER, })); } catch { diff --git a/tests/windows/windows-process-evidence.test.ts b/tests/windows/windows-process-evidence.test.ts index c7425bf204f..82430294723 100644 --- a/tests/windows/windows-process-evidence.test.ts +++ b/tests/windows/windows-process-evidence.test.ts @@ -169,6 +169,20 @@ describe("Windows process evidence: facades", () => { expect(await terminateWindowsProcessBound(20, 5_000, { timeoutMs: 3_000, run: async () => { throw new Error("x"); } })) .toEqual({ status: "unavailable" }); }); + + test("a fractional deadline-derived timeout reaches the fallback as whole milliseconds", async () => { + setWindowsProcessPlatformForTests("win32"); + setWindowsProcessNativeForTests(null); + const commands: WindowsEvidenceCommand[] = []; + const run = async (command: WindowsEvidenceCommand) => { commands.push(command); return "terminated\n"; }; + expect(await terminateWindowsProcessBound(20, 5_000, { timeoutMs: 1_250.5, run })) + .toEqual({ status: "terminated", exited: false }); + expect(commands.map(command => command.timeoutMs)).toEqual([1_250]); + expect(commands[0]!.args.at(-1)).toContain("$p.WaitForExit(750)"); + // Under 1 ms floors to 0, which execFile reads as "no timeout": nothing is spawned. + expect(await terminateWindowsProcessBound(20, 5_000, { timeoutMs: 0.5, run })).toEqual({ status: "unavailable" }); + expect(commands).toHaveLength(1); + }); }); const liveX64 = process.platform !== "win32" || process.arch !== "x64"; From b5d9910d02cd9a33601a6b7b9418db3dab747b21 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 01:29:15 +0900 Subject: [PATCH 03/53] test(service): pin each fail-closed rule of the checked Windows stop One test per fail-closed branch, each shown to fail when its rule is mutated: an unreadable parent PID never binds and, with the task registered, is unreadable; the child-ancestor lineage clause binds on its own; a terminate overrun, an unreadable task or WinSW state, an unreadable stray candidate and a denied bind-time watch end Part A as unknown or denied; an unnamed image carrying this home's path is unknown in Part B. Refs #7771 --- tests/windows/windows-checked-stop.test.ts | 57 ++++++++++++++++++++++ tests/windows/windows-stop-binding.test.ts | 30 ++++++++++++ tests/windows/windows-stop-confirm.test.ts | 14 +++++- 3 files changed, 100 insertions(+), 1 deletion(-) diff --git a/tests/windows/windows-checked-stop.test.ts b/tests/windows/windows-checked-stop.test.ts index f59cb8c844f..b9df43642a7 100644 --- a/tests/windows/windows-checked-stop.test.ts +++ b/tests/windows/windows-checked-stop.test.ts @@ -135,6 +135,56 @@ describe("checked Windows stop, part A", () => { expect(mutations(h.calls)).toEqual([]); }); + test("a wrapper with an unreadable parent PID under a registered task is unknown, never unbound", async () => { + const h = harness({ childPid: null }, [scheduleHost, { ...launcher(), parentPid: null }, loop()]); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: false }); + expect(mutations(h.calls)).toEqual([]); + }); + + test("a denied exit watch at bind is denied before the task is ended", async () => { + const h = harness(); + const open = h.deps.openWatch; + h.deps.openWatch = (pid, start) => (pid === 30 ? { status: "denied" } : open(pid, start)); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "denied", reason: "access-denied", changed: false }); + expect(mutations(h.calls)).toEqual([]); + }); + + test("a terminate that overruns its cap ends part A as unknown with no further terminate", async () => { + const h = harness(); + h.deps.terminate = pid => { h.calls.push("terminate:" + pid); return never(); }; + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: true }); + expect(mutations(h.calls)).toEqual(["end", "terminate:30"]); + }); + + test("an unreadable task state after the stop is unknown, never stopped", async () => { + const unreadable = harness({ taskState: () => "unknown" }); + expect(await stopWindowsServiceChecked(unreadable.deps)) + .toMatchObject({ outcome: "unknown", reason: "evidence-unreadable" }); + const failed = harness({ taskState: () => { throw new Error("schtasks failed"); } }); + expect(await stopWindowsServiceChecked(failed.deps)) + .toMatchObject({ outcome: "unknown", reason: "evidence-unreadable" }); + }); + + test("an unreadable WinSW state after the stop is unknown, never stopped", async () => { + const finalState = (last: () => WindowsWinswScmState | Promise) => { + let reads = 0; + // The SCM poll sees stopped; the one manager-state read after the wait cannot tell. + return () => (reads++ === 0 ? "stopped" : last()); + }; + for (const last of [() => "unknown" as const, () => never()]) { + const h = harness({ + probeScheduler: () => ({ status: "absent" }), + probeWinsw: () => "started" as WinswStatus, + winswState: finalState(last), + }); + expect(await stopWindowsServiceChecked(h.deps)) + .toMatchObject({ outcome: "unknown", reason: "evidence-unreadable" }); + } + }); + test("a stray wrapper with the task not running is bound through the Schedule host and ended", async () => { const h = harness({ childPid: null }, [scheduleHost, launcher(), loop()]); const result = await stopWindowsServiceChecked(h.deps); @@ -277,6 +327,13 @@ describe("checked Windows stop, part A", () => { expect(mutations(h.calls)).toEqual(["terminate:30", "terminate:20"]); }); + test("with nothing registered, an unreadable stray candidate is unknown, never absent", async () => { + const h = harness({ childPid: null, probeScheduler: () => ({ status: "absent" }) }, [scheduleHost, launcher(20, SCHEDULE_PID, null)]); + const result = await stopWindowsServiceChecked(h.deps); + expect(result).toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", changed: false }); + expect(mutations(h.calls)).toEqual([]); + }); + test("the updater's authorizer refuses at entry before any probe", async () => { const h = harness(); const refusal = new Error("supervision blocked"); diff --git a/tests/windows/windows-stop-binding.test.ts b/tests/windows/windows-stop-binding.test.ts index 0f391fe22d2..387a5b53571 100644 --- a/tests/windows/windows-stop-binding.test.ts +++ b/tests/windows/windows-stop-binding.test.ts @@ -78,6 +78,36 @@ describe("checked Windows stop binding", () => { expect(binding.unbound.map(wrapper => wrapper.pid).sort()).toEqual([20, 30]); }); + test("an ancestor of the identity-checked child binds on that clause alone", () => { + // The launcher's parent is a live, older shell, so neither the Schedule host clause nor + // "parent gone" applies; only 030's first lineage clause can bind it. + const shell = entry(500, 4, "explorer.exe", "explorer.exe", 2_000); + const binding = bindWindowsStopWrappers(context({ + processes: [scheduleHost, shell, launcher(20, 500), loop(), proxy()], + taskRegistered: false, + })); + expect(binding.bound.map(wrapper => wrapper.pid)).toEqual([30, 20]); + expect(binding.unbound).toEqual([]); + }); + + test("a wrapper whose parent PID is unreadable never binds, registered or not", () => { + const tree = [scheduleHost, { ...launcher(), parentPid: null }, loop()]; + expect(bindWindowsStopWrappers(context({ processes: tree, childPid: null })).bound).toEqual([]); + const unregistered = bindWindowsStopWrappers(context({ + processes: tree, childPid: null, taskRegistered: false, managerRegistered: false, + })); + expect(unregistered.bound).toEqual([]); + expect(unregistered.unbound.map(wrapper => wrapper.pid).sort()).toEqual([20, 30]); + }); + + test("with the task registered, a wrapper whose parent PID is unreadable is unreadable, not unbound", () => { + const binding = bindWindowsStopWrappers(context({ + processes: [scheduleHost, { ...launcher(), parentPid: null }, loop()], childPid: null, + })); + expect(binding.unreadable).toEqual([20]); + expect(binding.unbound.map(wrapper => wrapper.pid)).not.toContain(20); + }); + test("an orphan binds only while this home's task is registered", () => { const orphanTree = [launcher(20, 777), loop()]; const registered = bindWindowsStopWrappers(context({ processes: orphanTree, childPid: null })); diff --git a/tests/windows/windows-stop-confirm.test.ts b/tests/windows/windows-stop-confirm.test.ts index 061dda51d33..7b7416ba441 100644 --- a/tests/windows/windows-stop-confirm.test.ts +++ b/tests/windows/windows-stop-confirm.test.ts @@ -10,7 +10,7 @@ import { type WindowsPortSample, type WindowsStopConfirmDeps, } from "../../src/service/windows-stop-confirm"; -import { PATHS, entry, launcher, scheduleHost, virtualClock } from "../helpers/windows-checked-stop-fixtures"; +import { CMD, PATHS, entry, launcher, scheduleHost, virtualClock } from "../helpers/windows-checked-stop-fixtures"; const quiet = (): WindowsStopProcessEntry[] => [scheduleHost, entry(70, 4, "explorer.exe", "explorer.exe", 1_000)]; @@ -84,6 +84,18 @@ describe("checked Windows stop, part B", () => { expect((await confirmWindowsStopFreed(deps)).outcome).toBe("unknown"); }); + test("a process with this home's path and an unreadable image name is unknown, never freed or foreign", async () => { + const unnamed = entry(91, 4, null, 'cmd.exe /c "' + CMD + '"', 12_000); + const free = confirmDeps({ processes: [...quiet(), unnamed] }); + expect(await confirmWindowsStopFreed(free.deps)) + .toMatchObject({ outcome: "unknown", reason: "evidence-unreadable", lateWrappers: [] }); + const held = confirmDeps({ + processes: [...quiet(), unnamed], sample: () => ({ state: "listening", ownerPid: 777 }), identity: "foreign", + }); + expect(await confirmWindowsStopFreed(held.deps)) + .toMatchObject({ outcome: "unknown", portHeldByForeign: false, reason: "evidence-unreadable" }); + }); + test("a slow port release reports failure instead of overrunning the window", async () => { const { deps, clock } = confirmDeps({ remainingMs: 14_500, From 029ff62a89ece98f1d957f2b006679fb7b1ee4d0 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 01:29:15 +0900 Subject: [PATCH 04/53] docs(service): record the ARM64 decision and the uncapped steps ARM64 ships the strict variant in slice 2 and its adapter must gate it, because terminateWindowsProcessBound falls back to PowerShell on its own. Record the Schedule PID read before admission, that openWatch and the authorizer are not capped, and the route's need for an outer race. Refs #7771 --- .../060_implementation.md | 12 +++++++- src/service/windows-checked-stop.ts | 1 + src/service/windows-stop-budget.ts | 5 ++-- structure/ops/windows-checked-stop.md | 28 ++++++++++++++----- 4 files changed, 36 insertions(+), 10 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 9bd281b5bd4..85e153ad61d 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -43,10 +43,20 @@ Contract doc: `structure/ops/windows-checked-stop.md`. - **Unbound wrappers with nothing registered.** Binding finds no bound wrapper but an unbound one is alive. The result is `survived` / `unbound-wrapper`, not `absent`, matching 040's bullet; the 030 "absent" definition reads as "nothing bound" alone. Nothing is changed either way. - **Unreadable candidates with nothing registered.** An unreadable wrapper image above the child counts only while a manager is registered. A hand-started proxy under an elevated shell would otherwise lose the stop it gets today. A matching wrapper without a creation time stays `unknown` even with nothing registered. - **Snapshot type.** The snapshot type is `WindowsStopProcessEntry`, which is `WindowsProcessEntry` plus `startTime`, in the evidence module. `windowsProcessList` and the guarded inspection stay untouched until PR2 switches them. +- **The Schedule PID is read before admission.** Part A reads the Task Scheduler host PID from the SCM (cap 2 s) right after the registration probe, before admission, whenever a task is registered or a stray sweep runs. 030's tables do not list this read. Admission measures the time left on the caller's deadline, so the read's actual elapsed time is already taken out of what admission sees, the same way the step 1 stray enumeration is. +- **Not every effect is capped.** The registration probes, enumeration, Schedule PID, `/end`, WinSW stop and poll, terminates, manager state and `proxyAlive` run under caps. `openWatch` and the authorizer do not. `openWatch` is one `OpenProcess` per wrapper with no wait in it. The authorizer runs before each terminate, inside the shared 3 s terminate-and-wait window, so a slow authorizer eats that window: the next terminate gets less time, and an authorizer that never returns hangs Part A. PR2's authorizer (the updater's `mutationGate` check) has to be bounded by its caller. +- **Fractional timeouts.** A cap derived from the deadline (`windowEnd - now`) is fractional on the real clock. The spawned terminate fallback floors it to whole milliseconds before building the script and calling `execFile`, and spawns nothing when it floors to 0, because `execFile` reads a 0 timeout as "no timeout". + +## ARM64 decision for PR2 + +050 requires an ARM64 device run before the bounded fallback ships; without one, ARM64 ships the strict variant (report `unknown`, kill nothing). No ARM64 device run exists, so PR2 ships the strict variant. PR1 still contains the fallback script, and `terminateWindowsProcessBound` reaches it on its own: on win32 without the FFI primitive (ARM64, or an x64 whose DLL load failed) it runs the PowerShell script. PR2's terminate adapter therefore has to gate strict itself, for example by answering `unavailable` whenever `windowsProcessNative()` is null, and must not call `terminateWindowsProcessBound` there. Without a native API `openWindowsProcessWatch` already answers `unavailable`, so the wait reports `unknown` rather than proving an exit. + +## Notes for PR2 + +- **The route's caps.** 030 caps the `/api/stop` route's stray sweep at 4 s and its receipt-less Part A at 6 s. `stopWindowsServiceChecked` has no overall cap: its own caps add up to more (the stray path alone has a 5 s enumeration), and it never returns before its steps do. The route needs an outer race on its deadline that answers `service_state_unknown` on overrun, or cap overrides passed in, such as a 4 s enumeration for the sweep. ## Open questions - Is the closing-sample accounting acceptable, or should the hold be measured differently, for example by shortening the final enumeration to 4.5 s to keep 030's totals? - Should a WinSW-only stop also sweep a stray scheduler wrapper (that would add an enumeration and 3 s to the WinSW row)? -- ARM64 ships the bounded fallback here. 050 requires an ARM64 device run before release, or else the strict variant. PR2 needs that decision before the fallback is reachable. - `GetLastError` after a failed `OpenProcess` through Bun FFI separates "gone" from "denied". If the runtime clobbers it, the answer degrades to "unavailable" (`unknown`). The live x64 test in `tests/windows/windows-process-evidence.test.ts` checks this on the Windows CI lane. diff --git a/src/service/windows-checked-stop.ts b/src/service/windows-checked-stop.ts index 8af35fda589..f81eb759eec 100644 --- a/src/service/windows-checked-stop.ts +++ b/src/service/windows-checked-stop.ts @@ -9,6 +9,7 @@ * 1. Probe registration (task, WinSW; 2 s caps). An unreadable answer is unknown with * nothing changed. With nothing registered, one enumeration (5 s) looks for a stray * wrapper of a deleted task; unreadable, or nothing found, is absent. + * With a task registered or a stray sweep running, the Schedule host PID is read (2 s). * Admission runs here, after the probe and before anything changes. * 2. Bind (src/service/windows-stop-binding.ts) and open an exit watch per wrapper. * 3. Ask the manager: schtasks /end (2 s); WinSW a non-waiting stop (2 s) and an SCM poll diff --git a/src/service/windows-stop-budget.ts b/src/service/windows-stop-budget.ts index bc82fa239d6..ca937c6c851 100644 --- a/src/service/windows-stop-budget.ts +++ b/src/service/windows-stop-budget.ts @@ -4,8 +4,9 @@ * One deadline governs the whole stop. Admission runs after Part A's registration probe * and before anything changes: it adds up the worst case of every step still to come and * refuses (unverified-budget, nothing changed) when the time left cannot hold it. Every - * step after admission has a cap, and these constants are those caps, so an admitted stop - * cannot overrun its deadline and the graceful child exit wait is never shortened. + * step after admission that waits on the machine has a cap, and these constants are those + * caps. Opening the bind-time watches and the caller's authorizer are not capped; the + * authorizer runs inside the terminateAndWait window and is bounded by its caller. * * Pure arithmetic; the callers own the clock. */ diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 92209f543af..c6dc76270c9 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -32,7 +32,13 @@ trusted system directory with a cap. The ARM64 terminate is one trusted PowerShe re-reads `Get-Process -Id` StartTime, compares it with the bound time within 1 s (UTC), calls `Kill()` on the same `Process` object and reads the exit back. `Kill()` opens its own handle by PID, so a window between the in-script check and the kill remains; that window and the 1 s tolerance are -the accepted residuals of the fallback. Only integers reach a script. +the accepted residuals of the fallback. Only integers reach a script: a fractional timeout is +floored to whole milliseconds, and one that floors to 0 spawns nothing. + +With no ARM64 device run, 050 ships ARM64 with the strict variant (kill nothing, report `unknown`). +`terminateWindowsProcessBound` does not enforce that: on win32 without the native API it runs the +fallback script. The caller's terminate adapter gates strict itself; see +`devlog/_plan/261111_windows_service_stop/060_implementation.md`. ## Binding rule @@ -51,12 +57,13 @@ deepest first and carry pid, creation time, image and a SHA-256 digest of the co ## Part A and Part B `src/service/windows-checked-stop.ts` is Part A, `stopWindowsServiceChecked(deps, options)`. Every -external effect is injected, and every step runs under a cap that races a timer and rechecks the -elapsed time on the injected clock. It probes registration (2 s each). An unreadable answer is -`unknown` with nothing changed. With nothing registered it enumerates once for a stray wrapper of a -deleted task; an unreadable table or no wrapper is `absent`. Admission then compares the caller's -deadline with the worst case still to come and returns `unverified-budget` without changing -anything when it does not fit. It binds, ends the task (`schtasks /end`, 2 s) or stops WinSW (a +external effect is injected. It probes registration (2 s each). An unreadable answer is `unknown` +with nothing changed. With nothing registered it enumerates once for a stray wrapper of a deleted +task; an unreadable table or no wrapper is `absent`. With a task registered or a stray sweep +running, it reads the Schedule host PID (2 s). Admission then compares the time left on the +caller's deadline, which already excludes these first steps, with the worst case still to come, +and returns `unverified-budget` without changing anything when it does not fit. It binds, ends +the task (`schtasks /end`, 2 s) or stops WinSW (a 2 s stop command and an SCM poll capped at `` + 2 s), terminates bound survivors deepest first, waits on the bind-time watches (terminates and wait share 3 s), and reads the manager state once (1 s). Denied stops every further step. A cap overrun is `unknown`. A wrapper or @@ -65,6 +72,13 @@ is `survived` while the proxy is still alive after the service reports stopped. variant `guardedStopWindowsServiceChecked` runs its authorizer at entry and before each terminate and reports a refusal as `WindowsCheckedStopRefusedError` with what had already happened. +Each step that asks the machine runs under a cap that races a timer and rechecks the elapsed time +on the injected clock. Opening the bind-time watches and the authorizer are not capped. The +authorizer runs inside the shared 3 s terminate-and-wait window, so its time comes out of that +window, and an authorizer that never returns hangs Part A; its caller bounds it. Part A has no +overall cap either, so a caller with a tighter bound than the sum of the caps (the `/api/stop` +route's 4 s sweep and 6 s Part A in 030) needs an outer race or smaller caps. + `src/service/windows-stop-confirm.ts` is Part B, `confirmWindowsStopFreed(deps)`, run after the caller's graceful child stop. The port must stay free for a 6 s hold, measured between sample starts, inside a window of `min(12 s, remaining - 0.5 s - 5 s - reserve)` with a 6 s floor. A final From 07e08dd992fce1977b6946cc9d26b1846ec7f1f8 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 02:23:45 +0900 Subject: [PATCH 05/53] feat(service): wire the checked Windows stop into every stop path Every Windows stop path now uses the checked stop from #8564 (part A ends the manager and the wrappers it can bind; part B confirms the port stays free): - stopServiceIfInstalledDetailed / stopServiceIfInstalledReport are async and run part A on Windows; Task Scheduler no longer reports stopped-respawnable. - Plain ocx stop: a part A outcome other than stopped/absent sends no /api/stop POST, blocks the shared teardown and records service failed (or state-unknown), proxy unknown. After a stopped part A, part B replaces the respawn re-poll and the POST skips stopProxy's port wait; orphan discovery is capped at 3 s. The receipt carries the part A record. - Guarded (consent) path: the inspection reads windowsProcessSnapshot and the Schedule PID once, a not-running task with wrappers the binding rule ties to the approved PID is bound-stray (revalidation compares the wrapper set), part A binds on the revalidation's snapshot before signalApproved, and part B runs after settle. - ocx stop --deadline-ms and OCX_STOP_DEADLINE_MS (flag wins, empty or invalid is absent, 60 s default), measured from process start; admission refuses with unverified-budget before anything changes. - /api/stop skips part A for a receipt whose part A record names the registered managers, caps it at 6 s otherwise and sweeps a deleted task's stray wrapper in 4 s; WinSW keeps its in-process stop. - ocx service stop claims a teardown receipt before the deferred child stop and completes it after part B and the restore; service uninstall/remove and prepareServiceInstall refuse before deleting or replacing the task; install and switch stop when another program holds the port; ocx uninstall runs part B as its respawn window. - The updater runs guardedStopWindowsChecked for Task Scheduler; each of its three call sites aborts the restart unless the outcome is stopped or absent. - ocx-stop/1 gains the additive serviceStopReason and portHeldByForeign. - ARM64 ships strict: the terminate adapter answers unavailable whenever the native API is missing, so the PowerShell fallback is unreachable. - Interactive callers refuse part A when the stopper descends from this home's wrapper; /api/stop and the updater opt out by design. - Forced taskkill gets a 5 s timeout. The best-effort wrapper sweep has no remaining stop-path caller. Docs: structure/ops/windows-checked-stop.md (now wired), runtime, update-supervised-recovery and cli-management structure docs, the CLI lifecycle and management API references, and devlog 000/030/060 (030's admission minimums are now 48.5 / 56.5 / 42.5 s and the Desktop drain 70.5 s). Refs #7771 --- .../261111_windows_service_stop/000_plan.md | 2 +- .../261111_windows_service_stop/030_design.md | 6 +- .../060_implementation.md | 15 + .../content/docs/reference/cli/lifecycle.md | 41 +- .../content/docs/reference/management-api.md | 2 +- scripts/test-layout/layout.json | 4 +- src/cli/dispatch.ts | 5 +- src/cli/index.ts | 60 ++- src/cli/registry.ts | 3 +- src/cli/root.ts | 2 +- src/cli/stop-approval.ts | 85 ++++- src/cli/stop-report.ts | 23 ++ src/cli/stop-windows-checked.ts | 287 ++++++++++++++ src/cli/uninstall-plan.ts | 20 +- src/config/pending-teardown.ts | 48 +++ src/lib/process-control.ts | 13 +- src/server/management-api.ts | 4 +- src/server/stop-service-route.ts | 85 +++++ src/service.ts | 4 +- src/service/cli-windows-stop.ts | 133 +++++++ src/service/cli.ts | 37 +- src/service/guarded-manager-target.ts | 19 +- src/service/orchestration.ts | 161 ++++---- src/service/windows-checked-stop-live.ts | 326 ++++++++++++++++ src/service/windows-guarded-manager.ts | 67 +++- src/service/windows-ops.ts | 53 --- src/service/windows-service-stop.ts | 258 +++++++++++++ src/service/windows-stop-deadline.ts | 39 ++ src/update/guarded-runtime-ops.ts | 12 +- src/update/job-windows-restart.ts | 59 ++- src/update/job.ts | 17 +- structure/INDEX.md | 8 +- structure/cli-management.md | 2 +- structure/manifest.json | 10 +- structure/ops/update-supervised-recovery.md | 13 +- structure/ops/windows-checked-stop.md | 92 ++++- structure/runtime.md | 2 +- tests/cli/cli-dispatch.test.ts | 12 +- tests/cli/cli-ready.test.ts | 26 +- tests/cli/uninstall.test.ts | 10 +- .../desktop-supervised-restart.test.ts | 2 +- tests/fixtures/parent-stop-runner.ts | 19 +- tests/fixtures/test-layout-expected.json | 4 +- tests/helpers/update-guarded-io.mjs | 6 +- .../update-helper-control-contracts.ts | 3 +- tests/helpers/update-helper-inventory.ts | 2 +- tests/helpers/update-import-policy.ts | 10 +- tests/helpers/windows-stop-caller-fixtures.ts | 42 +++ tests/providers/xai/grok-lifecycle.test.ts | 56 +-- .../service-desktop-runtime-preflight.test.ts | 6 +- .../service-install-wrapper-cleanup.test.ts | 113 +++--- .../service/service-runtime-preflight.test.ts | 23 +- tests/service/service.test.ts | 62 ++- tests/service/stop-deferred-teardown.test.ts | 2 +- tests/update/update-effect-authority.test.ts | 36 +- .../update/update-recovery-boundaries.test.ts | 14 +- .../update-windows-wrapper-authority.test.ts | 64 +++- .../windows/windows-checked-stop-live.test.ts | 131 +++++++ .../windows-deploy-close-regressions.test.ts | 26 +- .../windows-service-stop-wiring.test.ts | 234 ++++++++++++ .../windows/windows-service-wrappers.test.ts | 15 +- tests/windows/windows-stop-callers.test.ts | 356 ++++++++++++++++++ tests/windows/windows-stop-ocx-stop.test.ts | 114 ++++++ 63 files changed, 2896 insertions(+), 509 deletions(-) create mode 100644 src/cli/stop-windows-checked.ts create mode 100644 src/server/stop-service-route.ts create mode 100644 src/service/cli-windows-stop.ts create mode 100644 src/service/windows-checked-stop-live.ts create mode 100644 src/service/windows-service-stop.ts create mode 100644 src/service/windows-stop-deadline.ts create mode 100644 tests/helpers/windows-stop-caller-fixtures.ts create mode 100644 tests/windows/windows-checked-stop-live.test.ts create mode 100644 tests/windows/windows-service-stop-wiring.test.ts create mode 100644 tests/windows/windows-stop-callers.test.ts create mode 100644 tests/windows/windows-stop-ocx-stop.test.ts diff --git a/devlog/_plan/261111_windows_service_stop/000_plan.md b/devlog/_plan/261111_windows_service_stop/000_plan.md index 58ade97dacd..82070aa0a1e 100644 --- a/devlog/_plan/261111_windows_service_stop/000_plan.md +++ b/devlog/_plan/261111_windows_service_stop/000_plan.md @@ -1,6 +1,6 @@ # 000 — Windows service stop that frees the port (#7771 Option B): design -Status: approved (030, bounded ARM64 fallback). Implementation is split into three pull requests (060); the primitives and the two operations land first, unwired. +Status: approved (030). Implementation is split into three pull requests (060): PR1 (primitives and the two operations, #8564) and PR3 (Desktop, #8560) have landed; PR2 wires every caller. ARM64 ships the strict variant until the 040 ARM64 device case runs (050, 060). ## Problem diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index b517be8c912..5e6e967ba1b 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -74,15 +74,15 @@ Worst case still to come, at admission: | POST answer | 10 s, always | never shortened. The receipt and its part A record are written after admission (the snapshotless receipt inside `stopWithDeferral`), so admission cannot know whether they will exist and always counts the full abort. With the record the route skips part A and answers at once; without it (the write failed, or an older CLI) the route keeps today's behavior inside the same 10 s | | child exit: graceful exit wait, then forced | `shutdownTimeoutMs` + 3 s + 10 s = 18 s by default | today's ladder, kept in sequence: the graceful exit wait (never shortened) runs after an answered POST; if the child has not exited, or the POST aborted, `stopProxy` forces (never after a refusal): `taskkill` with a new 5 s timeout (it has none today) plus today's `waitForExit` 5 s | | inherited-receipt probes | 3 s × N | N abandoned receipts, known at admission; each probe tries both loopback hosts at today's 1.5 s per host (`probeEndpointLiveness`) | -| part B | 6 s floor, up to `min(12 s, remaining − 5 s − reserve)` | `waitForStoppedPort` (today a separate 15 s wait inside `stopProxy`) is the same port-free predicate and on Windows becomes part B's first phase; `settle`'s `waitExit` and endpoint probe are its first sample; the final CIM enumeration (`managerState` and `guardFinalStopSummary` share it) runs after the hold, capped at 5 s, and is counted: part B's worst case is 6 s + 5 s = 11 s. An unreadable result or a wrapper found that was not there at bind is non-success. Not run after a guarded `absent` manager (where `stopManager` is not called, as today). When admission lands exactly at the minimum, part B has no slack beyond its floor, so a slow port release reports failure instead of overrunning. Part B keeps today's `reclaimListenPort` clearing of a listener left by the exited child, and a listener whose owner has exited is not classified as `portHeldByForeign` | +| part B | 6 s floor, up to `min(12 s, remaining − 5 s − reserve)` | `waitForStoppedPort` (today a separate 15 s wait inside `stopProxy`) is the same port-free predicate and on Windows becomes part B's first phase; `settle`'s `waitExit` and endpoint probe are its first sample; the final CIM enumeration (`managerState` and `guardFinalStopSummary` share it) runs after the hold, capped at 5 s, and is counted: part B's worst case is 6 s + 5 s + the 0.5 s closing sample = 11.5 s. An unreadable result or a wrapper found that was not there at bind is non-success. Not run after a guarded `absent` manager (where `stopManager` is not called, as today). When admission lands exactly at the minimum, part B has no slack beyond its floor, so a slow port release reports failure instead of overrunning. Part B keeps today's `reclaimListenPort` clearing of a listener left by the exited child, and a listener whose owner has exited is not classified as `portHeldByForeign` | | reserve | 3 s | shared restore (`restoreSharedClientStateAfterStop`; its worst case is measured in 040 and the reserve grows if it does not fit), `revertSystemEnv` (a real 1 s cap on what is a synchronous call today; an overrun is logged and left for the next start), and the JSON summary | -Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 + 18 + 11 + 3 = 48 s; snapshotless Task Scheduler 11 + 3 + 10 + 18 + 11 + 3 = 56 s; WinSW snapshotless 25 + 3 + 11 + 3 = 42 s. The stray enumeration of a deleted task (5 s) runs in step 1, before admission, and so comes out of the time left at admission. +Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 + 18 + 11.5 + 3 = 48.5 s; snapshotless Task Scheduler 11 + 3 + 10 + 18 + 11.5 + 3 = 56.5 s; WinSW snapshotless 25 + 3 + 11.5 + 3 = 42.5 s. Part B is counted at 11.5 s: the 6 s hold, the 5 s final enumeration, and the 0.5 s closing port sample that can start at the very end of the window (the hold is measured between sample starts; 060 "Where PR1 differs from 030"). The stray enumeration of a deleted task (5 s) runs in step 1, before admission, and so comes out of the time left at admission. | Caller | Deadline | Fits? | |---|---|---| | consented takeover (`--expect-*`) | remaining startup deadline − 5 s (a Windows approval adds 60 s) | yes in practice; otherwise refused before anything changes | -| Desktop exit/restart drain (plain `ocx stop --json`) | today `runtime_stop::DEADLINE` − 3 s = 27 s | NO for a registered service. The same change raises Desktop's exit-drain deadline on Windows to 75 s (72 s passed as `OCX_STOP_DEADLINE_MS`). That holds the largest case (56 s) plus an 8 s allowance for what runs inside the deadline before admission (CLI start, the 2 s registration probe, and in the stray case the 5 s step-1 enumeration) plus two inherited receipts (3 × 2 = 6 s): 56 + 8 + 6 = 70 s ≤ 72 s; beyond that the stop is refused up front and the operator stops from a terminal. An older Desktop passes no flag, the CLI uses 60 s and can outlive Desktop's 30 s wait, which ends in `DrainVerdict::Failed`; today's stop can already outlive it (10 s POST + 8 s exit wait + 15 s port wait), so this is not new, and it is listed in 050 | +| Desktop exit/restart drain (plain `ocx stop --json`) | today `runtime_stop::DEADLINE` − 3 s = 27 s | NO for a registered service. The same change raises Desktop's exit-drain deadline on Windows to 75 s (72 s passed as `OCX_STOP_DEADLINE_MS`). That holds the largest case (56.5 s) plus an 8 s allowance for what runs inside the deadline before admission (CLI start, the 2 s registration probe, and in the stray case the 5 s step-1 enumeration) plus two inherited receipts (3 × 2 = 6 s): 56.5 + 8 + 6 = 70.5 s ≤ 72 s; beyond that the stop is refused up front and the operator stops from a terminal. An older Desktop passes no flag, the CLI uses 60 s and can outlive Desktop's 30 s wait, which ends in `DrainVerdict::Failed`; today's stop can already outlive it (10 s POST + 8 s exit wait + 15 s port wait), so this is not new, and it is listed in 050 | | terminal, scripts, other interactive callers | 60 s | yes | | `/api/stop` route | receipt with a part A record: no part A; receipt without the record: today's service stop, now the async part A, capped at the route's 6 s (an overrun answers `service_state_unknown`); WinSW: today's in-process stop; stray of a deleted task, no receipt: 4 s sweep (its enumeration capped at 4 s instead of today's 5 s; an overrun answers `service_state_unknown`, which fails safe) + teardown before the answer | the parent's 10 s POST abort, never shortened | | updater | job budget | yes | diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 85e153ad61d..af37d39ccd5 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -55,6 +55,21 @@ Contract doc: `structure/ops/windows-checked-stop.md`. - **The route's caps.** 030 caps the `/api/stop` route's stray sweep at 4 s and its receipt-less Part A at 6 s. `stopWindowsServiceChecked` has no overall cap: its own caps add up to more (the stray path alone has a 5 s enumeration), and it never returns before its steps do. The route needs an outer race on its deadline that answers `service_state_unknown` on overrun, or cap overrides passed in, such as a 4 s enumeration for the sweep. +## Where PR2 differs from 030 + +PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("Callers") is the contract. Points where the wiring makes a decision 030 leaves open, or departs from it: + +- **Route caps.** The outer race lives in `runWindowsServiceStopPartA({ capMs })`: an overrun answers `unknown` at once, and from then on the live `/end` and WinSW stop are gated off and the authorizer refuses every terminate, so nothing changes after the answer. A step that already started (a `/end` in flight) still completes. +- **The receipt record is checked against registration only.** The route skips Part A when the record's managers match a fresh registration probe (2 s each). It does not re-enumerate the recorded wrapper set; that would be the second enumeration 030 rules out. +- **The ancestry guard (r2 review).** The binding rule excludes the stopper but not its ancestors, so interactive callers refuse a Part A whose stopper descends from this home's wrapper. `/api/stop` and the updater opt out because both run beneath that wrapper by design. The check reads the bind snapshot (or the guarded snapshot), so it costs no extra enumeration. +- **The guarded evidence snapshot (r1 review).** The guarded inspection reads `windowsProcessSnapshot` and the Schedule PID once per inspection instead of the CIM-only `windowsProcessList`, and `stopManager` binds on the revalidation's copy. A CIM-only table would make the x64 terminate answer `reused` and Part A report a live wrapper as exited. +- **Absent counts as a pass for the updater and for install.** 040 says call sites abort on a non-`stopped` outcome. `absent` (nothing registered, nothing bound) has nothing that could relaunch the proxy, so the updater, install and switch continue on it, as `ocx stop` does. Refusing it would block every update on a machine whose recorded service was deleted. +- **`ocx service stop` on Part A failure** refuses before the receipt and the child stop, the same rule as plain `ocx stop`, so it never restores under a wrapper that can relaunch the proxy. +- **Part B's foreign result** fails install and switch (030) but not `ocx stop`, `ocx service stop` or uninstall, which only need our processes gone. +- **Not capped here.** The guarded path's own pre-steps (`runResolve`, the tracked-record check, the two inspections and their 5 s snapshot) run before admission and are measured, as 030 says. `proveLiveProxyOwnedByHome` is not capped separately; orphan discovery's `findLiveProxy` is capped at 3 s, and a missed orphan is caught by Part B. `revertSystemEnv` returns at once off macOS, so its 1 s cap holds on Windows without a change. `restoreSharedClientStateAfterStop` and `performStopTeardown` were not measured on a device; 040 keeps those measurements. +- **The manager-state read.** `schtasks /query /fo CSV` first, `Get-ScheduledTask` within the same 1 s cap when the status word is localized. On a slow or non-English machine that can overrun and report `unknown`; the 040 device run decides whether the cap needs to grow. +- **ARM64 ships strict** as recorded above; `windowsStopTerminate` enforces it whenever the native API is missing. + ## Open questions - Is the closing-sample accounting acceptable, or should the hold be measured differently, for example by shortening the final enumeration to 4.5 s to keep 030's totals? diff --git a/docs-site/src/content/docs/reference/cli/lifecycle.md b/docs-site/src/content/docs/reference/cli/lifecycle.md index 910aeef8bdb..90fc89117f5 100644 --- a/docs-site/src/content/docs/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/reference/cli/lifecycle.md @@ -80,7 +80,7 @@ ocx start --port 10100 --socks5 ocx start --socks5-off ``` -### `ocx stop [--json]` +### `ocx stop [--json] [--deadline-ms ]` An unsupervised Link Child handling a shutdown signal restores native Codex and managed Grok before releasing its listener, rechecking the served connection before each write. Service/Desktop supervision, sibling ownership, restart/recycle handoff and pending parent stop receipts preserve integrations. Shutdown rejects bound restart with `409 runtime_stopping`. Forced termination can bypass signal handlers; parent `ocx stop` restoration remains necessary. To route Codex through Home again after such a stop, start the Child (`ocx start`, or `ocx service start` for an installed service) and then run `ocx sync --restart-codex`. Starting alone does not reapply routing. A sync run while the Child is stopped still reapplies routing, which points Codex at a listener that is not running, so start the Child first. @@ -141,6 +141,45 @@ and its catalog to the restarted proxy instead of restoring native Codex. With 8 down, and the restarted proxy releases the kept teardown after its first sync; if it never starts, the next `ocx stop` restores native Codex. +#### Windows service stop + +On Windows, `ocx stop` stops a registered Task Scheduler task or native WinSW service with a +checked stop. It ends the task, then ends only the `wscript.exe` and `cmd.exe` wrapper processes +it can tie to this installation (this home's launcher or service script, a parent chain that leads +to the proxy or to the Task Scheduler service, and a recorded start time), and waits for them to +exit. Only then does it stop the proxy gracefully. Afterwards it confirms that the port stays free +for about six seconds and that no wrapper started again, so a wrapper's restart loop cannot bring +the proxy back behind a successful stop. + +When the service cannot be proven stopped, `ocx stop` sends no stop request to the proxy, leaves +your client configuration alone, and fails. With `--json` the summary keeps `outcome: "failed"` +and `proxy: "unknown"`, and adds `serviceStopReason`: + +| `serviceStopReason` | Meaning | What to do | +|---|---|---| +| `access-denied` | Ending the task or a wrapper was denied, usually an elevated task stopped from an unelevated terminal | Run `ocx stop` from an elevated terminal | +| `unbound-wrapper` | A wrapper running this home's service script could not be tied to the service, for example one started by hand | End that `wscript.exe`/`cmd.exe`, then retry | +| `evidence-unreadable` | The service or process state could not be read, or the port could not be confirmed free | Check `ocx service status`, then retry | +| `unverified-budget` | The time left could not hold a verified stop, so nothing was changed | Retry from a terminal | + +If another program is listening on the port after this installation's processes are gone, the +stop still succeeds and `--json` reports `portHeldByForeign: true`. Both fields are additive: +they are omitted when there is nothing to report, and no new `outcome` value exists. + +`--deadline-ms ` bounds the whole stop, measured from the moment the command started. Before +changing anything the stop checks that the time left can hold its worst case (about 57 seconds for +a registered Task Scheduler service with default settings) and otherwise refuses with +`unverified-budget`. Without the flag the stop reads `OCX_STOP_DEADLINE_MS` (the desktop app sets +it) and otherwise uses 60 seconds. macOS and Linux ignore both. + +The stop ends the running instance only. The task's logon and unlock triggers stay enabled, so the +proxy comes back at the next logon or unlock; use `ocx service uninstall` to remove the service. +`ocx service stop`, `ocx service uninstall`, `ocx uninstall` and `ocx service install` use the +same checked stop: uninstall and install refuse before deleting or replacing the task when the +service cannot be proven stopped, and install also stops when another program holds the port. On +Windows on Arm the wrappers cannot be ended yet, so a stop that would need to end one reports +`evidence-unreadable`; end the wrapper by hand, then retry. + For a desktop-approved takeover, `ocx stop --json` also accepts the complete set of `--expect-*` values supplied by `ocx resolve --json`. On Windows, this guarded stop can stop a Task Scheduler task only when its registered definition, running state, and wrapper diff --git a/docs-site/src/content/docs/reference/management-api.md b/docs-site/src/content/docs/reference/management-api.md index 9c199eb0c09..c0c2fb57a6b 100644 --- a/docs-site/src/content/docs/reference/management-api.md +++ b/docs-site/src/content/docs/reference/management-api.md @@ -864,7 +864,7 @@ the 503 without reporting model capacity; provider overload errors retain their | --- | --- | --- | | `GET /api/system/memory` | Return scalar process, heap, stream, response-state, watchdog, and active-turn metrics. Response-state diagnostics include spill-write status, consecutive failures, fixed privacy-safe failure class, and last failure/success timestamps. `spillLastWriteFailureOrigin` is `retry_returned_timeout`, `timeout_memo_refusal`, or null; cumulative `spillAclRetryReturnedTimeouts` and `spillAclTimeoutMemoRefusals` count terminal failed publications; cumulative `spillCapacityRefusals` and `spillHeadroomEvictions` count continuations refused at the spill ceiling and continuations evicted to make room for a newer one. See [Windows spill diagnostics](/troubleshooting/windows-memory/) for process-local semantics. Raw errors and paths are never returned. | — | | `POST /api/system/restart` | Begin a drain-aware process restart without removing client injection; optional JSON `drainGraceMs` explicitly shortens the default 60s grace | Returns 202; repeated calls report the existing grace; 400 invalid body or grace | -| `POST /api/stop` | Stop the service, restore native Codex, remove managed Grok injection, and drain the proxy | 409 service ownership conflict; 409 `respawnable_service` when a Windows Task Scheduler wrapper could respawn the proxy and the caller is not `ocx stop` (nothing is changed); 409 `self_unload_service` when this proxy is running as the installed launchd/systemd service, because stopping the manager from inside it would end the process before native Codex is restored — run `ocx stop` instead (nothing is changed); 409 when the installed manager refuses to stop; 409 `service_state_unknown` when the Task Scheduler state cannot be read (nothing is changed; repair the query and retry) | +| `POST /api/stop` | Stop the service, restore native Codex, remove managed Grok injection, and drain the proxy | 409 service ownership conflict; 409 `respawnable_service` when a Windows Task Scheduler wrapper could respawn the proxy and the caller is not `ocx stop` (nothing is changed); 409 `self_unload_service` when this proxy is running as the installed launchd/systemd service, because stopping the manager from inside it would end the process before native Codex is restored — run `ocx stop` instead (nothing is changed); 409 when the installed manager refuses to stop; 409 `service_state_unknown` when the Task Scheduler state cannot be read (nothing is changed; repair the query and retry), or when the Windows checked service stop does not finish within the route's cap (6 s; 4 s for a leftover wrapper of a deleted task). On Windows a request from `ocx stop` whose receipt records that the CLI already ran the checked service stop does not repeat it | | `GET /api/system/codex-app-server` | Report whether running Codex app-servers predate the current model catalog | — | | `POST /api/system/codex-restart` | Refresh the catalog, then restart Codex app-servers. A running Codex desktop app is given a restart attempt so the model picker can reload; the `desktopApp` response fields report a refusal, surviving targets, or a failed relaunch. A shell newer than the catalog is still reported as `desktopApp.pickerState: "unknown"`, because its start time cannot prove which `model_catalog_json` binding it loaded; an earlier sync may have rebound an already-existing catalog. A shell no newer than the catalog, or one whose binding this request changed, is `stale`. A closed desktop returns `desktopApp.reason: "no_targets"`; its `pickerState` may be `not_running`, or `stale` when this request changed the catalog binding. When the proxy itself is running inside the Codex app, the desktop restart is refused rather than handed off. | Returns 200 with `code: partially_stopped` when a target survives | diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index c2ad2913d08..5240ae0f33d 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -1932,8 +1932,8 @@ "windows-deploy-close-regressions.test.ts": "windows", "windows-elevation-spawn.test.ts": "windows", "windows-elevation.test.ts": "windows", - "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", - "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", + "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", "windows-checked-stop-live.test.ts": "windows", "windows-service-stop-wiring.test.ts": "windows", + "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", "windows-stop-callers.test.ts": "windows", "windows-stop-ocx-stop.test.ts": "windows", "windows-process-priority.test.ts": "windows", "windows-scheduler-errors.test.ts": "windows", "windows-scheduler-install-verification.test.ts": "windows", "windows-lifecycle-root-acl.test.ts": "windows", diff --git a/src/cli/dispatch.ts b/src/cli/dispatch.ts index 6cbe047f2a1..0ca8e19a856 100644 --- a/src/cli/dispatch.ts +++ b/src/cli/dispatch.ts @@ -35,6 +35,7 @@ import { normalizeUpdateChannel, runGuiUpdateWorker } from "../update/job"; import { isJsonOption, takeFlag } from "./runtime-api"; import { printStopSummary, type StopOutcome } from "./stop-report"; import { parseStopApproval, type StopApproval } from "./stop-approval"; +import { resolveStopDeadlineMs, setStopDeadlineMs } from "../service/windows-stop-deadline"; import type { ResolveArgs } from "./resolve"; import type { ClientConnectionState } from "../client/state"; import { OCX_NATIVE_REPLAY_RECOVERY_NOTE } from "../responses/compaction"; @@ -110,9 +111,11 @@ const commandRunners: Record = { stop: async deps => { const parsed = parseStopApproval(deps.args.slice(1)); if (!parsed.ok) { - console.error("Usage: ocx stop [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); + console.error("Usage: ocx stop [--deadline-ms ] [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); return 64; } + // #7771: the Windows checked stop's deadline, from this process's start (flag, then env, then 60 s). + setStopDeadlineMs(resolveStopDeadlineMs(parsed.deadlineMs)); // Downtime warning lives HERE, not in handleStop: `restart`/tray-restart callers // re-start the proxy immediately, so warning there would contradict the next line. const warning = "⚠️ Codex/Claude requests through the proxy will fail until it is restarted ('ocx start' or 'ocx service start')."; diff --git a/src/cli/index.ts b/src/cli/index.ts index 10e429277bc..acb7964efd5 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -79,7 +79,7 @@ import { quarantinePendingTeardown, } from "../config/pending-teardown"; import { collectStatus, detectMissingCodexCatalogPath, hubStatusLines, missingCodexCatalogLines, remoteHubBannerLine, remoteHubStatusLines, runtimeSupervisorLine, statusDeadProxyLines, statusProxyUp, statusUnusedProxyLines } from "./status"; -import { endpointsToProve, everyEndpointProvenDownAsync, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; +import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; import { takeFlag } from "./runtime-api"; import { parseStartOptions, StartArgsError } from "./start-args"; @@ -124,12 +124,12 @@ import { type GuardedStopSnapshot, type StopApproval, } from "./stop-approval"; -import { inspectGuardedManagerTarget, observeGuardedManagerStopped } from "../service/guarded-manager-target"; import { summarizeStopRun, type StopOutcome, type StopRunRecord } from "./stop-report"; +import { confirmGuardedFreed, inspectGuardedTarget, noServiceStop, observeGuardedStopped, orphanDiscoveryCap, stopGuardedManager, stopServiceForOcxStop, stopServiceForUninstall, stopWindowFailed, verifyUninstallStopWindow, windowsChildStopHeld, windowsPartAHolds, windowsPartAReceiptRecord, windowsPartBDue } from "./stop-windows-checked"; import { runCli } from "./root"; import { isProcessAlive, ProxyOwnershipRefusedError, refusalNextStep, stopProxy } from "../lib/process-control"; import { startupDataPlaneToken } from "../lib/service-secrets"; -import { assertNotAdminToken, diagnoseService, isServiceOwnershipError, proxyStillLiveAfterStop, serviceCommand, serviceEnvironmentOwnedHere, serviceStartableFromTray, serviceStatePaths, serviceStatusSummary, stopServiceIfInstalledDetailed, uninstallServiceIfInstalled, uninstallServiceDetailed } from "../service"; +import { assertNotAdminToken, diagnoseService, isServiceOwnershipError, serviceCommand, serviceEnvironmentOwnedHere, serviceStartableFromTray, serviceStatePaths, serviceStatusSummary, uninstallServiceIfInstalled, uninstallServiceDetailed } from "../service"; import { acquireOwnershipMutationLease } from "../service/ownership-mutation-lease.mjs"; import { formatStartupRoutingDetail, startupHealthSummary } from "../codex/autostart-health"; import { injectSystemEnv, reconcileShellHook, revertSystemEnv, uninstallShellHook } from "../server/system-env"; @@ -1028,7 +1028,7 @@ async function handleStop(approval?: StopApproval) { catch { return null; } }, describeTrackedRuntimeTarget, - () => inspectGuardedManagerTarget(approval.pid, approval.port), + () => inspectGuardedTarget(approval.pid, approval.port), snapshot => handleStopUnlocked(snapshot), ); } @@ -1148,7 +1148,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { if (teardownNonce || stoppingSibling) return; if (connectedClientSkip) return; try { - teardownNonce = claimPendingTeardown(endpoint, endpointSource, process.pid, handoverRequested ? "update-restart" : undefined).nonce; + teardownNonce = claimPendingTeardown(endpoint, endpointSource, process.pid, handoverRequested ? "update-restart" : undefined, windowsPartAReceiptRecord()).nonce; } catch (err) { // Without a receipt the proxy performs its own teardown, which is the pre-#3008 // behaviour: correct for every backend that cannot respawn, and merely early for @@ -1175,6 +1175,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // Only an exact endpoint may direct the request; the configured fallback is a guess // good enough to record an obligation against, not to POST a stop to. runtimeEndpoint: exact ?? undefined, + skipPortWait: windowsPartBDue(), // #7771: Part B owns the Windows port wait after a checked stop. }); // A valid receipt means the proxy deferred shared teardown to this process. If the // receipt could not be written, the proxy restores it itself and the caller must not @@ -1192,22 +1193,23 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { try { const serviceStop = snapshot ? (guardedStep = await runGuardedManagerStep(snapshot, { - revalidateManager: () => inspectGuardedManagerTarget(snapshot.approval.pid, snapshot.approval.port), + revalidateManager: () => inspectGuardedTarget(snapshot.approval.pid, snapshot.approval.port), stopManager: () => { if (approvedEndpoint) claimTeardown(approvedEndpoint, "exact"); - return stopServiceIfInstalledDetailed(); + return stopGuardedManager({ approvedPort: snapshot.approval.port, receiptNonce: () => teardownNonce, inheritedReceipts: inheritedTeardowns.length }); }, signalApproved: () => stopWithDeferral(snapshot.approval.pid, approvedEndpoint), settle: () => settleApprovedTarget(snapshot.approval), - managerState: () => observeGuardedManagerStopped(snapshot.manager), + confirmFreed: () => confirmGuardedFreed(record), + managerState: () => observeGuardedStopped(snapshot.manager), })).service // A sibling never runs under a service manager: an installed one belongs to the live owner, // and its ownership check would fail this stop and skip the stale-record purge below. // #7423: a manager stop takes the proxy down itself, so a handover claims its receipt first. - : stoppingSibling || connectedClientAlreadyStopped ? "absent" : (claimHandoverBeforeManager(), stopServiceIfInstalledDetailed()); + : stoppingSibling || connectedClientAlreadyStopped ? noServiceStop() : (claimHandoverBeforeManager(), await stopServiceForOcxStop({ receiptNonce: () => teardownNonce, inheritedReceipts: inheritedTeardowns.length })); record.service = serviceStop; stoppedService = serviceStop === "stopped" || serviceStop === "stopped-respawnable"; - schedulerCanRespawn = serviceStop === "stopped-respawnable"; + schedulerCanRespawn = serviceStop === "stopped-respawnable" || windowsPartBDue(); // No "won't respawn" claim here: a stopped Task Scheduler can still respawn through // its wrapper, which the verification below is what actually settles. if (stoppedService) console.log("🛑 Service manager stopped."); @@ -1225,6 +1227,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { console.error("❌ The Windows Task Scheduler state could not be read, so this stop cannot tell whether a wrapper would respawn the proxy."); console.error(" Run 'ocx service status' to see the query error, repair Task Scheduler access, then retry."); } + ownershipBlocked = windowsPartAHolds(record) || ownershipBlocked; // #7771: no POST and no shared teardown. } catch (err) { record.service = "error"; if (isServiceOwnershipError(err)) { @@ -1239,7 +1242,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { if (snapshot) { if (guardedStep?.effect === "approval-changed") { - return approvalChanged(guardedStep.detail); + return approvalChanged(guardedStep.detail, guardedStep.stopReason); } if (guardedStep?.effect === "manager-still-active") { return managerStillActive(record.service, record); @@ -1255,7 +1258,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { record.proxy = "unknown"; console.error("The approved runtime could not be verified after the guarded stop step."); } - } else { + } else if (windowsChildStopHeld()) { /* #7771: Part A did not prove the manager stopped, so no POST. */ } else { const pid = readPid(); if (pid) { try { @@ -1296,7 +1299,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { const staleRuntimePid = readRuntimePort()?.pid ?? null; // Orphan recovery: a live proxy can outlive its pid file (crash, manual delete, // corrupt file). Identity-checked liveness still finds it via the runtime record. - const live = await findLiveProxy({ acceptPackageTreeFenced: true }); + const live = await findLiveProxy({ acceptPackageTreeFenced: true, ...orphanDiscoveryCap() }); if (siblingStopFoundOwner(siblingOfPort, live)) { record.proxy = "not-running"; console.log(`The sibling instance is already gone; the proxy on port ${siblingOfPort} was left running.`); @@ -1357,21 +1360,13 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // Environment ownership is independent from service ownership. Roll back current-home // variables (the helper refuses foreign markers on its own) unless a sibling never set them. if (!stoppingSibling) { try { revertSystemEnv(); } catch { /* best-effort */ } } - // A stopped Windows scheduler is not a proven-down proxy. `killWindowsSchedulerWrappers` - // is explicitly best-effort and the `:loop` wrapper respawns its child after ~5s, so an - // immediate probe can see a dead interval and an update can start replacing files right - // before the proxy comes back. Poll across the restart window before this stop is allowed - // to report anything but failure (#3008) — and ONLY for that backend, since making every - // launchd and systemd stop wait seven seconds would be a regression in ordinary use. + // A stopped Windows service is not a proven-down proxy: the `:loop` wrapper relaunches its + // child about 5 s later, so an immediate probe can see a dead interval (#764, #3008). On + // Windows Part B of the checked stop (#7771) holds the port free across that window and + // re-enumerates the wrappers; only then may the stop report anything but failure. launchd + // and systemd are down when they say so and pay no wait. if (schedulerCanRespawn && !ownershipBlocked) { - const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); - if (survivor) { - stopFailed = true; - record.proxy = "respawned"; - console.error(`❌ A proxy is still listening on port ${survivor.port} after the service stop; it is being respawned.`); - console.error(" Skipping shared teardown: restoring client config while the proxy runs leaves both pointing at each other."); - ownershipBlocked = true; - } + if (await stopWindowFailed(record)) { stopFailed = true; ownershipBlocked = true; } } // Recovering somebody else's abandoned obligation is not the same act as finishing this // run's own. This run stopped a proxy and verified the result; the inherited case has no @@ -1523,7 +1518,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { }; const summary = summarizeStopRun(record, signals); if (snapshot && !stopFailed) { - return guardFinalStopSummary(record, () => observeGuardedManagerStopped(snapshot.manager), + return guardFinalStopSummary(record, () => observeGuardedStopped(snapshot.manager), () => ({ ok: !stopFailed, summary })); } return { ok: !stopFailed, summary }; @@ -1569,8 +1564,8 @@ async function handleUninstall() { serviceRemoval: null, respawnWindowVerified: false, }; - await runStep("service stopped", () => { - const outcome = stopServiceIfInstalledDetailed(); + await runStep("service stopped", async () => { + const outcome = await stopServiceForUninstall(); observed.serviceStop = outcome; if (outcome === "absent") return false; if (outcome === "failed") { @@ -1626,10 +1621,9 @@ async function handleUninstall() { // Only Task Scheduler can respawn through a surviving wrapper, and removing the // registration does not prove the running one died. Poll the same window `ocx stop` does // before shared config is allowed down (#764, #3008). - if (observed.serviceStop === "stopped-respawnable") { + if (respawnWindowRequired(observed)) { await runStep("respawn window verified", async () => { - const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); - if (survivor) throw new Error(`a proxy is still listening on port ${survivor.port} after the service was removed; it is being respawned`); + await verifyUninstallStopWindow(); // Windows: Part B (#7771); otherwise the legacy re-poll. // A null from that poll is not proof either: its identity probe returns null on a // timeout, so a respawned-but-unresponsive proxy looks the same as none. Require the // tri-state probe to say dead on every candidate before calling the window verified. diff --git a/src/cli/registry.ts b/src/cli/registry.ts index 90863ab1efc..bd0047a83d2 100644 --- a/src/cli/registry.ts +++ b/src/cli/registry.ts @@ -51,11 +51,12 @@ export const CLI_COMMANDS: CliCommandEntry[] = [ }, { name: "stop", - usage: "ocx stop [--json]", + usage: "ocx stop [--json] [--deadline-ms ]", summary: "Stop the proxy; preserve connected-client integrations, restore native Codex config for standalone owners.", details: [ "--json keeps the stop path unchanged and prints one structured summary document on stdout; human output moves to stderr.", "Exit codes are identical with and without --json: 0, 1, 79 (history cleanup incomplete), 80 (teardown deferred).", + "--deadline-ms Windows: finish the service stop within of this command's start, or refuse up front with nothing changed (default 60000; OCX_STOP_DEADLINE_MS is read when the flag is absent).", ], }, { diff --git a/src/cli/root.ts b/src/cli/root.ts index 9628af27afc..81f7d8ca433 100644 --- a/src/cli/root.ts +++ b/src/cli/root.ts @@ -141,7 +141,7 @@ export async function runCli(argv: string[]): Promise { process.exit(1); } if (head.command === "stop" && !parseStopApproval(head.args.slice(1)).ok) { - console.error("Usage: ocx stop [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); + console.error("Usage: ocx stop [--deadline-ms ] [--json [--expect-pid --expect-port --expect-hostname --expect-config-home --expect-cli-version --expect-compatibility-token ]]"); process.exit(64); } const uninstallError = uninstallArgsError(head.command, head.args); diff --git a/src/cli/stop-approval.ts b/src/cli/stop-approval.ts index 0e45cca200f..770c65c6eef 100644 --- a/src/cli/stop-approval.ts +++ b/src/cli/stop-approval.ts @@ -4,7 +4,9 @@ import { waitForPortAvailable } from "../server/ports"; import { probeEndpointLiveness, probeHostname } from "../server/proxy-liveness"; import type { GuardedManagerTarget } from "../service/guarded-manager-target"; import type { ResolveJson } from "./resolve"; -import { STOP_SUMMARY_SCHEMA, type StopOutcome, type StopRunRecord, type StopServiceOutcome } from "./stop-report"; +import { STOP_SUMMARY_SCHEMA, stopReasonFields, type StopOutcome, type StopRunRecord, type StopServiceOutcome, type StopServiceReason } from "./stop-report"; + +type MaybePromise = T | Promise; export interface StopApproval { pid: number; @@ -36,15 +38,43 @@ function sameGuardedManager( current: GuardedManagerTarget, ): boolean { if (expected.kind === "absent") return current.kind === "absent"; + if (expected.kind === "bound-stray") { + // The bound wrapper set is the identity (#7771): same PIDs with the same creation times. + if (current.kind !== "bound-stray" || current.pid !== expected.pid) return false; + const key = (set: { pid: number; startTime: number }[]) => + set.map(wrapper => `${wrapper.pid}@${wrapper.startTime}`).sort().join(","); + return key(current.wrappers) === key(expected.wrappers); + } return current.kind === "bound" && current.pid === expected.pid && current.managerPid === expected.managerPid && current.backend === expected.backend; } -export type StopApprovalParse = { ok: true; json: boolean; approval: StopApproval | null } | { ok: false }; +export type StopApprovalParse = + | { ok: true; json: boolean; approval: StopApproval | null; deadlineMs: number | null } + | { ok: false }; -export function parseStopApproval(argv: string[]): StopApprovalParse { - if (argv.length === 0) return { ok: true, json: false, approval: null }; - if (argv.every(arg => arg === "--json")) return { ok: true, json: true, approval: null }; +/** + * --deadline-ms (#7771): a positive decimal number of milliseconds, measured from this + * process's start; at most once, anywhere in the argument list. Removed here so the + * approval grammar below stays exactly what it was. + */ +function takeStopDeadline(argv: string[]): { argv: string[]; deadlineMs: number | null } | null { + const at = argv.indexOf("--deadline-ms"); + if (at < 0) return { argv, deadlineMs: null }; + if (argv.indexOf("--deadline-ms", at + 1) >= 0) return null; + const raw = argv[at + 1]; + if (raw === undefined || !/^\d+$/.test(raw)) return null; + const value = Number(raw); + if (!Number.isSafeInteger(value) || value <= 0) return null; + return { argv: [...argv.slice(0, at), ...argv.slice(at + 2)], deadlineMs: value }; +} + +export function parseStopApproval(input: string[]): StopApprovalParse { + const taken = takeStopDeadline(input); + if (!taken) return { ok: false }; + const { argv, deadlineMs } = taken; + if (argv.length === 0) return { ok: true, json: false, approval: null, deadlineMs }; + if (argv.every(arg => arg === "--json")) return { ok: true, json: true, approval: null, deadlineMs }; const names = ["--expect-pid", "--expect-port", "--expect-hostname", "--expect-config-home", "--expect-cli-version", "--expect-compatibility-token"]; if (argv.filter(arg => arg === "--json").length !== 1) return { ok: false }; @@ -68,7 +98,7 @@ export function parseStopApproval(argv: string[]): StopApprovalParse { const cliVersion = values.get("--expect-cli-version") ?? ""; if (pid === null || port === null || !/^[a-f0-9]{64}$/.test(compatibilityToken) || !configHome || !cliVersion) return { ok: false }; - return { ok: true, json: true, approval: { + return { ok: true, json: true, deadlineMs, approval: { pid, port, hostname: values.get("--expect-hostname") ?? "", configHome, cliVersion, compatibilityToken, } }; @@ -104,23 +134,25 @@ export function stopApprovalMismatch(expected: StopApproval, now: ResolveJson): return null; } -export function approvalChanged(detail?: string): StopOutcome { +export function approvalChanged(detail?: string, serviceStopReason?: StopServiceReason): StopOutcome { return { ok: false, summary: { schema: STOP_SUMMARY_SCHEMA, ok: false, outcome: "approval-changed", exitCode: 1, runtimeDown: false, service: "absent", proxy: "unknown", sharedTeardown: "skipped", ...(detail ? { detail } : {}), + ...stopReasonFields({ serviceStopReason }), message: `The approved runtime or managing CLI changed${detail ? `: ${detail}` : ""}; no stop was attempted.`, } }; } export function managerStillActive( service: StopServiceOutcome, - record?: Pick, + record?: Pick, ): StopOutcome { return { ok: false, summary: { schema: STOP_SUMMARY_SCHEMA, ok: false, outcome: "manager-still-active", exitCode: 1, runtimeDown: false, service, proxy: record?.proxy ?? "unknown", sharedTeardown: record?.sharedTeardown ?? "skipped", + ...stopReasonFields(record ?? {}), message: "The approved runtime and service manager could not both be confirmed stopped.", } }; } @@ -129,14 +161,14 @@ export async function runApprovedStop( expected: StopApproval, read: () => Promise, tracked: () => TrackedRuntimeCheck, - managerTarget: () => GuardedManagerTarget, + managerTarget: () => MaybePromise, stop: (snapshot: GuardedStopSnapshot) => Promise, ): Promise { let snapshot: GuardedStopSnapshot | null = null; try { const now = await read(); const target = tracked(); - const manager = managerTarget(); + const manager = await managerTarget(); if (now === null) return approvalChanged("the runtime could not be re-resolved"); const mismatch = stopApprovalMismatch(expected, now); if (mismatch !== null) return approvalChanged(mismatch); @@ -149,9 +181,9 @@ export async function runApprovedStop( return approvalChanged("the tracked runtime record changed"); } if (manager.kind === "unknown") { - return approvalChanged(`the service manager could not be re-verified (${manager.reason})`); + return approvalChanged(`the service manager could not be re-verified (${manager.reason})`, manager.stopReason); } - if (manager.kind === "bound" && manager.pid !== expected.pid) { + if ((manager.kind === "bound" || manager.kind === "bound-stray") && manager.pid !== expected.pid) { return approvalChanged("a different manager owns the approved runtime"); } snapshot = { approval: expected, manager }; @@ -188,30 +220,40 @@ export async function settleApprovedTarget( export async function runGuardedManagerStep( snapshot: GuardedStopSnapshot, io: { - revalidateManager: () => GuardedManagerTarget; - stopManager: () => StopServiceOutcome; + revalidateManager: () => MaybePromise; + /** Async on every platform; on Windows it is Part A of the checked stop (#7771). */ + stopManager: () => MaybePromise; signalApproved: () => Promise; settle: () => Promise; + /** + * Windows Part B after settle (#7771): the approved endpoint stays free for the hold + * and no wrapper of this home came back. Runs only where stopManager ran. + */ + confirmFreed?: () => Promise; managerState: () => Promise<"inactive" | "active" | "unknown">; }, ): Promise<{ service: StopServiceOutcome; effect: "stopped" | "approval-changed" | "manager-still-active" | "failed"; - proxy: "stopped" | "unknown"; handledByProxy: boolean; detail?: string }> { + proxy: "stopped" | "unknown"; handledByProxy: boolean; detail?: string; stopReason?: StopServiceReason }> { try { - const current = io.revalidateManager(); + const current = await io.revalidateManager(); if (!sameGuardedManager(snapshot.manager, current)) { // An unprovable manager is not a proven change: name the observation failure, // not a swap, or the stop log loses why the revalidation could not answer. const detail = current.kind === "unknown" ? `the service manager could not be re-verified (${current.reason})` : "the service manager changed between approval and stop"; - return { service: "absent", effect: "approval-changed", proxy: "unknown", handledByProxy: false, detail }; + return { service: "absent", effect: "approval-changed", proxy: "unknown", handledByProxy: false, detail, + ...(current.kind === "unknown" && current.stopReason ? { stopReason: current.stopReason } : {}) }; } } catch { return { service: "absent", effect: "approval-changed", proxy: "unknown", handledByProxy: false, detail: "the service manager could not be re-validated" }; } - const service = snapshot.manager.kind === "absent" ? "absent" : io.stopManager(); - if (snapshot.manager.kind === "bound" + // The order is the #7771 contract: the manager and its wrappers end first (nothing can + // relaunch the child), then the approved PID gets its graceful signal and receipt. + const managed = snapshot.manager.kind !== "absent"; + const service = managed ? await io.stopManager() : "absent"; + if (managed && service !== "stopped" && service !== "stopped-respawnable") { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy: false }; } @@ -231,6 +273,11 @@ export async function runGuardedManagerStep( if (!await io.settle()) { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; } + if (managed && io.confirmFreed) { + try { + if (!await io.confirmFreed()) return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; + } catch { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; } + } try { if (await io.managerState() !== "inactive") { return { service, effect: "manager-still-active", proxy: "unknown", handledByProxy }; diff --git a/src/cli/stop-report.ts b/src/cli/stop-report.ts index 744c77e6d6f..831a81073c3 100644 --- a/src/cli/stop-report.ts +++ b/src/cli/stop-report.ts @@ -41,6 +41,12 @@ export type StopSharedTeardownOutcome = | "failed" | "skipped"; +/** + * Why the checked Windows service stop did not succeed (#7771). Additive on ocx-stop/1: + * absent when the CLI has no reason to name, so older documents read unchanged. + */ +export type StopServiceReason = "access-denied" | "evidence-unreadable" | "unbound-wrapper" | "unverified-budget"; + /** Facts recorded where the stop path already decides them. */ export interface StopRunRecord { /** What stopServiceIfInstalledDetailed returned, or "error" when it threw. */ @@ -53,6 +59,10 @@ export interface StopRunRecord { inheritedTeardownBlocks: boolean; /** A discharged receipt could not be removed from disk. */ receiptClearFailed: boolean; + /** Windows checked stop: the reason a Part A or Part B result was not a success. */ + serviceStopReason?: StopServiceReason; + /** Windows checked stop: Part B found another program on the port after ours were gone. */ + portHeldByForeign?: boolean; } /** The internal booleans that already pick the process exit code. */ @@ -81,6 +91,10 @@ export interface StopSummaryJson { * run can name one. Additive on `ocx-stop/1`; absent on every other outcome. */ detail?: string; + /** Additive (#7771): see StopRunRecord.serviceStopReason. Omitted when there is none. */ + serviceStopReason?: StopServiceReason; + /** Additive (#7771): true or false once Part B ran, omitted otherwise; never null. */ + portHeldByForeign?: boolean; } /** What handleStop returns: the pre-existing boolean plus the structured twin. */ @@ -126,6 +140,14 @@ function stopMessage(record: StopRunRecord, signals: StopRunSignals, outcome: St return "The stop failed."; } +/** The additive #7771 fields of a record, omitted when unset (never null). */ +export function stopReasonFields(record: Partial>): Pick { + return { + ...(record.serviceStopReason ? { serviceStopReason: record.serviceStopReason } : {}), + ...(typeof record.portHeldByForeign === "boolean" ? { portHeldByForeign: record.portHeldByForeign } : {}), + }; +} + /** Pure mapper from the recorded run to the wire document. */ export function summarizeStopRun(record: StopRunRecord, signals: StopRunSignals): StopSummaryJson { const outcome = stopOutcome(record, signals); @@ -139,6 +161,7 @@ export function summarizeStopRun(record: StopRunRecord, signals: StopRunSignals) proxy: record.proxy, sharedTeardown: record.sharedTeardown, message: stopMessage(record, signals, outcome), + ...stopReasonFields(record), }; } diff --git a/src/cli/stop-windows-checked.ts b/src/cli/stop-windows-checked.ts new file mode 100644 index 00000000000..f050d86806a --- /dev/null +++ b/src/cli/stop-windows-checked.ts @@ -0,0 +1,287 @@ +/** + * ocx stop's side of the checked Windows service stop (#7771, 030 "Two parts"). + * + * handleStopUnlocked in src/cli/index.ts keeps its structure; the Windows-specific + * decisions live here so that file does not grow: + * + * - snapshotless: Part A runs as the service stop, with admission against the stop + * deadline. A result other than stopped/absent holds the child stop (no POST), and a + * stopped result makes Part B replace the old respawn re-poll; + * - guarded: the manager inspection reads the evidence snapshot (windowsProcessSnapshot, + * native creation times on x64) once per inspection, and Part A binds against the + * revalidation's snapshot; Part B runs after settle on the approved endpoint, and its + * final enumeration is the one the manager-state checks share; + * - the teardown receipt carries the Part A record, so /api/stop skips a second Part A. + * + * Off Windows every function keeps today's behavior and timing. + */ +import { recordPendingTeardownPartA, type PendingTeardownPartA } from "../config/pending-teardown"; +import { readPid } from "../config/process-state"; +import { observeGuardedManagerStopped, inspectGuardedManagerTarget, type GuardedManagerTarget } from "../service/guarded-manager-target"; +// Through the barrel, like every other stop-path call, so the parent-stop fixtures can see it. +import { proxyStillLiveAfterStop, stopServiceIfInstalledDetailed, stopServiceIfInstalledReport } from "../service"; +import type { ServiceStopOutcome } from "../service/orchestration"; +import type { WindowsCheckedStopResult } from "../service/windows-checked-stop"; +import { readScheduleServicePid, windowsProcessSnapshot, type WindowsStopProcessEntry } from "../service/windows-process-evidence"; +import { + confirmWindowsServiceStop, + describeWindowsPartA, + describeWindowsPartB, + windowsPartARecord, + windowsServiceStopHooks, + windowsServiceStopPort, + windowsStopAdmission, + windowsStopReason, + windowsStopRemainingMs, +} from "../service/windows-service-stop"; +import type { StopRunRecord } from "./stop-report"; + +const EVIDENCE_SNAPSHOT_TIMEOUT_MS = 5_000; +const SCHEDULE_PID_TIMEOUT_MS = 2_000; +const ORPHAN_DISCOVERY_CAP_MS = 3_000; + +interface WindowsStopState { + platform: NodeJS.Platform; + result: WindowsCheckedStopResult | null; + snapshotless: boolean; + port: number | null; + /** The evidence the last guarded inspection read; the bind reuses it. */ + evidence: { snapshot: readonly WindowsStopProcessEntry[] | null; schedulePid: number | null } | null; + /** Part B's final enumeration, which the guarded manager-state checks share. */ + partBSnapshot: readonly WindowsStopProcessEntry[] | null; +} + +const state: WindowsStopState = { + platform: process.platform, result: null, snapshotless: false, port: null, evidence: null, partBSnapshot: null, +}; + +/** Test-only: simulate the platform the stop wiring acts for, and start from a clean run. */ +export function resetOcxWindowsStopForTests(platform: NodeJS.Platform | null = null): void { + state.platform = platform ?? process.platform; + state.result = null; + state.snapshotless = false; + state.port = null; + state.evidence = null; + state.partBSnapshot = null; +} + +function windows(): boolean { + return state.platform === "win32"; +} + +function begin(snapshotless: boolean): void { + state.result = null; + state.snapshotless = snapshotless; + state.port = null; + state.partBSnapshot = null; +} + +function annotate(nonce: string | undefined): void { + const record = state.result ? windowsPartARecord(state.result) : null; + if (!nonce || !record) return; + try { + recordPendingTeardownPartA(nonce, record); + } catch { + // Without the record the route runs its own capped Part A, which is today's behavior. + } +} + +/** A stop that never reaches the service manager (a sibling, an already-stopped client). */ +export function noServiceStop(): ServiceStopOutcome { + begin(false); + return "absent"; +} + +/** The snapshotless service stop: Part A on Windows, today's manager stop elsewhere. */ +export async function stopServiceForOcxStop(io: { + receiptNonce: () => string | undefined; + inheritedReceipts: number; +}): Promise { + begin(true); + if (!windows()) return stopServiceIfInstalledDetailed(); + state.port = windowsServiceStopPort(); + const report = await stopServiceIfInstalledReport({ + admission: windowsStopAdmission({ orphanDiscovery: readPid() === null, inheritedReceipts: io.inheritedReceipts }), + }, state.platform); + state.result = report.windows; + annotate(io.receiptNonce()); + return report.outcome; +} + +/** The record a receipt claimed after Part A carries (null before Part A or off Windows). */ +export function windowsPartAReceiptRecord(): PendingTeardownPartA | null { + return state.result ? windowsPartARecord(state.result) : null; +} + +/** Part A held the child stop; handleStopUnlocked skips the POST branch entirely. */ +export function windowsChildStopHeld(): boolean { + const outcome = state.result?.outcome; + return windows() && state.snapshotless && !!outcome && outcome !== "stopped" && outcome !== "absent"; +} + +/** ocx uninstall's service stop: Part A on Windows (60 s deadline), today's stop elsewhere. */ +export async function stopServiceForUninstall(): Promise { + begin(true); + if (!windows()) return stopServiceIfInstalledDetailed(); + state.port = windowsServiceStopPort(); + const report = await stopServiceIfInstalledReport({ + admission: windowsStopAdmission({ orphanDiscovery: readPid() === null }), + }, state.platform); + state.result = report.windows; + return report.outcome; +} + +/** + * ocx uninstall's restart-window check after the service was removed: Part B on Windows + * after a Part A stopped, the legacy re-poll otherwise. Throws when the window fails; a + * port held by another program does not stop an uninstall. + */ +export async function verifyUninstallStopWindow(): Promise { + if (!windowsPartBDue()) { + const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); + if (survivor) throw new Error(`a proxy is still listening on port ${survivor.port} after the service was removed; it is being respawned`); + return; + } + const result = await confirmWindowsServiceStop({ port: state.port ?? windowsServiceStopPort(), remainingMs: windowsStopRemainingMs() }); + if (result.outcome === "freed") return; + if (result.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(result)); + return; + } + throw new Error(describeWindowsPartB(result)); +} + +/** Snapshotless Windows only: Part A stopped, so Part B confirms the port afterwards. */ +export function windowsPartBDue(): boolean { + return windows() && state.snapshotless && state.result?.outcome === "stopped"; +} + +/** + * A Part A result other than stopped/absent holds the child stop (030 "Two parts"): no + * POST, the shared teardown skipped, proxy unknown. Records the additive reason. + */ +export function windowsPartAHolds(record: StopRunRecord): boolean { + const result = state.result; + if (!windows() || !result || result.outcome === "stopped" || result.outcome === "absent") return false; + const reason = windowsStopReason(result); + if (reason) record.serviceStopReason = reason; + console.error(" " + describeWindowsPartA(result)); + if (!state.snapshotless) return true; + record.proxy = "unknown"; + console.error(" The proxy was left running and the shared teardown skipped: the service could relaunch it."); + return true; +} + +/** Orphan discovery after a checked Windows stop is capped at 3 s (030 "Time budget"). */ +export function orphanDiscoveryCap(): { deadlineAt?: number } { + return windowsPartBDue() ? { deadlineAt: Date.now() + ORPHAN_DISCOVERY_CAP_MS } : {}; +} + +/** + * After the child stop: true when the stop has to fail and hold the shared teardown. + * Windows runs Part B; a legacy respawnable stop keeps the old re-poll. + */ +export async function stopWindowFailed(record: StopRunRecord): Promise { + if (!windowsPartBDue()) { + const survivor = await proxyStillLiveAfterStop({ canRespawn: true }); + if (!survivor) return false; + record.proxy = "respawned"; + console.error("❌ A proxy is still listening on port " + survivor.port + " after the service stop; it is being respawned."); + console.error(" Skipping shared teardown: restoring client config while the proxy runs leaves both pointing at each other."); + return true; + } + const result = await confirmWindowsServiceStop({ port: state.port ?? windowsServiceStopPort(), remainingMs: windowsStopRemainingMs() }); + record.portHeldByForeign = result.portHeldByForeign; + if (result.outcome === "freed") return false; + if (result.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(result)); + return false; + } + if (result.outcome === "survived") record.proxy = "respawned"; + else { + record.proxy = "unknown"; + record.serviceStopReason = "evidence-unreadable"; + } + console.error("❌ " + describeWindowsPartB(result)); + console.error(" Skipping shared teardown: restoring client config while the proxy runs leaves both pointing at each other."); + return true; +} + +async function readEvidence(): Promise> { + const hooks = windowsServiceStopHooks(); + const [snapshot, schedulePid] = await Promise.all([ + hooks?.snapshot ? hooks.snapshot() : windowsProcessSnapshot({ timeoutMs: EVIDENCE_SNAPSHOT_TIMEOUT_MS }), + hooks?.schedulePid ? hooks.schedulePid() : readScheduleServicePid({ timeoutMs: SCHEDULE_PID_TIMEOUT_MS }), + ]); + return { snapshot, schedulePid }; +} + +/** + * The guarded manager inspection. On Windows the process table is the evidence snapshot + * with creation times, read once here and kept: Part A binds against exactly this table. + */ +export async function inspectGuardedTarget( + approvedPid: number, + approvedPort: number, + inspect: typeof inspectGuardedManagerTarget = inspectGuardedManagerTarget, +): Promise { + if (!windows()) return inspect(approvedPid, approvedPort); + const evidence = await readEvidence(); + state.evidence = evidence; + return inspect(approvedPid, approvedPort, { + win: { + winProcs: () => (evidence.snapshot ? [...evidence.snapshot] : null), + winSchedulePid: () => evidence.schedulePid, + }, + }); +} + +/** The guarded stopManager: Part A on Windows, binding on the revalidation's snapshot. */ +export async function stopGuardedManager(io: { + approvedPort: number; + receiptNonce: () => string | undefined; + inheritedReceipts: number; +}): Promise { + begin(false); + if (!windows()) return stopServiceIfInstalledDetailed(); + state.port = io.approvedPort; + const report = await stopServiceIfInstalledReport({ + admission: windowsStopAdmission({ orphanDiscovery: false, inheritedReceipts: io.inheritedReceipts }), + ...(state.evidence?.snapshot ? { bindSnapshot: state.evidence.snapshot } : {}), + }, state.platform); + state.result = report.windows; + annotate(io.receiptNonce()); + return report.outcome; +} + +/** The guarded Part B, after settle, on the approved endpoint. Off Windows: nothing to do. */ +export async function confirmGuardedFreed(record: StopRunRecord): Promise { + if (!windows() || state.snapshotless || state.result?.outcome !== "stopped" || state.port === null) return true; + const result = await confirmWindowsServiceStop({ + port: state.port, + remainingMs: windowsStopRemainingMs(), + snapshot: async timeoutMs => { + const snapshot = await windowsProcessSnapshot({ timeoutMs }); + state.partBSnapshot = snapshot; + return snapshot; + }, + }); + record.portHeldByForeign = result.portHeldByForeign; + if (result.outcome === "freed") return true; + if (result.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(result)); + return true; + } + if (result.outcome !== "survived") record.serviceStopReason = "evidence-unreadable"; + console.error("❌ " + describeWindowsPartB(result)); + return false; +} + +/** The guarded manager-state checks; on Windows they share Part B's final enumeration. */ +export function observeGuardedStopped( + manager: Exclude, + observe: typeof observeGuardedManagerStopped = observeGuardedManagerStopped, +): ReturnType { + const snapshot = windows() ? state.partBSnapshot : null; + return snapshot ? observe(manager, { win: { winProcs: () => [...snapshot] } }) : observe(manager); +} diff --git a/src/cli/uninstall-plan.ts b/src/cli/uninstall-plan.ts index 4bfb9d4a7a6..c6f984b70a7 100644 --- a/src/cli/uninstall-plan.ts +++ b/src/cli/uninstall-plan.ts @@ -24,22 +24,32 @@ export type UninstallObservation = { /** Service removal outcome, or null when the step threw. */ serviceRemoval: "absent" | "removed" | "failed" | null; /** - * For a Task Scheduler backend: was the restart window verified AFTER removal? + * Was the restart window verified AFTER removal? Needed after an older CLI's + * `stopped-respawnable` and, on Windows, after every `stopped`: there it is Part B of the + * checked stop (#7771), which proves the port stays free and no wrapper came back. * * Deleting the registration does not prove an already-running `:loop` wrapper died — - * killing it is best-effort (#764). `ocx stop` polls across the window; uninstall has - * to do the same before it may take shared config down. + * `ocx stop` confirms across the window; uninstall has to do the same before it may take + * shared config down (#764). */ respawnWindowVerified: boolean; }; -export function sharedTeardownAuthorized(o: UninstallObservation): boolean { +/** Whether the restart window must be verified before shared config may come down. */ +export function respawnWindowRequired( + o: Pick, + platform: NodeJS.Platform = process.platform, +): boolean { + return o.serviceStop === "stopped-respawnable" || (platform === "win32" && o.serviceStop === "stopped"); +} + +export function sharedTeardownAuthorized(o: UninstallObservation, platform: NodeJS.Platform = process.platform): boolean { if (o.serviceStop === null) return false; // "absent" and a clean stop are the only service states that prove nothing is managing // the proxy. if (o.serviceStop === "failed" || o.serviceStop === "state-unknown") return false; // Removing the registration is not the same as proving the running wrapper is gone. - if (o.serviceStop === "stopped-respawnable" && !o.respawnWindowVerified) return false; + if (respawnWindowRequired(o, platform) && !o.respawnWindowVerified) return false; if (o.serviceRemoval === null || o.serviceRemoval === "failed") return false; return o.proxyProvenDown; } diff --git a/src/config/pending-teardown.ts b/src/config/pending-teardown.ts index 5632d0739e5..ea29289bd5c 100644 --- a/src/config/pending-teardown.ts +++ b/src/config/pending-teardown.ts @@ -63,6 +63,20 @@ export type PendingTeardownReceipt = { * replacement ever comes up, the next `ocx stop` finishes it like any abandoned receipt. */ handover?: "update-restart"; + /** + * Windows checked stop (#7771): what the owner's Part A did before it sent the stop. + * /api/stop skips its own Part A only for a receipt carrying this record whose manager + * is still the registered one; a receipt without it (an older CLI, or one written + * before Part A) gets the route's own capped Part A. Readers that predate it ignore it. + */ + partA?: PendingTeardownPartA; +}; + +/** The Part A record of a teardown receipt: the outcome, the manager, the ended wrappers. */ +export type PendingTeardownPartA = { + partA: "stopped" | "absent"; + manager: { scheduler: boolean; winsw: boolean }; + wrappers: { pid: number; startTime: number }[]; }; /** @@ -130,6 +144,7 @@ export function claimPendingTeardown( endpointSource: "exact" | "guessed", ownerPid: number = process.pid, handover?: "update-restart", + partA?: PendingTeardownPartA | null, ): PendingTeardownReceipt { const dir = getConfigDir(); assertNotRealHomeUnderTest(dir); @@ -137,11 +152,44 @@ export function claimPendingTeardown( const receipt: PendingTeardownReceipt = { ownerPid, nonce, createdAt: new Date().toISOString(), endpoint, endpointSource, ...(handover ? { handover } : {}), + ...(partA ? { partA } : {}), }; atomicWriteFile(pendingTeardownPathFor(nonce), JSON.stringify(receipt, null, 2) + "\n"); return receipt; } +/** + * Add the Part A record to a receipt this process claimed before Part A ran (the guarded + * stop and an update handover claim first). Only the owner's own valid receipt is + * rewritten; anything else is left alone and reported as false. + */ +export function recordPendingTeardownPartA(nonce: string, partA: PendingTeardownPartA, ownerPid: number = process.pid): boolean { + const read = readPendingTeardown(nonce); + if (read.state !== "valid" || read.receipt.ownerPid !== ownerPid) return false; + assertNotRealHomeUnderTest(getConfigDir()); + atomicWriteFile(pendingTeardownPathFor(nonce), JSON.stringify({ ...read.receipt, partA }, null, 2) + "\n"); + return true; +} + +/** The Part A record of the receipt a request names, when it is present and well formed. */ +export function pendingTeardownPartA(nonce: string | null): PendingTeardownPartA | null { + if (!nonce || !NONCE_RE.test(nonce)) return null; + const read = readPendingTeardown(nonce); + if (read.state !== "valid") return null; + const record = read.receipt.partA as unknown; + if (!record || typeof record !== "object") return null; + const value = record as Record; + const manager = value.manager as Record | undefined; + const ok = (value.partA === "stopped" || value.partA === "absent") + && !!manager && typeof manager === "object" + && typeof manager.scheduler === "boolean" && typeof manager.winsw === "boolean" + && Array.isArray(value.wrappers) + && value.wrappers.every(entry => !!entry && typeof entry === "object" + && Number.isSafeInteger((entry as { pid?: unknown }).pid) + && Number.isFinite((entry as { startTime?: unknown }).startTime)); + return ok ? read.receipt.partA! : null; +} + export function readPendingTeardown(nonce: string): PendingTeardownRead { if (!NONCE_RE.test(nonce)) return { state: "missing" }; let raw: string; diff --git a/src/lib/process-control.ts b/src/lib/process-control.ts index d2fe5b7471f..6677f441a47 100644 --- a/src/lib/process-control.ts +++ b/src/lib/process-control.ts @@ -61,6 +61,11 @@ export interface GracefulStopIo { * endpoint the stop never contacted — and recovery probes exactly that endpoint. */ runtimeEndpoint?: { hostname: string; port: number }; + /** + * Windows checked stop (#7771): Part B confirms the port after this stop returns and + * owns the wait, so the 15 s waitForStoppedPort here is skipped (030 "Time budget"). + */ + skipPortWait?: boolean; } /** @@ -319,7 +324,7 @@ function drainDeadlineMs(): number { /** Graceful-first stop: management-API drain, then the platform kill ladder. */ export async function stopProxy(pid: number, io: GracefulStopIo = {}): Promise { if (!isProcessAlive(pid)) return false; - const runtime = io.runtimeEndpoint ?? readRuntimePort(pid); + const runtime = io.skipPortWait ? null : io.runtimeEndpoint ?? readRuntimePort(pid); const { result: graceful, refusal } = await stopProxyGracefullyDetailed(pid, io); if (graceful === "refused") { // The proxy refused on purpose. Forcing would strip shared config while whatever owns @@ -367,6 +372,9 @@ async function waitForStoppedPort( } } +/** taskkill in the forced fallback; it had no timeout before #7771. */ +export const FORCED_TASKKILL_TIMEOUT_MS = 5_000; + export function killProxy(pid: number, mayTerminate?: () => boolean): void { if (!isProcessAlive(pid)) return; const authorize = () => { @@ -381,7 +389,8 @@ export function killProxy(pid: number, mayTerminate?: () => boolean): void { const args = ["/PID", String(pid), "/T", "/F"]; authorize(); try { - execFileSync(taskkill, args, { stdio: "pipe", windowsHide: true }); + // Capped (#7771): the checked stop's budget counts the forced kill at 5 s. + execFileSync(taskkill, args, { stdio: "pipe", windowsHide: true, timeout: FORCED_TASKKILL_TIMEOUT_MS }); } catch (err) { if (isProcessAlive(pid)) throw err; } diff --git a/src/server/management-api.ts b/src/server/management-api.ts index b397ab53934..4b319e7c92b 100644 --- a/src/server/management-api.ts +++ b/src/server/management-api.ts @@ -495,7 +495,9 @@ export async function handleManagementAPI( } let serviceStop: import("../service").ServiceStopOutcome; try { - serviceStop = sibling ? "absent" : stopServiceIfInstalledDetailed(); + // #7771: on Windows the route skips Part A for a receipt that records it, caps it at + // 6 s for one that does not, and sweeps a deleted task's stray wrapper in 4 s. + serviceStop = sibling ? "absent" : await (await import("./stop-service-route")).routeServiceStop(url, holdsReceipt); } catch (err) { if (isServiceOwnershipError(err)) { // The installed service belongs to another CODEX_HOME/OPENCODEX_HOME: it would respawn diff --git a/src/server/stop-service-route.ts b/src/server/stop-service-route.ts new file mode 100644 index 00000000000..4945940fa54 --- /dev/null +++ b/src/server/stop-service-route.ts @@ -0,0 +1,85 @@ +/** + * The service stop inside POST /api/stop (#7771, 030 "/api/stop"). + * + * The route runs inside the proxy that holds the port, so it can never confirm the port + * itself; Part B belongs to the parent ocx stop. On Windows the route splits by case: + * + * - a receipt whose Part A record names the manager that is still registered: the parent + * already ran Part A, so the route skips it (no second enumeration, /end or wait); + * - a receipt without that record (an older CLI, or one claimed before Part A): today's + * behavior, the service stop, now Part A capped at 6 s; an overrun is + * service_state_unknown; + * - no receipt (the caller already refused a respawnable task, so nothing is registered + * with the Task Scheduler): a registered WinSW service keeps today's in-process stop, + * because WinSW's own stop ends this very process and cannot be capped from inside it; + * otherwise Part A's stray sweep for a deleted task's wrapper, capped at 4 s, so the + * route still answers inside the parent's 10 s POST abort. + * + * Off Windows it is today's stopServiceIfInstalledDetailed. + */ +import { pendingTeardownPartA, type PendingTeardownPartA } from "../config/pending-teardown"; +import type { ServiceStopOutcome } from "../service/orchestration"; + +export const ROUTE_PART_A_CAP_MS = 6_000; +export const ROUTE_STRAY_SWEEP_CAP_MS = 4_000; +const REGISTRATION_PROBE_MS = 2_000; + +export interface RouteServiceStopIo { + platform?: NodeJS.Platform; + stop?: (request: { capMs?: number; allowUnderOwnWrapper?: boolean }) => Promise; + assertEnvironment?: () => void; + readRecord?: (nonce: string | null) => PendingTeardownPartA | null; + registration?: () => Promise<{ scheduler: "present" | "absent" | "unknown"; winsw: "started" | "stopped" | "nonexistent" | "unknown" }>; + winswRegistered?: () => boolean; + stopWinsw?: () => void; +} + +async function liveRegistration(): Promise>>> { + const { windowsServiceStopHooks } = await import("../service/windows-service-stop"); + const hooked = windowsServiceStopHooks()?.registration; + if (hooked) return hooked(); + const { windowsCheckedStopLiveDeps } = await import("../service/windows-checked-stop-live"); + const deps = windowsCheckedStopLiveDeps(); + const [scheduler, winsw] = await Promise.all([ + Promise.resolve(deps.probeScheduler(REGISTRATION_PROBE_MS)).then(probe => probe.status, () => "unknown" as const), + Promise.resolve(deps.probeWinsw(REGISTRATION_PROBE_MS)).catch(() => "unknown" as const), + ]); + return { scheduler, winsw }; +} + +/** The record still describes this machine: the same managers are registered. */ +export function partARecordMatchesRegistration( + record: PendingTeardownPartA, + current: Awaited>>, +): boolean { + if (current.scheduler === "unknown" || current.winsw === "unknown") return false; + return record.manager.scheduler === (current.scheduler === "present") + && record.manager.winsw === (current.winsw !== "nonexistent"); +} + +export async function routeServiceStop(url: URL, holdsReceipt: boolean, io: RouteServiceStopIo = {}): Promise { + const service = await import("../service"); + const stop = io.stop ?? (request => service.stopServiceIfInstalledDetailed(request)); + if ((io.platform ?? process.platform) !== "win32") return stop({}); + // The ownership check every service stop ran before still runs first. + (io.assertEnvironment ?? service.assertServiceEnvironmentMatchesInstall)(); + if (holdsReceipt) { + const record = (io.readRecord ?? pendingTeardownPartA)(url.searchParams.get("teardownNonce")); + if (record && partARecordMatchesRegistration(record, await (io.registration ?? liveRegistration)())) { + return record.partA; + } + // The route runs inside the proxy its wrapper supervises: ending that wrapper is the point. + return stop({ capMs: ROUTE_PART_A_CAP_MS, allowUnderOwnWrapper: true }); + } + const winsw = await import("../lib/winsw"); + const winswRegistered = io.winswRegistered ?? (() => winsw.statusWinswRaw() !== "nonexistent"); + if (winswRegistered()) { + try { + (io.stopWinsw ?? (() => winsw.stopWinswService()))(); + return "stopped"; + } catch { + return "failed"; + } + } + return stop({ capMs: ROUTE_STRAY_SWEEP_CAP_MS, allowUnderOwnWrapper: true }); +} diff --git a/src/service.ts b/src/service.ts index ceb10e010de..da13da0eb5a 100644 --- a/src/service.ts +++ b/src/service.ts @@ -25,13 +25,13 @@ export { decodeSchtasksOutput, setQuerySchtasksForTests, formatWindowsSchedulerS export type { WindowsSchedulerXmlState } from "./service/windows-taskxml"; export { buildWindowsServiceScript, buildWindowsSchtasksCreateArgs, buildWindowsSchtasksCreateArgsForXml, buildWindowsLauncherVbs, buildWindowsTaskXml, buildWindowsTaskXmlDocument, windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy, readWindowsSchedulerXmlState } from "./service/windows-taskxml"; export type { WindowsSchedulerRegistrationStageDeps, FreshWindowsSchedulerRegistrationDeps, RemoveNativeWindowsServiceDeps } from "./service/windows-ops"; -export { windowsListenPort, winswListenPort, writeServiceDefinitionFile, definitionCarriesCredential, stageWindowsSchedulerRegistrationXml, stageElevatedSchedulerRegistration, describeElevatedRegistrationFailure, registerFreshWindowsSchedulerTask, removeNativeWindowsServiceForScheduler, assertWindowsNativeServiceAccountSupported, isWindowsSchedulerEndBenign, stopWindows, stopWindowsChecked, classifyWindowsServiceStop } from "./service/windows-ops"; +export { windowsListenPort, winswListenPort, writeServiceDefinitionFile, definitionCarriesCredential, stageWindowsSchedulerRegistrationXml, stageElevatedSchedulerRegistration, describeElevatedRegistrationFailure, registerFreshWindowsSchedulerTask, removeNativeWindowsServiceForScheduler, assertWindowsNativeServiceAccountSupported, isWindowsSchedulerEndBenign, stopWindows } from "./service/windows-ops"; export type { ServiceRepairVerb, RepairServiceDeps } from "./service/repair"; export { repairService, foreignServiceOwnerRefusal, unknownServiceOwnerRefusal } from "./service/repair"; export type { ServiceChildOwnershipDecision } from "./service/service-child-ownership"; export { serviceChildOwnershipDecision, serviceChildStayOutExitCode } from "./service/service-child-ownership"; export type { ServiceInstallPreparationDeps, FreshWindowsSchedulerInstallDeps, ServiceStopOutcome, ServiceUninstallOutcome } from "./service/orchestration"; -export { proxyStillLiveAfterStop, prepareServiceInstall, installServiceSafely, installFreshWindowsSchedulerSafely, installedServiceRespawnRisk, stopServiceIfInstalledDetailed, setUninstallServiceHooksForTests, uninstallServiceDetailed, uninstallServiceIfInstalled, isServiceInstalled, isServiceViable } from "./service/orchestration"; +export { proxyStillLiveAfterStop, prepareServiceInstall, installServiceSafely, installFreshWindowsSchedulerSafely, installedServiceRespawnRisk, stopServiceIfInstalledDetailed, stopServiceIfInstalledReport, setUninstallServiceHooksForTests, uninstallServiceDetailed, uninstallServiceIfInstalled, isServiceInstalled, isServiceViable } from "./service/orchestration"; export type { ServiceDiagnostic, WindowsTaskDiagnosticIdentityDeps, WindowsServiceDiagnosticInputs, LaunchdServiceDiagnosticInputs } from "./service/diagnostics"; export { bakedServicePathsDiagnostic, serviceStartableFromTray, resolveWindowsTaskDiagnosticUserId, deriveWindowsServiceDiagnostic, deriveWindowsServiceDiagnosticForCurrentUser, deriveLaunchdServiceDiagnostic, diagnoseService, serviceStatusSummary, serviceStatusReport } from "./service/diagnostics"; export type { ParsedServiceArgs, ServiceInstallationState, ServiceInstallationProbe, ServiceInstallationProbeHooks, ServiceCommandPlan } from "./service/cli"; diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts new file mode 100644 index 00000000000..13f3ff8beee --- /dev/null +++ b/src/service/cli-windows-stop.ts @@ -0,0 +1,133 @@ +/** + * ocx service stop / uninstall / remove on Windows (#7771, 030 "Where it sits"). + * + * - stop: Part A instead of stopWindows; a result other than stopped/absent refuses and + * leaves the proxy alone. Then a teardown receipt is claimed before the tracked proxy is + * stopped, so the proxy defers the shared teardown to this command. When the claim fails + * while this home owns the shared teardown, no undeferred child stop is sent: a wrapper + * appearing later could relaunch the proxy after its own teardown ran. Part B replaces + * proxyStillLiveAfterStop after a Part A stopped; the receipt is completed only after the + * restore ran, and kept for recovery otherwise. + * - uninstall / remove: Part A before the task is deleted (a looping wrapper used to + * outlive its task, #764), refusing on anything but stopped/absent, then the tracked + * proxy stop and Part B. + */ +import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-teardown"; +import { readRuntimePort } from "../config/process-state"; +import { loadConfig } from "../config"; +import { readClientConnectionState } from "../client/state"; +import { proxyStillLiveAfterStop, stopTrackedProxyForServiceCommand } from "./orchestration"; +import { + confirmWindowsServiceStop, + describeWindowsPartA, + describeWindowsPartB, + runWindowsServiceStopPartA, + windowsPartARecord, + windowsServiceStopPort, + windowsStopAdmission, + windowsStopRemainingMs, +} from "./windows-service-stop"; + +export interface WindowsServiceStopCommandIo { + ownsSharedTeardown?: () => boolean; + claim?: typeof claimPendingTeardown; + stopTracked?: typeof stopTrackedProxyForServiceCommand; + survivor?: typeof proxyStillLiveAfterStop; +} + +function receiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { + const runtime = readRuntimePort(); + if (runtime?.port) return { endpoint: { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port }, source: "exact" }; + try { + const config = loadConfig(); + return { endpoint: { hostname: config.hostname ?? "127.0.0.1", port: typeof config.port === "number" && config.port > 0 ? config.port : 10100 }, source: "guessed" }; + } catch { + return { endpoint: { hostname: "127.0.0.1", port: 10100 }, source: "guessed" }; + } +} + +/** Shared teardown belongs to this home unless a client connection owns the routing. */ +function ownsSharedTeardownByDefault(): boolean { + try { + return readClientConnectionState().kind !== "connected"; + } catch { + return true; + } +} + +function withNonce(nonce: string | undefined): { ok: true; nonce?: string } { + return nonce ? { ok: true, nonce } : { ok: true }; +} + +/** + * Everything ocx service stop does before its restore. ok=false means the command fails + * and restores nothing; nonce is the receipt to complete once the restore succeeded. + */ +export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopCommandIo = {}): Promise<{ ok: boolean; nonce?: string }> { + const port = windowsServiceStopPort(); + const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); + if (partA.outcome !== "stopped" && partA.outcome !== "absent") { + console.error("❌ service stop did not take effect: the Windows service could not be proven stopped (" + partA.outcome + ")."); + console.error(" " + describeWindowsPartA(partA)); + console.error(" The proxy was left running and native Codex was NOT restored."); + return { ok: false }; + } + let nonce: string | undefined; + if ((io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + const { endpoint, source } = receiptEndpoint(); + try { + nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)).nonce; + } catch (error) { + console.error("❌ service stop could not record its deferred-teardown receipt: " + (error instanceof Error ? error.message : String(error))); + console.error(" The proxy was left running: stopping it without the receipt would let it tear down client config while a wrapper can still relaunch it."); + return { ok: false }; + } + } + await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + skipPortWait: partA.outcome === "stopped", + }); + if (partA.outcome === "absent") { + const survivor = await (io.survivor ?? proxyStillLiveAfterStop)(); + if (!survivor) return withNonce(nonce); + console.error("❌ service stop did not take effect: a proxy is still listening on port " + survivor.port + "."); + console.error(" Native Codex was NOT restored. Check for a manually started proxy, then re-run 'ocx service stop'."); + return { ok: false }; + } + const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); + if (partB.outcome === "freed") return withNonce(nonce); + if (partB.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(partB)); + return withNonce(nonce); + } + console.error("❌ service stop did not take effect: " + describeWindowsPartB(partB)); + console.error(" Native Codex was NOT restored; the teardown receipt is kept so a later stop can finish it."); + return { ok: false }; +} + +/** After a successful restore: the obligation the receipt recorded is discharged. */ +export function completeWindowsServiceStopReceipt(nonce: string | undefined, clear: (nonce: string) => boolean = clearPendingTeardown): void { + if (!nonce) return; + if (!clear(nonce)) console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); +} + +/** ocx service uninstall / remove: stop and confirm before the task is deleted. */ +export async function windowsServiceRemovalStop(io: Pick = {}): Promise { + const port = windowsServiceStopPort(); + const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); + if (partA.outcome !== "stopped" && partA.outcome !== "absent") { + console.error("❌ Service uninstall refused: the Windows service could not be proven stopped (" + partA.outcome + "), so the task was not deleted."); + console.error(" " + describeWindowsPartA(partA)); + return false; + } + await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ skipPortWait: partA.outcome === "stopped" }); + if (partA.outcome !== "stopped") return true; + const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); + if (partB.outcome === "freed") return true; + if (partB.outcome === "port-held-by-foreign") { + console.warn("⚠️ " + describeWindowsPartB(partB)); + return true; + } + console.error("❌ Service uninstall refused: " + describeWindowsPartB(partB) + " The task was not deleted."); + return false; +} diff --git a/src/service/cli.ts b/src/service/cli.ts index 2d7f8bd5d6c..19df439f594 100644 --- a/src/service/cli.ts +++ b/src/service/cli.ts @@ -32,6 +32,7 @@ import { runServiceClaim } from "./claim"; import { createSupervisionLatch, desktopServiceCommandRefusal } from "./desktop-command-guard"; import { linuxServiceBootHint } from "./linux-linger"; import { diagnoseDesktopStartup, desktopLoginStatusSummary } from "./desktop-startup"; +import { completeWindowsServiceStopReceipt, windowsServiceRemovalStop, windowsServiceStopBeforeRestore } from "./cli-windows-stop"; /** * `restart` is NO LONGER folded into `repair`. @@ -398,17 +399,26 @@ export async function serviceCommand(...args: (string | undefined)[]): Promise { +/** How the tracked proxy is stopped: the deferral receipt and the Windows port wait. */ +export interface TrackedProxyStopIo { + /** ocx service stop on Windows: the claimed receipt, so the proxy defers its teardown. */ + deferSharedTeardownNonce?: string; + /** Part B follows and owns the port wait (030), so stopProxy skips its own. */ + skipPortWait?: boolean; +} + +async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise { let stopped = false; const pid = readPid(); const trackedKillPid = verifiedKillTarget(pid); if (trackedKillPid !== null && isProcessAlive(trackedKillPid)) { - await stopProxy(trackedKillPid); + await stopProxy(trackedKillPid, io); removePid(trackedKillPid); removeRuntimePort(trackedKillPid); stopped = true; @@ -214,7 +224,7 @@ async function stopTrackedProxyIfRunning(): Promise { }); const liveKillPid = verifiedKillTarget(live?.pid); if (liveKillPid !== null) { - await stopProxy(liveKillPid); + await stopProxy(liveKillPid, io); removePid(liveKillPid); removeRuntimePort(liveKillPid); stopped = true; @@ -224,9 +234,9 @@ async function stopTrackedProxyIfRunning(): Promise { return "none"; } -export async function stopTrackedProxyForServiceCommand(): Promise { +export async function stopTrackedProxyForServiceCommand(io: TrackedProxyStopIo = {}): Promise { try { - return await stopTrackedProxyIfRunning(); + return await stopTrackedProxyIfRunning(io); } catch (err) { console.error(`⚠️ Failed to stop proxy: ${err instanceof Error ? err.message : String(err)}`); return "none"; @@ -236,13 +246,14 @@ export async function stopTrackedProxyForServiceCommand(): Promise ServiceDiagnostic; managerOps?: (backend: ServiceBackend) => ServiceInstallCleanupOps | null; /** - * Task Scheduler only: ends surviving launcher/wrapper processes of THIS - * installation. Scoped to canonical script/launcher tokens, never a bare PID. + * Windows: Part A of the checked stop (#7771) for every registered manager and the + * bound wrappers of this home. Replaces the manager stops plus the best-effort sweep. */ - killServiceWrappers?: () => void; + windowsPartA?: (request: WindowsPartARequest) => Promise; + /** Windows: Part B after the tracked-proxy stop; the port must be free and stay free. */ + windowsPartB?: (request: { port: number; remainingMs: number }) => Promise; stopTrackedProxy?: () => Promise; platform?: NodeJS.Platform; } @@ -258,44 +269,44 @@ export async function prepareServiceInstall( ): Promise { const latch = deps.supervisionLatch ?? createSupervisionLatch(); assertNoDesktopSupervision(deps.inspectSupervision, latch); - const diagnostic = (deps.diagnose ?? diagnoseService)(); const platform = deps.platform ?? process.platform; + if (platform === "win32") { + // Part A stops whichever managers are registered (both on a conflicting install) and + // ends the bound wrappers, or refuses with nothing deleted or replaced (#7771). It + // also covers a registration deleted mid-uninstall that left a detached wrapper + // looping: with nothing registered it binds no orphan, and an unbound survivor + // refuses instead of being killed by path alone. + const partA = await (deps.windowsPartA ?? runWindowsServiceStopPartA)({ + admission: windowsStopAdmission({ orphanDiscovery: true }), + }); + if (partA.outcome !== "stopped" && partA.outcome !== "absent") { + throw new Error(`The installed Windows service could not be proven stopped (${partA.outcome}); nothing was replaced. ${describeWindowsPartA(partA)}`); + } + // Part B owns the port wait after a stopped manager (030 "Time budget"). + await (deps.stopTrackedProxy ?? (() => stopTrackedProxyIfRunning({ skipPortWait: partA.outcome === "stopped" })))(); + if (partA.outcome === "stopped") { + const partB = await (deps.windowsPartB ?? confirmWindowsServiceStop)({ + port: windowsServiceStopPort(), + remainingMs: windowsStopRemainingMs(), + }); + if (partB.outcome !== "freed") { + throw new Error(`The service port is not free after the stop; nothing was published or started. ${describeWindowsPartB(partB)}`); + } + } + return; + } const resolveOps = deps.managerOps ?? platformServiceInstallCleanupOps; const backends: ServiceBackend[] = []; const addBackend = (backend: ServiceBackend) => { if (!backends.includes(backend)) backends.push(backend); }; - - if (platform === "win32") { - // The recorded backend owns the old installation and must be stopped first. - // A conflicting diagnostic means both managers exist, so stop both even when - // the requested backend happens to match the recorded one. - if (diagnostic.backend === "scheduler" || diagnostic.backend === "native") { - addBackend(diagnostic.backend); - if (diagnostic.conflict) addBackend(diagnostic.backend === "scheduler" ? "native" : "scheduler"); - } - addBackend(requestedBackend); - } else { - addBackend(requestedBackend); - } + addBackend(requestedBackend); for (const backend of backends) { const manager = resolveOps(backend); if (!manager) throw new Error(`Background service manager is unavailable for ${backend}.`); if (manager.status() !== null) manager.stop(); } - if (platform === "win32") { - // `schtasks /end` ends the task instance but leaves the cmd `:loop` wrapper - // alive to respawn the proxy mid-install — the same survivor - // `ocx stop` kills via killWindowsServiceWrapperProcesses. Run it - // after every manager has been asked to stop and before the tracked-proxy - // kill so a child respawned in that gap is still caught below. It also runs - // when no manager stop happened: a registration deleted mid-uninstall can - // leave the detached wrapper looping. A manager stop that refused propagates - // above before this runs, so the respawner of a still-running service is - // never killed. - (deps.killServiceWrappers ?? killWindowsServiceWrapperProcesses)(); - } await (deps.stopTrackedProxy ?? stopTrackedProxyIfRunning)(); } @@ -514,11 +525,12 @@ export function installedServiceRespawnRisk( * install (#3008). */ /** - * `stopped-respawnable` is Task Scheduler specifically: `schtasks /end` ends the task + * `stopped-respawnable` was Task Scheduler specifically: `schtasks /end` ends the task * instance while the `cmd :loop` wrapper survives and respawns its child seconds later - * (#764). Only that backend needs the restart-window wait — launchd, systemd and WinSW - * are down when they report stopped, and making them pay a seven-second poll would be a - * regression in every ordinary `ocx stop`. + * (#764). Since the checked Windows stop (#7771) no current backend returns it: Windows + * reports `stopped` only after Part A ended the manager and every bound wrapper, and its + * callers then run Part B unconditionally. The value stays for documents and receipts + * written by older CLIs, which callers still read. */ /** * `state-unknown` is kept apart from `failed` because the remedies differ. A manager that @@ -528,46 +540,41 @@ export function installedServiceRespawnRisk( */ export type ServiceStopOutcome = "absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown"; -export function stopServiceIfInstalledDetailed(): ServiceStopOutcome { +/** The outcome plus, on Windows, the Part A result behind it (reason, bound wrappers). */ +export interface ServiceStopReport { + outcome: ServiceStopOutcome; + windows: WindowsCheckedStopResult | null; +} + +/** + * Stop whatever manager is installed. On Windows this is Part A of the checked stop + * (#7771): both backends are probed, the registered managers stopped and the bound + * wrappers ended and waited for, with admission against the caller's deadline. The proxy + * child is left to the caller's graceful stop, and the caller runs Part B after it. + */ +export async function stopServiceIfInstalledReport( + request: WindowsPartARequest = {}, + platform: NodeJS.Platform = process.platform, +): Promise { assertServiceEnvironmentMatchesInstall(); - if (process.platform === "darwin") { + if (platform === "darwin") { if (existsSync(plistPath())) { - try { stopLaunchd(); return "stopped"; } catch { return "failed"; } + try { stopLaunchd(); return { outcome: "stopped", windows: null }; } catch { return { outcome: "failed", windows: null }; } } - } else if (process.platform === "win32") { - // Query BOTH backends regardless of state: a failed switch or stale state can leave - // two managers installed, and either one would respawn the proxy after `ocx stop`. - let stopped = false; - let failed = false; - let schedulerStopped = false; - let stateUnknown = false; - // `probeWindowsSchedulerTask` is tri-state on purpose: a query that THROWS is not the - // same as a task that is absent, and treating it as absent lets a live scheduler - // survive a "successful" stop. - const probe = probeWindowsSchedulerTask(); - if (probe.status === "present") { - if (stopWindowsChecked()) { stopped = true; schedulerStopped = true; } - else failed = true; - } else if (probe.status === "unknown") { - // Not "failed": nothing refused to stop. The query itself could not answer, which is - // a different problem with a different fix. - stateUnknown = true; - } - if (statusWinswRaw() !== "nonexistent") { - try { stopWinswService(); stopped = true; } catch { failed = true; } - } - // `schtasks /end` ends the task instance but the cmd `:loop` wrapper survives and - // respawns its child seconds later (issue #764), resurrecting the proxy during a - // stop or a tray restart. Kill the launcher/wrapper processes outright. - killWindowsServiceWrapperProcesses(); - // A failure on either backend wins: the other one stopping does not make the live one - // safe to update over. - const outcome = classifyWindowsServiceStop({ stopped, failed, schedulerStopped, stateUnknown }); - if (outcome !== "absent") return outcome; - } else if (process.platform === "linux" && isSystemd() && existsSync(unitPath())) { - try { stopSystemd(); return "stopped"; } catch { return "failed"; } + } else if (platform === "win32") { + // Both backends are probed regardless of state (a failed switch can leave two), an + // unreadable probe is state-unknown and never absence, and a wrapper that survives + // `schtasks /end` is ended through the binding rule instead of a path-only sweep. + const windows = await runWindowsServiceStopPartA(request); + return { outcome: serviceStopOutcomeFromWindows(windows), windows }; + } else if (platform === "linux" && isSystemd() && existsSync(unitPath())) { + try { stopSystemd(); return { outcome: "stopped", windows: null }; } catch { return { outcome: "failed", windows: null }; } } - return "absent"; + return { outcome: "absent", windows: null }; +} + +export async function stopServiceIfInstalledDetailed(request: WindowsPartARequest = {}): Promise { + return (await stopServiceIfInstalledReport(request)).outcome; } /** Delete install-state files; stale state would make `ocx update` "reinstall" a service that no longer exists. */ diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts new file mode 100644 index 00000000000..00bb4019eb9 --- /dev/null +++ b/src/service/windows-checked-stop-live.ts @@ -0,0 +1,326 @@ +/** + * Live dependencies of the checked Windows service stop (#7771, 060 "PR2: caller wiring"). + * + * Part A (src/service/windows-checked-stop.ts) and Part B (src/service/windows-stop-confirm.ts) + * take every effect injected. This module is the production side of those seams: capped + * asynchronous schtasks and sc.exe calls, the evidence snapshot with native creation times + * (src/service/windows-process-evidence.ts), the checked terminate, the bind-time exit watch, + * the listening-port sampler and the listener identity check. + * + * The bind snapshot MUST be windowsProcessSnapshot: on x64 it overlays the native creation + * time, which is the exact value the handle terminate compares. A CIM-only snapshot carries + * a millisecond-truncated CIM time, the terminate then answers "reused" and Part A treats a + * live wrapper as exited. + * + * Windows ARM64 ships the STRICT variant (050 "Rollout", step 3): no ARM64 device run has + * happened, so a terminate there is "unavailable" and Part A reports unknown with + * evidence-unreadable instead of killing through the bounded PowerShell fallback. The + * fallback stays in the evidence module, unreachable until the constant below flips after + * the 040 ARM64 device case is run. + */ +import { execFile } from "node:child_process"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; + +import { readPid, verifyPidIdentity } from "../config/process-state"; +import { isProcessAlive } from "../lib/process-control"; +import { isTestHomeGuardArmed } from "../lib/test-home-guard"; +import { resolveTrustedWindowsPowerShellExe, resolveTrustedWindowsSchtasksExe, resolveTrustedWindowsSystemDirectory, toWindowsSchtasksError } from "../lib/windows-elevation"; +import { WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; +import { decodeWindowsTextBytes } from "../lib/windows-text"; +import { scanListenPids } from "../server/port-reclaim"; +import { dropWindowsTcpRowsForLocalPort } from "../server/windows-tcp-drop"; +import { TASK, windowsLauncherVbsPath, windowsServiceScriptPath } from "./state"; +import { ancestorWrapperPids } from "./windows-guarded-manager"; +import type { WindowsCheckedStopDeps, WindowsTaskEndResult, WindowsWinswScmState, WindowsWinswStopResult } from "./windows-checked-stop"; +import { + openWindowsProcessWatch, + readScheduleServicePid, + terminateWindowsProcessBound, + windowsProcessSnapshot, + type WindowsStopProcessEntry, +} from "./windows-process-evidence"; +import { windowsProcessNative, type WindowsProcessNativeApi, type WindowsTerminateResult } from "./windows-process-native"; +import type { WindowsListenerIdentity, WindowsPortSample, WindowsStopConfirmDeps } from "./windows-stop-confirm"; +import { windowsSchedulerCsvIncludesTask } from "./windows-scheduler"; + +/** + * 050: without an ARM64 device run, ARM64 ships strict. Flip only after the 040 ARM64 device + * case passes; the bounded PowerShell terminate in windows-process-evidence.ts then runs. + */ +export const WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED = false; + +export interface WindowsStopCommandResult { + status: number | null; + stdout: string; + stderr: string; + timedOut: boolean; +} + +/** One spawned helper; tests inject it, production runs execFile with a hard timeout. */ +export type WindowsStopCommandRunner = ( + program: "schtasks" | "sc" | "powershell", + args: string[], + timeoutMs: number, +) => Promise; + +function programPath(program: "schtasks" | "sc" | "powershell"): string { + if (program === "schtasks") return resolveTrustedWindowsSchtasksExe(); + if (program === "powershell") return resolveTrustedWindowsPowerShellExe(); + const file = join(resolveTrustedWindowsSystemDirectory(), "sc.exe"); + if (!existsSync(file)) throw new Error("sc.exe was not found in the trusted system directory."); + return file; +} + +export const liveWindowsStopCommandRunner: WindowsStopCommandRunner = (program, args, timeoutMs) => new Promise(resolve => { + // execFile reads a 0 timeout as "no timeout": a deadline-derived fraction is floored, and a + // timeout that floors to 0 spawns nothing (the same rule as terminateWindowsProcessBound). + const timeout = Math.floor(timeoutMs); + if (!(timeout > 0)) { + resolve({ status: null, stdout: "", stderr: "", timedOut: true }); + return; + } + // Task Scheduler and the SCM are machine-global: an armed test process may only observe. + const mutating = program === "schtasks" ? args[0]?.toLowerCase() !== "/query" : program === "sc" && args[0] !== "query"; + if (isTestHomeGuardArmed() && mutating) { + resolve({ status: null, stdout: "", stderr: "refusing a machine-global service mutation from an armed test process", timedOut: false }); + return; + } + let file: string; + try { + file = programPath(program); + } catch (error) { + resolve({ status: null, stdout: "", stderr: error instanceof Error ? error.message : String(error), timedOut: false }); + return; + } + execFile(file, args, { encoding: "buffer", timeout, windowsHide: true, maxBuffer: 1024 * 1024 }, + (error, stdout, stderr) => { + const text = (value: Buffer | string | undefined): string => + value === undefined ? "" : typeof value === "string" ? value : decodeWindowsTextBytes(value); + const failure = error as (NodeJS.ErrnoException & { killed?: boolean; signal?: string | null; code?: number | string }) | null; + resolve({ + status: failure ? (typeof failure.code === "number" ? failure.code : null) : 0, + stdout: text(stdout), + stderr: text(stderr), + timedOut: !!failure?.killed && failure.signal !== null, + }); + }); +}); + +/** schtasks /end, mapped: already stopped is not-running; access denied is denied. */ +export function classifyTaskEnd(result: WindowsStopCommandResult): WindowsTaskEndResult { + if (result.timedOut) return "failed"; + if (result.status === 0) return "ended"; + const error = Object.assign(new Error(result.stderr || result.stdout), { status: result.status, stderr: result.stderr, stdout: result.stdout }); + const detail = (result.stderr + "\n" + result.stdout).toLowerCase(); + if (detail.includes("no running instance") || detail.includes("not currently running") || detail.includes("0x41330")) return "not-running"; + return toWindowsSchtasksError(error, ["/end", "/tn", TASK]).reason === "access-denied" ? "denied" : "failed"; +} + +/** schtasks /query /fo CSV /nh status column. Localized values fall back to PowerShell. */ +export function parseTaskCsvState(csv: string): "running" | "not-running" | "unknown" { + for (const line of csv.split(/\r?\n/)) { + if (!windowsSchedulerCsvIncludesTask(line, TASK)) continue; + const fields = [...line.matchAll(/"([^"]*)"/g)].map(match => match[1]!.trim()); + const status = fields[fields.length - 1] ?? ""; + if (/^running$/i.test(status)) return "running"; + if (/^(ready|disabled|queued)$/i.test(status)) return "not-running"; + return "unknown"; + } + return "unknown"; +} + +/** sc.exe query: the numeric STATE and the 1060 "does not exist" code are not localized. */ +export function parseScQueryState(result: WindowsStopCommandResult): WindowsWinswScmState { + const text = result.stdout + "\n" + result.stderr; + if (/\b1060\b/.test(text)) return "nonexistent"; + if (result.timedOut || result.status !== 0) return "unknown"; + const state = /^\s*STATE\s*:\s*(\d+)/m.exec(text); + if (!state) return "unknown"; + const value = Number(state[1]); + if (value === 1) return "stopped"; + if (value === 4) return "running"; + if (value >= 2 && value <= 7) return "pending"; + return "unknown"; +} + +/** sc.exe stop without waiting: 1062 (not started) is fine, 5 is denied. */ +export function classifyScStop(result: WindowsStopCommandResult): WindowsWinswStopResult { + if (result.timedOut) return "failed"; + const text = result.stdout + "\n" + result.stderr; + if (result.status === 0 || /\b1062\b/.test(text) || /\b1060\b/.test(text)) return "sent"; + if (/\bFAILED 5\b/.test(text) || result.status === 5) return "denied"; + return "failed"; +} + +export function winswStatusFromScState(state: WindowsWinswScmState): WinswStatus { + if (state === "nonexistent") return "nonexistent"; + if (state === "stopped") return "stopped"; + // A pending service is still an active manager: Part A stops it and polls. + if (state === "running" || state === "pending") return "started"; + return "unknown"; +} + +export function windowsStopCanonicalPaths(): string[] { + return [windowsLauncherVbsPath(), windowsServiceScriptPath()]; +} + +/** + * The terminate Part A uses. terminateWindowsProcessBound runs the PowerShell fallback by + * itself whenever the native API is missing (ARM64, or an x64 whose DLL load failed), so the + * strict variant is enforced HERE (060 "ARM64 decision for PR2"): without the native + * primitive the answer is "unavailable" and the fallback is never reached. + */ +export async function windowsStopTerminate( + pid: number, + startTime: number, + timeoutMs: number, + io: { native?: () => WindowsProcessNativeApi | null; terminate?: typeof terminateWindowsProcessBound } = {}, +): Promise { + if ((io.native ?? windowsProcessNative)() === null && !WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED) return { status: "unavailable" }; + return (io.terminate ?? terminateWindowsProcessBound)(pid, startTime, { timeoutMs }); +} + +/** Floor a deadline-derived timeout; null when it floors to 0 (nothing may be spawned). */ +export function wholeTimeoutMs(timeoutMs: number): number | null { + const floored = Math.floor(timeoutMs); + return floored > 0 ? floored : null; +} + +/** The evidence snapshot with a floored timeout; 0 ms reads nothing (null, unreadable). */ +export function windowsStopSnapshot(timeoutMs: number): Promise { + const whole = wholeTimeoutMs(timeoutMs); + return whole === null ? Promise.resolve(null) : windowsProcessSnapshot({ timeoutMs: whole }); +} + +/** + * Does the stopping process descend from a wrapper carrying this home's launcher or + * service-script path? The binding rule excludes the stopper itself but not its ancestors, + * so such a stop would end its own supervision chain (structure/ops/windows-checked-stop.md, + * "Who may run Part A"). + */ +export function stopperUnderHomeWrapper( + processes: readonly WindowsStopProcessEntry[], + canonicalPaths: readonly string[], + selfPid: number = process.pid, +): boolean { + return ancestorWrapperPids(selfPid, processes, canonicalPaths).length > 0; +} + +export interface WindowsCheckedStopLiveOptions { + run?: WindowsStopCommandRunner; + /** Checked before every manager command (the route's cap, the updater's authorizer). */ + gate?: () => boolean; + snapshot?: (timeoutMs: number) => Promise; +} + +/** The identity-checked proxy child, or null. */ +export function windowsStopChildPid(): number | null { + try { + const pid = readPid(); + return pid !== null && verifyPidIdentity(pid) === pid ? pid : null; + } catch { + return null; + } +} + +export function windowsCheckedStopLiveDeps(options: WindowsCheckedStopLiveOptions = {}): WindowsCheckedStopDeps { + const run = options.run ?? liveWindowsStopCommandRunner; + const gate = options.gate ?? (() => true); + const scQuery = async (timeoutMs: number) => parseScQueryState(await run("sc", ["query", WINSW_SERVICE_ID], timeoutMs)); + return { + canonicalPaths: windowsStopCanonicalPaths(), + childPid: windowsStopChildPid(), + async probeScheduler(timeoutMs) { + const started = Date.now(); + const direct = await run("schtasks", ["/query", "/tn", TASK], timeoutMs); + if (direct.status === 0 && direct.stdout.includes(TASK)) return { status: "present" }; + const listing = await run("schtasks", ["/query", "/fo", "CSV"], Math.max(1, timeoutMs - (Date.now() - started))); + if (listing.status !== 0 || listing.timedOut) return { status: "unknown", detail: "the Task Scheduler query did not answer" }; + return windowsSchedulerCsvIncludesTask(listing.stdout, TASK) ? { status: "present" } : { status: "absent" }; + }, + probeWinsw: async timeoutMs => winswStatusFromScState(await scQuery(timeoutMs)), + enumerate: timeoutMs => (options.snapshot ?? windowsStopSnapshot)(timeoutMs), + scheduleServicePid: timeoutMs => { + const whole = wholeTimeoutMs(timeoutMs); + return whole === null ? null : readScheduleServicePid({ timeoutMs: whole }); + }, + async endTask(timeoutMs) { + if (!gate()) return "failed"; + return classifyTaskEnd(await run("schtasks", ["/end", "/tn", TASK], timeoutMs)); + }, + async taskState(timeoutMs) { + const started = Date.now(); + const csv = await run("schtasks", ["/query", "/tn", TASK, "/fo", "CSV", "/nh"], timeoutMs); + const parsed = csv.status === 0 ? parseTaskCsvState(csv.stdout) : "unknown"; + if (parsed !== "unknown") return parsed; + // A localized status word: ask Get-ScheduledTask, whose State is an enum name. + const left = timeoutMs - (Date.now() - started); + if (left <= 0) return "unknown"; + const ps = await run("powershell", ["-NoProfile", "-NoLogo", "-NonInteractive", "-Command", + "(Get-ScheduledTask -TaskName '" + TASK + "' -TaskPath '\\' -ErrorAction Stop).State"], left); + const state = ps.stdout.trim(); + if (/^running$/i.test(state)) return "running"; + if (/^(ready|disabled|queued)$/i.test(state)) return "not-running"; + return "unknown"; + }, + async winswStop(timeoutMs) { + if (!gate()) return "failed"; + return classifyScStop(await run("sc", ["stop", WINSW_SERVICE_ID], timeoutMs)); + }, + winswState: scQuery, + proxyAlive: () => { + const pid = windowsStopChildPid(); + return pid === null ? false : isProcessAlive(pid); + }, + openWatch: (pid, startTime) => openWindowsProcessWatch(pid, startTime), + terminate: (pid, startTime, timeoutMs) => windowsStopTerminate(pid, startTime, timeoutMs), + }; +} + +/** One port sample. A listener whose owner has exited is cleared like reclaimListenPort does. */ +export function sampleWindowsStopPort( + port: number, + io: { scan?: typeof scanListenPids; alive?: (pid: number) => boolean; drop?: (port: number) => unknown; selfPid?: number } = {}, +): WindowsPortSample { + const scan = (io.scan ?? scanListenPids)(port); + if (!scan.ok) return { state: "unknown" }; + const pids = scan.pids.filter(pid => pid !== (io.selfPid ?? process.pid)); + if (pids.length === 0) return { state: "free" }; + const alive = io.alive ?? isProcessAlive; + const live = pids.filter(pid => alive(pid)); + if (live.length === 0) { + // Same clearing the 15 s waitForStoppedPort did: a ghost row left by the exited child. + try { (io.drop ?? dropWindowsTcpRowsForLocalPort)(port); } catch { /* the next sample decides */ } + } + return { state: "listening", ownerPid: live[0] ?? pids[0] ?? null }; +} + +/** ours = an opencodex process (the respawned proxy); exited = its owner is gone. */ +export function identifyWindowsStopListener( + ownerPid: number | null, + io: { alive?: (pid: number) => boolean; verify?: (pid: number) => number | null } = {}, +): WindowsListenerIdentity { + if (ownerPid === null) return "unknown"; + if (!(io.alive ?? isProcessAlive)(ownerPid)) return "exited"; + try { + return (io.verify ?? verifyPidIdentity)(ownerPid) === ownerPid ? "ours" : "foreign"; + } catch { + return "unknown"; + } +} + +export function windowsStopConfirmLiveDeps( + port: number, + remainingMs: number, + options: { snapshot?: (timeoutMs: number) => Promise } = {}, +): WindowsStopConfirmDeps { + return { + port, + remainingMs, + canonicalPaths: windowsStopCanonicalPaths(), + samplePort: samplePort => sampleWindowsStopPort(samplePort), + identifyListener: (_port, ownerPid) => identifyWindowsStopListener(ownerPid), + enumerate: timeoutMs => (options.snapshot ?? windowsStopSnapshot)(timeoutMs), + }; +} diff --git a/src/service/windows-guarded-manager.ts b/src/service/windows-guarded-manager.ts index f74c056e161..f6e929d5855 100644 --- a/src/service/windows-guarded-manager.ts +++ b/src/service/windows-guarded-manager.ts @@ -29,6 +29,7 @@ import { cachedCurrentWindowsIdentity, resolveCurrentWindowsPrincipal, WINDOWS_P import { statusWinswRaw, winswExePath, WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; import type { GuardedManagerStopped, GuardedManagerTarget } from "./guarded-manager-target"; import { TASK, windowsLauncherVbsPath, windowsServiceScriptPath } from "./state"; +import { bindWindowsStopWrappers } from "./windows-stop-binding"; import { probeWindowsSchedulerTask, querySchtasks, type WindowsSchedulerTaskProbe } from "./windows-scheduler"; import { windowsWscript } from "./windows-wscript"; import { windowsTaskRegistrationHealthy, windowsTaskRegistrationRefreshableLegacy } from "./windows-taskxml"; @@ -57,7 +58,14 @@ export interface WindowsWinswServiceInfo { * default implementation exits early off win32, mirroring probeWindowsSchedulerTask. */ export interface WindowsGuardedManagerDeps { + /** + * The table may carry creation times (windowsProcessSnapshot, #7771): only then can a + * surviving wrapper of a not-running task be bound (bound-stray). A table without them + * leaves every such wrapper unreadable, which stays unknown. + */ winProcs?: () => WindowsProcessEntry[] | null; + /** The Task Scheduler service host PID from the SCM, for the binding rule's lineage. */ + winSchedulePid?: () => number | null; winTaskXml?: () => string; winTaskState?: () => WindowsTaskState; /** @@ -257,6 +265,7 @@ function resolveWindowsDeps(deps: WindowsGuardedManagerDeps, live: boolean): Res winScriptPath: deps.winScriptPath ?? windowsServiceScriptPath, winLauncherPath: deps.winLauncherPath ?? windowsLauncherVbsPath, winWinswExePath: deps.winWinswExePath ?? winswExePath, + winSchedulePid: deps.winSchedulePid ?? (() => null), }; } @@ -380,8 +389,42 @@ export function unreadableWrappersOfManager(managerPid: number, processes: reado }).map(entry => entry.pid); } -function unknown(reason: string): GuardedManagerTarget { - return { kind: "unknown", reason }; +function unknown(reason: string, stopReason?: "unbound-wrapper" | "evidence-unreadable"): GuardedManagerTarget { + return stopReason ? { kind: "unknown", reason, stopReason } : { kind: "unknown", reason }; +} + +/** + * A registered task that is not running, with wrappers of this home still alive (#7771, + * 030 "Guarded (consent) path"). The checked stop's binding rule decides: when every + * surviving wrapper is bound (wrapper image, this home's path token, lineage, a creation + * time) the state is bound-stray and reaches Part A; anything it cannot bind stays + * unknown, naming unbound-wrapper or evidence-unreadable. + */ +function bindStrayWrappers( + approvedPid: number, + processes: readonly WindowsProcessEntry[], + strays: readonly number[], + io: ResolvedWindowsDeps, +): GuardedManagerTarget { + const binding = bindWindowsStopWrappers({ + processes: processes.map(entry => ({ ...entry, startTime: (entry as { startTime?: number | null }).startTime ?? null })), + canonicalPaths: wrapperPaths(io), + childPid: approvedPid, + taskRegistered: true, + managerRegistered: true, + scheduleServicePid: io.winSchedulePid(), + }); + if (binding.unreadable.length > 0) { + return unknown("a surviving scheduler wrapper has unreadable evidence", "evidence-unreadable"); + } + const bound = new Set(binding.bound.map(wrapper => wrapper.pid)); + if (binding.unbound.length > 0 || strays.some(pid => !bound.has(pid))) { + return unknown("a surviving scheduler wrapper could not be tied to the registered task", "unbound-wrapper"); + } + return { + kind: "bound-stray", pid: approvedPid, backend: "scheduler", childNeedsSeparateStop: true, + wrappers: binding.bound.map(wrapper => ({ pid: wrapper.pid, startTime: wrapper.startTime })), + }; } function inspectWindowsSchedulerManager( @@ -394,16 +437,20 @@ function inspectWindowsSchedulerManager( const ancestors = ancestorWrapperPids(approvedPid, processes, paths); const strays = wrapperProcessesAlive(processes, paths); if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { - return unknown("a possible scheduler wrapper has an unreadable command line"); + return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } // A registration that cannot run anything owes only the stray-wrapper check. // That covers BOTH a not-running task and a task proven deleted between the // caller's presence probe and this inspection: in either case nothing // registered can respawn the proxy, while a surviving wrapper still can — // schtasks /end would report success on the inert task and leave it alive. - const inertVerdict = (): GuardedManagerTarget => strays.length === 0 + // A task proven deleted keeps today's verdict; only a registered, not-running task can + // own the survivors it left behind. + const inertVerdict = (registered = false): GuardedManagerTarget => strays.length === 0 ? { kind: "absent" } - : unknown("a surviving scheduler wrapper could not be tied to the registered task"); + : registered + ? bindStrayWrappers(approvedPid, processes, strays, io) + : unknown("a surviving scheduler wrapper could not be tied to the registered task", "unbound-wrapper"); const state = io.winTaskState(); if (state === "unknown") { // Missing vs unreadable: the state query also fails for a task deleted @@ -413,7 +460,7 @@ function inspectWindowsSchedulerManager( ? inertVerdict() : unknown("the registered task's running state could not be proven"); } - if (state === "not-running") return inertVerdict(); + if (state === "not-running") return inertVerdict(true); // Only the running path needs the registered definition: an unreadable // registration cannot prove the task is ours, but it must not hold an // otherwise-provable inert verdict unknown. @@ -502,21 +549,21 @@ export function inspectWindowsGuardedManager( const processes = io.winProcs(); if (processes === null) return unknown("the Windows process list could not be read"); if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { - return unknown("a possible scheduler wrapper has an unreadable command line"); + return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } return wrapperProcessesAlive(processes, wrapperPaths(io)).length === 0 ? { kind: "absent" } - : unknown("a surviving scheduler wrapper could not be tied to an installed manager"); + : unknown("a surviving scheduler wrapper could not be tied to an installed manager", "unbound-wrapper"); } // scheduler absent + winsw nonexistent — still have to rule out a detached wrapper. const processes = io.winProcs(); if (processes === null) return unknown("the Windows process list could not be read"); if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { - return unknown("a possible scheduler wrapper has an unreadable command line"); + return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } return wrapperProcessesAlive(processes, wrapperPaths(io)).length === 0 ? { kind: "absent" } - : unknown("a surviving scheduler wrapper could not be tied to an installed manager"); + : unknown("a surviving scheduler wrapper could not be tied to an installed manager", "unbound-wrapper"); } export interface WindowsGuardedStopDeps { diff --git a/src/service/windows-ops.ts b/src/service/windows-ops.ts index 440b267726b..2a92405b39e 100644 --- a/src/service/windows-ops.ts +++ b/src/service/windows-ops.ts @@ -16,10 +16,8 @@ import { OCX_ELEVATED_STAGING_UNREADABLE, runWindowsElevatedScheduledTaskRegistr import { defaultWinswEntry, installWinswService, statusWinswRaw, uninstallWinswService, WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; import { forgetEphemeralSecretDir, forgetEphemeralSecretPath, hardenElevatedStageDir, hardenElevatedStagePath, hardenSecretDir } from "../lib/windows-secret-acl"; import { recordOwnedConfigPath } from "../lib/config-ownership"; -import { killWindowsSchedulerWrappers } from "../lib/windows-service-wrappers"; import { isTestHomeGuardArmed } from "../lib/test-home-guard"; import { writeServiceApiTokenFile } from "./guards"; -import type { ServiceStopOutcome } from "./orchestration"; import { TASK, windowsLauncherVbsPath, windowsTaskXmlPath, serviceStatePath, writeServiceInstallState, serviceSourceDir, cliEntry } from "./state"; import { decodeSchtasksOutput, querySchtasks, schtasks, type WindowsSchedulerTaskProbe, schtasksErrorDetail, probeWindowsSchedulerTask, rollbackWindowsSchedulerTaskOwnedByAttempt } from "./windows-scheduler"; import { buildWindowsServiceScript, buildWindowsSchtasksCreateArgs, buildWindowsSchtasksCreateArgsForXml, buildWindowsLauncherVbs, resolvedWindowsTaskSid, buildWindowsTaskXmlDocument, windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy } from "./windows-taskxml"; @@ -851,42 +849,10 @@ export function stopWindows(beforeMutation?: () => void): void { } } -/** - * `stopWindows` for callers that need to know whether it worked. - * - * The void form swallows a non-benign `/end` failure, which is right for best-effort - * teardown and wrong for deciding whether an update may replace files: a scheduler that - * refused to stop can respawn the proxy on top of a half-written install (#3008). - */ -export function stopWindowsChecked(): boolean { - try { - schtasks(["/end", "/tn", TASK]); - return true; - } catch (error) { - return isWindowsSchedulerEndBenign(error); - } -} - export function statusWindows(): string { try { return schtasks(["/query", "/tn", TASK]); } catch { return ""; } } export function statusWindowsXml(): string { try { return schtasks(["/query", "/tn", TASK, "/xml"]); } catch { return ""; } } -/** - * Best-effort termination of surviving Windows scheduler launcher/wrapper processes. - * `schtasks /end` ends the task instance but often leaves wscript/cmd running the - * `:loop` batch, which brings the proxy back during a stop or restart. - * - * The matching rule — canonical paths of THIS installation, as complete - * command-line tokens — lives in lib/windows-service-wrappers so the update job - * cannot drift away from it again. - */ -export function killWindowsServiceWrapperProcesses(): void { - killWindowsSchedulerWrappers({ - scriptPath: windowsServiceScriptPath(), - launcherPath: windowsLauncherVbsPath(), - }); -} - export function uninstallWindows(): void { const probe = probeWindowsSchedulerTask(TASK); if (probe.status === "present") { @@ -909,22 +875,3 @@ export function uninstallWindows(): void { if (existsSync(windowsLauncherVbsPath())) unlinkSync(windowsLauncherVbsPath()); if (existsSync(windowsTaskXmlPath())) unlinkSync(windowsTaskXmlPath()); } - -/** - * Collapse the Windows backend observations into one outcome. - * - * Extracted so the precedence is testable by calling it. The rule that matters: a readable - * failure outranks an unreadable state, and an unreadable state outranks success — a - * scheduler we cannot see may still respawn the proxy. - */ -export function classifyWindowsServiceStop(o: { - stopped: boolean; - failed: boolean; - schedulerStopped: boolean; - stateUnknown: boolean; -}): ServiceStopOutcome { - if (o.failed) return "failed"; - if (o.stateUnknown) return "state-unknown"; - if (o.stopped) return o.schedulerStopped ? "stopped-respawnable" : "stopped"; - return "absent"; -} diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts new file mode 100644 index 00000000000..ec5148e2085 --- /dev/null +++ b/src/service/windows-service-stop.ts @@ -0,0 +1,258 @@ +/** + * Caller side of the checked Windows service stop (#7771, 030 "Where it sits"). + * + * Every Windows stop path reaches Part A and Part B through this module: ocx stop with and + * without a consent snapshot, ocx uninstall, /api/stop, ocx service stop / uninstall / + * remove, prepareServiceInstall and the updater. It owns: + * + * - the stop deadline (030 "Time budget"): --deadline-ms or OCX_STOP_DEADLINE_MS for + * ocx stop, 60 s otherwise, measured from this process's start (performance.now() counts + * from performance.timeOrigin, the process start, so the deadline is an absolute value on + * the Part A clock); + * - the admission inputs and the mapping of Part A onto ServiceStopOutcome; + * - the Part A record a teardown receipt carries, so /api/stop can skip a second Part A; + * - a hook seam so caller tests run hermetically (no schtasks, PowerShell or FFI). + */ +import { loadConfig } from "../config"; +import type { PendingTeardownPartA } from "../config/pending-teardown"; +import { readRuntimePort } from "../config/process-state"; +import type { WinswStatus } from "../lib/winsw"; +import type { ServiceStopOutcome } from "./orchestration"; +import { guardedStopWindowsServiceChecked, stopWindowsServiceChecked, windowsStopClock, type WindowsCheckedStopDeps, type WindowsCheckedStopOptions, type WindowsCheckedStopResult, type WindowsStopAdmission, type WindowsStopReason } from "./windows-checked-stop"; +import { stopperUnderHomeWrapper, windowsCheckedStopLiveDeps, windowsStopConfirmLiveDeps, type WindowsCheckedStopLiveOptions } from "./windows-checked-stop-live"; +import { windowsListenPort } from "./windows-ops"; +import type { WindowsStopProcessEntry } from "./windows-process-evidence"; +import { confirmWindowsStopFreed, resolveWindowsStopPort, type WindowsStopConfirmResult } from "./windows-stop-confirm"; +import { windowsStopDeadlineAt } from "./windows-stop-deadline"; + +export { parseStopDeadlineMs, resolveStopDeadlineMs, setStopDeadlineMs, STOP_DEADLINE_ENV, windowsStopDeadlineAt } from "./windows-stop-deadline"; + +/** Time left on the stop deadline, measured on the Part A clock. */ +export function windowsStopRemainingMs(now: () => number = windowsStopClock.now): number { + return windowsStopDeadlineAt() - now(); +} + +export function configuredShutdownTimeoutMs(): number | undefined { + try { + const value = loadConfig().shutdownTimeoutMs; + return typeof value === "number" && value > 0 ? value : undefined; + } catch { + return undefined; + } +} + +export function windowsStopAdmission(input: { + orphanDiscovery: boolean; + inheritedReceipts?: number; + partB?: boolean; +}): WindowsStopAdmission { + return { + deadlineAt: windowsStopDeadlineAt(), + budget: { + orphanDiscovery: input.orphanDiscovery, + inheritedReceipts: input.inheritedReceipts ?? 0, + shutdownTimeoutMs: configuredShutdownTimeoutMs(), + partB: input.partB ?? true, + }, + }; +} + +export interface WindowsPartARequest { + admission?: WindowsStopAdmission; + /** The evidence snapshot (windowsProcessSnapshot) a guarded revalidation already read. */ + bindSnapshot?: readonly WindowsStopProcessEntry[]; + /** The updater's authorizer: throws to refuse, at entry and before each terminate. */ + authorize?: () => void | Promise; + /** + * A hard cap (the route: 6 s with a receipt, 4 s for a stray sweep). An overrun answers + * unknown, and nothing further is changed: the manager commands and every terminate are + * gated off once the cap has passed. + */ + capMs?: number; + /** + * The stopping process may descend from this home's wrapper. Only the /api/stop route + * (it IS the wrapped proxy) and the updater (its worker descends from the service-run + * proxy) pass true; every interactive caller refuses such a stop with nothing changed. + */ + allowUnderOwnWrapper?: boolean; +} + +export interface WindowsPartBRequest { + port: number; + remainingMs: number; + /** The final enumeration; the guarded path wraps it to share the table it read. */ + snapshot?: (timeoutMs: number) => Promise; +} + +export interface WindowsServiceStopHooks { + partA?: (request: WindowsPartARequest) => Promise; + /** Run the real Part A against these deps instead of the live ones. */ + liveDeps?: (options: WindowsCheckedStopLiveOptions) => WindowsCheckedStopDeps; + partB?: (request: WindowsPartBRequest) => Promise; + /** The evidence snapshot the guarded inspection and the bind share. */ + snapshot?: () => Promise; + schedulePid?: () => Promise; + /** The route's registration re-check against a receipt's Part A record. */ + registration?: () => Promise<{ scheduler: "present" | "absent" | "unknown"; winsw: WinswStatus }>; +} + +let hooks: WindowsServiceStopHooks | null = null; + +/** Test-only: replace Part A / Part B / the evidence reads with hermetic fakes. */ +export function setWindowsServiceStopHooksForTests(next: WindowsServiceStopHooks | null): void { + hooks = next; +} + +export function windowsServiceStopHooks(): WindowsServiceStopHooks | null { + return hooks; +} + +function unknownResult(detail: string): WindowsCheckedStopResult { + return { + outcome: "unknown", reason: "evidence-unreadable", detail, + registration: { scheduler: "unknown", winsw: "unknown" }, + bound: [], unbound: [], terminated: [], changed: false, + }; +} + +export const UNDER_OWN_WRAPPER_DETAIL = "this stop runs under this home's service wrapper; run it from a separate terminal"; + +/** Part A with the live dependencies (or the test hook), optionally capped. */ +export async function runWindowsServiceStopPartA(request: WindowsPartARequest = {}): Promise { + let expired = false; + let underOwnWrapper = false; + const options: WindowsCheckedStopOptions = { + ...(request.admission ? { admission: request.admission } : {}), + ...(request.bindSnapshot ? { bindSnapshot: request.bindSnapshot } : {}), + }; + const run = async (): Promise => { + if (hooks?.partA) return hooks.partA(request); + const live: WindowsCheckedStopLiveOptions = { gate: () => !expired }; + const base = (hooks?.liveDeps ?? windowsCheckedStopLiveDeps)(live); + const selfPid = base.selfPid ?? process.pid; + const guard = (table: readonly WindowsStopProcessEntry[] | null): boolean => + !request.allowUnderOwnWrapper && table !== null && stopperUnderHomeWrapper(table, base.canonicalPaths, selfPid); + // The guarded path binds on a snapshot it already read: check it before anything runs. + if (request.bindSnapshot && guard(request.bindSnapshot)) return unknownResult(UNDER_OWN_WRAPPER_DETAIL); + // Otherwise the bind enumeration is checked as it arrives; a hit reads as an unreadable + // table, which Part A answers before /end (registered) or as absent (nothing to end). + const deps = { + ...base, + enumerate: async (timeoutMs: number) => { + const table = await base.enumerate(timeoutMs); + if (!guard(table)) return table; + underOwnWrapper = true; + return null; + }, + }; + const authorize = async (): Promise => { + if (expired) throw new Error("the checked stop's cap has passed"); + await request.authorize?.(); + }; + const result = await (request.authorize || request.capMs !== undefined + ? guardedStopWindowsServiceChecked(authorize, deps, options) + : stopWindowsServiceChecked(deps, options)); + if (underOwnWrapper && result.outcome === "unknown") result.detail = UNDER_OWN_WRAPPER_DETAIL; + return result; + }; + if (request.capMs === undefined) return run(); + const work = run().catch(error => { + if (error && typeof error === "object" && "partial" in error) return (error as { partial: WindowsCheckedStopResult }).partial; + throw error; + }); + let timer: ReturnType | undefined; + const cap = new Promise(resolve => { timer = setTimeout(() => resolve(null), request.capMs); }); + try { + const result = await Promise.race([work, cap]); + if (result === null) { + expired = true; + work.catch(() => undefined); + return unknownResult("the checked stop did not finish within its cap"); + } + return result; + } finally { + if (timer) clearTimeout(timer); + } +} + +/** Part A onto the existing ServiceStopOutcome (030 "Outcomes"). */ +export function serviceStopOutcomeFromWindows(result: WindowsCheckedStopResult): ServiceStopOutcome { + switch (result.outcome) { + case "absent": return "absent"; + case "stopped": return "stopped"; + case "unknown": return "state-unknown"; + default: return "failed"; + } +} + +/** The additive ocx-stop/1 serviceStopReason for a Part A result, if it has one. */ +export function windowsStopReason(result: WindowsCheckedStopResult | null | undefined): WindowsStopReason | undefined { + if (!result || result.outcome === "stopped" || result.outcome === "absent") return undefined; + return result.reason ?? undefined; +} + +/** One operator-facing line for a Part A result that was not stopped/absent. */ +export function describeWindowsPartA(result: WindowsCheckedStopResult): string { + switch (result.reason) { + case "access-denied": + return "The Windows service stop was denied for this account; run the stop from an elevated terminal."; + case "unbound-wrapper": + return "A service wrapper of this home is running but could not be tied to the service; end the wscript.exe/cmd.exe running the opencodex service script, then retry."; + case "unverified-budget": + return "The remaining time cannot hold a verified Windows service stop, so nothing was changed; retry from a terminal."; + case "evidence-unreadable": + return "The Windows service or process state could not be read (" + result.detail + "); nothing that could not be verified was stopped."; + default: + return "The Windows service manager or its wrapper is still running (" + result.detail + ")."; + } +} + +/** What a teardown receipt records after Part A (030 "/api/stop"). */ +export function windowsPartARecord(result: WindowsCheckedStopResult): PendingTeardownPartA | null { + if (result.outcome !== "stopped" && result.outcome !== "absent") return null; + return { + partA: result.outcome, + manager: { + scheduler: result.registration.scheduler === "present", + winsw: result.registration.winsw !== "nonexistent" && result.registration.winsw !== "unknown", + }, + wrappers: result.bound.map(wrapper => ({ pid: wrapper.pid, startTime: wrapper.startTime })), + }; +} + +/** The port Part B watches: runtime record, then the wrapper's baked port, then the config. */ +export function windowsServiceStopPort(): number { + return resolveWindowsStopPort({ + runtimePort: () => readRuntimePort()?.port ?? null, + wrapperPort: () => windowsListenPort(), + configuredPort: () => { + try { + const port = loadConfig().port; + return typeof port === "number" && port > 0 ? port : 10100; + } catch { + return 10100; + } + }, + }).port; +} + +/** Part B with the live dependencies (or the test hook). */ +export async function confirmWindowsServiceStop(request: WindowsPartBRequest): Promise { + if (hooks?.partB) return hooks.partB(request); + return confirmWindowsStopFreed(windowsStopConfirmLiveDeps(request.port, request.remainingMs, + request.snapshot ? { snapshot: request.snapshot } : {})); +} + +/** One operator-facing line for a Part B result other than freed. */ +export function describeWindowsPartB(result: WindowsStopConfirmResult): string { + if (result.outcome === "port-held-by-foreign") { + const owner = result.listener?.ownerPid ? " (PID " + result.listener.ownerPid + ")" : ""; + return "Another program" + owner + " holds port " + result.port + " after the stop."; + } + if (result.outcome === "survived") { + return result.lateWrappers.length > 0 + ? "A service wrapper of this home started again after the stop; it can relaunch the proxy." + : "The proxy answers on port " + result.port + " again after the stop; it is being respawned."; + } + return "Port " + result.port + " could not be confirmed free after the stop (" + result.detail + ")."; +} diff --git a/src/service/windows-stop-deadline.ts b/src/service/windows-stop-deadline.ts new file mode 100644 index 00000000000..73f76d52ae5 --- /dev/null +++ b/src/service/windows-stop-deadline.ts @@ -0,0 +1,39 @@ +/** + * The checked Windows stop's deadline (#7771, 030 "Time budget"). Kept free of the stop + * machinery because the CLI dispatcher, which loads on every command, records it. + * + * ocx stop reads --deadline-ms or OCX_STOP_DEADLINE_MS (the flag wins; an empty or + * invalid variable counts as absent); every other caller uses 60 s. The deadline is + * measured from this process's start: performance.now() counts from performance.timeOrigin, + * so N ms after process start is the clock value N on the Part A clock. + */ +import { DEFAULT_WINDOWS_STOP_DEADLINE_MS } from "./windows-stop-budget"; + +export const STOP_DEADLINE_ENV = "OCX_STOP_DEADLINE_MS"; + +/** A positive decimal integer of milliseconds, or null (absent, empty or invalid). */ +export function parseStopDeadlineMs(raw: string | undefined | null): number | null { + if (raw === undefined || raw === null) return null; + const trimmed = raw.trim(); + if (!/^\d+$/.test(trimmed)) return null; + const value = Number(trimmed); + return Number.isSafeInteger(value) && value > 0 ? value : null; +} + +/** The flag wins; an empty or invalid variable is absent; neither means 60 s. */ +export function resolveStopDeadlineMs(flagMs: number | null | undefined, env: NodeJS.ProcessEnv = process.env): number { + if (typeof flagMs === "number" && Number.isSafeInteger(flagMs) && flagMs > 0) return flagMs; + return parseStopDeadlineMs(env[STOP_DEADLINE_ENV]) ?? DEFAULT_WINDOWS_STOP_DEADLINE_MS; +} + +let stopDeadlineMs: number | null = null; + +/** ocx stop records its resolved deadline here once its arguments are parsed. */ +export function setStopDeadlineMs(ms: number | null): void { + stopDeadlineMs = ms; +} + +/** The absolute deadline on the Part A clock (ms since process start). */ +export function windowsStopDeadlineAt(): number { + return stopDeadlineMs ?? DEFAULT_WINDOWS_STOP_DEADLINE_MS; +} diff --git a/src/update/guarded-runtime-ops.ts b/src/update/guarded-runtime-ops.ts index 289f801e17e..dd2eec192b0 100644 --- a/src/update/guarded-runtime-ops.ts +++ b/src/update/guarded-runtime-ops.ts @@ -4,8 +4,7 @@ import { atomicWriteFile } from "../config/atomic-write"; import { removePid, removeRuntimePort } from "../config/process-state"; import { dropWindowsTcpRowsForLocalPort } from "../server/windows-tcp-drop"; import { stopWinswService } from "../lib/winsw"; -import { stopWindows } from "../service"; -import { killWindowsSchedulerWrappers } from "../lib/windows-service-wrappers"; +import { runWindowsServiceStopPartA } from "../service/windows-service-stop"; import { guardedRun, guardedKill, guardedExecFileSync, materializeMutationData, UpdateMutationRefusedError, type UpdateMutationAuthorizer } from "./mutation-guard.mjs"; export type KillProxy = typeof killProxy; @@ -50,9 +49,8 @@ export function guardedDropWindowsTcp(authorize: (kind: "runtime") => boolean, p export function guardedStopWinsw(authorize: UpdateMutationAuthorizer) { return stopWinswService(mutationGate(authorize, "service")); } -export function guardedStopWindows(authorize: UpdateMutationAuthorizer) { - return stopWindows(mutationGate(authorize, "service")); -} -export function guardedKillWindowsWrappers(authorize: UpdateMutationAuthorizer, paths: Parameters[0]) { - return killWindowsSchedulerWrappers(materializeMutationData(paths), mutationGate(authorize, "service")); +/** Part A of the checked Windows stop (#7771), the service gate at entry and before each terminate. */ +export async function guardedStopWindowsChecked(authorize: UpdateMutationAuthorizer) { + // The update worker descends from the service-run proxy, so its own wrapper is fair game. + return (await runWindowsServiceStopPartA({ authorize: mutationGate(authorize, "service"), allowUnderOwnWrapper: true })).outcome; } diff --git a/src/update/job-windows-restart.ts b/src/update/job-windows-restart.ts index 89f4f45839d..526e72c91db 100644 --- a/src/update/job-windows-restart.ts +++ b/src/update/job-windows-restart.ts @@ -1,39 +1,36 @@ -import { join } from "node:path"; -import { getConfigDir } from "../config"; -import { guardedStopWinsw, guardedStopWindows, guardedKillWindowsWrappers } from "./guarded-runtime-ops"; +import { guardedStopWinsw, guardedStopWindowsChecked } from "./guarded-runtime-ops"; import { readServiceBackend } from "../service"; import type { UpdateMutationAuthorizer } from "./mutation-guard.mjs"; -/** Stop the installed Windows backend and best-effort kill surviving :loop wrappers. */ -export function stopWindowsServiceWrappersBestEffort(authorizeMutation: UpdateMutationAuthorizer): void { - if (process.platform !== "win32") return; - try { - if (readServiceBackend() === "native") { - if (!authorizeMutation("service")) return; - guardedStopWinsw(authorizeMutation); - return; - } - if (!authorizeMutation("service")) return; - guardedStopWindows(authorizeMutation); - } catch { /* already stopped */ } - if (!authorizeMutation("service")) return; - killWindowsServiceWrapperProcesses(authorizeMutation); -} +/** Why an automatic update restart stopped instead of starting a runtime over a live service. */ +export const WINDOWS_SERVICE_STOP_ABORT = "Automatic update restart stopped: the Windows service could not be proven stopped, " + + "so no runtime was started over it. Run 'ocx stop' from a terminal, then 'ocx start'."; /** - * Best-effort termination of surviving Windows scheduler launcher/wrapper processes. - * `schtasks /end` ends the task instance but often leaves wscript/cmd running the - * `:loop` batch, which brings the proxy back during post-update reclaim. + * Stop the installed Windows backend before the updater replaces or restarts the runtime. + * Returns false when the restart must abort; true when nothing can relaunch the proxy. * - * This used to match the bare filenames with -like '*name*', which could stop a - * wrapper belonging to a DIFFERENT OpenCodex home under the same account. The - * shared killer scopes to this home's canonical paths as complete tokens. + * Task Scheduler uses Part A of the checked stop (#7771) through the authorizer-gated + * variant: the service authorization runs at entry and before every terminate, and only + * stopped or absent lets the restart go on. The native WinSW backend keeps today's stop + * (WinSW has no :loop wrapper to end). */ -function killWindowsServiceWrapperProcesses(authorizeMutation: UpdateMutationAuthorizer): void { - const paths = { - scriptPath: join(getConfigDir(), "opencodex-service.cmd"), - launcherPath: join(getConfigDir(), "opencodex-service-launcher.vbs"), - }; - if (!authorizeMutation("service")) return; - guardedKillWindowsWrappers(authorizeMutation, paths); +export async function stopWindowsServiceForRestart(authorizeMutation: UpdateMutationAuthorizer): Promise { + if (process.platform !== "win32") return true; + // An unreadable backend record falls through to Part A, which probes both backends itself. + let backend: string; + try { backend = readServiceBackend(); } catch { backend = "scheduler"; } + if (backend === "native") { + if (!authorizeMutation("service")) return false; + try { guardedStopWinsw(authorizeMutation); } catch { /* already stopped */ } + return true; + } + if (!authorizeMutation("service")) return false; + try { + const outcome = await guardedStopWindowsChecked(authorizeMutation); + return outcome === "stopped" || outcome === "absent"; + } catch { + // A refusal by the service authorizer (at entry or before a terminate) aborts too. + return false; + } } diff --git a/src/update/job.ts b/src/update/job.ts index b5926fceb40..8d84ad2ad65 100644 --- a/src/update/job.ts +++ b/src/update/job.ts @@ -22,7 +22,7 @@ import { type HealthzIdentity, } from "../server/proxy-liveness"; import { isServiceInstalled, isServiceViable } from "../service"; -import { stopWindowsServiceWrappersBestEffort } from "./job-windows-restart"; +import { stopWindowsServiceForRestart, WINDOWS_SERVICE_STOP_ABORT } from "./job-windows-restart"; import { createUpdateServiceHomePolicy, inspectUpdateServiceHome, type UpdateServiceHomeInspector } from "./service-home-authority"; import { runUpdateRestartWithOwnershipLease, type ServiceOwnershipResolution } from "./restart-ownership"; import { restartServiceManagerAfterUpdate, unsettledManagerRestartFailure, unsupervisedRestartFailure, type ManagerRestartFn } from "./job-supervised-restart"; @@ -396,6 +396,10 @@ export function readUpdateJob(jobId?: string | null): UpdateJobState | null { function updateJob(job: UpdateJobState, patch: Partial, logLine?: string): UpdateJobState { const next = prepareUpdateJob(job, patch, logLine); writeJob(next); return next; } +/** A Windows service that could not be proven stopped aborts the restart (#7771). */ +function windowsServiceStopAborted(job: UpdateJobState): false { + updateJob(job, { status: "failed", restarted: false, error: WINDOWS_SERVICE_STOP_ABORT }, WINDOWS_SERVICE_STOP_ABORT); return false; +} function prepareUpdateJob(job: UpdateJobState, patch: Partial, logLine?: string): UpdateJobState { const current = readUpdateJob(job.id) ?? job; const next = { @@ -954,7 +958,8 @@ export interface RestartIo { serviceInstalledFn?: () => boolean; serviceHomeAuthorityFn?: UpdateServiceHomeInspector; serviceHomeBaseline?: ReturnType; // Worker delegation pins its original authority. - stopServiceWrappersFn?: (authorizeMutation: UpdateMutationAuthorizer) => void; + /** Windows service stop before a restart; an explicit false aborts the restart (#7771). */ + stopServiceWrappersFn?: (authorizeMutation: UpdateMutationAuthorizer) => boolean | void | Promise; readPidFn?: typeof readPid; /** * After a service reinstall exits 0, only trust the service path when this is true. @@ -1121,7 +1126,7 @@ async function restartAfterUpdate( if (!homePolicy.authorizeMutation("runtime")) return false; // a home change during the service-args import refuses before stop/reclaim // schtasks /end often leaves the hidden wrapper alive; its :loop respawns // `ocx start` and races reclaim, so end wrappers before the socket. - if (homePolicy.authorizeMutation("service")) (io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation); + if (homePolicy.authorizeMutation("service") && await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job); const preServiceAllow = reclaimKillAllowlist(); const freed = await waitFn(port, hostname, reclaimOptsFor(preServiceAllow)); if (!freed && !reclaimGate()) return false; // a latched denial already refused the job @@ -1249,7 +1254,7 @@ async function restartAfterUpdate( // below are the path that repairs stuck Windows listeners. } } - if (serviceInstalled && homePolicy.authorizeMutation("service")) (io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation); + if (serviceInstalled && homePolicy.authorizeMutation("service") && await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job); // Reclaim before the pinned start: a busy old socket used to hop ports or fail health. const directAllow = reclaimKillAllowlist(); const freed = await waitFn(port, hostname, reclaimOptsFor(directAllow)); @@ -1348,7 +1353,9 @@ async function restartAfterUpdate( lastChild = null; } if (!homePolicy.authorizeMutation("runtime")) return false; - if (preparePort(job, port, listPids, aliveFn, verifyOcx, () => { if (serviceInstalled && homePolicy.authorizeMutation("service")) (io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation); }, reclaimGate) === false) return false; + // #7771: the service stop runs before the synchronous preparation so its outcome can abort. + if (serviceInstalled && homePolicy.authorizeMutation("service") && await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job); + if (preparePort(job, port, listPids, aliveFn, verifyOcx, undefined, reclaimGate) === false) return false; const ready = await waitForGhost( port, hostname, diff --git a/structure/INDEX.md b/structure/INDEX.md index f7a9d3f6912..477818765a7 100644 --- a/structure/INDEX.md +++ b/structure/INDEX.md @@ -165,7 +165,7 @@ Background service, docs, release, and design discipline. | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | Durable Desktop ownership read, takeover-consent handling, live-supervision evidence, the Windows probe's trusted system-directory acquisition, and the supervision latch blocking contract consumed by service and update guards. | | [`ops/service-logging.md`](ops/service-logging.md) | Service-only console timestamps and bounded copy-truncate log retention. Producer bounds for combo positions, target providers and effort diagnostics. | | [`ops/service-health-recovery.md`](ops/service-health-recovery.md) | Opt-in sentinel that turns a hung but listening managed service proxy into an exit its manager restarts. | -| [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | Windows process evidence, the wrapper binding rule, and the checked manager stop and port confirmation that are not yet wired to a stop path. | +| [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | Windows process evidence, the wrapper binding rule, the checked manager stop and port confirmation, and their wiring into every Windows stop path (ocx stop, uninstall, /api/stop, service commands, install/switch, updater). | | [`ops/update-supervised-recovery.md`](ops/update-supervised-recovery.md) | Manager-first recovery after a failed post-update service refresh, and the non-zero unsupervised fallback. | | [`ops/plugins.md`](ops/plugins.md) | Plugin loading from OPENCODEX_HOME/plugins and the upstream rewrite slot plugins attach to. | | [`ops/docs-and-release.md`](ops/docs-and-release.md) | Docs site, workflow map, branch policy, release flow, and cross-platform CI. | @@ -266,6 +266,7 @@ A source area can be described by more than one doc, because these docs are orga | `src/cli/models-runtime.ts` | [`pricing.md`](pricing.md) | | `src/cli/status-codex-settings.ts` | [`remote-link.md`](remote-link.md) | | `src/cli/status-link.ts` | [`remote-link.md`](remote-link.md) | +| `src/cli/stop-windows-checked.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/cli/telemetry.ts` | [`telemetry.md`](telemetry.md) | | `src/cli/update-restart.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/client/` | [`runtime.md`](runtime.md)
[`fallback-policy.md`](fallback-policy.md)
[`clients/claude-desktop.md`](clients/claude-desktop.md) | @@ -510,10 +511,12 @@ A source area can be described by more than one doc, because these docs are orga | `src/server/safe-response-headers.ts` | [`data-planes/error-detail.md`](data-planes/error-detail.md) | | `src/server/search.ts` | [`providers/codex-installation-identity.md`](providers/codex-installation-identity.md) | | `src/server/sse-payload-rewrite.ts` | [`data-planes/error-detail.md`](data-planes/error-detail.md) | +| `src/server/stop-service-route.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/server/ws-bridge.ts` | [`providers/codex-installation-identity.md`](providers/codex-installation-identity.md) | | `src/service-manager-probe.ts` | [`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/service.ts` | [`runtime.md`](runtime.md)
[`ops/docs-and-release.md`](ops/docs-and-release.md) | | `src/service/` | [`runtime.md`](runtime.md) | +| `src/service/cli-windows-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/cli.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/desktop-command-guard.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/desktop-recovery-process.ts` | [`desktop-recovery.md`](desktop-recovery.md) | @@ -530,12 +533,15 @@ A source area can be described by more than one doc, because these docs are orga | `src/service/logging.ts` | [`ops/service-logging.md`](ops/service-logging.md) | | `src/service/orchestration.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | | `src/service/repair.ts` | [`ops/desktop-supervision.md`](ops/desktop-supervision.md) | +| `src/service/windows-checked-stop-live.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-checked-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-process-evidence.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-process-native.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-service-stop.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-stop-binding.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-stop-budget.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/service/windows-stop-confirm.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | +| `src/service/windows-stop-deadline.ts` | [`ops/windows-checked-stop.md`](ops/windows-checked-stop.md) | | `src/sidecar/` | [`ops/service-and-sidecars.md`](ops/service-and-sidecars.md) | | `src/sidecar/anthropic-binding.ts` | [`providers/anthropic-account-pool.md`](providers/anthropic-account-pool.md) | | `src/stall-timeout.ts` | [`transports/timeouts.md`](transports/timeouts.md)
[`runtime.md`](runtime.md) | diff --git a/structure/cli-management.md b/structure/cli-management.md index 6d7144ce887..9c09191e574 100644 --- a/structure/cli-management.md +++ b/structure/cli-management.md @@ -63,7 +63,7 @@ The agents/routing leaf declares local `message sessions` and `message send`, th ## Generated operating reference -The stop declaration in `src/cli/capabilities-observe-system.ts` covers the read-only preservation preflight and shutdown methods at `/api/stop`, both declared in `src/server/management/route-registry.ts`. Discovery and generated help describe the existing `ocx stop [--json]` envelope and ownership exceptions; the declaration grants no shutdown authority. [Runtime](runtime.md) owns refusal, absence and teardown semantics. +The stop declaration in `src/cli/capabilities-observe-system.ts` covers the read-only preservation preflight and shutdown methods at `/api/stop`, both declared in `src/server/management/route-registry.ts`. Discovery and generated help describe the existing `ocx stop [--json] [--deadline-ms ]` envelope and ownership exceptions; the declaration grants no shutdown authority. [Runtime](runtime.md) owns refusal, absence and teardown semantics. `scripts/generate-ocx-skill-surface.ts` renders the compact `skills/ocx/references/01_management_surface.md` index and eight flat domain chapters from the same capability data. The index retains canonical fragment forwarders and links to complete chapter entries. Counts derive from declarations; grouping does not choose runtime dispatch or grant authority. Unknown roots and duplicate anchors are rejected rather than silently omitted. diff --git a/structure/manifest.json b/structure/manifest.json index 05dd11805cc..cde11ba3e4b 100644 --- a/structure/manifest.json +++ b/structure/manifest.json @@ -1307,14 +1307,20 @@ "path": "ops/windows-checked-stop.md", "tier": 6, "title": "Windows Checked Service Stop", - "scope": "Windows process evidence, the wrapper binding rule, and the checked manager stop and port confirmation that are not yet wired to a stop path.", + "scope": "Windows process evidence, the wrapper binding rule, the checked manager stop and port confirmation, and their wiring into every Windows stop path (ocx stop, uninstall, /api/stop, service commands, install/switch, updater).", "documents": [ + "src/cli/stop-windows-checked.ts", + "src/server/stop-service-route.ts", + "src/service/cli-windows-stop.ts", "src/service/windows-checked-stop.ts", + "src/service/windows-checked-stop-live.ts", "src/service/windows-process-evidence.ts", "src/service/windows-process-native.ts", + "src/service/windows-service-stop.ts", "src/service/windows-stop-binding.ts", "src/service/windows-stop-budget.ts", - "src/service/windows-stop-confirm.ts" + "src/service/windows-stop-confirm.ts", + "src/service/windows-stop-deadline.ts" ] }, { diff --git a/structure/ops/update-supervised-recovery.md b/structure/ops/update-supervised-recovery.md index 070a71428b7..9e01dc3d8b2 100644 --- a/structure/ops/update-supervised-recovery.md +++ b/structure/ops/update-supervised-recovery.md @@ -103,9 +103,14 @@ is a home failure and an actually started unsupervised fallback remains a failur `src/update/job-pinned-start.ts` performs pinned-start I/O and checks after log setup, just before spawning. Its caller passes the same latched authorizer. `src/update/job-windows-restart.ts` receives that same authorizer at all worker call -sites. It rechecks service permission after backend observation, before each backend -stop and after the blocking stop or its catch path before wrapper termination. Path -setup precedes a final service check at the scoped scheduler-wrapper killer. +sites. It checks service permission after backend observation and before the backend stop. +WinSW keeps its stop; Task Scheduler runs Part A of the checked Windows stop +(`guardedStopWindowsChecked`, [Windows checked service stop](windows-checked-stop.md)), whose +authorizer is the service mutation gate at entry and again before every bound-wrapper +terminate. A refusal, or any outcome other than stopped or absent, returns false, and each of +the three call sites in `src/update/job.ts` then fails the restart before reclaim or a start; +the one in front of the synchronous pinned-start preparation runs before it so its outcome can +abort. The Node launcher, Bun updater and dashboard worker use the same transition policy in `src/update/service-home-policy.mjs`. Their direct recovery helpers reacquire the ownership @@ -212,7 +217,7 @@ restart refusal, including after a transient re-match; void-returning preparatio The manager helper checks independently after lease release, immediately before contacting the manager, and returns distinct `refused` for foreign or unknown homes; callers cannot treat this as an ordinary failed restart. `src/update/job-windows-restart.ts` owns scoped Windows -wrapper cleanup under the same latched service guard. Pinned direct recovery receives global service-stop authority only for a matching +service stops under the same latched service guard. Pinned direct recovery receives global service-stop authority only for a matching managed home; foreground recovery does not acquire that authority. The preparation callback tests the current `serviceAllowed` after preparation's fresh mutation gate: a matching registration disappearing still allows foreground PID/port cleanup, while diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index c6dc76270c9..34b4ad7a17a 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -2,11 +2,12 @@ The checked Windows service stop (#7771) is a library of primitives and two operations that stop a Task Scheduler or WinSW manager, end the wrappers that would relaunch the proxy, and -then confirm the port is free. **No stop path calls it yet.** `ocx stop`, `ocx service stop`, -`ocx uninstall`, service uninstall and install, `/api/stop` and the updater still use the -best-effort sweep in `src/lib/windows-service-wrappers.ts` and the guarded inspection in -`src/service/windows-guarded-manager.ts`, unchanged. The plan for wiring the callers lives in -`devlog/_plan/261111_windows_service_stop/`. +then confirm the port is free. Every Windows stop path uses it: `ocx stop` with and without a +consent snapshot, `ocx uninstall`, `/api/stop`, `ocx service stop` / `uninstall` / `remove`, +service install and backend switch, and the updater. The best-effort path-token sweep in +`src/lib/windows-service-wrappers.ts` is no longer called by any stop path. macOS and Linux keep +their manager stops and timing; only the signatures turned async. The design and its slices live +in `devlog/_plan/261111_windows_service_stop/`. ## Process evidence @@ -93,6 +94,79 @@ settings the minimum is 48.5 s for a guarded Task Scheduler stop, 56.5 s for a s and 42.5 s for WinSW. Each includes the 0.5 s closing port sample that Part B can take past its window. +## Callers + +`src/service/windows-checked-stop-live.ts` is the production side of every seam: capped +asynchronous `schtasks` and `sc.exe` calls through `execFile` (a deadline-derived timeout is +floored to whole milliseconds and one that floors to 0 spawns nothing, because `execFile` reads 0 +as no timeout), the evidence snapshot `windowsProcessSnapshot` for every bind, the checked +terminate, the exit watch, the Part B port sampler (`scanListenPids`; a row whose owner has exited +is cleared like `reclaimListenPort` does) and the listener identity (`verifyPidIdentity`: ours or +foreign). The task's running state is read from `schtasks /query /fo CSV`, falling back to +`Get-ScheduledTask` inside the same 1 s cap when the status word is localized. + +`src/service/windows-service-stop.ts` is what callers call: `runWindowsServiceStopPartA` (optional +admission, a reused bind snapshot, the updater's authorizer, a hard cap for the route), +`confirmWindowsServiceStop` (Part B), the mapping onto `ServiceStopOutcome` (absent, stopped; +survived, denied and unverified-budget are failed; unknown is state-unknown), the additive +`serviceStopReason` and the receipt record. The deadline lives in +`src/service/windows-stop-deadline.ts`: `ocx stop` takes `--deadline-ms `, else +`OCX_STOP_DEADLINE_MS` (empty or invalid counts as absent), else 60 s, measured from the CLI's own +process start (the Part A clock is `performance.now()`, which counts from the process start). + +- `stopServiceIfInstalledReport` / `stopServiceIfInstalledDetailed` (`src/service/orchestration.ts`) + are async and run Part A on Windows. On Windows Task Scheduler never yields `stopped-respawnable`. +- Plain `ocx stop` (`src/cli/stop-windows-checked.ts`) admits against the deadline with orphan + discovery and the inherited receipts counted. A Part A other than stopped or absent holds the + child: no `/api/stop` POST, the shared teardown skipped, `service` failed or state-unknown, + `proxy` unknown. After a stopped Part A, the POST skips `stopProxy`'s port wait, orphan discovery + is capped at 3 s, and Part B replaces the respawn re-poll: our identity back on the port is + `proxy: respawned`, an unconfirmed port is `evidence-unreadable`, and another program on the port + sets `portHeldByForeign` without failing the stop. +- The guarded path reads the evidence snapshot and the Schedule host PID once per inspection; + `stopManager` binds on the revalidation's snapshot (so its admission does not count a second + enumeration), Part B runs after `settle` on the approved endpoint, and the manager-state checks + reuse Part B's final enumeration. +- The teardown receipt carries the Part A record (`partA` stopped or absent, the registered + managers, the ended wrappers as pid + creation time): written at the claim, or added by the owner + for a receipt claimed before Part A. `/api/stop` (`src/server/stop-service-route.ts`) skips its + own Part A only for a record whose managers are still the registered ones; a receipt without it + runs Part A capped at 6 s; without a receipt a registered WinSW keeps its in-process stop and + anything else gets the stray sweep capped at 4 s. A cap overrun is `service_state_unknown`, and + the manager commands and terminates are gated off once the cap has passed. +- `ocx service stop` (`src/service/cli-windows-stop.ts`) refuses on a Part A other than stopped or + absent, claims a teardown receipt before the deferred tracked-proxy stop when this home owns the + shared teardown (a failed claim sends no child stop), runs Part B, and clears the receipt only + after the restore. `ocx service uninstall` / `remove` refuse before deleting the task unless + Part A stopped (or found nothing) and Part B did not find our identity or an unreadable port. +- `prepareServiceInstall` refuses before replacing anything on a Part A other than stopped or + absent and stops install and switch on any Part B result other than a free port, naming the + owner. `ocx uninstall` runs Part B as its respawn window after a stopped Part A. +- The updater (`src/update/job-windows-restart.ts`) uses `guardedStopWindowsChecked` for Task + Scheduler, with the service mutation gate as the authorizer; WinSW keeps its stop. Each of the + three call sites in `src/update/job.ts` aborts the restart unless the outcome is stopped or absent. + +## Who may run Part A + +The binding rule excludes the stopping process but not its ancestors. A stop that runs beneath a +wrapper carrying this home's launcher or service-script path would end its own supervision chain, +so `runWindowsServiceStopPartA` refuses such a stop with nothing changed (`unknown`, "runs under +this home's service wrapper") by checking the bind snapshot's ancestry of the current PID. Two +callers opt out on purpose: `/api/stop` runs inside the wrapped proxy, and the updater's worker +descends from the service-run proxy; both exist to end that wrapper, and ending an ancestor never +ends the stopper. Every interactive caller (`ocx stop`, `ocx uninstall`, `ocx service` +commands, install and switch) refuses. + +## ARM64 + +No ARM64 device run exists, so ARM64 ships the strict variant (050). +`terminateWindowsProcessBound` falls back to the PowerShell script whenever the native API is +missing, so the strict gate sits in the caller's adapter: `windowsStopTerminate` answers +`unavailable` whenever `windowsProcessNative()` is null (ARM64, or an x64 whose DLL load failed) +and never reaches the fallback, which leaves Part A `unknown` / `evidence-unreadable` wherever a +terminate would be needed. `WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED` turns the fallback on once +the 040 ARM64 device case has run. + ## Tests `tests/windows/windows-stop-binding.test.ts`, `tests/windows/windows-checked-stop.test.ts`, @@ -100,3 +174,11 @@ window. hermetic: injected deps, a virtual clock and simulated platforms, and no real `schtasks`, PowerShell or FFI call. The FFI primitive itself is exercised only by the cases gated on real Windows x64. + +The callers are covered by `tests/windows/windows-checked-stop-live.test.ts` (adapters, ARM64 +strict, whole-millisecond timeouts, the ancestry check), `tests/windows/windows-service-stop-wiring.test.ts` +(the real Part A through the wiring: the own-wrapper refusal, the cap, admission, the deadline and +the receipt record), `tests/windows/windows-stop-callers.test.ts` (the route, the service commands, +the guarded order, `bound-stray`, and the bind on the evidence snapshot) and +`tests/windows/windows-stop-ocx-stop.test.ts` (plain `ocx stop` through the real CLI with the +platform simulated). None of them reaches `schtasks`, PowerShell or FFI. diff --git a/structure/runtime.md b/structure/runtime.md index 50d4bebe1ad..dc8b0e5a39c 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -59,7 +59,7 @@ The [CLI management contract](cli-management.md#cli-readiness-diagnostics) owns `ocx resolve` (`src/cli/resolve.ts`) is the machine surface a desktop shell asks instead of resolving the config home, the port, and liveness itself: the home comes from `src/config/paths.ts`, the effective port is the live listener's when the identity-checked `findLiveProxy` answers and the configured `config.port ?? 10100` otherwise, and the liveness verdict is that same module's output (pid, runtime-versus-config provenance, version, role). Config reads go through `readConfigDiagnostics`, not `loadConfig`: a missing file is defaults, but an invalid file exits 1 instead of being repaired to defaults, because a defaulted port is a guess the caller must refuse. Liveness is three-valued: when `findLiveProxy` returns null, resolve re-asks the endpoints with `endpointsToProve` and `everyEndpointProvenDown`, using the in-process `probeEndpointLiveness` from `src/server/proxy-liveness.ts`. Only unanimous definitive "dead" becomes `absent-proven`; a timeout or reset stays unknown, exits 1 and never authorises a start. The transient exclusive-bind fallback in `src/update/proxy-liveness-probe.mjs` belongs only to the npm/Bun updater. Resolve stays in-process because a compiled standalone binary cannot run a JavaScript probe with `execPath -e`. Discovery borrows `START_OWNERSHIP_LIVENESS`, the start path's ownership budget — the verdict feeds the shell's launch decision, so the cost of a false "nobody listening" is the duplicate proxy (#5004). The verb is in `skipsCodexShimAutoRestore`, so a read-only lookup never triggers a shim repair. Arguments are pre-parsed in `src/cli/root.ts` and exit 64 before any preflight side effect, the same ordering `ocx ready` obeys. `ocx-resolve/1` also reports `ownership` (`none`, `owned`, or `unknown`) and `takeover` (`supported` with protocol version, minimum CLI version and compatibility token, or `blocked` with a reason). A known liveness verdict still exits 0 when ownership is unknown, but takeover is blocked. An unreadable second service-state read blocks takeover instead of becoming an absent registration. `src/service/managing-cli.ts` observes the registered and selected PATH CLIs, including Windows PATHEXT order and file validation, before compatibility is offered. A `--version` probe is retried once on Windows — one transient failure must not read as an unobservable CLI — and a Windows `.CMD`/`.BAT` shim is invoked through a single verbatim-quoted `cmd /c` line, so a spaced executable path and spaced arguments both survive cmd's re-parse. `src/cli/route-policy.ts` distinguishes a missing profile (exit 4) from malformed `show` arguments (exit 2); an upstream failure propagates through the management error mapping. -`ocx stop --json` is a reporting layer over the unchanged stop path. `src/cli/index.ts` threads a `StopRunRecord` through the existing receipt, drain, respawn-verification and restore flow, and `src/cli/stop-report.ts` maps the recorded facts plus the signals that already pick the exit code into one versioned document (`schema: "ocx-stop/1"`). With `--json` the human lines print on stderr and stdout carries only that document; exit codes 0/1/79/80/81 cross the process boundary unchanged. The desktop's opt-in guarded stop adds exact approved PID, endpoint, home, CLI version and compatibility-token expectations; plain `ocx stop` retains its ordinary behavior. Under the ownership mutation lease, `src/cli/stop-approval.ts` re-resolves them and `src/service/guarded-manager-target.ts` binds any installed manager's process tree to the approved PID before a stop action. The same guard re-reads `ocx.pid` and `runtime-port.json` through `src/config/tracked-runtime-inspection.ts`, a single-pass read-only probe that walks the same checks `readPid()` and `readRuntimePort(pid)` perform and reports which one failed, answering the recorded process's start-identity question through the same cache `readPid()` fills so it cannot disagree with the admission it diagnoses. A pre-action mismatch returns `approval-changed` without stopping, and the summary carries `detail` naming the single guard fact that failed — live PID/port/hostname/CLI drift, an unverifiable ownership claim, a blocked takeover, the compatibility fingerprint, or the tracked-record cause — an absent, denied, unreadable or invalid record, a dead, foreign or unverifiable recorded process, or a record naming a different PID — so a transient re-probe can be told from genuine drift in the caller's log. The tracked-record causes are fixed strings: denied is reported only on an observed EACCES/EPERM, an unreadable command line stays unverifiable rather than attributed to an account or elevation, and no probed path, command line, stderr or token reaches the detail. Immediately before any manager stop or direct PID signal, the CLI rechecks the same manager identity or proven absence; a change also returns `approval-changed`. A non-OpenCodex process replacing the OS job in the remaining instant before the manager command is a residual same-user risk outside the ownership lease. The guarded path stops the bound manager, or the approved PID directly when managers are proven absent, then uses a five-second shared deadline to verify PID exit, port availability and definitive endpoint absence. Only after settlement does a read-only manager probe require launchd `not-loaded` in both domains or systemd `inactive` with `MainPID=0`; the state is checked again before a success summary. Timeout, active or unreadable manager state returns terminal `manager-still-active`, even if the endpoint briefly refuses. Windows managers bind through `src/service/windows-guarded-manager.ts`: a Task Scheduler backend must prove a recognized OpenCodex registration against the bounded current-user SID/name lookup, a running task, and a wrapper ancestor carrying this home's canonical paths — presence and running state are read root-folder-scoped because every `/tn` operation resolves a bare name there and nowhere else, so a same-named task inside a task folder is a different registration neither the probe nor the stop can reach; the registered definition itself is read only on the running path, and a state or registration read that cannot answer re-probes presence so a task deleted between probes — which can respawn nothing — reads absent while a merely unreadable one stays unknown; an unreadable current identity leaves the manager unknown; a WinSW backend must prove this installation's binary and SCM service PID ancestry; either marks `childNeedsSeparateStop`, so the approved PID receives its own graceful signal only after a successful manager stop because `schtasks /end` does not reliably cascade to the wrapper's child (#764); a failed or unknown manager stop leaves the child untouched. Post-stop Windows inactivity means the task not running, no surviving wrapper process, and no started WinSW — never mere registration presence; a running-state read that cannot answer re-probes presence the same way, so a task deleted between probes reports inactive while a persisting one stays unknown. Every Windows manager command a verdict or stop depends on — schtasks, sc.exe, WinSW verbs — runs under a bounded `execFileSync` timeout so a wedged Task Scheduler or Service Control Manager cannot hang the guarded stop; a killed command classifies through the same fail-closed paths a nonzero exit would (the interactive `install /p` password prompt is the one deliberate exception). A `wscript.exe`, `cscript.exe` or `cmd.exe` whose command line CIM cannot read makes the state unprovable only when it is tied to the proxy: pre-stop, as an ancestor of the approved PID; post-stop, as the former bound manager process or a descendant of it. Unrelated unreadable shells, which a non-admin query returns for other users' and elevated processes, are not evidence either way. An unprovable Windows manager still blocks guarded takeover. The token checks snapshot consistency, not whether a person approved the desktop prompt. `src/cli/stop-restore.ts` owns the shared Codex/Grok restoration and its history-only, history-deferred, and other failure categories. The checked Windows stop that will replace the best-effort wrapper sweep and bind a stray wrapper by lineage and creation time is described in [Windows checked service stop](ops/windows-checked-stop.md); no stop path calls it yet. +`ocx stop --json` is a reporting layer over the unchanged stop path. `src/cli/index.ts` threads a `StopRunRecord` through the existing receipt, drain, respawn-verification and restore flow, and `src/cli/stop-report.ts` maps the recorded facts plus the signals that already pick the exit code into one versioned document (`schema: "ocx-stop/1"`). With `--json` the human lines print on stderr and stdout carries only that document; exit codes 0/1/79/80/81 cross the process boundary unchanged. The desktop's opt-in guarded stop adds exact approved PID, endpoint, home, CLI version and compatibility-token expectations; plain `ocx stop` retains its ordinary behavior. Under the ownership mutation lease, `src/cli/stop-approval.ts` re-resolves them and `src/service/guarded-manager-target.ts` binds any installed manager's process tree to the approved PID before a stop action. The same guard re-reads `ocx.pid` and `runtime-port.json` through `src/config/tracked-runtime-inspection.ts`, a single-pass read-only probe that walks the same checks `readPid()` and `readRuntimePort(pid)` perform and reports which one failed, answering the recorded process's start-identity question through the same cache `readPid()` fills so it cannot disagree with the admission it diagnoses. A pre-action mismatch returns `approval-changed` without stopping, and the summary carries `detail` naming the single guard fact that failed — live PID/port/hostname/CLI drift, an unverifiable ownership claim, a blocked takeover, the compatibility fingerprint, or the tracked-record cause — an absent, denied, unreadable or invalid record, a dead, foreign or unverifiable recorded process, or a record naming a different PID — so a transient re-probe can be told from genuine drift in the caller's log. The tracked-record causes are fixed strings: denied is reported only on an observed EACCES/EPERM, an unreadable command line stays unverifiable rather than attributed to an account or elevation, and no probed path, command line, stderr or token reaches the detail. Immediately before any manager stop or direct PID signal, the CLI rechecks the same manager identity or proven absence; a change also returns `approval-changed`. A non-OpenCodex process replacing the OS job in the remaining instant before the manager command is a residual same-user risk outside the ownership lease. The guarded path stops the bound manager, or the approved PID directly when managers are proven absent, then uses a five-second shared deadline to verify PID exit, port availability and definitive endpoint absence. Only after settlement does a read-only manager probe require launchd `not-loaded` in both domains or systemd `inactive` with `MainPID=0`; the state is checked again before a success summary. Timeout, active or unreadable manager state returns terminal `manager-still-active`, even if the endpoint briefly refuses. Windows managers bind through `src/service/windows-guarded-manager.ts`: a Task Scheduler backend must prove a recognized OpenCodex registration against the bounded current-user SID/name lookup, a running task, and a wrapper ancestor carrying this home's canonical paths — presence and running state are read root-folder-scoped because every `/tn` operation resolves a bare name there and nowhere else, so a same-named task inside a task folder is a different registration neither the probe nor the stop can reach; the registered definition itself is read only on the running path, and a state or registration read that cannot answer re-probes presence so a task deleted between probes — which can respawn nothing — reads absent while a merely unreadable one stays unknown; an unreadable current identity leaves the manager unknown; a WinSW backend must prove this installation's binary and SCM service PID ancestry; either marks `childNeedsSeparateStop`, so the approved PID receives its own graceful signal only after a successful manager stop because `schtasks /end` does not reliably cascade to the wrapper's child (#764); a failed or unknown manager stop leaves the child untouched. Post-stop Windows inactivity means the task not running, no surviving wrapper process, and no started WinSW — never mere registration presence; a running-state read that cannot answer re-probes presence the same way, so a task deleted between probes reports inactive while a persisting one stays unknown. Every Windows manager command a verdict or stop depends on — schtasks, sc.exe, WinSW verbs — runs under a bounded `execFileSync` timeout so a wedged Task Scheduler or Service Control Manager cannot hang the guarded stop; a killed command classifies through the same fail-closed paths a nonzero exit would (the interactive `install /p` password prompt is the one deliberate exception). A `wscript.exe`, `cscript.exe` or `cmd.exe` whose command line CIM cannot read makes the state unprovable only when it is tied to the proxy: pre-stop, as an ancestor of the approved PID; post-stop, as the former bound manager process or a descendant of it. Unrelated unreadable shells, which a non-admin query returns for other users' and elevated processes, are not evidence either way. An unprovable Windows manager still blocks guarded takeover. The token checks snapshot consistency, not whether a person approved the desktop prompt. `src/cli/stop-restore.ts` owns the shared Codex/Grok restoration and its history-only, history-deferred, and other failure categories. Every Windows stop path now runs the checked stop described in [Windows checked service stop](ops/windows-checked-stop.md) (#7771): on the guarded path Part A ends the bound manager and the wrappers bound by lineage and creation time before the approved PID gets its graceful signal, a registered task that is not running but left wrappers the binding rule ties to the approved PID is the provable `bound-stray` manager (revalidation compares its wrapper set; one the rule cannot bind stays unknown and the `approval-changed` summary carries `serviceStopReason` `unbound-wrapper` or `evidence-unreadable`), and Part B confirms the approved endpoint after settle. Plain `ocx stop` sends no `/api/stop` POST while Part A cannot prove the manager stopped, and runs Part B instead of the respawn re-poll after it did. Both paths honour `--deadline-ms` / `OCX_STOP_DEADLINE_MS` on Windows and add the optional `serviceStopReason` and `portHeldByForeign` fields to `ocx-stop/1` without a new outcome or proxy value. `ocx service status --json` (`src/service/status-json.ts`) is the passive machine form of the service report, read by peers such as GOAT's device inventory. `src/service/cli.ts` accepts `--json` for `status` only (`planServiceCommand` refuses it on every other verb before any installation probe) and answers it before `platformOps()` resolves, so no install, start, stop, repair or migration path is reachable; `skipsCodexShimAutoRestore` exempts `service status --json` so the CLI preflight repairs no Codex shim first, the collection runs inside `runPassiveRead` so the serving probe's config load neither hardens modes nor backs up a malformed file, and `--json` on any other verb is refused before the Desktop check and the Windows mutation lock; an unsupported platform or Docker gets a document instead of the text path's exit. The document reuses exactly the reads the text report makes — `diagnoseService()` (launchd `print`, systemd observation verbs, Task Scheduler `/query`, SCM/WinSW status) and, only when a registration exists, the 1.5-second loopback `confirmServiceServing` probe — plus the ownership-lease and Desktop login lines. `running` is the manager's verdict (the value `startup.serviceRunning` carries in `ocx status --json`); `serving` reports separately whether a proxy answers on the installed port. `backend` is `launchd`, `systemd`, `schtasks`, `winsw`, or `none` when nothing is registered. The systemd branch of `diagnoseService` derives through the pure `deriveSystemdServiceDiagnostic`, matching the launchd and Windows derivations. `tests/service/service-status-json.test.ts` snapshots each platform branch and records every subprocess entry point on darwin, linux and win32 against a read-only allowlist. `ocx status --json` carries an additive `install` verdict (`src/cli/status-install.ts`) mapped from the updater's own `detectInstallOwnership()`: `method` is `bun`, `npm`, `pnpm`, `mise`, `source`, `standalone` or `unknown` (detector failure), and `evidence.basis` names the package layout, the mise backend record (tool alias and backend only) or the standalone binary without printing a path (`tests/cli/status-install-method.test.ts`). `ocx service restart` reads the pid of the proxy answering on the installed port before it restarts (`servingRuntimePid`, `src/service/health.ts`) and passes it to the serving check as `replacing`. While that pid is set, only a proxy reporting a different pid counts as serving: a Windows wrapper that survived `schtasks /end` keeps its old child answering healthz, and the restart used to report "restarted and serving" over it (#7771). If the replaced pid is still the one answering at the deadline, the command prints that the previous runtime is still serving and the replacement was not confirmed, prints no checkmark, and exits 1; a restart that threw names either the surviving pid (kept, not replaced) or the new pid. An answer that reports no pid cannot be shown to be the old runtime and counts as serving. `ocx service repair` keeps the plain any-proxy check, because a repair may preserve a healthy runtime by design. `tests/service/service-restart-replacement.test.ts` covers each case. diff --git a/tests/cli/cli-dispatch.test.ts b/tests/cli/cli-dispatch.test.ts index 3a36a246f2d..13124916ede 100644 --- a/tests/cli/cli-dispatch.test.ts +++ b/tests/cli/cli-dispatch.test.ts @@ -639,7 +639,7 @@ describe("a sibling start leaves shared client routing to the live owner", () => const readAt = stop.indexOf("const siblingOfPort = readRuntimePort()?.siblingOfPort;"); expect(readAt).toBeGreaterThan(-1); // Read before the first stop can remove the record. - expect(readAt).toBeLessThan(stop.indexOf("stopServiceIfInstalledDetailed()")); + expect(readAt).toBeLessThan(stop.indexOf("stopServiceForOcxStop(")); expect(stop).toContain("if (teardownNonce || stoppingSibling) return;"); // #7563 adds a connected client to the same gate; the sibling term is unchanged. expect(stop).toContain("const restoreBlocked = ownershipBlocked || inheritedBlocks || nativeRestoreHandledByProxy || stoppingSibling || connectedClientSkip !== null;"); @@ -652,10 +652,12 @@ describe("a sibling start leaves shared client routing to the live owner", () => // it failed the ownership check from the sibling's home: exit 1, and after a hard kill the // stale sibling records were never purged. // #7423: the non-sibling branch claims an update-handover receipt first; the sibling term is unchanged. - expect(stop).toContain(': stoppingSibling || connectedClientAlreadyStopped ? "absent" : (claimHandoverBeforeManager(), stopServiceIfInstalledDetailed());'); + expect(stop).toContain(': stoppingSibling || connectedClientAlreadyStopped ? noServiceStop() : (claimHandoverBeforeManager(), await stopServiceForOcxStop({ receiptNonce: () => teardownNonce, inheritedReceipts: inheritedTeardowns.length }));'); // The only other caller is the guarded desktop step, which reaches the manager only when it - // provably owns the approved pid; a sibling's pid never is. - expect(stop.split("stopServiceIfInstalledDetailed()").length - 1).toBe(2); + // provably owns the approved pid; a sibling's pid never is. Both go through the #7771 wiring. + expect(stop).not.toContain("stopServiceIfInstalledDetailed("); + expect(stop.split("stopServiceForOcxStop(").length - 1).toBe(1); + expect(stop.split("stopGuardedManager(").length - 1).toBe(1); }); test("ocx stop of a hard-killed sibling never stops the live owner discovery falls back to", () => { @@ -672,7 +674,7 @@ describe("a sibling start leaves shared client routing to the live owner", () => // Wiring: the orphan path asks before it may stop anything it found, and reports success. const stop = slice("async function handleStopUnlocked(", "async function handleUninstall("); - const findAt = stop.indexOf("const live = await findLiveProxy({ acceptPackageTreeFenced: true });"); + const findAt = stop.indexOf("const live = await findLiveProxy({ acceptPackageTreeFenced: true, ...orphanDiscoveryCap() });"); const askAt = stop.indexOf("if (siblingStopFoundOwner(siblingOfPort, live)) {"); const attestAt = stop.indexOf('} else if (live?.pid && (await proveLiveProxyOwnedByHome(live)) !== "proven") {'); expect(findAt).toBeGreaterThan(-1); diff --git a/tests/cli/cli-ready.test.ts b/tests/cli/cli-ready.test.ts index 0c739b47020..92c2fb84466 100644 --- a/tests/cli/cli-ready.test.ts +++ b/tests/cli/cli-ready.test.ts @@ -872,13 +872,15 @@ describe("handleStart OCX_SERVICE exit guard (source-level)", () => { expect(nonService, "non-service refusal keeps the exit 1 conflict error").not.toBeNull(); }); - test("service.ts teardown kills surviving wrapper processes on stop", () => { + test("service.ts teardown ends surviving wrapper processes on stop through the checked stop", () => { const serviceSource = readFileSync(repoPath("src/service/orchestration.ts"), "utf8"); - expect(serviceSource).toMatch(/killWindowsServiceWrapperProcesses/); // The boolean `stopServiceIfInstalled` is gone — it collapsed a live manager into the // same false as "not installed" (#3008). The stop itself is the detailed function. - const callSite = serviceSource.match(/stopServiceIfInstalledDetailed[\s\S]{0,1600}?killWindowsServiceWrapperProcesses\(\)/); - expect(callSite, "wrapper kill must run during stopServiceIfInstalledDetailed").not.toBeNull(); + // #7771: on Windows it is Part A of the checked stop, which ends the bound wrappers and + // waits for them instead of the best-effort path-token sweep. + const callSite = serviceSource.match(/stopServiceIfInstalledReport[\s\S]{0,2400}?runWindowsServiceStopPartA\(request\)/); + expect(callSite, "Part A must run during the service stop on Windows").not.toBeNull(); + expect(serviceSource).not.toMatch(/killWindowsServiceWrapperProcesses/); }); test("wrapper kill matches the canonical paths of THIS installation, not bare filenames", () => { @@ -886,15 +888,15 @@ describe("handleStart OCX_SERVICE exit guard (source-level)", () => { // wrapper from another OpenCodex home (or any process whose command line // merely contains the name). The kill must target the exact canonical // paths windowsServiceScriptPath()/windowsLauncherVbsPath() produce. - const serviceSource = readFileSync(repoPath("src/service/windows-ops.ts"), "utf8"); - expect(serviceSource).toMatch(/windowsServiceScriptPath\(\)/); - expect(serviceSource).toMatch(/windowsLauncherVbsPath\(\)/); - const killBody = serviceSource.match(/function killWindowsServiceWrapperProcesses\(\)[\s\S]*?\n}/); - expect(killBody, "killWindowsServiceWrapperProcesses body must exist").not.toBeNull(); - expect(killBody![0]).toContain("windowsServiceScriptPath()"); - expect(killBody![0]).toContain("windowsLauncherVbsPath()"); + // Since #7771 the stop binds wrappers through the checked stop's binding rule, whose + // path set is exactly these canonical paths (tests/windows/windows-stop-binding.test.ts + // covers other homes, substrings and image names). + const live = readFileSync(repoPath("src/service/windows-checked-stop-live.ts"), "utf8"); + const paths = live.match(/export function windowsStopCanonicalPaths\(\)[\s\S]*?\n}/); + expect(paths, "windowsStopCanonicalPaths body must exist").not.toBeNull(); + expect(paths![0]).toContain("return [windowsLauncherVbsPath(), windowsServiceScriptPath()];"); // Bare wrapper filenames must NOT be the match target. - expect(killBody![0]).not.toMatch(/\$pats = @\('opencodex-service\.cmd'\)/); + expect(live).not.toMatch(/\$pats = @\('opencodex-service\.cmd'\)/); }); test("wrapper kill requires the canonical path as a complete command-line token", () => { diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 1391a33c62b..ac73d37ea16 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -51,7 +51,7 @@ describe("full uninstall command", () => { expect(exit).toBeGreaterThan(refusal); // Nothing before the check: every step that stops, removes or rewrites anything comes after the exit. expect(body.slice(0, refusal).split("\n").filter(line => line.trim() && !line.trim().startsWith("//"))).toEqual(["async function handleUninstall() {"]); - for (const step of ["await runStep(", "stopServiceIfInstalledDetailed", "uninstallServiceDetailed", "restoreNativeCodexAsync", "revertSystemEnv", "uninstallCodexShimAsync", "removeOwnedConfigAfterDesktopCleanup"]) { + for (const step of ["await runStep(", "stopServiceForUninstall", "uninstallServiceDetailed", "restoreNativeCodexAsync", "revertSystemEnv", "uninstallCodexShimAsync", "removeOwnedConfigAfterDesktopCleanup"]) { expect(body.indexOf(step)).toBeGreaterThan(exit); } }); @@ -220,7 +220,9 @@ describe("uninstall gates shared teardown on a proven service stop", () => { // stopServiceIfInstalled returns false for "not installed", "refused to stop" and // "state could not be read" alike, so this step reported "not installed" for a manager // that might still be running (#3008). - expect(fn).toContain("stopServiceIfInstalledDetailed()"); + // #7771: through the stop wiring, which is stopServiceIfInstalledDetailed off Windows and + // Part A of the checked stop on Windows; both return the detailed outcome. + expect(fn).toContain("const outcome = await stopServiceForUninstall();"); expect(fn).not.toContain("stopServiceIfInstalled()"); expect(fn).toContain('if (outcome === "absent") return false;'); expect(fn).toContain('if (outcome === "failed")'); @@ -239,7 +241,9 @@ describe("uninstall gates shared teardown on a proven service stop", () => { expect(fn).toContain("respawnWindowVerified: false,"); expect(fn).toContain("observed.serviceStop = outcome;"); expect(fn).toContain("observed.serviceRemoval = outcome;"); - expect(fn).toContain('if (observed.serviceStop === "stopped-respawnable")'); + // A legacy respawnable stop and, on Windows, every stopped one (Part B, #7771). + expect(fn).toContain("if (respawnWindowRequired(observed)) {"); + expect(fn).toContain("await verifyUninstallStopWindow();"); expect(fn).toContain("observed.respawnWindowVerified = true;"); const gateAt = fn.indexOf("if (sharedTeardownAuthorized(observed)) {"); expect(gateAt).toBeLessThan(fn.indexOf("native Codex restored", gateAt)); diff --git a/tests/clients/desktop-supervised-restart.test.ts b/tests/clients/desktop-supervised-restart.test.ts index d3ced166fd4..96120e043c6 100644 --- a/tests/clients/desktop-supervised-restart.test.ts +++ b/tests/clients/desktop-supervised-restart.test.ts @@ -260,7 +260,7 @@ describe("the dashboard's Stop under the desktop app", () => { const refusal = handler.indexOf("desktopSupervisedStopRefusal(principal)"); expect(refusal).toBeGreaterThan(-1); expect(handler.slice(refusal, refusal + 200)).toContain("return jsonResponse(desktopRefusal, 409, req, config)"); - for (const later of ["installedServiceRespawnRisk()", "stopServiceIfInstalledDetailed()", "noteExplicitShutdownRequested()", "performStopTeardown(url"]) { + for (const later of ["installedServiceRespawnRisk()", "routeServiceStop(url, holdsReceipt)", "noteExplicitShutdownRequested()", "performStopTeardown(url"]) { expect(handler.indexOf(later)).toBeGreaterThan(refusal); } }); diff --git a/tests/fixtures/parent-stop-runner.ts b/tests/fixtures/parent-stop-runner.ts index db621f4b751..cd0fc05a91e 100644 --- a/tests/fixtures/parent-stop-runner.ts +++ b/tests/fixtures/parent-stop-runner.ts @@ -51,6 +51,9 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { noRuntimeRecord?: boolean; /** Like the real readPid, report no PID once the process is gone (a manager stop exited it). */ deadPidReadsNull?: boolean; + /** #7771: simulate win32 and answer Part A (and Part B) of the checked Windows stop. */ + windowsPartA?: "stopped" | "absent" | "survived" | "denied" | "unknown" | "unverified-budget"; + windowsPartB?: "freed" | "survived" | "port-held-by-foreign" | "unconfirmed" | "unknown"; }; if (options.proxyProbePort) { // Deliberately bypass every fixture mock for the real proxy-environment regression. @@ -97,7 +100,9 @@ mock.module("../../src/client/lifecycle-lock", () => ({ ...lifecycle, })); const endpoint = { hostname: "127.0.0.1", port: 10100 }; const fakePid = 4242; -const calls = { killed: 0, native: 0, grok: 0, cleared: 0, exited: 0, service: 0, lifecycle: 0 }; +const calls = { killed: 0, native: 0, grok: 0, cleared: 0, exited: 0, service: 0, lifecycle: 0, partB: 0 }; +/** The Part A record each stop POST's receipt carried when the POST was sent (#7771). */ +const postPartA: unknown[] = []; let receiptBlocker: import("bun:sqlite").Database | undefined; const { Database } = await import("bun:sqlite"); const { prepareConfigMutationDatabasePathForWrite } = await import("../../src/config"); @@ -137,7 +142,7 @@ const receipts = await import("../../src/config/pending-teardown"); process.on("exit", () => { receiptBlocker?.close(); writeFileSync(join(home, "parent-stop-result.json"), JSON.stringify({ - calls, urls, probes, transports, unexpectedIo, nonce, + calls, urls, probes, transports, unexpectedIo, nonce, postPartA, receiptExists: nonce !== undefined && existsSync(receipts.pendingTeardownPathFor(nonce)), })); }); @@ -261,6 +266,7 @@ globalThis.fetch = (async (input: string | URL | Request, init?: RequestInit) => } if (!url.startsWith("http://127.0.0.1:10100/api/stop")) return unexpected(`fetch(${url})`); urls.push(url); + postPartA.push(receipts.pendingTeardownPartA(new URL(url).searchParams.get("teardownNonce"))); transports.push(viaDirect ? "direct" : "global"); if (options.connectDuringPost) writeHubConnection(); const body = options.serverTeardown && !options.apiFailure @@ -290,6 +296,15 @@ mock.module("../../src/server/direct-local-http", () => ({ ...direct, })); process.argv = [process.execPath, "ocx", "stop", ...(options.json ? ["--json"] : [])]; +if (options.windowsPartA) { + // The stop wiring acts for win32 while the host stays untouched: Part A and Part B answer here. + (await import("../../src/cli/stop-windows-checked")).resetOcxWindowsStopForTests("win32"); + const { partAResult, partBResult } = await import("../helpers/windows-stop-caller-fixtures"); + (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ + partA: async () => { calls.service++; return partAResult(options.windowsPartA!); }, + partB: async () => { calls.partB++; return partBResult(options.windowsPartB ?? "freed"); }, + }); +} if (options.handover) process.env.OCX_UPDATE_TEARDOWN_HANDOVER = "1"; if (options.exitCleanup) { delete process.env.OCX_SERVICE; diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 96a958e6a5b..401a96b1d74 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1529,8 +1529,8 @@ "windows-deploy-close-regressions.test.ts": "windows", "windows-elevation-spawn.test.ts": "windows", "windows-elevation.test.ts": "windows", - "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", - "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", + "windows-guarded-manager.test.ts": "windows", "windows-checked-stop.test.ts": "windows", "windows-stop-binding.test.ts": "windows", "windows-checked-stop-live.test.ts": "windows", "windows-service-stop-wiring.test.ts": "windows", + "windows-popup-fix.test.ts": "windows", "windows-stop-confirm.test.ts": "windows", "windows-process-evidence.test.ts": "windows", "windows-stop-callers.test.ts": "windows", "windows-stop-ocx-stop.test.ts": "windows", "windows-process-priority.test.ts": "windows", "windows-scheduler-errors.test.ts": "windows", "windows-scheduler-install-verification.test.ts": "windows", "windows-lifecycle-root-acl.test.ts": "windows", diff --git a/tests/helpers/update-guarded-io.mjs b/tests/helpers/update-guarded-io.mjs index f9cf329a9b9..a7586d62e27 100644 --- a/tests/helpers/update-guarded-io.mjs +++ b/tests/helpers/update-guarded-io.mjs @@ -23,8 +23,10 @@ export function guardedUpdateIo(deps) { guardedRemoveRuntimePort: auth => guardedRun(auth, "runtime", native("removeRuntimePort"), []), guardedDropWindowsTcp: (auth, ...args) => guardedRun(auth, "runtime", native("dropWindowsTcpRowsForLocalPort"), args), guardedStopWinsw: auth => guardedRun(auth, "service", native("stopWinswService"), []), - guardedStopWindows: auth => guardedRun(auth, "service", native("stopWindows"), []), - guardedKillWindowsWrappers: (auth, ...args) => guardedRun(auth, "service", native("killWindowsSchedulerWrappers"), args), + // #7771: Part A takes the service gate as its authorizer (entry and before each terminate). + guardedStopWindowsChecked: async auth => native("checkedStop")(() => { + if (auth("service") !== true) throw new UpdateMutationRefusedError(); + }), }; } diff --git a/tests/helpers/update-helper-control-contracts.ts b/tests/helpers/update-helper-control-contracts.ts index c94203ab44a..8578cd95da7 100644 --- a/tests/helpers/update-helper-control-contracts.ts +++ b/tests/helpers/update-helper-control-contracts.ts @@ -106,8 +106,7 @@ const baseHelperControlContracts: readonly Contract[] = [ ["src/update/guarded-runtime-ops.ts","guardedRemoveRuntimePort","removeRuntimePort(undefined,",1,"same runtime gate immediately before record and registry unlink"], ["src/update/guarded-runtime-ops.ts","guardedDropWindowsTcp","dropWindowsTcpRowsForLocalPort(target,",1,"same runtime gate immediately before each prepared SetTcpEntry"], ["src/update/guarded-runtime-ops.ts","guardedStopWinsw","stopWinswService(mutationGate(",1,"same service gate at prepared native command"], - ["src/update/guarded-runtime-ops.ts","guardedStopWindows","stopWindows(mutationGate(",1,"same service gate at prepared scheduler command"], - ["src/update/guarded-runtime-ops.ts","guardedKillWindowsWrappers","killWindowsSchedulerWrappers(materializeMutationData(paths),",1,"same service gate after path/script preparation"], + ["src/update/guarded-runtime-ops.ts","guardedStopWindowsChecked","runWindowsServiceStopPartA({ authorize: mutationGate(authorize, \"service\"),",1,"same service gate at Part A entry and again before each bound-wrapper terminate (#7771)"], ["src/update/guarded-tray-ops.ts","guardedStartWindowsTray","guardedRun(authorize, \"own-tray\", startWindowsTray, [])",1,"materialize plain data then synchronous authorization and primitive dispatch"], ["src/update/guarded-tray-ops.ts","guardedStopWindowsTray","stopWindowsTray(()",1,"same tray gate after heartbeat and command preparation"], ["src/update/mutation-guard.mjs","guardedSpawn","guardedRun(authorize, kind, spawn, [file, args, options])",1,"materialize plain data then synchronous authorization and primitive dispatch"], diff --git a/tests/helpers/update-helper-inventory.ts b/tests/helpers/update-helper-inventory.ts index 86749aaf8c6..30d2236d9b4 100644 --- a/tests/helpers/update-helper-inventory.ts +++ b/tests/helpers/update-helper-inventory.ts @@ -56,7 +56,7 @@ export function helperControlInventory(parser?: API) { const visit = (node: ts.Node) => { if (ts.isCallExpression(node) && !["spawnFailure", "registrySpawnTarget"].includes(compact(node.expression)) && !(ts.isPropertyAccessExpression(node.expression) && node.expression.name.text === "trim") - && (capabilityCalls.has(node) || /[sS]pawn|execFile|guardedRun|guardedRename|guardedKill|guardedExecFile|ownedArtifact|^native$|^operation$|(?:^|\.)kill$|^run$|^runPnpm$|^io\.(?:stop|start)$|^(?:removePid|removeRuntimePort|dropWindowsTcpRowsForLocalPort|stopWinswService|stopWindows|killWindowsSchedulerWrappers|stopWindowsTray)$|(?:^|\.)(?:rename(?:WithRetry)?|rm|unlink|writeFile|copyFile)(?:Sync)?$/.test(compact(node.expression)))) { + && (capabilityCalls.has(node) || /[sS]pawn|execFile|guardedRun|guardedRename|guardedKill|guardedExecFile|ownedArtifact|^native$|^operation$|(?:^|\.)kill$|^run$|^runPnpm$|^io\.(?:stop|start)$|^(?:removePid|removeRuntimePort|dropWindowsTcpRowsForLocalPort|stopWinswService|stopWindows|killWindowsSchedulerWrappers|runWindowsServiceStopPartA|stopWindowsTray)$|(?:^|\.)(?:rename(?:WithRetry)?|rm|unlink|writeFile|copyFile)(?:Sync)?$/.test(compact(node.expression)))) { let context = ""; for (let p: ts.Node | undefined = node.parent; p; p = p.parent) { if (ts.isFunctionDeclaration(p) && p.name) { context = p.name.text; break; } diff --git a/tests/helpers/update-import-policy.ts b/tests/helpers/update-import-policy.ts index 43c0fa1646e..c563854b541 100644 --- a/tests/helpers/update-import-policy.ts +++ b/tests/helpers/update-import-policy.ts @@ -87,8 +87,8 @@ const baseStaticImports: Readonly> = { "import { removePid, removeRuntimePort } from \"../config/process-state\";", "import { dropWindowsTcpRowsForLocalPort } from \"../server/windows-tcp-drop\";", "import { stopWinswService } from \"../lib/winsw\";", - "import { stopWindows } from \"../service\";", - "import { killWindowsSchedulerWrappers } from \"../lib/windows-service-wrappers\";", + // #7771: Part A of the checked Windows stop; its authorizer is the service mutation gate. + "import { runWindowsServiceStopPartA } from \"../service/windows-service-stop\";", "import { guardedRun, guardedKill, guardedExecFileSync, materializeMutationData, UpdateMutationRefusedError, type UpdateMutationAuthorizer } from \"./mutation-guard.mjs\";" ], "src/update/guarded-tray-ops.ts": [ @@ -166,9 +166,7 @@ const baseStaticImports: Readonly> = { "import { handoffWindowsTrayForUpdate, planWindowsTrayUpdate } from \"./tray-update-plan.mjs\";" ], "src/update/job-windows-restart.ts": [ - "import { join } from \"node:path\";", - "import { getConfigDir } from \"../config\";", - "import { guardedStopWinsw, guardedStopWindows, guardedKillWindowsWrappers } from \"./guarded-runtime-ops\";", + "import { guardedStopWinsw, guardedStopWindowsChecked } from \"./guarded-runtime-ops\";", "import { readServiceBackend } from \"../service\";" ], "src/update/job.ts": [ @@ -187,7 +185,7 @@ const baseStaticImports: Readonly> = { "import { guardedKillProxy, guardedRemovePid, guardedRemoveRuntimePort, guardedDropWindowsTcp, guardedPublishJob, type KillProxy } from \"./guarded-runtime-ops\";", "import { isHealthzVersion, isOpencodexHealthz, probeHostname, proxyIdentityAt, type HealthzIdentity, } from \"../server/proxy-liveness\";", "import { isServiceInstalled, isServiceViable } from \"../service\";", - "import { stopWindowsServiceWrappersBestEffort } from \"./job-windows-restart\";", + "import { stopWindowsServiceForRestart, WINDOWS_SERVICE_STOP_ABORT } from \"./job-windows-restart\";", "import { createUpdateServiceHomePolicy, inspectUpdateServiceHome, type UpdateServiceHomeInspector } from \"./service-home-authority\";", "import { runUpdateRestartWithOwnershipLease, type ServiceOwnershipResolution } from \"./restart-ownership\";", "import { restartServiceManagerAfterUpdate, unsettledManagerRestartFailure, unsupervisedRestartFailure, type ManagerRestartFn } from \"./job-supervised-restart\";", diff --git a/tests/helpers/windows-stop-caller-fixtures.ts b/tests/helpers/windows-stop-caller-fixtures.ts new file mode 100644 index 00000000000..cc59f458b8b --- /dev/null +++ b/tests/helpers/windows-stop-caller-fixtures.ts @@ -0,0 +1,42 @@ +/** + * Result builders for the caller-level tests of the checked Windows stop (#7771): what Part A + * and Part B return, so a caller can be driven through every outcome without the machine. + */ +import type { WindowsCheckedStopResult } from "../../src/service/windows-checked-stop"; +import type { WindowsStopConfirmResult } from "../../src/service/windows-stop-confirm"; + +export function partAResult( + outcome: WindowsCheckedStopResult["outcome"], + extra: Partial = {}, +): WindowsCheckedStopResult { + const reason: WindowsCheckedStopResult["reason"] = outcome === "denied" ? "access-denied" + : outcome === "unknown" ? "evidence-unreadable" + : outcome === "unverified-budget" ? "unverified-budget" : null; + return { + outcome, reason, detail: "fixture " + outcome, + registration: { scheduler: outcome === "absent" ? "absent" : "present", winsw: "nonexistent" }, + bound: outcome === "stopped" ? [{ pid: 30, startTime: 5_100, image: "cmd.exe", parentPid: 20, commandLineDigest: "d" }] : [], + unbound: [], terminated: outcome === "stopped" ? [30] : [], changed: outcome !== "absent" && outcome !== "unverified-budget", + ...extra, + }; +} + +export function partBResult( + outcome: WindowsStopConfirmResult["outcome"], + extra: Partial = {}, +): WindowsStopConfirmResult { + return { + outcome, + portHeldByForeign: outcome === "port-held-by-foreign", + reason: outcome === "unknown" ? "evidence-unreadable" : null, + detail: "fixture " + outcome, + port: 10100, + windowMs: 6_000, + listener: outcome === "port-held-by-foreign" ? { ownerPid: 777, identity: "foreign" } + : outcome === "survived" ? { ownerPid: 43, identity: "ours" } : null, + lateWrappers: [], + ...extra, + }; +} + +export const ALL_PART_A_FAILURES = ["survived", "denied", "unknown", "unverified-budget"] as const; diff --git a/tests/providers/xai/grok-lifecycle.test.ts b/tests/providers/xai/grok-lifecycle.test.ts index b3e5634357a..2c70dac7727 100644 --- a/tests/providers/xai/grok-lifecycle.test.ts +++ b/tests/providers/xai/grok-lifecycle.test.ts @@ -1,7 +1,9 @@ import { describe, expect, test } from "bun:test"; import { readFileSync } from "node:fs"; import { join } from "node:path"; -import { classifyWindowsServiceStop, installedServiceRespawnRisk, isServiceOwnershipError, ServiceOwnershipError } from "../../../src/service"; +import { installedServiceRespawnRisk, isServiceOwnershipError, ServiceOwnershipError } from "../../../src/service"; +import { serviceStopOutcomeFromWindows } from "../../../src/service/windows-service-stop"; +import type { WindowsCheckedStopResult } from "../../../src/service/windows-checked-stop"; import { decideStartExitTeardown } from "../../../src/cli/dispatch"; import { repoPath } from "../../helpers/repo-root"; @@ -193,26 +195,30 @@ describe("Grok fence lifecycle wiring", () => { test("a stopped scheduler is verified across the respawn window before stop succeeds", () => { const stopFn = sliceFn(CLI_SOURCE, "async function handleStop(", "async function handleUninstall("); - // killWindowsSchedulerWrappers is best-effort and the `:loop` wrapper respawns after - // ~5s, so "stopped" alone is not a proven-down proxy. An update that trusts it can - // start replacing files during the dead interval (#3008). - expect(stopFn).toContain("proxyStillLiveAfterStop({ canRespawn: true })"); + // The `:loop` wrapper respawns after ~5s, so "stopped" alone is not a proven-down + // proxy. An update that trusts it can start replacing files during the dead interval + // (#3008). On Windows the window is Part B of the checked stop (#7771); its behavior is + // covered in tests/windows/windows-stop-callers.test.ts and the parent-stop runner. + expect(stopFn).toContain("if (await stopWindowFailed(record)) { stopFailed = true; ownershipBlocked = true; }"); + const wiring = readFileSync(repoPath("src", "cli", "stop-windows-checked.ts"), "utf8"); + expect(wiring).toContain("proxyStillLiveAfterStop({ canRespawn: true })"); + expect(wiring).toContain("confirmWindowsServiceStop("); // A survivor is an ordinary failure AND blocks shared teardown: restoring client // config while the proxy runs leaves both pointing at each other. expect(stopFn).toContain("stopFailed = true;"); expect(stopFn).toContain("ownershipBlocked = true;"); }); - test("only Task Scheduler earns the respawn wait", () => { + test("only a stop that can be respawned earns the respawn wait", () => { const stopFn = sliceFn(CLI_SOURCE, "async function handleStop(", "async function handleUninstall("); const serviceSource = readFileSync(repoPath("src", "service", "orchestration.ts"), "utf8"); // schtasks /end leaves the `cmd :loop` wrapper alive to respawn its child (#764). // launchd, systemd and WinSW are down when they report stopped, so charging them a // seven-second poll on every ocx stop would be a regression in ordinary use. expect(serviceSource).toContain('"absent" | "stopped" | "stopped-respawnable" | "failed"'); - const windowsOps = readFileSync(repoPath("src", "service", "windows-ops.ts"), "utf8"); - expect(windowsOps).toContain('schedulerStopped ? "stopped-respawnable" : "stopped"'); expect(stopFn).toContain("if (schedulerCanRespawn && !ownershipBlocked)"); + // #7771: a legacy respawnable outcome or a Windows Part A that stopped (Part B follows). + expect(stopFn).toContain('schedulerCanRespawn = serviceStop === "stopped-respawnable" || windowsPartBDue();'); // The wait is gated on the scheduler flag, not on "a service stopped". expect(stopFn).not.toContain("if (stoppedService && !ownershipBlocked)"); }); @@ -438,7 +444,7 @@ describe("POST /api/stop teardown", () => { const handler = sliceFn(MANAGEMENT_SOURCE, '"/api/stop"', "/api/codex-auth/"); expect(handler).toContain("const sibling = siblingOfLivePort() !== null;"); expect(handler).toContain('const respawnRisk = holdsReceipt || sibling ? "none" : installedServiceRespawnRisk();'); - expect(handler).toContain('serviceStop = sibling ? "absent" : stopServiceIfInstalledDetailed();'); + expect(handler).toContain('serviceStop = sibling ? "absent" : await (await import("./stop-service-route")).routeServiceStop(url, holdsReceipt);'); }); test("strips the Grok fence on an accepted stop", () => { @@ -461,21 +467,19 @@ describe("POST /api/stop teardown", () => { // wrong thing, and `ocx stop` was the command the API told them to run (#3008). expect(serviceSource).toContain('"absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown"'); // Behavioural, because a source-text assertion cannot tell whether an unreadable probe - // is still being folded into the generic failure. - expect(classifyWindowsServiceStop({ stopped: false, failed: false, schedulerStopped: false, stateUnknown: true })) - .toBe("state-unknown"); - // A readable failure outranks it — something actually refused to stop. - expect(classifyWindowsServiceStop({ stopped: false, failed: true, schedulerStopped: false, stateUnknown: true })) - .toBe("failed"); - // And an unreadable state outranks success: a scheduler we cannot see may respawn. - expect(classifyWindowsServiceStop({ stopped: true, failed: false, schedulerStopped: true, stateUnknown: true })) - .toBe("state-unknown"); - expect(classifyWindowsServiceStop({ stopped: true, failed: false, schedulerStopped: true, stateUnknown: false })) - .toBe("stopped-respawnable"); - expect(classifyWindowsServiceStop({ stopped: true, failed: false, schedulerStopped: false, stateUnknown: false })) - .toBe("stopped"); - expect(classifyWindowsServiceStop({ stopped: false, failed: false, schedulerStopped: false, stateUnknown: false })) - .toBe("absent"); + // is still being folded into the generic failure. Since #7771 the Windows outcome is + // Part A's: unreadable evidence is state-unknown, a refusal or survivor is failed, and + // Task Scheduler no longer reports stopped-respawnable (Part B covers the window). + const partA = (outcome: WindowsCheckedStopResult["outcome"]) => serviceStopOutcomeFromWindows({ + outcome, reason: null, detail: "", registration: { scheduler: "present", winsw: "nonexistent" }, + bound: [], unbound: [], terminated: [], changed: false, + }); + expect(partA("unknown")).toBe("state-unknown"); + expect(partA("denied")).toBe("failed"); + expect(partA("survived")).toBe("failed"); + expect(partA("unverified-budget")).toBe("failed"); + expect(partA("stopped")).toBe("stopped"); + expect(partA("absent")).toBe("absent"); const stopFn = sliceFn(CLI_SOURCE, "async function handleStop(", "async function handleUninstall("); expect(stopFn).toContain('if (serviceStop === "state-unknown")'); const unknownBranch = stopFn.slice(stopFn.indexOf('if (serviceStop === "state-unknown")'), stopFn.indexOf('if (serviceStop === "state-unknown")') + 700); @@ -499,7 +503,7 @@ describe("POST /api/stop teardown", () => { // stopServiceIfInstalled collapses "failed" into the same false as "not installed", so // this route used to tear down shared config while a manager that refused to stop was // still there to respawn the proxy (#3008). - expect(handler).toContain("stopServiceIfInstalledDetailed()"); + expect(handler).toContain("routeServiceStop(url, holdsReceipt)"); expect(handler).not.toContain("stopServiceIfInstalled();"); expect(handler).toContain('if (serviceStop === "failed")'); expect(handler.indexOf('if (serviceStop === "failed")')).toBeLessThan(handler.indexOf("await performStopTeardown")); @@ -512,7 +516,7 @@ describe("POST /api/stop teardown", () => { // sends a bare request on every backend. expect(handler).toContain('const respawnRisk = holdsReceipt || sibling ? "none" : installedServiceRespawnRisk();'); expect(handler).toContain('code: "respawnable_service"'); - expect(handler.indexOf("installedServiceRespawnRisk()")).toBeLessThan(handler.indexOf("stopServiceIfInstalledDetailed()")); + expect(handler.indexOf("installedServiceRespawnRisk()")).toBeLessThan(handler.indexOf("routeServiceStop(url, holdsReceipt)")); // The refusal must say nothing was changed, because nothing was. expect(handler).toContain("Nothing was changed."); // An unreadable scheduler state is its own answer: sending that operator to `ocx stop` diff --git a/tests/service/service-desktop-runtime-preflight.test.ts b/tests/service/service-desktop-runtime-preflight.test.ts index e60e6f75c2a..340f01fc344 100644 --- a/tests/service/service-desktop-runtime-preflight.test.ts +++ b/tests/service/service-desktop-runtime-preflight.test.ts @@ -11,6 +11,7 @@ import { installFreshWindowsSchedulerSafely, installServiceSafely } from "../../ import { repairService } from "../../src/service/repair"; import { buildWindowsTaskXml } from "../../src/service/windows-taskxml"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; const roots: string[] = []; afterEach(() => { for (const root of roots.splice(0)) removeTreeWithRetry(root); }); @@ -57,9 +58,8 @@ function fixture(supervised: boolean, passes: boolean) { const run = (action: string) => { if (action === "install") return installServiceSafely("scheduler", consume("install"), { ...admission, inspectSupervision, - diagnose: () => { order.push("diagnose"); return diagnostic; }, - managerOps: () => ({ status: () => "present", stop: () => { order.push("stop"); } }), - killServiceWrappers: () => { order.push("wrappers"); }, + windowsPartA: async () => { order.push("stop"); return partAResult("stopped"); }, + windowsPartB: async () => { order.push("confirm"); return partBResult("freed"); }, stopTrackedProxy: async () => { order.push("proxy"); }, }); if (action === "fresh install") return installFreshWindowsSchedulerSafely({ diff --git a/tests/service/service-install-wrapper-cleanup.test.ts b/tests/service/service-install-wrapper-cleanup.test.ts index 0b4dab028fa..5c37883677a 100644 --- a/tests/service/service-install-wrapper-cleanup.test.ts +++ b/tests/service/service-install-wrapper-cleanup.test.ts @@ -1,92 +1,103 @@ import { describe, expect, test } from "bun:test"; -import { prepareServiceInstall } from "../../src/service"; +import { installServiceSafely, prepareServiceInstall } from "../../src/service"; +import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; +// #7771: on Windows, install and backend switch stop the old installation through the checked +// stop. Part A ends the managers and the bound wrappers (or refuses with nothing replaced), +// the tracked proxy stops next, and Part B must find the port free before anything is +// published or started. describe("service install preparation wrapper cleanup", () => { - test("install preparation kills surviving scheduler wrappers before the tracked proxy stop", async () => { - // The defect: `schtasks /end` ends the task instance but the detached cmd - // `:loop` wrapper survives and respawns its child seconds later — landing - // after the one-shot tracked-proxy kill and holding the install port. Model - // the respawner: while the wrapper lives, a stopped proxy comes back. + test("install preparation ends the respawning wrapper (Part A) before the tracked proxy stop", async () => { + // The defect: schtasks /end ends the task instance but the detached cmd :loop wrapper + // survives and respawns its child seconds later, landing after the one-shot tracked-proxy + // kill and holding the install port. Model the respawner: while it lives, a stopped proxy + // comes back. const calls: string[] = []; let wrapperAlive = true; let respawned = false; await prepareServiceInstall("scheduler", { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps: () => ({ - status: () => "present", - // `schtasks /end` ends the task instance; it cannot reach the detached wrapper. - stop: () => { calls.push("stop:scheduler"); }, - }), - killServiceWrappers: () => { calls.push("kill:wrappers"); wrapperAlive = false; }, + windowsPartA: async () => { calls.push("partA"); wrapperAlive = false; return partAResult("stopped"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); if (wrapperAlive) respawned = true; }, }); - expect(calls).toEqual(["stop:scheduler", "kill:wrappers", "stop:standalone"]); + expect(calls).toEqual(["partA", "stop:standalone", "partB"]); expect(respawned).toBe(false); }); - test("install preparation kills stray scheduler wrappers even when no manager stop runs", async () => { - // A registration deleted mid-uninstall leaves no task for `schtasks /end`, but - // the detached wrapper may still be looping: it must not respawn the old proxy - // onto the port the fresh install is about to claim. - let killed = false; + test("a stray wrapper of a deleted task that cannot be bound refuses instead of being killed by path", async () => { + // With nothing registered an orphaned wrapper is never bound (unbound-wrapper): the install + // stops and names the manual step rather than replacing assets under a live loop. const calls: string[] = []; - await prepareServiceInstall("scheduler", { + await expect(prepareServiceInstall("scheduler", { platform: "win32", - diagnose: () => ({ supported: true, installed: false, enabled: false, running: false, viable: false, startable: false, stale: false, conflict: false, backend: null, summary: "test" }), - managerOps: () => ({ - status: () => null, - stop: () => { calls.push("stop:scheduler"); throw new Error("no task to stop"); }, - }), - killServiceWrappers: () => { calls.push("kill:wrappers"); killed = true; }, + windowsPartA: async () => partAResult("survived", { reason: "unbound-wrapper", registration: { scheduler: "absent", winsw: "nonexistent" } }), stopTrackedProxy: async () => { calls.push("stop:standalone"); }, - }); - expect(killed).toBe(true); - expect(calls).toEqual(["kill:wrappers", "stop:standalone"]); + })).rejects.toThrow(/could not be proven stopped \(survived\)[\s\S]*could not be tied to the service/); + expect(calls).toEqual([]); }); - test("a manager that refuses to stop aborts install preparation before the wrapper kill", async () => { - // Propagating the refusal is the fail-closed contract — and killing the wrapper - // of a service whose task is still running would break that surviving service. + test("a manager that refuses to stop aborts install preparation before anything else", async () => { const calls: string[] = []; await expect(prepareServiceInstall("scheduler", { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps: () => ({ - status: () => "present", - stop: () => { calls.push("stop:scheduler"); throw new Error("end refused"); }, - }), - killServiceWrappers: () => { calls.push("kill:wrappers"); }, + windowsPartA: async () => { calls.push("partA"); return partAResult("denied"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + })).rejects.toThrow("could not be proven stopped (denied)"); + expect(calls).toEqual(["partA"]); + }); + + test("install and backend switch stop when another program holds the port, naming it", async () => { + for (const backend of ["scheduler", "native"] as const) { + let installed = false; + await expect(installServiceSafely(backend, () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => partAResult("stopped"), + windowsPartB: async () => partBResult("port-held-by-foreign"), + stopTrackedProxy: async () => {}, + })).rejects.toThrow(/nothing was published or started\. Another program \(PID 777\) holds port 10100/); + expect(installed).toBe(false); + } + await expect(prepareServiceInstall("scheduler", { + platform: "win32", windowsPartA: async () => partAResult("stopped"), + windowsPartB: async () => partBResult("survived"), stopTrackedProxy: async () => {}, + })).rejects.toThrow("nothing was published or started"); + }); + + test("nothing registered: no Part B and no hold", async () => { + const calls: string[] = []; + await prepareServiceInstall("scheduler", { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("absent"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, - })).rejects.toThrow("end refused"); - expect(calls).toEqual(["stop:scheduler"]); + }); + expect(calls).toEqual(["partA", "stop:standalone"]); }); - test("non-Windows install preparation never invokes the scheduler wrapper kill", async () => { - let killed = 0; + test("non-Windows install preparation never reaches the checked Windows stop", async () => { + let partA = 0; await prepareServiceInstall("scheduler", { platform: "linux", managerOps: () => ({ status: () => "present", stop: () => {} }), - killServiceWrappers: () => { killed++; }, + windowsPartA: async () => { partA++; return partAResult("stopped"); }, stopTrackedProxy: async () => {}, }); - expect(killed).toBe(0); + expect(partA).toBe(0); }); - test("live Desktop supervision refuses install preparation before the wrapper kill", async () => { - // The pre-mutation supervision refusal verified here stays ahead of every - // cleanup step: a desktop-owned runtime keeps its respawner until the - // operator quits Desktop and retries. + test("live Desktop supervision refuses install preparation before the checked stop", async () => { + // The pre-mutation supervision refusal stays ahead of every cleanup step: a desktop-owned + // runtime keeps its respawner until the operator quits Desktop and retries. const calls: string[] = []; await expect(prepareServiceInstall("scheduler", { platform: "win32", inspectSupervision: () => ({ kind: "desktop", runtimePid: 4242, supervisorPid: 4200, app: "opencodex-desktop", proxy: "ocx" }), - diagnose: () => { calls.push("diagnose"); return { supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }; }, - managerOps: () => { calls.push("manager"); return { status: () => "present", stop: () => {} }; }, - killServiceWrappers: () => { calls.push("kill:wrappers"); }, + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, })).rejects.toThrow(/OpenCodex Desktop supervises the running proxy/); expect(calls).toEqual([]); diff --git a/tests/service/service-runtime-preflight.test.ts b/tests/service/service-runtime-preflight.test.ts index c6a2aa4a9ad..211f79ec869 100644 --- a/tests/service/service-runtime-preflight.test.ts +++ b/tests/service/service-runtime-preflight.test.ts @@ -14,6 +14,7 @@ import { recordOwnedConfigPath, CONFIG_OWNER_FILE, CONFIG_UNINSTALL_MANIFEST } f import type { ServiceDiagnostic } from "../../src/service/diagnostics"; import { windowsEnvIndirectBatchValue } from "../../src/lib/win-paths"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; import { repoPath } from "../helpers/repo-root"; const runtime: DurableBunRuntime = { path: process.execPath, source: "override", overrideEnv: "OPENCODEX_BUN_PATH" }; @@ -43,9 +44,8 @@ describe("service selected-runtime admission", () => { const deny = () => { throw new RuntimePreflightError(reason); }; const deps = { platform: "win32" as const, selectRuntime: () => runtime, configDir: () => temporary(), assertRuntimeWritable: deny }; await expect(installServiceSafely("scheduler", () => { effects.push("install"); }, { - ...deps, diagnose: () => { effects.push("diagnose"); return diagnostic; }, - managerOps: () => ({ status: () => "present", stop: () => { effects.push("stop"); } }), - killServiceWrappers: () => {}, + ...deps, + windowsPartA: async () => { effects.push("stop"); return partAResult("stopped"); }, stopTrackedProxy: async () => { effects.push("proxy"); }, })).rejects.toMatchObject({ code: "OCX_RUNTIME_PREFLIGHT_FAILED", reason }); const home = join(temporary(), "absent-home"); @@ -89,13 +89,12 @@ describe("service selected-runtime admission", () => { }, { platform: "win32", configDir: temporary, selectRuntime: () => { selections++; return selected; }, assertRuntimeWritable: admitted => { order.push("preflight"); expect(admitted).toEqual(runtime); selected.path = "unprobed.exe"; }, - diagnose: () => diagnostic, - managerOps: () => ({ status: () => "present", stop: () => { order.push("stop"); } }), - killServiceWrappers: () => { order.push("wrappers"); }, + windowsPartA: async () => { order.push("stop"); return partAResult("stopped"); }, + windowsPartB: async () => { order.push("confirm"); return partBResult("freed"); }, stopTrackedProxy: async () => { order.push("proxy"); }, }); expect(selections).toBe(1); - expect(order).toEqual(["preflight", "stop", "wrappers", "proxy", "install"]); + expect(order).toEqual(["preflight", "stop", "proxy", "confirm", "install"]); }); for (const backend of ["scheduler", "native"] as const) { @@ -138,9 +137,9 @@ describe("service selected-runtime admission", () => { }; const ops = platformOps(backend, { platform: "win32", windowsInstall: effects })!; const result = installServiceSafely(backend, ops.install, { - ...admission, diagnose: () => ({ ...diagnostic, backend }), - managerOps: () => ({ status: () => "present", stop: () => { order.push("stop"); } }), - killServiceWrappers: () => { order.push("wrappers"); }, + ...admission, + windowsPartA: async () => { order.push("stop"); return partAResult("stopped"); }, + windowsPartB: async () => { order.push("confirm"); return partBResult("freed"); }, stopTrackedProxy: async () => { order.push("proxy"); }, }); if (refuses) { @@ -149,8 +148,8 @@ describe("service selected-runtime admission", () => { } else { await result; expect(order).toEqual(backend === "scheduler" - ? ["preflight", "stop", "wrappers", "proxy", "ownership", "remove-native", "installer-stop", "assets", "/create", "/run", "state"] - : ["preflight", "stop", "wrappers", "proxy", "account", "native-config", "/query", "winsw", "state"]); + ? ["preflight", "stop", "proxy", "confirm", "ownership", "remove-native", "installer-stop", "assets", "/create", "/run", "state"] + : ["preflight", "stop", "proxy", "confirm", "account", "native-config", "/query", "winsw", "state"]); expect(JSON.parse(readFileSync(join(home, "state.json"), "utf8"))).toMatchObject({ backend, bunPath: runtime.path }); } expect(probes).toBe(1); expect(selections).toBe(1); diff --git a/tests/service/service.test.ts b/tests/service/service.test.ts index 03c5f7bcbec..1dc9d854562 100644 --- a/tests/service/service.test.ts +++ b/tests/service/service.test.ts @@ -18,6 +18,7 @@ import { resolveCurrentWindowsPrincipal, setWindowsPrincipalRunnerForTests } fro import { setAsyncIcaclsRunnerForTests, setIcaclsRunnerForTests } from "../../src/lib/windows-secret-acl"; import type { OcxConfig } from "../../src/types"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { ALL_PART_A_FAILURES, partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; import { repoRoot } from "../helpers/repo-root"; const TEST_WINDOWS_TASK_SID = "S-1-5-21-111-222-333-1001"; @@ -2269,24 +2270,16 @@ describe("service lifecycle cleanup ordering", () => { ]); }); - test("service install stops the recorded backend, requested backend, and standalone before loading assets", async () => { + test("Windows service install runs the checked stop, the standalone stop and Part B before loading assets", async () => { const calls: string[] = []; - const managerOps = (backend: "scheduler" | "native") => ({ - status: () => { calls.push(`status:${backend}`); return "present"; }, - stop: () => { calls.push(`stop:${backend}`); }, - }); await installServiceSafely("native", () => { calls.push("install:native"); }, { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps, - killServiceWrappers: () => { calls.push("kill:wrappers"); }, + // #7771: Part A stops every registered manager (both on a conflicting install) and ends the bound wrappers. + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, }); - expect(calls).toEqual([ - "status:scheduler", "stop:scheduler", - "status:native", "stop:native", - "kill:wrappers", "stop:standalone", "install:native", - ]); + expect(calls).toEqual(["partA", "stop:standalone", "partB", "install:native"]); }); test("legacy systemd reports an absent unit without stopping cleanup or blocking install", async () => { @@ -2379,21 +2372,12 @@ describe("service lifecycle cleanup ordering", () => { }); test("service install fails closed before install on manager or standalone cleanup errors", async () => { - for (const failure of ["status", "stop", "standalone"] as const) { + for (const failure of ["partA", "partB", "standalone"] as const) { let installed = false; const run = installServiceSafely("scheduler", () => { installed = true; }, { platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: true, startable: true, stale: false, conflict: false, backend: "scheduler", summary: "test" }), - managerOps: () => ({ - status: () => { - if (failure === "status") throw new Error("status failed"); - return "present"; - }, - stop: () => { - if (failure === "stop") throw new Error("stop failed"); - }, - }), - killServiceWrappers: () => {}, + windowsPartA: async () => { if (failure === "partA") throw new Error("partA failed"); return partAResult("stopped"); }, + windowsPartB: async () => { if (failure === "partB") throw new Error("partB failed"); return partBResult("freed"); }, stopTrackedProxy: async () => { if (failure === "standalone") throw new Error("standalone failed"); }, @@ -2403,16 +2387,14 @@ describe("service lifecycle cleanup ordering", () => { } }); - test("conflicting Windows install preparation stops both managers", async () => { - const stopped: string[] = []; - await prepareServiceInstall("scheduler", { - platform: "win32", - diagnose: () => ({ supported: true, installed: true, enabled: true, running: true, viable: false, startable: false, stale: false, conflict: true, backend: "scheduler", summary: "test" }), - managerOps: backend => ({ status: () => "present", stop: () => { stopped.push(backend); } }), - killServiceWrappers: () => { stopped.push("wrappers"); }, - stopTrackedProxy: async () => {}, - }); - expect(stopped).toEqual(["scheduler", "native", "wrappers"]); + test("Windows install preparation refuses before replacing anything unless Part A stopped or found nothing", async () => { + for (const outcome of ALL_PART_A_FAILURES) { + const stopped: string[] = []; + await expect(prepareServiceInstall("scheduler", { platform: "win32", windowsPartA: async () => partAResult(outcome), + windowsPartB: async () => { stopped.push("partB"); return partBResult("freed"); }, + stopTrackedProxy: async () => { stopped.push("standalone"); } })).rejects.toThrow(`could not be proven stopped (${outcome})`); + expect(stopped).toEqual([]); + } }); test("direct service stop kills the tracked proxy before restoring native Codex", async () => { @@ -2474,7 +2456,7 @@ describe("service lifecycle cleanup ordering", () => { test("Windows service uninstall verifies task deletion before removing assets", async () => { const service = await readText("src/service/windows-ops.ts"); - const uninstallWindows = service.slice(service.indexOf("function uninstallWindows()"), service.indexOf("function classifyWindowsServiceStop(")); + const uninstallWindows = service.slice(service.indexOf("function uninstallWindows()")); expect(uninstallWindows).toContain("probeWindowsSchedulerTask(TASK)"); expect(uninstallWindows).toContain("windowsServiceScriptPath()"); @@ -2493,12 +2475,12 @@ describe("service lifecycle cleanup ordering", () => { expect(service).toContain('type TrackedProxyCleanupResult = "none" | "stale" | "stopped";'); const health = await readText("src/service/health.ts"); expect(health).toContain('import { proxyIdentityAt } from "../server/proxy-liveness";'); - expect(service).toContain("async function stopTrackedProxyIfRunning(): Promise"); + expect(service).toContain("async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise"); expect(service).toContain("...SERVICE_STOP_LIVENESS"); expect(service).toContain("deadlineAt:"); expect(service).toContain("SERVICE_STOP_LIVENESS"); - expect(service).toContain("await stopProxy(trackedKillPid);"); - expect(service).toContain("await stopProxy(liveKillPid);"); + expect(service).toContain("await stopProxy(trackedKillPid, io);"); + expect(service).toContain("await stopProxy(liveKillPid, io);"); expect(service).toContain("removePid(pid);"); expect(service).toContain('return "stopped";'); }); @@ -2540,7 +2522,7 @@ describe("service lifecycle cleanup ordering", () => { test("service command cleanup logs kill failures without skipping restore/delete", async () => { const service = await readText("src/service/orchestration.ts"); - expect(service).toContain("async function stopTrackedProxyForServiceCommand(): Promise"); + expect(service).toContain("async function stopTrackedProxyForServiceCommand(io: TrackedProxyStopIo = {}): Promise"); expect(service).toContain("catch (err)"); expect(service).toContain("Failed to stop proxy"); expect(service).toContain('return "none";'); diff --git a/tests/service/stop-deferred-teardown.test.ts b/tests/service/stop-deferred-teardown.test.ts index 86d418b9ac6..5d6bff20d4b 100644 --- a/tests/service/stop-deferred-teardown.test.ts +++ b/tests/service/stop-deferred-teardown.test.ts @@ -1284,7 +1284,7 @@ test("the route refuses a self-unload before the manager is touched", () => { expect(handler).toContain('code: "self_unload_service"'); // Same invariant the Windows guard carries: refuse BEFORE acting, and say so. expect(handler.indexOf('code: "self_unload_service"')) - .toBeLessThan(handler.indexOf("stopServiceIfInstalledDetailed()")); + .toBeLessThan(handler.indexOf("routeServiceStop(url, holdsReceipt)")); const branch = handler.slice(handler.indexOf('code: "self_unload_service"'), handler.indexOf('code: "self_unload_service"') + 600); expect(branch).toContain("Nothing was changed."); expect(branch).toContain("ocx stop"); diff --git a/tests/update/update-effect-authority.test.ts b/tests/update/update-effect-authority.test.ts index 0845f47f76f..4f07e4cc837 100644 --- a/tests/update/update-effect-authority.test.ts +++ b/tests/update/update-effect-authority.test.ts @@ -50,7 +50,7 @@ for (const phase of ["identity", "liveness", "kill-liveness", "escalation", "sta if (phase !== "stable") { home = "same"; expect(policy.authorizeMutation("publish")).toBe(false); } }); -for (const operation of ["wrapper", "winsw", "scheduler", "tray", "pid", "runtime", "registry"]) +for (const operation of ["winsw", "tray", "pid", "runtime", "registry"]) for (const changed of [true, false]) test(`leaf ${operation} gates after preparation (${changed})`, () => { let home: "same" | "unknown" = "same", effects = 0; const policy = createUpdateServiceHomePolicy(() => ({ kind: home }), () => {}); @@ -59,26 +59,12 @@ for (const operation of ["wrapper", "winsw", "scheduler", "tray", "pid", "runtim throw Object.assign(new Error("refused"), { code: "EUPDATEAUTHORITY" }); }; const effect = () => { effects++; return "stopped"; }; let invoke: () => unknown; - if (operation === "wrapper") { - const fn = load("src/lib/windows-service-wrappers.ts", "killWindowsSchedulerWrappers", { - process: { platform: "win32" }, resolveTrustedWindowsPowerShellExe: prepare, - windowsWrapperKillScript: () => "fixture", spawnSync: effect, - }); - const wrapped = adapter("guardedKillWindowsWrappers", { killWindowsSchedulerWrappers: fn }); - invoke = () => wrapped(policy.authorizeMutation, { scriptPath: "script", launcherPath: "launcher" }); - } else if (operation === "winsw") { + if (operation === "winsw") { const run = load("src/lib/winsw.ts", "runWinsw", { winswExePath: prepare, execFileSync: effect, SERVICE_STOPWAIT_TIMEOUT_MS: 1, SERVICE_COMMAND_TIMEOUT_MS: 1, }); const fn = load("src/lib/winsw.ts", "stopWinswService", { runWinsw: run, statusWinswRaw: () => "stopped" }); invoke = () => adapter("guardedStopWinsw", { stopWinswService: fn })(policy.authorizeMutation); - } else if (operation === "scheduler") { - const query = load("src/service/windows-scheduler.ts", "querySchtasks", { - isTestHomeGuardArmed: () => false, querySchtasksForTests: null, windowsSchtasks: prepare, runFile: effect, - }); - const schtasks = load("src/service/windows-scheduler.ts", "schtasks", { querySchtasks: query, toWindowsSchtasksError: (e: unknown) => e }); - const stop = load("src/service/windows-ops.ts", "stopWindows", { schtasks, TASK: "fixture", isWindowsSchedulerEndBenign: () => false }); - invoke = () => adapter("guardedStopWindows", { stopWindows: stop })(policy.authorizeMutation); } else if (operation === "tray") { const signal = load("src/tray/windows.ts", "signalTrayStop", { readHeartbeat: () => ({}), windowsPowerShellPath: prepare, currentEntry: () => ({}), @@ -107,6 +93,24 @@ for (const operation of ["wrapper", "winsw", "scheduler", "tray", "pid", "runtim if (changed) { home = "same"; expect(policy.authorizeMutation("publish")).toBe(false); } }); +// #7771: the scheduler stop and the wrapper sweep are one checked operation (Part A). Its service +// gate is the authorizer Part A runs at entry and before every terminate. +for (const changed of [true, false]) test(`leaf checked scheduler stop gates after preparation (${changed})`, async () => { + let home: "same" | "unknown" = "same", effects = 0; + const policy = createUpdateServiceHomePolicy(() => ({ kind: home }), () => {}); + const runWindowsServiceStopPartA = async (request: { authorize: () => void; allowUnderOwnWrapper?: boolean }) => { + expect(request.allowUnderOwnWrapper).toBe(true); + if (changed) home = "unknown"; // preparation: the registration probe and the bind + await request.authorize(); + effects++; + return { outcome: "stopped" }; + }; + const invoke = adapter("guardedStopWindowsChecked", { runWindowsServiceStopPartA })(policy.authorizeMutation); + if (changed) await expect(invoke).rejects.toThrow(); else expect(await invoke).toBe("stopped"); + expect(effects).toBe(changed ? 0 : 1); + if (changed) { home = "same"; expect(policy.authorizeMutation("publish")).toBe(false); } +}); + test("Windows kill refusal propagates even when a later liveness check would report exit", () => { let reads = 0; const fn = adapter("guardedKillProxy", { diff --git a/tests/update/update-recovery-boundaries.test.ts b/tests/update/update-recovery-boundaries.test.ts index bde387661c9..e3d4f41ac34 100644 --- a/tests/update/update-recovery-boundaries.test.ts +++ b/tests/update/update-recovery-boundaries.test.ts @@ -136,10 +136,12 @@ for (const outcome of ["false", "throw", "promise"] as const) test(`async transa test("production pinned caller checks current service permission inside its wrapper callback", async () => { const prepare = section(worker, "function preparePortForPinnedStart(", "function spawnDetachedStart("); - const caller = section(worker, ' if (!homePolicy.authorizeMutation("runtime")) return false;\n if (preparePort(', " const ready = await waitForGhost("); + // #7771: the service stop now runs right before the synchronous preparation, so its outcome can + // abort; the service permission is still read immediately before it. + const caller = section(worker, ' if (!homePolicy.authorizeMutation("runtime")) return false;\n // #7771', " const ready = await waitForGhost("); for (const changed of ["same", "absent", "unknown"] as const) { let reads = 0, wrappers = 0, cleanup = 0; - const homePolicy = createUpdateServiceHomePolicy(() => ({ kind: ++reads < 3 ? "same" : changed }), () => {}); + const homePolicy = createUpdateServiceHomePolicy(() => ({ kind: ++reads < 2 ? "same" : changed }), () => {}); const result = await execute(prepare + "const preparePort = preparePortForPinnedStart;" + caller, { homePolicy, reclaimGate: () => homePolicy.authorizeMutation("runtime"), job: {}, port: 19001, listPids: () => [], aliveFn: () => false, verifyOcx: () => null, serviceInstalled: true, @@ -241,7 +243,7 @@ for (const caller of callers) for (const outcome of ["vetoed", "refused", true] test("production Node full stop refuses lost service permission before Windows wrapper cleanup", async () => { const stop = section(node, " const stopNeeded =", " const replacementOwnership ="); const childStop = section(readFileSync(repoPath("src/service/orchestration.ts"), "utf8"), - "export function stopServiceIfInstalledDetailed()", "/** Delete install-state files").replace("export ", ""); + "export async function stopServiceIfInstalledReport(", "/** Delete install-state files").replaceAll("export ", ""); const guard = section(readFileSync(repoPath("src/service/guards.ts"), "utf8"), "export function assertServiceEnvironmentMatchesInstall(", "/**\n * Refuse a management").replace("export ", ""); for (const [captured, changed] of [[true, "same"], [true, "absent"], [true, "unknown"], [false, "absent"]] as const) { @@ -260,8 +262,10 @@ test("production Node full stop refuses lost service permission before Windows w resolveServiceState: () => ({ kind: changed === "same" ? "state" : "none", state: {} }), currentCodexHome: () => "", serviceCodexHomeMatchesInstall: () => true, currentOpenCodexHome: () => "", servicePathMatchesInstall: () => true, - probeWindowsSchedulerTask: () => ({ status: "absent" }), statusWinswRaw: () => "nonexistent", - killWindowsServiceWrapperProcesses: () => { wrappers++; }, classifyWindowsServiceStop: () => "absent", + // #7771: Part A of the checked stop is the Windows wrapper cleanup now; it is called + // synchronously at the first await, so a lost permission must refuse before it. + runWindowsServiceStopPartA: async () => { wrappers++; return { outcome: "absent" }; }, + serviceStopOutcomeFromWindows: () => "absent", spawnSync: () => { stops++; const js = new Bun.Transpiler({ loader: "ts" }).transformSync(guard + childStop); diff --git a/tests/update/update-windows-wrapper-authority.test.ts b/tests/update/update-windows-wrapper-authority.test.ts index 1b6456d71ba..258afcb9c9d 100644 --- a/tests/update/update-windows-wrapper-authority.test.ts +++ b/tests/update/update-windows-wrapper-authority.test.ts @@ -7,7 +7,7 @@ import { createUpdateServiceHomePolicy, type UpdateServiceHomeAuthority } from " const helper = readFileSync(repoPath("src/update/job-windows-restart.ts"), "utf8"); const worker = readFileSync(repoPath("src/update/job.ts"), "utf8"); -const helperBody = helper.slice(helper.indexOf("export function stopWindowsServiceWrappersBestEffort")).replace("export ", ""); +const helperBody = helper.slice(helper.indexOf("export const WINDOWS_SERVICE_STOP_ABORT")).replaceAll("export ", ""); function execute(body: string, deps: object): any { // Complete production bodies; module loading and external OS/store observations are injected. const js = new Bun.Transpiler({ loader: "ts" }).transformSync(body.replaceAll('await import("../service")', "await moduleIo.service()")); @@ -16,28 +16,42 @@ function execute(body: string, deps: object): any { const authority = (kind: "same" | "absent" | "unknown" | "foreign"): UpdateServiceHomeAuthority => kind === "foreign" ? { kind, subject: "a".repeat(64) } : { kind }; -for (const phase of ["stop", "throwing-stop", "backend-native", "backend-throw"] as const) { +/** + * Part A of the checked stop (#7771) as the updater sees it: the authorizer at entry, the + * manager stop, then the authorizer again before the wrapper terminate. + */ +function checkedStop(events: string[], home: () => UpdateServiceHomeAuthority, change: () => void) { + return async (authorize: () => void) => { + authorize(); + events.push(`stop:${home().kind}`); + change(); + authorize(); + events.push(`terminate:${home().kind}`); + return "stopped"; + }; +} + +for (const phase of ["checked", "backend-native", "backend-throw"] as const) { for (const changed of ["same", "absent", "unknown", "foreign"] as const) { - test(`Windows wrapper helper checks ${changed} service authority after ${phase}`, () => { + test(`Windows service stop checks ${changed} service authority after ${phase}`, async () => { let home = authority("same"); const events: string[] = []; const homePolicy = createUpdateServiceHomePolicy(() => home, () => {}); - execute(helperBody + ';stopWindowsServiceWrappersBestEffort(homePolicy.authorizeMutation);', { - homePolicy, process: { platform: "win32" }, join, getConfigDir: () => "/fixture/own", + const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { + homePolicy, process: { platform: "win32" }, readServiceBackend: () => { if (phase.startsWith("backend")) home = authority(changed); if (phase === "backend-throw") throw new Error("fixture backend read"); return phase === "backend-native" ? "native" : "scheduler"; }, stopWinswService: () => events.push(`native:${home.kind}`), - stopWindows: () => { - events.push(`stop:${home.kind}`); home = authority(changed); - if (phase === "throwing-stop") throw new Error("fixture scheduler stop"); - }, - killWindowsSchedulerWrappers: () => events.push(`kill:${home.kind}`), + checkedStop: checkedStop(events, () => home, () => { if (phase === "checked") home = authority(changed); }), }); - const stable = phase === "backend-native" ? ["native:same"] : phase === "backend-throw" ? ["kill:same"] : ["stop:same", "kill:same"]; - expect(events).toEqual(changed === "same" ? stable : phase.startsWith("backend") ? [] : ["stop:same"]); + // An unreadable backend falls through to the checked stop, which probes both backends. + const stable = phase === "backend-native" ? ["native:same"] : ["stop:same", "terminate:same"]; + expect(events).toEqual(changed === "same" ? stable : phase === "checked" ? ["stop:same"] : []); + // Anything but a proven stop aborts the restart. + expect(result).toBe(changed === "same"); // The same policy must retain unknown/foreign refusal even if registration re-matches. home = authority("same"); expect(homePolicy.authorizeMutation("service")).toBe(changed === "same" || changed === "absent"); @@ -45,10 +59,22 @@ for (const phase of ["stop", "throwing-stop", "backend-native", "backend-throw"] } } +for (const outcome of ["survived", "denied", "unknown", "unverified-budget", "absent", "stopped"] as const) { + test(`Windows service stop aborts the restart unless Part A is stopped or absent: ${outcome}`, async () => { + const homePolicy = createUpdateServiceHomePolicy(() => authority("same"), () => {}); + const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { + homePolicy, process: { platform: "win32" }, readServiceBackend: () => "scheduler", + checkedStop: async (authorize: () => void) => { authorize(); return outcome; }, + }); + expect(result).toBe(outcome === "stopped" || outcome === "absent"); + }); +} + for (const changed of ["same", "absent", "unknown", "foreign"] as const) { - test(`complete worker passes its latched authorizer through Windows wrapper recovery: ${changed}`, async () => { + test(`complete worker passes its latched authorizer through the Windows service stop: ${changed}`, async () => { let home = authority("same"); const events: string[] = []; + let aborted = 0; const from = worker.indexOf("async function restartAfterUpdate("), to = worker.indexOf("/** Compact listen-holder summary", from); expect(from).toBeGreaterThan(0); expect(to).toBeGreaterThan(from); const result = await execute(helperBody + worker.slice(from, to) @@ -56,18 +82,22 @@ for (const changed of ["same", "absent", "unknown", "foreign"] as const) { createUpdateServiceHomePolicy, join, getConfigDir: () => "/fixture/own", process: { platform: "win32", env: {}, pid: 99 }, loadConfig: () => ({}), restartCommand: () => ({ bin: "fixture", args: [], display: "fixture" }), updateJob() {}, + windowsServiceStopAborted: () => { aborted++; return false; }, RESTART_PORT_RECLAIM_MS: 30000, WINDOWS_SERVICE_REPAIR_TIMEOUT_MS: 1000, RESTART_TIMEOUT_MS: 1000, moduleIo: { service: async () => ({ serviceReinstallArgs: () => ["service", "repair"] }) }, readServiceBackend: () => "scheduler", stopWinswService() {}, - stopWindows: () => { events.push(`stop:${home.kind}`); home = authority(changed); }, - killWindowsSchedulerWrappers: () => events.push(`kill:${home.kind}`), + checkedStop: checkedStop(events, () => home, () => { home = authority(changed); }), io: { serviceHomeAuthorityFn: () => home, serviceInstalledFn: () => true, packageLauncherPathFn: () => "fixture", waitForPort: async (_p: unknown, _h: unknown, options: any) => options.mayMutateFn(), listListenPidsFn: () => [], verifyOcxFn: () => null, isAliveFn: () => false, serviceViableFn: () => false, runService: () => ({ status: 1 }), restartServiceManagerFn: async () => "failed", reacquireForDirectStartFn: () => ({ notice: "fixture ownership veto", failed: false }), readPidFn: () => null, spawnStart: () => events.push("spawn") }, }); - expect(events).toEqual(changed === "same" ? ["stop:same", "kill:same"] : ["stop:same"]); - if (changed === "unknown" || changed === "foreign") expect(result).toBe(false); + expect(events).toEqual(changed === "same" ? ["stop:same", "terminate:same"] : ["stop:same"]); + // A refused or unproven service stop aborts before reclaim or any start (#7771). + if (changed !== "same") { + expect(result).toBe(false); + expect(aborted).toBe(1); + } }); } diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts new file mode 100644 index 00000000000..4572f5cd17a --- /dev/null +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -0,0 +1,131 @@ +/** + * The live adapters of the checked Windows stop (#7771): parsing of the capped schtasks and + * sc.exe answers, the ARM64 strict gate on the terminate, the whole-millisecond timeouts, the + * Part B port sampler and listener identity, and the stopper-ancestry check. Hermetic: no + * command is spawned and no FFI is loaded. + */ +import { describe, expect, test } from "bun:test"; + +import { + classifyScStop, + classifyTaskEnd, + identifyWindowsStopListener, + liveWindowsStopCommandRunner, + parseScQueryState, + parseTaskCsvState, + sampleWindowsStopPort, + stopperUnderHomeWrapper, + wholeTimeoutMs, + windowsStopSnapshot, + windowsStopTerminate, + winswStatusFromScState, + WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED, +} from "../../src/service/windows-checked-stop-live"; +import type { WindowsProcessNativeApi } from "../../src/service/windows-process-native"; +import { PATHS, entry, launcher, loop, proxy, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; + +const result = (status: number | null, stdout = "", stderr = "", timedOut = false) => ({ status, stdout, stderr, timedOut }); + +describe("ARM64 strict variant (050, 060 'ARM64 decision for PR2')", () => { + test("ships strict: the bounded PowerShell fallback is disabled", () => { + expect(WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED).toBe(false); + }); + + test("without the native primitive the terminate is unavailable and the fallback is never called", async () => { + let fallbackCalls = 0; + const terminate = async () => { fallbackCalls++; return { status: "terminated" as const, exited: true }; }; + // ARM64 has no native API; an x64 whose DLL load failed is the same case. + expect(await windowsStopTerminate(30, 5_100, 3_000, { native: () => null, terminate })).toEqual({ status: "unavailable" }); + expect(fallbackCalls).toBe(0); + }); + + test("with the native primitive the checked terminate runs", async () => { + const calls: unknown[] = []; + const native = {} as WindowsProcessNativeApi; + const terminate = async (pid: number, start: number, options: { timeoutMs: number }) => { + calls.push([pid, start, options.timeoutMs]); + return { status: "terminated" as const, exited: true }; + }; + expect(await windowsStopTerminate(30, 5_100, 2_500, { native: () => native, terminate })) + .toEqual({ status: "terminated", exited: true }); + expect(calls).toEqual([[30, 5_100, 2_500]]); + }); +}); + +describe("whole-millisecond timeouts (execFile reads 0 as unlimited)", () => { + test("a fraction is floored and a value below 1 ms spawns nothing", async () => { + expect(wholeTimeoutMs(1999.9)).toBe(1999); + expect(wholeTimeoutMs(0.9)).toBeNull(); + expect(wholeTimeoutMs(0)).toBeNull(); + expect(wholeTimeoutMs(-5)).toBeNull(); + // The runner answers a sub-millisecond cap as timed out without resolving any program. + expect(await liveWindowsStopCommandRunner("sc", ["query", "x"], 0.4)).toEqual(result(null, "", "", true)); + // The snapshot reads nothing (unreadable) rather than running with no timeout. + expect(await windowsStopSnapshot(0.7)).toBeNull(); + }); +}); + +describe("capped command answers", () => { + test("schtasks /end", () => { + expect(classifyTaskEnd(result(0))).toBe("ended"); + expect(classifyTaskEnd(result(1, "", "ERROR: There is no running instance of the task."))).toBe("not-running"); + expect(classifyTaskEnd(result(1, "", "ERROR: Access is denied."))).toBe("denied"); + expect(classifyTaskEnd(result(1, "", "ERROR: something else"))).toBe("failed"); + expect(classifyTaskEnd(result(null, "", "", true))).toBe("failed"); + }); + + test("schtasks CSV state reads English values and leaves a localized word unknown", () => { + expect(parseTaskCsvState('"\\opencodex-proxy","N/A","Running"')).toBe("running"); + expect(parseTaskCsvState('"\\opencodex-proxy","N/A","Ready"')).toBe("not-running"); + expect(parseTaskCsvState('"\\opencodex-proxy","N/A","Wird ausgeführt"')).toBe("unknown"); + expect(parseTaskCsvState('"\\Tools\\opencodex-proxy","N/A","Running"')).toBe("unknown"); + }); + + test("sc.exe query and stop", () => { + expect(parseScQueryState(result(1060, "[SC] EnumQueryServicesStatus:OpenService FAILED 1060:"))).toBe("nonexistent"); + expect(parseScQueryState(result(0, " STATE : 4 RUNNING"))).toBe("running"); + expect(parseScQueryState(result(0, " STATE : 1 STOPPED"))).toBe("stopped"); + expect(parseScQueryState(result(0, " STATE : 3 STOP_PENDING"))).toBe("pending"); + expect(parseScQueryState(result(5, "[SC] OpenService FAILED 5:"))).toBe("unknown"); + expect(winswStatusFromScState("pending")).toBe("started"); + expect(winswStatusFromScState("unknown")).toBe("unknown"); + expect(classifyScStop(result(0))).toBe("sent"); + expect(classifyScStop(result(1062, "[SC] ControlService FAILED 1062:"))).toBe("sent"); + expect(classifyScStop(result(5, "[SC] OpenService FAILED 5:"))).toBe("denied"); + expect(classifyScStop(result(null, "", "", true))).toBe("failed"); + }); +}); + +describe("Part B port sampler and listener identity", () => { + test("free, listening, unreadable, and an exited owner's row is cleared", () => { + expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [] }) })).toEqual({ state: "free" }); + expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: false }) })).toEqual({ state: "unknown" }); + expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [77] }), alive: () => true })) + .toEqual({ state: "listening", ownerPid: 77 }); + const drops: number[] = []; + expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [77] }), alive: () => false, drop: port => drops.push(port) })) + .toEqual({ state: "listening", ownerPid: 77 }); + expect(drops).toEqual([10100]); + // The stopping process's own socket is not a listener on the stopped port. + expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [5] }), selfPid: 5 })).toEqual({ state: "free" }); + }); + + test("ours, foreign, exited, unknown", () => { + expect(identifyWindowsStopListener(42, { alive: () => true, verify: pid => pid })).toBe("ours"); + expect(identifyWindowsStopListener(42, { alive: () => true, verify: () => null })).toBe("foreign"); + expect(identifyWindowsStopListener(42, { alive: () => false })).toBe("exited"); + expect(identifyWindowsStopListener(null)).toBe("unknown"); + }); +}); + +describe("who may run Part A: the stopper's ancestry", () => { + test("a stopper under this home's wrapper is detected; one under a plain shell is not", () => { + const tree = supervisedTree(); + // ocx launched as a child of the looping cmd.exe (or of the proxy it supervises). + expect(stopperUnderHomeWrapper([...tree, entry(70, 30, "ocx.exe", "ocx stop", 6_000)], PATHS, 70)).toBe(true); + expect(stopperUnderHomeWrapper([...tree, entry(71, 42, "bun.exe", "bun worker", 6_000)], PATHS, 71)).toBe(true); + // ocx launched from a terminal. + const shell = [launcher(), loop(), proxy(), entry(60, 1, "pwsh.exe", "pwsh", 100), entry(61, 60, "ocx.exe", "ocx stop", 6_000)]; + expect(stopperUnderHomeWrapper(shell, PATHS, 61)).toBe(false); + }); +}); diff --git a/tests/windows/windows-deploy-close-regressions.test.ts b/tests/windows/windows-deploy-close-regressions.test.ts index d6ea97d0b92..04ea7c25421 100644 --- a/tests/windows/windows-deploy-close-regressions.test.ts +++ b/tests/windows/windows-deploy-close-regressions.test.ts @@ -50,21 +50,19 @@ describe("update-job restart avoids the shell-less .cmd EINVAL (Windows, bun/sou // update — the common Windows path — with a stale service it could have refreshed. expect(src).toContain('process.env.OCX_SERVICE === "1" && refreshRegisters'); expect(src).toContain('const refreshRegisters = (svcArgs ?? []).includes("install")'); - // Follow the extracted teardown helper through its import and restart call site. - expect(src).toContain('import { stopWindowsServiceWrappersBestEffort } from "./job-windows-restart"'); - expect(src).toContain("(io.stopServiceWrappersFn ?? stopWindowsServiceWrappersBestEffort)(homePolicy.authorizeMutation);"); + // Follow the extracted teardown helper through its import and restart call sites. Since + // #7771 every call site gets a return path: a service that is not proven stopped aborts. + expect(src).toContain('import { stopWindowsServiceForRestart, WINDOWS_SERVICE_STOP_ABORT } from "./job-windows-restart"'); + expect(src.split("await (io.stopServiceWrappersFn ?? stopWindowsServiceForRestart)(homePolicy.authorizeMutation) === false) return windowsServiceStopAborted(job);").length - 1).toBe(3); const restart = read("src/update/job-windows-restart.ts"); - expect(restart).toContain("export function stopWindowsServiceWrappersBestEffort(authorizeMutation: UpdateMutationAuthorizer)"); - expect(restart).toContain('if (process.platform !== "win32") return;'); - // Native WinSW installs stop and return before Task Scheduler /end or wrapper cleanup. - expect(restart).toMatch(/if \(readServiceBackend\(\) === "native"\) \{\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedStopWinsw\(authorizeMutation\);\s*return;\s*\}\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedStopWindows\(authorizeMutation\);/); - expect(restart.indexOf("killWindowsServiceWrapperProcesses(authorizeMutation);")).toBeGreaterThan(restart.indexOf("guardedStopWindows(authorizeMutation);")); - // The wrapper-killer script (and its self-exclusion) moved to the shared - // lib/windows-service-wrappers so the updater and the service teardown path - // cannot drift apart again. Follow the invariant to where it lives; the - // matching rules themselves are covered by windows-service-wrappers.test.ts. - expect(restart).toMatch(/const paths = \{\s*scriptPath: join\(getConfigDir\(\), "opencodex-service.cmd"\),\s*launcherPath: join\(getConfigDir\(\), "opencodex-service-launcher.vbs"\),\s*\};\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedKillWindowsWrappers\(authorizeMutation, paths\);/); - expect(read("src/lib/windows-service-wrappers.ts")).toContain("$_.ProcessId -eq $PID"); + expect(restart).toContain("export async function stopWindowsServiceForRestart(authorizeMutation: UpdateMutationAuthorizer): Promise"); + expect(restart).toContain('if (process.platform !== "win32") return true;'); + // Native WinSW installs keep today's stop and return before the checked Task Scheduler stop. + expect(restart).toMatch(/if \(backend === "native"\) \{\s*if \(!authorizeMutation\("service"\)\) return false;\s*try \{ guardedStopWinsw\(authorizeMutation\); \} catch \{[^}]*\}\s*return true;\s*\}\s*if \(!authorizeMutation\("service"\)\) return false;/); + // Task Scheduler: Part A ends the manager and the bound wrappers (the binding rule in + // windows-stop-binding.ts excludes the stopper itself); only stopped or absent continues. + expect(restart).toContain("const outcome = await guardedStopWindowsChecked(authorizeMutation);"); + expect(restart).toContain('return outcome === "stopped" || outcome === "absent";'); // Pinned-child cleanup is exercised through the retry loop in update/update-job.test.ts, // including exited children, live retry/final cleanup, and successful health probes. }); diff --git a/tests/windows/windows-service-stop-wiring.test.ts b/tests/windows/windows-service-stop-wiring.test.ts new file mode 100644 index 00000000000..3e5d0461538 --- /dev/null +++ b/tests/windows/windows-service-stop-wiring.test.ts @@ -0,0 +1,234 @@ +/** + * Caller side of the checked Windows stop (#7771): the real Part A run through the wiring in + * src/service/windows-service-stop.ts with every machine effect injected. Covers the ARM64 + * strict variant end to end, the own-wrapper refusal, the route's cap, admission against the + * stop deadline, the deadline flag and variable, the receipt's Part A record and the additive + * ocx-stop/1 fields. No schtasks, PowerShell or FFI call is made. + */ +import { afterEach, describe, expect, test } from "bun:test"; + +import { approvalChanged, managerStillActive, parseStopApproval } from "../../src/cli/stop-approval"; +import { summarizeStopRun, type StopRunRecord } from "../../src/cli/stop-report"; +import { claimPendingTeardown, clearPendingTeardown, pendingTeardownPartA, recordPendingTeardownPartA } from "../../src/config/pending-teardown"; +import type { WindowsCheckedStopDeps } from "../../src/service/windows-checked-stop"; +import { windowsStopTerminate } from "../../src/service/windows-checked-stop-live"; +import type { WindowsProcessWatch } from "../../src/service/windows-process-native"; +import { + runWindowsServiceStopPartA, + setStopDeadlineMs, + setWindowsServiceStopHooksForTests, + UNDER_OWN_WRAPPER_DETAIL, + windowsPartARecord, + windowsStopAdmission, + resolveStopDeadlineMs, + parseStopDeadlineMs, + windowsStopDeadlineAt, +} from "../../src/service/windows-service-stop"; +import { DEFAULT_WINDOWS_STOP_DEADLINE_MS } from "../../src/service/windows-stop-budget"; +import { PATHS, SCHEDULE_PID, entry, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; +import { partAResult } from "../helpers/windows-stop-caller-fixtures"; + +afterEach(() => { + setWindowsServiceStopHooksForTests(null); + setStopDeadlineMs(null); +}); + +interface FakeMachine { + deps: WindowsCheckedStopDeps; + calls: string[]; + terminated: number[]; +} + +/** A registered, running task with the supervised tree; terminate kills through the watch. */ +function machine(overrides: Partial & { gate?: () => boolean; extra?: ReturnType[] } = {}): FakeMachine { + const calls: string[] = []; + const terminated: number[] = []; + const dead = new Set(); + const deps: WindowsCheckedStopDeps = { + canonicalPaths: PATHS, + childPid: 42, + selfPid: 1, + probeScheduler: () => ({ status: "present" }), + probeWinsw: () => "nonexistent", + enumerate: () => { calls.push("enumerate"); return [...supervisedTree(), ...(overrides.extra ?? [])]; }, + scheduleServicePid: () => SCHEDULE_PID, + endTask: () => { calls.push("end"); return overrides.gate && !overrides.gate() ? "failed" : "ended"; }, + taskState: () => "not-running", + winswStop: () => "sent", + winswState: () => "stopped", + proxyAlive: () => false, + openWatch: (pid): WindowsProcessWatch => ({ status: "open", exited: () => dead.has(pid), close() {} }), + terminate: (pid) => { calls.push("terminate:" + pid); terminated.push(pid); dead.add(pid); return { status: "terminated", exited: true }; }, + ...overrides, + }; + return { deps, calls, terminated }; +} + +describe("Part A through the caller wiring", () => { + test("ends the bound wrappers with the live-style deps", async () => { + const fake = machine(); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("stopped"); + expect(fake.terminated).toEqual([30, 20]); + }); + + test("ARM64 strict: nothing is terminated and the stop is unknown / evidence-unreadable", async () => { + let fallback = 0; + const fake = machine({ + // No native API: no exit watch, and the strict gate answers the terminate. + openWatch: () => ({ status: "unavailable" }), + terminate: (pid, start, timeoutMs) => windowsStopTerminate(pid, start, timeoutMs, { + native: () => null, + terminate: async () => { fallback++; return { status: "terminated", exited: true }; }, + }), + }); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.reason).toBe("evidence-unreadable"); + expect(result.terminated).toEqual([]); + expect(fallback).toBe(0); + }); + + test("a stop launched under this home's wrapper refuses before /end, unless the caller is the wrapped proxy", async () => { + const under = entry(70, 30, "ocx.exe", "ocx stop", 6_000); + const refused = machine({ selfPid: 70, extra: [under] }); + setWindowsServiceStopHooksForTests({ liveDeps: () => refused.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(refused.calls).toEqual(["enumerate"]); + + // The guarded path binds on the snapshot it already read: refused before any probe. + const guarded = await runWindowsServiceStopPartA({ bindSnapshot: [...supervisedTree(), under] }); + expect(guarded.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(refused.calls).toEqual(["enumerate"]); + + const allowed = machine({ selfPid: 70, extra: [under] }); + setWindowsServiceStopHooksForTests({ liveDeps: () => allowed.deps }); + expect((await runWindowsServiceStopPartA({ allowUnderOwnWrapper: true })).outcome).toBe("stopped"); + expect(allowed.calls).toContain("end"); + }); + + test("a cap overrun is unknown at once, and nothing is ended or terminated after it", async () => { + let gate: () => boolean = () => true; + const fake = machine({ + endTask: async () => { + await Bun.sleep(120); + return gate() ? "ended" : "failed"; + }, + }); + setWindowsServiceStopHooksForTests({ liveDeps: options => { gate = options.gate ?? gate; return fake.deps; } }); + const started = performance.now(); + const result = await runWindowsServiceStopPartA({ capMs: 40 }); + expect(performance.now() - started).toBeLessThan(110); + expect(result.outcome).toBe("unknown"); + expect(result.detail).toBe("the checked stop did not finish within its cap"); + await Bun.sleep(250); + expect(gate()).toBe(false); + expect(fake.terminated).toEqual([]); + }); + + test("admission refuses with unverified-budget and changes nothing when the deadline cannot hold the stop", async () => { + const fake = machine(); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + setStopDeadlineMs(Math.ceil(performance.now()) + 20_000); + const result = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); + expect(result.outcome).toBe("unverified-budget"); + expect(result.reason).toBe("unverified-budget"); + expect(result.budget?.requiredMs).toBe(56_500); + expect(fake.calls.filter(call => call === "end" || call.startsWith("terminate"))).toEqual([]); + // The same stop with the default 60 s from process start is admitted early in a process. + setStopDeadlineMs(Math.ceil(performance.now()) + 60_000); + expect((await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) })).outcome).toBe("stopped"); + }); +}); + +describe("the stop deadline", () => { + test("--deadline-ms wins over OCX_STOP_DEADLINE_MS; empty or invalid is absent; neither is 60 s", () => { + expect(resolveStopDeadlineMs(72_000, { OCX_STOP_DEADLINE_MS: "30000" })).toBe(72_000); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "30000" })).toBe(30_000); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "" })).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "-5" })).toBe(60_000); + expect(resolveStopDeadlineMs(null, { OCX_STOP_DEADLINE_MS: "12.5" })).toBe(60_000); + expect(resolveStopDeadlineMs(null, {})).toBe(60_000); + expect(parseStopDeadlineMs(" 45000 ")).toBe(45_000); + expect(parseStopDeadlineMs("0")).toBeNull(); + }); + + test("measured from process start: the deadline is the Part A clock value", () => { + setStopDeadlineMs(72_000); + expect(windowsStopDeadlineAt()).toBe(72_000); + setStopDeadlineMs(null); + expect(windowsStopDeadlineAt()).toBe(60_000); + }); + + test("ocx stop parses --deadline-ms anywhere and keeps the approval grammar", () => { + expect(parseStopApproval(["--deadline-ms", "72000"])).toEqual({ ok: true, json: false, approval: null, deadlineMs: 72_000 }); + expect(parseStopApproval(["--json", "--deadline-ms", "5000"])).toEqual({ ok: true, json: true, approval: null, deadlineMs: 5_000 }); + expect(parseStopApproval(["--json"])).toEqual({ ok: true, json: true, approval: null, deadlineMs: null }); + for (const bad of [["--deadline-ms"], ["--deadline-ms", "0"], ["--deadline-ms", "1e4"], ["--deadline-ms", "-1"], + ["--deadline-ms", "5", "--deadline-ms", "6"], ["--json", "--deadline-ms", "abc"]]) { + expect(parseStopApproval(bad).ok).toBe(false); + } + const token = "a".repeat(64); + const approval = ["--json", "--expect-pid", "42", "--expect-port", "10100", "--expect-hostname", "127.0.0.1", + "--expect-config-home", "/h", "--expect-cli-version", "2.0.0", "--expect-compatibility-token", token]; + const parsed = parseStopApproval([...approval.slice(0, 3), "--deadline-ms", "65000", ...approval.slice(3)]); + expect(parsed.ok && parsed.approval?.pid === 42 && parsed.deadlineMs === 65_000).toBe(true); + }); +}); + +describe("the receipt's Part A record", () => { + test("only stopped or absent produce a record, with the bound wrapper set", () => { + expect(windowsPartARecord(partAResult("stopped"))).toEqual({ + partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }], + }); + expect(windowsPartARecord(partAResult("absent"))).toEqual({ partA: "absent", manager: { scheduler: false, winsw: false }, wrappers: [] }); + for (const outcome of ["survived", "denied", "unknown", "unverified-budget"] as const) { + expect(windowsPartARecord(partAResult(outcome))).toBeNull(); + } + }); + + test("written at claim, or added by the owner afterwards; read back only when well formed", () => { + const endpoint = { hostname: "127.0.0.1", port: 10100 }; + const record = windowsPartARecord(partAResult("stopped"))!; + const atClaim = claimPendingTeardown(endpoint, "exact", process.pid, undefined, record); + expect(pendingTeardownPartA(atClaim.nonce)).toEqual(record); + const before = claimPendingTeardown(endpoint, "exact", process.pid); + expect(pendingTeardownPartA(before.nonce)).toBeNull(); + expect(recordPendingTeardownPartA(before.nonce, record, process.pid + 1)).toBe(false); + expect(recordPendingTeardownPartA(before.nonce, record)).toBe(true); + expect(pendingTeardownPartA(before.nonce)).toEqual(record); + const malformed = claimPendingTeardown(endpoint, "exact", process.pid, undefined, { partA: "stopped" } as never); + expect(pendingTeardownPartA(malformed.nonce)).toBeNull(); + for (const nonce of [atClaim.nonce, before.nonce, malformed.nonce]) clearPendingTeardown(nonce); + }); +}); + +describe("ocx-stop/1 additive fields", () => { + const record = (extra: Partial): StopRunRecord => ({ + service: "failed", proxy: "unknown", sharedTeardown: "skipped", inheritedTeardownBlocks: false, receiptClearFailed: false, ...extra, + }); + const signals = { failed: true, historyOnly: false, historyDeferred: false, exitCode: 1 }; + + test("serviceStopReason and portHeldByForeign are omitted until set, and never null", () => { + const plain = summarizeStopRun(record({}), signals); + expect("serviceStopReason" in plain).toBe(false); + expect("portHeldByForeign" in plain).toBe(false); + const denied = summarizeStopRun(record({ serviceStopReason: "access-denied" }), signals); + expect(denied).toMatchObject({ outcome: "failed", service: "failed", proxy: "unknown", sharedTeardown: "skipped", serviceStopReason: "access-denied" }); + const foreign = summarizeStopRun(record({ service: "stopped", proxy: "stopped", portHeldByForeign: true }), + { failed: false, historyOnly: false, historyDeferred: false, exitCode: 0 }); + expect(foreign).toMatchObject({ outcome: "stopped", service: "stopped", portHeldByForeign: true }); + expect(summarizeStopRun(record({ portHeldByForeign: false }), signals).portHeldByForeign).toBe(false); + }); + + test("the guarded refusals carry them; no new outcome or proxy value appears", () => { + expect(managerStillActive("failed", record({ serviceStopReason: "unverified-budget" })).summary) + .toMatchObject({ outcome: "manager-still-active", serviceStopReason: "unverified-budget" }); + expect(approvalChanged("x", "unbound-wrapper").summary).toMatchObject({ outcome: "approval-changed", serviceStopReason: "unbound-wrapper" }); + expect("serviceStopReason" in approvalChanged("x").summary).toBe(false); + }); +}); diff --git a/tests/windows/windows-service-wrappers.test.ts b/tests/windows/windows-service-wrappers.test.ts index 3e91d5d0a61..e01bc6bf07e 100644 --- a/tests/windows/windows-service-wrappers.test.ts +++ b/tests/windows/windows-service-wrappers.test.ts @@ -129,17 +129,18 @@ describe("the generated script still implements that rule", () => { }); }); -describe("both teardown paths use the shared killer", () => { - test("neither file keeps a private matcher", () => { - for (const rel of ["src/service/windows-ops.ts", "src/update/job-windows-restart.ts"]) { +describe("every teardown path ends wrappers through the checked stop (#7771)", () => { + test("no stop path keeps a private matcher or the best-effort sweep", () => { + for (const rel of ["src/service/windows-ops.ts", "src/service/orchestration.ts", "src/update/job-windows-restart.ts", "src/update/guarded-runtime-ops.ts"]) { const src = read(rel); - if (rel === "src/update/job-windows-restart.ts") { - expect(src).toContain("killWindowsServiceWrapperProcesses(authorizeMutation);"); - expect(src).toMatch(/const paths = \{[\s\S]*?scriptPath: join\(getConfigDir\(\), "opencodex-service.cmd"\),[\s\S]*?launcherPath: join\(getConfigDir\(\), "opencodex-service-launcher.vbs"\),\s*\};\s*if \(!authorizeMutation\("service"\)\) return;\s*guardedKillWindowsWrappers\(authorizeMutation, paths\);/); - } else expect(src).toContain("killWindowsSchedulerWrappers({"); + expect(src).not.toContain("killWindowsSchedulerWrappers"); + expect(src).not.toContain("killWindowsServiceWrapperProcesses"); expect(src).not.toContain("-like ('*' + $p + '*')"); expect(src).not.toContain("$pats = @('opencodex-service.cmd'"); } + // The one matcher is the binding rule; the service stop and the updater reach it through Part A. + expect(read("src/service/orchestration.ts")).toContain("runWindowsServiceStopPartA(request)"); + expect(read("src/update/guarded-runtime-ops.ts")).toContain("runWindowsServiceStopPartA({ authorize: mutationGate(authorize, \"service\"), allowUnderOwnWrapper: true })"); }); }); diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts new file mode 100644 index 00000000000..a5ff66879b5 --- /dev/null +++ b/tests/windows/windows-stop-callers.test.ts @@ -0,0 +1,356 @@ +/** + * Caller-level wiring of the checked Windows stop (#7771, 030 "Where it sits"): the + * /api/stop route, ocx service stop / uninstall, the guarded (consent) path with its + * bound-stray state, and the evidence snapshot the guarded bind must use. Hermetic: Part A and + * Part B are answered by fixtures or run against injected deps; nothing reaches the machine. + */ +import { afterEach, describe, expect, test } from "bun:test"; + +import { inspectGuardedTarget, resetOcxWindowsStopForTests, stopGuardedManager } from "../../src/cli/stop-windows-checked"; +import { runGuardedManagerStep, type GuardedStopSnapshot } from "../../src/cli/stop-approval"; +import { ROUTE_PART_A_CAP_MS, ROUTE_STRAY_SWEEP_CAP_MS, routeServiceStop } from "../../src/server/stop-service-route"; +import { completeWindowsServiceStopReceipt, windowsServiceRemovalStop, windowsServiceStopBeforeRestore } from "../../src/service/cli-windows-stop"; +import { inspectGuardedManagerTarget, type GuardedManagerTarget } from "../../src/service/guarded-manager-target"; +import type { WindowsCheckedStopDeps } from "../../src/service/windows-checked-stop"; +import type { WindowsProcessWatch } from "../../src/service/windows-process-native"; +import type { WindowsStopProcessEntry } from "../../src/service/windows-process-evidence"; +import { runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, setStopDeadlineMs, setWindowsServiceStopHooksForTests } from "../../src/service/windows-service-stop"; +import { CMD, PATHS, SCHEDULE_PID, VBS, entry, launcher, loop, proxy, scheduleHost } from "../helpers/windows-checked-stop-fixtures"; +import { ALL_PART_A_FAILURES, partAResult, partBResult } from "../helpers/windows-stop-caller-fixtures"; + +afterEach(() => { + setWindowsServiceStopHooksForTests(null); + setStopDeadlineMs(null); + resetOcxWindowsStopForTests(null); +}); + +const RECORD = { partA: "stopped" as const, manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }; +const url = (nonce = "a".repeat(32)) => new URL("http://127.0.0.1:10100/api/stop?deferSharedTeardown=1&teardownNonce=" + nonce); + +describe("/api/stop service stop on Windows", () => { + const route = (holdsReceipt: boolean, io: Parameters[2] = {}) => { + const stops: unknown[] = []; + const run = routeServiceStop(url(), holdsReceipt, { + platform: "win32", assertEnvironment: () => {}, winswRegistered: () => false, + stop: async request => { stops.push(request); return "stopped"; }, + ...io, + }); + return { run, stops }; + }; + + test("a receipt whose Part A record names the registered manager skips Part A", async () => { + const { run, stops } = route(true, { readRecord: () => RECORD, registration: async () => ({ scheduler: "present", winsw: "nonexistent" }) }); + expect(await run).toBe("stopped"); + expect(stops).toEqual([]); + }); + + test("a receipt without the record, or naming another manager, re-runs Part A capped at 6 s", async () => { + for (const io of [ + { readRecord: () => null }, + { readRecord: () => RECORD, registration: async () => ({ scheduler: "absent" as const, winsw: "nonexistent" as const }) }, + { readRecord: () => RECORD, registration: async () => ({ scheduler: "unknown" as const, winsw: "nonexistent" as const }) }, + ]) { + const { run, stops } = route(true, io); + await run; + expect(stops).toEqual([{ capMs: ROUTE_PART_A_CAP_MS, allowUnderOwnWrapper: true }]); + } + expect(ROUTE_PART_A_CAP_MS).toBe(6_000); + }); + + test("no receipt: WinSW keeps today's in-process stop; otherwise the 4 s stray sweep", async () => { + let winsw = 0; + const native = route(false, { winswRegistered: () => true, stopWinsw: () => { winsw++; } }); + expect(await native.run).toBe("stopped"); + expect(native.stops).toEqual([]); + expect(winsw).toBe(1); + expect(await route(false, { winswRegistered: () => true, stopWinsw: () => { throw new Error("x"); } }).run).toBe("failed"); + const stray = route(false); + await stray.run; + expect(stray.stops).toEqual([{ capMs: ROUTE_STRAY_SWEEP_CAP_MS, allowUnderOwnWrapper: true }]); + expect(ROUTE_STRAY_SWEEP_CAP_MS).toBe(4_000); + }); + + test("off Windows the route keeps today's service stop", async () => { + const { run, stops } = route(true, { platform: "darwin", readRecord: () => RECORD }); + await run; + expect(stops).toEqual([{}]); + }); + + test("a capped Part A that overruns answers service_state_unknown's outcome without waiting for it", async () => { + setWindowsServiceStopHooksForTests({ partA: () => new Promise(() => {}) }); + const started = performance.now(); + const result = await runWindowsServiceStopPartA({ capMs: 30, allowUnderOwnWrapper: true }); + expect(performance.now() - started).toBeLessThan(500); + expect(serviceStopOutcomeFromWindows(result)).toBe("state-unknown"); + }); +}); + +describe("ocx service stop on Windows", () => { + test("a Part A other than stopped/absent refuses before the receipt and the child stop", async () => { + for (const outcome of ALL_PART_A_FAILURES) { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult(outcome) }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + claim: () => { steps.push("claim"); return { nonce: "n" } as never; }, + stopTracked: async () => { steps.push("child"); return "stopped"; }, + }); + expect(result.ok).toBe(false); + expect(steps).toEqual([]); + } + }); + + test("claims the receipt before the deferred child stop, and completes it only after Part B and the restore", async () => { + const steps: string[] = []; + let claimed: unknown; + setWindowsServiceStopHooksForTests({ + partA: async () => { steps.push("partA"); return partAResult("stopped"); }, + partB: async () => { steps.push("partB"); return partBResult("freed"); }, + }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + claim: (_endpoint, _source, _owner, _handover, partA) => { steps.push("claim"); claimed = partA; return { nonce: "n1" } as never; }, + stopTracked: async io => { steps.push("child:" + io?.deferSharedTeardownNonce + ":" + io?.skipPortWait); return "stopped"; }, + }); + expect(steps).toEqual(["partA", "claim", "child:n1:true", "partB"]); + expect(claimed).toEqual(RECORD); + expect(result).toEqual({ ok: true, nonce: "n1" }); + const cleared: string[] = []; + completeWindowsServiceStopReceipt(result.nonce, nonce => { cleared.push(nonce); return true; }); + expect(cleared).toEqual(["n1"]); + }); + + test("a claim failure while this home owns the teardown sends no undeferred child stop", async () => { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped") }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + claim: () => { throw new Error("disk full"); }, + stopTracked: async () => { steps.push("child"); return "stopped"; }, + }); + expect(result.ok).toBe(false); + expect(steps).toEqual([]); + }); + + test("the not-owned teardown path is unchanged: no receipt, an ordinary child stop", async () => { + const children: unknown[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => false, + claim: () => { throw new Error("must not claim"); }, + stopTracked: async io => { children.push(io); return "stopped"; }, + }); + expect(result).toEqual({ ok: true }); + expect(children).toEqual([{ skipPortWait: true }]); + }); + + test("Part B failure keeps the receipt (no restore); a foreign listener does not fail the stop", async () => { + for (const [outcome, ok] of [["survived", false], ["unknown", false], ["unconfirmed", false], ["port-held-by-foreign", true]] as const) { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult(outcome) }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, claim: () => ({ nonce: "n2" }) as never, stopTracked: async () => "stopped", + }); + expect(result.ok).toBe(ok); + } + }); + + test("absent keeps today's survivor check instead of Part B", async () => { + let partB = 0; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent"), partB: async () => { partB++; return partBResult("freed"); } }); + const live = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => false, stopTracked: async () => "none", survivor: async () => ({ port: 10100 }), + }); + expect(live.ok).toBe(false); + expect(partB).toBe(0); + }); +}); + +describe("ocx service uninstall / remove on Windows", () => { + test("refuses before deleting the task unless Part A stopped (or found nothing) and Part B freed the port", async () => { + for (const outcome of ALL_PART_A_FAILURES) { + let child = 0; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult(outcome) }); + expect(await windowsServiceRemovalStop({ stopTracked: async () => { child++; return "stopped"; } })).toBe(false); + expect(child).toBe(0); + } + for (const [outcome, ok] of [["freed", true], ["port-held-by-foreign", true], ["survived", false], ["unknown", false]] as const) { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult(outcome) }); + expect(await windowsServiceRemovalStop({ stopTracked: async () => "stopped" })).toBe(ok); + } + }); +}); + +describe("guarded (consent) path", () => { + const strayManager: Extract = { + kind: "bound-stray", pid: 42, backend: "scheduler", childNeedsSeparateStop: true, + wrappers: [{ pid: 30, startTime: 5_100 }, { pid: 20, startTime: 5_000 }], + }; + const snapshot = (manager: GuardedStopSnapshot["manager"]): GuardedStopSnapshot => ({ + approval: { pid: 42, port: 10100, hostname: "127.0.0.1", configHome: "/h", cliVersion: "2.0.0", compatibilityToken: "a".repeat(64) }, + manager, + }); + + test("order: the wrappers end before signalApproved; the approved PID still gets its graceful signal; Part B after settle", async () => { + const order: string[] = []; + const step = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: async () => { order.push("revalidate"); return strayManager; }, + stopManager: async () => { order.push("partA"); return "stopped"; }, + signalApproved: async () => { order.push("signal"); return false; }, + settle: async () => { order.push("settle"); return true; }, + confirmFreed: async () => { order.push("partB"); return true; }, + managerState: async () => { order.push("state"); return "inactive"; }, + }); + expect(order).toEqual(["revalidate", "partA", "signal", "settle", "partB", "state"]); + expect(step.effect).toBe("stopped"); + }); + + test("Part A not stopped is manager-still-active and the approved PID is never signalled (a refusal is never forced)", async () => { + for (const service of ["failed", "state-unknown"] as const) { + const order: string[] = []; + const step = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: () => strayManager, + stopManager: async () => service, + signalApproved: async () => { order.push("signal"); return true; }, + settle: async () => { order.push("settle"); return true; }, + confirmFreed: async () => { order.push("partB"); return true; }, + managerState: async () => "inactive", + }); + expect(step.effect).toBe("manager-still-active"); + expect(order).toEqual([]); + } + // A graceful signal that is refused (409) fails the step without settle or Part B. + const order: string[] = []; + const refused = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: () => strayManager, stopManager: async () => "stopped", + signalApproved: async () => { throw new Error("409"); }, + settle: async () => { order.push("settle"); return true; }, + confirmFreed: async () => { order.push("partB"); return true; }, + managerState: async () => "inactive", + }); + expect(refused.effect).toBe("failed"); + expect(order).toEqual([]); + }); + + test("Part B failure is manager-still-active; an absent manager runs neither Part A nor Part B", async () => { + const failed = await runGuardedManagerStep(snapshot(strayManager), { + revalidateManager: () => strayManager, stopManager: async () => "stopped", signalApproved: async () => false, + settle: async () => true, confirmFreed: async () => false, managerState: async () => "inactive", + }); + expect(failed.effect).toBe("manager-still-active"); + const order: string[] = []; + const absent = await runGuardedManagerStep(snapshot({ kind: "absent" }), { + revalidateManager: () => ({ kind: "absent" }), stopManager: async () => { order.push("partA"); return "stopped"; }, + signalApproved: async () => false, settle: async () => true, + confirmFreed: async () => { order.push("partB"); return true; }, managerState: async () => "inactive", + }); + expect(absent.effect).toBe("stopped"); + expect(order).toEqual([]); + }); + + test("revalidation compares the bound wrapper set; an unbindable survivor carries its reason", async () => { + const io = (current: GuardedManagerTarget) => ({ + revalidateManager: () => current, stopManager: async () => "stopped" as const, signalApproved: async () => false, + settle: async () => true, managerState: async () => "inactive" as const, + }); + const changed = await runGuardedManagerStep(snapshot(strayManager), io({ ...strayManager, wrappers: [{ pid: 30, startTime: 9_999 }, { pid: 20, startTime: 5_000 }] })); + expect(changed.effect).toBe("approval-changed"); + const reordered = await runGuardedManagerStep(snapshot(strayManager), io({ ...strayManager, wrappers: [...strayManager.wrappers].reverse() })); + expect(reordered.effect).toBe("stopped"); + const unbound = await runGuardedManagerStep(snapshot(strayManager), io({ kind: "unknown", reason: "x", stopReason: "unbound-wrapper" })); + expect(unbound).toMatchObject({ effect: "approval-changed", stopReason: "unbound-wrapper" }); + }); +}); + +describe("guarded inspection: the binding rule decides bound-stray", () => { + const inspect = (processes: WindowsStopProcessEntry[] | Omit[]) => + inspectGuardedManagerTarget(42, 10100, { + platform: "win32", verifyPid: pid => pid, + scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", + win: { + winProcs: () => processes as WindowsStopProcessEntry[], winTaskState: () => "not-running", + winSchedulePid: () => SCHEDULE_PID, winScriptPath: () => CMD, winLauncherPath: () => VBS, + }, + }); + // The task is registered but not running; its launcher and loop survive with the child. + const tree = () => [scheduleHost, launcher(), loop(), proxy()]; + + test("surviving wrappers bound by the rule are bound-stray with their pid + creation time", () => { + expect(inspect(tree())).toEqual({ + kind: "bound-stray", pid: 42, backend: "scheduler", childNeedsSeparateStop: true, + wrappers: [{ pid: 30, startTime: 5_100 }, { pid: 20, startTime: 5_000 }], + }); + }); + + test("a CIM-only table (no creation times) cannot bind: unknown / evidence-unreadable", () => { + const cimOnly = tree().map(({ startTime: _startTime, ...rest }) => rest); + expect(inspect(cimOnly)).toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + }); + + test("a hand-started launcher under a live shell stays unknown / unbound-wrapper", () => { + const shell = entry(60, 1, "pwsh.exe", "pwsh", 100); + // The task's own wrappers bind; a second launcher started by hand from a live shell is not + // an ancestor of the approved child, its parent is neither the Schedule host nor gone, and + // its loop descends only from it, so neither may be ended. + const handStarted = [...tree(), shell, launcher(50, 60, 7_000), loop(51, 50, 7_100)]; + const target = inspect(handStarted); + expect(target.kind).toBe("unknown"); + expect(target).toMatchObject({ stopReason: "unbound-wrapper" }); + }); +}); + +describe("the guarded bind uses the evidence snapshot (native creation times), never the CIM-only list", () => { + // On x64 the evidence snapshot carries the native creation time, which is exactly what the + // handle terminate compares. The CIM-only time differs below the millisecond, and a terminate + // checked against it answers "reused": Part A would then treat a live wrapper as exited. + const NATIVE = { 20: 5_000, 30: 5_100 } as Record; + const evidence = (): WindowsStopProcessEntry[] => [scheduleHost, launcher(20, SCHEDULE_PID, NATIVE[20]), loop(30, 20, NATIVE[30]), proxy()]; + const cimOnly = (): WindowsStopProcessEntry[] => [scheduleHost, launcher(20, SCHEDULE_PID, NATIVE[20]! - 1), loop(30, 20, NATIVE[30]! - 1), proxy()]; + + function machine() { + const terminated: number[] = []; + const reused: number[] = []; + const enumerations: string[] = []; + const dead = new Set(); + const deps: WindowsCheckedStopDeps = { + canonicalPaths: PATHS, childPid: 42, selfPid: 1, + probeScheduler: () => ({ status: "present" }), probeWinsw: () => "nonexistent", + // What a CIM-only enumeration would return; the guarded bind must not use it. + enumerate: () => { enumerations.push("cim"); return cimOnly(); }, + scheduleServicePid: () => SCHEDULE_PID, endTask: () => "not-running", taskState: () => "not-running", + winswStop: () => "sent", winswState: () => "stopped", proxyAlive: () => false, + openWatch: (pid): WindowsProcessWatch => ({ status: "open", exited: () => dead.has(pid), close() {} }), + terminate: (pid, startTime) => { + if (NATIVE[pid] !== startTime) { reused.push(pid); return { status: "reused" }; } + terminated.push(pid); dead.add(pid); return { status: "terminated", exited: true }; + }, + }; + return { deps, terminated, reused, enumerations }; + } + + test("inspectGuardedTarget reads the evidence snapshot once and stopGuardedManager binds on it", async () => { + resetOcxWindowsStopForTests("win32"); + setStopDeadlineMs(Math.ceil(performance.now()) + 120_000); + const fake = machine(); + setWindowsServiceStopHooksForTests({ snapshot: async () => evidence(), schedulePid: async () => SCHEDULE_PID, liveDeps: () => fake.deps }); + const target = await inspectGuardedTarget(42, 10100, (pid, port, deps) => inspectGuardedManagerTarget(pid, port, { + ...deps, platform: "win32", verifyPid: p => p, scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", + win: { ...deps?.win, winTaskState: () => "not-running", winScriptPath: () => CMD, winLauncherPath: () => VBS }, + })); + expect(target.kind).toBe("bound-stray"); + const outcome = await stopGuardedManager({ approvedPort: 10100, receiptNonce: () => undefined, inheritedReceipts: 0 }); + expect(outcome).toBe("stopped"); + expect(fake.enumerations).toEqual([]); + expect(fake.reused).toEqual([]); + expect(fake.terminated).toEqual([30, 20]); + }); + + test("the hazard it prevents: binding on the CIM-only list ends nothing yet reports stopped", async () => { + const fake = machine(); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(fake.enumerations).toEqual(["cim"]); + expect(fake.terminated).toEqual([]); + expect(fake.reused).toEqual([30, 20]); + expect(result.outcome).toBe("stopped"); + }); +}); diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts new file mode 100644 index 00000000000..96f3032f0dc --- /dev/null +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -0,0 +1,114 @@ +/** + * Plain ocx stop (no consent snapshot) on Windows (#7771, 030 "Two parts"), through the real + * CLI module graph in tests/fixtures/parent-stop-runner.ts with the platform simulated and + * Part A / Part B answered by fixtures. Nothing reaches schtasks, PowerShell or FFI. + * + * A Part A outcome other than stopped/absent holds the child stop: no /api/stop POST, + * ownershipBlocked, the shared teardown skipped, and the record says service failed (or + * state-unknown), proxy unknown, outcome failed. After a stopped Part A, Part B replaces the + * respawn re-poll; the receipt the POST carries already records Part A. + */ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { spawnSync } from "node:child_process"; +import { existsSync, mkdtempSync, readFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { fixturePath, repoPath } from "../helpers/repo-root"; + +let home: string; +beforeEach(() => { home = mkdtempSync(join(tmpdir(), "ocx-windows-plain-stop-")); }); +afterEach(() => { removeTreeWithRetry(home); }); + +const restored = { success: true, message: "native Codex restored", + artifacts: { config: { state: "restored" }, catalog: { state: "restored" }, history: { state: "restored" } } }; + +function runStop(windowsPartA: string, windowsPartB?: string) { + const child = spawnSync(process.execPath, [fixturePath("parent-stop-runner.ts")], { + cwd: repoPath(), env: { ...process.env, OPENCODEX_HOME: home }, + input: JSON.stringify({ receipt: true, json: true, response: { success: true, sharedTeardown: "deferred" }, + restore: restored, windowsPartA, ...(windowsPartB ? { windowsPartB } : {}) }), + encoding: "utf8", timeout: 20_000, windowsHide: true, + }); + expect(child.error, child.stderr).toBeUndefined(); + const reportPath = join(home, "parent-stop-result.json"); + expect(existsSync(reportPath), child.stderr).toBe(true); + const report = JSON.parse(readFileSync(reportPath, "utf8")) as { + calls: { native: number; grok: number; service: number; partB: number }; + urls: string[]; unexpectedIo: string[]; nonce?: string; receiptExists: boolean; postPartA: unknown[]; + }; + expect(report.unexpectedIo, child.stderr).toEqual([]); + const lines = child.stdout.trim().split("\n"); + const summary = JSON.parse(lines[lines.length - 1]!) as Record; + return { ...report, summary, exitCode: child.status, stderr: child.stderr }; +} + +describe("plain ocx stop on Windows: Part A holds the child stop", () => { + for (const [outcome, service, reason] of [ + ["denied", "failed", "access-denied"], + ["survived", "failed", undefined], + ["unknown", "state-unknown", "evidence-unreadable"], + ["unverified-budget", "failed", "unverified-budget"], + ] as const) { + test(`${outcome}: no POST, ownership blocked, shared teardown skipped`, () => { + const run = runStop(outcome); + expect(run.calls.service, run.stderr).toBe(1); + expect(run.urls, run.stderr).toEqual([]); + expect(run.calls.partB).toBe(0); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.nonce).toBeUndefined(); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ schema: "ocx-stop/1", outcome: "failed", service, proxy: "unknown", sharedTeardown: "skipped" }); + if (reason) expect(run.summary.serviceStopReason).toBe(reason); + else expect("serviceStopReason" in run.summary).toBe(false); + expect(run.stderr).toContain("The proxy was left running and the shared teardown skipped"); + }); + } +}); + +describe("plain ocx stop on Windows: Part B after a stopped Part A", () => { + test("freed: the POST's receipt records Part A, then the restore runs and the receipt clears", () => { + const run = runStop("stopped", "freed"); + expect(run.urls, run.stderr).toHaveLength(1); + expect(run.postPartA).toEqual([{ partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }]); + expect(run.calls.partB).toBe(1); + expect(run.calls.native).toBe(1); + expect(run.receiptExists).toBe(false); + expect(run.exitCode, run.stderr).toBe(0); + expect(run.summary).toMatchObject({ outcome: "stopped", service: "stopped", proxy: "stopped", portHeldByForeign: false }); + }); + + test("our identity back on the port: respawned, shared teardown skipped, the receipt kept", () => { + const run = runStop("stopped", "survived"); + expect(run.urls).toHaveLength(1); + expect(run.calls.native).toBe(0); + expect(run.receiptExists).toBe(true); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", proxy: "respawned", sharedTeardown: "skipped", portHeldByForeign: false }); + }); + + test("a foreign listener: the stop succeeds and says so", () => { + const run = runStop("stopped", "port-held-by-foreign"); + expect(run.calls.native).toBe(1); + expect(run.exitCode, run.stderr).toBe(0); + expect(run.summary).toMatchObject({ outcome: "stopped", portHeldByForeign: true }); + }); + + test("an unconfirmed port: failed with evidence-unreadable, nothing restored", () => { + const run = runStop("stopped", "unknown"); + expect(run.calls.native).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", proxy: "unknown", serviceStopReason: "evidence-unreadable" }); + }); + + test("absent: today's stop, the POST without Part B and without the 6 s hold", () => { + const run = runStop("absent"); + expect(run.urls).toHaveLength(1); + expect(run.postPartA).toEqual([{ partA: "absent", manager: { scheduler: false, winsw: false }, wrappers: [] }]); + expect(run.calls.partB).toBe(0); + expect(run.exitCode, run.stderr).toBe(0); + expect(run.summary).toMatchObject({ outcome: "stopped", service: "absent" }); + expect("portHeldByForeign" in run.summary).toBe(false); + }); +}); From 8d172a78a18b1fb2b1db805c8588ba3e2c744131 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 03:06:43 +0900 Subject: [PATCH 06/53] fix(service): the updater re-checks its authority right before each manager stop Refs #7771 --- .../261111_windows_service_stop/030_design.md | 2 +- .../040_verification.md | 2 +- .../060_implementation.md | 2 +- src/service/windows-checked-stop.ts | 7 +++-- src/service/windows-service-stop.ts | 2 +- structure/ops/windows-checked-stop.md | 2 +- tests/windows/windows-checked-stop.test.ts | 29 ++++++++++++++++++- 7 files changed, 38 insertions(+), 8 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 5e6e967ba1b..32b4b673276 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -19,7 +19,7 @@ Part A replaces, on Windows only, the "manager stop + best-effort wrapper kill" - `ocx service uninstall` / `remove` call part A before deleting the task (today they call `stopWindows` and kill no wrappers, so a looping wrapper can outlive its task, #764), then part B. - `prepareServiceInstall` (service install and backend switch) calls part A instead of its manager stop + wrapper kill, then its existing tracked-proxy stop, then part B. A part B result other than a free port (our identity back, or `portHeldByForeign`) stops install and switch before they publish assets or start the service, with the port owner named; the service outcome stays `stopped` and no enum changes. - `/api/stop`: when the caller presents a valid receipt, the parent `ocx stop` has already run part A, so the route skips part A (no second enumeration, `/end` or wait inside the proxy) and only performs its graceful shutdown; part B belongs to the parent, since the route cannot observe its own post-exit window. Only `ocx stop` and `ocx service stop` write the receipt, and each sends the POST only after its own part A returned `stopped` or `absent` (see "Two parts"). The receipt alone is not the proof: after part A the parent records its outcome in the receipt (`partA: "stopped" | "absent"`, with the bound manager and the wrapper set it ended as pid + creation time). The route skips part A only when that record is present for this attempt and the manager it names is still the registered one; a receipt without the record (an older CLI, or a receipt written before part A) gets today's route behavior, which re-runs the service stop and answers 409 on failure. Without a receipt the existing `respawnable_service` refusal stays for a respawnable Task Scheduler service. Where the respawn risk is `none` the route splits by case. For a registered WinSW service it keeps today's in-process service stop unchanged; part A is not run there, because WinSW's own stop ends this very process and cannot be capped from inside it. For a deleted task with a stray wrapper it runs only part A's wrapper sweep (bind, terminate, handle wait) with a 4 s cap, and then, before it answers, its own shared teardown (`performStopTeardown`, which a caller without a receipt still gets); the drain runs after the answer. Every parent POST gets the full 10 s abort, never a shortened one, and admission always counts it at 10 s; the implementation must measure the teardown's worst case and keep sweep plus teardown under those 10 s, and 040 carries that as a test. A sweep overrun answers `service_state_unknown` and changes nothing further. -- The updater's `stopWindowsServiceWrappersBestEffort` moves to part A in the same change (Task Scheduler backend; for the native WinSW backend it keeps today's behavior of stopping WinSW without a wrapper kill, because WinSW has no `:loop` wrapper), through an authorizer-gated variant (`guardedStopWindowsServiceChecked(authorize)`) that runs the authorizer once at entry and again before each terminate, as the existing `mutationGate` does, so an update cannot race a respawned runtime either. The updater keeps its own reclaim checks. Its three call sites in src/update/job.ts discard the result today, and one runs inside the synchronous `preparePort` callback; each gets a return path (the callback becomes async, or part A runs before it) so a non-`stopped` outcome really aborts the update before files are replaced. +- The updater's `stopWindowsServiceWrappersBestEffort` moves to part A in the same change (Task Scheduler backend; for the native WinSW backend it keeps today's behavior of stopping WinSW without a wrapper kill, because WinSW has no `:loop` wrapper), through an authorizer-gated variant (`guardedStopWindowsServiceChecked(authorize)`) that runs the authorizer once at entry and again before each manager stop and each terminate, as the existing `mutationGate` does, so an update cannot race a respawned runtime either. The updater keeps its own reclaim checks. Its three call sites in src/update/job.ts discard the result today, and one runs inside the synchronous `preparePort` callback; each gets a return path (the callback becomes async, or part A runs before it) so a non-`stopped` outcome really aborts the update before files are replaced. Desktop reaches it through two `ocx stop` children, both ordinary CLI processes run as the user: the consented takeover at startup (`--expect-*`, under the startup sequence's deadline, which `resolve_consent` extends by the consent wait plus 60 s on a Windows approval) and the exit/restart drain (plain `ocx stop --json` under `runtime_stop::DEADLINE`, 30 s, desktop/src-tauri/src/exit.rs). The #5443 boundary is unchanged and Desktop never elevates. diff --git a/devlog/_plan/261111_windows_service_stop/040_verification.md b/devlog/_plan/261111_windows_service_stop/040_verification.md index 7bc683508b9..a0b2dd6d17a 100644 --- a/devlog/_plan/261111_windows_service_stop/040_verification.md +++ b/devlog/_plan/261111_windows_service_stop/040_verification.md @@ -19,7 +19,7 @@ - `ocx uninstall` runs part B after a part A `stopped` and keeps client configuration on any other outcome; service uninstall/install refuse before deleting/replacing the task on a non-stopped outcome. - `/api/stop` with a receipt skips part A; the whole consented stop fits the stated budget with injected clocks. - Guarded path: a stray wrapper bound by the rule is `bound-stray` and reaches part A; an unbindable one stays approval-changed. -- Output mapping per path: `ServiceStopOutcome`, `ocx-stop/1` (no new outcome/proxy enum values; additive fields only), `/api/stop` codes, Desktop parser unit tests with old and new documents; updater variant refuses without authorization at entry and before each terminate. +- Output mapping per path: `ServiceStopOutcome`, `ocx-stop/1` (no new outcome/proxy enum values; additive fields only), `/api/stop` codes, Desktop parser unit tests with old and new documents; updater variant refuses without authorization at entry, before each manager stop and before each terminate. - Plain `ocx stop`: part A `denied`, `survived` and `unknown` each send no `/api/stop` POST, set `ownershipBlocked`, skip the shared restore, and record `outcome: failed`, `proxy: unknown`, `sharedTeardown: skipped`. - `absent`: no registered task or WinSW service and no bound wrapper → `ServiceStopOutcome` `absent`, no "Service manager stopped.", the POST is sent, part B does not run (no 6 s hold); the same with an unreadable process list → still `absent`, and plain `ocx stop` still stops the proxy. - Budget, with injected clocks: admission refuses with `unverified-budget` and changes nothing (no `/end`, terminate or POST) when the remaining time is below the worst case still to come; once admitted, every step at its cap ends before the deadline; the graceful exit wait is never shortened; part B never gets less than 6 s; `--deadline-ms` from both Desktop paths is honored, and a missing flag means 60 s; macOS/Linux ignore it. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index af37d39ccd5..33807f98daa 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -44,7 +44,7 @@ Contract doc: `structure/ops/windows-checked-stop.md`. - **Unreadable candidates with nothing registered.** An unreadable wrapper image above the child counts only while a manager is registered. A hand-started proxy under an elevated shell would otherwise lose the stop it gets today. A matching wrapper without a creation time stays `unknown` even with nothing registered. - **Snapshot type.** The snapshot type is `WindowsStopProcessEntry`, which is `WindowsProcessEntry` plus `startTime`, in the evidence module. `windowsProcessList` and the guarded inspection stay untouched until PR2 switches them. - **The Schedule PID is read before admission.** Part A reads the Task Scheduler host PID from the SCM (cap 2 s) right after the registration probe, before admission, whenever a task is registered or a stray sweep runs. 030's tables do not list this read. Admission measures the time left on the caller's deadline, so the read's actual elapsed time is already taken out of what admission sees, the same way the step 1 stray enumeration is. -- **Not every effect is capped.** The registration probes, enumeration, Schedule PID, `/end`, WinSW stop and poll, terminates, manager state and `proxyAlive` run under caps. `openWatch` and the authorizer do not. `openWatch` is one `OpenProcess` per wrapper with no wait in it. The authorizer runs before each terminate, inside the shared 3 s terminate-and-wait window, so a slow authorizer eats that window: the next terminate gets less time, and an authorizer that never returns hangs Part A. PR2's authorizer (the updater's `mutationGate` check) has to be bounded by its caller. +- **Not every effect is capped.** The registration probes, enumeration, Schedule PID, `/end`, WinSW stop and poll, terminates, manager state and `proxyAlive` run under caps. `openWatch` and the authorizer do not. `openWatch` is one `OpenProcess` per wrapper with no wait in it. The authorizer runs before each manager stop and before each terminate, inside the shared 3 s terminate-and-wait window, so a slow authorizer eats that window: the next terminate gets less time, and an authorizer that never returns hangs Part A. PR2's authorizer (the updater's `mutationGate` check) has to be bounded by its caller. - **Fractional timeouts.** A cap derived from the deadline (`windowEnd - now`) is fractional on the real clock. The spawned terminate fallback floors it to whole milliseconds before building the script and calling `execFile`, and spawns nothing when it floors to 0, because `execFile` reads a 0 timeout as "no timeout". ## ARM64 decision for PR2 diff --git a/src/service/windows-checked-stop.ts b/src/service/windows-checked-stop.ts index f81eb759eec..79f4043c064 100644 --- a/src/service/windows-checked-stop.ts +++ b/src/service/windows-checked-stop.ts @@ -98,7 +98,7 @@ export interface WindowsCheckedStopOptions { /** A snapshot that already ran (guarded: revalidateManager's), reused for the bind. */ bindSnapshot?: readonly WindowsStopProcessEntry[]; admission?: WindowsStopAdmission; - /** Throws to refuse. Runs at entry and before each terminate. */ + /** Throws to refuse. Runs at entry, before each manager stop (/end, the WinSW stop) and before each terminate. */ authorize?: () => MaybePromise; } @@ -298,6 +298,8 @@ export async function stopWindowsServiceChecked( // Step 3: ask the manager. if (taskRegistered) { + // The updater re-checks its authority right before the manager stop, as before each terminate. + await authorize(); result.changed = true; const ended = await runCapped(clock, C.taskEnd, deps.endTask); if (!ended.ok) return finish("unknown", "evidence-unreadable", "the task end command did not answer within its cap"); @@ -305,6 +307,7 @@ export async function stopWindowsServiceChecked( if (ended.value === "failed") return finish("unknown", "evidence-unreadable", "the task end command failed"); } if (winswStarted) { + await authorize(); result.changed = true; const stop = await runCapped(clock, C.winswStopCommand, deps.winswStop); if (!stop.ok) return finish("unknown", "evidence-unreadable", "the native service stop did not answer within its cap"); @@ -422,7 +425,7 @@ async function unboundOnlyVerdict( return unboundStillAlive(binding, watches); } -/** The updater's variant: the authorizer runs at entry and before every terminate. */ +/** The updater's variant: the authorizer runs at entry, before each manager stop and before every terminate. */ export function guardedStopWindowsServiceChecked( authorize: () => MaybePromise, deps: WindowsCheckedStopDeps, diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts index ec5148e2085..02830d2bcf1 100644 --- a/src/service/windows-service-stop.ts +++ b/src/service/windows-service-stop.ts @@ -61,7 +61,7 @@ export interface WindowsPartARequest { admission?: WindowsStopAdmission; /** The evidence snapshot (windowsProcessSnapshot) a guarded revalidation already read. */ bindSnapshot?: readonly WindowsStopProcessEntry[]; - /** The updater's authorizer: throws to refuse, at entry and before each terminate. */ + /** The updater's authorizer: throws to refuse, at entry, before each manager stop and before each terminate. */ authorize?: () => void | Promise; /** * A hard cap (the route: 6 s with a receipt, 4 s for a stray sweep). An overrun answers diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 34b4ad7a17a..8880aa962ee 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -70,7 +70,7 @@ deepest first, waits on the bind-time watches (terminates and wait share 3 s), a manager state once (1 s). Denied stops every further step. A cap overrun is `unknown`. A wrapper or manager still running is `survived` (`unbound-wrapper` when the survivor was never bound). WinSW is `survived` while the proxy is still alive after the service reports stopped. The authorizer -variant `guardedStopWindowsServiceChecked` runs its authorizer at entry and before each terminate and +variant `guardedStopWindowsServiceChecked` runs its authorizer at entry, before each manager stop (/end, the WinSW stop) and before each terminate and reports a refusal as `WindowsCheckedStopRefusedError` with what had already happened. Each step that asks the machine runs under a cap that races a timer and rechecks the elapsed time diff --git a/tests/windows/windows-checked-stop.test.ts b/tests/windows/windows-checked-stop.test.ts index 9580c9a7d84..d5690b243d5 100644 --- a/tests/windows/windows-checked-stop.test.ts +++ b/tests/windows/windows-checked-stop.test.ts @@ -383,7 +383,8 @@ describe("checked Windows stop, part A", () => { let checks = 0; const error = await guardedStopWindowsServiceChecked(() => { checks += 1; - if (checks === 3) throw new Error("respawned runtime"); + // Entry, before /end, before terminate:30, before terminate:20. + if (checks === 4) throw new Error("respawned runtime"); }, h.deps).catch(caught => caught); expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); expect((error as WindowsCheckedStopRefusedError).partial.terminated).toEqual([30]); @@ -391,6 +392,32 @@ describe("checked Windows stop, part A", () => { }); }); +describe("the updater's authorizer guards each manager stop", () => { + test("a refusal right before /end sends no /end and changes nothing", async () => { + const h = harness(); + let checks = 0; + const error = await guardedStopWindowsServiceChecked(() => { + checks += 1; + if (checks === 2) throw new Error("service home changed"); + }, h.deps).catch(caught => caught); + expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); + expect((error as WindowsCheckedStopRefusedError).partial.changed).toBe(false); + expect(mutations(h.calls)).toEqual([]); + }); + + test("a refusal right before the WinSW stop sends no stop and changes nothing", async () => { + const h = harness({ probeScheduler: () => ({ status: "absent" }), probeWinsw: () => "started" as WinswStatus }); + let checks = 0; + const error = await guardedStopWindowsServiceChecked(() => { + checks += 1; + if (checks === 2) throw new Error("service home changed"); + }, h.deps).catch(caught => caught); + expect(error).toBeInstanceOf(WindowsCheckedStopRefusedError); + expect((error as WindowsCheckedStopRefusedError).partial.changed).toBe(false); + expect(mutations(h.calls)).toEqual([]); + }); +}); + describe("checked Windows stop budget", () => { test("admission minimums with default settings", () => { // 030's 48 / 56 / 42 s plus the 0.5 s closing sample part B can take past its window. From 2082b453b15d528cdbb05db11dd8bf10e46bd702 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 03:09:42 +0900 Subject: [PATCH 07/53] refactor(service): move commandLineHasPathToken to a leaf module to break the guarded-manager import cycle Refs #7771 --- src/service/windows-command-line-token.ts | 23 +++++++++++++++++++++ src/service/windows-guarded-manager.ts | 25 +++++------------------ src/service/windows-stop-binding.ts | 2 +- src/service/windows-stop-confirm.ts | 2 +- 4 files changed, 30 insertions(+), 22 deletions(-) create mode 100644 src/service/windows-command-line-token.ts diff --git a/src/service/windows-command-line-token.ts b/src/service/windows-command-line-token.ts new file mode 100644 index 00000000000..54f51b8073c --- /dev/null +++ b/src/service/windows-command-line-token.ts @@ -0,0 +1,23 @@ +// Leaf module: whole-token path matching on a process command line, shared by the guarded +// manager inspection, the checked-stop binding rule and Part B without an import cycle. + +/** + * True when path appears in commandLine as a COMPLETE token — the same boundary + * rule the windows-service-wrappers kill list uses, ported so the read-only proof + * cannot accept a substring hit. A quoted path inside an argument string counts; + * a suffix or prefix glued to it does not. + */ +export function commandLineHasPathToken(commandLine: string | null | undefined, path: string): boolean { + if (!commandLine || !path) return false; + const haystack = commandLine.toLowerCase(); + const needle = path.toLowerCase(); + const boundary = (ch: string | undefined): boolean => ch === undefined || /[\s"']/.test(ch); + let index = 0; + for (;;) { + index = haystack.indexOf(needle, index); + if (index < 0) return false; + const end = index + needle.length; + if (boundary(haystack[index - 1]) && boundary(haystack[end])) return true; + index = end; + } +} diff --git a/src/service/windows-guarded-manager.ts b/src/service/windows-guarded-manager.ts index f6e929d5855..98edb2f8a45 100644 --- a/src/service/windows-guarded-manager.ts +++ b/src/service/windows-guarded-manager.ts @@ -33,6 +33,11 @@ import { bindWindowsStopWrappers } from "./windows-stop-binding"; import { probeWindowsSchedulerTask, querySchtasks, type WindowsSchedulerTaskProbe } from "./windows-scheduler"; import { windowsWscript } from "./windows-wscript"; import { windowsTaskRegistrationHealthy, windowsTaskRegistrationRefreshableLegacy } from "./windows-taskxml"; +import { commandLineHasPathToken } from "./windows-command-line-token"; + +// Re-exported for existing importers; the function lives in a leaf module to keep the binding rule +// (windows-stop-binding.ts) free of an import cycle with this inspection. +export { commandLineHasPathToken }; /** Runtime table of one Win32 process; a null parent means the chain is unreadable. */ export interface WindowsProcessEntry { @@ -224,26 +229,6 @@ export function currentWindowsTaskUserIds(): readonly string[] | null { } } -/** - * True when path appears in commandLine as a COMPLETE token — the same boundary - * rule the windows-service-wrappers kill list uses, ported so the read-only proof - * cannot accept a substring hit. A quoted path inside an argument string counts; - * a suffix or prefix glued to it does not. - */ -export function commandLineHasPathToken(commandLine: string | null | undefined, path: string): boolean { - if (!commandLine || !path) return false; - const haystack = commandLine.toLowerCase(); - const needle = path.toLowerCase(); - const boundary = (ch: string | undefined): boolean => ch === undefined || /[\s"']/.test(ch); - let index = 0; - for (;;) { - index = haystack.indexOf(needle, index); - if (index < 0) return false; - const end = index + needle.length; - if (boundary(haystack[index - 1]) && boundary(haystack[end])) return true; - index = end; - } -} function resolveWindowsDeps(deps: WindowsGuardedManagerDeps, live: boolean): ResolvedWindowsDeps { // "live" is false whenever the caller simulated platform win32 (tests or a future diff --git a/src/service/windows-stop-binding.ts b/src/service/windows-stop-binding.ts index b1c45dd6280..a2f7c2f0a18 100644 --- a/src/service/windows-stop-binding.ts +++ b/src/service/windows-stop-binding.ts @@ -26,7 +26,7 @@ */ import { createHash } from "node:crypto"; -import { commandLineHasPathToken } from "./windows-guarded-manager"; +import { commandLineHasPathToken } from "./windows-command-line-token"; import type { WindowsStopProcessEntry } from "./windows-process-evidence"; const WRAPPER_IMAGES = new Set(["wscript.exe", "cscript.exe", "cmd.exe"]); diff --git a/src/service/windows-stop-confirm.ts b/src/service/windows-stop-confirm.ts index 38d2c9baa6b..20d519c332e 100644 --- a/src/service/windows-stop-confirm.ts +++ b/src/service/windows-stop-confirm.ts @@ -18,7 +18,7 @@ import type { WindowsStopProcessEntry } from "./windows-process-evidence"; import { windowsHomeWrapperPids } from "./windows-stop-binding"; import { runCapped, windowsStopClock, type WindowsStopClock } from "./windows-checked-stop"; -import { commandLineHasPathToken } from "./windows-guarded-manager"; +import { commandLineHasPathToken } from "./windows-command-line-token"; import { WINDOWS_STOP_CAPS, windowsPartBWindowMs } from "./windows-stop-budget"; type MaybePromise = T | Promise; From a9ef1c85f8082f0a9e7600b024f5a8f337ec0c86 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 10:08:21 +0900 Subject: [PATCH 08/53] fix(service): the own-wrapper refusal is unknown even with nothing registered The guard turns the bind enumeration unreadable, and Part A's stray path read an unreadable table with nothing registered as absent: "no service, nothing to end". Plain `ocx stop` then stopped the child, restored shared config and recorded {partA:"absent"} beneath this home's own looping wrapper, which can relaunch the proxy over the restored config. The refusal now answers for itself in both paths: unknown, evidence-unreadable, UNDER_OWN_WRAPPER_DETAIL, changed false, with no /end and no terminate. Nothing has changed at that point, so nothing has to be undone. `allowUnderOwnWrapper` (the route and the updater) keeps its sweep. Refs #7771 --- src/service/windows-service-stop.ts | 13 +++++-- tests/fixtures/parent-stop-runner.ts | 36 +++++++++++++++++-- .../windows-service-stop-wiring.test.ts | 31 +++++++++++++++- tests/windows/windows-stop-ocx-stop.test.ts | 21 +++++++++-- 4 files changed, 93 insertions(+), 8 deletions(-) diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts index 02830d2bcf1..8a90035e52a 100644 --- a/src/service/windows-service-stop.ts +++ b/src/service/windows-service-stop.ts @@ -135,7 +135,7 @@ export async function runWindowsServiceStopPartA(request: WindowsPartARequest = // The guarded path binds on a snapshot it already read: check it before anything runs. if (request.bindSnapshot && guard(request.bindSnapshot)) return unknownResult(UNDER_OWN_WRAPPER_DETAIL); // Otherwise the bind enumeration is checked as it arrives; a hit reads as an unreadable - // table, which Part A answers before /end (registered) or as absent (nothing to end). + // table, which Part A answers before /end when a manager is registered. const deps = { ...base, enumerate: async (timeoutMs: number) => { @@ -152,7 +152,16 @@ export async function runWindowsServiceStopPartA(request: WindowsPartARequest = const result = await (request.authorize || request.capMs !== undefined ? guardedStopWindowsServiceChecked(authorize, deps, options) : stopWindowsServiceChecked(deps, options)); - if (underOwnWrapper && result.outcome === "unknown") result.detail = UNDER_OWN_WRAPPER_DETAIL; + // The refusal is the answer whatever the stray path would have concluded: with nothing + // registered an unreadable enumeration reads as absent ("nothing to end"), which would + // let the caller stop the child and restore shared config beneath this home's own looping + // wrapper. Nothing has been changed at this point in either path, so it is unknown. + if (underOwnWrapper) { + result.outcome = "unknown"; + result.reason = "evidence-unreadable"; + result.detail = UNDER_OWN_WRAPPER_DETAIL; + result.changed = false; + } return result; }; if (request.capMs === undefined) return run(); diff --git a/tests/fixtures/parent-stop-runner.ts b/tests/fixtures/parent-stop-runner.ts index cd0fc05a91e..0bd768bf844 100644 --- a/tests/fixtures/parent-stop-runner.ts +++ b/tests/fixtures/parent-stop-runner.ts @@ -54,6 +54,11 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { /** #7771: simulate win32 and answer Part A (and Part B) of the checked Windows stop. */ windowsPartA?: "stopped" | "absent" | "survived" | "denied" | "unknown" | "unverified-budget"; windowsPartB?: "freed" | "survived" | "port-held-by-foreign" | "unconfirmed" | "unknown"; + /** + * #7771: run the REAL Part A while this stop runs under this home's wrapper, with no task and + * no WinSW service registered, so the own-wrapper refusal has to answer for itself. + */ + windowsPartAUnderOwnWrapper?: boolean; }; if (options.proxyProbePort) { // Deliberately bypass every fixture mock for the real proxy-environment regression. @@ -100,7 +105,9 @@ mock.module("../../src/client/lifecycle-lock", () => ({ ...lifecycle, })); const endpoint = { hostname: "127.0.0.1", port: 10100 }; const fakePid = 4242; -const calls = { killed: 0, native: 0, grok: 0, cleared: 0, exited: 0, service: 0, lifecycle: 0, partB: 0 }; +const calls = { killed: 0, native: 0, grok: 0, cleared: 0, exited: 0, service: 0, lifecycle: 0, partB: 0, reclaim: 0 }; +/** The timeout each reclaimListenPort call asked for; #7771 makes a skipped 15 s wait visible. */ +const reclaimTimeouts: number[] = []; /** The Part A record each stop POST's receipt carried when the POST was sent (#7771). */ const postPartA: unknown[] = []; let receiptBlocker: import("bun:sqlite").Database | undefined; @@ -142,7 +149,7 @@ const receipts = await import("../../src/config/pending-teardown"); process.on("exit", () => { receiptBlocker?.close(); writeFileSync(join(home, "parent-stop-result.json"), JSON.stringify({ - calls, urls, probes, transports, unexpectedIo, nonce, postPartA, + calls, reclaimTimeouts, urls, probes, transports, unexpectedIo, nonce, postPartA, receiptExists: nonce !== undefined && existsSync(receipts.pendingTeardownPathFor(nonce)), })); }); @@ -230,7 +237,12 @@ mock.module("../../src/grok/inject", () => ({ const systemEnv = await import("../../src/server/system-env"); mock.module("../../src/server/system-env", () => ({ ...systemEnv, revertSystemEnv() {} })); const portReclaim = await import("../../src/server/port-reclaim"); -mock.module("../../src/server/port-reclaim", () => ({ ...portReclaim, reclaimListenPort: async () => {} })); +mock.module("../../src/server/port-reclaim", () => ({ ...portReclaim, + reclaimListenPort: async (_port: number, _host: string, options?: { timeoutMs?: number }) => { + calls.reclaim += 1; + reclaimTimeouts.push(options?.timeoutMs ?? 0); + }, +})); // Only shim preflight is unrelated to this stop contract; parsing and dispatch stay real. mock.module("../../src/cli/codex-shim-autorestore", () => ({ async maybeAutoRestoreCodexShimAsync() {} })); @@ -305,6 +317,24 @@ if (options.windowsPartA) { partB: async () => { calls.partB++; return partBResult(options.windowsPartB ?? "freed"); }, }); } +if (options.windowsPartAUnderOwnWrapper) { + // This stop runs under this home's wrapper: ocx stop <- cmd.exe (.cmd) <- wscript.exe (.vbs), + // with nothing registered. The real Part A answers the own-wrapper refusal. + (await import("../../src/cli/stop-windows-checked")).resetOcxWindowsStopForTests("win32"); + const { PATHS, launcher, loop } = await import("../helpers/windows-checked-stop-fixtures"); + const self = { pid: process.pid, parentPid: 30, name: "ocx.exe", commandLine: "ocx stop", startTime: 6_000 }; + (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ + liveDeps: () => ({ + canonicalPaths: PATHS, childPid: null, selfPid: process.pid, + probeScheduler: () => { calls.service++; return { status: "absent" }; }, probeWinsw: () => "nonexistent", + enumerate: () => [launcher(), loop(), self], + scheduleServicePid: () => null, endTask: () => unexpected("schtasks /end"), taskState: () => "not-running", + winswStop: () => "sent", winswState: () => "stopped", proxyAlive: () => false, + openWatch: () => ({ status: "gone" }), + terminate: () => unexpected("terminate"), + }), + }); +} if (options.handover) process.env.OCX_UPDATE_TEARDOWN_HANDOVER = "1"; if (options.exitCleanup) { delete process.env.OCX_SERVICE; diff --git a/tests/windows/windows-service-stop-wiring.test.ts b/tests/windows/windows-service-stop-wiring.test.ts index 3e5d0461538..454896127ef 100644 --- a/tests/windows/windows-service-stop-wiring.test.ts +++ b/tests/windows/windows-service-stop-wiring.test.ts @@ -22,10 +22,11 @@ import { windowsStopAdmission, resolveStopDeadlineMs, parseStopDeadlineMs, + serviceStopOutcomeFromWindows, windowsStopDeadlineAt, } from "../../src/service/windows-service-stop"; import { DEFAULT_WINDOWS_STOP_DEADLINE_MS } from "../../src/service/windows-stop-budget"; -import { PATHS, SCHEDULE_PID, entry, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; +import { PATHS, SCHEDULE_PID, entry, launcher, loop, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; import { partAResult } from "../helpers/windows-stop-caller-fixtures"; afterEach(() => { @@ -111,6 +112,34 @@ describe("Part A through the caller wiring", () => { expect(allowed.calls).toContain("end"); }); + test("under this home's wrapper with nothing registered: unknown, never absent, and nothing is ended", async () => { + // self 70 (ocx stop) <- cmd 30 (.cmd of this home) <- wscript 20 (.vbs of this home), with + // no task and no WinSW service. The own-wrapper guard makes the enumeration unreadable, and + // the stray path would otherwise read that as absent: no service registered, nothing to do. + const tree = () => [launcher(), loop(), entry(70, 30, "ocx.exe", "ocx stop", 6_000)]; + const calls: string[] = []; + const refused = machine({ + selfPid: 70, + probeScheduler: () => ({ status: "absent" }), + enumerate: () => { calls.push("enumerate"); return tree(); }, + }); + setWindowsServiceStopHooksForTests({ liveDeps: () => refused.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.reason).toBe("evidence-unreadable"); + expect(result.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(result.changed).toBe(false); + expect(result.terminated).toEqual([]); + expect(calls).toEqual(["enumerate"]); + expect(serviceStopOutcomeFromWindows(result)).toBe("state-unknown"); + expect(windowsPartARecord(result)).toBeNull(); + + // The same machine where the refusal is allowed (the route and the updater) still sweeps. + const allowed = machine({ selfPid: 70, probeScheduler: () => ({ status: "absent" }), enumerate: tree }); + setWindowsServiceStopHooksForTests({ liveDeps: () => allowed.deps }); + expect((await runWindowsServiceStopPartA({ allowUnderOwnWrapper: true })).outcome).not.toBe("unknown"); + }); + test("a cap overrun is unknown at once, and nothing is ended or terminated after it", async () => { let gate: () => boolean = () => true; const fake = machine({ diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts index 96f3032f0dc..879207a2990 100644 --- a/tests/windows/windows-stop-ocx-stop.test.ts +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -23,11 +23,11 @@ afterEach(() => { removeTreeWithRetry(home); }); const restored = { success: true, message: "native Codex restored", artifacts: { config: { state: "restored" }, catalog: { state: "restored" }, history: { state: "restored" } } }; -function runStop(windowsPartA: string, windowsPartB?: string) { +function runStop(windowsPartA: string | undefined, windowsPartB?: string, extra: Record = {}) { const child = spawnSync(process.execPath, [fixturePath("parent-stop-runner.ts")], { cwd: repoPath(), env: { ...process.env, OPENCODEX_HOME: home }, input: JSON.stringify({ receipt: true, json: true, response: { success: true, sharedTeardown: "deferred" }, - restore: restored, windowsPartA, ...(windowsPartB ? { windowsPartB } : {}) }), + restore: restored, ...(windowsPartA ? { windowsPartA } : {}), ...(windowsPartB ? { windowsPartB } : {}), ...extra }), encoding: "utf8", timeout: 20_000, windowsHide: true, }); expect(child.error, child.stderr).toBeUndefined(); @@ -68,6 +68,23 @@ describe("plain ocx stop on Windows: Part A holds the child stop", () => { }); describe("plain ocx stop on Windows: Part B after a stopped Part A", () => { + + test("under this home's wrapper with nothing registered: unknown, no POST, no shared teardown", () => { + // The real Part A refuses: this stop runs under this home's .cmd/.vbs wrapper and no + // service is registered. Reading that as absent would stop the child and restore shared + // config beneath a wrapper that can relaunch the proxy. + const run = runStop(undefined, undefined, { windowsPartAUnderOwnWrapper: true }); + expect(run.calls.service, run.stderr).toBe(1); + expect(run.urls, run.stderr).toEqual([]); + expect(run.calls.partB).toBe(0); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.nonce).toBeUndefined(); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", service: "state-unknown", proxy: "unknown", sharedTeardown: "skipped", serviceStopReason: "evidence-unreadable" }); + expect(run.stderr).toContain("this stop runs under this home's service wrapper"); + }); + test("freed: the POST's receipt records Part A, then the restore runs and the receipt clears", () => { const run = runStop("stopped", "freed"); expect(run.urls, run.stderr).toHaveLength(1); From 398b727a19299e06ebc915214d4f5f7c3e5d3f16 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 15:03:14 +0900 Subject: [PATCH 09/53] fix(cli): Part B owns the port wait on the guarded and uninstall paths too The skipPortWait key was windowsPartBDue(), which is snapshotless-only because it also gates the legacy respawn re-poll. The guarded path's Part B runs after settle, and uninstall's Part B runs as the respawn window, so both still ran stopProxy's legacy 15 s waitForStoppedPort on top of Part B. WindowsPartBOwnsPortWait() answers only the port-wait question for both Windows paths; windowsPartBDue() keeps the snapshotless-only respawn semantics. The two uninstall stopProxy calls pass the same predicate. Non-Windows is unchanged because both predicates require the simulated win32 platform. Refs #7771 --- src/cli/index.ts | 10 +++++---- src/cli/stop-windows-checked.ts | 12 ++++++++++ tests/cli/uninstall.test.ts | 20 +++++++++++++++-- tests/fixtures/parent-stop-runner.ts | 12 ++++++++-- tests/windows/windows-stop-ocx-stop.test.ts | 25 +++++++++++++++++++++ 5 files changed, 71 insertions(+), 8 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index 3e2082835e0..acc7b7b4c31 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -125,7 +125,7 @@ import { type StopApproval, } from "./stop-approval"; import { summarizeStopRun, type StopOutcome, type StopRunRecord } from "./stop-report"; -import { confirmGuardedFreed, inspectGuardedTarget, noServiceStop, observeGuardedStopped, orphanDiscoveryCap, stopGuardedManager, stopServiceForOcxStop, stopServiceForUninstall, stopWindowFailed, verifyUninstallStopWindow, windowsChildStopHeld, windowsPartAHolds, windowsPartAReceiptRecord, windowsPartBDue } from "./stop-windows-checked"; +import { confirmGuardedFreed, inspectGuardedTarget, noServiceStop, observeGuardedStopped, orphanDiscoveryCap, stopGuardedManager, stopServiceForOcxStop, stopServiceForUninstall, stopWindowFailed, verifyUninstallStopWindow, windowsChildStopHeld, windowsPartAHolds, windowsPartAReceiptRecord, windowsPartBDue, windowsPartBOwnsPortWait } from "./stop-windows-checked"; import { runCli } from "./root"; import { isProcessAlive, ProxyOwnershipRefusedError, refusalNextStep, stopProxy } from "../lib/process-control"; import { startupDataPlaneToken } from "../lib/service-secrets"; @@ -1173,7 +1173,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // Only an exact endpoint may direct the request; the configured fallback is a guess // good enough to record an obligation against, not to POST a stop to. runtimeEndpoint: exact ?? undefined, - skipPortWait: windowsPartBDue(), // #7771: Part B owns the Windows port wait after a checked stop. + skipPortWait: windowsPartBOwnsPortWait(), // #7771: Part B owns the Windows port wait on both paths. }); // A valid receipt means the proxy deferred shared teardown to this process. If the // receipt could not be written, the proxy restores it itself and the caller must not @@ -1596,11 +1596,13 @@ async function handleUninstall() { if (!live.pid) { throw new Error(`a proxy is answering on port ${live.port} but no process id could be resolved for it; stop it from the home that started it, then rerun`); } - await stopProxy(live.pid); + // #7771: on Windows the respawn-window step below is Part B, which owns the port + // wait; the legacy 15 s waitForStoppedPort must not run on top of it. + await stopProxy(live.pid, { skipPortWait: windowsPartBDue() }); observed.proxyProvenDown = true; return true; } - await stopProxy(pid); + await stopProxy(pid, { skipPortWait: windowsPartBDue() }); removePid(pid); removeRuntimePort(pid); observed.proxyProvenDown = true; diff --git a/src/cli/stop-windows-checked.ts b/src/cli/stop-windows-checked.ts index f050d86806a..db61631f033 100644 --- a/src/cli/stop-windows-checked.ts +++ b/src/cli/stop-windows-checked.ts @@ -156,6 +156,18 @@ export function windowsPartBDue(): boolean { return windows() && state.snapshotless && state.result?.outcome === "stopped"; } +/** + * Part B owns the port wait after a stopped Part A on BOTH Windows paths (#7771). + * + * `windowsPartBDue` stays snapshotless-only because it also gates the legacy respawn + * re-poll (`schedulerCanRespawn`). This predicate is only about the port wait: on the + * guarded path Part B runs after `settle`, so `stopProxy`'s legacy 15 s + * `waitForStoppedPort` must not run on top of it. + */ +export function windowsPartBOwnsPortWait(): boolean { + return windows() && state.result?.outcome === "stopped"; +} + /** * A Part A result other than stopped/absent holds the child stop (030 "Two parts"): no * POST, the shared teardown skipped, proxy unknown. Records the additive reason. diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index ab10de34df4..ac485a229c0 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -124,11 +124,27 @@ describe("full uninstall command", () => { expect(uninstallBody).toContain('runStep("service stopped"'); expect(uninstallBody).toContain('runStep("proxy stopped"'); expect(uninstallBody).toContain('runStep("service removed"'); - expect(uninstallBody).toContain("await stopProxy(pid);"); + expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBDue() });"); expect(uninstallBody).toContain("uninstallServiceDetailed()"); expect(uninstallBody.indexOf('runStep("service stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("proxy stopped"')); expect(uninstallBody.indexOf('runStep("proxy stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("service removed"')); - expect(uninstallBody.indexOf("await stopProxy(pid);")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); + expect(uninstallBody.indexOf("await stopProxy(pid, { skipPortWait: windowsPartBDue() });")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); + }); + + test("Windows uninstall leaves the port wait to Part B instead of stopProxy's legacy 15 s", async () => { + const cli = await readText("src/cli/index.ts"); + const uninstallBody = cli.slice(cli.indexOf("async function handleUninstall()"), cli.indexOf("type HealthCheck")); + // #7771: after a stopped Part A on Windows the respawn-window step IS Part B, which + // owns the port wait; running stopProxy's 15 s waitForStoppedPort on top of it spends + // the stop budget twice. Both tracked and orphan stops pass the same predicate. + expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBDue() });"); + expect(uninstallBody).toContain("await stopProxy(live.pid, { skipPortWait: windowsPartBDue() });"); + const checked = await readText("src/cli/stop-windows-checked.ts"); + // The predicate is about the port wait only; the legacy re-poll key stays snapshotless. + const portWait = checked.slice(checked.indexOf("export function windowsPartBOwnsPortWait")); + expect(portWait).toContain('return windows() && state.result?.outcome === "stopped";'); + const partBDue = checked.slice(checked.indexOf("export function windowsPartBDue"), checked.indexOf("export function windowsPartBOwnsPortWait")); + expect(partBDue).toContain("state.snapshotless"); }); test("restore forwards the explicit provider-table removal flag and warns before mutation", async () => { diff --git a/tests/fixtures/parent-stop-runner.ts b/tests/fixtures/parent-stop-runner.ts index 0bd768bf844..97abec961e7 100644 --- a/tests/fixtures/parent-stop-runner.ts +++ b/tests/fixtures/parent-stop-runner.ts @@ -21,6 +21,8 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { clientServer?: "legacy" | "unknown" | "wrong-pid" | "malformed" | "oversized" | "unauthorized" | "redirect" | "body-timeout"; managerCascades?: boolean; guarded?: boolean; + /** #7771: the guarded path on a simulated win32, with a bound scheduler manager (separate child stop). */ + windowsGuarded?: boolean; orphan?: boolean; exitCleanup?: boolean; nativeSignal?: boolean; @@ -182,7 +184,9 @@ mock.module("../../src/service", () => ({ ...service, stopServiceIfInstalledDeta } })); if (options.guarded) { const approval = { pid: fakePid, ...endpoint, configHome: home, cliVersion: "2.93.0", compatibilityToken: "fixture" }; - const manager = { kind: "bound", pid: fakePid, managerPid: fakePid + 1, backend: "launchd" }; + const manager = options.windowsGuarded + ? { kind: "bound", pid: fakePid, managerPid: fakePid + 1, backend: "scheduler", childNeedsSeparateStop: true } + : { kind: "bound", pid: fakePid, managerPid: fakePid + 1, backend: "launchd" }; const stopApproval = await import("../../src/cli/stop-approval"); mock.module("../../src/cli/stop-approval", () => ({ ...stopApproval, parseStopApproval: () => ({ ok: true, json: true, approval }), @@ -191,7 +195,8 @@ if (options.guarded) { })); const guarded = await import("../../src/service/guarded-manager-target"); mock.module("../../src/service/guarded-manager-target", () => ({ ...guarded, - inspectGuardedManagerTarget: () => manager, observeGuardedManagerStopped: () => "stopped", + inspectGuardedManagerTarget: () => manager, + observeGuardedManagerStopped: () => (options.windowsGuarded ? "inactive" : "stopped"), })); } if (options.orphan || options.alreadyStopped) { @@ -315,6 +320,9 @@ if (options.windowsPartA) { (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ partA: async () => { calls.service++; return partAResult(options.windowsPartA!); }, partB: async () => { calls.partB++; return partBResult(options.windowsPartB ?? "freed"); }, + // The guarded inspection reads evidence before it asks the mocked manager; nothing may + // reach the host's process table. + ...(options.guarded ? { snapshot: async () => [], schedulePid: async () => null } : {}), }); } if (options.windowsPartAUnderOwnWrapper) { diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts index 879207a2990..5dc72d6aaeb 100644 --- a/tests/windows/windows-stop-ocx-stop.test.ts +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -35,6 +35,8 @@ function runStop(windowsPartA: string | undefined, windowsPartB?: string, extra: expect(existsSync(reportPath), child.stderr).toBe(true); const report = JSON.parse(readFileSync(reportPath, "utf8")) as { calls: { native: number; grok: number; service: number; partB: number }; + /** The timeout each reclaimListenPort call asked for; #7771 makes a skipped legacy wait visible. */ + reclaimTimeouts: number[]; urls: string[]; unexpectedIo: string[]; nonce?: string; receiptExists: boolean; postPartA: unknown[]; }; expect(report.unexpectedIo, child.stderr).toEqual([]); @@ -87,6 +89,8 @@ describe("plain ocx stop on Windows: Part B after a stopped Part A", () => { test("freed: the POST's receipt records Part A, then the restore runs and the receipt clears", () => { const run = runStop("stopped", "freed"); + // Part B owns the port wait: stopProxy's legacy 15 s wait must not run on top of it. + expect(run.reclaimTimeouts, run.stderr).toEqual([]); expect(run.urls, run.stderr).toHaveLength(1); expect(run.postPartA).toEqual([{ partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }]); expect(run.calls.partB).toBe(1); @@ -119,6 +123,27 @@ describe("plain ocx stop on Windows: Part B after a stopped Part A", () => { expect(run.summary).toMatchObject({ outcome: "failed", proxy: "unknown", serviceStopReason: "evidence-unreadable" }); }); + test("guarded: Part B owns the port wait, so the legacy 15 s wait never runs", () => { + // The guarded path's Part B runs after settle; keying the skip to the snapshotless + // flag alone left the approved endpoint's 15 s waitForStoppedPort on top of it. + const run = runStop("stopped", "freed", { guarded: true, windowsGuarded: true }); + expect(run.reclaimTimeouts, run.stderr).toEqual([]); + expect(run.calls.partB).toBe(1); + expect(run.urls).toHaveLength(1); + expect(run.exitCode, run.stderr).toBe(0); + }); + + test("guarded: our identity back on the port still holds the shared teardown", () => { + // A surviving listener never reaches the forced-kill fallback either: the guarded step + // reports manager-still-active and nothing is restored. + const run = runStop("stopped", "survived", { guarded: true, windowsGuarded: true }); + expect(run.reclaimTimeouts, run.stderr).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.calls.killed).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "manager-still-active", proxy: "unknown", sharedTeardown: "skipped" }); + }); + test("absent: today's stop, the POST without Part B and without the 6 s hold", () => { const run = runStop("absent"); expect(run.urls).toHaveLength(1); From 3c560d9628f76411f15994809eab271fbd76d1e8 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 15:07:20 +0900 Subject: [PATCH 10/53] fix(service): removal claims its teardown receipt before the child stop Part A leaves the Task Scheduler task registered, so the receipt-less POST from ocx service uninstall / remove still answered 409 respawnable_service and the uninstall refused with the proxy running. Removal now claims a receipt carrying Part A's record before the tracked stop, exactly as ocx service stop does, sends no child stop when the claim fails, and keeps the receipt until the restore succeeds (cli.ts completes it there). windowsServiceRemovalStop returns { ok, nonce } instead of a bare boolean so the caller can discharge the obligation at the right boundary. Refs #7771 --- src/service/cli-windows-stop.ts | 46 ++++++++++++---- src/service/cli.ts | 9 +++- tests/windows/windows-stop-callers.test.ts | 63 +++++++++++++++++++++- 3 files changed, 105 insertions(+), 13 deletions(-) diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index 13f3ff8beee..fda9627b862 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -9,8 +9,10 @@ * proxyStillLiveAfterStop after a Part A stopped; the receipt is completed only after the * restore ran, and kept for recovery otherwise. * - uninstall / remove: Part A before the task is deleted (a looping wrapper used to - * outlive its task, #764), refusing on anything but stopped/absent, then the tracked - * proxy stop and Part B. + * outlive its task, #764), refusing on anything but stopped/absent, then a receipt that + * carries Part A's record before the tracked proxy stop (Part A leaves the task + * registered, so a receipt-less POST answers 409 respawnable_service), then Part B. The + * receipt is kept until the caller's restore succeeds. */ import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-teardown"; import { readRuntimePort } from "../config/process-state"; @@ -111,23 +113,47 @@ export function completeWindowsServiceStopReceipt(nonce: string | undefined, cle if (!clear(nonce)) console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); } -/** ocx service uninstall / remove: stop and confirm before the task is deleted. */ -export async function windowsServiceRemovalStop(io: Pick = {}): Promise { +/** + * ocx service uninstall / remove: stop and confirm before the task is deleted. + * + * Part A leaves the scheduler task registered, so a receipt-less POST still answers + * `respawnable_service` (409) and the uninstall then refuses with the proxy left running. + * The receipt is therefore claimed with Part A's record before the child stop, exactly as + * `ocx service stop` does, and kept until the caller's restore succeeds. + */ +export async function windowsServiceRemovalStop( + io: Pick = {}, +): Promise<{ ok: boolean; nonce?: string }> { const port = windowsServiceStopPort(); const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); if (partA.outcome !== "stopped" && partA.outcome !== "absent") { console.error("❌ Service uninstall refused: the Windows service could not be proven stopped (" + partA.outcome + "), so the task was not deleted."); console.error(" " + describeWindowsPartA(partA)); - return false; + return { ok: false }; } - await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ skipPortWait: partA.outcome === "stopped" }); - if (partA.outcome !== "stopped") return true; + let nonce: string | undefined; + if ((io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + const { endpoint, source } = receiptEndpoint(); + try { + nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)).nonce; + } catch (error) { + console.error("❌ Service uninstall could not record its deferred-teardown receipt: " + (error instanceof Error ? error.message : String(error))); + console.error(" The proxy was left running: stopping it without the receipt would let it tear down client config while the task can still relaunch it."); + return { ok: false }; + } + } + await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + skipPortWait: partA.outcome === "stopped", + }); + if (partA.outcome !== "stopped") return withNonce(nonce); const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); - if (partB.outcome === "freed") return true; + if (partB.outcome === "freed") return withNonce(nonce); if (partB.outcome === "port-held-by-foreign") { console.warn("⚠️ " + describeWindowsPartB(partB)); - return true; + return withNonce(nonce); } console.error("❌ Service uninstall refused: " + describeWindowsPartB(partB) + " The task was not deleted."); - return false; + console.error(" The teardown receipt is kept so a later stop can finish it."); + return { ok: false }; } diff --git a/src/service/cli.ts b/src/service/cli.ts index 19df439f594..a891f052655 100644 --- a/src/service/cli.ts +++ b/src/service/cli.ts @@ -477,9 +477,14 @@ export async function serviceCommand(...args: (string | undefined)[]): Promise { for (const outcome of ALL_PART_A_FAILURES) { let child = 0; setWindowsServiceStopHooksForTests({ partA: async () => partAResult(outcome) }); - expect(await windowsServiceRemovalStop({ stopTracked: async () => { child++; return "stopped"; } })).toBe(false); + expect((await windowsServiceRemovalStop({ stopTracked: async () => { child++; return "stopped"; } })).ok).toBe(false); expect(child).toBe(0); } for (const [outcome, ok] of [["freed", true], ["port-held-by-foreign", true], ["survived", false], ["unknown", false]] as const) { setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult(outcome) }); - expect(await windowsServiceRemovalStop({ stopTracked: async () => "stopped" })).toBe(ok); + expect((await windowsServiceRemovalStop({ stopTracked: async () => "stopped" })).ok).toBe(ok); } }); + + test("claims a receipt carrying the Part A record before the POST, so the route does not answer 409", async () => { + // Deleting the registration leaves the scheduler task's receipt-less POST answering + // respawnable_service, which fails the uninstall. `ocx service stop` already claims; + // removal has to do the same, and keep the receipt until the restore succeeds. + const steps: string[] = []; + let claimed: unknown; + setWindowsServiceStopHooksForTests({ + partA: async () => { steps.push("partA"); return partAResult("stopped"); }, + partB: async () => { steps.push("partB"); return partBResult("freed"); }, + }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, + claim: (_endpoint, _source, _owner, _handover, partA) => { steps.push("claim"); claimed = partA; return { nonce: "r1" } as never; }, + stopTracked: async io => { steps.push("child:" + io?.deferSharedTeardownNonce + ":" + io?.skipPortWait); return "stopped"; }, + }); + expect(steps).toEqual(["partA", "claim", "child:r1:true", "partB"]); + expect(claimed).toEqual(RECORD); + expect(result).toEqual({ ok: true, nonce: "r1" }); + // Keep until the restore runs: only the caller's completion clears it. + const cleared: string[] = []; + completeWindowsServiceStopReceipt(result.nonce, nonce => { cleared.push(nonce); return true; }); + expect(cleared).toEqual(["r1"]); + }); + + test("a claim failure sends no undeferred child stop", async () => { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped") }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, + claim: () => { throw new Error("disk full"); }, + stopTracked: async () => { steps.push("child"); return "stopped"; }, + }); + expect(result.ok).toBe(false); + expect(steps).toEqual([]); + }); + + test("a failed Part B keeps the receipt for recovery", async () => { + const steps: string[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("survived") }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, claim: () => ({ nonce: "r2" }) as never, + stopTracked: async io => { steps.push("child:" + io?.deferSharedTeardownNonce); return "stopped"; }, + }); + expect(result).toEqual({ ok: false }); + expect(steps).toEqual(["child:r2"]); + }); + + test("the not-owned teardown path is unchanged: no receipt, an ordinary child stop", async () => { + const children: unknown[] = []; + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + claim: () => { throw new Error("must not claim"); }, + stopTracked: async io => { children.push(io); return "stopped"; }, + }); + expect(result).toEqual({ ok: true }); + expect(children).toEqual([{ skipPortWait: true }]); + }); }); describe("guarded (consent) path", () => { From 968290c5656444e3591be58516674eb76748915c Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 16:15:27 +0900 Subject: [PATCH 11/53] fix(service): a not-running task promotes its survivors only once it is proven ours The guarded inspection read "the task is not running" as "this home's task is not running" and bound every surviving wrapper of this home as a terminable bound-stray. The state word says nothing about ownership: a same-named foreign registration would have made hand-started wrappers terminable. Read the definition and require it to be a recognised OpenCodex registration before the promotion; an unreadable definition stays unknown with evidence-unreadable and a foreign one stays unknown, exactly as the running path already decided. --- src/service/windows-guarded-manager.ts | 35 ++++++++++++------- tests/windows/windows-guarded-manager.test.ts | 18 ++++++++++ tests/windows/windows-stop-callers.test.ts | 8 ++++- 3 files changed, 48 insertions(+), 13 deletions(-) diff --git a/src/service/windows-guarded-manager.ts b/src/service/windows-guarded-manager.ts index 98edb2f8a45..91f60447f7b 100644 --- a/src/service/windows-guarded-manager.ts +++ b/src/service/windows-guarded-manager.ts @@ -424,18 +424,12 @@ function inspectWindowsSchedulerManager( if (unreadableAncestorWrapperPids(approvedPid, processes).length > 0) { return unknown("a possible scheduler wrapper has an unreadable command line", "evidence-unreadable"); } - // A registration that cannot run anything owes only the stray-wrapper check. - // That covers BOTH a not-running task and a task proven deleted between the - // caller's presence probe and this inspection: in either case nothing - // registered can respawn the proxy, while a surviving wrapper still can — - // schtasks /end would report success on the inert task and leave it alive. - // A task proven deleted keeps today's verdict; only a registered, not-running task can - // own the survivors it left behind. - const inertVerdict = (registered = false): GuardedManagerTarget => strays.length === 0 + // A registration that cannot run anything owes only the stray-wrapper check: nothing + // registered can respawn the proxy, while a surviving wrapper still can — schtasks /end + // would report success on the inert task and leave it alive. + const inertVerdict = (): GuardedManagerTarget => strays.length === 0 ? { kind: "absent" } - : registered - ? bindStrayWrappers(approvedPid, processes, strays, io) - : unknown("a surviving scheduler wrapper could not be tied to the registered task", "unbound-wrapper"); + : unknown("a surviving scheduler wrapper could not be tied to the registered task", "unbound-wrapper"); const state = io.winTaskState(); if (state === "unknown") { // Missing vs unreadable: the state query also fails for a task deleted @@ -445,7 +439,24 @@ function inspectWindowsSchedulerManager( ? inertVerdict() : unknown("the registered task's running state could not be proven"); } - if (state === "not-running") return inertVerdict(true); + if (state === "not-running") { + // Only a registration that is provably this installation's definition may promote its + // survivors to bound-stray (#7771). The state word alone says the task is inert; it does + // not say the task is ours — a same-named foreign task would otherwise make this home's + // hand-started wrappers terminable. Reading the definition costs nothing when nothing + // survived, so a clean inert task keeps the absent verdict without a read. + if (strays.length === 0) return { kind: "absent" }; + let xml: string; + try { + xml = io.winTaskXml(); + } catch { + return unknown("the Task Scheduler registration of the surviving wrappers could not be read", "evidence-unreadable"); + } + if (!io.winRegistrationOurs(xml, io.winTaskUserIds())) { + return unknown("the registered task is not a recognized OpenCodex definition"); + } + return bindStrayWrappers(approvedPid, processes, strays, io); + } // Only the running path needs the registered definition: an unreadable // registration cannot prove the task is ours, but it must not hold an // otherwise-provable inert verdict unknown. diff --git a/tests/windows/windows-guarded-manager.test.ts b/tests/windows/windows-guarded-manager.test.ts index 982746b5bba..8093257a7eb 100644 --- a/tests/windows/windows-guarded-manager.test.ts +++ b/tests/windows/windows-guarded-manager.test.ts @@ -242,6 +242,24 @@ describe("windows guarded manager target", () => { expect(target.kind).toBe("unknown"); }); + test("a not-running task only promotes its survivors to bound-stray when the registration is provably ours", () => { + // #7771: the not-running state alone does not make the registration this + // installation's. The definition has to be read and recognised before its + // surviving wrappers may be bound — otherwise any foreign task of the same + // name would turn this home's stray wrappers into terminable ones. + const inertTask = { winTaskState: () => "not-running" as const, winProcs: () => SUPERVISED }; + expect(inspectGuardedManagerTarget(42, 10100, schedulerDeps({ + win: { ...inertTask, winRegistrationOurs: () => false }, + }))).toMatchObject({ kind: "unknown" }); + expect(inspectGuardedManagerTarget(42, 10100, schedulerDeps({ + win: { ...inertTask, winTaskXml: () => { throw new Error("denied"); } }, + }))).toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + // Provably ours: the binding rule decides as before. These fixtures carry no creation + // times, so it stops at the unreadable-evidence refusal rather than the definition one. + expect(inspectGuardedManagerTarget(42, 10100, schedulerDeps({ win: inertTask }))) + .toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + }); + test("two registered managers stay unknown", () => { const target = inspectGuardedManagerTarget(42, 10100, schedulerDeps({ winsw: winswStarted })); diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index 325520b7842..5ece379255e 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -327,6 +327,9 @@ describe("guarded inspection: the binding rule decides bound-stray", () => { scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", win: { winProcs: () => processes as WindowsStopProcessEntry[], winTaskState: () => "not-running", + // #7771: a not-running task promotes its survivors only after the definition is read + // and recognised as this installation's. + winTaskXml: () => "xml", winRegistrationOurs: () => true, winSchedulePid: () => SCHEDULE_PID, winScriptPath: () => CMD, winLauncherPath: () => VBS, }, }); @@ -393,7 +396,10 @@ describe("the guarded bind uses the evidence snapshot (native creation times), n setWindowsServiceStopHooksForTests({ snapshot: async () => evidence(), schedulePid: async () => SCHEDULE_PID, liveDeps: () => fake.deps }); const target = await inspectGuardedTarget(42, 10100, (pid, port, deps) => inspectGuardedManagerTarget(pid, port, { ...deps, platform: "win32", verifyPid: p => p, scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", - win: { ...deps?.win, winTaskState: () => "not-running", winScriptPath: () => CMD, winLauncherPath: () => VBS }, + win: { + ...deps?.win, winTaskState: () => "not-running", winScriptPath: () => CMD, winLauncherPath: () => VBS, + winTaskXml: () => "xml", winRegistrationOurs: () => true, + }, })); expect(target.kind).toBe("bound-stray"); const outcome = await stopGuardedManager({ approvedPort: 10100, receiptNonce: () => undefined, inheritedReceipts: 0 }); From 7c2ec3ef09578f890960bcd546349959cc325236 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 17:22:07 +0900 Subject: [PATCH 12/53] fix(service): an unreadable listener identity is unknown, not foreign verifyPidIdentity answers null both for "this is not an ocx process" and for "the probe could not read it", and identifyWindowsStopListener mapped every null to foreign. A Part B closing sample that could not read an elevated or other-account listener therefore reported portHeldByForeign, and the caller would restore shared config with the proxy possibly still alive. inspectOcxStartProcessIdentity already answers the tri-state; only a positive "mismatch" is foreign now, and "unverifiable" or a throwing probe stays unknown so nothing further is changed. Refs #7771 --- src/service/windows-checked-stop-live.ts | 21 +++++++++++++++---- .../windows/windows-checked-stop-live.test.ts | 14 ++++++++++--- 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts index 00bb4019eb9..b105752a5ba 100644 --- a/src/service/windows-checked-stop-live.ts +++ b/src/service/windows-checked-stop-live.ts @@ -22,7 +22,7 @@ import { execFile } from "node:child_process"; import { existsSync } from "node:fs"; import { join } from "node:path"; -import { readPid, verifyPidIdentity } from "../config/process-state"; +import { inspectOcxStartProcessIdentity, readPid, verifyPidIdentity, type OcxStartIdentityObservation } from "../config/process-state"; import { isProcessAlive } from "../lib/process-control"; import { isTestHomeGuardArmed } from "../lib/test-home-guard"; import { resolveTrustedWindowsPowerShellExe, resolveTrustedWindowsSchtasksExe, resolveTrustedWindowsSystemDirectory, toWindowsSchtasksError } from "../lib/windows-elevation"; @@ -296,15 +296,28 @@ export function sampleWindowsStopPort( return { state: "listening", ownerPid: live[0] ?? pids[0] ?? null }; } -/** ours = an opencodex process (the respawned proxy); exited = its owner is gone. */ +/** + * ours = an opencodex process (the respawned proxy); exited = its owner is gone. + * + * Tri-state (#7771): only a positive "mismatch" is foreign. `verifyPidIdentity` collapses + * "not an ocx process" and "the probe could not answer" into the same null, and a probe that + * cannot read another account's or an elevated process's command line is not evidence that + * another program holds the port — Part B would then report portHeldByForeign and let the + * caller restore shared config with the proxy possibly still alive. Unreadable stays unknown. + */ export function identifyWindowsStopListener( ownerPid: number | null, - io: { alive?: (pid: number) => boolean; verify?: (pid: number) => number | null } = {}, + io: { + alive?: (pid: number) => boolean; + identity?: (pid: number) => OcxStartIdentityObservation; + } = {}, ): WindowsListenerIdentity { if (ownerPid === null) return "unknown"; if (!(io.alive ?? isProcessAlive)(ownerPid)) return "exited"; try { - return (io.verify ?? verifyPidIdentity)(ownerPid) === ownerPid ? "ours" : "foreign"; + const observed = (io.identity ?? inspectOcxStartProcessIdentity)(ownerPid); + if (observed === "match") return "ours"; + return observed === "mismatch" ? "foreign" : "unknown"; } catch { return "unknown"; } diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts index 4572f5cd17a..0d58c3bed21 100644 --- a/tests/windows/windows-checked-stop-live.test.ts +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -111,10 +111,18 @@ describe("Part B port sampler and listener identity", () => { }); test("ours, foreign, exited, unknown", () => { - expect(identifyWindowsStopListener(42, { alive: () => true, verify: pid => pid })).toBe("ours"); - expect(identifyWindowsStopListener(42, { alive: () => true, verify: () => null })).toBe("foreign"); - expect(identifyWindowsStopListener(42, { alive: () => false })).toBe("exited"); + expect(identifyWindowsStopListener(42, { alive: () => true, identity: () => "match" })).toBe("ours"); + expect(identifyWindowsStopListener(42, { alive: () => true, identity: () => "mismatch" })).toBe("foreign"); + // Unreadable is not foreign (#7771): a probe that cannot answer must not let the caller + // conclude that another program holds the port. + expect(identifyWindowsStopListener(42, { alive: () => true, identity: () => "unverifiable" })).toBe("unknown"); + expect(identifyWindowsStopListener(42, { alive: () => false, identity: () => "match" })).toBe("exited"); expect(identifyWindowsStopListener(null)).toBe("unknown"); + // A throwing probe is unreadable too, never foreign. + expect(identifyWindowsStopListener(42, { + alive: () => true, + identity: () => { throw new Error("denied"); }, + })).toBe("unknown"); }); }); From d2fc61e37e652c00b4fc61504ae94cd626a54b0a Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 17:26:40 +0900 Subject: [PATCH 13/53] fix(service): the Part B port sample honors its timeout sampleWindowsStopPort ignored the timeoutMs Part B passed it and called the synchronous scanListenPids, so the closing sample could stall the stop for the helper's own budget and never yielded to Part B's deadline. The sample is async now: the production scan runs netstat -ano from the trusted System32 path through execFile with the caller's timeout and SIGKILL, so a helper that overruns is terminated rather than abandoned. A scan that does not answer in time is an unknown sample, and the timeout is floored with a value below 1 ms sampling nothing, because execFile reads 0 as unlimited. Refs #7771 --- src/service/windows-checked-stop-live.ts | 96 +++++++++++++++++-- .../windows/windows-checked-stop-live.test.ts | 44 +++++++-- 2 files changed, 126 insertions(+), 14 deletions(-) diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts index b105752a5ba..edf3b00e570 100644 --- a/src/service/windows-checked-stop-live.ts +++ b/src/service/windows-checked-stop-live.ts @@ -28,7 +28,7 @@ import { isTestHomeGuardArmed } from "../lib/test-home-guard"; import { resolveTrustedWindowsPowerShellExe, resolveTrustedWindowsSchtasksExe, resolveTrustedWindowsSystemDirectory, toWindowsSchtasksError } from "../lib/windows-elevation"; import { WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; import { decodeWindowsTextBytes } from "../lib/windows-text"; -import { scanListenPids } from "../server/port-reclaim"; +import { parseListenPidsFromNetstat, type ListenPidScan } from "../server/port-reclaim"; import { dropWindowsTcpRowsForLocalPort } from "../server/windows-tcp-drop"; import { TASK, windowsLauncherVbsPath, windowsServiceScriptPath } from "./state"; import { ancestorWrapperPids } from "./windows-guarded-manager"; @@ -278,13 +278,70 @@ export function windowsCheckedStopLiveDeps(options: WindowsCheckedStopLiveOption }; } -/** One port sample. A listener whose owner has exited is cleared like reclaimListenPort does. */ -export function sampleWindowsStopPort( +/** One listener scan, bounded by the caller's timeout; a scan that misses it is unreadable. */ +export type WindowsStopPortScan = (port: number, timeoutMs: number) => ListenPidScan | Promise; + +const WINDOWS_STOP_SCAN_MAX_BUFFER = 4 * 1024 * 1024; + +/** + * The production listener scan: netstat -ano through execFile with the caller's timeout. + * + * `netstat -ano` is asked directly rather than through `cmd /c chcp 437`: the locale-safe + * wildcard-foreign rule in parseListenEntriesFromNetstat already recognizes a listen row, and + * running the program itself means the timeout terminates the only helper spawned. The table + * is read from the trusted System32 path — SystemRoot is caller-controlled. + */ +export function scanWindowsStopListenPids(port: number, timeoutMs: number): Promise { + const whole = wholeTimeoutMs(timeoutMs); + if (whole === null) return Promise.resolve({ ok: false, error: "no positive timeout to sample with" }); + let netstat: string; + try { + netstat = join(resolveTrustedWindowsSystemDirectory(), "netstat.exe"); + } catch (error) { + return Promise.resolve({ ok: false, error: error instanceof Error ? error.message : String(error) }); + } + return new Promise(resolve => { + // execFile's timeout terminates the helper; the callback then answers unreadable. + execFile(netstat, ["-ano", "-p", "tcp"], { + encoding: "buffer", + timeout: whole, + killSignal: "SIGKILL", + windowsHide: true, + maxBuffer: WINDOWS_STOP_SCAN_MAX_BUFFER, + }, (error, stdout) => { + if (error) { + resolve({ ok: false, error: error.message }); + return; + } + const text = typeof stdout === "string" ? stdout : decodeWindowsTextBytes(stdout); + resolve({ ok: true, pids: parseListenPidsFromNetstat(text, port) }); + }); + }); +} + +/** + * One port sample, capped by `timeoutMs`. A listener whose owner has exited is cleared like + * reclaimListenPort does. + * + * A scan that does not answer within the budget is "unknown": Part B's closing sample is + * bounded (030 "Time budget"), and a synchronous scan both ignored that cap and could stall + * the stop for the helper's own 5 s. The timeout is floored and a value below 1 ms samples + * nothing, because execFile reads 0 as unlimited — the same rule as the command runner. + */ +export async function sampleWindowsStopPort( port: number, - io: { scan?: typeof scanListenPids; alive?: (pid: number) => boolean; drop?: (port: number) => unknown; selfPid?: number } = {}, -): WindowsPortSample { - const scan = (io.scan ?? scanListenPids)(port); - if (!scan.ok) return { state: "unknown" }; + timeoutMs: number, + io: { + scan?: WindowsStopPortScan; + alive?: (pid: number) => boolean; + drop?: (port: number) => unknown; + selfPid?: number; + } = {}, +): Promise { + const whole = wholeTimeoutMs(timeoutMs); + if (whole === null) return { state: "unknown" }; + const scan = await runBoundedScan(port, whole, io.scan ?? scanWindowsStopListenPids); + if (scan === null || !scan.ok) return { state: "unknown" }; const pids = scan.pids.filter(pid => pid !== (io.selfPid ?? process.pid)); if (pids.length === 0) return { state: "free" }; const alive = io.alive ?? isProcessAlive; @@ -296,6 +353,29 @@ export function sampleWindowsStopPort( return { state: "listening", ownerPid: live[0] ?? pids[0] ?? null }; } +/** Resolve null once the budget is spent; a scan that never answers cannot hold the stop. */ +function runBoundedScan(port: number, timeoutMs: number, scan: WindowsStopPortScan): Promise { + return new Promise(resolve => { + let settled = false; + const finish = (result: ListenPidScan | null): void => { + if (settled) return; + settled = true; + clearTimeout(timer); + resolve(result); + }; + const timer = setTimeout(() => finish(null), timeoutMs); + timer.unref?.(); + let pending: ListenPidScan | Promise; + try { + pending = scan(port, timeoutMs); + } catch { + finish(null); + return; + } + Promise.resolve(pending).then(finish, () => finish(null)); + }); +} + /** * ours = an opencodex process (the respawned proxy); exited = its owner is gone. * @@ -332,7 +412,7 @@ export function windowsStopConfirmLiveDeps( port, remainingMs, canonicalPaths: windowsStopCanonicalPaths(), - samplePort: samplePort => sampleWindowsStopPort(samplePort), + samplePort: (samplePort, timeoutMs) => sampleWindowsStopPort(samplePort, timeoutMs), identifyListener: (_port, ownerPid) => identifyWindowsStopListener(ownerPid), enumerate: timeoutMs => (options.snapshot ?? windowsStopSnapshot)(timeoutMs), }; diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts index 0d58c3bed21..75576917ae4 100644 --- a/tests/windows/windows-checked-stop-live.test.ts +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -97,17 +97,49 @@ describe("capped command answers", () => { }); describe("Part B port sampler and listener identity", () => { - test("free, listening, unreadable, and an exited owner's row is cleared", () => { - expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [] }) })).toEqual({ state: "free" }); - expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: false }) })).toEqual({ state: "unknown" }); - expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [77] }), alive: () => true })) + test("free, listening, unreadable, and an exited owner's row is cleared", async () => { + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [] }) })).toEqual({ state: "free" }); + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: false }) })).toEqual({ state: "unknown" }); + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => true })) .toEqual({ state: "listening", ownerPid: 77 }); const drops: number[] = []; - expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [77] }), alive: () => false, drop: port => drops.push(port) })) + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => false, drop: port => drops.push(port) })) .toEqual({ state: "listening", ownerPid: 77 }); expect(drops).toEqual([10100]); // The stopping process's own socket is not a listener on the stopped port. - expect(sampleWindowsStopPort(10100, { scan: () => ({ ok: true, pids: [5] }), selfPid: 5 })).toEqual({ state: "free" }); + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [5] }), selfPid: 5 })).toEqual({ state: "free" }); + }); + + test("the sample honors its timeout: an unfinished scan is unknown, never a hung part", async () => { + const seen: number[] = []; + const started = Date.now(); + // The scan never answers; the sample must give up on the timeout. + const sample = await sampleWindowsStopPort(10100, 60, { + scan: (_port, timeoutMs) => { + seen.push(timeoutMs); + return new Promise(() => undefined); + }, + }); + expect(sample).toEqual({ state: "unknown" }); + expect(seen).toEqual([60]); + expect(Date.now() - started).toBeLessThan(2_000); + }); + + test("a sub-millisecond timeout samples nothing: execFile reads 0 as unlimited", async () => { + let scans = 0; + expect(await sampleWindowsStopPort(10100, 0.4, { scan: () => { scans++; return { ok: true, pids: [] }; } })) + .toEqual({ state: "unknown" }); + expect(await sampleWindowsStopPort(10100, 0, { scan: () => { scans++; return { ok: true, pids: [] }; } })) + .toEqual({ state: "unknown" }); + expect(scans).toBe(0); + }); + + test("the scan sees a floored whole-millisecond budget", async () => { + let seen = -1; + expect(await sampleWindowsStopPort(10100, 499.9, { + scan: (_port, timeoutMs) => { seen = timeoutMs; return { ok: true, pids: [] }; }, + })).toEqual({ state: "free" }); + expect(seen).toBe(499); }); test("ours, foreign, exited, unknown", () => { From 34c3c38a94d3daf4ea320f115585c90499008c5b Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 17:48:23 +0900 Subject: [PATCH 14/53] fix(service): orphan discovery takes the cap its caller admitted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit stopTrackedProxyIfRunning hard-coded a 7 s deadline for the identity-checked orphan discovery that follows the tracked stop, and applied it on every platform and to every caller — including one whose Part A and Part B were already paid for out of a checked Windows stop's admitted budget, which reserves 3 s for exactly this step. The stop could then run past the deadline its own admission had just proven. The helper takes an orphanDiscoveryCapMs and keeps 7 s when none is passed, so macOS and Linux stops are unchanged. The checked Windows callers — both service command stops and prepareServiceInstall — pass WINDOWS_STOP_CAPS.orphanDiscovery, and the plain ocx stop path reads the same constant instead of restating 3000. Refs #7771 --- src/cli/stop-windows-checked.ts | 3 +- src/service/cli-windows-stop.ts | 3 + src/service/orchestration.ts | 28 ++++++++- .../service/service-stop-verification.test.ts | 60 ++++++++++++++++++- tests/windows/windows-stop-callers.test.ts | 8 ++- 5 files changed, 95 insertions(+), 7 deletions(-) diff --git a/src/cli/stop-windows-checked.ts b/src/cli/stop-windows-checked.ts index db61631f033..4f66baf3634 100644 --- a/src/cli/stop-windows-checked.ts +++ b/src/cli/stop-windows-checked.ts @@ -34,11 +34,12 @@ import { windowsStopReason, windowsStopRemainingMs, } from "../service/windows-service-stop"; +import { WINDOWS_STOP_CAPS } from "../service/windows-stop-budget"; import type { StopRunRecord } from "./stop-report"; const EVIDENCE_SNAPSHOT_TIMEOUT_MS = 5_000; const SCHEDULE_PID_TIMEOUT_MS = 2_000; -const ORPHAN_DISCOVERY_CAP_MS = 3_000; +const ORPHAN_DISCOVERY_CAP_MS = WINDOWS_STOP_CAPS.orphanDiscovery; interface WindowsStopState { platform: NodeJS.Platform; diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index fda9627b862..d74573d2cb9 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -29,6 +29,7 @@ import { windowsStopAdmission, windowsStopRemainingMs, } from "./windows-service-stop"; +import { WINDOWS_STOP_CAPS } from "./windows-stop-budget"; export interface WindowsServiceStopCommandIo { ownsSharedTeardown?: () => boolean; @@ -88,6 +89,7 @@ export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopComm await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, }); if (partA.outcome === "absent") { const survivor = await (io.survivor ?? proxyStillLiveAfterStop)(); @@ -145,6 +147,7 @@ export async function windowsServiceRemovalStop( await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, }); if (partA.outcome !== "stopped") return withNonce(nonce); const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index 57979cd841a..ad326826e55 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -17,6 +17,7 @@ import type { ServiceBackend } from "./state"; import { unitPath, isSystemd, installSystemd, startSystemd, stopSystemd, statusSystemd, uninstallSystemd, systemdServiceInstallCleanupOps } from "./systemd"; import { writeWindowsSchedulerAssets, stageWindowsSchedulerRegistrationXml, removeWindowsSchedulerRegistrationStage, registerFreshWindowsSchedulerTask, recordWindowsSchedulerOwnership, removeNativeWindowsServiceForScheduler, commitWindowsInstall, commitWindowsNativeInstall, type WindowsServiceInstallDeps, startWindows, isWindowsSchedulerEndBenign, stopWindows, statusWindows, statusWindowsXml, uninstallWindows } from "./windows-ops"; import type { WindowsCheckedStopResult } from "./windows-checked-stop"; +import { WINDOWS_STOP_CAPS } from "./windows-stop-budget"; import type { WindowsStopConfirmResult } from "./windows-stop-confirm"; import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; import { schtasks, probeWindowsSchedulerTask, rollbackWindowsSchedulerTaskOwnedByAttempt, settleDelay } from "./windows-scheduler"; @@ -193,12 +194,29 @@ export async function proxyStillLiveAfterStop(deps: { } } +/** + * How long orphan discovery may take when the caller does not name a cap. + * + * Seven seconds is the legacy multi-candidate window. It stays the default so macOS and + * Linux stops behave exactly as before; a Windows caller that admitted a budget + * (030 "Time budget") passes its own 3 s cap instead. + */ +export const DEFAULT_ORPHAN_DISCOVERY_MS = 7_000; + /** How the tracked proxy is stopped: the deferral receipt and the Windows port wait. */ export interface TrackedProxyStopIo { /** ocx service stop on Windows: the claimed receipt, so the proxy defers its teardown. */ deferSharedTeardownNonce?: string; /** Part B follows and owns the port wait (030), so stopProxy skips its own. */ skipPortWait?: boolean; + /** + * Cap on the identity-checked orphan discovery that follows the tracked stop. + * + * The checked Windows stop admits `WINDOWS_STOP_CAPS.orphanDiscovery` (3 s) for this step + * (030 "Time budget"), and Part B runs inside the same deadline: discovery that spent the + * legacy 7 s would overrun the budget admission had just proven. Omitted keeps 7 s. + */ + orphanDiscoveryCapMs?: number; } async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise { @@ -216,10 +234,11 @@ async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise stopTrackedProxyIfRunning({ skipPortWait: partA.outcome === "stopped" })))(); + await (deps.stopTrackedProxy ?? (() => stopTrackedProxyIfRunning({ + skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, + })))(); if (partA.outcome === "stopped") { const partB = await (deps.windowsPartB ?? confirmWindowsServiceStop)({ port: windowsServiceStopPort(), diff --git a/tests/service/service-stop-verification.test.ts b/tests/service/service-stop-verification.test.ts index 5f228b9e2ad..f99da8cbe29 100644 --- a/tests/service/service-stop-verification.test.ts +++ b/tests/service/service-stop-verification.test.ts @@ -1,5 +1,7 @@ -import { describe, expect, test } from "bun:test"; +import { describe, expect, spyOn, test } from "bun:test"; +import * as proxyLiveness from "../../src/server/proxy-liveness"; import { proxyStillLiveAfterStop } from "../../src/service"; +import { WINDOWS_STOP_CAPS } from "../../src/service/windows-stop-budget"; /** * #764: `ocx service stop` reported success while the proxy kept running, and native Codex was @@ -126,3 +128,59 @@ describe("service stop verification (#764)", () => { } }); }); + +/** + * #7771: the tracked-proxy stop's orphan discovery is part of the checked Windows stop's + * admitted budget (030 "Time budget"), which reserves 3 s for it. The helper used to + * hard-code 7 s for every platform and every caller, so a Windows stop that had already + * paid for Part A and Part B could still spend 7 s on discovery — past the deadline its + * admission had just proven — and a non-Windows stop must keep today's 7 s. + */ +describe("orphan discovery is capped by the caller, not hard-coded (#7771)", () => { + // The helper calls findLiveProxy directly, so the spy is the seam: it records the + // deadline the helper asked for and answers "nothing live" so no process is touched. + const discoveredBudget = async (io: Record): Promise => { + const { stopTrackedProxyForServiceCommand } = await import("../../src/service/orchestration"); + let seen = -1; + const started = Date.now(); + const spy = spyOn(proxyLiveness, "findLiveProxy").mockImplementation((async (options: { deadlineAt?: number }) => { + seen = (options.deadlineAt ?? 0) - started; + return null; + }) as never); + try { + await stopTrackedProxyForServiceCommand(io as never); + } finally { + spy.mockRestore(); + } + return seen; + }; + + test("the default keeps today's 7 s window", async () => { + const budget = await discoveredBudget({}); + expect(budget).toBeGreaterThanOrEqual(6_900); + expect(budget).toBeLessThanOrEqual(7_100); + }); + + test("a Windows checked caller's admitted 3 s is what discovery gets", async () => { + const budget = await discoveredBudget({ orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery }); + expect(WINDOWS_STOP_CAPS.orphanDiscovery).toBe(3_000); + expect(budget).toBeGreaterThanOrEqual(2_900); + expect(budget).toBeLessThanOrEqual(3_100); + }); + + test("an absent Part A still names the cap: discovery is the only port proof left", async () => { + const { windowsServiceRemovalStop, windowsServiceStopBeforeRestore } = await import("../../src/service/cli-windows-stop"); + const { setWindowsServiceStopHooksForTests } = await import("../../src/service/windows-service-stop"); + const { partAResult } = await import("../helpers/windows-stop-caller-fixtures"); + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + try { + for (const run of [windowsServiceStopBeforeRestore, windowsServiceRemovalStop]) { + const seen: unknown[] = []; + await run({ ownsSharedTeardown: () => false, stopTracked: async io => { seen.push(io); return "stopped"; } }); + expect(seen).toEqual([{ skipPortWait: false, orphanDiscoveryCapMs: 3_000 }]); + } + } finally { + setWindowsServiceStopHooksForTests(null); + } + }); +}); diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index 5ece379255e..f1c06dd72a3 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -141,7 +141,9 @@ describe("ocx service stop on Windows", () => { stopTracked: async io => { children.push(io); return "stopped"; }, }); expect(result).toEqual({ ok: true }); - expect(children).toEqual([{ skipPortWait: true }]); + // #7771: the stop also names the admitted discovery cap, so the helper cannot spend 7 s + // of a budget whose 3 s for this step was already proven at admission. + expect(children).toEqual([{ skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); }); test("Part B failure keeps the receipt (no restore); a foreign listener does not fail the stop", async () => { @@ -235,7 +237,9 @@ describe("ocx service uninstall / remove on Windows", () => { stopTracked: async io => { children.push(io); return "stopped"; }, }); expect(result).toEqual({ ok: true }); - expect(children).toEqual([{ skipPortWait: true }]); + // #7771: the stop also names the admitted discovery cap, so the helper cannot spend 7 s + // of a budget whose 3 s for this step was already proven at admission. + expect(children).toEqual([{ skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); }); }); From 205e9ad12a12e73378fd64bce047fbdd849d9cc6 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 17:57:28 +0900 Subject: [PATCH 15/53] fix(service): only ocx stop measures its deadline from command start --deadline-ms and OCX_STOP_DEADLINE_MS are a command-start budget: the operator's clock begins when they typed the command. Every other caller reaches the stop after work it cannot charge to the stop -- the scheduler registration and the UAC prompt -- so a 60 s budget measured from process start could already be spent, and admission refused `unverified-budget` a stop that had its whole budget left. An explicit command deadline still wins and stays absolute; with none recorded, the first deadline read in a stop phase arms the default 60 s and every later read inside that phase keeps it, so one deadline governs the whole stop instead of sliding forward. Refs #7771 --- src/service/windows-service-stop.ts | 12 ++++--- src/service/windows-stop-deadline.ts | 35 ++++++++++++++----- .../windows-service-stop-wiring.test.ts | 30 ++++++++++++++-- 3 files changed, 60 insertions(+), 17 deletions(-) diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts index 8a90035e52a..aadeb24d996 100644 --- a/src/service/windows-service-stop.ts +++ b/src/service/windows-service-stop.ts @@ -6,9 +6,10 @@ * remove, prepareServiceInstall and the updater. It owns: * * - the stop deadline (030 "Time budget"): --deadline-ms or OCX_STOP_DEADLINE_MS for - * ocx stop, 60 s otherwise, measured from this process's start (performance.now() counts - * from performance.timeOrigin, the process start, so the deadline is an absolute value on - * the Part A clock); + * ocx stop, measured from the command's start (performance.now() counts from + * performance.timeOrigin, the process start, so the deadline is an absolute value on the + * Part A clock); every other caller starts the default 60 s when its stop phase begins, + * because registration and the UAC prompt can come first; * - the admission inputs and the mapping of Part A onto ServiceStopOutcome; * - the Part A record a teardown receipt carries, so /api/stop can skip a second Part A; * - a hook seam so caller tests run hermetically (no schtasks, PowerShell or FFI). @@ -27,9 +28,10 @@ import { windowsStopDeadlineAt } from "./windows-stop-deadline"; export { parseStopDeadlineMs, resolveStopDeadlineMs, setStopDeadlineMs, STOP_DEADLINE_ENV, windowsStopDeadlineAt } from "./windows-stop-deadline"; -/** Time left on the stop deadline, measured on the Part A clock. */ +/** Time left on the stop deadline, measured on the Part A clock (one clock read). */ export function windowsStopRemainingMs(now: () => number = windowsStopClock.now): number { - return windowsStopDeadlineAt() - now(); + const at = now(); + return windowsStopDeadlineAt(at) - at; } export function configuredShutdownTimeoutMs(): number | undefined { diff --git a/src/service/windows-stop-deadline.ts b/src/service/windows-stop-deadline.ts index 73f76d52ae5..8ab7c367b46 100644 --- a/src/service/windows-stop-deadline.ts +++ b/src/service/windows-stop-deadline.ts @@ -3,9 +3,12 @@ * machinery because the CLI dispatcher, which loads on every command, records it. * * ocx stop reads --deadline-ms or OCX_STOP_DEADLINE_MS (the flag wins; an empty or - * invalid variable counts as absent); every other caller uses 60 s. The deadline is - * measured from this process's start: performance.now() counts from performance.timeOrigin, - * so N ms after process start is the clock value N on the Part A clock. + * invalid variable counts as absent) and records the resolved budget as a command-start + * deadline: performance.now() counts from performance.timeOrigin, so N ms of budget at + * command start is the Part A clock value N. Every other caller starts the default 60 s + * when its stop phase begins, not when the process did -- installFreshWindowsSchedulerSafely + * reaches the stop only after registration and the UAC prompt, so measuring from process + * start would refuse `unverified-budget` a stop that had its whole budget left. */ import { DEFAULT_WINDOWS_STOP_DEADLINE_MS } from "./windows-stop-budget"; @@ -26,14 +29,28 @@ export function resolveStopDeadlineMs(flagMs: number | null | undefined, env: No return parseStopDeadlineMs(env[STOP_DEADLINE_ENV]) ?? DEFAULT_WINDOWS_STOP_DEADLINE_MS; } -let stopDeadlineMs: number | null = null; +/** ocx stop's command-start deadline, or null while no command recorded one. */ +let commandDeadlineMs: number | null = null; +/** The unconfigured caller's deadline, armed once when its stop phase first asks. */ +let phaseDeadlineAt: number | null = null; -/** ocx stop records its resolved deadline here once its arguments are parsed. */ +/** + * ocx stop records its resolved deadline here once its arguments are parsed. A new value + * is an absolute command-start deadline; null clears it and re-arms the stop phase, so the + * next unconfigured caller measures its own 60 s from when it gets there. + */ export function setStopDeadlineMs(ms: number | null): void { - stopDeadlineMs = ms; + commandDeadlineMs = ms; + phaseDeadlineAt = null; } -/** The absolute deadline on the Part A clock (ms since process start). */ -export function windowsStopDeadlineAt(): number { - return stopDeadlineMs ?? DEFAULT_WINDOWS_STOP_DEADLINE_MS; +/** + * The absolute deadline on the Part A clock (ms since process start). An explicit command + * deadline wins; otherwise the first call in a stop phase arms the default 60 s and every + * later call inside that phase keeps it -- one deadline governs the whole stop. + */ +export function windowsStopDeadlineAt(now: number = performance.now()): number { + if (commandDeadlineMs !== null) return commandDeadlineMs; + if (phaseDeadlineAt === null) phaseDeadlineAt = now + DEFAULT_WINDOWS_STOP_DEADLINE_MS; + return phaseDeadlineAt; } diff --git a/tests/windows/windows-service-stop-wiring.test.ts b/tests/windows/windows-service-stop-wiring.test.ts index 454896127ef..947a5f03b15 100644 --- a/tests/windows/windows-service-stop-wiring.test.ts +++ b/tests/windows/windows-service-stop-wiring.test.ts @@ -20,6 +20,7 @@ import { UNDER_OWN_WRAPPER_DETAIL, windowsPartARecord, windowsStopAdmission, + windowsStopRemainingMs, resolveStopDeadlineMs, parseStopDeadlineMs, serviceStopOutcomeFromWindows, @@ -186,11 +187,34 @@ describe("the stop deadline", () => { expect(parseStopDeadlineMs("0")).toBeNull(); }); - test("measured from process start: the deadline is the Part A clock value", () => { + test("ocx stop's deadline is the command-start clock value it recorded", () => { setStopDeadlineMs(72_000); - expect(windowsStopDeadlineAt()).toBe(72_000); + expect(windowsStopDeadlineAt(0)).toBe(72_000); + expect(windowsStopDeadlineAt(42_000)).toBe(72_000); + // Command start, not stop phase: 30 s of the budget are already spent at t=30_000. + expect(windowsStopRemainingMs(() => 30_000)).toBe(42_000); + }); + + test("every other caller starts its 60 s when the stop begins, not when the process did", () => { + // installFreshWindowsSchedulerSafely reaches the stop only after registration and the UAC + // prompt, so by then this process can be minutes old. Measuring from process start made + // admission refuse `unverified-budget` for a stop that had its whole budget left. + setStopDeadlineMs(null); + const phaseStart = 600_000; + expect(windowsStopRemainingMs(() => phaseStart)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + // One deadline governs the whole stop: a later read inside it does not slide forward. + expect(windowsStopDeadlineAt(phaseStart + 20_000)).toBe(phaseStart + DEFAULT_WINDOWS_STOP_DEADLINE_MS); + expect(windowsStopRemainingMs(() => phaseStart + 20_000)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS - 20_000); + }); + + test("recording a new command deadline clears an armed stop phase", () => { + setStopDeadlineMs(null); + expect(windowsStopRemainingMs(() => 5_000)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); + setStopDeadlineMs(9_000); + expect(windowsStopDeadlineAt(5_000)).toBe(9_000); + // And un-setting it again re-arms at the next stop phase rather than keeping 9_000. setStopDeadlineMs(null); - expect(windowsStopDeadlineAt()).toBe(60_000); + expect(windowsStopRemainingMs(() => 5_000)).toBe(DEFAULT_WINDOWS_STOP_DEADLINE_MS); }); test("ocx stop parses --deadline-ms anywhere and keeps the approval grammar", () => { From f7219cf3d29a045edf68779d64bef32d641b7211 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 18:03:48 +0900 Subject: [PATCH 16/53] test(cli): uninstall's authorization expectations name their platform The base case asserted that a clean `stopped` authorizes shared teardown without naming a platform, so it inherited process.platform. On Windows that input is refused: a clean stop there is Part B of the checked stop and owes a verified respawn window. The test therefore passed on darwin/linux hosts and failed on Windows for a reason that has nothing to do with the code under test. Every expectation now passes its platform, and Windows gets the full matrix: `stopped` needs the window, `absent` does not, `stopped-respawnable` needs it everywhere, and a failed/unknown stop, a failed/absent removal or an unproven proxy refuses even with the window. Refs #7771, #3008, #764 --- tests/cli/uninstall.test.ts | 56 +++++++++++++++++++++++++------------ 1 file changed, 38 insertions(+), 18 deletions(-) diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index ac485a229c0..c3e3006366a 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -166,24 +166,44 @@ describe("uninstall gates shared teardown on a proven service stop", () => { serviceRemoval: "removed" as const, respawnWindowVerified: false, }; - expect(sharedTeardownAuthorized(base)).toBe(true); - expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent" })).toBe(true); - expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "absent" })).toBe(true); - // Removing the registration does not prove an already-running wrapper died; killing it - // is best-effort (#764), so the restart window has to be polled first. - expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable" })).toBe(false); - expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable", respawnWindowVerified: true })).toBe(true); - // A manager that refused to stop, or one we could not read, may still be running. - expect(sharedTeardownAuthorized({ ...base, serviceStop: "failed" })).toBe(false); - expect(sharedTeardownAuthorized({ ...base, serviceStop: "state-unknown" })).toBe(false); - // The step itself threw: we know nothing. - expect(sharedTeardownAuthorized({ ...base, serviceStop: null })).toBe(false); - // A proxy that could not be PROVEN down — a live orphan with no pid, or an endpoint - // that would not answer — blocks it. A findLiveProxy miss is not proof. - expect(sharedTeardownAuthorized({ ...base, proxyProvenDown: false })).toBe(false); - // A removal that failed used to look like absence on darwin and linux. - expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "failed" })).toBe(false); - expect(sharedTeardownAuthorized({ ...base, serviceRemoval: null })).toBe(false); + // The platform is explicit in every expectation below. It is an input to the rule, not + // an ambient fact: a clean Windows stop is Part B of the checked stop (#7771) and owes a + // verified restart window, while a clean stop elsewhere owes nothing more. Asserting on + // the default platform made this test's verdict depend on the machine it ran on. + for (const platform of ["linux", "darwin"] as const) { + expect(sharedTeardownAuthorized(base, platform)).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent" }, platform)).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "absent" }, platform)).toBe(true); + // Removing the registration does not prove an already-running wrapper died; killing it + // is best-effort (#764), so the restart window has to be polled first. + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable" }, platform)).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable", respawnWindowVerified: true }, platform)).toBe(true); + // A manager that refused to stop, or one we could not read, may still be running. + expect(sharedTeardownAuthorized({ ...base, serviceStop: "failed" }, platform)).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "state-unknown" }, platform)).toBe(false); + // The step itself threw: we know nothing. + expect(sharedTeardownAuthorized({ ...base, serviceStop: null }, platform)).toBe(false); + // A proxy that could not be PROVEN down — a live orphan with no pid, or an endpoint + // that would not answer — blocks it. A findLiveProxy miss is not proof. + expect(sharedTeardownAuthorized({ ...base, proxyProvenDown: false }, platform)).toBe(false); + // A removal that failed used to look like absence on darwin and linux. + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "failed" }, platform)).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: null }, platform)).toBe(false); + } + // On Windows a clean stop is only half the proof: Part B has to have shown the port free + // and no wrapper back. Without that window the same input is refused. + expect(sharedTeardownAuthorized(base, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, respawnWindowVerified: true }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent" }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "absent", respawnWindowVerified: true }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable" }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "stopped-respawnable", respawnWindowVerified: true }, "win32")).toBe(true); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "failed" }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: "state-unknown" }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceStop: null }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, proxyProvenDown: false, respawnWindowVerified: true }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: "failed", respawnWindowVerified: true }, "win32")).toBe(false); + expect(sharedTeardownAuthorized({ ...base, serviceRemoval: null, respawnWindowVerified: true }, "win32")).toBe(false); }); test("a removal failure is distinguishable from nothing being installed", async () => { From 9f14ef82c9b4f9b739378aabe5d223792c0ea3cd Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 18:11:49 +0900 Subject: [PATCH 17/53] docs(windows): record the wiring review's timing and evidence rules The contract doc and the 030/060 design records now say what the wiring actually does: the port wait is Part B's on both Windows paths, the deadline's origin depends on the caller, orphan discovery takes the cap its caller admitted, the listener identity is tri-state, the Part B port sample is bounded, service removal claims its receipt before the child stop, and a not-running task needs its registration proof before bound-stray. Refs #7771 --- .../261111_windows_service_stop/030_design.md | 6 +- .../060_implementation.md | 7 ++ structure/ops/windows-checked-stop.md | 71 ++++++++++++++----- 3 files changed, 64 insertions(+), 20 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 32b4b673276..e43547f430a 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -60,7 +60,7 @@ Part A: Windows only. The async `stopManager` change reaches every platform, but macOS and Linux keep today's timing and ignore the deadline. -`ocx stop` takes the deadline as an input, `--deadline-ms ` or the environment variable `OCX_STOP_DEADLINE_MS` (decimal milliseconds), measured from its own process start. Desktop passes it through the environment variable only, because an older CLI rejects an unknown flag and ignores an unknown variable, and does not set a value that would not be positive. It passes it on both paths: for the consented takeover, the remaining startup deadline minus a 5 s margin for the silence probe and `claim::run` that Desktop still runs under that deadline after the stop (desktop/src-tauri/src/startup.rs); for the exit drain, its deadline minus a 3 s spawn/parse margin. Without either (a terminal, a script, an older Desktop) the deadline is 60 s. The other callers (`ocx service stop`, `ocx uninstall`, service uninstall/remove, install/switch) are interactive and use 60 s; the updater uses its job budget. +`ocx stop` takes the deadline as an input, `--deadline-ms ` or the environment variable `OCX_STOP_DEADLINE_MS` (decimal milliseconds), measured from its own process start — that is the operator's budget, and it is the only caller for which process start and command start are the same instant. Every other caller starts the default 60 s when its stop phase begins (PR2, 060): it reaches the stop after work it cannot charge to the stop, so a budget measured from process start can already be spent. Desktop passes it through the environment variable only, because an older CLI rejects an unknown flag and ignores an unknown variable, and does not set a value that would not be positive. It passes it on both paths: for the consented takeover, the remaining startup deadline minus a 5 s margin for the silence probe and `claim::run` that Desktop still runs under that deadline after the stop (desktop/src-tauri/src/startup.rs); for the exit drain, its deadline minus a 3 s spawn/parse margin. Without either (a terminal, a script, an older Desktop) the deadline is 60 s. The other callers (`ocx service stop`, `ocx uninstall`, service uninstall/remove, install/switch) are interactive and use 60 s; the updater uses its job budget. Admission. Part A's registration probe (step 1) runs first, so a machine with nothing registered returns `absent` and is never refused for budget. Then, before step 2 and after every pre-step has already run (on the consented path: `runResolve`, `describeTrackedRuntimeTarget` and the first `inspectGuardedManagerTarget` inside `runApprovedStop`, then `revalidateManager`, each with its existing CIM 5 s timeout and `schtasks`; their actual elapsed time is what counts), the stop computes the worst case of everything still to come from the table below. If the remaining time cannot hold it, the stop changes NOTHING (no `/end`, no WinSW stop, no terminate, no POST) and reports `serviceStopReason: "unverified-budget"` with `manager-still-active` (guarded) or `failed` (snapshotless). Every step after admission has a cap, and a step without a natural bound today gets one (below); the caps sum to at most the remaining time, so the graceful exit wait is never shortened and the stop does not overrun. @@ -70,7 +70,7 @@ Worst case still to come, at admission: |---|---|---| | part A, Task Scheduler | guarded 6 s (`/end` 2 s, handle wait 3 s, manager state 1 s; the bind reuses `revalidateManager`'s enumeration); snapshotless 11 s (its own enumeration, 5 s, plus the same 6 s) | the registration probe already ran before admission | | part A, WinSW | 25 s: stop command 2 s + SCM poll `` + 2 s = 22 s + manager state 1 s | WinSW stops the proxy itself, so the POST and exit-wait rows below do not apply to it. The stray enumeration of step 1 does not apply either: it runs only when nothing is registered | -| orphan discovery (snapshotless, no PID file; also after WinSW stopped the proxy) | 3 s | `findLiveProxy` and `proveLiveProxyOwnedByHome`, capped at 3 s together; an overrun is treated as "no proven orphan" and refuses, as an unproven owner does today | +| orphan discovery (snapshotless, no PID file; also after WinSW stopped the proxy) | 3 s | `findLiveProxy` and `proveLiveProxyOwnedByHome`, capped at 3 s together; an overrun is treated as "no proven orphan" and refuses, as an unproven owner does today. The cap is threaded to the helper (`orphanDiscoveryCapMs`) so a checked Windows caller passes exactly the 3 s admission priced; the helper's own default stays the legacy 7 s for macOS and Linux | | POST answer | 10 s, always | never shortened. The receipt and its part A record are written after admission (the snapshotless receipt inside `stopWithDeferral`), so admission cannot know whether they will exist and always counts the full abort. With the record the route skips part A and answers at once; without it (the write failed, or an older CLI) the route keeps today's behavior inside the same 10 s | | child exit: graceful exit wait, then forced | `shutdownTimeoutMs` + 3 s + 10 s = 18 s by default | today's ladder, kept in sequence: the graceful exit wait (never shortened) runs after an answered POST; if the child has not exited, or the POST aborted, `stopProxy` forces (never after a refusal): `taskkill` with a new 5 s timeout (it has none today) plus today's `waitForExit` 5 s | | inherited-receipt probes | 3 s × N | N abandoned receipts, known at admission; each probe tries both loopback hosts at today's 1.5 s per host (`probeEndpointLiveness`) | @@ -83,7 +83,7 @@ Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 |---|---|---| | consented takeover (`--expect-*`) | remaining startup deadline − 5 s (a Windows approval adds 60 s) | yes in practice; otherwise refused before anything changes | | Desktop exit/restart drain (plain `ocx stop --json`) | today `runtime_stop::DEADLINE` − 3 s = 27 s | NO for a registered service. The same change raises Desktop's exit-drain deadline on Windows to 75 s (72 s passed as `OCX_STOP_DEADLINE_MS`). That holds the largest case (56.5 s) plus an 8 s allowance for what runs inside the deadline before admission (CLI start, the 2 s registration probe, and in the stray case the 5 s step-1 enumeration) plus two inherited receipts (3 × 2 = 6 s): 56.5 + 8 + 6 = 70.5 s ≤ 72 s; beyond that the stop is refused up front and the operator stops from a terminal. An older Desktop passes no flag, the CLI uses 60 s and can outlive Desktop's 30 s wait, which ends in `DrainVerdict::Failed`; today's stop can already outlive it (10 s POST + 8 s exit wait + 15 s port wait), so this is not new, and it is listed in 050 | -| terminal, scripts, other interactive callers | 60 s | yes | +| terminal, scripts, other interactive callers | 60 s, from the start of the stop phase (PR2; `ocx stop` measures from command start) | yes | | `/api/stop` route | receipt with a part A record: no part A; receipt without the record: today's service stop, now the async part A, capped at the route's 6 s (an overrun answers `service_state_unknown`); WinSW: today's in-process stop; stray of a deleted task, no receipt: 4 s sweep (its enumeration capped at 4 s instead of today's 5 s; an overrun answers `service_state_unknown`, which fails safe) + teardown before the answer | the parent's 10 s POST abort, never shortened | | updater | job budget | yes | diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 33807f98daa..9ea0519469b 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -69,6 +69,13 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **Not capped here.** The guarded path's own pre-steps (`runResolve`, the tracked-record check, the two inspections and their 5 s snapshot) run before admission and are measured, as 030 says. `proveLiveProxyOwnedByHome` is not capped separately; orphan discovery's `findLiveProxy` is capped at 3 s, and a missed orphan is caught by Part B. `revertSystemEnv` returns at once off macOS, so its 1 s cap holds on Windows without a change. `restoreSharedClientStateAfterStop` and `performStopTeardown` were not measured on a device; 040 keeps those measurements. - **The manager-state read.** `schtasks /query /fo CSV` first, `Get-ScheduledTask` within the same 1 s cap when the status word is localized. On a slow or non-English machine that can overrun and report `unknown`; the 040 device run decides whether the cap needs to grow. - **ARM64 ships strict** as recorded above; `windowsStopTerminate` enforces it whenever the native API is missing. +- **The deadline's origin depends on the caller.** 030 prices `ocx stop` from process start, which is right for an operator's command and wrong for every other caller: `installFreshWindowsSchedulerSafely` reaches the stop only after registration and the UAC prompt, so a 60 s budget measured from process start can already be spent and admission refuses `unverified-budget` a stop that has its whole budget left. `windows-stop-deadline.ts` therefore keeps an explicit command deadline (set by the dispatcher from `--deadline-ms`/`OCX_STOP_DEADLINE_MS`) separate from a phase deadline that the first read in a stop phase arms. One deadline governs a phase; a later read does not slide it forward, and setting or clearing the command deadline re-arms. +- **The port wait is skipped on both Windows paths.** `windowsPartBDue` is snapshotless-only because it also gates the legacy respawn re-poll, so keying `skipPortWait` to it left the guarded path and `ocx uninstall`'s two `stopProxy` calls running the legacy 15 s `waitForStoppedPort` on top of Part B. `windowsPartBOwnsPortWait` (any Windows Part A `stopped`) now drives `skipPortWait`; the respawn semantics stay snapshotless. +- **Service removal claims its receipt first.** Part A leaves the scheduler task registered, so `stopTrackedProxyForServiceCommand`'s receipt-less POST answered `respawnable_service` (409) and the removal refused. `windowsServiceRemovalStop` now claims a receipt carrying Part A's record before the child stop, keeps it until the shared restore succeeds, and sends no child stop when the claim fails — the same rule `ocx service stop` already had. +- **Orphan discovery takes the cap its caller admitted.** `stopTrackedProxyIfRunning` hardcoded 7 s, which is more than the 3 s admission priced, so discovery could overrun the budget admission had just proven and Part B's window with it. The helper takes `orphanDiscoveryCapMs`; the checked Windows callers pass `WINDOWS_STOP_CAPS.orphanDiscovery` and the default keeps macOS and Linux unchanged. +- **Listener identity is tri-state.** `verifyPidIdentity`'s null covers both "not an ocx process" and "the probe could not answer", and mapping it to `foreign` reported a listener we merely could not read as another program holding the port — `portHeldByForeign` then let the caller restore shared config with the proxy possibly alive. Only `inspectOcxStartProcessIdentity`'s `mismatch` is foreign; `unverifiable` is unknown. +- **The Part B port sample honors its timeout.** `sampleWindowsStopPort` called the synchronous `scanListenPids` and ignored `timeoutMs`, so the closing sample could stall the stop past its cap. It is now async over `netstat.exe` through `execFile` with the caller's timeout and `SIGKILL`; a timeout or an unreadable scan samples `unknown`, and the timeout is floored because `execFile` reads 0 as unlimited. +- **A not-running task needs its registration proof before `bound-stray`.** The state word says the task is inert, not that it is ours; promoting on the word alone let a same-named foreign task make this home's hand-started wrappers terminable. The promotion now follows `winTaskXml` and `winRegistrationOurs`, and anything else keeps the previous verdict. ## Open questions diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 8880aa962ee..e3eeb9a0f04 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -100,33 +100,59 @@ window. asynchronous `schtasks` and `sc.exe` calls through `execFile` (a deadline-derived timeout is floored to whole milliseconds and one that floors to 0 spawns nothing, because `execFile` reads 0 as no timeout), the evidence snapshot `windowsProcessSnapshot` for every bind, the checked -terminate, the exit watch, the Part B port sampler (`scanListenPids`; a row whose owner has exited -is cleared like `reclaimListenPort` does) and the listener identity (`verifyPidIdentity`: ours or -foreign). The task's running state is read from `schtasks /query /fo CSV`, falling back to -`Get-ScheduledTask` inside the same 1 s cap when the status word is localized. +terminate, the exit watch, the Part B port sampler and the listener identity. The task's running +state is read from `schtasks /query /fo CSV`, falling back to `Get-ScheduledTask` inside the same +1 s cap when the status word is localized. + +The port sampler is `sampleWindowsStopPort(port, timeoutMs)`, async and bounded by the caller's +timeout: it runs `netstat.exe` from the trusted System32 directory through `execFile` with that +timeout and `SIGKILL`, and a scan that misses the timeout, or cannot be read, samples `unknown`. +A synchronous scan would ignore the cap and could stall the stop for the helper's own 5 s. A row +whose owner has exited is cleared like `reclaimListenPort` does. The listener identity is +tri-state over `inspectOcxStartProcessIdentity`: only a positive `mismatch` is `foreign`, and an +`unverifiable` probe (another account's or an elevated process's command line) stays `unknown`. +`verifyPidIdentity`'s null collapses those two, which would report a listener we merely could not +read as another program holding the port. `src/service/windows-service-stop.ts` is what callers call: `runWindowsServiceStopPartA` (optional admission, a reused bind snapshot, the updater's authorizer, a hard cap for the route), `confirmWindowsServiceStop` (Part B), the mapping onto `ServiceStopOutcome` (absent, stopped; survived, denied and unverified-budget are failed; unknown is state-unknown), the additive `serviceStopReason` and the receipt record. The deadline lives in -`src/service/windows-stop-deadline.ts`: `ocx stop` takes `--deadline-ms `, else -`OCX_STOP_DEADLINE_MS` (empty or invalid counts as absent), else 60 s, measured from the CLI's own -process start (the Part A clock is `performance.now()`, which counts from the process start). +`src/service/windows-stop-deadline.ts`, and its origin depends on who is stopping. `ocx stop` +takes `--deadline-ms `, else `OCX_STOP_DEADLINE_MS` (empty or invalid counts as absent), else +60 s, measured from the command's own start: that is the operator's budget, and +`performance.now()` counting from `performance.timeOrigin` makes it an absolute value on the Part +A clock. Every other caller starts the default 60 s when its stop phase begins, because it +reaches the stop after work it cannot charge to the stop — the scheduler registration and the UAC +prompt — and a budget measured from process start could already be spent, which made admission +refuse `unverified-budget` a stop that had its whole budget left. The first deadline read in a +stop phase arms it and later reads inside that phase keep it, so one deadline governs the whole +stop instead of sliding forward. - `stopServiceIfInstalledReport` / `stopServiceIfInstalledDetailed` (`src/service/orchestration.ts`) are async and run Part A on Windows. On Windows Task Scheduler never yields `stopped-respawnable`. - Plain `ocx stop` (`src/cli/stop-windows-checked.ts`) admits against the deadline with orphan discovery and the inherited receipts counted. A Part A other than stopped or absent holds the child: no `/api/stop` POST, the shared teardown skipped, `service` failed or state-unknown, - `proxy` unknown. After a stopped Part A, the POST skips `stopProxy`'s port wait, orphan discovery - is capped at 3 s, and Part B replaces the respawn re-poll: our identity back on the port is - `proxy: respawned`, an unconfirmed port is `evidence-unreadable`, and another program on the port - sets `portHeldByForeign` without failing the stop. + `proxy` unknown. After a stopped Part A, the POST skips `stopProxy`'s legacy 15 s + `waitForStoppedPort`, orphan discovery is capped at 3 s, and Part B replaces the respawn re-poll: + our identity back on the port is `proxy: respawned`, an unconfirmed port is + `evidence-unreadable`, and another program on the port sets `portHeldByForeign` without failing + the stop. The port wait is skipped on BOTH Windows paths, guarded included, through + `windowsPartBOwnsPortWait`; `windowsPartBDue` stays snapshotless-only because it also gates the + legacy respawn re-poll. `ocx uninstall`'s two `stopProxy` calls pass the same predicate. - The guarded path reads the evidence snapshot and the Schedule host PID once per inspection; `stopManager` binds on the revalidation's snapshot (so its admission does not count a second enumeration), Part B runs after `settle` on the approved endpoint, and the manager-state checks reuse Part B's final enumeration. +- A registered task whose state is `not-running` promotes its surviving wrappers to `bound-stray` + only after `winTaskXml` and `winRegistrationOurs` pass (`src/service/windows-guarded-manager.ts`). + The state word says the task is inert, not that it is ours; promoting on the word alone let a + same-named foreign task make this home's hand-started wrappers terminable. Nothing surviving + keeps the `absent` verdict without the read; an unreadable definition is `evidence-unreadable` + and a definition that is not this installation's is `unknown`, both leaving the previous + verdict. - The teardown receipt carries the Part A record (`partA` stopped or absent, the registered managers, the ended wrappers as pid + creation time): written at the claim, or added by the owner for a receipt claimed before Part A. `/api/stop` (`src/server/stop-service-route.ts`) skips its @@ -139,6 +165,14 @@ process start (the Part A clock is `performance.now()`, which counts from the pr shared teardown (a failed claim sends no child stop), runs Part B, and clears the receipt only after the restore. `ocx service uninstall` / `remove` refuse before deleting the task unless Part A stopped (or found nothing) and Part B did not find our identity or an unreadable port. + Removal claims the same receipt, carrying Part A's record, before its child stop: Part A leaves + the scheduler task registered, so a receipt-less POST answers `respawnable_service` (409) and + the removal refuses with the proxy left running. The receipt is discharged only after the + shared restore, and a failed claim sends no child stop. +- Every tracked-proxy stop takes its orphan-discovery cap from the caller + (`TrackedProxyStopIo.orphanDiscoveryCapMs`). The checked Windows callers pass the admitted 3 s + (`WINDOWS_STOP_CAPS.orphanDiscovery`), which is what admission priced; the omitted default stays + the legacy 7 s so macOS and Linux timing is unchanged. - `prepareServiceInstall` refuses before replacing anything on a Part A other than stopped or absent and stops install and switch on any Part B result other than a free port, naming the owner. `ocx uninstall` runs Part B as its respawn window after a stopped Part A. @@ -176,9 +210,12 @@ PowerShell or FFI call. The FFI primitive itself is exercised only by the cases Windows x64. The callers are covered by `tests/windows/windows-checked-stop-live.test.ts` (adapters, ARM64 -strict, whole-millisecond timeouts, the ancestry check), `tests/windows/windows-service-stop-wiring.test.ts` -(the real Part A through the wiring: the own-wrapper refusal, the cap, admission, the deadline and -the receipt record), `tests/windows/windows-stop-callers.test.ts` (the route, the service commands, -the guarded order, `bound-stray`, and the bind on the evidence snapshot) and -`tests/windows/windows-stop-ocx-stop.test.ts` (plain `ocx stop` through the real CLI with the -platform simulated). None of them reaches `schtasks`, PowerShell or FFI. +strict, whole-millisecond timeouts, the ancestry check, the bounded port sampler and the tri-state +listener identity), `tests/windows/windows-service-stop-wiring.test.ts` +(the real Part A through the wiring: the own-wrapper refusal, the cap, admission, the deadline's +two origins and the receipt record), `tests/windows/windows-stop-callers.test.ts` (the route, the +service commands, the guarded order, `bound-stray`, the orphan-discovery cap and the bind on the +evidence snapshot), `tests/windows/windows-guarded-manager.test.ts` (the not-running promotion +behind its registration proof) and `tests/windows/windows-stop-ocx-stop.test.ts` (plain `ocx stop` +through the real CLI with the platform simulated). None of them reaches `schtasks`, PowerShell or +FFI. From da2d55788007c41985df1bca865cb2f1a214d25f Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 21:10:00 +0900 Subject: [PATCH 18/53] fix(cli): uninstall's port wait asks the port-wait question The two `stopProxy` calls in `handleUninstall` were keyed to `windowsPartBDue()`, which is snapshotless-only because it also gates the legacy respawn re-poll. Uninstall's Part B is the respawn-window step that follows service removal, so keying the skip to the snapshotless re-poll rule was the wrong question: the calls skipped the legacy 15 s wait only by accident of that rule's current shape, while the contract doc already said they pass the same predicate as the guarded path. Both calls now pass `windowsPartBOwnsPortWait()`, the predicate that answers only "does Part B own the port wait". Non-Windows behavior is unchanged (the predicate requires the simulated win32 platform), and `windowsPartBDue` keeps its snapshotless-only respawn semantics. Fail-before: `bun test tests/cli/uninstall.test.ts` -> 2 fail (both `stopProxy(... skipPortWait: windowsPartBOwnsPortWait())` assertions), pass-after -> 53 pass, 0 fail. Refs #7771 --- src/cli/index.ts | 8 +++++--- tests/cli/uninstall.test.ts | 14 +++++++++----- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index acc7b7b4c31..5ecce0ed320 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -1597,12 +1597,14 @@ async function handleUninstall() { throw new Error(`a proxy is answering on port ${live.port} but no process id could be resolved for it; stop it from the home that started it, then rerun`); } // #7771: on Windows the respawn-window step below is Part B, which owns the port - // wait; the legacy 15 s waitForStoppedPort must not run on top of it. - await stopProxy(live.pid, { skipPortWait: windowsPartBDue() }); + // wait; the legacy 15 s waitForStoppedPort must not run on top of it. Ask the + // port-wait question itself: `windowsPartBDue` also gates the legacy re-poll and is + // snapshotless-only, which is a different rule. + await stopProxy(live.pid, { skipPortWait: windowsPartBOwnsPortWait() }); observed.proxyProvenDown = true; return true; } - await stopProxy(pid, { skipPortWait: windowsPartBDue() }); + await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() }); removePid(pid); removeRuntimePort(pid); observed.proxyProvenDown = true; diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index c3e3006366a..7ef3dd50ff2 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -124,11 +124,11 @@ describe("full uninstall command", () => { expect(uninstallBody).toContain('runStep("service stopped"'); expect(uninstallBody).toContain('runStep("proxy stopped"'); expect(uninstallBody).toContain('runStep("service removed"'); - expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBDue() });"); + expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() });"); expect(uninstallBody).toContain("uninstallServiceDetailed()"); expect(uninstallBody.indexOf('runStep("service stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("proxy stopped"')); expect(uninstallBody.indexOf('runStep("proxy stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("service removed"')); - expect(uninstallBody.indexOf("await stopProxy(pid, { skipPortWait: windowsPartBDue() });")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); + expect(uninstallBody.indexOf("await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() });")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); }); test("Windows uninstall leaves the port wait to Part B instead of stopProxy's legacy 15 s", async () => { @@ -136,9 +136,13 @@ describe("full uninstall command", () => { const uninstallBody = cli.slice(cli.indexOf("async function handleUninstall()"), cli.indexOf("type HealthCheck")); // #7771: after a stopped Part A on Windows the respawn-window step IS Part B, which // owns the port wait; running stopProxy's 15 s waitForStoppedPort on top of it spends - // the stop budget twice. Both tracked and orphan stops pass the same predicate. - expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBDue() });"); - expect(uninstallBody).toContain("await stopProxy(live.pid, { skipPortWait: windowsPartBDue() });"); + // the stop budget twice. Both tracked and orphan stops pass the port-wait predicate + // itself (not `windowsPartBDue`, which additionally gates the legacy re-poll): the + // uninstall step must keep asking the question it means, so a later change to the + // snapshotless rule cannot silently bring the 15 s wait back. + expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() });"); + expect(uninstallBody).toContain("await stopProxy(live.pid, { skipPortWait: windowsPartBOwnsPortWait() });"); + expect(uninstallBody).not.toContain("skipPortWait: windowsPartBDue()"); const checked = await readText("src/cli/stop-windows-checked.ts"); // The predicate is about the port wait only; the legacy re-poll key stays snapshotless. const portWait = checked.slice(checked.indexOf("export function windowsPartBOwnsPortWait")); From e7f11f8a49b7886c0661e4b314e30df2b23ee5d7 Mon Sep 17 00:00:00 2001 From: JUN Date: Thu, 12 Nov 2026 22:03:04 +0900 Subject: [PATCH 19/53] test(service): pin the not-running definition check with a bindable tree The guarded-manager test for this rule used fixtures without creation times, so binding refused as unknown whether or not the definition was checked: it passed with the fix reverted. This case uses the tree that binds to bound-stray when the registration is ours and shows a foreign or unreadable definition keeps it unknown. Fails with 968290c565's source change reverted. Refs #7771 --- tests/windows/windows-stop-callers.test.ts | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index f1c06dd72a3..90e58325c82 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -352,6 +352,27 @@ describe("guarded inspection: the binding rule decides bound-stray", () => { expect(inspect(cimOnly)).toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); }); + test("the same bindable survivors stay unknown when the not-running registration is not provably ours", () => { + // #7771: this tree binds to bound-stray when the definition is ours (test above). A + // same-named foreign task, or a definition that cannot be read, must not turn this + // home's surviving wrappers into terminable ones. + const withDefinition = (win: { winTaskXml?: () => string; winRegistrationOurs?: () => boolean }) => + inspectGuardedManagerTarget(42, 10100, { + platform: "win32", verifyPid: pid => pid, + scheduler: () => ({ status: "present" }), winsw: () => "nonexistent", + win: { + winProcs: () => tree(), winTaskState: () => "not-running", + winTaskXml: () => "xml", winRegistrationOurs: () => true, + winSchedulePid: () => SCHEDULE_PID, winScriptPath: () => CMD, winLauncherPath: () => VBS, + ...win, + }, + }); + expect(withDefinition({}).kind).toBe("bound-stray"); + expect(withDefinition({ winRegistrationOurs: () => false }).kind).toBe("unknown"); + expect(withDefinition({ winTaskXml: () => { throw new Error("access denied"); } })) + .toMatchObject({ kind: "unknown", stopReason: "evidence-unreadable" }); + }); + test("a hand-started launcher under a live shell stays unknown / unbound-wrapper", () => { const shell = entry(60, 1, "pwsh.exe", "pwsh", 100); // The task's own wrappers bind; a second launcher started by hand from a live shell is not From d5516a480048698b1fc7cad1b79c79456c8a8b68 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:28:14 +0900 Subject: [PATCH 20/53] fix(cli): uninstall's proxy stop claims the teardown receipt first MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ocx uninstall` sent /api/stop with no deferral, so the proxy performed the shared client teardown during the stop — before the Windows respawn-window check ran. If that check then failed, native Codex and the Grok fence were already restored under a wrapper that could still relaunch the proxy (#3008). The stop now lives in `src/cli/uninstall-stop.ts` so the decision is callable rather than only readable, and on Windows it claims a pending-teardown receipt carrying Part A's record before the child stop, sends its nonce on the POST, and leaves the receipt for recovery unless Part B and the shared restore both succeeded. Off Windows nothing changes. Refs #7771 --- src/cli/index.ts | 64 ++++++++++---------- src/cli/uninstall-stop.ts | 114 ++++++++++++++++++++++++++++++++++++ tests/cli/uninstall.test.ts | 103 ++++++++++++++++++++++++++++---- 3 files changed, 237 insertions(+), 44 deletions(-) create mode 100644 src/cli/uninstall-stop.ts diff --git a/src/cli/index.ts b/src/cli/index.ts index 5ecce0ed320..25b0ecfbcec 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -79,6 +79,7 @@ import { quarantinePendingTeardown, } from "../config/pending-teardown"; import { collectStatus, detectMissingCodexCatalogPath, hubStatusLines, missingCodexCatalogLines, remoteHubBannerLine, remoteHubStatusLines, runtimeSupervisorLine, statusDeadProxyLines, statusProxyUp, statusUnusedProxyLines } from "./status"; +import { stopUninstallProxy } from "./uninstall-stop"; import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; import { takeFlag } from "./runtime-api"; import { parseStartOptions, StartArgsError } from "./start-args"; @@ -130,6 +131,7 @@ import { runCli } from "./root"; import { isProcessAlive, ProxyOwnershipRefusedError, refusalNextStep, stopProxy } from "../lib/process-control"; import { startupDataPlaneToken } from "../lib/service-secrets"; import { assertNotAdminToken, diagnoseService, isServiceOwnershipError, serviceCommand, serviceEnvironmentOwnedHere, serviceStartableFromTray, serviceStatePaths, serviceStatusSummary, uninstallServiceIfInstalled, uninstallServiceDetailed } from "../service"; +import { completeWindowsServiceStopReceipt } from "../service/cli-windows-stop"; import { acquireOwnershipMutationLease } from "../service/ownership-mutation-lease.mjs"; import { formatStartupRoutingDetail, startupHealthSummary } from "../codex/autostart-health"; import { injectSystemEnv, reconcileShellHook, revertSystemEnv, uninstallShellHook } from "../server/system-env"; @@ -1562,6 +1564,9 @@ async function handleUninstall() { serviceRemoval: null, respawnWindowVerified: false, }; + // The receipt an uninstall's Windows proxy stop claimed; discharged only after Part B + // and the shared restore succeeded, kept for recovery otherwise (#7771, #3008). + let uninstallStopNonce: string | undefined; await runStep("service stopped", async () => { const outcome = await stopServiceForUninstall(); observed.serviceStop = outcome; @@ -1576,39 +1581,26 @@ async function handleUninstall() { }); await runStep("proxy stopped", async () => { - const pid = readPid(); - if (!pid) { - // A missing pid file is not proof that nothing is serving: a proxy can outlive its - // record (crash, manual delete, corrupt file), which is exactly why `ocx stop` falls - // back to identity-checked discovery. Without this, uninstall restored shared config - // and reported success while that proxy kept running (#3008). - const live = await findLiveProxy(); - if (!live) { - // A miss is not proof: `findLiveProxy` collapses a timeout and a transport failure - // into the same null as a dead endpoint. Ask the tri-state probe, which only says - // "dead" for a refused connection or a definitive non-OpenCodex answer (#3008). - observed.proxyProvenDown = await proxyEndpointProvenDown(); - if (!observed.proxyProvenDown) { - throw new Error("no proxy could be found, but its endpoint could not be confirmed down either; confirm nothing is serving, then rerun"); - } - return false; - } - if (!live.pid) { - throw new Error(`a proxy is answering on port ${live.port} but no process id could be resolved for it; stop it from the home that started it, then rerun`); - } - // #7771: on Windows the respawn-window step below is Part B, which owns the port - // wait; the legacy 15 s waitForStoppedPort must not run on top of it. Ask the - // port-wait question itself: `windowsPartBDue` also gates the legacy re-poll and is - // snapshotless-only, which is a different rule. - await stopProxy(live.pid, { skipPortWait: windowsPartBOwnsPortWait() }); - observed.proxyProvenDown = true; - return true; - } - await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() }); - removePid(pid); - removeRuntimePort(pid); - observed.proxyProvenDown = true; - return true; + // #7771 / #3008: on Windows this claims the teardown receipt before the child stop and + // forbids the forced fallback when Part A did not prove the manager stopped, so shared + // client config is never restored under a wrapper that can relaunch the proxy. + const result = await stopUninstallProxy({ + platform: process.platform, + serviceStop: observed.serviceStop, + readPid, + findLiveProxy, + endpointProvenDown: proxyEndpointProvenDown, + partARecord: windowsPartAReceiptRecord, + stop: (target, io) => stopProxy(target, io), + removeTracked: pid => { + removePid(pid); + removeRuntimePort(pid); + }, + portWaitOwned: windowsPartBOwnsPortWait, + }); + uninstallStopNonce = result.nonce; + observed.proxyProvenDown = result.provenDown; + return result.changed; }); await runStep("service removed", () => { @@ -1670,6 +1662,12 @@ async function handleUninstall() { console.error(" 'ocx restore' is an interim step if you need native routing back before then."); } + // The obligation the proxy stop deferred is discharged only now: Part B and the shared + // restore both succeeded. Otherwise the receipt stays for the next stop to finish. + if (uninstallStopNonce && failures.length === 0) { + completeWindowsServiceStopReceipt(uninstallStopNonce); + } + await runStep("system env vars reverted", () => { const r = revertSystemEnv(); if (!r.reverted && r.reason !== "no tracking file" && r.reason !== "not macOS") throw new Error(r.reason ?? "revert failed"); diff --git a/src/cli/uninstall-stop.ts b/src/cli/uninstall-stop.ts new file mode 100644 index 00000000000..bc85af4db86 --- /dev/null +++ b/src/cli/uninstall-stop.ts @@ -0,0 +1,114 @@ +/** + * `ocx uninstall`'s proxy stop, extracted from `handleUninstall` so the decision is + * callable rather than only readable (#7771, #3008). + * + * The step has two jobs that used to be implicit in the command body: find the proxy + * (tracked pid, or an identity-checked orphan), and stop it without letting a wrapper + * relaunch it after shared client config has already been restored. + * + * On Windows the stop is deferred exactly as `ocx stop` and `ocx service stop` do it: a + * teardown receipt is claimed BEFORE the child stop and its nonce travels on the POST, so + * the proxy does not restore shared client config while a Task Scheduler wrapper can still + * relaunch it. The caller discharges the receipt only after Part B and its own restore + * succeeded; otherwise it stays on disk for recovery. + * + */ +import { loadConfig } from "../config"; +import { claimPendingTeardown, type PendingTeardownPartA } from "../config/pending-teardown"; +import { readRuntimePort } from "../config/process-state"; +import type { GracefulStopIo } from "../lib/process-control"; + +export interface UninstallProxyStopIo { + platform: NodeJS.Platform; + /** Detailed Part A outcome, or null when the service-stop step threw. */ + serviceStop: "absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown" | null; + readPid: () => number | null; + findLiveProxy: () => Promise<{ pid?: number | null; port: number } | null>; + /** Tri-state proof that nothing answers on any endpoint this home could serve. */ + endpointProvenDown: () => Promise; + /** The Part A record the receipt carries, so /api/stop skips a second Part A. */ + partARecord?: () => PendingTeardownPartA | null; + claim?: ( + endpoint: { hostname: string; port: number }, + source: "exact" | "guessed", + ownerPid: number, + handover: "update-restart" | undefined, + partA: PendingTeardownPartA | null, + ) => { nonce: string }; + receiptEndpoint?: () => { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" }; + stop: (pid: number, io: GracefulStopIo) => Promise; + removeTracked: (pid: number) => void; + /** Whether Part B owns the Windows port wait (#7771). */ + portWaitOwned: () => boolean; +} + +export interface UninstallProxyStopResult { + changed: boolean; + provenDown: boolean; + /** The deferred-teardown receipt this step claimed, for the caller to discharge. */ + nonce?: string; +} + +/** The endpoint a receipt records: the runtime record when there is one, else the config. */ +function defaultReceiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { + const runtime = readRuntimePort(); + if (runtime?.port) return { endpoint: { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port }, source: "exact" }; + try { + const config = loadConfig(); + return { + endpoint: { hostname: config.hostname ?? "127.0.0.1", port: typeof config.port === "number" && config.port > 0 ? config.port : 10100 }, + source: "guessed", + }; + } catch { + return { endpoint: { hostname: "127.0.0.1", port: 10100 }, source: "guessed" }; + } +} + +export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise { + const windows = io.platform === "win32"; + let nonce: string | undefined; + if (windows) { + const { endpoint, source } = (io.receiptEndpoint ?? defaultReceiptEndpoint)(); + try { + nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, (io.partARecord ?? (() => null))()).nonce; + } catch (error) { + // Without the receipt the proxy performs its own teardown during the stop, which is + // the pre-#3008 behaviour — and on Windows a wrapper can relaunch it afterwards, so + // client config would already be restored under a live proxy. Refuse instead. + throw new Error( + "the deferred-teardown receipt could not be recorded, so the proxy was left running: " + + (error instanceof Error ? error.message : String(error)), + ); + } + } + const stopIo: GracefulStopIo = { + skipPortWait: io.portWaitOwned(), + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + }; + const pid = io.readPid(); + if (!pid) { + // A missing pid file is not proof that nothing is serving: a proxy can outlive its + // record (crash, manual delete, corrupt file), which is exactly why `ocx stop` falls + // back to identity-checked discovery. Without this, uninstall restored shared config + // and reported success while that proxy kept running (#3008). + const live = await io.findLiveProxy(); + if (!live) { + // A miss is not proof: `findLiveProxy` collapses a timeout and a transport failure + // into the same null as a dead endpoint. Ask the tri-state probe, which only says + // "dead" for a refused connection or a definitive non-OpenCodex answer (#3008). + const provenDown = await io.endpointProvenDown(); + if (!provenDown) { + throw new Error("no proxy could be found, but its endpoint could not be confirmed down either; confirm nothing is serving, then rerun"); + } + return { changed: false, provenDown, ...(nonce ? { nonce } : {}) }; + } + if (!live.pid) { + throw new Error(`a proxy is answering on port ${live.port} but no process id could be resolved for it; stop it from the home that started it, then rerun`); + } + await io.stop(live.pid, stopIo); + return { changed: true, provenDown: true, ...(nonce ? { nonce } : {}) }; + } + await io.stop(pid, stopIo); + io.removeTracked(pid); + return { changed: true, provenDown: true, ...(nonce ? { nonce } : {}) }; +} diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 7ef3dd50ff2..fc51371ddd8 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -18,6 +18,7 @@ import { INTEGRATION_CLIENTS } from "../../src/integrations/registry"; import { createIntegrationStateStore } from "../../src/integrations/store"; import { applyIntegration, disableIntegrationCoordinated } from "../../src/integrations/writer"; import type { OcxConfig } from "../../src/types"; +import { stopUninstallProxy } from "../../src/cli/uninstall-stop"; const root = pathToFileURL(repoRoot() + "/"); @@ -124,11 +125,12 @@ describe("full uninstall command", () => { expect(uninstallBody).toContain('runStep("service stopped"'); expect(uninstallBody).toContain('runStep("proxy stopped"'); expect(uninstallBody).toContain('runStep("service removed"'); - expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() });"); + // #7771 round 7: the stop itself lives in `uninstall-stop`, which the step calls. + expect(uninstallBody).toContain("await stopUninstallProxy({"); expect(uninstallBody).toContain("uninstallServiceDetailed()"); expect(uninstallBody.indexOf('runStep("service stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("proxy stopped"')); expect(uninstallBody.indexOf('runStep("proxy stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("service removed"')); - expect(uninstallBody.indexOf("await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() });")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); + expect(uninstallBody.indexOf("await stopUninstallProxy({")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); }); test("Windows uninstall leaves the port wait to Part B instead of stopProxy's legacy 15 s", async () => { @@ -140,9 +142,11 @@ describe("full uninstall command", () => { // itself (not `windowsPartBDue`, which additionally gates the legacy re-poll): the // uninstall step must keep asking the question it means, so a later change to the // snapshotless rule cannot silently bring the 15 s wait back. - expect(uninstallBody).toContain("await stopProxy(pid, { skipPortWait: windowsPartBOwnsPortWait() });"); - expect(uninstallBody).toContain("await stopProxy(live.pid, { skipPortWait: windowsPartBOwnsPortWait() });"); - expect(uninstallBody).not.toContain("skipPortWait: windowsPartBDue()"); + expect(uninstallBody).toContain("portWaitOwned: windowsPartBOwnsPortWait,"); + const stopHelper = await readText("src/cli/uninstall-stop.ts"); + // The helper asks the port-wait question itself rather than re-deciding it. + expect(stopHelper).toContain("skipPortWait: io.portWaitOwned(),"); + expect(stopHelper).not.toContain("windowsPartBDue"); const checked = await readText("src/cli/stop-windows-checked.ts"); // The predicate is about the port wait only; the legacy re-poll key stays snapshotless. const portWait = checked.slice(checked.indexOf("export function windowsPartBOwnsPortWait")); @@ -231,17 +235,16 @@ describe("uninstall gates shared teardown on a proven service stop", () => { }); test("a live orphan with no pid file blocks the teardown", async () => { - const cli = await readText("src/cli/index.ts"); - const at = cli.indexOf("async function handleUninstall("); - const fn = cli.slice(at, at + 9000); + const helper = await readText("src/cli/uninstall-stop.ts"); + const fn = helper.slice(helper.indexOf("export async function stopUninstallProxy(")); // A missing pid file is not proof that nothing is serving — the same discovery // `ocx stop` performs. Without it, uninstall restored shared config under a live proxy. - expect(fn).toContain("const live = await findLiveProxy();"); - expect(fn).toContain("observed.proxyProvenDown = await proxyEndpointProvenDown();"); + expect(fn).toContain("const live = await io.findLiveProxy();"); + expect(fn).toContain("const provenDown = await io.endpointProvenDown();"); expect(fn).toContain("no process id could be resolved for it"); // The orphan-with-no-pid branch THROWS, so `proxyProvenDown` stays false and the // authorization rule refuses the shared teardown. - const orphanBranch = fn.slice(fn.indexOf("const live = await findLiveProxy();"), fn.indexOf("const live = await findLiveProxy();") + 600); + const orphanBranch = fn.slice(fn.indexOf("const live = await io.findLiveProxy();"), fn.indexOf("const live = await io.findLiveProxy();") + 600); expect(orphanBranch).toContain("throw new Error("); // A findLiveProxy miss is not proof either: it goes through the tri-state probe first. expect(orphanBranch).toContain("could not be confirmed down either"); @@ -842,3 +845,81 @@ describe("uninstall restores recorded third-party integrations before deleting r } finally { rmSync(f.root, { recursive: true, force: true }); } }); }); + +describe("uninstall proxy stop defers the shared teardown through a receipt (#7771)", () => { + // Round 7 (A/B): the uninstall POST used to carry no deferral, so the proxy performed the + // shared client teardown itself BEFORE the Windows respawn window ran. If that window then + // failed, client config had already been restored under a wrapper that could relaunch the + // proxy. And when the service stop itself failed, the step could still fall through to + // taskkill /T /F. Both are decided here so they can be exercised without a machine. + const NONCE = "c".repeat(32); + const io = (over: Partial[0]> = {}): Parameters[0] => ({ + platform: "win32", + serviceStop: "stopped", + readPid: () => 4242, + findLiveProxy: async () => null, + endpointProvenDown: async () => true, + claim: () => ({ nonce: NONCE }), + stop: async () => true, + removeTracked: () => {}, + portWaitOwned: () => true, + ...over, + }); + + test("Windows: the receipt is claimed before the child stop and its nonce travels on the POST", async () => { + const steps: string[] = []; + const seen: unknown[] = []; + const result = await stopUninstallProxy(io({ + claim: () => { steps.push("claim"); return { nonce: NONCE }; }, + stop: async (pid, stopIo) => { steps.push("stop:" + pid); seen.push(stopIo); return true; }, + removeTracked: pid => { steps.push("remove:" + pid); }, + })); + expect(steps).toEqual(["claim", "stop:4242", "remove:4242"]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE }]); + expect(result).toEqual({ changed: true, provenDown: true, nonce: NONCE }); + }); + + test("an orphan stop carries the same nonce and leaves the tracked records alone", async () => { + const seen: unknown[] = []; + const removed: number[] = []; + const result = await stopUninstallProxy(io({ + readPid: () => null, + findLiveProxy: async () => ({ pid: 777, port: 12000 }), + stop: async (pid, stopIo) => { seen.push([pid, stopIo]); return true; }, + removeTracked: pid => { removed.push(pid); }, + })); + expect(seen).toEqual([[777, { skipPortWait: true, deferSharedTeardownNonce: NONCE }]]); + expect(removed).toEqual([]); + expect(result.nonce).toBe(NONCE); + }); + + test("a receipt that cannot be recorded refuses the stop instead of sending an undeferred POST", async () => { + let stopped = 0; + await expect(stopUninstallProxy(io({ + claim: () => { throw new Error("disk full"); }, + stop: async () => { stopped++; return true; }, + }))).rejects.toThrow(/deferred-teardown receipt could not be recorded[\s\S]*disk full/); + expect(stopped).toBe(0); + }); + + test("off Windows nothing changes: no receipt, no fallback narrowing, today's port wait", async () => { + const seen: unknown[] = []; + let claimed = 0; + const result = await stopUninstallProxy(io({ + platform: "linux", serviceStop: "failed", + claim: () => { claimed++; return { nonce: NONCE }; }, + portWaitOwned: () => false, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + expect(claimed).toBe(0); + expect(seen).toEqual([{ skipPortWait: false }]); + expect(result).toEqual({ changed: true, provenDown: true }); + }); + + test("the receipt is finished only after the shared restore ran", async () => { + const { completeWindowsServiceStopReceipt } = await import("../../src/service/cli-windows-stop"); + const cleared: string[] = []; + completeWindowsServiceStopReceipt(NONCE, nonce => { cleared.push(nonce); return true; }); + expect(cleared).toEqual([NONCE]); + }); +}); From 29419fd70b89c64f7b9aa1faeecb1442cd37d307 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:28:28 +0900 Subject: [PATCH 21/53] fix(cli): uninstall never forces a stop its Part A could not prove The proxy-stop step ran after a failed or unreadable service stop, and `stopProxy` falls back to `taskkill /T /F` when the POST times out, the transport fails, or the answer is not a 409. Under a manager that would not stop, that forced kill is the one outcome the checked Windows stop exists to prevent: the wrapper can relaunch the proxy after its teardown already ran. `stopProxy` gains `allowForcedFallback` (default true, so every other caller is unchanged) and uninstall sets it false when Part A did not end stopped/absent. The graceful POST still goes out, the uninstall failure stays recorded, and an unconfirmed stop no longer claims the proxy was proven down. A 409 refusal still never reaches the ladder. Refs #7771 --- src/cli/uninstall-stop.ts | 20 +++++++-- src/lib/process-control.ts | 22 +++++++++- tests/cli/uninstall.test.ts | 27 +++++++++++++ tests/lib/process-control-graceful.test.ts | 47 ++++++++++++++++++++++ 4 files changed, 111 insertions(+), 5 deletions(-) diff --git a/src/cli/uninstall-stop.ts b/src/cli/uninstall-stop.ts index bc85af4db86..9cb83706442 100644 --- a/src/cli/uninstall-stop.ts +++ b/src/cli/uninstall-stop.ts @@ -12,6 +12,10 @@ * relaunch it. The caller discharges the receipt only after Part B and its own restore * succeeded; otherwise it stays on disk for recovery. * + * A Part A that did not end stopped/absent still gets the graceful POST — the proxy may be + * running and a graceful stop is always preferable — but never the forced fallback: a + * `taskkill /T /F` under a manager that would not stop is the one outcome the checked stop + * exists to prevent. */ import { loadConfig } from "../config"; import { claimPendingTeardown, type PendingTeardownPartA } from "../config/pending-teardown"; @@ -66,6 +70,9 @@ function defaultReceiptEndpoint(): { endpoint: { hostname: string; port: number export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise { const windows = io.platform === "win32"; + // Part A not stopped/absent means a manager may still be running and could respawn the + // proxy. The graceful POST is still worth sending; the forced fallback is not. + const partAProven = io.serviceStop === "stopped" || io.serviceStop === "stopped-respawnable" || io.serviceStop === "absent"; let nonce: string | undefined; if (windows) { const { endpoint, source } = (io.receiptEndpoint ?? defaultReceiptEndpoint)(); @@ -81,10 +88,16 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise stopped || !forbidForcedFallback; const pid = io.readPid(); if (!pid) { // A missing pid file is not proof that nothing is serving: a proxy can outlive its @@ -105,10 +118,9 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise void; } /** @@ -346,7 +360,13 @@ export async function stopProxy(pid: number, io: GracefulStopIo = {}): Promise { + for (const serviceStop of ["failed", "state-unknown", null] as const) { + const seen: unknown[] = []; + await stopUninstallProxy(io({ serviceStop, stop: async (_pid, stopIo) => { seen.push(stopIo); return false; } })); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, allowForcedFallback: false }]); + } + // A Part A that ended stopped (or found nothing) keeps today's fallback available. + const allowed: unknown[] = []; + for (const serviceStop of ["stopped", "absent"] as const) { + await stopUninstallProxy(io({ serviceStop, stop: async (_pid, stopIo) => { allowed.push(stopIo); return false; } })); + } + expect(allowed).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE }, { skipPortWait: true, deferSharedTeardownNonce: NONCE }]); + }); + + test("a forbidden fallback that does not confirm withholds the proof of a stopped proxy", async () => { + // The graceful POST may stop the proxy, but when it does not and the ladder was + // forbidden there is no proof left: the caller's recorded Part A failure is what keeps + // shared config up. Off Windows an unconfirmed stop keeps today's proof (the respawn + // window follows). + const windows = await stopUninstallProxy(io({ serviceStop: "failed", stop: async () => false })); + expect(windows.provenDown).toBe(false); + const stopped = await stopUninstallProxy(io({ serviceStop: "failed", stop: async () => true })); + expect(stopped.provenDown).toBe(true); + const elsewhere = await stopUninstallProxy(io({ platform: "linux", serviceStop: "failed", stop: async () => false })); + expect(elsewhere.provenDown).toBe(true); + }); + test("a receipt that cannot be recorded refuses the stop instead of sending an undeferred POST", async () => { let stopped = 0; await expect(stopUninstallProxy(io({ diff --git a/tests/lib/process-control-graceful.test.ts b/tests/lib/process-control-graceful.test.ts index 9f20eb386c4..9678dbe944a 100644 --- a/tests/lib/process-control-graceful.test.ts +++ b/tests/lib/process-control-graceful.test.ts @@ -353,6 +353,53 @@ describe("409 refusal reporting", () => { expect((thrown as ProxyOwnershipRefusedError).message).toBe("wrapper owns it"); }); + test("a caller that forbids the forced fallback never reaches the kill ladder", async () => { + // #7771 round 7 (B): uninstall's proxy step runs after a Part A that did not prove the + // manager stopped. The graceful POST is still worth sending; taskkill /T /F under a + // wrapper that could respawn the proxy is exactly what the checked stop prevents. + // + // The pid is a real live process this test owns, and the ladder is injected: the rule + // is "was the ladder entered", which a mock of the whole stop could not show. + const { spawn } = await import("node:child_process"); + const child = spawn("sleep", ["60"], { stdio: "ignore" }); + await new Promise(resolve => child.once("spawn", () => resolve())); + const failing = (async () => new Response("nope", { status: 500 })) as typeof fetch; + try { + const forbidden: number[] = []; + const unconfirmed = await stopProxy(child.pid!, { + readRuntime: () => null, fetchFn: failing, waitExit: () => true, env: {}, + allowForcedFallback: false, forceKill: pid => { forbidden.push(pid); }, + }); + expect(unconfirmed).toBe(false); + expect(forbidden).toEqual([]); + + // Default behaviour is unchanged: an unconfirmed graceful stop still falls back. + const fallback: number[] = []; + await stopProxy(child.pid!, { + readRuntime: () => null, fetchFn: failing, waitExit: () => true, env: {}, + forceKill: pid => { fallback.push(pid); }, + }); + expect(fallback).toEqual([child.pid!]); + + // A 409 refusal is never forced, with or without the option. + for (const allowForcedFallback of [true, false]) { + const refused: number[] = []; + await expect(stopProxy(child.pid!, { + readRuntime: () => ({ port: 10100 }), + fetchFn: (async () => new Response( + JSON.stringify({ success: false, code: "respawnable_service", message: "wrapper owns it" }), + { status: 409, headers: { "content-type": "application/json" } }, + )) as typeof fetch, + waitExit: () => true, env: {}, allowForcedFallback, + forceKill: pid => { refused.push(pid); }, + })).rejects.toBeInstanceOf(ProxyOwnershipRefusedError); + expect(refused).toEqual([]); + } + } finally { + child.kill("SIGKILL"); + } + }); + test("no next step sends the operator back to the command that just refused", () => { // #4169's loop: POST /api/stop answers "the stop must be run by `ocx stop`", and // `ocx stop` reprints it. Whatever the cause, the next step has to be something else, From 940bc111cadd8e0313654a475d8595c9c8f8056f Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:30:39 +0900 Subject: [PATCH 22/53] fix(service): an absent Part A is not proof the proxy is down `windowsServiceRemovalStop` returned ok for a Part A "absent" without proving anything about the endpoint. "Absent" only says no manager is registered: `findLiveProxy` inside the tracked stop can return null when its discovery cap runs out while the endpoint is live, and the task was then deleted and shared config restored under a running proxy. Removal now asks the endpoint itself, the way `windowsServiceStopBeforeRestore` already does for absent, and refuses with the task intact when something still answers. No respawn hold is requested: with no manager there is nothing that could relaunch the proxy. Refs #7771 --- src/service/cli-windows-stop.ts | 14 ++++++++++-- tests/windows/windows-stop-callers.test.ts | 25 ++++++++++++++++++++++ 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index d74573d2cb9..a545db5cf5d 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -124,7 +124,7 @@ export function completeWindowsServiceStopReceipt(nonce: string | undefined, cle * `ocx service stop` does, and kept until the caller's restore succeeds. */ export async function windowsServiceRemovalStop( - io: Pick = {}, + io: Pick = {}, ): Promise<{ ok: boolean; nonce?: string }> { const port = windowsServiceStopPort(); const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); @@ -149,7 +149,17 @@ export async function windowsServiceRemovalStop( skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, }); - if (partA.outcome !== "stopped") return withNonce(nonce); + if (partA.outcome === "absent") { + // "absent" says no manager is registered, not that nothing is serving. `findLiveProxy` + // inside the tracked stop can return null when its discovery cap runs out while the + // endpoint is live, so ask the endpoint itself before the task is deleted. No respawn + // hold: with no manager there is nothing to relaunch the proxy. + const survivor = await (io.survivor ?? proxyStillLiveAfterStop)({ canRespawn: false }); + if (!survivor) return withNonce(nonce); + console.error("❌ Service uninstall refused: a proxy is still listening on port " + survivor.port + "."); + console.error(" The task was not deleted. Stop that proxy, then re-run 'ocx service uninstall'."); + return { ok: false }; + } const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); if (partB.outcome === "freed") return withNonce(nonce); if (partB.outcome === "port-held-by-foreign") { diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index 90e58325c82..3ba0fb76d65 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -181,6 +181,31 @@ describe("ocx service uninstall / remove on Windows", () => { } }); + test("an absent Part A is not proof the proxy is down: a live endpoint still refuses removal", async () => { + // Part A "absent" only says no manager is registered. `findLiveProxy` can return null + // when its discovery cap runs out while the endpoint is live, so removal / the shared + // restore must ask the endpoint itself before deleting the task. No respawn hold is + // needed — nothing can relaunch it — but the answer still has to be definitive. + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const survivors: unknown[] = []; + const refused = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + survivor: async io => { survivors.push(io); return { port: 10100 }; }, + }); + expect(refused.ok).toBe(false); + // The hold would be meaningless here; asking for it would spend 7 s for nothing. + expect(survivors).toEqual([{ canRespawn: false }]); + + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const absent = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + survivor: async () => null, + }); + expect(absent).toEqual({ ok: true }); + }); + test("claims a receipt carrying the Part A record before the POST, so the route does not answer 409", async () => { // Deleting the registration leaves the scheduler task's receipt-less POST answering // respawnable_service, which fails the uninstall. `ocx service stop` already claims; From 61087d87dc9335cde1ad50a3e053c86bd9828699 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:30:46 +0900 Subject: [PATCH 23/53] test(service): stop the absent-Part-A caller test waiting on the real probe The caller test ran `windowsServiceStopBeforeRestore` with an absent Part A, which falls through to the real `proxyStillLiveAfterStop`. On Windows that spends its full 7 s respawn window before answering, and with the removal path's new endpoint check it would wait twice. Both callers are now given `survivor`, which also pins that the removal path asks for no respawn hold while the stop path keeps its default, and that both consulted the injection instead of the machine. Refs #7771 --- tests/service/service-stop-verification.test.ts | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/tests/service/service-stop-verification.test.ts b/tests/service/service-stop-verification.test.ts index f99da8cbe29..51b51f8ba8f 100644 --- a/tests/service/service-stop-verification.test.ts +++ b/tests/service/service-stop-verification.test.ts @@ -174,11 +174,22 @@ describe("orphan discovery is capped by the caller, not hard-coded (#7771)", () const { partAResult } = await import("../helpers/windows-stop-caller-fixtures"); setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); try { + // The survivor probe is injected: both callers fall through to the real + // proxyStillLiveAfterStop, which on Windows spends its full 7 s window before + // answering, and this test is about the discovery budget, not about that wait. + const survivors: unknown[] = []; for (const run of [windowsServiceStopBeforeRestore, windowsServiceRemovalStop]) { const seen: unknown[] = []; - await run({ ownsSharedTeardown: () => false, stopTracked: async io => { seen.push(io); return "stopped"; } }); + await run({ + ownsSharedTeardown: () => false, + stopTracked: async io => { seen.push(io); return "stopped"; }, + survivor: async io => { survivors.push(io ?? "no-args"); return null; }, + }); expect(seen).toEqual([{ skipPortWait: false, orphanDiscoveryCapMs: 3_000 }]); } + // Both callers consulted the injection rather than the real 7 s probe, and the + // removal path asks for no respawn hold because nothing there can relaunch the proxy. + expect(survivors).toEqual(["no-args", { canRespawn: false }]); } finally { setWindowsServiceStopHooksForTests(null); } From 3608136cc6027e6103533af0a5e66f2fd32ae061 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:32:37 +0900 Subject: [PATCH 24/53] fix(service): install preparation resolves the stop port before the tracked cleanup Part B was reading windowsServiceStopPort() after the tracked cleanup deleted the runtime record, so it confirmed the wrapper's port (10100) instead of the stopped proxy's port (12000). Resolve the port once before Part A and carry it into Part B. Refs #7771 --- src/service/orchestration.ts | 12 +++++++++- .../service-install-wrapper-cleanup.test.ts | 23 +++++++++++++++++++ 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index ad326826e55..2f26d92fb71 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -273,6 +273,13 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { windowsPartA?: (request: WindowsPartARequest) => Promise; /** Windows: Part B after the tracked-proxy stop; the port must be free and stay free. */ windowsPartB?: (request: { port: number; remainingMs: number }) => Promise; + /** + * Windows: the port the stopped proxy was serving, resolved ONCE before Part A (#7771). + * + * Resolving it after the tracked cleanup reads a runtime record the cleanup just deleted, + * so Part B confirmed the wrapper's port while the proxy's own port was never checked. + */ + windowsStopPort?: () => number; stopTrackedProxy?: () => Promise; platform?: NodeJS.Platform; } @@ -290,6 +297,9 @@ export async function prepareServiceInstall( assertNoDesktopSupervision(deps.inspectSupervision, latch); const platform = deps.platform ?? process.platform; if (platform === "win32") { + // Resolve the stopped proxy's port BEFORE anything removes its runtime record: Part B + // has to confirm that port, and the wrapper's port is only a fallback (#7771). + const stopPort = (deps.windowsStopPort ?? windowsServiceStopPort)(); // Part A stops whichever managers are registered (both on a conflicting install) and // ends the bound wrappers, or refuses with nothing deleted or replaced (#7771). It // also covers a registration deleted mid-uninstall that left a detached wrapper @@ -308,7 +318,7 @@ export async function prepareServiceInstall( })))(); if (partA.outcome === "stopped") { const partB = await (deps.windowsPartB ?? confirmWindowsServiceStop)({ - port: windowsServiceStopPort(), + port: stopPort, remainingMs: windowsStopRemainingMs(), }); if (partB.outcome !== "freed") { diff --git a/tests/service/service-install-wrapper-cleanup.test.ts b/tests/service/service-install-wrapper-cleanup.test.ts index 5c37883677a..a6a8dc01861 100644 --- a/tests/service/service-install-wrapper-cleanup.test.ts +++ b/tests/service/service-install-wrapper-cleanup.test.ts @@ -68,6 +68,29 @@ describe("service install preparation wrapper cleanup", () => { })).rejects.toThrow("nothing was published or started"); }); + test("Part B checks the stopped proxy's runtime port, resolved before the tracked cleanup", async () => { + // The defect: `windowsServiceStopPort()` was resolved AFTER the tracked cleanup removed + // the runtime record, so it fell through to the wrapper's port (10100) and Part B + // confirmed the wrong port while the proxy's own 12000 was never checked. + const calls: string[] = []; + let runtimePort: number | null = 12000; + const ports: number[] = []; + await prepareServiceInstall("scheduler", { + platform: "win32", + windowsStopPort: () => { + calls.push("port"); + // The runtime record is what makes this the stopped proxy's port; the wrapper's + // 10100 is only a fallback once that record is gone. + return runtimePort ?? 10100; + }, + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + stopTrackedProxy: async () => { calls.push("stop:standalone"); runtimePort = null; }, + windowsPartB: async request => { calls.push("partB"); ports.push(request.port); return partBResult("freed"); }, + }); + expect(calls).toEqual(["port", "partA", "stop:standalone", "partB"]); + expect(ports).toEqual([12000]); + }); + test("nothing registered: no Part B and no hold", async () => { const calls: string[] = []; await prepareServiceInstall("scheduler", { From 3e9e4c1240bbafccceb7281efb23772f0bcb45fa Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:49:52 +0900 Subject: [PATCH 25/53] fix(service): Part A proves the WinSW registration is this home's before stopping it sc query answers that `opencodex-proxy-native` exists, not that it is this installation's service. A successful query was treated as ownership, so Part A could run `sc stop` on another installation's WinSW service under the fixed ID. The live probe now reads the registered BINARY_PATH_NAME with `sc qc` and requires this home's WinSW executable, the same proof the guarded manager path uses. An unreadable registration is unknown; a foreign one is reported as nonexistent, so nothing is stopped. Refs #7771 --- src/service/windows-checked-stop-live.ts | 47 ++++++++++++++++-- .../windows/windows-checked-stop-live.test.ts | 48 +++++++++++++++++++ 2 files changed, 92 insertions(+), 3 deletions(-) diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts index edf3b00e570..c0242e66d00 100644 --- a/src/service/windows-checked-stop-live.ts +++ b/src/service/windows-checked-stop-live.ts @@ -26,8 +26,9 @@ import { inspectOcxStartProcessIdentity, readPid, verifyPidIdentity, type OcxSta import { isProcessAlive } from "../lib/process-control"; import { isTestHomeGuardArmed } from "../lib/test-home-guard"; import { resolveTrustedWindowsPowerShellExe, resolveTrustedWindowsSchtasksExe, resolveTrustedWindowsSystemDirectory, toWindowsSchtasksError } from "../lib/windows-elevation"; -import { WINSW_SERVICE_ID, type WinswStatus } from "../lib/winsw"; +import { WINSW_SERVICE_ID, winswExePath, type WinswStatus } from "../lib/winsw"; import { decodeWindowsTextBytes } from "../lib/windows-text"; +import { parseScQcBinaryPathName, scBinaryPathNamesExecutable } from "../service-manager-probe"; import { parseListenPidsFromNetstat, type ListenPidScan } from "../server/port-reclaim"; import { dropWindowsTcpRowsForLocalPort } from "../server/windows-tcp-drop"; import { TASK, windowsLauncherVbsPath, windowsServiceScriptPath } from "./state"; @@ -81,7 +82,10 @@ export const liveWindowsStopCommandRunner: WindowsStopCommandRunner = (program, return; } // Task Scheduler and the SCM are machine-global: an armed test process may only observe. - const mutating = program === "schtasks" ? args[0]?.toLowerCase() !== "/query" : program === "sc" && args[0] !== "query"; + // `sc query` and `sc qc` only read the SCM row and its registration. + const mutating = program === "schtasks" + ? args[0]?.toLowerCase() !== "/query" + : program === "sc" && !["query", "qc"].includes((args[0] ?? "").toLowerCase()); if (isTestHomeGuardArmed() && mutating) { resolve({ status: null, stdout: "", stderr: "refusing a machine-global service mutation from an armed test process", timedOut: false }); return; @@ -209,11 +213,36 @@ export function stopperUnderHomeWrapper( export interface WindowsCheckedStopLiveOptions { run?: WindowsStopCommandRunner; + /** + * The `sc.exe qc` query that proves the registered WinSW service runs THIS installation's + * executable. Defaults to the capped runner; tests inject it. + */ + scQcRun?: (timeoutMs: number) => Promise; /** Checked before every manager command (the route's cap, the updater's authorizer). */ gate?: () => boolean; snapshot?: (timeoutMs: number) => Promise; } +/** + * Is the registered WinSW service this installation's? + * + * `sc stop opencodex-proxy-native` names a fixed ID, so a successful `sc query` is evidence + * that SOMETHING is registered, not that it is ours: another installation's WinSW service + * under the same ID would be stopped by this home's Part A. The registered + * `BINARY_PATH_NAME` must name this home's WinSW executable, the same proof the guarded + * manager path uses (`windowsWinswServiceInfo` + `commandLineHasPathToken`). Unreadable is + * unknown; a foreign executable is `nonexistent`, which keeps Part A from stopping it. + */ +export function classifyWinswOwnership( + qc: WindowsStopCommandResult | null, + winswExe: string = winswExePath(), +): WinswStatus | "foreign" { + if (qc === null || qc.timedOut || qc.status !== 0) return "unknown"; + const binaryPathName = parseScQcBinaryPathName(qc.stdout); + if (binaryPathName === null) return "unknown"; + return scBinaryPathNamesExecutable(binaryPathName, winswExe) ? "started" : "foreign"; +} + /** The identity-checked proxy child, or null. */ export function windowsStopChildPid(): number | null { try { @@ -228,6 +257,7 @@ export function windowsCheckedStopLiveDeps(options: WindowsCheckedStopLiveOption const run = options.run ?? liveWindowsStopCommandRunner; const gate = options.gate ?? (() => true); const scQuery = async (timeoutMs: number) => parseScQueryState(await run("sc", ["query", WINSW_SERVICE_ID], timeoutMs)); + const scQcRun = options.scQcRun ?? (async (timeoutMs: number) => run("sc", ["qc", WINSW_SERVICE_ID], timeoutMs)); return { canonicalPaths: windowsStopCanonicalPaths(), childPid: windowsStopChildPid(), @@ -239,7 +269,18 @@ export function windowsCheckedStopLiveDeps(options: WindowsCheckedStopLiveOption if (listing.status !== 0 || listing.timedOut) return { status: "unknown", detail: "the Task Scheduler query did not answer" }; return windowsSchedulerCsvIncludesTask(listing.stdout, TASK) ? { status: "present" } : { status: "absent" }; }, - probeWinsw: async timeoutMs => winswStatusFromScState(await scQuery(timeoutMs)), + async probeWinsw(timeoutMs) { + const state = await scQuery(timeoutMs); + const status = winswStatusFromScState(state); + // Absence, and anything already unreadable, needs no ownership read. + if (status !== "started") return status; + const started = Date.now(); + const qc = await scQcRun(Math.max(1, timeoutMs - (Date.now() - started))); + const ownership = classifyWinswOwnership(qc); + // Foreign is `nonexistent` for Part A: a manager this home must not stop. + if (ownership === "foreign") return "nonexistent"; + return ownership === "unknown" ? "unknown" : status; + }, enumerate: timeoutMs => (options.snapshot ?? windowsStopSnapshot)(timeoutMs), scheduleServicePid: timeoutMs => { const whole = wholeTimeoutMs(timeoutMs); diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts index 75576917ae4..32564ff338a 100644 --- a/tests/windows/windows-checked-stop-live.test.ts +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -19,9 +19,11 @@ import { windowsStopSnapshot, windowsStopTerminate, winswStatusFromScState, + windowsCheckedStopLiveDeps, WINDOWS_ARM64_BOUNDED_TERMINATE_ENABLED, } from "../../src/service/windows-checked-stop-live"; import type { WindowsProcessNativeApi } from "../../src/service/windows-process-native"; +import { winswExePath } from "../../src/lib/winsw"; import { PATHS, entry, launcher, loop, proxy, supervisedTree } from "../helpers/windows-checked-stop-fixtures"; const result = (status: number | null, stdout = "", stderr = "", timedOut = false) => ({ status, stdout, stderr, timedOut }); @@ -169,3 +171,49 @@ describe("who may run Part A: the stopper's ancestry", () => { expect(stopperUnderHomeWrapper(shell, PATHS, 61)).toBe(false); }); }); + +describe("WinSW ownership before Part A stops it (#7771 follow-up)", () => { + // `sc stop opencodex-proxy-native` names a fixed ID. A registration that runs another + // installation's WinSW executable must never be stopped by this home's Part A. + const qc = (binaryPathName: string) => result(0, [ + "SERVICE_NAME: opencodex-proxy-native", + " TYPE : 10 WIN32_OWN_PROCESS", + " BINARY_PATH_NAME : " + binaryPathName, + ].join("\r\n")); + + const probe = (answers: { query: ReturnType; qc?: ReturnType }) => + windowsCheckedStopLiveDeps({ + run: async program => { + if (program !== "sc") throw new Error("unexpected program " + program); + return answers.query; + }, + scQcRun: async () => answers.qc ?? result(1), + }).probeWinsw(2_000); + + test("a service registered against another installation is not this home's manager", async () => { + const foreign = "C:\\Users\\other\\.opencodex\\winsw\\opencodex-proxy-native.exe"; + expect(await probe({ query: result(0, " STATE : 4 RUNNING"), qc: qc(foreign) })) + .toBe("nonexistent"); + }); + + test("this home's registered executable is the manager Part A may stop", async () => { + expect(await probe({ query: result(0, " STATE : 4 RUNNING"), qc: qc(`"${winswExePath()}"`) })) + .toBe("started"); + }); + + test("an unreadable registration fails closed instead of claiming or stopping it", async () => { + expect(await probe({ query: result(0, " STATE : 4 RUNNING"), qc: result(5, "", "Access is denied.") })) + .toBe("unknown"); + expect(await probe({ query: result(0, " STATE : 4 RUNNING") })).toBe("unknown"); + }); + + test("a service that does not exist needs no ownership read", async () => { + let qcCalls = 0; + const status = await windowsCheckedStopLiveDeps({ + run: async () => result(1060, "[SC] EnumQueryServicesStatus:OpenService FAILED 1060:"), + scQcRun: async () => { qcCalls++; return result(0); }, + }).probeWinsw(2_000); + expect(status).toBe("nonexistent"); + expect(qcCalls).toBe(0); + }); +}); From 29f862da90a38856b319a5d346edddf38b85b038 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:51:23 +0900 Subject: [PATCH 26/53] fix(server): clear only an exited owner's TCP rows on the stopped port Part B's sampler cleared every IPv4 TCP row on the port once the listening PID was found exited. A program that took the port between scans owns its rows, so that reset could break a connection this stop does not own. parseTcpQuadsForLocalPort and dropWindowsTcpRowsForLocalPort accept an owner PID filter, and the sampler passes the exited PID(s); the hard-kill path keeps clearing the whole port. Refs #7771 --- src/server/windows-tcp-drop.ts | 25 ++++++++++++++++--- src/service/windows-checked-stop-live.ts | 10 +++++--- tests/server/port-reclaim.test.ts | 14 +++++++++++ .../windows/windows-checked-stop-live.test.ts | 9 ++++--- 4 files changed, 48 insertions(+), 10 deletions(-) diff --git a/src/server/windows-tcp-drop.ts b/src/server/windows-tcp-drop.ts index 518aae24904..0b646abbb20 100644 --- a/src/server/windows-tcp-drop.ts +++ b/src/server/windows-tcp-drop.ts @@ -13,8 +13,14 @@ export type TcpQuad = { state: string; }; -/** Parse netstat -ano rows whose local address uses `port`. Exported for tests. */ -export function parseTcpQuadsForLocalPort(output: string, port: number): TcpQuad[] { +/** + * Parse netstat -ano rows whose local address uses `port`. Exported for tests. + * + * `ownerPids` restricts the rows to those the listed processes own. A row left by a process + * that has exited is that process's to clear; a program that took the port between scans owns + * its own rows, and resetting them would break a connection this stop does not own. + */ +export function parseTcpQuadsForLocalPort(output: string, port: number, ownerPids?: readonly number[]): TcpQuad[] { const rows: TcpQuad[] = []; const portSuffix = `:${port}`; for (const rawLine of output.split(/\r?\n/)) { @@ -26,6 +32,10 @@ export function parseTcpQuadsForLocalPort(output: string, port: number): TcpQuad const remote = parts[2]!; const state = parts[3]!; if (!local.endsWith(portSuffix) && !local.endsWith(`]:${port}`)) continue; + if (ownerPids !== undefined) { + const rowPid = Number(parts[4]); + if (!Number.isSafeInteger(rowPid) || !ownerPids.includes(rowPid)) continue; + } const localAddr = stripPort(local, port); const remoteParsed = splitHostPort(remote); if (!localAddr || !remoteParsed) continue; @@ -134,10 +144,17 @@ function readNetstatAno(): string { /** * Force-delete IPv4 TCP rows bound to `localPort` via SetTcpEntry(DELETE_TCB). * Does not kill foreign processes — only resets sockets so the listen port can bind again. + * `ownerPids` restricts the deletion to rows those processes own (see + * parseTcpQuadsForLocalPort); omitted, every row on the port is cleared, which is what the + * hard-kill path wants and what a stale-port sweep must not do. * Bare IPv6 rows (`::1`, `::`, etc.) are skipped (not coerced into IPv4 wildcards); * IPv6 TCB reclamation is unsupported on this path. */ -export function dropWindowsTcpRowsForLocalPort(port: number, mayMutate?: () => boolean): WindowsTcpDropResult { +export function dropWindowsTcpRowsForLocalPort( + port: number, + mayMutate?: () => boolean, + ownerPids?: readonly number[], +): WindowsTcpDropResult { if (process.platform !== "win32" || !Number.isFinite(port) || port <= 0) { return { dropped: 0, skippedIpv6: 0, accessDenied: 0 }; } @@ -151,7 +168,7 @@ export function dropWindowsTcpRowsForLocalPort(port: number, mayMutate?: () => b return { dropped: 0, skippedIpv6: 0, accessDenied: 0 }; } - const rows = parseTcpQuadsForLocalPort(output, Math.trunc(port)); + const rows = parseTcpQuadsForLocalPort(output, Math.trunc(port), ownerPids); let dropped = 0; let skippedIpv6 = 0; let accessDenied = 0; diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts index c0242e66d00..c3d71a9e75b 100644 --- a/src/service/windows-checked-stop-live.ts +++ b/src/service/windows-checked-stop-live.ts @@ -375,7 +375,8 @@ export async function sampleWindowsStopPort( io: { scan?: WindowsStopPortScan; alive?: (pid: number) => boolean; - drop?: (port: number) => unknown; + /** Clears the ghost rows of the exited owners only; never another program's rows. */ + drop?: (port: number, ownerPids: readonly number[]) => unknown; selfPid?: number; } = {}, ): Promise { @@ -388,8 +389,11 @@ export async function sampleWindowsStopPort( const alive = io.alive ?? isProcessAlive; const live = pids.filter(pid => alive(pid)); if (live.length === 0) { - // Same clearing the 15 s waitForStoppedPort did: a ghost row left by the exited child. - try { (io.drop ?? dropWindowsTcpRowsForLocalPort)(port); } catch { /* the next sample decides */ } + // Clear only the ghost rows the exited PID(s) left: a program that took the port after + // the scan owns its rows, and resetting them would break a connection this stop does not own. + try { + (io.drop ?? ((target: number, owners: readonly number[]) => dropWindowsTcpRowsForLocalPort(target, undefined, owners)))(port, pids); + } catch { /* the next sample decides */ } } return { state: "listening", ownerPid: live[0] ?? pids[0] ?? null }; } diff --git a/tests/server/port-reclaim.test.ts b/tests/server/port-reclaim.test.ts index 21a26984d40..4b02092eaba 100644 --- a/tests/server/port-reclaim.test.ts +++ b/tests/server/port-reclaim.test.ts @@ -467,6 +467,20 @@ describe("parseTcpQuadsForLocalPort / IPv6", () => { ]); }); + test("an owner filter keeps only the rows the named processes own", () => { + const output = [ + " TCP 127.0.0.1:10100 0.0.0.0:0 LISTENING 18268", + " TCP 127.0.0.1:10100 127.0.0.1:60001 ESTABLISHED 18268", + " TCP 127.0.0.1:10100 127.0.0.1:60002 ESTABLISHED 20001", + ].join("\n"); + // The exited proxy's rows are cleared; the program that took the port keeps its own. + expect(parseTcpQuadsForLocalPort(output, 10100, [18268]).map(row => row.remotePort)).toEqual([0, 60001]); + expect(parseTcpQuadsForLocalPort(output, 10100, [20001]).map(row => row.remotePort)).toEqual([60002]); + expect(parseTcpQuadsForLocalPort(output, 10100, [])).toEqual([]); + // No filter keeps every row, which is what the hard-kill path wants. + expect(parseTcpQuadsForLocalPort(output, 10100)).toHaveLength(3); + }); + test("keeps IPv6 local rows parseable without coercing them into IPv4 wildcards", () => { const output = [ " TCP [::1]:10100 [::]:0 LISTENING 18268", diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts index 32564ff338a..9d9b2c46782 100644 --- a/tests/windows/windows-checked-stop-live.test.ts +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -104,10 +104,13 @@ describe("Part B port sampler and listener identity", () => { expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: false }) })).toEqual({ state: "unknown" }); expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => true })) .toEqual({ state: "listening", ownerPid: 77 }); - const drops: number[] = []; - expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => false, drop: port => drops.push(port) })) + const drops: Array<[number, readonly number[]]> = []; + expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [77] }), alive: () => false, drop: (port, pids) => drops.push([port, pids]) })) .toEqual({ state: "listening", ownerPid: 77 }); - expect(drops).toEqual([10100]); + // Only the exited owner's rows may be cleared: a program that took the port between + // scans owns its own rows, and resetting them would break a connection this stop does + // not own. + expect(drops).toEqual([[10100, [77]]]); // The stopping process's own socket is not a listener on the stopped port. expect(await sampleWindowsStopPort(10100, 400, { scan: () => ({ ok: true, pids: [5] }), selfPid: 5 })).toEqual({ state: "free" }); }); From 8319c316040fe389f166d6f2c13762edab3ecea4 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:53:07 +0900 Subject: [PATCH 27/53] docs: name repair as the remaining stop exception and record the wiring decisions structure/ops/windows-checked-stop.md claimed every Windows stop path uses the checked stop; ocx service repair still stops through stopWindows() (repair.ts:259) and does not restore shared state, so it is the remaining exception. 060 records it as its own follow-up slice and captures the decisions from the review round (uninstall receipt, no forced fallback, absent Part A, resolved stop port, WinSW ownership, owner-filtered TCP drop), and 030 gains the WinSW ownership probe. Refs #7771 --- .../261111_windows_service_stop/030_design.md | 2 +- .../060_implementation.md | 11 ++++++++ structure/ops/windows-checked-stop.md | 27 +++++++++++++------ 3 files changed, 31 insertions(+), 9 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index e43547f430a..56f01ae5008 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -50,7 +50,7 @@ New work, named explicitly (x64): the creation-time reader, the handle terminate Part A: -1. Probe registration: `probeWindowsSchedulerTask` and, for WinSW, the service status (both capped at 2 s; an overrun or unreadable answer is `unknown`, nothing changed). With nothing registered, the process enumeration that looks for a stray wrapper of a deleted task (capped at 5 s) also runs here, before admission. Nothing registered and nothing bound → `absent`, return. +1. Probe registration: `probeWindowsSchedulerTask` and, for WinSW, the service status (both capped at 2 s; an overrun or unreadable answer is `unknown`, nothing changed). A running WinSW service is only this home's manager when its registered `BINARY_PATH_NAME` (`sc qc`) names this installation's WinSW executable: the service ID is fixed, so a successful `sc query` alone would let Part A stop another installation's service. An unreadable registration is `unknown`; a foreign executable is treated as not registered for this home. With nothing registered, the process enumeration that looks for a stray wrapper of a deleted task (capped at 5 s) also runs here, before admission. Nothing registered and nothing bound → `absent`, return. 2. Bind (rule above) and record (pid, startTime, image, command-line digest). 3. Ask the manager. Task Scheduler: `schtasks /end` (cap 2 s). WinSW: a non-waiting stop (the stop command capped at 2 s instead of today's 15 s `SERVICE_COMMAND_TIMEOUT_MS`) followed by a capped SCM state poll (`sc query`) instead of today's `stopwait`; if WinSW reports stopped but the proxy is still alive, the outcome is `survived`; the poll's cap is the service's `` (20 s in the generated definition) plus 2 s, because WinSW itself stops the proxy within its stop timeout. Access denied → `denied`, stop there. A cap overrun → `unknown`, with nothing further changed (no terminate). 4. Terminate bound wrapper survivors through the handle primitive (`cmd.exe` running the loop first, then the launcher). WinSW has no wrapper, so this step is empty there. The proxy child is NOT terminated here: it stops through the existing graceful path (`stopProxy` / `stopWithDeferral` / `signalApproved`) with its drain, the #7423 teardown-deferral receipt, and the rule that a received refusal (409, `ProxyOwnershipRefusedError`) forbids a forced stop. It is force-killed only where today's code already forces after a graceful timeout, never after a refusal. This operation gives no new kill authority over the child. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 9ea0519469b..a26848fb3ed 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -77,6 +77,17 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **The Part B port sample honors its timeout.** `sampleWindowsStopPort` called the synchronous `scanListenPids` and ignored `timeoutMs`, so the closing sample could stall the stop past its cap. It is now async over `netstat.exe` through `execFile` with the caller's timeout and `SIGKILL`; a timeout or an unreadable scan samples `unknown`, and the timeout is floored because `execFile` reads 0 as unlimited. - **A not-running task needs its registration proof before `bound-stray`.** The state word says the task is inert, not that it is ours; promoting on the word alone let a same-named foreign task make this home's hand-started wrappers terminable. The promotion now follows `winTaskXml` and `winRegistrationOurs`, and anything else keeps the previous verdict. +- **Uninstall's proxy stop claims its receipt before the child stop.** `handleUninstall` sent `/api/stop` without a teardown receipt, so the route finished the shared teardown while the wrapper respawn window had not been checked yet; a failed `verifyUninstallStopWindow` then left shared config restored under a wrapper that could relaunch the proxy. On Windows the uninstall proxy stop is now `src/cli/uninstall-stop.ts`: it claims a receipt carrying Part A's record before the child stop, sends its nonce, and discharges it only after Part B and the shared restore. A failed claim sends no child stop; a failed later step keeps the receipt for recovery. +- **Uninstall never forces a stop its Part A could not prove.** When Part A did not end `stopped` or `absent`, the uninstall proxy stop still sends the graceful POST but passes `allowForcedFallback: false` to `stopProxy`, so a POST timeout, transport error or non-409 answer cannot fall through to `taskkill /T /F`. The uninstall failure stays recorded. A 409 refusal already forbids the forced path; this closes the remaining ones. +- **An absent Part A is not proof the proxy is down.** `windowsServiceRemovalStop` returned `ok` on Part A `absent` without checking the endpoint, and `findLiveProxy` answers null when its 3 s discovery cap runs out even while the endpoint is live. Removal now requires `proxyStillLiveAfterStop({ canRespawn: false })` to report the endpoint down before it deletes the task; a live endpoint refuses removal, and an unreadable answer is not success. No respawn hold is needed: a wait would price removal at Part B's 6 s for a manager that is not registered. +- **Install preparation resolves the stop port before Part A.** `windowsServiceStopPort()` ran after the tracked cleanup had already removed the runtime record, so Part B confirmed the wrapper's baked port (10100) while the proxy's runtime port (12000) was never checked. `prepareServiceInstall` resolves the port once before Part A (`windowsStopPort` is the test seam) and carries it into Part B. +- **Part A proves the WinSW registration is this home's.** `sc query` answering "running" says a service exists under the fixed `opencodex-proxy-native` ID, not that it is this installation's. The live probe now reads `BINARY_PATH_NAME` through `sc qc` and requires this home's `winswExePath()`, the same proof `inspectWindowsWinswManager` uses; an unreadable registration is `unknown` (`evidence-unreadable`, nothing changed) and a foreign executable is reported as `nonexistent`, so Part A does not stop it and reports `absent`. +- **Part B clears only the exited owner's TCP rows.** After a listener PID was found exited, the sampler called `dropWindowsTcpRowsForLocalPort(port)`, which resets every IPv4 row on the port — including rows of a program that took the port between scans. `parseTcpQuadsForLocalPort` and `dropWindowsTcpRowsForLocalPort` take an owner PID filter and the sampler passes the exited PID(s); the hard-kill path still clears the whole port. + +## Follow-up slices + +- **`ocx service repair` still stops through `stopWindows()`** (`src/service/repair.ts:259`), the unguarded `schtasks /end` path. Repair does not restore shared client state, so the #3008 relaunch window the checked stop closes does not apply and it is not wired here. Its own slice: decide whether repair needs Part A (it re-registers the task immediately after, so a surviving wrapper can be rebound by the new definition) and, if so, which Part B applies when repair then starts the task again. + ## Open questions - Is the closing-sample accounting acceptable, or should the hold be measured differently, for example by shortening the final enumeration to 4.5 s to keep 030's totals? diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index e3eeb9a0f04..d290c81e73f 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -2,12 +2,16 @@ The checked Windows service stop (#7771) is a library of primitives and two operations that stop a Task Scheduler or WinSW manager, end the wrappers that would relaunch the proxy, and -then confirm the port is free. Every Windows stop path uses it: `ocx stop` with and without a -consent snapshot, `ocx uninstall`, `/api/stop`, `ocx service stop` / `uninstall` / `remove`, -service install and backend switch, and the updater. The best-effort path-token sweep in -`src/lib/windows-service-wrappers.ts` is no longer called by any stop path. macOS and Linux keep -their manager stops and timing; only the signatures turned async. The design and its slices live -in `devlog/_plan/261111_windows_service_stop/`. +then confirm the port is free. Every Windows stop path that also restores shared client state +uses it: `ocx stop` with and without a consent snapshot, `ocx uninstall`, `/api/stop`, +`ocx service stop` / `uninstall` / `remove`, service install and backend switch, and the updater. +The best-effort path-token sweep in `src/lib/windows-service-wrappers.ts` is no longer called by +any stop path. `ocx service repair` is the one remaining exception: it still stops the manager +through `stopWindows()` (`src/service/repair.ts:259`) because it does not restore shared client +state, so the #3008 relaunch window this stop exists to close does not apply to it. Wiring repair +is its own follow-up slice (`devlog/_plan/261111_windows_service_stop/060_implementation.md`, +"Follow-up slices"). macOS and Linux keep their manager stops and timing; only the signatures +turned async. The design and its slices live in `devlog/_plan/261111_windows_service_stop/`. ## Process evidence @@ -102,13 +106,20 @@ floored to whole milliseconds and one that floors to 0 spawns nothing, because ` as no timeout), the evidence snapshot `windowsProcessSnapshot` for every bind, the checked terminate, the exit watch, the Part B port sampler and the listener identity. The task's running state is read from `schtasks /query /fo CSV`, falling back to `Get-ScheduledTask` inside the same -1 s cap when the status word is localized. +1 s cap when the status word is localized. A running WinSW service is only this home's manager +when `sc qc` shows a `BINARY_PATH_NAME` naming this installation's WinSW executable, the same +proof the guarded manager path uses: the service ID is fixed, so a successful `sc query` alone +would let Part A stop another installation's service. An unreadable registration is `unknown` +(`evidence-unreadable`, nothing changed); a foreign executable is reported as `nonexistent`, so +Part A stops nothing and answers `absent`. The port sampler is `sampleWindowsStopPort(port, timeoutMs)`, async and bounded by the caller's timeout: it runs `netstat.exe` from the trusted System32 directory through `execFile` with that timeout and `SIGKILL`, and a scan that misses the timeout, or cannot be read, samples `unknown`. A synchronous scan would ignore the cap and could stall the stop for the helper's own 5 s. A row -whose owner has exited is cleared like `reclaimListenPort` does. The listener identity is +whose owner has exited is cleared like `reclaimListenPort` does, but only the rows that exited +PID owns: `dropWindowsTcpRowsForLocalPort` takes an owner filter, so a program that took the port +between scans keeps its connections. The listener identity is tri-state over `inspectOcxStartProcessIdentity`: only a positive `mismatch` is `foreign`, and an `unverifiable` probe (another account's or an elevated process's command line) stays `unknown`. `verifyPidIdentity`'s null collapses those two, which would report a listener we merely could not From 70dd91ccb20a66d23643c3bda418e83f5adba5c6 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 06:57:57 +0900 Subject: [PATCH 28/53] test(service): follow the uninstall stop into its helper when pinning the transport default --- tests/service/stop-direct-fetch.test.ts | 40 ++++++++++++++++++++----- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/tests/service/stop-direct-fetch.test.ts b/tests/service/stop-direct-fetch.test.ts index 9ff9dfc3297..048a310277e 100644 --- a/tests/service/stop-direct-fetch.test.ts +++ b/tests/service/stop-direct-fetch.test.ts @@ -4,6 +4,28 @@ import { createServer, type Socket } from "node:net"; import { stopProxyGracefully } from "../../src/lib/process-control"; import { repoPath } from "../helpers/repo-root"; +/** Argument text of each `await (...)`, balanced so nested calls stay separate. */ +function callArguments(body: string, call: string): string[] { + const marker = `await ${call}(`; + const found: string[] = []; + let from = 0; + for (;;) { + const at = body.indexOf(marker, from); + if (at < 0) return found; + let depth = 0; + let index = at + marker.length - 1; + for (; index < body.length; index += 1) { + if (body[index] === "(") depth += 1; + else if (body[index] === ")") { + depth -= 1; + if (depth === 0) break; + } + } + found.push(body.slice(at + marker.length, index)); + from = index; + } +} + const stopRunner = ` const { stopProxyGracefully } = await import("./src/lib/process-control.ts"); const port = Number(process.env.OCX_TEST_STOP_PORT); @@ -108,10 +130,12 @@ describe("local stop transport", () => { // These paths mutate service/process state; exercise their shared real transport above // and verify each caller delegates without overriding it with the global fetch. - for (const [file, functionName, count] of [ - ["src/cli/index.ts", "handleStopUnlocked", 1], - ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 2], - ["src/cli/index.ts", "handleUninstall", 2], + // `handleUninstall` delegates its two stops to `stopUninstallProxy` (#7771), which owns + // the receipt and the no-forced-fallback rule; the transport default is still not overridden. + for (const [file, functionName, count, call] of [ + ["src/cli/index.ts", "handleStopUnlocked", 1, "stopProxy"], + ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 2, "stopProxy"], + ["src/cli/uninstall-stop.ts", "stopUninstallProxy", 2, "io.stop"], ] as const) { test(`${functionName} retains the direct stop default`, () => { const source = readFileSync(repoPath(file), "utf8"); @@ -119,12 +143,12 @@ describe("local stop transport", () => { expect(start).toBeGreaterThanOrEqual(0); const end = source.indexOf("\n}", start); expect(end).toBeGreaterThan(start); - const calls = [...source.slice(start, end).matchAll(/await stopProxy\(([\s\S]*?)\);/g)]; + const calls = callArguments(source.slice(start, end), call); expect(calls).toHaveLength(count); - for (const call of calls) { + for (const args of calls) { // Connected stops also retain the shared default's refusal-status handling. - expect(call[1]!).not.toMatch(/\bfetchFn\s*:/); - expect(call[1]!.replace(/as typeof fetch\b/g, "")).not.toMatch(/\bfetch\b/); + expect(args).not.toMatch(/\bfetchFn\s*:/); + expect(args.replace(/as typeof fetch\b/g, "")).not.toMatch(/\bfetch\b/); } }); } From 399e78100a32c823d5a69410632f296226744870 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 07:02:21 +0900 Subject: [PATCH 29/53] fix(service): the WinSW ownership read shares one cap with the state query The ownership read's budget was measured from after the state query, so the two reads together could take twice the probe's cap. Measure from before the state query and answer unknown when nothing is left. Refs #7771 --- src/service/windows-checked-stop-live.ts | 7 +++++-- .../windows/windows-checked-stop-live.test.ts | 20 +++++++++++++++++++ 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/src/service/windows-checked-stop-live.ts b/src/service/windows-checked-stop-live.ts index c3d71a9e75b..1580c109834 100644 --- a/src/service/windows-checked-stop-live.ts +++ b/src/service/windows-checked-stop-live.ts @@ -270,12 +270,15 @@ export function windowsCheckedStopLiveDeps(options: WindowsCheckedStopLiveOption return windowsSchedulerCsvIncludesTask(listing.stdout, TASK) ? { status: "present" } : { status: "absent" }; }, async probeWinsw(timeoutMs) { + // One cap covers both reads: the ownership query gets only what the state query left. + const started = Date.now(); const state = await scQuery(timeoutMs); const status = winswStatusFromScState(state); // Absence, and anything already unreadable, needs no ownership read. if (status !== "started") return status; - const started = Date.now(); - const qc = await scQcRun(Math.max(1, timeoutMs - (Date.now() - started))); + const remaining = Math.floor(timeoutMs - (Date.now() - started)); + if (remaining < 1) return "unknown"; + const qc = await scQcRun(remaining); const ownership = classifyWinswOwnership(qc); // Foreign is `nonexistent` for Part A: a manager this home must not stop. if (ownership === "foreign") return "nonexistent"; diff --git a/tests/windows/windows-checked-stop-live.test.ts b/tests/windows/windows-checked-stop-live.test.ts index 9d9b2c46782..9f6b05ab491 100644 --- a/tests/windows/windows-checked-stop-live.test.ts +++ b/tests/windows/windows-checked-stop-live.test.ts @@ -210,6 +210,26 @@ describe("WinSW ownership before Part A stops it (#7771 follow-up)", () => { expect(await probe({ query: result(0, " STATE : 4 RUNNING") })).toBe("unknown"); }); + test("the ownership read gets only the time the state query left of one cap", async () => { + const running = result(0, " STATE : 4 RUNNING"); + let qcTimeout = -1; + const status = await windowsCheckedStopLiveDeps({ + run: async () => { await Bun.sleep(120); return running; }, + scQcRun: async timeoutMs => { qcTimeout = timeoutMs; return qc(`"${winswExePath()}"`); }, + }).probeWinsw(400); + expect(status).toBe("started"); + expect(qcTimeout).toBeGreaterThan(0); + expect(qcTimeout).toBeLessThanOrEqual(400 - 100); + // A state query that spent the whole cap leaves nothing: unknown, and no ownership read. + let qcCalls = 0; + const spent = await windowsCheckedStopLiveDeps({ + run: async () => { await Bun.sleep(60); return running; }, + scQcRun: async () => { qcCalls++; return qc(`"${winswExePath()}"`); }, + }).probeWinsw(40); + expect(spent).toBe("unknown"); + expect(qcCalls).toBe(0); + }); + test("a service that does not exist needs no ownership read", async () => { let qcCalls = 0; const status = await windowsCheckedStopLiveDeps({ From 1f7381b6401a68b152840dc5e6102288848d0cb2 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 07:11:14 +0900 Subject: [PATCH 30/53] fix(lib): keep stopProxy's forced kill a literal killProxy(pid) call The forced-fallback seam rewrote the call as (io.forceKill ?? killProxy)(pid), which hid it from the source check that the 409 refusal is handled before any kill (grok-lifecycle 'a 409 does not escalate to a forced kill'). Call killProxy(pid) literally and use the seam only when a test injects it. Refs #7771 --- src/lib/process-control.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/lib/process-control.ts b/src/lib/process-control.ts index 38b20e66283..72ae2f1a15c 100644 --- a/src/lib/process-control.ts +++ b/src/lib/process-control.ts @@ -366,7 +366,10 @@ export async function stopProxy(pid: number, io: GracefulStopIo = {}): Promise Date: Fri, 13 Nov 2026 08:02:46 +0900 Subject: [PATCH 31/53] fix(cli): resolve the uninstall stop target before claiming its receipt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `stopUninstallProxy` claimed the deferred-teardown receipt from `defaultReceiptEndpoint()` before it knew what it would stop. That helper marked any runtime record "exact", so a stale record plus a proxy discovered on another port produced a receipt naming an endpoint the stop never contacted; and when Part A failed with no proxy at all, a "guessed" receipt was left outstanding that recovery refuses to finish. The target is now resolved first — the tracked pid with its own runtime record, or the discovered live proxy's endpoint — and only then is a receipt claimed, with the same endpoint handed to `stopProxy`. No target means no receipt. The tracked record is removed only for a stop that was proven, and the step reports `changed` only when state actually changed. Refs #7771 --- src/cli/uninstall-stop.ts | 94 ++++++++++++---------- tests/cli/uninstall.test.ts | 100 ++++++++++++++++++++++-- tests/service/stop-direct-fetch.test.ts | 4 +- 3 files changed, 150 insertions(+), 48 deletions(-) diff --git a/src/cli/uninstall-stop.ts b/src/cli/uninstall-stop.ts index 9cb83706442..b2a7ce5f1dc 100644 --- a/src/cli/uninstall-stop.ts +++ b/src/cli/uninstall-stop.ts @@ -17,7 +17,6 @@ * `taskkill /T /F` under a manager that would not stop is the one outcome the checked stop * exists to prevent. */ -import { loadConfig } from "../config"; import { claimPendingTeardown, type PendingTeardownPartA } from "../config/pending-teardown"; import { readRuntimePort } from "../config/process-state"; import type { GracefulStopIo } from "../lib/process-control"; @@ -27,7 +26,7 @@ export interface UninstallProxyStopIo { /** Detailed Part A outcome, or null when the service-stop step threw. */ serviceStop: "absent" | "stopped" | "stopped-respawnable" | "failed" | "state-unknown" | null; readPid: () => number | null; - findLiveProxy: () => Promise<{ pid?: number | null; port: number } | null>; + findLiveProxy: () => Promise<{ pid?: number | null; port: number; hostname?: string } | null>; /** Tri-state proof that nothing answers on any endpoint this home could serve. */ endpointProvenDown: () => Promise; /** The Part A record the receipt carries, so /api/stop skips a second Part A. */ @@ -39,7 +38,8 @@ export interface UninstallProxyStopIo { handover: "update-restart" | undefined, partA: PendingTeardownPartA | null, ) => { nonce: string }; - receiptEndpoint?: () => { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" }; + /** The tracked target's own runtime record; null when there is none for that pid. */ + runtimeEndpointFor?: (pid: number) => { hostname: string; port: number } | null; stop: (pid: number, io: GracefulStopIo) => Promise; removeTracked: (pid: number) => void; /** Whether Part B owns the Windows port wait (#7771). */ @@ -53,19 +53,10 @@ export interface UninstallProxyStopResult { nonce?: string; } -/** The endpoint a receipt records: the runtime record when there is one, else the config. */ -function defaultReceiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { - const runtime = readRuntimePort(); - if (runtime?.port) return { endpoint: { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port }, source: "exact" }; - try { - const config = loadConfig(); - return { - endpoint: { hostname: config.hostname ?? "127.0.0.1", port: typeof config.port === "number" && config.port > 0 ? config.port : 10100 }, - source: "guessed", - }; - } catch { - return { endpoint: { hostname: "127.0.0.1", port: 10100 }, source: "guessed" }; - } +/** The runtime record a tracked pid wrote, which is the only exact endpoint it can be reached on. */ +function trackedRuntimeEndpoint(pid: number): { hostname: string; port: number } | null { + const runtime = readRuntimePort(pid); + return runtime?.port ? { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port } : null; } export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise { @@ -73,33 +64,21 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise null))()).nonce; - } catch (error) { - // Without the receipt the proxy performs its own teardown during the stop, which is - // the pre-#3008 behaviour — and on Windows a wrapper can relaunch it afterwards, so - // client config would already be restored under a live proxy. Refuse instead. - throw new Error( - "the deferred-teardown receipt could not be recorded, so the proxy was left running: " - + (error instanceof Error ? error.message : String(error)), - ); - } - } const forbidForcedFallback = windows && !partAProven; - const stopIo: GracefulStopIo = { - skipPortWait: io.portWaitOwned(), - ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), - ...(forbidForcedFallback ? { allowForcedFallback: false } : {}), - }; // An unconfirmed stop normally still leaves this step's proof to the respawn window that // follows. With the forced ladder forbidden it must not: the process may be alive, and the // Part A failure the caller already recorded is what keeps shared config up. const confirmed = (stopped: boolean): boolean => stopped || !forbidForcedFallback; + + // Resolve the target BEFORE anything is recorded. The receipt is an obligation on a + // specific address, so claiming it from a runtime record the stop never contacts — or + // before knowing whether a stop will happen at all — writes a promise nobody can + // discharge: recovery probes exactly the endpoint the receipt names. + let target: { pid: number; tracked: boolean; endpoint: { hostname: string; port: number } | null }; const pid = io.readPid(); - if (!pid) { + if (pid) { + target = { pid, tracked: true, endpoint: (io.runtimeEndpointFor ?? trackedRuntimeEndpoint)(pid) }; + } else { // A missing pid file is not proof that nothing is serving: a proxy can outlive its // record (crash, manual delete, corrupt file), which is exactly why `ocx stop` falls // back to identity-checked discovery. Without this, uninstall restored shared config @@ -113,14 +92,45 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise null))()).nonce; + } catch (error) { + // Without the receipt the proxy performs its own teardown during the stop, which is + // the pre-#3008 behaviour — and on Windows a wrapper can relaunch it afterwards, so + // client config would already be restored under a live proxy. Refuse instead. + throw new Error( + "the deferred-teardown receipt could not be recorded, so the proxy was left running: " + + (error instanceof Error ? error.message : String(error)), + ); + } } - const provenDown = confirmed(await io.stop(pid, stopIo)); - io.removeTracked(pid); - return { changed: true, provenDown, ...(nonce ? { nonce } : {}) }; + const stopIo: GracefulStopIo = { + skipPortWait: io.portWaitOwned(), + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + ...(forbidForcedFallback ? { allowForcedFallback: false } : {}), + // The same endpoint the receipt names, so the POST and the obligation cannot diverge. + ...(target.endpoint ? { runtimeEndpoint: target.endpoint } : {}), + }; + const provenDown = confirmed(await io.stop(target.pid, stopIo)); + // Only a proven stop cleared the record it names. Removing it after a POST that timed out + // drops the only pointer to a process that may still be running, and reporting the step as + // changed would claim a state that was never observed. `provenDown` is how the caller + // learns it must record the failure (it keeps the shared teardown up). + if (target.tracked && provenDown) io.removeTracked(target.pid); + return { changed: target.tracked ? provenDown : true, provenDown, ...(nonce ? { nonce } : {}) }; } diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 0c38e508a5e..3d49d746afa 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -857,6 +857,7 @@ describe("uninstall proxy stop defers the shared teardown through a receipt (#77 platform: "win32", serviceStop: "stopped", readPid: () => 4242, + runtimeEndpointFor: pid => (pid === 4242 ? { hostname: "127.0.0.1", port: 10100 } : null), findLiveProxy: async () => null, endpointProvenDown: async () => true, claim: () => ({ nonce: NONCE }), @@ -875,7 +876,7 @@ describe("uninstall proxy stop defers the shared teardown through a receipt (#77 removeTracked: pid => { steps.push("remove:" + pid); }, })); expect(steps).toEqual(["claim", "stop:4242", "remove:4242"]); - expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE }]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }]); expect(result).toEqual({ changed: true, provenDown: true, nonce: NONCE }); }); @@ -888,7 +889,7 @@ describe("uninstall proxy stop defers the shared teardown through a receipt (#77 stop: async (pid, stopIo) => { seen.push([pid, stopIo]); return true; }, removeTracked: pid => { removed.push(pid); }, })); - expect(seen).toEqual([[777, { skipPortWait: true, deferSharedTeardownNonce: NONCE }]]); + expect(seen).toEqual([[777, { skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 12000 } }]]); expect(removed).toEqual([]); expect(result.nonce).toBe(NONCE); }); @@ -897,14 +898,17 @@ describe("uninstall proxy stop defers the shared teardown through a receipt (#77 for (const serviceStop of ["failed", "state-unknown", null] as const) { const seen: unknown[] = []; await stopUninstallProxy(io({ serviceStop, stop: async (_pid, stopIo) => { seen.push(stopIo); return false; } })); - expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, allowForcedFallback: false }]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, allowForcedFallback: false, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }]); } // A Part A that ended stopped (or found nothing) keeps today's fallback available. const allowed: unknown[] = []; for (const serviceStop of ["stopped", "absent"] as const) { await stopUninstallProxy(io({ serviceStop, stop: async (_pid, stopIo) => { allowed.push(stopIo); return false; } })); } - expect(allowed).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE }, { skipPortWait: true, deferSharedTeardownNonce: NONCE }]); + expect(allowed).toEqual([ + { skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }, + { skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }, + ]); }); test("a forbidden fallback that does not confirm withholds the proof of a stopped proxy", async () => { @@ -939,10 +943,96 @@ describe("uninstall proxy stop defers the shared teardown through a receipt (#77 stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, })); expect(claimed).toBe(0); - expect(seen).toEqual([{ skipPortWait: false }]); + expect(seen).toEqual([{ skipPortWait: false, runtimeEndpoint: { hostname: "127.0.0.1", port: 10100 } }]); expect(result).toEqual({ changed: true, provenDown: true }); }); + test("the receipt names the endpoint the stop actually contacts (tracked pid)", async () => { + // #7771 round 8: the receipt used to be claimed from `readRuntimePort()` with no + // expectation, so a stale record could name an endpoint the stop never contacted, and + // `stopProxy` re-read the record on its own. The receipt and the POST now carry the one + // endpoint resolved for the target. + const claimed: unknown[] = []; + const seen: unknown[] = []; + await stopUninstallProxy(io({ + runtimeEndpointFor: pid => (pid === 4242 ? { hostname: "127.0.0.1", port: 12000 } : null), + claim: (endpoint, source) => { claimed.push([endpoint, source]); return { nonce: NONCE }; }, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + expect(claimed).toEqual([[{ hostname: "127.0.0.1", port: 12000 }, "exact"]]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 12000 } }]); + }); + + test("a stale runtime record does not become the receipt for a proxy found on another port", async () => { + // The pid file is gone but a stale runtime record survives; discovery finds the live + // proxy elsewhere. Claiming the stale record marked it "exact" while the stop went to + // the discovered endpoint, so the obligation named an address nobody contacted. + const claimed: unknown[] = []; + const seen: unknown[] = []; + await stopUninstallProxy(io({ + readPid: () => null, + runtimeEndpointFor: () => ({ hostname: "127.0.0.1", port: 10999 }), + findLiveProxy: async () => ({ pid: 777, port: 12000, hostname: "127.0.0.1" }), + claim: (endpoint, source) => { claimed.push([endpoint, source]); return { nonce: NONCE }; }, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + expect(claimed).toEqual([[{ hostname: "127.0.0.1", port: 12000 }, "exact"]]); + expect(seen).toEqual([{ skipPortWait: true, deferSharedTeardownNonce: NONCE, runtimeEndpoint: { hostname: "127.0.0.1", port: 12000 } }]); + }); + + test("no target means no receipt: a guessed obligation recovery cannot resolve is not claimed", async () => { + // A missing pid file, no live proxy, and a definitively dead endpoint: nothing will be + // stopped, so nothing may be deferred. The old order claimed a "guessed" receipt first + // and left it outstanding, which recovery then refused to finish. + let claimed = 0; + const result = await stopUninstallProxy(io({ + readPid: () => null, + findLiveProxy: async () => null, + endpointProvenDown: async () => true, + claim: () => { claimed++; return { nonce: NONCE }; }, + })); + expect(claimed).toBe(0); + expect(result).toEqual({ changed: false, provenDown: true }); + }); + + test("a tracked target with no runtime record claims nothing rather than a guessed endpoint", async () => { + let claimed = 0; + const seen: unknown[] = []; + await stopUninstallProxy(io({ + runtimeEndpointFor: () => null, + claim: () => { claimed++; return { nonce: NONCE }; }, + stop: async (_pid, stopIo) => { seen.push(stopIo); return true; }, + })); + // With no endpoint there is no graceful POST to defer, so no receipt may name one. + expect(claimed).toBe(0); + expect(seen).toEqual([{ skipPortWait: true }]); + }); + + test("the tracked record is removed only for a stop that was proven", async () => { + // Keeping a record the stop did not clear leaves a live pid for the next command to + // chase; removing it on a failed POST claims a state change that did not happen. + const removed: number[] = []; + // Part A failed and the forced ladder was forbidden: nothing proves the proxy exited, so + // the record it names must stay for the next command to chase, and the step reports no + // change (the caller records the failure and keeps the shared teardown up). + const failed = await stopUninstallProxy(io({ + serviceStop: "failed", + stop: async () => false, + removeTracked: pid => { removed.push(pid); }, + })); + expect(removed).toEqual([]); + expect(failed.changed).toBe(false); + expect(failed.provenDown).toBe(false); + + const stopped = await stopUninstallProxy(io({ + stop: async () => true, + removeTracked: pid => { removed.push(pid); }, + })); + expect(removed).toEqual([4242]); + expect(stopped.changed).toBe(true); + expect(stopped.provenDown).toBe(true); + }); + test("the receipt is finished only after the shared restore ran", async () => { const { completeWindowsServiceStopReceipt } = await import("../../src/service/cli-windows-stop"); const cleared: string[] = []; diff --git a/tests/service/stop-direct-fetch.test.ts b/tests/service/stop-direct-fetch.test.ts index 048a310277e..010b57682ad 100644 --- a/tests/service/stop-direct-fetch.test.ts +++ b/tests/service/stop-direct-fetch.test.ts @@ -135,7 +135,9 @@ describe("local stop transport", () => { for (const [file, functionName, count, call] of [ ["src/cli/index.ts", "handleStopUnlocked", 1, "stopProxy"], ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 2, "stopProxy"], - ["src/cli/uninstall-stop.ts", "stopUninstallProxy", 2, "io.stop"], + // One site: the target is resolved before the receipt, so both the tracked and the + // discovered path reach the stop through the same call (#7771 round 8). + ["src/cli/uninstall-stop.ts", "stopUninstallProxy", 1, "io.stop"], ] as const) { test(`${functionName} retains the direct stop default`, () => { const source = readFileSync(repoPath(file), "utf8"); From 1f2b9dd8756a5e806ecd18afce5878839aa8c1bd Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 08:05:54 +0900 Subject: [PATCH 32/53] fix(lib): make an unconfirmed stop's port wait observational MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `waitForStoppedPort` reclaimed the port with `killOcxHolders` and `onlyKillPids: [pid]` on every path, so a stop that had forbidden the forced ladder could still terminate a verified-but-live listener through `killProxy` — the very forced stop the caller had just refused. When `allowForcedFallback` is false the wait is now observational: it runs for the same window, but with no kill eligible. The reclaim dependency is injectable (`reclaimPort`) so the rule is proven against a verified listener without signalling a process; the literal `killProxy(pid)` call in `stopProxy` stays as the refusal-ordering oracle requires. Refs #7771 --- src/lib/process-control.ts | 38 +++++++++++++---- tests/lib/process-control-graceful.test.ts | 48 ++++++++++++++++++++++ 2 files changed, 77 insertions(+), 9 deletions(-) diff --git a/src/lib/process-control.ts b/src/lib/process-control.ts index 72ae2f1a15c..ad79b814bd2 100644 --- a/src/lib/process-control.ts +++ b/src/lib/process-control.ts @@ -80,6 +80,15 @@ export interface GracefulStopIo { * process. Production callers omit it and get {@link killProxy}. */ forceKill?: (pid: number) => void; + /** + * The post-stop port wait, injectable so a test can present a verified listener without + * starting one. Production callers omit it and get `reclaimListenPort`. + */ + reclaimPort?: ( + port: number, + hostname: string, + opts: { timeoutMs?: number; intervalMs?: number; scanIntervalMs?: number; killOcxHolders?: boolean; onlyKillPids?: number[] }, + ) => Promise; } /** @@ -353,42 +362,53 @@ export async function stopProxy(pid: number, io: GracefulStopIo = {}): Promise { if (!runtime?.port) return; + const mayKill = io.allowForcedFallback !== false; try { - const { reclaimListenPort } = await import("../server/port-reclaim"); - await reclaimListenPort(runtime.port, runtime.hostname ?? "127.0.0.1", { + const reclaim = io.reclaimPort ?? (await import("../server/port-reclaim")).reclaimListenPort; + await reclaim(runtime.port, runtime.hostname ?? "127.0.0.1", { timeoutMs: 15_000, intervalMs: 100, scanIntervalMs: 500, // Only the process we just stopped — never kill a newly started twin proxy. - killOcxHolders: !!(stoppedPid && stoppedPid > 0), - onlyKillPids: stoppedPid && stoppedPid > 0 ? [stoppedPid] : [], + killOcxHolders: mayKill && !!(stoppedPid && stoppedPid > 0), + onlyKillPids: mayKill && stoppedPid && stoppedPid > 0 ? [stoppedPid] : [], }); } catch { /* best-effort — callers that need a hard guarantee reclaim again before bind */ diff --git a/tests/lib/process-control-graceful.test.ts b/tests/lib/process-control-graceful.test.ts index 9678dbe944a..451b1de9530 100644 --- a/tests/lib/process-control-graceful.test.ts +++ b/tests/lib/process-control-graceful.test.ts @@ -400,6 +400,54 @@ describe("409 refusal reporting", () => { } }); + test("a forbidden fallback's port wait never kills a verified listener", async () => { + // #7771 round 8: `waitForStoppedPort` reclaimed the port with `killOcxHolders` and + // `onlyKillPids: [pid]` even when the caller had forbidden the forced ladder. A + // verified-but-still-live listener was then killed through `killProxy` — the forced + // stop the caller had just refused. The wait is observational in that mode: it runs, + // it just may not terminate anything. + const { reclaimListenPort } = await import("../../src/server/port-reclaim"); + const { spawn } = await import("node:child_process"); + const child = spawn("sleep", ["60"], { stdio: "ignore" }); + await new Promise(resolve => child.once("spawn", () => resolve())); + const kills: number[] = []; + const pid = child.pid!; + const reclaimed: unknown[] = []; + const unconfirmed = await stopProxy(pid, { + runtimeEndpoint: { hostname: "127.0.0.1", port: 45999 }, + fetchFn: (async () => new Response("nope", { status: 500 })) as typeof fetch, + waitExit: () => true, + env: {}, + allowForcedFallback: false, + forceKill: target => { kills.push(target); }, + reclaimPort: async (port, hostname, opts) => { + reclaimed.push({ port, hostname, killOcxHolders: opts.killOcxHolders, onlyKillPids: opts.onlyKillPids }); + return reclaimListenPort(port, hostname, { + ...opts, + timeoutMs: 30, + intervalMs: 5, + scanIntervalMs: 0, + sleepMs: async () => {}, + isAvailableFn: async () => false, + listListenPidsFn: () => ({ ok: true, pids: [pid] }), + isAliveFn: () => true, + verifyOcxFn: candidate => candidate, + killFn: target => { kills.push(target); }, + dropTcpFn: () => {}, + }); + }, + }); + try { + expect(unconfirmed).toBe(false); + // The wait ran against the resolved endpoint (it is not silently skipped)... + expect(reclaimed).toEqual([{ port: 45999, hostname: "127.0.0.1", killOcxHolders: false, onlyKillPids: [] }]); + // ...and a verified listener that was still alive survived it. + expect(kills).toEqual([]); + } finally { + child.kill("SIGKILL"); + } + }); + test("no next step sends the operator back to the command that just refused", () => { // #4169's loop: POST /api/stop answers "the stop must be run by `ocx stop`", and // `ocx stop` reprints it. Whatever the cause, the next step has to be something else, From ac285b2e22e56d5b357680add019025e3fe88e00 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 08:09:18 +0900 Subject: [PATCH 33/53] fix(service): prove an absent-Part-A endpoint down before deleting the task MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `windowsServiceRemovalStop` read `proxyStillLiveAfterStop({canRespawn:false})` as the answer to "is anything still serving after a Part A that found no manager". Its null is returned for a probe that timed out or failed as well as for a dead endpoint, so an unreadable answer was treated as success and the task was deleted — and a claimed receipt reported as discharged. Removal now asks the captured endpoint itself with the same tri-state contract as `endpointProvenDown` / `proxyEndpointProvenDown`, and only a definitive "dead" permits deletion; "live" and "unknown" refuse and keep the receipt. The receipt and the probe share one resolved endpoint. Refs #7771 --- src/service/cli-windows-stop.ts | 27 +++++++--- .../service/service-stop-verification.test.ts | 15 +++--- tests/windows/windows-stop-callers.test.ts | 50 ++++++++++++------- 3 files changed, 61 insertions(+), 31 deletions(-) diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index a545db5cf5d..c5c8e39155a 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -18,6 +18,7 @@ import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-te import { readRuntimePort } from "../config/process-state"; import { loadConfig } from "../config"; import { readClientConnectionState } from "../client/state"; +import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; import { proxyStillLiveAfterStop, stopTrackedProxyForServiceCommand } from "./orchestration"; import { confirmWindowsServiceStop, @@ -36,6 +37,11 @@ export interface WindowsServiceStopCommandIo { claim?: typeof claimPendingTeardown; stopTracked?: typeof stopTrackedProxyForServiceCommand; survivor?: typeof proxyStillLiveAfterStop; + /** + * Tri-state probe of the endpoint the removal stop targeted. Only a definitive "dead" + * permits deleting the task; "live" and "unknown" refuse and keep the receipt. + */ + endpointLiveness?: (endpoint: { hostname: string; port: number }) => Promise; } function receiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { @@ -124,9 +130,12 @@ export function completeWindowsServiceStopReceipt(nonce: string | undefined, cle * `ocx service stop` does, and kept until the caller's restore succeeds. */ export async function windowsServiceRemovalStop( - io: Pick = {}, + io: Pick = {}, ): Promise<{ ok: boolean; nonce?: string }> { const port = windowsServiceStopPort(); + // Resolved once: the receipt and the absent-Part-A probe must name the same address, or + // the proof would be about an endpoint the stop never contacted. + const { endpoint, source } = receiptEndpoint(); const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); if (partA.outcome !== "stopped" && partA.outcome !== "absent") { console.error("❌ Service uninstall refused: the Windows service could not be proven stopped (" + partA.outcome + "), so the task was not deleted."); @@ -135,7 +144,6 @@ export async function windowsServiceRemovalStop( } let nonce: string | undefined; if ((io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { - const { endpoint, source } = receiptEndpoint(); try { nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)).nonce; } catch (error) { @@ -152,11 +160,16 @@ export async function windowsServiceRemovalStop( if (partA.outcome === "absent") { // "absent" says no manager is registered, not that nothing is serving. `findLiveProxy` // inside the tracked stop can return null when its discovery cap runs out while the - // endpoint is live, so ask the endpoint itself before the task is deleted. No respawn - // hold: with no manager there is nothing to relaunch the proxy. - const survivor = await (io.survivor ?? proxyStillLiveAfterStop)({ canRespawn: false }); - if (!survivor) return withNonce(nonce); - console.error("❌ Service uninstall refused: a proxy is still listening on port " + survivor.port + "."); + // endpoint is live, and `proxyStillLiveAfterStop` returns that same null for a probe + // that timed out or failed — which the old branch read as success. Ask the endpoint + // itself, tri-state, and delete the task only on a definitive "dead". No respawn hold: + // with no manager there is nothing to relaunch the proxy. + const liveness = await (io.endpointLiveness ?? probeEndpointLiveness)(endpoint) + .catch((): EndpointLiveness => "unknown"); + if (liveness === "dead") return withNonce(nonce); + console.error(liveness === "live" + ? "❌ Service uninstall refused: a proxy is still listening on port " + endpoint.port + "." + : "❌ Service uninstall refused: the endpoint on port " + endpoint.port + " could not be confirmed down."); console.error(" The task was not deleted. Stop that proxy, then re-run 'ocx service uninstall'."); return { ok: false }; } diff --git a/tests/service/service-stop-verification.test.ts b/tests/service/service-stop-verification.test.ts index 51b51f8ba8f..5111e825375 100644 --- a/tests/service/service-stop-verification.test.ts +++ b/tests/service/service-stop-verification.test.ts @@ -174,22 +174,25 @@ describe("orphan discovery is capped by the caller, not hard-coded (#7771)", () const { partAResult } = await import("../helpers/windows-stop-caller-fixtures"); setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); try { - // The survivor probe is injected: both callers fall through to the real - // proxyStillLiveAfterStop, which on Windows spends its full 7 s window before - // answering, and this test is about the discovery budget, not about that wait. + // The endpoint probes are injected: both callers would otherwise fall through to + // the real ones, which on Windows spend their full window before answering, and this + // test is about the discovery budget, not about that wait. const survivors: unknown[] = []; + const probed: unknown[] = []; for (const run of [windowsServiceStopBeforeRestore, windowsServiceRemovalStop]) { const seen: unknown[] = []; await run({ ownsSharedTeardown: () => false, stopTracked: async io => { seen.push(io); return "stopped"; }, survivor: async io => { survivors.push(io ?? "no-args"); return null; }, + endpointLiveness: async endpoint => { probed.push(endpoint); return "dead"; }, }); expect(seen).toEqual([{ skipPortWait: false, orphanDiscoveryCapMs: 3_000 }]); } - // Both callers consulted the injection rather than the real 7 s probe, and the - // removal path asks for no respawn hold because nothing there can relaunch the proxy. - expect(survivors).toEqual(["no-args", { canRespawn: false }]); + // The stop path consulted its survivor injection; the removal path asked the + // tri-state endpoint probe instead, because a null survivor is not proof of absence. + expect(survivors).toEqual(["no-args"]); + expect(probed.length).toBe(1); } finally { setWindowsServiceStopHooksForTests(null); } diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index 3ba0fb76d65..ea0a4fdaedc 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -181,29 +181,43 @@ describe("ocx service uninstall / remove on Windows", () => { } }); - test("an absent Part A is not proof the proxy is down: a live endpoint still refuses removal", async () => { + test("an absent Part A asks the endpoint itself: only a definitive down permits removal", async () => { // Part A "absent" only says no manager is registered. `findLiveProxy` can return null - // when its discovery cap runs out while the endpoint is live, so removal / the shared - // restore must ask the endpoint itself before deleting the task. No respawn hold is - // needed — nothing can relaunch it — but the answer still has to be definitive. - setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); - const survivors: unknown[] = []; - const refused = await windowsServiceRemovalStop({ - ownsSharedTeardown: () => false, - stopTracked: async () => "none", - survivor: async io => { survivors.push(io); return { port: 10100 }; }, - }); - expect(refused.ok).toBe(false); - // The hold would be meaningless here; asking for it would spend 7 s for nothing. - expect(survivors).toEqual([{ canRespawn: false }]); + // when its discovery cap runs out while the endpoint is live, so removal must ask the + // endpoint itself before deleting the task — and a probe that could not answer + // (timeout, transport failure) is not the answer "nothing is serving". Only the + // tri-state "dead" admits removal; "live" and "unknown" refuse and keep the receipt. + for (const [answer, ok] of [["dead", true], ["live", false], ["unknown", false]] as const) { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const probed: { port: number }[] = []; + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpointLiveness: async endpoint => { probed.push(endpoint); return answer; }, + }); + expect(result.ok).toBe(ok); + expect(probed.length).toBe(1); + expect(probed[0]!.port).toBeGreaterThan(0); + } + }); + test("an unreadable absent-Part-A probe refuses and keeps the receipt", async () => { + // The old branch read `proxyStillLiveAfterStop`'s null — its answer for a timed-out or + // failed probe — as success and deleted the task, and a claimed receipt was reported + // as discharged by the caller. setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); - const absent = await windowsServiceRemovalStop({ - ownsSharedTeardown: () => false, + let claimed = 0; + let probed = 0; + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => true, + claim: () => { claimed++; return { nonce: "r3" } as never; }, stopTracked: async () => "none", - survivor: async () => null, + endpointLiveness: async () => { probed++; throw new Error("probe timed out"); }, }); - expect(absent).toEqual({ ok: true }); + expect(claimed).toBe(1); + // The endpoint is what has to answer; a survivor probe's null never stood in for it. + expect(probed).toBe(1); + expect(result).toEqual({ ok: false }); }); test("claims a receipt carrying the Part A record before the POST, so the route does not answer 409", async () => { From 1fd6c7252eea2157c151d50370d3421131d4d54e Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 08:27:02 +0900 Subject: [PATCH 34/53] fix(cli): gate Windows service removal on a proven stop and re-check ownership MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ocx uninstall` deleted the Windows registration with `uninstallServiceDetailed()` even when Part A had failed to stop the manager, and the WinSW branch stopped and deleted whatever was registered under the fixed `opencodex-proxy-native` ID — which can be another installation's service. Removal now requires a Part A that ended stopped or absent (`serviceRemovalAllowed`, Windows only; other platforms unchanged) and records `failed` otherwise, so the shared teardown stays blocked as `ocx service uninstall` already did. `uninstallServiceDetailed` re-reads `sc qc` BINARY_PATH_NAME right before the native removal: foreign leaves the registration untouched, unreadable refuses and keeps install state. Refs #7771 --- src/cli/index.ts | 14 ++++++++- src/cli/uninstall-plan.ts | 18 +++++++++++ src/service/orchestration.ts | 45 ++++++++++++++++++++++++++-- tests/cli/uninstall.test.ts | 58 ++++++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 4 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index 25b0ecfbcec..77a00c9cdc9 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -80,7 +80,7 @@ import { } from "../config/pending-teardown"; import { collectStatus, detectMissingCodexCatalogPath, hubStatusLines, missingCodexCatalogLines, remoteHubBannerLine, remoteHubStatusLines, runtimeSupervisorLine, statusDeadProxyLines, statusProxyUp, statusUnusedProxyLines } from "./status"; import { stopUninstallProxy } from "./uninstall-stop"; -import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; +import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, serviceRemovalAllowed, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; import { takeFlag } from "./runtime-api"; import { parseStartOptions, StartArgsError } from "./start-args"; @@ -1604,6 +1604,18 @@ async function handleUninstall() { }); await runStep("service removed", () => { + // Windows only: a Part A that failed (or could not read the state, or left a wrapper + // that can still respawn) leaves a manager this uninstall cannot account for. Deleting + // its registration would remove the evidence without stopping the wrapper, and + // `ocx service uninstall` refuses in the same states. The failure stays recorded so the + // shared teardown below stays blocked. + if (!serviceRemovalAllowed(observed)) { + observed.serviceRemoval = "failed"; + throw new Error( + "the service manager could not be proven stopped, so its registration was left in place; " + + "resolve the 'service stopped' failure, then rerun 'ocx uninstall'", + ); + } const outcome = uninstallServiceDetailed(); observed.serviceRemoval = outcome; // "absent" and "removed" are both fine; a failure is not, and it used to look like diff --git a/src/cli/uninstall-plan.ts b/src/cli/uninstall-plan.ts index c6f984b70a7..d76d52675f6 100644 --- a/src/cli/uninstall-plan.ts +++ b/src/cli/uninstall-plan.ts @@ -43,6 +43,24 @@ export function respawnWindowRequired( return o.serviceStop === "stopped-respawnable" || (platform === "win32" && o.serviceStop === "stopped"); } +/** + * Whether the uninstall may remove the installed manager. + * + * On Windows the removal deletes a Task Scheduler registration and stops/deletes a WinSW + * service. Both are only safe once Part A proved the manager stopped or that none is + * registered: `ocx service uninstall` refuses in the same states, and removing the + * registration of a manager that could not be stopped hides the reason the proxy may + * come back instead of fixing it. Other platforms remove after their own stop attempt, as + * before. + */ +export function serviceRemovalAllowed( + o: Pick, + platform: NodeJS.Platform = process.platform, +): boolean { + if (platform !== "win32") return true; + return o.serviceStop === "stopped" || o.serviceStop === "absent"; +} + export function sharedTeardownAuthorized(o: UninstallObservation, platform: NodeJS.Platform = process.platform): boolean { if (o.serviceStop === null) return false; // "absent" and a clean stop are the only service states that prove nothing is managing diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index 2f26d92fb71..c6b228375a3 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -7,7 +7,8 @@ import { readPid, removePid, removeRuntimePort, verifyPidIdentity } from "../con import { isWslRuntime } from "../codex/home"; import { isProcessAlive, stopProxy } from "../lib/process-control"; import { randomUUID } from "node:crypto"; -import { startWinswService, stopWinswService, statusWinswRaw, uninstallWinswService, winswStatusSummary, type WinswStatus } from "../lib/winsw"; +import { startWinswService, stopWinswService, statusWinswRaw, uninstallWinswService, WINSW_SERVICE_ID, winswExePath, winswStatusSummary, type WinswStatus } from "../lib/winsw"; +import { queryWinswBinaryPathName, scBinaryPathNamesExecutable } from "../service-manager-probe"; import { diagnoseService } from "./diagnostics"; import { assertServiceEnvironmentMatchesInstall } from "./guards"; import { runLaunchctl, launchdEvictionTargets, launchctlBootoutBenign, probeLaunchdLoadState, installLaunchd, startLaunchd, stopLaunchd, statusLaunchd, uninstallLaunchd } from "./launchd"; @@ -620,6 +621,13 @@ type UninstallServiceHooksForTests = { probeWindowsTask: () => WindowsSchedulerTaskProbe; uninstallWindowsTask: () => void; nativeStatus: () => WinswStatus; + /** + * Is the registered `opencodex-proxy-native` service THIS home's? `sc stop/delete` name a + * fixed ID, so a registration is not evidence of ownership — another installation's + * WinSW service under the same ID would be stopped and deleted here. Production reads + * `sc qc` BINARY_PATH_NAME and compares it with this home's executable. + */ + nativeOwnership?: () => "owned" | "foreign" | "unknown"; uninstallNative: () => void; removeInstallState: () => void; launchdPlistPath?: () => string; @@ -648,6 +656,26 @@ export function setUninstallServiceHooksForTests(hooks: UninstallServiceHooksFor */ export type ServiceUninstallOutcome = "absent" | "removed" | "failed"; +/** + * Does the registered WinSW service run this home's executable? + * + * The same proof the guarded manager and Part A use (`sc qc` `BINARY_PATH_NAME`), reduced to + * the three answers removal needs: this home's (`owned`), another's (`foreign`), or one that + * could not be read (`unknown`, which refuses). + */ +function winswOwnershipOfThisHome( + query: () => string | null = queryWinswBinaryPathName, +): "owned" | "foreign" | "unknown" { + let binaryPathName: string | null; + try { + binaryPathName = query(); + } catch { + return "unknown"; + } + if (binaryPathName === null) return "unknown"; + return scBinaryPathNamesExecutable(binaryPathName, winswExePath()) ? "owned" : "foreign"; +} + /** * Remove installed manager assets, including orphaned launchd recovery files. * Keep install state on removal failure; Windows refusal and unknown probes may throw. @@ -676,8 +704,19 @@ export function uninstallServiceDetailed(): ServiceUninstallOutcome { removed = true; } if ((hooks?.nativeStatus ?? statusWinswRaw)() !== "nonexistent") { - (hooks?.uninstallNative ?? uninstallWinswService)(); - removed = true; + // Re-prove ownership right before the destructive step: `sc qc` may have changed + // since the install-state read, and a foreign registration must survive an uninstall + // of this home. Unreadable is not a licence to delete — fail closed and keep state. + const ownership = (hooks?.nativeOwnership ?? winswOwnershipOfThisHome)(); + if (ownership === "unknown") { + throw new Error( + `The native service registration's ownership could not be verified (sc qc ${WINSW_SERVICE_ID}); refusing to remove it. Check 'sc query ${WINSW_SERVICE_ID}' and retry.`, + ); + } + if (ownership === "owned") { + (hooks?.uninstallNative ?? uninstallWinswService)(); + removed = true; + } } if (removed) { (hooks?.removeInstallState ?? removeServiceInstallState)(); return "removed"; } } else if (platform === "linux" && existsSync(unitPath())) { diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 3d49d746afa..6086029c1dc 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -1,6 +1,7 @@ import { afterEach, describe, expect, test } from "bun:test"; import { setUninstallServiceHooksForTests, + uninstallServiceDetailed, uninstallServiceIfInstalled, } from "../../src/service"; import { pathToFileURL } from "node:url"; @@ -90,6 +91,7 @@ describe("full uninstall command", () => { probeWindowsTask: () => ({ status: "present" }), uninstallWindowsTask: () => { calls.push("scheduler"); }, nativeStatus: () => "started", + nativeOwnership: () => "owned", uninstallNative: () => { calls.push("native"); throw new Error("native removal failed"); @@ -102,6 +104,39 @@ describe("full uninstall command", () => { expect(stateRemovals).toBe(0); }); + test("a foreign WinSW registration is left alone and an unreadable one refuses removal", () => { + const calls: string[] = []; + let stateRemovals = 0; + const hooks = (ownership: "owned" | "foreign" | "unknown") => ({ + platform: "win32" as const, + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "absent" }) as never, + uninstallWindowsTask: () => { calls.push("scheduler"); }, + nativeStatus: () => "started" as const, + nativeOwnership: () => ownership, + uninstallNative: () => { calls.push("native"); }, + removeInstallState: () => { stateRemovals++; }, + }); + + // `sc stop/delete opencodex-proxy-native` names a fixed ID, so a registration under it + // is not proof it is this home's. Another installation's WinSW service must survive. + setUninstallServiceHooksForTests(hooks("foreign")); + expect(uninstallServiceDetailed()).toBe("absent"); + expect(calls).toEqual([]); + expect(stateRemovals).toBe(0); + + // Unreadable ownership is not absence: refuse and keep install state for a retry. + setUninstallServiceHooksForTests(hooks("unknown")); + expect(() => uninstallServiceDetailed()).toThrow("could not be verified"); + expect(calls).toEqual([]); + expect(stateRemovals).toBe(0); + + setUninstallServiceHooksForTests(hooks("owned")); + expect(uninstallServiceDetailed()).toBe("removed"); + expect(calls).toEqual(["native"]); + expect(stateRemovals).toBe(1); + }); + test("scheduler removal failure propagates without deleting install state", () => { let stateRemovals = 0; setUninstallServiceHooksForTests({ @@ -257,6 +292,29 @@ describe("uninstall gates shared teardown on a proven service stop", () => { return cli.slice(at, at + 9000); } + test("Windows removal is gated on a Part A that ended stopped or absent", async () => { + const { serviceRemovalAllowed } = await import("../../src/cli/uninstall-plan"); + for (const outcome of ["stopped", "absent"] as const) { + expect(serviceRemovalAllowed({ serviceStop: outcome }, "win32")).toBe(true); + } + // A Part A that failed, could not read the state, or left a respawnable wrapper still + // has a manager that can relaunch the proxy; `ocx service uninstall` refuses there, and + // deleting the registration would remove the evidence without stopping the wrapper. + for (const outcome of ["failed", "state-unknown", "stopped-respawnable", null] as const) { + expect(serviceRemovalAllowed({ serviceStop: outcome }, "win32")).toBe(false); + } + // Non-Windows removal is unchanged. + for (const platform of ["linux", "darwin"] as const) { + for (const outcome of ["stopped", "absent", "stopped-respawnable", "failed", "state-unknown", null] as const) { + expect(serviceRemovalAllowed({ serviceStop: outcome }, platform)).toBe(true); + } + } + const fn = await uninstallFn(); + const step = fn.slice(fn.indexOf('runStep("service removed"'), fn.indexOf('runStep("service removed"') + 800); + expect(step).toContain("serviceRemovalAllowed(observed)"); + expect(step.indexOf("serviceRemovalAllowed(observed)")).toBeLessThan(step.indexOf("uninstallServiceDetailed()")); + }); + test("the detailed outcome is consumed, not the boolean collapse", async () => { const fn = await uninstallFn(); // stopServiceIfInstalled returns false for "not installed", "refused to stop" and From ad343f62f101be9509b86085ced97f5c15ebd93d Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 08:28:06 +0900 Subject: [PATCH 35/53] docs(windows): record the uninstall stop wiring and removal gates structure/ops/windows-checked-stop.md and devlog 060 now describe items 1-5: the uninstall stop resolving its target before claiming, the observational wait under a forbidden fallback, the tri-state absent-Part-A probe, and the Windows removal gates (Part A outcome plus a fresh `sc qc` ownership check). Refs #7771 --- .../060_implementation.md | 6 +++++- structure/ops/windows-checked-stop.md | 21 ++++++++++++++++++- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index a26848fb3ed..85770fbd095 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -79,7 +79,11 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **Uninstall's proxy stop claims its receipt before the child stop.** `handleUninstall` sent `/api/stop` without a teardown receipt, so the route finished the shared teardown while the wrapper respawn window had not been checked yet; a failed `verifyUninstallStopWindow` then left shared config restored under a wrapper that could relaunch the proxy. On Windows the uninstall proxy stop is now `src/cli/uninstall-stop.ts`: it claims a receipt carrying Part A's record before the child stop, sends its nonce, and discharges it only after Part B and the shared restore. A failed claim sends no child stop; a failed later step keeps the receipt for recovery. - **Uninstall never forces a stop its Part A could not prove.** When Part A did not end `stopped` or `absent`, the uninstall proxy stop still sends the graceful POST but passes `allowForcedFallback: false` to `stopProxy`, so a POST timeout, transport error or non-409 answer cannot fall through to `taskkill /T /F`. The uninstall failure stays recorded. A 409 refusal already forbids the forced path; this closes the remaining ones. -- **An absent Part A is not proof the proxy is down.** `windowsServiceRemovalStop` returned `ok` on Part A `absent` without checking the endpoint, and `findLiveProxy` answers null when its 3 s discovery cap runs out even while the endpoint is live. Removal now requires `proxyStillLiveAfterStop({ canRespawn: false })` to report the endpoint down before it deletes the task; a live endpoint refuses removal, and an unreadable answer is not success. No respawn hold is needed: a wait would price removal at Part B's 6 s for a manager that is not registered. +- **An absent Part A is not proof the proxy is down.** `windowsServiceRemovalStop` returned `ok` on Part A `absent` without checking the endpoint, and `findLiveProxy` answers null when its 3 s discovery cap runs out even while the endpoint is live. Removal now probes the endpoint it captured, once, with the tri-state contract (`endpointLiveness`, the same one `endpointProvenDown` uses in `uninstall-stop` and `proxyEndpointProvenDown` in `src/cli/index.ts`); `proxyStillLiveAfterStop` looked down on a null, which its own probe produces on a timeout as well. Only `"dead"` permits deleting the task — `"live"` refuses and `"unknown"` (a timeout or transport failure) refuses too, and both keep the receipt. No respawn hold is needed: a wait would price removal at Part B's 6 s for a manager that is not registered. +- **Uninstall's receipt names the endpoint its stop contacts.** `stopUninstallProxy` claimed the receipt first and resolved its target afterwards, and `defaultReceiptEndpoint()` marks any runtime record "exact" — so a stale record plus a proxy discovered on another port left a receipt for a port the stop never contacted, and a failed Part A with no proxy left a "guessed" obligation outstanding that recovery refuses. It now resolves the target first (the tracked pid with its own runtime endpoint, or the discovered live proxy's endpoint), claims only when a stop will be attempted, and hands the receipt and `stopProxy` the same endpoint. No target, no receipt. +- **An unconfirmed stop removes no tracked state.** The tracked-pid path called `removeTracked(pid)` regardless of the stop's result, so a failed stop deleted the record that a rerun needed. It runs only when the stop was proven, and the step reports `changed` only when state actually changed. +- **A forbidden forced fallback waits without killing.** With `allowForcedFallback === false`, `waitForStoppedPort` still called `reclaimListenPort` with `killOcxHolders`/`onlyKillPids: [pid]`, so the wait could kill through `killProxy` — the path the flag exists to close. The wait now runs observationally (the reclaim dependency is injected, and the forbidden case passes a mode that samples without kill eligibility); `stopProxy` keeps its literal `killProxy(pid)` call, which `tests/providers/xai/grok-lifecycle.test.ts` reads as order evidence. +- **A foreign WinSW registration survives `ocx uninstall`.** Two gaps: on Windows the "service removed" step called `uninstallServiceDetailed()` even when Part A failed to stop the manager, and the native branch stopped and deleted whatever carried the fixed `opencodex-proxy-native` ID — including another installation's service. Removal is now gated on a Part A that ended stopped or absent (`serviceRemovalAllowed`, Windows only; other platforms unchanged), recording `failed` otherwise so the shared teardown stays blocked as `ocx service uninstall` already refuses. Immediately before the native removal, `uninstallServiceDetailed` re-reads `sc qc` `BINARY_PATH_NAME` and compares it with this home's WinSW executable: foreign leaves the registration untouched, unreadable refuses and keeps install state. - **Install preparation resolves the stop port before Part A.** `windowsServiceStopPort()` ran after the tracked cleanup had already removed the runtime record, so Part B confirmed the wrapper's baked port (10100) while the proxy's runtime port (12000) was never checked. `prepareServiceInstall` resolves the port once before Part A (`windowsStopPort` is the test seam) and carries it into Part B. - **Part A proves the WinSW registration is this home's.** `sc query` answering "running" says a service exists under the fixed `opencodex-proxy-native` ID, not that it is this installation's. The live probe now reads `BINARY_PATH_NAME` through `sc qc` and requires this home's `winswExePath()`, the same proof `inspectWindowsWinswManager` uses; an unreadable registration is `unknown` (`evidence-unreadable`, nothing changed) and a foreign executable is reported as `nonexistent`, so Part A does not stop it and reports `absent`. - **Part B clears only the exited owner's TCP rows.** After a listener PID was found exited, the sampler called `dropWindowsTcpRowsForLocalPort(port)`, which resets every IPv4 row on the port — including rows of a program that took the port between scans. `parseTcpQuadsForLocalPort` and `dropWindowsTcpRowsForLocalPort` take an owner PID filter and the sampler passes the exited PID(s); the hard-kill path still clears the whole port. diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index d290c81e73f..4ab0e307806 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -179,11 +179,30 @@ stop instead of sliding forward. Removal claims the same receipt, carrying Part A's record, before its child stop: Part A leaves the scheduler task registered, so a receipt-less POST answers `respawnable_service` (409) and the removal refuses with the proxy left running. The receipt is discharged only after the - shared restore, and a failed claim sends no child stop. + shared restore, and a failed claim sends no child stop. Part A `absent` says no manager is + registered, not that nothing serves, so removal then probes the endpoint it captured, tri-state: + only a definitive dead permits deleting the task, while live or unknown refuses and keeps the + receipt. - Every tracked-proxy stop takes its orphan-discovery cap from the caller (`TrackedProxyStopIo.orphanDiscoveryCapMs`). The checked Windows callers pass the admitted 3 s (`WINDOWS_STOP_CAPS.orphanDiscovery`), which is what admission priced; the omitted default stays the legacy 7 s so macOS and Linux timing is unchanged. +- `ocx uninstall` (`src/cli/uninstall-stop.ts`) resolves its stop target before it claims + anything: the tracked pid with its own runtime endpoint, or the discovered live proxy with the + endpoint discovery found it on. The receipt and `stopProxy` then name the same endpoint, so a + stale runtime record cannot produce a receipt for a port the stop never contacted, and no target + means no receipt at all (a guessed obligation recovery cannot resolve is worse than none). The + tracked record is removed only for a stop that was proven, and the step reports a change only + when state changed. When Part A did not end stopped or absent the graceful POST still goes out + with `allowForcedFallback: false`, and that unconfirmed wait is observational: it samples the + port without `reclaimListenPort`'s kill eligibility, so a forbidden fallback cannot kill through + the reclaim path either. +- `ocx uninstall` removes the Windows manager only after a Part A that ended stopped or absent + (`serviceRemovalAllowed` in `src/cli/uninstall-plan.ts`); otherwise the step records `failed` and + leaves the registration, as `ocx service uninstall` refuses. `uninstallServiceDetailed` + (`src/service/orchestration.ts`) re-reads `sc qc` `BINARY_PATH_NAME` immediately before the + native removal: a registration naming another installation's WinSW executable is left untouched, + and an unreadable one refuses and keeps install state. Non-Windows removal is unchanged. - `prepareServiceInstall` refuses before replacing anything on a Part A other than stopped or absent and stops install and switch on any Part B result other than a free port, naming the owner. `ocx uninstall` runs Part B as its respawn window after a stopped Part A. From 1ce8e8f194d5ff72d5fe9fd393a392e786f518bb Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 08:32:19 +0900 Subject: [PATCH 36/53] refactor(cli): keep the uninstall removal gate out of src/cli/index.ts The gate pushed src/cli/index.ts to 2005 lines, over the 2000-line ratchet. `uninstallServiceRemoval` now holds the rule and the call in `uninstall-plan`, so the command body stays one call and the refusal is testable directly. Refs #7771 --- src/cli/index.ts | 11 ++--------- src/cli/uninstall-plan.ts | 21 +++++++++++++++++++++ tests/cli/uninstall.test.ts | 18 ++++++++++++++---- 3 files changed, 37 insertions(+), 13 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index 77a00c9cdc9..df003ef0728 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -80,7 +80,7 @@ import { } from "../config/pending-teardown"; import { collectStatus, detectMissingCodexCatalogPath, hubStatusLines, missingCodexCatalogLines, remoteHubBannerLine, remoteHubStatusLines, runtimeSupervisorLine, statusDeadProxyLines, statusProxyUp, statusUnusedProxyLines } from "./status"; import { stopUninstallProxy } from "./uninstall-stop"; -import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, serviceRemovalAllowed, sharedTeardownAuthorized, type UninstallObservation } from "./uninstall-plan"; +import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, sharedTeardownAuthorized, uninstallServiceRemoval, type UninstallObservation } from "./uninstall-plan"; import { takeFlag } from "./runtime-api"; import { parseStartOptions, StartArgsError } from "./start-args"; @@ -1609,14 +1609,7 @@ async function handleUninstall() { // its registration would remove the evidence without stopping the wrapper, and // `ocx service uninstall` refuses in the same states. The failure stays recorded so the // shared teardown below stays blocked. - if (!serviceRemovalAllowed(observed)) { - observed.serviceRemoval = "failed"; - throw new Error( - "the service manager could not be proven stopped, so its registration was left in place; " - + "resolve the 'service stopped' failure, then rerun 'ocx uninstall'", - ); - } - const outcome = uninstallServiceDetailed(); + const outcome = uninstallServiceRemoval(observed, uninstallServiceDetailed); observed.serviceRemoval = outcome; // "absent" and "removed" are both fine; a failure is not, and it used to look like // absence on darwin and linux. diff --git a/src/cli/uninstall-plan.ts b/src/cli/uninstall-plan.ts index d76d52675f6..97bd1dc9feb 100644 --- a/src/cli/uninstall-plan.ts +++ b/src/cli/uninstall-plan.ts @@ -72,6 +72,27 @@ export function sharedTeardownAuthorized(o: UninstallObservation, platform: Node return o.proxyProvenDown; } +/** + * The "service removed" step: gate, then remove. + * + * Kept beside the rule so the command body stays a call, and so the refusal is testable + * without reading `handleUninstall`. A refused gate throws, which the step records as a + * failure and which keeps the shared teardown blocked. + */ +export function uninstallServiceRemoval( + o: Pick, + remove: () => "absent" | "removed" | "failed", + platform: NodeJS.Platform = process.platform, +): "absent" | "removed" | "failed" { + if (!serviceRemovalAllowed(o, platform)) { + throw new Error( + "the service manager could not be proven stopped, so its registration was left in place; " + + "resolve the 'service stopped' failure, then rerun 'ocx uninstall'", + ); + } + return remove(); +} + /** An endpoint an uninstall must account for before shared config comes down. */ export type ProbeEndpoint = { hostname: string; port: number }; diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 6086029c1dc..dcff9053fc3 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -162,10 +162,12 @@ describe("full uninstall command", () => { expect(uninstallBody).toContain('runStep("service removed"'); // #7771 round 7: the stop itself lives in `uninstall-stop`, which the step calls. expect(uninstallBody).toContain("await stopUninstallProxy({"); - expect(uninstallBody).toContain("uninstallServiceDetailed()"); + // The removal runs through the gate that keeps a Part A failure from deleting the + // registration; the gate's rule is exercised in the removal test below. + expect(uninstallBody).toContain("uninstallServiceRemoval(observed, uninstallServiceDetailed)"); expect(uninstallBody.indexOf('runStep("service stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("proxy stopped"')); expect(uninstallBody.indexOf('runStep("proxy stopped"')).toBeLessThan(uninstallBody.indexOf('runStep("service removed"')); - expect(uninstallBody.indexOf("await stopUninstallProxy({")).toBeLessThan(uninstallBody.indexOf("uninstallServiceDetailed()")); + expect(uninstallBody.indexOf("await stopUninstallProxy({")).toBeLessThan(uninstallBody.indexOf("uninstallServiceRemoval(observed, uninstallServiceDetailed)")); }); test("Windows uninstall leaves the port wait to Part B instead of stopProxy's legacy 15 s", async () => { @@ -311,8 +313,16 @@ describe("uninstall gates shared teardown on a proven service stop", () => { } const fn = await uninstallFn(); const step = fn.slice(fn.indexOf('runStep("service removed"'), fn.indexOf('runStep("service removed"') + 800); - expect(step).toContain("serviceRemovalAllowed(observed)"); - expect(step.indexOf("serviceRemovalAllowed(observed)")).toBeLessThan(step.indexOf("uninstallServiceDetailed()")); + // The command body calls the gate; the rule itself is exercised above. + expect(step).toContain("uninstallServiceRemoval(observed, uninstallServiceDetailed)"); + // A refused gate throws before the removal runs, and the step records the failure. + const { uninstallServiceRemoval } = await import("../../src/cli/uninstall-plan"); + let removed = 0; + expect(() => uninstallServiceRemoval({ serviceStop: "failed" }, () => { removed++; return "removed"; }, "win32")) + .toThrow("could not be proven stopped"); + expect(removed).toBe(0); + expect(uninstallServiceRemoval({ serviceStop: "stopped" }, () => { removed++; return "removed"; }, "win32")).toBe("removed"); + expect(removed).toBe(1); }); test("the detailed outcome is consumed, not the boolean collapse", async () => { From a10c07996d4814c6cb936786966793bf478b9578 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:06:15 +0900 Subject: [PATCH 37/53] fix(cli): prove the endpoint down when the uninstall stop is unconfirmed stopProxy answers false without contacting anything when the tracked pid is already dead, so an unconfirmed stop said nothing about a different proxy on the configured port. The tracked path now asks the same tri-state probe the discovery path requires before reporting provenDown, and still forbids the forced fallback. Refs #7771 --- src/cli/uninstall-stop.ts | 16 +++++++++++++-- tests/cli/uninstall.test.ts | 39 +++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 2 deletions(-) diff --git a/src/cli/uninstall-stop.ts b/src/cli/uninstall-stop.ts index b2a7ce5f1dc..709de299b77 100644 --- a/src/cli/uninstall-stop.ts +++ b/src/cli/uninstall-stop.ts @@ -68,7 +68,19 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise stopped || !forbidForcedFallback; + // + // An unconfirmed stop is also NOT evidence that this home's proxy is down. `stopProxy` + // answers false without contacting the endpoint when the tracked pid is already dead, so + // "the tracked proxy is gone" and "nothing else is serving on this home's ports" are two + // different claims; Part A saying `absent`/`stopped` covers only the manager it saw. A + // different proxy on the configured port would otherwise be missed, and shared config + // restored under it (#3008). The tri-state probe the discovery path already requires is + // what settles it, and only a definitive "nothing answers anywhere" keeps the proof. + const confirmed = async (stopped: boolean): Promise => { + if (stopped) return true; + if (forbidForcedFallback) return false; + return await io.endpointProvenDown(); + }; // Resolve the target BEFORE anything is recorded. The receipt is an obligation on a // specific address, so claiming it from a runtime record the stop never contacts — or @@ -126,7 +138,7 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise { + // #7771 round 9: `stopProxy` returns false without contacting anything when the tracked + // pid is already dead. `confirmed(false)` then reported provenDown:true on the strength of + // Part A alone, so a DIFFERENT proxy on the configured port was never probed and shared + // config could be restored under it. The tri-state probe is what the discovery path + // already requires; the tracked path has to ask the same question. + let probed = 0; + const dead = await stopUninstallProxy(io({ + serviceStop: "absent", + stop: async () => false, + endpointProvenDown: async () => { probed++; return true; }, + })); + expect(probed).toBe(1); + expect(dead.provenDown).toBe(true); + + // A live or unreadable endpoint is not proof, and the record stays for the next run. + for (const answer of [false]) { + const removed: number[] = []; + const result = await stopUninstallProxy(io({ + serviceStop: "stopped", + stop: async () => false, + endpointProvenDown: async () => answer, + removeTracked: pid => { removed.push(pid); }, + })); + expect(result.provenDown).toBe(false); + expect(result.changed).toBe(false); + expect(removed).toEqual([]); + } + + // A confirmed stop needs no second opinion. + let asked = 0; + const stopped = await stopUninstallProxy(io({ + stop: async () => true, + endpointProvenDown: async () => { asked++; return false; }, + })); + expect(stopped.provenDown).toBe(true); + expect(asked).toBe(0); + }); + test("a receipt that cannot be recorded refuses the stop instead of sending an undeferred POST", async () => { let stopped = 0; await expect(stopUninstallProxy(io({ From ff2e276369f3bde3ff00dfdb347a10fb0132a720 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:06:58 +0900 Subject: [PATCH 38/53] fix(service): prove every candidate endpoint before Windows removal With Part A absent only the receipt endpoint was probed, so a stale runtime record naming a closed port could prove removal while the configured port had a live listener. Removal now proves the same candidate set ocx uninstall proves, and any live or unreadable candidate refuses and keeps the receipt. Refs #7771 --- src/service/cli-windows-stop.ts | 65 ++++++++++++++++++---- tests/windows/windows-stop-callers.test.ts | 41 ++++++++++++++ 2 files changed, 95 insertions(+), 11 deletions(-) diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index c5c8e39155a..a97cfde273b 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -18,6 +18,7 @@ import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-te import { readRuntimePort } from "../config/process-state"; import { loadConfig } from "../config"; import { readClientConnectionState } from "../client/state"; +import { endpointsToProve, type ProbeEndpoint } from "../cli/uninstall-plan"; import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; import { proxyStillLiveAfterStop, stopTrackedProxyForServiceCommand } from "./orchestration"; import { @@ -38,10 +39,50 @@ export interface WindowsServiceStopCommandIo { stopTracked?: typeof stopTrackedProxyForServiceCommand; survivor?: typeof proxyStillLiveAfterStop; /** - * Tri-state probe of the endpoint the removal stop targeted. Only a definitive "dead" - * permits deleting the task; "live" and "unknown" refuse and keep the receipt. + * Tri-state probe of an endpoint. Only a definitive "dead" on EVERY candidate permits + * deleting the task; "live" and "unknown" refuse and keep the receipt. */ endpointLiveness?: (endpoint: { hostname: string; port: number }) => Promise; + /** + * Every endpoint this home could be serving on, for the absent-Part-A proof. Defaults to + * `endpointsToProve(readRuntimePort(), loadConfig())` — the set `ocx uninstall` proves. + */ + endpoints?: () => ProbeEndpoint[]; +} + +/** + * Every endpoint removal has to account for, exactly the set `ocx uninstall` proves. + * + * The receipt endpoint is ONE candidate; a stale runtime record naming a closed port must + * not stand in for the configured port where a live listener can sit (#3008, #7771). + */ +function removalEndpoints(): ProbeEndpoint[] { + let config: { port?: number; hostname?: string } = {}; + try { config = loadConfig(); } catch { /* defaults below */ } + return endpointsToProve(readRuntimePort(), config); +} + +/** + * Definitive absence on EVERY candidate. "unknown" is not absence: a listener that + * accepts connections but withholds /healthz, or one that times out, is exactly the state + * where deleting the registration is most harmful. + */ +async function endpointsAllDead( + endpoints: readonly ProbeEndpoint[], + probe: (endpoint: ProbeEndpoint) => Promise, +): Promise<{ dead: boolean; live: number[]; unknown: number[] }> { + const results = await Promise.all(endpoints.map(async endpoint => ( + await probe(endpoint).catch((): EndpointLiveness => "unknown") + ))); + const live: number[] = []; + const unknown: number[] = []; + endpoints.forEach((endpoint, index) => { + const answer = results[index]; + if (answer === "live") live.push(endpoint.port); + else if (answer !== "dead") unknown.push(endpoint.port); + }); + // No candidate is not proof of anything. + return { dead: endpoints.length > 0 && live.length === 0 && unknown.length === 0, live, unknown }; } function receiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { @@ -161,15 +202,17 @@ export async function windowsServiceRemovalStop( // "absent" says no manager is registered, not that nothing is serving. `findLiveProxy` // inside the tracked stop can return null when its discovery cap runs out while the // endpoint is live, and `proxyStillLiveAfterStop` returns that same null for a probe - // that timed out or failed — which the old branch read as success. Ask the endpoint - // itself, tri-state, and delete the task only on a definitive "dead". No respawn hold: - // with no manager there is nothing to relaunch the proxy. - const liveness = await (io.endpointLiveness ?? probeEndpointLiveness)(endpoint) - .catch((): EndpointLiveness => "unknown"); - if (liveness === "dead") return withNonce(nonce); - console.error(liveness === "live" - ? "❌ Service uninstall refused: a proxy is still listening on port " + endpoint.port + "." - : "❌ Service uninstall refused: the endpoint on port " + endpoint.port + " could not be confirmed down."); + // that timed out or failed — which the old branch read as success. Ask the endpoints + // themselves, tri-state, and delete the task only when EVERY candidate is definitively + // "dead": the receipt endpoint is only one of the addresses this home can serve on, and + // a stale runtime record pointing at a closed port used to prove removal while the + // configured port had a live listener. No respawn hold: with no manager registered + // there is nothing to relaunch the proxy. + const proof = await endpointsAllDead((io.endpoints ?? removalEndpoints)(), io.endpointLiveness ?? probeEndpointLiveness); + if (proof.dead) return withNonce(nonce); + console.error(proof.live.length > 0 + ? "❌ Service uninstall refused: a proxy is still listening on port " + proof.live.join(", ") + "." + : "❌ Service uninstall refused: the endpoint on port " + (proof.unknown.join(", ") || endpoint.port) + " could not be confirmed down."); console.error(" The task was not deleted. Stop that proxy, then re-run 'ocx service uninstall'."); return { ok: false }; } diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index ea0a4fdaedc..fb196aa526d 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -201,6 +201,47 @@ describe("ocx service uninstall / remove on Windows", () => { } }); + test("an absent Part A proves EVERY candidate endpoint dead, not just the receipt's", async () => { + // #7771 round 9: with Part A absent only the receipt endpoint was probed. A stale runtime + // record naming a closed port then "proved" removal while the configured port had a live + // listener, and the task was deleted under it. The candidate set is the one `ocx uninstall` + // proves (`endpointsToProve(readRuntimePort(), loadConfig())`), and every candidate has to + // answer definitively dead. + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const probed: { hostname: string; port: number }[] = []; + const result = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10999 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async endpoint => { + probed.push(endpoint); + return endpoint.port === 10999 ? "dead" : "live"; + }, + }); + expect(result.ok).toBe(false); + expect(probed.map(e => e.port)).toEqual([10999, 10100]); + + // One unreadable candidate refuses too, and nothing is deleted. + probed.length = 0; + const unknown = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10999 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async endpoint => { probed.push(endpoint); return endpoint.port === 10999 ? "dead" : "unknown"; }, + }); + expect(unknown.ok).toBe(false); + expect(probed).toHaveLength(2); + + // Every candidate dead is the only proof. + const dead = await windowsServiceRemovalStop({ + ownsSharedTeardown: () => false, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10999 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async () => "dead", + }); + expect(dead.ok).toBe(true); + }); + test("an unreadable absent-Part-A probe refuses and keeps the receipt", async () => { // The old branch read `proxyStillLiveAfterStop`'s null — its answer for a timed-out or // failed probe — as success and deleted the task, and a claimed receipt was reported From 56d563aa2bee7fa5dc13b0bd62346dae0b57272a Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:08:25 +0900 Subject: [PATCH 39/53] fix(service): probe Windows removal before deleting anything The scheduler task was deleted before the WinSW status and ownership probes, so an unreadable sc qc left a half-removed install with no task to retry from. Read both probes first and refuse on unknown before the first destructive step (Refs #7771). --- src/service/orchestration.ts | 26 +++++++++++------- tests/cli/uninstall.test.ts | 52 ++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 9 deletions(-) diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index c6b228375a3..4263d5033ef 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -699,24 +699,32 @@ export function uninstallServiceDetailed(): ServiceUninstallOutcome { if (scheduler.status === "unknown") { throw new Error(`Could not determine Task Scheduler state: ${scheduler.detail}`); } - if (scheduler.status === "present") { - (hooks?.uninstallWindowsTask ?? uninstallWindows)(); - removed = true; + // Every probe runs before the first destructive step. The task used to be deleted first, + // so an unreadable `sc qc` left a half-removed install: no scheduler task to retry from, + // native registration still on disk, and no way back (#7771 round 9). + const nativeStatus = (hooks?.nativeStatus ?? statusWinswRaw)(); + if (nativeStatus === "unknown") { + throw new Error("Native service status could not be verified."); } - if ((hooks?.nativeStatus ?? statusWinswRaw)() !== "nonexistent") { + let ownership: "owned" | "foreign" | "unknown" | null = null; + if (nativeStatus !== "nonexistent") { // Re-prove ownership right before the destructive step: `sc qc` may have changed // since the install-state read, and a foreign registration must survive an uninstall // of this home. Unreadable is not a licence to delete — fail closed and keep state. - const ownership = (hooks?.nativeOwnership ?? winswOwnershipOfThisHome)(); + ownership = (hooks?.nativeOwnership ?? winswOwnershipOfThisHome)(); if (ownership === "unknown") { throw new Error( `The native service registration's ownership could not be verified (sc qc ${WINSW_SERVICE_ID}); refusing to remove it. Check 'sc query ${WINSW_SERVICE_ID}' and retry.`, ); } - if (ownership === "owned") { - (hooks?.uninstallNative ?? uninstallWinswService)(); - removed = true; - } + } + if (scheduler.status === "present") { + (hooks?.uninstallWindowsTask ?? uninstallWindows)(); + removed = true; + } + if (ownership === "owned") { + (hooks?.uninstallNative ?? uninstallWinswService)(); + removed = true; } if (removed) { (hooks?.removeInstallState ?? removeServiceInstallState)(); return "removed"; } } else if (platform === "linux" && existsSync(unitPath())) { diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index f3c651166e8..64c346df129 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -137,6 +137,58 @@ describe("full uninstall command", () => { expect(stateRemovals).toBe(1); }); + test("an unreadable WinSW ownership removes nothing, scheduler task included", () => { + // #7771 round 9: the scheduler task was deleted before the ownership probe ran, so an + // `sc qc` that could not be read left a half-removed install: task gone, native + // registration still there, no retry that can put the task back. Every probe now runs + // before the first destructive step. + const calls: string[] = []; + let stateRemovals = 0; + setUninstallServiceHooksForTests({ + platform: "win32", + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "present" }), + uninstallWindowsTask: () => { calls.push("uninstall:task"); }, + nativeStatus: () => "stopped", + nativeOwnership: () => "unknown", + uninstallNative: () => { calls.push("uninstall:native"); }, + removeInstallState: () => { stateRemovals++; }, + }); + expect(() => uninstallServiceDetailed()).toThrow("could not be verified"); + expect(calls).toEqual([]); + expect(stateRemovals).toBe(0); + + // An unreadable native status is the same refusal, and it also precedes the task delete. + calls.length = 0; + setUninstallServiceHooksForTests({ + platform: "win32", + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "present" }), + uninstallWindowsTask: () => { calls.push("uninstall:task"); }, + nativeStatus: () => "unknown", + nativeOwnership: () => "owned", + uninstallNative: () => { calls.push("uninstall:native"); }, + removeInstallState: () => {}, + }); + expect(() => uninstallServiceDetailed()).toThrow("could not be verified"); + expect(calls).toEqual([]); + + // Both probes answer: the task and the owned service are removed. + calls.length = 0; + setUninstallServiceHooksForTests({ + platform: "win32", + assertEnvironment: () => {}, + probeWindowsTask: () => ({ status: "present" }), + uninstallWindowsTask: () => { calls.push("uninstall:task"); }, + nativeStatus: () => "started", + nativeOwnership: () => "owned", + uninstallNative: () => { calls.push("uninstall:native"); }, + removeInstallState: () => { calls.push("state"); }, + }); + expect(uninstallServiceDetailed()).toBe("removed"); + expect(calls).toEqual(["uninstall:task", "uninstall:native", "state"]); + }); + test("scheduler removal failure propagates without deleting install state", () => { let stateRemovals = 0; setUninstallServiceHooksForTests({ From ceafcadff1e5bc291f2c5baef5ae96c83546b100 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:08:47 +0900 Subject: [PATCH 40/53] test(lib): spawn the runtime's own binary for pid-taking stop tests spawn("sleep", ["60"]) depends on a POSIX sleep on PATH and hangs forever if the spawn fails, because only the spawn event was awaited. Use process.execPath with a short-lived -e script and reject on error (Refs #7771). --- tests/lib/process-control-graceful.test.ts | 26 +++++++++++++++++----- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/tests/lib/process-control-graceful.test.ts b/tests/lib/process-control-graceful.test.ts index 451b1de9530..9bf89fe055a 100644 --- a/tests/lib/process-control-graceful.test.ts +++ b/tests/lib/process-control-graceful.test.ts @@ -5,6 +5,24 @@ function okResponse(): Response { return new Response(JSON.stringify({ success: true, sharedTeardown: "performed" }), { status: 200 }); } +/** + * A live process this test owns, for the pid-taking stop paths. + * + * `spawn("sleep", ...)` depends on a POSIX `sleep` on PATH and, worse, waits forever when + * the spawn itself fails: the `spawn` event never fires and the `error` event is unhandled. + * The runtime's own binary is always present, and a failed start now rejects instead of + * hanging the suite. + */ +async function spawnIdleChild(): Promise { + const { spawn } = await import("node:child_process"); + const child = spawn(process.execPath, ["-e", "setTimeout(() => {}, 60_000)"], { stdio: "ignore" }); + await new Promise((resolve, reject) => { + child.once("spawn", () => resolve()); + child.once("error", reject); + }); + return child; +} + describe("gracefulStopHost", () => { test("loopback aliases and wildcard binds answer on IPv4 loopback", () => { for (const host of [undefined, "", " ", "localhost", "LOCALHOST", "127.0.0.1", "0.0.0.0", "::", "[::]"]) { @@ -360,9 +378,7 @@ describe("409 refusal reporting", () => { // // The pid is a real live process this test owns, and the ladder is injected: the rule // is "was the ladder entered", which a mock of the whole stop could not show. - const { spawn } = await import("node:child_process"); - const child = spawn("sleep", ["60"], { stdio: "ignore" }); - await new Promise(resolve => child.once("spawn", () => resolve())); + const child = await spawnIdleChild(); const failing = (async () => new Response("nope", { status: 500 })) as typeof fetch; try { const forbidden: number[] = []; @@ -407,9 +423,7 @@ describe("409 refusal reporting", () => { // stop the caller had just refused. The wait is observational in that mode: it runs, // it just may not terminate anything. const { reclaimListenPort } = await import("../../src/server/port-reclaim"); - const { spawn } = await import("node:child_process"); - const child = spawn("sleep", ["60"], { stdio: "ignore" }); - await new Promise(resolve => child.once("spawn", () => resolve())); + const child = await spawnIdleChild(); const kills: number[] = []; const pid = child.pid!; const reclaimed: unknown[] = []; From f2e42c08a9acdbb2fc1ea4b4c2cb11b755f9f128 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:26:14 +0900 Subject: [PATCH 41/53] fix(service): expose the candidate-endpoint seam on the removal stop's io type The absent-Part-A proof reads io.endpoints, but the Pick omitted it, so typecheck failed on the committed seam (Refs #7771). --- src/service/cli-windows-stop.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index a97cfde273b..3de026acb0f 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -171,7 +171,7 @@ export function completeWindowsServiceStopReceipt(nonce: string | undefined, cle * `ocx service stop` does, and kept until the caller's restore succeeds. */ export async function windowsServiceRemovalStop( - io: Pick = {}, + io: Pick = {}, ): Promise<{ ok: boolean; nonce?: string }> { const port = windowsServiceStopPort(); // Resolved once: the receipt and the absent-Part-A probe must name the same address, or From 18d2183941a7c8948823dc6a4e647fc60de1e9fc Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:26:31 +0900 Subject: [PATCH 42/53] fix(service): claim the install's teardown receipt before the stopped-part-A stop Part A leaves the scheduler task registered, so the tracked stop's /api/stop answered 409 respawnable_service without a receipt; stopProxy threw and the install aborted while the old proxy kept serving. prepareServiceInstall now claims a receipt carrying Part A's record before the stop and forwards its nonce. installServiceSafely clears it once the replacement is installed and started, and keeps it when install fails so the next ocx stop can finish (Refs #7771). --- src/service/orchestration.ts | 87 +++++++++++++++++-- src/service/windows-service-stop.ts | 15 ++++ .../service-install-wrapper-cleanup.test.ts | 64 ++++++++++++++ 3 files changed, 157 insertions(+), 9 deletions(-) diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index 4263d5033ef..6b39a231934 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -20,7 +20,8 @@ import { writeWindowsSchedulerAssets, stageWindowsSchedulerRegistrationXml, remo import type { WindowsCheckedStopResult } from "./windows-checked-stop"; import { WINDOWS_STOP_CAPS } from "./windows-stop-budget"; import type { WindowsStopConfirmResult } from "./windows-stop-confirm"; -import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; +import { claimPendingTeardown, clearPendingTeardown, type PendingTeardownReceipt } from "../config/pending-teardown"; +import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsPartARecord, windowsReceiptEndpoint, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; import { schtasks, probeWindowsSchedulerTask, rollbackWindowsSchedulerTaskOwnedByAttempt, settleDelay } from "./windows-scheduler"; import type { WindowsSchedulerTaskProbe } from "./windows-scheduler"; import { windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy } from "./windows-taskxml"; @@ -281,10 +282,51 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { * so Part B confirmed the wrapper's port while the proxy's own port was never checked. */ windowsStopPort?: () => number; - stopTrackedProxy?: () => Promise; + /** Receives the deferral nonce so the wrapper's POST does not run the teardown. */ + stopTrackedProxy?: (io?: TrackedProxyStopIo) => Promise; + /** The endpoint a Windows install's teardown receipt names (runtime record, else config). */ + teardownEndpoint?: typeof windowsReceiptEndpoint; + claimTeardown?: typeof claimPendingTeardown; + clearTeardown?: (nonce: string) => boolean; platform?: NodeJS.Platform; } +/** + * What an install preparation left behind. + * + * `deferredTeardownNonce` exists only on Windows after Part A stopped: the install stopped + * the old proxy with the shared teardown deferred, so it owes the receipt. The install that + * replaces the proxy keeps shared config pointed at OpenCodex and completes the receipt + * after the new service is installed and started; a failed install leaves it for the next + * `ocx stop` to finish. + */ +export interface ServiceInstallPreparationResult { + deferredTeardownNonce?: string; +} + +/** + * Claim the deferred-teardown receipt the install's tracked stop will name. + * + * Part A leaves the scheduler task registered, so a receipt-less POST answers 409 + * `respawnable_service` and the stop throws while the old proxy keeps serving. The + * receipt is claimed with Part A's record before the POST, exactly as `ocx service stop` + * does; without it no undeferred stop is sent. + */ +function claimInstallTeardownReceipt( + deps: ServiceInstallPreparationDeps, + partA: WindowsCheckedStopResult, +): PendingTeardownReceipt { + const { endpoint, source } = (deps.teardownEndpoint ?? windowsReceiptEndpoint)(); + try { + return (deps.claimTeardown ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)); + } catch (error) { + throw new Error( + "The deferred-teardown receipt could not be recorded, so the proxy was left running and nothing was replaced: " + + (error instanceof Error ? error.message : String(error)), + ); + } +} + /** * Stop every manager that could own the install port, then stop the tracked * standalone listener. Any unknown status or cleanup failure rejects, so callers @@ -293,7 +335,7 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { export async function prepareServiceInstall( requestedBackend: ServiceBackend, deps: ServiceInstallPreparationDeps = {}, -): Promise { +): Promise { const latch = deps.supervisionLatch ?? createSupervisionLatch(); assertNoDesktopSupervision(deps.inspectSupervision, latch); const platform = deps.platform ?? process.platform; @@ -312,11 +354,21 @@ export async function prepareServiceInstall( if (partA.outcome !== "stopped" && partA.outcome !== "absent") { throw new Error(`The installed Windows service could not be proven stopped (${partA.outcome}); nothing was replaced. ${describeWindowsPartA(partA)}`); } + let nonce: string | undefined; + if (partA.outcome === "stopped") { + // Before the tracked stop: a receipt-less POST would answer 409 and abort the install + // while the old proxy kept serving. + nonce = claimInstallTeardownReceipt(deps, partA).nonce; + } // Part B owns the port wait after a stopped manager (030 "Time budget"). - await (deps.stopTrackedProxy ?? (() => stopTrackedProxyIfRunning({ + await (deps.stopTrackedProxy ?? ((io?: TrackedProxyStopIo) => stopTrackedProxyIfRunning(io ?? { skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, - })))(); + })))({ + ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), + skipPortWait: partA.outcome === "stopped", + orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, + }); if (partA.outcome === "stopped") { const partB = await (deps.windowsPartB ?? confirmWindowsServiceStop)({ port: stopPort, @@ -325,8 +377,9 @@ export async function prepareServiceInstall( if (partB.outcome !== "freed") { throw new Error(`The service port is not free after the stop; nothing was published or started. ${describeWindowsPartB(partB)}`); } + return { deferredTeardownNonce: nonce }; } - return; + return {}; } const resolveOps = deps.managerOps ?? platformServiceInstallCleanupOps; const backends: ServiceBackend[] = []; @@ -341,6 +394,8 @@ export async function prepareServiceInstall( if (manager.status() !== null) manager.stop(); } await (deps.stopTrackedProxy ?? stopTrackedProxyIfRunning)(); + // No manager deferral off Windows: the launchd/systemd stop is down when it reports so. + return {}; } export async function installServiceSafely( @@ -353,9 +408,16 @@ export async function installServiceSafely( const runtime = Object.freeze({ ...(deps.selectRuntime ?? durableBunRuntime)() }); const configDir = (deps.configDir ?? getConfigDir)(); (deps.assertRuntimeWritable ?? assertSelectedRuntimeWritable)(runtime, configDir, { platform: deps.platform, rootWasAbsent: !existsSync(configDir) }); - await prepareServiceInstall(requestedBackend, { ...deps, supervisionLatch: latch }); + const prepared = await prepareServiceInstall(requestedBackend, { ...deps, supervisionLatch: latch }); assertNoDesktopSupervision(deps.inspectSupervision, latch); await install(runtime); + // The new service is installed and started, so shared config stays pointed at OpenCodex + // and the deferral is discharged. An install failure above throws before this line and + // keeps the receipt for the next stop. + const nonce = prepared.deferredTeardownNonce; + if (nonce && !(deps.clearTeardown ?? clearPendingTeardown)(nonce)) { + console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); + } } export interface FreshWindowsSchedulerInstallDeps extends RuntimePreflightDeps { @@ -364,7 +426,7 @@ export interface FreshWindowsSchedulerInstallDeps extends RuntimePreflightDeps { stageRegistrationXml?: (attemptNonce: string) => string; register?: (xmlPath: string, attemptNonce: string) => Promise; recordOwnership?: () => boolean; - prepare?: () => Promise; + prepare?: () => Promise; removeNativeService?: () => void; publishAssets?: (runtime: DurableBunRuntime) => void; verifyBeforeRun?: (attemptNonce: string) => void | Promise; @@ -446,7 +508,7 @@ export async function installFreshWindowsSchedulerSafely( ); } (deps.assertRuntimeWritable ?? assertSelectedRuntimeWritable)(runtime, configDir, { platform: deps.platform, rootWasAbsent: false }); - await prepare(); + const prepared = await prepare(); assertNoDesktopSupervision(deps.inspectSupervision, latch); removeNativeService(); publishAssets(runtime); @@ -455,6 +517,13 @@ export async function installFreshWindowsSchedulerSafely( runTask(); started = true; writeState(runtime); + // A prepare that stopped an old manager claimed the deferral receipt; the replacement + // is running now, so shared config stays pointed at OpenCodex and the obligation is + // discharged. A failure above keeps it for the next stop. + const nonce = prepared && "deferredTeardownNonce" in prepared ? prepared.deferredTeardownNonce : undefined; + if (nonce && !clearPendingTeardown(nonce)) { + console.warn("⚠️ The deferred-teardown receipt could not be removed; the next 'ocx stop' will finish it."); + } } catch (error) { const detail = error instanceof Error ? error.message : String(error); if (registered && !started) { diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts index aadeb24d996..c4176ce4b1a 100644 --- a/src/service/windows-service-stop.ts +++ b/src/service/windows-service-stop.ts @@ -231,6 +231,21 @@ export function windowsPartARecord(result: WindowsCheckedStopResult): PendingTea }; } +/** + * The endpoint a teardown receipt should name: the runtime record when it has one, the + * configured address otherwise. `exact` is the only source a stop may defer against. + */ +export function windowsReceiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { + const runtime = readRuntimePort(); + if (runtime?.port) return { endpoint: { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port }, source: "exact" }; + try { + const config = loadConfig(); + return { endpoint: { hostname: config.hostname ?? "127.0.0.1", port: typeof config.port === "number" && config.port > 0 ? config.port : 10100 }, source: "guessed" }; + } catch { + return { endpoint: { hostname: "127.0.0.1", port: 10100 }, source: "guessed" }; + } +} + /** The port Part B watches: runtime record, then the wrapper's baked port, then the config. */ export function windowsServiceStopPort(): number { return resolveWindowsStopPort({ diff --git a/tests/service/service-install-wrapper-cleanup.test.ts b/tests/service/service-install-wrapper-cleanup.test.ts index a6a8dc01861..78cf72cb2c0 100644 --- a/tests/service/service-install-wrapper-cleanup.test.ts +++ b/tests/service/service-install-wrapper-cleanup.test.ts @@ -102,6 +102,70 @@ describe("service install preparation wrapper cleanup", () => { expect(calls).toEqual(["partA", "stop:standalone"]); }); + test("a stopped Part A claims a receipt before the tracked stop and forwards its nonce", async () => { + // #7771 round 9: Part A leaves the scheduler task registered, so the tracked stop's + // /api/stop answers 409 respawnable_service without a receipt. stopProxy then throws and + // the install aborts while the old proxy keeps serving. The install claims the receipt + // with Part A's record before the stop, exactly as `ocx service stop` does. + const calls: string[] = []; + const claimed: Array<{ endpoint: { hostname: string; port: number }; partA: unknown }> = []; + const stopIo: Array<{ deferSharedTeardownNonce?: string }> = []; + const cleared: string[] = []; + let installed = false; + await installServiceSafely("scheduler", () => { calls.push("install"); installed = true; }, { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, + windowsStopPort: () => 12000, + teardownEndpoint: () => ({ endpoint: { hostname: "127.0.0.1", port: 12000 }, source: "exact" }), + claimTeardown: (endpoint, _source, _pid, _handover, partA) => { + calls.push("claim"); + claimed.push({ endpoint, partA }); + return { nonce: "nonce-1" }; + }, + clearTeardown: nonce => { calls.push("clear"); cleared.push(nonce); return true; }, + stopTrackedProxy: async io => { calls.push("stop:standalone"); stopIo.push(io ?? {}); }, + }); + expect(installed).toBe(true); + expect(calls).toEqual(["partA", "claim", "stop:standalone", "partB", "install", "clear"]); + expect(claimed).toEqual([{ + endpoint: { hostname: "127.0.0.1", port: 12000 }, + partA: { partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }, + }]); + expect(stopIo).toEqual([{ deferSharedTeardownNonce: "nonce-1", skipPortWait: true, orphanDiscoveryCapMs: expect.any(Number) }]); + // The install replaces the proxy, so shared config stays pointed at OpenCodex and the + // obligation is discharged only once the new service is installed and started. + expect(cleared).toEqual(["nonce-1"]); + }); + + test("an install that fails after the stop keeps the receipt for the next stop", async () => { + const cleared: string[] = []; + await expect(installServiceSafely("scheduler", () => { throw new Error("install failed"); }, { + platform: "win32", + windowsPartA: async () => partAResult("stopped"), + windowsPartB: async () => partBResult("freed"), + windowsStopPort: () => 12000, + teardownEndpoint: () => ({ endpoint: { hostname: "127.0.0.1", port: 12000 }, source: "exact" }), + claimTeardown: () => ({ nonce: "nonce-2" }), + clearTeardown: nonce => { cleared.push(nonce); return true; }, + stopTrackedProxy: async () => {}, + })).rejects.toThrow("install failed"); + expect(cleared).toEqual([]); + }); + + test("a receipt that cannot be claimed refuses before the tracked stop", async () => { + const calls: string[] = []; + await expect(prepareServiceInstall("scheduler", { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + windowsStopPort: () => 12000, + teardownEndpoint: () => ({ endpoint: { hostname: "127.0.0.1", port: 12000 }, source: "exact" }), + claimTeardown: () => { throw new Error("disk full"); }, + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + })).rejects.toThrow(/deferred-teardown receipt could not be recorded[\s\S]*disk full/); + expect(calls).toEqual(["partA"]); + }); + test("non-Windows install preparation never reaches the checked Windows stop", async () => { let partA = 0; await prepareServiceInstall("scheduler", { From e01a97375278da4d7da0b6c071ae7b7b1cd1c8b5 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:29:11 +0900 Subject: [PATCH 43/53] fix(service): prove every endpoint down when install preparation finds no manager With Part A absent, Part B is skipped and the tracked stop can return "none" after its orphan-discovery cap while an unresponsive proxy still holds the port, so the install published and started over it. prepareServiceInstall now resolves the candidate set before cleanup and requires every endpoint to be definitively dead; live or unknown refuses before the install callback. The candidate/probe helpers move to windows-service-stop so the removal and install paths share them (Refs #7771). --- src/service/cli-windows-stop.ts | 59 +++---------------- src/service/orchestration.ts | 22 ++++++- src/service/windows-service-stop.ts | 37 ++++++++++++ .../service-install-wrapper-cleanup.test.ts | 49 +++++++++++++++ 4 files changed, 114 insertions(+), 53 deletions(-) diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index 3de026acb0f..1e2b528dc76 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -15,18 +15,19 @@ * receipt is kept until the caller's restore succeeds. */ import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-teardown"; -import { readRuntimePort } from "../config/process-state"; -import { loadConfig } from "../config"; import { readClientConnectionState } from "../client/state"; -import { endpointsToProve, type ProbeEndpoint } from "../cli/uninstall-plan"; +import type { ProbeEndpoint } from "../cli/uninstall-plan"; import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; import { proxyStillLiveAfterStop, stopTrackedProxyForServiceCommand } from "./orchestration"; import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, + endpointsAllDead, runWindowsServiceStopPartA, windowsPartARecord, + windowsReceiptEndpoint, + windowsRemovalEndpoints, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, @@ -50,52 +51,6 @@ export interface WindowsServiceStopCommandIo { endpoints?: () => ProbeEndpoint[]; } -/** - * Every endpoint removal has to account for, exactly the set `ocx uninstall` proves. - * - * The receipt endpoint is ONE candidate; a stale runtime record naming a closed port must - * not stand in for the configured port where a live listener can sit (#3008, #7771). - */ -function removalEndpoints(): ProbeEndpoint[] { - let config: { port?: number; hostname?: string } = {}; - try { config = loadConfig(); } catch { /* defaults below */ } - return endpointsToProve(readRuntimePort(), config); -} - -/** - * Definitive absence on EVERY candidate. "unknown" is not absence: a listener that - * accepts connections but withholds /healthz, or one that times out, is exactly the state - * where deleting the registration is most harmful. - */ -async function endpointsAllDead( - endpoints: readonly ProbeEndpoint[], - probe: (endpoint: ProbeEndpoint) => Promise, -): Promise<{ dead: boolean; live: number[]; unknown: number[] }> { - const results = await Promise.all(endpoints.map(async endpoint => ( - await probe(endpoint).catch((): EndpointLiveness => "unknown") - ))); - const live: number[] = []; - const unknown: number[] = []; - endpoints.forEach((endpoint, index) => { - const answer = results[index]; - if (answer === "live") live.push(endpoint.port); - else if (answer !== "dead") unknown.push(endpoint.port); - }); - // No candidate is not proof of anything. - return { dead: endpoints.length > 0 && live.length === 0 && unknown.length === 0, live, unknown }; -} - -function receiptEndpoint(): { endpoint: { hostname: string; port: number }; source: "exact" | "guessed" } { - const runtime = readRuntimePort(); - if (runtime?.port) return { endpoint: { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port }, source: "exact" }; - try { - const config = loadConfig(); - return { endpoint: { hostname: config.hostname ?? "127.0.0.1", port: typeof config.port === "number" && config.port > 0 ? config.port : 10100 }, source: "guessed" }; - } catch { - return { endpoint: { hostname: "127.0.0.1", port: 10100 }, source: "guessed" }; - } -} - /** Shared teardown belongs to this home unless a client connection owns the routing. */ function ownsSharedTeardownByDefault(): boolean { try { @@ -124,7 +79,7 @@ export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopComm } let nonce: string | undefined; if ((io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { - const { endpoint, source } = receiptEndpoint(); + const { endpoint, source } = windowsReceiptEndpoint(); try { nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)).nonce; } catch (error) { @@ -176,7 +131,7 @@ export async function windowsServiceRemovalStop( const port = windowsServiceStopPort(); // Resolved once: the receipt and the absent-Part-A probe must name the same address, or // the proof would be about an endpoint the stop never contacted. - const { endpoint, source } = receiptEndpoint(); + const { endpoint, source } = windowsReceiptEndpoint(); const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); if (partA.outcome !== "stopped" && partA.outcome !== "absent") { console.error("❌ Service uninstall refused: the Windows service could not be proven stopped (" + partA.outcome + "), so the task was not deleted."); @@ -208,7 +163,7 @@ export async function windowsServiceRemovalStop( // a stale runtime record pointing at a closed port used to prove removal while the // configured port had a live listener. No respawn hold: with no manager registered // there is nothing to relaunch the proxy. - const proof = await endpointsAllDead((io.endpoints ?? removalEndpoints)(), io.endpointLiveness ?? probeEndpointLiveness); + const proof = await endpointsAllDead((io.endpoints ?? windowsRemovalEndpoints)(), io.endpointLiveness ?? probeEndpointLiveness); if (proof.dead) return withNonce(nonce); console.error(proof.live.length > 0 ? "❌ Service uninstall refused: a proxy is still listening on port " + proof.live.join(", ") + "." diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index 6b39a231934..a3ac97c42c2 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -21,7 +21,9 @@ import type { WindowsCheckedStopResult } from "./windows-checked-stop"; import { WINDOWS_STOP_CAPS } from "./windows-stop-budget"; import type { WindowsStopConfirmResult } from "./windows-stop-confirm"; import { claimPendingTeardown, clearPendingTeardown, type PendingTeardownReceipt } from "../config/pending-teardown"; -import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsPartARecord, windowsReceiptEndpoint, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; +import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; +import type { ProbeEndpoint } from "../cli/uninstall-plan"; +import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, endpointsAllDead, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsPartARecord, windowsReceiptEndpoint, windowsRemovalEndpoints, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; import { schtasks, probeWindowsSchedulerTask, rollbackWindowsSchedulerTaskOwnedByAttempt, settleDelay } from "./windows-scheduler"; import type { WindowsSchedulerTaskProbe } from "./windows-scheduler"; import { windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy } from "./windows-taskxml"; @@ -288,6 +290,9 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { teardownEndpoint?: typeof windowsReceiptEndpoint; claimTeardown?: typeof claimPendingTeardown; clearTeardown?: (nonce: string) => boolean; + /** Every endpoint an absent-manager install must prove down, resolved once. */ + installEndpoints?: () => ProbeEndpoint[]; + endpointLiveness?: (endpoint: ProbeEndpoint) => Promise; platform?: NodeJS.Platform; } @@ -354,6 +359,13 @@ export async function prepareServiceInstall( if (partA.outcome !== "stopped" && partA.outcome !== "absent") { throw new Error(`The installed Windows service could not be proven stopped (${partA.outcome}); nothing was replaced. ${describeWindowsPartA(partA)}`); } + // With no manager registered, Part B is skipped and the tracked stop can come back + // "none" after its discovery cap while an unresponsive proxy still holds the port. + // Resolve the candidates now: the cleanup below deletes the runtime record that is one + // of them. Nothing may be published over a candidate that is not definitively dead. + const absentProofEndpoints = partA.outcome === "absent" + ? (deps.installEndpoints ?? windowsRemovalEndpoints)() + : []; let nonce: string | undefined; if (partA.outcome === "stopped") { // Before the tracked stop: a receipt-less POST would answer 409 and abort the install @@ -379,6 +391,14 @@ export async function prepareServiceInstall( } return { deferredTeardownNonce: nonce }; } + if (partA.outcome === "absent") { + const proof = await endpointsAllDead(absentProofEndpoints, deps.endpointLiveness ?? probeEndpointLiveness); + if (!proof.dead) { + throw new Error(proof.live.length > 0 + ? `A proxy is still listening on port ${proof.live.join(", ")}; nothing was published or started. Check for a manually started proxy, then retry.` + : `The endpoint on port ${(proof.unknown.join(", ") || stopPort)} could not be confirmed down; nothing was published or started. Check for a manually started proxy, then retry.`); + } + } return {}; } const resolveOps = deps.managerOps ?? platformServiceInstallCleanupOps; diff --git a/src/service/windows-service-stop.ts b/src/service/windows-service-stop.ts index c4176ce4b1a..aac593c73b0 100644 --- a/src/service/windows-service-stop.ts +++ b/src/service/windows-service-stop.ts @@ -15,6 +15,8 @@ * - a hook seam so caller tests run hermetically (no schtasks, PowerShell or FFI). */ import { loadConfig } from "../config"; +import { endpointsToProve, type ProbeEndpoint } from "../cli/uninstall-plan"; +import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; import type { PendingTeardownPartA } from "../config/pending-teardown"; import { readRuntimePort } from "../config/process-state"; import type { WinswStatus } from "../lib/winsw"; @@ -231,6 +233,41 @@ export function windowsPartARecord(result: WindowsCheckedStopResult): PendingTea }; } +/** + * Every endpoint this home could be serving on, exactly the set `ocx uninstall` proves. + * + * The receipt endpoint is ONE candidate; a stale runtime record naming a closed port must + * not stand in for the configured port where a live listener can sit (#3008, #7771). + */ +export function windowsRemovalEndpoints(): ProbeEndpoint[] { + let config: { port?: number; hostname?: string } = {}; + try { config = loadConfig(); } catch { /* defaults below */ } + return endpointsToProve(readRuntimePort(), config); +} + +/** + * Definitive absence on EVERY candidate. "unknown" is not absence: a listener that + * accepts connections but withholds /healthz, or one that times out, is exactly the state + * where deleting a registration or installing over it is most harmful. + */ +export async function endpointsAllDead( + endpoints: readonly ProbeEndpoint[], + probe: (endpoint: ProbeEndpoint) => Promise = probeEndpointLiveness, +): Promise<{ dead: boolean; live: number[]; unknown: number[] }> { + const results = await Promise.all(endpoints.map(async endpoint => ( + await probe(endpoint).catch((): EndpointLiveness => "unknown") + ))); + const live: number[] = []; + const unknown: number[] = []; + endpoints.forEach((endpoint, index) => { + const answer = results[index]; + if (answer === "live") live.push(endpoint.port); + else if (answer !== "dead") unknown.push(endpoint.port); + }); + // No candidate is not proof of anything. + return { dead: endpoints.length > 0 && live.length === 0 && unknown.length === 0, live, unknown }; +} + /** * The endpoint a teardown receipt should name: the runtime record when it has one, the * configured address otherwise. `exact` is the only source a stop may defer against. diff --git a/tests/service/service-install-wrapper-cleanup.test.ts b/tests/service/service-install-wrapper-cleanup.test.ts index 78cf72cb2c0..953281b4ab5 100644 --- a/tests/service/service-install-wrapper-cleanup.test.ts +++ b/tests/service/service-install-wrapper-cleanup.test.ts @@ -97,6 +97,8 @@ describe("service install preparation wrapper cleanup", () => { platform: "win32", windowsPartA: async () => { calls.push("partA"); return partAResult("absent"); }, windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async () => "dead", stopTrackedProxy: async () => { calls.push("stop:standalone"); }, }); expect(calls).toEqual(["partA", "stop:standalone"]); @@ -166,6 +168,53 @@ describe("service install preparation wrapper cleanup", () => { expect(calls).toEqual(["partA"]); }); + test("an absent manager still proves every endpoint down before the install callback", async () => { + // #7771 round 9: with Part A absent, Part B is skipped and the tracked stop can return + // "none" after its orphan-discovery cap while an unresponsive proxy still holds the port. + // The install must not publish assets over it. + const calls: string[] = []; + let installed = false; + await expect(installServiceSafely("scheduler", () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => { calls.push("partA"); return partAResult("absent"); }, + windowsStopPort: () => 12000, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }, { hostname: "127.0.0.1", port: 10100 }], + endpointLiveness: async endpoint => (endpoint.port === 10100 ? "live" : "dead"), + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + })).rejects.toThrow(/port 10100/); + expect(installed).toBe(false); + expect(calls).toEqual(["partA", "stop:standalone"]); + }); + + test("an absent manager with an unreadable endpoint refuses too", async () => { + let installed = false; + await expect(installServiceSafely("scheduler", () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => partAResult("absent"), + windowsStopPort: () => 12000, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }], + endpointLiveness: async () => "unknown", + stopTrackedProxy: async () => {}, + })).rejects.toThrow(/could not be confirmed down/); + expect(installed).toBe(false); + }); + + test("an absent manager with every endpoint dead proceeds without a receipt", async () => { + const calls: string[] = []; + let installed = false; + await installServiceSafely("scheduler", () => { installed = true; }, { + platform: "win32", + windowsPartA: async () => partAResult("absent"), + windowsStopPort: () => 12000, + installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }], + endpointLiveness: async () => "dead", + claimTeardown: () => { calls.push("claim"); return { nonce: "unused" }; }, + stopTrackedProxy: async () => { calls.push("stop:standalone"); }, + }); + expect(installed).toBe(true); + expect(calls).toEqual(["stop:standalone"]); + }); + test("non-Windows install preparation never reaches the checked Windows stop", async () => { let partA = 0; await prepareServiceInstall("scheduler", { From 72e2f6c81e36fc6feb41a11f60db71b4390a6d7c Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 09:32:23 +0900 Subject: [PATCH 44/53] docs(structure): record the round-9 checked-stop wiring decisions Removal probes before deleting, install claims and completes its teardown receipt, an absent Part A proves every endpoint down, and an unconfirmed uninstall stop asks the endpoint (Refs #7771). --- .../060_implementation.md | 4 +++ structure/ops/windows-checked-stop.md | 33 +++++++++++++++---- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 85770fbd095..2b9fa8f9ea2 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -86,6 +86,10 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **A foreign WinSW registration survives `ocx uninstall`.** Two gaps: on Windows the "service removed" step called `uninstallServiceDetailed()` even when Part A failed to stop the manager, and the native branch stopped and deleted whatever carried the fixed `opencodex-proxy-native` ID — including another installation's service. Removal is now gated on a Part A that ended stopped or absent (`serviceRemovalAllowed`, Windows only; other platforms unchanged), recording `failed` otherwise so the shared teardown stays blocked as `ocx service uninstall` already refuses. Immediately before the native removal, `uninstallServiceDetailed` re-reads `sc qc` `BINARY_PATH_NAME` and compares it with this home's WinSW executable: foreign leaves the registration untouched, unreadable refuses and keeps install state. - **Install preparation resolves the stop port before Part A.** `windowsServiceStopPort()` ran after the tracked cleanup had already removed the runtime record, so Part B confirmed the wrapper's baked port (10100) while the proxy's runtime port (12000) was never checked. `prepareServiceInstall` resolves the port once before Part A (`windowsStopPort` is the test seam) and carries it into Part B. - **Part A proves the WinSW registration is this home's.** `sc query` answering "running" says a service exists under the fixed `opencodex-proxy-native` ID, not that it is this installation's. The live probe now reads `BINARY_PATH_NAME` through `sc qc` and requires this home's `winswExePath()`, the same proof `inspectWindowsWinswManager` uses; an unreadable registration is `unknown` (`evidence-unreadable`, nothing changed) and a foreign executable is reported as `nonexistent`, so Part A does not stop it and reports `absent`. +- **Install preparation claims its receipt before the stopped-Part-A stop.** Part A leaves the scheduler task registered, so the tracked stop's receipt-less POST answered `respawnable_service` (409), `stopProxy` threw, and the install aborted while the old proxy kept serving. `prepareServiceInstall` now claims a receipt carrying Part A's record before the stop and forwards its nonce. An install replaces the proxy, so shared config stays pointed at OpenCodex and `installServiceSafely` (and the fresh-scheduler path) clears the receipt only after the new service is installed and started; an install that fails after the stop keeps it for the next `ocx stop`. A failed claim sends no child stop. +- **An absent Part A still proves every endpoint down before install.** With no manager registered, Part B is skipped and the tracked stop can return `"none"` after its orphan-discovery cap while an unresponsive proxy still holds the port. `prepareServiceInstall` resolves `endpointsToProve(readRuntimePort(), loadConfig())` before cleanup and requires every candidate to be definitively dead; live or unknown throws like a Part B failure. The candidate and tri-state helpers live in `windows-service-stop.ts` so removal and install share them. +- **Windows removal probes before it deletes.** `uninstallServiceDetailed` deleted the scheduler task before reading the WinSW status and `sc qc` ownership, so an unreadable probe left a half-removed install with no task to retry from. Both probes now run first; unknown refuses with the existing error and removes nothing. +- **An unconfirmed uninstall stop asks the endpoint.** `stopProxy` answers false without contacting the endpoint when the tracked pid is already dead, so `confirmed(false)` reported `provenDown` with no endpoint check and a different proxy on the configured port was missed. The step now requires the tri-state `endpointProvenDown()` unless the forced fallback is forbidden, and still never forces. - **Part B clears only the exited owner's TCP rows.** After a listener PID was found exited, the sampler called `dropWindowsTcpRowsForLocalPort(port)`, which resets every IPv4 row on the port — including rows of a program that took the port between scans. `parseTcpQuadsForLocalPort` and `dropWindowsTcpRowsForLocalPort` take an owner PID filter and the sampler passes the exited PID(s); the hard-kill path still clears the whole port. ## Follow-up slices diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 4ab0e307806..9c4e645b900 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -180,9 +180,11 @@ stop instead of sliding forward. the scheduler task registered, so a receipt-less POST answers `respawnable_service` (409) and the removal refuses with the proxy left running. The receipt is discharged only after the shared restore, and a failed claim sends no child stop. Part A `absent` says no manager is - registered, not that nothing serves, so removal then probes the endpoint it captured, tri-state: - only a definitive dead permits deleting the task, while live or unknown refuses and keeps the - receipt. + registered, not that nothing serves, so removal then proves every candidate endpoint + (`endpointsToProve(readRuntimePort(), loadConfig())`, the set `ocx uninstall` proves) tri-state: + only a definitive dead on all of them permits deleting the task, while live or unknown refuses + and keeps the receipt. The receipt endpoint alone is not enough — a stale runtime record naming + a closed port must not stand in for the configured port where a live listener can sit. - Every tracked-proxy stop takes its orphan-discovery cap from the caller (`TrackedProxyStopIo.orphanDiscoveryCapMs`). The checked Windows callers pass the admitted 3 s (`WINDOWS_STOP_CAPS.orphanDiscovery`), which is what admission priced; the omitted default stays @@ -196,16 +198,33 @@ stop instead of sliding forward. when state changed. When Part A did not end stopped or absent the graceful POST still goes out with `allowForcedFallback: false`, and that unconfirmed wait is observational: it samples the port without `reclaimListenPort`'s kill eligibility, so a forbidden fallback cannot kill through - the reclaim path either. + the reclaim path either. An unconfirmed stop is not proof the proxy is down: `stopProxy` answers + false without contacting the endpoint when the tracked pid is already dead, so the step then + requires the tri-state `endpointProvenDown()` over every candidate before it reports + `provenDown`, exactly as the discovery path does. A different proxy on the configured port + would otherwise be missed and shared config restored under it (#3008). - `ocx uninstall` removes the Windows manager only after a Part A that ended stopped or absent (`serviceRemovalAllowed` in `src/cli/uninstall-plan.ts`); otherwise the step records `failed` and leaves the registration, as `ocx service uninstall` refuses. `uninstallServiceDetailed` - (`src/service/orchestration.ts`) re-reads `sc qc` `BINARY_PATH_NAME` immediately before the - native removal: a registration naming another installation's WinSW executable is left untouched, - and an unreadable one refuses and keeps install state. Non-Windows removal is unchanged. + (`src/service/orchestration.ts`) reads the scheduler probe, the native status and `sc qc` + `BINARY_PATH_NAME` before the first destructive step: a registration naming another + installation's WinSW executable is left untouched, and an unreadable status or ownership + refuses and keeps install state. The task used to be deleted first, so an unreadable `sc qc` + left a half-removed install with no task to retry from. Non-Windows removal is unchanged. - `prepareServiceInstall` refuses before replacing anything on a Part A other than stopped or absent and stops install and switch on any Part B result other than a free port, naming the owner. `ocx uninstall` runs Part B as its respawn window after a stopped Part A. +- Install and backend switch claim a teardown receipt before their tracked stop when Part A + ended `stopped`: the task is still registered, so a receipt-less POST answers 409 + `respawnable_service` and the install aborted while the old proxy kept serving. The receipt + carries Part A's record and its nonce travels on the POST. An install replaces the proxy, so + shared config stays pointed at OpenCodex and the receipt is cleared only after the new service + is installed and started; an install that fails after the stop keeps it for the next + `ocx stop`. A failed claim sends no child stop. +- With Part A `absent`, install preparation skips Part B, and the tracked stop can return + `"none"` after its orphan-discovery cap while an unresponsive proxy still holds the port. The + candidate set is resolved before cleanup and every endpoint must be definitively dead before + the install callback runs; live or unknown throws like a Part B failure. - The updater (`src/update/job-windows-restart.ts`) uses `guardedStopWindowsChecked` for Task Scheduler, with the service mutation gate as the authorizer; WinSW keeps its stop. Each of the three call sites in `src/update/job.ts` aborts the restart unless the outcome is stopped or absent. From d0110a1c679559fc5c2cef5e08ffa7effc8d49bb Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:11:26 +0900 Subject: [PATCH 45/53] Hold Windows child stop when Part A throws --- src/cli/index.ts | 4 ++-- src/cli/stop-windows-checked.ts | 2 +- structure/ops/windows-checked-stop.md | 2 +- tests/fixtures/parent-stop-runner.ts | 5 +++-- tests/windows/windows-stop-ocx-stop.test.ts | 8 ++++++++ 5 files changed, 15 insertions(+), 6 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index df003ef0728..6265881e113 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -1230,10 +1230,10 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { ownershipBlocked = windowsPartAHolds(record) || ownershipBlocked; // #7771: no POST and no shared teardown. } catch (err) { record.service = "error"; - if (isServiceOwnershipError(err)) { + if (isServiceOwnershipError(err) || windowsChildStopHeld()) { ownershipBlocked = true; stopFailed = true; - console.error(`❌ ${err.message}`); + console.error(`❌ ${err instanceof Error ? err.message : String(err)}`); console.error(" Skipping shared teardown (native Codex restore, Grok config): the installed service is still running."); } else { console.error(`⚠️ Service manager stop failed: ${err instanceof Error ? err.message : String(err)}`); diff --git a/src/cli/stop-windows-checked.ts b/src/cli/stop-windows-checked.ts index 4f66baf3634..97817982f02 100644 --- a/src/cli/stop-windows-checked.ts +++ b/src/cli/stop-windows-checked.ts @@ -117,7 +117,7 @@ export function windowsPartAReceiptRecord(): PendingTeardownPartA | null { /** Part A held the child stop; handleStopUnlocked skips the POST branch entirely. */ export function windowsChildStopHeld(): boolean { const outcome = state.result?.outcome; - return windows() && state.snapshotless && !!outcome && outcome !== "stopped" && outcome !== "absent"; + return windows() && state.snapshotless && outcome !== "stopped" && outcome !== "absent"; } /** ocx uninstall's service stop: Part A on Windows (60 s deadline), today's stop elsewhere. */ diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 9c4e645b900..a6bae2dadf0 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -146,7 +146,7 @@ stop instead of sliding forward. - Plain `ocx stop` (`src/cli/stop-windows-checked.ts`) admits against the deadline with orphan discovery and the inherited receipts counted. A Part A other than stopped or absent holds the child: no `/api/stop` POST, the shared teardown skipped, `service` failed or state-unknown, - `proxy` unknown. After a stopped Part A, the POST skips `stopProxy`'s legacy 15 s + `proxy` unknown. A thrown Part A also holds the child and reports a service error. After a stopped Part A, the POST skips `stopProxy`'s legacy 15 s `waitForStoppedPort`, orphan discovery is capped at 3 s, and Part B replaces the respawn re-poll: our identity back on the port is `proxy: respawned`, an unconfirmed port is `evidence-unreadable`, and another program on the port sets `portHeldByForeign` without failing diff --git a/tests/fixtures/parent-stop-runner.ts b/tests/fixtures/parent-stop-runner.ts index 97abec961e7..45d14f592fc 100644 --- a/tests/fixtures/parent-stop-runner.ts +++ b/tests/fixtures/parent-stop-runner.ts @@ -61,6 +61,7 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { * no WinSW service registered, so the own-wrapper refusal has to answer for itself. */ windowsPartAUnderOwnWrapper?: boolean; + windowsPartAReject?: boolean; }; if (options.proxyProbePort) { // Deliberately bypass every fixture mock for the real proxy-environment regression. @@ -313,12 +314,12 @@ mock.module("../../src/server/direct-local-http", () => ({ ...direct, })); process.argv = [process.execPath, "ocx", "stop", ...(options.json ? ["--json"] : [])]; -if (options.windowsPartA) { +if (options.windowsPartA || options.windowsPartAReject) { // The stop wiring acts for win32 while the host stays untouched: Part A and Part B answer here. (await import("../../src/cli/stop-windows-checked")).resetOcxWindowsStopForTests("win32"); const { partAResult, partBResult } = await import("../helpers/windows-stop-caller-fixtures"); (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ - partA: async () => { calls.service++; return partAResult(options.windowsPartA!); }, + partA: async () => { calls.service++; if (options.windowsPartAReject) throw new Error("wrapper watch rejected"); return partAResult(options.windowsPartA!); }, partB: async () => { calls.partB++; return partBResult(options.windowsPartB ?? "freed"); }, // The guarded inspection reads evidence before it asks the mocked manager; nothing may // reach the host's process table. diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts index 5dc72d6aaeb..c7269bb3310 100644 --- a/tests/windows/windows-stop-ocx-stop.test.ts +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -46,6 +46,14 @@ function runStop(windowsPartA: string | undefined, windowsPartB?: string, extra: } describe("plain ocx stop on Windows: Part A holds the child stop", () => { + test("a rejected Part A holds the child and shared teardown", () => { + const run = runStop(undefined, undefined, { windowsPartAReject: true }); + expect(run.urls).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ outcome: "failed", service: "error", proxy: "unknown", sharedTeardown: "skipped" }); + }); for (const [outcome, service, reason] of [ ["denied", "failed", "access-denied"], ["survived", "failed", undefined], From 2626100d04c8a2112d37aabbf7559038c1ee6991 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:12:35 +0900 Subject: [PATCH 46/53] Refuse Windows stop when teardown receipt fails --- src/cli/index.ts | 7 +++---- src/cli/stop-windows-checked.ts | 4 ++++ structure/ops/windows-checked-stop.md | 3 ++- tests/windows/windows-stop-ocx-stop.test.ts | 9 +++++++++ 4 files changed, 18 insertions(+), 5 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index 6265881e113..e62225e3461 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -126,7 +126,7 @@ import { type StopApproval, } from "./stop-approval"; import { summarizeStopRun, type StopOutcome, type StopRunRecord } from "./stop-report"; -import { confirmGuardedFreed, inspectGuardedTarget, noServiceStop, observeGuardedStopped, orphanDiscoveryCap, stopGuardedManager, stopServiceForOcxStop, stopServiceForUninstall, stopWindowFailed, verifyUninstallStopWindow, windowsChildStopHeld, windowsPartAHolds, windowsPartAReceiptRecord, windowsPartBDue, windowsPartBOwnsPortWait } from "./stop-windows-checked"; +import { confirmGuardedFreed, inspectGuardedTarget, noServiceStop, observeGuardedStopped, orphanDiscoveryCap, stopGuardedManager, stopServiceForOcxStop, stopServiceForUninstall, stopWindowFailed, verifyUninstallStopWindow, windowsChildStopHeld, windowsPartAHolds, windowsPartAReceiptRecord, windowsPartBDue, windowsPartBOwnsPortWait, windowsStopRequiresReceipt } from "./stop-windows-checked"; import { runCli } from "./root"; import { isProcessAlive, ProxyOwnershipRefusedError, refusalNextStep, stopProxy } from "../lib/process-control"; import { startupDataPlaneToken } from "../lib/service-secrets"; @@ -1150,9 +1150,8 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { try { teardownNonce = claimPendingTeardown(endpoint, endpointSource, process.pid, handoverRequested ? "update-restart" : undefined, windowsPartAReceiptRecord()).nonce; } catch (err) { - // Without a receipt the proxy performs its own teardown, which is the pre-#3008 - // behaviour: correct for every backend that cannot respawn, and merely early for - // Task Scheduler. Losing the deferral is far better than losing the stop. + // Windows wrappers can relaunch the proxy, so an undeferred stop is unsafe. + if (windowsStopRequiresReceipt()) { ownershipBlocked = true; stopFailed = true; throw err; } console.warn(`⚠️ Could not record the deferred-teardown receipt: ${err instanceof Error ? err.message : String(err)}`); } }; diff --git a/src/cli/stop-windows-checked.ts b/src/cli/stop-windows-checked.ts index 97817982f02..35299e01a11 100644 --- a/src/cli/stop-windows-checked.ts +++ b/src/cli/stop-windows-checked.ts @@ -70,6 +70,10 @@ function windows(): boolean { return state.platform === "win32"; } +export function windowsStopRequiresReceipt(): boolean { + return windows(); +} + function begin(snapshotless: boolean): void { state.result = null; state.snapshotless = snapshotless; diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index a6bae2dadf0..979f9561cf1 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -152,7 +152,8 @@ stop instead of sliding forward. `evidence-unreadable`, and another program on the port sets `portHeldByForeign` without failing the stop. The port wait is skipped on BOTH Windows paths, guarded included, through `windowsPartBOwnsPortWait`; `windowsPartBDue` stays snapshotless-only because it also gates the - legacy respawn re-poll. `ocx uninstall`'s two `stopProxy` calls pass the same predicate. + legacy respawn re-poll. A failed owner receipt claim refuses the child stop and shared teardown + on both paths. `ocx uninstall`'s two `stopProxy` calls pass the same predicate. - The guarded path reads the evidence snapshot and the Schedule host PID once per inspection; `stopManager` binds on the revalidation's snapshot (so its admission does not count a second enumeration), Part B runs after `settle` on the approved endpoint, and the manager-state checks diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts index c7269bb3310..519fe8476dd 100644 --- a/tests/windows/windows-stop-ocx-stop.test.ts +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -46,6 +46,15 @@ function runStop(windowsPartA: string | undefined, windowsPartB?: string, extra: } describe("plain ocx stop on Windows: Part A holds the child stop", () => { + for (const guarded of [false, true]) { + test(`receipt failure holds the ${guarded ? "guarded" : "plain"} child stop`, () => { + const run = runStop("stopped", "freed", { receipt: false, guarded, windowsGuarded: guarded }); + expect(run.urls).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.calls.grok).toBe(0); + expect(run.exitCode).toBe(1); + }); + } test("a rejected Part A holds the child and shared teardown", () => { const run = runStop(undefined, undefined, { windowsPartAReject: true }); expect(run.urls).toEqual([]); From 5474c6d2af3408bb152751071acbac7687f70331 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:14:12 +0900 Subject: [PATCH 47/53] Check wrappers for WinSW-only stops --- devlog/_plan/261111_windows_service_stop/030_design.md | 4 ++-- .../261111_windows_service_stop/060_implementation.md | 2 +- src/service/windows-checked-stop.ts | 4 ++-- src/service/windows-stop-budget.ts | 4 ++-- structure/ops/windows-checked-stop.md | 6 ++++-- tests/fixtures/parent-stop-runner.ts | 3 ++- tests/windows/windows-checked-stop.test.ts | 10 +++++----- tests/windows/windows-service-stop-wiring.test.ts | 10 ++++++++++ tests/windows/windows-stop-ocx-stop.test.ts | 8 ++++++++ 9 files changed, 36 insertions(+), 15 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 56f01ae5008..760aeda7bf0 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -69,7 +69,7 @@ Worst case still to come, at admission: | Step | Worst case | Notes | |---|---|---| | part A, Task Scheduler | guarded 6 s (`/end` 2 s, handle wait 3 s, manager state 1 s; the bind reuses `revalidateManager`'s enumeration); snapshotless 11 s (its own enumeration, 5 s, plus the same 6 s) | the registration probe already ran before admission | -| part A, WinSW | 25 s: stop command 2 s + SCM poll `` + 2 s = 22 s + manager state 1 s | WinSW stops the proxy itself, so the POST and exit-wait rows below do not apply to it. The stray enumeration of step 1 does not apply either: it runs only when nothing is registered | +| part A, WinSW | 33 s: enumeration 5 s + wrapper termination/wait 3 s + stop command 2 s + SCM poll `` + 2 s = 22 s + manager state 1 s | WinSW stops the proxy itself, so the POST and exit-wait rows below do not apply to it. Enumeration also checks ancestry and stray wrappers when WinSW is registered but stopped. | | orphan discovery (snapshotless, no PID file; also after WinSW stopped the proxy) | 3 s | `findLiveProxy` and `proveLiveProxyOwnedByHome`, capped at 3 s together; an overrun is treated as "no proven orphan" and refuses, as an unproven owner does today. The cap is threaded to the helper (`orphanDiscoveryCapMs`) so a checked Windows caller passes exactly the 3 s admission priced; the helper's own default stays the legacy 7 s for macOS and Linux | | POST answer | 10 s, always | never shortened. The receipt and its part A record are written after admission (the snapshotless receipt inside `stopWithDeferral`), so admission cannot know whether they will exist and always counts the full abort. With the record the route skips part A and answers at once; without it (the write failed, or an older CLI) the route keeps today's behavior inside the same 10 s | | child exit: graceful exit wait, then forced | `shutdownTimeoutMs` + 3 s + 10 s = 18 s by default | today's ladder, kept in sequence: the graceful exit wait (never shortened) runs after an answered POST; if the child has not exited, or the POST aborted, `stopProxy` forces (never after a refusal): `taskkill` with a new 5 s timeout (it has none today) plus today's `waitForExit` 5 s | @@ -77,7 +77,7 @@ Worst case still to come, at admission: | part B | 6 s floor, up to `min(12 s, remaining − 5 s − reserve)` | `waitForStoppedPort` (today a separate 15 s wait inside `stopProxy`) is the same port-free predicate and on Windows becomes part B's first phase; `settle`'s `waitExit` and endpoint probe are its first sample; the final CIM enumeration (`managerState` and `guardFinalStopSummary` share it) runs after the hold, capped at 5 s, and is counted: part B's worst case is 6 s + 5 s + the 0.5 s closing sample = 11.5 s. An unreadable result or a wrapper found that was not there at bind is non-success. Not run after a guarded `absent` manager (where `stopManager` is not called, as today). When admission lands exactly at the minimum, part B has no slack beyond its floor, so a slow port release reports failure instead of overrunning. Part B keeps today's `reclaimListenPort` clearing of a listener left by the exited child, and a listener whose owner has exited is not classified as `portHeldByForeign` | | reserve | 3 s | shared restore (`restoreSharedClientStateAfterStop`; its worst case is measured in 040 and the reserve grows if it does not fit), `revertSystemEnv` (a real 1 s cap on what is a synchronous call today; an overrun is logged and left for the next start), and the JSON summary | -Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 + 18 + 11.5 + 3 = 48.5 s; snapshotless Task Scheduler 11 + 3 + 10 + 18 + 11.5 + 3 = 56.5 s; WinSW snapshotless 25 + 3 + 11.5 + 3 = 42.5 s. Part B is counted at 11.5 s: the 6 s hold, the 5 s final enumeration, and the 0.5 s closing port sample that can start at the very end of the window (the hold is measured between sample starts; 060 "Where PR1 differs from 030"). The stray enumeration of a deleted task (5 s) runs in step 1, before admission, and so comes out of the time left at admission. +Admission minimum with default settings and N = 0: guarded Task Scheduler 6 + 10 + 18 + 11.5 + 3 = 48.5 s; snapshotless Task Scheduler 11 + 3 + 10 + 18 + 11.5 + 3 = 56.5 s; WinSW snapshotless 33 + 3 + 11.5 + 3 = 50.5 s. Part B is counted at 11.5 s: the 6 s hold, the 5 s final enumeration, and the 0.5 s closing port sample that can start at the very end of the window (the hold is measured between sample starts; 060 "Where PR1 differs from 030"). The stray enumeration of a deleted task (5 s) runs in step 1, before admission, and so comes out of the time left at admission. | Caller | Deadline | Fits? | |---|---|---| diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 2b9fa8f9ea2..72931fbb6a0 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -99,5 +99,5 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C ## Open questions - Is the closing-sample accounting acceptable, or should the hold be measured differently, for example by shortening the final enumeration to 4.5 s to keep 030's totals? -- Should a WinSW-only stop also sweep a stray scheduler wrapper (that would add an enumeration and 3 s to the WinSW row)? +- WinSW-only stops enumerate and bind wrappers as well; the WinSW row includes the enumeration and termination window. - `GetLastError` after a failed `OpenProcess` through Bun FFI separates "gone" from "denied". If the runtime clobbers it, the answer degrades to "unavailable" (`unknown`). The live x64 test in `tests/windows/windows-process-evidence.test.ts` checks this on the Windows CI lane. diff --git a/src/service/windows-checked-stop.ts b/src/service/windows-checked-stop.ts index 79f4043c064..950c5f57d54 100644 --- a/src/service/windows-checked-stop.ts +++ b/src/service/windows-checked-stop.ts @@ -257,7 +257,7 @@ export async function stopWindowsServiceChecked( } // Step 2: bind. - if (taskRegistered) { + if (taskRegistered || winswRegistered) { let processes = options.bindSnapshot ?? null; if (processes === null) { const snapshot = await runCapped(clock, C.enumeration, deps.enumerate); @@ -267,7 +267,7 @@ export async function stopWindowsServiceChecked( processes = snapshot.value; } binding = bindWindowsStopWrappers({ - ...bindContext, processes, taskRegistered: true, managerRegistered: true, + ...bindContext, processes, taskRegistered, managerRegistered: true, scheduleServicePid: schedulePid, }); } diff --git a/src/service/windows-stop-budget.ts b/src/service/windows-stop-budget.ts index ca937c6c851..344696ffb32 100644 --- a/src/service/windows-stop-budget.ts +++ b/src/service/windows-stop-budget.ts @@ -79,10 +79,10 @@ export function windowsPartAWorstCaseMs(input: Pick, so the SCM poll is capped at that plus a margin. + // The service stop itself is capped at plus a margin. ms += C.winswStopCommand + stopTimeout + C.winswPollMargin + C.managerState; } // Nothing registered: the stray enumeration already ran in the registration step. diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 979f9561cf1..29abb4b4ac7 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -65,7 +65,9 @@ deepest first and carry pid, creation time, image and a SHA-256 digest of the co external effect is injected. It probes registration (2 s each). An unreadable answer is `unknown` with nothing changed. With nothing registered it enumerates once for a stray wrapper of a deleted task; an unreadable table or no wrapper is `absent`. With a task registered or a stray sweep -running, it reads the Schedule host PID (2 s). Admission then compares the time left on the +running, it reads the Schedule host PID (2 s). A WinSW-only registration also binds against a +process enumeration, including when WinSW is stopped, so ancestry and stray-wrapper evidence +are checked before a stop can be confirmed. Admission then compares the time left on the caller's deadline, which already excludes these first steps, with the worst case still to come, and returns `unverified-budget` without changing anything when it does not fit. It binds, ends the task (`schtasks /end`, 2 s) or stops WinSW (a @@ -95,7 +97,7 @@ configured port. `src/service/windows-stop-budget.ts` holds the caps and the admission arithmetic. With default settings the minimum is 48.5 s for a guarded Task Scheduler stop, 56.5 s for a snapshotless one -and 42.5 s for WinSW. Each includes the 0.5 s closing port sample that Part B can take past its +and 50.5 s for WinSW. Each includes the 0.5 s closing port sample that Part B can take past its window. ## Callers diff --git a/tests/fixtures/parent-stop-runner.ts b/tests/fixtures/parent-stop-runner.ts index 45d14f592fc..6ae4191848e 100644 --- a/tests/fixtures/parent-stop-runner.ts +++ b/tests/fixtures/parent-stop-runner.ts @@ -61,6 +61,7 @@ const options = JSON.parse(readFileSync(0, "utf8")) as { * no WinSW service registered, so the own-wrapper refusal has to answer for itself. */ windowsPartAUnderOwnWrapper?: boolean; + windowsWinswStopped?: boolean; windowsPartAReject?: boolean; }; if (options.proxyProbePort) { @@ -335,7 +336,7 @@ if (options.windowsPartAUnderOwnWrapper) { (await import("../../src/service/windows-service-stop")).setWindowsServiceStopHooksForTests({ liveDeps: () => ({ canonicalPaths: PATHS, childPid: null, selfPid: process.pid, - probeScheduler: () => { calls.service++; return { status: "absent" }; }, probeWinsw: () => "nonexistent", + probeScheduler: () => { calls.service++; return { status: "absent" }; }, probeWinsw: () => options.windowsWinswStopped ? "stopped" : "nonexistent", enumerate: () => [launcher(), loop(), self], scheduleServicePid: () => null, endTask: () => unexpected("schtasks /end"), taskState: () => "not-running", winswStop: () => "sent", winswState: () => "stopped", proxyAlive: () => false, diff --git a/tests/windows/windows-checked-stop.test.ts b/tests/windows/windows-checked-stop.test.ts index d5690b243d5..06b58b24b3c 100644 --- a/tests/windows/windows-checked-stop.test.ts +++ b/tests/windows/windows-checked-stop.test.ts @@ -272,8 +272,8 @@ describe("checked Windows stop, part A", () => { }); const result = await stopWindowsServiceChecked(h.deps); expect(result.outcome).toBe("survived"); - expect(mutations(h.calls)).toEqual(["winsw-stop"]); - expect(h.calls).not.toContain("enumerate"); + expect(mutations(h.calls)).toEqual(["winsw-stop", "terminate:30", "terminate:20"]); + expect(h.calls).toContain("enumerate"); }); test("WinSW: an SCM state that never reports stopped is unknown after + 2 s", async () => { @@ -420,7 +420,7 @@ describe("the updater's authorizer guards each manager stop", () => { describe("checked Windows stop budget", () => { test("admission minimums with default settings", () => { - // 030's 48 / 56 / 42 s plus the 0.5 s closing sample part B can take past its window. + // The closing sample and WinSW's wrapper enumeration and termination are included. expect(windowsStopWorstCase({ managers: { scheduler: true, winsw: false, stray: false }, enumerationReused: true, orphanDiscovery: false, }).totalMs).toBe(48_500); @@ -429,7 +429,7 @@ describe("checked Windows stop budget", () => { }).totalMs).toBe(56_500); expect(windowsStopWorstCase({ managers: { scheduler: false, winsw: true, stray: false }, enumerationReused: false, orphanDiscovery: true, - }).totalMs).toBe(42_500); + }).totalMs).toBe(50_500); }); test("receipts, a longer shutdown and a longer WinSW stop timeout grow the worst case", () => { @@ -439,7 +439,7 @@ describe("checked Windows stop budget", () => { expect(windowsStopWorstCase({ managers: { scheduler: false, winsw: true, stray: false }, enumerationReused: false, orphanDiscovery: false, winswStopTimeoutMs: 30_000, - }).rows.find(row => row.step === "part-a")!.ms).toBe(35_000); + }).rows.find(row => row.step === "part-a")!.ms).toBe(43_000); expect(windowsStopWorstCase({ ...base, partB: false }).rows.map(row => row.step)).not.toContain("part-b"); }); diff --git a/tests/windows/windows-service-stop-wiring.test.ts b/tests/windows/windows-service-stop-wiring.test.ts index 947a5f03b15..66eae6f9b5c 100644 --- a/tests/windows/windows-service-stop-wiring.test.ts +++ b/tests/windows/windows-service-stop-wiring.test.ts @@ -67,6 +67,16 @@ function machine(overrides: Partial & { gate?: () => boo } describe("Part A through the caller wiring", () => { + test("WinSW registered and stopped still checks the caller's wrapper ancestry", async () => { + const under = entry(70, 30, "ocx.exe", "ocx stop", 6_000); + const fake = machine({ selfPid: 70, probeScheduler: () => ({ status: "absent" }), + probeWinsw: () => "stopped", extra: [under] }); + setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); + const result = await runWindowsServiceStopPartA(); + expect(result.outcome).toBe("unknown"); + expect(result.detail).toBe(UNDER_OWN_WRAPPER_DETAIL); + expect(result.changed).toBe(false); + }); test("ends the bound wrappers with the live-style deps", async () => { const fake = machine(); setWindowsServiceStopHooksForTests({ liveDeps: () => fake.deps }); diff --git a/tests/windows/windows-stop-ocx-stop.test.ts b/tests/windows/windows-stop-ocx-stop.test.ts index 519fe8476dd..5d8eaf29d15 100644 --- a/tests/windows/windows-stop-ocx-stop.test.ts +++ b/tests/windows/windows-stop-ocx-stop.test.ts @@ -88,6 +88,14 @@ describe("plain ocx stop on Windows: Part A holds the child stop", () => { describe("plain ocx stop on Windows: Part B after a stopped Part A", () => { + test("stopped WinSW and own wrapper: unknown and no POST", () => { + const run = runStop(undefined, undefined, { windowsPartAUnderOwnWrapper: true, windowsWinswStopped: true }); + expect(run.urls).toEqual([]); + expect(run.calls.native).toBe(0); + expect(run.exitCode).toBe(1); + expect(run.summary).toMatchObject({ service: "state-unknown", proxy: "unknown", sharedTeardown: "skipped" }); + }); + test("under this home's wrapper with nothing registered: unknown, no POST, no shared teardown", () => { // The real Part A refuses: this stop runs under this home's .cmd/.vbs wrapper and no // service is registered. Reading that as absent would stop the child and restore shared From 9cff82dd079c88aafeaae0d5d79018fca53fc27a Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:15:29 +0900 Subject: [PATCH 48/53] Guard updater WinSW stops with checked stop --- .../261111_windows_service_stop/030_design.md | 2 ++ .../060_implementation.md | 1 + src/update/job-windows-restart.ts | 16 +++--------- structure/ops/windows-checked-stop.md | 4 +-- .../update-windows-wrapper-authority.test.ts | 26 ++++++++++++------- .../windows-deploy-close-regressions.test.ts | 4 +-- 6 files changed, 27 insertions(+), 26 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 760aeda7bf0..2996675ca0a 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -5,6 +5,8 @@ The proxy child is stopped by each caller AFTER the service stop, and `/api/stop` runs inside the very proxy that holds the port. A port check inside the service operation would therefore always time out. The design is split: - Part A, `stopWindowsServiceChecked()` (async; the current `stopServiceIfInstalledDetailed` and the guarded `stopManager` slot become async on Windows): stop the manager, terminate bound wrappers, then wait until the manager is inactive and no bound wrapper is alive. Returns `absent` / `stopped` / `survived` / `denied` / `unknown` for the manager and wrappers only. `absent` means no OpenCodex task or WinSW service is registered for this home and no wrapper is bound: it maps to `ServiceStopOutcome` `absent`, prints no "Service manager stopped.", lets plain `ocx stop` send its POST, and skips part B, so a Windows install without a service keeps today's timing. With nothing registered, a process list that cannot be read is also `absent` (the wrapper sweep is skipped, as today's best-effort kill silently skips it), never `unknown`, so such a machine is never refused a stop it gets today; the residual is a stray wrapper of a deleted task that cannot be seen, which today's sweep also misses. `unknown` is reserved for evidence that cannot be read about a registered manager or a bound candidate, and for a registration probe that itself cannot answer: an unreadable `schtasks` query (today's `state-unknown`) stays `unknown`, it never becomes `absent`. + +The updater runs the authorizer-gated Part A for both WinSW and Task Scheduler; only `stopped` or `absent` permits restart. - Part B, `confirmWindowsStopFreed()`, run by the caller after its graceful child stop: the port is free and stays free for the hold, and a remaining listener is classified as ours or foreign. Part B replaces `proxyStillLiveAfterStop` in `ocx service stop`, the `schedulerCanRespawn` re-poll in `ocx stop`, and the respawn-window check in `ocx uninstall` (src/cli/index.ts, uninstall-plan.ts), which today run only on `stopped-respawnable`; on Windows they run part B unconditionally after a part A `stopped`. On the guarded path part B is added after the existing `settle`, which keeps its approved-PID exit check (`waitExit`) and the probe of the approved endpoint; there "the port" is the approved endpoint. - A part A outcome other than `stopped` aborts before anything destructive: `ocx service uninstall` / `remove` and `prepareServiceInstall` refuse with the outcome and reason before deleting or replacing the task (today uninstall only warns and continues), `ocx uninstall` keeps client configuration, and the updater aborts before replacing files. - Plain `ocx stop` (no consent snapshot) treats a part A outcome other than `stopped` or `absent` exactly as the guarded path treats `manager-still-active`: it does not stop the child (no `/api/stop` POST), sets `stopFailed` and `ownershipBlocked`, and skips the shared teardown, so client configuration is never restored under a wrapper that can relaunch the proxy (#3008). Today this path still sends the receipt-backed POST after a failed manager stop and relies on the route re-running the service stop and answering 409; once the route skips part A for a receipt holder (below), that safety net would be gone, so the parent must not send the POST at all. The record uses existing values only: `service: failed` (or `state-unknown`), `proxy: unknown` (the child stop was not attempted), `sharedTeardown: skipped`, `outcome: failed`. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 72931fbb6a0..d3807063c96 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -62,6 +62,7 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **Route caps.** The outer race lives in `runWindowsServiceStopPartA({ capMs })`: an overrun answers `unknown` at once, and from then on the live `/end` and WinSW stop are gated off and the authorizer refuses every terminate, so nothing changes after the answer. A step that already started (a `/end` in flight) still completes. - **The receipt record is checked against registration only.** The route skips Part A when the record's managers match a fresh registration probe (2 s each). It does not re-enumerate the recorded wrapper set; that would be the second enumeration 030 rules out. - **The ancestry guard (r2 review).** The binding rule excludes the stopper but not its ancestors, so interactive callers refuse a Part A whose stopper descends from this home's wrapper. `/api/stop` and the updater opt out because both run beneath that wrapper by design. The check reads the bind snapshot (or the guarded snapshot), so it costs no extra enumeration. +- **Updater WinSW stop.** The updater now runs the same authorized checked Part A for both backends and aborts unless the result is stopped or absent; a WinSW error cannot be treated as success. - **The guarded evidence snapshot (r1 review).** The guarded inspection reads `windowsProcessSnapshot` and the Schedule PID once per inspection instead of the CIM-only `windowsProcessList`, and `stopManager` binds on the revalidation's copy. A CIM-only table would make the x64 terminate answer `reused` and Part A report a live wrapper as exited. - **Absent counts as a pass for the updater and for install.** 040 says call sites abort on a non-`stopped` outcome. `absent` (nothing registered, nothing bound) has nothing that could relaunch the proxy, so the updater, install and switch continue on it, as `ocx stop` does. Refusing it would block every update on a machine whose recorded service was deleted. - **`ocx service stop` on Part A failure** refuses before the receipt and the child stop, the same rule as plain `ocx stop`, so it never restores under a wrapper that can relaunch the proxy. diff --git a/src/update/job-windows-restart.ts b/src/update/job-windows-restart.ts index 526e72c91db..5f7bda07647 100644 --- a/src/update/job-windows-restart.ts +++ b/src/update/job-windows-restart.ts @@ -1,5 +1,4 @@ -import { guardedStopWinsw, guardedStopWindowsChecked } from "./guarded-runtime-ops"; -import { readServiceBackend } from "../service"; +import { guardedStopWindowsChecked } from "./guarded-runtime-ops"; import type { UpdateMutationAuthorizer } from "./mutation-guard.mjs"; /** Why an automatic update restart stopped instead of starting a runtime over a live service. */ @@ -10,21 +9,12 @@ export const WINDOWS_SERVICE_STOP_ABORT = "Automatic update restart stopped: the * Stop the installed Windows backend before the updater replaces or restarts the runtime. * Returns false when the restart must abort; true when nothing can relaunch the proxy. * - * Task Scheduler uses Part A of the checked stop (#7771) through the authorizer-gated + * Both backends use Part A of the checked stop (#7771) through the authorizer-gated * variant: the service authorization runs at entry and before every terminate, and only - * stopped or absent lets the restart go on. The native WinSW backend keeps today's stop - * (WinSW has no :loop wrapper to end). + * stopped or absent lets the restart go on. */ export async function stopWindowsServiceForRestart(authorizeMutation: UpdateMutationAuthorizer): Promise { if (process.platform !== "win32") return true; - // An unreadable backend record falls through to Part A, which probes both backends itself. - let backend: string; - try { backend = readServiceBackend(); } catch { backend = "scheduler"; } - if (backend === "native") { - if (!authorizeMutation("service")) return false; - try { guardedStopWinsw(authorizeMutation); } catch { /* already stopped */ } - return true; - } if (!authorizeMutation("service")) return false; try { const outcome = await guardedStopWindowsChecked(authorizeMutation); diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index 29abb4b4ac7..c7c7c4ee241 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -228,8 +228,8 @@ stop instead of sliding forward. `"none"` after its orphan-discovery cap while an unresponsive proxy still holds the port. The candidate set is resolved before cleanup and every endpoint must be definitively dead before the install callback runs; live or unknown throws like a Part B failure. -- The updater (`src/update/job-windows-restart.ts`) uses `guardedStopWindowsChecked` for Task - Scheduler, with the service mutation gate as the authorizer; WinSW keeps its stop. Each of the +- The updater (`src/update/job-windows-restart.ts`) uses `guardedStopWindowsChecked` for both + backends, with the service mutation gate as the authorizer. Each of the three call sites in `src/update/job.ts` aborts the restart unless the outcome is stopped or absent. ## Who may run Part A diff --git a/tests/update/update-windows-wrapper-authority.test.ts b/tests/update/update-windows-wrapper-authority.test.ts index 258afcb9c9d..aff72dcb9a2 100644 --- a/tests/update/update-windows-wrapper-authority.test.ts +++ b/tests/update/update-windows-wrapper-authority.test.ts @@ -16,6 +16,20 @@ function execute(body: string, deps: object): any { const authority = (kind: "same" | "absent" | "unknown" | "foreign"): UpdateServiceHomeAuthority => kind === "foreign" ? { kind, subject: "a".repeat(64) } : { kind }; +for (const scenario of ["refused stop", "unreadable WinSW state", "conflicting scheduler task"]) { + test(`native backend aborts for ${scenario}`, async () => { + const homePolicy = createUpdateServiceHomePolicy(() => authority("same"), () => {}); + let checked = 0; + const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { + homePolicy, process: { platform: "win32" }, readServiceBackend: () => "native", + stopWinswService: () => { throw new Error("refused"); }, + checkedStop: async () => { checked++; return scenario === "refused stop" ? "denied" : "unknown"; }, + }); + expect(result).toBe(false); + expect(checked).toBe(1); + }); +} + /** * Part A of the checked stop (#7771) as the updater sees it: the authorizer at entry, the * manager stop, then the authorizer again before the wrapper terminate. @@ -31,7 +45,7 @@ function checkedStop(events: string[], home: () => UpdateServiceHomeAuthority, c }; } -for (const phase of ["checked", "backend-native", "backend-throw"] as const) { +for (const phase of ["checked"] as const) { for (const changed of ["same", "absent", "unknown", "foreign"] as const) { test(`Windows service stop checks ${changed} service authority after ${phase}`, async () => { let home = authority("same"); @@ -39,17 +53,11 @@ for (const phase of ["checked", "backend-native", "backend-throw"] as const) { const homePolicy = createUpdateServiceHomePolicy(() => home, () => {}); const result = await execute(helperBody + ";return stopWindowsServiceForRestart(homePolicy.authorizeMutation);", { homePolicy, process: { platform: "win32" }, - readServiceBackend: () => { - if (phase.startsWith("backend")) home = authority(changed); - if (phase === "backend-throw") throw new Error("fixture backend read"); - return phase === "backend-native" ? "native" : "scheduler"; - }, + readServiceBackend: () => "scheduler", stopWinswService: () => events.push(`native:${home.kind}`), checkedStop: checkedStop(events, () => home, () => { if (phase === "checked") home = authority(changed); }), }); - // An unreadable backend falls through to the checked stop, which probes both backends. - const stable = phase === "backend-native" ? ["native:same"] : ["stop:same", "terminate:same"]; - expect(events).toEqual(changed === "same" ? stable : phase === "checked" ? ["stop:same"] : []); + expect(events).toEqual(changed === "same" ? ["stop:same", "terminate:same"] : ["stop:same"]); // Anything but a proven stop aborts the restart. expect(result).toBe(changed === "same"); // The same policy must retain unknown/foreign refusal even if registration re-matches. diff --git a/tests/windows/windows-deploy-close-regressions.test.ts b/tests/windows/windows-deploy-close-regressions.test.ts index 04ea7c25421..8e039a511d7 100644 --- a/tests/windows/windows-deploy-close-regressions.test.ts +++ b/tests/windows/windows-deploy-close-regressions.test.ts @@ -57,8 +57,8 @@ describe("update-job restart avoids the shell-less .cmd EINVAL (Windows, bun/sou const restart = read("src/update/job-windows-restart.ts"); expect(restart).toContain("export async function stopWindowsServiceForRestart(authorizeMutation: UpdateMutationAuthorizer): Promise"); expect(restart).toContain('if (process.platform !== "win32") return true;'); - // Native WinSW installs keep today's stop and return before the checked Task Scheduler stop. - expect(restart).toMatch(/if \(backend === "native"\) \{\s*if \(!authorizeMutation\("service"\)\) return false;\s*try \{ guardedStopWinsw\(authorizeMutation\); \} catch \{[^}]*\}\s*return true;\s*\}\s*if \(!authorizeMutation\("service"\)\) return false;/); + // Both manager registrations are probed by the checked stop. + expect(restart).not.toContain("guardedStopWinsw"); // Task Scheduler: Part A ends the manager and the bound wrappers (the binding rule in // windows-stop-binding.ts excludes the stopper itself); only stopped or absent continues. expect(restart).toContain("const outcome = await guardedStopWindowsChecked(authorizeMutation);"); From 39933588ab6edac14bf8715436fb7d6629862637 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:16:30 +0900 Subject: [PATCH 49/53] Require dead endpoint proof before service restore --- .../261111_windows_service_stop/030_design.md | 1 + .../060_implementation.md | 1 + src/service/cli-windows-stop.ts | 17 +++++++++-------- structure/ops/windows-checked-stop.md | 3 ++- tests/windows/windows-stop-callers.test.ts | 14 ++++++++++++-- 5 files changed, 25 insertions(+), 11 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 2996675ca0a..9ef54d99603 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -7,6 +7,7 @@ The proxy child is stopped by each caller AFTER the service stop, and `/api/stop - Part A, `stopWindowsServiceChecked()` (async; the current `stopServiceIfInstalledDetailed` and the guarded `stopManager` slot become async on Windows): stop the manager, terminate bound wrappers, then wait until the manager is inactive and no bound wrapper is alive. Returns `absent` / `stopped` / `survived` / `denied` / `unknown` for the manager and wrappers only. `absent` means no OpenCodex task or WinSW service is registered for this home and no wrapper is bound: it maps to `ServiceStopOutcome` `absent`, prints no "Service manager stopped.", lets plain `ocx stop` send its POST, and skips part B, so a Windows install without a service keeps today's timing. With nothing registered, a process list that cannot be read is also `absent` (the wrapper sweep is skipped, as today's best-effort kill silently skips it), never `unknown`, so such a machine is never refused a stop it gets today; the residual is a stray wrapper of a deleted task that cannot be seen, which today's sweep also misses. `unknown` is reserved for evidence that cannot be read about a registered manager or a bound candidate, and for a registration probe that itself cannot answer: an unreadable `schtasks` query (today's `state-unknown`) stays `unknown`, it never becomes `absent`. The updater runs the authorizer-gated Part A for both WinSW and Task Scheduler; only `stopped` or `absent` permits restart. +When `ocx service stop` finds Part A absent, every candidate endpoint must be definitively dead before shared restoration. - Part B, `confirmWindowsStopFreed()`, run by the caller after its graceful child stop: the port is free and stays free for the hold, and a remaining listener is classified as ours or foreign. Part B replaces `proxyStillLiveAfterStop` in `ocx service stop`, the `schedulerCanRespawn` re-poll in `ocx stop`, and the respawn-window check in `ocx uninstall` (src/cli/index.ts, uninstall-plan.ts), which today run only on `stopped-respawnable`; on Windows they run part B unconditionally after a part A `stopped`. On the guarded path part B is added after the existing `settle`, which keeps its approved-PID exit check (`waitExit`) and the probe of the approved endpoint; there "the port" is the approved endpoint. - A part A outcome other than `stopped` aborts before anything destructive: `ocx service uninstall` / `remove` and `prepareServiceInstall` refuse with the outcome and reason before deleting or replacing the task (today uninstall only warns and continues), `ocx uninstall` keeps client configuration, and the updater aborts before replacing files. - Plain `ocx stop` (no consent snapshot) treats a part A outcome other than `stopped` or `absent` exactly as the guarded path treats `manager-still-active`: it does not stop the child (no `/api/stop` POST), sets `stopFailed` and `ownershipBlocked`, and skips the shared teardown, so client configuration is never restored under a wrapper that can relaunch the proxy (#3008). Today this path still sends the receipt-backed POST after a failed manager stop and relies on the route re-running the service stop and answering 409; once the route skips part A for a receipt holder (below), that safety net would be gone, so the parent must not send the POST at all. The record uses existing values only: `service: failed` (or `state-unknown`), `proxy: unknown` (the child stop was not attempted), `sharedTeardown: skipped`, `outcome: failed`. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index d3807063c96..6454bd95a27 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -63,6 +63,7 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **The receipt record is checked against registration only.** The route skips Part A when the record's managers match a fresh registration probe (2 s each). It does not re-enumerate the recorded wrapper set; that would be the second enumeration 030 rules out. - **The ancestry guard (r2 review).** The binding rule excludes the stopper but not its ancestors, so interactive callers refuse a Part A whose stopper descends from this home's wrapper. `/api/stop` and the updater opt out because both run beneath that wrapper by design. The check reads the bind snapshot (or the guarded snapshot), so it costs no extra enumeration. - **Updater WinSW stop.** The updater now runs the same authorized checked Part A for both backends and aborts unless the result is stopped or absent; a WinSW error cannot be treated as success. +- **Absent service stop.** Before restoring shared config, `ocx service stop` proves every candidate endpoint dead; live or unknown evidence keeps the receipt. - **The guarded evidence snapshot (r1 review).** The guarded inspection reads `windowsProcessSnapshot` and the Schedule PID once per inspection instead of the CIM-only `windowsProcessList`, and `stopManager` binds on the revalidation's copy. A CIM-only table would make the x64 terminate answer `reused` and Part A report a live wrapper as exited. - **Absent counts as a pass for the updater and for install.** 040 says call sites abort on a non-`stopped` outcome. `absent` (nothing registered, nothing bound) has nothing that could relaunch the proxy, so the updater, install and switch continue on it, as `ocx stop` does. Refusing it would block every update on a machine whose recorded service was deleted. - **`ocx service stop` on Part A failure** refuses before the receipt and the child stop, the same rule as plain `ocx stop`, so it never restores under a wrapper that can relaunch the proxy. diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index 1e2b528dc76..9ce62cfd117 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -6,8 +6,8 @@ * stopped, so the proxy defers the shared teardown to this command. When the claim fails * while this home owns the shared teardown, no undeferred child stop is sent: a wrapper * appearing later could relaunch the proxy after its own teardown ran. Part B replaces - * proxyStillLiveAfterStop after a Part A stopped; the receipt is completed only after the - * restore ran, and kept for recovery otherwise. + * the legacy survivor check after a Part A stopped. With an absent Part A every candidate + * endpoint must be proven dead before restore; the receipt is kept otherwise. * - uninstall / remove: Part A before the task is deleted (a looping wrapper used to * outlive its task, #764), refusing on anything but stopped/absent, then a receipt that * carries Part A's record before the tracked proxy stop (Part A leaves the task @@ -18,7 +18,7 @@ import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-te import { readClientConnectionState } from "../client/state"; import type { ProbeEndpoint } from "../cli/uninstall-plan"; import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; -import { proxyStillLiveAfterStop, stopTrackedProxyForServiceCommand } from "./orchestration"; +import { stopTrackedProxyForServiceCommand } from "./orchestration"; import { confirmWindowsServiceStop, describeWindowsPartA, @@ -38,7 +38,6 @@ export interface WindowsServiceStopCommandIo { ownsSharedTeardown?: () => boolean; claim?: typeof claimPendingTeardown; stopTracked?: typeof stopTrackedProxyForServiceCommand; - survivor?: typeof proxyStillLiveAfterStop; /** * Tri-state probe of an endpoint. Only a definitive "dead" on EVERY candidate permits * deleting the task; "live" and "unknown" refuse and keep the receipt. @@ -88,16 +87,18 @@ export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopComm return { ok: false }; } } + const endpoints = partA.outcome === "absent" ? (io.endpoints ?? windowsRemovalEndpoints)() : []; await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, }); if (partA.outcome === "absent") { - const survivor = await (io.survivor ?? proxyStillLiveAfterStop)(); - if (!survivor) return withNonce(nonce); - console.error("❌ service stop did not take effect: a proxy is still listening on port " + survivor.port + "."); - console.error(" Native Codex was NOT restored. Check for a manually started proxy, then re-run 'ocx service stop'."); + const proof = await endpointsAllDead(endpoints, io.endpointLiveness ?? probeEndpointLiveness); + if (proof.dead) return withNonce(nonce); + console.error(proof.live.length ? "❌ service stop did not take effect: a proxy is still listening on port " + proof.live.join(", ") + "." + : "❌ service stop did not take effect: endpoint liveness could not be confirmed on port " + proof.unknown.join(", ") + "."); + console.error(" Native Codex was NOT restored; the teardown receipt is kept for a later stop."); return { ok: false }; } const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index c7c7c4ee241..bc5e08afd49 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -177,7 +177,8 @@ stop instead of sliding forward. - `ocx service stop` (`src/service/cli-windows-stop.ts`) refuses on a Part A other than stopped or absent, claims a teardown receipt before the deferred tracked-proxy stop when this home owns the shared teardown (a failed claim sends no child stop), runs Part B, and clears the receipt only - after the restore. `ocx service uninstall` / `remove` refuse before deleting the task unless + after the restore. With Part A absent it requires every candidate endpoint to be definitively + dead; live or unknown keeps the receipt and blocks restoration. `ocx service uninstall` / `remove` refuse before deleting the task unless Part A stopped (or found nothing) and Part B did not find our identity or an unreadable port. Removal claims the same receipt, carrying Part A's record, before its child stop: Part A leaves the scheduler task registered, so a receipt-less POST answers `respawnable_service` (409) and diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index fb196aa526d..d92d0f044a9 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -156,11 +156,21 @@ describe("ocx service stop on Windows", () => { } }); - test("absent keeps today's survivor check instead of Part B", async () => { + test("absent refuses an unknown endpoint and keeps the receipt", async () => { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent") }); + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, claim: () => ({ nonce: "kept" }) as never, + stopTracked: async () => "none", + endpoints: () => [{ hostname: "127.0.0.1", port: 10100 }], endpointLiveness: async () => "unknown", + }); + expect(result.ok).toBe(false); + }); + + test("absent checks endpoints instead of Part B", async () => { let partB = 0; setWindowsServiceStopHooksForTests({ partA: async () => partAResult("absent"), partB: async () => { partB++; return partBResult("freed"); } }); const live = await windowsServiceStopBeforeRestore({ - ownsSharedTeardown: () => false, stopTracked: async () => "none", survivor: async () => ({ port: 10100 }), + ownsSharedTeardown: () => false, stopTracked: async () => "none", endpoints: () => [{ hostname: "127.0.0.1", port: 10100 }], endpointLiveness: async () => "live", }); expect(live.ok).toBe(false); expect(partB).toBe(0); From b9c70a5fe2eb7ee2e131e24e9f0d201577f678a9 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:21:00 +0900 Subject: [PATCH 50/53] Bind service stop receipts to resolved proxy targets --- .../261111_windows_service_stop/030_design.md | 1 + .../060_implementation.md | 1 + src/service/cli-windows-stop.ts | 35 ++++++------ src/service/orchestration.ts | 53 +++++++++++++++---- structure/ops/windows-checked-stop.md | 3 ++ .../service-install-wrapper-cleanup.test.ts | 28 ++++++++-- .../service/service-stop-verification.test.ts | 10 ++-- tests/service/stop-direct-fetch.test.ts | 2 +- tests/windows/windows-stop-callers.test.ts | 37 +++++++++++-- 9 files changed, 129 insertions(+), 41 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 9ef54d99603..4d0843f8537 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -8,6 +8,7 @@ The proxy child is stopped by each caller AFTER the service stop, and `/api/stop The updater runs the authorizer-gated Part A for both WinSW and Task Scheduler; only `stopped` or `absent` permits restart. When `ocx service stop` finds Part A absent, every candidate endpoint must be definitively dead before shared restoration. +Service stop, removal and install resolve the tracked or discovered target once and use its exact endpoint for both the receipt and stop; no target means no receipt or POST. - Part B, `confirmWindowsStopFreed()`, run by the caller after its graceful child stop: the port is free and stays free for the hold, and a remaining listener is classified as ours or foreign. Part B replaces `proxyStillLiveAfterStop` in `ocx service stop`, the `schedulerCanRespawn` re-poll in `ocx stop`, and the respawn-window check in `ocx uninstall` (src/cli/index.ts, uninstall-plan.ts), which today run only on `stopped-respawnable`; on Windows they run part B unconditionally after a part A `stopped`. On the guarded path part B is added after the existing `settle`, which keeps its approved-PID exit check (`waitExit`) and the probe of the approved endpoint; there "the port" is the approved endpoint. - A part A outcome other than `stopped` aborts before anything destructive: `ocx service uninstall` / `remove` and `prepareServiceInstall` refuse with the outcome and reason before deleting or replacing the task (today uninstall only warns and continues), `ocx uninstall` keeps client configuration, and the updater aborts before replacing files. - Plain `ocx stop` (no consent snapshot) treats a part A outcome other than `stopped` or `absent` exactly as the guarded path treats `manager-still-active`: it does not stop the child (no `/api/stop` POST), sets `stopFailed` and `ownershipBlocked`, and skips the shared teardown, so client configuration is never restored under a wrapper that can relaunch the proxy (#3008). Today this path still sends the receipt-backed POST after a failed manager stop and relies on the route re-running the service stop and answering 409; once the route skips part A for a receipt holder (below), that safety net would be gone, so the parent must not send the POST at all. The record uses existing values only: `service: failed` (or `state-unknown`), `proxy: unknown` (the child stop was not attempted), `sharedTeardown: skipped`, `outcome: failed`. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 6454bd95a27..11c1b21d4ca 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -64,6 +64,7 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **The ancestry guard (r2 review).** The binding rule excludes the stopper but not its ancestors, so interactive callers refuse a Part A whose stopper descends from this home's wrapper. `/api/stop` and the updater opt out because both run beneath that wrapper by design. The check reads the bind snapshot (or the guarded snapshot), so it costs no extra enumeration. - **Updater WinSW stop.** The updater now runs the same authorized checked Part A for both backends and aborts unless the result is stopped or absent; a WinSW error cannot be treated as success. - **Absent service stop.** Before restoring shared config, `ocx service stop` proves every candidate endpoint dead; live or unknown evidence keeps the receipt. +- **Receipt target.** Service stop, removal and install resolve the proxy once, claim only for that target's exact endpoint, and send the same endpoint to the child stop. No target means no receipt or POST. - **The guarded evidence snapshot (r1 review).** The guarded inspection reads `windowsProcessSnapshot` and the Schedule PID once per inspection instead of the CIM-only `windowsProcessList`, and `stopManager` binds on the revalidation's copy. A CIM-only table would make the x64 terminate answer `reused` and Part A report a live wrapper as exited. - **Absent counts as a pass for the updater and for install.** 040 says call sites abort on a non-`stopped` outcome. `absent` (nothing registered, nothing bound) has nothing that could relaunch the proxy, so the updater, install and switch continue on it, as `ocx stop` does. Refusing it would block every update on a machine whose recorded service was deleted. - **`ocx service stop` on Part A failure** refuses before the receipt and the child stop, the same rule as plain `ocx stop`, so it never restores under a wrapper that can relaunch the proxy. diff --git a/src/service/cli-windows-stop.ts b/src/service/cli-windows-stop.ts index 9ce62cfd117..dac0c2c097d 100644 --- a/src/service/cli-windows-stop.ts +++ b/src/service/cli-windows-stop.ts @@ -18,7 +18,7 @@ import { clearPendingTeardown, claimPendingTeardown } from "../config/pending-te import { readClientConnectionState } from "../client/state"; import type { ProbeEndpoint } from "../cli/uninstall-plan"; import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; -import { stopTrackedProxyForServiceCommand } from "./orchestration"; +import { resolveTrackedProxyTarget, stopTrackedProxyForServiceCommand } from "./orchestration"; import { confirmWindowsServiceStop, describeWindowsPartA, @@ -26,7 +26,6 @@ import { endpointsAllDead, runWindowsServiceStopPartA, windowsPartARecord, - windowsReceiptEndpoint, windowsRemovalEndpoints, windowsServiceStopPort, windowsStopAdmission, @@ -38,6 +37,7 @@ export interface WindowsServiceStopCommandIo { ownsSharedTeardown?: () => boolean; claim?: typeof claimPendingTeardown; stopTracked?: typeof stopTrackedProxyForServiceCommand; + resolveTarget?: typeof resolveTrackedProxyTarget; /** * Tri-state probe of an endpoint. Only a definitive "dead" on EVERY candidate permits * deleting the task; "live" and "unknown" refuse and keep the receipt. @@ -77,10 +77,11 @@ export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopComm return { ok: false }; } let nonce: string | undefined; - if ((io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { - const { endpoint, source } = windowsReceiptEndpoint(); + const target = await (io.resolveTarget ?? resolveTrackedProxyTarget)(); + if (target && (io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + if (!target.endpoint) { console.error("❌ service stop cannot defer teardown without the target's endpoint."); return { ok: false }; } try { - nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)).nonce; + nonce = (io.claim ?? claimPendingTeardown)(target.endpoint, "exact", process.pid, undefined, windowsPartARecord(partA)).nonce; } catch (error) { console.error("❌ service stop could not record its deferred-teardown receipt: " + (error instanceof Error ? error.message : String(error))); console.error(" The proxy was left running: stopping it without the receipt would let it tear down client config while a wrapper can still relaunch it."); @@ -88,7 +89,9 @@ export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopComm } } const endpoints = partA.outcome === "absent" ? (io.endpoints ?? windowsRemovalEndpoints)() : []; + if (target?.endpoint && partA.outcome === "absent" && !endpoints.some(e => e.hostname === target.endpoint!.hostname && e.port === target.endpoint!.port)) endpoints.push(target.endpoint); await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ + target, ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, @@ -101,7 +104,7 @@ export async function windowsServiceStopBeforeRestore(io: WindowsServiceStopComm console.error(" Native Codex was NOT restored; the teardown receipt is kept for a later stop."); return { ok: false }; } - const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); + const partB = await confirmWindowsServiceStop({ port: target?.endpoint?.port ?? port, remainingMs: windowsStopRemainingMs() }); if (partB.outcome === "freed") return withNonce(nonce); if (partB.outcome === "port-held-by-foreign") { console.warn("⚠️ " + describeWindowsPartB(partB)); @@ -127,12 +130,9 @@ export function completeWindowsServiceStopReceipt(nonce: string | undefined, cle * `ocx service stop` does, and kept until the caller's restore succeeds. */ export async function windowsServiceRemovalStop( - io: Pick = {}, + io: Pick = {}, ): Promise<{ ok: boolean; nonce?: string }> { const port = windowsServiceStopPort(); - // Resolved once: the receipt and the absent-Part-A probe must name the same address, or - // the proof would be about an endpoint the stop never contacted. - const { endpoint, source } = windowsReceiptEndpoint(); const partA = await runWindowsServiceStopPartA({ admission: windowsStopAdmission({ orphanDiscovery: true }) }); if (partA.outcome !== "stopped" && partA.outcome !== "absent") { console.error("❌ Service uninstall refused: the Windows service could not be proven stopped (" + partA.outcome + "), so the task was not deleted."); @@ -140,9 +140,13 @@ export async function windowsServiceRemovalStop( return { ok: false }; } let nonce: string | undefined; - if ((io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + const target = await (io.resolveTarget ?? resolveTrackedProxyTarget)(); + const endpoints = partA.outcome === "absent" ? (io.endpoints ?? windowsRemovalEndpoints)() : []; + if (target?.endpoint && partA.outcome === "absent" && !endpoints.some(e => e.hostname === target.endpoint!.hostname && e.port === target.endpoint!.port)) endpoints.push(target.endpoint); + if (target && (io.ownsSharedTeardown ?? ownsSharedTeardownByDefault)()) { + if (!target.endpoint) { console.error("❌ Service uninstall cannot defer teardown without the target's endpoint."); return { ok: false }; } try { - nonce = (io.claim ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)).nonce; + nonce = (io.claim ?? claimPendingTeardown)(target.endpoint, "exact", process.pid, undefined, windowsPartARecord(partA)).nonce; } catch (error) { console.error("❌ Service uninstall could not record its deferred-teardown receipt: " + (error instanceof Error ? error.message : String(error))); console.error(" The proxy was left running: stopping it without the receipt would let it tear down client config while the task can still relaunch it."); @@ -150,6 +154,7 @@ export async function windowsServiceRemovalStop( } } await (io.stopTracked ?? stopTrackedProxyForServiceCommand)({ + target, ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, @@ -164,15 +169,15 @@ export async function windowsServiceRemovalStop( // a stale runtime record pointing at a closed port used to prove removal while the // configured port had a live listener. No respawn hold: with no manager registered // there is nothing to relaunch the proxy. - const proof = await endpointsAllDead((io.endpoints ?? windowsRemovalEndpoints)(), io.endpointLiveness ?? probeEndpointLiveness); + const proof = await endpointsAllDead(endpoints, io.endpointLiveness ?? probeEndpointLiveness); if (proof.dead) return withNonce(nonce); console.error(proof.live.length > 0 ? "❌ Service uninstall refused: a proxy is still listening on port " + proof.live.join(", ") + "." - : "❌ Service uninstall refused: the endpoint on port " + (proof.unknown.join(", ") || endpoint.port) + " could not be confirmed down."); + : "❌ Service uninstall refused: the endpoint on port " + (proof.unknown.join(", ") || port) + " could not be confirmed down."); console.error(" The task was not deleted. Stop that proxy, then re-run 'ocx service uninstall'."); return { ok: false }; } - const partB = await confirmWindowsServiceStop({ port, remainingMs: windowsStopRemainingMs() }); + const partB = await confirmWindowsServiceStop({ port: target?.endpoint?.port ?? port, remainingMs: windowsStopRemainingMs() }); if (partB.outcome === "freed") return withNonce(nonce); if (partB.outcome === "port-held-by-foreign") { console.warn("⚠️ " + describeWindowsPartB(partB)); diff --git a/src/service/orchestration.ts b/src/service/orchestration.ts index a3ac97c42c2..cf817704b29 100644 --- a/src/service/orchestration.ts +++ b/src/service/orchestration.ts @@ -3,7 +3,7 @@ import { existsSync, unlinkSync } from "node:fs"; import { durableBunRuntime, type DurableBunRuntime } from "../lib/bun-runtime"; import { assertSelectedRuntimeWritable, type RuntimePreflightDeps } from "../lib/bun-runtime-preflight"; import { getConfigDir } from "../config"; -import { readPid, removePid, removeRuntimePort, verifyPidIdentity } from "../config/process-state"; +import { readPid, readRuntimePort, removePid, removeRuntimePort, verifyPidIdentity } from "../config/process-state"; import { isWslRuntime } from "../codex/home"; import { isProcessAlive, stopProxy } from "../lib/process-control"; import { randomUUID } from "node:crypto"; @@ -23,7 +23,7 @@ import type { WindowsStopConfirmResult } from "./windows-stop-confirm"; import { claimPendingTeardown, clearPendingTeardown, type PendingTeardownReceipt } from "../config/pending-teardown"; import { probeEndpointLiveness, type EndpointLiveness } from "../server/proxy-liveness"; import type { ProbeEndpoint } from "../cli/uninstall-plan"; -import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, endpointsAllDead, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsPartARecord, windowsReceiptEndpoint, windowsRemovalEndpoints, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; +import { confirmWindowsServiceStop, describeWindowsPartA, describeWindowsPartB, endpointsAllDead, runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, windowsPartARecord, windowsRemovalEndpoints, windowsServiceStopPort, windowsStopAdmission, windowsStopRemainingMs, type WindowsPartARequest } from "./windows-service-stop"; import { schtasks, probeWindowsSchedulerTask, rollbackWindowsSchedulerTaskOwnedByAttempt, settleDelay } from "./windows-scheduler"; import type { WindowsSchedulerTaskProbe } from "./windows-scheduler"; import { windowsTaskRegistrationOwnedByAttempt, windowsTaskRegistrationHealthy } from "./windows-taskxml"; @@ -209,6 +209,8 @@ export const DEFAULT_ORPHAN_DISCOVERY_MS = 7_000; /** How the tracked proxy is stopped: the deferral receipt and the Windows port wait. */ export interface TrackedProxyStopIo { + /** Pre-resolved Windows target; null means there is no child to stop. */ + target?: TrackedProxyTarget | null; /** ocx service stop on Windows: the claimed receipt, so the proxy defers its teardown. */ deferSharedTeardownNonce?: string; /** Part B follows and owns the port wait (030), so stopProxy skips its own. */ @@ -223,7 +225,33 @@ export interface TrackedProxyStopIo { orphanDiscoveryCapMs?: number; } +export interface TrackedProxyTarget { + pid: number; + endpoint: { hostname: string; port: number } | null; +} + +/** Resolve once before claiming a receipt, so its endpoint is the stop's endpoint. */ +export async function resolveTrackedProxyTarget(): Promise { + const pid = verifiedKillTarget(readPid()); + if (pid !== null && isProcessAlive(pid)) { + const runtime = readRuntimePort(pid); + return { pid, endpoint: runtime?.port ? { hostname: runtime.hostname ?? "127.0.0.1", port: runtime.port } : null }; + } + const live = await findLiveProxy({ ...SERVICE_STOP_LIVENESS, deadlineAt: Date.now() + WINDOWS_STOP_CAPS.orphanDiscovery, acceptPackageTreeFenced: true }); + if (!live) return null; + const livePid = verifiedKillTarget(live.pid); + if (livePid === null) throw new Error("A live proxy was found but its process identity could not be verified."); + return { pid: livePid, endpoint: { hostname: live.hostname ?? "127.0.0.1", port: live.port } }; +} + async function stopTrackedProxyIfRunning(io: TrackedProxyStopIo = {}): Promise { + if (io.target !== undefined) { + if (!io.target) return "none"; + await stopProxy(io.target.pid, { ...io, ...(io.target.endpoint ? { runtimeEndpoint: io.target.endpoint } : {}) }); + removePid(io.target.pid); + removeRuntimePort(io.target.pid); + return "stopped"; + } let stopped = false; const pid = readPid(); const trackedKillPid = verifiedKillTarget(pid); @@ -286,8 +314,7 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { windowsStopPort?: () => number; /** Receives the deferral nonce so the wrapper's POST does not run the teardown. */ stopTrackedProxy?: (io?: TrackedProxyStopIo) => Promise; - /** The endpoint a Windows install's teardown receipt names (runtime record, else config). */ - teardownEndpoint?: typeof windowsReceiptEndpoint; + resolveTarget?: typeof resolveTrackedProxyTarget; claimTeardown?: typeof claimPendingTeardown; clearTeardown?: (nonce: string) => boolean; /** Every endpoint an absent-manager install must prove down, resolved once. */ @@ -299,7 +326,7 @@ export interface ServiceInstallPreparationDeps extends RuntimePreflightDeps { /** * What an install preparation left behind. * - * `deferredTeardownNonce` exists only on Windows after Part A stopped: the install stopped + * `deferredTeardownNonce` exists on Windows when the install stopped a target: it stopped * the old proxy with the shared teardown deferred, so it owes the receipt. The install that * replaces the proxy keeps shared config pointed at OpenCodex and completes the receipt * after the new service is installed and started; a failed install leaves it for the next @@ -320,10 +347,10 @@ export interface ServiceInstallPreparationResult { function claimInstallTeardownReceipt( deps: ServiceInstallPreparationDeps, partA: WindowsCheckedStopResult, + endpoint: { hostname: string; port: number }, ): PendingTeardownReceipt { - const { endpoint, source } = (deps.teardownEndpoint ?? windowsReceiptEndpoint)(); try { - return (deps.claimTeardown ?? claimPendingTeardown)(endpoint, source, process.pid, undefined, windowsPartARecord(partA)); + return (deps.claimTeardown ?? claimPendingTeardown)(endpoint, "exact", process.pid, undefined, windowsPartARecord(partA)); } catch (error) { throw new Error( "The deferred-teardown receipt could not be recorded, so the proxy was left running and nothing was replaced: " @@ -366,24 +393,28 @@ export async function prepareServiceInstall( const absentProofEndpoints = partA.outcome === "absent" ? (deps.installEndpoints ?? windowsRemovalEndpoints)() : []; + const target = await (deps.resolveTarget ?? resolveTrackedProxyTarget)(); + if (target?.endpoint && partA.outcome === "absent" && !absentProofEndpoints.some(e => e.hostname === target.endpoint!.hostname && e.port === target.endpoint!.port)) absentProofEndpoints.push(target.endpoint); let nonce: string | undefined; - if (partA.outcome === "stopped") { + if (target) { // Before the tracked stop: a receipt-less POST would answer 409 and abort the install // while the old proxy kept serving. - nonce = claimInstallTeardownReceipt(deps, partA).nonce; + if (!target.endpoint) throw new Error("The target endpoint is unavailable, so deferred teardown cannot be recorded."); + nonce = claimInstallTeardownReceipt(deps, partA, target.endpoint).nonce; } // Part B owns the port wait after a stopped manager (030 "Time budget"). await (deps.stopTrackedProxy ?? ((io?: TrackedProxyStopIo) => stopTrackedProxyIfRunning(io ?? { skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, })))({ + target, ...(nonce ? { deferSharedTeardownNonce: nonce } : {}), skipPortWait: partA.outcome === "stopped", orphanDiscoveryCapMs: WINDOWS_STOP_CAPS.orphanDiscovery, }); if (partA.outcome === "stopped") { const partB = await (deps.windowsPartB ?? confirmWindowsServiceStop)({ - port: stopPort, + port: target?.endpoint?.port ?? stopPort, remainingMs: windowsStopRemainingMs(), }); if (partB.outcome !== "freed") { @@ -399,7 +430,7 @@ export async function prepareServiceInstall( : `The endpoint on port ${(proof.unknown.join(", ") || stopPort)} could not be confirmed down; nothing was published or started. Check for a manually started proxy, then retry.`); } } - return {}; + return nonce ? { deferredTeardownNonce: nonce } : {}; } const resolveOps = deps.managerOps ?? platformServiceInstallCleanupOps; const backends: ServiceBackend[] = []; diff --git a/structure/ops/windows-checked-stop.md b/structure/ops/windows-checked-stop.md index bc5e08afd49..4271a9fadbe 100644 --- a/structure/ops/windows-checked-stop.md +++ b/structure/ops/windows-checked-stop.md @@ -193,6 +193,9 @@ stop instead of sliding forward. (`TrackedProxyStopIo.orphanDiscoveryCapMs`). The checked Windows callers pass the admitted 3 s (`WINDOWS_STOP_CAPS.orphanDiscovery`), which is what admission priced; the omitted default stays the legacy 7 s so macOS and Linux timing is unchanged. +- Windows service stop, removal and install resolve one tracked or discovered proxy target before + claiming its receipt. The receipt and POST use that target's exact endpoint; Part B checks its + port. No target means no receipt or POST. An owning caller refuses a target without an endpoint. - `ocx uninstall` (`src/cli/uninstall-stop.ts`) resolves its stop target before it claims anything: the tracked pid with its own runtime endpoint, or the discovered live proxy with the endpoint discovery found it on. The receipt and `stopProxy` then name the same endpoint, so a diff --git a/tests/service/service-install-wrapper-cleanup.test.ts b/tests/service/service-install-wrapper-cleanup.test.ts index 953281b4ab5..50c3292b598 100644 --- a/tests/service/service-install-wrapper-cleanup.test.ts +++ b/tests/service/service-install-wrapper-cleanup.test.ts @@ -7,6 +7,23 @@ import { partAResult, partBResult } from "../helpers/windows-stop-caller-fixture // the tracked proxy stops next, and Part B must find the port free before anything is // published or started. describe("service install preparation wrapper cleanup", () => { + test("install claims the discovered orphan endpoint and carries it to the stop", async () => { + const endpoint = { hostname: "127.0.0.1", port: 12001 }; + let claimed: unknown; + let target: unknown; + let port = 0; + await prepareServiceInstall("scheduler", { + platform: "win32", windowsStopPort: () => 10100, + windowsPartA: async () => partAResult("stopped"), + resolveTarget: async () => ({ pid: 42, endpoint }), + claimTeardown: value => { claimed = value; return { nonce: "n" }; }, + stopTrackedProxy: async io => { target = io?.target; }, + windowsPartB: async request => { port = request.port; return partBResult("freed"); }, + }); + expect(claimed).toEqual(endpoint); + expect(target).toEqual({ pid: 42, endpoint }); + expect(port).toBe(12001); + }); test("install preparation ends the respawning wrapper (Part A) before the tracked proxy stop", async () => { // The defect: schtasks /end ends the task instance but the detached cmd :loop wrapper // survives and respawns its child seconds later, landing after the one-shot tracked-proxy @@ -84,6 +101,8 @@ describe("service install preparation wrapper cleanup", () => { return runtimePort ?? 10100; }, windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), + claimTeardown: () => ({ nonce: "fixture" }), stopTrackedProxy: async () => { calls.push("stop:standalone"); runtimePort = null; }, windowsPartB: async request => { calls.push("partB"); ports.push(request.port); return partBResult("freed"); }, }); @@ -119,7 +138,7 @@ describe("service install preparation wrapper cleanup", () => { windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, windowsPartB: async () => { calls.push("partB"); return partBResult("freed"); }, windowsStopPort: () => 12000, - teardownEndpoint: () => ({ endpoint: { hostname: "127.0.0.1", port: 12000 }, source: "exact" }), + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), claimTeardown: (endpoint, _source, _pid, _handover, partA) => { calls.push("claim"); claimed.push({ endpoint, partA }); @@ -134,7 +153,7 @@ describe("service install preparation wrapper cleanup", () => { endpoint: { hostname: "127.0.0.1", port: 12000 }, partA: { partA: "stopped", manager: { scheduler: true, winsw: false }, wrappers: [{ pid: 30, startTime: 5_100 }] }, }]); - expect(stopIo).toEqual([{ deferSharedTeardownNonce: "nonce-1", skipPortWait: true, orphanDiscoveryCapMs: expect.any(Number) }]); + expect(stopIo).toEqual([{ target: { pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }, deferSharedTeardownNonce: "nonce-1", skipPortWait: true, orphanDiscoveryCapMs: expect.any(Number) }]); // The install replaces the proxy, so shared config stays pointed at OpenCodex and the // obligation is discharged only once the new service is installed and started. expect(cleared).toEqual(["nonce-1"]); @@ -147,7 +166,7 @@ describe("service install preparation wrapper cleanup", () => { windowsPartA: async () => partAResult("stopped"), windowsPartB: async () => partBResult("freed"), windowsStopPort: () => 12000, - teardownEndpoint: () => ({ endpoint: { hostname: "127.0.0.1", port: 12000 }, source: "exact" }), + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), claimTeardown: () => ({ nonce: "nonce-2" }), clearTeardown: nonce => { cleared.push(nonce); return true; }, stopTrackedProxy: async () => {}, @@ -161,7 +180,7 @@ describe("service install preparation wrapper cleanup", () => { platform: "win32", windowsPartA: async () => { calls.push("partA"); return partAResult("stopped"); }, windowsStopPort: () => 12000, - teardownEndpoint: () => ({ endpoint: { hostname: "127.0.0.1", port: 12000 }, source: "exact" }), + resolveTarget: async () => ({ pid: 42, endpoint: { hostname: "127.0.0.1", port: 12000 } }), claimTeardown: () => { throw new Error("disk full"); }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, })).rejects.toThrow(/deferred-teardown receipt could not be recorded[\s\S]*disk full/); @@ -208,6 +227,7 @@ describe("service install preparation wrapper cleanup", () => { windowsStopPort: () => 12000, installEndpoints: () => [{ hostname: "127.0.0.1", port: 12000 }], endpointLiveness: async () => "dead", + resolveTarget: async () => null, claimTeardown: () => { calls.push("claim"); return { nonce: "unused" }; }, stopTrackedProxy: async () => { calls.push("stop:standalone"); }, }); diff --git a/tests/service/service-stop-verification.test.ts b/tests/service/service-stop-verification.test.ts index 5111e825375..1c37a17287f 100644 --- a/tests/service/service-stop-verification.test.ts +++ b/tests/service/service-stop-verification.test.ts @@ -177,22 +177,18 @@ describe("orphan discovery is capped by the caller, not hard-coded (#7771)", () // The endpoint probes are injected: both callers would otherwise fall through to // the real ones, which on Windows spend their full window before answering, and this // test is about the discovery budget, not about that wait. - const survivors: unknown[] = []; const probed: unknown[] = []; for (const run of [windowsServiceStopBeforeRestore, windowsServiceRemovalStop]) { const seen: unknown[] = []; await run({ ownsSharedTeardown: () => false, + resolveTarget: async () => null, stopTracked: async io => { seen.push(io); return "stopped"; }, - survivor: async io => { survivors.push(io ?? "no-args"); return null; }, endpointLiveness: async endpoint => { probed.push(endpoint); return "dead"; }, }); - expect(seen).toEqual([{ skipPortWait: false, orphanDiscoveryCapMs: 3_000 }]); + expect(seen).toEqual([{ target: null, skipPortWait: false, orphanDiscoveryCapMs: 3_000 }]); } - // The stop path consulted its survivor injection; the removal path asked the - // tri-state endpoint probe instead, because a null survivor is not proof of absence. - expect(survivors).toEqual(["no-args"]); - expect(probed.length).toBe(1); + expect(probed.length).toBe(2); } finally { setWindowsServiceStopHooksForTests(null); } diff --git a/tests/service/stop-direct-fetch.test.ts b/tests/service/stop-direct-fetch.test.ts index 010b57682ad..4d2cffde617 100644 --- a/tests/service/stop-direct-fetch.test.ts +++ b/tests/service/stop-direct-fetch.test.ts @@ -134,7 +134,7 @@ describe("local stop transport", () => { // the receipt and the no-forced-fallback rule; the transport default is still not overridden. for (const [file, functionName, count, call] of [ ["src/cli/index.ts", "handleStopUnlocked", 1, "stopProxy"], - ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 2, "stopProxy"], + ["src/service/orchestration.ts", "stopTrackedProxyIfRunning", 3, "stopProxy"], // One site: the target is resolved before the receipt, so both the tracked and the // discovered path reach the stop through the same call (#7771 round 8). ["src/cli/uninstall-stop.ts", "stopUninstallProxy", 1, "io.stop"], diff --git a/tests/windows/windows-stop-callers.test.ts b/tests/windows/windows-stop-callers.test.ts index d92d0f044a9..19ef54be61c 100644 --- a/tests/windows/windows-stop-callers.test.ts +++ b/tests/windows/windows-stop-callers.test.ts @@ -9,10 +9,14 @@ import { afterEach, describe, expect, test } from "bun:test"; import { inspectGuardedTarget, resetOcxWindowsStopForTests, stopGuardedManager } from "../../src/cli/stop-windows-checked"; import { runGuardedManagerStep, type GuardedStopSnapshot } from "../../src/cli/stop-approval"; import { ROUTE_PART_A_CAP_MS, ROUTE_STRAY_SWEEP_CAP_MS, routeServiceStop } from "../../src/server/stop-service-route"; -import { completeWindowsServiceStopReceipt, windowsServiceRemovalStop, windowsServiceStopBeforeRestore } from "../../src/service/cli-windows-stop"; +import { completeWindowsServiceStopReceipt, windowsServiceRemovalStop as removalStop, windowsServiceStopBeforeRestore as stopBeforeRestore } from "../../src/service/cli-windows-stop"; import { inspectGuardedManagerTarget, type GuardedManagerTarget } from "../../src/service/guarded-manager-target"; import type { WindowsCheckedStopDeps } from "../../src/service/windows-checked-stop"; import type { WindowsProcessWatch } from "../../src/service/windows-process-native"; + +const fixtureTarget = { pid: 4321, endpoint: { hostname: "127.0.0.1", port: 10100 } }; +const windowsServiceStopBeforeRestore: typeof stopBeforeRestore = io => stopBeforeRestore({ resolveTarget: async () => fixtureTarget, ...io }); +const windowsServiceRemovalStop: typeof removalStop = io => removalStop({ resolveTarget: async () => fixtureTarget, ...io }); import type { WindowsStopProcessEntry } from "../../src/service/windows-process-evidence"; import { runWindowsServiceStopPartA, serviceStopOutcomeFromWindows, setStopDeadlineMs, setWindowsServiceStopHooksForTests } from "../../src/service/windows-service-stop"; import { CMD, PATHS, SCHEDULE_PID, VBS, entry, launcher, loop, proxy, scheduleHost } from "../helpers/windows-checked-stop-fixtures"; @@ -86,6 +90,33 @@ describe("/api/stop service stop on Windows", () => { }); describe("ocx service stop on Windows", () => { + test("a discovered orphan's endpoint is used for both receipt and stop", async () => { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + const endpoint = { hostname: "127.0.0.1", port: 12001 }; + let claimed: unknown; + let stopped: unknown; + const result = await windowsServiceStopBeforeRestore({ + ownsSharedTeardown: () => true, + resolveTarget: async () => ({ pid: 4321, endpoint }), + claim: (address) => { claimed = address; return { nonce: "n" } as never; }, + stopTracked: async io => { stopped = io?.target; return "stopped"; }, + }); + expect(result.ok).toBe(true); + expect(claimed).toEqual(endpoint); + expect(stopped).toEqual({ pid: 4321, endpoint }); + }); + + test("no resolved target claims no receipt", async () => { + setWindowsServiceStopHooksForTests({ partA: async () => partAResult("stopped"), partB: async () => partBResult("freed") }); + let claimed = false; + let target: unknown = "unset"; + const result = await windowsServiceStopBeforeRestore({ resolveTarget: async () => null, + ownsSharedTeardown: () => true, claim: () => { claimed = true; return { nonce: "n" } as never; }, + stopTracked: async io => { target = io?.target; return "none"; } }); + expect(result.ok).toBe(true); + expect(claimed).toBe(false); + expect(target).toBeNull(); + }); test("a Part A other than stopped/absent refuses before the receipt and the child stop", async () => { for (const outcome of ALL_PART_A_FAILURES) { const steps: string[] = []; @@ -143,7 +174,7 @@ describe("ocx service stop on Windows", () => { expect(result).toEqual({ ok: true }); // #7771: the stop also names the admitted discovery cap, so the helper cannot spend 7 s // of a budget whose 3 s for this step was already proven at admission. - expect(children).toEqual([{ skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); + expect(children).toEqual([{ target: fixtureTarget, skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); }); test("Part B failure keeps the receipt (no restore); a foreign listener does not fail the stop", async () => { @@ -329,7 +360,7 @@ describe("ocx service uninstall / remove on Windows", () => { expect(result).toEqual({ ok: true }); // #7771: the stop also names the admitted discovery cap, so the helper cannot spend 7 s // of a budget whose 3 s for this step was already proven at admission. - expect(children).toEqual([{ skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); + expect(children).toEqual([{ target: fixtureTarget, skipPortWait: true, orphanDiscoveryCapMs: 3_000 }]); }); }); From 18a0787f44db7767ed83a26541faae139b5fa896 Mon Sep 17 00:00:00 2001 From: JUN Date: Fri, 13 Nov 2026 10:23:11 +0900 Subject: [PATCH 51/53] Align Windows stop design notes with checked behavior --- devlog/_plan/261111_windows_service_stop/030_design.md | 2 +- .../_plan/261111_windows_service_stop/060_implementation.md | 4 ++-- src/cli/index.ts | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index 4d0843f8537..cb09d7a94f5 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -57,7 +57,7 @@ Part A: 1. Probe registration: `probeWindowsSchedulerTask` and, for WinSW, the service status (both capped at 2 s; an overrun or unreadable answer is `unknown`, nothing changed). A running WinSW service is only this home's manager when its registered `BINARY_PATH_NAME` (`sc qc`) names this installation's WinSW executable: the service ID is fixed, so a successful `sc query` alone would let Part A stop another installation's service. An unreadable registration is `unknown`; a foreign executable is treated as not registered for this home. With nothing registered, the process enumeration that looks for a stray wrapper of a deleted task (capped at 5 s) also runs here, before admission. Nothing registered and nothing bound → `absent`, return. 2. Bind (rule above) and record (pid, startTime, image, command-line digest). 3. Ask the manager. Task Scheduler: `schtasks /end` (cap 2 s). WinSW: a non-waiting stop (the stop command capped at 2 s instead of today's 15 s `SERVICE_COMMAND_TIMEOUT_MS`) followed by a capped SCM state poll (`sc query`) instead of today's `stopwait`; if WinSW reports stopped but the proxy is still alive, the outcome is `survived`; the poll's cap is the service's `` (20 s in the generated definition) plus 2 s, because WinSW itself stops the proxy within its stop timeout. Access denied → `denied`, stop there. A cap overrun → `unknown`, with nothing further changed (no terminate). -4. Terminate bound wrapper survivors through the handle primitive (`cmd.exe` running the loop first, then the launcher). WinSW has no wrapper, so this step is empty there. The proxy child is NOT terminated here: it stops through the existing graceful path (`stopProxy` / `stopWithDeferral` / `signalApproved`) with its drain, the #7423 teardown-deferral receipt, and the rule that a received refusal (409, `ProxyOwnershipRefusedError`) forbids a forced stop. It is force-killed only where today's code already forces after a graceful timeout, never after a refusal. This operation gives no new kill authority over the child. +4. Terminate bound wrapper survivors through the handle primitive (`cmd.exe` running the loop first, then the launcher). WinSW-only registration also checks for wrappers left by a previous backend. The proxy child is NOT terminated here: it stops through the existing graceful path (`stopProxy` / `stopWithDeferral` / `signalApproved`) with its drain, the #7423 teardown-deferral receipt, and the rule that a received refusal (409, `ProxyOwnershipRefusedError`) forbids a forced stop. It is force-killed only where today's code already forces after a graceful timeout, never after a refusal. This operation gives no new kill authority over the child. 5. Wait (cap 3 s) until every bound wrapper has exited, by waiting on the process handles opened at bind time (no enumeration, no PowerShell during the wait), then read the manager state once with `schtasks /query` / `sc query` only (cap 1 s). The CIM enumeration that would find a wrapper started after bind is NOT repeated here; it runs once at the end of part B, where a late relaunch is what it has to catch. ## Time budget (one deadline, checked before anything changes) diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 11c1b21d4ca..81c81e75c04 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -37,8 +37,8 @@ Contract doc: `structure/ops/windows-checked-stop.md`. ## Where PR1 differs from 030 -- **Part B's closing sample.** The 6 s hold is measured between port sample starts, and the window floor is also 6 s, so the last sample starts at the window's end and can run up to its 0.5 s cap past it. The budget counts this, which raises the admission minimums to 48.5 / 56.5 / 42.5 s. The Desktop drain case becomes 70.5 s, which still fits in 72 s. -- **WinSW binds nothing.** WinSW-only part A neither enumerates nor binds. This follows 030's step 4 and its 25 s row: a stray scheduler wrapper left behind by an earlier backend is caught only when nothing is registered. "Proxy still alive after WinSW stopped" is an injected `proxyAlive` check inside the 1 s manager-state step. An answer of "cannot tell" leaves the decision to part B. +- **Part B's closing sample.** The 6 s hold is measured between port sample starts, and the window floor is also 6 s, so the last sample starts at the window's end and can run up to its 0.5 s cap past it. The budget counts this; the current admission minimums are 48.5 / 56.5 / 50.5 s, including the later WinSW wrapper check. The Desktop drain case becomes 70.5 s, which still fits in 72 s. +- **WinSW wrapper evidence.** WinSW-only Part A enumerates and binds wrappers, including with the service stopped, so the own-wrapper guard and stray evidence apply. "Proxy still alive after WinSW stopped" is an injected `proxyAlive` check inside the 1 s manager-state step. An answer of "cannot tell" leaves the decision to Part B. - **Terminates share the wait cap.** 030's tables do not price the terminate calls themselves. They share the 3 s handle-wait window, which matters for the ARM64 script (about 1 s each). - **Unbound wrappers with nothing registered.** Binding finds no bound wrapper but an unbound one is alive. The result is `survived` / `unbound-wrapper`, not `absent`, matching 040's bullet; the 030 "absent" definition reads as "nothing bound" alone. Nothing is changed either way. - **Unreadable candidates with nothing registered.** An unreadable wrapper image above the child counts only while a manager is registered. A hand-started proxy under an elevated shell would otherwise lose the stop it gets today. A matching wrapper without a creation time stays `unknown` even with nothing registered. diff --git a/src/cli/index.ts b/src/cli/index.ts index e62225e3461..66601f3c282 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -1265,8 +1265,8 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) { // the proxy's shutdown handlers actually run; taskkill /F is the fallback inside. // Shared teardown is deferred to this process: it happens after the respawn // verification below, so a survivor does not get its client config pulled first. - // The receipt goes down first — the proxy honours the deferral only when it can - // see one, so an unrecordable claim degrades to the child doing its own teardown. + // The receipt goes down first; Windows refuses if it cannot be recorded. + // Other platforms retain the existing undeferred fallback. nativeRestoreHandledByProxy = await stopWithDeferral(pid); console.log(`✅ Proxy (PID ${pid}) stopped.`); removePid(pid); From b8c24aa7b3b2d8636088e33a8de26a863114466c Mon Sep 17 00:00:00 2001 From: t Date: Fri, 13 Nov 2026 11:41:25 +0900 Subject: [PATCH 52/53] fix(cli): an absent-Part-A uninstall proves every candidate endpoint dead after a confirmed stop A confirmed tracked stop proved only the tracked proxy down. After an absent Part A on Windows no Part B follows, so a second proxy on another candidate endpoint went unseen and shared config could be restored under it (#3008). The stop now also requires every candidate endpoint to answer dead, with the candidate set captured before the stop removes the runtime record. After a stopped Part A, Part B confirms the port as before. Refs #7771 --- src/cli/index.ts | 15 +++--------- src/cli/uninstall-plan.ts | 23 ++++++++++++++++++ src/cli/uninstall-stop.ts | 6 ++++- tests/cli/uninstall.test.ts | 47 +++++++++++++++++++++++++++++++++---- 4 files changed, 74 insertions(+), 17 deletions(-) diff --git a/src/cli/index.ts b/src/cli/index.ts index 66601f3c282..03cee08f0e1 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -80,7 +80,7 @@ import { } from "../config/pending-teardown"; import { collectStatus, detectMissingCodexCatalogPath, hubStatusLines, missingCodexCatalogLines, remoteHubBannerLine, remoteHubStatusLines, runtimeSupervisorLine, statusDeadProxyLines, statusProxyUp, statusUnusedProxyLines } from "./status"; import { stopUninstallProxy } from "./uninstall-stop"; -import { endpointsToProve, everyEndpointProvenDownAsync, respawnWindowRequired, sharedTeardownAuthorized, uninstallServiceRemoval, type UninstallObservation } from "./uninstall-plan"; +import { captureUninstallEndpointProof, endpointsToProve, respawnWindowRequired, sharedTeardownAuthorized, uninstallServiceRemoval, type UninstallObservation } from "./uninstall-plan"; import { takeFlag } from "./runtime-api"; import { parseStartOptions, StartArgsError } from "./start-args"; @@ -1527,17 +1527,8 @@ async function handleUninstall() { // Before every teardown step: an elevated run must not act on a home another account can change. const privilegeRefusal = uninstallPrivilegeRefusal(getConfigDir()); if (privilegeRefusal) { console.error(`❌ ${privilegeRefusal}`); process.exit(1); } - /** Definitive "nothing is answering" on the endpoint this home would serve. */ - const proxyEndpointProvenDown = async (): Promise => { - try { - // Every candidate, not just the preferred one: a stale runtime record pointing at a - // closed port would otherwise "prove" a live proxy on the configured port is gone. - const endpoints = endpointsToProve(readRuntimePort(), loadConfig()); - return await everyEndpointProvenDownAsync(endpoints, probeEndpointLiveness); - } catch { - return false; - } - }; + /** Definitive "nothing is answering" on every endpoint this home could serve, captured before the stop. */ + const proxyEndpointProvenDown = captureUninstallEndpointProof(() => endpointsToProve(readRuntimePort(), loadConfig()), probeEndpointLiveness); const failures: string[] = []; const runStep = async (label: string, step: () => void | boolean | Promise) => { diff --git a/src/cli/uninstall-plan.ts b/src/cli/uninstall-plan.ts index 97bd1dc9feb..a1502b7a794 100644 --- a/src/cli/uninstall-plan.ts +++ b/src/cli/uninstall-plan.ts @@ -142,3 +142,26 @@ export async function everyEndpointProvenDownAsync( const results = await Promise.all(endpoints.map(e => probe(e))); return results.every(result => result === "dead"); } + +/** + * The uninstall's "nothing answers on any endpoint this home could serve" proof, with the + * candidate set captured NOW. The proxy stop removes the runtime record (and a stopped proxy + * may remove its own), so a set read after the stop would silently drop the stopped proxy's + * port. Every candidate, not just the preferred one: a stale runtime record pointing at a + * closed port would otherwise "prove" a live proxy on the configured port is gone. Any + * failure to read or probe is not proof. + */ +export function captureUninstallEndpointProof( + readCandidates: () => ProbeEndpoint[], + probe: (e: ProbeEndpoint) => Promise<"live" | "dead" | "unknown"> | "live" | "dead" | "unknown", +): () => Promise { + let candidates: ProbeEndpoint[] | null = null; + try { candidates = readCandidates(); } catch { candidates = null; } + return async () => { + try { + return candidates !== null && await everyEndpointProvenDownAsync(candidates, probe); + } catch { + return false; + } + }; +} diff --git a/src/cli/uninstall-stop.ts b/src/cli/uninstall-stop.ts index 709de299b77..d2aa2a31e0b 100644 --- a/src/cli/uninstall-stop.ts +++ b/src/cli/uninstall-stop.ts @@ -77,7 +77,11 @@ export async function stopUninstallProxy(io: UninstallProxyStopIo): Promise => { - if (stopped) return true; + // A confirmed stop proves the tracked proxy down, nothing more. After an "absent" Part A on + // Windows no Part B follows, so a second proxy on another candidate endpoint would go + // unseen and shared config would be restored under it (#3008): every candidate has to + // answer definitively dead. After "stopped", Part B confirms the port instead. + if (stopped) return !(windows && io.serviceStop === "absent") || await io.endpointProvenDown(); if (forbidForcedFallback) return false; return await io.endpointProvenDown(); }; diff --git a/tests/cli/uninstall.test.ts b/tests/cli/uninstall.test.ts index 64c346df129..5f3b18db9e6 100644 --- a/tests/cli/uninstall.test.ts +++ b/tests/cli/uninstall.test.ts @@ -12,7 +12,7 @@ import { dirname, join } from "node:path"; import { removeOwnedConfigAfterDesktopCleanup, type UninstallClientStateDeps } from "../../src/cli/uninstall-client-state"; import { cleanupOwnedIntegrationsBeforeUninstall } from "../../src/cli/uninstall-integrations"; import { assertClientLifecycleHeld, withClientLifecycle, withClientLifecycleSync, type ClientLifecycleHeld } from "../../src/client/lifecycle-lock"; -import type { UninstallObservation } from "../../src/cli/uninstall-plan"; +import { captureUninstallEndpointProof, type UninstallObservation } from "../../src/cli/uninstall-plan"; import type { DesktopDisconnectReceipt } from "../../src/claude/desktop-remote-store"; import type { ExportModel } from "../../src/clients/config-export"; import { INTEGRATION_CLIENTS } from "../../src/integrations/registry"; @@ -470,9 +470,13 @@ describe("uninstall gates shared teardown on a proven service stop", () => { expect(windowStep).toContain("could not be confirmed down either"); expect(windowStep.indexOf("if (!await proxyEndpointProvenDown())")) .toBeLessThan(windowStep.indexOf("observed.respawnWindowVerified = true;")); - // And the proof itself asks every candidate. - expect(fn).toContain("endpointsToProve(readRuntimePort(), loadConfig())"); - expect(fn).toContain("everyEndpointProvenDownAsync(endpoints, probeEndpointLiveness)"); + // And the proof itself asks every candidate, captured before the proxy stop removes the + // runtime record; the helper requires every captured candidate to answer dead. + expect(fn).toContain("captureUninstallEndpointProof(() => endpointsToProve(readRuntimePort(), loadConfig()), probeEndpointLiveness)"); + expect(fn.indexOf("captureUninstallEndpointProof(")).toBeLessThan(fn.indexOf('runStep("proxy stopped"')); + const plan = await readText("src/cli/uninstall-plan.ts"); + const helper = plan.slice(plan.indexOf("export function captureUninstallEndpointProof(")); + expect(helper).toContain("everyEndpointProvenDownAsync(candidates, probe)"); }); const safeTeardown: UninstallObservation = { @@ -1198,4 +1202,39 @@ describe("uninstall proxy stop defers the shared teardown through a receipt (#77 completeWindowsServiceStopReceipt(NONCE, nonce => { cleared.push(nonce); return true; }); expect(cleared).toEqual([NONCE]); }); + + test("after an absent Part A a confirmed tracked stop still proves every candidate endpoint dead", async () => { + // A confirmed stop proves the tracked proxy down, not that nothing else serves on this + // home's ports; with no manager registered no Part B follows, so a second proxy on the + // configured endpoint would otherwise go unseen and shared config be restored under it. + let probes = 0; + const live = await stopUninstallProxy(io({ + serviceStop: "absent", + stop: async () => true, + endpointProvenDown: async () => { probes++; return false; }, + })); + expect(probes).toBe(1); + expect(live.provenDown).toBe(false); + const dead = await stopUninstallProxy(io({ serviceStop: "absent", stop: async () => true, endpointProvenDown: async () => true })); + expect(dead.provenDown).toBe(true); + // After a stopped Part A, Part B confirms the port, so the stop's own proof stands. + probes = 0; + const stopped = await stopUninstallProxy(io({ serviceStop: "stopped", stop: async () => true, endpointProvenDown: async () => { probes++; return false; } })); + expect(probes).toBe(0); + expect(stopped.provenDown).toBe(true); + }); + + test("the uninstall endpoint proof reads its candidates once, before the stop removes the runtime record", async () => { + let reads = 0; + const probed: number[] = []; + let runtime: { port: number } | null = { port: 12000 }; + const proof = captureUninstallEndpointProof(() => { reads++; return runtime ? [{ hostname: "127.0.0.1", port: runtime.port }, { hostname: "127.0.0.1", port: 10100 }] : [{ hostname: "127.0.0.1", port: 10100 }]; }, + endpoint => { probed.push(endpoint.port); return endpoint.port === 12000 ? "dead" : "dead"; }); + runtime = null; // the tracked stop removed the runtime record + expect(await proof()).toBe(true); + expect(reads).toBe(1); + expect(probed).toEqual([12000, 10100]); + const unreadable = captureUninstallEndpointProof(() => { throw new Error("config unreadable"); }, () => "dead"); + expect(await unreadable()).toBe(false); + }); }); From 5b816d67196695b303e7dc5988b8ab618dc3b97a Mon Sep 17 00:00:00 2001 From: t Date: Fri, 13 Nov 2026 11:47:49 +0900 Subject: [PATCH 53/53] docs(service): describe every receipt writer and the all-endpoint proof Refs #7771 --- devlog/_plan/261111_windows_service_stop/030_design.md | 2 +- .../_plan/261111_windows_service_stop/060_implementation.md | 2 +- docs-site/src/content/docs/reference/cli/lifecycle.md | 5 +++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/devlog/_plan/261111_windows_service_stop/030_design.md b/devlog/_plan/261111_windows_service_stop/030_design.md index cb09d7a94f5..a4a4179cdea 100644 --- a/devlog/_plan/261111_windows_service_stop/030_design.md +++ b/devlog/_plan/261111_windows_service_stop/030_design.md @@ -22,7 +22,7 @@ Part A replaces, on Windows only, the "manager stop + best-effort wrapper kill" - `ocx service stop` calls part A instead of `stopWindows`, then claims a teardown receipt (as `ocx stop` does) before stopping the tracked proxy, so the route defers the shared teardown; it completes and clears the receipt only after part B passes and keeps it for recovery otherwise. If the claim fails while this home owns the shared teardown, it does not send an undeferred child stop at all: it reports the failure and leaves the proxy running, because part A only covers wrappers it bound and a wrapper that appears later could relaunch the proxy after an undeferred teardown. The existing refusals and the not-owned teardown path are unchanged. - `ocx service uninstall` / `remove` call part A before deleting the task (today they call `stopWindows` and kill no wrappers, so a looping wrapper can outlive its task, #764), then part B. - `prepareServiceInstall` (service install and backend switch) calls part A instead of its manager stop + wrapper kill, then its existing tracked-proxy stop, then part B. A part B result other than a free port (our identity back, or `portHeldByForeign`) stops install and switch before they publish assets or start the service, with the port owner named; the service outcome stays `stopped` and no enum changes. -- `/api/stop`: when the caller presents a valid receipt, the parent `ocx stop` has already run part A, so the route skips part A (no second enumeration, `/end` or wait inside the proxy) and only performs its graceful shutdown; part B belongs to the parent, since the route cannot observe its own post-exit window. Only `ocx stop` and `ocx service stop` write the receipt, and each sends the POST only after its own part A returned `stopped` or `absent` (see "Two parts"). The receipt alone is not the proof: after part A the parent records its outcome in the receipt (`partA: "stopped" | "absent"`, with the bound manager and the wrapper set it ended as pid + creation time). The route skips part A only when that record is present for this attempt and the manager it names is still the registered one; a receipt without the record (an older CLI, or a receipt written before part A) gets today's route behavior, which re-runs the service stop and answers 409 on failure. Without a receipt the existing `respawnable_service` refusal stays for a respawnable Task Scheduler service. Where the respawn risk is `none` the route splits by case. For a registered WinSW service it keeps today's in-process service stop unchanged; part A is not run there, because WinSW's own stop ends this very process and cannot be capped from inside it. For a deleted task with a stray wrapper it runs only part A's wrapper sweep (bind, terminate, handle wait) with a 4 s cap, and then, before it answers, its own shared teardown (`performStopTeardown`, which a caller without a receipt still gets); the drain runs after the answer. Every parent POST gets the full 10 s abort, never a shortened one, and admission always counts it at 10 s; the implementation must measure the teardown's worst case and keep sweep plus teardown under those 10 s, and 040 carries that as a test. A sweep overrun answers `service_state_unknown` and changes nothing further. +- `/api/stop`: when the caller presents a valid receipt, the parent `ocx stop` has already run part A, so the route skips part A (no second enumeration, `/end` or wait inside the proxy) and only performs its graceful shutdown; part B belongs to the parent, since the route cannot observe its own post-exit window. `ocx stop`, `ocx service stop`, `ocx service uninstall`/`remove`, `ocx uninstall` and service install/switch write the receipt before the child stop, naming the endpoint that stop contacts (the tracked pid's own runtime record or the discovered live proxy), and on Windows a receipt that cannot be recorded means no child stop. Each of them except `ocx uninstall` sends the POST only after its own part A returned `stopped` or `absent` (see "Two parts"); `ocx uninstall` still sends the graceful POST after an unproven part A, but with the forced fallback forbidden and the failure recorded, so shared teardown stays blocked. The receipt alone is not the proof: after part A the parent records its outcome in the receipt (`partA: "stopped" | "absent"`, with the bound manager and the wrapper set it ended as pid + creation time). The route skips part A only when that record is present for this attempt and the manager it names is still the registered one; a receipt without the record (an older CLI, or a receipt written before part A) gets today's route behavior, which re-runs the service stop and answers 409 on failure. Without a receipt the existing `respawnable_service` refusal stays for a respawnable Task Scheduler service. Where the respawn risk is `none` the route splits by case. For a registered WinSW service it keeps today's in-process service stop unchanged; part A is not run there, because WinSW's own stop ends this very process and cannot be capped from inside it. For a deleted task with a stray wrapper it runs only part A's wrapper sweep (bind, terminate, handle wait) with a 4 s cap, and then, before it answers, its own shared teardown (`performStopTeardown`, which a caller without a receipt still gets); the drain runs after the answer. Every parent POST gets the full 10 s abort, never a shortened one, and admission always counts it at 10 s; the implementation must measure the teardown's worst case and keep sweep plus teardown under those 10 s, and 040 carries that as a test. A sweep overrun answers `service_state_unknown` and changes nothing further. - The updater's `stopWindowsServiceWrappersBestEffort` moves to part A in the same change (Task Scheduler backend; for the native WinSW backend it keeps today's behavior of stopping WinSW without a wrapper kill, because WinSW has no `:loop` wrapper), through an authorizer-gated variant (`guardedStopWindowsServiceChecked(authorize)`) that runs the authorizer once at entry and again before each manager stop and each terminate, as the existing `mutationGate` does, so an update cannot race a respawned runtime either. The updater keeps its own reclaim checks. Its three call sites in src/update/job.ts discard the result today, and one runs inside the synchronous `preparePort` callback; each gets a return path (the callback becomes async, or part A runs before it) so a non-`stopped` outcome really aborts the update before files are replaced. Desktop reaches it through two `ocx stop` children, both ordinary CLI processes run as the user: the consented takeover at startup (`--expect-*`, under the startup sequence's deadline, which `resolve_consent` extends by the consent wait plus 60 s on a Windows approval) and the exit/restart drain (plain `ocx stop --json` under `runtime_stop::DEADLINE`, 30 s, desktop/src-tauri/src/exit.rs). The #5443 boundary is unchanged and Desktop never elevates. diff --git a/devlog/_plan/261111_windows_service_stop/060_implementation.md b/devlog/_plan/261111_windows_service_stop/060_implementation.md index 81c81e75c04..8bf7916cbb3 100644 --- a/devlog/_plan/261111_windows_service_stop/060_implementation.md +++ b/devlog/_plan/261111_windows_service_stop/060_implementation.md @@ -82,7 +82,7 @@ PR2 wires every caller listed above; `structure/ops/windows-checked-stop.md` ("C - **Uninstall's proxy stop claims its receipt before the child stop.** `handleUninstall` sent `/api/stop` without a teardown receipt, so the route finished the shared teardown while the wrapper respawn window had not been checked yet; a failed `verifyUninstallStopWindow` then left shared config restored under a wrapper that could relaunch the proxy. On Windows the uninstall proxy stop is now `src/cli/uninstall-stop.ts`: it claims a receipt carrying Part A's record before the child stop, sends its nonce, and discharges it only after Part B and the shared restore. A failed claim sends no child stop; a failed later step keeps the receipt for recovery. - **Uninstall never forces a stop its Part A could not prove.** When Part A did not end `stopped` or `absent`, the uninstall proxy stop still sends the graceful POST but passes `allowForcedFallback: false` to `stopProxy`, so a POST timeout, transport error or non-409 answer cannot fall through to `taskkill /T /F`. The uninstall failure stays recorded. A 409 refusal already forbids the forced path; this closes the remaining ones. -- **An absent Part A is not proof the proxy is down.** `windowsServiceRemovalStop` returned `ok` on Part A `absent` without checking the endpoint, and `findLiveProxy` answers null when its 3 s discovery cap runs out even while the endpoint is live. Removal now probes the endpoint it captured, once, with the tri-state contract (`endpointLiveness`, the same one `endpointProvenDown` uses in `uninstall-stop` and `proxyEndpointProvenDown` in `src/cli/index.ts`); `proxyStillLiveAfterStop` looked down on a null, which its own probe produces on a timeout as well. Only `"dead"` permits deleting the task — `"live"` refuses and `"unknown"` (a timeout or transport failure) refuses too, and both keep the receipt. No respawn hold is needed: a wait would price removal at Part B's 6 s for a manager that is not registered. +- **An absent Part A is not proof the proxy is down.** `windowsServiceRemovalStop` returned `ok` on Part A `absent` without checking the endpoint, and `findLiveProxy` answers null when its 3 s discovery cap runs out even while the endpoint is live. Removal now probes every candidate endpoint (the runtime record and the configured endpoint, the same set `ocx uninstall` proves, plus the resolved stop target) with the tri-state contract (`endpointLiveness`, the same one `endpointProvenDown` uses in `uninstall-stop` and `proxyEndpointProvenDown` in `src/cli/index.ts`); `proxyStillLiveAfterStop` looked down on a null, which its own probe produces on a timeout as well. Only `"dead"` on every candidate permits deleting the task — `"live"` refuses and `"unknown"` (a timeout or transport failure) refuses too, and both keep the receipt. No respawn hold is needed: a wait would price removal at Part B's 6 s for a manager that is not registered. - **Uninstall's receipt names the endpoint its stop contacts.** `stopUninstallProxy` claimed the receipt first and resolved its target afterwards, and `defaultReceiptEndpoint()` marks any runtime record "exact" — so a stale record plus a proxy discovered on another port left a receipt for a port the stop never contacted, and a failed Part A with no proxy left a "guessed" obligation outstanding that recovery refuses. It now resolves the target first (the tracked pid with its own runtime endpoint, or the discovered live proxy's endpoint), claims only when a stop will be attempted, and hands the receipt and `stopProxy` the same endpoint. No target, no receipt. - **An unconfirmed stop removes no tracked state.** The tracked-pid path called `removeTracked(pid)` regardless of the stop's result, so a failed stop deleted the record that a rerun needed. It runs only when the stop was proven, and the step reports `changed` only when state actually changed. - **A forbidden forced fallback waits without killing.** With `allowForcedFallback === false`, `waitForStoppedPort` still called `reclaimListenPort` with `killOcxHolders`/`onlyKillPids: [pid]`, so the wait could kill through `killProxy` — the path the flag exists to close. The wait now runs observationally (the reclaim dependency is injected, and the forbidden case passes a mode that samples without kill eligibility); `stopProxy` keeps its literal `killProxy(pid)` call, which `tests/providers/xai/grok-lifecycle.test.ts` reads as order evidence. diff --git a/docs-site/src/content/docs/reference/cli/lifecycle.md b/docs-site/src/content/docs/reference/cli/lifecycle.md index ab88b791eb6..bda02308126 100644 --- a/docs-site/src/content/docs/reference/cli/lifecycle.md +++ b/docs-site/src/content/docs/reference/cli/lifecycle.md @@ -163,8 +163,9 @@ and `proxy: "unknown"`, and adds `serviceStopReason`: | `unverified-budget` | The time left could not hold a verified stop, so nothing was changed | Retry from a terminal | If another program is listening on the port after this installation's processes are gone, the -stop still succeeds and `--json` reports `portHeldByForeign: true`. Both fields are additive: -they are omitted when there is nothing to report, and no new `outcome` value exists. +stop still succeeds and `--json` reports `portHeldByForeign: true`. Both fields are additive and +no new `outcome` value exists. `serviceStopReason` is omitted when there is nothing to report; +`portHeldByForeign` appears (`true` or `false`) whenever the port check ran, and is omitted otherwise. `--deadline-ms ` bounds the whole stop, measured from the moment the command started. Before changing anything the stop checks that the time left can hold its worst case (about 57 seconds for