Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/guides/web-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ is visible) and never forces an upstream refresh.
| **Native reasoning retention** | Independently opt into reasoning ciphertext forwarding on native model or account changes. Both default off; account retention is experimental. See [the configuration scope](/reference/configuration/server/#native-reasoning-retention). |
| **Sub-agent delegation** | Choose a native or routed model and optional reasoning effort shared by OpenCodex delegation guidance and the separate native-default opt-in. This is not a proxy-side per-spawn router; see below. |
| **Sidecars** | Choose the web-search model and effort plus the vision-description model. Changes apply on the next request. |
| **Maintenance** | Resync the Codex model catalog, inspect project-local config bypass warnings, check the latest or preview release, and run an update with optional proxy restart. In the desktop shell, its update entry opens the native app update page instead of running the package updater. |
| **Maintenance** | Resync the Codex model catalog, inspect project-local config bypass warnings, check the latest or preview release, and run an update with optional proxy restart. With restart on, Codex routing and its model catalog stay in place while the package is replaced, so a Codex app that loads them during the update still sees your routed models; the restarted proxy takes them over, and `ocx stop` restores native Codex if it never comes back. In the desktop shell, its update entry opens the native app update page instead of running the package updater. |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
| **Startup safety** | Show whether injected Codex routing survives a restart, with separate service and launcher-shim health plus exact repair commands. |
| **Windows tray** | Install a per-user login tray for one-click proxy start, stop, restart, dashboard access, and status. The tray is a controller, not a proxy restart service. |
| **Codex autostart** | Allow an already-installed Codex launcher shim to run `ocx ensure`. This toggle does not install a shim or background service. |
Expand Down
7 changes: 5 additions & 2 deletions docs-site/src/content/docs/reference/cli/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,11 @@ failure to restore shared client configuration keeps the stop failed and its rec
carries the outcome class (`stopped`, `not-running`, `history-incomplete`, `history-deferred`,
or `failed`), the service and proxy classifications, whether the runtime is down, and a stable
one-line message. Exit codes are identical with and without `--json`: 0 on success, 1 on failure,
79 when only Codex history cleanup did not complete, and 80 when the shared teardown was deferred
and is still owed.
79 when only Codex history cleanup did not complete, 80 when the shared teardown was deferred
and is still owed, and 81 when a dashboard update that will restart the proxy handed Codex routing
and its catalog to the restarted proxy instead of restoring native Codex. With 81 the proxy is
down, and the restarted proxy releases the kept teardown after its first sync; if it never starts,
the next `ocx stop` restores native Codex.

For a desktop-approved takeover, `ocx stop --json` also accepts the complete set of
`--expect-*` values supplied by `ocx resolve --json`. On Windows, this guarded stop can
Expand Down
2 changes: 1 addition & 1 deletion scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -1848,7 +1848,7 @@
"update-refresh.test.ts": "update",
"update-restart-lease.test.ts": "update",
"update-stop-classification.test.ts": "update",
"update-stop-first.test.ts": "update",
"update-stop-first.test.ts": "update", "update-teardown-handover.test.ts": "update",
"update-transactional-leftovers.test.ts": "update",
"update-transactional.test.ts": "update",
"update-tray-handoff.test.ts": "update",
Expand Down
14 changes: 13 additions & 1 deletion src/cli/cross-home-owner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ import { detectCodexRoutingDrift } from "../codex/routing-drift";
import { readBoundedCodexConfig } from "../codex/inject/bounded-config-reader";
import { currentExternalCodexModelProvider } from "../codex/inject";
import { reconcileJournal } from "../codex/journal";
import { shouldSyncCodexOnStart } from "../codex/desired-state";
import { loadConfig } from "../config";
import { updateTeardownHandoverPending } from "../update/teardown-handover";
import { markSiblingStart, siblingOfLivePort } from "../codex/sibling-start";
import { readClientConnectionState } from "../client/state";
import { findManagedRegion, resolveGrokHome } from "../grok/inject";
Expand Down Expand Up @@ -350,5 +353,14 @@ export function reconcileStartupJournal(): void {
const clientState = readClientConnectionState();
reconcileJournal(clientState.kind === "connected"
? { activeClientApiKeyId: clientState.value.apiKeyId }
: undefined);
: { adoptUpdateHandover: updateTeardownHandoverPending() && startupSyncEnabled() });
}

/**
* #7423: adoption keeps routing only for a startup that will re-publish it. A user who has
* since turned the Codex integration off gets the ordinary recovery, even with a stale
* handover receipt on disk; an unreadable config is treated the same way.
*/
function startupSyncEnabled(): boolean {
try { return shouldSyncCodexOnStart(loadConfig()); } catch { return false; }
}
27 changes: 21 additions & 6 deletions src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ import {
type RetainedCodexProviderTable,
} from "../codex/inject/restore";
import { stripGrokConfig } from "../grok/inject";
import { STOP_HISTORY_DEFERRED_EXIT_CODE, STOP_HISTORY_INCOMPLETE_EXIT_CODE } from "../update/stop-contract.mjs";
import { STOP_HISTORY_DEFERRED_EXIT_CODE, STOP_HISTORY_INCOMPLETE_EXIT_CODE, STOP_TEARDOWN_HANDED_OVER_EXIT_CODE } from "../update/stop-contract.mjs";
import { announceStopHandover, completeUpdateTeardownHandover, forgetUpdateTeardownHandover, stopHandsTeardownOver, updateTeardownHandoverRequested } from "../update/teardown-handover";
import {
describeHistoryJobFailure,
resolveCodexHistoryJobTarget,
Expand Down Expand Up @@ -143,7 +144,7 @@ import {
StartOwnershipRollbackUncertainError,
} from "./start-ownership-publication";
import { syncModelsToCodex } from "../codex/sync";
import { localClientSkipReason, shouldSyncGrokOnStart, syncCodexOnStartIfEnabled } from "../codex/desired-state";
import { localClientSkipReason, shouldSyncCodexOnStart, shouldSyncGrokOnStart, syncCodexOnStartIfEnabled } from "../codex/desired-state";
import { honorSiblingMarker, markSiblingStart, siblingOfLivePort, siblingRuntimeField, siblingStopFoundOwner, withoutSiblingMarker } from "../codex/sibling-start";
import { consumeSiblingHandoff } from "../codex/sibling-handoff";
import {
Expand Down Expand Up @@ -384,6 +385,7 @@ async function findProxyOwnerBeforeJournalRecovery(
}

async function handleStart(options: { block?: boolean } = {}) {
forgetUpdateTeardownHandover(); // #7423: a recovery start must not pass the marker to its update workers
updateRestartChild?.check();
// A supervised service child defers to a foreign recorded owner before doing
// anything else. 'ocx service start' refuses this activation path already, but
Expand Down Expand Up @@ -752,6 +754,8 @@ async function handleStart(options: { block?: boolean } = {}) {
},
);
if (!startupSync.ran) console.log(startupLeftCodexNativeLine(localClientSkipReason(config), server.port ?? port));
// #7423: routing an update kept in place for this proxy is ours once the startup sync published.
if (!siblingStart && startupSync.published) completeUpdateTeardownHandover(pid => isProcessAlive(pid) && isLikelyOcxProcess(pid));
Comment thread
lidge-jun marked this conversation as resolved.
Resolved
await refreshOwnedRaycastCatalog(config, port);
if (!cleaned && !shuttingDown) healthSentinel = startServiceHealthSentinel(config); // after startup reconciliation; never mid-boot or mid-drain
// #1046: one warning per startup, after BOTH writes. The server's cache
Expand Down Expand Up @@ -1130,11 +1134,14 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) {
const inheritedTeardowns = listPendingTeardowns()
.filter(read => isPendingTeardownAbandoned(read, teardownOwnerStillRunning));
let teardownNonce: string | undefined;
// #7423: an update that restarts the proxy asks this stop to leave Codex routing for the replacement.
const handoverRequested = updateTeardownHandoverRequested(process.env, () => shouldSyncCodexOnStart(loadConfig())) && !stoppingSibling && !connectedClientSkip;
let deferralHonoredByProxy = false;
const claimTeardown = (endpoint: { hostname: string; port: number }, endpointSource: "exact" | "guessed") => {
if (teardownNonce || stoppingSibling) return;
if (connectedClientSkip) return;
try {
teardownNonce = claimPendingTeardown(endpoint, endpointSource).nonce;
teardownNonce = claimPendingTeardown(endpoint, endpointSource, process.pid, handoverRequested ? "update-restart" : undefined).nonce;
} catch (err) {
// Without a receipt the proxy performs its own teardown, which is the pre-#3008
// behaviour: correct for every backend that cannot respawn, and merely early for
Expand All @@ -1161,12 +1168,15 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) {
// A valid receipt means the proxy deferred shared teardown to this process. If the
// receipt could not be written, the proxy restores it itself and the caller must not
// attempt a second restore after a graceful stop. A hard-kill always leaves restore
// to this process.
// to this process. #7423: only a confirmed deferral kept the proxy's own exit cleanup off.
if (graceful && teardownNonce) deferralHonoredByProxy = true;
return graceful && !teardownNonce;
};
const approvedEndpoint = snapshot
? { hostname: probeHostname(snapshot.approval.hostname || undefined), port: snapshot.approval.port }
: null;
// Only an exact runtime endpoint may back a handover; without one the ordinary restore runs.
const claimHandoverBeforeManager = () => { const exact = handoverRequested ? endpointOf(readRuntimePort()) : null; if (exact) claimTeardown(exact, "exact"); };
let guardedStep: Awaited<ReturnType<typeof runGuardedManagerStep>> | null = null;
try {
const serviceStop = snapshot
Expand All @@ -1182,7 +1192,8 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) {
})).service
// A sibling never runs under a service manager: an installed one belongs to the live owner,
// and its ownership check would fail this stop and skip the stale-record purge below.
: stoppingSibling || connectedClientAlreadyStopped ? "absent" : stopServiceIfInstalledDetailed();
// #7423: a manager stop takes the proxy down itself, so a handover claims its receipt first.
: stoppingSibling || connectedClientAlreadyStopped ? "absent" : (claimHandoverBeforeManager(), stopServiceIfInstalledDetailed());
record.service = serviceStop;
stoppedService = serviceStop === "stopped" || serviceStop === "stopped-respawnable";
schedulerCanRespawn = serviceStop === "stopped-respawnable";
Expand Down Expand Up @@ -1425,7 +1436,10 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) {
if (stoppingSibling) console.log(`↩️ Client routing stays on the proxy at port ${siblingOfPort}; this sibling had no shared teardown to run.`);
if (connectedClientSkip) console.log(connectedClientSkip);
const restoreBlocked = ownershipBlocked || inheritedBlocks || nativeRestoreHandledByProxy || stoppingSibling || connectedClientSkip !== null;
if (!restoreBlocked) {
const handedOver = stopHandsTeardownOver({ requested: handoverRequested, restoreBlocked, nonce: teardownNonce,
stopFailed, recoveredCount: recoveredNonces.length, proxy: record.proxy, service: record.service, deferralHonored: deferralHonoredByProxy });
if (handedOver) announceStopHandover();
else if (!restoreBlocked) {
if (recoveredNonces.length > 0) {
// A previous deferred stop died before restoring, and the probe says its endpoint is
// not answering. That is the whole point of leaving the receipt behind.
Expand Down Expand Up @@ -1503,6 +1517,7 @@ async function handleStopUnlocked(snapshot?: GuardedStopSnapshot) {
? STOP_HISTORY_DEFERRED_EXIT_CODE
: 1;
}
else if (handedOver) process.exitCode = STOP_TEARDOWN_HANDED_OVER_EXIT_CODE;
const signals = {
failed: stopFailed,
historyOnly: historyOnlyFailure,
Expand Down
7 changes: 6 additions & 1 deletion src/codex/desired-state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ export async function syncCodexOnStartIfEnabled(
config: LocalClientSyncConfig,
sync: CodexStartupSync = defaultStartupSync,
readinessGate?: ReadinessGate,
): Promise<{ ran: boolean; catalogWritten: boolean; cacheSynced: boolean }> {
): Promise<{ ran: boolean; catalogWritten: boolean; cacheSynced: boolean; published?: boolean }> {
if (!shouldSyncCodexOnStart(config)) {
// The user explicitly turned Codex off: there is nothing to sync, so the
// proxy is ready as soon as it is up. The gate is driven here so /readyz
Expand All @@ -277,10 +277,15 @@ export async function syncCodexOnStartIfEnabled(
const outcome = readinessGate
? await runStartupReadinessSync(readinessGate, async () => (await sync(port)) ?? null)
: await sync(port).catch(() => undefined);
// #7423: "published" means this proxy now owns what Codex reads: the sync succeeded and
// wrote config and catalog (or the catalog, when an external provider owns config.toml).
// A skipped sync (sibling, hub gate, desired OFF) published nothing of its own.
const status = (outcome as { status?: unknown } | null | undefined)?.status;
return {
ran: true,
catalogWritten: outcome?.catalogWritten === true,
cacheSynced: outcome?.cacheSynced === true,
published: outcome?.ok === true && (status === "applied" || status === "catalog-only"),
};
}

Expand Down
15 changes: 15 additions & 0 deletions src/codex/journal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,12 @@ function warnIfJournalRetained(): void {

export interface ReconcileJournalOptions {
activeClientApiKeyId?: string;
/**
* #7423: an update stopped the previous proxy and handed its Codex routing to this one.
* A process-owned journal whose owner is gone is then adopted rather than replayed, so
* startup does not restore native config in the seconds before its own sync re-injects.
*/
adoptUpdateHandover?: boolean;
}

export function reconcileJournal(options: ReconcileJournalOptions = {}): boolean {
Expand Down Expand Up @@ -527,6 +533,15 @@ export function reconcileJournal(options: ReconcileJournalOptions = {}): boolean
return false;
}
}
if (options.adoptUpdateHandover) {
// The original baseline and injected-state hashes stay as written; only ownership moves.
// Published like every other journal write, so a replaced Codex home is refused rather
// than overwritten; a refusal preserves both the journal and the config.
const adopted = JSON.stringify({ ...journal, pid: process.pid, owner: { kind: "process", pid: process.pid } });
try { publishCodexArtifact(JOURNAL_PATH, handle, (path, hooks) => atomicWriteFile(path, adopted, undefined, hooks)); }
catch { console.error("⚠️ The Codex journal handed over by an update could not be adopted; the journal and configuration were preserved."); }
return false;
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
const restored = restoreJournalState({ heldConfigWriteLock: handle });
if (restored.unverified) {
console.error("⚠️ Codex journal recovery was not verified; current configuration files and the journal were preserved.");
Expand Down
23 changes: 22 additions & 1 deletion src/config/pending-teardown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,15 @@ export type PendingTeardownReceipt = {
* guessed receipt therefore fails closed into manual recovery.
*/
endpointSource: "exact" | "guessed";
/**
* Set when the stop handed its teardown to a replacement proxy instead of performing it
* (#7423). An update that is about to restart the proxy keeps Codex routing and the
* routed catalog in place across the gap, so a Codex process that loads them while the
* package is replaced still sees the routed rows. The obligation is the same one: the
* replacement discharges it after its first successful publication, and if no
* replacement ever comes up, the next `ocx stop` finishes it like any abandoned receipt.
*/
handover?: "update-restart";
};

/**
Expand Down Expand Up @@ -111,6 +120,7 @@ function isReceipt(value: unknown, nonce: string): value is PendingTeardownRecei
&& receipt.nonce === nonce
&& typeof receipt.createdAt === "string"
&& (receipt.endpointSource === "exact" || receipt.endpointSource === "guessed")
&& (receipt.handover === undefined || receipt.handover === "update-restart")
&& endpointOk;
}

Expand All @@ -119,11 +129,15 @@ export function claimPendingTeardown(
endpoint: { hostname: string; port: number },
endpointSource: "exact" | "guessed",
ownerPid: number = process.pid,
handover?: "update-restart",
): PendingTeardownReceipt {
const dir = getConfigDir();
assertNotRealHomeUnderTest(dir);
const nonce = randomBytes(16).toString("hex");
const receipt: PendingTeardownReceipt = { ownerPid, nonce, createdAt: new Date().toISOString(), endpoint, endpointSource };
const receipt: PendingTeardownReceipt = {
ownerPid, nonce, createdAt: new Date().toISOString(), endpoint, endpointSource,
...(handover ? { handover } : {}),
};
atomicWriteFile(pendingTeardownPathFor(nonce), JSON.stringify(receipt, null, 2) + "\n");
return receipt;
}
Expand Down Expand Up @@ -323,3 +337,10 @@ export function deferralMatchesReceipt(nonce: string | null): boolean {
if (!nonce || !NONCE_RE.test(nonce)) return false;
return readPendingTeardown(nonce).state === "valid";
}

/** Does this request name a valid obligation that an update handed to its replacement (#7423)? */
export function isUpdateHandoverReceipt(nonce: string | null): boolean {
if (!nonce || !NONCE_RE.test(nonce)) return false;
const read = readPendingTeardown(nonce);
return read.state === "valid" && read.receipt.handover === "update-restart";
}
20 changes: 20 additions & 0 deletions src/server/stop-teardown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ export function stopCapabilities() {
export type StopTeardownIo = {
/** Does the nonce this request carries name a readable obligation on disk? */
ownsReceipt?: (nonce: string | null) => boolean;
/** Does that obligation hand the teardown to the proxy an update restarts (#7423)? */
isHandoverReceipt?: (nonce: string | null) => boolean;
/** Keep Codex, Grok and env fences in place when this process exits. */
keepClientConfigOnExit?: () => void;
restoreNativeCodex?: () => Promise<CodexNativeRestoreResult>;
stripGrok?: () => GrokStripResult;
};
Expand Down Expand Up @@ -96,6 +100,22 @@ export async function performStopTeardown(url: URL, io: StopTeardownIo = {}): Pr
}
const ownsReceipt = io.ownsReceipt ?? deferralMatchesReceipt;
if (deferralHonored(url, ownsReceipt)) {
// A handover outlives this process: the exit cleanup would otherwise restore native
// Codex and put the pristine catalog back right after the stop kept them (#7423).
// Exiting as a recycle leaves them for the replacement, as a dashboard restart does.
const nonce = url.searchParams.get("teardownNonce");
const handover = io.isHandoverReceipt
? io.isHandoverReceipt(nonce)
: (await import("../config/pending-teardown")).isUpdateHandoverReceipt(nonce);
if (handover) {
if (io.keepClientConfigOnExit) io.keepClientConfigOnExit();
else (await import("./lifecycle")).markRecyclingForExit();
return {
success: true,
message: "Proxy stopping; Codex routing and catalog kept for the updated proxy.",
sharedTeardown: "deferred",
};
}
// Not "native Codex restored": nothing was restored here, and claiming otherwise
// would be a success message the operator cannot verify.
return {
Expand Down
29 changes: 29 additions & 0 deletions src/update/bind-probe.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import { spawnSync } from "node:child_process";

/**
* Strict bind script: exit 0 only after listen+close. Any listen error (including
* Windows ghost-TCB failures under Bun) is busy — matches published `ocx start`
* probes that treat every listen error as unavailable.
*/
export function strictBindProbeScript(port: number, hostname: string): string {
return [
"const net=require('net');",
"const s=net.createServer();",
"s.once('error',()=>process.exit(2));",
`s.listen(${Math.trunc(port)},${JSON.stringify(hostname)},()=>s.close(()=>process.exit(0)));`,
"setTimeout(()=>process.exit(3),2500);",
].join("");
}

export function spawnBindProbe(bin: string, script: string): boolean {
try {
const r = spawnSync(bin, ["-e", script], {
windowsHide: true,
timeout: 4000,
stdio: "ignore",
});
return r.status === 0;
} catch {
return false;
}
}
Loading
Loading