Repository navigation
feat(remote): surface a script delivery failure on every caller surface - #4582
Conversation
Size Report
Startup median (7 runs, lower is better):
|
d03f541 to
4556e33
Compare
|
Thanks for this. I reviewed 4556e33 and found two things to fix before merge. CI is green (21 checks, none failing), and there are no conflicts. A script delivery failure that the daemon reports on a batch step never reaches the batch result.
Not blocking, and you can take or leave these: Could the batch side be much smaller? Keep the five SDK spreads and the serializer helper. Give The CodeQL thread on the polynomial regex in The first finding comes from reading the path from |
a385901 to
f09f36f
Compare
|
Yes — it could, and it now is. Line counts
The batch side is 286 lines in two places instead of 333 in one. The second file exists so those readers stay off the client's eager import path: Finding 1 — a step's daemon-side failure now reaches the batch payload The disclosure takes this client's own failure first, else the last finished step's failure whose Finding 2 — one normalizer Imported from Cross-check items
Non-blocking notes, taken
|
|
The PR is ready at f09f36f: the earlier findings from the review of 4556e33 (#4582 (comment)) are fixed and I found no new problems. Not blocking, take or leave: readBatchDisclosure picks warnings by matching their text against the failure message and hint, and the merge in daemon-artifacts.ts replaces the batch payload's own The [wait, close] failed-final-write path is traced statically; the unit case at daemon-artifacts-batch-script.test.ts:340 covers it with a hand-built step answer. The code-scanning thread on the polynomial regex no longer applies, since CI is green with 24 checks and none failing, and there are no conflicts. The PR is still a draft, so it needs to be marked ready for review before merge. |
|
All four are in at d14e68a. The root cause was the same in the three blockers: retirement and disclosure read the plan or the counts beside the answer's proof instead of using it.
On the non-blocking note: agreed, and it went further than the dropped warning. Hardening: the serializer table now calls each serializer with plain input and asserts both fields are absent, so a serializer that invents a disclosure fails; and a Production code is +47 lines over f09f36f: |
An action that records a remote script version keeps its success when that version cannot reach the caller file, so the caller needs one field that says the file is stale. `readScriptDeliveryFailure` reads it from a response, and every result that can carry a script version now spreads it beside the warning: the SDK open, close, snapshot and screenshot results, the CLI human output and `--json`, and the MCP tool text and structured content. A batch delivers the script versions its own steps record. Each step is published through the session-script owner under its own command name, in step order, so a step that closes the recording retires the caller record the way a standalone close does, and a step that only records a version leaves the record open for the next one. The destination still comes only from the record this client's own `open --save-script` wrote, so a response never chooses a path, and a step script of another publication still writes nothing. That retires the `remote_script_batch_close_unsupported` refusal, which existed only because a batch delivered nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tted A batch answer carries one entry per step, and the client delivered whatever the answer named. That made the response the authority over a caller-owned record: an entry planted in place of a `wait` step could name this client's publication and retire it, and its artifact list was unbounded, so one answer asking for a thousand duplicate versions spent a thousand downloads. Delivery now runs on the plan THIS client submitted: an entry must be the plan's own command at the plan's own position and a step that finished, and anything else is reported exactly as the daemon answered it, with no download, no destination and no retirement. Of the versions one payload names, only the newest is taken, and a duplicate or a version no newer than one already accepted is dropped before any request, which caps one answer at one download per submitted step. A `close` step that names this client's publication now reaches the owner even when it carries no artifact, so a recording the daemon could not finish is still forgotten instead of staying active beside a file that never gained its final version. A failed batch reports the steps it finished in the error's `partialResults`, which never reached the code that materializes a successful response. Those steps are delivered before the error is rehydrated, under the same plan bound, so `[wait, close, snapshot]` that fails on `snapshot` still writes the final script and still ends the recording; the batch error itself stands. A batch's own output reads the batch payload, so a step's warning and typed `scriptDeliveryFailure` are lifted onto it: the CLI `Warning:` line, the MCP tool text and `--json` and structured content now say the caller file is stale, instead of printing only "OK Closed: journey". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A batch answer is untrusted input, and the per-step delivery path read it step by step. A failing step's report could carry successful entries for steps that never ran, so a forged partial result retired a close the caller never ran. A malformed publication ID threw while it was interpolated, which turned a successful batch into a client failure. Filtering revisions re-scanned every earlier version, so a large answer blocked the caller. And a successful batch whose steps carried no script stopped before its own payload, leaving the caller's file stale behind a newer version the daemon had stamped. Deliver at most ONE version per batch. Every version is the whole recording, so only the newest of the open publication matters: one linear, bounded scan reads the plan's finished steps and the batch payload and keeps that one, and one delivery goes through the session-script owner. Derive the completed prefix from the submitted plan and cross-check it against the step and executed count the error names, so entries at or after the failure authorize no download and no retirement. Retire only when that prefix contains the close step, with or without an artifact. Validate each candidate with the publisher's own guards, so no response value reaches a record lookup, a path, or a download request. Reports #4564 review findings 1 to 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ints ADR 0016 said every recorded action costs one download without naming the batch case, and the ledger rationale still described the per-step path this module no longer takes. Reports #4564 review finding 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ler path The batch script owner is replaced by one delivery path: read the plan the caller submitted, take its finished prefix, pick the newest publishable version the answer names, and run the single-response owner on that one artifact. `sessionScriptDelivery` takes an explicit `closedPublicationId` instead of a faked `command: 'close'`, and `isPublishableScriptVersion` is one predicate used by the publisher and the batch scan. Step names resolve through `normalizeBatchCommandName`, so the plan check and the daemon's own step naming cannot drift apart. A finished step's `scriptDeliveryFailure` now reaches the batch payload: this client's own failure wins, otherwise the last finished step's failure for that publication, unless the saved version came from a later step. Only the warning written beside that failure rides up. A step entry outside the finished prefix, or one whose report does not describe the submitted plan, authorizes nothing. The failed-batch catch emits the `session_script_delivery_failed` diagnostic, and the batch path rewrites the answer's own artifact list the way a single response does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d prefix A batch answer proves which of the caller's steps ran. Retirement, disclosure and the newest version each read the plan or the counts beside that proof, so three findings followed: a failed batch that carried only step failures lost its disclosure, a foreign revision masked the version this client was owed, and a finished count that passed the plan's `close` entry retired a recording whose close nobody ran. `readBatchAnswer` now returns ONE validated prefix — the steps the answer proves ran, of the recording this client armed — and the publication, the newest version, the close decision and the disclosure all come from it. Candidates for the newest version must name a recording this client armed, so another client's revision cannot outrank the caller's own file. Retirement needs the plan's `close` position to be a validated step the answer reports as succeeded. Disclosure runs whatever a download did, and answers with the payload's own warnings plus this failure's line, so a daemon-side write failure and a client-side download failure are both reported and no warning text is matched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h closes Batch step delivery moves to #4595, where it waits on a daemon-side delivery receipt. This carries the caller-facing half only: every result that recorded a remote script version discloses a version that did not arrive, so no caller reads a stale file as current. `readScriptDeliveryFailure` reads the response field the daemon already answers, and `readScriptDeliveryDisclosure` pairs it with its warning. The SDK `open`, close, snapshot and screenshot results, the CLI human output and `--json`, and the MCP tool result each carry both, and the SDK close keeps the response's warnings the CLI prints. `remote_script_batch_close_unsupported` stands as it was: a batch delivers no step versions, so closing an armed remote recording inside one stays refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d14e68a to
0ebfbfb
Compare
|
Split per your call: this PR now carries the caller-facing half only. Batch step delivery moves to #4595, which is the shape that needs a daemon-side receipt. Removed: Kept: Docs: ADR 0016 keeps the batch limitation, now pointing at #4595; remote-proxy.md and replay-e2e.md describe the result fields and the refusal. The wire ledger is main's again, plus one additive digest for the new Five findings from the d14e68a cross-check are not fixed here — they belong to the batch reader, so they move with it to #4595. Net against main: 9 production files, +131/-7. Still a draft. |
Apex Sentinel · ✅ ApproveSince the last review: 0 new · 0 still open · 0 no longer raised No open findings. Reviewed with this repository's own Sentinel configuration ( 💬 Talking to SentinelIn a finding's thread, reply with:
On this conversation, comment Earlier reviewsReview of
|
…file Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
4 issues found across 18 files
You’re at about 96% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/cli/commands/__tests__/screenshot.test.ts">
<violation number="1" location="src/cli/commands/__tests__/screenshot.test.ts:95">
P3: The fixture's `warnings` should be the wire string the daemon writes, `${failure.message} ${failure.hint}`, not `failure.message` alone. Otherwise the assertion only checks the fixture against itself. Update both the fixture and the `warnings` assertion.</violation>
</file>
<file name="test/integration/provider-scenarios/remote-script-publication.test.ts">
<violation number="1" location="test/integration/provider-scenarios/remote-script-publication.test.ts:616">
P3: `createArtifactGate` repeats the inline artifact-refusing gate already written in the refresh test. Have that test call `createArtifactGate(proxy.baseUrl, () => refuseArtifacts)` so the forwarding logic lives in one place.</violation>
</file>
<file name="packages/client/src/index.ts">
<violation number="1" location="packages/client/src/index.ts:609">
P3: The leveled snapshot branch returns the raw payload, so a malformed `scriptDeliveryFailure` reaches the caller unchecked. Pass that field through `readScriptDeliveryFailure` on the leveled path too, as `presentedSurface` already is.</violation>
<violation number="2" location="packages/client/src/index.ts:621">
P3: Add `'scriptDeliveryFailure'` to the `Pick<CaptureSnapshotResult, ...>` union in `optionalSnapshotResponseFields`. The union is the documented list of fields this helper can return, and the new field is missing from it.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| path: '/tmp/shot.png', | ||
| width: 40, | ||
| height: 20, | ||
| warnings: [failure.message], |
There was a problem hiding this comment.
P3: The fixture's warnings should be the wire string the daemon writes, ${failure.message} ${failure.hint}, not failure.message alone. Otherwise the assertion only checks the fixture against itself. Update both the fixture and the warnings assertion.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/cli/commands/__tests__/screenshot.test.ts, line 95:
<comment>The fixture's `warnings` should be the wire string the daemon writes, `${failure.message} ${failure.hint}`, not `failure.message` alone. Otherwise the assertion only checks the fixture against itself. Update both the fixture and the `warnings` assertion.</comment>
<file context>
@@ -77,6 +77,37 @@ test('screenshot --json carries response-level warnings in the data', async () =
+ path: '/tmp/shot.png',
+ width: 40,
+ height: 20,
+ warnings: [failure.message],
+ scriptDeliveryFailure: failure,
+ });
</file context>
| * A gate in front of the daemon proxy that answers the artifact downloads of a refused moment with | ||
| * 503, which is how these scenarios make one delivery fail. | ||
| */ | ||
| async function createArtifactGate( |
There was a problem hiding this comment.
P3: createArtifactGate repeats the inline artifact-refusing gate already written in the refresh test. Have that test call createArtifactGate(proxy.baseUrl, () => refuseArtifacts) so the forwarding logic lives in one place.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At test/integration/provider-scenarios/remote-script-publication.test.ts, line 616:
<comment>`createArtifactGate` repeats the inline artifact-refusing gate already written in the refresh test. Have that test call `createArtifactGate(proxy.baseUrl, () => refuseArtifacts)` so the forwarding logic lives in one place.</comment>
<file context>
@@ -438,3 +450,197 @@ test('close delivers the final script after it stops an app log', async () => {
+ * A gate in front of the daemon proxy that answers the artifact downloads of a refused moment with
+ * 503, which is how these scenarios make one delivery fail.
+ */
+async function createArtifactGate(
+ upstreamUrl: string,
+ refused: () => boolean,
</file context>
| ...(snapshotDiagnostics ? { snapshotDiagnostics } : {}), | ||
| // `readSerializedSnapshotCaptureAnnotations` above already reads this response's `warnings`, so | ||
| // only the typed field needs adding here. | ||
| ...(scriptDeliveryFailure === undefined ? {} : { scriptDeliveryFailure }), |
There was a problem hiding this comment.
P3: Add 'scriptDeliveryFailure' to the Pick<CaptureSnapshotResult, ...> union in optionalSnapshotResponseFields. The union is the documented list of fields this helper can return, and the new field is missing from it.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/client/src/index.ts, line 621:
<comment>Add `'scriptDeliveryFailure'` to the `Pick<CaptureSnapshotResult, ...>` union in `optionalSnapshotResponseFields`. The union is the documented list of fields this helper can return, and the new field is missing from it.</comment>
<file context>
@@ -614,6 +616,9 @@ function optionalSnapshotResponseFields(
...(snapshotDiagnostics ? { snapshotDiagnostics } : {}),
+ // `readSerializedSnapshotCaptureAnnotations` above already reads this response's `warnings`, so
+ // only the typed field needs adding here.
+ ...(scriptDeliveryFailure === undefined ? {} : { scriptDeliveryFailure }),
...(typeof data.fallbackScreenshotPath === 'string'
? { fallbackScreenshotPath: data.fallbackScreenshotPath }
</file context>
| const snapshotDiagnostics = readSnapshotDiagnosticsSummary(data.snapshotDiagnostics); | ||
| const presentedSurface = readPresentedSurfaceFact(data); | ||
| const postGestureOutcome = readPostGestureOutcome(data.postGestureOutcome); | ||
| const { scriptDeliveryFailure } = readScriptDeliveryDisclosure(data); |
There was a problem hiding this comment.
P3: The leveled snapshot branch returns the raw payload, so a malformed scriptDeliveryFailure reaches the caller unchecked. Pass that field through readScriptDeliveryFailure on the leveled path too, as presentedSurface already is.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/client/src/index.ts, line 609:
<comment>The leveled snapshot branch returns the raw payload, so a malformed `scriptDeliveryFailure` reaches the caller unchecked. Pass that field through `readScriptDeliveryFailure` on the leveled path too, as `presentedSurface` already is.</comment>
<file context>
@@ -605,6 +606,7 @@ function optionalSnapshotResponseFields(
const snapshotDiagnostics = readSnapshotDiagnosticsSummary(data.snapshotDiagnostics);
const presentedSurface = readPresentedSurfaceFact(data);
const postGestureOutcome = readPostGestureOutcome(data.postGestureOutcome);
+ const { scriptDeliveryFailure } = readScriptDeliveryDisclosure(data);
return {
...(keyboard ? { keyboard } : {}),
</file context>
|
The earlier note on f09f36f found no problems. The code-scanning thread is now fixed, and one docs problem remains at 8f0ecc6. The new sentence in remote-proxy.md says every result that records a version prints a The code-scanning thread no longer applies, because this PR does not change There are no conflicts. Integration, unit and wire-compat checks pass on this head. The smoke tests and the review-bot checks are still running, and none has failed. Before merge, the docs sentence needs the fix above and those checks need to finish green. |
…urface Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
5de4c76: remote-proxy.md now says the CLI prints a warning, without promising a |
|
The earlier finding from 8f0ecc6 (#4582 (comment)) is fixed, and there are no new problems in 5de4c76. The only change is one docs line in Not blocking: four lower-priority review threads are still open and this docs change does not touch them: #4582 (comment), #4582 (comment), #4582 (comment) and #4582 (comment). |
Part of #4564; batch step delivery moves to #4595.
An action that records a remote script version keeps its success when that version cannot reach the caller file. The wire already answered
scriptDeliveryFailure; nothing above it passed the field on, so a caller read a stale file as current.readScriptDeliveryFailurein the kernel reads the field from a response, andreadScriptDeliveryDisclosurepairs it with the warning written beside it. Every result that can carry a script version now spreads both: the SDKopen,sessions.close,apps.close,snapshotandscreenshotresults, the CLI human output and--jsonpayload, and the MCP tool text and structured content. The SDK close also keeps the response'swarnings, which the CLI human output prints.remote_script_batch_close_unsupportedstands unchanged: a batch delivers no script versions from its steps, so closing an armed remote recording inside one stays refused until #4595 gives the daemon one receipt to deliver from.Tests: a table covers each serializer, including that plain input yields neither field, and the provider scenario discloses a failed final download on the SDK result, the CLI
--jsonand MCP. Production code is 9 files, +131/-7.🤖 Generated with Claude Code