Repository navigation
fix(computer): a frame an agent gets carries no capture time, which was the driver's real time - #703
Open
earakely-scale wants to merge 3 commits into
Open
earakely-scale wants to merge 3 commits into
earakely-scale wants to merge 3 commits into
Conversation
…as the driver's real time The driver keeps real time, so every screenshot result's captured_at gave an agent on a computer following the universe clock the real date and a UTC clock beside the universe's (2026-11-15T23:18Z, in a Saturday 6 February 2027 run). No agent needs it and nothing read it: FrameData drops it, and Frame, which only copied it to the result, too. Recordings keep stamping their own frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ch recordings line up with and agents aren't shown The recording test lines each screenshot up with the video by when the driver captured it, so that time stays, as `agentenv.io/capturedAt` in the tool answer's _meta (MCP clients don't show _meta to their model), not in the structured content an agent reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a, where a frame's capture time is Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
A frame an agent gets, from
computer_screenshotor any tool that captures one, no longer carriescaptured_atin what the agent reads. The capture time moves to the tool answer's_meta(agentenv.io/capturedAt), which MCP clients don't show their model.The driver deliberately keeps real time (its waits and recordings do), so
captured_atwas the real UTC time. On a computer that follows the universe clock, every screenshot result told the agent the real date and a UTC clock that disagreed with the desktop's own. In the FD-5483 pilot, a run whose universe was Saturday 6 February 2027 in Denver got"captured_at": "2026-11-15T23:18:43Z"on each frame.No agent needs the field. The recording test does: it lines each screenshot up with the video by the driver's capture time. So:
FrameData, the structured content, drops it;send()puts the frame's capture time in the answer's_meta, for screenshots and for inputs and waits that capture one;drive_deskverifier reads it from there;_metatoo.Recordings are unchanged: they stamp their own frames (
RecordedFrame).cua-driver's forwarded tools answer with its own daemon's results, which this doesn't touch.Testing
test_computer_frames.py: a screenshot's structured content has nocaptured_at, and its_metacarries the capture time, a crop's being its frame's.test_computer_spaces.pychecks the same through standalone FastMCP; it fails without the adapter fix.tst/unit/driversand the recorder test: 1776 passed.ruff checkis clean.desktop_example_test.py's scripted drive under a recording passed (61 s). It lines each screenshot up with the video's frames by the capture time from_meta, within one frame interval.🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge.Summary
Moves screenshot capture times out of agent-visible content and into
_meta.Reviews (5) · Last reviewed commit: "fix(protocol): a driver on standalone Fa..." · Reviewed by Greptile