Skip to content

feat(cli): watch a run full screen with evorca watch - #27

Open
kottinov wants to merge 3 commits into
mainfrom
feat/cli-watch
Open

kottinov wants to merge 3 commits into
mainfrom
feat/cli-watch

Conversation

@kottinov

Copy link
Copy Markdown
Collaborator

Why

A 6-hour evorca evaluate run prints one line per finished task, so you cannot see what the agent does while it works. evorca watch shows a run live in evorca's own terminal style: the run's progress, each task as a dot, and the commands, output, and reasoning of one task's agent. It reads only the run folder, so it attaches to a run that is already going and cannot disturb it.

What changed

  • evorca watch --benchmark=openrca [--scenario=...] [<run ID>] opens the view full screen. Without a run ID, it follows the newest run and moves to the next run of a --runs series when its folder appears.
  • watch.ts is a pure model: a board of tasks from run.json and the task folders, and a feed that grows one trajectory.jsonl line at a time. watch-folder.ts reads each growing file from the last byte offset and holds back a line that is still being written. watch-view.tsx draws them with the shell's grays, Mark, Hints, and Footer.
  • The run ID check and the task folder path move into runs.ts, so the reader and the runner share them.

Scope

  • Read only. No writes to the run folder, no Docker, and no signals.
  • Scores show only once scores.json exists. The view counts the tasks with readTasks, which drops the scoring points, and it never reads record.csv.
  • evorca evaluate itself is unchanged. The stacked PR opens this view from evaluate.

Blast Radius

evorca watch is new, and main.ts loads it on demand like the shell, so other commands still never load OpenTUI. The shell changes only by exporting four of its parts. Its tests pass unchanged.

Verification

  • pnpm check passes. 35 new tests drive the model, the reader on real folders that grow as the runner writes them, and the view in OpenTUI's test renderer. Each of 17 mutations to the reader, the view, and the model fails a test.
  • A replay of a finished react run, re-created as a live 2-run series at 12x with every trajectory line written in two halves, ran under pnpm evorca watch on a real pseudo-terminal. The feed grew, tasks finished on the board, the view moved to run 2, and it showed the scores once the run was scored. Switching tasks, folding, and pausing the tail all worked.
  • q and ctrl+c exit with code 0. After exit the terminal is back in canonical mode with echo, the alternate screen is closed, and the cursor shows. A task folder with no out/ and a trajectory that ends mid-line both render.

A frame from the replay, 7 seconds in. The task's elapsed time and the feed's clock times come from the recorded run, so they read hours off:

  evorca   telecom   20261118T140000Z-react   claude-sonnet-5 high
  0/4 tasks   $0.0479 of $50.00   7s elapsed

  • • • •

  task 000   task_2   running   5h 06m   $0.0205                                                        live   tailing

                    print(df)
                    "
                        cmdb_id          name         p95  ...        p01          mx         mn
                    0    os_001  CPU_util_pct    2.420779  ...   1.673454    4.849936   1.636268
                    1    os_002  CPU_util_pct   33.567388  ...  28.836894   34.448827  28.555985
                    2    os_003  CPU_util_pct    9.052831  ...   8.291235   11.481513   7.995870
                    3    os_004  CPU_util_pct    1.326398  ...   1.057648    3.627625   1.036659
                    4    os_005  CPU_util_pct    5.117221  ...   4.516727    7.836819   4.462440
                    … 8 more lines, o to show all

  07:20:42  turn    4   3.38 s   $0.0064
  07:20:42  think   I'm going to compare the time window data against the thresholds for each node to spot any
                    anomalies.
  07:20:42  $       cd /tmp && python3 -c "
                    import duckdb, pandas as pd
                    import pytz, datetime

                    # time range 2020-04-11 00:00 - 00:30 UTC+8 -> convert to ms epoch
                    tz = pytz.timezone('Asia/Shanghai')
                    start = tz.localize(datetime.datetime(2020,4,11,0,0,0))
                    end = tz.localize(datetime.datetime(2020,4,11,0,30,0))
                    start_ms = int(start.timestamp()*1000)
                    end_ms = int(end.timestamp()*1000)
                    print(start_ms, end_ms)
                    "
                    running…

  ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  ↑↓ scroll   end tail   o output   ←→ task   tab running   f live   q quit

  data/runs/openrca/telecom/20261118T140000Z-react                                                       evorca v0.0.0

…uns.ts

evorca watch reads run folders too, so the pattern of a run ID and the
path of a task's folder move next to the other run.json helpers.
The watch view draws with the same parts. Hints now takes its list, and
the shell passes its own.
evorca watch shows the progress of a run, each task as a dot, and the
trajectory of one task: the instruction, each response with its time and
cost, the thinking summaries, the commands with the output the agent
sees, and the answer. It reads the run folder once a second, only reads,
and follows the newest run of a series.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant