Repository navigation
Conversation
The runner takes the signal that stops it and returns once the run has stopped, instead of handling SIGINT and SIGTERM and exiting itself. The evaluate command owns the signals and the exit code, so another caller can run an evaluation in the same process and go on afterwards. A signal that comes before a run writes its folder now ends the command without the run.
In a terminal, evorca evaluate shows the run in the view of evorca watch, with the last lines that it printed under the feed. When the run ends or stops, the view closes and the command prints every line, so the scrollback holds the same text as before. --plain, or output without a terminal, prints the lines as the run goes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A 5-run
evorca evaluateseries takes hours, and you want to watch it in evorca's own terminal UI instead of a stream of lines. This PR builds on #27: in a terminal,evaluatenow opens theevorca watchview on the run it starts, and the lines it prints show under the feed. When the run ends or stops, the view closes and the scrollback gets the same text as before, score tables included.What changed
evorca evaluateshows the run full screen. It follows each run of a--runsseries as the runner starts it, and the runner's last four lines show under the feed.--plainkeeps the old line output.evaluate-view.tsxcapturesconsole.log,info,warn, anderror. When the run settles, the view closes, the terminal is restored, and every captured line goes to stdout or stderr in order.AbortSignaland anonRuncallback. It no longer handles SIGINT or SIGTERM and no longer callsprocess.exit. Theevaluatecommand now owns the signals: it prints the same "Stopping." line, aborts the run, and exits with 130 or 143 as before.kill -INTtake the same path. When a SIGINT or SIGTERM comes and nothing else handles it, as with the second ctrl+c, the view closes and prints the lines first. Then the signal ends the process.Watchtakes aloggetter and an optionalonExit. WithoutonExit, the hints showctrl+c stopinstead ofq quit.Scope
evaluatebehavior beyond the view. Plain output is the same as before, line for line.randomagent has no trajectory, so its feed stays empty. Areactrun fills it as in feat(cli): watch a run full screen with evorca watch #27./benchmarkcommand. This PR does not add that command.Blast Radius
evaluaterun goes through the new signal handling inmain.ts. A signal that comes before a run writes its folder, for example during the image build or between two runs of a series, now ends the command with 130 or 143 and writes nothing. Before, it killed the process at once. An error that the stop causes, such as a scorer that the terminal's SIGINT killed, still exits 130 as before. A second ctrl+c still stops at once.run.jsonis rewritten after each task, so a failing view loses no run data. If the view throws while it draws, the error shows in its place, and ctrl+c still stops the run because the view reads that key outside its components. While the view is open, OpenTUI catches uncaught errors. On one, the view closes and prints the lines and the error, and the run goes on with plain lines.&from a terminal still sees a terminal, so it opens the view. Setting up the terminal from a background job gets it suspended, as with any full-screen program. I did not try this. The README says to use--plainfor a run in the background.Verification
pnpm checkpasses. New tests cover the console capture, a reader that waits for the run the evaluation names (even when a newer run exists), and the view's log and stop hint. Each of 5 mutations to those parts fails a test.randomseries onmainand on this branch, with no terminal andEVORCA_SEED=7, gives the same stdout and stderr once run IDs, task seconds, the commit, and wall times are masked.randomagent:Stopped. Finish the 2 runs with: pnpm evorca evaluate --benchmark=openrca --resume=…. Undernode, the exit code was 130, as onmain.main. No containers were left.kill -TERM20 ms apart: the view closed, the lines reached the scrollback, and the process ended by SIGTERM.--plainwith one ctrl+c printed the same lines asmainand exited 130.The view during a run, after 5 of 51 tasks:
The scrollback after that ctrl+c: