Repository navigation
perf: stage replayed files in a single git.add call - #463
Open
SammyTourani wants to merge 1 commit into
Open
SammyTourani wants to merge 1 commit into
SammyTourani wants to merge 1 commit into
Conversation
GitCloneService.buildRepository replays every agent commit on top of the template base and staged the working tree with one git.add call per file. Each call re-reads and re-parses the index, reloads the config, and re-serializes and hashes the whole index, so every replayed commit cost O(files^2) index work. This replay runs inside the agent Durable Object during GitHub export and in the Worker during git clone. Pass all paths to a single git.add call per replayed commit. The staged set, ignore handling and resulting commits are unchanged (same commit OIDs); only the per-file index round-trips are removed. Add a focused test for the replay.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #312
In the replay loop of
GitCloneService.buildRepository(worker/agents/git/git-clone-service.ts), all working-tree files are now staged with onegit.add({ fs, dir: '/', filepath: allFiles })call per replayed commit, instead of one call per file.git.addaccepts an array and processes it under a single index acquisition, so the index is read and written once per commit instead of once per file.The output does not change:
.gitignorerules see the same working tree whatever the add order.I also added
worker/agents/git/git-clone-service.test.ts, a focused test for the replay.buildRepositoryhad no test coverage. The test checks:This roughly halves the CPU time of the step that exceeds the Durable Object limit. It does not remove the limit for very large histories.
Refs #312, #226
Verification
Command:
bunx vitest run --no-file-parallelismResult:
New test before and after. The new test passes on both the original and the patched code. It fails when staging is broken: I tried
filepath: []and staging only the agent's files, and both fail with the expected file-list mismatch.Same output. I ran the original and patched
buildRepositoryside by side on synthetic agent histories covering:.gitignore.gitignoreedits partway through the history (both unignoring and newly ignoring files)Both versions produced identical commit and tree OIDs for every commit in every scenario.
CPU time. I measured
buildRepositoryprocess CPU time under Node 26 (V8). The synthetic app had 80 template files and 40 agent files, with 1 to 5 files changed per commit. Values are the median of 3 runs, and HEAD OIDs were identical in all runs:Type check and lint.
bun run typecheck(tsc -b) passes with no errors. The worker config excludes*.test.ts, so I also type-checked the new test under the same compiler settings: no errors in it.eslintis clean on the changed source file.