Repository navigation
fix(agent-server): run one git init at a time - #6880
Conversation
EventService.start ran git rev-parse and, for a new workspace, git init as subprocesses on the event loop, for every conversation start. On macOS, where starting a process is slow, that froze every request and socket for about 60 ms per create. Run the check with asyncio.to_thread. On a Mac the worst /alive latency during a warm create went from 59-71 ms to 5-18 ms; the create itself is unchanged.
Agent-server conversation start benchmarkBase
How this is measuredEach create is a Canvas-shaped
A timing is flagged when the 95% CIs do not overlap and the medians differ by at least 50 ms, or when a row with at least 8 samples per side moves by at least 150 ms and 25%. A create stalled the server when its slowest |
Coverage Report •
|
||||||||||||||||||||
Off the event loop, two conversations starting on the same new directory ran git init at once; the losers failed on .git/config.lock and logged an error, although the directory ended up a repository.
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
⏳ Review waiting on checksThe current head
No action is needed. The scheduled scan retries once every check on the head reports a conclusion. This is an automated check - no AI was used to generate this comment. |
all-hands-bot
left a comment
There was a problem hiding this comment.
This review was posted by an AI agent (OpenHands).
Reviewed against the repository guidance (root and openhands-agent-server AGENTS.md, .agents/skills/custom-codereview-guide.md). The change is in scope: agent-server conversation-start lifecycle belongs in this repository, and the repo's own async-safety guidance explicitly directs moving synchronous subprocess/lock work off the event loop in async services, which is exactly what this does.
Verified on head 451fd59:
tests/agent_server/test_event_service.py -k "repo_off_the_event_loop or concurrent_repo_checks"passes; the full event-service suite (136 tests) and the conversation-service/router suites (268 tests) pass locally.- In an isolated base-branch worktree (
vasco/lease-lock-poll), both new tests fail as claimed: the off-loop check runs on the loop thread at base, and concurrent checks on a fresh directory race on.git/config.lock. The regression coverage is real. - Startup ordering is preserved: the check is awaited at the same point in
EventService.start, so moving it to a worker thread does not reorder the subsequentbase_state/LocalConversationsteps. The synchronousgitsubprocesses (rev-parse,init) behave identically off-loop. - The module-level
_GIT_INIT_LOCKis only held aroundgit init(never nested, subprocess has its own timeout), so it cannot reintroduce loop blocking or deadlock; it correctly serializes the now-concurrent inits that moving off the loop makes possible, and repeatedgit initon an already-initialized repo is harmless.
CI on this head: agent-server-tests, cross-tests, and the compatibility/REST checks pass. The Build & Push (python-arm64) failure is an infrastructure flake (ERROR CODE: BlobNotFound fetching a GitHub Actions cache blob during buildx), not a code defect — the same SHA builds successfully on the sibling run of that workflow.
No material correctness, security, or compatibility findings.
✅ APPROVED
…loop # Conflicts: # openhands-agent-server/openhands/agent_server/event_service.py
HUMAN:
The git init check at conversation start blocked the server for about 60 ms on macOS. This moves it to a thread.
AGENT:
Why
EventService.startrunsgit rev-parseand, for a new workspace,git initas subprocesses directly on the event loop, for every conversation start (#6868). On macOS, where starting a process is slow, that freezes every request and socket for about 60 ms per create. On Linux it is a few milliseconds, so the CI benchmark barely shows it.Summary
_prepare_workspace_dirruns inasyncio.to_thread). This PR keeps its regression test for that and adds the part main doesn't have:git initruns under a module lock: off the loop, conversations starting on the same new directory would otherwise race on.git/config.lockand log errors (test_concurrent_repo_checks_on_a_new_directory_all_succeed).Issue Number
Part of #6868
How to Test
/aliveper warm create 59–71 ms, with 16 loop stalls inside_run_git_subprocess(two per create);uv run pytest tests/agent_server/test_event_service.py -k "repo_off_the_event_loop or concurrent_repo_checks": the check runs on a worker thread, and concurrent checks on a new directory all succeed without errors; the concurrency test fails without the lock. The event-service, conversation-service and parent/child suites pass locally (310 tests).Type
Notes
Stacked on #6879.
🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdkpython-node-runtimepython-node-runtimepython-node-runtimegolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:1f260ad-pythonRun
All tags pushed for this build
About Multi-Architecture Support
1f260ad-python) is a multi-arch manifest supporting both amd64 and arm641f260ad-python-amd64) are also available if needed