Repository navigation
PERF: Prepare scenarios on the backend event loop - #3046
Richard Lundeen (richlundeen) wants to merge 2 commits into
Conversation
Retain serialized preparation and abandoned admission cleanup, offload blocking construction, and preserve runtime shutdown ownership. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8fabc130-e568-4583-9ff0-ec26491a6414
Keep upstream statistics and async-generator typing while removing the preparation executor. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8fabc130-e568-4583-9ff0-ec26491a6414
Richard Lundeen (richlundeen)
changed the title
| raise RuntimeError("Scenario run scheduling is stopping.") | ||
| resumed_from_cancelled = await self._is_run_cancelled_async(scenario_result_id=request.scenario_result_id) | ||
| if request.scenario_result_id: | ||
| self._preparing_run_ids.add(request.scenario_result_id) |
There was a problem hiding this comment.
🔴 Must Fix: scenario initialization still contains blocking construction. Moving the whole _prepare_run_async coroutine onto the backend loop also moves synchronous work inside initialize_async there. For example, RapidResponse._build_atomic_attacks_async calls MatrixAtomicAttackBuilder.build directly, and the many_shot factory reads the examples JSON and prompt YAML once per dataset. Generic Crescendo, TAP, and Skeleton Key construction also read templates synchronously. Offloading the scenario constructor and the Psychosocial Crescendo branch does not cover these paths.
I reproduced this with a real RapidResponse launch, real in-memory SQLite, seven local datasets, and an offline target. With 50 ms of simulated disk latency per many-shot JSON read, a 5 ms heartbeat stopped for 405 ms during preparation. The same launch with preparation on a worker had a 25 ms maximum gap. Even without injected latency, the current path had a 75 ms gap.
Please keep async initialization/database operations on the backend loop, but offload the synchronous attack-building portions too. A heartbeat regression test using a real matrix scenario would catch this; the new mocked-constructor tests do not exercise the work that now blocks health checks, progress polling, and cancellation.
This branch has not been deployed
2 participants
Prepare and execute scenarios on one backend event loop so async memory resources stay valid and the backend remains responsive.
We removed the preparation executor, temporary event loop, and per-launch memory disposal while preserving serialized preparation and FIFO execution. We moved blocking construction off-loop and retained cleanup for requests cancelled before scheduler admission, without changing HTTP disconnect shielding or runs already owned by the scheduler. We added regression coverage for cancellation during admission, construction responsiveness, shared SQLite loop resources, and shutdown ordering.