[WIP] feat: add SubprocessDriver for CRD-based agent execution#232
Draft
rioloc wants to merge 6 commits intolightspeed-core:mainfrom
Draft
[WIP] feat: add SubprocessDriver for CRD-based agent execution#232rioloc wants to merge 6 commits intolightspeed-core:mainfrom
rioloc wants to merge 6 commits intolightspeed-core:mainfrom
Conversation
…gration Introduce a generic `agents:` configuration block in system.yaml that makes the HTTP API one agent type among potentially many (e.g., Kubernetes CRD agents for Openshift Agentic Lightspeed). The key design ensures zero breakage of existing configs and code: - New Pydantic models: AgentsConfig, AgentDefaultConfig, HttpApiAgentConfig in a new `core/models/agents.py` module - SystemConfig gains an `agents: Optional[AgentsConfig]` field alongside the existing `api:` field - A model_validator auto-migrates `api:` to `agents:` when `agents:` is absent, so all existing system.yaml files continue working unchanged - `api.enabled=True` maps to `default.agent="http_api"`; `api.enabled=False` maps to `default.agent=None` - The `api` field is preserved — all downstream code reading `config.api` continues to work without modification - EvaluationData gains `agent` and `agent_config` optional fields for per-conversation-group agent selection and config overrides - Config resolution follows a 3-level priority chain: eval_data.agent_config > agents.<name> > agents.default - ConfigLoader passes `agents` YAML data through to SystemConfig Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace direct APIDataAmender usage with abstract AgentDriver interface, enabling per-conversation agent resolution and support for multiple agent types. Pipeline owns driver lifecycle; processor receives driver per call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace APIDataAmender/APIClient mocks with AgentDriver mocks in processor and pipeline tests. Add unit tests for AgentDriverRegistry, HttpApiDriver, and AgentDriver base class. All 113 evaluation tests pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add native agents: config format YAML files alongside legacy api: configs. Rename TestFullEvaluation to TestFullApiEvaluation and extend parametrization to cover both config formats (backward compat via migrate + native agents). Add get_agent_info() helper to resolve agent state from either format. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| import logging | ||
| import os | ||
| import shutil | ||
| import subprocess |
Comment on lines
+167
to
+168
| return subprocess.run( | ||
| [self._cli, *args], |
Collaborator
Author
There was a problem hiding this comment.
This is a safe usage of subprocess.run():
- No shell invocation — arguments are passed as a list, not a string, so
shell=False(the default) is in effect. No shell expansion or injection is possible. - CLI binary resolved via
shutil.which()— the binary path comes from system PATH resolution at driver init time, not from user input. - Command arguments are hardcoded — all args (
apply,get,delete,-f,-n,-o json,--ignore-not-found) are string literals built by the driver itself. - Data passes through stdin, not arguments — CR manifests are serialized with
json.dumps()and piped via theinput=parameter. Special characters in user-provided fields (quotes, newlines,$, backticks) never reach a shell. - Same pattern as existing code —
ScriptExecutionManager(core/script/manager.py:86) usessubprocess.run()identically and has been reviewed and accepted.
rioloc
force-pushed
the
feat/subprocess-agent-driver
branch
from
16c6d86 to
321c572
Compare
Introduce SubprocessDriver — a new AgentDriver that manages OpenShift Proposal CR lifecycle via oc/kubectl CLI. The driver builds and applies Proposal CRs, polls status conditions until terminal state, handles auto-approval via ProposalApproval resources, and cleans up on completion. Key design decisions: - Works directly with CRD conditions (the stable API contract) instead of replicating the operator's internal DerivePhase() logic - TerminalOutcome enum for driver-level terminal states - TurnData extended with proposal_spec, proposal_status, description, and expected_proposal_status fields (backward-compatible, all Optional) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rioloc
force-pushed
the
feat/subprocess-agent-driver
branch
from
321c572 to
3af4e56
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
SubprocessDriver— a newAgentDriverthat manages OpenShift Proposal CR lifecycle viaoc/kubectlCLI, enabling evaluation of agentic workloads running on the cluster.The driver builds and applies Proposal CRs, polls
.status.conditionsuntil a terminal state is reached, handles auto-approval viaProposalApprovalresources, and cleans up on completion. It works directly with CRD conditions (the stable API contract) rather than replicating the operator's internal phase derivation logic.Design document: https://gist.github.com/rioloc/c4b29b534f02e46d0e023599c90ed0f0
Milestones
SubprocessDriverclass withexecute_turn, polling loop, auto-approve, cleanupTerminalOutcomeenum for driver-level terminal statesTurnDataextended withproposal_spec,proposal_status,description,expected_proposal_status_is_terminal,_should_approve)SubprocessDriverinAgentDriverRegistrySubprocessAgentConfigtoagents.py"subprocess"toSUPPORTED_AGENT_TYPES__init__.pyTest plan
uv run pytest tests/unit/pipeline/evaluation/test_subprocess_driver.py -vmake testmake pre-commit🤖 Generated with Claude Code