Skip to content

feat: add AG2 multi-agent framework as native agent client#219

Open
faridun-ag2 wants to merge 1 commit intoTHUDM:mainfrom
faridun-ag2:feat/add-ag2-agent-client
Open

feat: add AG2 multi-agent framework as native agent client#219
faridun-ag2 wants to merge 1 commit intoTHUDM:mainfrom
faridun-ag2:feat/add-ag2-agent-client

Conversation

@faridun-ag2
Copy link
Copy Markdown

Summary

Adds AG2 (formerly AutoGen) as a native agent client for AgentBench evaluation. AG2 is an open-source multi-agent framework with 500K+ monthly PyPI downloads, 4,300+ GitHub stars, and 400+ contributors.

  • AG2Agent: Single AssistantAgent + UserProxyAgent pair (comparable to existing HTTPAgent)
  • AG2GroupChatAgent: Multi-agent GroupChat with Planner + Executor (demonstrates AG2's multi-agent advantage)

Dependency strategy

AG2 v0.11.4+ requires Python >=3.10 and pydantic >=2.6.1, which conflicts with AgentBench's core dependencies (Python 3.9, pydantic ~=1.10). Rather than modifying existing requirements, a separate requirements-ag2.txt is provided for an isolated venv:

python3.10 -m venv .venv-ag2
source .venv-ag2/bin/activate
pip install -r requirements-ag2.txt

The AG2 import in __init__.py is wrapped in try/except ImportError so the base AgentBench installation is unaffected.

New files

  • src/client/agents/ag2_agent.py — AG2Agent and AG2GroupChatAgent implementations
  • src/client/agents/test_ag2_agent.py — 23 tests (unit, integration, e2e with fake task server)
  • configs/agents/ag2-single.yaml — Config for single-agent evaluation
  • configs/agents/ag2-groupchat.yaml — Config for multi-agent evaluation
  • configs/agents/ag2-gpt4.yaml — Config for GPT-4 leaderboard runs
  • configs/agents/ag2_agents.yaml — Aggregated config for agent_test.py
  • requirements-ag2.txt — Isolated dependencies for AG2 environment

Modified files

  • src/client/agents/__init__.py — Optional AG2 agent registration

Test plan

  • 23 pytest tests pass (pytest src/client/agents/test_ag2_agent.py -v)
  • Interactive REPL test via python -m src.client.agent_test --config configs/agents/ag2_agents.yaml --agent ag2-single
  • AG2Agent returns correct responses from OpenAI API
  • No impact on existing agents when AG2 is not installed

AG2 (formerly AutoGen) is an open-source multi-agent framework with
500K+ monthly PyPI downloads. This adds two AgentClient implementations:

- AG2Agent: single AssistantAgent + UserProxyAgent pair
- AG2GroupChatAgent: multi-agent GroupChat with Planner + Executor

AG2 requires Python >=3.10 and pydantic >=2.6.1, which conflicts with
AgentBench's core deps (Python 3.9, pydantic v1). A separate
requirements-ag2.txt is provided for an isolated venv setup.

Includes 23 tests (unit, integration, e2e with fake task server).
@faridun-ag2
Copy link
Copy Markdown
Author

Hi @Xiao9905 @Longin-Yu!

Would love your review on this PR. It adds AG2 (formerly AutoGen) as a native agent client — the first multi-agent framework integration in AgentBench.

Key design decisions:

  • Separate requirements-ag2.txt instead of modifying the main requirements.txt, since AG2 needs Python >=3.10 and pydantic v2 (incompatible with AgentBench's current deps)
  • try/except ImportError in __init__.py so existing installations are unaffected
  • 23 tests covering unit, integration, and e2e (with a fake task server)

Happy to adjust anything based on your feedback. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant