Skip to content

fix(deps): skill-agent arms no longer fail with "No module named 'mcp'" - #897

Draft
xaleryb wants to merge 1 commit into
intel:mainfrom
xaleryb:fix/litellm-1.109.1
Draft

xaleryb wants to merge 1 commit into
intel:mainfrom
xaleryb:fix/litellm-1.109.1

Conversation

@xaleryb

@xaleryb xaleryb commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Draft until 2026-11-19: the locks pin a litellm release younger than their 7-day cooldown. See "Quarantine" below.

Summary

From a locked install, every skill-agent: arm answer fails with agent_failed: No module named 'mcp'. The run still prints [1/1] Q1 OK and arms run exits 0, so the batch looks healthy and the arm scores as if the agent had answered nothing.

Cause: litellm 1.106.0 through 1.108.3 import mcp.types on any completion(tools=...). main.py takes the if not skip_mcp_handler and tools: branch, which imports responses/mcp/litellm_proxy_mcp_handler, which imports proxy/_experimental/mcp_server/catalog.py, which does from mcp.types import CacheableResult. mcp is only in litellm's proxy/mcp extras, so a base install does not have it. It is fixed upstream in 1.108.4 and later (BerriAI/litellm#45961, backported in BerriAI/litellm#50605, merged 2026-11-09; reports: BerriAI/litellm#46045, BerriAI/litellm#50781). Our lock moved onto 1.106.0 on 2026-10-26 (#224). Since then, the agent arms have not worked from requirements.lock.txt or ci/conda-lock/.

The fix raises the floor to the first fixed release and locks 1.109.1. Rejected alternatives:

  • Add mcp as a runtime dependency: it brings 13 packages (starlette, uvicorn, sse-starlette, python-multipart among them) into the base install, which goes against fix(deps): take ragas, langchain and diskcache out of the CI/dev lock #245 (ragas moved out of the base install to reduce the CVE surface).
  • Pass litellm's _skip_mcp_handler: a private argument that can change in any release.
  • Guard the import in Dokimi: the import happens inside litellm, so a guard would change the error message and the call would still fail.
  • Cap litellm<1.106: this stays on an old release and blocks the upstream fix.

Changes

  • litellm>=1.108.4 (was >=1.105.0) in requirements.txt, requirements-test.txt, pyproject.toml and conda-recipe/meta.yaml. No other file lists the spec.
  • requirements.lock.txt: litellm==1.109.1 (was 1.106.6). The only other change is the python_full_version < '3.15' marker that 1.109.1 adds to its fastuuid requirement, plus a # via litellm line. Every other pin is the same as on main.
  • ci/conda-lock/linux-64-py3.{11,12,13,14}.txt: litellm 1.106.0 becomes conda-forge 1.109.1. async-timeout drops out because 1.109.1 no longer requires it on Python 3.11 and later. The # specs: header carries the new floor. Every other record is the same build as on main.
  • tests/test_agent_arm_dependencies.py: runs a skill_agent: arm through the real litellm.completion with mock_response. This sends nothing over the network, but it imports what a call with tools imports. The test fails on main's lock and passes with this one.
  • Changelog fragment (Fixed).

Quarantine

Why 1.109.1 and not 1.108.4: conda-forge skipped 1.108.x (1.106.0 on 2026-10-21, then 1.109.0 on 2026-11-11 and 1.109.1 on 2026-11-12). 1.109.1 is the oldest fixed release that both PyPI and conda-forge ship, so the pip lock and the conda locks pin the same version.

Why a draft: both locks admit only releases at least 7 days old (--exclude-newer=P7D in the uv header, --min-age-days 7 in ci/lock.py). The 1.109.1 wheels reached PyPI on 2026-11-12 02:00 UTC and the conda-forge builds on 2026-11-12 10:47 UTC, so both clear the cooldown on 2026-11-19 at 10:47 UTC (02:47 PST). Before that, this PR relaxes the cooldown for litellm only:

  • requirements.lock.txt: the header's command plus --upgrade-package litellm --exclude-newer-package litellm=false. The header line keeps the canonical command.
  • ci/conda-lock/: ci/lock.py --min-age-days 5, with every record except litellm held to main's build so that only litellm and what it requires could move. The header keeps # min-age-days: 7.

So both headers already describe the state they will have on 2026-11-19. If you run the canonical uv command today, it does not keep 1.109.1: it falls back to 1.108.5, which is fixed but is not on conda-forge.

On 2026-11-19, after 10:47 UTC:

  1. Regenerate requirements.lock.txt with the exact command in its header. It must come out identical. A run of that command with P7D replaced by its 2026-11-19 03:00 UTC cutoff already does.
  2. Run python ci/lock.py --if-stale. It must report all four conda locks current. Then re-run the litellm-only solve at --min-age-days 7 and expect the same four files byte for byte.
  3. Re-run the gates below, then mark this PR ready.

Risk

This changes litellm only, from 1.106.x to 1.109.1, for every LLM call Dokimi makes. The full suite passes on the new lock, and an agent arm answered end to end on Bedrock. pip install dokimi users with an older litellm get the new floor on their next upgrade. The changelog entry tells them to re-run affected arms run batches.

1.109.1 declares filelock<5, so the filelock>=4.2.1 override and the --no-deps installs that SECURITY.md describes are no longer needed (pip check is clean on the new lock). This PR leaves both alone. They are follow-up 1 below.

Testing

On Python 3.13.18 (Linux x86_64), in a venv built from the new requirements.lock.txt with pip install --no-deps --require-hashes -r requirements.lock.txt -r requirements-ci.lock.txt:

  • pytest -n auto --outcomes ... (the CI command): 22842 passed, 38 skipped, rc 0.
  • tests/test_agent_arm_dependencies.py: fails in a venv from main's lock (litellm 1.106.6) with ModuleNotFoundError: No module named 'mcp' from litellm/proxy/_experimental/mcp_server/catalog.py. Passes in the new venv, and passes in a conda env created from the new linux-64-py3.13.txt.
  • ruff check ., mypy (438 files), scripts/check_growth.py --report, scripts/check_comment_blocks.py, scripts/check_changelog_fragment.py, scripts/check_test_narrowing.py, scripts/check_duplicates.py, python -m dokimi.schema_compat --base origin/main, deptry ., scripts/check_docs_review_triggers.py: all clean.
  • pip-audit, run as pip-audit.yml runs it over all six locks: no known vulnerabilities. osv-scanner 2.7.1 with osv-scanner.toml on requirements.lock.txt: no issues.
  • micromamba create --file succeeds for all four ci/conda-lock files, and each env imports litellm 1.109.1.
  • End to end: cli.py arms run --product demo --questions q.json --arms skill-agent:<demo skill> --model us.anthropic.claude-haiku-5-5 --provider amazon-bedrock --max-iterations 2. On main the answer is agent_failed: No module named 'mcp' with Q1 OK and rc 0. On this branch the answer is "The demo widget uses port 4242." after 2 model calls, with no agent_failed.

Open questions

  • The conda locks change only litellm. A plain python ci/lock.py re-solve would also move about 15 unrelated packages per file (boto3, datasets, filelock 3.32.6 to 4.2.9, regex, ...). I left that to Monday's conda-lock.yml refresh. Say so if you would rather this PR carry the full re-solve.

Revert

Revert the commit. The floors and both locks return to litellm 1.105.0/1.106.x, and the agent arms fail again as described above. No stored records or schema versions are involved.

Follow-ups (separate PRs)

  1. Drop requirements-overrides.txt (filelock>=4.2.1), the --override flag in both lock headers and the --no-deps installs, and update the filelock paragraph of SECURITY.md: litellm 1.109.1 allows filelock 4.
  2. arms run reports OK and exits 0 when an arm answer is agent_failed. That is why this breakage went unnoticed for three weeks.

Checklist

  • Tests added/updated and pytest passes
  • ruff, mypy, and schema checks pass
  • Docs updated (or N/A): N/A here. The SECURITY.md filelock paragraph is follow-up 1
  • Self-reviewed per "Before opening a PR" in AGENTS.md
  • Each new raise/exit and test names the doc, contract or issue it traces to ([Bug]: "ModuleNotFoundError: No module named 'mcp'" BerriAI/litellm#46045)
  • No ad-hoc run artifacts committed
  • New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

litellm 1.106.0-1.108.3 import mcp on every completion(tools=...), so each
skill-agent arm answer from a locked install failed with No module named
'mcp' (BerriAI/litellm#46045). Raise the floor to the release with the
fix and lock 1.109.1, the first one conda-forge also ships.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
@xaleryb
xaleryb force-pushed the fix/litellm-1.109.1 branch from c56109d to 961ef71 Compare November 17, 2026 23:22
@xaleryb xaleryb changed the title fix(deps): require litellm>=1.108.4 and lock 1.109.1 fix(deps): skill-agent arms no longer fail with "No module named 'mcp'" Nov 17, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.79%. Comparing base (b62613a) to head (961ef71).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #897   +/-   ##
=======================================
  Coverage   95.79%   95.79%           
=======================================
  Files         432      432           
  Lines       57204    57204           
  Branches     9858     9858           
=======================================
  Hits        54796    54796           
  Misses       1497     1497           
  Partials      911      911           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants