Repository navigation
Description
Which server?
git (mcp-server-git)
Protocol era
Both, or not era-specific
MCP client
Custom SDK client: AgentGrade (open-source eval harness on the MCP Python SDK 2.3.0 client); model qwen3:4b-instruct via Ollama, temperature 0.7
The problem
Server: mcp-server-git==2026.10.10 (checked: 2026.11.11 has the same git_log description and the same empty output). Run with uvx mcp-server-git --repository <repo>, stdio, macOS.
git_log's description is "Shows the commit logs". It doesn't say results come newest first, or that start_timestamp / end_timestamp are optional. Asked "Who made the most recent commit on the main branch?", a small model repeatedly called:
{"repo_path": "<repo>", "max_count": 1, "start_timestamp": "now", "end_timestamp": "now"}The server returned an empty Commit history: and the agent concluded "the commit history is empty… unknown author". This happened in 5 of 6 runs of that question (two independent runs), even when the agent knew the correct repo path.
Measured on a deterministic repository (identical commit hashes every run), 5 read-only questions x 3 runs, answers checked in code, no LLM judge.
Case study (method, all tables, caveats): https://github.com/Tanishq67m/AgentGrade/blob/main/docs/CASE_STUDY.md
Raw reports: https://github.com/Tanishq67m/AgentGrade/tree/main/results
Solution you have in mind (optional)
Change only git_log's description to something like:
Shows the commit log, newest commit first. To see the latest commits, call it with just repo_path (and optionally max_count). start_timestamp and end_timestamp are OPTIONAL filters for a specific date range - leave them out unless the user asks about particular dates, because a narrow range returns no commits.
With only that change, the suite went from 13/15 to 15/15, and the "most recent commit" question from 1/6 to 3/3.
Two smaller ideas in the same spirit:
- When no commits match, say so with a hint ("No commits in this range; omit the timestamps to see the latest commits") instead of an empty
Commit history:. - The
repo_patherror already names the allowed repository, which is great, but small models usually stopped there instead of retrying (all 9 failures in one run), and a 120B model mis-copied the long path twice. A defaultrepo_path(the configured--repository) would remove that failure class.
(We also hit git_diff rejecting main..feature in 2026.10.10. Thanks for already fixing that in #5194.)
Which MCP feature does it demonstrate? (optional)
No response
Alternatives or workarounds you have tried
- Agent-side prompt advice ("If a tool returns an error, read it and retry"): 8/15 vs 7/15 baseline, so no real effect, and it made another task worse.
- Telling the agent the repo path in its system prompt: 7/15 → 13/15 overall, but the "most recent commit" question stayed at 1/6. The timestamp trap is independent of the path.
- A larger model (gpt-oss-120b) never used the timestamp filters and scored 14/15, so this mainly affects small, cheap models.
Already built it locally?
No server code was changed. The suggested description was tested by having the client show the model only a different git_log description, with the server binary unchanged and the same tasks and repeats. Verified by deterministic, code-checked answers over 15 runs per variant. The exact text is in "Solution" above.
Before you submit
- I searched existing issues and this is not a duplicate.
- This is a request for this repository or one of its servers, not a new server, the MCP specification or an SDK.
Activity
shivamm-gupta commentedon Nov 18, 2026
Put together a patch for this alongside #5553:
shivamm-gupta@b3c9386b
Summary of changes
git_logdescription insrc/git/src/mcp_server_git/server.pyandREADME.mdto specify results are in reverse chronological order (newest first), and clarifiedstart_timestamp/end_timestampare optional date range filters that should be left empty for recent commits.list_tools.jsontest snapshot to match.Prompt used
Per repo contribution guidelines, here is the prompt that produced the change:
cliffhall commentedon Nov 18, 2026
Triage: Priority Low (total 4)
git_logand get an empty history; omitting the filters worksBoard: Servers V2 (#43), Status
Incoming, awaiting maintainer review (no milestone yet).Tanishq67m commentedon Nov 18, 2026
I measured the patch for this one the same way (
b3c9386bvse1d154ee, mcp-server-git 2026.11.11,qwen3:4b-instruct, 5 tasks × 3 repeats, with the repository path given to the agent):git_logon 2026.11.11e1d154eeb3c9386bMain's one failure was exactly this bug: the agent concluded "the commit history is empty". Neither description variant hit it. On 2026.10.10 the same task failed 6 of 6 times with the path given, so 2026.11.11 already behaves much better here, and 15 runs can't show a significant difference. The patch looks good to me. It removes the failure we saw and doesn't regress anything else.
The other suggestion still stands. An empty
Commit history:with a hint ("No commits in this range; omit the timestamps to see the latest commits") would cover the case where a model still adds filters.Suite: https://github.com/Tanishq67m/AgentGrade/blob/main/suites/pr-5552-git.yaml
Report: https://github.com/Tanishq67m/AgentGrade/blob/main/results/2026-11-18-pr5552-git-patch.md