Skip to content

git: git_log description doesn't say results are newest-first or that timestamp filters are optional; small models filter to an empty window #5552

Open

Description

@Tanishq67m

Which server?

git (mcp-server-git)

Protocol era

Both, or not era-specific

MCP client

Custom SDK client: AgentGrade (open-source eval harness on the MCP Python SDK 2.3.0 client); model qwen3:4b-instruct via Ollama, temperature 0.7

The problem

Server: mcp-server-git==2026.10.10 (checked: 2026.11.11 has the same git_log description and the same empty output). Run with uvx mcp-server-git --repository <repo>, stdio, macOS.

git_log's description is "Shows the commit logs". It doesn't say results come newest first, or that start_timestamp / end_timestamp are optional. Asked "Who made the most recent commit on the main branch?", a small model repeatedly called:

{"repo_path": "<repo>", "max_count": 1, "start_timestamp": "now", "end_timestamp": "now"}

The server returned an empty Commit history: and the agent concluded "the commit history is empty… unknown author". This happened in 5 of 6 runs of that question (two independent runs), even when the agent knew the correct repo path.

Measured on a deterministic repository (identical commit hashes every run), 5 read-only questions x 3 runs, answers checked in code, no LLM judge.

Case study (method, all tables, caveats): https://github.com/Tanishq67m/AgentGrade/blob/main/docs/CASE_STUDY.md
Raw reports: https://github.com/Tanishq67m/AgentGrade/tree/main/results

Solution you have in mind (optional)

Change only git_log's description to something like:

Shows the commit log, newest commit first. To see the latest commits, call it with just repo_path (and optionally max_count). start_timestamp and end_timestamp are OPTIONAL filters for a specific date range - leave them out unless the user asks about particular dates, because a narrow range returns no commits.

With only that change, the suite went from 13/15 to 15/15, and the "most recent commit" question from 1/6 to 3/3.

Two smaller ideas in the same spirit:

  • When no commits match, say so with a hint ("No commits in this range; omit the timestamps to see the latest commits") instead of an empty Commit history:.
  • The repo_path error already names the allowed repository, which is great, but small models usually stopped there instead of retrying (all 9 failures in one run), and a 120B model mis-copied the long path twice. A default repo_path (the configured --repository) would remove that failure class.

(We also hit git_diff rejecting main..feature in 2026.10.10. Thanks for already fixing that in #5194.)

Which MCP feature does it demonstrate? (optional)

No response

Alternatives or workarounds you have tried

  • Agent-side prompt advice ("If a tool returns an error, read it and retry"): 8/15 vs 7/15 baseline, so no real effect, and it made another task worse.
  • Telling the agent the repo path in its system prompt: 7/15 → 13/15 overall, but the "most recent commit" question stayed at 1/6. The timestamp trap is independent of the path.
  • A larger model (gpt-oss-120b) never used the timestamp filters and scored 14/15, so this mainly affects small, cheap models.

Already built it locally?

No server code was changed. The suggested description was tested by having the client show the model only a different git_log description, with the server binary unchanged and the same tasks and repeats. Verified by deterministic, code-checked answers over 15 runs per variant. The exact text is in "Solution" above.

Before you submit

  • I searched existing issues and this is not a duplicate.
  • This is a request for this repository or one of its servers, not a new server, the MCP specification or an SDK.

Activity

  1. shivamm-gupta commented on Nov 18, 2026

    @shivamm-gupta

    Put together a patch for this alongside #5553:
    shivamm-gupta@b3c9386b

    Summary of changes

    • Updated git_log description in src/git/src/mcp_server_git/server.py and README.md to specify results are in reverse chronological order (newest first), and clarified start_timestamp/end_timestamp are optional date range filters that should be left empty for recent commits.
    • Updated the list_tools.json test snapshot to match.

    Prompt used

    Per repo contribution guidelines, here is the prompt that produced the change:

    In mcp-server-git (src/git/src/mcp_server_git/server.py and README.md), update git_log tool description to specify reverse chronological order (newest first) and clarify that start_timestamp and end_timestamp are optional date range filters (#5552). Update src/git/tests/snapshots/list_tools.json snapshot.

  2. added
    server-gitReference implementation for the Git MCP server - src/git
    on Nov 18, 2026
  3. cliffhall commented on Nov 18, 2026

    @cliffhall
    Member

    Triage: Priority Low (total 4)

    • Severity 2: small models pass narrow timestamp filters to git_log and get an empty history; omitting the filters works
    • Urgency 2: wanted eventually, nothing blocked on it
    • Bonuses: none

    Board: Servers V2 (#43), Status Incoming, awaiting maintainer review (no milestone yet).

  4. added this to the v2.6.0 milestone on Nov 18, 2026
  5. Tanishq67m commented on Nov 18, 2026

    @Tanishq67m
    Author

    I measured the patch for this one the same way (b3c9386b vs e1d154ee, mcp-server-git 2026.11.11, qwen3:4b-instruct, 5 tasks × 3 repeats, with the repository path given to the agent):

    git_log on 2026.11.11 Passed
    main e1d154ee 14/15
    patch b3c9386b 15/15
    main + the description suggested in this issue 15/15

    Main's one failure was exactly this bug: the agent concluded "the commit history is empty". Neither description variant hit it. On 2026.10.10 the same task failed 6 of 6 times with the path given, so 2026.11.11 already behaves much better here, and 15 runs can't show a significant difference. The patch looks good to me. It removes the failure we saw and doesn't regress anything else.

    The other suggestion still stands. An empty Commit history: with a hint ("No commits in this range; omit the timestamps to see the latest commits") would cover the case where a model still adds filters.

    Suite: https://github.com/Tanishq67m/AgentGrade/blob/main/suites/pr-5552-git.yaml
    Report: https://github.com/Tanishq67m/AgentGrade/blob/main/results/2026-11-18-pr5552-git-patch.md

  6. self-assigned this
    on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestserver-gitReference implementation for the Git MCP server - src/gitv2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions