Skip to content

feat: add docker-docs skill and Docker docs MCP server - #111

Draft
dvdksn wants to merge 1 commit into
mainfrom
docs-retrieval-skill
Draft

dvdksn wants to merge 1 commit into
mainfrom
docs-retrieval-skill

Conversation

@dvdksn

@dvdksn dvdksn commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

What I did

Adds an experimental docker-docs skill and registers the remote Docker docs MCP server (https://ai-backend-service.docker.com/mcp) in the plugin and extension manifests.

Important

Depends on docker/gordon#2603 being deployed. The skill and its checks refer to the tool as search_docker_docs; the live server still exposes it as knowledge_base until that PR ships. Keep this as a draft until then.

Skill: skills/docker-docs/

  • Answers questions about any Docker product from current, cited Docker Docs pages rather than from memory. It covers more than containers: Sandboxes (sbx, local and cloud, Sandboxes API/SDK), the Agentic Platform, DHI, Docker Agent, Model Runner, MCP Toolkit, and accounts, admin and billing.
  • Source order: the search_docker_docs MCP tool when it is available. Otherwise it falls back to docs.docker.com/llms.txt, then a local search of the llms-full.txt page index, then a page fetched as Markdown (a .md URL or Accept: text/markdown). A general web search limited to site:docs.docker.com is the last resort.
  • Borrows ideas from the OpenAI docs skill:
    • Search before answering.
    • Fetch the page instead of relying on search excerpts, and cite only pages it has read.
    • Keep the product name the user gave.
    • Don't guess at pricing or entitlements.
    • A missing tool is not a reason to install anything.
    • Read at most one reference.
  • references/product-map.md lists aliases and top-level docs sections only, to keep drift low. references/mcp-setup.md covers manual setup and troubleshooting, and the skill reads it only when the user asks.
  • Also adds checks/verification.md (manual runbook against the live endpoints), evals/docker-docs.md, dd-* eval checks, a catalog entry (0.1.0, experimental, Cross-Product) and CODEOWNERS rules.

MCP server registration

Client Declared in Verified
Claude Code .claude-plugin/plugin.json → plugin-mcp.json live install, connected
Codex .codex-plugin/plugin.json → plugin-mcp.json, plus agents/openai.yaml tool dependency live install, enabled
GitHub Copilot CLI .github/plugin/plugin.json → plugin-mcp.json live install, source: plugin
Gemini CLI inline httpUrl in gemini-extension.json live install (0.62.0), connected
Cursor .cursor-plugin/plugin.json → plugin-mcp.json from Cursor CLI plugin-loader source only; needs a logged-in session to confirm

The file is called plugin-mcp.json rather than .mcp.json on purpose. The plugin root is the repository root, so a .mcp.json would also be loaded as project MCP config by Claude Code and Copilot CLI for anyone working in this repo.

The skills CLI, Docker Sandboxes, Docker Agent and manual-copy installs get the skill only, not the server. For those, the skill uses the llms.txt and Markdown fallback.

Eval simulation

I ran the five prompts in evals/docker-docs.md with Claude Code (Opus 5.5) in two modes: with the docs MCP server connected, and without it, which forces the fallback. Because docker/gordon#2603 hasn't deployed yet, the runs used a scratch copy of the skill with the tool name set back to knowledge_base. Tools were limited to read-only.

Prompt With MCP Fallback
1. Allow one host, block everything else Fail Fail
2. DHI Node.js migration Pass Pass
3. Cloud sandbox GitHub token Pass Pass
4. Business plan vs cloud compute Pass Pass
5. Add the docs MCP server Pass Pass

In prompts 2-4, the agent searched or used the index, fetched pages as Markdown, and cited them. It also said when the docs don't settle something, such as Business-plan credits for cloud compute. The fallback runs searched llms-full.txt from a temp file and read only small slices.

Prompt 1 is evidence for this skill. In both modes the agent routed to docker-sandboxes-network-credentials and answered from that skill's bundled guidance without checking the docs. Compared with the current local policy docs, that answer is wrong on two points:

  • It warned that sbx policy init deny-all would cut the agent off from its own model API. The docs say built-in agent kits add per-sandbox allow rules even under Locked Down (deny-all).
  • It said hosts without a matching rule are blocked. The docs say that under Balanced and Locked Down, an unmatched request asks the user for approval instead of being denied outright.

Bundled CLI guidance goes stale between releases. Retrieving from the docs doesn't have that problem, as prompts 2-4 show.

Follow-ups to consider

  • Consider removing or slimming product skills whose value is mostly reference material that the docs already cover, starting with the sbx skills, in favor of docker-docs. Keep only workflow and safety guidance the docs don't cover, such as docker-destructive-guardrails. This PR doesn't change that routing: docker-docs defers to a more specific skill when one exists, which is why prompt 1 never consulted the docs.
  • Fix the outdated policy guidance in docker-sandboxes-network-credentials in the meantime.
  • docs.docker.com/llms.txt advertises https://mcp-docs.docker.com/mcp, which only returns llms.txt and has no search tool. It may be worth pointing it at this server.
  • CODEOWNERS assigns @docker/gordon only. A docs team may also want ownership.

Checklist

  • My commits are signed off (DCO).
  • task passes locally. The two PR-only checks, check_version_bumps.py and check_dco.py, were run directly against the merge base and both pass.
  • If this PR changes skills/<id>/, I increased that skill's version in both catalog.yaml and its skill.yaml. This is a new skill with matching initial version 0.1.0.

🤖 Generated with Claude Code

Add an experimental docker-docs skill that answers questions about any
Docker product from current, cited Docker Docs pages. It searches with
the search_docker_docs MCP tool when available and otherwise falls back
to docs.docker.com/llms.txt and per-page Markdown. A product map covers
aliases and top-level sections across Sandboxes (local and cloud), DHI,
Docker Agent, Model Runner, MCP Toolkit, and admin and billing docs.

Register the remote Docker docs MCP server in the Claude Code, Codex,
Cursor, and Copilot CLI plugin manifests through plugin-mcp.json, and
inline in gemini-extension.json. The file is not named .mcp.json so
clients don't load it as project config for this repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant