-
Notifications
You must be signed in to change notification settings - Fork 26
feat(document-review): add documentation review workflow #97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
fe19007
2c4f805
f721c17
d5d9cdf
1e2265f
9b641e6
ece44b5
834f7f4
7d359e7
f3d87c6
d294118
6938432
9aa0c18
2df5734
cbcce31
8ca0ba1
16d5af5
8d9c5c1
3f58639
ef2a467
cb50930
3633f16
6f0bbe9
bbedcd5
b3026fb
98ae4da
fb5cb4a
f295eb7
d0649b6
79c5443
2dd1458
9a9f32b
a4bc29c
742141b
9f655de
9e3cdc8
f353387
98ba76c
af0098e
daa867c
9c777c5
a9f81f5
cd72ee4
1bf821e
1aa846d
0352ff8
72c393a
04d9986
44a0d99
ab152ad
580b580
71b0104
c5ea86a
3ed265f
7ccc12c
986f9a0
37fc6f4
958abb0
b039bd4
b7d9f1a
887ce5e
b0d4dd9
a233022
d6134b5
43d4e78
60e346b
a166aa3
a337f5e
2275bd5
b63d43e
0f7134f
d981b0e
21e6e00
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "name": "Document Review", | ||
| "description": "Systematic workflow for reviewing a project's documentation — assessing quality, completeness, accuracy, and consistency, then generating actionable findings.", | ||
| "systemPrompt": "You are a documentation quality specialist for the Ambient Code Platform.\n\nYou are controlled by a workflow controller at:\n .claude/skills/controller/SKILL.md\n\nRead it at the start of the session. It defines how to execute phases, recommend next steps, and handle transitions.\n\nWORKSPACE NAVIGATION:\n**CRITICAL: Follow these rules to avoid fumbling when looking for files.**\n\nStandard file locations (from workflow root):\n- Controller: .claude/skills/controller/SKILL.md\n- Phase skills: .claude/skills/{name}/SKILL.md\n- Commands: .claude/commands/*.md\n- Output files: artifacts/*.md\n\nTool selection rules:\n- Use Read for: Known paths, standard files, files you just created\n- Use Glob for: Discovery (finding multiple files by pattern)\n- Use Grep for: Content search\n\nNever glob for standard files:\n✅ DO: Read .ambient/ambient.json\n❌ DON'T: Glob **/ambient.json", | ||
| "startupPrompt": "Welcome! I'm your documentation review specialist. I systematically review project documentation to identify quality issues and classify them by severity.\n\n## Available Commands\n\n- `/scan` — Discover and catalog all documentation in the project\n- `/quality-review` — Deep quality review of the entire documentation corpus\n- `/code-check` — Cross-reference documentation against source code\n- `/report` — Consolidate all findings into a deduplicated report\n- `/full-review` — Run scan → quality-review + code-check → report in one shot\n- `/jira` — Create a Jira epic with child bugs/tasks from the report\n\n## Getting Started\n\nPoint me at a project repository or documentation path and I'll get started. You can either:\n1. Provide a repository URL or local path\n2. Run `/scan` to auto-discover documentation in the current workspace\n3. Run `/full-review` for a complete review in one pass\n", | ||
| "results": { | ||
| "Inventory": "artifacts/inventory.md", | ||
| "Quality Review Findings": "artifacts/findings-quality-review.md", | ||
| "Code Check Findings": "artifacts/findings-code-check.md", | ||
| "Report": "artifacts/report.md" | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # /code-check — Cross-reference docs against source code | ||
|
|
||
| Verifies documentation claims against actual source code using parallel | ||
| discovery agents. Finds mismatches, undocumented features, and stale references. | ||
| Writes findings to `artifacts/findings-code-check.md`. | ||
|
|
||
| **Requires:** `/scan` must have been run first. | ||
|
|
||
| Read `.claude/skills/controller/SKILL.md` and follow it. | ||
|
|
||
| Dispatch the **code-check** phase. Context: | ||
|
|
||
| $ARGUMENTS | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,10 @@ | ||||||
| # /full-review — Run the complete review pipeline | ||||||
|
|
||||||
| Runs scan → quality-review + code-check (parallel) → report in one shot, | ||||||
| pausing only for critical decisions. Results are written to `artifacts/`. | ||||||
|
|
||||||
| Read `.claude/skills/controller/SKILL.md` and follow it. | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Controller path should not depend on CWD. If dispatcher runs from repo root, this reference may fail and block Proposed fix-Read `.claude/skills/controller/SKILL.md` and follow it.
+Read `workflows/document-review/.claude/skills/controller/SKILL.md` and follow it.📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
|
|
||||||
| Dispatch the **full-review** phase. Context: | ||||||
|
|
||||||
| $ARGUMENTS | ||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,53 @@ | ||||||||||||||||||||
| # /jira - Create Jira Issues From Report | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Purpose | ||||||||||||||||||||
|
|
||||||||||||||||||||
| Creates a Jira epic from the documentation review report, with a child bug or | ||||||||||||||||||||
| task for each finding. Bugs are for findings that impact external users or | ||||||||||||||||||||
| customers. Tasks are for findings that impact developers or are maintenance | ||||||||||||||||||||
| items. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Prerequisites | ||||||||||||||||||||
|
|
||||||||||||||||||||
| - `artifacts/report.md` must exist (run `/report` first) | ||||||||||||||||||||
| - `JIRA_URL`, `JIRA_EMAIL`, and `JIRA_API_TOKEN` environment variables must be set | ||||||||||||||||||||
|
Comment on lines
+10
to
+13
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add missing pandoc prerequisite. The 📝 Proposed addition ## Prerequisites
- `artifacts/report.md` must exist (run `/report` first)
+- `pandoc` must be installed (install via `pip install pypandoc_binary`)
- `JIRA_URL`, `JIRA_EMAIL`, and `JIRA_API_TOKEN` environment variables must be set📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Usage | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ```text | ||||||||||||||||||||
| /jira <project-key> [component=<name>] [labels=<a,b,c>] [team=<name>] [status=<name>] | ||||||||||||||||||||
| ``` | ||||||||||||||||||||
|
|
||||||||||||||||||||
| Arguments override environment variables. If no project key is given, the | ||||||||||||||||||||
| `JIRA_PROJECT` environment variable is used. Any field not provided via | ||||||||||||||||||||
| arguments or environment variables will be prompted for — the user must | ||||||||||||||||||||
| explicitly set a value or confirm it should be left blank. | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ### Environment Variables | ||||||||||||||||||||
|
|
||||||||||||||||||||
| | Variable | Purpose | | ||||||||||||||||||||
| |----------|---------| | ||||||||||||||||||||
| | `JIRA_PROJECT` | Default Jira project key | | ||||||||||||||||||||
| | `JIRA_COMPONENT` | Default component name | | ||||||||||||||||||||
| | `JIRA_LABELS` | Default comma-separated labels | | ||||||||||||||||||||
| | `JIRA_TEAM` | Default team name | | ||||||||||||||||||||
| | `JIRA_INITIAL_STATUS` | Workflow transition after creation (e.g., `Backlog`) | | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ### Examples | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ```text | ||||||||||||||||||||
| /jira DOCS component=documentation labels=docs,review team=docs-team status=Backlog | ||||||||||||||||||||
| /jira DOCS component=none labels=none team=none status=New | ||||||||||||||||||||
| ``` | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Process | ||||||||||||||||||||
|
|
||||||||||||||||||||
| 1. Read the skill at `.claude/skills/jira/SKILL.md` | ||||||||||||||||||||
| 2. Execute the skill's steps | ||||||||||||||||||||
|
|
||||||||||||||||||||
| ## Output | ||||||||||||||||||||
|
|
||||||||||||||||||||
| Jira issues created via the Jira REST API: | ||||||||||||||||||||
|
|
||||||||||||||||||||
| - One **Epic** summarizing the full review | ||||||||||||||||||||
| - One **Bug** or **Task** per finding, as children of the epic | ||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,13 @@ | ||||||
| # /quality-review — Deep quality analysis of documentation | ||||||
|
|
||||||
| Evaluates every document in the inventory against 7 quality dimensions | ||||||
| (accuracy, completeness, consistency, clarity, currency, structure, examples) | ||||||
| and writes findings to `artifacts/findings-quality-review.md`. | ||||||
|
|
||||||
| **Requires:** `/scan` must have been run first. | ||||||
|
|
||||||
| Read `.claude/skills/controller/SKILL.md` and follow it. | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Make controller reference deterministic (repo-root path). Current path is CWD-sensitive and may fail in common execution contexts. Proposed fix-Read `.claude/skills/controller/SKILL.md` and follow it.
+Read `workflows/document-review/.claude/skills/controller/SKILL.md` and follow it.📝 Committable suggestion
Suggested change
|
||||||
|
|
||||||
| Dispatch the **quality-review** phase. Context: | ||||||
|
|
||||||
| $ARGUMENTS | ||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,14 @@ | ||||||
| # /report — Consolidate findings into a deduplicated report | ||||||
|
|
||||||
| Merges all findings from `/quality-review` and `/code-check` into a single | ||||||
| report grouped by severity, with a dimension × severity summary table. | ||||||
| Writes to `artifacts/report.md`. | ||||||
|
|
||||||
| **Requires:** At least one of `/quality-review` or `/code-check` must have | ||||||
| been run first. | ||||||
|
|
||||||
| Read `.claude/skills/controller/SKILL.md` and follow it. | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fix controller skill path to avoid runtime resolution failures. Use a repo-root-qualified path here as well. Proposed fix-Read `.claude/skills/controller/SKILL.md` and follow it.
+Read `workflows/document-review/.claude/skills/controller/SKILL.md` and follow it.📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
|
|
||||||
| Dispatch the **report** phase. Context: | ||||||
|
|
||||||
| $ARGUMENTS | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # /scan — Discover and catalog all documentation | ||
|
|
||
| Discovers all documentation files in the project, classifies them by format, | ||
| topic, and audience, and writes a structured inventory to `artifacts/inventory.md`. | ||
|
|
||
| **Arguments:** Optional path or glob to limit the scan to specific files or | ||
| directories. If omitted, scans the entire project. | ||
|
|
||
| Read `.claude/skills/controller/SKILL.md` and follow it. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Make controller path unambiguous to avoid command breakage Line 3 hardcodes a relative path that only works if the runtime CWD is the workflow root. If commands run from repo root, controller loading fails and 🤖 Prompt for AI Agents |
||
|
|
||
| Dispatch the **scan** phase. Context: | ||
|
|
||
| $ARGUMENTS | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,225 @@ | ||
| --- | ||
| name: code-check | ||
| description: Cross-reference documentation claims against actual source code. | ||
| --- | ||
|
|
||
| # Code Check Documentation Skill | ||
|
|
||
| You are cross-referencing a project's documentation against its actual source | ||
| code to verify accuracy. This is a deeper accuracy check than the | ||
| `/quality-review` phase, which only evaluates documentation in isolation. | ||
|
|
||
| You work in three stages: | ||
|
|
||
| 1. **Reconnaissance** — Detect languages, frameworks, and components | ||
| 2. **Discovery** — Dispatch parallel agents to build a code inventory | ||
| 3. **Verification** — Cross-reference the code inventory against documentation | ||
|
|
||
| ## Critical Rules | ||
|
|
||
| - **Read the inventory first.** This phase requires `/scan` to have been run. | ||
| If `artifacts/inventory.md` does not exist, inform the user and recommend | ||
| running `/scan` first. | ||
| - **Read, don't run.** This is static analysis — read source code, don't | ||
| execute it. | ||
| - **Be precise.** Every finding must include a direct quote from the | ||
| documentation AND an actual code snippet. Use fenced code blocks with | ||
| language tags. | ||
| - **Flag undocumented features.** Code functionality with no documentation is | ||
| a High finding. When checking a code area (e.g., a struct or route group), | ||
| scan all fields/routes — not just what docs mention. | ||
| - **Separate uncertain findings.** Low-confidence findings go in a dedicated | ||
| section, not in the main findings. Fuzzy name matches (e.g., `MAAS_DB_HOST` | ||
| vs `DB_HOST`) belong in Low-Confidence. | ||
|
|
||
| ## Stage 1: Reconnaissance | ||
|
|
||
| Perform this yourself — no agents needed. This determines which discovery | ||
| agents to spawn in Stage 2. | ||
|
|
||
| ### Step 1.1: Language & Framework Detection | ||
|
|
||
| Use Glob to check for these signature files in the project root and | ||
| subdirectories: | ||
|
|
||
| | File | Language/Framework | | ||
| |------|-------------------| | ||
| | `go.mod` | Go | | ||
| | `Cargo.toml` | Rust | | ||
| | `package.json` | Node.js/TypeScript | | ||
| | `pyproject.toml`, `setup.py`, `requirements.txt` | Python | | ||
| | `pom.xml`, `build.gradle` | Java | | ||
| | `Gemfile` | Ruby | | ||
|
|
||
| For detected languages, check framework markers: | ||
|
|
||
| - Go: `go.mod`/`go.sum` for `controller-runtime`, `cobra`, `gin`, `echo`, | ||
| `chi` | ||
| - Python: `pyproject.toml`/`requirements.txt` for `fastapi`, `flask`, | ||
| `django`, `click` | ||
| - Node.js: `package.json` for `express`, `next`, `nestjs`, `commander` | ||
| - Java: `pom.xml`/`build.gradle` for `spring-boot`, `spring-web`, `quarkus`, | ||
| `micronaut`, `picocli`, `javax.ws.rs` (JAX-RS) | ||
| - Ruby: `Gemfile` for `rails`, `sinatra`, `grape`, `thor` | ||
|
|
||
| ### Step 1.2: Component Detection | ||
|
|
||
| Use Glob to find multiple `go.mod`, `package.json`, `Cargo.toml`, | ||
| `pyproject.toml` files. Each distinct directory containing one (that is NOT | ||
| the project root) is a component. Also check for directories with their own | ||
| `README.md` + `Makefile` or `Dockerfile`. | ||
|
|
||
| ### Step 1.3: Category Applicability | ||
|
|
||
| Determine which discovery agents to spawn: | ||
|
|
||
| | Category | Reference file | Spawn condition | | ||
| |----------|---------------|----------------| | ||
| | Env vars | `discovery-env-vars.md` | Always | | ||
| | CLI args | `discovery-cli-args.md` | Entry points found (`main.go`, `main.py`, `bin/`, CLI framework) | | ||
| | Config schema | `discovery-config-schema.md` | Config files or config library imports | | ||
| | API schema | `discovery-api-schema.md` | OpenAPI specs, protobuf, HTTP framework imports | | ||
| | Data models | `discovery-data-models.md` | CRD directories, migration files, ORM imports | | ||
| | File I/O | `discovery-file-io.md` | File write operations or output file references in docs | | ||
| | External deps | `discovery-external-deps.md` | Database drivers, HTTP clients, message queues | | ||
| | Build/deploy | `discovery-build-deploy.md` | Makefiles, Dockerfiles, CI configs | | ||
|
|
||
| ### Step 1.4: Compile Project Profile | ||
|
|
||
| Build a concise profile block to pass to discovery agents: | ||
|
|
||
| ```text | ||
| PROJECT PROFILE | ||
| Languages: Go, Python | ||
| Frameworks: controller-runtime, cobra | ||
| Components: | ||
| - maas-api (Go) — maas-api/ | ||
| - maas-controller (Go) — maas-controller/ | ||
| Discovery agents to spawn: env-vars, cli-args, api-schema, data-models, build-deploy | ||
| ``` | ||
|
|
||
| ## Stage 2: Discovery | ||
|
|
||
| Dispatch discovery agents in PARALLEL using the Agent tool with | ||
| `subagent_type: Explore`. | ||
|
|
||
| For each agent to spawn: | ||
|
|
||
| 1. Read the agent's prompt template from | ||
| `.claude/skills/code-check/references/discovery-{category}.md` | ||
| 2. Read the inventory format spec from | ||
| `.claude/skills/code-check/references/inventory-format.md` | ||
| 3. Construct the agent prompt by concatenating: | ||
| - The project profile from Stage 1 | ||
| - The agent's prompt template | ||
| - The inventory format spec | ||
| 4. Dispatch via Agent tool with `subagent_type: Explore` | ||
|
|
||
| Issue ALL Agent tool calls in a SINGLE response to maximize parallelism. | ||
|
|
||
| **Per-component splitting:** For projects with 4+ components, consider | ||
| spawning per-component agents for categories like API Schema or Data Models | ||
| (e.g., one API Schema agent for `maas-api/`, another for | ||
| `maas-controller/`). Include the component path scope in the agent prompt. | ||
|
|
||
| ### Merge Discovery Results | ||
|
|
||
| After all agents return: | ||
|
|
||
| 1. Collect all inventory fragments | ||
| 2. Empty fragments (zero items) are valid — record as "no items found" | ||
| 3. For failed agents: log the failure, continue with others | ||
| 4. Deduplicate: | ||
| - Primary match: item name (exact) + type | ||
| - Secondary match: same source file and line across agents | ||
| - Merge source locations into single entry | ||
| - For conflicting values: preserve conflict explicitly (e.g., | ||
| `Default: "30" (per config-schema agent) / "90" (per env-vars agent)`) | ||
| - Use entry with most non-null fields as base | ||
| - Use the env var name as the canonical item name when the item is an | ||
| environment variable | ||
| 5. Organize by workflow (installation, then usage, then both), then by | ||
| category within each section | ||
|
|
||
| Hold the merged code inventory in context for Stage 3. | ||
|
|
||
| ## Stage 3: Verification | ||
|
|
||
| Cross-reference the code inventory against the documentation files cataloged | ||
| in `artifacts/inventory.md`. | ||
|
|
||
| ### What to Check | ||
|
|
||
| For each documentation file, identify claims about code behavior and compare | ||
| against the code inventory: | ||
|
|
||
| **Accuracy — do docs match code?** | ||
|
|
||
| - API endpoints: routes, methods, paths, auth requirements | ||
| - CLI flags: names, types, defaults, help text | ||
| - Config options: field names, types, defaults | ||
| - Default values: what the code actually sets | ||
| - Behavior descriptions: auth flows, error handling, rate limiting | ||
|
|
||
| **Completeness — are code features documented?** | ||
|
|
||
| - Code inventory items with NO mention in any documentation file | ||
| - CRD fields, API endpoints, CLI flags, env vars missing from docs | ||
| - Runtime behaviors not explained | ||
|
|
||
| **Staleness — do docs reference things that no longer exist?** | ||
|
|
||
| - Env vars, CLI flags, API endpoints, config fields in docs but NOT in the | ||
| code inventory — confirm absence with Grep/Glob before reporting | ||
| - File paths that don't exist in the repo | ||
| - Components or modules that were removed | ||
| - Dead internal links | ||
|
|
||
| ### Verification Process | ||
|
|
||
| 1. Read each documentation file from the inventory | ||
| 2. For each verifiable claim, compare against the code inventory | ||
| 3. For items in the code inventory with no doc coverage, flag as undocumented | ||
| 4. For items in docs with no code inventory match, verify absence with | ||
| Grep/Glob before flagging as stale | ||
| 5. Record each result: | ||
| - **Match**: Doc accurately reflects code — no finding needed | ||
| - **Mismatch**: Doc contradicts code — typically Critical, but use judgment | ||
| - **Partial**: Doc is incomplete or imprecise — typically Low, but Medium | ||
| or higher if the gap affects a user-facing API or procedure | ||
| - **Undocumented**: Code feature not in docs — typically High, but Medium | ||
| for internal-only features or Low for minor config options | ||
| - **Stale**: Doc references removed functionality — typically High, but | ||
| Critical if users would follow broken instructions | ||
|
|
||
| ### Record Findings | ||
|
|
||
| Follow the template at `templates/findings-code-check.md`. Write to | ||
| `artifacts/findings-code-check.md`. | ||
|
|
||
| Each finding must include: | ||
|
|
||
| - **Severity**: Critical, High, Medium, or Low (use the guidance in the | ||
| verification process above, but assess each finding individually) | ||
| - **Dimension**: Accuracy or Completeness | ||
| - **File**: Doc file path and line (e.g., `README.md:85`) | ||
| - **Code location**: Source file and line | ||
| - **Documented claim**: What the docs say (direct quote) | ||
| - **Actual behavior**: What the code does | ||
| - **Evidence**: Code snippet in a fenced code block with language tag | ||
| - **Fix**: Correction, if known with high confidence (omit if unsure) | ||
|
|
||
| ## Output | ||
|
|
||
| - `artifacts/findings-code-check.md` | ||
|
|
||
| ## When This Phase Is Done | ||
|
|
||
| Report to the user: | ||
|
|
||
| - Total findings by type (mismatch, partial, undocumented, stale) | ||
| - Inventory coverage (which agents ran, items found per category) | ||
| - Top 3 most critical findings | ||
|
|
||
| Then **re-read the controller** (`.claude/skills/controller/SKILL.md`) for | ||
| next-step guidance. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use a repo-root-qualified controller path to prevent command breakage.
This relative path depends on current working directory and can fail outside
workflows/document-review/.Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents