[sjyangkevin]

Summary

• Why. The security skills and the security-management helpers ran Vulnogram's own commands (vulnogram-api-record-update, -record-fetch, …) and made decisions on Vulnogram's state names (state == REVIEW). tools/cve-tool/README.md already defined a tool-neutral interface, and the organization config already names the tool (cve_authority.tool), but no code implemented the interface or read the setting. A project on another CVE system had no way to plug in.
• What. A small cve-tool command now implements that interface. It reads cve_authority.tool, hands each call to that tool's adapter, and reports record states with the generic names (allocated, review-ready, publish-ready, public). Vulnogram is the only adapter and the ASF implementation; it calls the existing Vulnogram code unchanged. The helpers and skills now run cve-tool <operation> instead of Vulnogram's commands.
• For ASF projects nothing changes: the same Vulnogram calls run underneath, with the same flags, output and exit codes.

Closes #1800

Background

A CVE record goes through allocate → push updates → check state → publish. Two layers drive those steps:

• the skills (security-issue-sync, security-cve-allocate, security-issue-invalidate, security-issue-deduplicate): Markdown instructions the agent follows;
• the security-management helpers: Python that decides the next step and prints the exact command for the agent to run.

Before this PR, both named Vulnogram's commands directly, for example:

• tools/security-management/src/security_management/issue_sync/push.py hardcoded <framework>/tools/cve-tool-vulnogram/oauth-api and printed vulnogram-api-record-update …;
• skills/security-issue-sync/release-manager-comments.md gated the release-manager hand-off on state == REVIEW;
• skills/security-issue-sync/cve-record-comments.md pointed the reviewer-comment fetch at tools/<cve-tool>/oauth-api, a path that does not exist.

What changes

The PR has five commits, meant to be reviewed in order:

#	Commit	What it does
1	feat(cve-tool): add the cve-tool command…	New tools/cve-tool/ package. cve-tool [--tool NAME] <allocate|fetch|push|publish|retract|check|setup|which> [args…] resolves cve_authority.tool (--tool, then project.md, then the organization manifest, reusing release-config's reader) and passes the arguments to the adapter unchanged. The Vulnogram adapter calls the existing vulnogram_api.*.main() functions and translates states at the edge: fetch --state-only prints review-ready, not REVIEW, and publish --allow-state accepts generic states.
2	refactor(security-management): print cve-tool commands…	The helpers print cve-tool … commands, built in one place (shared/cve_tool.py). The post-push verify reads both the generic and the native state.
3	docs(security): run CVE-record steps through cve-tool…	issue-sync and cve-allocate run cve-tool and use the generic states. This is required, not cosmetic: fetch now prints review-ready, so a check written against REVIEW would no longer match. Vulnogram names stay only as ASF examples. The two step-5b eval fixtures follow the new commands.
4	docs(cve-tool): document the cve-tool command…	The contract README documents the command (operations, exit codes, adding an adapter), drops the stale "PR4 rename" notes, and corrects the consumer list. issue-deduplicate and issue-invalidate give a concrete cve-tool command where they named a contract method abstractly. The labels-and-capabilities row and the cve-tooling spec are updated.
5	test(evals): record the cross-model issue-sync run…	The journaled eval run and its generated run-history table.

How the operations map:

cve-tool	Contract method	Vulnogram adapter runs
allocate	allocate	vulnogram-api-allocate
fetch	fetch_current_state	vulnogram-api-record-fetch
push	push_update	vulnogram-api-record-update
publish	publish	vulnogram-api-record-publish
retract	retract	not implemented (exit 22, hint: reject in the Vulnogram UI)
check / setup	session probe / setup	vulnogram-api-check / vulnogram-api-setup

The interface adds three exit codes of its own, clear of Vulnogram's 0–7: 20 no tool declared, 21 the tool has no adapter, 22 the adapter lacks the operation.

What does not change

• ASF behaviour. Same Vulnogram calls, flags, output and exit codes underneath.
• The Vulnogram code (tools/cve-tool-vulnogram/oauth-api) is not modified.
• The CVE JSON generator (generate-cve-json) still emits Vulnogram-shaped JSON, including the native CNA_private.state. Making it tool-neutral is a separate step.
• Links to documents in tools/cve-tool-vulnogram/ (credit policies, templates) stay: they are documents the skills read, not commands they run.
• No new adapters (GHSA, MITRE form, CVE.org direct). This PR creates the slot they plug into.

Type of change

• Skill change (.claude/skills/<name>/) — eval fixtures updated below
• Tool / bridge contract (tools/<system>/*.md)
• Python package (tools/*/ with pyproject.toml)
• Groovy reference impl
• Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
• Documentation (docs/, README.md, CONTRIBUTING.md)
• Project template (projects/_template/)
• CI / dev loop (prek, workflows, validators)
• Other:

Test plan

• prek run --all-files passes
  • Run as the two pre-PR gates in AGENTS.md: prek run --all-files --skip workspace-pytest, then prek run workspace-pytest --from-ref origin/main --to-ref HEAD. The root pyproject.toml / uv.lock change makes the second one select all 54 workspace members. Both pass.
  • One local-only exception: check-claude-mods-local fails on ICON_MISSING for the root .claude-plugin/plugin.json with the locally installed Claude Code 2.1.323. This branch touches no file that hook watches.
• For Python packages touched: uv run pytest / ruff check / mypy passes
  • tools/cve-tool (new): 39 tests covering config lookup, dispatch and argument pass-through, exit-code pass-through, the 20/21/22 refusals, state mapping both ways, and every operation importing a real vulnogram_api entry point.
  • tools/security-management: 957 tests, with the expected commands updated and 3 new tests (generic fetched state, cve_tool_project path override).
  • tools/cve-tool-vulnogram/oauth-api: existing tests pass unchanged.
• For Groovy bridges touched: command-line invocation tested end-to-end
• For skill changes: eval suite passes for the affected skill
  (PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)

  • Cross-model run (Claude Sonnet 5.5 via claude -p) on the 46 security-issue-sync cases whose prompts include rewritten text, journaled in tools/skill-evals/evals/security-issue-sync/runs.jsonl:

    Step	Cases	Result
    step-5b-judged-gates	2	2 pass
    pointer-routing	10	10 pass
    pointer-routing-gather	7	7 pass
    step-1d-signals	7	7 pass
    step-2b-proposed-changes	12	11 pass (2 of them on re-run), 1 grader flake (below)
    step-2c-next-step	4	4 pass, including the reworded Step 13 next step
    step-handoff-delegate	4	4 pass

  • step-2b/case-5-pre-announce-distributors failed both runs on a grader "NO" whose stated reason says the candidate matches on the decision. On main (8d24b524) the same case failed once and passed once, and this branch changes no text it depends on (distributor pre-announcement). case-2 (a self-contradicting grader verdict) and case-6 (the claude CLI exiting before any output) passed on re-run.

  • Agent self-eval of step-5b-judged-gates: 2/2 match.

• For skill behaviour changes: a new or updated eval fixture is included in this PR
  (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
  • step-5b-judged-gates case-4 and case-5 now show the cve-tool check / push / fetch --state-only commands and the generic review-ready state the skill sees after this change.
• Other:
  • End to end against the real Vulnogram code, no network:
    • cve-tool --tool vulnogram check --credentials <missing file> → not-configured, exit 2 (Vulnogram's own);
    • cve-tool --tool vulnogram fetch --cve-id not-a-cve --state-only → Vulnogram's own refusal, exit 1;
    • cve-tool which resolves vulnogram for an ASF project and mitre-form for an independent one; mitre-form then exits 21, and retract exits 22.
  • No direct Vulnogram command call remains in the skills or in security-management/src (grep for vulnogram-api-; only adapter-doc links remain).
  • skill-token-count --write restamped security-cve-allocate, security-issue-deduplicate and security-issue-invalidate.
  • The cve-tooling spec gains acceptance criterion 10 for the command, and its validation block runs the new tests.

RFC-AI-0004 compliance

• HITL — any new mutation is gated on explicit user confirmation
• Sandbox — no new unrestricted host access; network reach declared in the adapter
• Vendor neutrality — placeholders (<PROJECT>, <tracker>, <upstream>, <security-list>) used in all skill / tool prose (the check-placeholders prek hook is the mechanical gate)
• Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
• Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
• Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Closes #1800

• Generated files, safe to skip: plugins/magpie-security-tools/tools/** (the mirror check-family-plugins.py --fix writes), uv.lock, the measured_tokens: stamps, and the eval run-history table. Hand-written diff: 41 files, +1041 / −193.

• Arguments pass through unchanged, so adapter-specific flags (--pmc, --comments-only) still appear in the skills. A tool-neutral flag set can come later without touching the skills' structure.

• One behaviour change outside the ASF. A project inheriting mitre-form (the independent default) used to be handed Vulnogram commands that could not work for it. It now gets exit 21, and the skill takes its manual path.

• cve_allocate/render.py now offers the source-tab paste fallback whenever source_tab_url_template is set, rather than only for Vulnogram. Only the Vulnogram config sets that template, so the output is the same today.

• tools/cve-tool depends on release-config to reuse its project → organization config reader rather than add a second parser. Both are stdlib-only.

• Known follow-ups, out of scope:

  • Vulnogram's adapter has no retract.
  • generate-cve-json is still Vulnogram-shaped.
  • The ASF allow-list recommendation Bash(vulnogram-api-record-fetch *) in permission-audit and setup verify matches neither the old nor the new uv run … spelling the skills use.

  The first and third are recorded under Known gaps in tools/spec-loop/specs/cve-tooling.md.

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Opus 5.5)

Generated-by: Claude Code (Opus 5.5)

🤖 Generated with Claude Code

[Kaap10]

This cleanly implements the tool-neutral CVE interface behind cve-tool and decouples the security skills and helpers from hardcoded Vulnogram states (REVIEW, READY) into generic contract states (review-ready, publish-ready). The config resolution reuse through release-config's reader is solid, and dual-tree parity between tools/ and plugins/magpie-security-tools is fully maintained.

A few technical observations on edge-case behavior and command construction below.

Smaller observations

• See inline comments on tools/cve-tool/src/cve_tool/adapters/vulnogram.py:99, tools/cve-tool/src/cve_tool/cli.py:114, and tools/security-management/src/security_management/shared/cve_tool.py:38.

[Kaap10]

Edge-case handling on unexpected stdout text

vulnogram-api-record-fetch --state-only prints the raw state string on stdout. If vulnogram_api.record_fetch.main() exits 0 but prints warning lines or logging along with the state (e.g. UserWarning: ...\nREVIEW), text.strip() evaluates to the entire multi-line string. In that case, generic_state(text.strip()) falls back to "unknown", which will cause push.py's verify step to fail closed and block handoff (fetched == 'unknown').

Consider extracting text.strip().splitlines()[-1] or testing that generic_state() can handle leading warning lines, so transient warnings on a successful fetch don't misclassify a valid state as unknown.

[sjyangkevin]

Thanks for checking this. After running some tests, on the --state-only path, record_fetch writes only the state to stdout. Its diagnostics go to stderr, as Python warnings and logging do by default, and stderr is not captured, so a successful fetch prints exactly one line.

If that ever changes, the result is unknown and push's verify step stops and hands off. That is the fail-closed outcome we want, so I'd rather keep it than read the last line, which would trust whatever happened to be printed last.

[Kaap10]

Makes total sense - agree that fail-closed is the right posture here rather than guessing. Thanks for clarifying!

[Kaap10]

Pass-through options ordering with argparse.REMAINDER

Because args uses nargs=argparse.REMAINDER, any flags intended for cve-tool itself (like --project-root or --config-dir) that appear after the operation name are treated as adapter arguments rather than cve-tool arguments.

For example, cve-tool fetch --project-root <dir> --cve-id <id> will pass --project-root directly to vulnogram_api.record_fetch, which does not recognize the flag and exits 2. This is documented in the docstring ("Everything after the operation goes to the adapter unchanged"), but it might be worth mentioning in tools/cve-tool/README.md under usage so callers know global options must precede <operation>.

[sjyangkevin]

Thanks for catching it. Rather than only documenting it, cve-tool now refuses --tool, --project-root and --config-dir after the operation with a usage error (exit 2) that names the option, before the adapter runs. That also fixes cve-tool which --project-root DIR, which used to ignore DIR. The README documents the order and reserves the three names for adapters.

[Kaap10]

Shell-quoting expectation for *args

w(project) correctly quotes the project path, but *args is joined raw with spaces. Callers in push.py properly quote their arguments before passing (e.g., w(cve_id) and w(json_file)), but if future callers pass unquoted paths with spaces (e.g. --json-file path with spaces.json), the generated bash command will split unexpectedly.

Either adding w(a) mapping over args or keeping an explicit type annotation/docstring note that args must already be shell-quoted will prevent bugs when new cve-tool operations are wired in.

[sjyangkevin]

Agreed, thanks. command() now quotes every argument itself, and the callers pass raw values. It uses shlex.quote rather than w(), because the allocation title comes from the tracker and w() leaves a <placeholder>-shaped value bare.