Skip to content

docs(adr): propose ADR-0061, ship gh-aw shared components as a staged advisory surface - #340

Open
mbeacom wants to merge 3 commits into
mainfrom
add-gh-aw
Open

mbeacom wants to merge 3 commits into
mainfrom
add-gh-aw

Conversation

@mbeacom

@mbeacom mbeacom commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

I'm proposing ADR-0061: ship adrkit's decision review as a set of GitHub Agentic Workflows (gh-aw) shared components. Adopting it would take one imports: line plus a pinned CLI and image. The model-backed review would run inside gh-aw's sandbox and firewall and spend gh-aw's AI credits, and it stays staged until calibrated, as ADR-0027 requires.

The record is proposed and conditional. Nothing under packages/adapters/gh-aw/ exists yet, and none should until it is ratified.

What the record decides

  • Location and versioning. The components live in packages/adapters/gh-aw/, versioned independently (ADR-0007, ADR-0030) on a gh-aw-v<semver> tag. The image is pinned by digest and the CLI and gh-aw compiler by exact version. CI diffs the compiled lock.
  • Default capabilities. The CLI on the host, a precomputed adr check --json, and a read-only checkout. The MCP server and the agent plugin are opt-in and off by default.
  • Safe outputs. The workflow may post a COMMENT review, review comments up to a cap, and a custom check run whose conclusion is hard-coded to neutral. Every verb that approves, requests changes, merges, labels or opens an issue is forbidden.
  • The gate. adr lint runs, then adr check. The agent runs only when something governs the change, and the gate fails closed to unverified.
  • Calibration before anything posts. The workflow is staged by default and runs unstaged only in adrkit-demo. The holdout is frozen before the first score, with at least 30 labeled conflicts and 90 other cases. Each must clear a Wilson lower bound (recall ≥0.70, precision ≥0.75) plus a cost and usefulness bar.
  • Trust posture. It runs on pull_request only, with no fork opt-in, and is never a required check.

Evidence so far (rung 1, plus a partial rung-2 smoke)

The spike is mbeacom/adrkit-demo#9, compiled with gh-aw v1.0.1 --strict.

  • I ran five staged fixture PRs against it. The gate skipped the README-only change and ran on the four governed ones.
  • All four verdicts matched the labels I wrote down before any run. That includes a superseded and a proposed record, both kept as context and never judged.
  • Nothing was posted.
  • The agent never called the MCP server or the plugin, so I rebuilt the spike without either and ran the same five fixtures again (Restrict CI workflow GITHUB_TOKEN permissions #15–feat: add read-only MCP server #19). All four verdicts matched again, every record citation used the real filename instead of a glob, and nothing was posted.
  • Dropping that context didn't make it cheaper: the median went from 8.11 to 8.35 AI credits, and the same change cost 7.52 in one round and 13.50 in the other. The record now sets the cost threshold from 8.35, at 16.7 credits.

What is still open, listed as action items in the record:

  • Remote subpath imports: and unstaged gh aw logs --outcomes haven't been measured.
  • The new partial gate summary hasn't fired yet, because the demo corpus has no package or entity matcher.

Notes for review

… advisory surface

Proposes GitHub Agentic Workflows shared components under packages/adapters/gh-aw/ as an advisory distribution surface for the decision review, staged until calibrated per ADR-0027. The decision is conditional on the spike in mbeacom/adrkit-demo#9; two of its four load-bearing measurements are in, and the record lists the rest as action items.

Signed-off-by: Mark Beacom <m@beacom.dev>
Copilot AI balanced review requested due to automatic review settings November 14, 2026 15:02
@mbeacom mbeacom self-assigned this Nov 14, 2026
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Decisions governing this change

2 accepted decisions govern this change

  • 0001 — Record architecture decisions as versioned markdown in git
    • via path: docs/adr/**
  • 0040 — Keep derived surfaces in lockstep with three mechanisms matched to three classes of drift
    • via path: MANIFEST.md

Links point at 4e5be44, this run's GITHUB_SHA. Run adr explain <path> locally to see why a file is governed.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The proposed gate can silently skip unresolved review inputs, and the ADR overstates containment of a PR-modifiable workflow that can publish required check contexts.

2 open issues
What changed in this PR

Proposes ADR-0061 for an independently versioned GitHub Agentic Workflows adapter that would extend adrkit’s advisory decision review into CI. This is documentation only; the proposal remains conditional and unratified.

Changes:

  • Defines component placement, version pins, default capabilities, output restrictions, and the deterministic activation gate.
  • Establishes staged defaults, calibration thresholds, spike evidence, and remaining validation work.
  • Regenerates the manifest to include ADR-0061 and update corpus totals.

Full ADR compliance was not verified; the adrkit MCP server was unavailable.

File Description
MANIFEST.md Updates the generated inventory to 60 records and adds proposed ADR-0061.
docs/​adr/​0061-ship-github-agentic-workflows-shared-components-as-an-advisory-distribution-surf.md Proposes the gh-aw distribution surface, safeguards, calibration requirements, and ratification conditions.

Review effort: Balanced

Comment on lines +273 to +276
`governs=true` only when lint exits `0`, check exits `0` or `1`, and
`.governing | length > 0`; otherwise `governs=false`. When lint or check
cannot complete, the step writes `unverified` with the exit code to the job
summary instead of skipping silently (ADR-0038). The agent job runs only on

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and confirmed against the code: check builds no snapshots, so package/entity matchers come back affects-unresolvable at info, and a capped marker scan leaves ok true. Fixed in b6a68c0. Clause 6 now writes partial to the job summary, with the inert record ids and the cap counts, whenever affects-unresolvable, marker-scan-capped, or marker-declarations-capped appears. It keys on those rule ids, not on severity. governs still follows .governing, so the exit-code contract doesn't change. Gate fixtures for each case are listed as tests to land, and the spike rebuild (action item 3) picks the check up too.

Comment on lines +327 to +329
It is advisory under ADR-0035: never a required check, so a same-repo PR that
edits its own workflow (fact 7) can at most change advisory output on its own
PR. The governing-decisions Action remains the CI authority (ADR-0045), and

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, the containment sentence overclaimed. Required checks match by name, and a PR that edits the job can use checks: write to publish any name. Fixed in b6a68c0. Clause 8 now states that residual risk and names workflow-path protection as the real control (ADR-0042 here). It also requires the consumer docs to ask for an equivalent ruleset or code-owner review on .github/workflows/**. I added it to Trade-offs as well.

…-0061

adr check passes no dependency or catalog snapshot and caps marker scans without changing ok, so the gate now writes partial to the job summary on affects-unresolvable, marker-scan-capped, or marker-declarations-capped, keyed on rule ids. Clause 8 drops the claim that an advisory check confines a PR that edits its own workflow: checks: write can publish any check name, so the record states the residual risk and requires workflow-path protection in consumer docs.

Signed-off-by: Mark Beacom <m@beacom.dev>
Adds fact 9: the spike rebuilt without the MCP server or the plugin reached the same four verdicts, cited real record filenames, and posted nothing, at a median 8.35 agent credits against 8.11 before. The cost baseline moves to 8.35 (threshold 16.7), action item 3 is done, and the partial-gate path is listed as unexercised because the demo corpus has no package or entity matcher.

Signed-off-by: Mark Beacom <m@beacom.dev>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants