Skip to content

v0.12.0: money-aware tool policies, testable policies and oversight reporting - #17

Merged
renezander030 merged 1 commit into
masterfrom
release/v0.12.0
Oct 15, 2026
Merged

renezander030 merged 1 commit into
masterfrom
release/v0.12.0

Conversation

@renezander030

Copy link
Copy Markdown
Owner

v0.12.0: money-aware tool policies, testable policies and oversight reporting

Bounds for money-shaped tool arguments, policies and pipelines you can test in CI, a preflight that checks the credentials themselves, and a report of who decided what.

Changes, in review order

# Change Why it matters
1 max_per_day on a numeric tool argument: a running total per tool argument per UTC day. A call that would pass the cap is handled like a mismatch (on_mismatch: deny refuses it as rule: max_per_day.<arg>, approve asks a human). Totals live in the state store and hold across restarts; capped decisions are made one at a time A per-call max cannot be bypassed by splitting one large amount into many allowed calls
2 draftcat gate check <tool> --args JSON and draftcat gate test cases.yaml: decide tool calls against tool_gate without the engine, with the same evaluation the endpoint uses. test exits 1 when a decision (or the deciding rule) differs from the case Policy changes are reviewed and tested in CI like code
3 draftcat doctor --online: read-only checks that Telegram accepts the token (getMe) and can reach the operator chat (getChat), the provider accepts the key, and the relay answers. Sends nothing and spends no tokens; credentials are redacted from every detail line A revoked token or wrong key shows up before go-live, not at the first approval
4 draftcat report: decisions per pipeline and step over a period, decisions by person vs. policy, per-operator counts, skip reasons, reported tool executions and receipt signature status, ending in a digest that matches the receipts of the same window. Markdown or --json; exits 1 on a tampered receipt A monthly record of human oversight for a client or an auditor
5 review_above on a numeric argument: values above it go to a human, even when the rule would allow the call. With max it gives three bands: allowed, asked, refused "Small invoices pass, medium ones are asked, large ones are refused" in one rule
6 fixtures/<pipeline>/_expect.json for draftcat test: assert the outcome, values in the final data, and text in approval drafts and operator messages. Unmet expectations are listed and the run exits 1; unknown keys are refused A prompt, skill or schema change that alters a pipeline's result fails in CI
7 budget status by model and by pipeline: each admitted model call records its provider model and pipeline in the admission row; status and --json (by_model, by_pipeline) split the day See which model and which pipeline the money goes to
8 tool_gate.notify_outcomes: failed|all: the operator hears what an allowed action did once the harness reports it, once per outcome An approved action that failed downstream does not go unnoticed
9 Halted runs name what did not run: the operator's error notice adds halted at: <step> and not run: <steps> It is clear which sends and writes did not happen

Prompts for capped tools show the running total (today: amount: 1200 allowed today, max_per_day 10000). Validation covers review_above (must be below max), max_per_day (not negative) and notify_outcomes. Guides: amount bands and daily totals, checking a policy offline, running Draftcat unattended, fixture expectations.

Compatibility

All new settings are optional, and configurations without them decide exactly as before. Rules without review_above or max_per_day keep their policy hashes, so permits bound before the upgrade still match (pinned by a test). The state store gains a tool_action_amounts table and model/pipeline columns on budget_calls at first start; budget status on a store the engine has not opened yet shows totals without the split. doctor without --online makes no network calls. The stored run error and the log keep each step's own error text; only the operator notice names the halted steps. budget status --json adds two arrays. Versions are synchronized to 0.12.0 in version.go, package.json and package-lock.json.

Verification

  • go build ./..., go vet ./..., gofmt: clean
  • go test -short -count=1 ./...: 16 packages ok; 495 top-level tests pass (469 on master), 0 failures
  • go test -count=1 ./... (non-short), go test -race -short ./..., go test -short -tags voice ./...: ok
  • golangci-lint run --new-from-rev=master (v2.5.0, repo config): 0 issues; the full run adds no finding to master's set
  • go run . validate: 0 errors; npm test: pass; npm pack --dry-run: draftcat-0.12.0.tgz
  • CLI help and usage of every existing subcommand unchanged
  • Local smoke with the built binary: two allowed calls of 400 and a third denied at max_per_day: 1000 through the live /gate/tool-call endpoint, then consume and a reported failure; report counts match; SIGTERM exits cleanly; no configured credential appears in the engine log. doctor --online with deliberately invalid credentials reports the Telegram token and the provider key as rejected without printing either

New tests cover the bands and daily totals (including revoked permits, restarts and several capped arguments), the store-less fail-closed path, the prompt text, gate check/gate test (pass, failing case, usage errors), doctor probes against a local Bot API and provider stand-in (accepted, rejected, transport errors redacted, no calls without --online), fixture expectations (pass, drift, skip outcome, unknown keys), the report (window, operators, skip reasons, digest, tampered receipt), outcome notices per mode with retry dedup, usage attribution (including older stores) and halted-run notes.

Review notes

  • Base: master @ 448be5a. No other open PRs.
  • Touched: tool gate decision path (now one policy evaluation shared with draftcat gate), tool rule constraints (internal/config), state store (amount totals, receipt queries, budget attribution), doctor, draftcat test, receipts verification helper, budget status, notices.
  • New files: tool_policy.go, gate_cmd.go, report_cmd.go, doctor_online.go, test_expect.go, halted_run.go, internal/state/{tool_amounts,report,budget_usage}.go.
  • Untouched: approval prompts and receipts format, the hitl/v0 protocol, webhook admission and signatures, scheduling, ZK/FHE commands.
  • Argument values under max_per_day are stored next to the action so totals can be summed; all other arguments remain hashed only.

…rsight reporting

- max_per_day on numeric tool arguments: a running total per tool
  argument per UTC day; a call past the cap is handled like a mismatch.
- review_above sends values above it to a human; with max it gives
  allowed, asked and refused bands.
- draftcat gate check|test decides tool calls offline with the same
  evaluation as the endpoint; test exits 1 when a decision differs.
- draftcat doctor --online confirms the Telegram token and chat, the
  provider key and relay reachability with read-only calls.
- draftcat report summarizes decisions by pipeline, step, person and
  policy, skip reasons, tool outcomes and receipt integrity.
- fixtures/<pipeline>/_expect.json turns draftcat test into a
  regression check.
- budget status splits the day by provider model and by pipeline.
- tool_gate.notify_outcomes tells the operator what allowed actions did.
- Error notices for a halted run name the steps that did not run.
@renezander030
renezander030 merged commit 0de5044 into master Oct 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant