feat(review): tools for labeling using Claude Managed Agents

.github/CODEOWNERS

@@ -35,6 +35,10 @@
 /REVIEWERS.md @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
 /tasks/scripts/check_review_readiness.py @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
 /tasks/scripts/check_review_readiness_test.py @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
+/tasks/scripts/managed_label_classifier.py @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
+/tasks/scripts/managed_label_classifier_test.py @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
+/tasks/scripts/labeling_agent_events.py @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
+/tasks/scripts/labeling_agent_events_test.py @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr
 
 # PR issue acceptance checks
 /tasks/scripts/check_issue_acceptance.cjs @NVIDIA/openshell-codeowners @mrunalp @sjenning @derekwaynecarr

.github/workflows/branch-checks.yml

@@ -605,6 +605,9 @@ jobs:
 
       - name: Test release qualification summary
         run: mise run test:qualification-summary
+
+      - name: Test labeling agent runtime
+        run: mise run test:labeling-agent
   sdk-go-freeze:
     # The Go SDK moved to ai-openshell/sdk-go (#5031). Its directory stays only
     # until the final deprecated sdk/go tag exists, so PRs must not add or change

tasks/scripts/README-labeling-agent.md

@@ -0,0 +1,106 @@
+# Labeling agent runtime integration
+
+These modules start Part 3 of [#5276](https://github.com/NVIDIA/OpenShell/issues/5276):
+select invocations and return an unvalidated JSON classification proposal. The
+shared `/label-issue` helper from [#5355](https://github.com/NVIDIA/OpenShell/pull/5355)
+owns eligibility, evidence, validation, label writes, checkpoints, and local
+reporting. No automatic workflow is installed.
+
+## Execution boundary
+
+Select an invocation with `labeling_agent_events.select_invocation`, then call the
+shared helper's preparation. Only `status: ready` with `classifier_required: true`
+creates a model session. Pass the prepared evidence and trusted classifier
+instructions to `managed_label_classifier.session_request`, call `classify`, and
+return its proposal to the shared preview/application helper. Follow the shared
+[command/result contract](https://github.com/NVIDIA/OpenShell/blob/70fa003b902416459b2c08131b5cc265294d35d6/.agents/skills/label-issue/references/result-contract.md);
+this adapter checks JSON shape, not classification validity.
+
+Include the result schema and referenced classification guidance in the trusted
+instructions: the runtime cannot fetch referenced files or expand evidence.
+The caller must handle required evidence expansion through the shared helper.
+Do not silently treat missing or partial evidence as complete.
+
+Trusted instructions become the session system prompt; evidence becomes the user
+message. Session overrides disable tools, MCP servers, and attached skills without
+changing the saved agent. The adapter supplies no GitHub credentials, repository
+mounts, or vaults.
+
+Use a dedicated single-agent CMA configuration and an environment without
+preinstalled secrets. Start with a Sonnet-level model; the adapter pins the agent
+version rather than choosing a model. Verify overrides in a live smoke test before
+enabling automation.
+
+## Smoke-test CLI
+
+This temporary harness previews or submits prepared inputs. Consolidate it into
+the real entry point when the shared-helper integration is available.
+
+```shell
+uv run --no-project python tasks/scripts/managed_label_classifier.py \
+  --instructions /path/to/trusted-classifier-instructions.md \
+  --evidence /path/to/prepared-evidence.json \
+  --agent-id agent_CONFIGURED_ID --agent-version 1 \
+  --environment-id env_CONFIGURED_ID
+```
+
+By default it prints the request without API calls or credentials. Output includes
+item evidence; keep it in the invoking session or appropriate artifacts.
+
+Add `--run` with `ANTHROPIC_API_KEY` in the environment to create a billable
+session. Never include credentials in evidence. Defaults are a 25-US-cent
+list-price ceiling (`--budget-cents`) and a 180-second deadline (`--timeout`).
+The result contains `session_id` and `proposal`; validate the proposal through
+the shared helper. Ambiguity belongs in local reports, never item comments.
+
+## Completion and recovery
+
+Success requires JSON `agent.message` followed by idle `end_turn`. Failure or
+timeout archives the session; archival failure reports its ID. Successful
+sessions remain available for review. Session creation is never retried
+automatically after an uncertain response. Polling is bounded and error bodies
+are suppressed. See the [session event API](https://platform.claude.com/docs/en/managed-agents/events-and-streaming).
+
+## Invocation selection
+
+`select_invocation` returns `Invocation(repository, number, force, source)` or no
+invocation:
+
+- `issues` / `pull_request_target`: `labeled` with `state:accepted`, including
+  automation-applied labels and drafts.
+- `issue_comment`: a new human PR comment containing only `/label-issue`,
+  `/label-issue force`, or `/label-issue --force`.
+- `workflow_dispatch`: a same-repository issue/PR number or URL and a Boolean
+  force input.
+
+Human commands require current write, maintain, or admin permission. Supply a live
+permission lookup that propagates failures; author association is insufficient.
+Force bypasses only unchanged detection in the shared helper. Other label and
+lifecycle events do not dispatch the classifier.
+
+## Pending wiring
+
+After observed skill runs succeed, connect these modules using #5276's integration
+plan and trusted default-branch code. All entry points need coordinated per-item
+execution; #5355's local lock does not serialize separate hosts.
+
+[#5260](https://github.com/NVIDIA/OpenShell/pull/5260) writes acceptance using
+`GITHUB_TOKEN`; its label event will not start another workflow. Invoke the shared
+entry point explicitly after a successful acceptance-label write, including for
+drafts, independently of later reviewer-routing success. An existing acceptance
+label alone is not current eligibility: #5260 does not refresh it. Use the shared
+helper's preflight, which checks the item's author rather than the triggering
+actor and propagates lookup failures.
+
+Remaining prerequisites are the shared interfaces, reviewed CMA configuration,
+acceptance-writer hook, label preparation, and observed end-to-end runs. See #5355
+for canonical-label and `meta:labeler-checkpoint` prerequisites.
+
+## Verification
+
+```shell
+mise run test:labeling-agent
+```
+
+Offline tests fake CMA and permissions and perform no remote writes. Live
+configuration and model quality require observed runs.

tasks/scripts/labeling_agent_events.py

@@ -0,0 +1,105 @@
+"""Select explicit acceptance and authorized human labeling-agent invocations."""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from urllib.parse import urlsplit
+
+REPOSITORY = re.compile(r"[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+")
+COMMENT_COMMAND = re.compile(r"/label-issue(?:[ \t]+(force|--force))?")
+HUMAN_PERMISSIONS = {"write", "maintain", "admin"}
+
+
+@dataclass(frozen=True)
+class Invocation:
+    repository: str
+    number: int
+    force: bool
+    source: str
+
+
+def item_number(item: str, repository: str) -> int:
+    """Resolve one number or same-repository issue/PR URL without network access."""
+    if not REPOSITORY.fullmatch(repository):
+        raise ValueError("expected an OWNER/REPO repository")
+    if re.fullmatch(r"[1-9][0-9]*", item):
+        return int(item)
+    url = urlsplit(item)
+    if (
+        url.scheme != "https"
+        or url.netloc != "github.com"
+        or url.query
+        or url.fragment
+    ):
+        raise ValueError("expected an item number or a same-repository GitHub URL")
+    match = re.fullmatch(r"/([^/]+/[^/]+)/(?:issues|pull)/([1-9][0-9]*)/?", url.path)
+    if not match or match[1].casefold() != repository.casefold():
+        raise ValueError("item must belong to the configured repository")
+    return int(match[2])
+
+
+def select_invocation(
+    event_name: str,
+    event: dict,
+    repository: str,
+    *,
+    permission,
+) -> Invocation | None:
+    """Route events only; the shared preflight must still verify current eligibility.
+
+    permission(login) returns the live repository permission or raises on lookup
+    failure. An event payload's author association is not an authorization check.
+    """
+    if not REPOSITORY.fullmatch(repository):
+        raise ValueError("expected an OWNER/REPO repository")
+    actual_repository = (event.get("repository") or {}).get("full_name", "")
+    if actual_repository.casefold() != repository.casefold():
+        raise ValueError("event repository does not match the configured repository")
+    if (
+        event_name in {"issues", "pull_request_target"}
+        and event.get("action") == "labeled"
+        and (event.get("label") or {}).get("name") == "state:accepted"
+    ):
+        item = event.get("issue" if event_name == "issues" else "pull_request") or {}
+        number = item.get("number")
+        if not isinstance(number, int) or isinstance(number, bool) or number < 1:
+            raise ValueError("acceptance event has no valid item number")
+        return Invocation(repository, number, False, "acceptance")
+    if event_name not in {"issue_comment", "workflow_dispatch"}:
+        return None
+    sender = event.get("sender") or {}
+    login = sender.get("login")
+    if sender.get("type") != "User" or not isinstance(login, str) or not login:
+        return None
+    if event_name == "issue_comment":
+        comment = event.get("comment") or {}
+        author = comment.get("user") or {}
+        issue = event.get("issue") or {}
+        if (
+            event.get("action") != "created"
+            or not issue.get("pull_request")
+            or author.get("type") != "User"
+            or author.get("login") != login
+        ):
+            return None
+        command = COMMENT_COMMAND.fullmatch((comment.get("body") or "").strip())
+        if not command:
+            return None
+        number = issue.get("number")
+        if not isinstance(number, int) or isinstance(number, bool) or number < 1:
+            raise ValueError("comment event has no valid PR number")
+        force = command[1] is not None
+    else:
+        inputs = event.get("inputs") or {}
+        number = item_number(str(inputs.get("item", "")), repository)
+        force_input = inputs.get("force", False)
+        if not (
+            isinstance(force_input, bool)
+            or isinstance(force_input, str) and force_input in {"true", "false"}
+        ):
+            raise ValueError("force must be true or false")
+        force = force_input is True or force_input == "true"
+    if permission(login) not in HUMAN_PERMISSIONS:
+        return None
+    return Invocation(repository, number, force, "human")