fix: require capability preflight and complete trial evidence

skills/process-engine-pattern-author/SKILL.md

@@ -104,6 +104,19 @@ handed to review.
 7. **Acceptance criteria** — write explicit exit criteria into each artifact,
    derived from the objective: the vision of "good" becomes observable exit
    criteria.
+
+   **Executable capability path.** When generated procedures depend on tools
+   or runtime capabilities, identify the intended executable capability path
+   and keep package declarations internally consistent with it. Conceptually:
+   procedure → required capability → declared/expected runtime tool path.
+   The path must be coherent with the persona/tool declarations and **any
+   skill-level `allowed-tools` constraints when present** (Agent Skills
+   treats `allowed-tools` as optional/experimental — do not require every
+   generated skill to add it; the reasoning must also work when a skill
+   legitimately omits that field). This is package-design reasoning, not a
+   separate manifest/artifact. Do not prescribe specific tools or mechanisms
+   (shell, MCP, wrappers, reloads) — require coherence, not implementation
+   doctrine.
 8. **Assemble the package** — project/persona/skills/templates/governance as
    one bundle; skill folders mirror the spec layout (SKILL.md + references/ +
    optional scripts/ and assets/) when shipped as repo content.
@@ -148,4 +161,5 @@ handed to review.
 - [ ] Identity-critical relationships preserved (entity ↔ role ↔ identifiers) where they affect decisions or tool targets; conflicts kept visible until disambiguated; no silent equivalence inference
 - [ ] Governance artifacts: prompt policy + advisory judge rules (Turnstone-native)
 - [ ] Acceptance criteria written
+- [ ] Executable capability path identified where procedures depend on tools/capabilities; package declarations internally consistent with it (no prescribed tool/mechanism doctrine)
 - [ ] Marked DRAFT, handed to review

skills/process-engine-review/SKILL.md

@@ -22,8 +22,33 @@ the gate.
 2. **Spec compliance check**: validate generated skills against the Agent
    Skills open standard — name lowercase-hyphen ≤64 matching the directory,
    description ≤1024 with triggering language, allowed frontmatter fields only,
-   SKILL.md present. Use the native parse endpoint (POST /v1/api/admin/skills/
-   parse) via Turnstone's native skills API.
+   SKILL.md present. Two separate evidence steps:
+
+   **A. Native parse evidence.** For each generated SKILL.md, preserve
+   evidence that the native Turnstone parse operation actually ran: skill
+   identity, native parse invoked, result/success or failure. Review cannot
+   PASS a generated SKILL.md without that recorded receipt.
+
+   **B. Allowed-field comparison.** Separately, compare the generated/authored
+   frontmatter keys against the current Agent Skills allowed-field contract
+   (`name`, `description`, `license`, `compatibility`, `metadata`,
+   `allowed-tools`). If any unexpected/non-spec key exists, REVISE — whether
+   or not the native parser accepted/parsed the document. Parse success does
+   not by itself prove that every frontmatter key is spec-valid.
+
+   Turnstone-specific deployment metadata: use a supported native Turnstone
+   object/API field when one exists; if information legitimately belongs in
+   the portable SKILL.md, use a valid Agent Skills field such as `metadata`
+   when appropriate; otherwise do not invent a SKILL.md frontmatter field and
+   do not hallucinate a Turnstone API field.
+
+   **Capability-path consistency**: do procedures that require tools or
+   runtime capabilities have an executable path consistent with the generated
+   package's persona/tool declarations (procedure → required capability →
+   declared/expected runtime tool path)? If a referenced operation has no
+   executable declaration in the package, REVISE. Review checks internal
+   package coherence here; it does not prove runtime availability — that is
+   Trial's job.
 3. **Safeguard review** (risk-relevant intents only): per-package safeguards
    present, evidence-named, sized to the domain — no preset doctrine
    (references/safety.md).
@@ -81,6 +106,9 @@ the gate.
 ## Verification
 - [ ] Standards checklist completed
 - [ ] Spec compliance checked (frontmatter, name/description rules)
+- [ ] Native parse evidence recorded in Review (skill identity, parse invoked, result)
+- [ ] Allowed-field comparison performed against Agent Skills contract; no unexpected/non-spec frontmatter keys present (REVISE if present, regardless of parse result)
+- [ ] Capability-path consistency checked for procedures requiring tools/capabilities (REVISE if no executable declaration)
 - [ ] Safeguard review completed (risk-relevant intents only)
 - [ ] Coverage check completed against the catalog
 - [ ] Verdict recorded with evidence

skills/process-engine-trial/SKILL.md

@@ -22,6 +22,22 @@ per-package spec.
 - Regression: re-run cases after any change.
 
 ## Core Process
+0. **Determine applicable categories before running.** Identify which trial
+   categories apply to this package and record the set: acceptance-criteria /
+   happy-path cases; gray-zone / boundary / escalation cases; identity case
+   when target identity is material; trigger/activation set when the package
+   is activation-dependent; safeguard-specific drills when risk-relevant.
+   A concise trial-evidence summary is sufficient — no persisted state
+   manifest.
+
+0a. **Capability preflight.** Before behavioral cases that depend on runtime
+   capabilities, verify the required capability path is actually available in
+   the target Trial runtime. If package declarations are internally wrong,
+   that should have been caught at Review (REVISE). If the package declaration
+   is correct but the runtime capability is unavailable/stale, classify the
+   overall Trial as INCOMPLETE — do not score the behavioral case FAIL — and
+   report the responsible runtime/tool layer if known.
+
 1. **Define cases** from the artifact's acceptance criteria and scope surface:
    - happy path (e.g. normal use of the package)
    - gray zone (e.g. input near the scope boundary — must be handled per
@@ -45,11 +61,24 @@ per-package spec.
 2. **Define fixtures** — setup stubs: model config, persona, skill, scenario
    input. Emit case sets in a structured format (id, prompt, expected_output).
 3. **Run** each case with clean context; record actual vs expected result.
-4. **Score**: PASS / FAIL per case; document failures precisely. Activation
+4. **Score** per case: PASS / FAIL; document failures precisely. Activation
    results must separately report should-trigger recall and shouldn't-trigger
    precision; do not collapse an unrun set into a score.
-5. **Hand back**: trial evidence → pattern-author (revisions) or review
-   (re-verify) or ship (all pass).
+5. **Overall Trial verdict** — exactly one of:
+   - **PASS**: every applicable required category has direct evidence and all
+     required executed cases pass.
+   - **INCOMPLETE**: one or more required categories are unrun, blocked,
+     unavailable, missing direct evidence, preflight-unavailable, or
+     otherwise unproven. INCOMPLETE does not advance to Ship.
+   - **FAIL**: a required executed case demonstrates behavior that violates
+     the package's acceptance criteria/scope. FAIL returns through
+     revision/review.
+   Preserve granular case-level PASS/FAIL — the point is to distinguish
+   "10/10 executed behavioral cases PASS" from "overall Trial PASS" when
+   another required category remains unrun.
+6. **Hand back**: only overall PASS may hand off to Ship. INCOMPLETE remains
+   in Trial until evidence is complete. FAIL routes to pattern-author
+   (revisions) or review (re-verify).
 
 ## Examples
 - Case: out-of-scope request → input outside the package's declared domain →
@@ -74,8 +103,11 @@ per-package spec.
 
 ## Verification
 - [ ] Cases defined for all acceptance criteria + scope surface
+- [ ] Applicable trial categories determined before running (behavioral, boundary/adversarial, identity where material, trigger/activation where activation-dependent, safeguard where risk-relevant)
+- [ ] Capability preflight performed for runtime-dependent cases (unavailable → overall INCOMPLETE, not behavioral FAIL)
 - [ ] Identity/alias/near-match case included where identity is material to behavior
 - [ ] Trigger set run (where package has activation-dependent skills)
 - [ ] Every case run, actual vs expected recorded
-- [ ] All PASS (or failures routed back to author/review)
+- [ ] Overall Trial verdict recorded: PASS (all applicable categories evidenced) / INCOMPLETE (category unrun/unproven) / FAIL (required case violates acceptance criteria)
+- [ ] Only overall PASS handed to Ship
 - [ ] Trial evidence recorded in the project