Skip to content

Downstream trial: OpenClaw Maintenance package - dual-target MCP management + skill-collision engine data (2026-08-10) #11

Open

Description

@RedEyeNinja-BKK

Downstream trial: OpenClaw Maintenance package — dual-target MCP management (2026-08-10)

Type: Downstream product-trial case study / engine-eval data (same pattern as prior TrueNAS Manager / Proxmox Manager runs).
Package: openclaw-maintenance-2026-08-10 — generated by Process Engine v1.9.6 (Pattern → Author → Review-prep → Trial-plan), currently DRAFT, not published.
Scope: Managing BOTH OpenClaw instances in a 3-agent stack through MCP only (no SSH): a LOCAL OpenClaw reached via one MCP gateway, and a REMOTE OpenClaw on a distant host reached via a second MCP gateway. Persona: "OpenClaw Manager". Skill family: "OpenClaw Management" (parent) + 6 children.


1. Proven capability boundary (from live testing, prior to PE engagement)

  • Single-command exec over MCP works on both gateways (openclaw --version, npm --version, openclaw update status --json). Update-status read returns full JSON; the remote is current, so no update was applied (correctly no-op).
  • Multi-command compound tasks can self-block under exec allowlist mode on the REMOTE gateway; the safe operating unit is one command at a time (matches local behavior).
  • Admin-RPC surface is parity across both gateways: methods supported on one are supported on the other; unsupported methods fail cleanly with HTTP 400 "method is not supported" on both (never hang).
  • Silent exec-approval prompts are a real failure mode: an un-approved command blocks until approved in the remote UI; tasks look hung. Pre-authorization (allowlist) or operator notification is required for multi-step flows.
  • Config value validation is critical: tools.exec.mode: "allow" is INVALID in the tested runtime — gateway exits with code 78 and its error message enumerates valid values (deny, allowlist, … full). Allowlist matching uses resolved paths or node-command hashes; bare binary paths silently fail to match.

2. What Process Engine produced (observed behavior)

  • Complete standards-shaped package: manifest, persona, parent skill + 6 child skills (spec-valid SKILL.md), advisory prompt policy, advisory judge rules, creation checklist, scripted 8-case trial plan, process report. No failure in generation.
  • Skill-name collision observation (important engine data): an existing skill named Openclaw Management (lowercase c, v1.0.0) already exists. PE did not refuse, edit, disable, or duplicate it. It created the new package in draft filesystem form with a spec-safe frontmatter name (openclaw-management) and the requested title ("OpenClaw Management"), and recorded the collision as ASSUMED with a proposed operator-gated, reversible migration.
  • Engine gap identified: the PE author stage could not inspect the exact existing skill object (catalog read capability not exposed to the generator task), so it could not prove runtime collision resolution (supersede vs duplicate vs refuse). Turnstone upgraded the observation to PROVEN afterward via a native skill-get. Recommendation: consider granting the author stage catalog inspection or a capability preflight so collision handling is proven at author time, not deferred to deploy time.

3. Trial results (read-only, live probes, all PASS)

  • T1 health (both gateways) PASS; T2 config.get (both) PASS; T3 cron.list (both; LOCAL 1 job, REMOTE 7 jobs) PASS; T4 single-command read-only exec (both) PASS; T5 admin-RPC parity / unsupported-method handling PASS; T6 collision handling (non-destructive draft) PASS with deploy half gated; T7 gated-update design PASS (no live mutation, correctly); T8 invalid exec.mode / resolved-path allowlist handling PASS.
  • No mutation performed; no SSH; no credentials exposed; package remains DRAFT.

4. Limitations / open items

  • Catalog supersession of the old lowercase-c skill is still an operator-gated deploy-time action; deploy-time collision resolution is UNPROVEN.
  • Package hash deferred to freeze (PE chose not to record a draft hash — good practice).
  • Live update flow not executed (correctly gated; no update available at time of test).

5. Ask for the reviewer

Please review the collision-handling gap (author-stage catalog inspection) and whether the engine should encode "prove-by-live-test" (capability = live proof, not config reading) and "validate config values against runtime enum before apply" as first-class standards. Detailed evidence lives in the private workspace (sanitized summary above); happy to expand any section.

Activity

  1. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    GitHub issue #11 — ship-evidence update comment (ready to post)

    How to post (operator): on #11 click "Comment" and paste the body below. Or, if you prefer, authorize a PAT with issues: write / repo scope and I'll post it via API.


    UPDATE (2026-08-10) — SHIPPED: package deployed to Turnstone and verified (full evidence loop)

    The OpenClaw Maintenance package has been fully deployed onto Turnstone and verified by read-back, completing the trial's evidence loop. Details below (sanitized; role-based host references only).

    Deployed artifacts (read-back verified)

    Artifact Result
    Persona "OpenClaw Manager" Created, enabled, MCP + memory on, tool allowlist = openclaw-gateway, openclaw-remote-gateway, turnstone-api. Verified present in personas list.
    Parent skill openclaw-management Created, enabled, risk_level safe (scanner-assigned).
    6 child skills (access-connectivity, health-observability, update-maintenance, exec-approval, troubleshooting, verification) Created, enabled, all risk_level safe.
    Prompt policy openclaw-manager-prompt-policy Created, priority 1, content-only, no tool_gate. Verified in policies list.

    All created through Turnstone's native mechanisms (persona via admin API, skills via native skills tool, policy via native policy API) — not filesystem shims. No credentials, no private host data, no SSH.

    Deploy-time collision trial (the important engine data)

    • The existing Openclaw Management skill (lowercase c, v1.0.0, previously enabled) was disabled via the native skills tool after operator GO.
    • Read-back: hidden from enabled discovery (find with enabled_only=true returns only the 7 new skills); get still returns full content with enabled: false → fully reversible (re-enable restores it).
    • Conclusion: at deploy time, Turnstone's native catalog does support reversible supersession: create-new-coexist → operator GO → disable-old. The stale "Hermes SSH delegation" remote-maintenance guidance inside the old skill is now archived out of active use, superseded by the MCP-only capability proven in this trial.
    • This closes the earlier ASSUMED gap: Process Engine's author stage could not inspect the catalog (assumed collision behavior); the deploy-time trial proved the actual behavior. Recommendation stands: give the author stage a catalog-inspection capability preflight so collision handling is proven earlier.

    Evidence loop (complete)

    1. Live capability proved (single-command MCP exec, admin-RPC parity, exec-approval/allowlist mechanics) — session evidence.
    2. PE generated the package (draft, standards-shaped).
    3. Trial: read-only live probes T1–T5/T8 PASS; T6/T7 gated/design PASS.
    4. Operator review + GO.
    5. Deployed via native mechanisms.
    6. Read-back verified (persona, 7 skills, policy present; old skill disabled reversibly).
    7. Deploy receipts + manifest stage=deployed recorded; rollback path documented.

    Ask for the reviewer (unchanged, now with ship evidence)

    1. Author-stage catalog inspection — should PE's author stage be granted a catalog-inspection/preflight capability so name-collision handling is proven at author time rather than deferred to deploy time?
    2. Prove-by-live-test as a standard — capability = live proof, not config reading (this trial's central lesson: an adapter allowlist ≠ remote-supported surface; a task_id ≠ completion).
    3. Validate config values against runtime enum before apply — tools.exec.mode: "allow" crashed the gateway (exit 78) because the runtime's valid set is deny/allowlist/…/full; skills should encode "check the runtime's own accepted values first."

    All ship evidence is in the private workspace (deploy receipts, ledger, evals); this issue is the sanitized public record.

  2. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    Reviewer prompt — Process Engine development (post on issue #11)

    How to post (operator): on #11 click "Comment" and paste the body below.


    Reviewer request: critical review, guidance, and steering for Process Engine development

    @maintainers / @reviewers — I'm asking for a deliberate, critical review of this trial and the three recommendations embedded in it, to steer Process Engine development. Please engage directly on this issue.

    Context (what this trial exercised)

    This was a downstream product trial: turn a real, hard-won operator session (remote OpenClaw management via MCP only, no SSH) into a Process Engine package — persona + skill family + governance helpers — then deploy it onto Turnstone and verify. The full loop ran: live capability proof → PE generation → read-only trial → operator GO → native deploy → read-back → supersession of an existing skill.

    Two engine behaviors were under the microscope:

    1. Skill name collision — an existing skill (Openclaw Management, lowercase c) collided with the requested package name (OpenClaw Management). PE's author stage could not inspect the catalog, recorded the collision as ASSUMED, and deferred resolution to deploy time. Deploy-time proved the native catalog supports reversible disable, but the author-stage blindness remains.
    2. Capability claims — the session that motivated this package repeatedly demonstrated that "config reading" and "adapter allowlists" are NOT proof of live capability (an adapter listed 30+ methods; the remote supported ~12; a task_id ≠ completion; a config value allow crashed the gateway because the runtime's valid set is deny/allowlist/…/full).

    The three recommendations I want you to stress-test

    R1 — Author-stage catalog inspection / capability preflight. Should PE's author stage be granted (or required to use) a catalog-inspection capability so name-collision and artifact-existence checks are PROVEN at author time, not deferred to deploy time? What are the risks (e.g., author-stage privilege, partial writes, evaluation cost) and the right boundary?

    R2 — "Prove-by-live-test" as a first-class standard. Should the engine encode, as a load-bearing standard (not a guideline), that capability claims must be backed by live test evidence — and that generated skills must include "probe the actual endpoint, not the wrapper" procedures? How should PE represent the difference between a PROVEN capability and an ASSUMED one in generated artifacts?

    R3 — Validate config values against runtime enum before apply. Should the engine's generated maintenance/troubleshooting skills include a mandatory "check the runtime's own accepted values before proposing or applying a config change" step, and should the review gate check for it? (Evidence: tools.exec.mode: "allow" → gateway exit 78; the gateway's own error enumerated the valid set.)

    What I'd value from you

    • Steering: which of R1–R3 is highest-value for Process Engine's near-term development, and why?
    • Pushback: where are these recommendations wrong, over-engineered, or premature? Where would they add ceremony without value?
    • Design: for the one you'd prioritize, what does the minimal correct implementation look like in Process Engine's prompt-first architecture (no code enforcement, per its current design)?
    • Triage lens: does this trial surface any other engine gap we haven't named?

    I'll take the guidance back into Process Engine development. The full sanitized evidence trail is available on request; this issue is the durable public record.

  3. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    Reviewer kickoff prompt — Process Engine critical review (post on issue #11)

    How to post (operator): on #11 click "Comment" and paste the body below. (Token now has issues:write, so I can also post via API if you prefer.)


    Kickoff: review and assessment requested

    This trial has reached a fully shipped and verified state — the OpenClaw Maintenance package (persona + skill family + governance) is deployed on Turnstone, read-back verified, the old skill superseded reversibly, and the full evidence loop closed. Now I'd like a deliberate, critical review to steer Process Engine development.

    The evidence (in this issue)

    • Original trial summary (first message)
    • Ship evidence (comment above): deployed artifacts, deploy-time collision trial PASS, 7-step evidence loop
    • Three recommendations to stress-test: R1 author-stage catalog inspection/preflight; R2 "prove-by-live-test" as a first-class standard; R3 validate config values against runtime enum before apply.

    What I'm asking you to do

    1. Steering — which of R1–R3 is highest-value for Process Engine's near-term development, and why?
    2. Pushback — where are these wrong, over-engineered, or premature? Where would they add ceremony without value?
    3. Design — for the one you'd prioritize, what's the minimal correct implementation in Process Engine's prompt-first architecture (no code enforcement)?
    4. Triage lens — does this trial surface any engine gap we haven't named?

    Your guidance goes directly back into Process Engine development. Please be direct — I'd rather have hard pushback than polite agreement.

  4. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    Reviewer steering — prioritize destination-state proof, not broader author privilege

    I reviewed the shipped evidence and the three recommendations against the current Process Engine contracts, including the PR #10 candidate work on executable capability paths and Trial PASS / INCOMPLETE / FAIL semantics.

    Recommendation order

    1. R1 is the highest-value marginal engine finding — but I would reframe it.

    The real gap is not “grant Pattern Author broader catalog privilege.” It is:

    Before finalizing named Turnstone artifacts, prove relevant destination-state facts that can change package design or deployment — especially exact/normalized-name collisions — using read-only native catalog inspection.

    This is cross-cutting and not OpenClaw-specific. The current core/intake path can search existing skills when the operator explicitly asks to find one, but there is no proactive destination-catalog preflight when Pattern has just chosen names for new Turnstone artifacts. That is why this package could know a collision existed yet could not prove the exact object until deploy time.

    Minimal prompt-first implementation:

    • Pattern Author: after artifact names/identities are chosen, and only when a Turnstone destination catalog is relevant, perform a read-only exact/normalized-name lookup for the named project/persona/skills/templates/policies that could collide. Treat the result as target-state evidence, not design authority. No writes, disables, supersession, or mutation at author time.
    • If the needed read surface is unavailable, record the collision/existence state as UNPROVEN rather than guessing. Do not invent a resolution.
    • Review: verify that material collisions are either (a) proven and have a reversible operator-gated disposition, or (b) explicitly UNPROVEN and deferred. Review should reject silent overwrite/duplicate assumptions, but it need not block a safe draft merely because final supersession is intentionally operator-gated.
    • Keep actual create/disable/supersede actions in Ship under operator approval. Do not add a new stage, manifest, collision database, or PE-owned catalog layer.

    If Turnstone must expose a catalog tool to make that prompt actionable, expose only the existing native read/find/get surface to the generator. That is Turnstone integration/configuration, not Process Engine runtime code.

    One caution: do not make “semantic similarity search the whole catalog” a mandatory author ritual. Exact/canonical/normalized identity collisions are the load-bearing case. Broader overlap can be a heads-up when useful.

    2. R2 is directionally right, but too broad as written — and PR #10 already covers much of it

    I would not make “every capability claim requires a live test” a universal standard. Static/specification claims can be supported by authoritative documentation; some live probes are unsafe, expensive, or impossible.

    The stronger bounded rule is:

    Environment/runtime-dependent capability claims that materially affect generated procedures must be verified in the target runtime before overall Trial PASS; when safe and appropriate, use a non-mutating end-to-end probe rather than treating configuration, an adapter allowlist, or a tool listing as proof of successful operation.

    PR #10 already moves PE substantially in this direction by requiring an executable capability path at Pattern/Review and runtime capability preflight at Trial. I would merge/live-test that first before adding more wording. The only refinement I would consider afterward, if the live reconciliation shows ambiguity, is one sentence making clear that declaration/listing alone is not end-to-end proof when the claimed procedure depends on actual target behavior.

    Do not encode “probe the actual endpoint, not the wrapper” literally as universal doctrine. Sometimes the wrapper is the intended governed runtime boundary. The question is whether the evidence proves the capability through the actual intended execution path.

    Also scope evidence to the target: proof on LOCAL does not silently prove REMOTE. This trial did the right thing by testing both.

    3. R3 is useful package guidance, but premature as a universal Process Engine rule

    The tools.exec.mode: "allow" → exit 78 incident is strong OpenClaw/runtime evidence, but “check the runtime enum” does not generalize cleanly: many systems expose schemas, validation endpoints, help text, typed APIs, or authoritative docs rather than an enumerable runtime set.

    The general form would be:

    For configuration mutations, validate proposed values against the target runtime's authoritative validation surface when one is available; otherwise do not invent accepted values and keep uncertainty explicit.

    That is a good technique for the generated OpenClaw maintenance skills because the source evidence supports it. I would not add it to universal PE standards yet. Watch for recurrence in a non-OpenClaw/non-infrastructure trial first.

    Additional engine signal not named: initiation ≠ completion

    Issue #11 contains a separate evidence lesson that should not be collapsed into R2:

    A task/job/request ID proves acceptance or initiation, not successful completion.

    For generated skills that invoke asynchronous operations, the package should not claim success until it observes the relevant terminal state/read-back; if terminal evidence is unavailable, report the operation as submitted/pending/UNPROVEN rather than complete.

    This looks potentially cross-domain (updates, CI jobs, cloud operations, exports, background workflows), but I would record it as a candidate rather than immediately patch PE from one trial. One more independent occurrence would justify a very small Pattern/Review/Trial acceptance criterion.

    One evidence caution on this shipped trial

    The public T1–T8 list does not show an independent should-trigger / shouldn't-trigger / parent→child routing set for the parent + six child skills. If that evidence exists privately, link/summarize it. If it does not, then under the v1.9.6 Trial contract the accurate statement is:

    • executed behavioral cases: PASS;
    • activation/routing: UNPROVEN;
    • full overall Trial PASS: not proven.

    That would be another instance of the completeness problem PR #10 is specifically fixing, not a new mechanism for Issue #11.

    Steering

    My current recommendation is:

    1. Do not start an R2/R3 product patch while PR fix: require capability preflight and complete trial evidence #10 is still being merged/reconciled. Avoid overlapping prompt changes before the new capability-preflight/Trial semantics are proven live.
    2. Preserve Issue Downstream trial: OpenClaw Maintenance package - dual-target MCP management + skill-collision engine data (2026-08-10) #11 as the next product-learning record.
    3. After PR fix: require capability preflight and complete trial evidence #10 is canonical and live-tested, define a small R1 destination-catalog preflight change if the catalog-read gap still exists in the reconciled generator.
    4. Keep R2 as supporting evidence for PR fix: require capability preflight and complete trial evidence #10 unless live use proves a remaining declaration-vs-end-to-end ambiguity.
    5. Keep R3 package-specific for now.
    6. Track “accepted/submitted ≠ completed” as a candidate engine lesson pending another independent example.

    Highest-value near-term: R1, reframed as read-only destination-state preflight, not broader author authority.

    Minimal likely PE files if later implemented: Pattern Author + Review only. Core/Trial/Ship should not need changes unless the post-PR10 live trial demonstrates a concrete gap.

  5. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    Issue #11 evidence audit against the now-live post-PR10 candidate (2026-08-10)

    References reviewer steering 5234968235. PR #10 (capability preflight + Trial PASS/INCOMPLETE/FAIL) is now merged (a6b81ff…) and live-proven (Issue #8 comment 5235101209: L1-L5 + cross-domain sanity all PASS on the persisted runtime).

    R1 - destination-catalog collision preflight (reframed): READY TO DEFINE (narrow)

    Catalog-capability audit (read-only live probe):

    • The generator's turnstone-api MCP surface exposes no native skills_list/skill_get - the native Turnstone skill catalog is available to another context (admin/API) but not the generator surface.
    • However, a read-only collision lookup IS possible today via the Hermes gateway catalog tool (hermes_skills_list): a live probe of the proposed name openclaw-management returned PROVEN PRESENT (Openclaw Management + openclaw-management both listed). A local filesystem search alone would have been UNPROVEN.
    • Pipeline point: artifact identity becomes sufficiently known at Pattern, after shape + canonical name are selected, before authoring/deployment.
    • Minimal future PE change (do NOT implement yet): Pattern - after names chosen and only when a Turnstone destination catalog is relevant, perform a read-only exact/normalized-name lookup; record PROVEN PRESENT / PROVEN ABSENT / UNPROVEN (never guess); no writes/disable/supersede at author time. Review - verify material collisions are (a) proven with a reversible operator-gated disposition or (b) explicitly UNPROVEN/deferred; reject silent overwrite/duplicate assumptions. Actual create/disable/supersede stays in Ship under operator approval. Exposing the native read/find/get surface to the generator is Turnstone integration/config, not PE runtime code. No new stage/manifest/collision database. Caution: do not make semantic similarity search of the whole catalog a mandatory ritual - exact/canonical/normalized identity collisions are the load-bearing case.

    R2 - prove-by-live-test: MOSTLY COVERED BY PR #10

    PR #10 now requires an executable capability path at Pattern/Review and runtime capability preflight at Trial; L3 (unavailable runtime capability → INCOMPLETE, not behavioral FAIL, layer reported) proves it live. Do not encode a universal "everything must be live-tested" rule: static/interface facts may be proven by authoritative source/schema/catalog evidence; destructive/unavailable probes stay ASSUMED/UNPROVEN with explicit evidence status. Do not encode "probe the endpoint, not the wrapper" literally - the governed wrapper/adapter may be the intended execution path; what matters is evidence through the actual intended path. Also scope evidence to target: LOCAL proof does not silently prove REMOTE.

    R3 - configuration value validation: PACKAGE-SPECIFIC / NEEDS MORE EVIDENCE

    The tools.exec.mode exit-78 incident is strong OpenClaw/runtime evidence, but "check the runtime enum" does not generalize cleanly (schemas, validation endpoints, help text, typed APIs, authoritative docs). General form if it ever generalizes: validate proposed config values against the target's authoritative validation surface when one exists; otherwise keep uncertainty explicit. Watch for recurrence in a non-infrastructure trial before adding to universal standards. Keep it in the generated OpenClaw maintenance skills for now.

    Async-completion candidate ("accepted/submitted ≠ completed"): PARTIAL / NEEDS MORE EVIDENCE

    Search of current PE prompts: PE already distinguishes acceptance vs completion implicitly via read-back/verification requirements (Ship requires read-back; Trial is evidence-driven), but there is no explicit Pattern/Review/Trial acceptance criterion stating that submission/acceptance is not completion and a success claim requires terminal-state evidence or post-operation read-back appropriate to the target. One more independent occurrence (updates, CI jobs, cloud ops, exports, background workflows) would justify a very small acceptance criterion. Not implemented in this gate.

    Retrospective Issue #11 activation/routing (post-PR10 completeness contract)

    Private trial receipts show T1-T8 behavioral cases with live/session evidence (all PASS) + deploy-time collision trial PASS + read-back. No independent should-trigger / shouldn't-trigger / parent→child routing set is recorded. Therefore, under the post-PR10 completeness contract:

    • recorded behavioral suite: PASS
    • activation/routing: UNPROVEN
    • retrospective overall Trial: INCOMPLETE (activation category unrun)
      The shipped package is not undone; operator GO + deploy read-back are valid. This is another instance of the completeness problem PR fix: require capability preflight and complete trial evidence #10 fixes.

    Exact candidate next action

    Define (do not implement) a small R1 destination-catalog preflight (Pattern + Review only, read-only, UNPROVEN fallback, no new stage/manifest). Requires exposing the native Turnstone read/find/get catalog surface to the generator (Turnstone integration). No new PE change was implemented in this gate.

  6. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    R1 change definition - destination-skill collision preflight (defined, NOT implemented)

    References: 5234968235, 5235103927. This is the definition only; no PE prompt change, no branch/PR, no Turnstone integration change performed.

    1. Scope narrowed to generated skills first

    Issue #11 evidence is a skill catalog collision (Openclaw Management vs openclaw-management/OpenClaw Management). Therefore R1 targets generated Turnstone skills only - not personas, projects, policies, templates, or all Turnstone objects. Those may later need the same treatment; Issue #11 does not establish that yet.

    2. Product problem

    PE can select a new generated skill identity without proving relevant existing destination-catalog state, so authoring/review proceed while a material name collision is only ASSUMED. Correction: once Pattern has selected a generated skill's canonical identity, inspect the destination Turnstone skill catalog through a native read-only surface before finalizing the draft. This is destination-state evidence, not design authority.

    3. Turnstone-native architecture

    PE must not depend on Hermes, filesystem search, a custom catalog DB, a PE registry, a duplicate manifest, or custom collision code. The required runtime surface is a Turnstone-native read-only skill-catalog capability exposed to the generator: conceptually find/search proposed skill identity + fetch/get a returned skill object. No write/update/enable/disable/delete authority.

    4. Absence semantics

    • PROVEN PRESENT: authoritative native catalog evidence identifies an existing material collision.
    • PROVEN ABSENT: an authoritative lookup with sufficient exact-identity semantics establishes no matching destination skill exists.
    • UNPROVEN: catalog capability unavailable; search semantics not authoritative enough to prove absence; lookup ambiguous; near-match unresolvable; returned evidence insufficient.
      search returned nothing ≠ PROVEN ABSENT unless the tool contract supports that conclusion.

    5. No invented normalization doctrine

    First implementation around: proposed Agent Skills canonical name; Turnstone's native skill identity/name fields; exact native lookup semantics; obvious returned identity conflicts. If the native catalog provides canonical/normalized matching, use it. No custom fuzzy/semantic matching machinery in PE. A broader similarity search may produce a heads-up when useful, but must not be mandatory ceremony.

    6. Pattern behavior

    After a generated skill's canonical name is chosen: (1) if destination Turnstone catalog read access is available, perform the bounded lookup; (2) fetch enough returned identity data to distinguish actual skill objects; (3) record PROVEN PRESENT / PROVEN ABSENT / UNPROVEN; (4) preserve distinct returned identities; (5) never silently choose overwrite/coexist/supersede; (6) do not mutate anything. If PROVEN PRESENT: surface the collision and propose dispositions (retain both / rename new / later supersede / intentionally replace if separately supported). No disposition is automatic.

    7. Review behavior

    • PROVEN ABSENT: collision dimension satisfied.
    • PROVEN PRESENT: Review requires an explicit, reversible, operator-gated disposition before PASS (does not execute it). A plan like create new → operator GO → disable old → read-back may be valid where Turnstone supports it.
    • UNPROVEN: for a deployment-bound package with a material unresolved destination identity, Review must not issue full PASS - return REVISE with the missing destination-state evidence identified. (Use existing Review outcome semantics; no new state machine.)

    8. Ship remains authority boundary

    Actual catalog mutation stays in Ship after Review PASS + Trial PASS + explicit operator approval. R1 grants Pattern/Review no create/update/enable/disable/delete/supersede. Ship unchanged.

    9. PE-product vs Turnstone-integration split

    • PE product change (future): Pattern Author + Review prompt instructions only.
    • Turnstone integration change (future): expose the minimum native read-only skill-catalog surface to the generator - Turnstone integration/config, not PE runtime code.
      Recommended sequence: define PE prompts against the intended native surface first (repository-first), then integrate/expose the surface; keep the two changes separable and reviewable.

    10. Acceptance criteria C1-C6

    • C1 proven collision: existing skill with proposed canonical identity returned from authoritative catalog → Pattern records PROVEN PRESENT + identity evidence; no mutation; Review requires explicit reversible disposition.
    • C2 proven absence: authoritative exact lookup proves no matching skill → Pattern records PROVEN ABSENT; Review does not invent a collision concern.
    • C3 catalog unavailable: generator lacks catalog read → Pattern records UNPROVEN, does not guess; Review does not full-PASS a deployment-bound package until resolved.
    • C4 ambiguous/near identity: catalog returns potentially conflicting skill with distinct identity/display spelling → Pattern fetches/preserves both identities; no silent collapse; ambiguity visible; Review requires resolution/evidence.
    • C5 no semantic-search ceremony: unrelated skill with vaguely similar purpose but distinct identity → no mandatory full-catalog semantic-overlap ritual; no false collision gate.
    • C6 mutation boundary: every C1-C5 case - Pattern/Review perform zero catalog writes; no enable/disable/delete/update; Ship/operator authority unchanged.

    11. Smallest affected PE artifact set

    Pattern Author + Review only. Core/Trial/Ship unchanged (Ship already owns mutation authority).

    R2 / R3 / async - retained classifications (unchanged)

    No implementation performed in this gate.

  7. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    R1 implementation evidence - PR #12 (repo-only; live integration held)

    References: 5234968235, 5235103927, 5235225699.

    Implementation PR #12: fix: preflight destination skill collisions

    • base a6b81ff349a650440c553b4a0c3da38b486958ca, head 80d200e935aceee2611d66925c686b77c15ab1e1
    • Files changed (2): Pattern Author (+37), Review (+18). Ship unchanged; Core/Trial/references/persona/templates/governance unchanged.
    • Structural validation PASS.

    Implemented: Pattern destination-skill preflight for generated Turnstone skills (PROVEN PRESENT / PROVEN ABSENT / UNPROVEN; no custom normalization/fuzzy engine; preserves distinct identities; no automatic disposition; zero mutation) + Review destination-skill evidence check (ABSENT ok; PRESENT requires explicit reversible operator-gated disposition; UNPROVEN → REVISE for deployment-bound; zero mutation).

    C1-C7 regression results (bounded fixtures, labeled not-live): all PASS - receipts in PR #12 comment 5235318860. C1 proven collision → PROVEN PRESENT + disposition required; C2 authoritative absence → PROVEN ABSENT; C3 catalog unavailable → UNPROVEN + REVISE (no filesystem fallback); C4 case-variant identities preserved (no silent equivalence); C5 distinct-but-similar → no false gate; C6 zero mutation across all; C7 persona generation does not trigger skill preflight (skills-only scope).

    Live generator integration is still NOT changed. The current generator does not yet expose the desired native Turnstone read-only skill-catalog surface; this PR is the repository prompt change only. Hermes is evidence the collision exists, not the permanent PE integration. Live integration remains held pending (a) drift containment (Issue #8 5235223880/5235291721: recurrence risk MEDIUM, mechanism UNKNOWN) and (b) a separately approved Turnstone catalog-read exposure gate.

    R2 / R3 / async classifications unchanged: R2 MOSTLY COVERED BY PR #10; R3 PACKAGE-SPECIFIC / NEEDS MORE EVIDENCE; accepted/submitted ≠ completed PARTIAL / NEEDS MORE EVIDENCE.

    No live PE mutation, no catalog exposure, no tool-assignment change, no version/release, no Issue closure.

  8. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    R1 identity-binding correction (PR #12, head 53c1963…)

    Independent review 4893388004 found a stale-identity evidence hole: Review could accept a rename plan using destination evidence gathered for the old canonical skill name. Correction implemented on PR #12 (Pattern + Review only):

    • Pattern: destination-skill evidence is valid only for the same canonical skill identity it queried (queried name + returned identity + PRESENT/ABSENT/UNPROVEN + source as one reviewable unit). If a collision disposition changes the generated canonical name, that is a revision: rename draft → re-run preflight for the new name → record fresh evidence → return to Review. No re-run when the disposition leaves the canonical name unchanged. Point-in-time semantics: PRESENT/ABSENT is preflight-time author/review evidence, not a Ship-time transactional guarantee (no locks/leases/Ship preflight).
    • Review: confirm the recorded destination-skill evidence corresponds to the current canonical name in the draft being reviewed; stale receipt or future rename-plan → REVISE (rename must happen in Pattern first with fresh evidence); completed rename with fresh receipt → evaluate normally.

    C8-C11 results (receipts in PR #12 comment 5235465675):

    • C8 stale receipt after rename → REVISE (identity mismatch; old evidence cannot satisfy) - PASS
    • C9 rename + fresh absence → PROVEN ABSENT bound to new identity; Review satisfied - PASS
    • C10 rename + fresh UNPROVEN → old receipt discarded; REVISE; no PASS from old identity - PASS
    • C11 unchanged-identity supersession plan → evidence corresponds to current identity; disposition adequate; no forced rename/re-preflight; zero mutation - PASS

    New PR #12 head: 53c19638875595ed4ca86d3af9f6e72aed68e687 (base a6b81ff…); structural validation PASS. Live integration remains held (drift MEDIUM/UNKNOWN; separate catalog-read exposure gate). R2/R3/async classifications unchanged (MOSTLY COVERED BY PR10 / PACKAGE-SPECIFIC / PARTIAL). No live PE mutation, no catalog exposure, no version/release, no Issue closure.

  9. RedEyeNinja-BKK commented on Aug 10, 2026

    @RedEyeNinja-BKK
    OwnerAuthor

    R1 controlled live integration + alignment gate - result (2026-08-10)

    PR #12 merged at 076dc4d323dba449cf416c1f1db948e997b0b334 from the exact reviewed head 53c19638875595ed4ca86d3af9f6e72aed68e687. Canonical repo implementation of R1 (destination-skill collision preflight) is complete on main. Released tag v1.9.6 remains at 65eaba888322609afa9986ff1d19e959317f5e28 (unchanged; no release/version decision made).

    Pre-live state was INTENTIONAL REPO-AHEAD - live Turnstone held the pre-PR12 canonical basis while the merged candidate awaited a separate live-integration gate.

    Controlled live integration (this gate)

    • Gate-entry live read matched the expected pre-PR12 canonical basis across all six pipeline skills (no fresh unexpected drift before mutation).
    • Rollback evidence captured before any write (pattern + review + generator-persona config; hashes, enabled state, rollback mechanism recorded) at sync-rollback-2026-08-10-r1-live/.
    • Pattern Author and Review reconciled from canonical 076dc4d…; all other PE objects untouched (Core/Trial/Ship/Triage, references, templates, generator persona content, development persona, policies/judge unchanged).
    • Exact persisted read-back: all six pipeline skills now match canonical 076dc4d… product content (normalized byte comparison; storage trims the final newline).

    Catalog capability (conceptual level, as required)

    R1 needs a Turnstone-native authoritative read-only skill-catalog surface. The existing native surface already provides it:

    • authoritative object identity = Turnstone template_id (admin read); canonical Agent Skills name is the queried identity;
    • exact-name get by canonical name is exact-match, case-sensitive, returns the full object, and returns an explicit not-found for a missing identity → this is the authoritative presence/absence surface;
    • the generic find/search query is BM25-ranked and cannot authoritatively prove absence (verified live: an exact-name row can be missed by find);
    • disabled-skill visibility: find supports enabled_only=false; catalog currently has zero disabled rows, so disabled-get behavior remains unproven live (contract supports it);
    • zero mutation surface: the generator receives no catalog write capability; the read actions used by the preflight are read-only by construction, and R1's Pattern/Review forbid any create/update/enable/disable/delete/supersede.

    No custom PE catalog service was introduced; no global tool-authority expansion; Hermes is not a dependency of R1.

    Controlled R1 smoke (S1-S4) - actual persisted generator

    Run through the real generator persona with the deployed Pattern skill and the real native catalog surface. No catalog writes.

    • S1 (existing identity) - proposed process-engine-core; Pattern invoked native read; recorded PROVEN PRESENT with queried name + returned destination identity + evidence source as one identity-bound unit; zero mutation; no automatic disposition.
    • S2 (absence) - unique non-deployed name; exact-name get returned explicit not-found → PROVEN ABSENT (native semantics genuinely prove it); find zero-result treated as corroboration only, not proof.
    • S3 (identity binding) - renamed the S1 collision draft; old process-engine-core receipt explicitly invalidated for the new name; fresh preflight recorded for the renamed identity; zero mutation.
    • S4 (non-skill scope) - document-template draft; destination-skill preflight correctly not run (skills-only scope); no unnecessary catalog ceremony.

    Alignment and risk

    • Final repo/live alignment: ALIGNED - GitHub canonical 076dc4d… = live six-skill PE content, with the generator receiving only the minimum native read-only catalog read capability (already present in its default toolset; no new write surface introduced).
    • Historical drift risk remains MEDIUM / mechanism UNKNOWN. This gate's clean persistence does not establish that the unknown writer is solved; a recurrence watch remains warranted.

    Issue #11 remains open; no issue closure performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions