Repository navigation
Description
Downstream trial: OpenClaw Maintenance package — dual-target MCP management (2026-08-10)
Type: Downstream product-trial case study / engine-eval data (same pattern as prior TrueNAS Manager / Proxmox Manager runs).
Package: openclaw-maintenance-2026-08-10 — generated by Process Engine v1.9.6 (Pattern → Author → Review-prep → Trial-plan), currently DRAFT, not published.
Scope: Managing BOTH OpenClaw instances in a 3-agent stack through MCP only (no SSH): a LOCAL OpenClaw reached via one MCP gateway, and a REMOTE OpenClaw on a distant host reached via a second MCP gateway. Persona: "OpenClaw Manager". Skill family: "OpenClaw Management" (parent) + 6 children.
1. Proven capability boundary (from live testing, prior to PE engagement)
- Single-command exec over MCP works on both gateways (
openclaw --version,npm --version,openclaw update status --json). Update-status read returns full JSON; the remote is current, so no update was applied (correctly no-op). - Multi-command compound tasks can self-block under exec allowlist mode on the REMOTE gateway; the safe operating unit is one command at a time (matches local behavior).
- Admin-RPC surface is parity across both gateways: methods supported on one are supported on the other; unsupported methods fail cleanly with HTTP 400 "method is not supported" on both (never hang).
- Silent exec-approval prompts are a real failure mode: an un-approved command blocks until approved in the remote UI; tasks look hung. Pre-authorization (allowlist) or operator notification is required for multi-step flows.
- Config value validation is critical:
tools.exec.mode: "allow"is INVALID in the tested runtime — gateway exits with code 78 and its error message enumerates valid values (deny,allowlist, …full). Allowlist matching uses resolved paths or node-command hashes; bare binary paths silently fail to match.
2. What Process Engine produced (observed behavior)
- Complete standards-shaped package: manifest, persona, parent skill + 6 child skills (spec-valid SKILL.md), advisory prompt policy, advisory judge rules, creation checklist, scripted 8-case trial plan, process report. No failure in generation.
- Skill-name collision observation (important engine data): an existing skill named
Openclaw Management(lowercase c, v1.0.0) already exists. PE did not refuse, edit, disable, or duplicate it. It created the new package in draft filesystem form with a spec-safe frontmatter name (openclaw-management) and the requested title ("OpenClaw Management"), and recorded the collision as ASSUMED with a proposed operator-gated, reversible migration. - Engine gap identified: the PE author stage could not inspect the exact existing skill object (catalog read capability not exposed to the generator task), so it could not prove runtime collision resolution (supersede vs duplicate vs refuse). Turnstone upgraded the observation to PROVEN afterward via a native skill-get. Recommendation: consider granting the author stage catalog inspection or a capability preflight so collision handling is proven at author time, not deferred to deploy time.
3. Trial results (read-only, live probes, all PASS)
- T1 health (both gateways) PASS; T2 config.get (both) PASS; T3 cron.list (both; LOCAL 1 job, REMOTE 7 jobs) PASS; T4 single-command read-only exec (both) PASS; T5 admin-RPC parity / unsupported-method handling PASS; T6 collision handling (non-destructive draft) PASS with deploy half gated; T7 gated-update design PASS (no live mutation, correctly); T8 invalid exec.mode / resolved-path allowlist handling PASS.
- No mutation performed; no SSH; no credentials exposed; package remains DRAFT.
4. Limitations / open items
- Catalog supersession of the old lowercase-c skill is still an operator-gated deploy-time action; deploy-time collision resolution is UNPROVEN.
- Package hash deferred to freeze (PE chose not to record a draft hash — good practice).
- Live update flow not executed (correctly gated; no update available at time of test).
5. Ask for the reviewer
Please review the collision-handling gap (author-stage catalog inspection) and whether the engine should encode "prove-by-live-test" (capability = live proof, not config reading) and "validate config values against runtime enum before apply" as first-class standards. Detailed evidence lives in the private workspace (sanitized summary above); happy to expand any section.
Activity
RedEyeNinja-BKK commentedon Aug 10, 2026
GitHub issue #11 — ship-evidence update comment (ready to post)
How to post (operator): on #11 click "Comment" and paste the body below. Or, if you prefer, authorize a PAT with
issues: write/reposcope and I'll post it via API.UPDATE (2026-08-10) — SHIPPED: package deployed to Turnstone and verified (full evidence loop)
The OpenClaw Maintenance package has been fully deployed onto Turnstone and verified by read-back, completing the trial's evidence loop. Details below (sanitized; role-based host references only).
Deployed artifacts (read-back verified)
openclaw-gateway,openclaw-remote-gateway,turnstone-api. Verified present in personas list.openclaw-managementopenclaw-manager-prompt-policyAll created through Turnstone's native mechanisms (persona via admin API, skills via native skills tool, policy via native policy API) — not filesystem shims. No credentials, no private host data, no SSH.
Deploy-time collision trial (the important engine data)
Openclaw Managementskill (lowercase c, v1.0.0, previously enabled) was disabled via the native skills tool after operator GO.findwith enabled_only=true returns only the 7 new skills);getstill returns full content withenabled: false→ fully reversible (re-enable restores it).Evidence loop (complete)
Ask for the reviewer (unchanged, now with ship evidence)
tools.exec.mode: "allow"crashed the gateway (exit 78) because the runtime's valid set isdeny/allowlist/…/full; skills should encode "check the runtime's own accepted values first."All ship evidence is in the private workspace (deploy receipts, ledger, evals); this issue is the sanitized public record.
RedEyeNinja-BKK commentedon Aug 10, 2026
Reviewer prompt — Process Engine development (post on issue #11)
How to post (operator): on #11 click "Comment" and paste the body below.
Reviewer request: critical review, guidance, and steering for Process Engine development
@maintainers / @reviewers — I'm asking for a deliberate, critical review of this trial and the three recommendations embedded in it, to steer Process Engine development. Please engage directly on this issue.
Context (what this trial exercised)
This was a downstream product trial: turn a real, hard-won operator session (remote OpenClaw management via MCP only, no SSH) into a Process Engine package — persona + skill family + governance helpers — then deploy it onto Turnstone and verify. The full loop ran: live capability proof → PE generation → read-only trial → operator GO → native deploy → read-back → supersession of an existing skill.
Two engine behaviors were under the microscope:
Openclaw Management, lowercase c) collided with the requested package name (OpenClaw Management). PE's author stage could not inspect the catalog, recorded the collision as ASSUMED, and deferred resolution to deploy time. Deploy-time proved the native catalog supports reversible disable, but the author-stage blindness remains.allowcrashed the gateway because the runtime's valid set isdeny/allowlist/…/full).The three recommendations I want you to stress-test
R1 — Author-stage catalog inspection / capability preflight. Should PE's author stage be granted (or required to use) a catalog-inspection capability so name-collision and artifact-existence checks are PROVEN at author time, not deferred to deploy time? What are the risks (e.g., author-stage privilege, partial writes, evaluation cost) and the right boundary?
R2 — "Prove-by-live-test" as a first-class standard. Should the engine encode, as a load-bearing standard (not a guideline), that capability claims must be backed by live test evidence — and that generated skills must include "probe the actual endpoint, not the wrapper" procedures? How should PE represent the difference between a PROVEN capability and an ASSUMED one in generated artifacts?
R3 — Validate config values against runtime enum before apply. Should the engine's generated maintenance/troubleshooting skills include a mandatory "check the runtime's own accepted values before proposing or applying a config change" step, and should the review gate check for it? (Evidence:
tools.exec.mode: "allow"→ gateway exit 78; the gateway's own error enumerated the valid set.)What I'd value from you
I'll take the guidance back into Process Engine development. The full sanitized evidence trail is available on request; this issue is the durable public record.
RedEyeNinja-BKK commentedon Aug 10, 2026
Reviewer kickoff prompt — Process Engine critical review (post on issue #11)
How to post (operator): on #11 click "Comment" and paste the body below. (Token now has issues:write, so I can also post via API if you prefer.)
Kickoff: review and assessment requested
This trial has reached a fully shipped and verified state — the OpenClaw Maintenance package (persona + skill family + governance) is deployed on Turnstone, read-back verified, the old skill superseded reversibly, and the full evidence loop closed. Now I'd like a deliberate, critical review to steer Process Engine development.
The evidence (in this issue)
What I'm asking you to do
Your guidance goes directly back into Process Engine development. Please be direct — I'd rather have hard pushback than polite agreement.
RedEyeNinja-BKK commentedon Aug 10, 2026
Reviewer steering — prioritize destination-state proof, not broader author privilege
I reviewed the shipped evidence and the three recommendations against the current Process Engine contracts, including the PR #10 candidate work on executable capability paths and Trial
PASS / INCOMPLETE / FAILsemantics.Recommendation order
1. R1 is the highest-value marginal engine finding — but I would reframe it.
The real gap is not “grant Pattern Author broader catalog privilege.” It is:
This is cross-cutting and not OpenClaw-specific. The current core/intake path can search existing skills when the operator explicitly asks to find one, but there is no proactive destination-catalog preflight when Pattern has just chosen names for new Turnstone artifacts. That is why this package could know a collision existed yet could not prove the exact object until deploy time.
Minimal prompt-first implementation:
If Turnstone must expose a catalog tool to make that prompt actionable, expose only the existing native read/find/get surface to the generator. That is Turnstone integration/configuration, not Process Engine runtime code.
One caution: do not make “semantic similarity search the whole catalog” a mandatory author ritual. Exact/canonical/normalized identity collisions are the load-bearing case. Broader overlap can be a heads-up when useful.
2. R2 is directionally right, but too broad as written — and PR #10 already covers much of it
I would not make “every capability claim requires a live test” a universal standard. Static/specification claims can be supported by authoritative documentation; some live probes are unsafe, expensive, or impossible.
The stronger bounded rule is:
PR #10 already moves PE substantially in this direction by requiring an executable capability path at Pattern/Review and runtime capability preflight at Trial. I would merge/live-test that first before adding more wording. The only refinement I would consider afterward, if the live reconciliation shows ambiguity, is one sentence making clear that declaration/listing alone is not end-to-end proof when the claimed procedure depends on actual target behavior.
Do not encode “probe the actual endpoint, not the wrapper” literally as universal doctrine. Sometimes the wrapper is the intended governed runtime boundary. The question is whether the evidence proves the capability through the actual intended execution path.
Also scope evidence to the target: proof on LOCAL does not silently prove REMOTE. This trial did the right thing by testing both.
3. R3 is useful package guidance, but premature as a universal Process Engine rule
The
tools.exec.mode: "allow"→ exit 78 incident is strong OpenClaw/runtime evidence, but “check the runtime enum” does not generalize cleanly: many systems expose schemas, validation endpoints, help text, typed APIs, or authoritative docs rather than an enumerable runtime set.The general form would be:
That is a good technique for the generated OpenClaw maintenance skills because the source evidence supports it. I would not add it to universal PE standards yet. Watch for recurrence in a non-OpenClaw/non-infrastructure trial first.
Additional engine signal not named: initiation ≠ completion
Issue #11 contains a separate evidence lesson that should not be collapsed into R2:
For generated skills that invoke asynchronous operations, the package should not claim success until it observes the relevant terminal state/read-back; if terminal evidence is unavailable, report the operation as submitted/pending/UNPROVEN rather than complete.
This looks potentially cross-domain (updates, CI jobs, cloud operations, exports, background workflows), but I would record it as a candidate rather than immediately patch PE from one trial. One more independent occurrence would justify a very small Pattern/Review/Trial acceptance criterion.
One evidence caution on this shipped trial
The public T1–T8 list does not show an independent should-trigger / shouldn't-trigger / parent→child routing set for the parent + six child skills. If that evidence exists privately, link/summarize it. If it does not, then under the v1.9.6 Trial contract the accurate statement is:
That would be another instance of the completeness problem PR #10 is specifically fixing, not a new mechanism for Issue #11.
Steering
My current recommendation is:
Highest-value near-term: R1, reframed as read-only destination-state preflight, not broader author authority.
Minimal likely PE files if later implemented: Pattern Author + Review only. Core/Trial/Ship should not need changes unless the post-PR10 live trial demonstrates a concrete gap.
RedEyeNinja-BKK commentedon Aug 10, 2026
Issue #11 evidence audit against the now-live post-PR10 candidate (2026-08-10)
References reviewer steering
5234968235. PR #10 (capability preflight + Trial PASS/INCOMPLETE/FAIL) is now merged (a6b81ff…) and live-proven (Issue #8 comment5235101209: L1-L5 + cross-domain sanity all PASS on the persisted runtime).R1 - destination-catalog collision preflight (reframed): READY TO DEFINE (narrow)
Catalog-capability audit (read-only live probe):
turnstone-apiMCP surface exposes no nativeskills_list/skill_get- the native Turnstone skill catalog is available to another context (admin/API) but not the generator surface.hermes_skills_list): a live probe of the proposed nameopenclaw-managementreturned PROVEN PRESENT (Openclaw Management+openclaw-managementboth listed). A local filesystem search alone would have been UNPROVEN.R2 - prove-by-live-test: MOSTLY COVERED BY PR #10
PR #10 now requires an executable capability path at Pattern/Review and runtime capability preflight at Trial; L3 (unavailable runtime capability → INCOMPLETE, not behavioral FAIL, layer reported) proves it live. Do not encode a universal "everything must be live-tested" rule: static/interface facts may be proven by authoritative source/schema/catalog evidence; destructive/unavailable probes stay ASSUMED/UNPROVEN with explicit evidence status. Do not encode "probe the endpoint, not the wrapper" literally - the governed wrapper/adapter may be the intended execution path; what matters is evidence through the actual intended path. Also scope evidence to target: LOCAL proof does not silently prove REMOTE.
R3 - configuration value validation: PACKAGE-SPECIFIC / NEEDS MORE EVIDENCE
The
tools.exec.modeexit-78 incident is strong OpenClaw/runtime evidence, but "check the runtime enum" does not generalize cleanly (schemas, validation endpoints, help text, typed APIs, authoritative docs). General form if it ever generalizes: validate proposed config values against the target's authoritative validation surface when one exists; otherwise keep uncertainty explicit. Watch for recurrence in a non-infrastructure trial before adding to universal standards. Keep it in the generated OpenClaw maintenance skills for now.Async-completion candidate ("accepted/submitted ≠ completed"): PARTIAL / NEEDS MORE EVIDENCE
Search of current PE prompts: PE already distinguishes acceptance vs completion implicitly via read-back/verification requirements (Ship requires read-back; Trial is evidence-driven), but there is no explicit Pattern/Review/Trial acceptance criterion stating that submission/acceptance is not completion and a success claim requires terminal-state evidence or post-operation read-back appropriate to the target. One more independent occurrence (updates, CI jobs, cloud ops, exports, background workflows) would justify a very small acceptance criterion. Not implemented in this gate.
Retrospective Issue #11 activation/routing (post-PR10 completeness contract)
Private trial receipts show T1-T8 behavioral cases with live/session evidence (all PASS) + deploy-time collision trial PASS + read-back. No independent should-trigger / shouldn't-trigger / parent→child routing set is recorded. Therefore, under the post-PR10 completeness contract:
The shipped package is not undone; operator GO + deploy read-back are valid. This is another instance of the completeness problem PR fix: require capability preflight and complete trial evidence #10 fixes.
Exact candidate next action
Define (do not implement) a small R1 destination-catalog preflight (Pattern + Review only, read-only, UNPROVEN fallback, no new stage/manifest). Requires exposing the native Turnstone read/find/get catalog surface to the generator (Turnstone integration). No new PE change was implemented in this gate.
RedEyeNinja-BKK commentedon Aug 10, 2026
R1 change definition - destination-skill collision preflight (defined, NOT implemented)
References:
5234968235,5235103927. This is the definition only; no PE prompt change, no branch/PR, no Turnstone integration change performed.1. Scope narrowed to generated skills first
Issue #11 evidence is a skill catalog collision (
Openclaw Managementvsopenclaw-management/OpenClaw Management). Therefore R1 targets generated Turnstone skills only - not personas, projects, policies, templates, or all Turnstone objects. Those may later need the same treatment; Issue #11 does not establish that yet.2. Product problem
PE can select a new generated skill identity without proving relevant existing destination-catalog state, so authoring/review proceed while a material name collision is only ASSUMED. Correction: once Pattern has selected a generated skill's canonical identity, inspect the destination Turnstone skill catalog through a native read-only surface before finalizing the draft. This is destination-state evidence, not design authority.
3. Turnstone-native architecture
PE must not depend on Hermes, filesystem search, a custom catalog DB, a PE registry, a duplicate manifest, or custom collision code. The required runtime surface is a Turnstone-native read-only skill-catalog capability exposed to the generator: conceptually find/search proposed skill identity + fetch/get a returned skill object. No write/update/enable/disable/delete authority.
4. Absence semantics
search returned nothing≠PROVEN ABSENTunless the tool contract supports that conclusion.5. No invented normalization doctrine
First implementation around: proposed Agent Skills canonical
name; Turnstone's native skill identity/name fields; exact native lookup semantics; obvious returned identity conflicts. If the native catalog provides canonical/normalized matching, use it. No custom fuzzy/semantic matching machinery in PE. A broader similarity search may produce a heads-up when useful, but must not be mandatory ceremony.6. Pattern behavior
After a generated skill's canonical name is chosen: (1) if destination Turnstone catalog read access is available, perform the bounded lookup; (2) fetch enough returned identity data to distinguish actual skill objects; (3) record PROVEN PRESENT / PROVEN ABSENT / UNPROVEN; (4) preserve distinct returned identities; (5) never silently choose overwrite/coexist/supersede; (6) do not mutate anything. If PROVEN PRESENT: surface the collision and propose dispositions (retain both / rename new / later supersede / intentionally replace if separately supported). No disposition is automatic.
7. Review behavior
create new → operator GO → disable old → read-backmay be valid where Turnstone supports it.8. Ship remains authority boundary
Actual catalog mutation stays in Ship after Review PASS + Trial PASS + explicit operator approval. R1 grants Pattern/Review no create/update/enable/disable/delete/supersede. Ship unchanged.
9. PE-product vs Turnstone-integration split
Recommended sequence: define PE prompts against the intended native surface first (repository-first), then integrate/expose the surface; keep the two changes separable and reviewable.
10. Acceptance criteria C1-C6
11. Smallest affected PE artifact set
Pattern Author + Review only. Core/Trial/Ship unchanged (Ship already owns mutation authority).
R2 / R3 / async - retained classifications (unchanged)
No implementation performed in this gate.
RedEyeNinja-BKK commentedon Aug 10, 2026
R1 implementation evidence - PR #12 (repo-only; live integration held)
References:
5234968235,5235103927,5235225699.Implementation PR #12:
fix: preflight destination skill collisionsa6b81ff349a650440c553b4a0c3da38b486958ca, head80d200e935aceee2611d66925c686b77c15ab1e1Implemented: Pattern destination-skill preflight for generated Turnstone skills (PROVEN PRESENT / PROVEN ABSENT / UNPROVEN; no custom normalization/fuzzy engine; preserves distinct identities; no automatic disposition; zero mutation) + Review destination-skill evidence check (ABSENT ok; PRESENT requires explicit reversible operator-gated disposition; UNPROVEN → REVISE for deployment-bound; zero mutation).
C1-C7 regression results (bounded fixtures, labeled not-live): all PASS - receipts in PR #12 comment
5235318860. C1 proven collision → PROVEN PRESENT + disposition required; C2 authoritative absence → PROVEN ABSENT; C3 catalog unavailable → UNPROVEN + REVISE (no filesystem fallback); C4 case-variant identities preserved (no silent equivalence); C5 distinct-but-similar → no false gate; C6 zero mutation across all; C7 persona generation does not trigger skill preflight (skills-only scope).Live generator integration is still NOT changed. The current generator does not yet expose the desired native Turnstone read-only skill-catalog surface; this PR is the repository prompt change only. Hermes is evidence the collision exists, not the permanent PE integration. Live integration remains held pending (a) drift containment (Issue #8
5235223880/5235291721: recurrence risk MEDIUM, mechanism UNKNOWN) and (b) a separately approved Turnstone catalog-read exposure gate.R2 / R3 / async classifications unchanged: R2 MOSTLY COVERED BY PR #10; R3 PACKAGE-SPECIFIC / NEEDS MORE EVIDENCE; accepted/submitted ≠ completed PARTIAL / NEEDS MORE EVIDENCE.
No live PE mutation, no catalog exposure, no tool-assignment change, no version/release, no Issue closure.
RedEyeNinja-BKK commentedon Aug 10, 2026
R1 identity-binding correction (PR #12, head
53c1963…)Independent review
4893388004found a stale-identity evidence hole: Review could accept a rename plan using destination evidence gathered for the old canonical skill name. Correction implemented on PR #12 (Pattern + Review only):name, that is a revision: rename draft → re-run preflight for the new name → record fresh evidence → return to Review. No re-run when the disposition leaves the canonical name unchanged. Point-in-time semantics: PRESENT/ABSENT is preflight-time author/review evidence, not a Ship-time transactional guarantee (no locks/leases/Ship preflight).namein the draft being reviewed; stale receipt or future rename-plan → REVISE (rename must happen in Pattern first with fresh evidence); completed rename with fresh receipt → evaluate normally.C8-C11 results (receipts in PR #12 comment
5235465675):New PR #12 head:
53c19638875595ed4ca86d3af9f6e72aed68e687(basea6b81ff…); structural validation PASS. Live integration remains held (drift MEDIUM/UNKNOWN; separate catalog-read exposure gate). R2/R3/async classifications unchanged (MOSTLY COVERED BY PR10 / PACKAGE-SPECIFIC / PARTIAL). No live PE mutation, no catalog exposure, no version/release, no Issue closure.RedEyeNinja-BKK commentedon Aug 10, 2026
R1 controlled live integration + alignment gate - result (2026-08-10)
PR #12 merged at
076dc4d323dba449cf416c1f1db948e997b0b334from the exact reviewed head53c19638875595ed4ca86d3af9f6e72aed68e687. Canonical repo implementation of R1 (destination-skill collision preflight) is complete onmain. Released tagv1.9.6remains at65eaba888322609afa9986ff1d19e959317f5e28(unchanged; no release/version decision made).Pre-live state was INTENTIONAL REPO-AHEAD - live Turnstone held the pre-PR12 canonical basis while the merged candidate awaited a separate live-integration gate.
Controlled live integration (this gate)
sync-rollback-2026-08-10-r1-live/.076dc4d…; all other PE objects untouched (Core/Trial/Ship/Triage, references, templates, generator persona content, development persona, policies/judge unchanged).076dc4d…product content (normalized byte comparison; storage trims the final newline).Catalog capability (conceptual level, as required)
R1 needs a Turnstone-native authoritative read-only skill-catalog surface. The existing native surface already provides it:
template_id(admin read); canonical Agent Skillsnameis the queried identity;getby canonical name is exact-match, case-sensitive, returns the full object, and returns an explicit not-found for a missing identity → this is the authoritative presence/absence surface;find/search query is BM25-ranked and cannot authoritatively prove absence (verified live: an exact-name row can be missed byfind);findsupportsenabled_only=false; catalog currently has zero disabled rows, so disabled-get behavior remains unproven live (contract supports it);No custom PE catalog service was introduced; no global tool-authority expansion; Hermes is not a dependency of R1.
Controlled R1 smoke (S1-S4) - actual persisted generator
Run through the real generator persona with the deployed Pattern skill and the real native catalog surface. No catalog writes.
process-engine-core; Pattern invoked native read; recorded PROVEN PRESENT with queried name + returned destination identity + evidence source as one identity-bound unit; zero mutation; no automatic disposition.getreturned explicit not-found → PROVEN ABSENT (native semantics genuinely prove it);findzero-result treated as corroboration only, not proof.process-engine-corereceipt explicitly invalidated for the new name; fresh preflight recorded for the renamed identity; zero mutation.Alignment and risk
076dc4d…= live six-skill PE content, with the generator receiving only the minimum native read-only catalog read capability (already present in its default toolset; no new write surface introduced).Issue #11 remains open; no issue closure performed.