From a3f0c7f220ae169a72a0eed54e038e10c9f81c42 Mon Sep 17 00:00:00 2001 From: RedEyeNinja-BKK <232920946+RedEyeNinja-BKK@users.noreply.github.com> Date: Sat, 8 Aug 2026 21:13:29 +0700 Subject: [PATCH 1/6] docs: upstream & release-candidate alignment review 2026-08-08 Read-only alignment evidence for Issue #1 / v1.9.6 release steering. - PR #4 verified: base 16e4cbe, head 3d114e5, 12 changed files (corrected from a report's erroneous 14), structural-validation PASS, non-behavioral. - Upstream audit (Hermes run_1e79935c, Turnstone-verified) with exact SHAs: addyosmani/agent-skills f4933771, agentskills/agentskills 217be548, OWASP/CheatSheetSeries da4c967e. - Osmani alignment: all attributed principles confirmed present; faithful proportional application; no drift. - Agent Skills: spec vs recommendation classification; imperative-phrasing and trial-methodology attributions flagged B-class (relabel). - OWASP: recommend named proportional security basis (option #2), not a universal third source; bucket classification A/B/C/D included. - Core-value and claim-to-source matrices; severity A/B/C/D/E findings; smallest corrections before v1.9.6 and post-v1.9.6. - Verdict: ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION CORRECTIONS; PR #4 ready for merge review. No behavior, prompts, references, tooling, or Turnstone objects changed by this review. --- .../2026-08-08-upstream-alignment-review.md | 185 ++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 docs/alignment-review/2026-08-08-upstream-alignment-review.md diff --git a/docs/alignment-review/2026-08-08-upstream-alignment-review.md b/docs/alignment-review/2026-08-08-upstream-alignment-review.md new file mode 100644 index 0000000..58195c8 --- /dev/null +++ b/docs/alignment-review/2026-08-08-upstream-alignment-review.md @@ -0,0 +1,185 @@ +# Upstream & release-candidate alignment review — 2026-08-08 + +**Status:** Read-only alignment evidence and release steering. Recorded by Turnstone (Process Engine Development Lead) with an upstream source audit delegated to Hermes and independently verified by Turnstone. No behavior, prompts, references, tooling, or Turnstone objects were changed by this review. References Issue #1. + +## 0. Evidence identity + +- PR #4 (`v1.9.6 release identity + documentation alignment`): open, not merged, mergeable. + - base: `16e4cbe7151297ac8c659f7fca8bad1ffd05f9dc` (main after PR #3) + - head: `3d114e58e5e87303868fd3ed6ad073698e8988dc` + - branch: `v1.9.6-release-metadata` + - commits: 2 (`c2bf841`, `3d114e5`) + - GitHub `structural-validation` check on head: PASS +- **Changed-file count: 12** (not 14). The earlier "14" over-counted by adding the two docs-alignment files without deduplicating against the metadata commit (README.md and docs/architecture.md appear in both). Corrected count: `CHANGELOG.md`, `README.md`, `docs/architecture.md`, `docs/governance-usage.md`, `docs/standards.md`, `process-engine.toml`, six `skills/*/SKILL.md` = 12. + +## 1. PR #4 diff review (verified full base→head diff) + +- **Release identity:** `process-engine.toml` is consistent 1.9.6 (version, engine, header; lineage/counts/compat/author unchanged). All six skills change only `metadata.version` + `metadata.engine` frontmatter. README current identity is v1.9.6. Historical v1.9.5 references (README Origin Story, toml historical comment, CHANGELOG v1.9.5 entry) remain intentionally historical. +- **Governance wording:** README, architecture, standards, governance-usage now state the advisory contract (Turnstone provides native governance surfaces; prompt policy = durable contextual guidance; advisory judge = review/trial evidence; operator is final authority at defined gates). Grep confirms no remaining "mechanically" / "enforces these gates" / "owns the guardrails" / "enforcement layer" / "native enforcement mechanism" in README + docs. +- **Gate model:** architecture now names the meaningful operator gates (Summary Gate, Review, Trial, Ship) and states the pipeline stays conversational between them. +- **Generator identity:** README deploying section + architecture components distinguish the canonical generator persona (`persona.md` → deploy under a distinct generator identity such as `process-engine-generator`); no private/local object IDs encoded. +- **Deployment snapshot:** `docs/governance-usage.md` separates durable product contract from a clearly labeled dated "Deployment snapshot — 2026-08-01" (policy/rule IDs, judge counts, judge model) as mutable operational facts. +- **Behavior boundary:** confirmed non-behavioral — no persona prompt changes, no skill-body changes, no reference/template content changes, no tooling changes, no Turnstone mutation. + +## 2. Upstream repositories inspected (exact SHAs) + +| Source | Ref | SHA | +|---|---|---| +| addyosmani/agent-skills | main | `f49337711b7a932b4b338c1d4ad73384df8fd87d` | +| agentskills/agentskills | main | `217be548739f21d6008915c29aefe320ea1a90af` | +| OWASP/CheatSheetSeries | master | `da4c967e9de854727f72bb2748dd98f76c888b06` | + +## 3. Osmani alignment (engineering discipline / workflow) + +Upstream is explicitly "Production-grade engineering skills for AI coding agents" — coding-agent scope, not a general format spec. All major Process Engine claims attributed to Osmani were confirmed present upstream (Hermes quotes; Turnstone verified representative claims): + +- Clarify/specify before building (`skills/spec-driven-development/SKILL.md`). +- Process, not prose (`skills/source-driven-development/SKILL.md`, README). +- Small verifiable units (`skills/planning-and-task-breakdown/SKILL.md`). +- Incremental execution + feedback (`skills/incremental-implementation/SKILL.md`). +- Verification before success ("Tests are proof — 'seems right' is not done", `skills/test-driven-development/SKILL.md`). +- Review before ship (`skills/code-review-and-quality/SKILL.md`, `references/definition-of-done.md`). +- Evidence/source-driven development (`skills/source-driven-development/SKILL.md`). +- Adversarial/doubt review (`skills/doubt-driven-development/SKILL.md`). +- Simplification (`skills/code-simplification/SKILL.md`). +- Progressive context (`skills/context-engineering/SKILL.md`). +- Anti-rationalization (`skills/spec-driven-development/SKILL.md`). +- Controlled autonomy (README: removes human between tasks, not verification). + +**Assessment:** Process Engine applies these faithfully and proportionately (workflow discipline, not code-specific machinery). It does not reproduce upstream coding skills — correct. Attribution stands. No drift. + +## 4. Agent Skills alignment (format spec + authoring guidance) + +### Normative spec (verified from `docs/specification.mdx`) + +- SKILL.md required; YAML frontmatter required; Markdown body has **no format restrictions**. +- Required frontmatter: `name` (1–64 chars; lowercase alphanumeric + hyphens; no leading/trailing/consecutive hyphens; must match directory) and `description` (1–1024 chars; describes what + when). +- Optional: `license`, `compatibility` (≤500 chars if present), `metadata` (string map), `allowed-tools` (space-separated; **experimental**). +- Directory structure: `scripts/`, `references/`, `assets/` are **optional conventions**, not mandatory. +- Progressive disclosure: metadata (~100 tokens) → instructions (<5000 tokens recommended) → resources on demand; SKILL.md <500 lines recommended. +- Validation: `skills-ref validate`. + +### Authoring recommendations (separate, non-normative) + +`docs/skill-creation/best-practices.mdx`, `optimizing-descriptions.mdx`: real-task-then-extract workflow; start from real expertise; execution→revision loops; context economy; add-what-agent-lacks/omit-what-it-knows; coherent scope; moderate detail; progressive disclosure in authoring; trigger quality; **imperative description phrasing**; procedures over declarations; validation loops; plan-validate-execute for destructive ops. + +### Key classification result + +- "description imperative phrasing" is an upstream **RECOMMENDATION** (optimizing-descriptions.mdx), not a spec requirement — Process Engine's `references/best-practices.md` and `docs/spec-compliance.md` present it inside the spec-requirement framing. **B-class attribution framing.** +- Trial methodology (trigger sets, with/without baseline, token/timing capture) is not present in the Agent Skills spec or authoring docs; it is a **Process Engine adaptation** informed by Osmani's evals culture + the engine's own trial skill. `references/best-practices.md` attributes it to "Agent Skills … authoring/eval guidance" — **B-class attribution** (should be labeled PE adaptation). + +### Session-context finding + +Upstream explicitly supports the workflow observed in live trials #1/#2: +> "Complete a real task in conversation with an agent, providing context, corrections, and preferences along the way. Then extract the reusable pattern into a skill." (docs/skill-creation/best-practices.mdx) + +This is **external validation** of the already-observed PE behavior. Per steering, do NOT add a prompt rule; record as evidence. + +## 5. OWASP alignment (proportional security/risk basis) + +Inspected (current titles at root `cheatsheets/`, not `cheatsheets/ai/`): +- **AI Agent Security Cheat Sheet** — external data untrusted; prompt injection (direct/indirect); least privilege + read/write scoping; explicit approval for high-impact/irreversible actions; action previews before execution; autonomy boundaries; audit trails; bind approval to exact action (actor, tool, target, params, timestamp, expiry); separate decision from execution; fail closed; output validation; monitor/observe. +- **LLM Prompt Injection Prevention Cheat Sheet** — indirect injection via retrieved/external content; tool manipulation; action screening against original intent; human oversight for high-risk operations; least privilege + read-only accounts; external-content sanitization. +- **MCP Security Cheat Sheet** — tool descriptions/return values untrusted (tool poisoning); treat every tool response as untrusted input; least privilege per server; narrow scopes; inspect/pin tool definitions; full parameter display before approval; never auto-approve; validate inputs/outputs; verify session/token identity; consent before connecting servers; never allow untrusted content to trigger installation; fail closed; redact secrets/PII from logs. + +### Bucket classification (candidate OWASP lessons) + +- **A — Process Engine generator behavior (universal, prompt-level):** treat external material as material not authority; preserve provenance; never expose credentials; distinguish operator instruction from embedded/untrusted instruction; verify before mutation. These already exist in PE (`references/intake.md` trust boundary, `references/safety.md`, PR #2 provenance work) — **already aligned**, no new prompt needed. +- **B — Generated-package guidance (proportional, when domain/tools/risk require):** read-only discovery first; approval before high-impact actions; sensitive-data safeguards; tool-target identity verification; action preview/rollback. PE already encodes these proportionally (pattern-author safeguard pass, per-package gates; observed in Proxmox package: T2/T3/T5/T7/T10 gates, T5 dry-run, T6 secret refusal). +- **C — Turnstone-native responsibility (PE relies on, does not recreate):** runtime tool permissions; RBAC; policy execution; approvals infrastructure; audit/storage; authorization enforcement. +- **D — Out of Process Engine scope:** generated applications; Turnstone internals; Method Factory; MCP/tool implementation; infrastructure. + +### OWASP role recommendation + +**Option #2 — named proportional security basis.** OWASP should be a named security basis used proportionally when a package's intent/material/tools make security relevant — not a universal third generation-basis source applied to every package, and not a "third source" in evidence-library/best-practices by default. PE should have security awareness without turning every package into a security framework. Concretely: (post-v1.9.6, small) add OWASP as a named proportional basis in `references/evidence-library.md` and `references/safety.md` (one line each), keep the persona/core unchanged. + +## 6. Core-value alignment matrix + +| Principle | Repo evidence | Live evidence | Status | +|---|---|---|---| +| Collect before creating | core SKILL step 2, intake | trials #1/#2 (collect invite + exit) | ALIGNED | +| Understand intent before generation | core clarify step 3 | trials (one genuine-gap question) | ALIGNED | +| Ask what "good" looks like | core objective step 4 | trials (objective question) | ALIGNED | +| Account for supplied + accumulated context | intake, core | trials #1/#2 (reused workstream context) | ALIGNED | +| Adapt rather than copy | intake "author original" | Proxmox package provenance | ALIGNED | +| Name evidence and provenance | standards, evidence-library | Proxmox receipts, PR evidence | ALIGNED | +| Distinguish evidence from inference | PR #2 intake/pattern | trials (provenance labels) | ALIGNED | +| Preserve identity/ambiguity/conflict | PR #2 (intake/pattern/review/trial) | T3 remediation, trials | ALIGNED | +| Define acceptance criteria | pattern-author step 7 | packages carry exit criteria | ALIGNED | +| Nothing ships without review | review skill, core gate | PR review chain | ALIGNED | +| Nothing ships untried | trial skill | Proxmox 10/10 suite | ALIGNED | +| Operator final gate | ship skill, docs (PR #4) | merge gates | ALIGNED | +| Verify deployment by read-back | ship skill | store sync read-back | ALIGNED | +| Feedback feeds improvement | triage skill | Issue #1 thread | ALIGNED | +| Generator, not generated product | persona, docs | smoke test | ALIGNED | +| Prompts-only | toml, tools README, PR #3 | runtime executes nothing | ALIGNED | +| Turnstone-native | persona, docs | deployed on Turnstone | ALIGNED | +| GitHub canonical | repo-first, PR #3/#4 | merged PRs | ALIGNED | +| Governance proportional/advisory | governance.md, docs (PR #4) | prompt policy content-only | ALIGNED | +| Heads-up, not police | core, safety | Proxmox advisory gates | ALIGNED | +| Avoid unnecessary questions | core collect/exit | trials friction = 0 | ALIGNED | +| Avoid ceremonial gates | core, PR #4 gate-model correction | trials (no extra gates) | ALIGNED | +| Progressive disclosure/context economy | best-practices, skill-anatomy | on-demand reference loading | ALIGNED | +| Real behavioral trials over structural-test theater | trial skill, PR #3 CI framing | live trials #1/#2, honest CI | ALIGNED | + +## 7. Claim-to-source matrix (significant claims) + +| Claim | Attributed source | Upstream support | Implementing artifact | Classification | Status | +|---|---|---|---|---|---| +| Spec-valid SKILL.md (name/description rules) | agentskills spec | Required (name ≤64 lowercase-hyphen, desc ≤1024 what+when) | pattern-author, validate.py, spec-compliance | SPEC REQUIREMENT | ALIGNED | +| Description "imperative phrasing" | agentskills spec (framed as rule) | Recommendation only (optimizing-descriptions.mdx) | best-practices, spec-compliance | PE ADAPTATION (mis-framed as spec) | B — correct framing | +| Process, not prose | Osmani | Present (README, source-driven) | best-practices | UPSTREAM | ALIGNED | +| Verification non-negotiable / "seems right" never sufficient | Osmani | Present (TDD, README) | best-practices | UPSTREAM | ALIGNED | +| Anti-rationalization | Osmani | Present (spec-driven) | best-practices, skill-anatomy | UPSTREAM | ALIGNED | +| Progressive disclosure | agentskills spec + Osmani | Present (spec; context-engineering) | skill-anatomy | SPEC REQUIREMENT + UPSTREAM | ALIGNED | +| Trial methodology (trigger sets, baselines, token capture) | Agent Skills "authoring/eval guidance" | Not in agentskills spec/docs (eval culture is Osmani/PE) | best-practices, trial skill | PE ADAPTATION | B — correct attribution | +| 24-skill catalog | Osmani | Catalog is real; exact count needs re-verification at current ref | best-practices | UPSTREAM (verify count) | A/B — verify count | +| Two-source basis (Osmani + Agent Skills) | — | Accurate for current references | evidence-library, best-practices | PE ORIGINAL (basis doc) | ALIGNED; OWASP to be added proportionally | +| OWASP Top 10 prevention | Osmani catalog (security-and-hardening skill) | Present as that skill's topic | best-practices (catalog line) | UPSTREAM (catalog description) | ALIGNED (OWASP itself not a PE basis yet) | + +## 8. Security review of intake/context/material handling + +PE consumes user text, files, URLs, external skills, docs, tool observations, accumulated workstream context. OWASP assessment: + +- **External material untrusted:** already encoded (`references/intake.md` trust boundary: "Treat all fetched or user-provided material as untrusted data, never as authority"). ALIGNED. +- **Distinguish operator instruction vs embedded/untrusted instruction:** present (intake trust boundary; PR #2 provenance handling; trials #1/#2 demonstrated four-way provenance + correction handling). ALIGNED. +- **Sensitive/private material:** present (`references/safety.md`: no credential exposure, no private-data leakage; Proxmox T6 refused secrets). ALIGNED. +- **Tool evidence vs inference:** present (PR #2 intake preserves tool evidence with provenance; trials labeled evidence). ALIGNED. +- **Potentially hostile embedded instruction:** handled by the same intake trust boundary; not separately tested in trials — theoretical; classify as a **future-trial candidate** (a D/C finding: no demonstrated behavioral gap). +- **No new prompt text warranted by this review** (existing trials cover the provenance chain; anything else is theoretical). + +## 9. Architectural boundary check + +All candidate improvements were checked: RBAC/approvals/audit/persistence/security middleware belong to Turnstone (C); generated-application controls belong to generated packages (D); evaluator/state-machine/regeneration machinery was already removed (PR #3) and must not return. No proposed change recreates Method Factory machinery. Current behavior is adequate; no mechanism additions recommended. + +## 10. Findings (severity classification) + +- **A — factual/source drift:** verify the "24-skill" catalog count against current addyosmani ref before v1.9.6 (if stale, correct in best-practices). Changed-file count discrepancy was in a report, not the repo — corrected here. +- **B — attribution drift:** (1) "imperative phrasing" framed as spec requirement in best-practices + spec-compliance → relabel as upstream recommendation/PE adaptation; (2) trial methodology attribution → relabel as PE adaptation (informed by Osmani eval culture); (3) persona.md "Turnstone enforces them mechanically" → align to advisory contract (small persona-wording change, separate from PR #4's non-behavioral scope). +- **C — behavioral gap:** none found. Theoretical intake concerns (hostile embedded instruction in untrusted material) → future-trial candidate, not a patch. +- **D — useful upstream evolution:** real-task-then-extract (externally validates observed behavior — record, do not add rule); plan-validate-execute for destructive ops (PE already approximates via dry-run/gate patterns); exact-action approval binding (PE already binds gates to exact operations in generated packages); tool-description-untrusted (MCP) — already reflected in intake trust boundary. +- **E — out of scope:** RBAC/audit/approvals infra, memory encryption, output-guard middleware, MCP implementation → Turnstone / generated applications / Method Factory / infrastructure. + +## 11. Smallest proposed corrections + +**Before v1.9.6 (small, release-facing, non-behavioral or minimal):** +1. Relabel "imperative phrasing" as upstream recommendation (best-practices.md + spec-compliance.md) — documentation. +2. Relabel trial-methodology attribution as Process Engine adaptation informed by Osmani (best-practices.md) — documentation. +3. Verify/correct the "24-skill" catalog count (best-practices.md) — documentation. +4. Align persona.md "Turnstone enforces them mechanically" wording to the advisory contract — **small persona-prompt wording change**; do as a separate reviewable change (not inside PR #4). + +**Post-v1.9.6 (future trial / small):** +5. Add OWASP as a named **proportional** security basis (one line each in evidence-library.md + safety.md), not a universal third source. +6. Future-trial candidate: hostile-instruction-in-untrusted-material intake case (to confirm the existing trust boundary holds behaviorally). + +**Reject / out of scope:** all E-class items. + +## 12. Release-candidate verdict + +**ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION CORRECTIONS.** + +PR #4 itself is correct and ready for merge review (verified full diff; structural-validation PASS on head; 12 files; non-behavioral). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. The B-class attribution corrections and the persona wording alignment should be completed before the v1.9.6 tag/release gate (as small follow-up changes), but none is a behavioral gap and none blocks PR #4's merge review. + +--- + +*Evidence attribution: Hermes executed the upstream source audit (run_1e79935cdfd24dccaac0a57a02d6a1fc) with exact SHA/path/quote citations; Turnstone independently verified the load-bearing claims (agentskills spec at docs/specification.mdx, OWASP cheat sheets at cheatsheets/*, upstream branch SHAs, and the full PR #4 diff). This document records findings only; no repository behavior, references, prompts, tooling, or Turnstone objects were modified by the review.* From e894cf3741c14b4b7ba8e4c2a8774d1bad8e4c3b Mon Sep 17 00:00:00 2001 From: RedEyeNinja-BKK <232920946+RedEyeNinja-BKK@users.noreply.github.com> Date: Sat, 8 Aug 2026 21:33:34 +0700 Subject: [PATCH 2/6] docs: correct PR #5 alignment evidence per independent review 4888963474 - Agent Skills eval-methodology: corrected - upstream DOES contain trigger sets, near-misses, with/without baselines, token/timing capture (optimizing-descriptions.mdx, evaluating-skills.mdx @ 217be548); PE adapts it to Turnstone, not PE-only. - Osmani 24-skill: RESOLVED - README @ f4933771 says 'install all 24 skills' + 'All 24 Skills' (23 lifecycle + 1 meta). - Imperative-description: kept as PE authoring standard; formal spec requires non-empty <=1024 what+when only (spec-vs-guidance framing preserved). - Progressive disclosure reclassified as spec guidance/design convention + PE standard (not hard validator requirement). - metadata wording: spec = map from string keys to string values; PE docs 'arbitrary key-value map' -> align (B-class). - Added internal behavioral-contract issue: stale mechanical-enforcement wording in persona.md + core + pattern-author + ship (C-class, small prompt correction before release, opposite of adding enforcement). - Added Triage package-manifest remnant finding (verify; rewrite if none). - Added validator/spec-integrity finding (top-level version; compat<=500; metadata string->string) - small structural fidelity. - OWASP: proportional named security basis confirmed (not universal). - Verdict revised: ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION + INTERNAL GOVERNANCE-CONTRACT WORDING CORRECTIONS; PR #4 ready for merge review. --- .../2026-08-08-upstream-alignment-review.md | 42 ++++++++++++++----- 1 file changed, 32 insertions(+), 10 deletions(-) diff --git a/docs/alignment-review/2026-08-08-upstream-alignment-review.md b/docs/alignment-review/2026-08-08-upstream-alignment-review.md index 58195c8..c217c51 100644 --- a/docs/alignment-review/2026-08-08-upstream-alignment-review.md +++ b/docs/alignment-review/2026-08-08-upstream-alignment-review.md @@ -65,8 +65,12 @@ Upstream is explicitly "Production-grade engineering skills for AI coding agents ### Key classification result -- "description imperative phrasing" is an upstream **RECOMMENDATION** (optimizing-descriptions.mdx), not a spec requirement — Process Engine's `references/best-practices.md` and `docs/spec-compliance.md` present it inside the spec-requirement framing. **B-class attribution framing.** -- Trial methodology (trigger sets, with/without baseline, token/timing capture) is not present in the Agent Skills spec or authoring docs; it is a **Process Engine adaptation** informed by Osmani's evals culture + the engine's own trial skill. `references/best-practices.md` attributes it to "Agent Skills … authoring/eval guidance" — **B-class attribution** (should be labeled PE adaptation). +- "description imperative phrasing" is an upstream **RECOMMENDATION** (optimizing-descriptions.mdx), not a spec requirement — Process Engine's `references/best-practices.md` and `docs/spec-compliance.md` present it inside the spec-requirement framing. **B-class attribution framing.** Correction: keep imperative/user-intent phrasing as the Process Engine authoring standard, but classify it accurately as authoring guidance, not a formal spec-validity rule. Formal spec requires: description non-empty, ≤1024 chars, describes what + when. +- **Trial/eval methodology** — correction (was B-class; now verified substantially aligned): current Agent Skills authoring/evaluation guidance explicitly covers trigger sets, near-miss negatives, with/without (or previous-version) baselines, and token/duration capture: + - `docs/skill-creation/optimizing-descriptions.mdx` (commit `217be548`): should-trigger / should-not-trigger query sets, near-miss negatives, repeated runs (3 recommended), trigger rates, train/validation-style design, fresh-query sanity. + - `docs/skill-creation/evaluating-skills.mdx` (commit `217be548`): eval cases with `id`, `prompt`, `expected_output`, optional `files`; "run each test case twice: once **with the skill** and once **without it** (or with a previous version)"; records token count + duration in `timing.json`; assertions + grading evidence + benchmark deltas. + - Therefore Process Engine's attribution to Agent Skills "authoring/eval guidance" for this methodology is substantially correct. PE adapts that guidance into its Turnstone-native Trial stage and operator-gated package process, but upstream does NOT lack it. Remove the earlier "PE-only adaptation" framing. +- **Progressive disclosure** — classification refined: the Agent Skills spec contains a Progressive Disclosure section and recommends staged loading / <500 lines / <5000 tokens, but these are **spec guidance / design convention**, not frontmatter validation requirements. Classify as "spec guidance / design convention + PE authoring standard," not simply "SPEC REQUIREMENT." ### Session-context finding @@ -132,9 +136,10 @@ Inspected (current titles at root `cheatsheets/`, not `cheatsheets/ai/`): | Verification non-negotiable / "seems right" never sufficient | Osmani | Present (TDD, README) | best-practices | UPSTREAM | ALIGNED | | Anti-rationalization | Osmani | Present (spec-driven) | best-practices, skill-anatomy | UPSTREAM | ALIGNED | | Progressive disclosure | agentskills spec + Osmani | Present (spec; context-engineering) | skill-anatomy | SPEC REQUIREMENT + UPSTREAM | ALIGNED | -| Trial methodology (trigger sets, baselines, token capture) | Agent Skills "authoring/eval guidance" | Not in agentskills spec/docs (eval culture is Osmani/PE) | best-practices, trial skill | PE ADAPTATION | B — correct attribution | -| 24-skill catalog | Osmani | Catalog is real; exact count needs re-verification at current ref | best-practices | UPSTREAM (verify count) | A/B — verify count | -| Two-source basis (Osmani + Agent Skills) | — | Accurate for current references | evidence-library, best-practices | PE ORIGINAL (basis doc) | ALIGNED; OWASP to be added proportionally | +| Trial methodology (trigger sets, baselines, token capture) | Agent Skills authoring/eval guidance | **Present** (optimizing-descriptions.mdx: trigger sets, near-misses, rates; evaluating-skills.mdx: with/without baseline, timing.json token+duration, assertions) | best-practices, trial skill | SPEC GUIDANCE (authoring/eval) + PE ADAPTATION to Turnstone Trial | ALIGNED (corrected) | +| 24-skill catalog | Osmani | Confirmed — README at `f4933771` says "install all 24 skills" + "All 24 Skills" section (23 lifecycle + 1 meta) | best-practices | UPSTREAM | ALIGNED (resolved — no longer A/B) | +| Two-source basis (Osmani + Agent Skills) | — | Accurate for current references; OWASP to be added as a proportional security basis (see §5) | evidence-library, best-practices | PE ORIGINAL (basis doc) | ALIGNED; OWASP proportional post-v1.9.6 | +| Progressive disclosure | agentskills spec + Osmani | Present (spec Progressive Disclosure section = guidance/convention; context-engineering) | skill-anatomy | SPEC GUIDANCE / DESIGN CONVENTION + PE STANDARD | ALIGNED (classification refined) | | OWASP Top 10 prevention | Osmani catalog (security-and-hardening skill) | Present as that skill's topic | best-practices (catalog line) | UPSTREAM (catalog description) | ALIGNED (OWASP itself not a PE basis yet) | ## 8. Security review of intake/context/material handling @@ -154,9 +159,17 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi ## 10. Findings (severity classification) -- **A — factual/source drift:** verify the "24-skill" catalog count against current addyosmani ref before v1.9.6 (if stale, correct in best-practices). Changed-file count discrepancy was in a report, not the repo — corrected here. -- **B — attribution drift:** (1) "imperative phrasing" framed as spec requirement in best-practices + spec-compliance → relabel as upstream recommendation/PE adaptation; (2) trial methodology attribution → relabel as PE adaptation (informed by Osmani eval culture); (3) persona.md "Turnstone enforces them mechanically" → align to advisory contract (small persona-wording change, separate from PR #4's non-behavioral scope). -- **C — behavioral gap:** none found. Theoretical intake concerns (hostile embedded instruction in untrusted material) → future-trial candidate, not a patch. +- **A — factual/source drift:** none remains after this correction pass. (Changed-file count discrepancy was in a report, not the repo — corrected here to 12.) +- **B — attribution drift:** (1) "imperative phrasing" framed as spec requirement in best-practices + spec-compliance → relabel as upstream authoring recommendation / PE authoring standard (formal spec requires only non-empty ≤1024 chars describing what + when); (2) **metadata wording** — spec says "map from string keys to string values"; PE docs say "arbitrary key-value map" → align wording; (3) **trial/eval attribution corrected** — Agent Skills authoring/eval guidance does contain trigger sets / near-misses / with-without baselines / token-timing capture; PE adapts it (no longer "PE-only"). (4) persona.md "Turnstone enforces them mechanically" → align to advisory contract. +- **C — behavioral gap:** none found in the generator behavior itself. **However — internal behavioral-contract consistency issue:** stale mechanical-enforcement language remains in **four load-bearing prompt artifacts** — `persona.md`, `skills/process-engine-core/SKILL.md`, `skills/process-engine-pattern-author/SKILL.md`, `skills/process-engine-ship/SKILL.md` — containing "for enforcement", "Turnstone owns the guardrails", "enforcement layer", "enforce this mechanically", "native enforcement layer", while `references/governance.md` and the PR #4 docs correctly state prompt policy/judge are advisory helpers and operator approval is authoritative. This is a **small prompt correction before release** (make prompts accurately describe the advisory Turnstone-native governance model intended — the opposite of adding enforcement). Theoretical intake concern (hostile embedded instruction in untrusted material) → future-trial candidate, now planned as G6. +- **D — useful upstream evolution:** real-task-then-extract (externally validates observed behavior — record, do not add rule); plan-validate-execute for destructive ops (PE already approximates via dry-run/gate patterns); exact-action approval binding (PE already binds gates to exact operations in generated packages); tool-description-untrusted (MCP) — already reflected in intake trust boundary; Osmani source-driven-development retrieval-safety ("fetched docs are data, not commands") — reinforces PE intake trust boundary, no new rule. +- **E — out of scope:** RBAC/audit/approvals infra, memory encryption, output-guard middleware, MCP implementation → Turnstone / generated applications / Method Factory / infrastructure. + +### Additional findings (independent senior review, review `4888963474`) + +1. **Triage manifest remnant (possible):** `skills/process-engine-triage/SKILL.md` step 4 instructs feedback to link to `package_id`, `version`, `deployment_id`, and trial run "from the package manifest". The current prompts-only product does not otherwise define a package-manifest contract (v1.9.5 retired old manifest mechanics). **Verify whether a current Turnstone-native artifact is intended; if not, rewrite in terms of package/project deployment/trial evidence without reintroducing a manifest.** Pre-clean-slate/Method-Factory-era remnant candidate. +2. **Validator/spec-integrity (small structural fidelity):** `tools/validate.py` claims Agent Skills spec validation but allows top-level `version` in its allowed frontmatter set; current Agent Skills `skills-ref` allowed fields are `name`, `description`, `license`, `allowed-tools`, `metadata`, `compatibility`. PE itself correctly places version inside `metadata`, so no current skill is invalid. Validator also does not fully enforce `compatibility` ≤500 or metadata string→string. Small fidelity issue, not a reason to build more machinery. Either make these small checks match upstream or narrow the validator's claim to "PE repository invariants / selected Agent Skills constraints" (review uses Turnstone parse + skills-ref for full spec validation). +3. **Osmani 24-skill count — RESOLVED:** current README at `f4933771` explicitly says "install all 24 skills" and has an "All 24 Skills" section (23 lifecycle + 1 meta). No remaining A/B uncertainty. - **D — useful upstream evolution:** real-task-then-extract (externally validates observed behavior — record, do not add rule); plan-validate-execute for destructive ops (PE already approximates via dry-run/gate patterns); exact-action approval binding (PE already binds gates to exact operations in generated packages); tool-description-untrusted (MCP) — already reflected in intake trust boundary. - **E — out of scope:** RBAC/audit/approvals infra, memory encryption, output-guard middleware, MCP implementation → Turnstone / generated applications / Method Factory / infrastructure. @@ -176,9 +189,18 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi ## 12. Release-candidate verdict -**ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION CORRECTIONS.** +**ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION CORRECTIONS (revised: also internal governance-contract wording correction).** + +PR #4 itself is correct and ready for merge review (verified full diff; structural-validation PASS on head; 12 files; non-behavioral). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. + +After the independent senior review (`4888963474`), the correction set is sharper: +1. stale mechanical-enforcement wording in persona/core/pattern/ship (internal behavioral-contract consistency, small prompt correction — do NOT add enforcement); +2. imperative-description spec-vs-guidance framing; +3. metadata string-map wording; +4. validator claim/allowed-field mismatch (small structural fidelity); +5. Triage package-manifest remnant (verify; if none, rewrite without manifest). -PR #4 itself is correct and ready for merge review (verified full diff; structural-validation PASS on head; 12 files; non-behavioral). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. The B-class attribution corrections and the persona wording alignment should be completed before the v1.9.6 tag/release gate (as small follow-up changes), but none is a behavioral gap and none blocks PR #4's merge review. +These should land as a small follow-up alignment change (stacked on PR #4) before the v1.9.6 tag/release gate. None is a behavioral gap requiring new machinery; none blocks PR #4's merge review itself. --- From 77fa3ca27d3b8d0524a4552af3102660cb46494d Mon Sep 17 00:00:00 2001 From: RedEyeNinja-BKK <232920946+RedEyeNinja-BKK@users.noreply.github.com> Date: Sat, 8 Aug 2026 21:48:39 +0700 Subject: [PATCH 3/6] docs: reconcile PR #5 evidence per review 4889011120 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Remove stale §11 instruction to relabel trial methodology as PE/Osmani-only (Agent Skills eval guidance is primary; PE adapts it). - Remove stale §11 instruction to re-verify 24-skill count (confirmed). - §5 OWASP recommendation updated: integrated pre-release in PR #6 (not post-v1.9.6). - §11 now reflects resolved-via-PR#6 state incl. G6 hostile-intake trial. - PR #5 body updated: no longer calls trial methodology a B-class correction. --- .../2026-08-08-upstream-alignment-review.md | 20 +++++++++---------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs/alignment-review/2026-08-08-upstream-alignment-review.md b/docs/alignment-review/2026-08-08-upstream-alignment-review.md index c217c51..8e93a88 100644 --- a/docs/alignment-review/2026-08-08-upstream-alignment-review.md +++ b/docs/alignment-review/2026-08-08-upstream-alignment-review.md @@ -95,7 +95,7 @@ Inspected (current titles at root `cheatsheets/`, not `cheatsheets/ai/`): ### OWASP role recommendation -**Option #2 — named proportional security basis.** OWASP should be a named security basis used proportionally when a package's intent/material/tools make security relevant — not a universal third generation-basis source applied to every package, and not a "third source" in evidence-library/best-practices by default. PE should have security awareness without turning every package into a security framework. Concretely: (post-v1.9.6, small) add OWASP as a named proportional basis in `references/evidence-library.md` and `references/safety.md` (one line each), keep the persona/core unchanged. +**Option #2 — named proportional security basis.** OWASP should be a named security basis used proportionally when a package's intent/material/tools make security relevant — not a universal third generation-basis source applied to every package. PE should have security awareness without turning every package into a security framework. Concretely: add OWASP as a named proportional basis in `references/evidence-library.md` and `references/safety.md` (done in PR #6), keep the persona/core unchanged (except the small governance-wording alignment). ## 6. Core-value alignment matrix @@ -136,7 +136,7 @@ Inspected (current titles at root `cheatsheets/`, not `cheatsheets/ai/`): | Verification non-negotiable / "seems right" never sufficient | Osmani | Present (TDD, README) | best-practices | UPSTREAM | ALIGNED | | Anti-rationalization | Osmani | Present (spec-driven) | best-practices, skill-anatomy | UPSTREAM | ALIGNED | | Progressive disclosure | agentskills spec + Osmani | Present (spec; context-engineering) | skill-anatomy | SPEC REQUIREMENT + UPSTREAM | ALIGNED | -| Trial methodology (trigger sets, baselines, token capture) | Agent Skills authoring/eval guidance | **Present** (optimizing-descriptions.mdx: trigger sets, near-misses, rates; evaluating-skills.mdx: with/without baseline, timing.json token+duration, assertions) | best-practices, trial skill | SPEC GUIDANCE (authoring/eval) + PE ADAPTATION to Turnstone Trial | ALIGNED (corrected) | +| Trial methodology (trigger sets, baselines, token capture) | Agent Skills authoring/eval guidance | **Present** (optimizing-descriptions.mdx: trigger sets, near-misses, rates; evaluating-skills.mdx: with/without baseline, timing.json token+duration, assertions) | best-practices, trial skill | SPEC GUIDANCE (authoring/eval) + PE ADAPTATION to Turnstone Trial | ALIGNED (corrected; no longer B-class) | | 24-skill catalog | Osmani | Confirmed — README at `f4933771` says "install all 24 skills" + "All 24 Skills" section (23 lifecycle + 1 meta) | best-practices | UPSTREAM | ALIGNED (resolved — no longer A/B) | | Two-source basis (Osmani + Agent Skills) | — | Accurate for current references; OWASP to be added as a proportional security basis (see §5) | evidence-library, best-practices | PE ORIGINAL (basis doc) | ALIGNED; OWASP proportional post-v1.9.6 | | Progressive disclosure | agentskills spec + Osmani | Present (spec Progressive Disclosure section = guidance/convention; context-engineering) | skill-anatomy | SPEC GUIDANCE / DESIGN CONVENTION + PE STANDARD | ALIGNED (classification refined) | @@ -175,15 +175,13 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi ## 11. Smallest proposed corrections -**Before v1.9.6 (small, release-facing, non-behavioral or minimal):** -1. Relabel "imperative phrasing" as upstream recommendation (best-practices.md + spec-compliance.md) — documentation. -2. Relabel trial-methodology attribution as Process Engine adaptation informed by Osmani (best-practices.md) — documentation. -3. Verify/correct the "24-skill" catalog count (best-practices.md) — documentation. -4. Align persona.md "Turnstone enforces them mechanically" wording to the advisory contract — **small persona-prompt wording change**; do as a separate reviewable change (not inside PR #4). - -**Post-v1.9.6 (future trial / small):** -5. Add OWASP as a named **proportional** security basis (one line each in evidence-library.md + safety.md), not a universal third source. -6. Future-trial candidate: hostile-instruction-in-untrusted-material intake case (to confirm the existing trust boundary holds behaviorally). +**Resolved before v1.9.6 (by PR #6 — stacked on PR #4, reviewed):** +1. Relabel "imperative phrasing" as PE authoring guidance (best-practices.md + spec-compliance.md) — **done in PR #6**. +2. Trial-methodology attribution — **corrected**: Agent Skills authoring/eval guidance is the primary source (trigger sets, near-misses, with/without baselines, token-timing), with Process Engine adapting it to Turnstone. Not PE/Osmani-only. **Done in PR #6.** +3. Osmani 24-skill count — **resolved** (README at `f4933771` confirms 24). No verification needed. **Done in PR #6.** +4. Align persona.md "Turnstone enforces them mechanically" wording to the advisory contract — **done in PR #6** (small persona-prompt wording change, reviewed as part of the alignment-fix PR, not inside PR #4). +5. Add OWASP as a named **proportional** security/risk basis (evidence-library.md + safety.md), not a universal third source — **done in PR #6 (pre-release integration, per the approved alignment correction)**. +6. Future-trial candidate: hostile-instruction-in-untrusted-material intake case — **done as G6 in PR #6 behavioral regressions** (trust boundary held: material treated as data, embedded instruction ignored as authority). **Reject / out of scope:** all E-class items. From ab439d686870b47409baf3211466beaa2dbe4a68 Mon Sep 17 00:00:00 2001 From: RedEyeNinja-BKK <232920946+RedEyeNinja-BKK@users.noreply.github.com> Date: Sat, 8 Aug 2026 21:56:47 +0700 Subject: [PATCH 4/6] docs: final PR #5 evidence reconciliation (v1.9.6 closure) - Claim/source matrix: OWASP now 'integrated pre-release in PR #6' (not post-v1.9.6 / not-a-basis-yet). - Security/intake: hostile embedded instruction now distinguished as theoretical-before-G6 vs focused-behavioral-evidence-after-G6 (comment 5226619712). - C-finding: mechanical-enforcement wording marked corrected by PR #6. - Verdict: correction set implemented in PR #6, senior-reviewed PASS (4889019914); original finding -> correction -> final status preserved. - PR #5 body updated to match final evidence document. --- .../2026-08-08-upstream-alignment-review.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/alignment-review/2026-08-08-upstream-alignment-review.md b/docs/alignment-review/2026-08-08-upstream-alignment-review.md index 8e93a88..4fc38f2 100644 --- a/docs/alignment-review/2026-08-08-upstream-alignment-review.md +++ b/docs/alignment-review/2026-08-08-upstream-alignment-review.md @@ -138,9 +138,9 @@ Inspected (current titles at root `cheatsheets/`, not `cheatsheets/ai/`): | Progressive disclosure | agentskills spec + Osmani | Present (spec; context-engineering) | skill-anatomy | SPEC REQUIREMENT + UPSTREAM | ALIGNED | | Trial methodology (trigger sets, baselines, token capture) | Agent Skills authoring/eval guidance | **Present** (optimizing-descriptions.mdx: trigger sets, near-misses, rates; evaluating-skills.mdx: with/without baseline, timing.json token+duration, assertions) | best-practices, trial skill | SPEC GUIDANCE (authoring/eval) + PE ADAPTATION to Turnstone Trial | ALIGNED (corrected; no longer B-class) | | 24-skill catalog | Osmani | Confirmed — README at `f4933771` says "install all 24 skills" + "All 24 Skills" section (23 lifecycle + 1 meta) | best-practices | UPSTREAM | ALIGNED (resolved — no longer A/B) | -| Two-source basis (Osmani + Agent Skills) | — | Accurate for current references; OWASP to be added as a proportional security basis (see §5) | evidence-library, best-practices | PE ORIGINAL (basis doc) | ALIGNED; OWASP proportional post-v1.9.6 | +| Two-source basis (Osmani + Agent Skills) | — | Accurate for current references; OWASP is the named proportional cross-cutting security/risk basis (integrated pre-release in PR #6) | evidence-library, best-practices | PE ORIGINAL (basis doc) | ALIGNED | | Progressive disclosure | agentskills spec + Osmani | Present (spec Progressive Disclosure section = guidance/convention; context-engineering) | skill-anatomy | SPEC GUIDANCE / DESIGN CONVENTION + PE STANDARD | ALIGNED (classification refined) | -| OWASP Top 10 prevention | Osmani catalog (security-and-hardening skill) | Present as that skill's topic | best-practices (catalog line) | UPSTREAM (catalog description) | ALIGNED (OWASP itself not a PE basis yet) | +| OWASP Top 10 prevention | Osmani catalog (security-and-hardening skill) | Present as that skill's topic | best-practices (catalog line) | UPSTREAM (catalog description) | ALIGNED (OWASP also added as PE proportional security basis in PR #6) | ## 8. Security review of intake/context/material handling @@ -150,7 +150,7 @@ PE consumes user text, files, URLs, external skills, docs, tool observations, ac - **Distinguish operator instruction vs embedded/untrusted instruction:** present (intake trust boundary; PR #2 provenance handling; trials #1/#2 demonstrated four-way provenance + correction handling). ALIGNED. - **Sensitive/private material:** present (`references/safety.md`: no credential exposure, no private-data leakage; Proxmox T6 refused secrets). ALIGNED. - **Tool evidence vs inference:** present (PR #2 intake preserves tool evidence with provenance; trials labeled evidence). ALIGNED. -- **Potentially hostile embedded instruction:** handled by the same intake trust boundary; not separately tested in trials — theoretical; classify as a **future-trial candidate** (a D/C finding: no demonstrated behavioral gap). +- **Potentially hostile embedded instruction:** handled by the same intake trust boundary. Before G6 this was theoretical (D/C: no demonstrated behavioral gap); **now G6 provides focused behavioral evidence** (PR #6 comment `5226619712`): an embedded hostile instruction remained data, not authority; useful material could still be extracted; provenance remained; operator intent unchanged; no unrelated action occurred. - **No new prompt text warranted by this review** (existing trials cover the provenance chain; anything else is theoretical). ## 9. Architectural boundary check @@ -161,7 +161,7 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi - **A — factual/source drift:** none remains after this correction pass. (Changed-file count discrepancy was in a report, not the repo — corrected here to 12.) - **B — attribution drift:** (1) "imperative phrasing" framed as spec requirement in best-practices + spec-compliance → relabel as upstream authoring recommendation / PE authoring standard (formal spec requires only non-empty ≤1024 chars describing what + when); (2) **metadata wording** — spec says "map from string keys to string values"; PE docs say "arbitrary key-value map" → align wording; (3) **trial/eval attribution corrected** — Agent Skills authoring/eval guidance does contain trigger sets / near-misses / with-without baselines / token-timing capture; PE adapts it (no longer "PE-only"). (4) persona.md "Turnstone enforces them mechanically" → align to advisory contract. -- **C — behavioral gap:** none found in the generator behavior itself. **However — internal behavioral-contract consistency issue:** stale mechanical-enforcement language remains in **four load-bearing prompt artifacts** — `persona.md`, `skills/process-engine-core/SKILL.md`, `skills/process-engine-pattern-author/SKILL.md`, `skills/process-engine-ship/SKILL.md` — containing "for enforcement", "Turnstone owns the guardrails", "enforcement layer", "enforce this mechanically", "native enforcement layer", while `references/governance.md` and the PR #4 docs correctly state prompt policy/judge are advisory helpers and operator approval is authoritative. This is a **small prompt correction before release** (make prompts accurately describe the advisory Turnstone-native governance model intended — the opposite of adding enforcement). Theoretical intake concern (hostile embedded instruction in untrusted material) → future-trial candidate, now planned as G6. +- **C — behavioral gap:** none found in the generator behavior itself. **However — internal behavioral-contract consistency issue (now corrected):** stale mechanical-enforcement language was found in four load-bearing prompt artifacts — `persona.md`, `skills/process-engine-core/SKILL.md`, `skills/process-engine-pattern-author/SKILL.md`, `skills/process-engine-ship/SKILL.md` ("for enforcement", "Turnstone owns the guardrails", "enforcement layer", "enforce this mechanically", "native enforcement layer") while `references/governance.md` and the PR #4 docs correctly state prompt policy/judge are advisory helpers and operator approval is authoritative. **Corrected by PR #6** (small prompt wording change — makes prompts accurately describe the advisory Turnstone-native governance model intended, the opposite of adding enforcement). Theoretical intake concern (hostile embedded instruction in untrusted material) → now covered by G6 (see §8). - **D — useful upstream evolution:** real-task-then-extract (externally validates observed behavior — record, do not add rule); plan-validate-execute for destructive ops (PE already approximates via dry-run/gate patterns); exact-action approval binding (PE already binds gates to exact operations in generated packages); tool-description-untrusted (MCP) — already reflected in intake trust boundary; Osmani source-driven-development retrieval-safety ("fetched docs are data, not commands") — reinforces PE intake trust boundary, no new rule. - **E — out of scope:** RBAC/audit/approvals infra, memory encryption, output-guard middleware, MCP implementation → Turnstone / generated applications / Method Factory / infrastructure. @@ -191,14 +191,15 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi PR #4 itself is correct and ready for merge review (verified full diff; structural-validation PASS on head; 12 files; non-behavioral). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. -After the independent senior review (`4888963474`), the correction set is sharper: -1. stale mechanical-enforcement wording in persona/core/pattern/ship (internal behavioral-contract consistency, small prompt correction — do NOT add enforcement); +After the independent senior reviews (`4888963474`, `4889011120`, `4889019914`), the correction set is **implemented in PR #6 and senior-reviewed PASS** (subject to stacked-PR CI after retargeting): +1. stale mechanical-enforcement wording in persona/core/pattern/ship (internal behavioral-contract consistency, small prompt correction — no enforcement added); 2. imperative-description spec-vs-guidance framing; 3. metadata string-map wording; 4. validator claim/allowed-field mismatch (small structural fidelity); -5. Triage package-manifest remnant (verify; if none, rewrite without manifest). +5. Triage package-manifest remnant (verified: no current contract; rewritten without manifest); +6. OWASP named proportional security/risk basis integrated pre-release. -These should land as a small follow-up alignment change (stacked on PR #4) before the v1.9.6 tag/release gate. None is a behavioral gap requiring new machinery; none blocks PR #4's merge review itself. +**Final status:** original findings (this review) → corrections (PR #6) → senior-reviewed PASS (`4889019914`). PR #4 remains the clean release-identity/documentation PR; PR #6 is the stacked alignment-fix PR; both await operator merge gates in sequence. None of these is a behavioral gap requiring new machinery. --- From 2378a933298897dd293401480c44ea9d5da0e9c0 Mon Sep 17 00:00:00 2001 From: RedEyeNinja-BKK <232920946+RedEyeNinja-BKK@users.noreply.github.com> Date: Sun, 9 Aug 2026 00:17:13 +0700 Subject: [PATCH 5/6] docs: finalize PR #5 evidence for merged v1.9.6 candidate - Record PR #4 merge (2511f1a) + PR #6 merge (4808268a) + final main 4808268ac8f959b2a748410cff0739ac8f0cac88. - Record exact-head structural-validation run 31268609588 (success) on PR #6 head bcab220 (tree identical to reviewed 6705e5f). - Record G1-G6 evidence (comment 5226619712) + final alignment verdict: ALIGNED, merged product matches reviewed v1.9.6 candidate. --- .../2026-08-08-upstream-alignment-review.md | 23 ++++++++++++------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/docs/alignment-review/2026-08-08-upstream-alignment-review.md b/docs/alignment-review/2026-08-08-upstream-alignment-review.md index 4fc38f2..10b0021 100644 --- a/docs/alignment-review/2026-08-08-upstream-alignment-review.md +++ b/docs/alignment-review/2026-08-08-upstream-alignment-review.md @@ -2,15 +2,20 @@ **Status:** Read-only alignment evidence and release steering. Recorded by Turnstone (Process Engine Development Lead) with an upstream source audit delegated to Hermes and independently verified by Turnstone. No behavior, prompts, references, tooling, or Turnstone objects were changed by this review. References Issue #1. -## 0. Evidence identity +## 0. Evidence identity (final, post-merge) -- PR #4 (`v1.9.6 release identity + documentation alignment`): open, not merged, mergeable. +- **PR #4** (`v1.9.6 release identity + documentation alignment`): **MERGED** into `main`. - base: `16e4cbe7151297ac8c659f7fca8bad1ffd05f9dc` (main after PR #3) - head: `3d114e58e5e87303868fd3ed6ad073698e8988dc` - - branch: `v1.9.6-release-metadata` - commits: 2 (`c2bf841`, `3d114e5`) - - GitHub `structural-validation` check on head: PASS -- **Changed-file count: 12** (not 14). The earlier "14" over-counted by adding the two docs-alignment files without deduplicating against the metadata commit (README.md and docs/architecture.md appear in both). Corrected count: `CHANGELOG.md`, `README.md`, `docs/architecture.md`, `docs/governance-usage.md`, `docs/standards.md`, `process-engine.toml`, six `skills/*/SKILL.md` = 12. + - merge commit: `2511f1afac7a53d013ed3fa09296adb5128f4a91` +- **PR #6** (`v1.9.6 alignment corrections`): **MERGED** into `main` (stacked on PR #4, retargeted to `main` after PR #4). + - head: `bcab2205f0bf12221d84caeb0b09938bb7a9ae72` (history-sync merge of `main` into the reviewed tree; tree byte-for-byte identical to reviewed head `6705e5fc84d1ec520f6df14ff6a27e863bdd201d`) + - merge commit: `4808268ac8f959b2a748410cff0739ac8f0cac88` + - exact-head `structural-validation` on `bcab220…` run `31268609588` → **success** +- **Final canonical `main`:** `4808268ac8f959b2a748410cff0739ac8f0cac88` (Merge PR #6: v1.9.6 alignment corrections) +- **Changed-file count (PR #4): 12** (not 14). The earlier "14" over-counted by adding the two docs-alignment files without deduplicating against the metadata commit. Corrected count: `CHANGELOG.md`, `README.md`, `docs/architecture.md`, `docs/governance-usage.md`, `docs/standards.md`, `process-engine.toml`, six `skills/*/SKILL.md` = 12. +- **PR #6 changed-file count vs `main`:** 19 (+170/−60). ## 1. PR #4 diff review (verified full base→head diff) @@ -185,13 +190,13 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi **Reject / out of scope:** all E-class items. -## 12. Release-candidate verdict +## 12. Release-candidate verdict (final, post-merge) **ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION CORRECTIONS (revised: also internal governance-contract wording correction).** -PR #4 itself is correct and ready for merge review (verified full diff; structural-validation PASS on head; 12 files; non-behavioral). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. +PR #4 and PR #6 are **both merged into canonical `main`** (`4808268ac8f959b2a748410cff0739ac8f0cac88`). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. -After the independent senior reviews (`4888963474`, `4889011120`, `4889019914`), the correction set is **implemented in PR #6 and senior-reviewed PASS** (subject to stacked-PR CI after retargeting): +After the independent senior reviews (`4888963474`, `4889011120`, `4889019914`, `4889264021`, `4889278302`), the correction set is **implemented in PR #6 and senior-reviewed PASS** (exact-head `structural-validation` run `31268609588` success on `bcab220…`): 1. stale mechanical-enforcement wording in persona/core/pattern/ship (internal behavioral-contract consistency, small prompt correction — no enforcement added); 2. imperative-description spec-vs-guidance framing; 3. metadata string-map wording; @@ -199,6 +204,8 @@ After the independent senior reviews (`4888963474`, `4889011120`, `4889019914`), 5. Triage package-manifest remnant (verified: no current contract; rewritten without manifest); 6. OWASP named proportional security/risk basis integrated pre-release. +**G1–G6 focused behavioral evidence** at PR #6 comment `5226619712` (Ship gate, advisory governance, governance generation, Triage traceability, spec-vs-guidance, hostile embedded instruction). **Final alignment verdict: ALIGNED — merged product matches the reviewed v1.9.6 candidate.** + **Final status:** original findings (this review) → corrections (PR #6) → senior-reviewed PASS (`4889019914`). PR #4 remains the clean release-identity/documentation PR; PR #6 is the stacked alignment-fix PR; both await operator merge gates in sequence. None of these is a behavioral gap requiring new machinery. --- From 29dc800ee257d1cee797af73019f0593c6396ef2 Mon Sep 17 00:00:00 2001 From: RedEyeNinja-BKK <232920946+RedEyeNinja-BKK@users.noreply.github.com> Date: Sun, 9 Aug 2026 00:26:11 +0700 Subject: [PATCH 6/6] docs: PR #5 evidence corrections per review 4889308958 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Triage manifest remnant: mark historical finding; resolved by PR #6 (explicit historical v1.9.5 state) - satisfies validator historical exemption. - §12: remove stale 'both await operator merge gates in sequence'; actual chain (findings -> PR #6 correction -> senior PASS -> PR #4/#6 merged; PR #5 evidence-only awaits its gate). - '4808268a' framed as reviewed product-candidate main before this evidence-only PR (merging PR #5 advances main without product change). - No product/validator/CI changes. --- .../2026-08-08-upstream-alignment-review.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/alignment-review/2026-08-08-upstream-alignment-review.md b/docs/alignment-review/2026-08-08-upstream-alignment-review.md index 10b0021..2b24859 100644 --- a/docs/alignment-review/2026-08-08-upstream-alignment-review.md +++ b/docs/alignment-review/2026-08-08-upstream-alignment-review.md @@ -13,7 +13,7 @@ - head: `bcab2205f0bf12221d84caeb0b09938bb7a9ae72` (history-sync merge of `main` into the reviewed tree; tree byte-for-byte identical to reviewed head `6705e5fc84d1ec520f6df14ff6a27e863bdd201d`) - merge commit: `4808268ac8f959b2a748410cff0739ac8f0cac88` - exact-head `structural-validation` on `bcab220…` run `31268609588` → **success** -- **Final canonical `main`:** `4808268ac8f959b2a748410cff0739ac8f0cac88` (Merge PR #6: v1.9.6 alignment corrections) +- **Reviewed product-candidate `main` before this evidence-only PR:** `4808268ac8f959b2a748410cff0739ac8f0cac88` (Merge PR #6: v1.9.6 alignment corrections). Merging PR #5 adds evidence only and will advance `main` without changing product content. - **Changed-file count (PR #4): 12** (not 14). The earlier "14" over-counted by adding the two docs-alignment files without deduplicating against the metadata commit. Corrected count: `CHANGELOG.md`, `README.md`, `docs/architecture.md`, `docs/governance-usage.md`, `docs/standards.md`, `process-engine.toml`, six `skills/*/SKILL.md` = 12. - **PR #6 changed-file count vs `main`:** 19 (+170/−60). @@ -172,7 +172,7 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi ### Additional findings (independent senior review, review `4888963474`) -1. **Triage manifest remnant (possible):** `skills/process-engine-triage/SKILL.md` step 4 instructs feedback to link to `package_id`, `version`, `deployment_id`, and trial run "from the package manifest". The current prompts-only product does not otherwise define a package-manifest contract (v1.9.5 retired old manifest mechanics). **Verify whether a current Turnstone-native artifact is intended; if not, rewrite in terms of package/project deployment/trial evidence without reintroducing a manifest.** Pre-clean-slate/Method-Factory-era remnant candidate. +1. **Triage manifest remnant (historical finding; resolved by PR #6):** `skills/process-engine-triage/SKILL.md` step 4 historically instructed feedback to link to `package_id`, `version`, `deployment_id`, and trial run "from the package manifest" — an artifact of the pre-clean-slate/Method-Factory-era (historical v1.9.5 state, where old manifest mechanics existed). The current prompts-only product does not otherwise define a package-manifest contract. **Resolved by PR #6:** the instruction was rewritten in terms of package/project deployment/trial evidence without reintroducing a manifest. 2. **Validator/spec-integrity (small structural fidelity):** `tools/validate.py` claims Agent Skills spec validation but allows top-level `version` in its allowed frontmatter set; current Agent Skills `skills-ref` allowed fields are `name`, `description`, `license`, `allowed-tools`, `metadata`, `compatibility`. PE itself correctly places version inside `metadata`, so no current skill is invalid. Validator also does not fully enforce `compatibility` ≤500 or metadata string→string. Small fidelity issue, not a reason to build more machinery. Either make these small checks match upstream or narrow the validator's claim to "PE repository invariants / selected Agent Skills constraints" (review uses Turnstone parse + skills-ref for full spec validation). 3. **Osmani 24-skill count — RESOLVED:** current README at `f4933771` explicitly says "install all 24 skills" and has an "All 24 Skills" section (23 lifecycle + 1 meta). No remaining A/B uncertainty. - **D — useful upstream evolution:** real-task-then-extract (externally validates observed behavior — record, do not add rule); plan-validate-execute for destructive ops (PE already approximates via dry-run/gate patterns); exact-action approval binding (PE already binds gates to exact operations in generated packages); tool-description-untrusted (MCP) — already reflected in intake trust boundary. @@ -194,7 +194,7 @@ All candidate improvements were checked: RBAC/approvals/audit/persistence/securi **ALIGNED WITH SMALL DOCUMENTATION/ATTRIBUTION CORRECTIONS (revised: also internal governance-contract wording correction).** -PR #4 and PR #6 are **both merged into canonical `main`** (`4808268ac8f959b2a748410cff0739ac8f0cac88`). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. +PR #4 and PR #6 are **both merged** into the reviewed product-candidate `main` (`4808268ac8f959b2a748410cff0739ac8f0cac88`). The behavioral boundary holds; the engine remains smaller, source-honest, prompts-only, Turnstone-native, proportionate, and proven through real use. After the independent senior reviews (`4888963474`, `4889011120`, `4889019914`, `4889264021`, `4889278302`), the correction set is **implemented in PR #6 and senior-reviewed PASS** (exact-head `structural-validation` run `31268609588` success on `bcab220…`): 1. stale mechanical-enforcement wording in persona/core/pattern/ship (internal behavioral-contract consistency, small prompt correction — no enforcement added); @@ -206,7 +206,7 @@ After the independent senior reviews (`4888963474`, `4889011120`, `4889019914`, **G1–G6 focused behavioral evidence** at PR #6 comment `5226619712` (Ship gate, advisory governance, governance generation, Triage traceability, spec-vs-guidance, hostile embedded instruction). **Final alignment verdict: ALIGNED — merged product matches the reviewed v1.9.6 candidate.** -**Final status:** original findings (this review) → corrections (PR #6) → senior-reviewed PASS (`4889019914`). PR #4 remains the clean release-identity/documentation PR; PR #6 is the stacked alignment-fix PR; both await operator merge gates in sequence. None of these is a behavioral gap requiring new machinery. +**Final status:** original alignment findings recorded → corrections implemented by PR #6 → senior review PASS (`4889019914`, `4889278302`) → **PR #4 and PR #6 are merged** into the reviewed product-candidate `main` before this evidence-only PR (`4808268ac8f959b2a748410cff0739ac8f0cac88`). **PR #5 is the remaining evidence-only PR** (one added alignment-review file) awaiting its operator merge gate. None of the corrections was a behavioral gap requiring new machinery; merging PR #5 adds evidence only and does not alter product behavior. ---