diff --git a/examples/adversarial-implementation/README.md b/examples/adversarial-implementation/README.md index eaf55140d..2cafd78dc 100644 --- a/examples/adversarial-implementation/README.md +++ b/examples/adversarial-implementation/README.md @@ -38,6 +38,8 @@ The workflow enters the prepared checkout and artifact directory through absolut `Dir` paths. Enable `--allow-absolute` for these runs; the flag retains existing read/write restrictions. +For an ordinary ticket, pass `--props-max-steps 1` to finish one role call and inspect its next gate. Each step joins the role, its settlement and continuation publication before returning. Omitting the limit preserves the normal run. An explicit `maxSteps` on a foundation/migration launch holds before the foundation; that route does not expose this Ticket step boundary. + ## Select role providers and budgets The launch task requires `providers` for all three original roles and explicit @@ -53,11 +55,19 @@ work, select: } ``` -Planner and Implementor can independently select `claude` instead. The fixed -profiles below determine model and effort; provider selection never substitutes -another provider when one refuses. Architect uses `codex`. Missing or invalid -allocation or budget fields hold before paid work. The resolved Ticket manifest -retains the selected providers and all three budgets through continuation. +Planner and Implementor can independently select `claude` instead. Architect uses `codex`. Omit `implementationModel` to keep the default profile in Responsibilities and models. To select a different Implementor model, set `implementationModel` to an exact model ID advertised by the selected provider. For example: + +```json +{ + "providers": {"architect":"codex","planner":"codex","implementor":"claude"}, + "implementationModel": "sonnet", + "maxCorrections": 3, + "maxApiRepairs": 3, + "maxResponseCorrections": 0 +} +``` + +The explicit model applies only to the Implementor. It preserves that provider's existing effort policy and all other role profiles. Missing or invalid allocation or budget fields hold before paid work. Session verifies the provider's advertised model and effort before prompting. A refused or unavailable selection never substitutes another model, provider or conversation. The resolved Ticket manifest retains the model selection, role providers and all three budgets through continuation and correction. `maxCorrections` bounds source corrections and `maxApiRepairs` bounds consumer-driven API repairs, each from zero through ten. `maxResponseCorrections` @@ -197,7 +207,7 @@ Session resolution retains its existing provider setup behavior. This candidate writes `phase-session.v2` receipts. Older receipts refuse without changes; they are not migrated. Existing tickets keep their pinned workflow and -original receipts. +original receipts. An explicitly prepared model amendment uses a new pinned workflow source only after the prior host entry has joined. It preserves the saved Ticket source and identity receipts. It changes no active entry's includes or running role. Ordinary Tickets and API foundation/migration work share their original identity directory across attempts. For standalone `phase.md` requests, retain @@ -513,6 +523,8 @@ raw reply, result, status and acceptance receipt beneath the request's attempt. Source and role checks run before a prompt. Reusing an occupied attempt refuses; inspect its saved effects before selecting another attempt. +A standalone role request can likewise supply `implementationModel`; it applies only when that request runs the Implementor. Omission keeps the default. Response-format corrections reuse the same selected model and original conversation. + Set `maxResponseCorrections` to zero to validate the reply without another model prompt. A valid reply saves its canonical result. A rejected reply saves the raw text, rejection and `response-disposition.json`, then fails without a @@ -545,6 +557,24 @@ deno task xmd run --allow-absolute examples/adversarial-implementation/continue. --include examples/adversarial-implementation/components --raw ``` +To select Sonnet for an original Claude Implementor that has never been dispatched, first join the prior host entry and inspect the confirmed paused `implementation` handoff. Supply the user's authorization as readable text under a fresh reference, then continue with both options: + +```bash +xmd run --allow-absolute /absolute/path/to/pinned-workflow/examples/adversarial-implementation/continue.md \ + --props-record /absolute/path/to/ticket-record.json --props-steps 1 \ + --props-implementation-model sonnet \ + --props-implementation-model-evidence /absolute/path/to/sonnet-approval.md \ + --include /absolute/path/to/pinned-workflow/examples/adversarial-implementation/components --raw +``` + +In a native REPL, keep the lexical Repository and Worktree around the entry. Select the pinned workflow components for that new entry; do not change the includes of work still running. + +The amendment requires both original Implementor initialization flags to be false, no prior Implementor call or identity receipt and an unused next call reservation. Other gates, held or completed records, uncertain attempts and previously dispatched Implementors refuse before a role starts. Both options are required together. The selected provider stays the original provider; this operation does not change it. + +The authorization text is retained through the existing document snapshots. The workflow publishes a new continuation with the selected model and an `implementationModelSelection` receipt linking its prior continuation, previous model, selected model and authorization snapshot. It preserves prior bytes, sessions, captured inputs, call ordinals and consumed allowances. Only a confirmed publication can advance through the same requested step limit. Omit both options for an ordinary continuation. + +A model field or session name is selection evidence. Actual model use requires the production Session confirmation and canonical Prompt configuration for that attempt. The normal `delivery.json` profile and saved call profile retain the selected model after a successful raw reply; they do not turn a refused model request into a provider acknowledgement. + The envelope names a confirmed continuation by path, UTF-8 byte length and SHA-256. The protected continuation retains the original manifest, sessions, call ordinals, delivered-input accounting and correction counts. Each explicit step runs at most @@ -687,12 +717,12 @@ determine the work's outcome. | Owner | Work | Model and effort | | --- | --- | --- | | Planner | Ordinary plan/evidence; on migration, implement the API, validate representative consumers and repair API conflicts | Selected Codex `gpt-6.1-sol` or Claude `opus`; ordinary planning uses medium, high for a concrete recorded risk; migration foundation design uses high | -| Implementor | Implement accepted work; on migration, challenge the API, convert consumers and delete obsolete machinery | Selected Codex `gpt-6.1-sol`, medium (high for a concrete recorded risk), or Claude `opus`, high | +| Implementor | Implement accepted work; on migration, challenge the API, convert consumers and delete obsolete machinery | Default: selected Codex `gpt-6.1-sol`, medium (high for a concrete recorded risk), or Claude `opus`, high. Explicit `implementationModel` selects the advertised model of that same provider and preserves its effort policy. | | Planner reviewer | Inspect the migration against the accepted outcome and evidence | Same selected provider and original Planner session, medium; high for a concrete recorded risk | | Architect | Answer a concrete structural question or review structural impact | Codex `gpt-6.1-sol`, medium; high for a concrete recorded risk | | Sidekick running Codex | Author and apply issue/PR titles and descriptions; own publication and delivery | Existing coordinating Sidekick | -Aliases select the provider's available Opus model. Coding roles +Default aliases select the provider's available Opus model. An explicit model uses the provider's exact advertised ID. Coding roles do not draft or edit issue/PR titles or descriptions. The phase runner refuses editorial assignments; technical handoffs and API documentation remain part of the roles' work. diff --git a/examples/adversarial-implementation/components/Adversarial/Run.md b/examples/adversarial-implementation/components/Adversarial/Run.md index aaed2462e..894041831 100644 --- a/examples/adversarial-implementation/components/Adversarial/Run.md +++ b/examples/adversarial-implementation/components/Adversarial/Run.md @@ -111,7 +111,7 @@ mkdir -- {quoted} input.path),props.plannerGuide.path,props.implementorGuide.path]} /> [session,paths.filter(path => currentReferences.includes(path))]))} /> - + diff --git a/examples/adversarial-implementation/components/Adversarial/Schema.md b/examples/adversarial-implementation/components/Adversarial/Schema.md index ee81bb726..d8d5bc3ff 100644 --- a/examples/adversarial-implementation/components/Adversarial/Schema.md +++ b/examples/adversarial-implementation/components/Adversarial/Schema.md @@ -75,6 +75,7 @@ returns: {type: object} }, providers:{type:"object",properties:{architect:{const:"codex"},planner:{enum:["claude","codex"]},implementor:{enum:["claude","codex"]}},required:["architect","planner","implementor"],additionalProperties:false}, maxResponseCorrections:{type:"integer",minimum:0,maximum:3}, + implementationModel:{type:"string",minLength:1}, "maxCorrections": { "type": "integer", "minimum": 0, @@ -244,6 +245,7 @@ returns: {type: object} sessions:{type:"object",properties:{planner:text,implementor:text,architect:text},required:["planner","implementor","architect"],additionalProperties:false}, providers:{type:"object",properties:{architect:{const:"codex"},planner:{enum:["claude","codex"]},implementor:{enum:["claude","codex"]}},required:["architect","planner","implementor"],additionalProperties:false}, maxResponseCorrections:{type:"integer",minimum:0,maximum:3}, + implementationModel:{type:"string",minLength:1}, maxCorrections:{type:"integer",minimum:0,maximum:10}, maxApiRepairs:{type:"integer",minimum:0,maximum:10} },required:["work","repository","directory","request","sessions","providers","maxCorrections","maxApiRepairs","maxResponseCorrections"],additionalProperties:false}, diff --git a/examples/adversarial-implementation/components/Entry/Continue.md b/examples/adversarial-implementation/components/Entry/Continue.md index 98c587f5a..1711d472a 100644 --- a/examples/adversarial-implementation/components/Entry/Continue.md +++ b/examples/adversarial-implementation/components/Entry/Continue.md @@ -4,6 +4,8 @@ props: properties: record: {type: string} steps: {type: integer, minimum: 1, default: 1} + implementationModel: {type: string, minLength: 1} + implementationModelEvidence: {type: string, pattern: '^/'} required: [record] additionalProperties: false returns: {type: object} @@ -12,11 +14,11 @@ returns: {type: object} {text} - + {continued.output} - + !/^\s+at /.test(line)).join("\n").slice(0,1024),data:null}})}} as="bounded" /> diff --git a/examples/adversarial-implementation/components/Entry/Phase.md b/examples/adversarial-implementation/components/Entry/Phase.md index 852d4dfe2..2d0f68731 100644 --- a/examples/adversarial-implementation/components/Entry/Phase.md +++ b/examples/adversarial-implementation/components/Entry/Phase.md @@ -15,13 +15,13 @@ returns: {type: object} -{text} +{text} documents[path]),roleInputs:{},roles:{}},documents,calls:[]}} /> - + diff --git a/examples/adversarial-implementation/components/Entry/Run.md b/examples/adversarial-implementation/components/Entry/Run.md index 82236cd94..2a758d944 100644 --- a/examples/adversarial-implementation/components/Entry/Run.md +++ b/examples/adversarial-implementation/components/Entry/Run.md @@ -3,6 +3,7 @@ props: type: object properties: task: {type: string, minLength: 1} + maxSteps: {type: integer, minimum: 1} required: [task] additionalProperties: false returns: {type: object} @@ -28,6 +29,7 @@ input or existing record you need to reconcile before another launch. task.scope.migration.remainingConsumers.includes(path)))}> + ```bash exec as="source" git -C {repositoryPath} rev-parse --show-toplevel @@ -88,7 +90,7 @@ test -f {referencePath} -a -r {referencePath} - + ```bash exec as="reserved" mkdir -- {directoryPath} @@ -99,9 +101,9 @@ mkdir -- {directoryPath} - + - + diff --git a/examples/adversarial-implementation/components/Phase/Correct.md b/examples/adversarial-implementation/components/Phase/Correct.md index cd69431ab..4c643d721 100644 --- a/examples/adversarial-implementation/components/Phase/Correct.md +++ b/examples/adversarial-implementation/components/Phase/Correct.md @@ -10,6 +10,7 @@ props: profile: {type: object} plannerProvider: {enum: [claude, codex], default: claude} implementationProvider: {enum: [claude, codex]} + implementationModel: {type: string, minLength: 1} inputs: {type: array, items: {type: object}} maxResponseCorrections: {type: integer, minimum: 0, maximum: 3, default: 3} required: [assignment, role, session, attempt, raw, profile, implementationProvider, inputs] @@ -18,7 +19,7 @@ returns: {type: object} --- - + @@ -77,7 +78,7 @@ shasum -a 256 -- {quoted}/request.json {quoted}/diagnostic.json {quoted}/origina ``` {identities.stdout} - + {raw} diff --git a/examples/adversarial-implementation/components/Phase/Profile.md b/examples/adversarial-implementation/components/Phase/Profile.md index e267cedf8..84f3a3dac 100644 --- a/examples/adversarial-implementation/components/Phase/Profile.md +++ b/examples/adversarial-implementation/components/Phase/Profile.md @@ -1,5 +1,5 @@ --- -description: 'Select the fixed role model. Use .' +description: 'Select the role profile and optional explicit Implementor model. Use .' props: type: object properties: @@ -8,10 +8,11 @@ props: design: {type: boolean, default: false} plannerProvider: {enum: [claude, codex], default: claude} implementationProvider: {enum: [claude, codex], default: claude} + implementationModel: {type: string, minLength: 1} required: [role, assessment] additionalProperties: false returns: {type: object} --- - + diff --git a/examples/adversarial-implementation/components/Phase/Role.md b/examples/adversarial-implementation/components/Phase/Role.md index 1f9f6f046..6b0982460 100644 --- a/examples/adversarial-implementation/components/Phase/Role.md +++ b/examples/adversarial-implementation/components/Phase/Role.md @@ -55,6 +55,7 @@ props: implementationProvider: enum: [claude, codex] default: claude + implementationModel: {type: string, minLength: 1} required: - role - session @@ -62,7 +63,7 @@ props: additionalProperties: false --- - + diff --git a/examples/adversarial-implementation/components/Phase/Run.md b/examples/adversarial-implementation/components/Phase/Run.md index 946e45632..9e4fbcf14 100644 --- a/examples/adversarial-implementation/components/Phase/Run.md +++ b/examples/adversarial-implementation/components/Phase/Run.md @@ -45,6 +45,7 @@ props: implementationProvider: enum: [claude, codex] default: claude + implementationModel: {type: string, minLength: 1} maxResponseCorrections: type: integer minimum: 0 @@ -129,14 +130,14 @@ mkdir -- {quotedAttempt} - + {raw} - + - + diff --git a/examples/adversarial-implementation/components/Ticket/Continue.md b/examples/adversarial-implementation/components/Ticket/Continue.md index 61817ec4a..3e4dd8eaf 100644 --- a/examples/adversarial-implementation/components/Ticket/Continue.md +++ b/examples/adversarial-implementation/components/Ticket/Continue.md @@ -4,9 +4,21 @@ props: properties: record: {type: object} maxSteps: {type: integer, minimum: 1} + implementationModel: {type: string, minLength: 1} + implementationModelEvidence: {type: string, pattern: '^/'} required: [record] additionalProperties: false returns: {type: object} --- + + + + + + + + + + diff --git a/examples/adversarial-implementation/components/Ticket/Schema.md b/examples/adversarial-implementation/components/Ticket/Schema.md index 035efc2e0..b5cdffcdf 100644 --- a/examples/adversarial-implementation/components/Ticket/Schema.md +++ b/examples/adversarial-implementation/components/Ticket/Schema.md @@ -23,6 +23,17 @@ returns: {type: object} "identityDirectory": {"type":"string","pattern":"^/"}, "api": {"type": "object"}, "delivery": {"type": "object"}, + "implementationModelSelection": { + "type": "object", + "properties": { + "previousContinuation": {"type":"object","properties":{"path":{"type":"string","minLength":1},"utf8Bytes":{"type":"integer","minimum":0},"sha256":{"type":"string","pattern":"^[0-9a-f]{64}$"}},"required":["path","utf8Bytes","sha256"],"additionalProperties":false}, + "previousModel": {"type": ["string", "null"], "minLength": 1}, + "model": {"type": "string", "minLength": 1}, + "evidence": {"type":"object","properties":{"path":{"type":"string","minLength":1},"utf8Bytes":{"type":"integer","minimum":0},"sha256":{"type":"string","pattern":"^[0-9a-f]{64}$"},"snapshot":{"type":"string","pattern":"^/"},"observation":{"enum":["file","supplied"]}},"required":["path","utf8Bytes","sha256","snapshot","observation"],"additionalProperties":false} + }, + "required": ["previousContinuation", "previousModel", "model", "evidence"], + "additionalProperties": false + }, "assignment": phases.assignment, "maxApiRepairs": {"type":"integer","minimum":0,"maximum":10}, "maxResponseCorrections": {"type":"integer","minimum":0,"maximum":3}, @@ -94,6 +105,10 @@ returns: {type: object} "claude" ] }, + "model": { + "type": "string", + "minLength": 1 + }, "initialized": { "type": "boolean" } diff --git a/examples/adversarial-implementation/components/Ticket/SelectModel.md b/examples/adversarial-implementation/components/Ticket/SelectModel.md new file mode 100644 index 000000000..1a9336723 --- /dev/null +++ b/examples/adversarial-implementation/components/Ticket/SelectModel.md @@ -0,0 +1,52 @@ +--- +description: >- + Select the model of an unused original Implementor at a confirmed paused implementation + handoff and publish the amended continuation without calling a role. Use + , + then continue the returned envelope normally. +props: + type: object + properties: + record: {type: object} + model: {type: string, minLength: 1} + evidence: {type: string, pattern: '^/'} + required: [record, model, evidence] + additionalProperties: false +returns: {type: object} +--- + +The amendment covers one boundary: the confirmed, dispatchable, paused implementation +gate whose original Implementor has no initialization standing, no call of any status, +no identity receipt and no occupied next call reservation. The provider, sessions, +source anchor, captured inputs, call ordinals and consumed allowances stay as recorded. +The caller's authorization text enters the retained documents under a fresh reference +before a new continuation records the selection and links the one it replaces. + + + + + + call.role === "implementor")}> + + + + + + + + + + + + + + + + + + + + + + + diff --git a/examples/adversarial-implementation/components/Ticket/Step.md b/examples/adversarial-implementation/components/Ticket/Step.md index c4bba2c45..58e10a2e8 100644 --- a/examples/adversarial-implementation/components/Ticket/Step.md +++ b/examples/adversarial-implementation/components/Ticket/Step.md @@ -46,7 +46,7 @@ mkdir -- {recordDirectory} diff --git a/examples/adversarial-implementation/components/Ticket/Unstarted.ts b/examples/adversarial-implementation/components/Ticket/Unstarted.ts new file mode 100644 index 000000000..4583529a1 --- /dev/null +++ b/examples/adversarial-implementation/components/Ticket/Unstarted.ts @@ -0,0 +1,26 @@ +import { API } from "@executablemd/runtime"; +import { z } from "zod"; + +export const props = { + type: "object", + properties: { paths: { type: "array", items: { type: "string", pattern: "^/" } } }, + required: ["paths"], + additionalProperties: false, +}; + +export default function* Unstarted(value: unknown) { + const selected = z + .object({ paths: z.array(z.string().startsWith("/")) }) + .strict() + .parse(value); + try { + for (const path of selected.paths) { + if ((yield* API.Fs.operations.lstat(path)).exists) { + return false; + } + } + return true; + } catch { + return false; + } +} diff --git a/examples/adversarial-implementation/components/Ticket/control.ts b/examples/adversarial-implementation/components/Ticket/control.ts index efc4a2f0a..8b58655bc 100644 --- a/examples/adversarial-implementation/components/Ticket/control.ts +++ b/examples/adversarial-implementation/components/Ticket/control.ts @@ -18,6 +18,15 @@ const identity = z initialized: z.boolean(), }) .strict(); +const implementorIdentity = identity.extend({ model: z.string().min(1).optional() }).strict(); +const modelSelection = z + .object({ + previousContinuation: descriptor, + previousModel: z.string().min(1).nullable(), + model: z.string().min(1), + evidence: snapshot, + }) + .strict(); export const envelope = z .object({ schema: z.literal("ticket-return.v1"), @@ -64,7 +73,9 @@ export const state = z directory: z.string().startsWith("/"), identityDirectory: z.string().startsWith("/").optional(), assignment: jsonObject, - roles: z.object({ architect: identity, planner: identity, implementor: identity }).strict(), + roles: z + .object({ architect: identity, planner: identity, implementor: implementorIdentity }) + .strict(), acceptedInputs: z.array(snapshot), roleInputs: z.partialRecord(role, z.array(snapshot)).optional(), deliveredInputs: z.record(z.string(), z.array(z.string())).optional(), @@ -73,6 +84,7 @@ export const state = z maxResponseCorrections: z.number().int(), api: jsonObject.optional(), delivery: jsonObject.optional(), + implementationModelSelection: modelSelection.optional(), }) .strict(), checkpoint: z @@ -182,7 +194,10 @@ export function confirmLinks(saved: z.infer, selected: z.infer, selected: z.infer + diff --git a/examples/adversarial-implementation/run.md b/examples/adversarial-implementation/run.md index 7ee86053f..503e8987d 100644 --- a/examples/adversarial-implementation/run.md +++ b/examples/adversarial-implementation/run.md @@ -3,9 +3,10 @@ props: type: object properties: task: {type: string, minLength: 1} + maxSteps: {type: integer, minimum: 1} required: [task] additionalProperties: false --- - + diff --git a/examples/adversarial-implementation/tests/Implementation.test.md b/examples/adversarial-implementation/tests/Implementation.test.md index 586bc0744..d917872cc 100644 --- a/examples/adversarial-implementation/tests/Implementation.test.md +++ b/examples/adversarial-implementation/tests/Implementation.test.md @@ -42,3 +42,10 @@ printf '%s' "$XMD_EXAMPLE_TEST_COMMAND" {tester} {group} ``` + + + "'" + value.replaceAll("'","'\"'\"'") + "'").join(" ")} /> +```bash exec +{tester} {group} +``` + diff --git a/examples/adversarial-implementation/tests/LaunchPolicy.test.md b/examples/adversarial-implementation/tests/LaunchPolicy.test.md index c49644f6a..0701b7afe 100644 --- a/examples/adversarial-implementation/tests/LaunchPolicy.test.md +++ b/examples/adversarial-implementation/tests/LaunchPolicy.test.md @@ -8,7 +8,7 @@ printf '%s' "$XMD_EXAMPLE_COMMAND" - + @@ -28,7 +28,7 @@ git rev-parse HEAD Accepted repeated production friction in wide and the difficult modal case; substantial remaining conversion in consumer.sh. - + @@ -83,6 +83,9 @@ git rev-parse HEAD + + + call.role !== "architect").every(call => call.provider === "codex")} expected={true} /> @@ -112,6 +115,7 @@ git rev-parse HEAD ...["providers","maxCorrections","maxApiRepairs","maxResponseCorrections"].map(field => ({name:"missing-" + field,value:Object.fromEntries(Object.entries(task).filter(([key]) => key !== field)),field})), ...[-1,0.5,4,"0"].map(limit => ({name:"invalid-response-" + limit,value:{...task,maxResponseCorrections:limit},field:"maxResponseCorrections"})), {name:"invalid-provider",value:{...task,providers:{...task.providers,planner:"other"}},field:"planner"}, + ...["",1,null].map(model => ({name:"invalid-model-" + String(model),value:{...task,implementationModel:model},field:"implementationModel"})), {name:"missing-member",value:{...task,scope:{...task.scope,implementation:{evidence:[evidence]}}},field:"ready"}, {name:"string-boolean",value:{...task,scope:{...task.scope,implementation:{...task.scope.implementation,ready:"false"}}},field:"ready"}, {name:"unknown-scope",value:{...task,scope:{...task.scope,settled:true}},field:"settled"}, @@ -186,3 +190,38 @@ cd -- {directory} + + + + + +printf '%s' 'wide production fixture' + +printf '%s' 'remaining production fixture' + +```bash exec as="base" +git add wide.sh consumer.sh +git commit -qm 'existing production consumers' +git rev-parse HEAD +``` + + + +Accepted repeated production friction. + + + + + +{child.stdout} + + + +```bash exec as="reserved" +test -e {quotedDirectory} -o -L {quotedDirectory} +``` + + + + + diff --git a/examples/adversarial-implementation/tests/ProductionRoleSonnet.test.md b/examples/adversarial-implementation/tests/ProductionRoleSonnet.test.md new file mode 100644 index 000000000..36317e26d --- /dev/null +++ b/examples/adversarial-implementation/tests/ProductionRoleSonnet.test.md @@ -0,0 +1,67 @@ +```bash exec as="location" +pwd -P +``` + + + + + + + + + + + + + + + + + + + + + + + + + +```bash exec as="head" +git -C {quoted} rev-parse HEAD +``` +```bash exec as="temporary" +pwd -P +``` + +{"{}"} + + + + + + + + + + + + +{text} + + +{text} + + +{identityText} + + + + + + + + + + + + diff --git a/examples/adversarial-implementation/tests/agents/sonnet-correction.md b/examples/adversarial-implementation/tests/agents/sonnet-correction.md new file mode 100644 index 000000000..3d92781a6 --- /dev/null +++ b/examples/adversarial-implementation/tests/agents/sonnet-correction.md @@ -0,0 +1,10 @@ + + +{text} +{assignment.request} + + + +{text} + + diff --git a/examples/adversarial-implementation/tests/components/Fixture/Entry.md b/examples/adversarial-implementation/tests/components/Fixture/Entry.md index 94fd208bc..1d7ac8676 100644 --- a/examples/adversarial-implementation/tests/components/Fixture/Entry.md +++ b/examples/adversarial-implementation/tests/components/Fixture/Entry.md @@ -7,6 +7,8 @@ props: raw: {type: string} taskPath: {type: [string, 'null'], default: null} steps: {type: integer, minimum: 1} + maxSteps: {type: integer, minimum: 1} + extraProps: {type: array, items: {type: string}, default: []} repository: {type: string} command: {type: string} root: {type: string} @@ -20,7 +22,7 @@ returns: {type: object} ```bash exec as="location" pwd -P ``` - "'" + value.replaceAll("'","'\"'\"'") + "'").join(" ")} /> + "'" + value.replaceAll("'","'\"'\"'") + "'").join(" ")} /> ```bash exec as="child" set -e diff --git a/examples/adversarial-implementation/tests/components/Phase/Role.md b/examples/adversarial-implementation/tests/components/Phase/Role.md index 18fb4cb72..f07d1c62b 100644 --- a/examples/adversarial-implementation/tests/components/Phase/Role.md +++ b/examples/adversarial-implementation/tests/components/Phase/Role.md @@ -4,6 +4,7 @@ props: properties: plannerProvider: {enum: [claude, codex], default: claude} implementationProvider: {enum: [claude, codex], default: claude} + implementationModel: {type: string, minLength: 1} role: {type: string} session: {type: string} assignment: {type: object} @@ -16,7 +17,7 @@ props: - + {original.json} @@ -24,7 +25,7 @@ props: - input.path),inputs:props.inputs,request:assignment.request,phase:assignment.phase,contract:assignment.contract}} /> + input.path),inputs:props.inputs,request:assignment.request,phase:assignment.phase,contract:assignment.contract}} /> Use the existing format API for the remaining consumer. Remove legacy.fixture, run bash consumer.sh, and return its signed feedback commit. diff --git a/examples/adversarial-implementation/tests/implementation-selection.md b/examples/adversarial-implementation/tests/implementation-selection.md new file mode 100644 index 000000000..e27ccacd0 --- /dev/null +++ b/examples/adversarial-implementation/tests/implementation-selection.md @@ -0,0 +1,342 @@ +```bash exec as="location" +pwd -P +``` + +```bash exec as="commandSetting" +printf '%s' "$XMD_EXAMPLE_COMMAND" +``` + + + + + + + +format() { printf '%s' "$1"; } + +```bash exec as="base" +git add api.sh +git commit -qm 'existing API' +git rev-parse HEAD +``` + + +Evidence A: accepted format behavior and focused consumer checks. + + + + + + +{child.stdout} + + + + + + + call.role === "implementor").attempt + "/called.json"} as="defaultText" /> +{defaultText} + + + + + + + call.role)} expected={["implementor"]} /> + + call.role)} expected={["planner"]} /> + + + +Evidence B: mutable original after the bounded start. + + +{again.stdout} + + call.role)} expected={["planner","implementor"]} /> + + + + +{implementorText} + + + input.path === evidence).content} expected="Evidence A: accepted format behavior and focused consumer checks." /> + + + + +{readyText} + + + + + + + + + + + + + + +format() { printf '%s' "$1"; } + +```bash exec as="base" +git add api.sh +git commit -qm 'existing API' +git rev-parse HEAD +``` + + + + +Accepted plan retention contract.The user selected Claude Sonnet for the unused Implementor. + + + + +{child.stdout} + + + call.role)} expected={["planner"]} /> + + +```bash exec as="priorBefore" +shasum -a 256 -- {quotedPrior} +``` + + + + + +{child.stdout} + + + + +```bash exec as="priorAfter" +shasum -a 256 -- {quotedPrior} +``` + +Mutated after the selection was retained. + + + + + + + [path,amended.documents[path]]))} expected={planned.documents} /> + + + + + + + + + + + + +{child.stdout} + + + call.role)} expected={["planner","implementor"]} /> + + +{implementorText} + + input.path === reference).content} expected="Accepted plan retention contract." /> + + +{child.stdout} + + call.role)} expected={["planner","implementor","architect","planner"]} /> + + + +{child.stdout} + + call.role)} expected={["planner","implementor","architect","planner","implementor"]} /> + + + +{correctionText} + + + + + + + + + + + +format() { printf '%s' "$1"; } + +```bash exec as="base" +git add api.sh +git commit -qm 'existing API' +git rev-parse HEAD +``` + + + + +Accepted plan retention contract.The user selected Claude Sonnet for the unused Implementor. + + + + +{child.stdout} + + +```bash exec as="occupied" +mkdir -- {quotedDirectory}/attempts/0001 +``` + + + +```bash exec as="occupied" +mkdir -- {quotedDirectory}/calls/0001 +``` + + + +{"{}"} + + + +```bash exec as="priorBefore" +shasum -a 256 -- {quotedPrior} +``` + + +{child.stdout} + + + + + + +```bash exec as="priorAfter" +shasum -a 256 -- {quotedPrior} +``` + + + + + + + + + + + +format() { printf '%s' "$1"; } + +```bash exec as="base" +git add api.sh +git commit -qm 'existing API' +git rev-parse HEAD +``` + + + + +Accepted plan retention contract.The user selected Claude Sonnet for the unused Implementor. + + + + +{child.stdout} + + +```bash exec as="locked" +chmod 555 {continuations} +``` + + +```bash exec as="unlocked" +chmod 755 {continuations} +``` + + +{child.stdout} + + + + + + + + + + + + + +format() { printf '%s' "$1"; } + +```bash exec as="base" +git add api.sh +git commit -qm 'existing API' +git rev-parse HEAD +``` + + + + +Accepted plan retention contract.The user selected Claude Sonnet for the unused Implementor. + + + + +{child.stdout} + + + call.role)} expected={["planner"]} /> + + + + + + +```bash exec as="blocked" +mkdir -p {occupied} +chmod 000 {calls} +``` + + +```bash exec as="restored" +chmod 755 {calls} +``` + + + + + + + path !== "calls/0001")} expected={before} /> + + + diff --git a/packages/cli/tests/adversarial-session-continuity.test.ts b/packages/cli/tests/adversarial-session-continuity.test.ts index 8206ba899..a741c4126 100644 --- a/packages/cli/tests/adversarial-session-continuity.test.ts +++ b/packages/cli/tests/adversarial-session-continuity.test.ts @@ -550,6 +550,76 @@ describe("production workflow provider scope", () => { } }); + it("configures an explicit Sonnet Implementor before its prompt and retains it with its identity across cold reentry", function* () { + const fake = createFakeAcp(); + fake.configuration(catalog(["fixture-default", "opus", "sonnet"], ["medium", "high"])); + const harness = yield* reviewWorld(fake); + const assignment = implementorAssignment(harness); + yield* ensureDir(assignment.artifactDirectory); + const props = { assignment }; + const first = `${sonnetRoleSource}`; + const continued = `${sonnetContinuedRoleSource}`; + for (const text of [first, continued]) { + fake.script({ reply: "Completed the assigned work." }); + const ran = yield* harness.run(text, props); + if (!ran.ok) { + throw new Error(ran.text); + } + const ensured = fake.ensured.at(-1)!; + expect(ensured.requestedModel).toBe("sonnet"); + expect(fake.running(ensured.sessionKey)).toEqual({ model: "sonnet", effort: "high" }); + } + expect(fake.configCalls.filter((call) => call.startsWith("set "))).toEqual([ + "set model=sonnet", + "set effort=high", + ]); + expect(fake.turns).toHaveLength(2); + expect(new Set(fake.turns.map((entry) => entry.handle.sessionKey)).size).toBe(1); + const saved = receipt.parse( + JSON.parse(yield* readTextFile(join(harness.root, "sessions", "sonnet-owner.json"))), + ); + expect(saved).toMatchObject({ agent: "claude", name: "sonnet-owner", attempted: true }); + expect(saved.identity).not.toBe(null); + }); + + it("sends no Implementor prompt when Sonnet is unavailable, its write is refused or its readback differs", function* () { + for (const variant of ["missing", "refused", "ignored"] as const) { + const fake = createFakeAcp(); + fake.configuration( + catalog( + variant === "missing" ? ["fixture-default", "opus"] : ["fixture-default", "sonnet"], + ["medium", "high"], + ), + ); + const harness = yield* reviewWorld( + fake, + variant === "missing" ? fake.create : configurationFault(fake, variant), + ); + const assignment = implementorAssignment(harness); + yield* ensureDir(assignment.artifactDirectory); + fake.script({ reply: "This prompt must not be sent." }); + const ran = yield* harness.run( + `${sonnetRoleSource}`, + { + assignment, + }, + ); + expect(ran.ok).toBe(false); + expect(ran.text).toContain( + { + missing: 'Unknown model "sonnet"', + refused: "The provider refused the model write.", + ignored: 'did not apply the model "sonnet"', + }[variant], + ); + expect(fake.prompts).toEqual([]); + expect(fake.turns).toEqual([]); + expect(fake.configCalls.filter((call) => call === "set model=opus")).toEqual([]); + const running = fake.ensured.map((entry) => fake.running(entry.sessionKey)); + expect(running.every((state) => state?.model !== "opus")).toBe(true); + } + }); + it("keeps ordinary acpx unchanged and refuses to retrofit its retained owner", function* () { const fake = createFakeAcp(); fake.configuration(catalog(["opus"], ["high"])); @@ -573,6 +643,50 @@ describe("production workflow provider scope", () => { }); }); +const sonnetRoleSource = + '' + + ''; +const sonnetContinuedRoleSource = + ''; + +function implementorAssignment(harness: ReviewWorld) { + return { + repository: String(harness.delivery.repository), + artifactDirectory: join(harness.root, "submission"), + identityDirectory: join(harness.root, "sessions"), + phase: "implementation", + work: "sonnet-selection", + revision, + request: "Use the saved evidence.", + assessment: { + mechanical: false, + architecture: false, + acceptance: true, + risk: null, + reason: "Inspect the accepted behavior.", + evidence: ["accepted.md"], + }, + }; +} + +function configurationFault(fake: FakeAcp, fault: "refused" | "ignored") { + return (options: AcpRuntimeOptions): ProbeCapableRuntime => { + const runtime = fake.create(options); + const write = runtime.setConfigOption?.bind(runtime); + return { + ...runtime, + setConfigOption(input: Parameters>[0]) { + if (input.key !== "model") { + return write?.(input) ?? Promise.resolve(); + } + return fault === "refused" + ? Promise.reject(new Error("The provider refused the model write.")) + : Promise.resolve(); + }, + }; + }; +} + function catalog(models: string[], efforts: string[] = ["low", "high"]) { return { model: { selected: models[0]!, choices: models.map((id) => ({ id })) }, diff --git a/scripts/lib/runtime-tests.ts b/scripts/lib/runtime-tests.ts index 195b4ab5a..5c9dbc98d 100644 --- a/scripts/lib/runtime-tests.ts +++ b/scripts/lib/runtime-tests.ts @@ -96,7 +96,12 @@ function cliEntry(runtime: Runtime): string[] { } /** The suites whose roles are the production ones rather than the general fixture components. */ -const PRODUCTION_SUITES = new Set(["Preparation", "ProductionRole", "Submission"]); +const PRODUCTION_SUITES = new Set([ + "Preparation", + "ProductionRole", + "ProductionRoleSonnet", + "Submission", +]); /** * Each suite's whole-launch deadline, in milliseconds. Implementation has none: @@ -113,6 +118,7 @@ export const MARKDOWN_DEADLINES: Record = { LaunchPolicy: 120_000, Preparation: 420_000, ProductionRole: 120_000, + ProductionRoleSonnet: 120_000, Submission: 120_000, Workflow: 120_000, }; diff --git a/scripts/tests/runtime-exclusions.test.ts b/scripts/tests/runtime-exclusions.test.ts index 4983974b4..a67994def 100644 --- a/scripts/tests/runtime-exclusions.test.ts +++ b/scripts/tests/runtime-exclusions.test.ts @@ -186,6 +186,7 @@ describe("the applicable corpus", () => { "LaunchPolicy", "Preparation", "ProductionRole", + "ProductionRoleSonnet", "Submission", "Workflow", ].map((name) => `examples/adversarial-implementation/tests/${name}.test.md`); @@ -336,7 +337,7 @@ describe("the one-file Markdown commands", () => { }); it("gives the production-role suites their production components", function* () { - for (const name of ["Preparation", "ProductionRole", "Submission"]) { + for (const name of ["Preparation", "ProductionRole", "ProductionRoleSonnet", "Submission"]) { const file = `examples/adversarial-implementation/tests/${name}.test.md`; expect(oneFileCommand("bun", file, ROOT).arguments.slice(-4)).toEqual([ "--include", @@ -395,6 +396,7 @@ describe("the one-file Markdown commands", () => { LaunchPolicy: 120_000, Preparation: 420_000, ProductionRole: 120_000, + ProductionRoleSonnet: 120_000, Submission: 120_000, Workflow: 120_000, }); diff --git a/specs/testing-spec.md b/specs/testing-spec.md index 8286823dd..29ce99c9c 100644 --- a/specs/testing-spec.md +++ b/specs/testing-spec.md @@ -985,7 +985,7 @@ The adversarial suites append the following arguments to the source entrypoint: --include /examples/adversarial-implementation/tests/components --include /examples/adversarial-implementation/components`, with the include paths absolute so that a nested `` cannot change where components are found. -`Preparation`, `ProductionRole` and `Submission` include +`Preparation`, `ProductionRole`, `ProductionRoleSonnet` and `Submission` include `tests/production-components` in place of `tests/components`. The child's environment also receives `XMD_EXAMPLE_COMMAND` and `XMD_EXAMPLE_TEST_COMMAND`. Each is the prepared Deno CLI above followed by `run --allow-absolute` or