Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions external_plugins/autoresearch/.mcp.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,18 @@
"autoresearch": {
"type": "stdio",
"command": "node",
"args": ["${ZCODE_PLUGIN_ROOT}/mcp/server.ts"],
"args": [
"${ZCODE_PLUGIN_ROOT}/scripts/launcher.mjs",
"${ZCODE_PLUGIN_ROOT}/mcp/server.ts"
],
"cwd": "${ZCODE_PROJECT_DIR}",
"env": {
"AR_MAX_ITERATIONS": "${user_config.maxIterations}",
"AR_BENCHMARK_TIMEOUT_MS": "${user_config.benchmarkTimeoutMs}",
"AR_CHECKS_TIMEOUT_MS": "${user_config.checksTimeoutMs}"
},
"enabled": true,
"timeoutMs": 60000
"timeoutMs": 600000
}
}
}
3 changes: 2 additions & 1 deletion external_plugins/autoresearch/.zcode-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@
"en": "Autonomous experiment loop for ZCode: set a goal, pick a mechanical metric, and let the agent iterate. It measures, keeps what works, reverts what doesn't, and repeats.",
"zh-CN": "ZCode 自主实验循环:设定目标与机械度量,让 agent 迭代。每一轮测量、保留有效改动、回滚无效改动,如此往复。"
},
"version": "0.1.0",
"version": "0.2.0",
"author": {
"name": "luochang212"
},
"skills": "./skills",
"keywords": [
"autoresearch",
"experiment",
Expand Down
96 changes: 53 additions & 43 deletions external_plugins/autoresearch/README.md

Large diffs are not rendered by default.

96 changes: 53 additions & 43 deletions external_plugins/autoresearch/README_CN.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions external_plugins/autoresearch/commands/autoresearch.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
description: Enter autoresearch mode. Resumes from .auto/prompt.md when a session exists, otherwise sets one up from your goal. Usage: /autoresearch:autoresearch <goal or resume context>
description: Enter autoresearch mode. Resumes when .auto/log.jsonl exists (the ledger is the session), otherwise sets one up from your goal. Usage: /autoresearch:autoresearch <goal or resume context>
---

Enter autoresearch mode for this workspace.

1. If `.auto/log.jsonl` and `.auto/prompt.md` exist, **resume**: read the charter and the ledger, then continue the loop (one focused change → `run_experiment` → `log_experiment`).
1. If `.auto/log.jsonl` exists, **resume** (the tools and hooks treat the ledger as the session — do not re-run setup: the frozen files already exist and the hook will deny rewriting them): first clear `autoresearchOff` if it is set (set it to false in `.auto/config.json`), then read the charter (`.auto/prompt.md`) and the ledger and continue the loop (one focused change → `run_experiment` → `log_experiment`). If `.auto/prompt.md` is missing, reconstruct a minimal charter from the ledger's config rows and the frozen goal/scope files before continuing.
2. Otherwise **set up a new session** from the goal in $ARGUMENTS:
- Load the skill `autoresearch` (or read `skills/autoresearch/SKILL.md`).
- Follow `references/setup-guide.md`: pick a mechanical metric, create `.auto/measure.sh` (prints `METRIC name=value`), optional `.auto/checks.sh`, write `.auto/prompt.md` charter.
- Follow `references/setup-guide.md`: pick a mechanical metric, create `.auto/measure.sh` (prints `METRIC name=value`), optional `.auto/checks.sh`, record the user's deep goal verbatim in `.auto/goal.md` and the agreed scope boundary in `.auto/scope.md`, write `.auto/prompt.md` charter (its goal/scope sections quote the two files verbatim). Re-targeting after a `/autoresearch:clear`: the old frozen files still exist and the guard denies their rewrites — remove them first (user-approved) when the new target needs a different benchmark or gate.
- `init_experiment` with metric name and direction, run a baseline, then start the loop.
3. Remind the user: keep the benchmark frozen (`.auto/measure.sh` / `.auto/checks.sh` are write-protected by the plugin hook), and use `/autoresearch:export` for a dashboard.
3. Remind the user: keep the session anchors frozen (`.auto/measure.sh` / `.auto/checks.sh` / `.auto/goal.md` / `.auto/scope.md` are write-protected by the plugin hook once created), and use `/autoresearch:export` for a dashboard.

$ARGUMENTS
6 changes: 3 additions & 3 deletions external_plugins/autoresearch/commands/clear.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
description: Clear the autoresearch session by deleting .auto/log.jsonl, then start fresh. Keeps measure.sh / checks.sh / prompt.md. Usage: /autoresearch:clear
description: Clear the autoresearch session by deleting .auto/log.jsonl, then start fresh. All other .auto/ files are kept. Usage: /autoresearch:clear
---

Clear the current autoresearch session.

1. Confirm with the user that they want to wipe the experiment history. This cannot be undone: the session state is gone, though the ledger and all `experiment:` commits remain in git history.
1. Confirm with the user that they want to wipe the experiment history. This cannot be undone: the ledger is deleted, while all `experiment:` commits remain in git history.
2. Call the `clear_experiments` tool.
3. **Scheduled wakeups**: run `CronList` and `CronDelete` every remaining `[autoresearch wake]` task of this session (scheduled tasks cannot be deleted from automation turns, so this must happen in a user turn).
4. Report the result. A fresh target can now start with `/autoresearch:autoresearch <goal>` or `init_experiment`.
4. Report the result. A fresh target can now start with `/autoresearch:autoresearch <goal>` or `init_experiment`. If the new target needs a different benchmark or correctness gate, the old frozen files (`.auto/measure.sh`, `.auto/checks.sh`, and the old `goal.md`/`scope.md`) must be removed first — clear keeps them and the write guard still denies rewrites of files that exist (user-deleted, or an explicitly user-approved command).

Note: kept `experiment:` commits remain in git history; only the `.auto/` session ledger is reset.
4 changes: 2 additions & 2 deletions external_plugins/autoresearch/commands/export.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ description: Render .auto/log.jsonl into autoresearch-dashboard.html. Usage: /au

Export the autoresearch experiment dashboard.

1. Call the `export_dashboard` tool (prefer the MCP tool; the same export logic also lives in `${ZCODE_PLUGIN_ROOT}/mcp/server.ts`).
1. Call the `export_dashboard` tool (prefer the MCP tool; the same export logic also lives in `${ZCODE_PLUGIN_ROOT}/mcp/server.ts`). Besides writing the static file, it starts a live SSE dashboard on `127.0.0.1` (auto-refreshing) and returns its `url`.
2. If the tool is unavailable, fall back to reading `.auto/log.jsonl` yourself, summarizing experiments (status, metric, delta vs baseline, direction), and writing a self-contained `autoresearch-dashboard.html` in the workspace root.
3. Tell the user the file path (`autoresearch-dashboard.html`) and a 2-3 line summary of progress (experiments run, kept, best metric).
3. Open the live `url` **once** in the user's browser (`open` / `xdg-open` / `cmd /c start`) — once per session, not per export; SSE auto-refreshes the page afterward. Skip the open command when headless (SSH / CI / remote executor) or when the user declined watching — then hand over the URL in text only. Always tell the user the static file path (`autoresearch-dashboard.html`; an export-time snapshot, not per-round updated) plus a 2-3 line summary of progress (experiments run, kept, best metric).

If there is no `.auto/log.jsonl`, say so and suggest `/autoresearch:autoresearch` to start a session first.
5 changes: 3 additions & 2 deletions external_plugins/autoresearch/commands/finalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Split kept experiments into clean topic branches you can PR. Usage:
Finalize the experiment session into clean, PR-able topic branches.

1. Read `.auto/log.jsonl`, collect the **kept** experiments (status=keep with a commit).
2. Group them by file dependency: two experiments may share a branch only if their changed files overlap; group small, keep order.
2. Group experiments that touch the same files into the same branch; separate branches must have disjoint file sets (the script rejects overlap). Keep groups small and in commit order. The script verifies that the union of the group branches equals **every** change between the trunk merge-base and HEAD (session files excluded) — manual, non-experiment commits included — so either fold ungrouped commits into a group or the verify step fails and rolls everything back.
3. Write `groups.json` at the project root:
```json
{
Expand All @@ -24,6 +24,7 @@ Finalize the experiment session into clean, PR-able topic branches.
`last_commit` must be the full kept commit hash (`git rev-parse <short>`).
4. Run `bash ${ZCODE_PLUGIN_ROOT}/scripts/finalize.sh <project dir> <path to groups.json>`.
5. **Scheduled wakeups**: run `CronList` and `CronDelete` every remaining `[autoresearch wake]` task of this session (scheduled tasks cannot be deleted from automation turns, so this must happen in a user turn).
6. Report: the created branches (`autoresearch/<goal>/NN-<slug>`), the overall metric improvement, and cleanup notes (`git branch -D` + `rm -r .auto` when done).
6. **Experience review (before any cleanup)**: if `.auto/experience.md` exists, read it with the user and fold the surviving lessons into the project's `AGENTS.md` (drop `NO_ENTRY` stubs and lessons the user rejects). This step MUST complete before deleting the session directory — `experience.md` only lives in `.auto/` and is lost otherwise.
7. Report: the created branches (`autoresearch/<goal>/NN-<slug>`), the overall metric improvement, and cleanup notes (`git branch -D` + `rm -r .auto`) — cleanup runs **only after** the experience review above is done.

If the script reports a file appearing in multiple groups, merge those groups or re-split and rerun.
6 changes: 3 additions & 3 deletions external_plugins/autoresearch/commands/off.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
description: Stop the auto-resume hints while keeping the session. Resume anytime with /autoresearch:autoresearch. Usage: /autoresearch:off
description: Pause the loop and its hints while keeping the session. Resume anytime with /autoresearch:autoresearch. Usage: /autoresearch:off
---

Pause autoresearch without wiping the session.

1. Set `autoresearchOff: true` in `.auto/config.json` (create the file if missing). The SessionStart hook will stop injecting "resume" hints for this workspace.
1. Set `autoresearchOff: true` in `.auto/config.json` (create the file if missing). Edit the single key — do not rewrite the whole file: server-managed keys (`benchmarkHashes`, `harnessHashes`, `pending*`) live here, and a full-file rewrite silently disarms drift detection and the harness trace. Stop continuation, memory injection and the SessionStart resume hint all stop for this workspace.
2. The ledger and all experiment commits stay intact.
3. **Scheduled wakeups**: if unattended mode was active, run `CronList` and `CronDelete` every remaining `[autoresearch wake]` task of this session (scheduled tasks cannot be deleted from automation turns, so this must happen in a user turn; until deleted or maxRuns is hit, leftover wakes idle with one cheap reply).
4. To resume: run `/autoresearch:autoresearch` (it ignores the off marker), or clear the marker (`autoresearchOff: false`) for hints again.
4. To resume: run `/autoresearch:autoresearch` (it clears the off marker), or set `autoresearchOff: false` manually.
5. To start completely fresh: `/autoresearch:clear`.
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,13 @@ const run = p.run_entry;
const session = p.session;
const title = `autoresearch run ${run.run}: ${run.status}`;
const body = `metric=${run.metric ?? '—'} (best=${session?.best_metric ?? '—'}) ${run.description ?? ''}`;
// AppleScript string literal: escape backslashes FIRST (else the quote escape
// itself gets escaped), then quotes; fold newlines — a literal newline inside
// an AppleScript string is a syntax error.
const esc = (s) =>
s.replace(/\\/g, '\\\\').replace(/"/g, '\\"').replace(/\r?\n/g, ' ');
try {
execFileSync('osascript', ['-e', `display notification "${body.replace(/"/g, '\\"')}" with title "${title.replace(/"/g, '\\"')}"`], { stdio: 'ignore' });
execFileSync('osascript', ['-e', `display notification "${esc(body)}" with title "${esc(title)}"`], { stdio: 'ignore' });
} catch {
/* no osascript (non-macOS) → silent */
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
set -euo pipefail

readonly STREAK_THRESHOLD=3
readonly WINDOW=5

payload="$(cat)"

Expand All @@ -21,14 +20,16 @@ const tail = fs.readFileSync(log, 'utf8')
.filter(e => e && e.type === 'run')
.slice(-5);

// Consecutive-failure semantics match the guardrails spec: only real failures
// count; keep AND noop both break the chain.
let streak = 0;
for (const r of [...tail].reverse()) {
if (r.status === 'keep') break;
if (r.status === 'keep' || r.status === 'noop') break;
streak += 1;
}
if (streak < 3) process.exit(0);

console.log(`⚠️ ${streak} consecutive non-keep results. Consider:`);
console.log(`⚠️ ${streak} consecutive failed results. Consider:`);
console.log(' - re-reading .auto/prompt.md and the benchmark script');
console.log(' - something structurally different, not another variation of the same idea');
console.log(' - measuring where time/space is actually spent before the next change');
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#!/usr/bin/env bash
# idea-rotator: pick the next untried idea from .auto/ideas.md (one idea per
# line, lines starting with '#' are ignored) and steer the agent to try it.
# Also checks the ledger for a shallow streak: when the recent tagged
# hypotheses (asi.hypothesis tier prefixes) are all [tune]/[ensemble] with no
# [struct], nudge the agent back to the ideation list.
# before hook. Silent when there is no ideas file or no untried ideas left.
set -euo pipefail

Expand All @@ -10,17 +13,43 @@ node - "$payload" <<'NODE'
const p = JSON.parse(process.argv[2]);
const fs = require('fs');
const ideasFile = `${p.cwd}/.auto/ideas.md`;
if (!fs.existsSync(ideasFile)) process.exit(0);
if (fs.existsSync(ideasFile)) {
const lines = fs.readFileSync(ideasFile, 'utf8')
.split('\n')
.map(l => l.trim())
.filter(l => l && !l.startsWith('#'));

const lines = fs.readFileSync(ideasFile, 'utf8')
.split('\n')
.map(l => l.trim())
.filter(l => l && !l.startsWith('#'));
if (lines.length > 0) {
// Rotate using the run counter so each experiment surfaces a different idea.
const idx = (p.session?.run_count ?? 0) % lines.length;
console.log(`💡 untried idea to consider: ${lines[idx]}`);
console.log(' (add/remove lines in .auto/ideas.md to control the pool)');
}
}

if (lines.length === 0) process.exit(0);

// Rotate using the run counter so each experiment surfaces a different idea.
const idx = (p.session?.run_count ?? 0) % lines.length;
console.log(`💡 untried idea to consider: ${lines[idx]}`);
console.log(' (add/remove lines in .auto/ideas.md to control the pool)');
// Shallow-streak check over the ledger's last (up to 6) run rows.
// Only tagged hypotheses count; untagged ones stay invisible (backward
// compatible with older ledgers). Advisory: this is a nudge, not a gate.
try {
const ledger = `${p.cwd}/.auto/log.jsonl`;
if (!fs.existsSync(ledger)) process.exit(0);
const rows = fs.readFileSync(ledger, 'utf8')
.split('\n')
.filter(l => l.trim())
.map(l => { try { return JSON.parse(l); } catch { return null; } })
.filter(e => e && e.type === 'run')
.slice(-6);
const tagged = rows
.map(r => String(r?.asi?.hypothesis || ''))
.filter(h => /^\[(struct|tune|ensemble)\]/.test(h));
if (tagged.length < 3) process.exit(0);
const shallow = tagged.filter(h => !h.startsWith('[struct]'));
if (shallow.length === tagged.length) {
console.log(' ⚠️ shallow streak: the recent tagged hypotheses are all [tune]/[ensemble].');
console.log(' Rebalance before burning more runs: pick or invent one [struct] direction');
console.log(' (see the cold-start list in .auto/ideas.md).');
}
} catch {
// fail open: never block the loop over ledger-reading trouble
}
NODE
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
# self-check: wall-clock self-assessment for long sessions. When more than
# CHECK_INTERVAL seconds have passed since the last self-check (tracked in
# .auto/selfcheck.state, created on demand), inject a short diagnostic
# checklist — am I still doing structural work, or drifting into shallow
# tuning? Most useful in unattended/overnight runs; harmless otherwise.
# before hook. Silent when the interval has not elapsed or the session has
# no started_at epoch (older ledgers).
set -euo pipefail

payload="$(cat)"

CHECK_INTERVAL="${AUTORESEARCH_SELF_CHECK_INTERVAL:-3600}"

node - "$payload" "$CHECK_INTERVAL" <<'NODE'
const p = JSON.parse(process.argv[2]);
const interval = Number(process.argv[3]) || 3600;
const fs = require('fs');

const started = p.session?.started_at_epoch;
if (!started) process.exit(0); // older ledger: no segment clock, stay silent

const stateFile = `${p.cwd}/.auto/selfcheck.state`;
let last = 0;
try { last = Number(fs.readFileSync(stateFile, 'utf8').trim()) || 0; } catch {}

const now = Math.floor(Date.now() / 1000);
if (now - last < interval) process.exit(0);

try { fs.writeFileSync(stateFile, String(now)); } catch {}

console.log('🕐 hourly self-check — answer these briefly before the next run:');
console.log(' 1. Is my current direction structural work, or shallow tuning in disguise?');
console.log(' 2. Did recent failures get a root-cause analysis, or did I just reach for the safest next move?');
console.log(' 3. Is there an insight in the ledger (a near-miss, an unexplained delta) I have not acted on?');
console.log(' 4. Given the time left: is my frontrunner truly my best candidate, or just the most familiar one?');
console.log(' If any answer points at drift, course-correct now (cold-start list in .auto/ideas.md).');
NODE
Loading
Loading