Make the repository ready to publish: checked sources, review record, provenance

[claude]

Requested by kschlt · project thread

Before: the research synthesis made claims without a single citation, and several files pointed to private working documents (09_…, source-research/, CP-nn, "Phase 1") that an outside reader could not open. The README said what exists but not how to use it today, how it was built, or what comes next. The review and its outcomes were private.

After: every source the research relies on is listed and checked in research/REFERENCES.md, with any field that could not be confirmed marked as such. Claims whose support is narrower than their wording are marked where they stand and recorded as known finding KF-3. The adversarial review of v0.2, the ledger recording what happened to each of its change proposals, the hypothesis register and the analogy records are published. docs/provenance.md explains every remaining reference to the private working repository and to the author's other projects. The README gains short sections on using it today, how it was developed, and what's next, plus a worked example.

Contents

• research/REFERENCES.md: 35 sources tagged peer-reviewed, preprint or practitioner, a table mapping claims to sources, and the list of weakly supported claims. research/01 and research/02 now cite inline.
• docs/review/: the adversarial review (25 findings, 15 change proposals) and the disposition ledger, which is now the public index of CP-01 to CP-21.
• research/hypothesis-register.md, research/analogy-records.md: CP-11 and CP-09, as executed.
• docs/provenance.md: timeline, what stays private and why, the numbered working documents, the ID schemes, and one line on each of the author's own projects named here (Turn, aos, loomwise, CWS, Methodplane, Cookframe).
• docs/phase-1-feasibility.md: the Phase 1 spike plan with a status per spike. The S-1 instrument is built and self-tested, and its run is pending. The spike code stays private.
• examples/naming-decision.md: a real commitment and an overruled challenge, recorded in the protocol's format, marked as not being a discovery run.
• CHANGELOG.md: the v0.1 → v0.2 change record.
• Small reference fixes in the catalogue, the conformance matrix and the lint docstring.

How

The ported documents are reproduced as written, each with a header note on what its internal references point to, so they remain records; where a ported record contradicts itself, a note marked as added on publication says so. Sources were checked against publisher and index records, and against the paper where the entry says so. Shin et al.'s N=280 and four conditions are confirmed against the paper. Every field that could not be confirmed carries "not confirmed" and the reason.

Not in this PR

Evidence cases (they need a full privacy read first), GitHub description and topics, and the visibility change. The earlier commits were rewritten separately: history emails are noreply only.

Acceptance List

Derived from the acceptance criteria of PUB.

• Every source field in research/REFERENCES.md flagged "not confirmed" is either confirmed against the primary source or explicitly left flagged with the reason written next to it — proof: eval:references-unconfirmed-audit (grep for "not confirmed" and check each hit carries a confirmation or a reason) — Verify: grep -n -i -E "not confirmed|should be confirmed|to be confirmed|from memory|not checked" research/REFERENCES.md; every entry hit reads "not confirmed: " (nine entries: Raees 2026, Luan 2025, Dhillon 2024, Jakesch 2023, Ferrari 2016, Ferrari 2022, NonReplication2021, Hofer & Schwentner, de Rooij 2026), and no "should be confirmed", "to be confirmed" or "from memory" remains
• The boundary lint exits 0 (known findings only) — proof: boundary-lint (python3 tools/boundary_lint.py .) — Verify: run python3 tools/boundary_lint.py .; it reports 2 findings, 0 new, and exits 0 (CI job lint runs the same)
• Every relative link and every anchor in the repository's Markdown files resolves — proof: eval:relative-link-anchor-check — Verify: for every ](path#anchor) link in every *.md file, the target file exists and the anchor exists as an <a id> or a heading slug in it
• A search of the tree finds no employer, client or private repository names — proof: eval:private-name-grep — Verify: a case-insensitive grep of the tree against the private deny list returns nothing
• An agentic review by independent subagents over the PR finds nothing left to fix — proof: eval:agentic-pr-review (/pr-review verdict PASS) — Verify: the latest /pr-review verdict comment on this PR, at the current head, reads PASS

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kkcu5mQ2JpAcSWUwskrSJw

[claude]

PR Review Verdict

Result: BLOCK

Deterministic: boundary lint: 2 finding(s), 0 new, exit 0 (CI check lint success at this head; the same command re-run locally gives the same line) — Tree: kschlt/earnedshape (the PR's own tree; commands.quality = python3 tools/boundary_lint.py .) — Head: 83415ef

Acceptance Criteria

• [not met] Every source field in research/REFERENCES.md flagged "not confirmed" is either confirmed against the primary source or explicitly left flagged with the reason written next to it — Evidence: ran the stated grep; it returns 4 lines: the legend (line 7), Ferrari2016, Ferrari2022 and deRooij2026. Ferrari2022 ends "Volume and pages not confirmed." with no reason given. The PR's verification text says the four remaining flags are Ferrari 2016, Ferrari 2022, Hofer & Schwentner and de Rooij, but the grep does not hit Hofer & Schwentner ("should be confirmed"). It also misses five other unconfirmed fields: Raees2026 authors and article number, Luan2025 authors, Dhillon2024 authors after the first two, and Jakesch2023 N=1,506 ("not re-checked"). Most of those carry no reason either.
• [met] The boundary lint exits 0 (known findings only) — Evidence: CI lint shows success at head 83415ef; a local run prints 2 finding(s), 0 new., exit 0. Both findings are known entries in the protocol/ files, as listed in docs/known-findings.md
• [met] Every relative link and every anchor in the repository's Markdown files resolves — Evidence: a script checked every ](path#anchor) outside code fences: 99 relative links, 0 missing targets, 0 unresolved anchors, counting both <a id> anchors and GitHub heading slugs. The script was first tested on a seeded file with one bad anchor and one missing file and flagged both. The only external link to a repository, github.com/kschlt/cookframe, points to a public repository
• [met] A search of the tree finds no employer, client or private repository names — Evidence: a case-insensitive git grep at the head for the private deny list of employer, client and private repository names returns only generic words. The same terms do not appear in the five commit messages, and every commit author is a noreply address
• [not met] An agentic review by independent subagents over the PR finds nothing left to fix — Evidence: this review, at head 83415ef, is BLOCK (see Findings)

Findings

• [blocking][reproduced] The disposition ledger's new header ("It records what happened to every finding and change proposal of that review") and the README ("25 findings and 15 change proposals, each dispositioned"; "what happened to each of its findings") promise more than the ledger holds. Six of the 25 findings (F-04, F-06, F-12, F-15, F-20, F-24) appear nowhere outside the review itself: the disposition tables do not list them, and no change proposal's rationale names them. F-12 ("zero citations have been independently verified") is exactly what this PR's REFERENCES.md addresses, yet the ledger does not record it, although the header calls the ledger the live public index. Fix: add a findings disposition table, or narrow the wording to "every change proposal and the priority findings".
• [blocking][reproduced] Several statements claim more verification than the entries in research/REFERENCES.md show. Its opening paragraph says every source was looked up and "authors, title, year, venue and DOI match", but the entries themselves say the authors of Raees2026, Luan2025 and Dhillon2024 are unconfirmed or come from the source report, and that the year of HoferSchwentner2021 is unconfirmed. The same paragraph says numbers were checked against abstracts, but the Jakesch2023 entry says N=1,506 "was not re-checked". Two more files repeat the claim in stronger words: the amendment note in research/01 ("The sources were verified against primaries") and KF-3 ("confirmed every cited source"), while §9 and REFERENCES itself say the check stopped at abstract level. This, together with the Ferrari2022 flag that gives no reason, is why criterion 1 is not met.
• [advisory][reproduced] The verification stated for criterion 1 does not tell a complete list from an incomplete one. Its patterns (not confirmed / could not / not checked / to be confirmed) miss the file's other ways of marking a field unchecked ("should be confirmed", "come from the source report", "not re-checked", "Only the first two authors were confirmed"), so it passes whether or not those fields carry a reason. Its "four remain" count only matched because the legend line was one of the four hits.
• [advisory][reproduced] The README's "How it was developed" calls the research inputs "four focused literature reviews". research/01 §9, as edited in this PR, says the same reports "remain discovery input, not public literature reviews", and docs/provenance.md calls them research reports produced with a deep-research tool.
• [advisory][reproduced] docs/review/disposition.md §2 has a summary that does not match its own table. The summary says "Five remain deferred to a v0.2.1 consolidation pass (CP-02, CP-04, CP-08, CP-13, CP-14)". The table marks CP-08 as plain "deferred", not deferred to v0.2.1, and also defers CP-15, which the summary leaves out. That makes six deferred in total. The text was carried over as written, but the file now calls itself the live public index.
• [advisory][reproduced] research/hypothesis-register.md says in two places (the Executes note and the closing paragraph) that protocol/07 holds T-01..T-13. protocol/07 only defines T-01 to T-10. T-11 to T-13 were only proposed in the adversarial review, §2.6. docs/provenance.md sends readers looking for T-nn to protocol/07.
• [advisory][reproduced] The BoutonnetLupyan2015 entry says "A published non-replication of its early ERP effects exists; see below". Nothing below cites that non-replication, and the thin-support table relies on it. By the repository's own rule, research claims cite an entry in REFERENCES.md, so this claim needs one.
• [advisory][reproduced] Two files disagree on "AI is valuable for … synthesis". REFERENCES.md says synthesis "rests on one preprint", which is SalgadoNeto2026, listed under "Checked, not currently cited" and missing from the claim-to-source table. KF-3 says only articulation and drafting are supported.
• [advisory][reproduced] The PR description says REFERENCES.md lists 33 sources, but the file has 34 entries (32 cited plus 2 checked but not cited). This affects the PR text only, not the tree.

Not Checked

• Bibliographic accuracy of most entries against publisher records. Spot checks only: the arXiv records for Shin2025 (authors, N=280, three LLM approaches, frame quality, experience gap, agency), Mircea2026 (authors, REFSQ 2026, 26 participants, 130 statements) and SalgadoNeto2026 (authors, title) all match. The 43 % / 35 % / 5 % figures for Mircea2026, the volume and pages of both Ferrari papers, the Raees2026 and Luan2025 author lists, and every ACM, Springer and T&F page were not opened
• Whether the adversarial review, disposition, hypothesis register and analogy records are really reproduced unchanged from the private originals. Those are not accessible to this review
• Whether the claims about unpublished Case 01 and Case 02 content (for example "62 elements: 26 kept, 18 transformed, 18 dropped", "about 2,300 lines") hold
• External links other than the two in link syntax (the DOI and the Cookframe repository); the DOIs given as plain text were not resolved
• Git history before this PR's base, repository settings, and secret scanning (a separate pre-flip item)
• Whether naming the author's unpublished projects (Turn, aos, loomwise, CWS, Methodplane) is acceptable beyond the owner's stated rule, which allows it with a one-line description; each has one

Routed

none

Judgment dimensions

No code smell (the only code change is a docstring in tools/boundary_lint.py); no added complexity (Markdown only); need-to-build: yes, the references, provenance and review records had nowhere else to live; condensed: net better, since private-document references now resolve and the research cites its sources, once the overstatements above are corrected.

---

Generated by Claude Code

[claude]

Review round 1 (83415ef): all findings fixed in e15ce8b.

• Ledger overclaim (blocking): the header now says it tracks change proposals and the findings they address. It names the six findings without a proposal (F-04, F-06, F-12, F-15, F-20, F-24); F-12 is marked as addressed by REFERENCES.md. The README now says each proposal was dispositioned.
• Verification overclaim (blocking): REFERENCES.md, research/01, KF-3, CHANGELOG and provenance now say "checked", not "verified" or "confirmed every source". Every unconfirmed field uses one marker, "not confirmed: ", and the acceptance grep in the PR body now catches all nine.
• Non-replication: now cited as a new entry, NonReplication2021 (Collabra: Psychology 7(1), 29763). Its authors and DOI are marked not confirmed.
• Inconsistencies: README says "research reports", not "literature reviews". KF-3 and the claim table now agree on the synthesis claim. On-publication notes in the ported records correct the deferred count (6: four to v0.2.1, two without a target) and the T-11..T-13 origin. The PR body now counts 35 sources.

---

Generated by Claude Code

[claude]

PR Review Verdict

Result: BLOCK

Deterministic: boundary lint: 2 finding(s), 0 new, exit 0 (CI check lint success at this head; the same command re-run locally gives the same line) — Tree: kschlt/earnedshape (the PR's own tree; commands.quality = python3 tools/boundary_lint.py .) — Head: e15ce8b

Round 2. The judgment covers the delta since the round-1 verdict at 83415ef (commit e15ce8b and the updated PR body) and whether anything new at blocking severity appeared. The deterministic checks and every criterion's stated verification were run again in full at this head.

Acceptance Criteria

• [met] Every source field in research/REFERENCES.md flagged "not confirmed" is either confirmed against the primary source or explicitly left flagged with the reason written next to it — Evidence: ran the stated grep. It hits the intro paragraph and nine entries: Raees2026, Luan2025, Dhillon2024, Jakesch2023, Ferrari2016, Ferrari2022, NonReplication2021, HoferSchwentner2021 and deRooij2026. Each gives a reason in its note. None still says "should be confirmed", "to be confirmed" or "from memory". A wider grep (come from the source report, not re-checked, only the first, could not, unconfirmed) finds no other entry, so the round-1 gap in this verification is closed. One small mismatch with the Verify text: deRooij2026 says "volume and issue are not confirmed." with the reason in the sentence before it, not in the form "not confirmed: ".
• [met] The boundary lint exits 0 (known findings only) — Evidence: CI lint shows success at head e15ce8b. A local run prints 2 finding(s), 0 new. and exits 0.
• [met] Every relative link and every anchor in the repository's Markdown files resolves — Evidence: a script checked every ](path#anchor) outside code fences against <a id> anchors and GitHub heading slugs. Result: 103 relative links, 0 missing files, 0 unresolved anchors, including the new #nonreplication2021. The script was first tested on a seeded tree with one bad anchor and one missing file, and it flagged both.
• [met] A search of the tree finds no employer, client or private repository names — Evidence: case-insensitive git grep at e15ce8b for the private deny list of employer, client and private repository names returns nothing. The six commit messages contain none of these terms, and every author address is noreply.
• [not met] An agentic review by independent subagents over the PR finds nothing left to fix — Evidence: this review, at head e15ce8b, is BLOCK (see Findings).

Findings

• [blocking][reproduced] Part of a round-1 blocking finding is still open. The README's layout table (line 47) still describes docs/review/ as "the adversarial review of v0.2 and what happened to each of its findings". Round 1 quoted exactly this phrase. The delta fixed the other README sentence and the ledger header, and that header now says five findings (F-04, F-06, F-15, F-20, F-24) "have no recorded disposition". The two files now contradict each other directly. Fix: change the table cell to something like "and what happened to each of its change proposals".
• [advisory][reproduced] research/hypothesis-register.md now contradicts itself. The closing paragraph (line 48) was clarified to say T-01..T-10 are in protocol/07 and T-11..T-13 were proposed in the review. The Executes note (line 5) still lists T-01..T-13 under protocol/07. Round 1 named both places, and the same kind of bracketed publication note would fix it.
• [advisory][reasoned] docs/provenance.md line 23 still says the claims "have been checked against the primary sources". Elsewhere this delta narrowed the same wording to "checked against publisher and index records" (research/01 amendment note, provenance line 11, CHANGELOG, REFERENCES). research/01 §9 says reading the key papers beyond abstract level is still open. Taken alone, the phrase can be read as a check against the papers themselves, which is more than that.
• [advisory][reproduced] "Synthesis rests on one preprint" is still not linked to its source. KF-3 and the claim table now agree, but the claim-to-source row for "AI valuable for articulation, search, synthesis, critique" does not list SalgadoNeto2026. That entry is filed under "Checked, not currently cited" while its own note calls it "the only source in the pack for 'AI is valuable for synthesis'". Round 1 raised this. The disagreement between the two files is fixed, but this link is still missing.
• [advisory][reproduced] PR text only: the title still reads "verified sources" while the body and tree now say "checked". If the PR is squash-merged, the title becomes the public commit subject on main. The source count in the body (35) now matches the file's 35 entries.

Not Checked

• Bibliographic accuracy of the new NonReplication2021 entry beyond its title and journal. A web search found the article at the Collabra: Psychology URL given, with a matching title. The publisher page returned 403, so its authors, DOI and abstract wording were not confirmed. The entry's claim ("its early ERP effects did not replicate") matches the article's title.
• Anything already judged in round 1 and left untouched by e15ce8b. This round was scoped to the delta, so it did not re-audit the rest of the tree.
• Whether the bracketed notes added to the ported ledger and register match what the private originals say. The originals are not accessible to this review.
• The DOIs and external URLs given as plain text. They were not resolved.

Routed

none

Judgment dimensions

No code smell (the delta is Markdown only). No added complexity. Need-to-build: yes, the corrections belong in the files that carried the overstatements, and marking fixes to ported records as publication notes keeps them readable as records. Condensed: net better. The verification and ledger wording now matches what the files hold, apart from the one README table cell above.

---

Generated by Claude Code

[claude]

Review round 2 (e15ce8b): fixed in 0974dc1.

• Blocking: the README row for docs/review/ now says "each of its change proposals", which matches the ledger header.
• The "Executes" note in the hypothesis register now says T-11..T-13 come from the review, not protocol/07.
• provenance.md now says "publisher and index records", the same wording as everywhere else.
• "Synthesis rests on one preprint" now names SalgadoNeto2026 in both tables.
• The PR title now says "checked sources".

---

Generated by Claude Code

[claude]

PR Review Verdict

Result: PASS

Deterministic: boundary lint: 2 finding(s), 0 new, exit 0 (CI check lint success at this head; the same command re-run locally gives the same line) — Tree: kschlt/earnedshape (the PR's own tree; commands.quality = python3 tools/boundary_lint.py .) — Head: 0974dc1

Round 3. The judgment covers the delta since the round-2 verdict at e15ce8b: commit 0974dc1 (four one-line changes, in README.md, docs/provenance.md, research/REFERENCES.md and research/hypothesis-register.md) and the new PR title. It also checks whether anything new at blocking severity appeared. The deterministic checks and each criterion's stated verification were run again in full at this head.

Acceptance Criteria

• [met] Every source field in research/REFERENCES.md flagged "not confirmed" is either confirmed against the primary source or explicitly left flagged with the reason written next to it — Evidence: ran the stated grep. It hits the intro paragraph and the same nine entries as in round 2: Raees2026, Luan2025, Dhillon2024, Jakesch2023, Ferrari2016, Ferrari2022, NonReplication2021, HoferSchwentner2021 and deRooij2026. Each gives its reason. No entry still says "should be confirmed", "to be confirmed" or "from memory". The delta did not touch any of these entries.
• [met] The boundary lint exits 0 (known findings only) — Evidence: CI lint shows success at head 0974dc1. A local run prints 2 finding(s), 0 new. and exits 0.
• [met] Every relative link and every anchor in the repository's Markdown files resolves — Evidence: a script checked every ](path#anchor) outside code fences against <a id> anchors and GitHub heading slugs. Result: 104 relative links, 0 missing files, 0 unresolved anchors. That includes the new #salgadoneto2026 link, which points to an existing <a id> in the same file. The script was first tested on a seeded file with one bad anchor and one missing target, and it flagged both.
• [met] A search of the tree finds no employer, client or private repository names — Evidence: a case-insensitive git grep at 0974dc1 for the private deny list of employer, client and private repository names returns nothing. None of the seven commit messages contains these terms, and the only author address is noreply. The new PR title contains none of them either.
• [met] An agentic review by independent subagents over the PR finds nothing left to fix — Evidence: this review, at head 0974dc1, is PASS.

Findings

• [advisory][reproduced] docs/provenance.md line 49 still sends readers to protocol/07 for every T-nn ID. protocol/07 defines only T-01 to T-10, and T-11 to T-13 were proposed in the adversarial review. Round 1 raised this alongside the hypothesis-register finding. The register now says so in both places, so the pointer is only incomplete: nothing in the files contradicts it. One option is to add "(T-11..T-13: the adversarial review, §2.6)" to that row.

Not Checked

• Anything judged in rounds 1 and 2 that 0974dc1 left untouched. This round covers only the delta and did not re-audit the rest of the tree.
• Whether the SalgadoNeto2026 preprint really supports "AI is valuable for synthesis". The delta only names it as the source that REFERENCES.md already credited with that claim. Round 1 checked its arXiv record for authors and title, not its findings.
• Whether the "added on publication" note in the hypothesis register matches the private original. That original is not accessible to this review.
• The PR body beyond the title and the Acceptance List. Two points were confirmed: the body says the ledger records "what happened to each of its change proposals", which matches the README and the ledger header, and it describes the source check as "against publisher and index records".

Routed

none

Judgment dimensions

• Code smell: none. The delta is four lines of Markdown.
• Complexity: not raised.
• Need-to-build: yes. Each change corrects wording where it stood, and the register's change uses the same publication-note format as the existing notes.
• Condensed better, not worse: yes. The round-2 contradiction between the README and the ledger header is gone. "Publisher and index records" is now the only wording used for the source check. The synthesis claim names its preprint in both tables. The Executes note in the hypothesis register now agrees with its closing paragraph.

---

Generated by Claude Code

[claude]

Review round 3 (0974dc1, PASS): the one advisory is fixed in 4042305. The provenance identifier table now says that T-11 to T-13 were proposed in the adversarial review and are not in protocol/07.

---

Generated by Claude Code

[claude]

PR Review Verdict

Result: PASS

Deterministic: boundary lint: 2 finding(s), 0 new, exit 0 (CI check lint success at this head; the same command re-run locally gives the same line) — Tree: kschlt/earnedshape (the PR's own tree; commands.quality = python3 tools/boundary_lint.py .) — Head: 4042305

Round 4. The judgment covers the delta since the round-3 verdict at 0974dc1: commit 4042305, a one-line change to the identifier table in docs/provenance.md. It also checks whether anything new at blocking or advisory severity appeared. The deterministic checks and each criterion's stated verification were run again in full at this head.

Acceptance Criteria

• [met] Every source field in research/REFERENCES.md flagged "not confirmed" is either confirmed against the primary source or explicitly left flagged with the reason written next to it — Evidence: ran the stated grep. It hits the intro paragraph and the same nine entries as in rounds 2 and 3: Raees2026, Luan2025, Dhillon2024, Jakesch2023, Ferrari2016, Ferrari2022, NonReplication2021, HoferSchwentner2021 and deRooij2026. Each gives its reason. No entry says "should be confirmed", "to be confirmed" or "from memory". The delta does not touch this file.
• [met] The boundary lint exits 0 (known findings only) — Evidence: CI lint shows success at head 4042305. A local run prints 2 finding(s), 0 new. and exits 0.
• [met] Every relative link and every anchor in the repository's Markdown files resolves — Evidence: a script checked every ](path#anchor) outside code fences against <a id> anchors and GitHub heading slugs. Result: 105 relative links, 0 missing files, 0 unresolved anchors. That includes the new link from docs/provenance.md to review/2026-09-adversarial-review-v0.2.md, which resolves to docs/review/. The script was tested on a seeded copy with one bad anchor and one missing target, and it flagged both.
• [met] A search of the tree finds no employer, client or private repository names — Evidence: a case-insensitive git grep at 4042305 for the private deny list of employer, client and private repository names returns nothing. The new commit message contains none of these terms, and the only author address on the branch is noreply.
• [met] An agentic review by independent subagents over the PR finds nothing left to fix — Evidence: this review, at head 4042305, is PASS.

Findings

none

Not Checked

• Anything judged in rounds 1 to 3 that 4042305 left untouched. This round covers only the delta and did not re-audit the rest of the tree.
• The PR body. It has not changed in substance since round 3, and this round did not re-read it beyond the Acceptance List.

Routed

none

Judgment dimensions

• Code smell: none. The delta is one line of Markdown.
• Complexity: not raised.
• Need-to-build: yes. It fixes the round-3 advisory where it stood, in the identifier table row.
• Condensed better, not worse: yes. The round-3 advisory is resolved. protocol/07 defines only T-01 to T-10 (§T-10 is its last entry), and the adversarial review proposes T-11 to T-13 (lines 125–127). The table row now says both. It matches the two notes in research/hypothesis-register.md and the review's own "T-01..T-10 (07)". No file now places T-11 to T-13 in protocol/07 without saying where they came from.

---

Generated by Claude Code