diff --git a/README.md b/README.md index 6e00039..0c9f3a9 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ The schema follows a few rules, informed by how agent-maintained wikis actually - **Trust is structural, not remembered.** Immutable `raw/`, append-only `log.md`, a git commit per operation, a pre-commit hook that lints the staged snapshot (exactly the bytes that will land) and makes lint errors uncommittable, and commit-pinned verification fields. Agent-maintained judgment metadata decays, so `confidence:` is reduced to the two states a lint can actually check (`low` = uncited, `contested` = sources disagree and the body must explain), inferred claims are marked inline with `(inferred)`, and tags are validated against a `taxonomy.md`, whose `## Page types` section also describes every allowed page type with a one-line meaning, so each bundle self-describes its type vocabulary to OKF consumers. - **Contested is a state to exit.** The documented failure mode of agent wikis is contradictions accumulating faster than they resolve. Lint flags contested pages older than 30 days; the reconcile workflow rewrites in place, moving losing claims to a dated "Superseded claims" section instead of deleting them. - **Autonomous but reversible.** The maintenance workflow runs unattended on a `maintenance` branch with an exhaustively-listed set of safe actions (mechanical fixes, index rebuild, unambiguous cross-links); everything else becomes a proposal. The human reviews the branch diff and merges. Nothing automated ever lands on main directly. -- **Native OKF conformance.** Every wiki is an [OKF v0.2](https://github.com/GoogleCloudPlatform/open-knowledge-format/blob/main/SPEC.md) bundle: markdown files with YAML frontmatter, ordinary markdown links as the edge form (bundle-absolute `[title](/dir/page.md)`, the form v0.2 §6.1 recommends), reserved `index.md` (stamped with `okf_version` frontmatter by `rebuild-index`, the mechanism §12 specifies) and `log.md` (date-grouped headings, bold-action-word entries). `check_okf` enforces the spec's three conformance rules: parseable frontmatter on every non-reserved `.md`, a non-empty `type`, and reserved-file structure. Provenance uses the spec's `sources` shape (§5.1): mapping entries with a required `resource`, enforced by `check_sources`, with pre-0.2 string entries downgraded to warnings. Scriptorium's schema is a strict superset of OKF's (the spec's `generated.at` is our `updated`; its `resource` is our `source_path`; its `title` is optional, with the index prettifying filenames when absent). One deliberate deviation: `raw/` is excluded from conformance because prime directive 1 makes those sources immutable, and OKF has no concept of a non-concept directory. +- **Native OKF conformance.** Every wiki is an [OKF v0.2](https://github.com/GoogleCloudPlatform/open-knowledge-format/blob/main/SPEC.md) bundle: markdown files with YAML frontmatter, ordinary markdown links as the edge form (bundle-absolute `[title](/dir/page.md)`, the form v0.2 §6.1 recommends), reserved `index.md` (stamped with `okf_version` frontmatter by `rebuild-index`, the mechanism §12 specifies) and `log.md` (date-grouped headings, bold-action-word entries). `check_okf` enforces the spec's three conformance rules: parseable frontmatter on every non-reserved `.md`, a non-empty `type`, and reserved-file structure. Provenance uses the spec's `sources` shape (§5.1): mapping entries with a required `resource`, enforced by `check_sources`, with pre-0.2 string entries downgraded to warnings. Pages carry OKF's own page-level keys directly: a required `title`, `generated: { by, at }` with an offset datetime (§5.2; the spec's last-meaningful-change record), and the §5.4 `status` values (`draft | stable | deprecated`), with `check_okf_fields` validating their shape and a page named in another page's `supersedes` required to be `deprecated`. Everything else (`created`, `confidence`, `supersedes`, and the spec's `resource` as our `source_path`) is a producer extension key, so the schema stays a strict superset of OKF's. One deliberate deviation: `raw/` is excluded from conformance because prime directive 1 makes those sources immutable, and OKF has no concept of a non-concept directory. - **Opt-in extension points for non-wiki trees.** The engine can also lint markdown trees that aren't wikis (a findings folder, a labs journal): `okf_conformance: False` turns off the OKF rules for trees that aren't bundles, `non_page_allowed` accepts glob patterns, `index_file`/`index_body_fn` relocate and reshape the generated index (`index_file: None` disables it), `extra_secret_patterns`/`secret_allow_res` extend the secrets scan, `extra_checks` runs custom callables, and `extra_commands` registers extra `lint.py ` subcommands (`{verb: (callable(root) -> exit code, help)}`) which dispatch ahead of the wiki-root guard and appear in `lint.py help` alongside the engine's own verbs, so a wiki never has to intercept `argv` and end up with a second, partial usage string. Every knob's default lives once in `wikilint/settings.py` (`DEFAULTS`) and a wiki's `lint.py` lists a key only to override it. `DEFAULTS` covers *every* key the engine reads, in two documented tiers: `EXTENSION_DEFAULTS` (the extension points above) preserves the original wiki behavior, so a wiki written before an extension existed keeps behaving as it did; `CORE_DEFAULTS` (the schema knobs — page dirs, required fields, staleness, ADRs, mermaid, coverage, ...) defaults to the neutral/disabled value, so a default can only ever silence a check, never invent one. A minimal config is therefore a handful of lines rather than a full key list. Bad values (a malformed regex, an out-of-tree `index_file`, a non-callable check) are rejected at startup with a clear message rather than a mid-run traceback. ## Unattended maintenance diff --git a/tests/extension_configs.py b/tests/extension_configs.py index 11a7963..b2a3641 100644 --- a/tests/extension_configs.py +++ b/tests/extension_configs.py @@ -6,7 +6,9 @@ def index_entry_extra(fields): - return f"(updated {fields.get('updated', '?')})" + generated = fields.get("generated") + at = generated.get("at", "") if isinstance(generated, dict) else "" + return f"(updated {at[:10] or '?'})" # knobs: infra @@ -41,11 +43,12 @@ def index_entry_extra(fields): 'contested_max_days': 30, 'reverse_fields': ['depends_on'], 'membership': {'member_type': 'component', + 'status_field': 'component_status', 'active_statuses': ['active'], 'container_type': 'topology', 'container_field': 'includes'}, - 'required_fields': ['type', 'created', 'updated', 'description', 'tags'], - 'type_required': {'component': ['component_kind', 'status', 'last_verified'], + 'required_fields': ['type', 'created', 'generated.at', 'description', 'tags'], + 'type_required': {'component': ['component_kind', 'component_status', 'last_verified'], 'topology': ['scope', 'includes'], 'concept': [], 'runbook': [], @@ -62,7 +65,7 @@ def index_entry_extra(fields): 'subnet', 'peer', 'volume'], - 'status': ['active', + 'component_status': ['active', 'planned', 'deprecated', 'retired']}, @@ -137,13 +140,14 @@ def index_entry_extra(fields): 'producers', 'consumers'], 'membership': {'member_type': 'module', + 'status_field': 'module_status', 'active_statuses': ['active'], 'container_type': 'architecture', 'container_field': 'includes'}, - 'required_fields': ['type', 'created', 'updated', 'description', 'tags'], + 'required_fields': ['type', 'created', 'generated.at', 'description', 'tags'], 'type_required': {'module': ['source_path', 'language', - 'status', + 'module_status', 'last_verified_commit', 'last_verified'], 'service': ['source_path', @@ -159,7 +163,7 @@ def index_entry_extra(fields): 'storage', 'last_verified_commit'], 'architecture': ['scope', 'includes'], - 'adr': ['adr_number', 'status', 'date'], + 'adr': ['adr_number', 'adr_status', 'date'], 'concept': [], 'runbook': [], 'postmortem': [], @@ -167,7 +171,7 @@ def index_entry_extra(fields): 'source': [], 'query': []}, 'enum_fields': {'confidence': ['low', 'contested']}, - 'type_enum_fields': {'module': {'status': ['active', + 'type_enum_fields': {'module': {'module_status': ['active', 'experimental', 'deprecated', 'removed']}, @@ -190,7 +194,7 @@ def index_entry_extra(fields): 'data-flow', 'deployment', 'request-flow']}, - 'adr': {'status': ['proposed', + 'adr': {'adr_status': ['proposed', 'accepted', 'superseded', 'rejected']}}, @@ -270,7 +274,7 @@ def index_entry_extra(fields): 'membership': None, 'required_fields': ['type', 'created', - 'updated', + 'generated.at', 'description', 'tags', 'subsystem'], @@ -278,7 +282,7 @@ def index_entry_extra(fields): 'owners': [], 'module': ['source_path', 'language', - 'status', + 'module_status', 'criticality', 'last_verified_commit', 'last_verified', @@ -296,7 +300,7 @@ def index_entry_extra(fields): 'storage', 'last_verified_commit'], 'architecture': ['scope', 'includes'], - 'adr': ['adr_number', 'status', 'date'], + 'adr': ['adr_number', 'adr_status', 'date'], 'concept': [], 'runbook': [], 'postmortem': [], @@ -304,7 +308,7 @@ def index_entry_extra(fields): 'source': [], 'query': []}, 'enum_fields': {'confidence': ['low', 'contested']}, - 'type_enum_fields': {'module': {'status': ['active', + 'type_enum_fields': {'module': {'module_status': ['active', 'experimental', 'deprecated', 'removed'], @@ -335,7 +339,7 @@ def index_entry_extra(fields): 'data-flow', 'deployment', 'request-flow']}, - 'adr': {'status': ['proposed', + 'adr': {'adr_status': ['proposed', 'accepted', 'superseded', 'rejected']}}, diff --git a/tests/helpers.py b/tests/helpers.py index 873e9c8..3d47fe6 100644 --- a/tests/helpers.py +++ b/tests/helpers.py @@ -47,9 +47,12 @@ def use_variant(variant): def page(ptype, description, extra_fm="", body="Body.\n", tags="[alpha]", created=TODAY, updated=None): + """A valid page in the template's OKF shape. `updated` is the date of + the last edit, written as `generated.at` (an offset datetime).""" updated = updated or created return ( - f"---\ntype: {ptype}\ncreated: {created}\nupdated: {updated}\n" + f"---\ntype: {ptype}\ntitle: Test Page\ncreated: {created}\n" + f"generated: {{ by: claude-code/test, at: {updated}T00:00:00Z }}\n" f"description: {description}\ntags: {tags}\nsources: []\n{extra_fm}---\n\n{body}" ) diff --git a/tests/test_checks.py b/tests/test_checks.py index 6a48a04..78685db 100644 --- a/tests/test_checks.py +++ b/tests/test_checks.py @@ -40,7 +40,7 @@ def test_empty_required_list_is_missing(self): "concepts/empty-tags.md": page("concept", "Empty tags.", tags=""), }) (root / "concepts/empty-tags.md").write_text( - "---\ntype: concept\ncreated: 2026-07-01\nupdated: 2026-07-01\n" + "---\ntype: concept\ntitle: T\ncreated: 2026-07-01\ngenerated: { by: claude-code/test, at: 2026-07-01T00:00:00Z }\n" "description: Empty tags.\ntags:\nsources: []\n---\n\nBody.\n" ) report = gather(root) @@ -65,7 +65,7 @@ def test_trailing_space_delimiter_keeps_body(self): use_variant("infra") root = make_wiki(self.tmp, variant="infra", files={ "topology/lan.md": ( - "---\ntype: topology\ncreated: 2026-07-01\nupdated: 2026-07-01\n" + "---\ntype: topology\ntitle: T\ncreated: 2026-07-01\ngenerated: { by: claude-code/test, at: 2026-07-01T00:00:00Z }\n" "description: LAN map.\ntags: [alpha]\nsources: []\nscope: l2\n" "includes: [topology/lan.md]\n--- \n\n```mermaid\nflowchart LR\n a[A] --> b[B]\n```\n" ), @@ -120,7 +120,7 @@ class TestMembership(WikiTest): def test_active_component_outside_topology_flagged(self): # Review regression: the old "topology orphans" rule had been dropped. component = page("component", "A switch.", tags="[alpha]", - extra_fm="component_kind: device\nstatus: active\n" + extra_fm="component_kind: device\ncomponent_status: active\n" f"last_verified: {TODAY}\ndepends_on: []\n") root = make_wiki(self.tmp, variant="infra", files={ "components/switch-01.md": component, @@ -134,7 +134,7 @@ def test_included_component_passes(self): root = make_wiki(self.tmp, variant="infra", files={ "components/switch-01.md": page( "component", "A switch.", - extra_fm="component_kind: device\nstatus: active\n" + extra_fm="component_kind: device\ncomponent_status: active\n" f"last_verified: {TODAY}\ndepends_on: []\n"), "topology/lan.md": page( "topology", "LAN.", extra_fm="scope: l2\nincludes: [components/switch-01.md]\n", @@ -245,7 +245,7 @@ def test_pinning_quotes_block_lists_and_decoy_fence(self): def test_sync_drift_uses_unquoted_commit(self): module = page("module", "Auth module.", tags="[alpha]", - extra_fm="source_path: libs/auth/\nlanguage: go\nstatus: active\n" + extra_fm="source_path: libs/auth/\nlanguage: go\nmodule_status: active\n" "depends_on: []\nlast_verified_commit: e4f5g6h\n" f"last_verified: {TODAY}\n") root = make_wiki(self.tmp, variant="pinned-repo", @@ -267,7 +267,7 @@ def test_all_relationship_fields_reversed(self): root = make_wiki(self.tmp, variant="pinned-repo", files={ "modules/auth.md": page( "module", "Auth.", extra_fm="source_path: libs/auth/\nlanguage: go\n" - f"status: active\ndepends_on: []\nlast_verified_commit: abc\nlast_verified: {TODAY}\n"), + f"module_status: active\ndepends_on: []\nlast_verified_commit: abc\nlast_verified: {TODAY}\n"), "apis/auth-http.md": page( "api", "Auth API.", extra_fm="api_kind: http\ndefined_in: modules/auth.md\n" "stability: stable\nlast_verified_commit: abc\n"), @@ -301,9 +301,9 @@ def test_hot_core_cap(self): class TestAdrsAndStaleness(WikiTest): def test_adr_gap_and_missing_forward_link(self): adr = page("adr", "Choose postgres.", tags="[alpha]", - extra_fm="adr_number: 0001\nstatus: superseded\ndate: 2026-07-01\n") + extra_fm="adr_number: 0001\nadr_status: superseded\ndate: 2026-07-01\n") adr3 = page("adr", "Choose sqlite.", tags="[alpha]", - extra_fm="adr_number: 0003\nstatus: accepted\ndate: 2026-07-02\n") + extra_fm="adr_number: 0003\nadr_status: accepted\ndate: 2026-07-02\n") root = make_wiki(self.tmp, variant="pinned-repo", files={ "adrs/0001-choose-postgres.md": adr, "adrs/0003-choose-sqlite.md": adr3, @@ -312,11 +312,21 @@ def test_adr_gap_and_missing_forward_link(self): self.assertEqual(len(findings(report, "adr", "ERROR")), 1) # no forward link self.assertEqual(len(findings(report, "adr", "WARNING")), 1) # gap 0002 + def test_legacy_adr_status_still_checked(self): + """Review regression: ADR trees predating adr_status keep their + superseded-without-forward-link check.""" + adr = page("adr", "Legacy ADR.", + extra_fm="adr_number: 0001\nstatus: superseded\ndate: 2026-07-01\n") + root = make_wiki(self.tmp, variant="pinned-repo", + files={"adrs/0001-legacy.md": adr}) + msgs = [m for _, _, _, m in findings(gather(root), "adr", "ERROR")] + self.assertTrue(any("superseded_by" in m for m in msgs), msgs) + def test_last_verified_staleness(self): root = make_wiki(self.tmp, variant="infra", files={ "components/old-box.md": page( "component", "Old box.", - extra_fm="component_kind: host\nstatus: active\n" + extra_fm="component_kind: host\ncomponent_status: active\n" f"last_verified: {DAYS_AGO_95}\ndepends_on: []\n"), }) report = gather(root) @@ -335,7 +345,7 @@ def test_coverage_anchored_and_grouped_by_subsystem(self): module = page( "module", "Auth.", extra_fm="subsystem: auth\nsource_path: libs/auth\nlanguage: go\n" - "status: active\ncriticality: load-bearing\ndepends_on: []\n" + "module_status: active\ncriticality: load-bearing\ndepends_on: []\n" f"last_verified_commit: abc\nlast_verified: {TODAY}\n" "verification_method: full\n") root = make_wiki(str(wiki), variant="sharded-repo", @@ -360,8 +370,8 @@ def test_broken_link_flagged_with_file_accurate_line(self): }) hits = findings(gather(root), "link", "ERROR") self.assertEqual(len(hits), 1) - # frontmatter is 8 lines + 1 blank: the body's first line is file line 10 - self.assertTrue(hits[0][2].endswith(":10"), hits[0][2]) + # frontmatter is 9 lines + 1 blank: the body's first line is file line 11 + self.assertTrue(hits[0][2].endswith(":11"), hits[0][2]) def test_bundle_absolute_links_resolve_against_root(self): root = make_wiki(self.tmp, files={ @@ -492,7 +502,7 @@ def test_secret_allowlist_suppresses_a_real_match(self): use_variant_with("wiki", secret_allow_res=[r"\$\{[^}]*\}"]) hits = findings(gather(root), "secrets", "ERROR") self.assertEqual(len(hits), 1) - self.assertTrue(hits[0][2].endswith(":11"), hits[0][2]) # the plain line + self.assertTrue(hits[0][2].endswith(":12"), hits[0][2]) # the plain line def test_iso_date_fields_configurable(self): root = make_wiki(self.tmp, files={ @@ -544,14 +554,14 @@ def test_check_log_reports_the_configured_path(self): self.assertTrue(hits) self.assertTrue(all(h[2].startswith("ops-journal.md:") for h in hits), hits) - def test_created_updated_ordering_independent_of_iso_date_fields(self): + def test_created_generated_ordering_independent_of_iso_date_fields(self): root = make_wiki(self.tmp, files={ "concepts/a-note.md": page("concept", "A.", created="2026-05-01", updated="2026-04-01"), }) use_variant_with("wiki", iso_date_fields=["date"]) # excludes the pair msgs = [m for _, _, _, m in findings(gather(root), "frontmatter", "ERROR")] - self.assertIn("updated is older than created", msgs) + self.assertIn("generated.at is older than created", msgs) def test_index_file_none_disables_index_without_crashing(self): root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) @@ -594,6 +604,15 @@ def test_index_body_fn_leading_newline_no_perpetual_drift(self): self._rebuild(root) self.assertEqual(findings(gather(root), "index"), []) + def test_membership_rule_missing_keys_raises_config_error(self): + """Review regression: a rule predating status_field must fail at + startup with a clear message, not KeyError mid-run.""" + from wikilint.settings import ConfigError + rule = {"member_type": "component", "active_statuses": ["active"], + "container_type": "topology", "container_field": "includes"} + with self.assertRaisesRegex(ConfigError, "status_field"): + use_variant_with("infra", membership=rule) + def test_bad_secret_regex_raises_config_error(self): from wikilint.settings import ConfigError make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) @@ -834,7 +853,7 @@ def test_index_requires_okf_version(self): self.assertEqual(findings(gather(root), "okf", "ERROR"), []) text = (root / "index.md").read_text() self.assertTrue(text.startswith('---\nokf_version: "0.2"\n---\n'), text[:60]) - self.assertIn("* [A Note](/concepts/a-note.md) - A.", text) + self.assertIn("* [Test Page](/concepts/a-note.md) - A.", text) def test_rebuild_index_does_not_accumulate_frontmatter(self): root = make_wiki(self.tmp, files={ @@ -858,7 +877,7 @@ class TestSourcesShape(WikiTest): bundle paths use the bundle-absolute form. The pre-0.2 plain-string shape still resolves but warns.""" - SRC = "---\ntype: source\ncreated: 2026-07-01\nupdated: 2026-07-01\n" \ + SRC = "---\ntype: source\ntitle: T\ncreated: 2026-07-01\ngenerated: { by: claude-code/test, at: 2026-07-01T00:00:00Z }\n" \ "description: A source.\ntags: [alpha]\nsources: []\n" \ "source_path: raw/articles/foo.md\n---\n\nBody.\n" @@ -866,7 +885,7 @@ def _wiki(self, citing_fm): return make_wiki(self.tmp, files={ "sources/foo-article.md": self.SRC, "concepts/cites.md": ( - "---\ntype: concept\ncreated: 2026-07-01\nupdated: 2026-07-01\n" + "---\ntype: concept\ntitle: T\ncreated: 2026-07-01\ngenerated: { by: claude-code/test, at: 2026-07-01T00:00:00Z }\n" f"description: Cites foo.\ntags: [alpha]\n{citing_fm}---\n\n" "Body cites [foo](/sources/foo-article.md).\n" ), diff --git a/tests/test_okf_fields.py b/tests/test_okf_fields.py new file mode 100644 index 0000000..99fd7c5 --- /dev/null +++ b/tests/test_okf_fields.py @@ -0,0 +1,203 @@ +"""OKF v0.2 page-level frontmatter the template adopts: `title`, +`generated { by, at }` (SPEC §5.2, offset datetimes per §5), and +`status` (§5.4), plus the parser shapes those keys need.""" + +import tempfile +import unittest +from datetime import date, timedelta + +from helpers import ( + DAYS_AGO_41, TODAY, findings, gather, make_wiki, page, use_variant_with, +) +from wikilint.model import parse_frontmatter, parse_okf_datetime + + +def messages(report, check=None, severity=None): + return [i[3] for i in findings(report, check, severity)] + + +class OkfFieldTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + + +class TestParserMappings(unittest.TestCase): + def test_flow_mapping_parses_to_dict(self): + fields, err = parse_frontmatter( + "---\ntype: concept\n" + "generated: { by: claude-code/opus, at: 2026-10-08T14:00:00Z }\n---\n") + self.assertIsNone(err) + self.assertEqual(fields["generated"], + {"by": "claude-code/opus", "at": "2026-10-08T14:00:00Z"}) + + def test_block_mapping_parses_to_dict(self): + fields, err = parse_frontmatter( + "---\ntype: concept\ngenerated:\n by: human:jack\n" + " at: 2026-10-08T14:00:00+05:30\ntags: [a]\n---\n") + self.assertIsNone(err) + self.assertEqual(fields["generated"], + {"by": "human:jack", "at": "2026-10-08T14:00:00+05:30"}) + self.assertEqual(fields["tags"], ["a"]) + + def test_empty_key_still_parses_as_list(self): + fields, _ = parse_frontmatter( + "---\nsources:\n - resource: /sources/a.md\nsupersedes:\n---\n") + self.assertEqual(fields["sources"], [{"resource": "/sources/a.md"}]) + self.assertEqual(fields["supersedes"], []) + + def test_braced_scalar_stays_a_string(self): + """Review regression: `{todo}`-style placeholders parsed as plain + strings before mappings existed and must keep doing so.""" + fields, err = parse_frontmatter( + "---\ntype: concept\ntitle: {draft}\ndescription: {a, b}\n---\n") + self.assertIsNone(err) + self.assertEqual(fields["title"], "{draft}") + self.assertEqual(fields["description"], "{a, b}") + + +class TestOkfDatetime(unittest.TestCase): + def test_offset_forms_accepted(self): + self.assertIsNotNone(parse_okf_datetime("2026-10-08T14:00:00Z")) + self.assertIsNotNone(parse_okf_datetime("2026-10-08T14:00:00+05:30")) + + def test_forms_python_39_fromisoformat_rejects(self): + """Review regression: 3.9's fromisoformat refuses these RFC 3339 + forms that 3.11+ accepts; results must not depend on the Python.""" + for value in ("2026-10-08T14:00:00.5Z", "2026-10-08T14:00:00.123456789Z", + "2026-10-08T14:00:00+0000", "2026-10-08T14:00:00.25-0530"): + self.assertIsNotNone(parse_okf_datetime(value), value) + + def test_date_only_and_naive_rejected(self): + self.assertIsNone(parse_okf_datetime("2026-10-08")) + self.assertIsNone(parse_okf_datetime("2026-10-08T14:00:00")) + self.assertIsNone(parse_okf_datetime("soon")) + self.assertIsNone(parse_okf_datetime(None)) + + +class TestTemplateRequiresOkfKeys(OkfFieldTest): + def test_helper_page_passes(self): + root = make_wiki(self.tmp.name, files={ + "concepts/a.md": page("concept", "A page.")}) + self.assertEqual(findings(gather(root), "frontmatter"), []) + self.assertEqual(findings(gather(root), "okf"), []) + + def test_title_and_generated_at_required(self): + bare = ("---\ntype: concept\ncreated: 2026-07-01\n" + "description: d\ntags: [alpha]\nsources: []\n---\n\nBody.\n") + root = make_wiki(self.tmp.name, files={"concepts/a.md": bare}) + msgs = messages(gather(root), "frontmatter") + self.assertIn("missing or empty required field: title", msgs) + self.assertIn("missing or empty required field: generated.at", msgs) + + +class TestGeneratedShape(OkfFieldTest): + def lint_generated(self, line): + root = make_wiki(self.tmp.name, files={ + "concepts/a.md": page("concept", "A page.").replace( + page("concept", "A page.").split("\n")[4], line)}) + return messages(gather(root), "okf") + + def test_date_only_at_rejected(self): + msgs = self.lint_generated("generated: { by: claude-code/opus, at: 2026-07-01 }") + self.assertTrue(any("explicit offset" in m for m in msgs), msgs) + + def test_missing_by_rejected(self): + msgs = self.lint_generated("generated: { at: 2026-07-01T00:00:00Z }") + self.assertTrue(any("generated.by" in m for m in msgs), msgs) + + def test_scalar_generated_rejected(self): + msgs = self.lint_generated("generated: 2026-07-01T00:00:00Z") + self.assertTrue(any("mapping" in m for m in msgs), msgs) + + def test_generated_before_created_rejected(self): + root = make_wiki(self.tmp.name, files={"concepts/a.md": page( + "concept", "A page.", created="2026-07-02", updated="2026-07-01")}) + self.assertIn("generated.at is older than created", + messages(gather(root), "frontmatter")) + + +class TestStatus(OkfFieldTest): + def test_non_okf_status_value_rejected(self): + root = make_wiki(self.tmp.name, files={ + "concepts/a.md": page("concept", "A page.", extra_fm="status: active\n")}) + msgs = messages(gather(root), "okf") + self.assertTrue(any("status" in m and "§5.4" in m for m in msgs), msgs) + + def test_okf_status_values_pass(self): + files = {f"concepts/{s}.md": page("concept", "A page.", extra_fm=f"status: {s}\n") + for s in ("draft", "stable", "deprecated")} + root = make_wiki(self.tmp.name, files=files) + self.assertEqual(findings(gather(root), "okf"), []) + + def test_status_check_follows_okf_gate(self): + root = make_wiki(self.tmp.name, files={ + "concepts/a.md": page("concept", "A page.", extra_fm="status: active\n")}) + use_variant_with("wiki", okf_conformance=False) + self.assertEqual(findings(gather(root), "okf"), []) + + def test_superseded_page_must_be_deprecated(self): + files = { + "concepts/old.md": page("concept", "Old."), + "concepts/new.md": page("concept", "New.", + extra_fm="supersedes: [/concepts/old.md]\n"), + } + root = make_wiki(self.tmp.name, files=files) + msgs = messages(gather(root), "supersedes", "ERROR") + self.assertEqual(len(msgs), 1, msgs) + self.assertIn("status: deprecated", msgs[0]) + + def test_deprecated_superseded_page_passes(self): + files = { + "concepts/old.md": page("concept", "Old.", extra_fm="status: deprecated\n"), + "concepts/new.md": page("concept", "New.", + extra_fm="supersedes: [/concepts/old.md]\n"), + } + root = make_wiki(self.tmp.name, files=files) + self.assertEqual(findings(gather(root), "supersedes"), []) + + +class TestLastModified(OkfFieldTest): + def test_contested_age_reads_generated_at(self): + root = make_wiki(self.tmp.name, files={"concepts/a.md": page( + "concept", "A page.", extra_fm="confidence: contested\n", + created=DAYS_AGO_41, updated=DAYS_AGO_41)}) + self.assertEqual(len(findings(gather(root), "contested")), 1) + + def test_contested_age_falls_back_to_legacy_updated(self): + """Non-OKF trees configured with a legacy `updated` key keep their + contested-age check (OKF §13.1 likewise falls back to `timestamp`).""" + legacy = (f"---\ntype: concept\ncreated: {DAYS_AGO_41}\nupdated: {DAYS_AGO_41}\n" + "description: d\ntags: [alpha]\nsources: []\n" + "confidence: contested\n---\n\nBody.\n") + root = make_wiki(self.tmp.name, files={"concepts/a.md": legacy}) + use_variant_with("wiki", required_fields=["type", "created", "updated"]) + self.assertEqual(len(findings(gather(root), "contested")), 1) + + def test_recent_generated_at_not_flagged(self): + root = make_wiki(self.tmp.name, files={"concepts/a.md": page( + "concept", "A page.", extra_fm="confidence: contested\n")}) + self.assertEqual(findings(gather(root), "contested"), []) + + +class TestIndexUsesOkfKeys(OkfFieldTest): + def test_index_entry_shows_title_and_generated_date(self): + from wikilint.derived import rebuild_index + root = make_wiki(self.tmp.name, files={"concepts/zt.md": page( + "concept", "A page.", updated="2026-09-30")}) + rebuild_index(root) + index = (root / "index.md").read_text() + self.assertIn("[Test Page](/concepts/zt.md)", index) + self.assertIn("(updated 2026-09-30)", index) + + def test_untitled_page_falls_back_to_prettified_stem(self): + from wikilint.derived import rebuild_index + untitled = page("concept", "A page.").replace("title: Test Page\n", "") + root = make_wiki(self.tmp.name, files={"concepts/a-note.md": untitled}) + use_variant_with("wiki", required_fields=["type", "created"]) + rebuild_index(root) + self.assertIn("[A Note](/concepts/a-note.md)", (root / "index.md").read_text()) + + +if __name__ == "__main__": + unittest.main() diff --git a/wiki/CLAUDE.md b/wiki/CLAUDE.md index 83be57a..3a530cc 100644 --- a/wiki/CLAUDE.md +++ b/wiki/CLAUDE.md @@ -50,25 +50,29 @@ Every wiki page (everything outside `raw/`) starts with YAML frontmatter: ```yaml --- type: source | entity | concept | synthesis | query +title: Zero Trust Networking created: 2026-04-10 -updated: 2026-04-10 +generated: { by: claude-code/, at: 2026-04-10T14:00:00Z } description: One line saying what this page covers, used for scanning. tags: [tag1, tag2] sources: - resource: /sources/foo.md supersedes: [] +status: deprecated # optional; absent means stable confidence: low | contested # optional; absent means normal --- ``` Rules: -- `created` is set once and never changed. `updated` changes on every edit. +- `title` is the page's human-readable name, shown in the index. +- `created` is a date, set once and never changed. `generated` records the last meaningful edit and changes on every edit: `by` is the actor (`claude-code/` for you, `human:` when the human wrote the change), `at` is a UTC datetime from `date -u +%Y-%m-%dT%H:%M:%SZ`. Date-only `at` values fail lint. +- `status` follows OKF: absent means stable, `draft` marks an incomplete page, and `deprecated` marks a page kept for history. A page another page `supersedes` must carry `status: deprecated` (lint-enforced). - `description` is one sentence. It is how you and the index find this page without opening it. Keep it accurate on every edit. -- `sources` lists every source page that supports claims on this page; each entry is a mapping whose `resource` is the bundle-absolute page path (OKF v0.2 shape). External URLs are allowed as `resource` for material with no source page. +- `sources` lists every source page that supports claims on this page; each entry is a mapping whose `resource` is the bundle-absolute page path (OKF v0.2 shape). External URLs are allowed as `resource` for material with no source page. OKF's optional per-entry keys (`id`, `title`, `author`) may be added; `id` is worth adding when the body cites the source by name. - `confidence` is absent on normal pages. `low` means claims lack citations. `contested` means two or more sources disagree, and the page body must explain the disagreement. Contested is a state to exit, not a resting place: reconcile it. - Every tag must appear in `taxonomy.md`. Introducing a tag means adding it there, with a one-line meaning, in the same commit. The allowed page types are described there too, under '## Page types'. - Mark claims you inferred rather than read with `(inferred)` inline; a page containing any carries `confidence: low`. -- `supersedes` lists older pages whose claims this page replaces. The old page stays but gets a banner pointing forward. +- `supersedes` lists older pages whose claims this page replaces. The old page stays, gets `status: deprecated`, and gets a banner pointing forward. - Use markdown links with bundle-absolute targets for all internal references: `[Zero Trust](/concepts/zero-trust-networking.md)`. Never use bare, unlinked paths in prose. - File names are kebab-case, lowercase, descriptive. `zero-trust-networking.md`, not `ZTN.md`. - One concept per page. If a page is becoming two things, split it and ask for confirmation. diff --git a/wiki/lint.py b/wiki/lint.py index 12438e7..7b32128 100644 --- a/wiki/lint.py +++ b/wiki/lint.py @@ -32,8 +32,9 @@ "reverse_fields": [], # Active members that must appear in a container page (None disables). "membership": None, - # Frontmatter required on every page. - "required_fields": ["type", "created", "updated", "description", "tags"], + # Frontmatter required on every page. A dotted name reads into a + # mapping: generated.at is OKF's last-meaningful-change datetime. + "required_fields": ["type", "title", "created", "generated.at", "description", "tags"], # type -> extra required fields for that type. "type_required": { "source": [], @@ -51,6 +52,8 @@ "type_enum_fields": {}, # Frontmatter fields whose values are wiki paths that must exist. "path_fields": ["sources", "supersedes"], + # Pages listed in this field must carry OKF `status: deprecated`. + "supersedes_field": "supersedes", # Fields that define dependency edges (stored once, on the depender). # The reverse map is derived, never stored. "edge_fields": [], @@ -91,9 +94,10 @@ def index_entry_extra(fields): - """Trailing annotation for an index entry, by page type.""" - updated = fields.get("updated", "?") - return f"(updated {updated})" + """Trailing annotation for an index entry: the date of the last edit.""" + generated = fields.get("generated") + at = generated.get("at", "") if isinstance(generated, dict) else "" + return f"(updated {at[:10] or '?'})" if __name__ == "__main__": diff --git a/wiki/wikilint/checks.py b/wiki/wikilint/checks.py index bc9884d..7061f69 100644 --- a/wiki/wikilint/checks.py +++ b/wiki/wikilint/checks.py @@ -2,14 +2,15 @@ builder in derived.py).""" import re -from datetime import date, datetime +from datetime import date, datetime, timezone from urllib.parse import unquote from .model import ( ADR_FILE_RE, FILENAME_EXEMPT, KEBAB_RE, LOG_DATE_HEADER_RE, LOG_ENTRY_RE, MD_LINK_RE, OKF_VERSION, SECRET_PATTERNS, blank_frontmatter, blank_images, - blank_inline_code, build_link_index, discover_okf_bundle, line_at, - parse_frontmatter, parse_index_pinning, parse_iso_date, resolve_link, + OKF_STATUSES, blank_inline_code, build_link_index, discover_okf_bundle, + field_value, last_modified, line_at, parse_frontmatter, parse_index_pinning, + parse_iso_date, parse_okf_datetime, resolve_link, ) from .settings import CONFIG @@ -33,7 +34,7 @@ def check_frontmatter(pages, report): report.error("frontmatter", p.rel, f"invalid or missing type: {ptype!r}") continue for field in CONFIG["required_fields"] + CONFIG["type_required"][ptype]: - if missing_value(p.fields.get(field)): + if missing_value(field_value(p.fields, field)): report.error("frontmatter", p.rel, f"missing or empty required field: {field}") check_enums(p, ptype, report) check_dates(p, report) @@ -63,6 +64,9 @@ def check_dates(p, report): updated = parse_iso_date(p.fields.get("updated")) if created and updated and updated < created: report.error("frontmatter", p.rel, "updated is older than created") + generated_at = parse_okf_datetime(field_value(p.fields, "generated.at")) + if created and generated_at and generated_at.date() < created: + report.error("frontmatter", p.rel, "generated.at is older than created") def check_filenames(pages, report): @@ -198,7 +202,7 @@ def check_membership(pages, report): for p in pages: if p.type != rule["member_type"] or not p.fields: continue - if p.fields.get("status") not in rule["active_statuses"]: + if p.fields.get(rule["status_field"]) not in rule["active_statuses"]: continue if p.rel not in contained: report.warning( @@ -364,14 +368,14 @@ def check_skills(pages, report, root): def check_contested_age(pages, report): - today = date.today() + now = datetime.now(timezone.utc) for p in pages: if (p.fields or {}).get("confidence") != "contested": continue - updated = parse_iso_date(p.fields.get("updated")) - if updated is None: + changed = last_modified(p.fields) + if changed is None: continue - age = (today - updated).days + age = (now - changed).days if age > CONFIG["contested_max_days"]: report.warning( "contested", p.rel, @@ -555,8 +559,10 @@ def check_adr_dir(dir_path, pages, report, root): report.warning("adr", p.rel, "not in NNNN-slug.md form") continue numbers[int(m.group(1))] = p - status = (p.fields or {}).get("status") - if status == "superseded" and not (p.fields or {}).get("superseded_by"): + # adr_status since OKF claimed `status`; legacy ADR trees keep theirs. + fields = p.fields or {} + status = fields.get("adr_status", fields.get("status")) + if status == "superseded" and not fields.get("superseded_by"): report.error("adr", p.rel, "superseded ADR has no superseded_by forward link") if numbers: expected = set(range(min(numbers), max(numbers) + 1)) diff --git a/wiki/wikilint/cli.py b/wiki/wikilint/cli.py index 7e90986..273313c 100644 --- a/wiki/wikilint/cli.py +++ b/wiki/wikilint/cli.py @@ -8,7 +8,7 @@ import sys from pathlib import Path -from . import checks +from . import checks, okf_fields from .derived import check_index_drift, rebuild_index, run_coverage, run_reverse_deps from .model import Report, discover_pages from .settings import BUILTIN_COMMANDS, CONFIG, ConfigError, configure @@ -54,6 +54,8 @@ def gather_report(root): checks.check_log(root, report) checks.check_sources(pages, report) checks.check_okf(pages, report, root) + okf_fields.check_okf_fields(pages, report) + okf_fields.check_supersedes(pages, report) check_index_drift(pages, report, root) for extra in CONFIG["extra_checks"]: # A third-party check must not abort the run: a raise here would diff --git a/wiki/wikilint/model.py b/wiki/wikilint/model.py index 4c2cbae..5b2516e 100644 --- a/wiki/wikilint/model.py +++ b/wiki/wikilint/model.py @@ -1,7 +1,7 @@ """Page model, parsers, and shared helpers.""" import re -from datetime import date +from datetime import date, datetime, timezone from fnmatch import fnmatch from pathlib import Path, PurePosixPath @@ -11,6 +11,8 @@ # OKF (Open Knowledge Format) version this engine produces and validates. OKF_VERSION = "0.2" +# OKF v0.2 §5.4 lifecycle values; absent means stable. +OKF_STATUSES = ("draft", "stable", "deprecated") # Pages allowed to break kebab-case and to have no inbound links. FILENAME_EXEMPT = ("README.md", "OWNERS.md") @@ -47,6 +49,10 @@ # required whitespace after the colon keeps bare URLs (scheme:// has none) # parsing as plain strings. LIST_MAP_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_-]*):\s+(\S.*)$") +# Datetime pieces Python 3.9's fromisoformat rejects: a fraction that is not +# exactly 3 or 6 digits, and a colon-less +HHMM offset. +ISO_FRACTION_RE = re.compile(r"\.(\d+)(?=[+-]\d|$)") +ISO_COMPACT_OFFSET_RE = re.compile(r"([+-])(\d{2})(\d{2})$") class Report: @@ -101,12 +107,29 @@ def as_list(value): return value +def parse_flow_mapping(raw_val): + """One-level `{ k: v, k2: v2 }` flow mapping (the OKF `generated` + shape). Values may not contain commas; returns None when any part is + not a `k: v` pair.""" + inner = raw_val[1:-1].strip() + mapping = {} + for part in (inner.split(",") if inner else []): + m = LIST_MAP_RE.match(part.strip()) + if not m: + return None + mapping[m.group(1)] = unquote(m.group(2)) + return mapping + + def parse_frontmatter(text): - """Minimal YAML frontmatter parser: flat keys, inline and block lists. + """Minimal YAML frontmatter parser: flat keys, inline and block lists, + and one-level mappings. Block-list items may be one-level mappings ("- resource: /x.md" plus - same-item continuation lines), the OKF v0.2 sources shape. Deeper - nesting is not supported by design; the page schemas stay flat. + same-item continuation lines), the OKF v0.2 sources shape. A key may also + hold a one-level mapping, inline (`generated: { by: x, at: y }`) or as + indented `k: v` lines under a bare key. Deeper nesting is not supported + by design; the page schemas stay shallow. Returns (fields, error). """ @@ -143,9 +166,22 @@ def parse_frontmatter(text): [unquote(v) for v in inner.split(",") if v.strip()] if inner else [] ) + elif raw_val.startswith("{") and raw_val.endswith("}"): + # Not k: v pairs (a `{todo}` placeholder): keep the scalar. + mapping = parse_flow_mapping(raw_val) + fields[key] = mapping if mapping else unquote(raw_val) else: fields[key] = unquote(raw_val) - elif stripped.startswith("- ") and current_list_key is not None: + elif (current_list_key is not None and not stripped.startswith("- ") + and (fields[current_list_key] == [] + or isinstance(fields[current_list_key], dict)) + and LIST_MAP_RE.match(stripped)): + # Indented `k: v` under a bare key: a block mapping, not a list. + m = LIST_MAP_RE.match(stripped) + fields[current_list_key] = { + **dict(fields[current_list_key]), m.group(1): unquote(m.group(2))} + elif (stripped.startswith("- ") and current_list_key is not None + and isinstance(fields[current_list_key], list)): item_text = stripped[2:] m = LIST_MAP_RE.match(item_text) if m: @@ -168,6 +204,51 @@ def parse_iso_date(value): return None +def parse_okf_datetime(value): + """An OKF timestamp (SPEC §5): ISO 8601 datetime with an explicit UTC + offset. Date-only and offset-less values return None, as the spec tells + consumers to ignore them. Z, fractions of any length and +HHMM offsets + are normalized first: datetime.fromisoformat accepts them only from + Python 3.11, and results must not depend on the interpreter.""" + if not isinstance(value, str) or "T" not in value: + return None + text = value.strip() + if text[-1:] in ("Z", "z"): + text = text[:-1] + "+00:00" + text = ISO_FRACTION_RE.sub(lambda m: "." + m.group(1)[:6].ljust(6, "0"), text) + text = ISO_COMPACT_OFFSET_RE.sub(r"\1\2:\3", text) + try: + parsed = datetime.fromisoformat(text) + except ValueError: + return None + return parsed if parsed.tzinfo is not None else None + + +def last_modified(fields): + """When a page's content last changed, as an aware datetime: OKF's + `generated.at`, falling back to a legacy `updated` date at midnight UTC + (the way SPEC §13.1 falls back to v0.1's `timestamp`). None if neither.""" + generated = (fields or {}).get("generated") + if isinstance(generated, dict): + at = parse_okf_datetime(generated.get("at")) + if at is not None: + return at + updated = parse_iso_date((fields or {}).get("updated")) + if updated is None: + return None + return datetime(updated.year, updated.month, updated.day, tzinfo=timezone.utc) + + +def field_value(fields, name): + """A frontmatter value by name; a dotted name (`generated.at`) reads one + level into a mapping. None when any step is absent.""" + head, _, rest = name.partition(".") + value = (fields or {}).get(head) + if not rest: + return value + return value.get(rest) if isinstance(value, dict) else None + + def strip_code_blocks(body): """Blank fenced code blocks (preserving line count) so wikilinks and mention hints inside examples don't count but line numbers stay right.""" diff --git a/wiki/wikilint/okf_fields.py b/wiki/wikilint/okf_fields.py new file mode 100644 index 0000000..4463010 --- /dev/null +++ b/wiki/wikilint/okf_fields.py @@ -0,0 +1,75 @@ +"""OKF v0.2 page-level trust and lifecycle keys: the shape of `generated` +(SPEC §5.2, timestamps per §5) and the `status` value set (§5.4), plus the +supersedes -> deprecated rule tying our edge field to OKF's lifecycle.""" + +from .model import ( + OKF_STATUSES, build_link_index, parse_okf_datetime, resolve_link, +) +from .settings import CONFIG + +DATETIME_EXAMPLE = "2026-10-08T14:00:00Z" + + +def check_okf_fields(pages, report): + """Validate OKF-defined keys wherever a page carries them. Presence is + the schema's business (required_fields); this checks only that a key OKF + defines means what OKF says, so foreign consumers read it correctly.""" + if not CONFIG["okf_conformance"]: + return + for p in pages: + if not p.fields: + continue + if "generated" in p.fields: + check_generated(p, report) + status = p.fields.get("status") + if status is not None and status not in OKF_STATUSES: + report.error( + "okf", p.rel, + f"status: {status!r} is not an OKF lifecycle value " + f"{list(OKF_STATUSES)} (OKF v0.2 §5.4)", + ) + + +def check_generated(p, report): + generated = p.fields["generated"] + if not isinstance(generated, dict): + report.error( + "okf", p.rel, + "generated must be a mapping, e.g. " + f"{{ by: claude-code/, at: {DATETIME_EXAMPLE} }} (OKF v0.2 §5.2)", + ) + return + if not generated.get("by"): + report.error( + "okf", p.rel, + "generated.by is required within generated: an actor such as " + "claude-code/ or human: (OKF v0.2 §5.2, §7)", + ) + at = generated.get("at") + if at and parse_okf_datetime(at) is None: + report.error( + "okf", p.rel, + f"generated.at {at!r} must be an ISO 8601 datetime with an explicit " + f"offset, e.g. {DATETIME_EXAMPLE} (OKF v0.2 §5)", + ) + + +def check_supersedes(pages, report): + """A page another page supersedes is OKF's `deprecated`: kept for links + and history, no longer current. Requiring the status makes the forward + banner machine-readable to any OKF consumer.""" + field = CONFIG["supersedes_field"] + if not field: + return + by_stem, by_rel = build_link_index(pages) + for p in pages: + for value in p.path_list(field): + target = resolve_link(value, by_stem, by_rel) + if target is None or target.rel == p.rel: + continue # dangling references are check_links_and_orphans' job + if (target.fields or {}).get("status") != "deprecated": + report.error( + "supersedes", target.rel, + f"superseded by {p.rel}; set status: deprecated " + "(OKF v0.2 §5.4)", + ) diff --git a/wiki/wikilint/settings.py b/wiki/wikilint/settings.py index ed4cfe2..81fa294 100644 --- a/wiki/wikilint/settings.py +++ b/wiki/wikilint/settings.py @@ -39,8 +39,13 @@ "edge_fields": [], # Fields `reverse-deps` inverts into a reverse-dependency map. "reverse_fields": [], - # Container-page membership rule; None disables check_membership. + # Container-page membership rule; None disables check_membership. A rule + # is {member_type, status_field, active_statuses, container_type, + # container_field}. "membership": None, + # Edge field whose targets must carry OKF `status: deprecated` (SPEC + # §5.4), e.g. "supersedes"; None disables check_supersedes. + "supersedes_field": None, # Per-field staleness rules: [{field, types, max_days, severity}]. "staleness": [], # Days a `confidence: contested` page may sit untouched. @@ -176,6 +181,25 @@ def _validate_commands(commands): raise ConfigError(f"extra_commands[{verb!r}] needs a non-empty help string") +MEMBERSHIP_KEYS = ("member_type", "status_field", "active_statuses", + "container_type", "container_field") + + +def _validate_membership(rule): + """A membership rule must name every key check_membership reads, so an + older rule (status_field arrived when OKF claimed `status`) fails here + with the fix rather than as a KeyError mid-run.""" + if rule is None: + return + if not isinstance(rule, dict): + raise ConfigError("membership must be None or a dict") + missing = [k for k in MEMBERSHIP_KEYS if k not in rule] + if missing: + raise ConfigError( + f"membership rule missing {missing}; status_field names the member " + "lifecycle field (not `status`, which OKF v0.2 §5.4 defines)") + + def _validate(cfg): """Validate user-supplied extension values at the config boundary and precompile the secret regexes so a bad pattern fails here, once, with a @@ -190,6 +214,7 @@ def _validate(cfg): f"index_file must stay within the wiki root: {index_file!r}") if cfg["log_file"] is not None and not isinstance(cfg["log_file"], str): raise ConfigError("log_file must be None or a relative path string") + _validate_membership(cfg["membership"]) if not isinstance(cfg["types_glossary"], bool): raise ConfigError("types_glossary must be a bool") skills_dir = cfg["skills_dir"] diff --git a/wiki/workflows/ingest.md b/wiki/workflows/ingest.md index 6d7feb1..1cf62a2 100644 --- a/wiki/workflows/ingest.md +++ b/wiki/workflows/ingest.md @@ -10,7 +10,7 @@ Triggered by "ingest `raw/articles/foo.md`" or similar. Default to one source at 2. **Discuss before writing.** Tell the human the three to five most important takeaways and ask what to emphasize. Wait for their response before touching the wiki. 3. **Create the source page** at `sources/.md`. Include: bibliographic info, a one-paragraph summary, key claims as a bulleted list, notable quotes (under 15 words each), and open questions raised. 4. **Identify affected pages.** Scan `index.md` and page frontmatter for entities, concepts, and synthesis pages this source touches, then ripgrep for anything missed. Open bodies only for the candidates. -5. **Update each affected page.** Integrate the new information, add the source to its frontmatter, bump `updated`, and refresh `description` if the page's scope shifted. If the source contradicts an existing claim, do not silently overwrite: set `confidence: contested` and structure the body latest-evidence-first, newest claim on top with its date, older claims below with theirs, so `workflows/reconcile.md` has clean input. +5. **Update each affected page.** Integrate the new information, add the source to its frontmatter, refresh `generated` (your actor and the current UTC datetime), and refresh `description` if the page's scope shifted. If the source contradicts an existing claim, do not silently overwrite: set `confidence: contested` and structure the body latest-evidence-first, newest claim on top with its date, older claims below with theirs, so `workflows/reconcile.md` has clean input. 6. **Create new pages where needed.** If the source introduces an entity or concept without a page, apply the page-worthiness test below and create the page only if all four parts hold. 7. **Rebuild the index and lint.** Run `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix any errors it reports. 8. **Append to `log.md`** using the format below, then commit: `ingest: `. diff --git a/wiki/workflows/reconcile.md b/wiki/workflows/reconcile.md index e912263..027df1f 100644 --- a/wiki/workflows/reconcile.md +++ b/wiki/workflows/reconcile.md @@ -9,7 +9,7 @@ Triggered by "reconcile " or by lint flagging a long-unreconciled conteste 1. **Gather the conflict.** Read the page and every source it cites. List each competing claim with its source and the source's date. 2. **Propose a resolution.** Rank claims by source recency first, then source authority (primary sources over secondary, the human's direct word over both). State which claim should win and why, show the proposed rewrite, and wait for approval. 3. **Rewrite in place.** The winning claim goes in the main body, stated plainly. Each losing claim moves to a "Superseded claims" section at the bottom: one dated line per claim with its source and a one-line rationale for why it lost. Never silently delete the loser. -4. **Update frontmatter.** Remove `confidence: contested` (downgrade to `low` instead if the winner is itself uncited). Bump `updated` and refresh `description` if the page's conclusion changed. +4. **Update frontmatter.** Remove `confidence: contested` (downgrade to `low` instead if the winner is itself uncited). Refresh `generated` (your actor and the current UTC datetime) and `description` if the page's conclusion changed. 5. **Check the neighbors.** Follow the page's links one hop out. Any page repeating the losing claim gets the same treatment in this session. 6. **Lint, log, commit.** Run `python3 lint.py check`, append to `log.md`, commit: `reconcile: `.