Skip to content

docs: organize Dokimi features into an illustrated guide - #627

Merged
napetrov merged 45 commits into
intel:mainfrom
napetrov:docs/feature-cards-and-map
Nov 16, 2026
Merged

napetrov merged 45 commits into
intel:mainfrom
napetrov:docs/feature-cards-and-map

Conversation

@napetrov

@napetrov napetrov commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Make Dokimi’s capabilities easier to discover and understand: start with a concise README, explore a linked feature tree, and open individual feature cards for details.

What changes

  • Add 39 illustrated feature cards across eight topic guides.
  • Give each feature a description, implementation status, limitations, and links to detailed documentation.
  • Add a feature tree connecting the overview to individual cards.
  • Shorten the root README and update the package README, documentation home, and MkDocs navigation.
  • Correct outdated descriptions of repository evaluation, task generation, coverage, and adherence.
  • Extend documentation link checks to cover nested feature pages and the root README.

Feature availability

The guide distinguishes functionality available on main, partial implementations, open proposals, and research directions. Availability was reviewed against the code and open feature PRs on November 11, 2026.

“Available on main” does not imply inclusion in a published release. Illustrations explain behavior; they are not product screenshots or measured results.

Validation

  • Strict MkDocs build passed.
  • All 67 documentation checks passed.
  • Verified 39 feature anchors, 39 image assets, and all nine new feature pages.
  • Confirmed that a broken link in a nested feature page fails the link checks.

The full test run finished with 18,554 passed, 18 skipped, and 10 failed. All 10 failures also reproduce on the unchanged base in the same environment.

Full local CI remains non-green due to those failures and additional environment/tooling failures in package and smoke-test steps. Keeping this PR as a draft pending CI review.

No product implementation changes.

Add 48 addressable feature cards, eight guide groups and a linked feature tree. Refresh README and documentation entry points, distinguish working features from proposed extensions, and reconcile stale journey claims.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov
napetrov requested a review from a team as a code owner November 11, 2026 23:25
Copilot AI balanced review requested due to automatic review settings November 11, 2026 23:25
@napetrov napetrov changed the title Docs/feature cards and map docs: organize Dokimi features into an illustrated guide Nov 11, 2026
@exai-bot

exai-bot commented Nov 11, 2026 •

Copy link
Copy Markdown

OpenCodeReview: Commit 7118eec was not reviewed. This PR has reached its review limit (6 reviews).

CI at `7118eec`: 11 still running, 2 passing, 1 skipped
Check Status
Build docs (strict) still running
CodeQL (actions) still running
CodeQL (python) still running
Dependency review still running
External catalog smoke test (uxlfoundation/skills) still running
Platforms and Python versions still running
Static checks (ruff, schema, zizmor, licenses, pre-commit, bandit, secrets, changelog, schema com... still running
Windows smoke test (py3.13) still running
benchmark still running
mypy still running

1 more not listed.

To review this commit: re-request review from exai-bot in the Reviewers list.

<details><summary>Last review (f0cd431): no findings</summary>

Open threads from the bot: 0

</details>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Documentation still mislabels knowledge retention as task value and conflates answer-grading red-team checks with adversarial controls for executable-task verifiers.

2 open issues
What changed in this PR

2 new findings.

This PR adds a question-based capability map for Dokimi’s agent-context evaluation workflows, distinguishing implemented features from partial workflows and planned work.

Changes:

  • Adds 48 feature cards across eight topics, with illustrations, availability boundaries and guide links.
  • Updates README entry points, measurement guidance and repository-evaluation status.
  • Adds feature navigation and expands link checking to nested documentation and both READMEs.
File Description
README.md Introduces question-based navigation and free starting checks.
dokimi/​tests/​test_docs_links.py Checks nested documentation and both READMEs.
dokimi/​README.md Links the capability map and updates implementation status.
dokimi/​mkdocs.yml Adds feature pages to site navigation.
dokimi/​docs/​what-it-measures.md Links the map and clarifies novelty interpretation.
dokimi/​docs/​user-journeys/​journeys.md Updates task-generation, coverage and adherence boundaries.
dokimi/​docs/​repo-evaluation-contract.md Identifies the implemented repository-evaluation core.
dokimi/​docs/​README.md Streamlines the documentation home around capabilities and evidence.
dokimi/​docs/​features/​traces.md Documents context inspection and execution diagnostics.
dokimi/​docs/​features/​skills.md Covers skill analysis, shortening and validation.
dokimi/​docs/​features/​repositories.md Covers repository guidance and evaluation workflows.
dokimi/​docs/​features/​README.md Indexes all 48 cards and defines availability labels.
dokimi/​docs/​features/​judging.md Explains grading methods and judge checks.
dokimi/​docs/​features/​experiments.md Covers controlled comparisons, harnesses and cost.
dokimi/​docs/​features/​eval-design.md Covers task creation, coverage and grader controls.
dokimi/​docs/​features/​discovery.md Covers skill selection and instruction loading.
dokimi/​docs/​features/​continuous-evaluation.md Covers reports, regression checks and development directions.

Review effort: Balanced

Comment thread dokimi/docs/features/eval-design.md Resolved
Comment thread dokimi/docs/features/skills.md Resolved
@codecov-commenter

codecov-commenter commented Nov 11, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.76%. Comparing base (47f89ed) to head (7118eec).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #627   +/-   ##
=======================================
  Coverage   95.76%   95.76%           
=======================================
  Files         416      416           
  Lines       55624    55624           
  Branches     9553     9553           
=======================================
  Hits        53267    53267           
  Misses       1456     1456           
  Partials      901      901           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add role-based routes, result interpretation, scoped feature cards and a downloadable PR documentation preview. Preserve existing overview links.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Comment thread dokimi/docs/quickstart.md Resolved
Document development directions, CLI migrations, reliability limits and all 109 open work items. Correct feature inputs, workflow links, shortening semantics, resource budgets and free versus paid steps after independent review.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
napetrov and others added 10 commits November 11, 2026 18:35
The card claimed adversarial controls for executable tasks, which no code
implements. It now lists null, oracle and isolation checks, and points
answer-only grading to eval redteam.

Thread: PRRT_kwDOVBg7g862ZmzX
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Thread: PRRT_kwDOVBg7g862aHgu
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Thread: PRRT_kwDOVBg7g862aHgu
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…es-only route

Thread: PRRT_kwDOVBg7g862aHgu
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
- 48 -> 41 cards: fold 03 into 02, 47 into 04, 08 into 07, 45 into 12,
  16 into 13, 22 into 21 and 17 into 35; old anchors kept as aliases
- new images for 05, 24, 32, 38 and 39, which showed another card's art
- Open PR / Research badges on 18, 29, 34, 41, 43 and 44
- fix line numbering in images 44 and 48

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wg6N1cwy55Kuq6gxjrgQm1
Signed-off-by: Claude <noreply@anthropic.com>
- fold 38 into 27 (documentation as input) and 39 into 40 (stored results
  and regression checks); 41 -> 39 cards, old anchors kept
- every card image's alt text now starts with the card heading

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wg6N1cwy55Kuq6gxjrgQm1
Signed-off-by: Claude <noreply@anthropic.com>
… claude/pr-627-review-fixes-bet7ht

Signed-off-by: Claude <noreply@anthropic.com>

# Conflicts:
#	dokimi/docs/features/eval-design.md
#	dokimi/docs/quickstart.md
…gn conflict

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wg6N1cwy55Kuq6gxjrgQm1
Signed-off-by: Claude <noreply@anthropic.com>
docs: address intel#627 review threads and self-review findings
napetrov added a commit to napetrov/dokimi that referenced this pull request Nov 12, 2026
docs: address intel#627 review threads and self-review findings
Feature cards now state what main does (ablation candidates, routing
scope, trace inputs, review and explore results, CI coverage); the start
guides name the flag spellings 0.2.0 still uses; contributor instructions
name the new index; card links point at the pages that cover each feature.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
@xaleryb

xaleryb commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Ran: the start guides followed on Linux with dokimi 0.2.0 from PyPI and with this branch; the cards checked against the code at main; mkdocs build --strict, the docs link tests and the docs check gate (passes, score 0.6256 before this commit, 0.6267 after).

Pushed f0cd431:

  • docs/quickstart.md, docs/start/README.md: the seed example's --out-dir fails on 0.2.0 (unrecognized arguments), and 0.2.0 is what pip install dokimi installs. Both pages now give the 0.2.0 spelling, and development/interfaces.md gains a row for the merged output-flag rename.
  • AGENTS.md, CONTRIBUTING.md, docs/features/README.md, docs/start/contributors.md: they told contributors to link a new page from docs/README.md. The index test now reads documentation-index.md, so a contributor who followed them turned test_the_index_links_every_doc_in_the_directory red.
  • docs/features/traces.md card 37 had the candidates reversed: status_of makes a rule the sessions never brought up the candidate, and a rule they triggered or acted on in_use.
  • Wrong behaviour statements: traces cards 35/36 (input is stream-json output, and without --outcomes a pass only means the harness exited cleanly); discovery card 14 (novelty routing scores only should-select queries, lexically without --model; negatives and real selection are catalog trigger-probe); skills cards 10 (the routing axis is always 1.0), 11 (--ignore takes finding codes only) and 46 (per-atom relevance for one skill); eval-design cards 26 (file/line F1, no grader) and 30 (eval check needs a repo manifest and refuses Harbor directories); experiments card 04 (tasks run takes one --model); continuous-evaluation card 40 (the reusable workflow runs validate, scan and eval check only); repositories card 24 input; architecture.md (the contract is no longer marked PROPOSED); start/experiments.md (arms run and tasks run have no cap); development/limitations.md (on main atoms and probe keep capped work).
  • Commands that fail or mislead as written: quickstart step 6 (dashboard generate reads results/ by default and exits 2 after step 4; now --results-dir .); quickstart content setup (onetbb ships with the package and preflight from an empty directory fails only on the keys; --questions-root is named); start/README.md (linked guides still write python cli.py); start/repositories.md (the text output lists only fail claims; repo validate alone passes a manifest eval check refuses; --harness is named); start/skills.md (skill check for one skill); start/automation.md (the reusable repo-eval.yml workflow); features/README.md and docs/README.md (--version in a checkout prints the last release).
  • Wrong links: dokimi/README.md, the PyPI readme, called docs/README.md the full explanation and the index; card links on 02, 07, 18, 21, 25, 26, 31, 32 and 33 pointed at pages or modules that do not cover the feature (methodology.md has nothing on headroom, the leak gate or judge validation); --plugins, --no-shorten and the dokimi-design-evals skill had no prose page left; documentation-index.md cited an unrelated PR (fix(judge): read a qualified or negated figure verdict #67); skill-novelty.md still said three free steps where the walkthrough says two.
  • Statuses: discovery card 18 is Partial (catalog check already prints free merge/split hints); the flow map's J4 adherence edge now says what exists (declared process contracts).

Left for the author (images, not touched here):

  • 30 shows a verifier challenge (redteam_print_pass.py) that the card says is not implemented.
  • 31 shows a run_eval notebook that dokimi does not have.
  • Crop artifacts: 25, 26, 35, 43 and 44 show the edge of a neighbouring frame or lose their own.
  • 25 carries the GitHub mark and 27 an "Office" label; generic icons would avoid vendor marks.
  • The alt texts repeat the heading and subtitle instead of describing the scene.

DCO fails: 49f1f17 ("docs: address PR #627 review threads and self-review findings") has no Signed-off-by line, so the PR cannot merge until that commit is signed off.

#472, #482, #489 and #588 add pages linked from docs/README.md; after this PR the index is docs/documentation-index.md, so they need an entry there.

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at f0cd431. 3 new findings: 1 must-fix, 2 nits. In the line comments. Reorganizes the docs into an illustrated feature tree of 39 cards and shortens the docs home. The docs link tests pass on the PR and it merges without conflict, but the eight topic guides under docs/features/ are missing from the documentation index that the docs say must link them, and the docs home drops sections while keeping their anchors. The PR text says 48 cards and 48 images; the tree holds 39 of each.

Verified

  • Docs link tests: All 11 tests in dokimi/tests/test_docs_links.py pass on the PR head, including the relative-link and same-page anchor checks over the changed pages. The index test passes only because it covers top-level pages (see the first finding).
  • Merge: The PR merges onto main with no conflict.
  • Card count: The feature tree links 39 distinct cards (dokimi/docs/features/README.md), and dokimi/docs/assets/features/ holds 39 .webp images. dokimi/docs/README.md:32 and dokimi/docs/documentation-index.md:27 both say 39. The PR description's 48 cards and 48 images do not match the tree.

Not verified

  • The availability labels on the 39 cards ("available on main", partial, proposed) were not traced to code feature by feature; only the commands and flags the cards cite were looked at: Cite the file or test that shows each "available on main" label on its card, or mark the label as the author's reading.

Comment thread dokimi/docs/documentation-index.md Resolved
Comment thread dokimi/docs/README.md Resolved
Comment thread dokimi/docs/quickstart.md Resolved
@napetrov

Copy link
Copy Markdown
Contributor Author

@xaleryb two open questions before I change anything here:

  1. Card count: the PR text says 48 cards and 48 images, but the tree links and ships 39 of each (vchamarthi's review). Should I (a) correct the description to 39, or (b) add the 9 missing cards and images?
  2. Commit 49f1f17 has no Signed-off-by, so DCO blocks the PR. OK to re-sign it (history rewrite on this branch, lease push), or should the commit be squashed into a signed one?

The feature tree says a new topic page is linked from the index, but the
index test only looked at top-level pages, so the eight guides under
docs/features/ were missing unnoticed. The test now walks every subfolder.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
@xaleryb

xaleryb commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

@napetrov on your two questions:

  1. (a): correct the description to 39. The tree, docs/README.md:32 and documentation-index.md:27 already say 39, and the 9 merged cards were the duplicate-image pairs, so adding cards back to reach 48 would only restore what the merge removed.
  2. 49f1f17 is authored as Claude <noreply@anthropic.com>, and DCO checks the sign-off against the commit's author, so --signoff alone will not pass. Amend it with git commit --amend --reset-author -s (an interactive rebase that stops at that commit), replay the later commits and push with --force-with-lease. The repository merges by squash only, so the rewritten branch history does not reach main.

Re-checked f0cd431 merged with current main: the full suite, ruff, mypy, bandit and docs check --strict.

  • Pushed 1c5c042 for vchamarthi's must-fix: documentation-index.md linked none of the eight docs/features/*.md guides that features/README.md:72 points to, and test_the_index_links_every_doc_in_the_directory checked only top-level pages, so it stayed green. The index now links the eight guides, the test walks every subfolder, and a new test checks that an unlinked page in a subfolder is reported.
  • Left for the author: images 30 and 31, the crops on 25/26/35/43/44, the vendor marks on 25 and 27, and the alt texts, as listed earlier.

…ards are available

Thread: PRRT_kwDOVBg7g863eJeI
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…e old one as fallback

Thread: PRR_kwDOVBg7g88AAAABX--A5w
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… page

The docs home dropped these sections but kept their anchors, so the links
pointed at nothing. The sections now live in how-evaluation-works.md, which
the home and the documentation index link. The orphan anchors are gone.

Thread: PRRT_kwDOVBg7g8623NNy
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Cards 29, 34 and 41 and the development pages still called merged work
open (intel#455, intel#459, intel#472, intel#556, intel#557, intel#588, intel#590, intel#594, intel#595, intel#596). Their
availability, next steps and the work index now match the merged state, and
the routing and cost-control notes name the new commands.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Routing is free without --model and calls a model with it. analyze keeps a
partial record at the cap on main. intel#624 is closed. intel#454 is merged.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Merged PRs are annotated where the docs link them. The work index marks
merged PRs as Merged and issues closed by 14 November 2026 as Closed.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The work index marks merged PRs Merged in every row. Three limitation rows
name only closed or merged items; they now say so instead of calling the
defects open.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
intel#702 makes a capped skill shorten keep its verdicts when a response cache is
set, and a rerun replays them. The row and the budget sentence in interfaces.md
say so, and cite intel#630 and intel#702.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… it was rechecked

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov
napetrov force-pushed the docs/feature-cards-and-map branch from 2caa3ed to e993757 Compare November 14, 2026 19:59

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Everything from my earlier review is fixed: the merged PRs are marked merged on the cards and development pages, the discovery card no longer names novelty routing, the docs-new-page check reads the new index, and every commit now carries a Signed-off-by. Four nits, two inline and two below.

What I ran, on a Linux review host at e993757:

  • make ci-local: 33 passed, 0 failed, 14 skipped (the skips need Docker, network or another Python). Run tests and Coverage of changed lines pass; mkdocs build --strict passes.
  • The docs link tests against 6 planted breakages (broken nested link, broken nested anchor, unindexed page in features/, unindexed page in start/, broken root README link, README.md missing from the skip set): each one fails a test. Replacing rglob with glob in SOURCES fails none (inline).
  • Every command and flag the cards, the start guides and the quickstart name, against the CLI parser. Every PR link in features/ and development/ against the base's git log: each "merged" PR is there, each "open" one is not.
  • oracle.patch of the docs-new-page task on the task's pinned base: applies, and the updated check.py grades it all true.

Nits not anchored inline:

  1. Three images are orphaned. dokimi/docs/assets/readme/annotated-skill.png, shortening-strategies.png and three-questions.png were only referenced from the two READMEs this PR rewrites (about 527 KB, still copied into the site). git grep for the three names at the head finds nothing. Delete them or reuse them.
  2. The PR description is stale. It says 48 cards and 48 images; the tree has 39 of each (git ls-files dokimi/docs/assets/features lists 39), as the docs themselves say.

Checked, no issues

  • Pins: .github/workflows/docs.yml uses the same upload-artifact SHA as the other workflows, runs only on pull_request, and zizmor reports nothing.
  • Index rule: AGENTS.md, CONTRIBUTING.md and start/contributors.md name documentation-index.md, which is what _unindexed checks; every new page is in nav:.
  • Links: all 39 card images and anchors resolve.

Comment thread dokimi/tests/test_docs_links.py Resolved
Comment thread dokimi/docs/how-evaluation-works.md Resolved
antonwolfy and others added 3 commits November 14, 2026 21:38
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
The skills card said `novelty view` re-renders the annotated outcomes.
It calls attribute_spans without outcomes or categories, so it labels
spans by structure only (unread, protected, empty, unscored); outcome and
category colouring needs the render made during `skill novelty analyze`.
Pin the wording in the docs tests.

Signed-off-by: vchamarthi <123653452+vchamarthi@users.noreply.github.com>
@vchamarthi

Copy link
Copy Markdown
Contributor

Pushed 1 commit to docs/feature-cards-and-map (3cf74da..f9a7a17).

  • features/skills.md states novelty view colours by span and category only.
  • features/skills.md states novelty view labels spans by structure only, without outcome or category colours.
    Run: the full suite with main merged in, 20491 passed.

The PR description needs these corrections:

  • "Add 48 illustrated feature cards across eight topic guides." should read: Add 39 illustrated feature cards across eight topic guides.
  • "Verified 48 feature anchors, 48 image assets, and all nine new feature pages." should read: Verified 39 feature anchors, 39 image assets, and all nine new feature pages.

@antonwolfy

Copy link
Copy Markdown
Contributor

Pushed two commits for my nits: 3cf74da pins the widened SOURCES in tests/test_docs_links.py, and 1c62e3f rewords "Everything above" in how-evaluation-works.md. I resolved both threads. I also read f9a7a17 (the novelty view wording in features/skills.md and its test): it does what its message says and I found no problem in it. I am leaving this as a comment without an approval, since the other reviewers' review is still pending.

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at f9a7a17: Changes needed: 1 must-fix, 1 earlier finding still fails.

Blocking

  • Still fails: earlier finding – The PR description still says 48 cards and 48 images; dokimi/docs/assets/features/ holds 39 .webp files on the head. For @napetrov.
Verified on f9a7a17: 7 checks pass
Area What holds How
Docs link tests All 14 tests in dokimi/tests/test_docs_links.py pass on the head, including the nested-page index test and the skills-card wording test tests on the head
Merge onto main The PR merges onto current main with no conflict merge onto main
Mkdocs build The mkdocs build step passes with main merged in full suite, main merged in
Skills card wording Changing coloured by span structure only in dokimi/docs/features/skills.md to coloured by outcome and category fails test_skills_card_says_novelty_view_has_no_outcome_colours broke the code, tests caught it
Nested link sources Changing DOCS.rglob("*.md") to DOCS.glob("*.md") in SOURCES fails test_sources_cover_nested_pages_and_both_readmes broke the code, tests caught it
Novelty view labels With no outcomes or categories passed, attribute_spans labels a span unread, protected, empty or unscored, as the new card sentence says ran it
Task base layout Without docs/documentation-index.md, a page linked only from docs/README.md gives in_docs_index: true, as the task base expects ran it

@vchamarthi

Copy link
Copy Markdown
Contributor

Re-reviewed at f9a7a17. Card text now matches main, thanks. The status fixes for 29/34/41/43/44/12/14/28/31/40 all check out, and all 52 "Open issue" rows in the work index are open. test_docs_links.py passes (14 passed).

The images haven't changed, though, so several of them now contradict their own cards.

Images vs cards

  • 29, 34, 43, 44: the images still show an "Open PR · not on main" badge, while the cards say "Available on main". The sentence on each page, "Open PR and Research images carry a badge", now misleads. Drop the badges or regenerate the images.
  • 21: repo-eval.yaml shows task/type/patch/tests/review. The real repo_manifest.v1 has eval_dir/tasks/entry_points, its tasks are task.json files, and it lives at .agent-eval/repo-eval.yaml (repos/manifest.py:42-58).
  • 24: the decoy shows as "ignored · 0 false matches". A decoy hit is counted as a false positive (decoys_flagged, repos/graders/review.py:198-206).
  • 30: the image shows a red-team verifier control (print("PASS") → should fail). The card says "No adversarial controls are implemented".
  • 37: the image marks an in-use rule ("Run package tests") as the ablation candidate. Candidates are rules the sessions rarely or never bring up (--in-use-rate / --rare-max).
  • 11: the results.jsonl preview uses codes and keys that don't exist (missing_file, review_routing, ready_to_inspect). Real codes are hyphenated, e.g. path-missing. Its Batch/Jobs/Resume buttons stand in for CLI flags.
  • 25: one task is built from a PR diff, review.json and scip.json together. --from takes one of graph|prs|replay, and the graph mode reads a binary SCIP --index, not a scip.json.
  • 31: eval.ipynb / run_eval("held-out") don't exist. The real command is eval headroom.
  • 07: shows "Already known / New knowledge / Conflicting advice"; the outcomes are redundant / added / degraded / still-missing.
  • 23: shows "More evidence"; the verdict is insufficient-evidence.
  • Minor:
    • 01 shows an AGENTS.md before/after, but the card is arms run, and the uncertainty the card stresses isn't shown.
    • 05's task layout has no task.toml/task.yaml or tests/test.sh.
    • 33 shows two judges disagreeing, not the order/wording/degradation probes.
    • 42 shows an auto-accept loop with no Research badge.

Docs

  • development/work-index.md:46 (docs: judge-panel study of the skill-novelty headline numbers #482) and :132 (refactor(docker): one run_spec builder for every docker run argv #623) say Open PR; both are merged on main.
  • repo-evaluation-contract.md:17-22: the new banner says the core is implemented, but the text below still says "Implemented so far (slice S1)".
  • Cards 29, 34, 41, 43 and 44 have no Implementation link now that they're available.
  • start/experiments.md:22: the list of caps leaves out --max-usd on the repo hits judge (commands/repo_hits.py:211) and on eval skill-coverage (commands/skill_coverage.py:138).
  • assets/readme/annotated-skill.png, shortening-strategies.png and three-questions.png are no longer referenced anywhere.

Tests

  • test_skills_card_says_novelty_view_has_no_outcome_colours asserts exact wording in skills.md, so any reword breaks CI.
  • _relative_links still skips raw HTML href/src and [x](<path>) links, and an anchor on a directory link raises IsADirectoryError instead of reporting it.

PR

  • Still conflicts with main in dokimi/docs/README.md.
  • The description still says 48 cards / 48 images; there are 39 of each (48 is the anchor count, including the aliases).

…ure-cards-and-map

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

# Conflicts:
#	dokimi/docs/README.md
A page linked only from docs/README.md counted as indexed. The check now
reads docs/documentation-index.md when it exists, and falls back to
docs/README.md for the task base.

Thread: PRRT_kwDOVBg7g863qICo
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The docs link test requires every page in the docs directory to be
indexed; judge-panel-study.md came in with the merge of main.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approved · re-reviewed 7118eec

Next: nothing required from @napetrov.

Verified on 7118eec: ✅ changed tests: 14 passed, fail on main · ✅ full suite with main merged in: 21769 passed, its only failures are main's too · ✅ merges onto main · 1 edge input tried

3 checks behind it, each with its evidence
Area What holds How
Docs-new-page check With docs/documentation-index.md present, a page linked only from docs/README.md gives in_docs_index: false; the check reads the index only ran it, edge input (paths)
Index links documentation-index.md links all eight topic guides under docs/features/ ran it
Merge onto main The PR merges onto current main with no conflict merge onto main

@napetrov
napetrov merged commit d92be0c into intel:main Nov 16, 2026
31 checks passed
napetrov added a commit to napetrov/dokimi that referenced this pull request Nov 16, 2026
…le-arm-knowledge-run

Resolve docs/README.md: main moved the doc index to documentation-index.md (intel#627); the knowledge-runs link goes there.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
antonwolfy added a commit that referenced this pull request Nov 16, 2026
Settles the review point on #627 (B2.1) that `annotated-skill.png`, `shortening-strategies.png` and `three-questions.png` in `dokimi/docs/assets/readme/` are orphaned.

`git grep` for each name and stem across the tree (docs, README, mkdocs.yml, tests) finds no reference, so the three files are deleted with no other change.

Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants