Repository navigation
docs: organize Dokimi features into an illustrated guide - #627
Conversation
Add 48 addressable feature cards, eight guide groups and a linked feature tree. Refresh README and documentation entry points, distinguish working features from proposed extensions, and reconcile stale journey claims. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
OpenCodeReview: Commit 7118eec was not reviewed. This PR has reached its review limit (6 reviews). CI at `7118eec`: 11 still running, 2 passing, 1 skipped
1 more not listed. To review this commit: re-request review from exai-bot in the Reviewers list. <details><summary>Last review (f0cd431): no findings</summary> Open threads from the bot: 0 </details> |
There was a problem hiding this comment.
🟡 Changes recommended
Documentation still mislabels knowledge retention as task value and conflates answer-grading red-team checks with adversarial controls for executable-task verifiers.
2 open issues
What changed in this PR
2 new findings.
This PR adds a question-based capability map for Dokimi’s agent-context evaluation workflows, distinguishing implemented features from partial workflows and planned work.
Changes:
- Adds 48 feature cards across eight topics, with illustrations, availability boundaries and guide links.
- Updates README entry points, measurement guidance and repository-evaluation status.
- Adds feature navigation and expands link checking to nested documentation and both READMEs.
| File | Description |
|---|---|
README.md |
Introduces question-based navigation and free starting checks. |
dokimi/tests/test_docs_links.py |
Checks nested documentation and both READMEs. |
dokimi/README.md |
Links the capability map and updates implementation status. |
dokimi/mkdocs.yml |
Adds feature pages to site navigation. |
dokimi/docs/what-it-measures.md |
Links the map and clarifies novelty interpretation. |
dokimi/docs/user-journeys/journeys.md |
Updates task-generation, coverage and adherence boundaries. |
dokimi/docs/repo-evaluation-contract.md |
Identifies the implemented repository-evaluation core. |
dokimi/docs/README.md |
Streamlines the documentation home around capabilities and evidence. |
dokimi/docs/features/traces.md |
Documents context inspection and execution diagnostics. |
dokimi/docs/features/skills.md |
Covers skill analysis, shortening and validation. |
dokimi/docs/features/repositories.md |
Covers repository guidance and evaluation workflows. |
dokimi/docs/features/README.md |
Indexes all 48 cards and defines availability labels. |
dokimi/docs/features/judging.md |
Explains grading methods and judge checks. |
dokimi/docs/features/experiments.md |
Covers controlled comparisons, harnesses and cost. |
dokimi/docs/features/eval-design.md |
Covers task creation, coverage and grader controls. |
dokimi/docs/features/discovery.md |
Covers skill selection and instruction loading. |
dokimi/docs/features/continuous-evaluation.md |
Covers reports, regression checks and development directions. |
Review effort: Balanced
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #627 +/- ##
=======================================
Coverage 95.76% 95.76%
=======================================
Files 416 416
Lines 55624 55624
Branches 9553 9553
=======================================
Hits 53267 53267
Misses 1456 1456
Partials 901 901 🚀 New features to boost your workflow:
|
Add role-based routes, result interpretation, scoped feature cards and a downloadable PR documentation preview. Preserve existing overview links. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Document development directions, CLI migrations, reliability limits and all 109 open work items. Correct feature inputs, workflow links, shortening semantics, resource budgets and free versus paid steps after independent review. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The card claimed adversarial controls for executable tasks, which no code implements. It now lists null, oracle and isolation checks, and points answer-only grading to eval redteam. Thread: PRRT_kwDOVBg7g862ZmzX Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Thread: PRRT_kwDOVBg7g862aHgu Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Thread: PRRT_kwDOVBg7g862aHgu Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…es-only route Thread: PRRT_kwDOVBg7g862aHgu Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
- 48 -> 41 cards: fold 03 into 02, 47 into 04, 08 into 07, 45 into 12, 16 into 13, 22 into 21 and 17 into 35; old anchors kept as aliases - new images for 05, 24, 32, 38 and 39, which showed another card's art - Open PR / Research badges on 18, 29, 34, 41, 43 and 44 - fix line numbering in images 44 and 48 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wg6N1cwy55Kuq6gxjrgQm1 Signed-off-by: Claude <noreply@anthropic.com>
- fold 38 into 27 (documentation as input) and 39 into 40 (stored results and regression checks); 41 -> 39 cards, old anchors kept - every card image's alt text now starts with the card heading Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wg6N1cwy55Kuq6gxjrgQm1 Signed-off-by: Claude <noreply@anthropic.com>
… claude/pr-627-review-fixes-bet7ht Signed-off-by: Claude <noreply@anthropic.com> # Conflicts: # dokimi/docs/features/eval-design.md # dokimi/docs/quickstart.md
…gn conflict Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wg6N1cwy55Kuq6gxjrgQm1 Signed-off-by: Claude <noreply@anthropic.com>
docs: address intel#627 review threads and self-review findings
docs: address intel#627 review threads and self-review findings
Feature cards now state what main does (ablation candidates, routing scope, trace inputs, review and explore results, CI coverage); the start guides name the flag spellings 0.2.0 still uses; contributor instructions name the new index; card links point at the pages that cover each feature. Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
|
Ran: the start guides followed on Linux with dokimi 0.2.0 from PyPI and with this branch; the cards checked against the code at main; Pushed f0cd431:
Left for the author (images, not touched here):
DCO fails: 49f1f17 ("docs: address PR #627 review threads and self-review findings") has no #472, #482, #489 and #588 add pages linked from |
vchamarthi
left a comment
There was a problem hiding this comment.
Review at f0cd431. 3 new findings: 1 must-fix, 2 nits. In the line comments. Reorganizes the docs into an illustrated feature tree of 39 cards and shortens the docs home. The docs link tests pass on the PR and it merges without conflict, but the eight topic guides under docs/features/ are missing from the documentation index that the docs say must link them, and the docs home drops sections while keeping their anchors. The PR text says 48 cards and 48 images; the tree holds 39 of each.
Verified
- Docs link tests: All 11 tests in
dokimi/tests/test_docs_links.pypass on the PR head, including the relative-link and same-page anchor checks over the changed pages. The index test passes only because it covers top-level pages (see the first finding). - Merge: The PR merges onto main with no conflict.
- Card count: The feature tree links 39 distinct cards (
dokimi/docs/features/README.md), anddokimi/docs/assets/features/holds 39.webpimages.dokimi/docs/README.md:32anddokimi/docs/documentation-index.md:27both say 39. The PR description's 48 cards and 48 images do not match the tree.
Not verified
- The availability labels on the 39 cards ("available on main", partial, proposed) were not traced to code feature by feature; only the commands and flags the cards cite were looked at: Cite the file or test that shows each "available on main" label on its card, or mark the label as the author's reading.
|
@xaleryb two open questions before I change anything here:
|
The feature tree says a new topic page is linked from the index, but the index test only looked at top-level pages, so the eight guides under docs/features/ were missing unnoticed. The test now walks every subfolder. Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
|
@napetrov on your two questions:
Re-checked f0cd431 merged with current main: the full suite, ruff, mypy, bandit and
|
…ards are available Thread: PRRT_kwDOVBg7g863eJeI Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…e old one as fallback Thread: PRR_kwDOVBg7g88AAAABX--A5w Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… page The docs home dropped these sections but kept their anchors, so the links pointed at nothing. The sections now live in how-evaluation-works.md, which the home and the documentation index link. The orphan anchors are gone. Thread: PRRT_kwDOVBg7g8623NNy Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Cards 29, 34 and 41 and the development pages still called merged work open (intel#455, intel#459, intel#472, intel#556, intel#557, intel#588, intel#590, intel#594, intel#595, intel#596). Their availability, next steps and the work index now match the merged state, and the routing and cost-control notes name the new commands. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Merged PRs are annotated where the docs link them. The work index marks merged PRs as Merged and issues closed by 14 November 2026 as Closed. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The work index marks merged PRs Merged in every row. Three limitation rows name only closed or merged items; they now say so instead of calling the defects open. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… it was rechecked Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
2caa3ed to
e993757
Compare
antonwolfy
left a comment
There was a problem hiding this comment.
LGTM. Everything from my earlier review is fixed: the merged PRs are marked merged on the cards and development pages, the discovery card no longer names novelty routing, the docs-new-page check reads the new index, and every commit now carries a Signed-off-by. Four nits, two inline and two below.
What I ran, on a Linux review host at e993757:
make ci-local: 33 passed, 0 failed, 14 skipped (the skips need Docker, network or another Python).Run testsandCoverage of changed linespass;mkdocs build --strictpasses.- The docs link tests against 6 planted breakages (broken nested link, broken nested anchor, unindexed page in
features/, unindexed page instart/, broken root README link,README.mdmissing from the skip set): each one fails a test. ReplacingrglobwithglobinSOURCESfails none (inline). - Every command and flag the cards, the start guides and the quickstart name, against the CLI parser. Every PR link in
features/anddevelopment/against the base's git log: each "merged" PR is there, each "open" one is not. oracle.patchof thedocs-new-pagetask on the task's pinned base: applies, and the updatedcheck.pygrades it all true.
Nits not anchored inline:
- Three images are orphaned.
dokimi/docs/assets/readme/annotated-skill.png,shortening-strategies.pngandthree-questions.pngwere only referenced from the two READMEs this PR rewrites (about 527 KB, still copied into the site).git grepfor the three names at the head finds nothing. Delete them or reuse them. - The PR description is stale. It says 48 cards and 48 images; the tree has 39 of each (
git ls-files dokimi/docs/assets/featureslists 39), as the docs themselves say.
Checked, no issues
- Pins:
.github/workflows/docs.ymluses the sameupload-artifactSHA as the other workflows, runs only onpull_request, andzizmorreports nothing. - Index rule:
AGENTS.md,CONTRIBUTING.mdandstart/contributors.mdnamedocumentation-index.md, which is what_unindexedchecks; every new page is innav:. - Links: all 39 card images and anchors resolve.
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
The skills card said `novelty view` re-renders the annotated outcomes. It calls attribute_spans without outcomes or categories, so it labels spans by structure only (unread, protected, empty, unscored); outcome and category colouring needs the render made during `skill novelty analyze`. Pin the wording in the docs tests. Signed-off-by: vchamarthi <123653452+vchamarthi@users.noreply.github.com>
|
Pushed 1 commit to
The PR description needs these corrections:
|
|
Pushed two commits for my nits: 3cf74da pins the widened |
vchamarthi
left a comment
There was a problem hiding this comment.
Re-review at f9a7a17: Changes needed: 1 must-fix, 1 earlier finding still fails.
Blocking
- Still fails: earlier finding – The PR description still says 48 cards and 48 images;
dokimi/docs/assets/features/holds 39.webpfiles on the head. For @napetrov.
Verified on f9a7a17: 7 checks pass
| Area | What holds | How |
|---|---|---|
| Docs link tests | All 14 tests in dokimi/tests/test_docs_links.py pass on the head, including the nested-page index test and the skills-card wording test |
tests on the head |
| Merge onto main | The PR merges onto current main with no conflict | merge onto main |
| Mkdocs build | The mkdocs build step passes with main merged in |
full suite, main merged in |
| Skills card wording | Changing coloured by span structure only in dokimi/docs/features/skills.md to coloured by outcome and category fails test_skills_card_says_novelty_view_has_no_outcome_colours |
broke the code, tests caught it |
| Nested link sources | Changing DOCS.rglob("*.md") to DOCS.glob("*.md") in SOURCES fails test_sources_cover_nested_pages_and_both_readmes |
broke the code, tests caught it |
| Novelty view labels | With no outcomes or categories passed, attribute_spans labels a span unread, protected, empty or unscored, as the new card sentence says |
ran it |
| Task base layout | Without docs/documentation-index.md, a page linked only from docs/README.md gives in_docs_index: true, as the task base expects |
ran it |
|
Re-reviewed at f9a7a17. Card text now matches main, thanks. The status fixes for 29/34/41/43/44/12/14/28/31/40 all check out, and all 52 "Open issue" rows in the work index are open. The images haven't changed, though, so several of them now contradict their own cards. Images vs cards
Docs
Tests
PR
|
…ure-cards-and-map Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com> # Conflicts: # dokimi/docs/README.md
A page linked only from docs/README.md counted as indexed. The check now reads docs/documentation-index.md when it exists, and falls back to docs/README.md for the task base. Thread: PRRT_kwDOVBg7g863qICo Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The docs link test requires every page in the docs directory to be indexed; judge-panel-study.md came in with the merge of main. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
vchamarthi
left a comment
There was a problem hiding this comment.
🟢 Approved · re-reviewed 7118eec
Next: nothing required from @napetrov.
Verified on 7118eec: ✅ changed tests: 14 passed, fail on main · ✅ full suite with main merged in: 21769 passed, its only failures are main's too · ✅ merges onto main · 1 edge input tried
3 checks behind it, each with its evidence
| Area | What holds | How |
|---|---|---|
| Docs-new-page check | With docs/documentation-index.md present, a page linked only from docs/README.md gives in_docs_index: false; the check reads the index only |
ran it, edge input (paths) |
| Index links | documentation-index.md links all eight topic guides under docs/features/ |
ran it |
| Merge onto main | The PR merges onto current main with no conflict | merge onto main |
…le-arm-knowledge-run Resolve docs/README.md: main moved the doc index to documentation-index.md (intel#627); the knowledge-runs link goes there. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Settles the review point on #627 (B2.1) that `annotated-skill.png`, `shortening-strategies.png` and `three-questions.png` in `dokimi/docs/assets/readme/` are orphaned. `git grep` for each name and stem across the tree (docs, README, mkdocs.yml, tests) finds no reference, so the three files are deleted with no other change. Signed-off-by: Anton Volkov <antonwolfy@gmail.com>

Make Dokimi’s capabilities easier to discover and understand: start with a concise README, explore a linked feature tree, and open individual feature cards for details.
What changes
Feature availability
The guide distinguishes functionality available on main, partial implementations, open proposals, and research directions. Availability was reviewed against the code and open feature PRs on November 11, 2026.
“Available on main” does not imply inclusion in a published release. Illustrations explain behavior; they are not product screenshots or measured results.
Validation
The full test run finished with 18,554 passed, 18 skipped, and 10 failed. All 10 failures also reproduce on the unchanged base in the same environment.
Full local CI remains non-green due to those failures and additional environment/tooling failures in package and smoke-test steps. Keeping this PR as a draft pending CI review.
No product implementation changes.