Repository navigation
Conversation
…pelling The registry compared option strings only. Each use of a canonical spelling is now compared with its registry row: a switch or a value (kind), the metavar when one is set, choices (kind `choice`), and the type for the int, float and usd kinds. Known clashes are listed in MEANING_EXCEPTIONS with a reason; a row that stops clashing fails, as the allowlist does. --judge is a switch in arms run, arms matrix-run and subjects run and a PROVIDER/MODEL elsewhere: the clash issue intel#94 names (sec. 2.4, D10), allowlisted until F1b. --context-framing is declared as a choice, which its parsers already are. No parser, default or help text changes. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 3 selected item(s). CI at Review details
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #844 +/- ##
==========================================
- Coverage 95.75% 95.74% -0.01%
==========================================
Files 423 423
Lines 56416 56416
Branches 9714 9714
==========================================
- Hits 54019 54015 -4
- Misses 1489 1492 +3
- Partials 908 909 +1
🚀 New features to boost your workflow:
|
vlad-perevezentsev
left a comment
There was a problem hiding this comment.
Review at 5f0fcf6. The meaning check works and runs on the real tree. Every clash it finds today is either fixed in the registry (--context-framing) or listed with a reason. The two gaps are in the line comments, and neither blocks the merge.
What I ran, at 5f0fcf6:
make ci-local --offline: 22,194 passed, 0 failed, 72 skipped; all checks pass.tests/test_cli_flags.pyon this branch merged into currentmainand then with each open PR that touchescli_flags.py(#489, #849, #838, #811, #783): 23 passed every time, so none of them adds a new clash.- I listed every use of a registry flag on the tree to see what each aspect checks.
Checked, no issues
- Exceptions: each one matches a real clash on the tree. A stale row, a row with no reason and a clash with no row each fail the test.
- Metavar: it is compared only when a leaf sets one, which avoids false clashes from argparse's default
DESTmetavar. The docstring says so. - Changelog: an internal-only change, under the right category; no CLI, default or help text changes.
| clashes.append((*where, "switch", f"is {shape} here, but the registry kind is `{flag.kind}`")) | ||
| if flag.kind != "flag" and use.metavar is not None and use.metavar != flag.value: | ||
| clashes.append((*where, "metavar", f"has metavar `{use.metavar}`, the registry says `{flag.value}`")) | ||
| if (flag.kind == "choice") != use.choices: |
There was a problem hiding this comment.
The choices aspect checks only that a leaf has choices, not that they are the ones the registry row lists.
--context-framing (reference|neutral), --delegation (native|none) and --fail-on ({warn,fail}) list their values in value. A leaf with choices=["native", "none", "auto"], or one that drops neutral, passes this check today. They all match now, so this only matters for new leaves.
Suggestion: when value names the values (a|b or {a,b}), compare the set with action.choices too, and keep FORMAT-style rows on the presence check. Or say in the docstring and the changelog that the values are not compared.
| _TYPES_BY_KIND: Mapping[str, frozenset[str]] = { | ||
| "int": frozenset({"int", "positive_int"}), | ||
| "float": frozenset({"float"}), | ||
| "usd": frozenset({"usd_amount", "_usd"}), | ||
| } |
There was a problem hiding this comment.
Nit: _TYPES_BY_KIND has no row for text, so --timeout (kind text, value [PHASE=]SEC) is not checked on any leaf.
catalog trigger-probe takes a plain float, and repo reach, repo run and repo skill-reach take positive_int. None of them accepts PHASE=, and none sets a metavar, so the contract reports nothing. The changelog says every use is compared on type.
Suggestion: list these four in MEANING_EXCEPTIONS (or give --timeout its own kind), so F1b sees them along with the other clashes.
vchamarthi
left a comment
There was a problem hiding this comment.
🟡 Not approved yet · reviewed 5f0fcf6
Next: @napetrov: settle 1 open question (below). 3 nits optional.
Blocking
- Open question for @napetrov:
--timeoutis declaredtextindokimi/dokimi/cli_flags.py:239, butrepo reach,repo runandrepo skill-reachparse it aspositive_intandcatalog trigger-probeasfloat. Give the row the kind the four leaves share, or keeptextand say in thetests/test_cli_flags.pydocstring thattextrows are not type-checked.
Nits
-
dokimi/CONTRIBUTING.md:275– The contributor guide says the ratchet compares spellings only; the test now compares switch or value, metavar, choices and int, float and usd types.Input: the sentence at line 275,
The ratchet compares spellings only: a reviewer checks that the option's type matches the kind of its concept.Head:meaning_clashesintests/test_cli_flags.pyalso compares switch or value, metavar, choices, and the types of int, float and usd rows. Expected: the sentence names what the ratchet compares now and what a reviewer still checks.Suggestion: Replace the sentence with: The ratchet compares spellings, switch or value, metavar, choices, and the int, float and usd types; a reviewer checks the other kinds.
Patch, run on
5f0fcf6: the quoted text is gone; tests are no worse.Patch
--- a/dokimi/CONTRIBUTING.md +++ b/dokimi/CONTRIBUTING.md @@ -272,8 +272,10 @@ - **Snapshot.** From `dokimi/`, after `make dev`, run `make cli-snapshot` and commit `docs/reference/cli-interface.json` with the change. CI's `typecheck` job and `tests/test_cli_flags.py` check it against the code, so a flag change without a - regenerated snapshot fails the PR. The ratchet compares spellings only: a reviewer checks - that the option's type matches the `kind` of its concept. + regenerated snapshot fails the PR. The ratchet compares spellings, and for each use of a + canonical spelling its switch or value, metavar, choices and, for `int`, `float` and `usd` + kinds, its type. A reviewer checks that the type of every other kind matches the `kind` of + its concept. - **Non-canonical spelling.** Only if the flag must keep it, add one row to `tests/data/cli_flags_allowlist.txt` with the reason after `#`, for example `--sample-size # the spelling the external runner already passes`. An allowlist row
Verified on 5f0fcf6: ✅ changed tests: 23 passed, fail on main · ✅ full suite with main merged in: 22223 passed · ✅ merges onto main · 6 mutations tried · 3 edge inputs tried
16 checks behind it, each with its evidence
| Area | What holds | How |
|---|---|---|
| Context-framing row | --context-framing is declared choice, which its parser matches (choices on dokimi/commands/novelty.py:637); declaring it text fails the tree check |
broke the code, tests caught it |
| Int type aspect | Narrowing the int kind to int alone fails the tree check on the positive_int rows |
broke the code, tests caught it |
| Switch aspect | A switch where the kind takes a value, and a value where it takes a switch, is reported; with that check removed the suite fails | broke the code, tests caught it |
| Stale exception row | A MEANING_EXCEPTIONS row that no longer clashes fails; with that check removed the suite fails |
broke the code, tests caught it |
| Empty exception reason | An exception row with a blank reason fails; with that check removed the suite fails | broke the code, tests caught it |
| Exception count | Eleven known clashes are listed in MEANING_EXCEPTIONS, as the description says |
ran it |
| Repeatability | Use has no repeatability field, so a flag's repeatability is not compared, as the description says |
ran it |
| Metavar text | A metavar is compared as exact text: "", " N", "N ", "N\r\n" and "n" each give one metavar clash against N, and "N" gives none |
ran it, edge input (text) |
| Unset metavar | A use with no metavar set (None) gives no metavar clash, so the registry value is not compared for it |
ran it, edge input (none) |
| Tuple metavar | A tuple metavar is joined with a space before the comparison, so ("PROVIDER", "MODEL") clashes with PROVIDER/MODEL |
ran it, edge input (tuple) |
| Banned spellings | A banned spelling is not in the canonical map, so --max-tokens gives no meaning clash, as the description says |
ran it |
| Evidence count | The three flag test files give 110 passed on the head, as the description says | ran it |
| Changed tests | The 23 tests of tests/test_cli_flags.py pass on the head, including the tree-wide meaning check, the leaf-scoped --judge case and the --context-framing row |
tests on the head |
| New tests on main | The tree-wide meaning check and the leaf-scoped --judge case fail on main |
same tests on main |
| Full check run | The full repository check run passes with the PR merged onto main, including the CLI snapshot and mypy, so no flag, default or help text changes | full suite, main merged in |
| Merge | The PR merges onto main with no conflict | merge onto main |
| # The other direction: a switch where the kind takes a value. | ||
| switch = [Use("dokimi a", "--n", "_StoreTrueAction", "int", "n", 0, None, False)] | ||
| assert [what for *_, what in meaning_clashes(switch, concepts)] == [ | ||
| "is a switch here, but the registry kind is `int`"] |
There was a problem hiding this comment.
🟡 Nit: Limiting the type check to int rows still passes all 23 tests, so a usd or float row whose parser takes the wrong type is not reported.
Input: --max-usd, registered as kind usd, with a parser of type=float, through meaning_clashes. Head: reports a type clash for it. Limiting the type comparison to int rows leaves all 23 tests green, and no test pins a usd or float clash. Expected: a case that fails when that comparison is skipped.
Fix, run on 5f0fcf6: the tests now catch the broken variant; tests and lint are no worse.
| "is a switch here, but the registry kind is `int`"] | |
| "is a switch here, but the registry kind is `int`"] | |
| def test_a_usd_or_float_row_reports_a_wrong_type(): | |
| concepts = (cli_flags.Concept(name="x", help="", canonical=( | |
| cli_flags.Flag("--max-usd", "USD", "usd"), | |
| cli_flags.Flag("--rate", "RATE", "float"), | |
| )),) | |
| found = [ | |
| Use("dokimi a", "--max-usd", "_StoreAction", "usd_amount", "max_usd", None, "USD", False), | |
| Use("dokimi a", "--max-usd", "_StoreAction", "float", "max_usd", None, "USD", False), | |
| Use("dokimi a", "--rate", "_StoreAction", "float", "rate", None, "RATE", False), | |
| Use("dokimi a", "--rate", "_StoreAction", "int", "rate", None, "RATE", False), | |
| ] | |
| flagged = [(option, aspect) for _leaf, option, aspect, _what in meaning_clashes(found, concepts)] | |
| assert flagged == [("--max-usd", "type"), ("--rate", "type")] |
| @@ -0,0 +1,3 @@ | |||
| ### Changed | |||
|
|
|||
| - **Internal: the flag contract also checks what a flag means, not only its spelling.** `tests/test_cli_flags.py` compares every use of a registered spelling with its registry row: switch or value, metavar, choices and type. The known clashes (for example `--judge` as a switch in `arms run`) are listed with a reason and do not change the CLI. `--context-framing` is declared as a choice in the registry. No command, flag, default or help text changes. | |||
There was a problem hiding this comment.
🟡 Nit: The changelog fragment says the flag contract compares type for every use; only int, float and usd rows are compared.
Input: --timeout on repo reach, registered as kind text, with a parser of type=positive_int. Head: no type comparison, since _TYPES_BY_KIND lists only int, float and usd. Expected: the fragment and cli_flags.py:13 name the three kinds the type comparison covers.
Also at: dokimi/dokimi/cli_flags.py:13.
Suggestion: Write type (int, float and usd rows) in the fragment and in the cli_flags.py docstring.
Patch, run on 5f0fcf6: the quoted text is gone; tests and lint are no worse.
Patch
--- a/dokimi/changelog.d/20261116_120000_napetrov_arch-c12n-flag-meaning.md
+++ b/dokimi/changelog.d/20261116_120000_napetrov_arch-c12n-flag-meaning.md
@@ -1,3 +1,3 @@
### Changed
-- **Internal: the flag contract also checks what a flag means, not only its spelling.** `tests/test_cli_flags.py` compares every use of a registered spelling with its registry row: switch or value, metavar, choices and type. The known clashes (for example `--judge` as a switch in `arms run`) are listed with a reason and do not change the CLI. `--context-framing` is declared as a choice in the registry. No command, flag, default or help text changes.
+- **Internal: the flag contract also checks what a flag means, not only its spelling.** `tests/test_cli_flags.py` compares every use of a registered spelling with its registry row: switch or value, metavar, choices, and the type of `int`, `float` and `usd` rows. The known clashes (for example `--judge` as a switch in `arms run`) are listed with a reason and do not change the CLI. `--context-framing` is declared as a choice in the registry. No command, flag, default or help text changes.
--- a/dokimi/dokimi/cli_flags.py
+++ b/dokimi/dokimi/cli_flags.py
@@ -9,9 +9,9 @@
spelling that is neither canonical here nor on ``tests/data/cli_flags_allowlist.txt``
fails the build, and so does a banned synonym whose concept has a canonical spelling,
unless that spelling is on the allowlist. The same test also compares every use of a canonical
-spelling with its registry row: switch or value (``kind``), metavar (``value``), choices and
-type (``kind``). ``meaning_clashes`` in ``tests/test_cli_flags.py`` says what is compared; a
-flag's repeatability is not compared.
+spelling with its registry row: switch or value (``kind``), metavar (``value``), choices, and the
+type of an ``int``, ``float`` or ``usd`` kind. ``meaning_clashes`` in ``tests/test_cli_flags.py``
+says what is compared; a flag's repeatability is not compared.
A concept is one row of the §2.4 audit. Its ``canonical`` spellings are the ones new
commands use. ``banned`` lists the synonyms the audit found for it. A banned spelling
Summary
The flag contract (
tests/test_cli_flags.py) now compares what a canonical flag means on every leaf, not only its spelling:--judgeas a switch inarms run,arms matrix-runandsubjects run(CLI API: the evaluation loop at the top level, subject and eval groups below, one enforced flag vocabulary (proposal v2.1) #94 §2.4, D10; F1b moves it to--judge P/M).--context-framingis declaredchoice, since its parser already passeschoices=.--max-tokens) are not meaning-checked here.Evidence
cli_flagschecks spelling only;--judgeand--context-framingclashes go unreported.After:
pytest tests/test_cli_flags.py tests/test_cli_flag_moves.py tests/test_cli_skill_catalog_moves.pyon Python 3.13: 110 passed (new tests cover each aspect, a second clash on an excepted pair, and the empty-reason and stale-row checks).make ci-local(local run on the head): 0 failed, 34 passed, 14 skipped. One earlier run had a timing failure intest_example_runner_processes.pythat passed on rerun.run_idis normalised. No parser, default or help change.Merge Danger
Door: two-way. Registry data and a test; no runtime code path changes.
Blast Radius: contributors only. A new flag that clashes with its registry row now fails
test_every_registered_flag_has_one_meaning_on_the_treeuntil it is fixed or listed inMEANING_EXCEPTIONS.Follow-ups:
--judge P/M(#94 F1b); the other clash rows (--from,--format,--arm,--cache,--network,--questions) are F1b work; repeatability and banned-spelling meaning are not checked yet.🤖 Generated with Claude Code