Repository navigation
refactor(cli): one cli_types module for the copied argparse type= validators - #872
Conversation
…efuse Characterization rows for the flags whose argparse types are private copies today (context.py, evaluate.py, leak.py, tools.py and novelty_run.usd_amount): the accepted values and the line argparse prints with exit 2 for each refused one. Refs intel#229 Signed-off-by: Vladislav Perevezentsev <vladislav.perevezentsev@intel.com>
The argparse validators that commands copied each into their own module
now live in dokimi/cli_types.py, one per kind: positive_int,
non_negative_int, positive_usd, positive_fraction ((0, 1]) and
unit_interval ([0, 1]). context.py, evaluate.py, leak.py, tools.py and
novelty_run.py (usd_amount, also read by skill_coverage.py) import them
and keep no copy.
Every flag accepts and refuses the same values as before, with exit 2.
A kind has one message now, the one most of its flags printed:
- trace prefilter --rare-max: "must be 0 or more, not -1" becomes
"must be >= 0, got -1"; a non-integer says "expected an integer"
instead of "invalid _non_negative_int value".
- eval leak --verbatim-words: "expected a whole number of words, got
'0'" becomes "must be >= 1, got 0" ("expected an integer" for text).
- tools preference --max-cost-usd: "must be > 0, got inf" becomes
"must be a positive number of USD, got 'inf'".
- a value that is no number at all on --in-use-rate and --max-usd gets
the flag's own message instead of "invalid <function> value".
The snapshot changes only in the type of the moved flags.
commands/repo_run.py's _usd and cli.py's positive_int are left for a
follow-up.
Refs intel#229
Signed-off-by: Vladislav Perevezentsev <vladislav.perevezentsev@intel.com>
…ve_int tasks_generate._count was one more copy of the non-negative integer validator (--references, --derives). It goes; the flags accept the same values, and a negative now reads "must be >= 0, got -1" like the other integer flags instead of "must be 0 or more, not -1". Refs intel#229 Signed-off-by: Vladislav Perevezentsev <vladislav.perevezentsev@intel.com>
…) is given Commands get positive_int as a parameter of register() from cli.py, and cli_types.py holds the other kinds; the line said every kind comes from cli_types.py, which a reader copying a register() would not find. Refs intel#229 Signed-off-by: Vladislav Perevezentsev <vladislav.perevezentsev@intel.com>
The converters it covered left leak.py for cli_types; --verbatim-words many and --min-fraction half are pinned through the parser in tests/test_cli_types.py, so a message change had two tests to edit. Test-narrowed: the same refusals are checked end to end in test_cli_types.py. Refs intel#229 Signed-off-by: Vladislav Perevezentsev <vladislav.perevezentsev@intel.com>
The fragment said a refused number reads the same on every flag that takes one; repo run, repo hits judge and the positive_int flags keep their own copies, so it now names the commands this change touches. Refs intel#229 Signed-off-by: Vladislav Perevezentsev <vladislav.perevezentsev@intel.com>
CLI interface change detectedThis PR changes dokimi's user-facing CLI surface (commands and/or options): ~ `dokimi catalog routing` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi eval graph` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi eval judge validate` parameters changed:
~ changed option `--typed-max-len`: type: 'dokimi.commands.evaluate._non_negative_int#95d3497f2297' -> 'dokimi.cli_types.non_negative_int#02fbc9d0cba5'
~ `dokimi eval leak` parameters changed:
~ changed option `--min-fraction`: type: 'dokimi.commands.leak._fraction#949dcee9b366' -> 'dokimi.cli_types.positive_fraction#ff4bcafbe278'
~ changed option `--verbatim-words`: type: 'dokimi.commands.leak._positive_int#fdae49cd27e9' -> 'dokimi.cli_types.positive_int#96184e0d3f3b'
~ `dokimi eval skill-coverage` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi eval tasks generate` parameters changed:
~ changed option `--derives`: type: 'dokimi.commands.tasks_generate._count#62fd583ee4e8' -> 'dokimi.cli_types.non_negative_int#02fbc9d0cba5'
~ changed option `--references`: type: 'dokimi.commands.tasks_generate._count#62fd583ee4e8' -> 'dokimi.cli_types.non_negative_int#02fbc9d0cba5'
~ `dokimi skill novelty analyze` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi skill novelty atoms` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi skill novelty probe` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi skill routing` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi skill shorten` parameters changed:
~ changed option `--max-usd`: type: 'dokimi.commands.novelty_run.usd_amount#af10a4ceaa08' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi tools preference` parameters changed:
~ changed option `--max-cost-usd`: type: 'dokimi.commands.tools._positive_usd#ee3ece1dfb71' -> 'dokimi.cli_types.positive_usd#3f7e5c8902a3'
~ `dokimi trace prefilter` parameters changed:
~ changed option `--in-use-rate`: type: 'dokimi.commands.context._unit_rate#9396aeedb2c0' -> 'dokimi.cli_types.unit_interval#3988979284cb'
~ changed option `--rare-max`: type: 'dokimi.commands.context._non_negative_int#3a500817a521' -> 'dokimi.cli_types.non_negative_int#02fbc9d0cba5'Updated on every push; the |
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 14 selected item(s). CI at `531dba3`: 3 failing, 22 passing, 3 skippedNot fully read: job Test Linux Wheels Python 3.11: HTTP 404: The specified blob does not exist.; job Tests (py3.11): HTTP 404: The specified blob does not exist.; job Tests (py3.14): HTTP 404: The specified blob does not exist.
Review details
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #872 +/- ##
=======================================
Coverage 95.76% 95.76%
=======================================
Files 428 429 +1
Lines 56703 56697 -6
Branches 9766 9762 -4
=======================================
- Hits 54299 54297 -2
+ Misses 1494 1491 -3
+ Partials 910 909 -1
🚀 New features to boost your workflow:
|
antonwolfy
left a comment
There was a problem hiding this comment.
LGTM. The refactor holds up: each moved validator accepts and refuses what its copy did, and I found no code finding.
What I ran, on a Linux review host at 531dba3:
make ci-local: 34 passed, 0 failed, 14 skipped. 22,561 tests passed, 19 skipped. It ran on Python 3.13 only.test_cli_types.py,test_import_architecture.py,test_cli_flags.pyandtest_leak_command.pypass alone.
GitHub checks. Tests (py3.11), Tests (py3.14) and Test Linux Wheels Python 3.11 were red at this head. The log of Tests (py3.11) says the runner lost communication with the server, and a rerun of the failed jobs passed for all three, so I take them as infrastructure failures, not the PR.
Checked, no issues
- Behaviour: each moved validator accepts and refuses the same values as its copy. The integer bound,
isfinite, the(0, 1]and[0, 1]ranges, andnanandinfare all handled as before. Only the message text differs, as the description lists. - Changelog: the
Changedfragment is present and names the flags whose message changed. Every commit carries aSigned-off-by. - Architecture:
dokimi.cli_typesis added to the foundation layer intest_import_architecture.py, and the snapshot incli-interface.jsonchanges only thetypefields. - Coverage: changed lines are 100% covered in
cli_types.pyandcontext.py.
Summary
#229 P3-2: the argparse
type=validators that command modules copied each for themselves now live in one module,dokimi/dokimi/cli_types.py, next to the #94 F1a flag registry (cli_flags.py). One function per kind of value:cli_typespositive_intleak._positive_int(eval leak --verbatim-words)non_negative_intcontext._non_negative_int(trace prefilter --rare-max),evaluate._non_negative_int(eval judge validate --typed-max-len),tasks_generate._count(eval tasks generate --references,--derives)positive_usdtools._positive_usd(tools preference --max-cost-usd),novelty_run.usd_amount(--max-usdofskill novelty analyze/atoms/probe,skill shorten,skill routing,catalog routing,eval graph,eval skill-coverage)positive_fractionleak._fraction(eval leak --min-fraction)unit_intervalcontext._unit_rate(trace prefilter --in-use-rate)context.py,evaluate.py,leak.py,tools.py,tasks_generate.pyandnovelty_run.pykeep no private validator.dokimi/CONTRIBUTING.md("Adding or renaming a CLI flag") gets one line on where a flag takes its value check from.Every flag accepts and refuses the same values as before, with exit 2: the copies of one kind already agreed on the range (all USD copies refuse
0, negatives,nanandinfand take1e3; all integer copies refuse1.5and1e3). What differed is the message, and a kind now has one: the one most of its flags already printed (expected an integer, got 'x'/must be >= N, got V, ascli.py'spositive_intprints for about 80 flags;must be a positive number of USD, got 'x', as--max-usdprinted). A value that is not a number at all used to fall through to argparse'sinvalid <function name> valueon some flags; each kind now gives its own message for it. Hence aChangedfragment rather thanskip-changelog.Refs #229 P3-2, #94 F1a.
Changed output
Only the text after
argument --flag:of a refused value, exit code 2 unchanged:--typed-max-lenand--min-fractionprint exactly what they printed.docs/reference/cli-interface.jsonchanges in thetypefield of the 16 moved flags only (dokimi.commands.<module>.<copy>#hash->dokimi.cli_types.<kind>#hash); no other field.Evidence
tests/test_cli_types.py(first commit, onmain96dfccf: passes) runs each flag throughbuild_parser(): 14 accepted values with the number and type they parse to (1e3->1000.0for USD), 32 refused values with exit 2 and the exactargument --flag: ...line. The refactor commits change only the rows of the messages listed above; the--referencesrows were run against the oldtasks_generate.pybefore their message was updated.0,-1,-01,1.5,1e3,nan,inf, text): the accept/refuse column is identical; the message diff is the one above.cli_types.py, each failing the tests:<-><=in the integer bound (3 failed);isfinitedropped frompositive_usd(4);0let intopositive_fraction(1);unit_intervalre-raising theValueError(1); the integer message printing the typed value instead of the parsed one (1, after a-01row was added for it).tasks_generate._countwas a missed copy (replaced); theCONTRIBUTING.mdline did not match theregister(sub, positive_int)pattern (reworded; whycli.pystays is below); a leak test repeatingtest_cli_typesrows (removed,Test-narrowed:in its commit).git merge-treeof this branch with each open PR on the same files (fix(cli): exit codes for repo run crashes, cross-judge verdicts, unread traces (#94 A2b, #275) #810context.py, feat(llm): codex-cli and copilot-cli answering providers, one-condition report #806evaluate.py, fix: settle two leftovers of the reviews of #628 and #828 #856tasks_generate.py, feat(provenance): optional version and provenance on tasks and question-set items #800test_import_architecture.py, 11 PRs oncli-interface.json): no conflict this branch adds.make ci-local --offline: 0 failed, 23 passed, 20 skipped (pytest 22501 passed, 79 skipped). Not run locally: the other Python legs, the network steps (pre-commit, gitleaks, package and Windows smoke).Not in this PR
cli.py'spositive_int, defined inbuild_parser()and handed to everyregister(sub, positive_int): moving it tocli_typesrewrites thetypeof 92 snapshot lines, andgit merge-treethen showscli-interface.jsonconflicts with feat(report): one summary over cli_output.v1 documents, the leaves it needs, and actions/report (#749) #850, feat(llm): codex-cli and copilot-cli answering providers, one-condition report #806 and feat(repo): fingerprint the host with collectors; --label-host runs a missed host #781 that this branch otherwise does not cause. Until then twopositive_ints exist; they differ only in echoing-01as typed (cli.py) or as-1(cli_types).commands/repo_run.py's_usd(repo run --budget,--max-batch-usd,repo hits judge --max-usd): five open PRs on the file. Same range and message aspositive_usd; only a non-number readsinvalid _usd value.Flag.kindincli_flags.py(int,usd, ...) could name thecli_typesfunction for its kind, so the ratchet compares an option's type too;cli_flags.pyhas three open PRs.repo_security._alpha(open (0, 1)),eval_coverage._floor(METRIC=X),baseline_regression.threshold(Decimal).type=int/type=floatthat take any value, e.g.eval questions generate --count/--top-k,eval personas discover --count,--rerank-threshold,--min-accuracy,repo reach --budget(checked later in the command).Checklist
pytestpasses —tests/test_cli_types.py; 22501 passed inmake ci-localruff,mypy, and schema checks pass —ci-localstatic and typecheck rowsCONTRIBUTING.mdline, regenerateddocs/reference/cli-interface.json,ChangedfragmentAGENTS.md— mutations, other sites of the copies listed above, bad inputs (nan,inf, text,-01) in the testsraise/exit and test names the doc, contract or issue it traces to — theraisesites moved from the copies, module docstring cites Consolidate duplicated subsystems (stats, LLM routing, judge loop, containers, digests, writers) #229 P3-2; no new refusal.github/.licenserc.yaml) —cli_types.py,test_cli_types.py, the fragment