Repository navigation
Conversation
… libraries `skill novelty ground` could only veto long options from a tool's --help. A closed C library (oneMKL) has no such help, so an answer naming an invented function (cblas_dgemmx, LAPACKE_dgesvv) went to the judge. --header PATH (glob ok) and --library PATH (read with nm -D --defined-only), both repeatable, add the API's names as the known set of an identifiers veto. --prefix (a C identifier ending in '_', repeatable) is required with them and scopes which names are checked, so an ordinary word is never read. Each input is pinned by the sha256 of its bytes, as the help snapshot is. A set with no prefixed name is refused, so a wrong header cannot veto everything. Key verification (meta.key_status) covers these names too. The flags are registered as the api_header and symbol_prefix concepts. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… input errors symbols_check([]) built an empty alternation that matches every word of an answer; the CLI refused this already, the function did not. An unreadable --header raised a bare PermissionError, which printed a traceback instead of the one-line error the other input refusals give. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…state the refusal per set add_checks keyed a symbol check by the literal "symbols", so a --tool named symbols was deleted on a re-run that added --header or --library. The key is now a tuple. grading-tiers.md now says the refusal applies to the sources together, and that one file with no prefixed name is not reported. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…refixes keeps the first veto add_checks replaced every symbol check on the atom, so a second ground run with other --prefix values dropped the first run's veto. Symbol checks are now keyed by their prefixes. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
OpenCodeReview: Commit 0691210 was not reviewed. This PR has reached its review limit (6 reviews). CI at `0691210`: 10 still running, 3 passing, 1 skipped
To review this commit: re-request review from exai-bot in the Reviewers list. <details><summary>Last review (40c8061): 4 medium</summary> Open threads from the bot: 0 </details> |
CLI interface change detectedThis PR changes dokimi's user-facing CLI surface (commands and/or options): ~ `dokimi skill novelty ground` parameters changed:
+ added option `--header`
+ added option `--prefix`
+ added option `--shared-lib`
~ changed option `--tool`: required: True -> FalseUpdated on every push; the |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #811 +/- ##
========================================
Coverage 95.79% 95.79%
========================================
Files 432 433 +1
Lines 57204 57368 +164
Branches 9858 9890 +32
========================================
+ Hits 54796 54957 +161
- Misses 1497 1500 +3
Partials 911 911
🚀 New features to boost your workflow:
|
vchamarthi
left a comment
There was a problem hiding this comment.
🔴 Changes needed: 3 must-fixes, 1 nit · reviewed 7fc1317
Next: @napetrov: fix the 3 must-fix findings; then push for a re-review. 1 nit optional.
Verified on 7fc1317: ✅ changed tests: 136 passed, fail on main · ✅ full suite with main merged in: 21801 passed, its only failures are main's too · ✅ merges onto main · 4 edge inputs tried
3 checks behind it, each with its evidence
| Area | What holds | How |
|---|---|---|
| Changed tests | tests/test_novelty_checks.py and tests/test_novelty_ground_symbols.py pass on the head (136 passed, 2 skipped) |
tests on the head |
| Merge | The head merges onto main with no conflict | merge onto main |
| Whole suite | The whole check run on the head merged onto main passes | full suite, main merged in |
…und-symbols Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…ed libraries A header or library path that exists is read as it is, even with glob characters in its name. A `//` comment continued by a backslash ends with its next line, so a name in that line is not declared. A library path starting with `-` reaches nm after `--`. A symbol check replaces any older one that shares a prefix, not only one with the same prefix set, so a stale older check cannot veto a name the newer headers declare. Thread: PRRT_kwDOVBg7g8630sjS Thread: PRRT_kwDOVBg7g8630sjc Thread: PRRT_kwDOVBg7g8630sji Thread: PRRT_kwDOVBg7g8630YKj Review: PRR_kwDOVBg7g88AAAABYKZxaA Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
A macro's replacement list counts as a name the header spells, but a name the preprocessor builds (token pasting) does not, so a library is the workaround. A header with no prefixed name is recorded with its sha256 and gives no warning. Thread: PRRT_kwDOVBg7g8630YKe Thread: PRRT_kwDOVBg7g8630sju Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
…und-symbols Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…t API names are checked A nested prefix (mkl_ and mkl_sparse_) now replaces the stale check, and a name after an operator is an API name when it is called or follows an operand; a member, option or path stays excluded. Thread: PRRT_kwDOVBg7g864GSzn Thread: PRRT_kwDOVBg7g864GSzs Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
An invented call after an operator, written with a space or a newline before its argument list, was not in the key scope, so the veto missed it. Thread: PRRT_kwDOVBg7g864Ix9M Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…und-symbols Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
A symbol check was dropped whole when a new symbol check overlapped one of its prefixes, so names in its other scopes (for example mkl_vml_*) were no longer checked. Now a symbol check with a prefix the new check does not cover is kept, and takes the new check's names for the shared scope; names outside that scope stay as they were. A check whose prefixes are all covered by the new check is still replaced, and a kept check left with no name is dropped. Thread: PRRT_kwDOVBg7g864GSzn Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
vlad-perevezentsev
left a comment
There was a problem hiding this comment.
Review at 3e5ed8d. The symbol check works for the documented cblas_ / LAPACKE_ case, and the earlier must-fixes (glob paths, // splices, --named libraries, nested prefixes, operator-adjacent calls) are fixed. One gap needs a change before merge: a broad prefix like mkl_ vetoes correct link instructions. The rest are small and in the line comments.
Agreed with @exai-bot: the lost sha256 of an overwritten header on a partial refresh; the member access with a space after -> (real, but rare: low).
What I ran, at 3e5ed8d:
make ci-local --offline: 0 failed, 23 passed, 20 skipped (network and matrix jobs).- Repros through
grade_check/add_checks: link sentences withmkl_, the#errorapostrophe case, a subset-prefix refresh, a same-path overwrite, and answers of 8 KB, 42 KB and 126 KB for timing.
Checked, no issues
- Refresh rules: a check whose prefixes the new run covers is replaced; a broader one keeps its other names and takes the new ones; a disjoint one is kept as is.
- Input refusals: an empty prefix list,
--prefixwithout a source, a source without--prefix, a file set with no prefixed name, andnmerrors are input errors, not tracebacks or empty vetoes. - Docs and changelog: the fragment is under Added and says what to run; the macro and
#includelimits are documented. The check is opt-in, so no published score changes.
| A C API has no `--help`, so its check reads names instead. `--prefix` (a C identifier ending in | ||
| `_`, repeatable) says which names are checked: an answer's `cblas_dgemmx` is vetoed when no |
There was a problem hiding this comment.
With --prefix mkl_, a correct link instruction is vetoed: library names share the prefix but are no symbols.
With a header declaring mkl_set_num_threads:
Link with mkl_intel_lp64, mkl_sequential and mkl_core.→not in source: mkl_intel_lp64, mkl_sequential, mkl_coreUse the mkl_rt single dynamic library.→not in source: mkl_rttarget_link_libraries(app MKL::mkl_core)→not in source: mkl_core
No --library exports a symbol named after itself, and the tests use mkl_ as a prefix.
Suggestion: say in the docs that a prefix must not be shared with library or file names (mkl_sparse_, cblas_, not mkl_), with the link sentence as a test; or skip names in a link context (-l, ::, after link).
There was a problem hiding this comment.
Waiting on a decision before I change this: fix the --prefix mkl_ link veto in the docs (a) or in code (b)? My recommendation: (a), a rule that a prefix must not be shared with library or file names, with the link sentence as a test. The thread stays open until then.
A partial refresh over a same-path overwrite keeps both snapshot hashes for the names that still depend on the old one. A subset-prefix refresh no longer adds a second check over the same names. A member access after -> may have blanks before its name; a name after . is not vetoed, since a sentence's period and a space look the same. A C short character literal is the only literal skipped; an apostrophe in a directive is not. The scan is linear in the answer length, and a clause window cut inside a word negates nothing. --tool-version without --tool is refused. Thread: PRRT_kwDOVBg7g864UA60 Thread: PRRT_kwDOVBg7g864UA67 Thread: PRRT_kwDOVBg7g864aBl- Thread: PRRT_kwDOVBg7g864aBl4 Thread: PRRT_kwDOVBg7g864aBly Thread: PRRT_kwDOVBg7g864aBmJ Review: PRR_kwDOVBg7g88AAAABYelOBQ Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…und-symbols Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
Re: review by vlad-perevezentsev
|
…s --library The shared library of skill novelty ground gets its own concept (shared_library, next to api_header) and the spelling --shared-lib PATH. --library is not on the ground command any more, so the inputs concept's ban needs no allowlist row for it. Thread: PRRT_kwDOVBg7g864aBlm Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Summary
dokimi skill novelty groundcan now take a C library's API as its source of names:--header PATH(repeatable, glob ok) reads the prefixed identifiers of each header's code, skipping comments and string literals.--library PATH(repeatable) reads the defined names ofnm -D --defined-only, with the@@VERtag dropped.--prefix PREFIX(repeatable, a C identifier ending in_, for examplecblas_) says which names are checked. It is required with a source, and refused without one.The result is one
identifiersveto (role: veto,tier: verifiable) per run. An answer that names a prefixed API name no source declares or exports is vetoed without a judge. Each input is pinned by the sha256 of its bytes.verify_keyswritesmeta.key_statusfor a key that names a missing API name asnot_found, and a key that names a known one asverified.Other changes in the commit series:
dokimi/novelty/symbols.py;add_checks(atoms, checks)takes each check's mention from the check itself (its tool command line, or its symbol prefixes);api_headerandsymbol_prefix, withdocs/reference/cli-interface.jsonregenerated;grading-tiers.md,answer-formats.md,skill-novelty.md; a changelog fragment.Evidence
tests/test_novelty_ground_symbols.pycovers header and library names, prefix and negation vetoes (parametrised), key verification (not_foundandverified), sha256 pinning, prefix validation, glob and no-match refusal, the CLI end to end, and flag combinations. Each fix in the series came with a test that failed first.ccand reads it throughnm. Exported prefixed names are known; a static function is not.nm_namessurvived. The clause was dead (the field-count check already drops those lines), so it was removed rather than tested.make ci-local CI_LOCAL_ARGS='--offline --jobs static,typecheck'gives 0 failed (static, mypy, CLI snapshot, deptry); novelty and CLI tests: 4138 passed, 4 skipped. The full pytest run (21766 passed on 7fc1317, before the sync) is not repeated here; the CI run on the pushed head is the gate.pr-review, one pass): its code findings are none; its two description findings are fixed here.Merge danger
add_checksloses itstoolsparameter. The in-tree callers are updated; an external caller breaks.--toolis no longer required by argparse.Risks and known limits
_DENIED_AFTERineval/reference_checks.py.##in a macro is not read from a header. A correct answer naming it is vetoed. Probe:#define LAPACKE_DECL(n) LAPACKE_##nwithLAPACKE_DECL(dgesv)gives known=['LAPACKE_DECL']. Workaround: also pass the library, sincenmlists the exported name.#includeof another header is not read; pass each header.--prefixthat no source declares vetoes every mention with it. This follows from the spec; the prefix must match the API./,-,.or>, or directly followed by.word, is not matched (paths, members, file names).cblas_dgemm/cblas_dgemmxmisses the second name.--libraryis on the existing CLI-flag allowlist, because the subject concept bans it for another command.🤖 Generated with Claude Code