Skip to content

fix(novelty): a partial atoms file with no usable message names its stop - #877

Open
antonwolfy wants to merge 2 commits into
intel:mainfrom
antonwolfy:fix/atoms-partial-message
Open

antonwolfy wants to merge 2 commits into
intel:mainfrom
antonwolfy:fix/atoms-partial-message

Conversation

@antonwolfy

@antonwolfy antonwolfy commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

load_atoms_record refuses a partial atoms file whose partial.reason is no_price or permanent_error by printing the block's message as it is. On main a block with no usable message gives an empty or junk slot:

partial block Refusal on main
{"reason": "no_price"} <file> is a partial atoms file: , so it is not a finished atom set. Drop --max-usd to run unmetered
{"reason": "no_price", "message": " "} ... file: , so it ... (blank slot)
{"reason": "permanent_error", "message": 5} ... file: 5, so it ...
{"reason": "permanent_error", "message": "​"} ... file: <zero-width space>, so it ...

When the message is not a string with a printable, non-space character, the refusal now names the stop: a call with no known price stopped the run for no_price, a permanent provider error stopped the run for permanent_error. This is the fix the issue proposes, the way novelty/items.py::_refuse_partial falls back. The texts are atoms' own: an atoms file is not a measurement, and the refusal already ends with its advice (Drop --max-usd to run unmetered / Fix the error and rerun), so the probe texts would say it twice. A real message is printed as is. The check keeps a message made only of an invisible-looking printable character such as U+3164 (the slot then looks blank); partial_atoms_refusal's docstring says only what the check covers, and the rejection of such characters is tracked in #881.

Other readers of a partial block's message were read and run on this branch with absent, blank, non-text and zero-width messages. load_artifact("novelty", ...) never prints the file's message (its refusal is fixed text by reason), and novelty items already falls back. novelty_budget.py (save_analyze_partial, save_atoms_run) and novelty_routing_report.py::stop_if_partial format a block the running process just built, with a message the code wrote, not one read from a file. No further site.

Closes #833

Changes

  • dokimi/dokimi/novelty/atoms_refusal.py (new): partial_atoms_refusal(path, partial) builds the refusal sentence for every reason, with the fallback for no_price and permanent_error. load_atoms_record in novelty/atoms.py calls it. The spend_cap text, the no-block text and the advice sentences are unchanged. A reason that is not text (a list, a number) is still read as a cap stop, as before. The sentence moved out of atoms.py because the file sits at the 800-line cap.
  • tests/test_atoms_partial_status.py: the refusal sentence for both reasons with no message, empty, spaces, whitespace, 5, null, a list and a zero-width space; a real message printed as is; a non-text reason read as a cap stop. Each cites docs/skill-novelty.md:1216-1217 (probe and the other atom readers refuse a partial atoms file). No doc line describes the wording of this refusal.
  • Changelog fragment (Fixed).

Risk

Only the text of an error changes, and only for a partial atoms file of those two reasons whose message is absent, blank or not text; the exception type and every other refusal are the same. A script that matched the old empty slot would no longer match; a real message is printed exactly as before.

Testing

On a Linux review host, at d79f9b3 (on main b768023):

  • make ci-local: 34 passed, 0 failed, 14 skipped (docker external-catalog smoke, pinned Claude Code install, jobs that do not run on a PR, the matrix legs, conda-forge, atheris fuzz, semgrep, the CLI-surface package step). 22561 tests passed, 19 skipped.
  • On main's code the new test file fails 16 of its 29 cases (every no-message case of both reasons); the first assertion that fails is the exact refusal sentence, which has an empty or junk slot. The real-message and non-text-reason cases pass on main, as they pin behaviour that must not change.
  • Mutants, each failing the new tests: dropping the fallback (16 failed), swapping the two reasons' fallbacks (16 failed), loosening the visible-character test to bool(message) (6 failed: spaces, whitespace and zero-width), dropping the isinstance(reason, str) guard (2 failed, TypeError on a list reason), always using the fallback (3 failed: the real-message cases and the older no price for m test).
  • The Test Linux Wheels Python 3.11 failure of tests/test_repo_batch_lifecycle.py::test_ctrl_c_makes_the_repo_run_process_exit_nonzero is a test this PR does not touch; run alone on this branch it passes, and Flaky on CI: tests/test_repo_batch_lifecycle.py::test_ctrl_c_makes_the_repo_run_process_exit_nonzero #871 tracks its flakiness.
  • Not run: Python 3.11, 3.12 and 3.14 (the matrix legs), Windows and macOS.

Open questions

None.

Revert

A plain revert restores the old refusal text (an empty slot for a message-less block). Nothing is stored.

Checklist

  • Tests added/updated and pytest passes
  • ruff, mypy, and schema checks pass
  • Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
  • Self-reviewed per "Before opening a PR" in AGENTS.md
  • Each new raise/exit and test names the doc, contract or issue it traces to
  • No ad-hoc run artifacts committed
  • New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

…top (intel#833)

load_atoms_record printed the partial block's message as it was for a no_price or permanent_error stop, so a block with no message was refused as "is a partial atoms file: , so it is not a finished atom set", and a blank, zero-width or non-text message (5, a list) gave a blank or junk slot.

The refusal sentence now lives in novelty/atoms_refusal.py. When the message is not a string with a printable, non-space character it names the stop instead: "a call with no known price stopped the run" for no_price, "a permanent provider error stopped the run" for permanent_error. A real message is printed as before, and the spend_cap text and the advice sentences are unchanged. The text is atoms' own, since an atoms file is not a measurement and the advice already follows the slot.

The move also takes the function back under the 800-line cap.

Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
@antonwolfy
antonwolfy requested a review from a team as a code owner November 17, 2026 17:23
@antonwolfy antonwolfy self-assigned this Nov 17, 2026
@exai-bot

exai-bot commented Nov 17, 2026 •

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 4 selected item(s).

CI at `d79f9b3`: 1 failing, 24 passing, 3 skipped

CI has one failing check: the Linux wheel Python 3.11 suite reports an interrupt/cancellation timing failure in unchanged repo-batch code; 22,641 tests passed. The diff only changes partial-atoms refusal wording and its tests, so the observed failure is consistent with a scheduling race rather than a regression from this change. Three other checks were intentionally skipped by workflow...

Check Status Assessment
Test Linux Wheels Python 3.11 failing at step Install the wheel and run ci/test.py Infrastructure or flaky. The SIGINT test sends an interrupt from a worker and immediately continues running cells, while the main thread must handle KeyboardInterrupt and cancel queued futures. Cells r3 and r4 completed before cancellation in this run, exposing a timing race in the unchanged test/runner; the changed...
Review details
  • Commit reviewed: d79f9b3 against merge base b768023
  • Coverage: 4 files in 1 groups
  • Engine: OpenCodeReview v1.13.5-exai.4.8f73eada, us.openai.gpt-6.1-sol
  • Usage: 39k tokens (17k from cache), 8s
  • Tool calls: 5 (4 file_read, 1 ci_status)
  • Atoms partial-message fallback and refusal handling: 4 files

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I checked bc0e6d5. The fix is right: a no_price or permanent_error block whose message is missing, blank, zero-width or not a string now names its stop instead of leaving an empty slot, and a real message and every other reason read as before.

Merge: safe now: it merges cleanly into current main, 1 commit behind (#724, which touches only repos/), and no other open PR touches its files.

What I ran, at bc0e6d5:

  • make ci-local --offline: 22,508 passed, 0 failed, 72 skipped; all static checks pass.
  • The new test file on the merge-base: the 16 fallback cases fail there and pass here; the real-message and non-text-reason cases pass on both, as they pin the unchanged behaviour.
  • The four failing GitHub jobs (Linux Python 3.11/3.14, Tests py3.11/3.14) are not from the PR: their annotation is "The self-hosted runner lost communication with the server". A re-run should clear them.

Checked, no issues

  • Behaviour kept: the --max-usd sentence and the two advice texts are unchanged; the isinstance check runs before the _STOPS lookup, so a list or dict reason cannot raise TypeError there, and the new test covers it.
  • New module: atoms.py was at the 800-line limit, so the move to atoms_refusal.py is what the growth rule asks for; it has the SPDX header.
  • Writers: _stop_at_cap always writes a string message, so the fallback is only for truncated or hand-edited files, as the changelog says.
  • Changelog: the fragment is under Fixed and says what the user now sees.

@antonwolfy

Copy link
Copy Markdown
Contributor Author

(approved) LGTM

@vlad-perevezentsev (#877 (review)): Approval with no change requested. No commit was added this round; the branch stays at bc0e6d5.

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Ready to approve · reviewed bc0e6d5

Next: nothing required from @antonwolfy. 1 nit optional.

Follow-ups, not blocking

  • dokimi/dokimi/novelty/atoms.py:790 – A message of only U+3164, U+115F or U+0301 counts as visible, so the refusal prints it as the slot and the slot is blank (main has it too).

Verified on bc0e6d5: ✅ changed tests: 29 passed, fail on main · ✅ full suite with main merged in: 22568 passed · ✅ merges onto main · ✅ 4 broken variants of the changed code: 4 caught by the tests · 1 mutation tried · 4 edge inputs tried

8 checks behind it, each with its evidence
Area What holds How
Blank message Messages of empty, spaces, a tab and newline, CR and CRLF get the no_price stop text on the head; main prints them raw, as the slot, with the advice ran it, edge input (text)
Real message Messages " bad key ", straße ✓, a 5000-character line and a two-line message print as they are, with the same sentence as on main ran it, edge input (text)
Non-text input A message of 5, ["x"] or a lone surrogate gets the permanent_error stop text; a reason of ["no_price"], {"a": 1} or 3 still gives the --max-usd cap sentence, and both raise InputError ran it, edge input (wrong-type)
Partial refusal tests The 29 cases of tests/test_atoms_partial_status.py pass on the head tests on the head
Partial refusal tests on main On main 16 of the 29 cases fail: the no-message cases of both reasons. The real-message and non-text reason cases pass there, as the PR intends same tests on main
Refusal mutants Negating the visible-character test on atoms_refusal.py:26 makes 19 of the 29 cases fail, so the tests pin that line broke the code, tests caught it
Full local ci The whole local CI passes with main merged in, with no new failures full suite, main merged in
Merge The PR merges onto current main with no conflict merge onto main

Comment thread dokimi/dokimi/novelty/atoms_refusal.py Resolved
… non-empty slot

Settles the review nit on atoms_refusal.py: a message of only U+3164 is printable and not a space, so the check keeps it and the slot looks blank. The docstring now says only what the check covers.

Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
@antonwolfy

Copy link
Copy Markdown
Contributor Author

(commented) 🟢 Ready to approve · reviewed bc0e6d5

@vchamarthi (#877 (review)): The review's one follow-up (a message of only U+3164, U+115F or U+0301 counts as visible) is the nit in thread 4603801781: I reproduced it for U+3164, fixed the docstring in d79f9b3 and track the check in #881.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.79%. Comparing base (b768023) to head (d79f9b3).
⚠️ Report is 8 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #877      +/-   ##
==========================================
+ Coverage   95.76%   95.79%   +0.03%     
==========================================
  Files         429      430       +1     
  Lines       56714    56830     +116     
  Branches     9766     9784      +18     
==========================================
+ Hits        54310    54442     +132     
+ Misses       1494     1485       -9     
+ Partials      910      903       -7     
Files with missing lines Coverage Δ
dokimi/dokimi/novelty/atoms.py 98.79% <100.00%> (-0.01%) ⬇️
dokimi/dokimi/novelty/atoms_refusal.py 100.00% <100.00%> (ø)

... and 19 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@antonwolfy

Copy link
Copy Markdown
Contributor Author

Ready for another look at d79f9b3; failed: Test Linux Wheels Python 3.11.

Declined, with the evidence in the reply:

Follow-up issues opened:

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not approved yet · re-reviewed d79f9b3

Next: @antonwolfy: settle 1 open question (below). 1 nit optional.

Blocking

  • Open question for @antonwolfy: The two fallback sentences are new wording. Cite the doc or issue line that asks for these sentences, or list the wording as an open question in the PR description.

Verified on d79f9b3: ✅ changed tests: 29 passed, fail on main · ✅ full suite with main merged in: 22849 passed · ✅ merges onto main · ✅ 4 broken variants of the changed code: 4 caught by the tests · 1 mutation tried · 5 edge inputs tried

9 checks behind it, each with its evidence
Area What holds How
Stop text for unusable text partial_atoms_refusal returns the reason's stop text for a no_price message that is empty, spaces only, CR, CRLF or zero-width, with the advice sentence after it ran it, edge input (text)
Non-text values A permanent_error message that is not text (5, a list, a dict, null) gets the permanent-error stop text, and a reason that is not text (a list, a dict, 3, null) or spend_cap gets the cap sentence ran it, edge input (wrong type)
Real text kept A real no_price message of ß, a two-line CRLF message or 300 characters is printed as it is, with the same sentence main gives ran it, edge input (text)
Bytes that are not utf-8 An atoms file with the byte \xff in its message is refused before the partial check, as is not UTF-8 text, on the head and on main ran it, edge input (text)
Visible-character check pinned Negating the not any( printable-character check in atoms_refusal.py makes 11 of the 29 cases of tests/test_atoms_partial_status.py fail, so the tests pin that line broke the code, tests caught it
No-message cases On the head, tests/test_atoms_partial_status.py passes: 29 passed, including the 16 no-message cases of both reasons tests on the head
Main behaviour On main 16 of the 29 cases fail, every no-message case of both reasons; the real-message and non-text reason cases pass on main, so the advice sentences and real-message text are unchanged same tests on main
Full suite make ci-local passes with main merged in: 22849 passed, 0 failed full suite, main merged in
Merge The PR merges onto main with no conflict merge onto main

@@ -0,0 +1,3 @@
### Fixed

- **A partial atoms file with no usable message now says why the run stopped** — for a `no_price` or `permanent_error` stop, the refusal of `load_atoms_record` printed the block's `message` as it was, so a block with no message gave an empty slot (`is a partial atoms file: , so it ...`) and a blank, zero-width or non-text message gave junk. It now names the stop (`a call with no known price stopped the run`, `a permanent provider error stopped the run`) when the message is not text with a visible character; a real message is printed as before. (#833)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: The changelog says a message with no visible character gets the stop text, but U+3164 alone is kept and prints blank.

Input {"reason": "no_price", "message": "ㅤ"} (U+3164 only). On the head it gives ... file: ㅤ, so it is not a finished atom set., a blank slot. Expected: the fragment states the check the code runs, as the docstring now does (printable, non-space character). The check itself stays; #881 tracks it.

Fix, run on d79f9b3: the quoted text is gone; tests are no worse.

Suggested change
- **A partial atoms file with no usable message now says why the run stopped** — for a `no_price` or `permanent_error` stop, the refusal of `load_atoms_record` printed the block's `message` as it was, so a block with no message gave an empty slot (`is a partial atoms file: , so it ...`) and a blank, zero-width or non-text message gave junk. It now names the stop (`a call with no known price stopped the run`, `a permanent provider error stopped the run`) when the message is not text with a visible character; a real message is printed as before. (#833)
- **A partial atoms file with no usable message now says why the run stopped** — for a `no_price` or `permanent_error` stop, the refusal of `load_atoms_record` printed the block's `message` as it was, so a block with no message gave an empty slot (`is a partial atoms file: , so it ...`) and a blank, zero-width or non-text message gave junk. It now names the stop (`a call with no known price stopped the run`, `a permanent provider error stopped the run`) when the message is not text with a printable, non-space character; a real message is printed as before. (#833)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

load_atoms_record: a no_price partial atoms file with no message is refused with an empty slot

5 participants