Repository navigation
fix(repo): refuse a --cache-dir that is not a usable directory with exit 2 - #828
Conversation
…xit 2 (intel#765) A `repo` command given a `--cache-dir` that names a regular file (or a path below one) reached `ensure_clone`, whose `mkdir` raised `FileExistsError`; the command printed `✗ [Errno 17] File exists: '<path>'` and exited 1, a failed run in `--json`, without naming the flag. `usable_cache_dir` moves from `commands/novelty_run.py` to the new `commands/cache_dir.py` (still importable from `novelty_run`) with a parameter for the closing words of its message: the novelty text is unchanged, and the repo commands say "pass a directory the repository clones can be cached in". `_cache_dir` in `commands/repo.py` calls it, and `_parse_source` calls it so every command that parses a source refuses a bad value first, also with a local-directory source, which never used the cache. InputError is a ValueError, so the clone handlers of `fetch`, `reach` and `_source_tree`, which were three copies, are one `_open_tree` that reads the flag before its `try`. Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 5 selected item(s). CI at Review details
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #828 +/- ##
==========================================
- Coverage 95.79% 95.75% -0.04%
==========================================
Files 419 424 +5
Lines 55879 56403 +524
Branches 9604 9714 +110
==========================================
+ Hits 53527 54007 +480
- Misses 1452 1487 +35
- Partials 900 909 +9
🚀 New features to boost your workflow:
|
napetrov
left a comment
There was a problem hiding this comment.
Change class: not computed (no receipt profile for this repo, or receipt.py failed)
Verdict: approve. No blocking findings; one question and two nits. Head a383e96, merge base b3225e1, lane L2. Ran: a probe of usable_cache_dir on head with an empty read-only directory (non-root uid, so os.access is real); the passes ran the cache-dir edge cases (probes/edges.sh, output in probes/edges.out), which I did not re-run.
CI: 0 failing Actions jobs on head a383e96; merge base is behind base tip (not a red).
| # | Finding | Label | Provenance | Scope | Probe | Expected |
|---|---|---|---|---|---|---|
| 1 | An empty read-only --cache-dir passes the check; the clone then fails with exit 1 |
question | - | new surface | probes/edges.out | the Summary's "refuse a directory that is not usable" vs "also a read-only one" |
| 2 | _parse_source docstring lists the exit-2 cases without a bad --cache-dir |
nit | introduced | local | grep on head | the docstring matches the code |
| 3 | Changelog fragment has no author segment | nit | introduced | local | - | <timestamp>_<author>_<branch>.md |
Raised before
No prior row makes a claim (the bot summary and the coverage notice are dropped).
Design
None.
Coverage
Checked: whole-PR and edge passes; the cache-dir edge cases (unset, empty, regular file, child of a file, trailing slash, existing writable and read-only directories, missing dir under a writable and an unwritable parent, non-ASCII path, symlink loop, long name) per the passes' probes/edges.out (they report the same result on base and head or what the body announces); a read-only probe on head. Not checked: the full pytest suite, mypy, the gate scripts that diff against origin/main, root-run permission cases (as root os.access returns True), Windows.
Follow-up issues
_SOURCE_EXITincommands/repo_run.pyand therepo run/eval checkhelp text still list the old exit-2 cases (finding 2, outside this PR's diff).
vchamarthi
left a comment
There was a problem hiding this comment.
🟡 Not approved yet · reviewed a383e96
Next: @antonwolfy: answer 3 open threads from napetrov (below). 1 nit optional.
Blocking
- Open question, not answered: thread by @napetrov –
usable_cache_diraccepts an empty read-only--cache-dir(cache_dir.py:43);repo fetchon it exits 1 withgit clone failed: ... Permission denied, as main does, while the title and the_cache_dirdocstring say such a value is refused. - Still holds on this head: thread by @napetrov – The
_parse_sourcedocstring (repo.py:43) still omits the--cache-dirusage error thatrepo.py:53now raises. - Still holds on this head: thread by @napetrov – The fragment
20261116_150000_repo-cache-dir-usable.mdhas no author segment, althoughchangelog.d/README.mdnames fragments<timestamp>_<you>_<branch>.md.
Verified on a383e96: ✅ changed tests: 9 passed, fail on main · ❌ full suite with main merged in fails at tests/test_fuzz_targets.py::test_a_bare_deep_reply_is_a_parse_error_not_a_crash[extract_json_array-text0], ci.yml:test / Run tests · ✅ merges onto main ·
12 checks behind it, each with its evidence
| Area | What holds | How |
|---|---|---|
| Refusal exit code | repo validate, repo inspect, repo budget and eval tasks generate --from prs given a regular file as --cache-dir exit 2 with Error: --cache-dir <file> is not a directory; pass a directory the repository clones can be cached in; main exits 1 for the same input |
ran it, edge input (cli) |
| Changed tests | The nine cases in tests/test_repo_cache_dir_usable.py pass on the head: a regular file, a child of one, and a bad flag next to a local-directory scan exit 2 naming the flag for repo fetch, scan, reach, eval check and tasks generate --from replay; the _cache_dir cases pass through an unset value and an existing or missing directory |
tests on the head |
| Main result | On main 8 of the 9 cases in tests/test_repo_cache_dir_usable.py fail, so they pin the change |
same tests on main |
| Unset value | _cache_dir returns None for an unset --cache-dir and for an empty one; the _cache_dir case of the changed tests pins both |
tests on the head, edge input (none) |
| Paths | On the head a regular file, a child of one, a dangling symlink and a 300-character name are refused with InputError naming the flag and the path, and a missing directory under a writable parent is returned and not created |
ran it, edge input (paths) |
| Text values | usable_cache_dir(" ") returns Path(" "), and a non-ASCII name under a writable parent is returned unchanged; both are taken as literal paths |
ran it, edge input (text) |
| Permissions | A missing directory under a read-only parent, an existing directory without read permission and a path below a directory without search permission are each refused with is not writable, is not readable and cannot inspect (cache_dir.py:43, :33) |
ran it, edge input (permissions) |
| Structured output | repo fetch --json with a regular file as --cache-dir reports status failed and the flag's message, not exit 1: see stderr |
tests on the head, edge input (structured) |
| Clone failure exit | A clone failure in _open_tree (repo.py:64 to 68) prints its message and exits 1; the raise that the mutant adds at repo.py:67 follows sys.exit(1), so that variant behaves the same |
ran it, edge input (cli) |
| Refusal pinned | Dropping the _cache_dir(args) call in _parse_source (repo.py:53) fails the changed tests, so the refusal before eval check and tasks generate is pinned |
broke the code, tests caught it |
| Novelty text | novelty_run.usable_cache_dir gives the same refusal text as before the move: pass a directory the retention verdicts can be stored in |
ran it |
| Merge | The PR merges onto main with no conflict | merge onto main |
… by author, pin a usable cache dir Settles the review threads on the PR. An existing read-only --cache-dir stays accepted on purpose (a populated cache replays without writing); the `_cache_dir` docstring and the changelog fragment now say that an empty read-only one fails at the clone with exit 1. The `_parse_source` docstring lists the --cache-dir usage error. The fragment is renamed to carry the author segment, as changelog.d/README.md names it. New test: `repo fetch` of a pinned file:// source under a writable --cache-dir prints the SHA, so `_open_tree` returning no tree fails a test. Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
@napetrov (#828 (review)): An approval with one question and two nits; they are the threads 4591749039, 4591749050 and 4591749057, answered above. |
@vchamarthi (#828 (review)): The review lists the open threads 4591749039, 4591749050 and 4591749057 and the |
|
Ready for another look at Declined, with the evidence in the reply:
|
* docs(formal): describe the batch result write as the code does it Settles review point B1.1 of #628: the formal-model README and the RepoBatch.tla comment said the batch result is written with mkstemp + os.replace, but the code writes a `<name>.<pid>.<thread>.tmp` sibling of the target and then calls os.replace (artifacts.py). Only the README row and the TLA+ comment change; no state or action of the model is touched. Signed-off-by: Anton Volkov <antonwolfy@gmail.com> * fix(cli): list a bad --cache-dir among the exit-2 cases of repo run and eval check Settles the review of #828 that left the exit-2 text stale: `_SOURCE_EXIT` (the help of `repo run` and `eval check`), the `tasks generate` help and the repository-evaluation contract now name a `--cache-dir` that is not a usable directory, which these commands refuse with exit 2. A test pins the phrase in the help of both commands. Signed-off-by: Anton Volkov <antonwolfy@gmail.com> * test(repos): pin the batch label of the mixed-execution refusal Settles review point B2.1 of #628: nothing pinned the `{batch or where}` label that starts the refusal in refuse_mixed_executions, so replacing it with `{where}` kept the suite green. The new test checks that the message starts with the batch name the cells record, and with `where` when they record none. Adds the changelog fragment for this and the help text. Signed-off-by: Anton Volkov <antonwolfy@gmail.com> * fix(docs): restore mkstemp wording; name the --cache-dir exit 2 in two more docs Settles the review threads on #856: the formal-model README row and the RepoBatch.tla comment describe the write of result.json as mkstemp + os.replace again, since the runner writes through batch_ledger.write_json, which uses mkstemp. The --from graph section of the repository-evaluation contract (§4.7) and the exit-code paragraph of the quickstart now list a --cache-dir that is not a usable directory among the exit-2 cases. The help test also covers the eval tasks listing that holds the generate description, and the changelog fragment drops the bullet about the pinned mixed-execution label, which changes nothing a user sees. Signed-off-by: Anton Volkov <antonwolfy@gmail.com> --------- Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
Summary
repocommands pass--cache-dirtoopen_source, which creates it withmkdiron the first clone. Onmain,python cli.py repo fetch file:///tmp/nothere@HEAD --cache-dir /tmp/cachefile, where/tmp/cachefileis a regular file, prints✗ [Errno 17] File exists: '/tmp/cachefile'and exits 1 (with--json:"status": "failed","reason": "exit 1: see stderr"). A bad flag value reads as a failed run, and the message does not name--cache-dir. The same happens for a path below a regular file, and forscan,reach,validate,inspect,budget,eval checkandeval tasks generate --from replay|prs.Now every one of them exits 2 before any clone with
Error: --cache-dir /tmp/cachefile is not a directory; pass a directory the repository clones can be cached in(--jsonreports a failed status with that message). Fix option taken: validate the flag up front with the existingusable_cache_dircheck, as #766 did for the novelty cache; it fails before any clone starts. CatchingOSErrorinensure_clonewas not taken: it would editrepos/source.py, which open PR #724 rewrites.Closes #765
The same cause in
--cache-root/--cache NAME=PATHofeval checkandrepo runis tracked in #826, and inDOKIMI_REPO_CACHEin #827.Changes
usable_cache_dirmoves fromcommands/novelty_run.pyto the newcommands/cache_dir.py, with a parameter for the closing words of its message. The novelty callers keep their text byte for byte (novelty_runre-exports the function); the repo commands say "pass a directory the repository clones can be cached in"._cache_dirincommands/repo.pycalls it, so a regular file, a child of one, an unreadable existing directory or an unwritable parent is a usage error (2) naming--cache-dirand the path._parse_sourcecalls it as well, so the commands ofrepo_run.py(which handleOSErrorandValueErroras exit 1 around the clone) refuse the flag first.InputErroris aValueError, so the clone handlers offetch,reachand_source_treewould have turned it into exit 1. Those three copies of the same handler become one_open_tree, which reads the flag before itstry. This also takescommands/repo.pyfrom 800 lines to 785.tests/test_repo_cache_dir_usable.py.--cache-diris accepted on purpose and the_cache_dirdocstring and the fragment say so: a populated one replays without writing (repo fetchof a cached pinned source printed its SHA), an empty one fails at the clone withgit clone failed: ... Permission deniedand exit 1, since the check does not know the clone URL.Risk
A bad
--cache-dirnext to a local-directory source (never cloned) now also exits 2, where it used to be ignored. That is intended: a bad flag value is an input error, as in the novelty commands. A missing cache directory under a writable parent, and an existing directory (also a read-only one) still pass through unchanged. Novelty messages and exit codes are unchanged (tests/test_novelty_cache_dir_usable.pypasses without edits). Open PR #489 edits other parts ofcommands/novelty_run.py(the judge-panel arguments), not the lines this PR removes, so the two merge without a conflict.Testing
On a Linux review host, at
e8ac28d(onmainb3225e1):make ci-local: 34 passed, 0 failed, 14 skipped (docker and network steps, the other Python versions, conda-forge, fuzz, semgrep, jobs that do not run on a PR). 22068 tests passed, 19 skipped.main's code the new test file gives8 failed, 1 passed; the first failure is the exit code,assert 1 == 2, forrepo fetchwith--cache-dirnaming a regular file._cache_dirwithout the check (6 failed), the repo message with the novelty wording (3 failed), the flag read inside the clone'stry(4 failed),_parse_sourcewithout the check (the 2evalcommand tests fail), a missing child returned as unset (1 failed), an unset flag returned as a path (1 failed), a child of a file reported as the file itself (1 failed),_open_treereturning no tree (1 failed:repo fetchof a pinnedfile://source under a writable--cache-dirprints its SHA).Open questions
None.
Revert
A plain revert restores the old behaviour (exit 1 with the bare
[Errno 17]message) and movesusable_cache_dirback intonovelty_run.py. Nothing is stored.Checklist
pytestpassesruff,mypy, and schema checks passAGENTS.mdraise/exit and test names the doc, contract or issue it traces to.github/.licenserc.yaml)