Repository navigation
fix(repo): refuse a --cache-root, --cache or $DOKIMI_REPO_CACHE that is not a usable directory with exit 2 - #891
antonwolfy wants to merge 2 commits into
Conversation
…ctory with exit 2, before the image build (intel#826) `eval check` and `repo run` made each cache directory with `mkdir(parents=True, exist_ok=True)` after the image build, so a `--cache-root` or `--cache NAME=PATH` naming a regular file (or lying below one, or a `--cache-root` holding a regular file named after a cache) ended in `✗ [Errno 20] Not a directory` with exit 1, and the message did not name the flag. `_context` now checks those paths with the shared `usable_cache_dir` right after the manifest is known and before the export and the build, and raises `InputError` naming the flag, the path and what to pass, so the command exits 2. `usable_cache_dir` takes the name of the setting as a `label` keyword (default `--cache-dir`, so every existing message is unchanged). A `--cache` naming no cache of the manifest keeps its existing message. `repo run` and `eval check` let the `InputError` through their `ValueError` handler instead of printing it as a failed run. Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
… with exit 2 naming the variable (intel#827) With no `--cache-dir`, `default_cache_dir()` returned `$DOKIMI_REPO_CACHE` unchecked and `ensure_clone` then ran `mkdir(parents=True, exist_ok=True)` on it, so a variable naming a regular file ended in `✗ [Errno 17] File exists` with exit 1, and the message did not name the variable. `_cache_dir` in `commands/repo.py` now checks a non-empty `$DOKIMI_REPO_CACHE` with `usable_cache_dir` when `--cache-dir` is unset, labelled `$DOKIMI_REPO_CACHE`, so every repo command that clones (they all go through `_cache_dir`) exits 2 before any clone, with the variable and the path in the message. A `--cache-dir` that is given wins and the variable is not looked at; an empty variable counts as unset; `repos/source.py` is untouched. Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 11 selected item(s). CI at `18360fd`: 6 failing, 24 passing, 3 skipped
Review details
|
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #891 +/- ##
==========================================
- Coverage 95.79% 95.78% -0.02%
==========================================
Files 429 430 +1
Lines 56842 57164 +322
Branches 9787 9858 +71
==========================================
+ Hits 54452 54754 +302
- Misses 1487 1496 +9
- Partials 903 914 +11
🚀 New features to boost your workflow:
|
| With no ``--cache-dir``, a non-empty ``$DOKIMI_REPO_CACHE`` is checked the same way and returned, so a bad | ||
| variable is a usage error naming it (#827); a flag that is given wins and the variable is not looked at. | ||
| An existing read-only directory passes on purpose: a populated one replays without writing. An empty one | ||
| is not told apart without the clone URL, so its clone fails later with exit 1. |
There was a problem hiding this comment.
CI check Test Linux Python 3.11 fails: The seven-line addition to repo.py pushes the CI tree over the production file-length cap; the sole failed test is the growth ratchet. Proposed fix, not run by the bot.
CI check Test Linux Python 3.14 fails for the same reason.
CI check Test Linux Wheels Python 3.11 fails for the same reason.
CI check Test Linux Wheels Python 3.14 fails for the same reason.
CI check Tests (py3.11) fails for the same reason.
CI check Tests (py3.14) fails for the same reason.
| With no ``--cache-dir``, a non-empty ``$DOKIMI_REPO_CACHE`` is checked the same way and returned, so a bad | |
| variable is a usage error naming it (#827); a flag that is given wins and the variable is not looked at. | |
| An existing read-only directory passes on purpose: a populated one replays without writing. An empty one | |
| is not told apart without the clone URL, so its clone fails later with exit 1. | |
| Without the flag, non-empty ``$DOKIMI_REPO_CACHE`` is checked and returned; a bad value is a usage error naming it (#827). The flag wins. | |
| Read-only directories pass for cached replay; an empty one cannot be detected without the clone URL and fails later with exit 1. |
Summary
Two unusable cache paths still reached an unguarded
mkdir, the cause #765 fixed for--cache-dir. Onmain:maineval check <repo> --cache-root <regular file>(andrepo run)building dokimi/toy:... (tasks removed), then✗ [Errno 20] Not a directory: '<file>/build-cache'eval check <repo> --cache build-cache=<regular file>(andrepo run)✗ [Errno 17] File exists: '<file>'DOKIMI_REPO_CACHE=<regular file> dokimi repo fetch file:///.../nothere@HEAD(andscan,reach,eval check)✗ [Errno 17] File exists: '<file>'Neither message names the flag or the variable, and the image is built before the first two fail. Now each is refused up front with exit 2, naming the setting and the path, e.g.
--cache-root /p/file is not a directory; pass a directory the build caches can be kept inand$DOKIMI_REPO_CACHE /p/file is not a directory; pass a directory the repository clones can be cached in.For #826 the PR takes the first fix option (check with the shared
usable_cache_dirbefore the image build), because it fails before anything is exported or built. For #827 it takes the first option too (check the variable in_cache_dir), because every repo command that clones goes through it and it needs no change torepos/source.py, which open PR #783 edits.Closes #826
Closes #827
Not fixed here, and not the same cause: the three other sites that make a cache directory from a setting (
DOKIMI_RESPONSE_CACHE,XDG_CACHE_HOMEfor the CLI work dir,$HOMEfor the Harbor export cache) already name the problem or are not a dokimi setting; nothing to track.Changes
commands/cache_dir.py:usable_cache_dirtakes a keywordlabel(default--cache-dir) that starts each of its four messages; every existing message is unchanged. New constantCACHE_USE.commands/repo_run.py(eval check and repo run: --cache-root or --cache naming a regular file exits 1 with a bare [Errno 20] after the image build #826):_check_cache_paths(args, manifest)is called in_contextright after the manifest is known and before the export and the image build. It checks--cache-root, each--cache NAME=PATHfor a cache the manifest declares (label--cache NAME), and<--cache-root>/<name>for each declared cache without an override. A--cachenaming an undeclared cache keeps its existing message.eval checkandrepo runlet theInputErrorthrough theirValueErrorhandler, which would otherwise print it as✗ ...with exit 1.commands/repo.py(DOKIMI_REPO_CACHE naming a regular file makes repo commands exit 1 with a bare [Errno 17] #827):_cache_dirchecks a non-empty$DOKIMI_REPO_CACHEwithusable_cache_dir(label$DOKIMI_REPO_CACHE) when--cache-diris unset or empty. A flag that is given wins and the variable is not looked at.eval check,repo runandeval tasks generate,docs/repo-evaluation-contract.md(the exit-2 paragraph ofeval check/repo runand §9 Running) anddocs/quickstart.mdname the new cases.tests/test_repo_run_cache_paths_usable.py,tests/test_repo_cache_env_usable.py,tests/test_cache_dir_label.py. No existing test changed.Risk
Inputs that were taken before are refused now with exit 2: a
--cache-root,--cacheor$DOKIMI_REPO_CACHEthat is a regular file, below one, or (for--cache-root) holds a regular file named after a cache. These failed later with exit 1 anyway, except for a$DOKIMI_REPO_CACHEnext to a local-directory source or a command that never clones (for examplerepo validateof a local path), which used to ignore the variable and now exit 2, as--cache-dirdoes since #765. A read-only existing directory still passes. The default cache root (no--cache-root) is not checked in_context; it follows$DOKIMI_REPO_CACHE, which_cache_dirnow checks earlier in the same command. The diff touchesrepo_run.pyonly in the helper, its one call, twoexcept InputError: raiselines and one help string, to merge cleanly with the open PRs on that file: #781, #782, #787 and #810 add flags and_run_lockedlines (#810 also edits therepo runexception handlers, next to theexcept InputError: raiseline), and #783 edits_contextaround_require_grade; none changes the lines this diff edits, so a textual conflict is possible only at the handler ofrepo run.Testing
On Linux, at
18360fd(onmaina98a0e7):make ci-local: 36 passed, 0 failed, 14 skipped (steps that need a PR job, micromamba, semgrep, the fuzz run and the other-Python matrix legs). 22751 tests passed, 19 skipped; changed-line coverage 100% (25 lines); mutation of changed lines: 4 of 4 killed.main's code the new tests fail where the bug is:test_repo_run_cache_paths_usable.py8 failed, 5 passed (assert 1 == 2: exit 1, not 2, afterbuilding ...),test_repo_cache_env_usable.py6 failed, 2 passed (same),test_cache_dir_label.py3 failed, 1 passed (TypeError: nolabelparameter).Open questions
None.
Revert
A plain revert restores the old behaviour (exit 1 with a bare
[Errno ...], after the image build for the first two). Nothing is stored.Checklist
pytestpassesruff,mypy, and schema checks passAGENTS.mdraise/exit and test names the doc, contract or issue it traces to.github/.licenserc.yaml)