Repository navigation
fix: docling reruns replace their files, keyword bullets kept, sidecar rm checked #876
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
vlad-perevezentsev
wants to merge
14
commits into
intel:main
Choose a base branch
from
vlad-perevezentsev:fix/687-low-outside-repos
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+508
−105
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
ba1e5ef
fix(personas): keep a use-case bullet that names the section keyword
vlad-perevezentsev 8757142
fix(harness): a sidecar whose removal failed is not marked stopped
vlad-perevezentsev 544fbef
fix(ingest): a docling rerun replaces its own files instead of copying
vlad-perevezentsev 206c379
fix(ingest): key a docling source by its path, not its link target
vlad-perevezentsev 586fc1c
fix(ingest): a docling source whose name holds a backslash imports again
vlad-perevezentsev 0f07d27
fix(ingest): a new docling source skips names the record gives others
vlad-perevezentsev 34c0729
docs(changelog): a failed sidecar removal errors the cell, not as a t…
vlad-perevezentsev 11bf40b
docs(artifacts): name docling_sources in the module's list of kinds
vlad-perevezentsev 24538a1
refactor(personas): read the README once in the PyGithub analysis
vlad-perevezentsev 83c1a9a
fix(ingest): a docling name whose file and source are gone is free again
vlad-perevezentsev dc147f4
fix(ingest): find a recorded docling source when --out-dir is a link
vlad-perevezentsev b0374a3
fix(ingest): refuse a directory at the docling record, match names by…
vchamarthi b3534d2
Merge origin/main into fix/687-low-outside-repos
vchamarthi 1965fdd
fix(ingest): refuse a directory at the docling record, match names by…
vchamarthi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
22 changes: 22 additions & 0 deletions
22
dokimi/changelog.d/20261117_120000_vlad-perevezentsev_687-low-reruns.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| <!-- | ||
| Copyright (C) 2026 Intel Corporation | ||
| SPDX-License-Identifier: Apache-2.0 | ||
| --> | ||
|
|
||
| ### Fixed | ||
|
|
||
| - **`data import docling` reruns replace their own files** — a rerun into the same `--out-dir` | ||
| wrote `<stem>-<ext>-2.md` beside the file the first run wrote, so the docs dir filled with | ||
| copies. The output directory now keeps `.dokimi-docling.json` (`docling_sources.v1`), which | ||
| names each file's source; a rerun replaces that file atomically, and another source with the | ||
| same name still gets the next number, also across runs. A file the record does not name and a | ||
| link are still never written over. A directory converted before this change gets one more | ||
| numbered copy on its first rerun, then stays put (#687). | ||
| - **`eval personas discover` keeps a use-case bullet that names its section** — a README bullet | ||
| such as `- Web applications` under `## Use cases` was read as a section header and dropped, | ||
| and the same bullet outside a use-case section opened one (#687). | ||
| - **Docker harness cells: an egress sidecar whose removal failed is no longer taken as gone** — | ||
| a `docker rm` of the sidecar that exited non-zero still marked it stopped, so the verifier ran | ||
| beside it and teardown did not try the removal again. The cell now errors before the | ||
| verifier runs, as for any failed docker step, and teardown retries the removal before it drops | ||
| the network (#687). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Must-fix — case-sensitive source identity breaks Windows reruns. Trigger: on a case-insensitive Windows filesystem, import
raw/manual.pdf, change its contents, then import the same file asRAW/MANUAL.PDFinto the same output directory. Evidence:_source_id()preserves those spellings, this comparison rejects the existing owner, and the casefolded filename reservation allocatesMANUAL-pdf-2.md; the caller does not normalize casing. Impact: the old Markdown remains alongside the updated conversion, so downstream corpus ingestion includes stale, duplicate content instead of replacing the source's file. Normalize source identities/comparisons using Windows path semantics (including the ownership cleanup at line 99), without resolving symlinks, and add a regression for case-varied source paths. Workaround: use identical source-path casing on every import. Severity: medium, a bounded rerun-contract failure involving regenerable Markdown, not loss of original documents.