Repository navigation
fix: skip binary and non-UTF-8 files opened as source text - #429
Open
vibgrate-team wants to merge 1 commit into
Open
vibgrate-team wants to merge 1 commit into
vibgrate-team wants to merge 1 commit into
Conversation
Walks for vg build and vg scan no longer decode a blob just because its name looks like source. Those files are skipped, one stderr notice names them and shows a vg --exclude example, and the bytes stay out of the map and the scan artifact. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: vibgrate-team <vibgrate-team@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
vg buildandvg scanno longer treat a binary, media, or non-UTF-8 file as source text just because it sits under the walk or has a source-like name (payload.js,README.md).Those bytes are not decoded. The file is left out of the code map and out of scan text. One stderr notice names the count and the first few root-relative paths, and shows how to ignore the file. The notice never includes file contents. A file already covered by
.gitignoreor--excludestays quiet. Valid UTF-8 source, including non-ASCII text, is still read. This is separate from the filesystem-root guard and from symlink handling.When
vg scanalso builds the code map, that build can print its own notice for the same paths. Each line is still one summary.How to verify
The fixture writes a small binary blob (PNG-like header, a NUL, invalid UTF-8, and an ASCII marker) to
payload.jsandREADME.md, plus a realsrc/keep.tsand alogo.png. It asserts:buildGraphdo not throwkeep, not the blob.gitignoreand--exclude 'payload.js'omit the file from the noticerunCoreScanfinishes, and the artifact does not contain the markerRelated issues
Closes #294
Checklist
pnpm testpassespnpm lintis cleanpnpm typecheckis cleangraph.json/ report output (content-hashed IDs, stable sorts; no time, randomness, or filesystem-order dependence)git commit -s, DCO)Notes for reviewers
Known media extensions were already dropped from the scan walk. This change is the case where the file would have been opened as text: a misleading extension, or a context file such as
README.md. A NUL, a UTF-16 BOM, or any non-UTF-8 sequence is enough to skip. Files above the per-file cap are judged on an 8 KiB prefix so a large blob is not loaded just to be refused.