Repository navigation
feat(code-index): index authenticated Git submodules - #4801
Conversation
Keep parent repository source admission unchanged; use one shared walker for exclusion evidence instead of a second filesystem scan. Refs #4593
|
|
Refs #4593 — validation blocker at exact head The repository-gates job failed before Rust execution: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42433144833/job/128586224987 . The actual failures are:
No source fix, Rust test execution, independent review, branch update, merge or issue closure was completed in this follow-up. This environment has no executable repository checkout or Bazel; direct repository retrieval failed, and no usable remote execution/workflow-dispatch capability was exposed. This is a validation limitation, not evidence that the implementation is correct. #4801 remains diagnostic-only and draft. It does not yet satisfy authenticated submodule admission, pinned child-tree capture, child-aware freshness, or consistent retrieval/graph behavior. Retain Coordination: the dedicated #4593 owner remains responsible for implementation; PR maintenance owns CI assistance without racing its branch. Do not repair unrelated decision-model generated paths by importing forbidden work. #4776 and all Kev/decision-model work remain draft and unmergeable by instruction; #4763 remains read-only. No protections or tests were weakened. |
|
Refs #4593 — exact-head CI repair The PR is still partial: it adds skipped-nested-repository diagnostics and regression source, but NOT authenticated submodule indexing, freshness, or indexed retrieval. HOLD MERGE even if its current-head diagnostics tests pass: keep this same PR open to complete #4593. Being reviewable or passing preliminary tests does not mean the issue's acceptance is met. No complete-implementation, native macOS or independent-review proof exists. Keep |
Limit skipped-repository samples during traversal, honor directory policy before collecting diagnostics, and quote bounded path labels. Propagate source-walk failures instead of returning an empty successful audit. Add behavioral coverage for bounded sampling, policy scope, preserved ecosystem caveats, Unicode/control-character paths, and missing roots. Refs #4593
|
Refs #4593 — published source repair Fixed unbounded skipped-repository collection, diagnostics escaping configured directory exclusions, unquoted/unbounded path labels, and swallowed filesystem-walk failures. Added six behavioral regression cases (one Unix-only filename case); existing initialized-submodule and linked/nested-repository exclusions are preserved. Only Current-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42442123433 — queued when inspected. These Rust regressions have NOT executed; author diff review is NOT independent review. The prior successful draft run skipped Rust/Clippy/native jobs, so it is not acceptance evidence for these changes. The existing merge hold remains. This repairs the current audit implementation, not the still-unimplemented optional authenticated submodule indexing feature. Keep the PR draft/open and #4593 open. Continue the integrated admission/capture/freshness/retrieval implementation here rather than opening a duplicate or treating more diagnostic fixes as feature completion. |
|
Refs #4593 — published nonforce, expected-head fast-forward |
Use one source boundary for interactive walks, Git candidate classification, worktree capture (including ignored-file admissions), and freshness. Parent tracked files hidden by a new nested .git marker become tombstones, invalidate clean HEAD-tree reuse, and return when the marker disappears. Reject source paths passing through symlinked directories or lexical escapes. Add real Git classification and filesystem regressions for nested directory, gitdir-file, symlink, path traversal and removal behavior. Refs #4593
|
Refs #4593 — published source repair |
|
Refs #4593 — published exact-head CI repair |
Sample .git marker evidence for tracked and explicitly admitted source ancestors even when Git dirwalk never recurses into tracked-only folders. Test marker creation and removal against a real Git parent index. Refs #4593
Directory metadata changes when the marker is removed, so the original cached roster must not be treated as restored merely by marker absence. Refs #4593
A registered gitlink remains an untrusted nested source boundary even if its child .git marker is absent. Keep live grep and audit in default-deny parity with the parent index; include/glob cannot lift repository trust. Exercise a real registered Git submodule with its marker removed. Refs #4593
|
Refs #4593 — published narrow default-deny source fix Current-head run: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42466514013 (queued at check). NO Bazel behavioral Rust pass or independent review claim. Do not merge or mark ready. Feature acceptance remains integrated authenticated opt-in, pinned historical child capture, child status/freshness and cross-boundary graph/index/live retrieval, with nonzero Bazel/native validation. Preserve #4800 and protected Kev/release lanes. |
Apply the two exact rustfmt_check_2024 hunks from hosted run 42466514013. No source behavior changes; authenticated submodule indexing remains pending. Refs #4593
|
Refs #4593 — narrow hosted Bazel formatting repair published at Exact-head run 42466514013, repository-gates job 128687191524, failed only HOLD MERGE: this PR remains DRAFT and contains only exclusion/diagnostic/freshness hardening. The requested default-off authenticated submodule inventory, selected historical child-tree capture, dirty child freshness, and graph/indexed/live retrieval parity remain unimplemented. No independent review. #4593 stays open. Please do not merge or change the draft selection to conceal missing behavior, and preserve #4800/#4508, protected #4776 Kev and read-only #4763. |
|
Refs #4593 — new source-review boundary mismatch, regression needed; not a claimed exploit or passing test (reviewed at exact head The default-deny live walker now checks parent stage-0 gitlink roots through Test the concrete scenario with a real parent 160000 gitlink under Current-head hosted workflow 42468887836 has passed repository gates, but Rust native/OS tests and Clippy were skipped for this draft; the Dashboard job remained queued when checked. Do not push a speculative fix that cancels that run. Fix via the one canonical authenticated submodule boundary, not a third independent scanner, and keep DRAFT/HOLD MERGE, |
A parent stage-0 gitlink remains tracked repository source even when its child checkout loses the .git marker. Check every indexed ancestor before an ignored dependency is admitted or revalidated, preserving the existing proof for ordinary ignored sources. Add a real local Git submodule regression that removes the child marker and checks both the admission proof and production source-read refusal. Refs #4593
|
Refs #4593 — published The ignored-dependency gix proof now checks every parent index ancestor for mode-160000 gitlinks, not only the exact candidate path. This closes the omitted-marker admission path discussed in comment 6720823044 for verified ignored dependencies, including the production source read and existing ignored-roster revalidation. Added a real local Git submodule regression source: verified stage-0 gitlink and No Rust regression has executed through Bazel, and there is no independent review. Current-head hosted CI 42482445184 was queued. Previous-head run 42468887836 completed successfully after re-running a Dashboard job blocked by a transient download timeout; draft native Rust/Clippy remained skipped. I independently checked the disposable Git fixture commands locally, but that is not a repository Rust test or acceptance evidence. HOLD MERGE / keep DRAFT / #4593 OPEN. The default-off authenticated submodule inventory, selected child-commit historical capture, child-aware freshness, and indexed/graph/live retrieval parity remain undone. Protected Kev/release PRs, #4800/#4508, and CI protections were not changed. |
Refs #4593. Full authenticated child indexing still incomplete.
|
Refs #4593 — guarded source repair on existing DRAFT #4801, HOLD MERGE. Commit One file changed: The integrated canonical default-off authenticated submodule opt-in, shared inventory across child candidate capture/freshness, pinned historical submodule trees, and parent-relative graph/retrieval are not implemented. Do not mark ready, merge, or close #4593. Protected #4776 Kev, #4763 release and #4800/#4508 peers untouched. |
Refs #4593. Reuse the parent's index-plus-HEAD boundary for ignored sources and cover the real staged-removal checkout scenario. The authenticated opt-in indexing feature remains incomplete.
|
Refs #4593 — source repair published on existing DRAFT PR; HOLD MERGE. Commit Ignored-source proof now calls the same index-plus-HEAD parent gitlink boundary as the source walker. The real-Git fixture commits a parent with a Verification limit: read back the exact commit and diff; no nonzero Bazel Rust test executed locally. Current-head hosted run 42503394287 was queued and chooses |
Summary
Refs #4593.
Enable direct Git submodules through the default-off project setting
index.include_submodules.v1, followed by a daemon restart. The parent-relative child paths participate in source walking, grep, structural search, unmounted-file audit, and graph indexing. Parent-to-child call edges remain in one graph.Source authority
.gitmodules, and child checkout to the parent-owned Git module store.Repairs and verification
The earlier blanket DRAFT / HOLD MERGE wording is superseded by the maintainer's request to finish and verify this PR. The prerequisite dashboard fixes and scheduler progress work are integrated. The PR is ready for review; this does not authorize bypassing required checks.
Earlier locally verified commit:
6ca67c5b3669d72a60a1e6dd1379e765e355f6a2.Current head and hosted validation
The current head
76da516eintegrates actual masteraa5b722a, including the beta-103 metadata and reviewed release-runner recovery. Its tree is48bcaa1350a7f5f571e4c3f9d956554c281f043d. It also includes the reviewed failure-only snapshot assertion context: if the existing materialization assertion fails, the fixture reports the already-collected scan-budget evidence. No scan, clock, deadline, transaction, or success oracle changes.Independent full-tree review verified the release integration and the subsequent disjoint runner-recovery integration. The complete submodule/source-boundary policy, revision-23 canonical golden, all 4,211 Rust source postimages, and lockfile remain unchanged from the reviewed pre-runner integration. Actual commit-message validation and diff checks passed. All preceding ordinary and native Windows jobs were terminal, and their complete logs were preserved before this non-force update.
Ordinary CI run 43637394154 passed Linux 180/180 targets, with 60 fresh executions, on synthetic checkout
41323489, whose tree exactly matches this PR head. The canonical source-index suite passed freshly; the index unit and runtime unit results were cached. The ordinary Windows job 132585189472 also passed 467 build/typecheck targets and 26,371 actions; its complete log is preserved and independently hash-verified. All 14 ordinary workflow jobs are terminal with success or configured skip conclusions. Native workflow 43637859699, dispatched once on the actual PR branch at exact76da516e, is now complete and successful. Its Linux job passed 180/180 targets, 37 fresh. Its native Windows job 132587107941 passed all three complete owning targets through the normal Bazel cache: 0/3 executed freshly. Cached results report index unit 391/391, canonical index suite 227/227, and runtime 628 passed with two pre-existing ignored cases. These are current-head cached qualifications, not fresh native executions. The complete Linux and native Windows logs are preserved and independently hash-verified.Previous head
d3327bf0The previous head
d3327bf0integrates actual masterd2650cdd, after the repository owner merged #4827. Its tree is99f2b1805b52ea794e976865ccf0fa9bdb25c5a7. Independent review verified the complete merge: all 57 PR paths and 32 incoming master paths retain their exact respective contents, modes, and types. The paths do not overlap, and no manual resolution was required.The actual retained-baseline update includes the incoming database batch-execution and retention changes. All PR-specific submodule, source-boundary, fixture, TypeScript, and private-filesystem repairs remain unchanged. Pinned formatting, actual commit-message validation, and diff checks passed. All jobs on the prior head were terminal, and both complete Windows logs were saved and independently verified before this update. Ordinary CI run 43623216974 completed Linux with 177/180 targets passed, 88 fresh executions. Automation has two retained-publication wait failures; global database exceeded its existing 300-second bound after 448 of 548 reported case completions; the retained OpenCode snapshot fixture produced zero materialized records where one was required. The count does not identify an individually stalled database case. Daemon-service separately passed all 443 cases freshly.
The native workflow's Linux job completed 176/180 targets, 89 fresh executions. Its automation, sessions, and global-database owning logs are byte-identical to the ordinary run, so those are one observation, not independent reproductions. It additionally failed the provider descendant-pipe deadline assertion at 10.692 seconds; that same case passed in the fresh ordinary job. Code-index-runtime passed 714 cases with two pre-existing ignored cases freshly. The index unit and canonical round-trip suite passes in this run were cached.
Complete terminal Linux logs and artifacts are preserved. Bounded source review has not established a cause for the publication waits, snapshot failure, database timeout, or descendant-pipe delay. The current head now includes the independently reviewed failure-only snapshot assertion context described above; this does not establish a causal repair for the historical snapshot failure. The exact-head native Windows job 132547384675 passed all three complete targets freshly: source-index unit 391/391 in 26.62 seconds, canonical source-index suite 227/227 in 17.95 seconds, and code-index-runtime 628 passed with two pre-existing ignored cases in 176.04 seconds. The canonical round-trip case explicitly passed. Its complete 608,916-byte log was saved and independently verified. The ordinary Windows job 132548180704 also passed: 467 targets and 26,371 actions. Its complete 586,089-byte log was saved and independently verified. All jobs in both workflows are now terminal; ordinary Windows remains build/typecheck evidence, separate from the native execution above. The manual workflow remains failed because of its Linux result.
Separate measurement run 43631589275 at
53f788dacompleted with 179/180 targets passed, 108 fresh executions. Automation, daemon, sessions, runtime acceptance, global database, MCP, and composition all passed freshly. The snapshot case passed without failure-only scan-budget output. The sole failing target was the hooks/LSP suite (26/29 cases): two receipt-drain waits and the final linked-worktree publication condition. The observed elapsed timings do not establish a cause for the earlier publication, process, database, or snapshot failures. Complete logs and artifacts were preserved, then the temporary retained-publication and process tracing was removed by exact inverse82a4410d, restoring the entire reviewed premeasurement tree. Only the permanent failure-only snapshot assertion context is included in this PR; no temporary capture remains.Verified results on prior head
b77eb943The authentic revision-23 manifest golden correction executed successfully. In manual run 43617768798, Linux's complete source-index suite passed 227/227 freshly, including the canonical digest and every subsequent segment-roster, census, and round-trip assertion; source-index unit tests passed 402/402 freshly. The prior expected digest was corrected only to the encoder output actually observed on parent
342f9040:sha256:b8669b6269e0a571d9f6fa58fb9f9f7e254fe47c03c1dc231a7d71623abcfac5.The exact-head native Windows job 132523404460 passed all three complete targets freshly: source-index unit 391/391, canonical source-index suite 227/227, and code-index-runtime 628 passed with two pre-existing ignored. The complete 601,061-byte log was saved and independently verified. This satisfies the native requirement for historical head
b77eb943; the integrated current head requires its own qualification.The manual workflow's Linux job finished 179/180 targets, 57 fresh executions. Its only failure was the dense-reference rename fixture's initial-publication deadline, before the rename preview assertions ran. Its final status was failure despite the passing native Windows job.
Ordinary run 43618532983 finished Linux at 179/180 targets, 21 fresh executions. Runtime acceptance passed 141 of 142 cases; the remaining LSP journey timed out awaiting acknowledgement 10000. The MCP suite separately passed 682/682 freshly. Bounded source and trace review did not establish a causal repair for these timing failures. Workloads, deadlines, retry policy, and assertions remain unchanged.
The ordinary Windows job 132524492543 passed build/typecheck for all 467 targets, with 26,371 actions. Its complete 552,379-byte log was saved and independently verified. This ordinary job does not execute native Windows tests.
Earlier repairs retained
At
e04035c0, Clippy and Linux/Windows compilation identified the same graph context-path caller mismatch:sealed_source_bytesrequired a canonicalLiveSourceBoundariesowner. The reviewed caller repair keeps the actual policy and that owner within the existing blocking worker across sealed reads, preserving source-digest checks, cancellation, deadlines, and refusal handling. Parent342f9040compiled and reached the owning suites. The global-database timeout on that parent remains a historical failed run; no causal repair is inferred from later successful controls.The temporary dashboard subscriber, fixture invocations, and fact-add timing spans remain removed. The earlier full passing run at
34e92220and its complete Windows receipt were preserved before subsequent source updates.Remaining merge checks
Required hosted CI and explicitly dispatched native Windows indexing suites must pass on the latest head. The native target set is the source-index unit suite, the canonical
code_index_suiteowning the digest/round-trip oracle, and the code-index-runtime unit suite. The Windows runtime suite includes child-HEAD/add/delete transition, historical pinned graph, and restart journeys. New actionable review findings must be addressed before merge.The current-head native qualification, run 43637859699 at exact
76da516efdcd7e487ae268f5ff13e5598ef0aa39, completed successfully with all three full Windows targets resolved from the normal Bazel cache. The ordinary workflow has also completed successfully; final review and merge checks still apply. The d332 and b77 fresh native executions above remain historical. Any later source correction requires qualification of the resulting final head. The prior b77 filter-only diagnostic objects remain unreferenced; no temporary diagnostic capture is included in this PR.No approval, merge, or completion of #4593 is claimed by these local results.