Skip to content

feat(agents): package fully scanned self-contained snapshots - #168

Draft
tee8z wants to merge 3 commits into
tee8z/sandbox-snapshot-formatfrom
tee8z/sandbox-snapshot-archive
Draft

tee8z wants to merge 3 commits into
tee8z/sandbox-snapshot-formatfrom
tee8z/sandbox-snapshot-archive

Conversation

@tee8z

@tee8z tee8z commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Package a local snapshot archive without depending on forge credentials or silently omitting local history. buzz-agent-snapshot emits self-contained git bundle --all HEAD histories and the tracked/non-ignored working copy, using the checkpoint exclusions and immutable gitleaks policy. It scans the exact staged files, ref names and raw archived objects, so deleted secrets, merge-only blobs and annotated-tag messages cannot escape a scan limited to current files or commit diffs.

This producer-only foundation depends on snapshot wire #167. It adds no request handler, upload, extraction command or relay API; the caller must authorize and contain the job in subsequent layers. The unpublished atomic-claim/server work will follow this layer in the linear stack. The diff is 1,162 changed lines after the review fix. The narrow native CLI inspector and producer admission must ship together to validate the exact captured pack before import; splitting that correction would leave either an unavailable required helper or the resource race.

Source Git configuration cannot enable network fetch, hooks, fsmonitor, replacement/graft rewriting or unbounded pack worker pools. Neutral imports validate object integrity and retain explicit shallow boundaries. Directory-descriptor reads prevent a source symlink change from redirecting copied files outside the selected repo. Bundles, subprocess output and gzip bytes are admitted before writes; source, object/file counts, metadata, deadlines, actual free space and configured scratch budgets are bounded. Failed creation removes its partial output; existing files are never overwritten.

Artifact behavior Result
History Full locally available history, local-only branches, detached HEAD support and separate nested-repo bundles
Working copy Dirty/untracked files and real deletions; private/runtime paths omitted
Shallow repositories Explicit boundary metadata, no implicit fetch/unshallow; undeclared missing parents fail integrity checks
Secret policy Fixed image configuration; current, deleted, merge-only and tag content scanned; ignore/env overrides cannot weaken it
Output New mode-0600 archive; metadata-only success/failure receipt; no private subprocess diagnostics
Limits 256 MiB source/expanded objects, 200,000 objects, 50,000 files/references, 64 repos, 180-second creation deadline, ciphertext-compatible 100 MiB archive cap

The user chose full scanned history: a versioned historical .env.example remains in the bundle while the working-tree copy excludes it. The later extractor must hide intentional omissions from Git diff via skip-worktree while preserving actual source deletions. The scratch budget defaults to 512 MiB and must be configured below the actual platform quota after other data/reserves; statvfs alone cannot detect a Kubernetes emptyDir limit. See producer format and runbook.

Validation: the packaged helper and actual scanner pass nine hardened image journeys with a read-only root, UID 10001, no network, dropped capabilities and 1 GiB memory. Coverage includes SHA-1/SHA-256, dirty/untracked/deleted/executable files, relative links, bounded PAX names, outer+nested repositories, shallow restoration/missing-parent refusal, current/deleted/merge/tag secrets and bypass attempts, capacity/source refusal, existing-output preservation, unborn refusal, streaming admission and descriptor-relative containment. Seven disposable-image mutations fail the real workflow assertions: copied-file scan, raw historical-object scan, exclusions, gzip admission, subprocess output admission, source-directory nofollow and discovery count. Python compilation, shell syntax and diff checks pass. Local image validation used a source/CLI overlay on the existing agent image with the exact read-only Nix runtime libraries required by the local GNU binary, avoiding a duplicate Rust image build. The normal production image still builds its bundled static Sprig. A controlled source-commit race accepted a 20 MiB work budget while writing 50 MiB on the old image; the fixed image refuses before raw writes and succeeds with a 128 MiB control. Removing captured-cost admission reproduces the defect. The fix also passes 511 CLI tests, all-target Clippy, and actual hidden/no-auth/no-relay command checks.

Independent review is in progress; human integrated snapshot testing awaits the extraction and runtime consumers. Keep this PR in draft. Full local repository CI remains blocked by missing native Tauri GLib libraries from the parent task's run, with unrelated base CI fixes tracked separately. No deployment, paid model call, quarantine fallback, tunnel, broker or substrate exec is added.

Signed-off-by: Tee8z <tee8z@protonmail.com>
@tee8z tee8z self-assigned this Nov 10, 2026
@coderabbitai

coderabbitai Bot commented Nov 10, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@tee8z

tee8z commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

Blocking: scratch admission can use a stale source object inventory. If an active agent commits after the initial inventory, the bundle imported into the neutral repository can contain more objects and bytes than were reserved before ObjectStream writes its raw review files. A large compressible history-only file can therefore exceed the configured scratch budget while producing a successful small archive.

A controlled run against the exact packaged helper at 6504ed331 reproduced this: configured work budget 20,971,520 bytes; actual raw review objects 52,429,376 bytes; snapshot accepted with a 1,175-byte archive. The source writer added and then removed a committed 50 MiB file between inventory and bundle capture. This can exhaust the intended emptyDir allowance during otherwise authorized capture.

Fix: reconcile actual neutral object bytes/counts with the aggregate limits, and reserve any additional raw-object/block cost before starting ObjectStream. Keep the source preflight. Add a real capture-race fixture that refuses before oversized raw review writes, with a successful within-budget control.

Independent validation: the nine packaged-image cases pass with no network, UID 10001, a read-only root, dropped capabilities, and a 1 GiB cap. Installed helper SHA-256 matches signed source. The nested-repository trailing-slash issue found during review is already fixed and covered by a real outer/nested repository fixture. Transfer and extraction remain separate pending layers.

Scores: minimalism 9/10, elegance 9/10, correctness 8/10 until the concrete budget defect above is fixed. This is a plain review comment; draft status is preserved.

Signed-off-by: Tee8z <tee8z@protonmail.com>
@tee8z

tee8z commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

Fixed the resource-race blocker with native SDK preflight of the captured bundle before Git import, reservation of actual expansion/object/ref/index/block costs, and neutral-inventory equality before raw review writes. The source inventory remains conservative. The exact independent 50 MiB probe accepted a 20 MiB budget on the old source; fixed source refuses before any raw review bytes, while the 128 MiB control succeeds. Removing actual-cost admission reintroduces the failure. Eleven hardened packaged-image journeys and all 511 CLI tests pass; all-target Clippy, formatting and diff checks pass. The local-only native inspector requires no auth/relay configuration, is hidden from normal help, and has no missing/invalid fallback. Independent follow-up review and human integrated testing remain pending; PR stays draft.

@tee8z

tee8z commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

The budget blocker is resolved at 6e7ceba13.

I inspected the additive fix and independently reran the exact 50 MiB concurrent-source probe against the packaged fixed helper. The 20 MiB work budget is now refused before raw review writes (0 bytes). The helper hash matches signed source. All eleven hardened packaged-image journeys also pass independently, including the within-budget control and missing/invalid-inspector refusal.

The native SDK now preflights the actual captured bundle before Git imports it. The producer reserves actual object/ref/index costs, checks aggregate limits, and requires the neutral inventory to match before writing raw review objects. The source-inventory fallback is removed. The hidden local inspector requires no identity or relay configuration.

No remaining concrete blocker in this producer layer. Scores: minimalism 9/10, elegance 9/10, correctness 9/10. Snapshot extraction and transfer remain separate layers; human confirmation is still pending and this PR remains draft.

Signed-off-by: Tee8z <tee8z@protonmail.com>
@tee8z

tee8z commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

Hosted image CI exposed a missing Docker build-context entry for the new snapshot helper. Commit 1a1d563de adds that exact file to the agent Dockerfile-specific whitelist.

I built the actual Dockerfile source stage before and after the fix. Before, the existing checkpoint helper was present and snapshot.py was absent. Afterward, snapshot.py is present and its SHA-256 matches checked-out source (775ff6de...). No exclusion for private files was changed. Full final image/runtime checks will still run against the completed stack.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant