Skip to content

feat(cli): apply shipped baselines from local ZIP inputs - #932

Open
chrisknvidia wants to merge 1 commit into
NVIDIA:mainfrom
chrisknvidia:feat/christopherk/zip-shipped-baseline
Open

chrisknvidia wants to merge 1 commit into
NVIDIA:mainfrom
chrisknvidia:feat/christopherk/zip-shipped-baseline

Conversation

@chrisknvidia

@chrisknvidia chrisknvidia commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #931.

--use-shipped-baseline now applies the canonical baseline at a safely extracted local ZIP root, including a wrapper directory. It remains opt-in, explicit --baseline takes precedence, and the root choice does not propagate to transitive child scans. Baseline loading and notice failures clean up extracted files.

Refreshed onto main ea5aeef6; current head 4ca0915d. The feature patch is unchanged. Current-head CI is running; the historical failed run is not current-head validation.

Validation at this head:

  • 1,046 affected tests passed; one Linux-only test skipped on macOS. This includes eight linked ZIP integration tests.
  • 64 actual CLI runs passed across source and a fresh noneditable wheel: 56 matrix runs, six notice-failure cleanup controls, and two observed HTTPS controls. Four runs in total performed actual HTTPS retrieval.
  • Covered consent, explicit precedence, malformed/missing baselines, exact-fingerprint suppression, JSON/SARIF, strict finding/completeness exits, exclusions and transitive scope. Input archives/baselines remained unchanged and temporary extractions were removed.
  • All 28 source/wheel behavior pairs matched. Ten SARIF outputs passed the project models and the SARIF 2.1.0 JSON schema. All 148 package files and 158 installed distribution versions matched across frozen environments.
  • Full Ruff lint/format and independent code/security review passed without actionable findings.

Validation used static scans on macOS. No live LLM, customer production, Windows qualification or full integration suite was run. Dependency-inspection cases verify missing-lock/malformed-manifest reporting; successful Linux dependency-worker execution is not claimed. Notice controls inject only the notice callback. Native typed review bots were unavailable. Consumed-baseline output protection is handled separately in #986.

@chrisknvidia
chrisknvidia marked this pull request as ready for review October 28, 2026 09:06

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Value and readiness

The change delivers #931: explicit opt-in loads a local ZIP root baseline after safe extraction, including wrapper directories. Ready for final maintainer/PIC review; no required changes found.

Material findings

None blocking. Explicit --baseline takes precedence; root opt-in is not passed to transitive child scans. Both baseline-load failure and interrupted post-graph notices clean up extraction.

PIC tradeoffs

Author-controlled suppressions alter risk scores, so the existing explicit opt-in and audit trail remain essential; they are preserved. No additional PIC decision identified.

Verification and gaps

This was a static review of the complete diff, pinned-head source, added regression assertions, discussion/review history and hosted check results. No contributor code or tests were executed. Traced CLI flag → graph state → resolve_input → baseline exclusion/suppression and transitive call boundaries. Reviewed JSON/SARIF, wrapped/unwrapped, explicit/malformed baseline and interruption cases. All six hosted checks succeeded at this head. Author-reported source/wheel and live-provider results were not independently rerun.

Reviewed head: 50b30b55c982a82d64220b0e885e8a6600dc67fc.

@chrisknvidia
chrisknvidia force-pushed the feat/christopherk/zip-shipped-baseline branch 3 times, most recently from eafc1a8 to 089840f Compare November 2, 2026 19:30
@chrisknvidia
chrisknvidia force-pushed the feat/christopherk/zip-shipped-baseline branch from 089840f to 46dad66 Compare November 2, 2026 20:08

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Value and readiness

Ready for maintainer review. Explicit --use-shipped-baseline now loads the canonical baseline from a safely extracted local ZIP root, including a wrapper directory. The earlier marked approval raised no required changes; the rebased complete change remains sound.

Material findings

No required changes found. cli.py:931-949 selects local ZIP opt-in only without an explicit baseline; resolve_input.py:83-93 loads after materialization and within the handler's BaseException cleanup boundary. The post-graph notice at cli.py:2698-2707 also cleans extraction if interrupted. Root state carries suppression visibility and the bounded loader; transitive child calls at line 2283 do not receive shipped-baseline opt-in. The merged report retains the root baseline. Added regressions cover wrapped/unwrapped JSON/SARIF, explicit precedence, malformed baseline trust, baseline-load interruption and notice failure.

PIC tradeoffs

Author-supplied suppressions affect risk scores, so explicit opt-in and report audit trails matter; both are preserved. No additional PIC decision identified.

Verification and gaps

Read the complete six-file diff, exact-head CLI/state/resolver and supporting input-handler/baseline-loader source, all marked review history and comments/threads, and added assertions. Exact-head CI run 41516386255: changes, lint, OpenCode TypeScript Tests, DCO and docker-smoke succeeded. test-unit reports 17,818 passed and 15 failed; the failed test IDs exactly match main e14c450b run 41513800480, all existing PowerShell/quote-ownership cases. No new failing test ID was observed; the suite remains red and GitHub reports MERGEABLE/UNSTABLE. No required-check names are configured. Author-reported runtime results were not independently rerun. Static review only; no contributor code, tests, imports, builds, setup or scanner invocations were executed.

Reviewed head: 46dad66fda765b02d338f6de316b0e1d266e1096.

Signed-off-by: Christopher Kevin <christopherk@nvidia.com>
@chrisknvidia
chrisknvidia force-pushed the feat/christopherk/zip-shipped-baseline branch from 46dad66 to 4ca0915 Compare November 9, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support opted-in shipped baselines in local ZIP skill inputs

2 participants