Skip to content

feat(cli): nullify deps analyze — CI-aware dependency malware-analysis workflow#142

Open
tim-thacker-nullify wants to merge 4 commits into
mainfrom
feat/deps-analyze-workflow
Open

feat(cli): nullify deps analyze — CI-aware dependency malware-analysis workflow#142
tim-thacker-nullify wants to merge 4 commits into
mainfrom
feat/deps-analyze-workflow

Conversation

@tim-thacker-nullify
Copy link
Copy Markdown
Member

@tim-thacker-nullify tim-thacker-nullify commented Apr 19, 2026

Claude

Summary

  • Hand-written `nullify deps analyze` workflow — detects changed dependencies in a PR/push and requests malware analysis via `scpm` (which fans out to the platform-singleton `vuln-database` service; see monorepo PR #2280 for the server-side pieces).
  • New `internal/ci/` package — extensible `Provider` interface with 9 concrete implementations (GitHubActions, GitLabCI, CircleCI, BitbucketPipelines, Jenkins, AzureDevOps, GoogleCloudBuild, AWSCodeBuild, Local fallback). Each returns `benchmarks.PipelinePlatform` enum for consistency.
  • New `internal/scan/manifest/` — five lockfile parsers (npm `package-lock.json` v2/v3, pypi `requirements.txt`, go `go.sum`, Rust `Cargo.lock`, Ruby `Gemfile.lock`).
  • `internal/scan/diff.go` — `git diff` + per-ecosystem parse + set-diff → `[]ChangedDep{added, bumped, removed}`.
  • `internal/commands/deps_analyze.go` — the cobra workflow. CI-detection → ref resolution → manifest diff → scpm POST → render (text/json) → exit per `--fail-on` policy (none/vulnerable/suspicious/malicious, stable exit codes 0/10/20/30).
  • Registered at top level (`nullify deps analyze`) in `cmd/cli/cmd/root.go`.

Related

Depends on monorepo PR https://github.com/Nullify-Platform/nullify/pull/2280 for the scpm endpoints (`/scpm/dependencies/analyze`, `/scpm/dependencies/query/batch`, `/scpm/containers/analyze`) and the `vuln-database` service those endpoints fan out to.

Test plan

  • `go build ./...` clean
  • `go test ./internal/ci/... ./internal/scan/... ./internal/commands/...`
  • Smoke: `NULLIFY_REPO_SLUG=owner/name nullify deps analyze --base HEAD~1 --head HEAD` against a test repo with a changed `package-lock.json` — verifies CI provider detection (Local fallback) + manifest diff + scpm POST path.
  • Smoke: run in GitHub Actions to confirm `GITHUB_*` env detection + `X-Nullify-CI-Provider` header stamping.

🤖 Generated with Claude Code

@tim-thacker-nullify @claude
feat(cli): nullify deps analyze — CI-aware dependency malware-analysi…
65ffb1c 
…s workflow

Hand-written workflow command (not generated from OpenAPI) that wires
the Phase 5a/5b malware analyzer into customer CI pipelines via scpm's
/scpm/dependencies/analyze endpoint.

Components:

- internal/ci/ — extensible CI provider registry. Nine concrete
  Providers: GitHubActions, GitLabCI, CircleCI, BitbucketPipelines,
  Jenkins, AzureDevOps, GoogleCloudBuild, AWSCodeBuild, Local (last;
  always matches). Each exposes Platform() (matches the
  benchmarks.PipelinePlatform enum), Detect() (env-var signature),
  BaseRef / HeadRef (context-aware; fall back to
  `git rev-parse origin/HEAD` when CI doesn't expose a base),
  PRNumber, RepoSlug, and EnrichHeader (stamps CI metadata on outbound
  HTTP so scpm's audit log can tie back to the CI run).

- internal/scan/manifest/ — per-ecosystem lockfile parsers:
  npm (package-lock.json v2/v3), pypi (requirements.txt), go (go.sum),
  rust (Cargo.lock), ruby (Gemfile.lock). Each implements Parser and is
  registered in manifest.DefaultParsers(). Deliberately tight:
  pnpm-lock, poetry.lock, pom.xml, composer.lock, packages.lock.json
  are follow-up work. Every parser produces the same flat Entry
  (Ecosystem, Name, Version, File, Direct) shape so diff.go can
  compare them uniformly.

- internal/scan/diff.go — the (base, head) diff. Uses
  `git diff --name-only base..head` to list changed files, filters to
  those a parser matches, then `git show <ref>:<path>` on each side +
  parse + set-difference. Produces ChangedDep{added, bumped, removed}.

- internal/commands/deps_analyze.go — the cobra command. Flow:
    1. Detect CI provider; resolve base (--base or provider.BaseRef)
       and head (--head or provider.HeadRef) to commit SHAs.
    2. scan.Diff → list of changed deps.
    3. For each added/bumped dep, POST to
       {client.BaseURL}/scpm/dependencies/analyze with the CI-provider
       headers + an idempotency key derived from CI run ID + package
       tuple (so re-runs of the same CI build hit cached jobs).
    4. Render results as text (default) or json.
    5. Exit per --fail-on (none|vulnerable|suspicious|malicious):
       0=clean, 10=vulnerable, 20=suspicious, 30=malicious, 2=invalid
       invocation, 1=transient.

The scpm calls are hand-rolled HTTP POSTs rather than generated
api.Client methods because scpm's OpenAPI regen happens in a separate
pipeline — the CLI's scripts/generate/main.go will supersede these
once the spec lands. Follows the RegisterContextPushCommand pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@tim-thacker-nullify tim-thacker-nullify force-pushed the feat/deps-analyze-workflow branch from e646727 to 65ffb1c Compare May 25, 2026 10:20
@tim-thacker-nullify tim-thacker-nullify added the minor Minor version updates (features) label May 25, 2026
tim-thacker-nullify and others added 2 commits May 25, 2026 20:22
@tim-thacker-nullify @claude
style(cli): gofmt doc-comment blocks in CI provider files
357a255 
golangci-lint v2.12.0 gofmt flagged the space-indented env-var tables in
the AWS CodeBuild / Azure DevOps / Bitbucket provider doc comments; reflow
to the Go 1.19 preformatted form (blank // + tab indent).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@tim-thacker-nullify @claude
style(cli): gofmt remaining deps-analyze files
ceff969 
Reflow the rest of the CI-provider doc-comment tables to Go 1.19
preformatted form, and fix const/var/struct-tag alignment in
deps_analyze.go and npm_lock.go. golangci-lint caps gofmt reports at 3 per
run, so these surfaced only after the first batch was fixed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@tim-thacker-nullify tim-thacker-nullify marked this pull request as ready for review May 25, 2026 10:27
@tim-thacker-nullify
Copy link
Copy Markdown
Member Author

tim-thacker-nullify commented May 25, 2026

Claude

Refreshed this PR to merge-ready:

  • Rebased onto main (was ~10 commits behind — that's what build-test was failing on). Its server-side dependency, monorepo #2280 (vuln-database + scpm/dependencies/analyze endpoints), has since merged, so the backend this workflow calls now exists.
  • Fixed gofmt across the new internal/ci/* and internal/scan/* files — the doc-comment env-var tables needed the Go 1.19 preformatted form, plus const/var/struct-tag alignment in deps_analyze.go / npm_lock.go. (golangci caps gofmt reports at 3/run, so they surfaced in two batches.)
  • Added the minor semver label.

build-test and require-labels are green; go build ./... and go test ./... pass locally. Only REVIEW_REQUIRED remains.

@tim-thacker-nullify @claude
fix(cli): make deps-analyze gate functional + add tests
74c27aa 
Addresses review of the deps-analyze workflow.

Blockers:
- Wire ExitCodeFromError in main.go so the severity exit codes
  (10/20/30) actually fire — previously every result exited 1, so the
  CI merge-gate was inert. Set SilenceUsage so a verdict failure doesn't
  print cobra usage.
- Fix --fail-on=none, which always exited 10: checkFailOn now treats a
  zero-rank threshold as "never fail".

Correctness:
- renderResults no longer index-zips two parallel slices (a mid-list
  analyze failure misattributed verdicts to the wrong package and
  dropped the tail). One analyzedDep{dep,resp,err} entry per dep now.
- Unknown/unrecognised non-empty verdicts fail closed: warn and treat as
  malicious so a server-side rename can't silently bypass the gate.
- Provider BaseRef/HeadRef + resolveRef take repoPath and set cmd.Dir,
  so --repo is honoured (ref resolution no longer runs in the CWD).

Feature:
- Implement --wait (was parsed but unused): poll by re-POSTing the
  idempotent analyze request until the job is terminal or --wait-timeout
  elapses; adds --wait-timeout/--wait-interval. An incomplete analysis
  now exits transiently (1) instead of greenlighting the merge.

Cleanup:
- Typed enums replace magic strings: ci.Platform, scan.ChangeKind,
  manifest.Ecosystem, commands.Verdict.
- Remove the `var _ = fmt.Sprintf` hack and the //nolint:errcheck.
- Correct the doc comments that claimed a benchmarks.PipelinePlatform
  enum and a `make lint-ci-providers` target that don't exist.
- Fix the misleading idempotency-key comment; handle SSH repo URLs in
  AWSCodeBuild.RepoSlug; note fetch-depth:0 for GitHub push diffs.

Tests:
- Add unit tests for the five lockfile parsers, diffEntries, CI provider
  detection/parsing, and the verdict/gate/render/wait/exit-code logic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

minor Minor version updates (features)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@tim-thacker-nullify