Add a workflow to auto-bump vulnerable dependencies #5437
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| name: Bump vulnerable dependencies | ||
|
|
||
| on: | ||
| schedule: | ||
| # Run daily at 05:30 UTC, just after the Go toolchain bumper. | ||
| - cron: "30 5 * * *" | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| bump-vuln-deps: | ||
| runs-on: | ||
| group: databricks-protected-runner-group-large | ||
| labels: linux-ubuntu-latest-large | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 | ||
| with: | ||
| # vulnbump lives in the tools module, which is what this job compiles. | ||
| go-version-file: tools/go.mod | ||
|
|
||
| - name: Build vulnbump | ||
| run: go -C tools/vulnbump build -o "$RUNNER_TEMP/vulnbump" . | ||
|
|
||
| - name: Bump vulnerable dependencies | ||
| id: bump | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| # govulncheck is pinned as a tool dependency in tools/go.mod; -modfile | ||
| # resolves it from there while it scans the root module (the working | ||
| # directory). Only the root module ships; tools/ and | ||
| # bundle/internal/tf/codegen are build- and CI-only, so they are not | ||
| # scanned. Its vulnerability database is fetched from vuln.go.dev at | ||
| # runtime, so the pinned binary still uses the latest advisories. | ||
| # | ||
| # -scan module reports every advisory affecting a required module, | ||
| # regardless of whether the vulnerable symbol is reachable. In JSON | ||
| # mode govulncheck exits 0 on success whether or not it finds anything, | ||
| # and non-zero only on a real error; a failure must abort the job | ||
| # rather than be silently mistaken for "no vulnerabilities". | ||
| scan="$(mktemp)" | ||
| if ! go tool -modfile=tools/go.mod govulncheck -scan module -format json > "$scan"; then | ||
| echo "::error::govulncheck failed" | ||
| exit 1 | ||
| fi | ||
|
|
||
| summary_file="$(mktemp)" | ||
| "$RUNNER_TEMP/vulnbump" . < "$scan" > "$summary_file" | ||
|
|
||
| if git diff --quiet; then | ||
| echo "No vulnerable dependencies to bump." | ||
| echo "needed=false" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "needed=true" >> "$GITHUB_OUTPUT" | ||
| { | ||
| echo "summary<<SUMMARY_EOF" | ||
| cat "$summary_file" | ||
| echo "SUMMARY_EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Show diff | ||
| if: steps.bump.outputs.needed == 'true' | ||
| run: git diff | ||
|
|
||
| - name: Create pull request | ||
| if: steps.bump.outputs.needed == 'true' | ||
| uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1 | ||
| with: | ||
| branch: auto/bump-vuln-deps | ||
|
Contributor
There was a problem hiding this comment. do we need branch-suffix as well?
|
||
| commit-message: "Bump dependencies with known vulnerabilities" | ||
| title: "Bump dependencies with known vulnerabilities" | ||
|
Contributor
There was a problem hiding this comment. nit: we could also write small file with list of upgraded deps and pass it to title |
||
| body: | | ||
| Bump dependencies flagged by `govulncheck -scan module` to their fixed versions. | ||
|
|
||
| Each CVE links to its Go advisory page. | ||
|
|
||
| ${{ steps.bump.outputs.summary }} | ||
|
|
||
| Vulnerabilities in the Go standard library are left to the `Bump Go toolchain` workflow. | ||
|
|
||
| If a bump promotes a new direct dependency, double-check its license annotation in `go.mod` and `NOTICE`. | ||
| reviewers: simonfaltum,andrewnester,anton-107,denik,janniklasrose,pietern,shreyas-goenka | ||
| labels: dependencies | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| /vulnbump |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "encoding/json" | ||
| "errors" | ||
| "fmt" | ||
| "io" | ||
| "maps" | ||
| "slices" | ||
| "strings" | ||
|
|
||
| "golang.org/x/mod/semver" | ||
| ) | ||
|
|
||
| // stdlibModule is govulncheck's module name for the Go standard library. | ||
| // Standard-library fixes map to a toolchain version and are handled by the | ||
| // separate "Bump Go toolchain" workflow, so we skip them here. | ||
| const stdlibModule = "stdlib" | ||
|
|
||
| // advisoryBaseURL is the canonical advisory page for a GO-YYYY-NNNN ID. The | ||
| // page cross-links the CVE alias and the per-package fixed version. | ||
| const advisoryBaseURL = "https://pkg.go.dev/vuln/" | ||
|
|
||
| // advisory identifies a Go vulnerability and its CVE alias. | ||
| type advisory struct { | ||
| ID string // GO-YYYY-NNNN, always present. | ||
| CVE string // CVE-YYYY-NNNN, empty until one is assigned (it can lag the Go advisory by days). | ||
| } | ||
|
|
||
| // label is the link text for an advisory: the CVE when known, else the Go ID. | ||
| func (a advisory) label() string { | ||
| if a.CVE != "" { | ||
| return a.CVE | ||
| } | ||
| return a.ID | ||
| } | ||
|
|
||
| // bump is a single dependency upgrade: the highest fixed version across all | ||
| // advisories affecting a module, plus the advisories it resolves. | ||
| type bump struct { | ||
| Module string | ||
| FixedVersion string | ||
| Advisories []advisory | ||
| } | ||
|
|
||
| // govulncheckFinding mirrors the "finding" message emitted by | ||
| // `govulncheck -format json`. See https://pkg.go.dev/golang.org/x/vuln/internal/govulncheck#Finding. | ||
| type govulncheckFinding struct { | ||
| OSV string `json:"osv"` | ||
| FixedVersion string `json:"fixed_version"` | ||
| Trace []struct { | ||
| Module string `json:"module"` | ||
| } `json:"trace"` | ||
| } | ||
|
|
||
| // govulncheckOSV mirrors the "osv" message emitted by `govulncheck -format json`, | ||
| // which carries the full advisory record including its CVE aliases. | ||
| type govulncheckOSV struct { | ||
| ID string `json:"id"` | ||
| Aliases []string `json:"aliases"` | ||
| } | ||
|
|
||
| // parseBumps reads the concatenated JSON stream from `govulncheck -format json` | ||
| // and reduces it to one bump per module, choosing the highest fixed version and | ||
| // collecting the advisories it resolves. The standard library is excluded. | ||
| func parseBumps(r io.Reader) ([]bump, error) { | ||
| // One accumulator per module: the highest fixed version seen and the set of | ||
| // advisories resolved by upgrading to it. | ||
| type acc struct { | ||
| version string | ||
| advisories map[string]struct{} | ||
| } | ||
| byModule := map[string]*acc{} | ||
| // govulncheck emits a separate "osv" message per advisory; collect the CVE | ||
| // aliases so findings (which carry only the Go ID) can be labelled with it. | ||
| cveByID := map[string]string{} | ||
|
|
||
| dec := json.NewDecoder(r) | ||
| for { | ||
| var msg struct { | ||
| Finding *govulncheckFinding `json:"finding"` | ||
| OSV *govulncheckOSV `json:"osv"` | ||
| } | ||
| err := dec.Decode(&msg) | ||
| if errors.Is(err, io.EOF) { | ||
| break | ||
| } | ||
| if err != nil { | ||
| return nil, fmt.Errorf("decode govulncheck output: %w", err) | ||
| } | ||
|
|
||
| if o := msg.OSV; o != nil { | ||
| cveByID[o.ID] = firstCVE(o.Aliases) | ||
| } | ||
|
|
||
| f := msg.Finding | ||
| if f == nil || f.FixedVersion == "" || len(f.Trace) == 0 { | ||
| continue | ||
| } | ||
| // trace[0] is the most specific frame; in module scan mode it carries | ||
| // the vulnerable module itself. | ||
| module := f.Trace[0].Module | ||
| if module == "" || module == stdlibModule { | ||
| continue | ||
| } | ||
|
|
||
| a := byModule[module] | ||
| if a == nil { | ||
| a = &acc{advisories: map[string]struct{}{}} | ||
| byModule[module] = a | ||
| } | ||
| if a.version == "" || semver.Compare(f.FixedVersion, a.version) > 0 { | ||
| a.version = f.FixedVersion | ||
| } | ||
| a.advisories[f.OSV] = struct{}{} | ||
| } | ||
|
|
||
| bumps := make([]bump, 0, len(byModule)) | ||
| for module, a := range byModule { | ||
| var advisories []advisory | ||
| for _, id := range slices.Sorted(maps.Keys(a.advisories)) { | ||
| advisories = append(advisories, advisory{ID: id, CVE: cveByID[id]}) | ||
| } | ||
| bumps = append(bumps, bump{ | ||
| Module: module, | ||
| FixedVersion: a.version, | ||
| Advisories: advisories, | ||
| }) | ||
| } | ||
| slices.SortFunc(bumps, func(a, b bump) int { | ||
| return strings.Compare(a.Module, b.Module) | ||
| }) | ||
| return bumps, nil | ||
| } | ||
|
|
||
| // firstCVE returns the first CVE alias in the list, or "" if there is none. | ||
| func firstCVE(aliases []string) string { | ||
| for _, a := range aliases { | ||
| if strings.HasPrefix(a, "CVE-") { | ||
| return a | ||
| } | ||
| } | ||
| return "" | ||
| } | ||
|
|
||
| // renderSummary formats the bumps as Markdown list items for the PR body. It | ||
| // uses reference-style links so the list lines stay short instead of wrapping, | ||
| // with the advisory link definitions collected in a block at the end. | ||
| func renderSummary(bumps []bump) string { | ||
| var list, refs strings.Builder | ||
| seen := map[string]struct{}{} | ||
|
|
||
| for _, bump := range bumps { | ||
| labels := make([]string, len(bump.Advisories)) | ||
| for i, adv := range bump.Advisories { | ||
| labels[i] = "[" + adv.label() + "]" | ||
| if _, ok := seen[adv.label()]; ok { | ||
| continue | ||
| } | ||
| seen[adv.label()] = struct{}{} | ||
| fmt.Fprintf(&refs, "[%s]: %s%s\n", adv.label(), advisoryBaseURL, adv.ID) | ||
| } | ||
| fmt.Fprintf(&list, "- %s → %s (fixes %s)\n", | ||
| bump.Module, bump.FixedVersion, strings.Join(labels, ", ")) | ||
| } | ||
|
|
||
| if refs.Len() == 0 { | ||
| return list.String() | ||
| } | ||
| return list.String() + "\n" + refs.String() | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
q: would not this be enough? given you already have set -e