diff --git a/README.md b/README.md index 7b2c0e6..618f733 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ tamis does a first-pass triage of private vulnerability reports on GitHub. When For each report, tamis: -- lists public duplicates, and names an embargoed advisory only when it is an exact duplicate, +- lists exact duplicates, and names an embargoed one only once a separate check confirms it, - finds public issues and pull requests that already fix or harden the same code, - checks that the files and functions the report cites exist, - reads the security policy and flags what may exclude the report, @@ -25,7 +25,7 @@ Scanned on 2026-10-24 at 14:32 UTC on `main` at `21a7561`. Plausible signed `char` passed to `isxdigit()` in `php_url_decode_ex()`, the given reproducer never reaches it. -#### Possible duplicates +#### Duplicates | Advisory | Status | Confidence | |---|---|---| diff --git a/action.yml b/action.yml index 5c50075..f2e7353 100644 --- a/action.yml +++ b/action.yml @@ -55,7 +55,7 @@ inputs: description: Post the triage as a comment on the advisory. Off by default, every triage then only reaches `project`, which must be set. default: "false" references: - description: Whitespace-separated local git references, including each supported release branch. The first is analysed, the cited code is located on all of them. All must have been fetched. + description: Whitespace-separated local git references, including each supported release branch. The first is analysed, and the cited code is located on it. All must have been fetched. required: true packs: description: Whitespace-separated built-in guidance packs added to the agent's method, such as `php-src`. None by default. diff --git a/docs/configuration.md b/docs/configuration.md index a4efd0b..e676cd9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -6,7 +6,7 @@ |---|---|---| | `api-key` | | Required. API key of the model provider, or a Codex access token with `engine: openai`. | | `github-token` | | Required. Token able to read and comment the repository advisories. | -| `references` | | Required, at most ten. Whitespace-separated local git references, one per supported branch. tamis analyses the first and locates the cited code on all of them. A missing reference fails the analysis. | +| `references` | | Required, at most ten. Whitespace-separated local git references, one per supported branch. tamis analyses the first and locates the cited code on it. A missing reference fails the analysis. | | `packs` | | Whitespace-separated [guidance packs](#guidance-packs) added to the agent's method, such as `php-src`. An unknown pack fails the run. | | `since` | | Triage only advisories created after this date, `2026-10-22` or RFC 3339. Required with `comment: true` unless `advisory` is set. Without it, every advisory in triage goes on the board. | | `advisory` | | Triage this GHSA only, whatever its state and creation date. | @@ -33,7 +33,7 @@ A local run alone also reads `FORCE=true`, which triages every selected advisory Fetch the references with `fetch-depth: 0`. For php-src, list the default branch and each PHP branch your disclosure policy covers, such as `master origin/PHP-8.4`, and update the list as releases enter or leave support. -tamis analyses each reference on its own and records its exact commit in both copies of the result. It does not guess the maintained branches from the reporter's prose, and it never executes the source code. +tamis analyses the first reference and records the exact commit of each in both copies of the result. It does not guess the maintained branches from the reporter's prose, and it never executes the source code. ## Private reports @@ -42,7 +42,7 @@ The private artifact holds, per advisory: | File | Contents | |---|---| | `.md` | The full report, with every duplicate the global search proposed and any duplicate search or confirmation that could not complete | -| `.comment.md` | The text the reporter sees, limited to published duplicates and to the unpublished exact duplicates that passed the [one-candidate confirmation](how-it-works.md#3-duplicate-search) | +| `.comment.md` | The text the reporter sees, limited to the published exact duplicates and to the unpublished exact duplicates that passed the [one-candidate confirmation](how-it-works.md#3-duplicate-search) | | `.agent-error.txt` | Written when an agent run fails. It keeps the agent's error output and the error events of its JSON output, which may quote the report. | Each report names the source commit and the input revision it covers. diff --git a/docs/how-it-works.md b/docs/how-it-works.md index ce7804e..5330557 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -26,7 +26,7 @@ tamis rejects an unknown or mistyped field in the JSON and sends the error back When the run stops on its budget or on an error, tamis keeps a valid report already written in the private artifact and retries the advisory. The last attempt posts that report with a note that it may be incomplete. -The agent analyses the first configured reference. tamis then locates the cited code on each other reference, and the final report names every exact commit. A missing branch fails the triage. +The agent analyses the first configured reference, where tamis locates the cited code, and the final report names the exact commit of every reference. A missing branch fails the triage. ## 3. Duplicate search @@ -55,10 +55,10 @@ Before posting, tamis checks each claim against GitHub and the checkout: | Claim | Check | |---|---| -| Duplicates | Each one must belong to the advisories the search received. The private report keeps the complete result. An unpublished advisory appears in the comment, by identifier only, once its exact, high-confidence result passes the one-candidate confirmation. | +| Duplicates | Each one must belong to the advisories the search received. The private report keeps the complete result. The comment lists exact duplicates only, and an unpublished advisory appears there, by identifier only, once its exact, high-confidence result passes the one-candidate confirmation. | | Issues and pull requests | Each one must exist. Titles and links come from GitHub, not from the model. | | Cited code | tamis keeps only references named in the report, each marked as found in the repository or not, and keeps a cited line and its quoted code only when the report contains them. It looks the quote up in the file at the scanned commit, nearest the cited line. When the quote is gone, `git log -S` names the commit that last changed it, since reports often target another version. | -| CWE and CVSS | Each must be well-formed. CWE names come from the MITRE catalogue. | +| CWE and CVSS | Each must be well-formed. CWE names come from the MITRE catalogue. The comment leaves the CVSS vector out when the faulty code comes from an upstream project. | tamis drops the claims that fail a check, and the comment counts them. @@ -75,7 +75,7 @@ Scanned on 2026-10-24 at 14:32 UTC on `main` at `21a7561`. Plausible signed `char` passed to `isxdigit()` in `php_url_decode_ex()`, the given reproducer never reaches it. -#### Possible duplicates +#### Duplicates | Advisory | Status | Confidence | |---|---|---| diff --git a/docs/setup.md b/docs/setup.md index d98147a..41db84b 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -54,7 +54,7 @@ Copy the workflow from the [README](../README.md#quick-start), then: - replace `your-org`, `your-project`, `your-app-slug` and the commit placeholder, - pin the full SHA of a reviewed commit, since the action refuses a branch or tag, and review later updates in the private repository, - set `since` to today's date, so tamis only processes advisories in the `triage` state created after it, -- set `references` to every branch your security policy covers, default branch first. tamis analyses the report on the first one and locates the cited code on each. +- set `references` to every branch your security policy covers, default branch first. tamis analyses the report and locates the cited code on the first one. Keep `persist-credentials: false` on the checkout so no token lands in `.git/config`, and `fetch-depth: 0` so the agent can read the history. diff --git a/internal/report/report.go b/internal/report/report.go index 7013157..432f76d 100644 --- a/internal/report/report.go +++ b/internal/report/report.go @@ -86,8 +86,6 @@ type Report struct { Removed int `json:"-"` // Incomplete holds why the agent stopped before finishing the report it wrote. Incomplete error `json:"-"` - // Elsewhere holds the code references located on the other source references. - Elsewhere []Report `json:"-"` // Located says the faulty statement exists on the analysed source, as the duplicate search clues require. Located bool `json:"-"` } @@ -121,8 +119,8 @@ type Duplicate struct { // visible says whether the reporter may see d. func (d Duplicate) visible() bool { // A reporter learns nothing from an exact duplicate of their own bug, a partial match on an - // unpublished advisory would reveal another report. - return d.Published || d.Confirmed && d.Exact && d.Confidence == "high" + // unpublished advisory would reveal another report, and the maintainers only act on exact ones. + return d.Exact && (d.Published || d.Confirmed && d.Confidence == "high") } type Related struct { @@ -504,6 +502,10 @@ func abs(n int) int { // Render formats the report as the advisory comment. func Render(r Report) string { r.Duplicates = slices.DeleteFunc(slices.Clone(r.Duplicates), func(d Duplicate) bool { return !d.visible() }) + // A CVSS vector of the repository says little about a flaw of the bundled project. + if r.Upstream != "" { + r.CVSS = CVSS{} + } return RenderPrivate(r) } @@ -523,7 +525,7 @@ func RenderPrivate(r Report) string { fmt.Fprintf(&b, "\nThe faulty code seems to come from `%s`, which the repository bundles, so the fix may belong upstream.\n", r.Upstream) } - b.WriteString("\n#### Possible duplicates\n\n") + b.WriteString("\n#### Duplicates\n\n") switch { case r.DuplicateSearchFailed: b.WriteString("The duplicate search could not complete.\n") @@ -553,7 +555,7 @@ func RenderPrivate(r Report) string { // No title: the reporter reads this comment, and other advisories may still be under embargo. fmt.Fprintf(&b, "| %s | %s | %s |\n", advisory, label(d.State), confidence) } - b.WriteString("\n_The reporter-visible comment only lists unpublished advisories after a separate exact-duplicate confirmation, without their title. A match covers the shared vulnerability; review any additional findings before closing the report._\n") + b.WriteString("\n_The reporter-visible comment only lists exact duplicates, and unpublished ones after a separate confirmation, without their title. A match covers the shared vulnerability; review any additional findings before closing the report._\n") b.WriteString("\n#### Related issues and pull requests\n\n") if len(r.Related) == 0 { @@ -614,19 +616,14 @@ func RenderPrivate(r Report) string { return b.String() } -// cited formats the table of the code the report cites, with a column for each source it was located on. +// cited formats the table of the code the report cites. func (r Report) cited() string { if len(r.CodeReferences) == 0 { return "" } - sources := append([]Report{r}, r.Elsewhere...) var b strings.Builder - b.WriteString("\n#### Code cited by the report\n\n| Cited |") - for _, s := range sources { - fmt.Fprintf(&b, " In the repository%s |", s.scope()) - } - b.WriteString("\n|---|" + strings.Repeat("---|", len(sources)) + "\n") - for i, ref := range r.CodeReferences { + fmt.Fprintf(&b, "\n#### Code cited by the report\n\n| Cited | In the repository%s |\n|---|---|\n", r.scope()) + for _, ref := range r.CodeReferences { cited := ref.Path if cited != "" && ref.Line > 0 { cited += ":" + strconv.Itoa(ref.Line) @@ -637,11 +634,7 @@ func (r Report) cited() string { parts = append(parts, code(p)) } } - fmt.Fprintf(&b, "| %s |", strings.Join(parts, " ")) - for _, s := range sources { - fmt.Fprintf(&b, " %s |", s.where(s.CodeReferences[i])) - } - b.WriteString("\n") + fmt.Fprintf(&b, "| %s | %s |\n", strings.Join(parts, " "), r.where(ref)) } return b.String() diff --git a/internal/report/report_test.go b/internal/report/report_test.go index 8882780..5223148 100644 --- a/internal/report/report_test.go +++ b/internal/report/report_test.go @@ -170,7 +170,7 @@ func TestRenderFoldsTheAnalysisDetails(t *testing.T) { }) order := []string{ - "#### Possible duplicates", + "#### Duplicates", "#### Related issues and pull requests", "\n
\nShow more details from the analysis\n\n#### Code cited by the report", "#### Security policy", @@ -208,7 +208,7 @@ func TestRender(t *testing.T) { "| Advisory | Status | Confidence |", "| [GHSA-real](https://x/GHSA-real) | Closed | High • Proposed exact duplicate |", "| [GHSA-near](https://x/GHSA-near) | Published | Low |", - "_The reporter-visible comment only lists unpublished advisories after a separate exact-duplicate confirmation, without their title. A match covers the shared vulnerability; review any additional findings before closing the report._", + "_The reporter-visible comment only lists exact duplicates, and unpublished ones after a separate confirmation, without their title. A match covers the shared vulnerability; review any additional findings before closing the report._", "| Issue or PR | Title | Confidence |", "| [PR #12](https://x/12) | Harden | Medium |", "| [Issue #13](https://x/13) | Crash | Low |", @@ -230,11 +230,13 @@ func TestRenderShowsOnlyExactUnpublishedMatches(t *testing.T) { {GHSAID: "EXACT-ID", Confidence: "high", Exact: true, Confirmed: true}, {GHSAID: "UNSURE-ID", Confidence: "medium", Exact: true}, {GHSAID: "PARTIAL-ID", Confidence: "high"}, - {GHSAID: "PUBLIC-ID", Confidence: "low", Published: true}, + {GHSAID: "PUBLIC-ID", Confidence: "low", Exact: true, Published: true}, + {GHSAID: "PUBLISHED-NEAR-ID", Confidence: "high", Published: true}, }} body := Render(r) require.Contains(t, body, "EXACT-ID", "an exact duplicate is missing from the comment") - require.Contains(t, body, "PUBLIC-ID", "a published duplicate is missing from the comment") + require.Contains(t, body, "PUBLIC-ID", "a published exact duplicate is missing from the comment") + require.NotContains(t, body, "PUBLISHED-NEAR-ID", "a partial published match reached the comment") require.NotContains(t, body, "UNSURE-ID", "an unpublished match below high confidence reached the comment") require.NotContains(t, body, "PARTIAL-ID", "a partial unpublished match reached the comment") require.Contains(t, RenderPrivate(r), "PARTIAL-ID", "public rendering modified the private result") @@ -254,7 +256,7 @@ func TestRenderMarksTheOldestReport(t *testing.T) { assert.Equal(t, 1, strings.Count(got, "(oldest report)"), got) for _, r := range []Report{{CreatedAt: day(1), Duplicates: duplicates}, {Duplicates: []Duplicate{{GHSAID: "GHSA-undated", Confidence: "high", Exact: true, Published: true}}}, {Duplicates: duplicates[2:3]}} { - assert.NotContains(t, Render(r), "(oldest report)", "%+v", r) + assert.NotContains(t, RenderPrivate(r), "(oldest report)", "%+v", r) } } @@ -349,6 +351,21 @@ func TestRenderPolicyPointEndingWithAPeriod(t *testing.T) { assert.NotContains(t, got, ".:") } +func TestRenderLeavesOutTheCVSSVectorOfUpstreamCode(t *testing.T) { + r := Report{CWE: []CWE{{ID: "CWE-787", Reason: "write"}}, CVSS: CVSS{Vector: "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:N/SI:N/SA:N", Reason: "remote"}} + assert.Contains(t, Render(r), "| CVSS 4.0 |") + + r.Upstream = "lexbor" + got := Render(r) + assert.NotContains(t, got, "CVSS") + assert.Contains(t, got, "| CWE | [CWE-787]") + assert.Equal(t, "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:N/SI:N/SA:N", r.CVSS.Vector, "rendering cleared the vector of the report") + assert.Contains(t, RenderPrivate(r), "| CVSS 4.0 |", "the private report lost the vector") + + r.CWE = nil + assert.Contains(t, Render(r), "#### Weakness and severity\n\nNo estimate.\n") +} + func TestRenderNamesTheUpstreamProject(t *testing.T) { assert.Contains(t, Render(Report{Summary: "Overflow in the tokenizer.", Upstream: "lexbor"}), "Overflow in the tokenizer.\n\nThe faulty code seems to come from `lexbor`, which the repository bundles, so the fix may belong upstream.\n") assert.NotContains(t, Render(Report{Summary: "Overflow in the tokenizer."}), "upstream") @@ -376,7 +393,7 @@ func TestVerifyKeepsOnlyAProjectNameAsUpstream(t *testing.T) { func TestRenderFailedDuplicateSearch(t *testing.T) { got := Render(Report{Summary: "s", DuplicateSearchFailed: true}) - assert.Contains(t, got, "#### Possible duplicates\n\nThe duplicate search could not complete.\n") + assert.Contains(t, got, "#### Duplicates\n\nThe duplicate search could not complete.\n") assert.NotContains(t, got, "None found among the repository advisories.") } diff --git a/internal/triage/revisions.go b/internal/triage/revisions.go index 8e628bf..19cdcdd 100644 --- a/internal/triage/revisions.go +++ b/internal/triage/revisions.go @@ -91,12 +91,6 @@ func triageRevisions(ctx context.Context, cfg Config, gh GitHub, reader Reader, } sources += fmt.Sprintf("Source: `%s` at `%s`.\n", reference, commit) if i > 0 { - cited := report.Report{CodeReferences: slices.Clone(r.CodeReferences)} - if err := (report.Checker{Repo: cfg.Repo, Code: code}).Locate(&cited); err != nil { - return false, err - } - cited.Branch = reference - r.Elsewhere = append(r.Elsewhere, cited) continue } local := cfg diff --git a/internal/triage/triage_test.go b/internal/triage/triage_test.go index f7d13e6..6b76642 100644 --- a/internal/triage/triage_test.go +++ b/internal/triage/triage_test.go @@ -419,7 +419,7 @@ func TestRunLimitsEachReporterPerExecution(t *testing.T) { require.Len(t, eng.role(""), 1, "one reporter monopolized the run") } -func TestRunLocatesTheCitedCodeOnEveryConfiguredBranch(t *testing.T) { +func TestRunLocatesTheCitedCodeOnTheAnalysedBranch(t *testing.T) { gh, eng := newFixture(), &fakeEngine{reader: `{"summary":"s","code_references":[{"path":"head.c"}]}`} gh.advisories[1].Description = "Crash in head.c" dir := gitRepo(t, map[string]string{"stable.c": "stable"}) @@ -439,9 +439,10 @@ func TestRunLocatesTheCitedCodeOnEveryConfiguredBranch(t *testing.T) { require.Greater(t, provenance, strings.Index(posted, "
"), "the provenance is outside the details block") require.Less(t, provenance, strings.Index(posted, "
"), "the provenance is outside the details block") require.Equal(t, 1, strings.Count(posted, "#### Code cited by the report"), "the cited code was rendered once per branch") - require.Regexp(t, "\\| Cited \\| In the repository on `HEAD` at `[0-9a-f]{7}` \\| In the repository on `stable` at `[0-9a-f]{7}` \\|", posted) - row := strings.Index(posted, "| `head.c` | Found | Not found |") - require.Positive(t, row, "the cited code was not located on each branch") + require.Regexp(t, "\\| Cited \\| In the repository on `HEAD` at `[0-9a-f]{7}` \\|\n", posted) + require.NotContains(t, posted, "In the repository on `stable`", "the cited code was located on another branch") + row := strings.Index(posted, "| `head.c` | Found |\n") + require.Positive(t, row, "the cited code was not located on the analysed branch") require.Less(t, strings.Index(posted, "
"), row, "the cited code is outside the details block") require.Less(t, row, strings.Index(posted, "
"), "the cited code is outside the details block") out, err = exec.Command("git", "-C", dir, "worktree", "list", "--porcelain").CombinedOutput() @@ -768,7 +769,7 @@ func TestRunLeavesTheAdvisoriesToTheNextRunOnceItsTimeIsSpent(t *testing.T) { require.Contains(t, logs.String(), "the run time is spent, the remaining advisories wait for the next run") } -func TestRunHidesPartialMatchesOnUnpublishedAdvisories(t *testing.T) { +func TestRunHidesPartialMatches(t *testing.T) { gh := newFixture() gh.advisories = append(gh.advisories, github.Advisory{GHSAID: "GHSA-pub", State: "published", HTMLURL: "https://x/GHSA-pub", Summary: "foo php_foo"}) eng := &fakeEngine{comparator: `{"duplicates":[{"ghsa_id":"GHSA-old","confidence":"low","exact":false},{"ghsa_id":"GHSA-pub","confidence":"low","exact":false}]}`} @@ -779,7 +780,7 @@ func TestRunHidesPartialMatchesOnUnpublishedAdvisories(t *testing.T) { assert.NotContains(t, body, "GHSA-old", "partial matches on unpublished advisories must not show, even as a count") assert.NotContains(t, body, "GHSA-draft", "partial matches on unpublished advisories must not show, even as a count") assert.NotContains(t, body, "removed", "partial matches on unpublished advisories must not show, even as a count") - assert.Contains(t, body, "| [GHSA-pub](https://x/GHSA-pub) | Published | Low |", "published duplicates are listed whatever their confidence") + assert.NotContains(t, body, "GHSA-pub", "a partial match on a published advisory reached the comment") assert.Contains(t, privateReports(t, gh), "GHSA-old", "the global private match was discarded") assert.Empty(t, eng.confirmations(), "a non-exact match spent an exact confirmation") }