Skip to content

extension-duplicate-check: Remove backend names from author messages #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
0xdhrv merged 1 commit into from
Nov 2, 2026
Merged

extension-duplicate-check: Remove backend names from author messages #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions extension-duplicate-check/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ ghq, so they are worth a reviewer's look anyway. Of the 17 duplicates it misses

The relation also words the comment. `same` becomes "covers the same core functionality" and `enhancement` asks the
author to contribute the new features to the existing extension, as reviewers do by hand. Retries cover network
errors, `429`, `529` and server errors. If TypeSafe still fails, the action falls back to `lexical` and says so in the
comment.
errors, `429`, `529` and server errors. If TypeSafe still fails, the action falls back to `lexical` and adds a warning
to the workflow run. The comment and job summary never name the scorer or other backends.

A request is about 5,000 input tokens. At $0.042 per million input tokens, with output free, roughly 800 runs a month
cost about $0.20. The 394-pull-request backtest took 25 seconds.
Expand Down
13 changes: 9 additions & 4 deletions extension-duplicate-check/__tests__/comment.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ async function report(name: string): Promise<ExtensionReport> {
};
}

const options = { threshold: 0.6, scorer: "lexical", unavailable: [] };
const options = { threshold: 0.6, unavailable: [] };

describe("renderComment", () => {
it("greets the author and lists flagged matches in the reviewers' voice", async () => {
Expand Down Expand Up @@ -55,19 +55,24 @@ describe("renderComment", () => {
it("words the comment from the relation a model judged", async () => {
const spotify = await report("spotify-controls");
spotify.store = spotify.store.map((match, index) => ({ ...match, relation: index === 0 ? "enhancement" : "different" }));
const body = renderComment([spotify], { ...options, scorer: "jev" });
const body = renderComment([spotify], options);
expect(body).toContain("We found an extension with the same main purpose as **Spotify Controls**:");
expect(body).toContain("Your extension seems to add features on top of it. These look like natural enhancements");
expect(body).toContain("| Extension | Where | Relation | Score |");
expect(body).toContain("Same purpose, adds features");
expect(body).toContain("TypeSafe's Jev model");

spotify.store[0] = { ...spotify.store[0]!, relation: "same" };
expect(renderComment([spotify], { ...options, scorer: "jev" })).toContain(
expect(renderComment([spotify], options)).toContain(
"We found an extension that covers the same core functionality as **Spotify Controls**:",
);
});

it("doesn't name the backends behind the check", async () => {
const body = renderComment([await report("spotify-controls")], options);
expect(body).toContain("Scores range from 0 to 1");
expect(body).not.toMatch(/jev|lexical|typesafe|redash|scorer|model/i);
});

it("reports unavailable sources, no matches, and no new extension", async () => {
const body = renderComment([await report("svg-optimizer")], { ...options, unavailable: ["Store not checked."] });
expect(body).toContain("nothing to do here");
Expand Down
15 changes: 7 additions & 8 deletions extension-duplicate-check/dist/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24642,7 +24642,7 @@ function renderComment(reports, options) {
lines.push(...renderTable([...report.store, ...report.pullRequests], options.threshold), "");
}
lines.push(
options.scorer === "jev" ? `Scores come from TypeSafe's Jev model, which judged the closest matches of the \`lexical\` scorer: how likely an extension already does the main task of this one, unless the two are built for different apps or services. Matches from ${options.threshold} are flagged.` : `Scores range from 0 to 1 and come from the \`${options.scorer}\` scorer, which compares titles, descriptions, commands and keywords, and the similarity of names. Matches from ${options.threshold} are flagged.`,
`Scores range from 0 to 1 and show how closely an existing extension matches this one. Matches from ${options.threshold} are flagged.`,
"",
"</details>",
""
Expand Down Expand Up @@ -25611,7 +25611,6 @@ async function check(inputs) {
throw error2;
}
warning(`The ${scorer.name} scorer failed, falling back to lexical: ${errorMessage(error2)}`);
unavailable.push(`The \`${scorer.name}\` scorer was unavailable, so these matches come from the \`lexical\` scorer.`);
scorer = createScorer("lexical");
results = await scorer.score(added, candidates);
}
Expand All @@ -25625,7 +25624,7 @@ async function check(inputs) {
});
const hasDuplicates = reports.some((report) => isDuplicate(report, inputs.threshold));
setOutputs(reports, hasDuplicates);
const body = renderComment(reports, commentOptions(inputs, unavailable, pullRequest.author, scorer.name));
const body = renderComment(reports, commentOptions(inputs, unavailable, pullRequest.author));
if (process.env.GITHUB_STEP_SUMMARY) {
await summary.addRaw(body).write();
}
Expand Down Expand Up @@ -25655,7 +25654,7 @@ async function resolvePullRequest(octokit, repository, number) {
async function loadStore(inputs, unavailable) {
if (!inputs.redashApiKey) {
warning("redash-api-key is not set, skipping the store catalog.");
unavailable.push("The store catalog was not checked because no Redash API key is configured.");
unavailable.push("Store extensions could not be checked this time.");
return [];
}
try {
Expand All @@ -25666,7 +25665,7 @@ async function loadStore(inputs, unavailable) {
});
} catch (error2) {
warning(`Could not load the store catalog: ${errorMessage(error2)}`);
unavailable.push("The store catalog could not be loaded, so store extensions were not checked.");
unavailable.push("Store extensions could not be checked this time.");
return [];
}
}
Expand All @@ -25678,12 +25677,12 @@ async function loadOpenPullRequests(octokit, repository, current, inputs, unavai
return await fetchOpenPullRequestExtensions(octokit, repository, current);
} catch (error2) {
warning(`Could not load open pull requests: ${errorMessage(error2)}`);
unavailable.push("Open pull requests could not be loaded, so they were not checked.");
unavailable.push("Open pull requests could not be checked this time.");
return [];
}
}
function commentOptions(inputs, unavailable, author, scorer = inputs.scorer) {
return { threshold: inputs.threshold, scorer, unavailable, author };
function commentOptions(inputs, unavailable, author) {
return { threshold: inputs.threshold, unavailable, author };
}
function setOutputs(reports, hasDuplicates) {
setOutput("has-duplicates", hasDuplicates ? "true" : "false");
Expand Down
10 changes: 2 additions & 8 deletions extension-duplicate-check/src/comment.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ export interface ExtensionReport {

export interface CommentOptions {
threshold: number;
scorer: string;
/** Sources that could not be searched, with the reason. */
unavailable: string[];
/** GitHub login of the pull request author, greeted at the top of the comment. */
Expand Down Expand Up @@ -74,13 +73,8 @@ export function renderComment(reports: ExtensionReport[], options: CommentOption
lines.push(...renderTable([...report.store, ...report.pullRequests], options.threshold), "");
}
lines.push(
options.scorer === "jev"
? "Scores come from TypeSafe's Jev model, which judged the closest matches of the `lexical` scorer: how likely " +
"an extension already does the main task of this one, unless the two are built for different apps or " +
`services. Matches from ${options.threshold} are flagged.`
: `Scores range from 0 to 1 and come from the \`${options.scorer}\` scorer, which compares titles, ` +
"descriptions, commands and keywords, and the similarity of names. Matches from " +
`${options.threshold} are flagged.`,
"Scores range from 0 to 1 and show how closely an existing extension matches this one. Matches from " +
`${options.threshold} are flagged.`,
"",
"</details>",
"",
Expand Down
15 changes: 7 additions & 8 deletions extension-duplicate-check/src/main.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,8 @@ async function check(inputs: Inputs): Promise<void> {
if (scorer.name === "lexical") {
throw error;
}
// A model outage shouldn't block the check: fall back to the in-process scorer and say so.
// A model outage shouldn't block the check: fall back to the in-process scorer and warn in the run.
core.warning(`The ${scorer.name} scorer failed, falling back to lexical: ${errorMessage(error)}`);
unavailable.push(`The \`${scorer.name}\` scorer was unavailable, so these matches come from the \`lexical\` scorer.`);
scorer = createScorer("lexical");
results = await scorer.score(added, candidates);
}
Expand All @@ -133,7 +132,7 @@ async function check(inputs: Inputs): Promise<void> {
const hasDuplicates = reports.some((report) => isDuplicate(report, inputs.threshold));
setOutputs(reports, hasDuplicates);

const body = renderComment(reports, commentOptions(inputs, unavailable, pullRequest.author, scorer.name));
const body = renderComment(reports, commentOptions(inputs, unavailable, pullRequest.author));
if (process.env.GITHUB_STEP_SUMMARY) {
await core.summary.addRaw(body).write();
}
Expand Down Expand Up @@ -170,7 +169,7 @@ async function resolvePullRequest(
async function loadStore(inputs: Inputs, unavailable: string[]): Promise<Extension[]> {
if (!inputs.redashApiKey) {
core.warning("redash-api-key is not set, skipping the store catalog.");
unavailable.push("The store catalog was not checked because no Redash API key is configured.");
unavailable.push("Store extensions could not be checked this time.");
return [];
}
try {
Expand All @@ -181,7 +180,7 @@ async function loadStore(inputs: Inputs, unavailable: string[]): Promise<Extensi
});
} catch (error) {
core.warning(`Could not load the store catalog: ${errorMessage(error)}`);
unavailable.push("The store catalog could not be loaded, so store extensions were not checked.");
unavailable.push("Store extensions could not be checked this time.");
return [];
}
}
Expand All @@ -200,13 +199,13 @@ async function loadOpenPullRequests(
return await fetchOpenPullRequestExtensions(octokit, repository, current);
} catch (error) {
core.warning(`Could not load open pull requests: ${errorMessage(error)}`);
unavailable.push("Open pull requests could not be loaded, so they were not checked.");
unavailable.push("Open pull requests could not be checked this time.");
return [];
}
}

function commentOptions(inputs: Inputs, unavailable: string[], author?: string, scorer = inputs.scorer) {
return { threshold: inputs.threshold, scorer, unavailable, author };
function commentOptions(inputs: Inputs, unavailable: string[], author?: string) {
return { threshold: inputs.threshold, unavailable, author };
}

function setOutputs(reports: ExtensionReport[], hasDuplicates: boolean) {
Expand Down
Loading