Skip to content

Make Go support organizationally viable - #119

Open
irskep wants to merge 21 commits into
mainfrom
go-attribution-and-provider-factory
Open

irskep wants to merge 21 commits into
mainfrom
go-attribution-and-provider-factory

Conversation

@irskep

@irskep irskep commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

I'm working on Go improvements at Atticus and found some annoying API gaps that this PR fixes.


Running Dependicus over a pnpm workspace and one Go module, the Go side was there but not usable: every dependency said it was used by "golang", and the team pages for Go existed but nothing linked to them.

Go records which of your packages use what. go list -m treats a module as one unit, so every Go dependency belongs to the module. A new source writes two dependency facts instead: goImportedBy, the packages that import a dependency, and goBinaries, the main packages that reach it by following imports through your own packages. goBinaries is the one you want. A thin cmd/ binary reaches its dependencies through the internal/ packages it imports, so anything stopping at direct imports attributes almost nothing.

What counts as a service is a question about your repo, so the provider doesn't guess. It reports the graph and a grouping decides. Attribution itself doesn't change.

The Used By column can be grouped by owner. getUsedByGroups lets a plugin return a map of label to packages, so a dependency shows up under each team that uses it instead of under one combined label. The column and its count come from that map, so sorting and filtering match what's on screen.

Groupings that span ecosystems get one set of pages. They were written once per provider and the nav only linked the first, so the Go pages were built and unreachable. They now live at the site root, built from the merged dependency list, with each row's detail link resolved to the provider holding that dependency's pages. Groupings that set ecosystems still get pages per matching provider. URLs for the rest move from <provider>/<grouping>/ to <grouping>/, which breaks existing links to them.

Rollup pages split by ecosystem. Once a grouping covers every provider, its pages mix Go and npm in one alphabetical run, which reads worse than the per-provider pages they replace. Both the index and each value's page now use a heading and count per ecosystem, named the way someone would say it, so gomod reads as Go. The index is where this matters most: when values are apps and services, no single value mixes ecosystems but the list of them does. Pages covering one ecosystem keep the flat list.

GroupingConfig.getValue gets the ecosystem. The store is already scoped to it but never said which one, so a plugin tracking owners per ecosystem couldn't tell a Go module from an npm package of the same name. Existing two-argument groupings keep working.

Checked on a generated pnpm + go site: pages at teams/, nav pointing there rather than at pnpm/teams/, detail links resolving to ../go/details/ and ../pnpm/details/, and a Go dependency's Used By reading {"Services":["billing"]}.

Also fixes two plugin-doc examples that showed the pre-0.2.0 column signature and don't compile, and the Go version floor: go list -json=<fields> needs 1.19, not the 1.16 the page claimed.

Geese don't file under one flock, and neither should a backend.

stevelandey-byte and others added 2 commits October 1, 2026 11:20
A Go module is one unit to `go list -m`, so every dependency landed on
the module and a backend with many binaries under cmd/ read as a single
consumer. Groupings had no per-team view of it, unlike the Node side
where dependencies attribute to the workspace packages naming them.

GoProvider takes a `consumerOf` callback naming the consumer each package
directory belongs to. When given one it reads each package's imports,
resolves them to the module providing them by longest prefix, and groups
the result per consumer. Test-only imports become dev dependencies.
Modules nothing imports stay on the module rather than disappearing.
Opt-in, since reading imports needs the module sources rather than only
the go.mod files `go list -m all` fetches, and it falls back to the old
attribution when the package list can't be read.

Configuring that needs a provider the CLI didn't build, which meant
constructing all of them, and the provider classes weren't exported.
They are now, and `providers` accepts a factory handed the CacheService
and the resolved repo root and cache dir, which the CLI only has after
reading its flags.

GroupingConfig.getValue now takes a context carrying the ecosystem. The
scoped store doesn't name the ecosystem it belongs to, so a plugin
holding ownership per ecosystem couldn't tell a Go service from an npm
app sharing its name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`go list -m` sees a module as one unit, so every dependency is
attributed to the module: a backend with many binaries under cmd/ reads
as a single consumer, and groupings have no per-team view of it the way
they do for a pnpm workspace.

Deciding what a service is belongs to the repo, not to the provider, so
rather than taking a callback the Go provider records what Go knows and
leaves the policy downstream. A new source writes two dependency facts:
goImportedBy, the module's own packages that import it, and goBinaries,
the main packages that reach it through those packages. The second is
the useful one for ownership, since a thin cmd/ binary reaches its
dependencies through the internal packages it imports rather than
importing them itself.

Attribution is unchanged. The facts are skipped, with a message, when
the module's sources aren't present.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@irskep irskep changed the title Let a repo attribute Go dependencies to its own services Publish the Go import graph as facts, and let a repo build its own provider list Oct 1, 2026
stevelandey-byte and others added 7 commits October 1, 2026 11:37
The factory only existed to hand callers a CacheService the provider
constructors ask for and never use at construction. MiseProvider takes
one purely to match the shape and ignores it. Documenting that as a
design was wrong, and building an API around it was worse.

Exporting the provider classes was the other half of the original
request and covers the same need on its own, so `providers` goes back to
a plain array and ProviderFactory goes away.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With the import graph published as facts, the stock Go provider already
does the job, so nothing needs constructing and there is no reason to
widen the public API.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FactStore is a type-only export, so importing it as a value breaks for
any consumer using isolatedModules.

The grouping section didn't say getValue can return several values, which
the Go docs now rely on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Groupings key on whatever a plugin decides, so a generic explanation
shouldn't lean on teams as if they were a Dependicus concept. The Go
example also claimed three services land on three team pages, which its
own code contradicts: it maps binaries to teams through a Set, so
several services can be one team.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`go list ./...` exits non-zero when any one package has an unresolvable
import, which is routine mid-change, but it still writes valid JSON for
every package it did read. That output was being discarded, dropping the
module's whole graph, and the message blamed missing sources for what is
usually a broken package. Use the partial output, and report the actual
reason when there is none.

The grouping example in the Go docs was written against the context-object
signature from an abandoned revision and doesn't compile.

`go list -json=<fields>` needs Go 1.19, not the 1.16 the page claimed.

The longest-prefix test couldn't fail: no dependency in its fixture was a
prefix of another, so reversing the resolution loop passed it. Replaced
with one that catches that, and renamed the old assertion to the
test-import exclusion it was really checking. Added coverage for
XTestImports, for first-party packages never counting as dependencies,
and for the partial-output path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sources run together under Promise.all, so a synchronous `go list` on a
large module stalled the network-bound sources beside it. Use the
promisified execFile the rest of the codebase uses for this.

A bare execFile mock has no promisify.custom, so the test helper hands
the callback the { stdout, stderr } shape the real one resolves to. It
also now pins the command and its field list, which nothing checked: the
field list could have been wrong and every test still passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dependicus attributes dependencies to packages; teams are something a
plugin's grouping decides. Say what is actually missing, which is that
one Go module is one consumer where the Node providers give you the
workspace packages that name each dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@irskep irskep changed the title Publish the Go import graph as facts, and let a repo build its own provider list Publish the Go import graph as facts Oct 1, 2026
@irskep
irskep requested review from mblair and srubin October 1, 2026 20:18
stevelandey-byte and others added 3 commits October 1, 2026 13:46
getUsedByGroupKey only labels the set: one key, every consumer under it.
That renders as a comma-joined label over a single pill, and it is the
reason a Go dependency shows one "golang" chip no matter how many
services reach it. The npm side degrades the same way when a package is
used across several teams.

getUsedByGroups returns the map instead, so a dependency can appear under
several owners with the right consumers beneath each. The browser
formatter already renders a multi-key map; nothing ever fed it one. The
flat Used By column and its count now come from the map's values, so
sorting and filtering match what's on screen. Without the hook nothing
changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Grouping pages were written once per provider, and the nav could only
point at one of them, so a grouping's Go pages existed under go/ with
nothing linking to them. Which provider won was whichever came first.

A grouping that names no ecosystems covers them all, so it now gets a
single tree at the root built from the merged dependency list, with each
row's detail link resolved to the provider directory holding that
dependency's pages. Groupings that do name ecosystems still get a tree
per matching provider, and nav entries carry their own prefix so both
kinds are reachable from any page.

Grouping values are read through each dependency's own scope rather than
one provider's, since a merged page spans ecosystems.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@irskep irskep changed the title Publish the Go import graph as facts Make Go support organizationally viable Oct 1, 2026
stevelandey-byte and others added 3 commits October 1, 2026 14:34
A rollup page that covers several providers listed Go modules and npm
packages in one alphabetical run, which is hard to read and hides how
much of a team's surface is in which language. Split the list under a
heading per ecosystem, named the way someone would say it rather than
by its identifier, so gomod reads as Go.

Pages covering one ecosystem keep the flat list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
When a grouping's values are apps and services, the index is where
ecosystems mix even though no single value does: a page listing
client-ui next to wal_streamer gives a reader nothing to tell an app
from a Go service. Only the detail pages were split.

The index now uses the same headings and counts. A value whose
dependencies span ecosystems is listed under each, which is rare and
more use than hiding it under whichever came first. The entry total at
the top stays the total.

Also pluralizes the per-value dependency count, which read "1
dependencies" on a line this rewrites.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Listing a value under every ecosystem it spans is right when values are
individual packages, and degenerate when they aren't: a team-per-value
rollup where every team owns both Go and npm repeated the entire list
under each heading.

Split only when some value covers a strict subset of the index's
ecosystems. Each value's set is a subset of the union, so that is just a
smaller one. The detail pages are unaffected, since a dependency belongs
to exactly one ecosystem.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@irskep

irskep commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator Author

I know I added a bunch of stuff to this after adding reviewers but I swear it's done now!

Might tweak docs a bit.

stevelandey-byte and others added 6 commits October 2, 2026 09:31
Each of these led with machinery and made the reader assemble the point
from it: what `go list -m` does before what you get, the old hook's
limitation before the new one's use, four ideas in one paragraph about
groupings. Lead with what the feature is for, then say how it works.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The example guarded on `ecosystem !== 'gomod'` and read a Go fact, which
makes the hook look Go-specific on a page about plugins in general. It
isn't; the guard was only there because `goBinaries` is a Go fact.

Group the consumers the dependency already has by their owning team,
which is the ordinary use and needs no ecosystem check. The Go version
of this is already on the Go provider page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
It was written for someone auditing the API: here is a hook, here is
what it returns, here is the older hook it replaces. Someone reading
the plugin docs wants their dashboard to show who owns what.

Lead with the problem, say what the cell looks like afterward, then the
code. The notes now cover what an author actually hits: `Unknown` sorts
last, a single label drops the expander, sorting follows the groups.
Drops the aside about `getUsedByGroupKey`, which only matters to someone
already using it, and the API reference has it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bullets stand on their own after the example.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moving merged groupings to the site root broke every link a plugin
built: a dependency's page lives under its provider's directory, so the
`../details/…` a plugin writes resolves to nothing. BasicCompliancePlugin
built its flagged entries that way.

A plugin can't know where those pages sit, and the answer differs
between a grouping's own tree and a provider's, so it shouldn't be
constructing the path at all. GroupingFlag.detailLink is now optional
and filled in from the flag's own name and version. For a link inside a
section's HTML, which the writer can't rewrite, GroupingDetailContext
carries detailLinkFor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
These read like instructions to an implementer: don't do this, do that
instead. Docs and release notes describe what the thing does and leave
the reader to decide.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants