From c89fa7984bd8dd76831e9e980149f343ead05354 Mon Sep 17 00:00:00 2001 From: Jeff Crouse Date: Mon, 2 Nov 2026 18:56:47 -0500 Subject: [PATCH 1/3] ADR-0023: the commons closes, and its data stays published Accepted 2026-11-02. The maintainer's Familiar has stopped looking up and contributing (Familiar ADR-0159 point 6). Nothing on the server has changed. Co-Authored-By: Claude Opus 5.5 --- ...ons-closes-and-its-data-stays-published.md | 191 ++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md diff --git a/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md new file mode 100644 index 0000000..2a5c63e --- /dev/null +++ b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md @@ -0,0 +1,191 @@ +# ADR-0023: The Commons Closes, and Its Data Stays Published + +Status: accepted + +Date: 2026-11-02 + +Implementation: + +- **Point 3, first half, done 2026-11-02.** The maintainer's Familiar has lookup and contribution + switched off (Familiar `ADR-0159` point 6). No contribution arrives after this. Familiar's + `ADR-0159` code change and release are outstanding; nothing on this side has changed yet. + +Supersedes [ADR-0003](ADR-0003-the-commons-runs-on-one-small-server.md), +[ADR-0011](ADR-0011-the-commons-is-what-other-tools-plug-into.md) point 1, +[ADR-0013](ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md) points 3 and 6, and +[ADR-0022](ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md). Rejects +[ADR-0021](ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md). Executed together +with Familiar's `ADR-0159`, which goes first. + +## Context + +`ADR-0001` set out to build a public commons of CLAP embeddings: computed once, shared, worth +running while empty. Every accepted record since has been about making that commons correct, +legible and reachable. **It never got a second contributor.** The project's own notes have called +that the bottleneck since 2026-09-20: "the next step is somebody else replying." + +**What the commons is, measured 2026-11-02.** From the public API and the weekly export: + +| | | +|---|---| +| Rows | **26,401**, all one pipeline identity (`laion/clap-htsat-unfused+frontend1+artifact1+pool1+fp32`) | +| Contributors | One: Familiar, on the maintainer's library. The rows grew 25,930 → 26,401 since 2026-09-19, which is that library's new tracks | +| Last contribution | 2026-10-26 | +| Recording-keyed rows (`ADR-0020`) | **0**. The "first recording-keyed row" `ADR-0020` was waiting on never came | +| AcoustID-id claims (`ADR-0019` point 6) | none. `acoustid_claims.csv.gz` in the 2026-11-01 export is a 66-byte header | +| Picard plugin zip, `picard-v0.2.0` | **0 downloads** | +| GitHub | 1 star, 0 forks, no issues from anyone else, 30 of 30 PRs by the maintainer | +| PyPI, last month, mirrors excluded | `clapback-client` 884, `clapback-cli` 597, `beets-clapback` 493, `clapback-embed` 213 | + +The PyPI figures are not users. A package with no installations reporting into the corpus, +and no second pipeline identity, still shows hundreds of monthly downloads from CI and indexers. +`clapback-embed`'s share is mostly Familiar's own builds. The corpus is the measure that counts, +and it has one contributor. + +**What the outreach produced, read 2026-11-02.** Of `ADR-0011` point 5's four approaches: + +- **beets#7032**: closed 2026-09-15, too new to list, and against beets' AI policy. +- **picard-plugins#436**: no reply since it was opened on 2026-09-15. +- **dj-track-similarity#2**: on 2026-09-16 the maintainer wrote "not saying no, but it is too + early for me to take it". Nothing since. +- **KalinkaPlayer#128**, the prospective contributor who prompted `ADR-0013` to `ADR-0020`: + **parked on 2026-09-21 and closed as not planned on 2026-10-29.** The project's notes still + recorded "waiting on madenvel" six weeks after madenvel answered. That premise was wrong and is + recorded here so nobody goes on waiting. + +**madenvel's reasons are worth keeping, because they are structural and not about effort.** The +first is the cold start: a new pipeline identity starts at zero rows, and "the early ones pay for +the late ones". The second decided it. `export.sh` writes one file per pipeline identity, so **an +identity with one or two contributors exports a file that *is* one person's library**, joinable +to MBIDs through `claims.csv.gz` and dated to the day. The anonymity `ADR-0013` point 4 provides +comes from crowd size, and a new identity has no crowd. The period when contributing gives a user +nothing back is the period when it discloses the most about them. madenvel also asked for +contributions to be shuffled, and for a minimum contributor count before an identity is exported. +Neither exists. + +**That argument applies to the corpus as it stands.** The one identity in the export has one +contributor, so the published CC0 file is the maintainer's library, every week. That is the +maintainer's own data and was published knowingly, which is why it is acceptable. It also means +the commons could not have offered a second contributor what madenvel asked for without new work. + +**A guard `ADR-0013` relied on does not exist.** `packages/server/deploy/export.sh:58` says +"tests/test_export_script.py reads this file and fails if one appears", meaning a `client_id` +column. There is no `test_export_script.py` anywhere in the repository, and nothing else exercises +the script. madenvel found this. Checked 2026-11-02. The rule held because the script is correct, +not because anything enforced it. + +**What it costs.** `ADR-0003` records the instance as Lightsail `small_3_0` at **$12 a month** +(bundle prices checked 2026-09-04). Backup and export storage is cents. The live bill was not read +for this record, because the AWS session had expired. The bundle should be confirmed with +`aws lightsail get-instances` before the instance is deleted. + +**What depends on the server.** Familiar looks up every new track by default +(`community_cache_enabled = True`, URL hardcoded) and contributes on the maintainer's installation. +It falls back to local analysis on any failure. But a lookup to a host that **resolves and does not +answer** waits out a 10-second timeout three times (`MAX_RETRIES = 3` in Familiar's +`community_cache.py`), about 32 s per lookup and two lookups per track. That is roughly a minute +added to a 36.6 s median analysis (Familiar `ADR-0144`). A host that **does not resolve** fails instantly. How the server is +turned off decides which one every Familiar installation gets. The three plug-ins and +`clapback-cli` fail the same way and matter less: none has a measured user. + +## Decision + +1. **The public commons at `clapback.seethroughlab.com` is shut down.** The Lightsail instance is + deleted, with its static IP, and any snapshots are removed. The site, the API, the backup, + export and disk-alert timers all go with it. + +2. **The DNS record is deleted, not left pointing anywhere.** The `A` record for `clapback` at + Porkbun is removed in the same sitting as the instance, so every client fails at resolution, + instantly, instead of timing out against an address that no longer answers. A released static IP + can also be reassigned to somebody else's server. A client still configured with the old name + would then send its requests, with a `client_id` and library hashes, to a stranger. + +3. **Familiar stops calling the commons first** (Familiar `ADR-0159`). The maintainer's + installation has lookup and contribution switched off through the settings API before anything + on this side changes, so the last contribution is known before the final export is taken. + +4. **One final export is taken and left published** (supersedes `ADR-0013` points 3 and 6). + After point 3, `clapback-export.service` is run once more by hand. That export stays in + `s3://clapback-export` permanently, and the weekly ones before it are deleted. Its file URLs and + SHA256s go in the repository README, because the `/export` page that lists them today is on the + instance. The CC0 dedication (`ADR-0013` points 1 and 2) stands. This is what keeps the promise + `ADR-0013` was written to make to the first prospective contributor, that the data would outlive + the box. + +5. **One final `pg_dump` is kept in `s3://clapback-backup`**, private, and older dumps are + deleted. It is the only copy carrying what the export strips (`client_id`, agreement history, + `ip_stats`), and it is what a revival would restore from. + +6. **The repository stays open, and `clapback-embed` stays maintained.** Familiar depends on it + (`clapback-embed>=0.1.0,<0.2`, Familiar `ADR-0105`) and has no other CLAP runtime. **No package is + yanked.** Yanking `clapback-embed` would break Familiar's installs, and yanking the others gains + nothing. Their READMEs and the GitHub repository description say the commons is closed and point + to the final export. `clapback-client`, `clapback-cli` and `beets-clapback` get no further + releases. The Picard release's notes say the same. + +7. **The server code stays in `packages/server/`, unbuilt and undeployed.** `ADR-0013` point 7's + self-hosting path, restoring from an export, still works from it. That is the whole revival + plan, and it costs nothing to keep. + +8. **Notes to the two open threads are written by the maintainer** (picard-plugins#436 and + dj-track-similarity#2), per `ADR-0011`'s Implementation block. KalinkaPlayer#128 is already + closed and needs nothing, though madenvel's export finding deserves a thank-you if one is sent. + +9. **Execution order:** point 3 (Familiar's settings now, Familiar `ADR-0159` built and released) + → point 4's final export, and its URLs in the README → point 5 → confirm the bundle and the bill + → points 2 and 1 in one sitting, DNS first → point 6's READMEs and repository description → + point 8. `CLAUDE.md`'s "what is running" section is rewritten last, as "what was running". + +## Alternatives Considered + +**Keep running it at $12 a month and wait.** The commons costs little, the work is done, and +dj-track-similarity#2 is open, not refused. Rejected because waiting has been the plan since +2026-09-20 and has produced one refusal. The one prospective contributor who engaged in depth +named conditions this project has not built: a minimum contributor count before an identity is +exported, coarsened dates, shuffled contributions. Meeting them is new work for a population of +zero. A real second contributor can still restore the commons from point 5's dump, and would arrive +to a better-designed one. + +**Shrink it rather than close it**: the cheapest Lightsail bundle, or scale-to-zero on another +host. Rejected because RAM is `ADR-0003` point 3's binding constraint, and the HNSW index wants +its pages resident. The 2 GB bundle is already the small end. A cold-starting similarity service +answers the one client it has more slowly, for a saving of a few dollars. It is still a server, +backups and an alert channel for one user. Familiar's cache hits are mostly its own work coming +back, which a local database already has. + +**Fold the commons into Familiar's own server**, as a table beside its library. That is in effect +what it already is: one library's vectors, which Familiar keeps in `TrackAnalysis` anyway. +Rejected as a *decision* because there is nothing to fold. Familiar loses nothing it does not +already hold. + +**Delete everything, the export included.** Simplest, and it removes the one public file that is +one person's library. Rejected because `ADR-0013` made a public promise that the data would outlive +the box, the data is CC0 and copies may already exist, and the person whose library it is chose to +publish it. If that choice changes, deleting the final export is one command and needs no record. + +**Hand the commons to someone else to host.** Rejected for lack of a taker, not on principle. +Point 7 keeps it possible. + +## Consequences + +- **Positive:** About $12 a month, plus the maintenance of a public server (TLS, backups, the + disk alert, Tailscale-only SSH) for an audience of one. +- **Positive:** The data outlives the box, as `ADR-0013` promised, in the one form that needs no + server. +- **Positive:** Every Familiar installation fails fast once point 2 is done, including ones never + updated to `ADR-0159`. +- **Tradeoff:** A cache hit, about 28 s saved per already-analysed track, is gone for Familiar. In + practice that is the maintainer's own earlier work, re-served after a re-analysis or reinstall. +- **Tradeoff:** `ADR-0001` to `ADR-0022`, written between 2026-09-01 and 2026-09-20, now describe + a system that is not running. They stay as written. The records most worth re-reading in a revival are `ADR-0007` + (attestation, deliberately unbuilt), `ADR-0008`, `ADR-0019` (the key is path-dependent) and + `ADR-0021`'s measurement. +- **Tradeoff:** The published export is one person's library under CC0, permanently. That is + accepted here knowingly, and it is the condition madenvel would not ask an early adopter to + accept. +- **Follow-up:** If the commons is ever revived, madenvel's three conditions come before any + outreach: suppress an identity from the export below a minimum number of distinct clients, + coarsen `created` for young identities, and let clients trickle contributions in shuffled order. + So does the missing export test. +- **Follow-up:** `CLAUDE.md` and the memory notes that call the second contributor the bottleneck + are rewritten when this is executed. From 8bc88fe0ac55ec1c1b0a2bcc8d9137ea677c2e9b Mon Sep 17 00:00:00 2001 From: Jeff Crouse Date: Tue, 3 Nov 2026 22:17:51 -0500 Subject: [PATCH 2/3] ADR-0023 executed: the commons is closed The final export (2026-11-03) is in the README with SHA256s; the server, DNS record, static IP and snapshots are deleted. The README, the four server-dependent package READMEs and CLAUDE.md say so; ADR-0023's Implementation block records each step and two corrections (bucket expiry rules, the import script that never existed); superseded and rejected records' Status lines are updated. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 47 +++++++---- README.md | 81 ++++++++++++++----- ...03-the-commons-runs-on-one-small-server.md | 2 +- ...e-commons-is-what-other-tools-plug-into.md | 2 +- ...-public-data-not-just-a-public-endpoint.md | 2 +- ...hes-bridges-between-pipeline-identities.md | 2 +- ...-pitched-on-discovery-not-saved-compute.md | 2 +- ...ons-closes-and-its-data-stays-published.md | 30 ++++++- packages/beets-clapback/README.md | 4 + packages/cli/README.md | 4 + packages/client/README.md | 4 + packages/picard-clapback/README.md | 4 + 12 files changed, 144 insertions(+), 40 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 11cdf15..dbfd430 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,6 +5,14 @@ embedding is a 512-dimensional vector describing what a recording sounds like: e trivial to compare, and therefore worth computing once and sharing — provided everybody computes the same thing. +**The commons is closed, as of 2026-11-03** (`ADR-0023`). The server, its DNS record and its +snapshots are deleted. What remains: this repository, `clapback-embed` (still maintained, because +Familiar depends on it), one permanent CC0 export in `s3://clapback-export/exports/2026-11-03/` +(URLs and SHA256s in the README), and one private final dump, +`s3://clapback-backup/postgres/clapback-2026-11-03.sql.gz`. Neither bucket expires anything current +any more. Most of what follows is the record of the project as it ran from September to November +2026, in the present tense it was written in. + ## Architecture - **Server** (`app/`): FastAPI + PostgreSQL (pgvector) + SQLAlchemy 2.0 async, Alembic migrations. @@ -226,14 +234,21 @@ discovery. Familiar still never calls `/v1/similar`; that is Familiar's decision `ADR-0008` closes `ADR-0001` deferred item 2 — the part the cross-machine measurement did not answer, being where the agreement threshold sits and what the corpus does with it. -**What is still owed, in execution order:** - -| ADR-0001 item | Decision | Why here | -|---|---|---| -| 4 | The recording-id key | `ADR-0002` point 4 makes it a prerequisite: similarity search over a hash-keyed corpus returns hashes nobody can resolve | -| 6 | The rename, and what the domain serves | Cheap, and last on purpose — nothing above depends on it | - -### What is actually running, as of 2026-09-19 +**`ADR-0023` was accepted 2026-11-02 and executed 2026-11-03: the commons closed.** In two months +it had one contributor, the maintainer's Familiar: 26,401 rows, one pipeline identity, 0 +recording-keyed rows, 0 Picard downloads. None of the four outreach threads produced a second +contributor. KalinkaPlayer#128 was parked on 2026-09-21 and closed on 2026-10-29, for reasons +worth keeping: the cold start, and that an export of a young identity is one person's library. +Familiar `ADR-0159` (released in Familiar `v0.2.0-beta22`) removed Familiar's client first. Then +came the final export and dump, the DNS record deleted before the instance so old clients fail +instantly, and the instance, IP and snapshots deleted. The Implementation block records each step, +and two corrections: the buckets' expiry rules would have deleted the "permanent" export, and +`ADR-0013` point 7's import script never existed. `ADR-0021` is rejected. **Nothing is owed except +the GitHub description, the Picard release notes, and the maintainer's notes on picard-plugins#436 and +dj-track-similarity#2.** A revival starts with `ADR-0023`'s Follow-ups: madenvel's three export +conditions and the missing export test, before any outreach. + +### What was running, until 2026-11-03 (as written 2026-09-19) The commons is public at **https://clapback.seethroughlab.com** — one AWS instance, TLS via Caddy, nightly `pg_dump` to `s3://clapback-backup`, and a corpus of **25,930 rows (2026-09-19), every one of which @@ -291,8 +306,8 @@ which it was. The Implementation block records what that corrected — **message maintainers are written by Jeff, AI use is disclosed where a project asks, and a project's AI policy is read before anything is sent.** -**The bottleneck is a second contributor, not a decision, and not a build.** Everything queued -that this project can do alone is done; the next step is somebody else replying. +**The bottleneck was a second contributor, not a decision, and not a build.** Nobody replied in a +way that produced one, and `ADR-0023` closed the commons on 2026-11-03. **The pattern worth keeping.** For most of this project's life the decisions ran far ahead of the code, and the `Implementation:` block is the only thing that kept that legible. Write one the day @@ -301,7 +316,8 @@ and are marked so rather than quietly overtaken. ## Compatibility with Familiar -[Familiar](https://github.com/seethroughlab/familiar) is the only client. `ADR-0001` point 1 says it +[Familiar](https://github.com/seethroughlab/familiar) was the only client of the server, and is +still the only dependant of the package. `ADR-0001` point 1 says it does not own this project; it does constrain it. **Check both surfaces before changing either, and verify against the sibling checkout rather than from memory** — the audit in `ADR-0005` is dated and will go stale. @@ -313,10 +329,11 @@ from the top level, and reaches into two submodules by path: `clapback_embed.art is part of the contract**, not an implementation detail. `ADR-0005` point 11 widens `__all__` to match. -**The server.** Familiar calls `/v1/embeddings`, `/v1/features` and `/v1/analysis-detail` — GET and -POST on each — plus `/health`. **The features and analysis-detail endpoints are legacy for the -corpus but live for the client.** `ADR-0001` point 7 decided no existing feature rows migrate; that -is a statement about what the corpus carries, and it does not license removing the endpoints. +**The server.** Familiar no longer calls it: Familiar `ADR-0159` (Familiar `v0.2.0-beta22`) +removed its client. Before that it called `/v1/embeddings`, `/v1/features` and +`/v1/analysis-detail` (GET and POST on each) plus `/health`. Installations older than beta22 still +try. They fail instantly, because the name no longer resolves, and analyse locally. **The package +surface above is the only contract left.** **The API is the only way in.** Clients and tools — Familiar, the CLI of `ADR-0001` point 3, anything else — reach the corpus over HTTP and never over a database connection. Every guarantee the diff --git a/README.md b/README.md index 3b0d609..d817363 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,40 @@ A public commons of **CLAP audio embeddings**, and the reference implementation that produces them. +> **The commons is closed.** The server at `clapback.seethroughlab.com` was shut down in +> November 2026 because no library besides its maintainer's ever contributed +> ([`ADR-0023`](docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). +> Its data stays published under CC0 (see [The final export](#the-final-export) below). The +> embedder, `clapback-embed`, is still maintained. The other packages get no further releases. + +## The final export + +Taken 2026-11-03 13:24 UTC, after the last contribution (2026-10-26). These files are +permanent and are the only form the corpus now takes. Licence: [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/). +No `client_id` and no address is included +([`ADR-0013`](docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md)). + +| File | Rows | SHA256 | +|---|---|---| +| [`embeddings-laion_clap_htsat_unfused_frontend1_artifact1_pool1_fp32.csv.gz`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/embeddings-laion_clap_htsat_unfused_frontend1_artifact1_pool1_fp32.csv.gz) | 26,401 | `cd76e5a7e715e3386e63ae749c2397010ddf7834090f4d8e155ba02340d5ef0e` | +| [`claims.csv.gz`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/claims.csv.gz) | 23,648 | `fd648291e8ba9ce9a47f562a721b71ac3420ebf28af7517533471d586b686644` | +| [`acoustid_claims.csv.gz`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/acoustid_claims.csv.gz) | 0 | `ede39b651189ac67d75201c4de650a5836c7d4185ad4b2cdea61aca943447b45` | +| [`manifest.json`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/manifest.json) | | `49e0b57af4cadfaac12fb661ada193dae5b02080c1cea69f285f4f887875c4a2` | + +Every embedding is from one pipeline identity, +`laion/clap-htsat-unfused+frontend1+artifact1+pool1+fp32`. Each row is keyed on the SHA256 of a +canonical AcoustID fingerprint. Rows that name a MusicBrainz recording do so through +`claims.csv.gz`. A revival would start from the server code in `packages/server/` and the +`COPY` statements in its `deploy/RUNBOOK.md`. The import script `ADR-0013` point 7 promised was +never written. + +## What this was + +Everything below describes the commons as it was built and run from September to November +2026. None of it is running. The API section documents a server that no longer exists, kept +because `packages/server/` still implements it. + + A streaming service can recommend music because it holds the catalogue. A tool that manages music its user *owns* — beets, Picard, a self-hosted player — holds one library, so its recommendations stop at the last track. This corpus is the catalogue those tools @@ -15,17 +49,20 @@ reverse ([`ADR-0022`](docs/decisions/ADR-0022-the-commons-is-pitched-on-discover A result is a MusicBrainz recording you can open, or a hash when nobody has named it yet. Where to hear it is the tool's job; the commons holds no titles, no audio and no links. -The commons is public at **https://clapback.seethroughlab.com**. Reads need no key and no -account. Contribution is open, opt-in in every client, and sends a vector and a one-way +The commons was public at `https://clapback.seethroughlab.com`. Reads needed no key and no +account. Contribution was open, opt-in in every client, and sent a vector and a one-way hash — never audio, never a filename. -## Take part +## The packages -The commons is the product. It is meant to be reached from the tools people already run, -and the way in is a package with no dependency beyond the standard library +The commons was meant to be reached from the tools people already run, through a package with +no dependency beyond the standard library ([`ADR-0011`](docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md)). +**Every package below except `clapback-embed` needs the server, and gets no further +releases.** They stay on PyPI. `clapback-embed` needs no server and is still maintained: +Familiar depends on it. -| If you… | Install | What it is | +| If you… | Install | What it was | |---|---|---| | use **beets** | `pip install beets-clapback` | `absubmit` reborn: look up, else embed; contribute if you say so; `beet clapback-similar`. [README](packages/beets-clapback/) | | use **Picard** | the zip from a [`picard-v*` release](https://github.com/seethroughlab/clapback/releases?q=picard-v) | Right-click: look up, name, contribute; *what sounds like this*. Works with no embedder installed, and says so. [README](packages/picard-clapback/) | @@ -33,8 +70,8 @@ and the way in is a package with no dependency beyond the standard library | want a **command line** | `pip install clapback-cli` | The reference client: index a directory, search it by description, find duplicates, `clapback similar` — what sounds like a track, including what you don’t own — and contribute. [README](packages/cli/) | | need the **embedder** | `pip install clapback-embed` | The reference pipeline. ONNX Runtime, no `torch`. [README](packages/embed/) | -A tool that already computes CLAP vectors can contribute under its own pipeline -identity; they sit beside the reference's rather than being compared with it. +A tool that already computed CLAP vectors could contribute under its own pipeline +identity; they sat beside the reference's rather than being compared with it. ## The reference pipeline @@ -76,9 +113,16 @@ machine computed them. See [`packages/embed/`](packages/embed/) for the details [`packages/embed/scripts/compare_vectors.py`](packages/embed/scripts/compare_vectors.py) for the cross-machine check. -## Where this actually stands +## Where it ended + +At shutdown, measured 2026-11-02: **26,401 embeddings, all from one pipeline identity and one +contributor** (the maintainer's Familiar), 0 recording-keyed rows, and 0 downloads of the Picard +plugin. Four maintainer proposals went out on 2026-09-15; none led to a second contributor. +[`ADR-0023`](docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md) has the +numbers and the reasons, including a structural one: an export of a young pipeline identity is +one person's library. -Early, and worth being plain about. As of 2026-09-15: +What this section said on 2026-09-15: - **25,886 embeddings, all from a single contributor.** Every row declares the pipeline that produced it and is keyed on a hash any client can reproduce from the audio — @@ -129,13 +173,13 @@ Two licences, because they cover two different things. vector, and a recording id if you attach one. There is nobody to attribute — contributors are installations, not people — so no attribution or share-alike licence could be complied with. [`ADR-0013`](docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md) - is the decision; it also commits to a weekly public export and a documented path to running - your own copy, both of which are owed rather than built as of 2026-09-15. + is the decision. The weekly export it committed to ran from 2026-09-15 to 2026-11-03; the + final one is [above](#the-final-export) and is permanent. The import script it promised was + never written. ## API -Full reference at [`/api`](https://clapback.seethroughlab.com/api); schema at -[`/docs`](https://clapback.seethroughlab.com/docs). +The server that answered these is gone. This is what `packages/server/` implements. | Endpoint | Method | Description | |----------|--------|-------------| @@ -317,8 +361,9 @@ Read the relevant ones before changing anything they govern. [`ADR-0003`](docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md) chose one small AWS instance running both Postgres and the application, sized by index RAM rather -than corpus size, with the upgrade path written down. That is what runs: TLS via Caddy, -nightly `pg_dump` to S3, a disk alert that delivers to a phone. `docker-compose.aws.yml` +than corpus size, with the upgrade path written down. That is what ran, from 2026-09-04 until +it was deleted on 2026-11-03: TLS via Caddy, nightly `pg_dump` to S3, a disk alert that +delivered to a phone. `docker-compose.aws.yml` is the configuration and [`deploy/RUNBOOK.md`](packages/server/deploy/RUNBOOK.md) is how it was brought up and how it is operated. @@ -370,5 +415,5 @@ Environment variables (prefix: `CACHE_`): - **API**: FastAPI + uvicorn, SQLAlchemy 2.0 async, Alembic migrations. - **Database**: PostgreSQL with pgvector; `(fingerprint_hash, pipeline_version)` is the key, HNSW over the vectors, and recording ids in a claims table beside them. -- **Hosting**: one AWS Lightsail instance, per `ADR-0003`. No auto-deploy; the runbook - is the deploy. +- **Hosting**: one AWS Lightsail instance, per `ADR-0003`, deleted 2026-11-03 (`ADR-0023`). + No auto-deploy; the runbook was the deploy. diff --git a/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md b/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md index 449afbf..e1dc11c 100644 --- a/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md +++ b/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md @@ -1,6 +1,6 @@ # ADR-0003: The Commons Runs on One Small Server -Status: accepted +Status: superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-03 diff --git a/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md b/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md index 0112e9f..d505e74 100644 --- a/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md +++ b/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md @@ -1,6 +1,6 @@ # ADR-0011: The Commons Is What Other Tools Plug Into -Status: accepted +Status: accepted — point 1 superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-13 diff --git a/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md b/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md index 3b96431..0189271 100644 --- a/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md +++ b/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md @@ -1,6 +1,6 @@ # ADR-0013: The Corpus Is Public Data, Not Just a Public Endpoint -Status: accepted +Status: accepted — points 3 and 6 superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-15 diff --git a/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md b/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md index be09a19..1a752fe 100644 --- a/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md +++ b/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md @@ -1,6 +1,6 @@ # ADR-0021: The Corpus Publishes Bridges Between Pipeline Identities -Status: proposed +Status: rejected by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-19 diff --git a/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md b/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md index 92755cf..5953f87 100644 --- a/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md +++ b/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md @@ -1,6 +1,6 @@ # ADR-0022: The Commons Is Pitched on Discovery, Not Saved Compute -Status: accepted +Status: superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-20 diff --git a/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md index 2a5c63e..9d99b60 100644 --- a/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md +++ b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md @@ -7,8 +7,34 @@ Date: 2026-11-02 Implementation: - **Point 3, first half, done 2026-11-02.** The maintainer's Familiar has lookup and contribution - switched off (Familiar `ADR-0159` point 6). No contribution arrives after this. Familiar's - `ADR-0159` code change and release are outstanding; nothing on this side has changed yet. + switched off (Familiar `ADR-0159` point 6). No contribution arrives after this. +- **Point 3, second half, done 2026-11-03.** Familiar `ADR-0159` shipped in Familiar + `v0.2.0-beta22` (released 2026-11-03 01:24 UTC). The maintainer's installation runs it. +- **Point 4 done 2026-11-03.** The final export ran by hand at 13:24 UTC: 26,401 embeddings and + 23,648 claims, matching `GET /v1/pipelines` (last contribution 2026-10-26). Each file was + downloaded through its public URL and hashed, and the URLs and SHA256s are in the README. The + five earlier weekly exports, the `monthly/` copies and `latest/` were deleted. + **The point missed something:** both buckets had expiry lifecycle rules (`exports/` after 35 + days, dumps after 90), which would have deleted the "permanent" export around 2026-12-08. Both + rules were replaced with ones that expire only noncurrent versions and stale delete markers. + The export and backup timers were disabled before the instance went, so neither wrote again. +- **Point 5 done 2026-11-03.** `postgres/clapback-2026-11-03.sql.gz` (04:12 UTC, after the last + contribution) is kept, checked by decompressing it in full: 8 tables. The 59 older dumps were + deleted. +- **Bill confirmed 2026-11-03.** October's Lightsail charge was $13.73: $11.99 for the + `small_3_0` bundle, $1.73 for snapshots. The two buckets cost cents. +- **Points 2 and 1 done 2026-11-03, in that order.** The `A` record `clapback` → + `184.192.136.180` was deleted at Porkbun. All four Porkbun nameservers, 1.1.1.1 and 8.8.8.8 + answered `NXDOMAIN` before the instance was touched. Then the instance (with its seven automatic + snapshots), the static IP `clapback-ip` and the manual snapshot + `clapback-pre-hardening-2026-10-02` were deleted. Lightsail holds nothing. +- **Point 7 overstated what is kept.** It says the self-hosting path "still works" from + `packages/server/`. `ADR-0013` point 7's import script was never written. What exists is the + `COPY` SQL in `deploy/RUNBOOK.md`. A revival restores from point 5's dump, or writes the script. +- **Point 6 done 2026-11-03** in this repository: the README carries the final export and says + the commons is closed, and the four server-dependent packages' READMEs say so. Still owed: the + GitHub repository description and the `picard-v0.2.0` release notes. +- **Point 8 owed.** The maintainer writes both notes. Supersedes [ADR-0003](ADR-0003-the-commons-runs-on-one-small-server.md), [ADR-0011](ADR-0011-the-commons-is-what-other-tools-plug-into.md) point 1, diff --git a/packages/beets-clapback/README.md b/packages/beets-clapback/README.md index fa69ffe..d6a80bc 100644 --- a/packages/beets-clapback/README.md +++ b/packages/beets-clapback/README.md @@ -1,5 +1,9 @@ # beets-clapback +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). This plugin's lookup, `clapback-similar` and contribution all need that server, so it does nothing useful now. It stays on PyPI and gets no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + Look up, embed, and — if you say so — contribute CLAP embeddings for your beets library to the [clapback](https://clapback.seethroughlab.com) commons. `absubmit` reborn, with a payload that can actually be reconciled across contributors. diff --git a/packages/cli/README.md b/packages/cli/README.md index 521a420..77a3770 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,5 +1,9 @@ # clapback +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). `clapback similar` and `clapback contribute` need that server and no longer work. `index`, `search` and `duplicates` run locally and still do. The package stays on PyPI and gets no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + Search your own music by description, find duplicates across formats and masters, and ask the commons what sounds like a track — including recordings you don't own. diff --git a/packages/client/README.md b/packages/client/README.md index a367143..1fa7b5b 100644 --- a/packages/client/README.md +++ b/packages/client/README.md @@ -1,5 +1,9 @@ # clapback-client +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). This package exists to talk to that server. It stays on PyPI and gets no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + The contract a tool follows to take part in the [clapback](https://clapback.seethroughlab.com) commons — look up before you contribute, send what the records require, back off when told to — with no dependency beyond the standard library. diff --git a/packages/picard-clapback/README.md b/packages/picard-clapback/README.md index 8a2e3c1..1963c63 100644 --- a/packages/picard-clapback/README.md +++ b/packages/picard-clapback/README.md @@ -1,5 +1,9 @@ # Clapback for MusicBrainz Picard +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). This plugin's lookup, naming, contribution and *what sounds like this* all need that server, so it does nothing useful now. The release zip stays up, and there will be no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + Look up, name, and — if you say so — contribute CLAP audio embeddings for your files in the [clapback](https://clapback.seethroughlab.com) commons, and ask what sounds like a track across every library it holds. `ADR-0011` point 5's second integration: Picard already fingerprints every From 991df8250b7bfbf01a9bfce5edf01d4b6303cebe Mon Sep 17 00:00:00 2001 From: Jeff Crouse Date: Tue, 3 Nov 2026 22:18:05 -0500 Subject: [PATCH 3/3] ADR-0023: repository description and Picard release notes done Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 3 +-- ...DR-0023-the-commons-closes-and-its-data-stays-published.md | 4 ++-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index dbfd430..d1f7c3f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -244,8 +244,7 @@ came the final export and dump, the DNS record deleted before the instance so ol instantly, and the instance, IP and snapshots deleted. The Implementation block records each step, and two corrections: the buckets' expiry rules would have deleted the "permanent" export, and `ADR-0013` point 7's import script never existed. `ADR-0021` is rejected. **Nothing is owed except -the GitHub description, the Picard release notes, and the maintainer's notes on picard-plugins#436 and -dj-track-similarity#2.** A revival starts with `ADR-0023`'s Follow-ups: madenvel's three export +the maintainer's notes on picard-plugins#436 and dj-track-similarity#2.** A revival starts with `ADR-0023`'s Follow-ups: madenvel's three export conditions and the missing export test, before any outreach. ### What was running, until 2026-11-03 (as written 2026-09-19) diff --git a/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md index 9d99b60..8ea8718 100644 --- a/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md +++ b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md @@ -32,8 +32,8 @@ Implementation: `packages/server/`. `ADR-0013` point 7's import script was never written. What exists is the `COPY` SQL in `deploy/RUNBOOK.md`. A revival restores from point 5's dump, or writes the script. - **Point 6 done 2026-11-03** in this repository: the README carries the final export and says - the commons is closed, and the four server-dependent packages' READMEs say so. Still owed: the - GitHub repository description and the `picard-v0.2.0` release notes. + the commons is closed, and the four server-dependent packages' READMEs say so. The GitHub + repository description and the `picard-v0.2.0` release notes say the same. - **Point 8 owed.** The maintainer writes both notes. Supersedes [ADR-0003](ADR-0003-the-commons-runs-on-one-small-server.md),