diff --git a/CLAUDE.md b/CLAUDE.md index 11cdf15..d1f7c3f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,6 +5,14 @@ embedding is a 512-dimensional vector describing what a recording sounds like: e trivial to compare, and therefore worth computing once and sharing — provided everybody computes the same thing. +**The commons is closed, as of 2026-11-03** (`ADR-0023`). The server, its DNS record and its +snapshots are deleted. What remains: this repository, `clapback-embed` (still maintained, because +Familiar depends on it), one permanent CC0 export in `s3://clapback-export/exports/2026-11-03/` +(URLs and SHA256s in the README), and one private final dump, +`s3://clapback-backup/postgres/clapback-2026-11-03.sql.gz`. Neither bucket expires anything current +any more. Most of what follows is the record of the project as it ran from September to November +2026, in the present tense it was written in. + ## Architecture - **Server** (`app/`): FastAPI + PostgreSQL (pgvector) + SQLAlchemy 2.0 async, Alembic migrations. @@ -226,14 +234,20 @@ discovery. Familiar still never calls `/v1/similar`; that is Familiar's decision `ADR-0008` closes `ADR-0001` deferred item 2 — the part the cross-machine measurement did not answer, being where the agreement threshold sits and what the corpus does with it. -**What is still owed, in execution order:** - -| ADR-0001 item | Decision | Why here | -|---|---|---| -| 4 | The recording-id key | `ADR-0002` point 4 makes it a prerequisite: similarity search over a hash-keyed corpus returns hashes nobody can resolve | -| 6 | The rename, and what the domain serves | Cheap, and last on purpose — nothing above depends on it | - -### What is actually running, as of 2026-09-19 +**`ADR-0023` was accepted 2026-11-02 and executed 2026-11-03: the commons closed.** In two months +it had one contributor, the maintainer's Familiar: 26,401 rows, one pipeline identity, 0 +recording-keyed rows, 0 Picard downloads. None of the four outreach threads produced a second +contributor. KalinkaPlayer#128 was parked on 2026-09-21 and closed on 2026-10-29, for reasons +worth keeping: the cold start, and that an export of a young identity is one person's library. +Familiar `ADR-0159` (released in Familiar `v0.2.0-beta22`) removed Familiar's client first. Then +came the final export and dump, the DNS record deleted before the instance so old clients fail +instantly, and the instance, IP and snapshots deleted. The Implementation block records each step, +and two corrections: the buckets' expiry rules would have deleted the "permanent" export, and +`ADR-0013` point 7's import script never existed. `ADR-0021` is rejected. **Nothing is owed except +the maintainer's notes on picard-plugins#436 and dj-track-similarity#2.** A revival starts with `ADR-0023`'s Follow-ups: madenvel's three export +conditions and the missing export test, before any outreach. + +### What was running, until 2026-11-03 (as written 2026-09-19) The commons is public at **https://clapback.seethroughlab.com** — one AWS instance, TLS via Caddy, nightly `pg_dump` to `s3://clapback-backup`, and a corpus of **25,930 rows (2026-09-19), every one of which @@ -291,8 +305,8 @@ which it was. The Implementation block records what that corrected — **message maintainers are written by Jeff, AI use is disclosed where a project asks, and a project's AI policy is read before anything is sent.** -**The bottleneck is a second contributor, not a decision, and not a build.** Everything queued -that this project can do alone is done; the next step is somebody else replying. +**The bottleneck was a second contributor, not a decision, and not a build.** Nobody replied in a +way that produced one, and `ADR-0023` closed the commons on 2026-11-03. **The pattern worth keeping.** For most of this project's life the decisions ran far ahead of the code, and the `Implementation:` block is the only thing that kept that legible. Write one the day @@ -301,7 +315,8 @@ and are marked so rather than quietly overtaken. ## Compatibility with Familiar -[Familiar](https://github.com/seethroughlab/familiar) is the only client. `ADR-0001` point 1 says it +[Familiar](https://github.com/seethroughlab/familiar) was the only client of the server, and is +still the only dependant of the package. `ADR-0001` point 1 says it does not own this project; it does constrain it. **Check both surfaces before changing either, and verify against the sibling checkout rather than from memory** — the audit in `ADR-0005` is dated and will go stale. @@ -313,10 +328,11 @@ from the top level, and reaches into two submodules by path: `clapback_embed.art is part of the contract**, not an implementation detail. `ADR-0005` point 11 widens `__all__` to match. -**The server.** Familiar calls `/v1/embeddings`, `/v1/features` and `/v1/analysis-detail` — GET and -POST on each — plus `/health`. **The features and analysis-detail endpoints are legacy for the -corpus but live for the client.** `ADR-0001` point 7 decided no existing feature rows migrate; that -is a statement about what the corpus carries, and it does not license removing the endpoints. +**The server.** Familiar no longer calls it: Familiar `ADR-0159` (Familiar `v0.2.0-beta22`) +removed its client. Before that it called `/v1/embeddings`, `/v1/features` and +`/v1/analysis-detail` (GET and POST on each) plus `/health`. Installations older than beta22 still +try. They fail instantly, because the name no longer resolves, and analyse locally. **The package +surface above is the only contract left.** **The API is the only way in.** Clients and tools — Familiar, the CLI of `ADR-0001` point 3, anything else — reach the corpus over HTTP and never over a database connection. Every guarantee the diff --git a/README.md b/README.md index 3b0d609..d817363 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,40 @@ A public commons of **CLAP audio embeddings**, and the reference implementation that produces them. +> **The commons is closed.** The server at `clapback.seethroughlab.com` was shut down in +> November 2026 because no library besides its maintainer's ever contributed +> ([`ADR-0023`](docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). +> Its data stays published under CC0 (see [The final export](#the-final-export) below). The +> embedder, `clapback-embed`, is still maintained. The other packages get no further releases. + +## The final export + +Taken 2026-11-03 13:24 UTC, after the last contribution (2026-10-26). These files are +permanent and are the only form the corpus now takes. Licence: [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/). +No `client_id` and no address is included +([`ADR-0013`](docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md)). + +| File | Rows | SHA256 | +|---|---|---| +| [`embeddings-laion_clap_htsat_unfused_frontend1_artifact1_pool1_fp32.csv.gz`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/embeddings-laion_clap_htsat_unfused_frontend1_artifact1_pool1_fp32.csv.gz) | 26,401 | `cd76e5a7e715e3386e63ae749c2397010ddf7834090f4d8e155ba02340d5ef0e` | +| [`claims.csv.gz`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/claims.csv.gz) | 23,648 | `fd648291e8ba9ce9a47f562a721b71ac3420ebf28af7517533471d586b686644` | +| [`acoustid_claims.csv.gz`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/acoustid_claims.csv.gz) | 0 | `ede39b651189ac67d75201c4de650a5836c7d4185ad4b2cdea61aca943447b45` | +| [`manifest.json`](https://clapback-export.s3.us-east-1.amazonaws.com/exports/2026-11-03/manifest.json) | | `49e0b57af4cadfaac12fb661ada193dae5b02080c1cea69f285f4f887875c4a2` | + +Every embedding is from one pipeline identity, +`laion/clap-htsat-unfused+frontend1+artifact1+pool1+fp32`. Each row is keyed on the SHA256 of a +canonical AcoustID fingerprint. Rows that name a MusicBrainz recording do so through +`claims.csv.gz`. A revival would start from the server code in `packages/server/` and the +`COPY` statements in its `deploy/RUNBOOK.md`. The import script `ADR-0013` point 7 promised was +never written. + +## What this was + +Everything below describes the commons as it was built and run from September to November +2026. None of it is running. The API section documents a server that no longer exists, kept +because `packages/server/` still implements it. + + A streaming service can recommend music because it holds the catalogue. A tool that manages music its user *owns* — beets, Picard, a self-hosted player — holds one library, so its recommendations stop at the last track. This corpus is the catalogue those tools @@ -15,17 +49,20 @@ reverse ([`ADR-0022`](docs/decisions/ADR-0022-the-commons-is-pitched-on-discover A result is a MusicBrainz recording you can open, or a hash when nobody has named it yet. Where to hear it is the tool's job; the commons holds no titles, no audio and no links. -The commons is public at **https://clapback.seethroughlab.com**. Reads need no key and no -account. Contribution is open, opt-in in every client, and sends a vector and a one-way +The commons was public at `https://clapback.seethroughlab.com`. Reads needed no key and no +account. Contribution was open, opt-in in every client, and sent a vector and a one-way hash — never audio, never a filename. -## Take part +## The packages -The commons is the product. It is meant to be reached from the tools people already run, -and the way in is a package with no dependency beyond the standard library +The commons was meant to be reached from the tools people already run, through a package with +no dependency beyond the standard library ([`ADR-0011`](docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md)). +**Every package below except `clapback-embed` needs the server, and gets no further +releases.** They stay on PyPI. `clapback-embed` needs no server and is still maintained: +Familiar depends on it. -| If you… | Install | What it is | +| If you… | Install | What it was | |---|---|---| | use **beets** | `pip install beets-clapback` | `absubmit` reborn: look up, else embed; contribute if you say so; `beet clapback-similar`. [README](packages/beets-clapback/) | | use **Picard** | the zip from a [`picard-v*` release](https://github.com/seethroughlab/clapback/releases?q=picard-v) | Right-click: look up, name, contribute; *what sounds like this*. Works with no embedder installed, and says so. [README](packages/picard-clapback/) | @@ -33,8 +70,8 @@ and the way in is a package with no dependency beyond the standard library | want a **command line** | `pip install clapback-cli` | The reference client: index a directory, search it by description, find duplicates, `clapback similar` — what sounds like a track, including what you don’t own — and contribute. [README](packages/cli/) | | need the **embedder** | `pip install clapback-embed` | The reference pipeline. ONNX Runtime, no `torch`. [README](packages/embed/) | -A tool that already computes CLAP vectors can contribute under its own pipeline -identity; they sit beside the reference's rather than being compared with it. +A tool that already computed CLAP vectors could contribute under its own pipeline +identity; they sat beside the reference's rather than being compared with it. ## The reference pipeline @@ -76,9 +113,16 @@ machine computed them. See [`packages/embed/`](packages/embed/) for the details [`packages/embed/scripts/compare_vectors.py`](packages/embed/scripts/compare_vectors.py) for the cross-machine check. -## Where this actually stands +## Where it ended + +At shutdown, measured 2026-11-02: **26,401 embeddings, all from one pipeline identity and one +contributor** (the maintainer's Familiar), 0 recording-keyed rows, and 0 downloads of the Picard +plugin. Four maintainer proposals went out on 2026-09-15; none led to a second contributor. +[`ADR-0023`](docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md) has the +numbers and the reasons, including a structural one: an export of a young pipeline identity is +one person's library. -Early, and worth being plain about. As of 2026-09-15: +What this section said on 2026-09-15: - **25,886 embeddings, all from a single contributor.** Every row declares the pipeline that produced it and is keyed on a hash any client can reproduce from the audio — @@ -129,13 +173,13 @@ Two licences, because they cover two different things. vector, and a recording id if you attach one. There is nobody to attribute — contributors are installations, not people — so no attribution or share-alike licence could be complied with. [`ADR-0013`](docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md) - is the decision; it also commits to a weekly public export and a documented path to running - your own copy, both of which are owed rather than built as of 2026-09-15. + is the decision. The weekly export it committed to ran from 2026-09-15 to 2026-11-03; the + final one is [above](#the-final-export) and is permanent. The import script it promised was + never written. ## API -Full reference at [`/api`](https://clapback.seethroughlab.com/api); schema at -[`/docs`](https://clapback.seethroughlab.com/docs). +The server that answered these is gone. This is what `packages/server/` implements. | Endpoint | Method | Description | |----------|--------|-------------| @@ -317,8 +361,9 @@ Read the relevant ones before changing anything they govern. [`ADR-0003`](docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md) chose one small AWS instance running both Postgres and the application, sized by index RAM rather -than corpus size, with the upgrade path written down. That is what runs: TLS via Caddy, -nightly `pg_dump` to S3, a disk alert that delivers to a phone. `docker-compose.aws.yml` +than corpus size, with the upgrade path written down. That is what ran, from 2026-09-04 until +it was deleted on 2026-11-03: TLS via Caddy, nightly `pg_dump` to S3, a disk alert that +delivered to a phone. `docker-compose.aws.yml` is the configuration and [`deploy/RUNBOOK.md`](packages/server/deploy/RUNBOOK.md) is how it was brought up and how it is operated. @@ -370,5 +415,5 @@ Environment variables (prefix: `CACHE_`): - **API**: FastAPI + uvicorn, SQLAlchemy 2.0 async, Alembic migrations. - **Database**: PostgreSQL with pgvector; `(fingerprint_hash, pipeline_version)` is the key, HNSW over the vectors, and recording ids in a claims table beside them. -- **Hosting**: one AWS Lightsail instance, per `ADR-0003`. No auto-deploy; the runbook - is the deploy. +- **Hosting**: one AWS Lightsail instance, per `ADR-0003`, deleted 2026-11-03 (`ADR-0023`). + No auto-deploy; the runbook was the deploy. diff --git a/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md b/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md index 449afbf..e1dc11c 100644 --- a/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md +++ b/docs/decisions/ADR-0003-the-commons-runs-on-one-small-server.md @@ -1,6 +1,6 @@ # ADR-0003: The Commons Runs on One Small Server -Status: accepted +Status: superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-03 diff --git a/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md b/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md index 0112e9f..d505e74 100644 --- a/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md +++ b/docs/decisions/ADR-0011-the-commons-is-what-other-tools-plug-into.md @@ -1,6 +1,6 @@ # ADR-0011: The Commons Is What Other Tools Plug Into -Status: accepted +Status: accepted — point 1 superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-13 diff --git a/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md b/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md index 3b96431..0189271 100644 --- a/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md +++ b/docs/decisions/ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md @@ -1,6 +1,6 @@ # ADR-0013: The Corpus Is Public Data, Not Just a Public Endpoint -Status: accepted +Status: accepted — points 3 and 6 superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-15 diff --git a/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md b/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md index be09a19..1a752fe 100644 --- a/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md +++ b/docs/decisions/ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md @@ -1,6 +1,6 @@ # ADR-0021: The Corpus Publishes Bridges Between Pipeline Identities -Status: proposed +Status: rejected by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-19 diff --git a/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md b/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md index 92755cf..5953f87 100644 --- a/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md +++ b/docs/decisions/ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md @@ -1,6 +1,6 @@ # ADR-0022: The Commons Is Pitched on Discovery, Not Saved Compute -Status: accepted +Status: superseded by [ADR-0023](ADR-0023-the-commons-closes-and-its-data-stays-published.md) Date: 2026-09-20 diff --git a/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md new file mode 100644 index 0000000..8ea8718 --- /dev/null +++ b/docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md @@ -0,0 +1,217 @@ +# ADR-0023: The Commons Closes, and Its Data Stays Published + +Status: accepted + +Date: 2026-11-02 + +Implementation: + +- **Point 3, first half, done 2026-11-02.** The maintainer's Familiar has lookup and contribution + switched off (Familiar `ADR-0159` point 6). No contribution arrives after this. +- **Point 3, second half, done 2026-11-03.** Familiar `ADR-0159` shipped in Familiar + `v0.2.0-beta22` (released 2026-11-03 01:24 UTC). The maintainer's installation runs it. +- **Point 4 done 2026-11-03.** The final export ran by hand at 13:24 UTC: 26,401 embeddings and + 23,648 claims, matching `GET /v1/pipelines` (last contribution 2026-10-26). Each file was + downloaded through its public URL and hashed, and the URLs and SHA256s are in the README. The + five earlier weekly exports, the `monthly/` copies and `latest/` were deleted. + **The point missed something:** both buckets had expiry lifecycle rules (`exports/` after 35 + days, dumps after 90), which would have deleted the "permanent" export around 2026-12-08. Both + rules were replaced with ones that expire only noncurrent versions and stale delete markers. + The export and backup timers were disabled before the instance went, so neither wrote again. +- **Point 5 done 2026-11-03.** `postgres/clapback-2026-11-03.sql.gz` (04:12 UTC, after the last + contribution) is kept, checked by decompressing it in full: 8 tables. The 59 older dumps were + deleted. +- **Bill confirmed 2026-11-03.** October's Lightsail charge was $13.73: $11.99 for the + `small_3_0` bundle, $1.73 for snapshots. The two buckets cost cents. +- **Points 2 and 1 done 2026-11-03, in that order.** The `A` record `clapback` → + `184.192.136.180` was deleted at Porkbun. All four Porkbun nameservers, 1.1.1.1 and 8.8.8.8 + answered `NXDOMAIN` before the instance was touched. Then the instance (with its seven automatic + snapshots), the static IP `clapback-ip` and the manual snapshot + `clapback-pre-hardening-2026-10-02` were deleted. Lightsail holds nothing. +- **Point 7 overstated what is kept.** It says the self-hosting path "still works" from + `packages/server/`. `ADR-0013` point 7's import script was never written. What exists is the + `COPY` SQL in `deploy/RUNBOOK.md`. A revival restores from point 5's dump, or writes the script. +- **Point 6 done 2026-11-03** in this repository: the README carries the final export and says + the commons is closed, and the four server-dependent packages' READMEs say so. The GitHub + repository description and the `picard-v0.2.0` release notes say the same. +- **Point 8 owed.** The maintainer writes both notes. + +Supersedes [ADR-0003](ADR-0003-the-commons-runs-on-one-small-server.md), +[ADR-0011](ADR-0011-the-commons-is-what-other-tools-plug-into.md) point 1, +[ADR-0013](ADR-0013-the-corpus-is-public-data-not-just-a-public-endpoint.md) points 3 and 6, and +[ADR-0022](ADR-0022-the-commons-is-pitched-on-discovery-not-saved-compute.md). Rejects +[ADR-0021](ADR-0021-the-corpus-publishes-bridges-between-pipeline-identities.md). Executed together +with Familiar's `ADR-0159`, which goes first. + +## Context + +`ADR-0001` set out to build a public commons of CLAP embeddings: computed once, shared, worth +running while empty. Every accepted record since has been about making that commons correct, +legible and reachable. **It never got a second contributor.** The project's own notes have called +that the bottleneck since 2026-09-20: "the next step is somebody else replying." + +**What the commons is, measured 2026-11-02.** From the public API and the weekly export: + +| | | +|---|---| +| Rows | **26,401**, all one pipeline identity (`laion/clap-htsat-unfused+frontend1+artifact1+pool1+fp32`) | +| Contributors | One: Familiar, on the maintainer's library. The rows grew 25,930 → 26,401 since 2026-09-19, which is that library's new tracks | +| Last contribution | 2026-10-26 | +| Recording-keyed rows (`ADR-0020`) | **0**. The "first recording-keyed row" `ADR-0020` was waiting on never came | +| AcoustID-id claims (`ADR-0019` point 6) | none. `acoustid_claims.csv.gz` in the 2026-11-01 export is a 66-byte header | +| Picard plugin zip, `picard-v0.2.0` | **0 downloads** | +| GitHub | 1 star, 0 forks, no issues from anyone else, 30 of 30 PRs by the maintainer | +| PyPI, last month, mirrors excluded | `clapback-client` 884, `clapback-cli` 597, `beets-clapback` 493, `clapback-embed` 213 | + +The PyPI figures are not users. A package with no installations reporting into the corpus, +and no second pipeline identity, still shows hundreds of monthly downloads from CI and indexers. +`clapback-embed`'s share is mostly Familiar's own builds. The corpus is the measure that counts, +and it has one contributor. + +**What the outreach produced, read 2026-11-02.** Of `ADR-0011` point 5's four approaches: + +- **beets#7032**: closed 2026-09-15, too new to list, and against beets' AI policy. +- **picard-plugins#436**: no reply since it was opened on 2026-09-15. +- **dj-track-similarity#2**: on 2026-09-16 the maintainer wrote "not saying no, but it is too + early for me to take it". Nothing since. +- **KalinkaPlayer#128**, the prospective contributor who prompted `ADR-0013` to `ADR-0020`: + **parked on 2026-09-21 and closed as not planned on 2026-10-29.** The project's notes still + recorded "waiting on madenvel" six weeks after madenvel answered. That premise was wrong and is + recorded here so nobody goes on waiting. + +**madenvel's reasons are worth keeping, because they are structural and not about effort.** The +first is the cold start: a new pipeline identity starts at zero rows, and "the early ones pay for +the late ones". The second decided it. `export.sh` writes one file per pipeline identity, so **an +identity with one or two contributors exports a file that *is* one person's library**, joinable +to MBIDs through `claims.csv.gz` and dated to the day. The anonymity `ADR-0013` point 4 provides +comes from crowd size, and a new identity has no crowd. The period when contributing gives a user +nothing back is the period when it discloses the most about them. madenvel also asked for +contributions to be shuffled, and for a minimum contributor count before an identity is exported. +Neither exists. + +**That argument applies to the corpus as it stands.** The one identity in the export has one +contributor, so the published CC0 file is the maintainer's library, every week. That is the +maintainer's own data and was published knowingly, which is why it is acceptable. It also means +the commons could not have offered a second contributor what madenvel asked for without new work. + +**A guard `ADR-0013` relied on does not exist.** `packages/server/deploy/export.sh:58` says +"tests/test_export_script.py reads this file and fails if one appears", meaning a `client_id` +column. There is no `test_export_script.py` anywhere in the repository, and nothing else exercises +the script. madenvel found this. Checked 2026-11-02. The rule held because the script is correct, +not because anything enforced it. + +**What it costs.** `ADR-0003` records the instance as Lightsail `small_3_0` at **$12 a month** +(bundle prices checked 2026-09-04). Backup and export storage is cents. The live bill was not read +for this record, because the AWS session had expired. The bundle should be confirmed with +`aws lightsail get-instances` before the instance is deleted. + +**What depends on the server.** Familiar looks up every new track by default +(`community_cache_enabled = True`, URL hardcoded) and contributes on the maintainer's installation. +It falls back to local analysis on any failure. But a lookup to a host that **resolves and does not +answer** waits out a 10-second timeout three times (`MAX_RETRIES = 3` in Familiar's +`community_cache.py`), about 32 s per lookup and two lookups per track. That is roughly a minute +added to a 36.6 s median analysis (Familiar `ADR-0144`). A host that **does not resolve** fails instantly. How the server is +turned off decides which one every Familiar installation gets. The three plug-ins and +`clapback-cli` fail the same way and matter less: none has a measured user. + +## Decision + +1. **The public commons at `clapback.seethroughlab.com` is shut down.** The Lightsail instance is + deleted, with its static IP, and any snapshots are removed. The site, the API, the backup, + export and disk-alert timers all go with it. + +2. **The DNS record is deleted, not left pointing anywhere.** The `A` record for `clapback` at + Porkbun is removed in the same sitting as the instance, so every client fails at resolution, + instantly, instead of timing out against an address that no longer answers. A released static IP + can also be reassigned to somebody else's server. A client still configured with the old name + would then send its requests, with a `client_id` and library hashes, to a stranger. + +3. **Familiar stops calling the commons first** (Familiar `ADR-0159`). The maintainer's + installation has lookup and contribution switched off through the settings API before anything + on this side changes, so the last contribution is known before the final export is taken. + +4. **One final export is taken and left published** (supersedes `ADR-0013` points 3 and 6). + After point 3, `clapback-export.service` is run once more by hand. That export stays in + `s3://clapback-export` permanently, and the weekly ones before it are deleted. Its file URLs and + SHA256s go in the repository README, because the `/export` page that lists them today is on the + instance. The CC0 dedication (`ADR-0013` points 1 and 2) stands. This is what keeps the promise + `ADR-0013` was written to make to the first prospective contributor, that the data would outlive + the box. + +5. **One final `pg_dump` is kept in `s3://clapback-backup`**, private, and older dumps are + deleted. It is the only copy carrying what the export strips (`client_id`, agreement history, + `ip_stats`), and it is what a revival would restore from. + +6. **The repository stays open, and `clapback-embed` stays maintained.** Familiar depends on it + (`clapback-embed>=0.1.0,<0.2`, Familiar `ADR-0105`) and has no other CLAP runtime. **No package is + yanked.** Yanking `clapback-embed` would break Familiar's installs, and yanking the others gains + nothing. Their READMEs and the GitHub repository description say the commons is closed and point + to the final export. `clapback-client`, `clapback-cli` and `beets-clapback` get no further + releases. The Picard release's notes say the same. + +7. **The server code stays in `packages/server/`, unbuilt and undeployed.** `ADR-0013` point 7's + self-hosting path, restoring from an export, still works from it. That is the whole revival + plan, and it costs nothing to keep. + +8. **Notes to the two open threads are written by the maintainer** (picard-plugins#436 and + dj-track-similarity#2), per `ADR-0011`'s Implementation block. KalinkaPlayer#128 is already + closed and needs nothing, though madenvel's export finding deserves a thank-you if one is sent. + +9. **Execution order:** point 3 (Familiar's settings now, Familiar `ADR-0159` built and released) + → point 4's final export, and its URLs in the README → point 5 → confirm the bundle and the bill + → points 2 and 1 in one sitting, DNS first → point 6's READMEs and repository description → + point 8. `CLAUDE.md`'s "what is running" section is rewritten last, as "what was running". + +## Alternatives Considered + +**Keep running it at $12 a month and wait.** The commons costs little, the work is done, and +dj-track-similarity#2 is open, not refused. Rejected because waiting has been the plan since +2026-09-20 and has produced one refusal. The one prospective contributor who engaged in depth +named conditions this project has not built: a minimum contributor count before an identity is +exported, coarsened dates, shuffled contributions. Meeting them is new work for a population of +zero. A real second contributor can still restore the commons from point 5's dump, and would arrive +to a better-designed one. + +**Shrink it rather than close it**: the cheapest Lightsail bundle, or scale-to-zero on another +host. Rejected because RAM is `ADR-0003` point 3's binding constraint, and the HNSW index wants +its pages resident. The 2 GB bundle is already the small end. A cold-starting similarity service +answers the one client it has more slowly, for a saving of a few dollars. It is still a server, +backups and an alert channel for one user. Familiar's cache hits are mostly its own work coming +back, which a local database already has. + +**Fold the commons into Familiar's own server**, as a table beside its library. That is in effect +what it already is: one library's vectors, which Familiar keeps in `TrackAnalysis` anyway. +Rejected as a *decision* because there is nothing to fold. Familiar loses nothing it does not +already hold. + +**Delete everything, the export included.** Simplest, and it removes the one public file that is +one person's library. Rejected because `ADR-0013` made a public promise that the data would outlive +the box, the data is CC0 and copies may already exist, and the person whose library it is chose to +publish it. If that choice changes, deleting the final export is one command and needs no record. + +**Hand the commons to someone else to host.** Rejected for lack of a taker, not on principle. +Point 7 keeps it possible. + +## Consequences + +- **Positive:** About $12 a month, plus the maintenance of a public server (TLS, backups, the + disk alert, Tailscale-only SSH) for an audience of one. +- **Positive:** The data outlives the box, as `ADR-0013` promised, in the one form that needs no + server. +- **Positive:** Every Familiar installation fails fast once point 2 is done, including ones never + updated to `ADR-0159`. +- **Tradeoff:** A cache hit, about 28 s saved per already-analysed track, is gone for Familiar. In + practice that is the maintainer's own earlier work, re-served after a re-analysis or reinstall. +- **Tradeoff:** `ADR-0001` to `ADR-0022`, written between 2026-09-01 and 2026-09-20, now describe + a system that is not running. They stay as written. The records most worth re-reading in a revival are `ADR-0007` + (attestation, deliberately unbuilt), `ADR-0008`, `ADR-0019` (the key is path-dependent) and + `ADR-0021`'s measurement. +- **Tradeoff:** The published export is one person's library under CC0, permanently. That is + accepted here knowingly, and it is the condition madenvel would not ask an early adopter to + accept. +- **Follow-up:** If the commons is ever revived, madenvel's three conditions come before any + outreach: suppress an identity from the export below a minimum number of distinct clients, + coarsen `created` for young identities, and let clients trickle contributions in shuffled order. + So does the missing export test. +- **Follow-up:** `CLAUDE.md` and the memory notes that call the second contributor the bottleneck + are rewritten when this is executed. diff --git a/packages/beets-clapback/README.md b/packages/beets-clapback/README.md index fa69ffe..d6a80bc 100644 --- a/packages/beets-clapback/README.md +++ b/packages/beets-clapback/README.md @@ -1,5 +1,9 @@ # beets-clapback +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). This plugin's lookup, `clapback-similar` and contribution all need that server, so it does nothing useful now. It stays on PyPI and gets no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + Look up, embed, and — if you say so — contribute CLAP embeddings for your beets library to the [clapback](https://clapback.seethroughlab.com) commons. `absubmit` reborn, with a payload that can actually be reconciled across contributors. diff --git a/packages/cli/README.md b/packages/cli/README.md index 521a420..77a3770 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,5 +1,9 @@ # clapback +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). `clapback similar` and `clapback contribute` need that server and no longer work. `index`, `search` and `duplicates` run locally and still do. The package stays on PyPI and gets no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + Search your own music by description, find duplicates across formats and masters, and ask the commons what sounds like a track — including recordings you don't own. diff --git a/packages/client/README.md b/packages/client/README.md index a367143..1fa7b5b 100644 --- a/packages/client/README.md +++ b/packages/client/README.md @@ -1,5 +1,9 @@ # clapback-client +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). This package exists to talk to that server. It stays on PyPI and gets no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + The contract a tool follows to take part in the [clapback](https://clapback.seethroughlab.com) commons — look up before you contribute, send what the records require, back off when told to — with no dependency beyond the standard library. diff --git a/packages/picard-clapback/README.md b/packages/picard-clapback/README.md index 8a2e3c1..1963c63 100644 --- a/packages/picard-clapback/README.md +++ b/packages/picard-clapback/README.md @@ -1,5 +1,9 @@ # Clapback for MusicBrainz Picard +> **The clapback commons is closed.** Its server, `clapback.seethroughlab.com`, was shut down on +> 2026-11-03 ([`ADR-0023`](../../docs/decisions/ADR-0023-the-commons-closes-and-its-data-stays-published.md)). This plugin's lookup, naming, contribution and *what sounds like this* all need that server, so it does nothing useful now. The release zip stays up, and there will be no further releases. The corpus's data stays published under CC0 as +> [a final export](https://github.com/seethroughlab/clapback#the-final-export). + Look up, name, and — if you say so — contribute CLAP audio embeddings for your files in the [clapback](https://clapback.seethroughlab.com) commons, and ask what sounds like a track across every library it holds. `ADR-0011` point 5's second integration: Picard already fingerprints every