Skip to content

release: v0.18.2 - #871

Merged
dgenio merged 9 commits into
mainfrom
release/v0.18.2
Aug 29, 2026
Merged

dgenio merged 9 commits into
mainfrom
release/v0.18.2

Conversation

@dgenio

@dgenio dgenio commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Summary

Prepare v0.18.2 from the fixed main release path.

This patch release:

  • includes the environment-independent API manifest / BM25 release-verification fix from fix: make release verification independent of optional extras #869;
  • repairs scripts/prepare_release.py for the current D1-first README shape and adds focused regression coverage;
  • updates version metadata, changelog, release benchmark history/trend, and generated LLM docs through the normal preparation workflow.

Validation

Prepare release branch completed successfully on the prepared branch, including:

  • isolated deterministic benchmark evidence;
  • release metadata/artifact generation;
  • README/security/version metadata checks;
  • release-readiness validation;
  • benchmark trend and generated-doc drift checks.

The short-lived .release-target.json was removed by the successful preparation commit.

Refs #849 and #869.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark delta (vs main)

Soft regression feedback only — this comment never blocks the PR.
Latency budget: ⚠️ when head > base × 1.3. Accuracy budget: ⚠️ when head < base - 1pp.

Routing summary (single backend × catalog sizes)

size recall@k (head Δ vs base) MRR (head Δ vs base) p99 (ms)
50 ✅ 0.5649 (+0.0000) ✅ 0.4978 (+0.0000) ✅ 0.521 (base 0.650)
83 ✅ 0.3825 (+0.0000) ✅ 0.3242 (+0.0000) ✅ 0.786 (base 1.205)
1000 ✅ 0.1475 (+0.0000) ✅ 0.1456 (+0.0000) ✅ 36.978 (base 47.855)

Per-backend × per-size matrix

backend size recall@k (Δ) MRR (Δ) p99 (ms)
bm25 100 ✅ 0.3825 (+0.0000) ✅ 0.3399 (+0.0000) ✅ 6.563 (base 8.399)
bm25 500 ✅ 0.2250 (+0.0000) ✅ 0.2165 (+0.0000) ✅ 30.468 (base 41.865)
bm25 1000 ✅ 0.1575 (+0.0000) ✅ 0.1525 (+0.0000) ✅ 86.944 (base 112.387)
embedding_hashing 100 ✅ 0.5175 (+0.0000) ✅ 0.4360 (+0.0000) ✅ 7.529 (base 7.218)
embedding_hashing 500 ✅ 0.2700 (+0.0000) ✅ 0.2674 (+0.0000) ✅ 42.324 (base 43.171)
embedding_hashing 1000 ✅ 0.2000 (+0.0000) ✅ 0.1931 (+0.0000) ✅ 98.359 (base 102.979)
embedding_st 100 skipped (skipped: missing sentence-transformers) — —
embedding_st 500 skipped (skipped: missing sentence-transformers) — —
embedding_st 1000 skipped (skipped: missing sentence-transformers) — —
fuzzy 100 skipped (skipped: missing rapidfuzz) — —
fuzzy 500 skipped (skipped: missing rapidfuzz) — —
fuzzy 1000 skipped (skipped: missing rapidfuzz) — —
tfidf 100 ✅ 0.3825 (+0.0000) ✅ 0.3220 (+0.0000) ✅ 1.007 (base 1.193)
tfidf 500 ✅ 0.2325 (+0.0000) ✅ 0.2314 (+0.0000) ✅ 10.138 (base 10.803)
tfidf 1000 ✅ 0.1475 (+0.0000) ✅ 0.1456 (+0.0000) ✅ 35.925 (base 42.986)

Context pipeline (per scenario)

scenario tokens dropped dedup
large_catalog 1480 (base 1480, Δ+0) 0 (base 0, Δ+0) 0 (base 0, Δ+0)
long_conversation 2500 (base 2500, Δ+0) 0 (base 0, Δ+0) 0 (base 0, Δ+0)
mixed_payload 488 (base 488, Δ+0) 0 (base 0, Δ+0) 0 (base 0, Δ+0)
short_conversation 487 (base 487, Δ+0) 0 (base 0, Δ+0) 0 (base 0, Δ+0)
stress_conversation 6590 (base 6590, Δ+0) 11 (base 11, Δ+0) 4 (base 4, Δ+0)
tiny_payload 256 (base 256, Δ+0) 0 (base 0, Δ+0) 0 (base 0, Δ+0)

Numbers come from make benchmark / make benchmark-matrix.
Latency is hardware-dependent — treat the markers as a rough guide.
See benchmarks/scorecard.md for the full picture.

@dgenio
dgenio merged commit f8c43d3 into main Aug 29, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant