Issues

• Add BEAM benchmark dataset (ICLR 2026 — long-term memory evaluation)

  enhancement
  basicmachines-co/basic-memory-benchmarks#12 · Filter by author bm-clawd opened on Mar 24, 2026

• Benchmark: Add QMD comparison (local-first knowledge search baseline)

  basicmachines-co/basic-memory-benchmarks#3 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Benchmark: Add agent mode evaluation (multi-round retrieval via MCP)

  basicmachines-co/basic-memory-benchmarks#5 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Benchmark: Verify LoCoMo category assignments match source code, not paper

  basicmachines-co/basic-memory-benchmarks#4 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Benchmark: Track token usage per query for cost comparison

  basicmachines-co/basic-memory-benchmarks#6 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Benchmark: Test with multiple eval LLMs to isolate memory quality from model capability

  basicmachines-co/basic-memory-benchmarks#7 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Benchmark: Adopt Backboard's LoCoMo methodology for reproducible comparison

  basicmachines-co/basic-memory-benchmarks#8 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Benchmark: Add LLM-as-Judge evaluation (GPT-4.1) for LoCoMo

  basicmachines-co/basic-memory-benchmarks#9 · Filter by author bm-clawd opened on Feb 27, 2026
  1

• Investigate low content hit rate for bm-local (15.5% vs Mem0 34.3%)

  basicmachines-co/basic-memory-benchmarks#2 · Filter by author bm-clawd opened on Feb 26, 2026

• Add MCP stdio provider for warm-connection benchmarks

  basicmachines-co/basic-memory-benchmarks#1 · Filter by author bm-clawd opened on Feb 26, 2026
  1

• Standalone benchmark suite for retrieval quality evaluation

  enhancement
  basicmachines-co/basic-memory-benchmarks#10 · Filter by author bm-clawd opened on Feb 25, 2026
  3