Feat/hermes memory semantic recall

CLAUDE.md

@@ -8,7 +8,8 @@ starting work; it captures the non-obvious decisions and gotchas that aren't vis
 **Everything runs on Upstash Redis. No vector database, ever.** All "semantic"/similarity features use
 **Upstash Redis Search** with the **`$smart` fuzzy operator** — via `@upstash/redis`'s `redis.search`
 API, **not** RediSearch `FT.*` and **not** `@upstash/vector`. Matching is lexical/fuzzy (BM25), not
-embeddings — keep that in mind when naming/among scoring.
+embeddings — keep that in mind when naming/among scoring. The one exception is `upstash-hermes-memory`, whose memory
+`text` is a Redis Search `HYBRID` field with managed embeddings (still no separate vector database).
 
 ## Packages
 

python/hermes-memory/CHANGELOG.md

@@ -4,6 +4,22 @@ Written by hand: this package is not versioned by Changesets. To release, bump `
 `pyproject.toml`, run `node scripts/sync-version.mjs` and `uv lock`, and add a `## <version>`
 section here. The release workflow publishes any version PyPI does not have yet.
 
+## 0.3.0
+
+### Minor Changes
+
+- **Breaking:** recall is hybrid only. The memory `text` field is a Redis Search `HYBRID` field and
+  recall merges keyword (`$smart`) and semantic (`$ann`) results with `$rrf`. An index from 0.2.x
+  (`text` as `TEXT`) is refused with an error: stop every client and run `SEARCH.DROP <index>`.
+  Documents are kept and indexed again on the next start. Stores with their own `prefix` change too.
+- `AgentMemory.recall` takes `mode="auto"|"keyword"`; records carry `retrieval`. A failed hybrid
+  query falls back to keyword once. `min_score` applies to keyword results only.
+- Automatic recall injects 3 memories instead of 5.
+- `upstash_memory_search` results include `retrieval`.
+- Fix: an empty result from a retriever query no longer sends `SEARCH.COUNT` with that query
+  (Search rejects `$ann` in COUNT); the missing-index check counts `{}` instead.
+- Docs: the shared `agentkit` index is Python-only; the TS SDK creates its own indexes.
+
 ## 0.2.2
 
 ### Patch Changes

python/hermes-memory/README.md

@@ -7,6 +7,7 @@ your AI SDK app saves is recalled by your Hermes agent, and the other way round.
 ## What it does in Hermes
 
 - **Recall:** before each turn, the memories closest to the user's message are added to the prompt.
+  By keyword and by meaning; see [Semantic recall](#semantic-recall).
 - **Chat history:** after each turn, the turn is appended in place to the session's searchable
   transcript document, so a write stays the size of one turn however long the session runs.
 - **Mirroring:** facts the built-in `memory` tool adds are also saved to Redis, and when Hermes
@@ -40,6 +41,31 @@ these):
 | `UPSTASH_HERMES_NAMESPACE` | Key namespace. Default `agentkit`, so memories sit at the TS SDK's default keys and are shared with AgentKit apps on the same database. |
 | `UPSTASH_HERMES_TELEMETRY` | Set to `1` to send telemetry. Off by default; see [Telemetry](#telemetry). |
 
+## Semantic recall
+
+Keyword recall (`$smart`: phrase, term and fuzzy matching) only finds a memory that shares words
+with the message: "where is the assistant hosted" misses "Hermes runs on the ops Box". So the
+memory `text` field is a `HYBRID` field, which Upstash Redis Search embeds itself (no separate
+vector database), and recall merges a keyword query with a semantic `$ann` query using `$rrf`
+([Hybrid Search](https://upstash.com/docs/redis/search/hybrid-search)).
+
+- **Upgrading from 0.2.x:** the old index has `text` as `TEXT` and is not supported. Recall fails
+  until you replace it: stop every client of the namespace (Hermes gateway, desktop app, cron jobs,
+  other profiles on it) and run `SEARCH.DROP agentkit` (or your namespace's index). Your memories
+  and chats are kept, and the next start creates the new index and indexes them again.
+- Automatic recall injects up to **3** memories: the semantic side has no relevance
+  threshold and returns up to its limit even for unrelated text, and in a small-store test its
+  similarity scores did not separate related from unrelated memories well enough for a cutoff. `upstash_memory_search` keeps its limit.
+- Results carry `retrieval: "hybrid"` or `"keyword"`. Hybrid scores are rank-fusion values that
+  only order one search's results; `min_score` applies to keyword results only.
+- If a hybrid query fails, recall falls back to keyword search for that call (logged at most
+  every 10 minutes).
+- Chat history stays keyword-only: a `HYBRID` field gets one embedding for its whole value, and a
+  chat document holds a whole session.
+- Every memory in the index is embedded, whichever user or profile wrote it, and each recall
+  sends the message to Upstash Redis Search to be embedded. In a small-store test a hybrid query
+  took about 150 to 300 ms, against a few ms for keyword recall.
+
 ## Telemetry
 
 Off by default. Set `UPSTASH_HERMES_TELEMETRY=1` to opt in: requests to your Upstash database then

python/hermes-memory/plugin.yaml

@@ -3,7 +3,7 @@
 # https://hermes-agent.nousresearch.com/docs/developer-guide/plugins/catalog-submission
 name: upstash
 # Copied from pyproject.toml by scripts/sync-version.mjs.
-version: "0.2.2"
+version: "0.3.0"
 description: "Upstash Redis memory provider: long-term memory, searchable chat history and recall on Redis Search, sharing keys with @upstash/agentkit-sdk."
 # Also declared in the catalog entry; older Hermes releases only enforce this one (catalog rule 14).
 requires_hermes: ">=0.19"

python/hermes-memory/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "upstash-hermes-memory"
-version = "0.2.2"
+version = "0.3.0"
 description = "Upstash Redis memory provider for Hermes Agent: long-term memory, searchable chat history and recall on Redis Search, with no separate vector database."
 readme = "README.md"
 license = "MIT"

python/hermes-memory/src/upstash_hermes_memory/_search.py

@@ -1,7 +1,9 @@
 """A Redis Search index that is created on first use, shared by AgentMemory and ChatHistory.
 
-The commands are the ones `ReactiveSearchIndex` sends from the TS SDK, argument for argument, so
-an index created from either language is the same index.
+A store with its own `prefix` sends the commands `ReactiveSearchIndex` sends from the TS SDK,
+argument for argument, except that memory `text` is HYBRID here (the TS SDK still creates it as
+TEXT), so the two languages cannot share an index over the same name. The default shared index is
+Python-only.
 """
 
 import json
@@ -25,13 +27,18 @@ def dumps(value: Any) -> str:
 
 DEFAULT_NAMESPACE = "agentkit"
 
-# The one index AgentMemory and ChatHistory share by default (the TS `SharedIndexSchema`, field
-# for field and in the same order). Upstash's Free plan allows one search index per database.
-# `kind` ("memory" or "chat") keeps the two apart in every query.
+# Retrievers ($rrf, $ann) must be the only key of the query object and cannot be counted.
+RETRIEVERS = frozenset({"$rrf", "$ann"})
+
+# The one index AgentMemory and ChatHistory share by default (Python only: the TS SDK gives each
+# store its own index). Upstash's Free plan allows one search index per database.
+# `kind` ("memory" or "chat") keeps the two apart in every query. Only the memory `text` is HYBRID
+# (full-text plus an embedding for `$ann`): chat transcripts stay TEXT, since a HYBRID field gets
+# one embedding for its whole value and `userMessages` / `modelMessages` hold a whole session each.
 SHARED_SCHEMA: list[Field] = [
     ("kind", "TEXT", True),
     ("userId", "TEXT", True),
-    ("text", "TEXT", False),
+    ("text", "HYBRID", False),
     ("sessionId", "TEXT", True),
     ("userMessages", "TEXT", False),
     ("modelMessages", "TEXT", False),
@@ -42,7 +49,7 @@ def shared_index(redis: Any, namespace: str) -> "SearchIndex":
     """The shared index of a namespace, covering `<namespace>:memory:` and `<namespace>:chat:`.
 
     Both stores build this same definition, so whichever reads first creates it and the other's
-    EXISTSOK create is a no-op; the TS SDK builds it identically.
+    EXISTSOK create is a no-op.
     """
     return SearchIndex(
         redis,
@@ -52,6 +59,10 @@ def shared_index(redis: Any, namespace: str) -> "SearchIndex":
     )
 
 
+def _is_config_mismatch(error: Exception) -> bool:
+    return "different configuration" in str(error)
+
+
 class SearchIndex:
     def __init__(
         self, redis: Any, *, name: str, prefix: str | Sequence[str], schema: Sequence[Field]
@@ -72,20 +83,38 @@ def create_command(self) -> list[str]:
         return cmd
 
     def ensure(self) -> None:
-        """Create the index if it does not exist yet. A no-op after the first call."""
+        """Create the index if it does not exist yet. A no-op after the first call.
+
+        An index of this name with another definition (one from 0.2.x has `text` as TEXT) is never
+        dropped here, since other clients may be using it; the error says how to replace it.
+        """
         if self._ready:
             return
         with self._lock:
             if self._ready:
                 return
-            self.redis.execute(self.create_command())
-            self.redis.execute(["SEARCH.WAITINDEXING", self.name])
+            try:
+                self.redis.execute(self.create_command())
+            except Exception as error:
+                if not _is_config_mismatch(error):
+                    raise
+                raise RuntimeError(
+                    f"search index {self.name} exists with a different definition, for example "
+                    f"from upstash-hermes-memory 0.2.x. Stop every client, run `SEARCH.DROP "
+                    f"{self.name}` (documents are kept and indexed again) and start again."
+                ) from error
+            # 1 once pending updates are searchable; 0 means the index is gone (dropped right
+            # after the create): stay not-ready, so the next call creates it again.
+            if self.redis.execute(["SEARCH.WAITINDEXING", self.name]) != 1:
+                raise RuntimeError(f"search index {self.name} disappeared while it was created")
             self._ready = True
 
     def query(self, filter: dict[str, Any], limit: int | None = None) -> list[Any]:
         self.ensure()
         rows = self._query(filter, limit)
-        if not rows and self._raw_count(filter) < 0:
+        # COUNT cannot take a retriever, so a retriever query checks the index with `{}` instead.
+        probe = {} if RETRIEVERS & set(filter) else filter
+        if not rows and self._raw_count(probe) < 0:
             # The Python client turns a missing index into [], so an empty result is ambiguous.
             # COUNT says -1 when the index is gone (another client dropped it): recreate, retry once.
             self._ready = False
@@ -114,9 +143,15 @@ def _raw_count(self, filter: dict[str, Any]) -> int:
         return int(getattr(result, "count", result))
 
     def wait_indexing(self) -> None:
-        """Block until every write so far is searchable (and every delete has left the index)."""
+        """Block until every write so far is searchable (and every delete has left the index).
+
+        WAITINDEXING answers 0 when the index is gone (another client dropped it): it is created
+        again (its documents are indexed by the create-time scan) and waited for once more.
+        """
         self.ensure()
-        self.redis.execute(["SEARCH.WAITINDEXING", self.name])
+        if self.redis.execute(["SEARCH.WAITINDEXING", self.name]) != 1:
+            self._ready = False
+            self.ensure()
 
     def drop(self) -> None:
         self.redis.execute(["SEARCH.DROP", self.name])

python/hermes-memory/src/upstash_hermes_memory/_version.py

@@ -1,2 +1,2 @@
 # Generated by scripts/sync-version.mjs from pyproject.toml — do not edit by hand.
-VERSION = "0.2.2"
+VERSION = "0.3.0"

python/hermes-memory/src/upstash_hermes_memory/memory.py

@@ -1,7 +1,9 @@
 """Python port of `AgentMemory` from `@upstash/agentkit-sdk` (packages/sdk/src/memory.ts).
 
-Same keys, same documents, same index: a memory saved by the TS SDK is recalled here and the other
-way round, as long as both sides use the same namespace (default `agentkit`).
+Same keys and documents as the TS SDK: a memory saved by the TS SDK is recalled here and the other
+way round, as long as both sides use the same namespace (default `agentkit`). The search index is
+not shared: Python indexes memories and chats together in one index (`agentkit`), the TS SDK
+creates its own (`agentkit_memory`), and both index the same keys.
 
     key    <namespace>:memory:<userId>:<id>
     value  JSON {"text", "userId", "createdAt" (epoch ms), ...metadata}, exactly what the TS SDK writes
@@ -13,16 +15,31 @@
 Passing `prefix` (or `index_name`) gives the store its own index over `<prefix>:` instead.
 """
 
+import logging
 import time
 import uuid
 from dataclasses import dataclass, field
-from typing import Any
+from typing import Any, Literal
+
+import httpx
+from upstash_redis.errors import UpstashError
 
 from ._search import DEFAULT_NAMESPACE, SearchIndex, dumps, index_name_for, shared_index
 from ._telemetry import add_telemetry
 
+logger = logging.getLogger(__name__)
+
 DEFAULT_PREFIX = f"{DEFAULT_NAMESPACE}:memory"
 
+# How often a failing hybrid query is logged, per store.
+_WARN_EVERY_S = 600.0
+
+RecallMode = Literal["auto", "keyword"]
+
+# What a failed hybrid query can raise: a server error (e.g. the intermittent "dot metric only
+# accepts unit-length vectors") or a transport failure. Anything else is a bug and propagates.
+_FALLBACK_ERRORS = (UpstashError, httpx.HTTPError)
+
 # The document's own fields. Metadata is spread next to them, so it must not replace them: a
 # `userId` in metadata would move the memory into another user's searches.
 RESERVED_FIELDS = frozenset({"kind", "text", "userId", "createdAt"})
@@ -47,6 +64,10 @@ class MemoryRecord:
     created_at: int
     score: float | None = None
     metadata: dict[str, Any] = field(default_factory=dict)
+    # How a recalled record was found: "keyword" ($smart, BM25 score) or "hybrid" ($rrf of $smart
+    # and $ann; the score is a rank-fusion value, only comparable within one query). None for
+    # records that were not searched for (get, list, an empty query).
+    retrieval: str | None = None
 
 
 class AgentMemory:
@@ -60,9 +81,11 @@ def __init__(
         min_score: float = 0,
         enable_telemetry: bool = False,
     ):
+        """`min_score` applies to keyword results only; hybrid scores are not on the same scale."""
         self.redis = redis
         self.min_score = min_score
         self.shared = prefix is None and index_name is None
+        self._warned_at: dict[str, float] = {}
         if self.shared:
             self.prefix = f"{namespace}:memory"
             self.index = shared_index(redis, namespace)
@@ -72,7 +95,7 @@ def __init__(
                 redis,
                 name=index_name or index_name_for(self.prefix),
                 prefix=f"{self.prefix}:",
-                schema=[("text", "TEXT", False), ("userId", "TEXT", True)],
+                schema=[("text", "HYBRID", False), ("userId", "TEXT", True)],
             )
         add_telemetry(redis, enabled=enable_telemetry)
 
@@ -110,18 +133,57 @@ def add(
         self.redis.execute(["JSON.SET", self._key(user_id, id), "$", dumps(doc)])
         return MemoryRecord(id=id, text=text, created_at=created_at, metadata=metadata or {})
 
-    def recall(self, query: str, *, user_id: str, top_k: int = 5) -> list[MemoryRecord]:
-        """The `top_k` memories closest to `query` ($smart: full-text plus fuzzy matching)."""
+    def recall(
+        self, query: str, *, user_id: str, top_k: int = 5, mode: RecallMode = "auto"
+    ) -> list[MemoryRecord]:
+        """The `top_k` memories closest to `query`.
+
+        `mode="auto"` uses hybrid search ($rrf of $smart and $ann). A failed hybrid query falls
+        back to keyword search ($smart: full-text plus fuzzy matching) once. `mode="keyword"`
+        always uses keyword search, for callers that need COUNT and QUERY to describe the same
+        set.
+        """
+        if mode not in ("auto", "keyword"):
+            raise ValueError(f"mode must be 'auto' or 'keyword' (got {mode!r})")
         _check_id(user_id, "user_id")
-        filter = self._filter(user_id)
-        has_query = bool(query and query.strip())
-        if has_query:
-            filter["text"] = {"$smart": query}
-        rows = self._memories(self.index.query(filter, limit=top_k), user_id)
-        min_score = self.min_score if has_query else 0
-        return [
-            r for r in (self._record(row, user_id) for row in rows) if (r.score or 0) >= min_score
-        ]
+        base = self._filter(user_id)
+        if not (query and query.strip()):
+            rows = self._memories(self.index.query(base, limit=top_k), user_id)
+            return [self._record(row, user_id) for row in rows]
+        if mode == "auto":
+            hybrid = {
+                "$rrf": {
+                    "retrievers": [
+                        {**base, "text": {"$smart": query}},
+                        {
+                            "$ann": {
+                                "field": "text",
+                                "query": query,
+                                "filter": self._filter(user_id),
+                            }
+                        },
+                    ]
+                }
+            }
+            try:
+                rows = self._memories(self.index.query(hybrid, limit=top_k), user_id)
+            except _FALLBACK_ERRORS as error:
+                self._warn(
+                    "hybrid", "upstash memory hybrid recall failed, using keyword: %s", error
+                )
+            else:
+                return [self._record(row, user_id, "hybrid") for row in rows]
+        rows = self._memories(
+            self.index.query({**base, "text": {"$smart": query}}, limit=top_k), user_id
+        )
+        records = [self._record(row, user_id, "keyword") for row in rows]
+        return [r for r in records if (r.score or 0) >= self.min_score]
+
+    def _warn(self, what: str, message: str, *args: Any) -> None:
+        now = time.monotonic()
+        if now - self._warned_at.get(what, -_WARN_EVERY_S) >= _WARN_EVERY_S:
+            self._warned_at[what] = now
+            logger.warning(message, *args)
 
     def list(self, *, user_id: str, limit: int = 100) -> list[MemoryRecord]:
         _check_id(user_id, "user_id")
@@ -145,10 +207,11 @@ def forget(self, id: str, *, user_id: str) -> bool:
         _check_memory_id(id)
         return bool(self.redis.delete(self._key(user_id, id)))
 
-    def _record(self, row: Any, user_id: str) -> MemoryRecord:
+    def _record(self, row: Any, user_id: str, retrieval: str | None = None) -> MemoryRecord:
         id = row.key[len(self._key(user_id, "")) :]
         record = self._from_doc(id, row.data if isinstance(row.data, dict) else {})
         record.score = row.score
+        record.retrieval = retrieval
         return record
 
     @staticmethod