Skip to content

fix(integrations): recall with the end of a long prompt in Hermes, LangChain, LangGraph, and Pydantic AI - #2647

Open
Fengzdadi wants to merge 1 commit into
oceanbase:masterfrom
Fengzdadi:fix/inprocess-adapters-recall-query-tail
Open

Fengzdadi wants to merge 1 commit into
oceanbase:masterfrom
Fengzdadi:fix/inprocess-adapters-recall-query-tail

Conversation

@Fengzdadi

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Part of #2250 ("② fix divergences": automatic recall query bound for the in-process Python adapters).

Rationale for this change

POST /v1/context/prepare accepts a query of at most 8,192 characters. Claude Code, Codex, WorkBuddy, Kimi Code, ZCode, Pi, OpenCode, DSH, and Bub send a long prompt's end (#2252, tests/fixtures/recall_query_bound.json). Hermes, LangChain, LangGraph, and Pydantic AI still sent query[:8192], the start. A prompt longer than 8,192 characters lost its end, which is usually the question in hand, so recall ran on preamble or pasted context instead of the request.

What changes are included in this PR?

Behavior

Adapter Automatic recall path Before After
Hermes prefetch, queue_prefetch (and the evaluation-trace query) first 8,192 characters, untrimmed shared rule
LangChain PowerContextMiddleware model-call recall (and the optional Langfuse recall observation) first 8,192 characters shared rule
LangGraph PowerContextRecall first 8,192 characters shared rule
Pydantic AI PowerContext capability recall first 8,192 characters shared rule

The shared rule is bounded_recall_query in src/powercontext_integration_lifecycle/prepared_context.py: trim; keep text within 8,192 characters and 8,192 UTF-8 bytes as is; otherwise keep the last 8,192 characters, then the last 8,192 bytes, without splitting a character.

Pydantic AI also stops cutting the latest user text at 8,192 characters before using it. The user_prompt capture event falls back to that text when ctx.prompt has none, so the fallback now keeps the whole prompt; the event keeps its own CAPTURE_MAX_BYTES limit.

How the adapters get the rule

  • Hermes ships a generated copy, integrations/hermes/plugins/powercontext/prepared_context.py, beside powercontext_client_config.py, because its plugin directory installs without the PowerContext package.
  • The SDK adapters install with powercontext[client], but they support released cores that predate powercontext_integration_lifecycle (first shipped in 1.5.1). The Verify standalone LangChain adapter with released core job pins core 1.5.0, and the text-assembly guide documents legacy recall with core 0.2.0. Importing the package would raise that floor, so each adapter ships a private generated copy (_prepared_context.py) instead. The wheels contain it and import nothing new.
  • scripts/generate_integration_lifecycle.py writes all four copies; make check (integration-lifecycle-check) rejects drift. pyproject.toml lints the copies as Python 3.9 like their source.

Tests

  • tests/inprocess_adapter_turns.py runs one turn through each adapter's public entry point (Hermes PowerContextMemoryProvider, LangChain create_agent with the middleware, LangGraph create_react_agent with PowerContextRecall, Pydantic AI Agent with PowerContext) and its real HTTP client, against a recording Server on an ephemeral loopback port.
  • tests/test_recall_query_bound.py checks the query each adapter sends for every case in recall_query_bound.json. Against the previous adapters, 25 of the 32 cases fail: every over-the-bound case for all four adapters, and Hermes's untrimmed within-the-bounds case.
  • tests/langgraph_adapter/test_recall.py::test_recall_completes_graph_for_over_limit_prompt asserted that recall fires on a long prompt whose question came first. Its prompt now ends with the question; the assertions are unchanged.

Docs

  • docs/{en,zh}/docs/integrations/{hermes,langchain,langgraph,pydantic-ai}.md describe the query the adapter sends.

Not included

Overlap with #2550

Are there any user-facing changes?

Yes. A prompt longer than 8,192 characters, or 8,192 UTF-8 bytes, now recalls with its end instead of its start in these four adapters. Hermes also trims surrounding whitespace from the query. No API or persisted format changes; capture is unchanged.

How was this change tested?

  • uv run pytest tests/test_recall_query_bound.py (41 passed); with the adapter sources reverted, 25 fail as described above.
  • uv run pytest tests/langchain_middleware tests/langgraph_adapter tests/pydantic_ai_adapter tests/integrations/test_hermes_provider.py tests/test_hermes_cli.py tests/integration_conformance tests/test_recall_query_bound.py tests/test_automatic_prompt_privacy.py tests/test_integration_lifecycle.py: 395 passed, 2 skipped.
  • make check passes. make test: 10,628 passed, 16 failed. Three failures were load-related (two test_artifact_processing cases and an asyncio timeout in tests/e2e/test_langgraph_chain.py) and passed when rerun serially. The other 13 (test_ci_installation_roles, test_jev_rerank_comparison, and one test_gateway_authentication case) fail the same way on unmodified upstream/master 195be9b.
  • Built the three SDK wheels offline and checked they include _prepared_context.py; imported the three adapters with powercontext_integration_lifecycle blocked from import. The CI job against released core 1.5.0 was not run locally.
  • Not run against a real Hermes, LangChain, LangGraph, or Pydantic AI application or a real Server; the recording Server checks only what the adapters send.

Self-review against REVIEW.md, plus an independent Opus review. Neither found a blocking issue. The independent review checked that a credential near the cut cannot survive as a fragment once the follow-up scrubs first. It also confirmed that the LangGraph regression now fails under head truncation. Remaining non-blocking notes:

  • The unlisted-copy scan in scripts/generate_integration_lifecycle.py covers only integrations/*/plugins/*. A stray hand-kept copy inside an SDK adapter's src/ would not be flagged. The listed copies are drift-checked.
  • The LangChain synchronous invoke path and Hermes queue_prefetch share the tested code path, but the new tests do not drive them. The reviewer checked LangChain invoke with a scratch script.

AI usage statement

Claude Code (Claude Opus 5.5) wrote the change, the tests, and this description, and ran the checks above.

…ngChain, LangGraph, and Pydantic AI

The four in-process Python adapters cut an automatic recall query to its first 8,192 characters,
so a long prompt lost its end, which is usually the question in hand. They now send the same query
as the hook hosts and Bub: a prompt within the bounds trimmed, a longer one cut to its most recent
text within 8,192 characters and 8,192 UTF-8 bytes without splitting a character.

The rule comes from powercontext_integration_lifecycle.prepared_context. Hermes ships a generated
copy beside its other plugin helpers. The SDK adapters install with the core but also support
released cores that predate that package (the LangChain job pins core 1.5.0), so importing it
would raise their floor; each ships a private generated copy instead, and make check rejects drift.

Pydantic AI no longer cuts the latest user text before using it, so the user_prompt capture
fallback keeps the whole prompt; capture still applies its own byte limit.

tests/test_recall_query_bound.py now drives one real turn through each adapter's public entry
point against a recording Server and checks the query for every fixture case. The LangGraph
over-limit regression places its question at the end of the prompt, where users put it.

Explicit search commands and tools (Hermes commands.py, LangGraph tools.py) are unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant