Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 24 additions & 14 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,14 @@ produces nothing of its own and exists to put a question to this turn's Agent Pa
merge what they answer (`ask_intake`, `ask_advice`, `ask_review`, `ask_salvage`, `ask_judge`,
`ask_system_addendum`, `ask_archive`, `ask_select_tools`). Replacing a role therefore means two
different things: a new way to produce, or a new rule for adjudicating what the products say.
Frozen per Generation: the tool array is the prompt-cache prefix, so the set a turn runs on
does not move between two of its model calls -- except for tools the turn's own `plugin` call
connected, which join it on purpose (`ToolRegistry.admit_to_this_turn`).
Frozen per Session: the tool array is the prompt-cache prefix, so the set a session
assembles with does not move between its model calls or between its turns. It is
captured on the session's first assembly and recaptured only at a refresh point -- a
new session, a compaction (cashed by the next assembly), a `/clear`, or a release. A
live switch, a missing credential, or a selected base changes availability, not the
array, and is reported in the runtime context; a tool the turn's own `plugin` call
connects is admitted to `offers` (reachable through `tool_call`) and joins the array at
the next refresh.

**Harness Curator** (`experimental/curator/`):
Experimental generation of a worker's Harness from its task, the materials handed to it and
Expand Down Expand Up @@ -719,9 +724,11 @@ user declaring the agent owns nothing and is never refilled.
Two things never carry ownership. The generic row (`Raven-General`) claims none whatever a config
says, because it carries no capability bias and a line about it would prohibit the agent
reading it from doing its own work. And the section is withheld entirely on a turn holding
no dispatch tool — `spawn` and `run_subagent_dag` can both be withheld by
`tools.disabledTools`, and a prohibition outliving every means of handing the work over
leaves a request with no compliant action at all. Only the paths the turn does hold are named.
no dispatch tool — a hard-subset deploy or a charter narrowing can leave the array without
`spawn` and `run_subagent_dag`, and a prohibition outliving every means of handing the work
over leaves a request with no compliant action at all. A live `tools.disabledTools` switch
keeps the schema and reports the tool unavailable in the runtime context instead. Only the
paths the turn does hold are named.
_Avoid_: putting it in `description` — that copy is spliced into the tool descriptions, where
the model reads it only once it is already choosing an agent.

Expand Down Expand Up @@ -926,8 +933,10 @@ Which interchangeable backend each web tool calls: `tools.web.search.provider` s
so a selection names a vendor and never a URL. `web_search` is registered whatever the config
holds and *withheld* from the offered set while the selected vendor's key does not resolve
(Keenable searches without one and is offered keyless, as Jina reads pages keyless), so a key
pasted mid-run surfaces it without a restart -- read live from the vendor slot and, for Serper,
from the pre-vendor leaf, in the order `WebToolsConfig.vendor_key` resolves them, and an empty
pasted mid-run makes it callable again without a restart -- the schema was already in the
session's frozen array, and the runtime context clears the notice -- read live from the vendor
slot and, for Serper, from the pre-vendor leaf, in the order `WebToolsConfig.vendor_key`
resolves them, and an empty
slot is a revocation rather than a miss (except for a borrowing vendor, whose empty slot is the
ordinary state and whose LLM provider key answers instead); `web_fetch` is always offered,
falling back to Jina, which reads pages without a key. The two sub-agent launchers inherit the host's selection
Expand Down Expand Up @@ -2349,12 +2358,13 @@ whose change implies WORK rather than a different answer (constructing a provide
an MCP handshake, moving a workspace) keeps its apply path -- a door, the pool's
fingerprint cache, or the generation swap. Timing is asymmetric by design: a revocation
binds the next read (a tightened deny pattern gates the very next tool call), while an
addition to the model's tool array lands on the next turn -- `ToolRegistry.turn_scope`
freezes both registry membership and the withheld set at turn entry, because the array
is the prompt-cache prefix and must not move between two model calls of one turn. The one
exception is an addition the turn's own call produced: `ToolRegistry.admit_to_this_turn`
lets the tools a `plugin` connect or authorize registered join that turn from its next
model call, at the cost of one rebuilt prefix.
addition to the model's tool array is held until a session refresh -- `ToolRegistry` freezes
the array on the session's first assembly and recaptures it only at a new session, a
compaction, a `/clear`, or a release, because the array is the prompt-cache prefix.
`ToolRegistry.turn_scope` still freezes registry membership and the withheld set at turn
entry, so a mid-turn arrival cannot change what `offers` or `execute` admits. The turn's own
`plugin` call is the one admission: `ToolRegistry.admit_to_this_turn` lets the tools it
connected reach `tool_call` from the next model call, without moving the array.
_Avoid_: reading `config.json` keys ad hoc outside this module; treating a live
preference as a door (doors reconcile members after a durable write; this lane never
touches member identity).
Expand Down
10 changes: 9 additions & 1 deletion raven/agent/loop/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,14 @@ def __init__(
# Asked once per assembled tool array, so an off switch flipped now is
# honoured by the next request rather than the next restart.
self.tools.set_withheld_source(self._withheld_tool_names)
# The same switch, asked for its reason: the runtime-context notice and
# the refusal of a call to an advertised-but-withheld tool say which
# setting or credential puts the tool back.
self.tools.set_withheld_reason_source(self._withheld_tool_reasons)
# The constructor's list is the one off switch that cannot move while
# the process runs, so it stays a hard removal from the session array
# instead of becoming a notice.
self.tools.set_hard_withheld_source(self._hard_withheld_names)

# Context engine — the single ContextAssembler.
# Constructed here (after self.tools) so the factory can capture
Expand Down Expand Up @@ -465,7 +473,7 @@ def __init__(
# rebuilt on a hot config apply, so anything captured now would be
# either missing or stale by the time a turn asks for it.
list_subagents=lambda: self.subagents.list_agents(),
get_tool_notices=self._mcp_tool_notices,
get_tool_notices=self._tool_notices,
# Read at call time: the board is wired further down this constructor.
get_session_notices=lambda key: self.task_board.turn_notes(key) if self.task_board else [],
now_fn=now_fn,
Expand Down
167 changes: 130 additions & 37 deletions raven/agent/loop/mcp_glue.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,47 @@ async def _swallow_terminal_cancel(task: "asyncio.Task") -> None:
pass


_MCP_AUTH_REMAIN = (
"installed, awaiting authorization. Its tools remain in this conversation's tool list, but "
"calls to them fail until it is authorized -- by the `plugin` tool's authorize action, or by "
"the user in the plugin panel."
)
_MCP_CONNECTING_REMAIN = (
"installed, still connecting. Its tools remain in this conversation's tool list, but calls to "
"them fail until the handshake finishes."
)
_MCP_DISCONNECTED_REMAIN = (
"disconnected. Its tools remain in this conversation's tool list, and calls to them fail until it reconnects."
)
_MCP_DISABLED_REMAIN = (
"disabled in settings. Its tools remain in this conversation's tool list, and calls to them "
"fail until it is enabled again."
)
_MCP_REMOVED_REMAIN = (
"no longer loaded. Its tools remain in this conversation's tool list, and calls to them fail "
"until it is added back."
)
_MCP_AUTH_LIVE = (
"installed, awaiting authorization. Its tools are absent from this turn's definitions until "
"it is authorized -- by the `plugin` tool's authorize action, or by the user in the plugin "
"panel."
)
_MCP_AUTH_JOIN = (
"installed, awaiting authorization. Its tools are absent from this conversation's tool list; "
"after it is authorized they join at the next compaction or new conversation -- by the "
"`plugin` tool's authorize action, or by the user in the plugin panel."
)
_MCP_CONNECTING_LIVE = (
"installed, still connecting. Its tools are absent from this turn's definitions and register "
"themselves when the handshake finishes, so they are available from a later turn without "
"anyone acting."
)
_MCP_CONNECTING_JOIN = (
"installed, still connecting. Its tools join this conversation's tool list at the next "
"compaction or new conversation."
)


class McpGlueMixin:
"""MCP lifecycle glue: connect, sync, prewarm, apply, close. Bodies moved
verbatim from main.py."""
Expand Down Expand Up @@ -171,8 +212,8 @@ def _sync_mcp_meta_tools(self) -> None:
def _mcp_tool_notices(self) -> list[str]:
"""Host facts about MCP tools the definitions cannot carry.

One line per enabled server whose tools are absent from this turn for a
reason the model cannot see, and the three reasons need different lines:
One line per server whose tools the model cannot use, and the states
need different lines:

* ``auth_required`` -- the wait is on a person, and the answer names who
can end it. Without this the model reads an unauthorized plugin as a
Expand All @@ -185,20 +226,29 @@ def _mcp_tool_notices(self) -> list[str]:
names an installed server, so a model that does not think to search
reports a capability it holds as missing. ``tool_search``'s own
description says a catalog exists; only this says what is in it.

This line states an inventory and promises no route, which is what keeps
it true. This block is rendered once per turn, while the fold is decided
per model call against an array a ``before_iteration`` hook may narrow
first (``HookDecision.modified_tools``) -- so a turn-level sentence about
what this request folded, or about reaching it through ``tool_search`` /
``tool_call``, can be falsified after it is written. "This server is
connected and offers N tools" cannot.

The count is the server's tools this turn's array carries, not the
manager's ``tool_count``: that one counts every wrapper the server
registered, including one the operator switched off, which no surface
offers. The fold gate is only about noise -- below it every schema is in
the array and the line tells the model nothing it cannot see.
* connected, disconnected, or disabled while its tools are OUTSIDE the
session's frozen array -- the array is not recomputed until a new
session or a compaction, so the line says what state the server is in
and when its tools join. A server that connects mid-session is
reachable through ``tool_call`` only if the model knows the name and
arguments; the honest fact is the refresh point.

The array is the session's snapshot (``ToolRegistry.get_definitions``):
captured once and refreshed only at a new session, a compaction, or a
log id change. This line states an inventory and promises no route,
which is what keeps it true. This block is rendered once per turn, while
the fold is decided per model call against an array a
``before_iteration`` hook may narrow first
(``HookDecision.modified_tools``) -- so a turn-level sentence about what
this request folded, or about reaching it through ``tool_search`` /
``tool_call``, can be falsified after it is written. "This server is
connected and offers N tools" cannot.

The folded count is the server's tools the array carries, not the
manager's ``tool_count``: that one counts every wrapper the server
registered, including one the operator switched off, which no surface
offers. The fold gate is only about noise -- below it every schema is in
the array and the line tells the model nothing it cannot see.

Rendered into the runtime-context block, not the system prompt -- the
set changes turn to turn and must never be cached with the prefix. This
Expand All @@ -224,32 +274,75 @@ def _mcp_tool_notices(self) -> list[str]:
offered: set[str] = set()
if folded:
offered = {d["function"]["name"] for d in self.tools.get_definitions()}
# None outside a turn: the pre-freeze behaviour, where the array is the
# live view and a server's tools appear as soon as it connects.
snapshot = self.tools.session_definition_names()
snapshot_servers = self.tools.session_definition_servers() if snapshot is not None else frozenset()
seen: set[str] = set()
notices = []
for snap in mgr.status():
name = snap["name"]
seen.add(name)
snap_names = set(self.tools.session_definition_names_of_server(name)) if snapshot is not None else set()
live_names = set(self.tools.names_from(name))
if not snap.get("enabled", True):
if snap_names:
notices.append(mcp_notice(name, _MCP_DISABLED_REMAIN))
continue
if snap["state"] == "auth_required":
notices.append(
mcp_notice(
snap["name"],
"installed, awaiting authorization. Its tools are absent from this turn's "
"definitions until it is authorized -- by the `plugin` tool's authorize "
"action, or by the user in the plugin panel.",
)
)
elif snap["state"] == "connecting":
notices.append(
mcp_notice(
snap["name"],
"installed, still connecting. Its tools are absent from this turn's "
"definitions and register themselves when the handshake finishes, so "
"they are available from a later turn without anyone acting.",
state = snap["state"]
if state == "auth_required":
if snapshot is None:
text = _MCP_AUTH_LIVE
elif snap_names:
text = _MCP_AUTH_REMAIN
else:
text = _MCP_AUTH_JOIN
notices.append(mcp_notice(name, text))
elif state == "connecting":
if snapshot is None:
text = _MCP_CONNECTING_LIVE
elif snap_names:
text = _MCP_CONNECTING_REMAIN
else:
text = _MCP_CONNECTING_JOIN
notices.append(mcp_notice(name, text))
elif snap["connected"]:
added = live_names - snap_names
gone = snap_names - live_names
if snapshot is not None and snap_names and (added or gone):
# A reconnect can change the server's tool set; the frozen
# array kept the old names. Name both halves rather than
# reporting the connection as fully in the list.
parts: list[str] = []
if added:
parts.append(
f"{len(added)} new tool(s) join this conversation's tool list at the next "
"compaction or new conversation"
)
if gone:
parts.append(
f"{len(gone)} tool(s) that are no longer registered remain in the list and "
"calls to them fail"
)
notices.append(mcp_notice(name, "connected; " + "; ".join(parts) + "."))
elif snapshot is not None and not snap_names and live_names:
notices.append(
mcp_notice(
name,
f"connected, offering {len(live_names)} tool(s); they join this conversation's "
"tool list at the next compaction or new conversation.",
)
)
)
elif folded and snap["connected"]:
mine = [n for n in self.tools.names_from(snap["name"]) if n in offered]
if mine:
notices.append(mcp_notice(snap["name"], f"connected, offering {len(mine)} tool(s)."))
elif folded:
mine = [n for n in live_names if n in offered]
if mine:
notices.append(mcp_notice(name, f"connected, offering {len(mine)} tool(s)."))
elif snap_names:
notices.append(mcp_notice(name, _MCP_DISCONNECTED_REMAIN))
# A server removed outright has no row in status() at all, but its names
# are still advertised by the frozen array.
for name in sorted(snapshot_servers - seen):
notices.append(mcp_notice(name, _MCP_REMOVED_REMAIN))
return notices

def prewarm_mcp(self) -> None:
Expand Down
12 changes: 12 additions & 0 deletions raven/agent/loop/organ_glue.py
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,13 @@ async def _assemble_context_messages(
model answer for a routed one.
"""
from raven.context_engine import TurnContext # deferred — see module note
from raven.context_engine.history.markers import HISTORY_COMPACTED_META

# The session log this turn reads. ``/clear`` mints a new log id on the
# same key; a frozen array captured under the old one is dropped and
# recaptured here, before the engine budgets with it.
revision = session.revision if isinstance(session, Session) else None
self.tools.bind_session_revision(session_key, revision.log_id if revision is not None else None)

# Reset the metadata stash BEFORE calling the engine. If
# ``engine.assemble`` raises partway, the next caller falls back to
Expand All @@ -248,6 +255,11 @@ async def _assemble_context_messages(
session_revision=session.revision if isinstance(session, Session) else None,
),
)
if assembled.metadata.get(HISTORY_COMPACTED_META):
# Mark, not invalidate: this turn's array is the one its notices and
# system-prompt derivations were built from. The next assembly
# cashes the mark, so the array, notices and text all move together.
self.tools.schedule_definitions_refresh(session_key)
# Stash the engine's injected_skill_ids so the after-turn
# feedback dispatcher can read the source-qualified ids the
# unified engine populates via SkillForgeRouter. If the key is absent
Expand Down
Loading
Loading