Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions docs/architecture/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,45 @@ session and components, and links verified citations into the memory and documen

## The bridge: each caller's own identity

Before checking model spend, the bridge reads the authorized namespace's public
mirrored-source registrations. Any active registration marks the complete answer,
even when its tools admit no document. The chat route sends `data: {"mirrored":true}`
as its first SSE event, or `mirrored: true` as its whole response's first field.
`assistant.ask` carries the same first field. A marked refusal carries it too. This
decision also routes the answer's cost row to private storage and suppresses public
diagnostics before subsequent model, budget or tool work.

Marked operation results carry the same server-owned flag after their typed output is
serialized. HTTP success responses retain it before post-handler rows. MCP success
responses retain it in the outer envelope, tool result, structured content and canonical
JSON text. The search tool's prose summary follows a separate first JSON marker block,
so consuming its text blocks in order observes protection before the summary. Ordinary
unmarked results keep their existing shapes.

The answer token keeps the caller's `session_protection` promise and a separate
`answer_mirrors` admission bit. Only an already marked answer for a declared caller
receives positive admission. Missing or false admission disables its mirror legs,
including when a source becomes active during an initially ordinary answer. Ordinary
namespace reads retain their authorization. Current ACL checks still decide each
mirrored item; the bit grants no access by itself. Ordinary delegated children omit
both claims and cannot regain mirror admission through a request header. An answer
token or any descendant cannot start another assistant answer.

Streaming output is provisional until `[DONE]`. The response owns its required
accounting lifetime beyond HTTP headers: it closes and joins model and citation work,
captures final usage once, and confirms private cost delivery before sending `[DONE]`
or a terminal error. Disconnects join producer cleanup and required delivery before
propagating cancellation. A failed cost write cannot publish a completed answer.
Whole answers and checked tool responses retain their accounting-before-publication
boundary. Incremental text precedes final usage, so its completion boundary is the
terminal event. A streaming client must reject EOF without `[DONE]` as incomplete.

Mirrored-source creation stays unavailable until every serving build and consumer
implements the admission, first-output marker, session and accounting boundaries.
An older verifier accepting an unknown claim does not enforce it; schema equality
does not establish this capability. The fleet and rollback requirements in
[rolling upgrades](rolling-upgrades.md) still apply.

LibreChat's agents API authenticates a LibreChat API key and runs the agent as that
key's LibreChat user. The agent's tools are Loom's MCP server, and what they may read is
decided by the token on each tool call, never by the LibreChat user. LibreChat v0.8.8
Expand Down
12 changes: 12 additions & 0 deletions docs/architecture/document-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,18 @@ their contributing proofs. A raw download owns the same required marked accounti
attempt as a registered operation. Cancellation and disconnects still propagate
through owned cleanup. These checks end at the server's response-send boundary.

The assistant's incremental stream has a terminal accounting boundary because final
usage is unavailable before its early text. Its server-owned first event is
`data: {"mirrored":true}` when marked. Partial text remains provisional until the
response closes and joins its producers, captures final usage and confirms its
required private cost row before `[DONE]` or a terminal error. Disconnect cleanup
retains and joins that required delivery. Whole answers and checked source frames
keep the accounting-before-publication boundary above. The answer token's distinct
`answer_mirrors` bit fixes mirror admission before model work, so a source activated
during a negative answer cannot introduce an unmarked tool leg. Missing admission is
false; ordinary delegation drops it. See the
[assistant boundary](assistant.md#the-bridge-each-callers-own-identity).

A multi-entry MCP batch refuses every response-bearing marked entry with the same
instruction to send it individually, including denied and empty entries. Notifications
remain silent. It discards each marked entry's complete frame before buffering it or
Expand Down
1 change: 1 addition & 0 deletions docs/reference/operations/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Ask the Loom assistant one question about this namespace and get one cited answe

| Output field | Type | Meaning |
| --- | --- | --- |
| `mirrored` | `bool` | True when this answer is private to the caller's protected session; false by default. |
| `answer` | `str` | The assistant's answer, with its citations as the agent wrote them. |
| `run_id` | `str` | The answer's id: the run its tool calls are ledgered under (agent_events.run_id). |
| `model` | `str` | The model that answered. |
Expand Down
10 changes: 9 additions & 1 deletion sdk/python/loom/assistant.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,16 @@ async def stream(
the text written so far."""
body = {"stream": True, "messages": _messages(question, history)}
reading = _Reading()
complete = False
async with self._agent.transport.stream(
ROUTE, json=body, headers=self._headers(namespace), read_timeout_s=read_timeout_s
) as resp:
request_id = resp.headers.get("X-Request-Id", "")
async for line in resp.aiter_lines():
if not line.startswith("data: ") or line == "data: [DONE]":
if line == "data: [DONE]":
complete = True
continue
if complete or not line.startswith("data: "):
continue
try:
event = json.loads(line[6:])
Expand All @@ -196,6 +200,10 @@ async def stream(
err = LoomError(f"{ROUTE}: the stream ended before the answer began")
err.request_id = request_id
raise err
if not complete:
err = LoomError(f"{ROUTE}: the stream ended before completion")
err.request_id = request_id
raise err
yield reading.answer()

async def ask(
Expand Down
1 change: 1 addition & 0 deletions sdk/python/loom/ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -5367,6 +5367,7 @@ class AssistantAskBody(TypedDict, total=False):


class AssistantAskOutput(TypedDict, total=False):
mirrored: NotRequired[bool]
answer: NotRequired[str]
run_id: NotRequired[str]
model: NotRequired[str]
Expand Down
25 changes: 25 additions & 0 deletions sdk/typescript/src/assistant.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,31 @@ describe("loom.assistant.stream", () => {
await expect(collect(loom)).rejects.toThrow(/before the answer began/);
});

it("keeps partial text incomplete when EOF arrives without DONE", async () => {
const { loom, sent } = client(() => new Response(ANSWER.replace(/data: \[DONE\]\n\n$/, "")));
const seen: AssistantEvent[] = [];
const read = async () => {
for await (const event of loom.assistant.stream({ question: "q" })) seen.push(event);
};
await expect(read()).rejects.toThrow(/before completion/);
expect(seen).toEqual([
{ type: "text", text: "Deploys freeze " },
{ type: "text", text: "on Fridays." },
]);
expect(sent).toHaveLength(1);
});

it("ignores text received after DONE", async () => {
const { loom } = client(() => new Response(ANSWER + sse(chunk("discarded"))));
const out = await collect(loom);
expect(out.slice(0, 2)).toEqual([
{ type: "text", text: "Deploys freeze " },
{ type: "text", text: "on Fridays." },
]);
expect(out).toHaveLength(3);
expect(out[2]?.type === "answer" && out[2].answer.text).toBe("Deploys freeze on Fridays.");
});

it("throws a connection cut mid-stream as NETWORK", async () => {
const { loom } = client(() => new Response(body(sse(chunk("Deploys ")), [5], new Error("reset"))));
const err = await collect(loom).then(
Expand Down
8 changes: 7 additions & 1 deletion sdk/typescript/src/assistant.ts
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,14 @@ export function assistantChat(transport: Transport): AssistantChat {
steps: [],
stepsCut: false,
};
let complete = false;
try {
for await (const data of events(res, transport.baseUrl, requestId, cut.signal, touch)) {
if (data === "[DONE]") continue;
if (data === "[DONE]") {
complete = true;
continue;
}
if (complete) continue;
const event = parse(data);
if (!event) continue;
if (isObject(event.error)) {
Expand All @@ -173,6 +178,7 @@ export function assistantChat(transport: Transport): AssistantChat {
}
if (!answer.runId)
throw new LoomError(`${ASSISTANT_ROUTE}: the stream ended before the answer began`, { requestId });
if (!complete) throw new LoomError(`${ASSISTANT_ROUTE}: the stream ended before completion`, { requestId });
pause();
yield { type: "answer", answer };
} finally {
Expand Down
2 changes: 2 additions & 0 deletions sdk/typescript/src/generated/ops.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ export interface AssistantCitation {
}

export interface AssistantAskOutput {
/** True when this answer is private to the caller's protected session; false by default. */
mirrored?: boolean;
answer?: string;
run_id?: string;
model?: string;
Expand Down
6 changes: 6 additions & 0 deletions server/assistant/api/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,12 @@ class AssistantCitation(BaseModel):


class AssistantAskOutput(BaseModel):
mirrored: bool = Field(
default=False,
description=(
"True when this answer is private to the caller's protected session; false by default."
),
)
# The assistant's answer, with its citations as the agent wrote them.
answer: str = ""
# The answer's id: the run its tool calls are ledgered under (agent_events.run_id).
Expand Down
1 change: 1 addition & 0 deletions server/assistant/app/ask.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ async def handle_assistant_ask(inp: AssistantAskInput) -> AssistantAskOutput:
messages = [m.model_dump() for m in inp.history] + [{"role": "user", "content": inp.question}]
answer = await bridge.ask(auth, inp.org, inp.namespace, messages)
return AssistantAskOutput(
mirrored=answer.mirrored,
answer="".join(answer.text),
run_id=answer.run_id,
citations=answer.citations,
Expand Down
Loading
Loading