[CharmingGroot]

Stacked on # — merge that first.

Teams already route LLM traffic through Envoy or LiteLLM. Rather than become another gateway, pasu now plugs into those. One Judge answers three front-ends, selected with --frontend, with the same policy, audit stream and HITL.

Before building, I surveyed how others integrate:

• Envoy-family gateways standardise on ext_proc: Envoy Gateway, agentgateway, GKE service extensions, NeMo Guardrails.
• None of those guardrails blocks tool calls in responses. Agent Router's GuardrailPolicy and agentgateway's webhook see text only.
• LiteLLM already ships tool_permission (name/argument regex). So pasu's value there is the single policy shared with the kernel layer, Ask, Korean PII redaction and one audit stream, not name matching.

Judge

The reverse proxy answered three questions inline: may this endpoint be used, may this prompt leave, may these tool calls reach the agent. Judge holds them transport-free. ProxyState is now Judge plus where to forward.

While doing this I found a redaction bug. Findings from every message were pooled, so a span found in one message overwrote the same offset in every other. Each text is now redacted with its own findings (regression test, mutation-checked).

--frontend ext-proc (Envoy)

• Request headers route the endpoint and ask the upstream for identity encoding.
• The request body is screened. A rewrite carries its new content-length, because Envoy checks it.
• The response body is parsed and its tool calls judged.
• Every refusal is an ImmediateResponse sent before the client sees a byte.

Bodies must be BUFFERED. pasu reads the body modes Envoy reports and refuses (500) a completion it would never be shown. Otherwise, with response_body_mode: NONE, every tool call passes unseen: on Envoy 1.37 a build without this check let a denied shell call through with a 200.

--frontend litellm

This speaks LiteLLM's Generic Guardrail API:

• Prompts are screened and redacted in order; LiteLLM writes them back by position.
• Proposed tool calls are judged.
• History tool calls on a request are not judged; they already ran.
• An optional shared key is read from PASU_GUARDRAIL_API_KEY.

LiteLLM consults a guardrail while it streams. With the built-in integration, a denied call's chunks reached the client before the block (measured on LiteLLM 1.91). integrations/litellm/pasu_guardrail.py fixes this: it subclasses generic_guardrail_api and holds a streamed completion until pasu has judged it, as LiteLLM's own tool_permission does. In the same run, no chunk left.

Verified against the real thing

	Envoy 1.37	LiteLLM 1.91 (built-in / PasuGuardrail)
text, stream and not	200	200 / 200
denied tool call	403	400 / 400
denied tool call, streamed	403, no chunk	chunks leaked / 400, no chunk
Korean RRN in prompt	403, provider never saw it	400 / 400
redaction	provider got [REDACTED:ko-rrn]	same
pasu down	500	refused (fail_closed)

integrations/ has tested example configs and a coverage table. Two things in it are worth knowing: LiteLLM 1.91 wants a top-level guardrails: key (its own example config fails to start), and Envoy's 200 ms default message_timeout is too short for HITL.

Dependencies

The ext-proc feature, on by default, pins envoy-types 0.7.3 and tonic 0.14.5, the last releases within our 1.86 MSRV. Their licenses (Apache-2.0 and MIT) are on the deny.toml allowlist.

Docs

• docs/attach.svg: three ways into one judge.
• docs/pipeline.svg: the four checks, and where each runs in every front-end.
• docs/litellm-streaming.svg: the streaming comparison.
• The README (ko/en), deployment.md and adapting.md are updated.

🤖 Generated with Claude Code