Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
ccf533d
fix(guardrails): run work-queue items in the submitter's context
Nurozen Oct 15, 2026
6824435
feat(guardrails): add a request-scoped conversation id for rails
Nurozen Oct 15, 2026
d038f71
feat(server): set the conversation id from guardrails.thread_id
Nurozen Oct 19, 2026
df46301
feat(langchain): use the LangGraph thread ID as the conversation ID
Nurozen Oct 22, 2026
913a148
feat(guardrails): give each model call a round-trip scope
Nurozen Oct 22, 2026
8e873ec
fix(langchain): read the thread ID from the LangGraph run config
Nurozen Oct 24, 2026
ae4e6bf
fix(server): reject a thread ID that cannot leave the request
Nurozen Oct 22, 2026
c315d3f
test(langchain): expect the run-config thread ID only on Python 3.11+
Nurozen Oct 26, 2026
fa27b3f
refactor(guardrails): drop dead branches left by the request-ID refac…
Nurozen Oct 26, 2026
f6f9f37
fix(llmrails): carry the request IDs into streams over an external ge…
Nurozen Nov 2, 2026
5eaa9ad
fix(server): accept a thread ID in any script
Nurozen Nov 7, 2026
10a8772
fix(langchain): keep the before-model hook a no-op without input rails
Nurozen Nov 7, 2026
e9efc5a
fix(langchain): key a round trip on the message it answers
Nurozen Nov 10, 2026
381f6ff
fix(server): apply the thread's conversation ID on /v1/checks
Nurozen Nov 10, 2026
914de81
feat(library): add HiddenLayer request, token and verdict modules
Nurozen Oct 22, 2026
488f112
feat(library): add HiddenLayer rail config and actions
Nurozen Oct 22, 2026
abf9d10
feat(library): add HiddenLayer rail manifest, flows, docs and tests
Nurozen Oct 22, 2026
58f91d3
fix(library): evaluate tool results whose parts carry no text
Nurozen Oct 22, 2026
513a3b4
fix(library): drop a conversation ID no header can carry
Nurozen Oct 22, 2026
d252a65
test(library): pin the request ID in the round-trip payload tests
Nurozen Oct 22, 2026
9c227ce
docs(library): name the HiddenLayer rail's evaluations, not legs
Nurozen Oct 23, 2026
fbded25
refactor(library): send HiddenLayer the checks NeMo hands the rail
Nurozen Oct 23, 2026
66ce562
fix(library): stop sending HiddenLayer checks it will reject
Nurozen Oct 23, 2026
8bee41e
test(library): pin a multimodal user turn through the HiddenLayer gen…
Nurozen Oct 24, 2026
c38b3d8
fix(library): map a round-trip ID HiddenLayer cannot take onto one it…
Nurozen Oct 24, 2026
f5d46ec
fix(library): keep a token a concurrent refresh already replaced
Nurozen Nov 2, 2026
3608eed
refactor(library): bring the HiddenLayer rail in line with add-librar…
Nurozen Nov 6, 2026
dd963b5
fix(tests): filter the HiddenLayer identifying headers, not every HL-…
Nurozen Nov 6, 2026
21beba1
fix(library): keep the HiddenLayer rail's header lookup to itself
Nurozen Nov 7, 2026
6f3c67a
test(library): record the HiddenLayer rail's e2e suite
Nurozen Nov 7, 2026
246ed65
fix(library): never allow a verdict the HiddenLayer rail cannot apply
Nurozen Nov 10, 2026
e1a25e9
fix(library): reject HiddenLayer identities no header can carry
Nurozen Nov 10, 2026
dbf67b7
docs(library): say what fail_open actually covers
Nurozen Nov 10, 2026
a749c8f
fix(library): stop claiming the HiddenLayer rail sends a tool declara…
Nurozen Nov 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/configure-rails/actions/action-parameters.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,30 @@ async def check_preference(context: Optional[dict] = None):
return preference == "dark_mode"
```

## Reading the Conversation ID

The conversation ID is not an action parameter.
An action reads the ID that the caller set with `conversation_id()` by calling `get_conversation_id()`, which returns `None` when the caller set no ID.

```python
import logging

from nemoguardrails.actions import action
from nemoguardrails.context import get_conversation_id

log = logging.getLogger(__name__)

@action()
async def audit_message(text: str):
log.info("Checked a message in conversation %s", get_conversation_id())
return True
```

For how a caller sets the ID, refer to [Checking Messages Against Rails: Setting a Conversation ID](/run-guardrailed-inference/using-python-apis/check-messages#setting-a-conversation-id).

`get_roundtrip_id()` reads the ID of the model call the check belongs to in the same way, so an action can pair the request leg of a call with its response leg.
For how a caller sets that ID, refer to [Checking Messages Against Rails: Setting a Round-Trip ID](/run-guardrailed-inference/using-python-apis/check-messages#setting-a-round-trip-id).

## The `events` Parameter

The `events` parameter provides the complete event history:
Expand Down
272 changes: 272 additions & 0 deletions docs/configure-rails/guardrail-catalog/community/hiddenlayer.mdx

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions docs/configure-rails/guardrail-catalog/third-party.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,26 @@ rails:

For more details, check out the [Zscaler AI Guard Integration](/configure-guardrails/guardrail-catalog/third-party/zscaler-aiguard) page.

## HiddenLayer AI Runtime Security

The NeMo Guardrails library supports using [HiddenLayer AI Runtime Security](https://hiddenlayer.com/aisec-platform/ai-runtime-security/) for input, output, tool-call and tool-result checks. It detects prompt injection, jailbreaks, personal data and policy violations, and can return redacted text for the input and output surfaces.

To activate the protection, set the `HIDDENLAYER_CLIENT_ID` and `HIDDENLAYER_CLIENT_SECRET` environment variables. The tool surfaces run on `IORails` only.

### Example usage

```yaml
rails:
input:
flows:
- hiddenlayer check input
output:
flows:
- hiddenlayer check output
```

For more details, check out the [HiddenLayer AI Runtime Security Integration](/configure-guardrails/guardrail-catalog/third-party/hiddenlayer) page.

## HuggingFace Classifier

The NeMo Guardrails library supports using HuggingFace text classification models for fast, prompt-free content moderation on input, output, and retrieval flows. Classifiers can run locally via `transformers` or connect to remote inference servers (vLLM, KServe, FMS).
Expand Down
3 changes: 3 additions & 0 deletions docs/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,9 @@ navigation:
- page: GuardrailsAI
path: configure-rails/guardrail-catalog/community/guardrails-ai.mdx
slug: guardrails-ai
- page: HiddenLayer
path: configure-rails/guardrail-catalog/community/hiddenlayer.mdx
slug: hiddenlayer
- page: HuggingFace Classifier
path: configure-rails/guardrail-catalog/community/hf-classifier.mdx
slug: hf-classifier
Expand Down
6 changes: 5 additions & 1 deletion docs/integration/langchain/agent-middleware.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,10 @@ result2 = agent.invoke(
)
```

The middleware runs its input and output checks with the thread ID as the conversation ID, so rails that group checks by conversation see one conversation per thread.
A conversation ID that your application already set with `conversation_id()` is kept.
For more information, refer to [Checking Messages Against Rails: Setting a Conversation ID](/run-guardrailed-inference/using-python-apis/check-messages#setting-a-conversation-id).

---

## Known Limitations
Expand Down Expand Up @@ -327,7 +331,7 @@ Convenience subclass that only runs input rails. The `aafter_model` hook is a no

### OutputRailsMiddleware

Convenience subclass that only runs output rails. The `abefore_model` hook is a no-op.
Convenience subclass that only runs output rails. The `abefore_model` hook runs no rails and only opens the round trip that the output check reports under.

### GuardrailViolation

Expand Down
22 changes: 13 additions & 9 deletions docs/reference/rail-engine-support.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ content:
type: "reference"
---

The NVIDIA NeMo Guardrails library ships 36 built-in rails under `nemoguardrails/library`.
The NVIDIA NeMo Guardrails library ships 37 built-in rails under `nemoguardrails/library`.
Each rail declares its behavior in a [rail manifest](/reference/rail-manifests), and each manifest exposes one or more *surfaces*.
A surface is the flow name you list under `rails.input.flows`, `rails.output.flows`, or `rails.retrieval.flows` in `config.yml`.

The 36 manifests declare 86 surfaces in total.
`LLMRails` runs 80 of them through its Colang runtime, in both Colang 1.0 and Colang 2.x.
`IORails` runs 67 of them by compiling the manifest directly, without Colang.
The 37 manifests declare 90 surfaces in total.
`LLMRails` runs 82 of them through its Colang runtime, in both Colang 1.0 and Colang 2.x.
`IORails` runs 71 of them by compiling the manifest directly, without Colang.

This page lists every surface and the engine that can run it.
For capability areas that are not tied to a specific rail, such as streaming, observability, and the generation API, refer to [Engine Feature Support](/reference/engine-feature-support).
Expand All @@ -37,12 +37,12 @@ The following table summarizes surface support by direction:

| Direction | Surfaces | LLMRails | IORails |
|---|---:|---:|---:|
| Input | 34 | 34 | 33 |
| Output | 36 | 36 | 29 |
| Input | 35 | 35 | 34 |
| Output | 37 | 37 | 30 |
| Retrieval | 11 | 11 | 0 |
| Tool_call | 3 | 0 | 3 |
| Tool_result | 3 | 0 | 3 |
| **Total** | **87** | **81** | **68** |
| Tool_call | 4 | 0 | 4 |
| Tool_result | 4 | 0 | 4 |
| **Total** | **91** | **83** | **72** |

`IORails` does not accept a `rails.retrieval` section at all, so every retrieval surface is an `LLMRails` capability.
A configuration that declares one routes to `LLMRails` by default.
Expand Down Expand Up @@ -73,6 +73,7 @@ The following table lists the input surfaces:
| `gliner detect pii on input` | GLiNER | ✓ | ✓ | |
| `gliner mask pii on input` | GLiNER | ✓ | ✓ | Rewrites `user_message` |
| `guardrailsai check input` | Guardrails AI | ✓ | ✓ | |
| `hiddenlayer check input` | HiddenLayer AI Runtime Security | ✓ | ✓ | Rewrites `user_message` |
| `hf classifier check input` | Hugging Face Classifier | ✓ | ✓ | |
| `jailbreak detection heuristics` | Jailbreak Detection | ✓ | ✗ | Backend ambiguity, [issue #2285](https://github.com/NVIDIA-NeMo/Guardrails/issues/2285) |
| `jailbreak detection model` | Jailbreak Detection | ✓ | ✓ | |
Expand Down Expand Up @@ -116,6 +117,7 @@ The following table lists the output surfaces:
| `gliner mask pii on output` | GLiNER | ✓ | ✓ | Rewrites `bot_message` |
| `guardrailsai check output` | Guardrails AI | ✓ | ✓ | |
| `self check hallucination` | Hallucination Detection | ✓ | ✗ | Reads `_last_bot_prompt` |
| `hiddenlayer check output` | HiddenLayer AI Runtime Security | ✓ | ✓ | Rewrites `bot_message` |
| `hf classifier check output` | Hugging Face Classifier | ✓ | ✓ | |
| `injection detection` | Injection Detection | ✓ | ✓ | Rewrites `bot_message` |
| `llama guard check output` | Llama Guard | ✓ | ✓ | |
Expand Down Expand Up @@ -164,6 +166,7 @@ Per-tool checks are an `IORails`-only capability by design:
| Flow | Rail | LLMRails | IORails | Notes |
|---|---|:---:|:---:|---|
| `cel check tool call` | Common Expression Language (CEL) | ✗ | ✓ | IORails-only capability by design |
| `hiddenlayer check tool call` | HiddenLayer AI Runtime Security | ✗ | ✓ | IORails-only capability by design |
| `regex check tool call` | Regex Detection | ✗ | ✓ | IORails-only capability by design |
| `tool safety check tool call` | Tool Safety Check | ✗ | ✓ | IORails-only capability by design |

Expand All @@ -175,6 +178,7 @@ Per-tool checks are an `IORails`-only capability by design:
| Flow | Rail | LLMRails | IORails | Notes |
|---|---|:---:|:---:|---|
| `cel check tool result` | Common Expression Language (CEL) | ✗ | ✓ | IORails-only capability by design |
| `hiddenlayer check tool result` | HiddenLayer AI Runtime Security | ✗ | ✓ | IORails-only capability by design |
| `regex check tool result` | Regex Detection | ✗ | ✓ | IORails-only capability by design |
| `tool safety check tool result` | Tool Safety Check | ✗ | ✓ | IORails-only capability by design |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ This is useful when you can only send the latest message rather than the full hi

<Tip>

The `thread_id` must be between 16 and 255 characters long.
The `thread_id` must be between 16 and 255 characters long, and must consist of printable ASCII characters with no leading or trailing whitespace.

</Tip>

Expand All @@ -355,6 +355,9 @@ response = requests.post(f"{base_url}/v1/chat/completions", json={
# The assistant remembers "Alice"
```

Rails and actions that run for the request see the `thread_id` as the conversation ID that `get_conversation_id()` returns.
For more information, refer to [Checking Messages Against Rails: Setting a Conversation ID](/run-guardrailed-inference/using-python-apis/check-messages#setting-a-conversation-id).

<Note>
The `thread_id` is currently not implemented in the NeMo Guardrails microservices.
</Note>
Expand Down
47 changes: 46 additions & 1 deletion docs/run-rails/using-python-apis/check-messages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
title: "Checking Messages Against Rails"
sidebar-title: "Check Messages"
description: "Validate messages against input, output, and tool rails using check_async and check methods on the LLMRails and IORails engines."
keywords: ["check_async", "check", "RailsResult", "RailStatus", "RailType", "input rails", "output rails", "tool rails", "ToolViolation", "validation", "IORails", "LLMRails"]
keywords: ["check_async", "check", "RailsResult", "RailStatus", "RailType", "input rails", "output rails", "tool rails", "ToolViolation", "validation", "IORails", "LLMRails", "conversation_id"]
content:
type: "reference"
---
Expand Down Expand Up @@ -335,6 +335,51 @@ result = await rails.check_async([

For more information about context variables, refer to [Core Classes: Passing Context](/run-guardrailed-inference/using-python-apis/core-classes#passing-context).

### Setting a Conversation ID

To tell rails which conversation a call belongs to, wrap the call in `conversation_id()` from `nemoguardrails.context`.
Every rail and action that runs for a call made inside the block sees the ID.
This works on both engines and for `check_async()`, `check()`, `generate_async()`, and `stream_async()`.
The ID is not added to the messages and is not sent to the main model.

```python
from nemoguardrails.context import conversation_id

with conversation_id("thread-42"):
result = await rails.check_async([
{"role": "user", "content": "I need help with my account"}
])
```

The previous ID is restored when the block exits, and concurrent requests each see only the ID they were made with.
Use an ID that identifies one conversation, such as a chat thread ID.
The server sets the ID from the `thread_id` of a [chat completion request](/run-guardrailed-inference/using-fastapi-server/chat-with-guardrailed-model#conversation-threads), unless the application that runs the server already set one.
A rail that reports checks to an external service can send the ID so that the service groups the checks of one conversation; the rail's catalog page describes whether it does.
For how a custom action reads the ID, refer to [Action Parameters: Reading the Conversation ID](/configure-guardrails/actions/action-parameters#reading-the-conversation-id).

### Setting a Round-Trip ID

A round trip is one model call: the request leg that checks what goes to the model and the response leg that checks what comes back.
Both legs report under the same round-trip ID, which lets a rail that reports to an external service pair them.
`generate_async()`, `stream_async()`, and `check_async()` open a round trip and mint an ID for it, so a caller that runs a whole model call through one of them has nothing to set.

Set the ID yourself with `roundtrip_id()` from `nemoguardrails.context` when the application knows where a model call starts and ends and NeMo Guardrails cannot:
an agent framework runs the checks of one call as separate `check_async()` calls, each of which would otherwise open a round trip of its own.

```python
from nemoguardrails.context import roundtrip_id
from nemoguardrails.rails.llm.options import RailType

with roundtrip_id(call_id):
checked = await rails.check_async(messages, rail_types=[RailType.INPUT])
# ... call the model ...
await rails.check_async(messages + [response], rail_types=[RailType.OUTPUT])
```

An ID you set is never overwritten, and the previous one is restored when the block exits.
NeMo Guardrails does not interpret the ID; a rail that requires a particular format converts it.
An action reads the ID with `get_roundtrip_id()`.

### Checking on the IORails Engine

Construct the `Guardrails` facade to run checks on the IORails engine.
Expand Down
2 changes: 1 addition & 1 deletion docs/telemetry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ In this context, a *session* is the lifetime of a single Python process running

Each value identifies a built-in library feature. The built-in rail manifests are the authoritative source. Related manifests can share a stable feature identifier so telemetry remains comparable across releases.

`activefence`, `ai_defense`, `autoalign`, `cel`, `clavata`, `cleanlab`, `content_safety`, `context_bloat_detection`, `crowdstrike_aidr`, `f5`, `factchecking`, `fiddler`, `gcp_moderate_text`, `gliner`, `guardrails_ai`, `hallucination`, `hf_classifier`, `injection_detection`, `jailbreak_detection`, `language_detector`, `llama_guard`, `pangea`, `patronusai`, `policyai`, `polygraf`, `prompt_security`, `regex`, `self_check`, `sensitive_data_detection`, `tool_safety_check`, `topic_safety`, `trend_micro`, `zscaler_aiguard`.
`activefence`, `ai_defense`, `autoalign`, `cel`, `clavata`, `cleanlab`, `content_safety`, `context_bloat_detection`, `crowdstrike_aidr`, `f5`, `factchecking`, `fiddler`, `gcp_moderate_text`, `gliner`, `guardrails_ai`, `hallucination`, `hf_classifier`, `hiddenlayer`, `injection_detection`, `jailbreak_detection`, `language_detector`, `llama_guard`, `pangea`, `patronusai`, `policyai`, `polygraf`, `prompt_security`, `regex`, `self_check`, `sensitive_data_detection`, `tool_safety_check`, `topic_safety`, `trend_micro`, `zscaler_aiguard`.

## Data Not Collected by Telemetry

Expand Down
24 changes: 24 additions & 0 deletions examples/configs/hiddenlayer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# HiddenLayer AI Runtime Security Example

This example checks user input, bot output, and the calls to and results from the `run_sql` tool with HiddenLayer AI Runtime Security.

Set the credentials, and set `project_id` in `config.yml` to your HiddenLayer project ID or alias:

```bash
export HIDDENLAYER_CLIENT_ID=...
export HIDDENLAYER_CLIENT_SECRET=...
export OPENAI_API_KEY=...
```

The tool rails run on the IORails engine, which the `Guardrails` class selects for this configuration:

```python
from nemoguardrails import Guardrails, RailsConfig

config = RailsConfig.from_path("examples/configs/hiddenlayer")
rails = Guardrails(config)
```

The `run_sql` tool is declared on the main model, so the per-tool rails can validate its calls.
The `rails.output.streaming` block buffers a streamed response and checks it once at the end, which lets HiddenLayer redact streamed output.
Refer to the HiddenLayer page in the guardrail catalog for the full configuration reference.
47 changes: 47 additions & 0 deletions examples/configs/hiddenlayer/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
models:
- type: main
engine: openai
model: gpt-5-nano
parameters:
tools:
- type: function
function:
name: run_sql
description: Run a read-only SQL query against the orders database.
parameters:
type: object
properties:
query:
type: string
required:
- query

instructions:
- type: general
content: You are a helpful assistant.

rails:
config:
hiddenlayer:
project_id: my-project
input:
flows:
- hiddenlayer check input
output:
flows:
- hiddenlayer check output
streaming:
enabled: true
stream_first: false
context_size: 0
chunk_size: 1000000
tool_call:
flows:
- tool call validation
per_tool:
run_sql: [hiddenlayer check tool call]
tool_result:
flows:
- tool result validation
per_tool:
run_sql: [hiddenlayer check tool result]
Loading
Loading