Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
20 changes: 1 addition & 19 deletions .github/workflows/framework-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,6 @@ on:
- 'docs-site/go.mod'
- 'docs-site/go.sum'
- 'docs-site/hugo.yaml'
# Layered on top of hugo.yaml by the build step below, so a change to it
# changes what this job renders. Drop this line when the overlay goes
# away at the 1.0 release.
- 'docs-site/hugo.preview.yaml'
- 'docs-site/.docs-test.toml'
- '.github/workflows/framework-tests.yml'
workflow_dispatch:
Expand Down Expand Up @@ -122,21 +118,7 @@ jobs:
npm ci
# Capture the build log so hugo-warnings.spec can scan it for ERROR /
# WARN lines; surface it and fail if the build itself fails.
#
# The preview overlay and -D are what put content/kagent/1.x/ in this
# build at all. That section is withheld from production by a draft
# cascade in 1.x/_index.md, so a plain `hugo --config hugo.yaml`
# renders ZERO 1.x pages and the `content` scanners below pass green
# having walked none of them — which is the whole reason this job
# includes docs-site/content/** in its path filter. The overlay also
# restores the 1.x entry in params.versions, without which every
# {{< version include-if="1.x" >}} renders empty and hugo-warnings
# sees a clean log for a build that is quietly missing a docset.
# Both are temporary: at the 1.0 release the draft keys and
# hugo.preview.yaml go away, and this line reverts to
# `hugo --config hugo.yaml --gc --minify`. links.yml and the
# Makefile's serve-docs target layer the overlay the same way.
hugo --config hugo.yaml,hugo.preview.yaml -D --gc --minify > .build.log 2>&1 || { cat .build.log; exit 1; }
hugo --config hugo.yaml --gc --minify > .build.log 2>&1 || { cat .build.log; exit 1; }
- name: Install harness dependencies
working-directory: docs-theme-extras
Expand Down
25 changes: 15 additions & 10 deletions .github/workflows/update-ref-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ concurrency:

# The docs are a Hugo site under docs-site/ (served at /docs). This workflow
# generates the reference pages directly as Hugo markdown into docs-site/content:
# - kagent CRD API ref -> docs-site/content/kagent/1.x/reference/api-ref.md
# - kagent CRD API ref -> docs-site/content/kagent/latest/reference/api-ref.md
# - kmcp CRD API ref -> docs-site/content/kmcp/reference/api-ref.md
# - kagent Helm ref -> docs-site/content/kagent/1.x/reference/helm.md
# - kagent Helm ref -> docs-site/content/kagent/latest/reference/helm.md
# Each page uses plain Hugo YAML frontmatter (no MDX `export const metadata`).
#
# It also refreshes the version conrefs under
Expand All @@ -48,7 +48,11 @@ concurrency:
# 0.10.x is the stable line and still takes patches -- v0.10.2 landed on
# 2026-09-23 -- so it is not frozen, and the by-hand regeneration it used to
# need is what let its pages drift in the first place. Publishing a docs
# release tagged v0.10.2 regenerates 0.x; v1.0.0-alphaN regenerates 1.x.
# release tagged v0.10.2 regenerates 0.x; v1.0.N regenerates latest.
#
# main/ is not generated here. It documents unreleased work, so no release tag
# describes it, and it took its reference pages from latest/ when 1.0.0 was
# cut. Its version-split conref spans are asserted unchanged on every run.
#
# The two must never write each other's values. Every conref a page of either
# line reads is version-split, and the conref helper refuses to continue if a
Expand Down Expand Up @@ -166,13 +170,14 @@ jobs:
fi
# Which documented line this release belongs to. Both are live: 0.10.x
# is the stable line and still takes patches, 1.0 is the alpha line.
# still takes patches, and 1.x is the current release line, documented under
# latest/.
# Everything downstream -- page paths, API version, frontmatter,
# which conrefs are in scope -- follows from this.
case "$KAGENT_VERSION" in
0.*) LINE=0.x ;;
1.*) LINE=1.x ;;
*) echo "Error: $KAGENT_VERSION belongs to no documented line. Only 0.x and 1.x exist."; exit 1 ;;
1.*) LINE=latest ;;
*) echo "Error: $KAGENT_VERSION belongs to no documented line. Only 0.x and latest (1.x) exist."; exit 1 ;;
esac
echo "Documenting the $LINE line."
echo "line=$LINE" >> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -304,7 +309,7 @@ jobs:
WITH_RUNTIME_IMAGES=false
WITH_KMCP=false
;;
1.x)
latest)
SECTION=reference
API_DIR=go/api/v1alpha3
API_TITLE="API reference"
Expand All @@ -317,9 +322,9 @@ jobs:
HELM_WEIGHT=20
# --weight 30 slots CLI docs after api-ref (10) and helm (20).
CLI_WEIGHT=30
# Version-root-relative path behind the `link` shortcode. 1.x
# Version-root-relative path behind the `link` shortcode. latest
# pages are rebased into the enterprise docs hub, so their
# cross-links must not hardcode the 1.x segment.
# cross-links must not hardcode the version segment.
KAGENT_CLI_LINK_PREFIX=reference/cli
WITH_RUNTIME_IMAGES=true
WITH_KMCP=true
Expand Down Expand Up @@ -609,7 +614,7 @@ jobs:
}
# Version-split conrefs. Each carries one span per documented line:
# {{< version include-if="0.x" >}}A{{< /version >}}{{< version include-if="1.x" >}}B{{< /version >}}
# {{< version include-if="0.x" >}}A{{< /version >}}{{< version include-if="latest" >}}B{{< /version >}}{{< version include-if="main" >}}C{{< /version >}}
# Only this run's span is written, and the helper refuses to continue
# if any other span moved -- that assertion is the whole safety net
# keeping a 0.10.x run from rewriting 1.x values and vice versa.
Expand Down
2 changes: 1 addition & 1 deletion DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ To preview the docs alone at http://localhost:1313/docs/, use one of these comma
| Command | Builds |
|---|---|
| `make serve-docs` | Every docs version in `docs-site/content/`. |
| `make serve-docs VERSION=<linkVersion>` | Only one kagent version, such as `VERSION=1.x` or `VERSION=latest`, for a faster preview. kmcp always builds. Links into other versions do not resolve locally. |
| `make serve-docs VERSION=<linkVersion>` | Only one kagent version, such as `VERSION=main` or `VERSION=latest`, for a faster preview. kmcp always builds. Links into other versions do not resolve locally. |
| `make serve-docs NO_SEARCH=1` | Same as `make serve-docs`, without the search index. The search box does nothing, but the build is faster. Combine with `VERSION=`. |

To preview the docs and marketing site together the way they deploy (so
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ inject-docs: ## Copy built docs into public/docs (preserves tracked assets, e.g.
# back but the stylesheets never do, and the preview degrades edit by edit
# instead of failing outright. Rendering to memory removes the shared directory.
#
# VERSION=<linkVersion> (for example VERSION=1.x, or VERSION=latest) builds only
# VERSION=<linkVersion> (for example VERSION=main, or VERSION=latest) builds only
# that docs version, for a faster preview. scripts/local-version-config.py
# derives the skip list from docs-site/hugo.yaml's `params.versions` and writes
# docs-site/hugo-local-version.yaml (gitignored). NO_SEARCH=1 builds without the
Expand Down
6 changes: 3 additions & 3 deletions docs-site/.docs-test.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ scanRoots = [
"./content",
]

# kagent docs are versioned (0.x frozen, 1.x in-progress rewrite), nested
# under /docs/kagent/<version>/... . kmcp, the other product built into this
# kagent docs are versioned (main in development, latest released, 0.x frozen),
# nested under /docs/kagent/<version>/... . kmcp, the other product built into this
# same Hugo site, stays flat/unversioned — see the sections.kagent.versions
# comment in hugo.yaml for why versions are scoped per-section rather than
# site-wide.
[versioning]
versionFromPath = "^/docs/kagent/(?<version>[^/]+)/"
versions = ["0.x", "1.x"]
versions = ["main", "latest", "0.x"]

# All checks default to enabled. No [checks] overrides are needed: kagent runs
# the full default set. (The former `smoke`/`crossBrowser` toggles were removed
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{{< version include-if="0.x" >}}0.0.9{{< /version >}}{{< version include-if="1.x" >}}1.0.0{{< /version >}}
{{< version include-if="0.x" >}}0.0.9{{< /version >}}{{< version include-if="latest" >}}1.0.0{{< /version >}}{{< version include-if="main" >}}1.0.0{{< /version >}}
2 changes: 1 addition & 1 deletion docs-site/assets/kagent-docs/versions/agent-substrate.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{{< version include-if="0.x" >}}0.0.9{{< /version >}}{{< version include-if="1.x" >}}1.0.0{{< /version >}}
{{< version include-if="0.x" >}}0.0.9{{< /version >}}{{< version include-if="latest" >}}1.0.0{{< /version >}}{{< version include-if="main" >}}1.0.0{{< /version >}}
2 changes: 1 addition & 1 deletion docs-site/assets/kagent-docs/versions/kagent.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{{< version include-if="0.x" >}}0.10.6{{< /version >}}{{< version include-if="1.x" >}}1.0.0-rc3{{< /version >}}
{{< version include-if="0.x" >}}0.10.6{{< /version >}}{{< version include-if="latest" >}}1.0.0-rc3{{< /version >}}{{< version include-if="main" >}}1.0.0-rc3{{< /version >}}
2 changes: 1 addition & 1 deletion docs-site/assets/kagent-docs/versions/max-kube.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{{< version include-if="0.x" >}}1.35{{< /version >}}{{< version include-if="1.x" >}}1.35{{< /version >}}
{{< version include-if="0.x" >}}1.35{{< /version >}}{{< version include-if="latest" >}}1.35{{< /version >}}{{< version include-if="main" >}}1.35{{< /version >}}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: kagent
linkTitle: "kagent 1.x"
linkTitle: "kagent 1.0"
description: Explore concepts, guides, and reference docs for running declarative AI agents on Kubernetes with Agent Substrate, from install through day-two operations.
weight: 10
author: kagent.dev
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
title: Agent Substrate architecture
description: See how Agent Substrate runs, suspends, and resumes the Actors behind every Session.
weight: 40
author: kagent.dev
aliases:
- /kagent/latest/about/agent-substrate/
---

{{< reuse "kagent-docs/snippets/about/agent-substrate-intro.md" >}}

## ActorTemplate

An **ActorTemplate** is the definition that Agent Substrate creates an Actor from. It states the agent's container image, the command that starts the agent, the agent's environment, the sandbox class to isolate it with, and the WorkerPool to schedule it onto. The kagent controller compiles an {{< gloss "Agent" >}}Agent{{< /gloss >}} and the resources that it names into an ActorTemplate.

Agent Substrate rejects any change to an ActorTemplate's spec after it is created, so a running Session stays on a fixed definition. The controller creates a new ActorTemplate for every compiled {{< gloss "Revision" >}}revision{{< /gloss >}} instead of editing an existing one, and it reclaims an old ActorTemplate once nothing references its revision. An Agent references one for as long as the revision is its desired or its latest successful one, and a Session references one for as long as it runs on that revision.

## Workers and WorkerPools

Each Actor runs on a **Worker**, a pod that Agent Substrate starts in advance. A Session that needs an Actor gets one on a Worker that is already running, so no Actor waits for a new pod. The sandbox boundary sits around the Actor, not around the Worker pod that hosts it.

### WorkerPools

Workers come from a **WorkerPool**, a Kubernetes custom resource that an operator provisions before any Agent can compile. A WorkerPool declares how many Workers to keep running and which sandbox technology those Workers use.

An operator never creates a Worker directly. Agent Substrate manages them and keeps enough ready in each WorkerPool for an Actor to start or resume immediately.

A WorkerPool exposes a Kubernetes scale subresource, so a HorizontalPodAutoscaler can size a pool from a metric instead of a fixed replica count. Agent Substrate publishes how many Workers in each pool have no free Actor slot. An autoscaler that watches this number adds Workers before the pool fills, instead of after placements fail.

<!-- REVIEW (runtime reviewer): queue-depth scaling is still unconfirmed and is deliberately not described here. Confirm whether it is supported before adding it. -->
<!-- REVIEW (runtime reviewer): ate-api-server publishes this signal as ate_workerpool_workers{state="at_capacity"}. Only the Kind-only demos/autoscaled-workerpool demo wires it into an HPA, through prometheus-adapter, and the chart ships no HPA. Confirm that this series is the supported autoscaling signal before documenting a worked HPA example. -->

### Worker capacity

One Worker hosts many Actors at the same time. Each Worker reports its maximum Actor capacity when it starts. A Worker at that capacity refuses further Actors, and Agent Substrate places them on another Worker in the pool. The Worker's Actor capacity is a property of the Worker build, not a setting on the pool. The gVisor and micro-VM Workers that Agent Substrate ships both report a capacity of 1000 Actors.

### Actor placement

Agent Substrate checks two things before it places an Actor on a Worker:
* Whether the Worker has a free Actor slot. This check always runs.
* Whether the Worker has enough spare CPU and memory. This check runs only when the ActorTemplate states how much CPU and memory one Actor needs.

Every ActorTemplate that the kagent controller compiles states a CPU and memory limit for one Actor, so both checks run for a Session's Actor. The `config.runtimeDefaults.cpu` and `config.runtimeDefaults.memory` Helm values set this limit for every agent, and default to `1` CPU and `1Gi` of memory.

The CPU and memory check compares that limit against the capacity that the Worker reports. A WorkerPool sets no resource limits on its Workers by default, and a Worker with no limits reports its whole node as its own capacity. The check then ignores every other pod on the node, so node pressure surfaces as eviction rather than as a placement failure. To make the check reflect the Worker, [set resource limits on Workers]({{< link path="operations/tune-agent-substrate#set-resource-requests-on-workers" >}}).

Agent Substrate first narrows the pool to the Workers that can take the Actor. It then picks two of those Workers at random and places the Actor on the less loaded of the two. A Worker's load is the higher of two shares: the share of its Actor slots that are filled, and the share of its CPU and memory that is committed. Sampling two Workers keeps the decision cheap, and it avoids the pile-up that follows when every placement aims at the Worker that looks emptiest.

<!-- REVIEW (runtime reviewer): a diagram of pods, Workers, pools, and Actor placement, with the sandbox boundaries marked, is wanted in this section. -->

## Atespaces

An **atespace** is the isolation boundary that an Actor belongs to, and the first half of its identity. Agent Substrate addresses an Actor by its atespace and its name together, so the same Actor name can exist in two atespaces without colliding.

An atespace resembles a Kubernetes namespace, but it is not one. Agent Substrate defines it as its own cluster-wide resource. No namespace contains it, a namespace's RBAC does not reach it, and deleting a Kubernetes namespace does not delete the atespace that kagent named after it.

Actors and Workers sit differently against this boundary. An Actor belongs to exactly one atespace for its entire life, whereas a Worker belongs to none. A Worker hosts whichever Actors Agent Substrate places on it, and those Actors can come from different atespaces.

kagent names each atespace after the Kubernetes namespace of the Agent that owns the Actor, and creates the atespace the first time a Session in that namespace needs an Actor. The Actor's own name is `session-` followed by the Session's identifier. A Session on an Agent in the `kagent` namespace therefore runs on an Actor in the `kagent` atespace. Both halves of this identity appear in the address that traffic uses to reach the Actor, which [Sandboxing]({{< link path="about/substrate-runtime/sandboxing#how-traffic-reaches-a-sandboxed-actor" >}}) covers.

## Sandboxing

Because an Actor often runs a model-directed agent that calls tools and executes commands, Agent Substrate runs each Actor in an isolated sandbox rather than a plain container. A WorkerPool's `sandboxClass` field selects the sandbox technology for its Workers: [gVisor](https://gvisor.dev), or a micro-VM that runs the workload under [Cloud Hypervisor](https://www.cloudhypervisor.org) with a [Kata Containers](https://katacontainers.io) kernel and root image. Both technologies isolate an Actor from its Worker's host kernel, and both support suspend and resume operations.

kagent compiles each ActorTemplate to the sandbox class of the WorkerPool that hosts it, so a default installation runs kagent agents in a {{< gloss "gVisor" >}}gVisor{{< /gloss >}} sandbox, and a pool on the `microvm` class runs them under the micro-VM. A WorkerPool that backs kagent Harnesses must run the Worker build that matches its class. For what each class isolates, see [Sandboxing]({{< link path="about/substrate-runtime/sandboxing" >}}).

## Suspend, snapshot, and resume

An Actor spends most of its time idle, waiting on a person or a large language model (LLM) to respond. Agent Substrate suspends an idle Actor, reclaims the capacity that it held, and resumes the Actor when traffic arrives. A WorkerPool can therefore run far more Actors than it has room for at any one moment.

The following diagram traces an Actor through one suspend-and-resume cycle, and shows the second path that opens up once the resulting snapshot is tagged.
</br></br>

{{< reuse "kagent-docs/snippets/snapshot-cycle-diagram.md" >}}

A **WorkerPool** keeps **Workers** running and ready, and a Worker hosts the **Actor** alongside other Actors while its Session is active. Suspending this Actor writes its durable data to an immutable **ActorSnapshot** and releases the capacity that it held on its Worker.

The diagram forks at this snapshot, because a snapshot serves two purposes.

- **Resume** restores the same Actor onto **any Worker in the pool with room**, which is not necessarily the Worker that it ran on before. Agent Substrate starts new containers for the Actor and restores the durable data from the snapshot. The conversation continues from where it left off. Every idle agent takes this path.
- A **{{< gloss "Tag" >}}Tag{{< /gloss >}}** pins this snapshot so that Agent Substrate does not collect it. To fork a Session from a checkpoint, kagent seeds a **New Actor** from that tag. Resuming an existing Actor never goes through a tag.

A tag gives a snapshot a stable name, so callers do not track Agent Substrate's internal snapshot identity. The tag names one snapshot permanently, and only its visibility scope can change afterward. A tag is also a retention pin, so Agent Substrate does not delete a snapshot while a tag still names it.

For example, an agent partway through a long incident investigation reaches a state worth keeping. Creating a [checkpoint]({{< link path="about/substrate-runtime/suspend-and-resume#checkpoints" >}}) tags the snapshot that the agent most recently suspended to, which holds that one snapshot in place while the agent carries on and writes newer ones. Without the tag, Agent Substrate collects this snapshot once a newer one supersedes it.

<!-- REVIEW (performance reviewer): an earlier draft stated a 100 ms target at the ninety-fifth percentile, measured from traffic arrival to the moment the Actor can receive it. Removed pending confirmation of the approved measurement and whether it applies to kagent. Do not restore it as a latency guarantee. -->

<!-- REVIEW (runtime/storage reviewers): expand this section with pause versus suspend, golden snapshots, where snapshots are stored, and what resume restores, and extend the snapshot-cycle diagram to show those distinctions. Do not equate a filesystem snapshot with full execution-state restoration without confirmation. -->
Loading
Loading