diff --git a/plugins/agent-browser/.agents/skills/agent-browser/SKILL.md b/plugins/agent-browser/.agents/skills/agent-browser/SKILL.md index 67e8a49a..da0e6dcb 100644 --- a/plugins/agent-browser/.agents/skills/agent-browser/SKILL.md +++ b/plugins/agent-browser/.agents/skills/agent-browser/SKILL.md @@ -16,11 +16,12 @@ Install: `npm i -g agent-browser && agent-browser install` This file is a discovery stub, not the usage guide. Before running any `agent-browser` command, load the actual workflow content from the CLI: ```bash -agent-browser skills get core # start here — workflows, common patterns, troubleshooting -agent-browser skills get core --full # include full command reference and templates +agent-browser skills get core # start here: the loop, waits, common tasks, troubleshooting ``` -The CLI serves skill content that always matches the installed version, so instructions never go stale. The content in this stub cannot change between releases, which is why it just points at `skills get core`. +The core guide ends with a table of references to load for detailed commands and flags. Use `agent-browser skills get core --full` only when you need every reference at once; it is several times larger. + +The CLI serves skill content that matches the installed version. This stub cannot change between releases, which is why it only points at `skills get core`. ## Specialized skills @@ -31,6 +32,7 @@ agent-browser skills get electron # Electron desktop apps (VS Code, Sla agent-browser skills get slack # Slack workspace automation agent-browser skills get dogfood # Exploratory testing / QA / bug hunts agent-browser skills get derive-client # Record a HAR, derive a standalone API client for a site +agent-browser skills get webmcp-gen # Build and validate WebMCP tools for a page agent-browser skills get vercel-sandbox # agent-browser inside Vercel Sandbox microVMs agent-browser skills get protected-vercel-deployments # Access protected Vercel deployments agent-browser skills get agentcore # AWS Bedrock AgentCore cloud browsers @@ -46,7 +48,3 @@ Run `agent-browser skills list` to see everything available on the installed ver - Accessibility-tree snapshots with element refs for reliable interaction - Sessions, authentication vault, state persistence, video recording - Specialized skills for Electron apps, Slack, exploratory testing, cloud providers - -## Observability Dashboard - -The dashboard runs independently of browser sessions on port 4848 and can also be opened through a proxied or forwarded URL such as `https://dashboard.agent-browser.localhost`. Agents should stay on the dashboard origin: session tabs, status, and stream traffic are proxied internally, so session ports do not need to be exposed. diff --git a/plugins/agent-browser/.agents/skills/dogfood/SKILL.md b/plugins/agent-browser/.agents/skills/dogfood/SKILL.md index dcd7d4d0..ac81a140 100644 --- a/plugins/agent-browser/.agents/skills/dogfood/SKILL.md +++ b/plugins/agent-browser/.agents/skills/dogfood/SKILL.md @@ -51,7 +51,7 @@ Start a named session: ```bash agent-browser --session {SESSION} open {TARGET_URL} -agent-browser --session {SESSION} wait --load networkidle +agent-browser --session {SESSION} wait --load domcontentloaded ``` ### 2. Authenticate @@ -64,7 +64,11 @@ agent-browser --session {SESSION} snapshot -i agent-browser --session {SESSION} fill @e1 "{EMAIL}" agent-browser --session {SESSION} fill @e2 "{PASSWORD}" agent-browser --session {SESSION} click @e3 -agent-browser --session {SESSION} wait --load networkidle +# Replace this with the target app's post-login URL, text, or JS condition: +agent-browser --session {SESSION} wait --url "{POST_LOGIN_URL_PATTERN}" +# Or: +# agent-browser --session {SESSION} wait --text "{POST_LOGIN_TEXT}" +# agent-browser --session {SESSION} wait --fn "{POST_LOGIN_CONDITION}" ``` For OTP/email codes: ask the user, wait for their response, then enter the code. diff --git a/plugins/agent-browser/.agents/skills/slack/SKILL.md b/plugins/agent-browser/.agents/skills/slack/SKILL.md index f64eeb48..91e2bca8 100644 --- a/plugins/agent-browser/.agents/skills/slack/SKILL.md +++ b/plugins/agent-browser/.agents/skills/slack/SKILL.md @@ -83,7 +83,9 @@ agent-browser snapshot -i # Look for channel name in the list (e.g., "engineering", "product-design") # Click on the channel treeitem ref agent-browser click @e94 # Example: engineering channel ref -agent-browser wait --load networkidle +# Replace `engineering` with the clicked channel's name so the old channel +# header cannot satisfy the wait while the new channel is still rendering. +agent-browser wait --fn "document.querySelector('[data-qa=\"channel_header\"]')?.innerText.includes('engineering')" agent-browser screenshot channel.png ``` @@ -95,7 +97,8 @@ agent-browser snapshot -i agent-browser click @e5 # Search button (typical ref) agent-browser fill @e_search "keyword" agent-browser press Enter -agent-browser wait --load networkidle +# Match the submitted query so a repeated search cannot reuse the old route. +agent-browser wait --fn "location.pathname.includes('/search') && new URL(location.href).searchParams.get('q') === 'keyword'" agent-browser screenshot search-results.png ``` diff --git a/plugins/agent-browser/.agents/skills/slack/references/slack-tasks.md b/plugins/agent-browser/.agents/skills/slack/references/slack-tasks.md index c9161731..a520aff7 100644 --- a/plugins/agent-browser/.agents/skills/slack/references/slack-tasks.md +++ b/plugins/agent-browser/.agents/skills/slack/references/slack-tasks.md @@ -101,7 +101,8 @@ Find all messages/threads mentioning specific terms. # Identify search input ref from snapshot agent-browser fill @e_search_input "your keyword" agent-browser press Enter - agent-browser wait --load networkidle + # Match the submitted query so a repeated search cannot reuse the old route. + agent-browser wait --fn "location.pathname.includes('/search') && new URL(location.href).searchParams.get('q') === 'your keyword'" ``` 3. **Capture results** @@ -141,7 +142,8 @@ Watch a channel and capture new messages/engagement. agent-browser snapshot -i # Find channel ref from sidebar agent-browser click @e_channel_ref - agent-browser wait --load networkidle + # Replace `engineering` with the clicked channel's name, or wait for its known URL: + agent-browser wait --fn "document.querySelector('[data-qa=\"channel_header\"]')?.innerText.includes('engineering')" ``` 2. **Check channel info** @@ -342,7 +344,6 @@ If you can't find an element: 4. **Wait for page to load** ```bash - agent-browser wait --load networkidle - agent-browser wait 1000 + agent-browser wait --fn "document.querySelector('[role=main]')" agent-browser snapshot -i ``` diff --git a/plugins/agent-browser/agent/skills/agent-browser/SKILL.md b/plugins/agent-browser/agent/skills/agent-browser/SKILL.md index 603308f0..02001fba 100644 --- a/plugins/agent-browser/agent/skills/agent-browser/SKILL.md +++ b/plugins/agent-browser/agent/skills/agent-browser/SKILL.md @@ -25,11 +25,12 @@ Install: `npm i -g agent-browser && agent-browser install` This file is a discovery stub, not the usage guide. Before running any `agent-browser` command, load the actual workflow content from the CLI: ```bash -agent-browser skills get core # start here — workflows, common patterns, troubleshooting -agent-browser skills get core --full # include full command reference and templates +agent-browser skills get core # start here: the loop, waits, common tasks, troubleshooting ``` -The CLI serves skill content that always matches the installed version, so instructions never go stale. The content in this stub cannot change between releases, which is why it just points at `skills get core`. +The core guide ends with a table of references to load for detailed commands and flags. Use `agent-browser skills get core --full` only when you need every reference at once; it is several times larger. + +The CLI serves skill content that matches the installed version. This stub cannot change between releases, which is why it only points at `skills get core`. ## Specialized skills @@ -40,6 +41,7 @@ agent-browser skills get electron # Electron desktop apps (VS Code, Sla agent-browser skills get slack # Slack workspace automation agent-browser skills get dogfood # Exploratory testing / QA / bug hunts agent-browser skills get derive-client # Record a HAR, derive a standalone API client for a site +agent-browser skills get webmcp-gen # Build and validate WebMCP tools for a page agent-browser skills get vercel-sandbox # agent-browser inside Vercel Sandbox microVMs agent-browser skills get protected-vercel-deployments # Access protected Vercel deployments agent-browser skills get agentcore # AWS Bedrock AgentCore cloud browsers @@ -55,7 +57,3 @@ Run `agent-browser skills list` to see everything available on the installed ver - Accessibility-tree snapshots with element refs for reliable interaction - Sessions, authentication vault, state persistence, video recording - Specialized skills for Electron apps, Slack, exploratory testing, cloud providers - -## Observability Dashboard - -The dashboard runs independently of browser sessions on port 4848 and can also be opened through a proxied or forwarded URL such as `https://dashboard.agent-browser.localhost`. Agents should stay on the dashboard origin: session tabs, status, and stream traffic are proxied internally, so session ports do not need to be exposed. diff --git a/plugins/agent-browser/skills-lock.json b/plugins/agent-browser/skills-lock.json index 06a1728f..a841e0a5 100644 --- a/plugins/agent-browser/skills-lock.json +++ b/plugins/agent-browser/skills-lock.json @@ -5,22 +5,25 @@ "source": "vercel-labs/agent-browser", "sourceType": "github", "skillPath": "skills/agent-browser/SKILL.md", - "computedHash": "79add54dd6de82a262a88c94744d7decde000159c1d7f67c78eb9d4e523d54bb" + "computedHash": "60a9cbe5af176150582c019e16e5d3703a41bb02d88438e881349876120d7944" }, "dogfood": { "source": "vercel-labs/agent-browser", "sourceType": "github", - "computedHash": "055e73cddeec68e7eb75eab0c3f8490415384f72e98ac05405821427ed0e74b1" + "skillPath": "skill-data/dogfood/SKILL.md", + "computedHash": "b7770c9814c6cdfd439ca312bfa238494817d722866b636a2e2cb47cedfd4746" }, "electron": { "source": "vercel-labs/agent-browser", "sourceType": "github", + "skillPath": "skill-data/electron/SKILL.md", "computedHash": "9c6ee7f028f409be8ba695dd9a6493c7de9a3802ee538ee905baaceea419aba9" }, "slack": { "source": "vercel-labs/agent-browser", "sourceType": "github", - "computedHash": "187f9b0c41e418377a16ccb535e5aa04d55c482dae9a82da41755c6f5ee268f6" + "skillPath": "skill-data/slack/SKILL.md", + "computedHash": "d977e04a42876d2c95bbe88b15346e9b45f4bdf8e95179c489e54f86870f8a71" } } } diff --git a/plugins/ai-sdk/skills-lock.json b/plugins/ai-sdk/skills-lock.json index b745916c..1e17675f 100644 --- a/plugins/ai-sdk/skills-lock.json +++ b/plugins/ai-sdk/skills-lock.json @@ -5,7 +5,7 @@ "source": "vercel/ai", "sourceType": "github", "skillPath": "skills/use-ai-sdk/SKILL.md", - "computedHash": "aee5bf7ecb066b54eae76799610fa8a9be9ff1ca9a000ea92d17c14ec862938b" + "computedHash": "d6910e7d8bb527d2823ba58b6225b304ae649c135fc7e43ec2d2152251c218db" } } } diff --git a/plugins/ast-grep/.agents/skills/ast-grep/SKILL.md b/plugins/ast-grep/.agents/skills/ast-grep/SKILL.md index 1c6e3afd..0e1dddc6 100644 --- a/plugins/ast-grep/.agents/skills/ast-grep/SKILL.md +++ b/plugins/ast-grep/.agents/skills/ast-grep/SKILL.md @@ -354,3 +354,12 @@ Contains detailed documentation for ast-grep rule syntax: - `rule_reference.md`: Comprehensive ast-grep rule documentation covering atomic rules, relational rules, composite rules, and metavariables Load these references when detailed rule syntax information is needed. + +### Online documentation +When a rule still fails after `--debug-query` and the rule reference, read +the official docs: +- `https://ast-grep.github.io/llms.txt` indexes every docs page; fetch the + one page you need (each is linked as a `.md` file, such as + `https://ast-grep.github.io/guide/rule-config/relational-rule.md`). +- `https://ast-grep.github.io/llms-full.txt` holds all the docs in one + file of about 850 KB. Download it and search it; never read it whole. diff --git a/plugins/ast-grep/agent/skills/ast-grep/SKILL.md b/plugins/ast-grep/agent/skills/ast-grep/SKILL.md index 3e3c770f..8293c054 100644 --- a/plugins/ast-grep/agent/skills/ast-grep/SKILL.md +++ b/plugins/ast-grep/agent/skills/ast-grep/SKILL.md @@ -358,3 +358,12 @@ Contains detailed documentation for ast-grep rule syntax: - `rule_reference.md`: Comprehensive ast-grep rule documentation covering atomic rules, relational rules, composite rules, and metavariables Load these references when detailed rule syntax information is needed. + +### Online documentation +When a rule still fails after `--debug-query` and the rule reference, read +the official docs: +- `https://ast-grep.github.io/llms.txt` indexes every docs page; fetch the + one page you need (each is linked as a `.md` file, such as + `https://ast-grep.github.io/guide/rule-config/relational-rule.md`). +- `https://ast-grep.github.io/llms-full.txt` holds all the docs in one + file of about 850 KB. Download it and search it; never read it whole. diff --git a/plugins/ast-grep/skills-lock.json b/plugins/ast-grep/skills-lock.json index 65d04023..24eb3f3b 100644 --- a/plugins/ast-grep/skills-lock.json +++ b/plugins/ast-grep/skills-lock.json @@ -5,7 +5,7 @@ "source": "ast-grep/agent-skill", "sourceType": "github", "skillPath": "ast-grep/skills/ast-grep/SKILL.md", - "computedHash": "472c7cd092f9fad72e0ad0c58b5f6af627844730ecc420c009e25a835a5d6e81" + "computedHash": "55ab409e6afd238ebab8f29d9a5230d513b6d0102e0eb9f71c554a95d1344bd6" } } } diff --git a/plugins/better-auth/.agents/skills/create-auth/SKILL.md b/plugins/better-auth/.agents/skills/create-auth/SKILL.md new file mode 100644 index 00000000..f57bf6b2 --- /dev/null +++ b/plugins/better-auth/.agents/skills/create-auth/SKILL.md @@ -0,0 +1,390 @@ +--- +name: create-auth +description: Scaffold and implement authentication in TypeScript/JavaScript apps using Better Auth. Detect frameworks, configure database adapters, set up route handlers, add OAuth providers, and create auth UI pages. Use when users want to add login, sign-up, or authentication to a new or existing project with Better Auth. +--- + +# Create Auth Skill + +Guide for adding authentication to TypeScript/JavaScript applications using Better Auth. + +**For code examples and syntax, see [better-auth.com/docs](https://better-auth.com/docs).** + +--- + +## Phase 1: Planning (REQUIRED before implementation) + +Before writing any code, gather requirements by scanning the project and asking the user structured questions. This ensures the implementation matches their needs. + +### Step 1: Scan the project + +Analyze the codebase to auto-detect: +- **Framework** — Look for `next.config`, `svelte.config`, `nuxt.config`, `astro.config`, `vite.config`, or Express/Hono entry files. +- **Database/ORM** — Look for `prisma/schema.prisma`, `drizzle.config.ts`, `package.json` deps (`pg`, `postgres`, `@neondatabase/serverless`, `mysql2`, `better-sqlite3`, `mongoose`, `mongodb`). If `drizzle.config.ts` exists, read its `dialect` field to determine the DB type (e.g., `"postgresql"` → Drizzle + Postgres). Also check which Drizzle driver is installed (`drizzle-orm/node-postgres` → `pg`, `drizzle-orm/postgres-js` → `postgres`, `drizzle-orm/neon-http` → Neon). +- **Existing auth** — Look for existing auth libraries (`next-auth`, `lucia`, `clerk`, `supabase/auth`, `firebase/auth`) in `package.json` or imports. +- **Package manager** — Check for `pnpm-lock.yaml`, `yarn.lock`, `bun.lockb`, or `package-lock.json`. + +Use what you find to pre-fill defaults and skip questions you can already answer. + +### Step 2: Ask planning questions + +Use the `AskQuestion` tool to ask the user **all applicable questions in a single call**. Skip any question you already have a confident answer for from the scan. Group them under a title like "Auth Setup Planning". + +**Questions to ask:** + +1. **Project type** (skip if detected) + - Prompt: "What type of project is this?" + - Options: New project from scratch | Adding auth to existing project | Migrating from another auth library + +2. **Framework** (skip if detected) + - Prompt: "Which framework are you using?" + - Options: Next.js (App Router) | Next.js (Pages Router) | SvelteKit | Nuxt | Astro | Express | Hono | SolidStart | Other + +3. **Database & ORM** (skip if detected) + - Prompt: "Which database setup will you use?" + - Options: PostgreSQL (Prisma) | PostgreSQL (Drizzle) | PostgreSQL (pg driver) | MySQL (Prisma) | MySQL (Drizzle) | MySQL (mysql2 driver) | SQLite (Prisma) | SQLite (Drizzle) | SQLite (better-sqlite3 driver) | MongoDB (Mongoose) | MongoDB (native driver) + +4. **Authentication methods** (always ask, allow multiple) + - Prompt: "Which sign-in methods do you need?" + - Options: Email & password | Social OAuth (Google, GitHub, etc.) | Magic link (passwordless email) | Passkey (WebAuthn) | Phone number + - `allow_multiple: true` + +5. **Social providers** (only if they selected Social OAuth above — ask in a follow-up call) + - Prompt: "Which social providers do you need?" + - Options: Google | GitHub | Apple | Microsoft | Discord | Twitter/X + - `allow_multiple: true` + +6. **Email verification** (only if Email & password was selected above — ask in a follow-up call) + - Prompt: "Do you want to require email verification?" + - Options: Yes | No + +7. **Email provider** (only if email verification is Yes, or if Password reset is selected in features — ask in a follow-up call) + - Prompt: "How do you want to send emails?" + - Options: Resend | Mock it for now (console.log) + +8. **Features & plugins** (always ask, allow multiple) + - Prompt: "Which additional features do you need?" + - Options: Two-factor authentication (2FA) | Organizations / teams | Admin dashboard | API bearer tokens | Password reset | None of these + - `allow_multiple: true` + +9. **Auth pages** (always ask, allow multiple — pre-select based on earlier answers) + - Prompt: "Which auth pages do you need?" + - Options vary based on previous answers: + - Always available: Sign in | Sign up + - If Email & password selected: Forgot password | Reset password + - If email verification enabled: Email verification + - `allow_multiple: true` + +10. **Auth UI style** (always ask) + - Prompt: "What style do you want for the auth pages? Pick one or describe your own." + - Options: Minimal & clean | Centered card with background | Split layout (form + hero image) | Floating / glassmorphism | Other (I'll describe) + +### Step 3: Summarize the plan + +After collecting answers, present a concise implementation plan as a markdown checklist. Example: + +``` +## Auth Implementation Plan + +- **Framework:** Next.js (App Router) +- **Database:** PostgreSQL via Prisma +- **Auth methods:** Email/password, Google OAuth, GitHub OAuth +- **Plugins:** 2FA, Organizations, Email verification +- **UI:** Custom forms + +### Steps +1. Install `better-auth` (the CLI ships as the `auth` package and is run via `npx auth@latest`, no install needed) +2. Create `lib/auth.ts` with server config +3. Create `lib/auth-client.ts` with React client +4. Set up route handler at `app/api/auth/[...all]/route.ts` +5. Configure Prisma adapter and generate schema +6. Add Google & GitHub OAuth providers +7. Enable `twoFactor` and `organization` plugins +8. Set up email verification handler +9. Run migrations +10. Create sign-in / sign-up pages +``` + +Ask the user to confirm the plan before proceeding to Phase 2. + +--- + +## Phase 2: Implementation + +Only proceed here after the user confirms the plan from Phase 1. + +Follow the decision tree below, guided by the answers collected above. + +``` +Is this a new/empty project? +├─ YES → New project setup +│ 1. Install better-auth (+ scoped packages per plan) +│ 2. Create auth.ts with all planned config +│ 3. Create auth-client.ts with framework client +│ 4. Set up route handler +│ 5. Set up environment variables +│ 6. Run CLI migrate/generate +│ 7. Add plugins from plan +│ 8. Create auth UI pages +│ +├─ MIGRATING → Migration from existing auth +│ 1. Audit current auth for gaps +│ 2. Plan incremental migration +│ 3. Install better-auth alongside existing auth +│ 4. Migrate routes, then session logic, then UI +│ 5. Remove old auth library +│ 6. See migration guides in docs +│ +└─ ADDING → Add auth to existing project + 1. Analyze project structure + 2. Install better-auth + 3. Create auth config matching plan + 4. Add route handler + 5. Run schema migrations + 6. Integrate into existing pages + 7. Add planned plugins and features +``` + +At the end of implementation, guide users thoroughly on remaining next steps (e.g., setting up OAuth app credentials, deploying env vars, testing flows). + +--- + +## Installation + +**Core:** `npm install better-auth` + +**Scoped packages (as needed):** +| Package | Use case | +|---------|----------| +| `@better-auth/passkey` | WebAuthn/Passkey auth | +| `@better-auth/sso` | SAML/OIDC enterprise SSO | +| `@better-auth/stripe` | Stripe payments | +| `@better-auth/scim` | SCIM user provisioning | +| `@better-auth/expo` | React Native/Expo | + +--- + +## Environment Variables + +```env +BETTER_AUTH_SECRET=<32+ chars, generate with: openssl rand -base64 32> +BETTER_AUTH_URL=http://localhost:3000 +DATABASE_URL= +``` + +Add OAuth secrets as needed: `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`, `GOOGLE_CLIENT_ID`, etc. + +--- + +## Server Config (auth.ts) + +**Location:** `lib/auth.ts` or `src/lib/auth.ts` + +**Minimal config needs:** +- `database` - Connection or adapter +- `emailAndPassword: { enabled: true }` - For email/password auth + +**Standard config adds:** +- `socialProviders` - OAuth providers (google, github, etc.) +- `emailVerification.sendVerificationEmail` - Email verification handler +- `emailAndPassword.sendResetPassword` - Password reset handler + +**Full config adds:** +- `plugins` - Array of feature plugins +- `session` - Expiry, cookie cache settings +- `account.accountLinking` - Multi-provider linking +- `rateLimit` - Rate limiting config + +**Export types:** `export type Session = typeof auth.$Infer.Session` + +--- + +## Client Config (auth-client.ts) + +**Import by framework:** +| Framework | Import | +|-----------|--------| +| React/Next.js | `better-auth/react` | +| Vue | `better-auth/vue` | +| Svelte | `better-auth/svelte` | +| Solid | `better-auth/solid` | +| Vanilla JS | `better-auth/client` | + +**Client plugins** go in `createAuthClient({ plugins: [...] })`. + +**Common exports:** `signIn`, `signUp`, `signOut`, `useSession`, `getSession` + +--- + +## Route Handler Setup + +| Framework | File | Handler | +|-----------|------|---------| +| Next.js App Router | `app/api/auth/[...all]/route.ts` | `toNextJsHandler(auth)` → export `{ GET, POST }` | +| Next.js Pages | `pages/api/auth/[...all].ts` | `toNextJsHandler(auth)` → default export | +| Express | Any file | `app.all("/api/auth/*", toNodeHandler(auth))` | +| SvelteKit | `src/hooks.server.ts` | `svelteKitHandler(auth)` | +| SolidStart | Route file | `solidStartHandler(auth)` | +| Hono | Route file | `auth.handler(c.req.raw)` | + +**Next.js Server Components:** Add `nextCookies()` plugin to auth config. + +--- + +## Database Migrations + +| Adapter | Command | +|---------|---------| +| Built-in Kysely | `npx auth@latest migrate` (applies directly) | +| Prisma | `npx auth@latest generate --output prisma/schema.prisma` then `npx prisma migrate dev` | +| Drizzle (dev) | `npx auth@latest generate --output src/db/auth-schema.ts` then `npx drizzle-kit push` | +| Drizzle (prod) | `npx auth@latest generate --output src/db/auth-schema.ts` then `npx drizzle-kit generate` then `npx drizzle-kit migrate` | + +> **Note:** `drizzle-kit push` skips migration files and is only safe for development. Use `drizzle-kit generate` + `drizzle-kit migrate` in production. + +**Re-run after adding plugins.** + +--- + +## Database Adapters + +| Database | Setup | +|----------|-------| +| SQLite | Pass `better-sqlite3` or `bun:sqlite` instance directly | +| PostgreSQL | Pass `pg.Pool` instance directly | +| MySQL | Pass `mysql2` pool directly | +| Prisma | `prismaAdapter(prisma, { provider: "postgresql" })` from `better-auth/adapters/prisma` | +| Drizzle (pg) | `drizzleAdapter(db, { provider: "pg" })` from `better-auth/adapters/drizzle` | +| Drizzle (mysql) | `drizzleAdapter(db, { provider: "mysql" })` from `better-auth/adapters/drizzle` | +| Drizzle (sqlite) | `drizzleAdapter(db, { provider: "sqlite" })` from `better-auth/adapters/drizzle` | +| MongoDB | `mongodbAdapter(db)` from `better-auth/adapters/mongodb` | + +### Drizzle + PostgreSQL Setup + +Before using `drizzleAdapter`, initialize the `db` instance: + +```ts +// Option 1: node-postgres (pg) +import { drizzle } from "drizzle-orm/node-postgres" +import { Pool } from "pg" +import * as schema from "./auth-schema" + +const pool = new Pool({ connectionString: process.env.DATABASE_URL }) +export const db = drizzle(pool, { schema }) +``` + +```ts +// Option 2: postgres.js +import { drizzle } from "drizzle-orm/postgres-js" +import postgres from "postgres" +import * as schema from "./auth-schema" + +const client = postgres(process.env.DATABASE_URL!) +export const db = drizzle(client, { schema }) +``` + +```ts +// Option 3: Neon serverless +import { drizzle } from "drizzle-orm/neon-http" +import { neon } from "@neondatabase/serverless" +import * as schema from "./auth-schema" + +const sql = neon(process.env.DATABASE_URL!) +export const db = drizzle(sql, { schema }) +``` + +Then pass to Better Auth: + +```ts +import { betterAuth } from "better-auth" +import { drizzleAdapter } from "better-auth/adapters/drizzle" +import { db } from "./db" + +export const auth = betterAuth({ + database: drizzleAdapter(db, { provider: "pg" }), + // ... +}) +``` + +### Drizzle Config (`drizzle.config.ts`) + +Required for `drizzle-kit` commands to find your schema: + +```ts +import { defineConfig } from "drizzle-kit" + +export default defineConfig({ + schema: "./src/db/auth-schema.ts", + out: "./drizzle", + dialect: "postgresql", + dbCredentials: { + url: process.env.DATABASE_URL!, + }, +}) +``` + +--- + +## Common Plugins + +| Plugin | Server Import | Client Import | Purpose | +|--------|---------------|---------------|---------| +| `twoFactor` | `better-auth/plugins` | `twoFactorClient` | 2FA with TOTP/OTP | +| `organization` | `better-auth/plugins` | `organizationClient` | Teams/orgs | +| `admin` | `better-auth/plugins` | `adminClient` | User management | +| `bearer` | `better-auth/plugins` | - | API token auth | +| `openAPI` | `better-auth/plugins` | - | API docs | +| `passkey` | `@better-auth/passkey` | `passkeyClient` | WebAuthn | +| `sso` | `@better-auth/sso` | - | Enterprise SSO | + +**Plugin pattern:** Server plugin + client plugin + run migrations. + +--- + +## Auth UI Implementation + +**Sign in flow:** +1. `signIn.email({ email, password })` or `signIn.social({ provider, callbackURL })` +2. Handle `error` in response +3. Redirect on success + +**Session check (client):** `useSession()` hook returns `{ data: session, isPending }` + +**Session check (server):** `auth.api.getSession({ headers: await headers() })` + +**Protected routes:** Check session, redirect to `/sign-in` if null. + +--- + +## Security Checklist + +- [ ] `BETTER_AUTH_SECRET` set (32+ chars) +- [ ] `advanced.useSecureCookies: true` in production +- [ ] `trustedOrigins` configured +- [ ] Rate limits enabled +- [ ] Email verification enabled +- [ ] Password reset implemented +- [ ] 2FA for sensitive apps +- [ ] CSRF protection NOT disabled +- [ ] `account.accountLinking` reviewed + +--- + +## Troubleshooting + +| Issue | Fix | +|-------|-----| +| "Secret not set" | Add `BETTER_AUTH_SECRET` env var | +| "Invalid Origin" | Add domain to `trustedOrigins` | +| Cookies not setting | Check `baseURL` matches domain; enable secure cookies in prod | +| OAuth callback errors | Verify redirect URIs in provider dashboard | +| Type errors after adding plugin | Re-run CLI generate/migrate | + +--- + +## Resources + +- [Docs](https://better-auth.com/docs) +- [Examples](https://github.com/better-auth/examples) +- [Plugins](https://better-auth.com/docs/concepts/plugins) +- [CLI](https://better-auth.com/docs/concepts/cli) +- [Migration Guides](https://better-auth.com/docs/guides) diff --git a/plugins/better-auth/.claude/skills/create-auth b/plugins/better-auth/.claude/skills/create-auth new file mode 120000 index 00000000..d6e37db5 --- /dev/null +++ b/plugins/better-auth/.claude/skills/create-auth @@ -0,0 +1 @@ +../../.agents/skills/create-auth \ No newline at end of file diff --git a/plugins/better-auth/agent/skills/create-auth/SKILL.md b/plugins/better-auth/agent/skills/create-auth/SKILL.md new file mode 100644 index 00000000..0754d591 --- /dev/null +++ b/plugins/better-auth/agent/skills/create-auth/SKILL.md @@ -0,0 +1,393 @@ +--- +name: create-auth +description: Scaffold and implement authentication in TypeScript/JavaScript apps + using Better Auth. Detect frameworks, configure database adapters, set up + route handlers, add OAuth providers, and create auth UI pages. Use when users + want to add login, sign-up, or authentication to a new or existing project + with Better Auth. +--- +# Create Auth Skill + +Guide for adding authentication to TypeScript/JavaScript applications using Better Auth. + +**For code examples and syntax, see [better-auth.com/docs](https://better-auth.com/docs).** + +--- + +## Phase 1: Planning (REQUIRED before implementation) + +Before writing any code, gather requirements by scanning the project and asking the user structured questions. This ensures the implementation matches their needs. + +### Step 1: Scan the project + +Analyze the codebase to auto-detect: +- **Framework** — Look for `next.config`, `svelte.config`, `nuxt.config`, `astro.config`, `vite.config`, or Express/Hono entry files. +- **Database/ORM** — Look for `prisma/schema.prisma`, `drizzle.config.ts`, `package.json` deps (`pg`, `postgres`, `@neondatabase/serverless`, `mysql2`, `better-sqlite3`, `mongoose`, `mongodb`). If `drizzle.config.ts` exists, read its `dialect` field to determine the DB type (e.g., `"postgresql"` → Drizzle + Postgres). Also check which Drizzle driver is installed (`drizzle-orm/node-postgres` → `pg`, `drizzle-orm/postgres-js` → `postgres`, `drizzle-orm/neon-http` → Neon). +- **Existing auth** — Look for existing auth libraries (`next-auth`, `lucia`, `clerk`, `supabase/auth`, `firebase/auth`) in `package.json` or imports. +- **Package manager** — Check for `pnpm-lock.yaml`, `yarn.lock`, `bun.lockb`, or `package-lock.json`. + +Use what you find to pre-fill defaults and skip questions you can already answer. + +### Step 2: Ask planning questions + +Use the `AskQuestion` tool to ask the user **all applicable questions in a single call**. Skip any question you already have a confident answer for from the scan. Group them under a title like "Auth Setup Planning". + +**Questions to ask:** + +1. **Project type** (skip if detected) + - Prompt: "What type of project is this?" + - Options: New project from scratch | Adding auth to existing project | Migrating from another auth library + +2. **Framework** (skip if detected) + - Prompt: "Which framework are you using?" + - Options: Next.js (App Router) | Next.js (Pages Router) | SvelteKit | Nuxt | Astro | Express | Hono | SolidStart | Other + +3. **Database & ORM** (skip if detected) + - Prompt: "Which database setup will you use?" + - Options: PostgreSQL (Prisma) | PostgreSQL (Drizzle) | PostgreSQL (pg driver) | MySQL (Prisma) | MySQL (Drizzle) | MySQL (mysql2 driver) | SQLite (Prisma) | SQLite (Drizzle) | SQLite (better-sqlite3 driver) | MongoDB (Mongoose) | MongoDB (native driver) + +4. **Authentication methods** (always ask, allow multiple) + - Prompt: "Which sign-in methods do you need?" + - Options: Email & password | Social OAuth (Google, GitHub, etc.) | Magic link (passwordless email) | Passkey (WebAuthn) | Phone number + - `allow_multiple: true` + +5. **Social providers** (only if they selected Social OAuth above — ask in a follow-up call) + - Prompt: "Which social providers do you need?" + - Options: Google | GitHub | Apple | Microsoft | Discord | Twitter/X + - `allow_multiple: true` + +6. **Email verification** (only if Email & password was selected above — ask in a follow-up call) + - Prompt: "Do you want to require email verification?" + - Options: Yes | No + +7. **Email provider** (only if email verification is Yes, or if Password reset is selected in features — ask in a follow-up call) + - Prompt: "How do you want to send emails?" + - Options: Resend | Mock it for now (console.log) + +8. **Features & plugins** (always ask, allow multiple) + - Prompt: "Which additional features do you need?" + - Options: Two-factor authentication (2FA) | Organizations / teams | Admin dashboard | API bearer tokens | Password reset | None of these + - `allow_multiple: true` + +9. **Auth pages** (always ask, allow multiple — pre-select based on earlier answers) + - Prompt: "Which auth pages do you need?" + - Options vary based on previous answers: + - Always available: Sign in | Sign up + - If Email & password selected: Forgot password | Reset password + - If email verification enabled: Email verification + - `allow_multiple: true` + +10. **Auth UI style** (always ask) + - Prompt: "What style do you want for the auth pages? Pick one or describe your own." + - Options: Minimal & clean | Centered card with background | Split layout (form + hero image) | Floating / glassmorphism | Other (I'll describe) + +### Step 3: Summarize the plan + +After collecting answers, present a concise implementation plan as a markdown checklist. Example: + +``` +## Auth Implementation Plan + +- **Framework:** Next.js (App Router) +- **Database:** PostgreSQL via Prisma +- **Auth methods:** Email/password, Google OAuth, GitHub OAuth +- **Plugins:** 2FA, Organizations, Email verification +- **UI:** Custom forms + +### Steps +1. Install `better-auth` (the CLI ships as the `auth` package and is run via `npx auth@latest`, no install needed) +2. Create `lib/auth.ts` with server config +3. Create `lib/auth-client.ts` with React client +4. Set up route handler at `app/api/auth/[...all]/route.ts` +5. Configure Prisma adapter and generate schema +6. Add Google & GitHub OAuth providers +7. Enable `twoFactor` and `organization` plugins +8. Set up email verification handler +9. Run migrations +10. Create sign-in / sign-up pages +``` + +Ask the user to confirm the plan before proceeding to Phase 2. + +--- + +## Phase 2: Implementation + +Only proceed here after the user confirms the plan from Phase 1. + +Follow the decision tree below, guided by the answers collected above. + +``` +Is this a new/empty project? +├─ YES → New project setup +│ 1. Install better-auth (+ scoped packages per plan) +│ 2. Create auth.ts with all planned config +│ 3. Create auth-client.ts with framework client +│ 4. Set up route handler +│ 5. Set up environment variables +│ 6. Run CLI migrate/generate +│ 7. Add plugins from plan +│ 8. Create auth UI pages +│ +├─ MIGRATING → Migration from existing auth +│ 1. Audit current auth for gaps +│ 2. Plan incremental migration +│ 3. Install better-auth alongside existing auth +│ 4. Migrate routes, then session logic, then UI +│ 5. Remove old auth library +│ 6. See migration guides in docs +│ +└─ ADDING → Add auth to existing project + 1. Analyze project structure + 2. Install better-auth + 3. Create auth config matching plan + 4. Add route handler + 5. Run schema migrations + 6. Integrate into existing pages + 7. Add planned plugins and features +``` + +At the end of implementation, guide users thoroughly on remaining next steps (e.g., setting up OAuth app credentials, deploying env vars, testing flows). + +--- + +## Installation + +**Core:** `npm install better-auth` + +**Scoped packages (as needed):** +| Package | Use case | +|---------|----------| +| `@better-auth/passkey` | WebAuthn/Passkey auth | +| `@better-auth/sso` | SAML/OIDC enterprise SSO | +| `@better-auth/stripe` | Stripe payments | +| `@better-auth/scim` | SCIM user provisioning | +| `@better-auth/expo` | React Native/Expo | + +--- + +## Environment Variables + +```env +BETTER_AUTH_SECRET=<32+ chars, generate with: openssl rand -base64 32> +BETTER_AUTH_URL=http://localhost:3000 +DATABASE_URL= +``` + +Add OAuth secrets as needed: `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`, `GOOGLE_CLIENT_ID`, etc. + +--- + +## Server Config (auth.ts) + +**Location:** `lib/auth.ts` or `src/lib/auth.ts` + +**Minimal config needs:** +- `database` - Connection or adapter +- `emailAndPassword: { enabled: true }` - For email/password auth + +**Standard config adds:** +- `socialProviders` - OAuth providers (google, github, etc.) +- `emailVerification.sendVerificationEmail` - Email verification handler +- `emailAndPassword.sendResetPassword` - Password reset handler + +**Full config adds:** +- `plugins` - Array of feature plugins +- `session` - Expiry, cookie cache settings +- `account.accountLinking` - Multi-provider linking +- `rateLimit` - Rate limiting config + +**Export types:** `export type Session = typeof auth.$Infer.Session` + +--- + +## Client Config (auth-client.ts) + +**Import by framework:** +| Framework | Import | +|-----------|--------| +| React/Next.js | `better-auth/react` | +| Vue | `better-auth/vue` | +| Svelte | `better-auth/svelte` | +| Solid | `better-auth/solid` | +| Vanilla JS | `better-auth/client` | + +**Client plugins** go in `createAuthClient({ plugins: [...] })`. + +**Common exports:** `signIn`, `signUp`, `signOut`, `useSession`, `getSession` + +--- + +## Route Handler Setup + +| Framework | File | Handler | +|-----------|------|---------| +| Next.js App Router | `app/api/auth/[...all]/route.ts` | `toNextJsHandler(auth)` → export `{ GET, POST }` | +| Next.js Pages | `pages/api/auth/[...all].ts` | `toNextJsHandler(auth)` → default export | +| Express | Any file | `app.all("/api/auth/*", toNodeHandler(auth))` | +| SvelteKit | `src/hooks.server.ts` | `svelteKitHandler(auth)` | +| SolidStart | Route file | `solidStartHandler(auth)` | +| Hono | Route file | `auth.handler(c.req.raw)` | + +**Next.js Server Components:** Add `nextCookies()` plugin to auth config. + +--- + +## Database Migrations + +| Adapter | Command | +|---------|---------| +| Built-in Kysely | `npx auth@latest migrate` (applies directly) | +| Prisma | `npx auth@latest generate --output prisma/schema.prisma` then `npx prisma migrate dev` | +| Drizzle (dev) | `npx auth@latest generate --output src/db/auth-schema.ts` then `npx drizzle-kit push` | +| Drizzle (prod) | `npx auth@latest generate --output src/db/auth-schema.ts` then `npx drizzle-kit generate` then `npx drizzle-kit migrate` | + +> **Note:** `drizzle-kit push` skips migration files and is only safe for development. Use `drizzle-kit generate` + `drizzle-kit migrate` in production. + +**Re-run after adding plugins.** + +--- + +## Database Adapters + +| Database | Setup | +|----------|-------| +| SQLite | Pass `better-sqlite3` or `bun:sqlite` instance directly | +| PostgreSQL | Pass `pg.Pool` instance directly | +| MySQL | Pass `mysql2` pool directly | +| Prisma | `prismaAdapter(prisma, { provider: "postgresql" })` from `better-auth/adapters/prisma` | +| Drizzle (pg) | `drizzleAdapter(db, { provider: "pg" })` from `better-auth/adapters/drizzle` | +| Drizzle (mysql) | `drizzleAdapter(db, { provider: "mysql" })` from `better-auth/adapters/drizzle` | +| Drizzle (sqlite) | `drizzleAdapter(db, { provider: "sqlite" })` from `better-auth/adapters/drizzle` | +| MongoDB | `mongodbAdapter(db)` from `better-auth/adapters/mongodb` | + +### Drizzle + PostgreSQL Setup + +Before using `drizzleAdapter`, initialize the `db` instance: + +```ts +// Option 1: node-postgres (pg) +import { drizzle } from "drizzle-orm/node-postgres" +import { Pool } from "pg" +import * as schema from "./auth-schema" + +const pool = new Pool({ connectionString: process.env.DATABASE_URL }) +export const db = drizzle(pool, { schema }) +``` + +```ts +// Option 2: postgres.js +import { drizzle } from "drizzle-orm/postgres-js" +import postgres from "postgres" +import * as schema from "./auth-schema" + +const client = postgres(process.env.DATABASE_URL!) +export const db = drizzle(client, { schema }) +``` + +```ts +// Option 3: Neon serverless +import { drizzle } from "drizzle-orm/neon-http" +import { neon } from "@neondatabase/serverless" +import * as schema from "./auth-schema" + +const sql = neon(process.env.DATABASE_URL!) +export const db = drizzle(sql, { schema }) +``` + +Then pass to Better Auth: + +```ts +import { betterAuth } from "better-auth" +import { drizzleAdapter } from "better-auth/adapters/drizzle" +import { db } from "./db" + +export const auth = betterAuth({ + database: drizzleAdapter(db, { provider: "pg" }), + // ... +}) +``` + +### Drizzle Config (`drizzle.config.ts`) + +Required for `drizzle-kit` commands to find your schema: + +```ts +import { defineConfig } from "drizzle-kit" + +export default defineConfig({ + schema: "./src/db/auth-schema.ts", + out: "./drizzle", + dialect: "postgresql", + dbCredentials: { + url: process.env.DATABASE_URL!, + }, +}) +``` + +--- + +## Common Plugins + +| Plugin | Server Import | Client Import | Purpose | +|--------|---------------|---------------|---------| +| `twoFactor` | `better-auth/plugins` | `twoFactorClient` | 2FA with TOTP/OTP | +| `organization` | `better-auth/plugins` | `organizationClient` | Teams/orgs | +| `admin` | `better-auth/plugins` | `adminClient` | User management | +| `bearer` | `better-auth/plugins` | - | API token auth | +| `openAPI` | `better-auth/plugins` | - | API docs | +| `passkey` | `@better-auth/passkey` | `passkeyClient` | WebAuthn | +| `sso` | `@better-auth/sso` | - | Enterprise SSO | + +**Plugin pattern:** Server plugin + client plugin + run migrations. + +--- + +## Auth UI Implementation + +**Sign in flow:** +1. `signIn.email({ email, password })` or `signIn.social({ provider, callbackURL })` +2. Handle `error` in response +3. Redirect on success + +**Session check (client):** `useSession()` hook returns `{ data: session, isPending }` + +**Session check (server):** `auth.api.getSession({ headers: await headers() })` + +**Protected routes:** Check session, redirect to `/sign-in` if null. + +--- + +## Security Checklist + +- [ ] `BETTER_AUTH_SECRET` set (32+ chars) +- [ ] `advanced.useSecureCookies: true` in production +- [ ] `trustedOrigins` configured +- [ ] Rate limits enabled +- [ ] Email verification enabled +- [ ] Password reset implemented +- [ ] 2FA for sensitive apps +- [ ] CSRF protection NOT disabled +- [ ] `account.accountLinking` reviewed + +--- + +## Troubleshooting + +| Issue | Fix | +|-------|-----| +| "Secret not set" | Add `BETTER_AUTH_SECRET` env var | +| "Invalid Origin" | Add domain to `trustedOrigins` | +| Cookies not setting | Check `baseURL` matches domain; enable secure cookies in prod | +| OAuth callback errors | Verify redirect URIs in provider dashboard | +| Type errors after adding plugin | Re-run CLI generate/migrate | + +--- + +## Resources + +- [Docs](https://better-auth.com/docs) +- [Examples](https://github.com/better-auth/examples) +- [Plugins](https://better-auth.com/docs/concepts/plugins) +- [CLI](https://better-auth.com/docs/concepts/cli) +- [Migration Guides](https://better-auth.com/docs/guides) diff --git a/plugins/better-auth/skills-lock.json b/plugins/better-auth/skills-lock.json index 0a208379..d6ed3d75 100644 --- a/plugins/better-auth/skills-lock.json +++ b/plugins/better-auth/skills-lock.json @@ -11,7 +11,7 @@ "source": "better-auth/skills", "sourceType": "github", "skillPath": "better-auth/create-auth/SKILL.md", - "computedHash": "31c027d02dbf1faa37f58191f574b9443143042fb2b684fe32df53ab0270487c" + "computedHash": "9a9b498bc144367b8e237233f89def63fb2bd69e8cafe173eaa58fe3a863cdc4" }, "email-and-password-best-practices": { "source": "better-auth/skills", diff --git a/plugins/better-auth/skills/create-auth b/plugins/better-auth/skills/create-auth new file mode 120000 index 00000000..0d7ae1d2 --- /dev/null +++ b/plugins/better-auth/skills/create-auth @@ -0,0 +1 @@ +../.agents/skills/create-auth \ No newline at end of file diff --git a/plugins/chat-sdk/skills-lock.json b/plugins/chat-sdk/skills-lock.json index 7e529544..fa5e3b75 100644 --- a/plugins/chat-sdk/skills-lock.json +++ b/plugins/chat-sdk/skills-lock.json @@ -5,7 +5,7 @@ "source": "vercel/chat", "sourceType": "github", "skillPath": "skills/chat/SKILL.md", - "computedHash": "15ee91013fc51b443d2de8f0e81831b4f45d73ef59f0028472eeef62fd2dd22d" + "computedHash": "4b4063fbbb30242d68ccb5d179d404e9c35a588fc7a4cfe8cc059bb117fee28c" } } } diff --git a/plugins/dev3000/.agents/skills/d3k/SKILL.md b/plugins/dev3000/.agents/skills/d3k/SKILL.md index 04b27ee4..5d1c287e 100644 --- a/plugins/dev3000/.agents/skills/d3k/SKILL.md +++ b/plugins/dev3000/.agents/skills/d3k/SKILL.md @@ -1,6 +1,6 @@ --- name: d3k -description: Run and debug a local web app with a managed browser and a shared server/browser evidence timeline. Use when asked to develop, test, or reproduce a browser issue with d3k. +description: Test and debug a web app in its existing d3k browser, including local Chrome connected to a Vercel Sandbox over CDP. Use for UI checks, browser interactions, screenshots, and reproducing app issues when d3k is running. --- # d3k @@ -9,8 +9,26 @@ Own the project's dev server and monitored browser through a small background ru Use `node /scripts/runtime.mjs` as `runtime` below, from the **target project root**. In a development checkout the helper runs the source with Bun; installed skills use the JavaScript package through `pnpm dlx` (Node.js 24+ and pnpm required). Package acquisition runs in an isolated temporary directory so the target app’s pnpm workspace settings do not affect it; runtime commands still run against the target project. To try unreleased source from another project, set `D3K_RUNTIME_ROOT` to the dev3000 checkout. Do not use a previously published package to test unreleased changes. +## Desktop MCP connection + +When d3k MCP tools are available, use them directly instead of the shell runtime below. They manage a remote project without a local checkout. Call `sandbox_sessions`, then `sandbox_start` with `project: "team/project"` or `resume: ""`. Poll `sandbox_status` until `phase: "ready"`; startup is asynchronous. If another companion owns a saved session, report it instead of taking over or killing it. + +An explicit request for a remote sandbox takes precedence over a linked local repository. Use `project` and the requested `branch` to create a fresh remote checkout. Reuse with `resume` or `attach` only when requested by the user. If creation fails, report its actual error; do not assume the repository is private, attach to an old checkout, discard the requested branch, or fall back to editing locally. + +For a phone user or an explicit phone-preview request, use `sandbox_start` with `browser: "link"`, then present the ready `sandbox_status.preview.url` as a clickable link and its `expiresAt`. Never give the phone a `.localhost` URL. Companion Chrome runs headlessly for agent testing; it cannot control the phone's browser. The link grants session app access, so share it only in the user's chat and keep it out of PRs/public logs. It expires with the session and is regenerated on resume; use `browser: "link"` again when resuming. Local stdio carries no phone User-Agent: use explicit user/client context, not the emulated viewport. The macOS companion and access to its local MCP server are still required; this does not provide phone-only ChatGPT connectivity. App sign-in may require allowing the preview origin's exact callback URL with the auth provider. + +Use `sandbox_exec`, `sandbox_read_file`, and `sandbox_write_file` for all project files and commands remotely. Read remote AGENTS.md before editing. Use `sandbox_runtime` with `args: ["status"]` and cursor-based events for evidence. Use `sandbox_browser` with `args: ["snapshot", "-i"]` and other agent-browser interaction arguments for the existing local Chrome; no CDP discovery or separate browser is needed. `sandbox_github` publishes Git objects and PRs through local repository-scoped GitHub authentication. Keep credentials out of remote commands. Do not clone or save project code/env files locally. + +If the dev runtime stops or the app returns 502, use `sandbox_recover` with `action: "start"`, then poll `action: "status"` until recovered or failed. Recovery keeps the agent, local Chrome and remote files; it does not clone, reinstall, replay commands, renew Gateway keys or extend compute lifetime. Reread runtime status and reproduce afterward; old in-memory evidence is lost on restart. + +Keep the session running during user testing. `sandbox_stop` returns stopping; poll until stopped. Stopping or disconnecting the MCP server closes Chrome and d3k-owned compute, preserves remote files, and leaves a resumable session. One active sandbox per server. No terminal harness or Gateway key is created; the desktop client supplies the model. The desktop conversation may retain source returned by tools. + +For one-time desktop setup, use `d3k mcp --print-config`. Configure its local stdio command/args/environment in ChatGPT Desktop's MCP settings or merge the generated entry into Claude Desktop's `mcpServers` configuration, preserving other servers. The command uses absolute paths and a GUI-compatible PATH; regenerate it after replacing the installation. Local Vercel login and canonical Portless must be ready before starting a session. The CLI's `mcp` entry point also supplies these instructions when the standalone skill isn't installed. + ## Prepare +For an already connected d3k session, use its existing runtime and browser. The dev server may be running remotely in the sandbox even when the harness UI is local. Inspect status through the connected session before inferring that a server needs starting; do not start a local server to verify the remote app. After a user-visible UI edit, verify the changed page in the paired browser and read the relevant events before reporting completion. Report the actual command/tool error if verification is blocked. + 1. Run `runtime portless status`. Canonical HTTPS must be ready (`canonical: true`, `serviceInstalled: true`, `setupRequired: false`) before starting the app or browser. If needed run `runtime portless setup`, wait for the one-time administrator authorization, then check again. Stop if setup is unavailable; never silently substitute direct localhost. 2. Run `runtime status`. Reuse a ready runtime for this project. If it is starting, wait for readiness. Do not start another dev server. 3. Start `runtime start` in a retained background tool session. Keep its process handle. The runtime detects the package manager and `dev` script; use `--command ''` when needed. Poll `runtime status` until `ready: true` and `browserConnected: true` with a port-free `https://...localhost` URL. Inspect `runtime events` if startup fails while the runtime is still alive, and the retained process output if it exits. @@ -19,12 +37,31 @@ Use `node /scripts/runtime.mjs` as `runtime` below, from the ** - For “let me test,” give the user the app URL and leave the browser running while they reproduce the issue. - For autonomous debugging, drive the same browser with `runtime browser snapshot -i`, `runtime browser click @e2`, and other agent-browser commands. The runtime supplies the CDP connection; never open a separate browser or profile. +- For page-load metrics, use `runtime browser metrics https://example.com/ --new-tab` (MCP: `sandbox_browser` with `args: ["metrics", "https://example.com/", "--new-tab"]`). This navigates a new tab with observers installed first and returns Web Vitals, navigation timing, long tasks, and resource summaries as JSON. Prefer this to custom eval scripts. Omit the URL to inspect the current page without reloading; CLS and long tasks then report unavailable. Metrics are a provisional snapshot using the existing cache and throttling, not Lighthouse or field data. `null` means unavailable; zero transfer bytes alone do not establish cache usage. +- After responsive/device testing, restore window-sized rendering with `runtime browser set viewport 0 0 0` (MCP: `sandbox_browser` with `args: ["set", "viewport", "0", "0", "0"]`). The zeros restore native dimensions and device scale. Verify the page fills the headed window before handing it back. Do not restore to a guessed desktop size such as 1280×900; that leaves a fixed viewport when the user resizes the window. - Before a reproduction, read `runtime status` and keep its `sessionId` and `cursor`. - After the interaction, read `runtime events --session --after `. Server output, console messages, exceptions, network requests, and navigation share sequence numbers. Requests and responses carry CDP request IDs; adjacent server lines are chronological context, not proof of causation. +- Network events include redacted headers, wire-header events, protocol/cache metadata, transfer size, and response timing. Derived durations are milliseconds; `cdpTiming` preserves CDP units (`requestTime` in monotonic seconds, phase offsets in milliseconds; `-1` means unavailable). App-origin Fetch/XHR JSON and form bodies have bounded, credential-redacted previews (8 KiB; bodies over 64 KiB are omitted). `response-body` events can arrive after `network-complete`; read a fresh interval after completion before concluding a body is absent. Cross-origin, binary/streaming, unavailable, and oversized bodies report omission reasons. Redaction is best-effort; payloads may still contain sensitive application data. - Continue from the returned `cursor` while `hasMore` is true. Use `--limit 1..500`, `--source server|browser|runtime`, or `--level error` for focused reads. For error context, read the unfiltered interval as well. Filters advance the cursor through examined events; retain the original cursor if you need another view of the same interval. - `reset: true` means the runtime restarted. `dropped` reports evicted events. Never claim complete evidence across either gap. Replay the interaction if necessary. - Fix the code and repeat the same interaction, then inspect the new interval. Keep the runtime alive across edits. +## Pull requests + +When the user asks for a PR in a cloud d3k sandbox, that authorizes committing the intended changes, publishing a feature branch and opening a draft PR. Proceed without another confirmation. Read project instructions, inspect the diff and status, and stage only intended files with explicit paths; leave unrelated changes and untracked artifacts out. Include newly created source files only when they are part of the requested change. Create a feature branch from the current base unless already on the task's feature branch; preserve intended working changes and do not reset or discard other work. Confirm `git rev-parse HEAD` before publishing. + +Use the user's configured author/committer preference, including the Git identity supplied by the companion. Inspect the effective identity before committing. If no preference is configured, ask once for their name and email and remember it in the checkout's Git configuration for resumed sessions. Never invent a placeholder identity. Run project-required checks and an available build or lint; report any checks that were not run and why. + +After committing, use the deterministic publisher from the remote project root: + +```sh +/tmp/d3k-tools/node_modules/.bin/d3k-runtime publish --base main --title "Describe the change" --body-file /tmp/pr-body.md +``` + +Use the actual base branch and write the description file remotely. Local/desktop harnesses run this through `sandbox_exec`. This command reads committed Git objects directly, preserves message newlines and author/committer timestamps, verifies every uploaded SHA before updating a ref, and creates a draft PR or returns the existing open PR. It checks branch and PR state on retry, never force-pushes, and keeps GitHub credentials on the laptop. Do not manually reconstruct Git Data requests or try an unauthenticated push first. It publishes only the current committed feature branch; uncommitted changes and untracked files stay out. A connected laptop companion with working local GitHub authentication is required. + +The publisher supports up to 100 linear, unsigned feature commits with UTF-8 commit text/paths, at most 64 MiB / 10,000 uploaded objects, and individual requests below the bridge's 1 MiB limit. Signed/merge commits, extra commit headers, oversized objects or SHA mismatches fail without changing the branch ref. Use existing managed Git authentication for those cases when available; report the concrete limitation rather than inventing alternate metadata or asking for tokens. Never force-push or run `gh auth login`. A requested PR does not authorize merging it. + ## Lifecycle Keep the runtime's project-stable profile for sign-in state. Use `--headless` for CI or when requested; `--servers-only` deliberately omits the browser. Automatic screenshots and persistent log files are off; request a screenshot through `runtime browser screenshot ` when visual evidence is needed. @@ -33,22 +70,28 @@ Stop with `runtime stop` or the retained process handle when asked. Leaving the ## Vercel Sandbox with local Chrome -For a remote project with no local checkout, run `runtime sandbox --project --scope --name --no-shell` on the laptop in a retained background session. The launcher requires Vercel CLI login, Bun, Node.js 24+, and macOS. d3k downloads Chrome for Testing automatically on first use and reuses it afterward. It creates a sandbox from the project's linked public GitHub repository, pulls Development env through Vercel CLI, installs dependencies, and starts the remote skill runtime. For a private repository, use `--attach ` with an existing sandbox that already contains the checkout. Private-repository access must be configured separately. Do not supply GitHub or Vercel management tokens to the app environment. +For a remote project with no local checkout, run `runtime sandbox --project --scope --name --no-shell` on the laptop in a retained background session. The launcher requires Vercel CLI login, Bun, Node.js 24+, and macOS. d3k uses the installed standard Google Chrome binary with a dedicated per-project profile and filesystem confinement; browser updates remain managed by the existing installation or IT. It creates a sandbox from the project's linked public GitHub repository, pulls Development env through Vercel CLI, installs dependencies, and starts the remote skill runtime. For a private repository, use `--attach ` with an existing sandbox that already contains the checkout. Private-repository access must be configured separately. Do not supply GitHub or Vercel management tokens to the app environment. The app uses `https://.d3k.localhost`; `--hostname songbook` selects `https://songbook.d3k.localhost`. Canonical Portless is required on the laptop before launch, with no Portless inside the VM. A loopback streaming proxy forwards verified HTTPS app traffic and a session token to the sandbox. Add the exact local auth callback URL to the provider allowlist. The laptop companion opens a dedicated local Chrome and maintains an authenticated outbound WSS bridge; project source and dev compute stay remote. Dev commands must honor `PORT`. Keep the companion alive throughout debugging. -Run status, browser, events, and stop commands **inside the sandbox's project directory**, using `/tmp/d3k-tools/node_modules/.bin/d3k-runtime`. For example, use `vercel sandbox exec --project --scope --workdir -- /tmp/d3k-tools/node_modules/.bin/d3k-runtime browser snapshot -i`. Supply a fresh project OIDC token obtained from `vercel project token --scope --format json` as the CLI process's `VERCEL_OIDC_TOKEN` environment variable if needed; capture it without printing it or putting it in command arguments. The interactive launcher handles this authentication automatically. Read the remote `status` before driving Chrome and reuse that runtime; do not start another runtime or browser. An interactive `d3k --project team/project` (or `d3k sandbox`) enters the remote project directory and adds tools to PATH. Type `fx` there after authenticating the harness in the sandbox; d3k installs fx if absent but does not provision Gateway keys or forward laptop credentials. `--no-fx` skips installation. Project processes share the harness identity, so manually configured credentials are accessible to project code. Without a TTY, use `vercel sandbox exec` or `vercel sandbox connect` separately. All remote harnesses need their own authentication. The interactive shell does not inherit app env automatically. +For authorized pushes and PRs, follow the Pull requests section above: commit intended files, then run `d3k-runtime publish --base --title --body-file <remote-description-file>` in the remote project root. Local/desktop harnesses use `sandbox_exec` for that command. The lower-level `sandbox_github` / `runtime github` API remains available for repository-scoped inspection; the publisher owns object construction, SHA verification, safe branch updates and draft PR creation. If the companion is disconnected, leave publishing pending until it reconnects. + +Run status, browser, events, and stop commands **inside the sandbox's project directory**, using `/tmp/d3k-tools/node_modules/.bin/d3k-runtime`. For example, use `vercel sandbox exec <name> --project <project-id> --scope <team> --workdir <remote-project-root> -- /tmp/d3k-tools/node_modules/.bin/d3k-runtime browser snapshot -i`. Supply a fresh project OIDC token obtained from `vercel project token <project-id> --scope <team> --format json` as the CLI process's `VERCEL_OIDC_TOKEN` environment variable if needed; capture it without printing it or putting it in command arguments. The interactive launcher handles this authentication automatically. Read the remote `status` before driving Chrome and reuse that runtime; do not start another runtime or browser. New interactive `d3k --project team/project` (or `d3k sandbox`) sessions default to Eve, with remote execution and its native terminal UI locally. Eve uses pinned dependencies, a $1 Gateway cap and up to 30 minutes of access; `/model` switches models. Cross-launch conversation resume and laptop-close integration remain incomplete. `--no-shell` omits the harness. The following fx details apply when using fx or an existing plain-shell session: the shell enters the remote project directory and adds tools to PATH. Type `fx` there; d3k installs fx if absent and automatically connects it to the project’s team Gateway with a project-scoped key that expires automatically after `--timeout` minutes (default: 60, shared with sandbox compute); d3k prints its expiration time when closing the session. Its non-renewing budget is $5 including BYOK usage; use `--gateway-budget <usd>` to change it or `--no-gateway` for manual authentication. Laptop harness credentials and Vercel management tokens stay local. `--no-fx` skips installation. Project processes share the harness identity and can access its session Gateway key; use trusted projects. Without a TTY, use `vercel sandbox exec` or `vercel sandbox connect` separately. Remote Codex and Claude need their own authentication. The interactive shell does not inherit app env automatically. App traffic streams through the local proxy to the sandbox, preserving the browser origin, redirects, cookies, and WebSockets. Batch browser operations to reduce CDP round trips, select a nearby sandbox region, and avoid continuous screenshots. HMR works through the in-VM proxy. A bridge disconnect records an evidence gap and closes pending channels without replaying commands; reread status and reproduce after reconnecting before claiming complete evidence. Stopping the local launcher stops its runtime and a sandbox it created while preserving the sandbox filesystem. `--attach` leaves the existing VM running. The sandbox agent can control the dedicated Chrome profile, and the public sandbox app endpoint requires the companion session token; application authentication remains the app's responsibility. Keep agent credentials separate from infrastructure credentials; never describe remote browser control as adding no security risk. +The launcher retries failed runtime-status requests without interrupting the coding shell. A confirmed remote runtime exit keeps an already-open interactive shell alive; exit and resume to restore the dev server and browser bridge. The sandbox timeout still applies. Avoid broad process-kill commands or stopping the shared runtime during dependency upgrades. + +The sandbox terminal sends WebSocket keepalives. A socket closure without a remote exit frame leaves the companion and sandbox running and prints a reconnect command. Reconnect from another terminal; do not automatically restart a harness or replay its commands because its last action may have completed. Ctrl-C in the retained launcher closes the session. + ## CLI aliases and evidence views `d3k`, `dev3000`, and `d3k-runtime` use the same implementation. Local startup is `start`; root options are for sandbox mode. Timeout flags use milliseconds. `logs --count <1..500> --type server|browser|runtime|all` and `errors --context` read recent events from the same live timeline; use `--json` to retain evidence-gap metadata. Prefer cursor-based `events` for reproductions. The runtime does not install agent packages, write log files, or automatically upgrade. Use `browser` rather than the removed `agent-browser` subcommand. -Sandbox resume: `d3k --resume` lists locally saved session IDs, projects, sandbox names, branches, URLs, and last-used times. `d3k --resume <id>` restores that sandbox checkout and hostname, refreshes Development env and dependencies, restarts the dev runtime, and pairs the retained browser profile using fresh bridge/proxy tokens. No repository checkout is needed on the laptop. Ctrl-C closes the browser and prints the resume command to stdout. Saved entries contain discovery metadata, not credentials or past terminal output; remote availability is checked on resume. Optional `--command`, `--hostname`, `--timeout`, `--no-shell`, `--no-fx`, and `--headless` override launch settings. Sandboxes originally created by d3k stop compute again on resumed exit; externally attached sandboxes remain running. +Sandbox resume: `d3k --resume` lists locally saved session IDs, projects, sandbox names, branches, URLs, and last-used times. `d3k --resume <id>` restores that sandbox checkout and hostname, refreshes Development env and dependencies, restarts the dev runtime, and pairs the retained browser profile using fresh bridge/proxy tokens. No repository checkout is needed on the laptop. Ctrl-C closes the browser and prints the resume command to stdout. Saved entries contain discovery metadata, not credentials or past terminal output; remote availability is checked on resume. Optional `--command`, `--hostname`, `--timeout`, `--harness-location`, `--no-shell`, `--no-fx`, `--no-gateway`, `--gateway-budget`, and `--headless` override launch settings. Sandboxes originally created by d3k stop compute again on resumed exit; externally attached sandboxes remain running. -Use `d3k start --harness codex` locally or `d3k --project my-team/my-project --harness fx` remotely to launch a coding harness after readiness. The supported choices are `fx`, `codex`, and `claude`; commands, paths, and extra arguments are rejected. An interactive terminal is required, and exiting the harness closes a session started by that invocation. Harnesses must already be installed and authenticated, except for existing sandbox fx setup. Codex/Claude receive a prompt to read the bundled d3k skill and reuse the runtime; fx receives a temporary skill context through `--add-dir` (its interactive CLI has no startup prompt). No project instruction files are overwritten. Local harnesses use local credentials; sandbox fx can use the temporary team Gateway key. `--resume <id>` remembers the harness choice. +Use `d3k start --harness codex` locally or `d3k --project my-team/my-project --harness fx` remotely to launch a coding harness after readiness. Eve is the default for new interactive sandbox sessions; `fx`, `codex`, and `claude` are alternatives. Eve always runs remotely with a local native client and does not accept `--harness-location sandbox`. For the alternatives, commands, paths, and extra arguments are rejected. An interactive terminal is required, and exiting the harness closes a session started by that invocation. For fx, Codex and Claude in a sandbox project, `--harness` runs locally by default; `--harness-location sandbox` selects the remote harness. Local fx uses a stable control-directory identity recorded per d3k session; other local harnesses use a temporary control directory. Both receive preloaded routing instructions and a remote-command helper; Codex/Claude also receive per-launch MCP configuration. Use sandbox_exec/read_file/write_file for remote code, sandbox_runtime for remote logs, and sandbox_browser for direct local Chrome control. Use sandbox_github for the linked repository’s Git Data and PR APIs with local gh authentication; do not forward GitHub credentials into the sandbox. No project checkout or env file is saved locally, but harness conversations may retain code returned by tools. Resume retains the selected location. The following remote installation and context details apply to `--harness-location sandbox` or the plain remote shell. Harnesses must already be installed and authenticated, except for existing sandbox fx setup. Codex/Claude receive the connected-session skill directly in their startup prompt. fx receives a temporary skill context through `--add-dir`; in the sandbox, a temporary managed block in `~/.fx/AGENTS.md` also preloads the connected-session skill, including the browser connection, exact remote commands, and evidence workflow, including when typing `fx` from the shell. Existing instruction content is preserved and the block is removed on exit. No project instruction files are overwritten. Local harnesses use local credentials; sandbox fx can use the temporary team Gateway key. `--resume <id>` remembers the harness choice and passes `fx -c` when launching fx, restoring its remembered conversation in the same workspace. fx owns conversation persistence. Other harness conversations are not automatically resumed; in a plain remote shell, type `fx -c` yourself. Local connection instructions are regenerated and broker credentials removed on exit. Sandbox Chrome and its children run under macOS filesystem confinement. Do not bypass it or substitute an unrestricted browser. The outer policy replaces Chrome's inner sandbox because macOS cannot nest them. Only Chrome, required system files, and the dedicated profile and temporary files are readable; unrelated laptop files are denied. The profile uses a local password store rather than the laptop keychain. Browser sessions remain accessible to sandbox automation. Other platforms fail closed until equivalent isolation is available. diff --git a/plugins/dev3000/agent/skills/d3k/SKILL.md b/plugins/dev3000/agent/skills/d3k/SKILL.md index b1546209..5f1ae550 100644 --- a/plugins/dev3000/agent/skills/d3k/SKILL.md +++ b/plugins/dev3000/agent/skills/d3k/SKILL.md @@ -1,8 +1,9 @@ --- name: d3k -description: Run and debug a local web app with a managed browser and a shared - server/browser evidence timeline. Use when asked to develop, test, or - reproduce a browser issue with d3k. +description: Test and debug a web app in its existing d3k browser, including + local Chrome connected to a Vercel Sandbox over CDP. Use for UI checks, + browser interactions, screenshots, and reproducing app issues when d3k is + running. --- # d3k @@ -10,8 +11,26 @@ Own the project's dev server and monitored browser through a small background ru Use `node <skill-directory>/scripts/runtime.mjs` as `runtime` below, from the **target project root**. In a development checkout the helper runs the source with Bun; installed skills use the JavaScript package through `pnpm dlx` (Node.js 24+ and pnpm required). Package acquisition runs in an isolated temporary directory so the target app’s pnpm workspace settings do not affect it; runtime commands still run against the target project. To try unreleased source from another project, set `D3K_RUNTIME_ROOT` to the dev3000 checkout. Do not use a previously published package to test unreleased changes. +## Desktop MCP connection + +When d3k MCP tools are available, use them directly instead of the shell runtime below. They manage a remote project without a local checkout. Call `sandbox_sessions`, then `sandbox_start` with `project: "team/project"` or `resume: "<saved-id>"`. Poll `sandbox_status` until `phase: "ready"`; startup is asynchronous. If another companion owns a saved session, report it instead of taking over or killing it. + +An explicit request for a remote sandbox takes precedence over a linked local repository. Use `project` and the requested `branch` to create a fresh remote checkout. Reuse with `resume` or `attach` only when requested by the user. If creation fails, report its actual error; do not assume the repository is private, attach to an old checkout, discard the requested branch, or fall back to editing locally. + +For a phone user or an explicit phone-preview request, use `sandbox_start` with `browser: "link"`, then present the ready `sandbox_status.preview.url` as a clickable link and its `expiresAt`. Never give the phone a `.localhost` URL. Companion Chrome runs headlessly for agent testing; it cannot control the phone's browser. The link grants session app access, so share it only in the user's chat and keep it out of PRs/public logs. It expires with the session and is regenerated on resume; use `browser: "link"` again when resuming. Local stdio carries no phone User-Agent: use explicit user/client context, not the emulated viewport. The macOS companion and access to its local MCP server are still required; this does not provide phone-only ChatGPT connectivity. App sign-in may require allowing the preview origin's exact callback URL with the auth provider. + +Use `sandbox_exec`, `sandbox_read_file`, and `sandbox_write_file` for all project files and commands remotely. Read remote AGENTS.md before editing. Use `sandbox_runtime` with `args: ["status"]` and cursor-based events for evidence. Use `sandbox_browser` with `args: ["snapshot", "-i"]` and other agent-browser interaction arguments for the existing local Chrome; no CDP discovery or separate browser is needed. `sandbox_github` publishes Git objects and PRs through local repository-scoped GitHub authentication. Keep credentials out of remote commands. Do not clone or save project code/env files locally. + +If the dev runtime stops or the app returns 502, use `sandbox_recover` with `action: "start"`, then poll `action: "status"` until recovered or failed. Recovery keeps the agent, local Chrome and remote files; it does not clone, reinstall, replay commands, renew Gateway keys or extend compute lifetime. Reread runtime status and reproduce afterward; old in-memory evidence is lost on restart. + +Keep the session running during user testing. `sandbox_stop` returns stopping; poll until stopped. Stopping or disconnecting the MCP server closes Chrome and d3k-owned compute, preserves remote files, and leaves a resumable session. One active sandbox per server. No terminal harness or Gateway key is created; the desktop client supplies the model. The desktop conversation may retain source returned by tools. + +For one-time desktop setup, use `d3k mcp --print-config`. Configure its local stdio command/args/environment in ChatGPT Desktop's MCP settings or merge the generated entry into Claude Desktop's `mcpServers` configuration, preserving other servers. The command uses absolute paths and a GUI-compatible PATH; regenerate it after replacing the installation. Local Vercel login and canonical Portless must be ready before starting a session. The CLI's `mcp` entry point also supplies these instructions when the standalone skill isn't installed. + ## Prepare +For an already connected d3k session, use its existing runtime and browser. The dev server may be running remotely in the sandbox even when the harness UI is local. Inspect status through the connected session before inferring that a server needs starting; do not start a local server to verify the remote app. After a user-visible UI edit, verify the changed page in the paired browser and read the relevant events before reporting completion. Report the actual command/tool error if verification is blocked. + 1. Run `runtime portless status`. Canonical HTTPS must be ready (`canonical: true`, `serviceInstalled: true`, `setupRequired: false`) before starting the app or browser. If needed run `runtime portless setup`, wait for the one-time administrator authorization, then check again. Stop if setup is unavailable; never silently substitute direct localhost. 2. Run `runtime status`. Reuse a ready runtime for this project. If it is starting, wait for readiness. Do not start another dev server. 3. Start `runtime start` in a retained background tool session. Keep its process handle. The runtime detects the package manager and `dev` script; use `--command '<dev command>'` when needed. Poll `runtime status` until `ready: true` and `browserConnected: true` with a port-free `https://...localhost` URL. Inspect `runtime events` if startup fails while the runtime is still alive, and the retained process output if it exits. @@ -20,12 +39,31 @@ Use `node <skill-directory>/scripts/runtime.mjs` as `runtime` below, from the ** - For “let me test,” give the user the app URL and leave the browser running while they reproduce the issue. - For autonomous debugging, drive the same browser with `runtime browser snapshot -i`, `runtime browser click @e2`, and other agent-browser commands. The runtime supplies the CDP connection; never open a separate browser or profile. +- For page-load metrics, use `runtime browser metrics https://example.com/ --new-tab` (MCP: `sandbox_browser` with `args: ["metrics", "https://example.com/", "--new-tab"]`). This navigates a new tab with observers installed first and returns Web Vitals, navigation timing, long tasks, and resource summaries as JSON. Prefer this to custom eval scripts. Omit the URL to inspect the current page without reloading; CLS and long tasks then report unavailable. Metrics are a provisional snapshot using the existing cache and throttling, not Lighthouse or field data. `null` means unavailable; zero transfer bytes alone do not establish cache usage. +- After responsive/device testing, restore window-sized rendering with `runtime browser set viewport 0 0 0` (MCP: `sandbox_browser` with `args: ["set", "viewport", "0", "0", "0"]`). The zeros restore native dimensions and device scale. Verify the page fills the headed window before handing it back. Do not restore to a guessed desktop size such as 1280×900; that leaves a fixed viewport when the user resizes the window. - Before a reproduction, read `runtime status` and keep its `sessionId` and `cursor`. - After the interaction, read `runtime events --session <id> --after <cursor>`. Server output, console messages, exceptions, network requests, and navigation share sequence numbers. Requests and responses carry CDP request IDs; adjacent server lines are chronological context, not proof of causation. +- Network events include redacted headers, wire-header events, protocol/cache metadata, transfer size, and response timing. Derived durations are milliseconds; `cdpTiming` preserves CDP units (`requestTime` in monotonic seconds, phase offsets in milliseconds; `-1` means unavailable). App-origin Fetch/XHR JSON and form bodies have bounded, credential-redacted previews (8 KiB; bodies over 64 KiB are omitted). `response-body` events can arrive after `network-complete`; read a fresh interval after completion before concluding a body is absent. Cross-origin, binary/streaming, unavailable, and oversized bodies report omission reasons. Redaction is best-effort; payloads may still contain sensitive application data. - Continue from the returned `cursor` while `hasMore` is true. Use `--limit 1..500`, `--source server|browser|runtime`, or `--level error` for focused reads. For error context, read the unfiltered interval as well. Filters advance the cursor through examined events; retain the original cursor if you need another view of the same interval. - `reset: true` means the runtime restarted. `dropped` reports evicted events. Never claim complete evidence across either gap. Replay the interaction if necessary. - Fix the code and repeat the same interaction, then inspect the new interval. Keep the runtime alive across edits. +## Pull requests + +When the user asks for a PR in a cloud d3k sandbox, that authorizes committing the intended changes, publishing a feature branch and opening a draft PR. Proceed without another confirmation. Read project instructions, inspect the diff and status, and stage only intended files with explicit paths; leave unrelated changes and untracked artifacts out. Include newly created source files only when they are part of the requested change. Create a feature branch from the current base unless already on the task's feature branch; preserve intended working changes and do not reset or discard other work. Confirm `git rev-parse HEAD` before publishing. + +Use the user's configured author/committer preference, including the Git identity supplied by the companion. Inspect the effective identity before committing. If no preference is configured, ask once for their name and email and remember it in the checkout's Git configuration for resumed sessions. Never invent a placeholder identity. Run project-required checks and an available build or lint; report any checks that were not run and why. + +After committing, use the deterministic publisher from the remote project root: + +```sh +/tmp/d3k-tools/node_modules/.bin/d3k-runtime publish --base main --title "Describe the change" --body-file /tmp/pr-body.md +``` + +Use the actual base branch and write the description file remotely. Local/desktop harnesses run this through `sandbox_exec`. This command reads committed Git objects directly, preserves message newlines and author/committer timestamps, verifies every uploaded SHA before updating a ref, and creates a draft PR or returns the existing open PR. It checks branch and PR state on retry, never force-pushes, and keeps GitHub credentials on the laptop. Do not manually reconstruct Git Data requests or try an unauthenticated push first. It publishes only the current committed feature branch; uncommitted changes and untracked files stay out. A connected laptop companion with working local GitHub authentication is required. + +The publisher supports up to 100 linear, unsigned feature commits with UTF-8 commit text/paths, at most 64 MiB / 10,000 uploaded objects, and individual requests below the bridge's 1 MiB limit. Signed/merge commits, extra commit headers, oversized objects or SHA mismatches fail without changing the branch ref. Use existing managed Git authentication for those cases when available; report the concrete limitation rather than inventing alternate metadata or asking for tokens. Never force-push or run `gh auth login`. A requested PR does not authorize merging it. + ## Lifecycle Keep the runtime's project-stable profile for sign-in state. Use `--headless` for CI or when requested; `--servers-only` deliberately omits the browser. Automatic screenshots and persistent log files are off; request a screenshot through `runtime browser screenshot <absolute-path>` when visual evidence is needed. @@ -34,22 +72,28 @@ Stop with `runtime stop` or the retained process handle when asked. Leaving the ## Vercel Sandbox with local Chrome -For a remote project with no local checkout, run `runtime sandbox --project <name-or-id> --scope <team> --name <sandbox-name> --no-shell` on the laptop in a retained background session. The launcher requires Vercel CLI login, Bun, Node.js 24+, and macOS. d3k downloads Chrome for Testing automatically on first use and reuses it afterward. It creates a sandbox from the project's linked public GitHub repository, pulls Development env through Vercel CLI, installs dependencies, and starts the remote skill runtime. For a private repository, use `--attach <name>` with an existing sandbox that already contains the checkout. Private-repository access must be configured separately. Do not supply GitHub or Vercel management tokens to the app environment. +For a remote project with no local checkout, run `runtime sandbox --project <name-or-id> --scope <team> --name <sandbox-name> --no-shell` on the laptop in a retained background session. The launcher requires Vercel CLI login, Bun, Node.js 24+, and macOS. d3k uses the installed standard Google Chrome binary with a dedicated per-project profile and filesystem confinement; browser updates remain managed by the existing installation or IT. It creates a sandbox from the project's linked public GitHub repository, pulls Development env through Vercel CLI, installs dependencies, and starts the remote skill runtime. For a private repository, use `--attach <name>` with an existing sandbox that already contains the checkout. Private-repository access must be configured separately. Do not supply GitHub or Vercel management tokens to the app environment. The app uses `https://<project>.d3k.localhost`; `--hostname songbook` selects `https://songbook.d3k.localhost`. Canonical Portless is required on the laptop before launch, with no Portless inside the VM. A loopback streaming proxy forwards verified HTTPS app traffic and a session token to the sandbox. Add the exact local auth callback URL to the provider allowlist. The laptop companion opens a dedicated local Chrome and maintains an authenticated outbound WSS bridge; project source and dev compute stay remote. Dev commands must honor `PORT`. Keep the companion alive throughout debugging. -Run status, browser, events, and stop commands **inside the sandbox's project directory**, using `/tmp/d3k-tools/node_modules/.bin/d3k-runtime`. For example, use `vercel sandbox exec <name> --project <project-id> --scope <team> --workdir <remote-project-root> -- /tmp/d3k-tools/node_modules/.bin/d3k-runtime browser snapshot -i`. Supply a fresh project OIDC token obtained from `vercel project token <project-id> --scope <team> --format json` as the CLI process's `VERCEL_OIDC_TOKEN` environment variable if needed; capture it without printing it or putting it in command arguments. The interactive launcher handles this authentication automatically. Read the remote `status` before driving Chrome and reuse that runtime; do not start another runtime or browser. An interactive `d3k --project team/project` (or `d3k sandbox`) enters the remote project directory and adds tools to PATH. Type `fx` there after authenticating the harness in the sandbox; d3k installs fx if absent but does not provision Gateway keys or forward laptop credentials. `--no-fx` skips installation. Project processes share the harness identity, so manually configured credentials are accessible to project code. Without a TTY, use `vercel sandbox exec` or `vercel sandbox connect` separately. All remote harnesses need their own authentication. The interactive shell does not inherit app env automatically. +For authorized pushes and PRs, follow the Pull requests section above: commit intended files, then run `d3k-runtime publish --base <actual-base> --title <title> --body-file <remote-description-file>` in the remote project root. Local/desktop harnesses use `sandbox_exec` for that command. The lower-level `sandbox_github` / `runtime github` API remains available for repository-scoped inspection; the publisher owns object construction, SHA verification, safe branch updates and draft PR creation. If the companion is disconnected, leave publishing pending until it reconnects. + +Run status, browser, events, and stop commands **inside the sandbox's project directory**, using `/tmp/d3k-tools/node_modules/.bin/d3k-runtime`. For example, use `vercel sandbox exec <name> --project <project-id> --scope <team> --workdir <remote-project-root> -- /tmp/d3k-tools/node_modules/.bin/d3k-runtime browser snapshot -i`. Supply a fresh project OIDC token obtained from `vercel project token <project-id> --scope <team> --format json` as the CLI process's `VERCEL_OIDC_TOKEN` environment variable if needed; capture it without printing it or putting it in command arguments. The interactive launcher handles this authentication automatically. Read the remote `status` before driving Chrome and reuse that runtime; do not start another runtime or browser. New interactive `d3k --project team/project` (or `d3k sandbox`) sessions default to Eve, with remote execution and its native terminal UI locally. Eve uses pinned dependencies, a $1 Gateway cap and up to 30 minutes of access; `/model` switches models. Cross-launch conversation resume and laptop-close integration remain incomplete. `--no-shell` omits the harness. The following fx details apply when using fx or an existing plain-shell session: the shell enters the remote project directory and adds tools to PATH. Type `fx` there; d3k installs fx if absent and automatically connects it to the project’s team Gateway with a project-scoped key that expires automatically after `--timeout` minutes (default: 60, shared with sandbox compute); d3k prints its expiration time when closing the session. Its non-renewing budget is $5 including BYOK usage; use `--gateway-budget <usd>` to change it or `--no-gateway` for manual authentication. Laptop harness credentials and Vercel management tokens stay local. `--no-fx` skips installation. Project processes share the harness identity and can access its session Gateway key; use trusted projects. Without a TTY, use `vercel sandbox exec` or `vercel sandbox connect` separately. Remote Codex and Claude need their own authentication. The interactive shell does not inherit app env automatically. App traffic streams through the local proxy to the sandbox, preserving the browser origin, redirects, cookies, and WebSockets. Batch browser operations to reduce CDP round trips, select a nearby sandbox region, and avoid continuous screenshots. HMR works through the in-VM proxy. A bridge disconnect records an evidence gap and closes pending channels without replaying commands; reread status and reproduce after reconnecting before claiming complete evidence. Stopping the local launcher stops its runtime and a sandbox it created while preserving the sandbox filesystem. `--attach` leaves the existing VM running. The sandbox agent can control the dedicated Chrome profile, and the public sandbox app endpoint requires the companion session token; application authentication remains the app's responsibility. Keep agent credentials separate from infrastructure credentials; never describe remote browser control as adding no security risk. +The launcher retries failed runtime-status requests without interrupting the coding shell. A confirmed remote runtime exit keeps an already-open interactive shell alive; exit and resume to restore the dev server and browser bridge. The sandbox timeout still applies. Avoid broad process-kill commands or stopping the shared runtime during dependency upgrades. + +The sandbox terminal sends WebSocket keepalives. A socket closure without a remote exit frame leaves the companion and sandbox running and prints a reconnect command. Reconnect from another terminal; do not automatically restart a harness or replay its commands because its last action may have completed. Ctrl-C in the retained launcher closes the session. + ## CLI aliases and evidence views `d3k`, `dev3000`, and `d3k-runtime` use the same implementation. Local startup is `start`; root options are for sandbox mode. Timeout flags use milliseconds. `logs --count <1..500> --type server|browser|runtime|all` and `errors --context` read recent events from the same live timeline; use `--json` to retain evidence-gap metadata. Prefer cursor-based `events` for reproductions. The runtime does not install agent packages, write log files, or automatically upgrade. Use `browser` rather than the removed `agent-browser` subcommand. -Sandbox resume: `d3k --resume` lists locally saved session IDs, projects, sandbox names, branches, URLs, and last-used times. `d3k --resume <id>` restores that sandbox checkout and hostname, refreshes Development env and dependencies, restarts the dev runtime, and pairs the retained browser profile using fresh bridge/proxy tokens. No repository checkout is needed on the laptop. Ctrl-C closes the browser and prints the resume command to stdout. Saved entries contain discovery metadata, not credentials or past terminal output; remote availability is checked on resume. Optional `--command`, `--hostname`, `--timeout`, `--no-shell`, `--no-fx`, and `--headless` override launch settings. Sandboxes originally created by d3k stop compute again on resumed exit; externally attached sandboxes remain running. +Sandbox resume: `d3k --resume` lists locally saved session IDs, projects, sandbox names, branches, URLs, and last-used times. `d3k --resume <id>` restores that sandbox checkout and hostname, refreshes Development env and dependencies, restarts the dev runtime, and pairs the retained browser profile using fresh bridge/proxy tokens. No repository checkout is needed on the laptop. Ctrl-C closes the browser and prints the resume command to stdout. Saved entries contain discovery metadata, not credentials or past terminal output; remote availability is checked on resume. Optional `--command`, `--hostname`, `--timeout`, `--harness-location`, `--no-shell`, `--no-fx`, `--no-gateway`, `--gateway-budget`, and `--headless` override launch settings. Sandboxes originally created by d3k stop compute again on resumed exit; externally attached sandboxes remain running. -Use `d3k start --harness codex` locally or `d3k --project my-team/my-project --harness fx` remotely to launch a coding harness after readiness. The supported choices are `fx`, `codex`, and `claude`; commands, paths, and extra arguments are rejected. An interactive terminal is required, and exiting the harness closes a session started by that invocation. Harnesses must already be installed and authenticated, except for existing sandbox fx setup. Codex/Claude receive a prompt to read the bundled d3k skill and reuse the runtime; fx receives a temporary skill context through `--add-dir` (its interactive CLI has no startup prompt). No project instruction files are overwritten. Local harnesses use local credentials; sandbox fx can use the temporary team Gateway key. `--resume <id>` remembers the harness choice. +Use `d3k start --harness codex` locally or `d3k --project my-team/my-project --harness fx` remotely to launch a coding harness after readiness. Eve is the default for new interactive sandbox sessions; `fx`, `codex`, and `claude` are alternatives. Eve always runs remotely with a local native client and does not accept `--harness-location sandbox`. For the alternatives, commands, paths, and extra arguments are rejected. An interactive terminal is required, and exiting the harness closes a session started by that invocation. For fx, Codex and Claude in a sandbox project, `--harness` runs locally by default; `--harness-location sandbox` selects the remote harness. Local fx uses a stable control-directory identity recorded per d3k session; other local harnesses use a temporary control directory. Both receive preloaded routing instructions and a remote-command helper; Codex/Claude also receive per-launch MCP configuration. Use sandbox_exec/read_file/write_file for remote code, sandbox_runtime for remote logs, and sandbox_browser for direct local Chrome control. Use sandbox_github for the linked repository’s Git Data and PR APIs with local gh authentication; do not forward GitHub credentials into the sandbox. No project checkout or env file is saved locally, but harness conversations may retain code returned by tools. Resume retains the selected location. The following remote installation and context details apply to `--harness-location sandbox` or the plain remote shell. Harnesses must already be installed and authenticated, except for existing sandbox fx setup. Codex/Claude receive the connected-session skill directly in their startup prompt. fx receives a temporary skill context through `--add-dir`; in the sandbox, a temporary managed block in `~/.fx/AGENTS.md` also preloads the connected-session skill, including the browser connection, exact remote commands, and evidence workflow, including when typing `fx` from the shell. Existing instruction content is preserved and the block is removed on exit. No project instruction files are overwritten. Local harnesses use local credentials; sandbox fx can use the temporary team Gateway key. `--resume <id>` remembers the harness choice and passes `fx -c` when launching fx, restoring its remembered conversation in the same workspace. fx owns conversation persistence. Other harness conversations are not automatically resumed; in a plain remote shell, type `fx -c` yourself. Local connection instructions are regenerated and broker credentials removed on exit. Sandbox Chrome and its children run under macOS filesystem confinement. Do not bypass it or substitute an unrestricted browser. The outer policy replaces Chrome's inner sandbox because macOS cannot nest them. Only Chrome, required system files, and the dedicated profile and temporary files are readable; unrelated laptop files are denied. The profile uses a local password store rather than the laptop keychain. Browser sessions remain accessible to sandbox automation. Other platforms fail closed until equivalent isolation is available. diff --git a/plugins/dev3000/skills-lock.json b/plugins/dev3000/skills-lock.json index 8f04047c..54a99638 100644 --- a/plugins/dev3000/skills-lock.json +++ b/plugins/dev3000/skills-lock.json @@ -5,7 +5,7 @@ "source": "vercel-labs/dev3000", "sourceType": "github", "skillPath": "skills/d3k/SKILL.md", - "computedHash": "f5072c92a35fc845e7c888f260cdb6be0e17725cdb90be6f6e8d31f012e8bb04" + "computedHash": "0d6bee960f4b4e8fd906ffc0320f78e3cda6fc8401058ab1581ab894a233457a" } } } diff --git a/plugins/emulate/skills-lock.json b/plugins/emulate/skills-lock.json index eb55d1db..91b19198 100644 --- a/plugins/emulate/skills-lock.json +++ b/plugins/emulate/skills-lock.json @@ -5,73 +5,73 @@ "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/apple/SKILL.md", - "computedHash": "69ab6f2682bfd5090e6b268ea0fd6e274c9c48e2ca4100f32e6e3b0f6946a10e" + "computedHash": "2b2953c420cffa7944760ae657e958b325d6248ad45c83d67155978e170ceea5" }, "aws": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/aws/SKILL.md", - "computedHash": "18ef961539949106b17fda5fdf3c514c99374a99613f76d38757a2062339044c" + "computedHash": "e2e2f8b9914d8367eb9eec60d3d5c6c5e6e4c7753a6d1d25abeebe1ab6b22e62" }, "emulate": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/emulate/SKILL.md", - "computedHash": "015f578db9195b15af246453405c9c185db843f9138a9192f7e24afae6e7248b" + "computedHash": "3b3eb34ff198ea3cf9ed09cfea243ebe36e8f0634b4254654763c635585e93bb" }, "github": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/github/SKILL.md", - "computedHash": "187a308c36d8b4f7563409d899257b96c316e5f406812c9bf72d4ab450da4c1c" + "computedHash": "346028942ae3aa4ba666102f8be9c6d71a443b04baff6eab3406d159e51771ad" }, "google": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/google/SKILL.md", - "computedHash": "4177214ab7832ee08c20aa2b4a02e344bb1438a57b4f6c505ce338e8d16474ac" + "computedHash": "10345fb5eaf170d1bfd3eb1d17e31551375a3e84832d62eb0b918a9ff1900f8b" }, "linear": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/linear/SKILL.md", - "computedHash": "48d3b6c7869a98661c5bbd3a7b7df09ca4bbbae6b86450d6e57853edee822316" + "computedHash": "1afff000e71e09a81e88679eec026c6be7461a6adc47ff49141b9d18fa5867e9" }, "microsoft": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/microsoft/SKILL.md", - "computedHash": "028538190fd0e34e00df9ba026a9d1f165c5d2388f7c71a6a2a92c8ed285b461" + "computedHash": "acb9e5acf2dd45a4d1492dcedff563729d80f8d5efc4c51acb34173b3ccbd62a" }, "next": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/next/SKILL.md", - "computedHash": "c581384e91811b427855b502280ee3fb143ed3b0d29c409afb0290c12697eb9a" + "computedHash": "5d4b8fe47c5c359dd6f621a52185f54e90b2e8d2dbe86c1c5673ccf0c8749d7d" }, "resend": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/resend/SKILL.md", - "computedHash": "8bb5e578779dbdf54c4bfc7646c2795769ff08986f7512f2b7f9371ace5eef68" + "computedHash": "0d77f1da34df3103c3ce4e89f5f17276b9f8d8649a9551097004f88fdeafad1f" }, "slack": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/slack/SKILL.md", - "computedHash": "2e58613063c123bb7e3f02b5f400e6a2be341c2bbc1b80a9f8a8a83e1fb77dc5" + "computedHash": "d29856bd44e77b9e53a661500ee5e5c60e983cbc0bd4c56f6f9ba7ad23605699" }, "stripe": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/stripe/SKILL.md", - "computedHash": "03845f355c42838653296330cdcf6ea91dec9929b29c12a1d26861f47d5f6b9e" + "computedHash": "5070639f3195e29bbed03ba86040a37a925baa540147b95c0bc53dd62712a035" }, "vercel": { "source": "vercel-labs/emulate", "sourceType": "github", "skillPath": "skills/vercel/SKILL.md", - "computedHash": "cba57cf4bfd387a81a826056e2c89af2961833f9664deee109c45f2cb08b1571" + "computedHash": "5f38ac1fb261b5bcab5941dfb3afa73b6d081926911a60b8dadc64ee8186c9c7" } } } diff --git a/plugins/eve/skills-lock.json b/plugins/eve/skills-lock.json index 79f6134d..2b750e7f 100644 --- a/plugins/eve/skills-lock.json +++ b/plugins/eve/skills-lock.json @@ -5,7 +5,7 @@ "source": "vercel/eve", "sourceType": "github", "skillPath": "skills/eve/SKILL.md", - "computedHash": "4833349453c45c606f79202e495b7bbc42ea380f329f3beee36b9927763f19ef" + "computedHash": "79a1e606ecf33eed33c1f3adc1cc266de391e19aa9237ca37e4f3aa3fba6b228" } } } diff --git a/plugins/evlog/.agents/skills/review-logging-patterns/SKILL.md b/plugins/evlog/.agents/skills/review-logging-patterns/SKILL.md index cc8c5ba3..7d31a211 100644 --- a/plugins/evlog/.agents/skills/review-logging-patterns/SKILL.md +++ b/plugins/evlog/.agents/skills/review-logging-patterns/SKILL.md @@ -603,12 +603,12 @@ const app = Fastify({ logger: false }) await app.register(evlog) app.get('/api/users', async (request) => { - request.log.set({ users: { count: 42 } }) + request.evlog.set({ users: { count: 42 } }) return { users: [] } }) ``` -`request.log` is the evlog wide-event logger (shadows Fastify's built-in pino logger on the request). Fastify's pino logger remains accessible via `fastify.log`. +`request.evlog` is the evlog wide-event logger. TypeScript keeps Fastify's own logger type on `request.log`, so `request.log.set()` does not type-check: use `request.evlog` or `useLogger()`. Fastify's pino logger remains accessible via `fastify.log`. Use `useLogger()` to access the logger from anywhere in the call stack without passing `request`: @@ -1131,6 +1131,24 @@ evlog: { | `bearer` | `Bearer sk_live_abc...` | `Bearer ***` | | `iban` | `FR76 3000 6000 ...189` | `FR76****189` | +**Opt-in credential patterns** match the value under any key and inside longer strings. They are off by default, and `builtins` runs only the names it lists, so add them next to the defaults: + +| Pattern | Example Input | Masked Output | +|---------|---------------|---------------| +| `apiKeys` | `ghp_` followed by 36 characters | `ghp_***` | +| `privateKey` | `-----BEGIN RSA PRIVATE KEY-----` block | `-----BEGIN RSA PRIVATE KEY-----***` | +| `urlPassword` | `postgres://app:hunter2@db` | `postgres://app:***@db` | + +```typescript +evlog: { + redact: { + builtins: ['creditCard', 'email', 'ipv4', 'phone', 'jwt', 'bearer', 'iban', 'secrets', 'apiKeys', 'privateKey', 'urlPassword'], + } +} +``` + +`apiKeys` covers GitHub, npm, Anthropic, OpenAI, Stripe, Slack and AWS key IDs by their fixed prefix. Suggest them when the code logs provider tokens, private keys or connection strings, even under keys that `secrets` does not cover (`apiKey`, `authorization`, `dsn`). + Works in all frameworks: Nuxt (`evlog` config), Nitro (`evlog()` module options), Next.js (`createEvlog()`), standalone (`initLogger()`), and all middleware integrations (Hono, Express, Fastify, Elysia, NestJS). --- diff --git a/plugins/evlog/agent/skills/review-logging-patterns/SKILL.md b/plugins/evlog/agent/skills/review-logging-patterns/SKILL.md new file mode 100644 index 00000000..0363ccca --- /dev/null +++ b/plugins/evlog/agent/skills/review-logging-patterns/SKILL.md @@ -0,0 +1,1328 @@ +--- +name: review-logging-patterns +description: Review code for logging patterns and suggest evlog adoption. + Optionally use the evlog CLI, shipped with the evlog package (`evlog init` to + wire evlog, `evlog agents` to write the conventions into AGENTS.md, `evlog + map` to score entry-point coverage, `--baseline` to gate regressions in CI) on + Nuxt, Nitro, Next.js, TanStack Start, Hono, Express, and Fastify. Guides setup + on those plus SvelteKit, React Router, NestJS, Express, Fastify, Elysia, oRPC, + Effect, Cloudflare Workers, AWS Lambda, Astro, and standalone TypeScript. + Detects console.log spam, unstructured errors, and missing context. Covers + wide events, structured errors, drain adapters (Axiom, OTLP, HyperDX, Sazabi, + PostHog, Sentry, Better Stack, Datadog, Loki, ClickHouse, NuxtHub, Memory), + sampling, enrichers, and AI SDK integration. +license: MIT +metadata: + author: HugoRCD + version: "0.9" +--- +# Review logging patterns + +Review and improve logging patterns in TypeScript/JavaScript codebases. Transform scattered console.logs into structured wide events and convert generic errors into self-documenting structured errors. + +## When to Use + +- Setting up evlog in a new or existing project (any supported framework) +- Reviewing code for logging best practices +- Converting console.log statements to structured logging +- Improving error handling with better context +- Configuring log draining, sampling, or enrichment + +## Quick Reference + +| Working on... | Resource | +| ----------------------- | ------------------------------------------------------------------ | +| Setup (CLI) | [`evlog init`](https://www.evlog.dev/cli/init): wire evlog into the project | +| Project conventions (CLI) | [`evlog agents`](https://www.evlog.dev/cli/agents): write the evlog block into the project's AGENTS.md | +| Coverage map (CLI) | [`evlog map`](https://www.evlog.dev/cli/map): score dark entry points | +| CI gating (CLI) | [`evlog map --min-score / --baseline`](https://www.evlog.dev/cli/ci): gate regressions | +| Wide events patterns | [references/wide-events.md](references/wide-events.md) | +| Error handling | [references/structured-errors.md](references/structured-errors.md) | +| Code review checklist | [references/code-review.md](references/code-review.md) | +| Drain pipeline | [references/drain-pipeline.md](references/drain-pipeline.md) | +| Audit logs | [build-audit-logs](../build-audit-logs/SKILL.md) skill + [docs](https://www.evlog.dev/use-cases/audit/overview) | + +## Audit logs + +For security-sensitive actions (auth, billing, admin, data export), use evlog's audit layer: a typed `audit` field on wide events, not a parallel logger. See the **`build-audit-logs`** skill for end-to-end setup (`log.audit`, `withAudit`, denials, `auditEnricher`, `auditOnly`, `signed`, `mockAudit`). + +```typescript +log.audit({ + action: 'invoice.refund', + actor: { type: 'user', id: user.id }, + target: { type: 'invoice', id: invoice.id }, + outcome: 'success', +}) +``` + +Docs: https://www.evlog.dev/use-cases/audit/overview + +## Installation + +```bash +npm install evlog +``` + +## Use the CLI (recommended on Nuxt, Nitro, Next.js, TanStack Start, Hono, Express, Fastify) + +The `evlog` executable ships with the `evlog` package: `pnpm evlog`, `npx evlog`, `bunx evlog`. It runs `@evlog/cli` when installed and fetches it on demand otherwise (no dependency added to the project), so it works on a project with nothing yet. Early but worth trying. It reads the project on disk (no traffic, no config needed). On the seven supported frameworks it covers the whole loop: **wire evlog in** (`init`), **score coverage** (`map`), **lock the score in CI** (`--min-score`, `--baseline`). If the CLI is unavailable, the framework has no adapter yet, or the user declines, continue with the manual sections below; the skill does not depend on it. **Ask before installing anything**. + +### 1. Setup: `evlog init` + +On a project that doesn't use evlog yet, prefer `init` over hand-writing the setup, since it detects the framework, reads what the project already has, and generates config, drains, enrichers, and extras in one pass. It is fully scriptable for agents: + +```bash +# preview everything without writing (always start here) +npx evlog init --dry-run --yes + +# then apply — flags instead of prompts +npx evlog init --yes \ + --service my-app \ + --drain fs \ + --prodDrain axiom \ + --extras enrichers,pipeline,sampling \ + --sampling medium +``` + +Useful flags: `--framework` (override detection: `nuxt`, `nitro`, `next`, `tanstack-start`, `hono`, `express`, `fastify`), `--prodDrain` (comma-separated: `axiom`, `otlp`, `posthog`, `sentry`, `better-stack`, `datadog`, `hyperdx`, `sazabi`, `loki`, `clickhouse`), `--extras` (`enrichers`, `pipeline`, `sampling`, `vite`, `error-catalog`, `audit-catalog`, `ai`, `better-auth`), `--enrichers`, `--sampling` (traffic tier: `all`, `low`, `medium`, `high`, `very-high`), `--apps` (monorepo: which workspace packages), `--no-install`. Review the `--dry-run` output with the user before applying. Docs: https://www.evlog.dev/cli/init + +### 2. Score: `evlog map` + +```bash +npx evlog map --no-write +# agents: npx evlog map --json --no-write +``` + +What you get: + +- A project score and which entry points are still dark +- **FIX FIRST**: the three most valuable places to fix +- **GOING FURTHER**: opportunities (catalogs, audit coverage, AI logging, auth identity) that never cost points +- Per-file inspect: `npx evlog map <file> --no-write` shows the shape the handler could take +- Re-run after fixes and watch the score move + +Work FIX FIRST in order, keep changes minimal (`useLogger()`, `log.set()`, `log.audit()`, `createError({ why, fix })`), then re-run with `--no-write`. Omit `--no-write` only when the user wants `evlog.map.json` written. + +### 3. Lock it in CI: `--min-score` and `--baseline` + +On GitHub Actions, suggest the [evlog action](https://github.com/evloghq/action) (`uses: evloghq/action@v1` with `min-score`): it scans the pull request base itself, so no map file has to be committed, and puts regressions on the diff and in one comment. The commands below are the same gate from a `run:` step. + +After fixing, propose making the score durable. This is where the CLI earns its keep: + +```bash +# in CI, after pnpm add -D @evlog/cli (pinned by the lockfile; ask first) +pnpm exec evlog map --min-score 80 # absolute gate: exits 1 below the threshold +pnpm exec evlog map --baseline # ratchet: exits 1 if this PR made things worse +``` + +`--baseline` compares the fresh scan against the committed `evlog.map.json`, **per entry point and per requirement**, so a refactor that instruments one route and breaks another fails even if the total score is unchanged. Disabling a passing check with a comment counts as a regression too. New uninstrumented routes are listed as `NEW AND DARK` without failing. Workflow: commit `evlog.map.json` once, add the `--baseline` run to CI, then re-run `map` without `--baseline` to accept an intentional change. Docs: https://www.evlog.dev/cli/ci + +When the project has an `evlog.config.ts`, the gate can live there instead of in the CI step: `map: { minScore: 80, baseline: true }` makes a bare `evlog map` gate the same way, and a flag still wins. The same `map` block turns checks off for every entry point (`rules: { 'error-catalog': 'off' }`) and leaves entry points out by file glob (`ignore`). The CLI reads these values without running the file, so they must be literals. Docs: https://www.evlog.dev/cli/config + +Early days: adapters and rules are still evolving; expect scores to move between releases. Docs: https://www.evlog.dev/cli/map · Rules: https://www.evlog.dev/cli/rules + +--- + +## Framework Setup + +### Nuxt + +```typescript +// nuxt.config.ts +export default defineNuxtConfig({ + modules: ['evlog/nuxt'], + evlog: { + env: { service: 'my-app' }, + include: ['/api/**'], + }, +}) +``` + +`useLogger`, `log`, and `parseError` are **auto-imported**. `createError` is not: a bare one resolves to h3's, which drops `why`, `fix`, and `link`. Import it from `evlog`. + +```typescript +// server/api/checkout.post.ts — no imports needed +export default defineEventHandler(async (event) => { + const log = useLogger(event) + log.set({ user: { id: user.id, plan: user.plan } }) + return { success: true } +}) +``` + +Drain, enrich, sampling and redaction can live in `evlog.config.ts` at the app root. The module loads it on the server, and the `evlog` key in `nuxt.config.ts` overrides it. The browser logger reads only the `evlog` key. + +```typescript +// evlog.config.ts +import { defineEvlog } from 'evlog' +import { createAxiomDrain } from 'evlog/axiom' + +export default defineEvlog({ + service: 'my-app', + drain: createAxiomDrain(), +}) +``` + +Nitro hooks in server plugins keep working next to the file, and an event reaches both drains: + +```typescript +// server/plugins/evlog-drain.ts +import { createAxiomDrain } from 'evlog/axiom' + +export default defineNitroPlugin((nitroApp) => { + nitroApp.hooks.hook('evlog:drain', createAxiomDrain()) +}) +``` + +Client transport (auto-configured Vue plugin): + +```typescript +// nuxt.config.ts +evlog: { + transport: { enabled: true }, // logs sent to /api/_evlog/ingest +} +``` + +Client-side: `log`, `setIdentity`, `clearIdentity` are auto-imported in components. + +### Next.js + +**Step 1: Create central config.** All exports come from here: + +```typescript +// lib/evlog.ts +import type { DrainContext } from 'evlog' +import { createEvlog } from 'evlog/next' +import { createUserAgentEnricher, createRequestSizeEnricher } from 'evlog/enrichers' +import { createDrainPipeline } from 'evlog/pipeline' + +const enrichers = [createUserAgentEnricher(), createRequestSizeEnricher()] +const pipeline = createDrainPipeline<DrainContext>({ batch: { size: 50, intervalMs: 5000 } }) +const drain = pipeline(createAxiomDrain({ dataset: 'logs', apiKey: process.env.AXIOM_API_KEY! })) + +export const { withEvlog, useLogger, log, createError } = createEvlog({ + service: 'my-app', + sampling: { + rates: { info: 10 }, + keep: [{ status: 400 }, { duration: 1000 }], + }, + routes: { + '/api/auth/**': { service: 'auth-service' }, + '/api/checkout/**': { service: 'checkout-service' }, + }, + keep: (ctx) => { + const user = ctx.context.user as { premium?: boolean } | undefined + if (user?.premium) ctx.shouldKeep = true + }, + enrich: (ctx) => { + for (const enricher of enrichers) enricher(ctx) + }, + drain, +}) +``` + +**Step 2: Wrap route handlers** with `withEvlog()`: + +```typescript +// app/api/checkout/route.ts +import { withEvlog, useLogger } from '@/lib/evlog' + +export const POST = withEvlog(async (request: Request) => { + const log = useLogger() // Zero arguments — uses AsyncLocalStorage + log.set({ user: { id: 'user_123', plan: 'enterprise' } }) + log.set({ cart: { items: 3, total: 14999 } }) + return Response.json({ success: true }) +}) +``` + +**Step 3: Server Actions.** Same `withEvlog()` wrapper: + +```typescript +// app/actions.ts +'use server' +import { withEvlog, useLogger } from '@/lib/evlog' + +export const checkout = withEvlog(async (formData: FormData) => { + const log = useLogger() + log.set({ action: 'checkout', source: 'server-action' }) + return { success: true } +}) +``` + +**Step 4: Middleware** (optional, sets `x-request-id` + timing headers): + +```typescript +// proxy.ts +import { evlogMiddleware } from 'evlog/next' +export const proxy = evlogMiddleware() +export const config = { matcher: ['/api/:path*'] } +``` + +**Step 5: Client Provider.** Wrap the root layout: + +```tsx +// app/layout.tsx +import { EvlogProvider } from 'evlog/next/client' + +export default function Layout({ children }: { children: React.ReactNode }) { + return ( + <html lang="en"> + <body> + <EvlogProvider service="my-app" transport={{ enabled: true, endpoint: '/api/evlog/ingest' }}> + {children} + </EvlogProvider> + </body> + </html> + ) +} +``` + +**Step 6: Client logging.** In any client component: + +```tsx +'use client' +import { log, setIdentity, clearIdentity } from 'evlog/next/client' + +setIdentity({ userId: 'usr_123' }) +log.info({ action: 'checkout_click' }) +clearIdentity() +``` + +**Step 7 (optional): Instrumentation.** Startup plus global `onRequestError` (SSR/RSC errors outside `withEvlog`). Use `defineNodeInstrumentation(() => import('./lib/evlog'))` in root `instrumentation.ts` to gate Node + cache the import, **or** write `register`/`onRequestError` manually. Both are valid. For custom logic, wrap evlog’s `register`/`onRequestError` inside `lib/evlog.ts` (compose with your own init or metrics), then re-export. + +Export `createInstrumentation()` from `lib/evlog.ts` alongside `createEvlog()`. See framework docs for coexistence with `lockLogger`. + +**Step 8: Client ingest endpoint.** Receives client logs: + +```typescript +// app/api/evlog/ingest/route.ts +import { NextRequest } from 'next/server' + +const VALID_LEVELS = ['info', 'error', 'warn', 'debug'] as const + +export async function POST(request: NextRequest) { + const origin = request.headers.get('origin') + const host = request.headers.get('host') + if (origin && new URL(origin).host !== host) { + return Response.json({ error: 'Invalid origin' }, { status: 403 }) + } + const body = await request.json() + if (!body?.timestamp || !body?.level || !VALID_LEVELS.includes(body.level)) { + return Response.json({ error: 'Invalid payload' }, { status: 400 }) + } + const { service: _, ...sanitized } = body + console.log('[CLIENT LOG]', JSON.stringify({ ...sanitized, service: 'my-app', source: 'client' })) + return new Response(null, { status: 204 }) +} +``` + +### SvelteKit + +```typescript +// src/hooks.server.ts +import { initLogger } from 'evlog' +import { createEvlogHooks } from 'evlog/sveltekit' + +initLogger({ env: { service: 'my-app' } }) + +export const { handle, handleError } = createEvlogHooks() +``` + +Access the logger via `event.locals.log` in route handlers or `useLogger()` from anywhere in the call stack: + +```typescript +// src/routes/api/users/[id]/+server.ts +import { json } from '@sveltejs/kit' + +export const GET = ({ locals, params }) => { + locals.log.set({ user: { id: params.id } }) + return json({ id: params.id }) +} +``` + +```typescript +import { useLogger } from 'evlog/sveltekit' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +export const { handle, handleError } = createEvlogHooks({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +}) +``` + +### Nitro v3 + +```typescript +// nitro.config.ts +import { defineConfig } from 'nitro' +import evlog from 'evlog/nitro/v3' + +export default defineConfig({ + modules: [evlog({ env: { service: 'my-api' } })], +}) +``` + +```typescript +// routes/api/checkout.post.ts +import { defineHandler } from 'nitro/h3' +import { useLogger } from 'evlog/nitro/v3' + +export default defineHandler(async (event) => { + const log = useLogger(event) + log.set({ action: 'checkout' }) + return { ok: true } +}) +``` + +### TanStack Start + +TanStack Start uses Nitro v3. Install evlog and add a `nitro.config.ts`: + +```typescript +// nitro.config.ts +import { defineConfig } from 'nitro' +import evlog from 'evlog/nitro/v3' + +export default defineConfig({ + experimental: { asyncContext: true }, + modules: [evlog({ env: { service: 'my-app' } })], +}) +``` + +Add the error handling middleware to `__root.tsx`: + +```typescript +// src/routes/__root.tsx +import { createMiddleware } from '@tanstack/react-start' +import { evlogErrorHandler } from 'evlog/nitro/v3' + +export const Route = createRootRoute({ + server: { + middleware: [createMiddleware().server(evlogErrorHandler)], + }, +}) +``` + +Use `useRequest()` from `nitro/context` to access the logger: + +```typescript +import { useRequest } from 'nitro/context' +import type { RequestLogger } from 'evlog' + +const req = useRequest() +const log = req.context.log as RequestLogger +log.set({ user: { id: 'user_123' } }) +``` + +### Nitro v2 + +```typescript +// nitro.config.ts +import { defineNitroConfig } from 'nitropack/config' +import evlog from 'evlog/nitro' + +export default defineNitroConfig({ + modules: [evlog({ env: { service: 'my-api' } })], +}) +``` + +Import `useLogger` from `evlog/nitro` in routes. On Nitro v2 and v3, an `evlog.config.ts` at the app root is loaded by the module, and the options passed to `evlog()` override it. + +### NestJS + +```typescript +// src/app.module.ts +import { Module } from '@nestjs/common' +import { EvlogModule } from 'evlog/nestjs' + +@Module({ + imports: [EvlogModule.forRoot()], +}) +export class AppModule {} +``` + +`EvlogModule.forRoot()` registers a global middleware, on both `@nestjs/platform-express` and `@nestjs/platform-fastify`. `req.log` in controllers is the evlog logger on both platforms. Do not also register `evlog/fastify` on a Nest Fastify app, or every request emits two wide events. Use `useLogger()` to access the request-scoped logger from any controller or service: + +```typescript +import { useLogger } from 'evlog/nestjs' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +EvlogModule.forRoot({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +}) +``` + +For async configuration with NestJS DI, use `forRootAsync()`: + +```typescript +EvlogModule.forRootAsync({ + imports: [ConfigModule], + inject: [ConfigService], + useFactory: (config) => ({ + drain: createAxiomDrain({ apiKey: config.get('AXIOM_API_KEY') }), + }), +}) +``` + +### Express + +```typescript +import express from 'express' +import { initLogger } from 'evlog' +import { evlog, useLogger } from 'evlog/express' + +initLogger({ env: { service: 'my-api' } }) + +const app = express() +app.use(evlog()) + +app.get('/api/users', (req, res) => { + req.log.set({ users: { count: 42 } }) + res.json({ users: [] }) +}) +``` + +Use `useLogger()` to access the logger from anywhere in the call stack without passing `req`: + +```typescript +import { useLogger } from 'evlog/express' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +app.use(evlog({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +})) +``` + +### Hono + +```typescript +import { Hono } from 'hono' +import { initLogger } from 'evlog' +import { evlog, type EvlogVariables } from 'evlog/hono' + +initLogger({ env: { service: 'my-api' } }) + +const app = new Hono<EvlogVariables>() +app.use(evlog()) + +app.get('/api/users', (c) => { + const log = c.get('log') + log.set({ users: { count: 42 } }) + return c.json({ users: [] }) +}) +``` + +Access the logger via `c.get('log')` in handlers. Use `useLogger()` from `evlog/hono` in the layers underneath (services, repositories) where `c` is not in hand. Both return the same logger: + +```typescript +import { useLogger } from 'evlog/hono' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +On Cloudflare Workers, `useLogger()` needs the `nodejs_compat` (or `nodejs_als`) compatibility flag; `c.get('log')` works with or without it. + +Structured errors: throw `createError()`, then in `app.onError` use `parseError()` and pass `parsed.status as ContentfulStatusCode` to `c.json()` (Hono types the status argument as `ContentfulStatusCode`, not `number`). + +```typescript +import { createError, parseError } from 'evlog' +import type { ContentfulStatusCode } from 'hono/utils/http-status' + +app.onError((error, c) => { + c.get('log').error(error) + const parsed = parseError(error) + return c.json( + { message: parsed.message, why: parsed.why, fix: parsed.fix, link: parsed.link }, + parsed.status as ContentfulStatusCode, + ) +}) +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +app.use(evlog({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +})) +``` + +### Fastify + +```typescript +import Fastify from 'fastify' +import { initLogger } from 'evlog' +import { evlog, useLogger } from 'evlog/fastify' + +initLogger({ env: { service: 'my-api' } }) + +const app = Fastify({ logger: false }) +await app.register(evlog) + +app.get('/api/users', async (request) => { + request.evlog.set({ users: { count: 42 } }) + return { users: [] } +}) +``` + +`request.evlog` is the evlog wide-event logger. TypeScript keeps Fastify's own logger type on `request.log`, so `request.log.set()` does not type-check: use `request.evlog` or `useLogger()`. Fastify's pino logger remains accessible via `fastify.log`. + +Use `useLogger()` to access the logger from anywhere in the call stack without passing `request`: + +```typescript +import { useLogger } from 'evlog/fastify' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +await app.register(evlog, { + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +}) +``` + +### Elysia + +```typescript +import { Elysia } from 'elysia' +import { initLogger } from 'evlog' +import { evlog, useLogger } from 'evlog/elysia' + +initLogger({ env: { service: 'my-api' } }) + +const app = new Elysia() + .use(evlog()) + .get('/api/users', ({ log }) => { + log.set({ users: { count: 42 } }) + return { users: [] } + }) + .listen(3000) +``` + +Use `useLogger()` to access the logger from anywhere in the call stack: + +```typescript +import { useLogger } from 'evlog/elysia' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +app.use(evlog({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +})) +``` + +### React Router + +```typescript +// react-router.config.ts +import type { Config } from '@react-router/dev/config' + +export default { + future: { + v8_middleware: true, + }, +} satisfies Config +``` + +```typescript +// app/root.tsx +import { initLogger } from 'evlog' +import { evlog } from 'evlog/react-router' + +initLogger({ env: { service: 'my-api' } }) + +export const middleware: Route.MiddlewareFunction[] = [ + evlog(), +] +``` + +Access the logger via `context.get(loggerContext)` in loaders and actions: + +```typescript +// app/routes/api.users.$id.tsx +import { loggerContext } from 'evlog/react-router' + +export async function loader({ params, context }: Route.LoaderArgs) { + const log = context.get(loggerContext) + log.set({ user: { id: params.id } }) + return { users: [] } +} +``` + +Use `useLogger()` to access the logger from anywhere in the call stack without passing context: + +```typescript +import { useLogger } from 'evlog/react-router' + +async function findUsers() { + const log = useLogger() + log.set({ db: { query: 'SELECT * FROM users' } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +export const middleware: Route.MiddlewareFunction[] = [ + evlog({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, + }), +] +``` + +### oRPC + +```typescript +import { os } from '@orpc/server' +import { RPCHandler } from '@orpc/server/fetch' +import { initLogger } from 'evlog' +import { evlog, withEvlog, type EvlogOrpcContext } from 'evlog/orpc' + +initLogger({ env: { service: 'my-rpc' } }) + +const base = os.$context<EvlogOrpcContext>().use(evlog()) + +const router = { + ping: base.handler(({ context }) => { + context.log.set({ pinged: true }) + return { ok: true } + }), +} + +const handler = withEvlog(new RPCHandler(router)) + +export default async function fetch(request: Request) { + const { matched, response } = await handler.handle(request, { prefix: '/rpc' }) + return matched ? response : new Response('Not Found', { status: 404 }) +} +``` + +`withEvlog()` wraps the handler so each matched request emits one wide event; `os.use(evlog())` exposes `context.log` on every procedure that descends from `base` and tags the wide event with `operation` (the procedure path joined with `.`). + +Use `useLogger()` to access the logger from utility modules: + +```typescript +import { useLogger } from 'evlog/orpc' + +async function chargeCard(amount: number) { + const log = useLogger() + log.set({ payment: { amount } }) +} +``` + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +const handler = withEvlog(new RPCHandler(router), { + include: ['/rpc/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +}) +``` + +### Effect + +Effect 4 (`HttpRouter`, `HttpApi`). `routerLayer` installs the `evlog()` middleware on every route and provides the `EvlogLogger` service. `initLoggerLayer` calls `initLogger` when the layer builds and flushes a `createDrainPipeline` drain when its scope closes: + +```typescript +import { createServer } from 'node:http' +import { NodeHttpServer, NodeRuntime } from '@effect/platform-node' +import { Effect, Layer } from 'effect' +import { HttpRouter, HttpServerResponse } from 'effect/http' +import { EvlogLogger, initLoggerLayer, routerLayer } from 'evlog/effect' + +const Routes = HttpRouter.add('GET', '/api/users', Effect.gen(function* () { + const log = yield* EvlogLogger + log.set({ users: { count: 42 } }) + return HttpServerResponse.jsonUnsafe({ users: [] }) +})) + +const Server = HttpRouter.serve(Routes.pipe(Layer.provide(routerLayer())), { disableLogger: true }).pipe( + Layer.provide(NodeHttpServer.layer(createServer, { port: 3000 })), + Layer.provide(initLoggerLayer({ env: { service: 'my-api' } })), +) + +NodeRuntime.runMain(Layer.launch(Server)) +``` + +There is no `AsyncLocalStorage` and no `log.fork()`. The event lives in the Effect context: `yield* EvlogLogger` in handlers and services (the requirement shows up in the effect's type), `yield* useLogger<Fields>()` for typed fields, and `yield* set(fields)` in library code that should not require `EvlogLogger` (a no-op outside a unit). It runs on Cloudflare Workers without `nodejs_compat`. + +```typescript +import { Effect } from 'effect' +import { useLogger } from 'evlog/effect' + +const chargeCard = (amount: number) => Effect.gen(function* () { + const log = yield* useLogger<{ payment: { amount: number } }>() + log.set({ payment: { amount } }) +}) +``` + +Failures are read from the `Cause` (the primary failure as `error`, the rest under `error.suppressed`, a pure interruption as `status: 499`). Effect's router answers a failed `createError({ status: 402 })` with 500, so catch it in the handler with `Effect.catch`, call `log.error(error)`, and return `HttpServerResponse.jsonUnsafe(body, { status: parsed.status })` built from `parseError(error)`. `HttpApi` encodes declared errors as responses before the middleware: the event gets the status but no `error` field. + +Non-HTTP work (jobs, queue consumers, Durable Object alarms) uses `.pipe(withLogger('action-name', options))`. A Durable Object `fetch` uses `.pipe(withRequestLogger(request, options))`. On Workers, provide `WaitUntil` per request (`handler(request, Context.make(WaitUntil, ctx.waitUntil.bind(ctx)))` with `HttpRouter.toWebHandler`) so drains outlive the response. `loggerLayer()` folds `Effect.log*` calls into the current wide event. Each unit continues an incoming `traceparent`, and `withTracePropagation(client)` and `traceHeaders` forward it on outgoing calls. + +Full pipeline with drain, enrich, and tail sampling: + +```typescript +import { createAxiomDrain } from 'evlog/axiom' + +routerLayer({ + include: ['/api/**'], + drain: createAxiomDrain(), + enrich: (ctx) => { ctx.event.region = process.env.FLY_REGION }, + keep: (ctx) => { + if (ctx.duration && ctx.duration > 2000) ctx.shouldKeep = true + }, +}) +``` + +### Cloudflare Workers + +```typescript +import { initWorkersLogger, withEvlog } from 'evlog/workers' + +initWorkersLogger({ env: { service: 'edge-api' } }) + +export default withEvlog(async (request, _env, _ctx, log) => { + log.set({ action: 'handle_request' }) + return Response.json({ ok: true }) +}) +``` + +`withEvlog` emits one wide event per request when the handler returns, with no manual `log.emit()`. Async drains are registered with `waitUntil` so they survive the response; streaming responses defer the emit until the body completes. `requestId` comes from `x-request-id` (fallback `cf-ray`); `method`, `path`, `cf-ray`, `traceparent`, and the safe subset of `request.cf` are captured automatically. It accepts the same options (`drain`, `enrich`, `keep`, `include`, `exclude`, `routes`) as every other integration. For manual control (scheduled handlers, queues), `createWorkersLogger(request)` + `log.emit()` remains available. No ALS-based `useLogger()` on Workers, so pass `log` explicitly. + +### AWS Lambda + +Lambda has no HTTP middleware lifecycle, so evlog behaves like standalone TypeScript, with one critical rule: **one logger per invocation**, never a shared module-level logger (Lambda reuses execution environments, so a shared instance leaks fields between invocations). + +```typescript +import { initLogger, createLogger } from 'evlog' + +initLogger({ env: { service: 'my-fn' } }) // once at module load (cold start) + +export async function handler(event: SQSEvent) { + for (const record of event.Records) { + const log = createLogger({ messageId: record.messageId }) + try { + log.set({ queue: { source: record.eventSourceARN } }) + await processMessage(record) + } catch (error) { + log.error(error as Error) + throw error + } finally { + log.emit() + } + } +} +``` + +### Astro + +```typescript +// src/middleware.ts +import { defineMiddleware } from 'astro:middleware' +import { initLogger, createRequestLogger } from 'evlog' + +initLogger({ env: { service: 'my-astro-app' } }) + +export const onRequest = defineMiddleware(async ({ request, locals }, next) => { + const url = new URL(request.url) + const log = createRequestLogger({ method: request.method, path: url.pathname }) + locals.log = log + + try { + const response = await next() + log.emit() + return response + } catch (error) { + log.error(error instanceof Error ? error : new Error(String(error))) + log.emit() + throw error + } +}) +``` + +Type `locals.log` in `src/env.d.ts` (`interface Locals { log: RequestLogger }`). Pair with the Vite plugin (below) for auto-imports and build-time DX. + +### Vite Plugin (any Vite-based framework) + +For any Vite-based project (SvelteKit, Astro, SolidStart, React+Vite, etc.), use the Vite plugin for auto-init, auto-imports, and build-time features: + +```typescript +// vite.config.ts +import evlog from 'evlog/vite' + +export default defineConfig({ + plugins: [ + evlog({ + service: 'my-app', + autoImports: true, // auto-import log, createEvlogError, parseError + strip: ['debug'], // remove log.debug() in production + sourceLocation: true, // inject file:line in dev + prod + client: { // client-side logging + transport: { endpoint: '/api/logs' }, + }, + }), + ], +}) +``` + +Server-side middleware (drain, enrich, keep, routes) is still configured in the framework integration (e.g., `evlog()` middleware for Hono/Express/SvelteKit). The Vite plugin handles build-time DX only. + +### Standalone TypeScript + +```typescript +import { initLogger, createRequestLogger } from 'evlog' + +initLogger({ env: { service: 'my-worker', environment: 'production' } }) + +const log = createRequestLogger({ jobId: job.id }) +log.set({ source: job.source, recordsSynced: 150 }) +log.emit() // Manual emit required in standalone +``` + +--- + +## Configuration Options + +All options work in Nuxt (`evlog` key), Nitro (passed to `evlog()`), Next.js (`createEvlog()`), and standalone (`initLogger()`). + +They can also live in `evlog.config.ts`, written with `defineEvlog()`. Nuxt and Nitro load the file on their own, with module options overriding it. Other frameworks import it and pass `toLoggerConfig(config)` to `initLogger()` and `toMiddlewareOptions(config)` to the middleware. `evlog init` writes the file, `evlog config` prints what it resolves to, and `extends` shares a base config across apps (one level). Docs: https://www.evlog.dev/cli/config + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `env.service` / `service` | `string` | `'app'` | Service name in logs | +| `enabled` | `boolean` | `true` | Global toggle (no-ops when false) | +| `pretty` | `boolean` | `true` in dev | Pretty tree format vs JSON | +| `silent` | `boolean` | `false` | Suppress console output. Events still go to drains | +| `include` | `string[]` | All routes | Route glob patterns to log | +| `exclude` | `string[]` | None | Route patterns to exclude (takes precedence) | +| `routes` | `Record<string, { service }>` | -- | Route-specific service names | +| `minLevel` | `'debug' \| 'info' \| 'warn' \| 'error'` | `'debug'` | Hard threshold for the global `log` API and client `log` (not request wide events). Use `sampling.rates` for probabilistic volume on requests | +| `sampling.rates` | `object` | -- | Head sampling: `{ info: 10, warn: 50 }` (0-100%) | +| `sampling.keep` | `array` | -- | Tail sampling: `[{ status: 400 }, { duration: 1000 }]` | +| `drain` | `(ctx) => void` | -- | Drain callback (Next.js, standalone) | +| `enrich` | `(ctx) => void` | -- | Enrich callback (Next.js) | +| `keep` | `(ctx) => void` | -- | Custom tail sampling callback (Next.js) | +| `redact` | `boolean \| RedactConfig` | `true` in production | Enabled by default in production. `false` to disable. Object for fine-grained control | + +### Nitro Hooks (Nuxt, Nitro v2/v3) + +| Hook | When | Use | +|------|------|-----| +| `evlog:drain` | After enrichment | Send events to external services | +| `evlog:enrich` | After emit, before console output and drain | Add derived context | +| `evlog:emit:keep` | During emit | Custom tail sampling logic | +| `close` | Server shutdown | Flush drain pipeline buffers | + +--- + +## Drain Adapters + +| Adapter | Import | Env Vars | +|---------|--------|----------| +| Axiom | `evlog/axiom` | `AXIOM_API_KEY`, `AXIOM_DATASET` | +| OTLP | `evlog/otlp` | `OTLP_ENDPOINT` (or the standard `OTEL_EXPORTER_OTLP_[LOGS_]ENDPOINT`, `_HEADERS`, `_COMPRESSION`, `_PROTOCOL`, `OTEL_RESOURCE_ATTRIBUTES`). `protocol: 'http/protobuf'` sends binary protobuf. `semanticConventions: true` adds OTel attribute names (`http.*`, `exception.*`, `gen_ai.*`) | +| HyperDX | `evlog/hyperdx` | `HYPERDX_API_KEY` (optional `HYPERDX_OTLP_ENDPOINT`; defaults to `https://in-otel.hyperdx.io`) | +| Sazabi | `evlog/sazabi` | `SAZABI_INTAKE_URL` (the log stream's intake URL; its key is part of the URL), optional `SAZABI_SERVICE_NAME` | +| Grafana Loki | `evlog/loki` | `LOKI_ENDPOINT`, optional `LOKI_API_KEY`, `LOKI_USER` | +| ClickHouse | `evlog/clickhouse` | `CLICKHOUSE_ENDPOINT` (the `evlog_events` table must exist), optional `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, `CLICKHOUSE_DATABASE`, `CLICKHOUSE_TABLE` | +| PostHog | `evlog/posthog` | `POSTHOG_API_KEY`, `POSTHOG_HOST` | +| Sentry | `evlog/sentry` | `SENTRY_DSN` | +| Better Stack | `evlog/better-stack` | `BETTER_STACK_API_KEY` | +| Datadog | `evlog/datadog` | `DD_API_KEY` or `DATADOG_API_KEY`, optional `DD_SITE` / `DATADOG_LOGS_URL` | +| Grafana Loki | `evlog/loki` | `LOKI_ENDPOINT`, optional `LOKI_API_KEY` + `LOKI_USER` (Grafana Cloud) or `LOKI_TENANT_ID` (multi-tenant) | +| ClickHouse | `evlog/clickhouse` | `CLICKHOUSE_ENDPOINT`, optional `CLICKHOUSE_USER` / `CLICKHOUSE_PASSWORD` / `CLICKHOUSE_DATABASE` / `CLICKHOUSE_TABLE` | +| File System | `evlog/fs` | None (local file system) | +| Memory | `evlog/memory` | None (in-process ring buffer; optional `EVLOG_MEMORY_STORE`, `EVLOG_MEMORY_MAX_EVENTS`). Read back with `readMemoryLogs()`, ideal for dev-only log endpoints agents can query | +| NuxtHub | `@evlog/nuxthub` (separate package, Nuxt module) | None: stores wide events in the NuxtHub database with retention-based cleanup (set `evlog.retention: '7d'` in the module options; accepts `d`/`h`/`m`) | +| HTTP (browser ingest) | `evlog/http` | None (configure `endpoint` in code). `evlog/browser` is deprecated; same API, removed next major | + +Use canonical env var names (e.g. `AXIOM_API_KEY`, `BETTER_STACK_API_KEY`), and the same names work in every framework. + +Setup pattern per framework: + +```typescript +// Nuxt/Nitro: server/plugins/evlog-drain.ts +import { createAxiomDrain } from 'evlog/axiom' +export default defineNitroPlugin((nitroApp) => { + nitroApp.hooks.hook('evlog:drain', createAxiomDrain()) +}) + +// Hono / Express / Elysia: pass drain in middleware options +import { createAxiomDrain } from 'evlog/axiom' +app.use(evlog({ drain: createAxiomDrain() })) + +// Fastify: pass drain in plugin options +import { createAxiomDrain } from 'evlog/axiom' +await app.register(evlog, { drain: createAxiomDrain() }) + +// NestJS: pass drain in module options +import { createAxiomDrain } from 'evlog/axiom' +EvlogModule.forRoot({ drain: createAxiomDrain() }) + +// Next.js: pass drain to createEvlog() +import { createAxiomDrain } from 'evlog/axiom' +import { createDrainPipeline } from 'evlog/pipeline' +const pipeline = createDrainPipeline<DrainContext>({ batch: { size: 50 } }) +const drain = pipeline(createAxiomDrain()) +// then: createEvlog({ ..., drain }) + +// Standalone: pass drain to initLogger() +initLogger({ env: { service: 'my-app' }, drain: createAxiomDrain() }) +``` + +See [references/drain-pipeline.md](references/drain-pipeline.md) for batching, retry, and buffer overflow config. + +--- + +## Enrichers + +Built-in: `createUserAgentEnricher()`, `createGeoEnricher()`, `createRequestSizeEnricher()`, `createTraceContextEnricher()`, all from `evlog/enrichers`. Each accepts `{ overwrite?: boolean }` (default `false`). Use `createDefaultEnrichers()` to compose all four in one call: + +```typescript +import { createDefaultEnrichers } from 'evlog/enrichers' +app.use(evlog({ enrich: createDefaultEnrichers() })) +``` + +```typescript +// Nuxt/Nitro: server/plugins/evlog-enrich.ts +import { createUserAgentEnricher, createGeoEnricher } from 'evlog/enrichers' +export default defineNitroPlugin((nitroApp) => { + const enrichers = [createUserAgentEnricher(), createGeoEnricher()] + nitroApp.hooks.hook('evlog:enrich', (ctx) => { + for (const enricher of enrichers) enricher(ctx) + }) +}) + +// Next.js: in lib/evlog.ts +createEvlog({ + enrich: (ctx) => { + for (const enricher of enrichers) enricher(ctx) + ctx.event.region = process.env.VERCEL_REGION + }, +}) +``` + +--- + +## Auto-Redaction (PII Protection) + +Built-in redaction scrubs sensitive data from wide events **before** console output and **before** any drain sees the data. **Enabled by default in production** (`NODE_ENV === 'production'`), disabled in development. Uses **smart partial masking**, preserving enough context for debugging. + +```typescript +// Disable in production (opt-out) +evlog: { redact: false } + +// Add custom paths on top of built-ins +evlog: { + redact: { + paths: ['user.password', 'headers.authorization'], + } +} + +// Only specific built-ins +evlog: { + redact: { + builtins: ['email', 'creditCard'], + } +} + +// No built-ins, only custom (uses flat [REDACTED] replacement) +evlog: { + redact: { + builtins: false, + paths: ['user.ssn'], + patterns: [/SECRET_\w+/g], + } +} +``` + +**Built-in patterns** with smart masking output: + +| Pattern | Example Input | Masked Output | +|---------|---------------|---------------| +| `creditCard` | `4111111111111111` | `****1111` | +| `email` | `alice@example.com` | `a***@***.com` | +| `ipv4` | `192.168.1.100` | `***.***.***.100` | +| `phone` | `+33 6 12 34 56 78` | `+33 ****5678` | +| `jwt` | `eyJhbGciOi...` | `eyJ***.***` | +| `bearer` | `Bearer sk_live_abc...` | `Bearer ***` | +| `iban` | `FR76 3000 6000 ...189` | `FR76****189` | + +**Opt-in credential patterns** match the value under any key and inside longer strings. They are off by default, and `builtins` runs only the names it lists, so add them next to the defaults: + +| Pattern | Example Input | Masked Output | +|---------|---------------|---------------| +| `apiKeys` | `ghp_` followed by 36 characters | `ghp_***` | +| `privateKey` | `-----BEGIN RSA PRIVATE KEY-----` block | `-----BEGIN RSA PRIVATE KEY-----***` | +| `urlPassword` | `postgres://app:hunter2@db` | `postgres://app:***@db` | + +```typescript +evlog: { + redact: { + builtins: ['creditCard', 'email', 'ipv4', 'phone', 'jwt', 'bearer', 'iban', 'secrets', 'apiKeys', 'privateKey', 'urlPassword'], + } +} +``` + +`apiKeys` covers GitHub, npm, Anthropic, OpenAI, Stripe, Slack and AWS key IDs by their fixed prefix. Suggest them when the code logs provider tokens, private keys or connection strings, even under keys that `secrets` does not cover (`apiKey`, `authorization`, `dsn`). + +Works in all frameworks: Nuxt (`evlog` config), Nitro (`evlog()` module options), Next.js (`createEvlog()`), standalone (`initLogger()`), and all middleware integrations (Hono, Express, Fastify, Elysia, NestJS). + +--- + +## AI SDK Integration + +Capture token usage, tool calls, model info, streaming metrics, tool execution timing, cost estimation, and embedding metadata from the Vercel AI SDK into wide events. Import from `evlog/ai`. Requires `ai >=6.0.168 <8.0.0` as a peer dependency. + +### Basic setup (middleware) + +```typescript +import { createAILogger } from 'evlog/ai' + +const log = useLogger(event) // or any RequestLogger +const ai = createAILogger(log) + +const result = streamText({ + model: ai.wrap('anthropic/claude-sonnet-4.6'), // accepts string or model object + messages, +}) +``` + +`ai.wrap()` uses model middleware to transparently capture all LLM calls. Works with `generateText`, `streamText`, and `ToolLoopAgent`. + +### Telemetry integration (deeper observability) + +For tool execution timing, success/failure tracking, and total generation wall time, add `createEvlogIntegration()`: + +```typescript +import { createAILogger, createEvlogIntegration } from 'evlog/ai' + +const ai = createAILogger(log) + +const agent = new ToolLoopAgent({ + model: ai.wrap('anthropic/claude-sonnet-4.6'), + tools: { searchWeb, queryDatabase }, + stopWhen: stepCountIs(5), + telemetry: { + integrations: [createEvlogIntegration(ai)], + }, +}) +``` + +This adds `ai.tools` (per-tool `{ name, durationMs, success, error? }`) and `ai.totalDurationMs` to the wide event. + +### Embeddings + +```typescript +const { embedding, usage } = await embed({ model: embeddingModel, value: query }) +ai.captureEmbed({ usage, model: 'text-embedding-3-small', dimensions: 1536 }) +``` + +For `embedMany`, pass the batch count: + +```typescript +ai.captureEmbed({ usage, model: 'text-embedding-3-small', count: documents.length }) +``` + +### Cost estimation + +Pass a pricing map to get `ai.estimatedCost` in the wide event: + +```typescript +const ai = createAILogger(log, { + cost: { + 'claude-sonnet-4.6': { input: 3, output: 15 }, + 'gpt-4o': { input: 2.5, output: 10 }, + }, +}) +``` + +### Wide event `ai` field + +Includes: `calls`, `model`, `provider`, `inputTokens`, `outputTokens`, `totalTokens`, `cacheReadTokens`, `reasoningTokens`, `finishReason`, `toolCalls`, `steps`, `msToFirstChunk`, `msToFinish`, `tokensPerSecond`, `error`, `tools` (via telemetry integration), `totalDurationMs` (via telemetry integration), `embedding` (via `captureEmbed`), `estimatedCost` (via `cost` option). + +Anti-patterns to detect: + +| Anti-Pattern | Fix | +|--------------|-----| +| Manual token tracking in `onFinish` | `ai.wrap()`: middleware captures automatically | +| `console.log('tokens:', result.usage)` | `ai.wrap()`: structured `ai.*` fields in wide event | +| No AI observability | Add `createAILogger(log)` + `ai.wrap()` | +| No tool execution timing | Add `createEvlogIntegration(ai)` to `telemetry.integrations` | +| Manual cost calculation | Use `cost` option in `createAILogger()` | + +--- + +## Related Capabilities (pointers) + +Recommend these when the review surfaces the matching need. Each has full docs on evlog.dev: + +| Need spotted | evlog answer | Docs | +|--------------|--------------|------| +| Auth'd user identity missing from events (Better Auth apps) | `evlog/better-auth`: `identifyUser()`, `createAuthMiddleware()`, client identity sync | https://www.evlog.dev/use-cases/better-auth/overview | +| Ad-hoc field names drifting across the codebase | Typed fields + error/audit catalogs (`evlog/catalog`) | https://www.evlog.dev/learn/typed-fields · https://www.evlog.dev/learn/catalogs | +| Cross-cutting hooks (request start/finish, client logs, logger extension) | Plugins: `definePlugin` | https://www.evlog.dev/extend/plugins | +| Tail logs live during dev / build a log viewer | `createStreamDrain` (`evlog/stream`, SSE) + `readFsLogs` / `tailFsLogs` (`evlog/fs`) | https://www.evlog.dev/extend/stream | +| Agents need to query logs over HTTP in dev | Memory adapter + `readMemoryLogs()` behind a dev-only endpoint | https://www.evlog.dev/integrate/adapters/self-hosted/memory | + +--- + +## Structured Errors + +```typescript +import { createError } from 'evlog' // required in Nuxt too: a bare createError is h3's + +// Minimal +throw createError({ message: 'Database connection failed', status: 500 }) + +// Standard +throw createError({ message: 'Payment failed', status: 402, why: 'Card declined by issuer' }) + +// Complete +throw createError({ + message: 'Payment failed', + status: 402, + why: 'Card declined by issuer - insufficient funds', + fix: 'Please use a different payment method or contact your bank', + link: 'https://docs.example.com/payments/declined', + cause: originalError, +}) + +// Backend-only context (wide events / drains — never HTTP body or parseError()) +throw createError({ + message: 'Not allowed', + status: 403, + why: 'Insufficient permissions', + internal: { correlationId: 'req_abc', resourceId: 'proj_123' }, +}) +``` + +Frontend: extract user-facing fields with `parseError()` (`internal` is never returned to clients): + +```typescript +import { parseError } from 'evlog' + +const error = parseError(err) +// error.message, error.status, error.why, error.fix, error.link +``` + +See [references/structured-errors.md](references/structured-errors.md) for common patterns and templates. + +--- + +## Anti-Patterns to Detect + +| Anti-Pattern | Fix | +|--------------|-----| +| Multiple `console.log` in one function | Single wide event with `log.set()` | +| `throw new Error('...')` | `throw createError({ message, status, why, fix })` | +| `console.error(e); throw e` | `log.error(e); throw createError(...)` | +| No logging in request handlers | Add `useLogger(event)` / `useLogger()` / `createRequestLogger()` | +| Flat log data `{ uid, n, t }` | Grouped objects: `{ user: {...}, cart: {...} }` | +| Logging sensitive data `log.set({ user: body })` | Explicit fields: `{ user: { id: body.id, plan: body.plan } }` + enable `redact: true` | +| Putting support-only IDs in `why` / `message` | Use `createError({ ..., internal: { ... } })` for non-user-facing diagnostics | + +See [references/code-review.md](references/code-review.md) for the full checklist. + +--- + +## Loading Reference Files + +Load based on what you're working on, and **do not load all at once**: + +- Designing wide events → [references/wide-events.md](references/wide-events.md) +- Improving errors → [references/structured-errors.md](references/structured-errors.md) +- Full code review → [references/code-review.md](references/code-review.md) +- Drain pipeline setup → [references/drain-pipeline.md](references/drain-pipeline.md) diff --git a/plugins/evlog/agent/skills/review-logging-patterns/references/code-review.md b/plugins/evlog/agent/skills/review-logging-patterns/references/code-review.md new file mode 100644 index 00000000..8f379d46 --- /dev/null +++ b/plugins/evlog/agent/skills/review-logging-patterns/references/code-review.md @@ -0,0 +1,342 @@ +# Code Review Checklist + +Use this checklist when reviewing code for logging best practices and evlog adoption. + +## Prefer `evlog map` when you can + +On **Nuxt, Nitro, Next.js App Router, TanStack Start, Hono, Express, and Fastify**, start with the `evlog` CLI (it ships with the `evlog` package) if the user is open to it: one command finds dark entry points and names the fixes: + +```bash +npx evlog map --no-write +npx evlog map <file> --no-write # suggested shape for one entry point +``` + +**Requirements** (move the score) map to the anti-patterns below: + +| Map rule id | Weight | What it expects | Related anti-pattern | +|-------------|--------|-----------------|----------------------| +| `wide-event` | 40 | `useLogger()` / request logger | No logging in handlers | +| `audit` | 25 | `log.audit(...)` on sensitive routes | Missing audit on auth/billing | +| `structured-errors` | 20 | `createError({ why, fix })` | `throw new Error('...')` | +| `page-error-handling` | 20 | fetch error handling on pages | Unhandled page fetches | +| `context` | 15 | `log.set(...)` | Flat / missing request context | +| `error-handling` | 15 | log or rethrow in `catch` | `console.error(e); throw e` | + +**Opportunities** (never cost points; fire only when the project already uses the feature). Surface them as suggestions, not defects: + +| Map rule id | Fires when | Related skill section | +|-------------|-----------|-----------------------| +| `error-catalog` | A catalog is declared and the same inline error appears in 2+ files | Related Capabilities → catalogs | +| `audit-coverage` | The project records audits and a state-changing handler has none | Audit logs | +| `ai-logging` | `ai` is a dependency and the AI SDK is called without `evlog/ai` | AI SDK Integration | +| `auth-identity` | `better-auth` is a dependency without `evlog/better-auth` | Related Capabilities → Better Auth | + +Full rule reference: https://www.evlog.dev/cli/rules + +Map tells you the **shape** is present, not that the context is useful at runtime. Keep the scans below for frameworks without adapters, for quality of context, drains, redaction, and AI SDK usage. If the user skips the CLI, use this checklist alone. + +## Quick Scan + +Run through these checks first to identify improvement opportunities: + +### 1. Console Statement Audit + +Search for these patterns: + +```typescript +// ❌ Patterns to find and transform +console.log(...) +console.error(...) +console.warn(...) +console.info(...) +console.debug(...) +``` + +**Questions to ask:** + +- Are there multiple console statements in one function? +- Are they logging request/response data? +- Could they be consolidated into a wide event? + +### 2. Error Pattern Audit + +Search for these patterns: + +```typescript +// ❌ Generic errors +throw new Error('...') +throw Error('...') + +// ❌ Re-throwing without context +catch (error) { + throw error +} + +// ❌ Logging and throwing +catch (error) { + console.error(error) + throw error +} +``` + +**Questions to ask:** + +- Does the error message explain what happened? +- Is there a `why` explaining the root cause? +- Is there a `fix` suggesting a solution? +- Is the original error preserved as `cause`? + +### 3. Request Handler Audit + +For each API route/handler, check: + +```typescript +// ❌ Missing request context +export default defineEventHandler(async (event) => { + // No logging at all, or scattered console.logs +}) +``` + +**Questions to ask:** + +- Is there a request-scoped logger? +- Is context accumulated throughout the request? +- Is there a single emit at the end? + +## Detailed Review + +### Console.log Transformations + +#### Single Debug Log + +```typescript +// ❌ Before +console.log('Processing user:', userId) + +// ✅ After - if part of a larger operation +log.set({ user: { id: userId } }) + +// ✅ After - if standalone debug +log.debug('user', `Processing user ${userId}`) +``` + +#### Multiple Related Logs + +```typescript +// ❌ Before +console.log('Starting checkout') +console.log('User:', user.id) +console.log('Cart items:', cart.items.length) +console.log('Total:', cart.total) + +// ✅ After +log.info({ + action: 'checkout', + user: { id: user.id }, + cart: { items: cart.items.length, total: cart.total }, +}) +``` + +#### Request Lifecycle Logs + +```typescript +// server/api/process.post.ts + +// ❌ Before +export default defineEventHandler(async (event) => { + console.log('Request started') + const user = await getUser(event) + console.log('User loaded') + const result = await processData(user) + console.log('Processing complete') + return result +}) + +// ✅ After (Nuxt - auto-imported, no import needed) +// For Nitro v3: import { useLogger } from 'evlog/nitro/v3' +// For Nitro v2: import { useLogger } from 'evlog/nitro' + +export default defineEventHandler(async (event) => { + const log = useLogger(event) + + const user = await getUser(event) + log.set({ user: { id: user.id } }) + + const result = await processData(user) + log.set({ result: { id: result.id } }) + + return result + // emit() called automatically +}) +``` + +### Error Transformations + +#### Generic Error + +```typescript +// ❌ Before +throw new Error('Failed to create user') + +// ✅ After +throw createError({ + message: 'Failed to create user', + why: 'Email address already registered', + fix: 'Use a different email or log in to existing account', + link: 'https://your-app.com/docs/registration', +}) +``` + +#### Wrapped Error Without Context + +```typescript +// ❌ Before +try { + await externalApi.call() +} catch (error) { + throw new Error('API call failed') +} + +// ✅ After +try { + await externalApi.call() +} catch (error) { + throw createError({ + message: 'External API call failed', + why: `API returned: ${error.message}`, + fix: 'Check API credentials and try again', + link: 'https://api-docs.example.com/errors', + cause: error, + }) +} +``` + +#### Log-and-Throw Anti-pattern + +```typescript +// ❌ Before +try { + await riskyOperation() +} catch (error) { + console.error('Operation failed:', error) + throw error +} + +// ✅ After +try { + await riskyOperation() +} catch (error) { + log.error(error, { step: 'riskyOperation' }) + throw createError({ + message: 'Operation failed', + why: error.message, + fix: 'Check input and retry', + cause: error, + }) +} +``` + +### Request Handler Transformations + +#### No Logging + +```typescript +// server/api/orders.post.ts + +// ❌ Before +export default defineEventHandler(async (event) => { + const body = await readBody(event) + const result = await processOrder(body) + return result +}) + +// ✅ After (Nuxt - auto-imported, no import needed) +// For Nitro v3: import { useLogger } from 'evlog/nitro/v3' +// For Nitro v2: import { useLogger } from 'evlog/nitro' +import { createError } from 'evlog' + +export default defineEventHandler(async (event) => { + const log = useLogger(event) + + const body = await readBody(event) + log.set({ order: { items: body.items?.length } }) + + try { + const result = await processOrder(body) + log.set({ result: { orderId: result.id, status: result.status } }) + return result + } catch (error) { + log.error(error, { step: 'processOrder' }) + throw createError({ + message: 'Order processing failed', + why: error.message, + fix: 'Check the order data and try again', + }) + } + // emit() called automatically +}) +``` + +## Review Checklist Summary + +### Logging + +- [ ] No raw `console.log` statements in production code +- [ ] Request handlers use `useLogger(event)` (Nuxt/Nitro) or `createRequestLogger()` (standalone) +- [ ] Context is accumulated with `log.set()` throughout the request +- [ ] `emit()` is automatic with `useLogger()`, manual with `createRequestLogger()` +- [ ] Wide events include: user, business context, outcome + +### Errors + +- [ ] All errors use `createError()` instead of `new Error()` (import from `evlog`) +- [ ] Every error has a clear `message` and appropriate `status` code +- [ ] Complex errors include `why` explaining root cause +- [ ] Fixable errors include `fix` with actionable steps +- [ ] Documented errors include `link` to docs +- [ ] Wrapped errors preserve `cause` +- [ ] Support-only or sensitive diagnostics use `internal`, not `message` / `why` / `fix` + +### Frontend Error Handling + +- [ ] API errors are caught and displayed with full context (message, why, fix) +- [ ] Toasts or error components use the structured data from `error.data.data` +- [ ] Links to documentation are actionable (buttons/links in toasts) + +### Context + +- [ ] User context includes: id, plan/subscription, relevant business data +- [ ] Request context includes: method, path, requestId +- [ ] Business context is domain-specific and useful for debugging +- [ ] No sensitive data in logs (passwords, tokens, full card numbers) + +## Anti-Pattern Summary + +| Anti-Pattern | Fix | +|--------------|-----| +| Multiple `console.log` in one function | Single wide event with `useLogger(event).set()` | +| `throw new Error('...')` | `throw createError({ message, status, why, fix })` | +| `console.error(e); throw e` | `log.error(e); throw createError(...)` | +| No logging in request handlers | Add `useLogger(event)` (Nuxt/Nitro) or `createRequestLogger()` (standalone) | +| Flat log data | Grouped objects: `{ user: {...}, cart: {...} }` | +| Abbreviated field names | Descriptive names: `userId` not `uid` | + +## Suggested Review Comments + +Use these when leaving review feedback: + +### Console.log Found + +> Consider using evlog's wide event pattern here. Instead of multiple console.log statements, use `useLogger(event)` to accumulate context and emit a single comprehensive event. + +### Generic Error + +> This error would benefit from evlog's structured error pattern. Consider using `import { createError } from 'evlog'` and `createError({ message, status, why, fix })` to provide more debugging context. + +### Missing Request Context + +> This handler would benefit from request-scoped logging. Add `useLogger(event)` at the start to capture context throughout the request lifecycle. + +### Good Logging (Positive Feedback) + +> Nice use of wide events here! The context is well-structured and will be very useful for debugging. diff --git a/plugins/evlog/agent/skills/review-logging-patterns/references/drain-pipeline.md b/plugins/evlog/agent/skills/review-logging-patterns/references/drain-pipeline.md new file mode 100644 index 00000000..2f1748f9 --- /dev/null +++ b/plugins/evlog/agent/skills/review-logging-patterns/references/drain-pipeline.md @@ -0,0 +1,166 @@ +# Drain Pipeline Reference + +The drain pipeline wraps any adapter to add batching, retry with backoff, and buffer overflow protection. Use it in production to reduce network overhead and handle transient failures. + +## When to Recommend + +- The user has a drain adapter (Axiom, OTLP, custom) and is deploying to production +- High-throughput scenarios where one HTTP request per event is wasteful +- The user needs retry logic for unreliable backends +- The user is implementing batching manually with `setInterval` and arrays + +## Basic Setup + +```typescript +// server/plugins/evlog-drain.ts +import type { DrainContext } from 'evlog' +import { createDrainPipeline } from 'evlog/pipeline' +import { createAxiomDrain } from 'evlog/axiom' + +export default defineNitroPlugin((nitroApp) => { + const pipeline = createDrainPipeline<DrainContext>() + const drain = pipeline(createAxiomDrain()) + + nitroApp.hooks.hook('evlog:drain', drain) + nitroApp.hooks.hook('close', () => drain.flush()) +}) +``` + +**Important:** Always call `drain.flush()` on server `close` hook. Without it, buffered events are lost when the process exits. + +## Full Configuration + +```typescript +const pipeline = createDrainPipeline<DrainContext>({ + batch: { + size: 50, // Max events per batch (default: 50) + intervalMs: 5000, // Max wait before flushing partial batch (default: 5000) + }, + retry: { + maxAttempts: 3, // Total attempts including first (default: 3) + backoff: 'exponential', // 'exponential' | 'linear' | 'fixed' (default: 'exponential') + initialDelayMs: 1000, // Base delay for first retry (default: 1000) + maxDelayMs: 30000, // Upper bound for any retry delay (default: 30000) + }, + maxBufferSize: 1000, // Max buffered events; oldest dropped on overflow (default: 1000) + onDropped: (events, error) => { + // Called when events are dropped (overflow or retry exhaustion) + console.error(`[evlog] Dropped ${events.length} events:`, error?.message) + }, +}) +``` + +## How It Works + +1. `drain(ctx)` pushes a single event into the buffer +2. When `buffer.length >= batch.size`, the batch is flushed immediately +3. If the batch isn't full, a timer starts; after `intervalMs`, whatever is buffered gets flushed +4. On flush, the drain function receives `T[]` (always an array) +5. Adapter drains built with `defineDrain` / `defineHttpDrain` are unwrapped to their throwing `raw` variant, which defaults to one HTTP attempt per pipeline attempt: the pipeline owns retries, unless `retries` is explicitly configured on the adapter, which then still applies inside each pipeline attempt +6. If the drain throws, the batch is retried with the configured backoff +7. After `maxAttempts` failures, `onDropped` is called and the batch is discarded (logged when no `onDropped` is configured) +8. If the buffer exceeds `maxBufferSize`, the oldest event is dropped and `onDropped` is called + +## Backoff Strategies + +| Strategy | Delay Pattern | Best For | +|----------|--------------|----------| +| `exponential` | 1s, 2s, 4s, 8s... | Default. Transient failures needing recovery time | +| `linear` | 1s, 2s, 3s, 4s... | Predictable delay growth | +| `fixed` | 1s, 1s, 1s, 1s... | Rate-limited APIs with known cooldown | + +## Returned Drain Function API + +```typescript +const drain = pipeline(myDrainFn) + +drain(ctx) // Push a single event (synchronous, non-blocking) +await drain.flush() // Force-flush all buffered events +await drain.settled() // Resolves once everything buffered is delivered or dropped, without forcing a flush +drain.pending // Number of events currently buffered (readonly) +``` + +## Serverless + +`drain(ctx)` returns after buffering, so delivery must outlive the response. When the pipeline is passed as the `drain` option of a framework integration, the shared middleware registers `drain.settled()` with the runtime's `waitUntil` (Next `after()`, Workers `ctx.waitUntil`) automatically: the instance stays alive until the batch is delivered or dropped. Next.js global drains configured through `createInstrumentation()` start inside `after()` and await `drain.settled()` after enqueueing. `batch.intervalMs` bounds when the first delivery attempt starts; retry backoff can extend the lifetime past it. Nitro's `evlog:drain` hook path can't do this for you; on Cloudflare register `drain.settled()` with the execution context's `waitUntil` where the hook is wired. + +## Common Patterns + +### With multiple adapters + +```typescript +const axiom = createAxiomDrain() +const otlp = createOTLPDrain() + +const pipeline = createDrainPipeline<DrainContext>() +const drain = pipeline(async (batch) => { + await Promise.allSettled([axiom(batch), otlp(batch)]) +}) +``` + +### Custom drain function + +```typescript +const pipeline = createDrainPipeline<DrainContext>({ batch: { size: 100 } }) +const drain = pipeline(async (batch) => { + await fetch('https://your-service.com/logs', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(batch.map(ctx => ctx.event)), + }) +}) +``` + +### Low-traffic with longer interval + +```typescript +const pipeline = createDrainPipeline<DrainContext>({ + batch: { size: 10, intervalMs: 30000 }, // Flush every 30s or 10 events +}) +``` + +## Anti-Patterns + +### Manual batching with setInterval + +```typescript +// ❌ No retry, no overflow protection, no flush on shutdown +const batch: WideEvent[] = [] +setInterval(() => { + if (batch.length > 0) fetch(...) +}, 5000) +``` + +**Transform to:** + +```typescript +// ✅ Use the pipeline +const pipeline = createDrainPipeline<DrainContext>() +const drain = pipeline(async (batch) => { await fetch(...) }) +nitroApp.hooks.hook('close', () => drain.flush()) +``` + +### Missing flush on shutdown + +```typescript +// ❌ Buffered events lost on process exit +nitroApp.hooks.hook('evlog:drain', drain) +``` + +**Fix:** + +```typescript +// ✅ Always flush on close +nitroApp.hooks.hook('evlog:drain', drain) +nitroApp.hooks.hook('close', () => drain.flush()) +``` + +## Review Checklist + +- [ ] Pipeline wraps the adapter for production use +- [ ] `drain.flush()` called on server `close` hook (long-lived servers) +- [ ] On serverless, the pipeline is passed as the integration's `drain` option (settled() registration is automatic); Nitro-on-Cloudflare wires `drain.settled()` into `waitUntil` manually +- [ ] `onDropped` callback logs or reports dropped events +- [ ] Retries are owned by one layer: no explicit `retries` on the adapter config when the pipeline wraps it +- [ ] Batch size and interval are appropriate for the traffic volume +- [ ] `maxBufferSize` is set to prevent memory leaks under load diff --git a/plugins/evlog/agent/skills/review-logging-patterns/references/structured-errors.md b/plugins/evlog/agent/skills/review-logging-patterns/references/structured-errors.md new file mode 100644 index 00000000..3d841909 --- /dev/null +++ b/plugins/evlog/agent/skills/review-logging-patterns/references/structured-errors.md @@ -0,0 +1,466 @@ +# Structured Errors Guide + +Structured errors provide context that helps developers understand **what** happened, **why** it happened, and **how to fix it**. + +## The Problem with Generic Errors + +```typescript +// ❌ Useless errors +throw new Error('Something went wrong') +throw new Error('Failed') +throw new Error('Invalid input') + +// ❌ Missing context +throw new Error('Payment failed') // Why? How do I fix it? +``` + +When these errors reach your logs or monitoring, you have no idea: + +- What actually failed +- Why it failed +- How to fix it +- Where to find more information + +## Structured Error Anatomy + +```typescript +import { createError } from 'evlog' + +throw createError({ + message: 'Payment failed', // What happened + status: 402, // HTTP status code + why: 'Card declined by issuer', // Why it happened + fix: 'Try a different payment method', // How to fix it + link: 'https://docs.example.com/...', // More information + cause: originalError, // Original error + data: { // Optional: extra payload for the client + orderId: 'ord_8x2k', + retryAfter: 30, + }, + internal: { // Optional: backend / logs only + correlationId: 'pay_abc', + processorCode: 'card_declined', + }, +}) +``` + +### `data` (client payload) + +- Use `data` for values the client branches on, an order id, a retry deadline, a list of invalid fields. `why`, `fix`, and `link` stay for the strings a human reads. +- The payload is merged into the response body's `data` object next to `code`, `why`, `fix`, and `link`, and those four win on a key collision. Read it client-side as `parseError(error).data`. +- Same field name as h3's `createError({ data })`, and evlog's Nitro error handler forwards it for h3 errors too. + +### `internal` (backend-only) + +- Use `internal` for IDs, gateway codes, or diagnostics that **must not** appear in HTTP error bodies or in client-side `parseError()` results. +- Access in server code via **`error.internal`**. Values are omitted from **`toJSON()`** and from framework serializers; they are included on wide events under **`error.internal`** when the error is captured with **`log.error()`** (or equivalent automatic capture). +- Stored with a non-enumerable symbol so `JSON.stringify(error)` does not leak `internal`; devtools may show it as `[Symbol(evlog.error.internal)]`. + +### Console Output (Development) + +``` +Error: Payment failed +Why: Card declined by issuer +Fix: Try a different payment method +More info: https://docs.example.com/payments/declined + +Caused by: StripeCardError: card_declined +``` + +### JSON Output (Production) + +`EvlogError.toJSON()`, returned as-is by the Next.js handler: + +```json +{ + "name": "EvlogError", + "message": "Payment failed", + "status": 402, + "data": { + "orderId": "ord_8x2k", + "retryAfter": 30, + "why": "Card declined by issuer", + "fix": "Try a different payment method", + "link": "https://docs.example.com/payments/declined" + }, + "cause": { + "name": "StripeCardError", + "message": "card_declined" + } +} +``` + +The Nitro, Nuxt, and SvelteKit handlers return a Nitro-shaped body instead, with `url`, `statusCode`, `statusText`, and `error: true` around the same `message`, `status`, and `data`. Every serializer agrees on the part clients read: `why`, `fix`, and `link` sit under `data`, not at the top level, and neither `internal` nor `stack` is ever included. On the client, `parseError()` flattens both shapes. + +## Field Guidelines + +### `message` - What Happened + +User-facing description of what went wrong. + +```typescript +// ✅ Good - clear, actionable +message: 'Failed to sync repository' +message: 'Unable to process payment' +message: 'User not found' + +// ❌ Bad - vague, unhelpful +message: 'Error' +message: 'Something went wrong' +message: 'Failed' +``` + +### `why` - Why It Happened + +Technical explanation for debugging. + +```typescript +// ✅ Good - specific, technical +why: 'GitHub API rate limit exceeded (403)' +why: 'Card declined by issuer: insufficient_funds' +why: 'No user with ID "user_123" exists in database' + +// ❌ Bad - just restating the message +why: 'It failed' +why: 'Error occurred' +``` + +### `fix` - How to Fix It + +Actionable steps to resolve the issue. + +```typescript +// ✅ Good - specific actions +fix: 'Wait 1 hour or use a different API token' +fix: 'Use a different payment method or contact your bank' +fix: 'Check the user ID and try again' + +// ❌ Bad - not actionable +fix: 'Fix the error' +fix: 'Try again' +``` + +### `link` - More Information + +Documentation URL for detailed troubleshooting. + +```typescript +// ✅ Good - specific documentation +link: 'https://docs.github.com/en/rest/rate-limit' +link: 'https://docs.stripe.com/declines/codes' +link: 'https://your-app.com/docs/errors/user-not-found' +``` + +### `cause` - Original Error + +The underlying error that triggered this one. + +```typescript +try { + await stripe.charges.create(...) +} catch (error) { + throw createError({ + message: 'Payment failed', + why: `Stripe error: ${error.code}`, + fix: 'Contact support with error code', + cause: error, // Preserves original stack trace + }) +} +``` + +## Common Error Patterns + +### API/External Service Errors + +```typescript +// Rate limiting +throw createError({ + message: 'GitHub sync temporarily unavailable', + status: 429, + why: 'API rate limit exceeded (5000/hour)', + fix: 'Wait until rate limit resets or use authenticated requests', + link: 'https://docs.github.com/en/rest/rate-limit', + cause: error, +}) + +// Authentication +throw createError({ + message: 'Unable to connect to Stripe', + status: 503, + why: 'Invalid API key provided', + fix: 'Check STRIPE_SECRET_KEY environment variable', + link: 'https://docs.stripe.com/keys', + cause: error, +}) + +// Network +throw createError({ + message: 'Failed to fetch user data', + status: 504, + why: 'Connection timeout after 30s', + fix: 'Check network connectivity and try again', + cause: error, +}) +``` + +### Validation Errors + +```typescript +// Missing required field +throw createError({ + message: 'Invalid checkout request', + status: 400, + why: 'Required field "email" is missing', + fix: 'Include a valid email address in the request body', + link: 'https://your-api.com/docs/checkout#request-body', +}) + +// Invalid format +throw createError({ + message: 'Invalid email format', + status: 422, + why: `"${email}" is not a valid email address`, + fix: 'Provide an email in the format user@example.com', +}) + +// Business rule violation +throw createError({ + message: 'Cannot cancel subscription', + status: 409, + why: 'Subscription has already been cancelled', + fix: 'No action needed - subscription is already inactive', +}) +``` + +### Database Errors + +```typescript +// Not found +throw createError({ + message: 'User not found', + status: 404, + why: `No user with ID "${userId}" exists`, + fix: 'Verify the user ID is correct', +}) + +// Constraint violation +throw createError({ + message: 'Cannot create duplicate account', + status: 409, + why: `User with email "${email}" already exists`, + fix: 'Use a different email or log in to existing account', + link: 'https://your-app.com/login', +}) + +// Connection +throw createError({ + message: 'Database unavailable', + status: 503, + why: 'Connection pool exhausted', + fix: 'Reduce concurrent connections or increase pool size', + cause: error, +}) +``` + +### Permission Errors + +```typescript +throw createError({ + message: 'Access denied', + status: 403, + why: 'User lacks "admin" role required for this action', + fix: 'Contact an administrator to request access', + link: 'https://your-app.com/docs/permissions', +}) +``` + +## Transformation Examples + +### Before: Generic Error + +```typescript +async function processPayment(cart, user) { + try { + return await stripe.charges.create({ + amount: cart.total, + currency: 'usd', + source: user.paymentMethodId, + }) + } catch (error) { + throw new Error('Payment failed') // ❌ No context + } +} +``` + +### After: Structured Error + +```typescript +async function processPayment(cart, user) { + try { + return await stripe.charges.create({ + amount: cart.total, + currency: 'usd', + source: user.paymentMethodId, + }) + } catch (error) { + throw createError({ + message: 'Payment failed', + why: getStripeErrorReason(error), + fix: getStripeErrorFix(error), + link: 'https://docs.stripe.com/declines/codes', + cause: error, + }) + } +} + +function getStripeErrorReason(error) { + const reasons = { + card_declined: 'Card was declined by the issuer', + insufficient_funds: 'Card has insufficient funds', + expired_card: 'Card has expired', + // ... + } + return reasons[error.code] ?? `Stripe error: ${error.code}` +} + +function getStripeErrorFix(error) { + const fixes = { + card_declined: 'Try a different payment method or contact your bank', + insufficient_funds: 'Use a different card or add funds', + expired_card: 'Update your card details with a valid expiration date', + // ... + } + return fixes[error.code] ?? 'Contact support with error code' +} +``` + +## Integration with Wide Events + +Structured errors integrate seamlessly with wide events: + +```typescript +// server/api/checkout.post.ts +// Nuxt: useLogger is auto-imported, createError is not +// Nitro v3: import { useLogger } from 'evlog/nitro/v3' +// Nitro v2: import { useLogger } from 'evlog/nitro' +import { createError } from 'evlog' + +export default defineEventHandler(async (event) => { + const log = useLogger(event) + + try { + // ... business logic ... + } catch (error) { + // EvlogError fields are automatically captured + log.error(error, { step: 'payment' }) + throw createError({ + message: 'Payment failed', + why: error.message, + fix: 'Try a different payment method', + }) + } + // emit() called automatically +}) +``` + +The wide event will include: + +```json +{ + "error": { + "name": "EvlogError", + "message": "Payment failed", + "why": "Card declined by issuer", + "fix": "Try a different payment method", + "link": "https://docs.stripe.com/declines/codes", + "internal": { + "stripeRequestId": "req_123" + } + }, + "step": "payment" +} +``` + +If you use `createError({ ..., internal: { ... } })` without calling `log.error(error)` yourself, framework integrations that attach thrown errors to the wide event still merge **`internal`** into **`error.internal`** on emit. + +## Best Practices + +### Do + +- Always provide `message` and `why` at minimum +- Include `fix` when there's an actionable solution +- Add `link` to documentation for complex errors +- Preserve `cause` when wrapping errors +- Be specific about what failed and why +- Put operator-only or sensitive diagnostics in `internal`, not in `why`/`fix`/`message` + +### Don't + +- Use generic messages like "Error" or "Failed" +- Leak sensitive data (passwords, tokens, PII) +- Expect `internal` in HTTP JSON or in `parseError()`. It is for server logs and drains only +- Make `why` and `message` identical +- Suggest fixes that aren't actually possible +- Create errors without any context + +## Nitro Compatibility + +evlog errors work with any Nitro-powered framework. When thrown in an API route, the error is automatically converted to an HTTP response: + +```typescript +// Backend - just throw +throw createError({ + message: 'Payment failed', + status: 402, + why: 'Card declined', + fix: 'Try another card', + link: 'https://docs.example.com/payments', +}) + +// HTTP Response: +// Status: 402 +// Body: { +// statusCode: 402, +// message: "Payment failed", +// data: { why: "Card declined", fix: "Try another card", link: "..." } +// } +``` + +### Frontend Integration + +Use `parseError()` to extract all fields at the top level: + +```typescript +import { parseError } from 'evlog' + +try { + await $fetch('/api/checkout') +} catch (err) { + const error = parseError(err) + + // Direct access: error.message, error.why, error.fix, error.link + toast.add({ + title: error.message, + description: error.why, + color: 'error', + actions: error.link + ? [{ label: 'Learn more', onClick: () => window.open(error.link) }] + : undefined, + }) + + if (error.fix) console.info(`💡 Fix: ${error.fix}`) +} +``` + +**The difference**: A generic error shows "An error occurred". A structured error shows the message, explains why, suggests a fix, and links to documentation. + +## Error Message Templates + +Common patterns, with the fields adapted to each case: + +| Pattern | Status | Fields | +|---------|--------|--------| +| Resource not found | 404 | `why`: what's missing, `fix`: verify identifier | +| External service failure | 503 | `why`: service error, `fix`: actionable step, `link`: service docs, `cause`: original error | +| Validation failure | 400 | `why`: what's invalid, `fix`: expected format | +| Permission denied | 403 | `why`: what's required, `fix`: how to get access | diff --git a/plugins/evlog/agent/skills/review-logging-patterns/references/wide-events.md b/plugins/evlog/agent/skills/review-logging-patterns/references/wide-events.md new file mode 100644 index 00000000..b54bc0e2 --- /dev/null +++ b/plugins/evlog/agent/skills/review-logging-patterns/references/wide-events.md @@ -0,0 +1,338 @@ +# Wide Events Guide + +Wide events are comprehensive log entries that capture all context for a single logical operation (usually a request) in one place. + +## Why Wide Events? + +Traditional logging scatters information across many log lines: + +``` +10:23:45.001 Request received POST /checkout +10:23:45.012 User authenticated: user_123 +10:23:45.045 Cart loaded: 3 items, $99.99 +10:23:45.089 Payment initiated: Stripe +10:23:45.234 Payment failed: card_declined +10:23:45.235 Request completed: 500 +``` + +During an incident, you're grep-ing through thousands of these trying to reconstruct what happened. + +**Wide events emit once with everything:** + +Development (pretty format): +``` +10:23:45.235 ERROR [api] POST /checkout 500 in 234ms + ├─ user: id=user_123 plan=premium accountAge=847 + ├─ cart: items=3 total=9999 + ├─ payment: provider=stripe method=card + └─ error: code=card_declined retriable=false +``` + +Production (JSON format): +```json +{ + "timestamp": "2025-01-24T10:23:45.235Z", + "level": "error", + "service": "api", + "method": "POST", + "path": "/checkout", + "duration": "234ms", + "user": { "id": "user_123", "plan": "premium", "accountAge": 847 }, + "cart": { "items": 3, "total": 9999 }, + "payment": { "provider": "stripe", "method": "card" }, + "error": { "code": "card_declined", "retriable": false } +} +``` + +## When to Use Wide Events + +| Scenario | Use Wide Event? | +|----------|----------------| +| HTTP request handling | Yes - one event per request | +| Background job execution | Yes - one event per job | +| Database query | No - use simple log | +| Cache hit/miss | No - include in parent wide event | +| User action (login, checkout) | Yes - one event per action | +| Debug statements | No - remove in production | + +## Required Fields + +Every wide event should include: + +### Request Context + +```typescript +log.set({ + method: 'POST', + path: '/api/checkout', + requestId: 'req_abc123', // For tracing + traceId: 'trace_xyz', // Distributed tracing +}) +``` + +### User Context + +```typescript +log.set({ + user: { + id: 'user_123', + plan: 'premium', // Business-relevant + accountAge: 847, // Days since signup + subscription: 'annual', + } +}) +``` + +### Business Context + +Add domain-specific data relevant to the operation: + +```typescript +// E-commerce checkout +log.set({ + cart: { id: 'cart_xyz', items: 3, total: 9999 }, + payment: { method: 'card', provider: 'stripe' }, + order: { id: 'order_123', status: 'created' }, +}) + +// API rate limiting +log.set({ + rateLimit: { + limit: 1000, + remaining: 42, + resetAt: '2025-01-24T11:00:00Z', + } +}) + +// File upload +log.set({ + upload: { + filename: 'document.pdf', + size: 1024000, + mimeType: 'application/pdf', + } +}) +``` + +### Outcome + +```typescript +// Success +log.set({ + status: 200, + // duration is added automatically by emit() +}) + +// Error +log.error(error, { + step: 'payment', + retriable: false, +}) +``` + +## Pattern: Request Logger in API Routes + +### Nuxt/Nitro (Recommended) + +With the evlog module, use `useLogger(event)` - it's auto-created and auto-emitted: + +```typescript +// server/api/checkout.post.ts +// Nuxt: useLogger is auto-imported, createError is not +// Nitro v3: import { useLogger } from 'evlog/nitro/v3' +// Nitro v2: import { useLogger } from 'evlog/nitro' +import { createError } from 'evlog' + +export default defineEventHandler(async (event) => { + const log = useLogger(event) // Auto-created by evlog + + const user = await requireAuth(event) + log.set({ user: { id: user.id, plan: user.plan } }) + + const cart = await getCart(user.id) + log.set({ cart: { items: cart.items.length, total: cart.total } }) + + try { + const payment = await processPayment(cart, user) + log.set({ payment: { id: payment.id, method: payment.method } }) + } catch (error) { + log.error(error, { step: 'payment' }) + throw createError({ + message: 'Payment failed', + why: error.message, + fix: 'Try a different payment method', + }) + } + + const order = await createOrder(cart, user) + log.set({ order: { id: order.id, status: order.status } }) + + return order + // log.emit() is called automatically at request end +}) +``` + +### Standalone TypeScript (Scripts, Workers) + +Without Nuxt/Nitro, use `createRequestLogger()` and call `emit()` manually: + +```typescript +// scripts/sync-job.ts +import { initLogger, createRequestLogger } from 'evlog' + +initLogger({ env: { service: 'sync-worker', environment: 'production' } }) + +async function processJob(job: Job) { + const log = createRequestLogger({ jobId: job.id, type: 'sync' }) + + try { + log.set({ source: job.source, target: job.target }) + + const result = await performSync(job) + log.set({ recordsSynced: result.count }) + + return result + } catch (error) { + log.error(error, { step: 'sync' }) + throw error + } finally { + log.emit() // Manual emit required + } +} +``` + +## Transformation Examples + +### Before: Console.log Spam + +```typescript +// server/api/checkout.post.ts + +export default defineEventHandler(async (event) => { + console.log('Checkout started') + + const user = await getUser(event) + console.log('User loaded:', user.id) + + const cart = await getCart(user.id) + console.log('Cart loaded:', cart.items.length, 'items') + + try { + const payment = await processPayment(cart) + console.log('Payment successful:', payment.id) + return { orderId: payment.orderId } + } catch (error) { + console.error('Payment failed:', error.message) + throw error + } +}) +``` + +### After: Single Wide Event + +```typescript +// server/api/checkout.post.ts +// Nuxt: useLogger is auto-imported, createError is not +// Nitro v3: import { useLogger } from 'evlog/nitro/v3' +// Nitro v2: import { useLogger } from 'evlog/nitro' +import { createError } from 'evlog' + +export default defineEventHandler(async (event) => { + const log = useLogger(event) + + const user = await getUser(event) + log.set({ user: { id: user.id, plan: user.plan } }) + + const cart = await getCart(user.id) + log.set({ cart: { items: cart.items.length, total: cart.total } }) + + try { + const payment = await processPayment(cart) + log.set({ payment: { id: payment.id }, order: { id: payment.orderId } }) + + return { orderId: payment.orderId } + } catch (error) { + log.error(error, { step: 'payment' }) + throw createError({ + message: 'Payment failed', + why: error.message, + fix: 'Try a different payment method', + }) + } + // emit() called automatically +}) +``` + +## Best Practices + +### Do + +- Include business-relevant context (user plan, cart value, etc.) +- Add enough context to debug without looking elsewhere +- Use consistent field names across your codebase +- Let `emit()` calculate duration automatically + +### Don't + +- Log sensitive data (passwords, tokens, full credit card numbers) +- Create multiple wide events for one logical operation +- Forget to call `emit()` (or use the Nuxt module for auto-emit) +- Include debugging logs inside wide events (remove them) + +## Security: Preventing Sensitive Data Leakage + +Always explicitly select which fields to log: + +```typescript +// ❌ DANGEROUS - logs everything including password +log.set({ user: body }) + +// ✅ SAFE - explicitly select fields +log.set({ + user: { + id: body.id, + email: maskEmail(body.email), + // password: body.password ← NEVER include + }, +}) +``` + +**Never log:** passwords, API keys, tokens, secrets, full card numbers, CVV, SSN, PII, session tokens, JWTs. + +**Sanitization helpers:** + +```typescript +// server/utils/sanitize.ts +export function maskEmail(email: string): string { + const [local, domain] = email.split('@') + if (!domain) return '***' + return `${local[0]}***@${domain[0]}***.${domain.split('.')[1]}` +} + +export function maskCard(card: string): string { + return `****${card.slice(-4)}` +} +``` + +See [code-review.md](code-review.md) for the full security review checklist. + +## Field Naming Conventions + +Use consistent, descriptive field names: + +```typescript +// ✅ Good - grouped, descriptive +log.set({ + user: { id, plan, accountAge }, + cart: { items, total }, + payment: { method, provider }, +}) + +// ❌ Bad - flat, abbreviated +log.set({ + uid: '123', + n: 3, + t: 9999, + pm: 'card', +}) +``` diff --git a/plugins/evlog/skills-lock.json b/plugins/evlog/skills-lock.json index 05fc75cc..5a43add8 100644 --- a/plugins/evlog/skills-lock.json +++ b/plugins/evlog/skills-lock.json @@ -17,7 +17,7 @@ "source": "evloghq/evlog", "sourceType": "github", "skillPath": "skills/review-logging-patterns/SKILL.md", - "computedHash": "96858a9f88502382975c0b5e84f04561ddfaed29bc26ae27d03e6ff38440e246" + "computedHash": "af44df7af91eed44d0929c3b530e23049bf40aee2794d838be87027eedf66b55" } } } diff --git a/plugins/firebase/.agents/skills/firebase-app-hosting-basics/SKILL.md b/plugins/firebase/.agents/skills/firebase-app-hosting-basics/SKILL.md index e311b203..f3d77d9d 100644 --- a/plugins/firebase/.agents/skills/firebase-app-hosting-basics/SKILL.md +++ b/plugins/firebase/.agents/skills/firebase-app-hosting-basics/SKILL.md @@ -70,8 +70,8 @@ This is the recommended flow for most users. npx -y firebase-tools@latest apphosting:backends:create --backend my-app-id --primary-region us-central1 --root-dir / ``` -1. If the app needs safe access to sensitive keys, set and grant access after - creating the backend: +1. For sensitive keys/secrets (e.g., from `.env*`), set and grant access after + creating the backend and reference via `secret:` (never `value:`): ```bash printf '%s' '<value>' | npx -y firebase-tools@latest apphosting:secrets:set <secret-name> --data-file - --force diff --git a/plugins/firebase/.agents/skills/firebase-app-hosting-basics/references/configuration.md b/plugins/firebase/.agents/skills/firebase-app-hosting-basics/references/configuration.md index 1307f551..e932387f 100644 --- a/plugins/firebase/.agents/skills/firebase-app-hosting-basics/references/configuration.md +++ b/plugins/firebase/.agents/skills/firebase-app-hosting-basics/references/configuration.md @@ -51,7 +51,8 @@ Defines environment variables available during build and/or runtime (local `.env*` files are not uploaded, so declare required variables or secrets here). - `variable`: The name of the env var (e.g., `NEXT_PUBLIC_API_URL`). -- `value`: A literal string value. +- `value`: A literal string value for non-sensitive config only (use `secret:` + for sensitive keys). - `secret`: The name of a secret in Cloud Secret Manager. use `npx -y firebase-tools@latest apphosting:secrets:set` to create these. - `availability`: Where the variable is needed. diff --git a/plugins/firebase/.agents/skills/firebase-firestore/SKILL.md b/plugins/firebase/.agents/skills/firebase-firestore/SKILL.md index d6c809a7..b73b4eaa 100644 --- a/plugins/firebase/.agents/skills/firebase-firestore/SKILL.md +++ b/plugins/firebase/.agents/skills/firebase-firestore/SKILL.md @@ -1,7 +1,7 @@ --- name: firebase-firestore description: >- - Sets up, manages, queries, and configures Cloud Firestore databases (Standard/Enterprise edition), including data modeling, security rules, indexes, and SDK integrations (Web, Python, iOS, Android, Flutter). Use when creating/listing Firestore databases, defining data models/indexes, writing SDK queries, or integrating Firestore SDKs. For authoring or modifying Firestore Security Rules (firestore.rules), delegate to the firestore-rules-author subagent if subagent delegation is available, or use firestore-rules-creation otherwise. Don't use for Firebase Hosting, Data Connect, Auth, Storage/GCS, Crashlytics, Functions, or BigQuery. + Sets up, manages, queries, and configures Cloud Firestore databases (Standard/Enterprise edition), including data modeling, security rules, indexes, and SDK integrations (Web, Admin, Python, iOS, Android, Flutter). Use when adding or migrating app databases, creating/listing Firestore databases, defining data models/indexes, writing SDK queries, or integrating Firestore SDKs. For authoring or modifying Firestore Security Rules (firestore.rules), delegate to the firestore-rules-author subagent if subagent delegation is available, or use firestore-rules-creation otherwise. Don't use for Data Connect, BigQuery, or standalone Hosting, Auth, Storage/GCS, Crashlytics, or Functions tasks. compatibility: This skill is best used with the Firebase CLI, but does not require it. Firebase CLI can be accessed through `npx -y firebase-tools@latest`. metadata: author: Google LLC @@ -10,8 +10,12 @@ metadata: # Cloud Firestore Database and Operations -> [!IMPORTANT] **Security Rules Authoring (`firestore.rules`)** Whenever your -> task requires creating, authoring, or modifying `firestore.rules`: +> [!IMPORTANT] **Security Rules Authoring (`firestore.rules`)** Whenever setting +> up a Firestore-backed app or creating/modifying `firestore.rules`, configure +> `"firestore": { "rules": "firestore.rules" }` in `firebase.json` and deploy +> `firestore.rules` +> (`npx -y firebase-tools@latest deploy --only firestore:rules`), even if server +> routes use the Admin SDK: > > - **If subagent delegation AND the `firestore-rules-author` subagent are > available**: delegate authoring `firestore.rules` to the diff --git a/plugins/firebase/agent/skills/firebase-app-hosting-basics/SKILL.md b/plugins/firebase/agent/skills/firebase-app-hosting-basics/SKILL.md index 0eb9c188..fddf69da 100644 --- a/plugins/firebase/agent/skills/firebase-app-hosting-basics/SKILL.md +++ b/plugins/firebase/agent/skills/firebase-app-hosting-basics/SKILL.md @@ -73,8 +73,8 @@ This is the recommended flow for most users. npx -y firebase-tools@latest apphosting:backends:create --backend my-app-id --primary-region us-central1 --root-dir / ``` -1. If the app needs safe access to sensitive keys, set and grant access after - creating the backend: +1. For sensitive keys/secrets (e.g., from `.env*`), set and grant access after + creating the backend and reference via `secret:` (never `value:`): ```bash printf '%s' '<value>' | npx -y firebase-tools@latest apphosting:secrets:set <secret-name> --data-file - --force diff --git a/plugins/firebase/agent/skills/firebase-app-hosting-basics/references/configuration.md b/plugins/firebase/agent/skills/firebase-app-hosting-basics/references/configuration.md index 1307f551..e932387f 100644 --- a/plugins/firebase/agent/skills/firebase-app-hosting-basics/references/configuration.md +++ b/plugins/firebase/agent/skills/firebase-app-hosting-basics/references/configuration.md @@ -51,7 +51,8 @@ Defines environment variables available during build and/or runtime (local `.env*` files are not uploaded, so declare required variables or secrets here). - `variable`: The name of the env var (e.g., `NEXT_PUBLIC_API_URL`). -- `value`: A literal string value. +- `value`: A literal string value for non-sensitive config only (use `secret:` + for sensitive keys). - `secret`: The name of a secret in Cloud Secret Manager. use `npx -y firebase-tools@latest apphosting:secrets:set` to create these. - `availability`: Where the variable is needed. diff --git a/plugins/firebase/agent/skills/firebase-firestore/SKILL.md b/plugins/firebase/agent/skills/firebase-firestore/SKILL.md index f47f3229..187dfed8 100644 --- a/plugins/firebase/agent/skills/firebase-firestore/SKILL.md +++ b/plugins/firebase/agent/skills/firebase-firestore/SKILL.md @@ -2,13 +2,14 @@ name: firebase-firestore description: Sets up, manages, queries, and configures Cloud Firestore databases (Standard/Enterprise edition), including data modeling, security rules, - indexes, and SDK integrations (Web, Python, iOS, Android, Flutter). Use when - creating/listing Firestore databases, defining data models/indexes, writing - SDK queries, or integrating Firestore SDKs. For authoring or modifying - Firestore Security Rules (firestore.rules), delegate to the - firestore-rules-author subagent if subagent delegation is available, or use - firestore-rules-creation otherwise. Don't use for Firebase Hosting, Data - Connect, Auth, Storage/GCS, Crashlytics, Functions, or BigQuery. + indexes, and SDK integrations (Web, Admin, Python, iOS, Android, Flutter). Use + when adding or migrating app databases, creating/listing Firestore databases, + defining data models/indexes, writing SDK queries, or integrating Firestore + SDKs. For authoring or modifying Firestore Security Rules (firestore.rules), + delegate to the firestore-rules-author subagent if subagent delegation is + available, or use firestore-rules-creation otherwise. Don't use for Data + Connect, BigQuery, or standalone Hosting, Auth, Storage/GCS, Crashlytics, or + Functions tasks. compatibility: This skill is best used with the Firebase CLI, but does not require it. Firebase CLI can be accessed through `npx -y firebase-tools@latest`. @@ -18,8 +19,12 @@ metadata: --- # Cloud Firestore Database and Operations -> [!IMPORTANT] **Security Rules Authoring (`firestore.rules`)** Whenever your -> task requires creating, authoring, or modifying `firestore.rules`: +> [!IMPORTANT] **Security Rules Authoring (`firestore.rules`)** Whenever setting +> up a Firestore-backed app or creating/modifying `firestore.rules`, configure +> `"firestore": { "rules": "firestore.rules" }` in `firebase.json` and deploy +> `firestore.rules` +> (`npx -y firebase-tools@latest deploy --only firestore:rules`), even if server +> routes use the Admin SDK: > > - **If subagent delegation AND the `firestore-rules-author` subagent are > available**: delegate authoring `firestore.rules` to the diff --git a/plugins/firebase/skills-lock.json b/plugins/firebase/skills-lock.json index eaa74999..9eb5c62f 100644 --- a/plugins/firebase/skills-lock.json +++ b/plugins/firebase/skills-lock.json @@ -17,7 +17,7 @@ "source": "firebase/agent-skills", "sourceType": "github", "skillPath": "skills/firebase-app-hosting-basics/SKILL.md", - "computedHash": "5a982e11348d2e1373ce3dfec6e314861e39ccd03139de7cbd8a492942947243" + "computedHash": "ccd7d537c01e42826033df6b2b6c19215fdec0eb5fb096a499a7c70d15a934b2" }, "firebase-auth-basics": { "source": "firebase/agent-skills", @@ -47,7 +47,7 @@ "source": "firebase/agent-skills", "sourceType": "github", "skillPath": "skills/firebase-firestore/SKILL.md", - "computedHash": "2c9989b46ad19f1469b0de9b48c6734f4c2f2a2536b8066098ab3ec6e0058ab1" + "computedHash": "f12f1a41365344fc025eba3fd05c0a2d9d1a2c01359f95d8939ccd323f5125be" }, "firebase-hosting-basics": { "source": "firebase/agent-skills", diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/SKILL.md b/plugins/genkit/.agents/skills/developing-genkit-dart/SKILL.md index 43f1c080..c829fb55 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/SKILL.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/SKILL.md @@ -1,6 +1,10 @@ --- name: developing-genkit-dart -description: Generates code and provides documentation for the Genkit Dart SDK. Use when the user asks to build AI agents in Dart, use Genkit flows, or integrate LLMs into Dart/Flutter applications. +description: >- + Provides guidance on developing AI-powered applications using the Genkit Dart + SDK. Use when the user asks to build AI agents in Dart, use Genkit flows, or + integrate LLMs into Dart/Flutter applications. Don't use for Genkit in + JavaScript/TypeScript, Go, or Python. metadata: version: "1.0.0" category: AiAndMachineLearning diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/a2ui.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/a2ui.md index cd3a86d7..fbff6124 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/a2ui.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/a2ui.md @@ -13,6 +13,16 @@ An A2UI-enabled agent streams more than prose. It streams interactive UI as the model responds. The entire server-side integration is a single model middleware: add `a2ui()` to an agent's `use` list and nothing else changes. +- [Install](#install): Lines 26-49 +- [Server: add the a2ui() middleware](#server-add-the-a2ui-middleware): Lines 51-124 +- [Client: render surfaces (Flutter + genui)](#client-render-surfaces-flutter--genui): Lines 126-181 +- [Handling user actions](#handling-user-actions): Lines 183-242 +- [Custom catalogs](#custom-catalogs): Lines 245-323 +- [Security and the trust boundary](#security-and-the-trust-boundary): Lines 325-346 +- [How it works](#how-it-works): Lines 348-367 + +--- + ## Install On the **server** (the shelf app that hosts the agent), add `genkit_a2ui` diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-artifacts.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-artifacts.md index ed7eb421..3e792e5e 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-artifacts.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-artifacts.md @@ -26,6 +26,14 @@ name) and are returned in `res.artifacts` / tracked on the client's > session artifacts for conversation-scoped deliverables, `filesystem()` for > persistent on-disk work. +- [Give the model artifact tools](#give-the-model-artifact-tools): Lines 37-99 +- [How artifacts flow](#how-artifacts-flow): Lines 101-113 +- [The Artifact shape](#the-artifact-shape): Lines 115-124 +- [Programmatic access (inside tools / custom agents)](#programmatic-access-inside-tools--custom-agents): Lines 126-142 +- [Reading artifacts on the client](#reading-artifacts-on-the-client): Lines 144-171 + +--- + ## Give the model artifact tools ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-background.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-background.md index 65245fca..fcaa70b9 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-background.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-background.md @@ -9,6 +9,13 @@ and returns a `snapshotId` **immediately**, keeps processing, then updates the snapshot to a terminal status (`completed` / `failed` / `aborted` / `expired`). The client polls for completion and can abort. +- [Define the agent (store required)](#define-the-agent-store-required): Lines 19-46 +- [Client-side: detach + poll + abort](#client-side-detach--poll--abort): Lines 48-87 +- [Server-side: detach + wait](#server-side-detach--wait): Lines 89-101 +- [Status values](#status-values): Lines 103-114 + +--- + ## Define the agent (store required) ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-branching.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-branching.md index 65f1573e..1341a7be 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-branching.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-branching.md @@ -10,6 +10,12 @@ snapshot creates a new, independent snapshot; the original is unchanged. To branch, open a new `chat` attached to an earlier snapshot via `agent.chat(snapshotId: ...)`. +- [Server-side branching](#server-side-branching): Lines 19-46 +- [Client-side branching ("pick a variant")](#client-side-branching-pick-a-variant): Lines 48-79 +- [Restoring history from a snapshot](#restoring-history-from-a-snapshot): Lines 81-109 + +--- + ## Server-side branching ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-custom.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-custom.md index 631fefc3..86ae2ed7 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-custom.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-custom.md @@ -9,6 +9,13 @@ the turn** — multiple sequential model calls, custom logic between them, manua message/state management, or custom progress streaming — use `ai.defineCustomAgent`. You provide the handler that runs the turn. +- [When to use it](#when-to-use-it): Lines 19-29 +- [Signature](#signature): Lines 31-62 +- [Example: multi-step research agent](#example-multi-step-research-agent): Lines 64-174 +- [Custom status streaming](#custom-status-streaming): Lines 176-199 + +--- + ## When to use it Reach for `defineCustomAgent` when a turn needs to: diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-deployment.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-deployment.md index 398b260e..bbfb4d8e 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-deployment.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-deployment.md @@ -19,6 +19,14 @@ match the `remoteAgent` client defaults (`${url}/getSnapshot`, `${url}/abort`), so a client only needs the base `url`. `hideGetSnapshot: true` / `hideAbort: true` remove supported routes; they can't force unsupported ones. +- [Serving several agents (shelf)](#serving-several-agents-shelf): Lines 30-78 +- [Auth](#auth): Lines 80-84 +- [CORS for browser clients](#cors-for-browser-clients): Lines 86-90 +- [Registering agents/flows](#registering-agentsflows): Lines 92-98 +- [Notes](#notes): Lines 100-110 + +--- + ## Serving several agents (shelf) ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-human-in-the-loop.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-human-in-the-loop.md index 5718a0d5..557a58a1 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-human-in-the-loop.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-human-in-the-loop.md @@ -36,6 +36,14 @@ back into the resume). Flow: `chat.send(text: ...)` → response has `res.interrupts` → collect human input → `chat.resume(respond: [...])`. +- [Define an interrupt (a tool that interrupts)](#define-an-interrupt-a-tool-that-interrupts): Lines 47-111 +- [Detect and resume (server-side)](#detect-and-resume-server-side): Lines 113-169 +- [Client-side (browser) interrupts](#client-side-browser-interrupts): Lines 171-202 +- [Interrupts with tool-approval middleware](#interrupts-with-tool-approval-middleware): Lines 204-229 +- [Notes & gotchas](#notes--gotchas): Lines 231-245 + +--- + ## Define an interrupt (a tool that interrupts) The example below uses tools that return `.interrupt(...)`, which pauses the diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-multi-agent.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-multi-agent.md index 416e5c18..249bbd9f 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-multi-agent.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-multi-agent.md @@ -19,6 +19,15 @@ import 'package:genkit_middleware/agents.dart'; final ai = Genkit(plugins: [googleAI(), AgentsPlugin()]); // retry() is built in ``` +- [1. Define the sub-agents](#1-define-the-sub-agents): Lines 31-57 +- [2. Wire up the orchestrator](#2-wire-up-the-orchestrator): Lines 59-101 +- [agents() options](#agents-options): Lines 103-113 +- [Background (async) delegation](#background-async-delegation): Lines 115-143 +- [Sharing artifacts between agents](#sharing-artifacts-between-agents): Lines 145-154 +- [Other middleware](#other-middleware): Lines 156-166 + +--- + ## 1. Define the sub-agents Give each sub-agent a `description` — it's auto-discovered from registry metadata diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-sessions.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-sessions.md index 3b63904c..71368e74 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-sessions.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-sessions.md @@ -12,6 +12,14 @@ conversation state forward. A store also enables [branching](agents-branching.md and [background execution](agents-background.md). (Interrupts work with or without a store — see [human-in-the-loop](agents-human-in-the-loop.md).) +- [Pick a store](#pick-a-store): Lines 23-71 +- [Typed session state](#typed-session-state): Lines 73-112 +- [Interrupts (human-in-the-loop)](#interrupts-human-in-the-loop): Lines 114-119 +- [Firestore session store (scalable)](#firestore-session-store-scalable): Lines 121-161 +- [Implementing a custom SessionStore](#implementing-a-custom-sessionstore): Lines 163-200 + +--- + ## Pick a store ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-state.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-state.md index 7aad1957..ed98f914 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-state.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents-state.md @@ -8,6 +8,13 @@ and mutate it during a turn, and it is automatically synced to the [`remoteAgent`](agents.md#consume-an-agent-from-a-client-remoteagent) client via `customPatch` chunks (so the client's tracked state stays live mid-stream). +- [Declare the state shape](#declare-the-state-shape): Lines 18-55 +- [Read & mutate state inside tools](#read--mutate-state-inside-tools): Lines 57-157 +- [Seed and read state (server-side)](#seed-and-read-state-server-side): Lines 159-182 +- [Auto-sync to the remoteAgent client](#auto-sync-to-the-remoteagent-client): Lines 184-222 + +--- + ## Declare the state shape Pass a `stateSchema` (a schemantic `.$schema`) to `defineAgent`. It's validated diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents.md index c058490f..3ec6b07c 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/agents.md @@ -46,6 +46,18 @@ Progressive disclosure — read the file for the level you need: - [Advanced custom agents](agents-custom.md): `defineCustomAgent` for full turn control. - [Deploying agents](agents-deployment.md): serving multiple agents over HTTP with `GenkitRouter.addAgent`, shelf, CORS. +- [Setup](#setup): Lines 61-83 +- [Define an agent](#define-an-agent): Lines 85-153 +- [Agents and middleware go hand in hand](#agents-and-middleware-go-hand-in-hand): Lines 155-196 +- [Chat with an agent (server-side)](#chat-with-an-agent-server-side): Lines 198-240 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 242-255 +- [Serve an agent over HTTP](#serve-an-agent-over-http): Lines 257-278 +- [Consume an agent from a client (remoteAgent)](#consume-an-agent-from-a-client-remoteagent): Lines 280-314 +- [Using from Flutter](#using-from-flutter): Lines 316-390 +- [Client-managed state (no server store)](#client-managed-state-no-server-store): Lines 392-431 + +--- + ## Setup Register the plugins your agents need on a shared `Genkit` instance. `retry` is diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/dotprompt.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/dotprompt.md index 44302ff3..fb23c002 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/dotprompt.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/dotprompt.md @@ -1,5 +1,17 @@ # Dotprompt (.prompt files) — Genkit Dart +- [What it is](#what-it-is): Lines 15-19 +- [Where files live](#where-files-live): Lines 21-34 +- [File format](#file-format): Lines 36-81 +- [Loading and calling a prompt](#loading-and-calling-a-prompt): Lines 83-157 +- [Registering named schemas](#registering-named-schemas): Lines 159-191 +- [Variants](#variants): Lines 193-201 +- [Partials](#partials): Lines 203-213 +- [Tools, tool-loop control, and middleware](#tools-tool-loop-control-and-middleware): Lines 215-266 +- [Backing an agent with a .prompt file](#backing-an-agent-with-a-prompt-file): Lines 268-282 + +--- + ## What it is `.prompt` files combine YAML frontmatter (model, config, schemas, tools, diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit.md index 0575e7fc..99dbe946 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit.md @@ -2,6 +2,23 @@ Genkit Dart is an AI SDK for Dart that provides a unified interface for text generation, structured output, tool calling, and agentic workflows. +- [Initialization](#initialization): Lines 22-33 +- [Reliability: retry transient errors](#reliability-retry-transient-errors): Lines 35-55 +- [Generate Text](#generate-text): Lines 57-67 +- [Stream Responses](#stream-responses): Lines 69-79 +- [Embed Text](#embed-text): Lines 81-95 +- [Define Tools](#define-tools): Lines 97-143 +- [Structured Output](#structured-output): Lines 145-166 +- [Errors and finish reasons](#errors-and-finish-reasons): Lines 168-225 +- [Cancellation](#cancellation): Lines 227-254 +- [Telemetry (instrumentation)](#telemetry-instrumentation): Lines 256-276 +- [Define Flows](#define-flows): Lines 278-326 +- [Calling remote Flows from a dart client](#calling-remote-flows-from-a-dart-client): Lines 328-368 +- [Calling remote Flows from a Javascript client](#calling-remote-flows-from-a-javascript-client): Lines 370-424 +- [Data Models](#data-models): Lines 426-565 + +--- + ## Initialization ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_google_genai.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_google_genai.md index 1828902c..1c724e46 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_google_genai.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_google_genai.md @@ -2,6 +2,15 @@ The Google AI plugin provides an interface against the official Google AI Gemini API. +- [Usage](#usage): Lines 14-34 +- [Model refs and Gemma](#model-refs-and-gemma): Lines 36-48 +- [Embeddings](#embeddings): Lines 50-61 +- [Config types are provider-prefixed](#config-types-are-provider-prefixed): Lines 63-85 +- [Image Generation](#image-generation): Lines 87-112 +- [Text-to-Speech (TTS)](#text-to-speech-tts): Lines 114-146 + +--- + ## Usage ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_mcp.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_mcp.md index 71da3478..12cb885f 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_mcp.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_mcp.md @@ -9,6 +9,12 @@ initialization-based peers automatically. > name is preserved in `metadata.originalName` and tool requests still resolve > correctly. +- [MCP Host (Recommended)](#mcp-host-recommended): Lines 18-73 +- [MCP Client (Advanced / Single Server)](#mcp-client-advanced--single-server): Lines 75-106 +- [MCP Server](#mcp-server): Lines 108-160 + +--- + ## MCP Host (Recommended) Connect to one or more MCP servers and aggregate their capabilities into the Genkit registry automatically. diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_middleware.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_middleware.md index 0d4d4e6f..2f9223bc 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_middleware.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_middleware.md @@ -17,6 +17,13 @@ void main() { } ``` +- [Filesystem Middleware](#filesystem-middleware): Lines 27-40 +- [Skills Middleware](#skills-middleware): Lines 42-55 +- [Tool Approval Middleware](#tool-approval-middleware): Lines 57-99 +- [Custom Middleware (core package:genkit)](#custom-middleware-core-packagegenkit): Lines 101-162 + +--- + ## Filesystem Middleware Allows the agent to list, read, write, and search/replace files within a restricted root directory. diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_shelf.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_shelf.md index c07797e7..54895b27 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_shelf.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/genkit_shelf.md @@ -7,6 +7,16 @@ HTTP serving lives in core `package:genkit/io.dart` (`GenkitRouter`, plain > There is no `startFlowServer` and no `FlowWithContextProvider`. Do not add > `shelf_cors_headers`; CORS is `CorsOptions`. +- [Standalone server (no shelf)](#standalone-server-no-shelf): Lines 20-54 +- [Existing shelf app](#existing-shelf-app): Lines 56-86 +- [Own dart:io server](#own-dartio-server): Lines 88-102 +- [Auth: contextProvider](#auth-contextprovider): Lines 104-119 +- [Agents](#agents): Lines 121-125 +- [CORS](#cors): Lines 127-132 +- [Older clients](#older-clients): Lines 134-142 + +--- + ## Standalone server (no shelf) ```dart diff --git a/plugins/genkit/.agents/skills/developing-genkit-dart/references/schemantic.md b/plugins/genkit/.agents/skills/developing-genkit-dart/references/schemantic.md index 2d132e97..7907df35 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-dart/references/schemantic.md +++ b/plugins/genkit/.agents/skills/developing-genkit-dart/references/schemantic.md @@ -2,6 +2,17 @@ Schemantic is a general-purpose Dart library used for defining strongly typed data classes that automatically bind to reusable runtime JSON schemas. It is standard for the `genkit-dart` framework but works independently as well. +- [Core Concepts](#core-concepts): Lines 16-22 +- [Installation](#installation): Lines 24-42 +- [Basic Usage](#basic-usage): Lines 44-97 +- [Primitive Schemas](#primitive-schemas): Lines 99-111 +- [Union Types (AnyOf)](#union-types-anyof): Lines 113-130 +- [Field Annotations](#field-annotations): Lines 132-153 +- [Recursive Schemas](#recursive-schemas): Lines 155-166 +- [Non-nullable getters throw on partial data](#non-nullable-getters-throw-on-partial-data): Lines 168-176 + +--- + ## Core Concepts Always use `schemantic` when strongly typed JSON parsing or programmatic schema validation is required. diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/SKILL.md b/plugins/genkit/.agents/skills/developing-genkit-go/SKILL.md index f9537a17..e8d597ab 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/SKILL.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/SKILL.md @@ -1,6 +1,11 @@ --- name: developing-genkit-go -description: Develop AI-powered applications using Genkit in Go. Use when the user asks to build AI features, agents, flows, or tools in Go using Genkit, or when working with Genkit Go code involving generation, prompts, streaming, tool calling, or model providers. +description: >- + Provides guidance on developing AI-powered applications using Genkit in Go. + Use when the user asks to build AI features, agents, flows, or tools in Go + using Genkit, or when working with Genkit Go code involving generation, + prompts, streaming, tool calling, or model providers. Don't use for Genkit in + JavaScript/TypeScript, Python, or Dart. metadata: version: "1.0.0" category: AiAndMachineLearning @@ -53,7 +58,7 @@ Load the appropriate reference based on what you need: | Feature | Reference | When to load | | --- | --- | --- | | Initialization | [references/getting-started.md](references/getting-started.md) | Setting up `genkit.Init`, plugins, the `*Genkit` instance pattern | -| Generation | [references/generation.md](references/generation.md) | `Generate`, `GenerateText`, `GenerateData`, streaming, output formats | +| Generation | [references/generation.md](references/generation.md) | `Generate`, `GenerateText`, `GenerateData`, streaming, output formats, `Embed` | | Prompts | [references/prompts.md](references/prompts.md) | `DefinePrompt`, `DefineDataPrompt`, `.prompt` files, schemas | | Tools | [references/tools.md](references/tools.md) | `DefineTool`, `DefineResumableTool` interrupts (`Interrupted` claim, `Restart`/`Respond`, `WithResume`), `ai/tool` helpers (`tool.Interrupt`, `tool.Fail`, `tool.AttachParts`) | | Middleware | [references/middleware.md](references/middleware.md) | `ai.Middleware`, `ai.WithUse`, `Hooks` (Generate/Model/Tool), built-ins (`Retry`, `Fallback`, `ToolApproval`, `Filesystem`, `Skills`, `SoftToolErrors`, `ContextCompression`, `Budget`) | diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/a2ui.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/a2ui.md index 5de54c5b..a005c921 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/a2ui.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/a2ui.md @@ -15,6 +15,15 @@ An A2UI-enabled agent streams more than prose. It streams interactive UI as the model responds. The entire Go integration is a single model middleware: add `&a2uix.Surfaces{}` to a generate/agent's `ai.WithUse` and nothing else changes. +- [Go is server-only: bring your own client](#go-is-server-only-bring-your-own-client): Lines 27-50 +- [Server: add the a2ui middleware](#server-add-the-a2ui-middleware): Lines 52-116 +- [Serve the agent over HTTP](#serve-the-agent-over-http): Lines 118-164 +- [Custom catalogs](#custom-catalogs): Lines 166-220 +- [Security and the trust boundary](#security-and-the-trust-boundary): Lines 222-242 +- [How it works](#how-it-works): Lines 244-263 + +--- + ## Go is server-only: bring your own client **Go has no A2UI client/renderer.** The Go plugin is the server half: it injects diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-artifacts.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-artifacts.md index 843ca999..6ecfaf97 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-artifacts.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-artifacts.md @@ -15,6 +15,13 @@ import ( ) ``` +- [Give the model artifact tools](#give-the-model-artifact-tools): Lines 25-67 +- [The Artifact shape](#the-artifact-shape): Lines 69-80 +- [Programmatic access (inside tools / custom agents)](#programmatic-access-inside-tools--custom-agents): Lines 82-110 +- [Sharing artifacts across agents](#sharing-artifacts-across-agents): Lines 112-128 + +--- + ## Give the model artifact tools The `middlewarex.Artifacts` middleware (see [using middleware](middleware.md)) diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-background.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-background.md index 1f8bb7a6..39851063 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-background.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-background.md @@ -11,6 +11,15 @@ and returns its `SnapshotID` **immediately**, keeps processing, then rewrites th snapshot to a terminal status (`completed` / `failed` / `aborted`). A reader waits on the snapshot (or polls it) and can `Abort` in the meantime. +- [Define the agent (store required)](#define-the-agent-store-required): Lines 23-33 +- [Detach a turn](#detach-a-turn): Lines 35-53 +- [Wait (or poll)](#wait-or-poll): Lines 55-79 +- [Abort and resume](#abort-and-resume): Lines 81-107 +- [Serve detach over HTTP](#serve-detach-over-http): Lines 109-127 +- [Status values (aix.SnapshotStatus)](#status-values-aixsnapshotstatus): Lines 129-142 + +--- + ## Define the agent (store required) ```go diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-branching.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-branching.md index 0d470a30..f68a8229 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-branching.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-branching.md @@ -10,6 +10,13 @@ snapshot creates a new, independent snapshot; the original is unchanged. To branch, run a turn resuming from an earlier snapshot via `aix.WithSnapshotID(id)`. +- [Branch from a checkpoint](#branch-from-a-checkpoint): Lines 20-47 +- ["Pick a variant"](#pick-a-variant): Lines 49-77 +- [Resume by session vs. by snapshot](#resume-by-session-vs-by-snapshot): Lines 79-84 +- [Restore history from a snapshot](#restore-history-from-a-snapshot): Lines 86-109 + +--- + ## Branch from a checkpoint ```go diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-custom.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-custom.md index 7da0384f..d347512f 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-custom.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-custom.md @@ -9,6 +9,14 @@ the turn** — multiple sequential model calls, custom logic between them, manua message/state management, or custom progress streaming — use `genkitx.DefineCustomAgent`. You provide the function that runs the turn loop. +- [When to use it](#when-to-use-it): Lines 20-30 +- [Signature](#signature): Lines 32-66 +- [Example: multi-step research agent](#example-multi-step-research-agent): Lines 68-157 +- [Custom status streaming](#custom-status-streaming): Lines 159-189 +- [Notes](#notes): Lines 191-202 + +--- + ## When to use it Reach for `DefineCustomAgent` when a turn needs to: diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-deployment.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-deployment.md index 03ce103d..ca50878a 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-deployment.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-deployment.md @@ -14,6 +14,15 @@ companion actions: `nil` unless the store implements `aix.SnapshotSubscriber` (both `localstore` stores do). +- [Request/response shape](#requestresponse-shape): Lines 26-41 +- [Serve with the built-in route layout (recommended)](#serve-with-the-built-in-route-layout-recommended): Lines 43-73 +- [A reusable helper for several agents (manual wiring)](#a-reusable-helper-for-several-agents-manual-wiring): Lines 75-121 +- [CORS for browser clients](#cors-for-browser-clients): Lines 123-149 +- [Registering agents](#registering-agents): Lines 151-157 +- [Verifying locally](#verifying-locally): Lines 159-180 + +--- + ## Request/response shape The handler accepts `{"data": <AgentInput>, "init": <AgentInit>}`: diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-human-in-the-loop.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-human-in-the-loop.md index 3389e714..21a7a21b 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-human-in-the-loop.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-human-in-the-loop.md @@ -20,6 +20,13 @@ The tool side is identical to plain `Generate` (see [tools.md](tools.md)): `Interrupted` method. Only the delivery differs: an agent takes the resume parts in an `aix.ToolResume` via `conn.SendResume` or `AgentInput.Resume`. +- [Define a resumable tool](#define-a-resumable-tool): Lines 30-101 +- [Detect, claim, and resume (streaming connection)](#detect-claim-and-resume-streaming-connection): Lines 103-165 +- [Single-turn resume with Run](#single-turn-resume-with-run): Lines 167-190 +- [Notes & gotchas](#notes--gotchas): Lines 192-208 + +--- + ## Define a resumable tool `genkit.DefineResumableTool[In, Out, Res]` defines a tool that can pause by diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-multi-agent.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-multi-agent.md index af07817e..17f6a942 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-multi-agent.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-multi-agent.md @@ -18,6 +18,17 @@ import ( ) ``` +- [1. Define the sub-agents](#1-define-the-sub-agents): Lines 32-55 +- [2. Wire up the orchestrator](#2-wire-up-the-orchestrator): Lines 57-94 +- [middlewarex.Agents fields](#middlewarexagents-fields): Lines 96-114 +- [Background delegation (Async: true)](#background-delegation-async-true): Lines 116-155 +- [Continuing a task (continue_task)](#continuing-a-task-continue_task): Lines 157-173 +- [Delegating to a remote agent](#delegating-to-a-remote-agent): Lines 175-197 +- [Sharing artifacts between agents](#sharing-artifacts-between-agents): Lines 199-211 +- [Combining with retries](#combining-with-retries): Lines 213-235 + +--- + ## 1. Define the sub-agents Give each sub-agent a description via `aix.WithDescription` — it's diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-sessions.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-sessions.md index bc5ca3e8..492da2be 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-sessions.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-sessions.md @@ -10,6 +10,16 @@ snapshot chain carries conversation state forward. A store also enables (Interrupts work with or without a store — see [human-in-the-loop](agents-human-in-the-loop.md).) +- [Pick a store](#pick-a-store): Lines 23-103 +- [Turns persist automatically](#turns-persist-automatically): Lines 105-115 +- [Resume a conversation](#resume-a-conversation): Lines 117-139 +- [Read a snapshot without running a turn](#read-a-snapshot-without-running-a-turn): Lines 141-158 +- [Typed session state](#typed-session-state): Lines 160-188 +- [Interrupts (human-in-the-loop)](#interrupts-human-in-the-loop): Lines 190-195 +- [Implementing a custom SessionStore](#implementing-a-custom-sessionstore): Lines 197-243 + +--- + ## Pick a store `ai/exp/localstore` ships two single-process stores. Both are generic over the diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-state.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-state.md index 66f5e05a..630e3c10 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-state.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents-state.md @@ -7,6 +7,15 @@ own structured data (a task list, a workflow status, counters, etc.). Tools read and mutate it during a turn, and every mutation is streamed to a connected client as a JSON Patch so the client's view stays live. +- [Declare the state shape](#declare-the-state-shape): Lines 19-44 +- [Read & mutate state inside tools](#read--mutate-state-inside-tools): Lines 46-88 +- [Seed and read state](#seed-and-read-state): Lines 90-107 +- [Live state on a connection](#live-state-on-a-connection): Lines 109-136 +- [Prompt templating with state](#prompt-templating-with-state): Lines 138-145 +- [Custom status streaming from a custom agent](#custom-status-streaming-from-a-custom-agent): Lines 147-152 + +--- + ## Declare the state shape The custom-state type is the `State` type parameter of the agent (and its store). diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents.md index db7a9c4b..bb84ddef 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/agents.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/agents.md @@ -27,6 +27,18 @@ Read the file for the level you need: - [Advanced custom agents](agents-custom.md): `DefineCustomAgent` for full turn control. - [Deploying agents](agents-deployment.md): serving agents over HTTP. +- [Packages and import aliases](#packages-and-import-aliases): Lines 42-59 +- [Setup](#setup): Lines 61-72 +- [Define an agent](#define-an-agent): Lines 74-166 +- [Agents and middleware go hand in hand](#agents-and-middleware-go-hand-in-hand): Lines 168-193 +- [Run an agent (single turn)](#run-an-agent-single-turn): Lines 195-222 +- [Chat with an agent (multi-turn)](#chat-with-an-agent-multi-turn): Lines 224-267 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 269-284 +- [Serve an agent over HTTP](#serve-an-agent-over-http): Lines 286-324 +- [Client-managed vs server-managed state](#client-managed-vs-server-managed-state): Lines 326-366 + +--- + ## Packages and import aliases Agents span a few packages. These aliases are used throughout the agent docs: diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/flows-and-http.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/flows-and-http.md index 92f99a22..a52e7d69 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/flows-and-http.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/flows-and-http.md @@ -1,5 +1,12 @@ # Flows & HTTP +- [DefineFlow](#defineflow): Lines 10-29 +- [DefineStreamingFlow](#definestreamingflow): Lines 31-99 +- [Named Sub-Steps](#named-sub-steps): Lines 101-123 +- [HTTP Handlers](#http-handlers): Lines 125-190 + +--- + ## DefineFlow Wrap AI logic in a flow for observability, tracing, and HTTP deployment. diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/generation.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/generation.md index 5934575c..b740845f 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/generation.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/generation.md @@ -1,5 +1,15 @@ # Generation +- [GenerateText](#generatetext): Lines 13-22 +- [Generate](#generate): Lines 24-37 +- [GenerateData (Structured Output)](#generatedata-structured-output): Lines 39-54 +- [Streaming](#streaming): Lines 56-114 +- [Common Options](#common-options): Lines 116-132 +- [Output Formats](#output-formats): Lines 134-186 +- [Embeddings](#embeddings): Lines 188-223 + +--- + ## GenerateText Simplest form. Returns a string. @@ -174,3 +184,40 @@ genkit.DefinePrompt(g, "characters", ai.WithOutputFormat(ai.OutputFormatJSONL), ) ``` + +## Embeddings + +```go +res, err := genkit.Embed(ctx, g, + ai.WithEmbedderName("googleai/gemini-embedding-001"), + ai.WithTextDocs("The sky is blue.", "Grass is green."), // one embedding per doc +) +if err != nil { + return err +} +vector := res.Embeddings[0].Embedding // []float32 + +// Optional tuning (default is 3072 dimensions) +res, err = genkit.Embed(ctx, g, + ai.WithEmbedderName("googleai/gemini-embedding-001"), + ai.WithTextDocs("The sky is blue."), + ai.WithConfig(&genai.EmbedContentConfig{ + OutputDimensionality: genai.Ptr[int32](768), + TaskType: "RETRIEVAL_DOCUMENT", + }), +) +``` + +Common embedders: `googleai/gemini-embedding-001` (text), +`googleai/gemini-embedding-2` (multimodal: text, image, video). + +Multimodal input goes in a document's media parts: + +```go +res, err := genkit.Embed(ctx, g, + ai.WithEmbedderName("googleai/gemini-embedding-2"), + ai.WithDocs(&ai.Document{Content: []*ai.Part{ + ai.NewMediaPart("image/png", "data:image/png;base64,..."), + }}), +) +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/getting-started.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/getting-started.md index 01e36177..f8d74b8d 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/getting-started.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/getting-started.md @@ -1,5 +1,12 @@ # Getting Started +- [Project Setup](#project-setup): Lines 10-24 +- [Initialization](#initialization): Lines 26-90 +- [Genkit CLI](#genkit-cli): Lines 92-136 +- [Import Paths](#import-paths): Lines 138-147 + +--- + ## Project Setup ```bash diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/middleware.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/middleware.md index 027d7bfd..b766d15e 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/middleware.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/middleware.md @@ -2,6 +2,21 @@ Middleware wraps `Generate` calls to add cross-cutting behavior (retries, logging, fallback, gating, sandboxed tools) without touching the flow that uses it. Middleware composes, so a single `Generate` call can stack several behaviors. Built-ins ship in the `plugins/middleware` package; custom middleware is just a Go struct with two methods. +- [The mental model](#the-mental-model): Lines 20-44 +- [When each hook fires](#when-each-hook-fires): Lines 46-56 +- [Composition order](#composition-order): Lines 58-70 +- [Per-call state](#per-call-state): Lines 72-120 +- [Plugin-level state](#plugin-level-state): Lines 122-157 +- [Composition with WithUse](#composition-with-withuse): Lines 159-172 +- [Inline middleware](#inline-middleware): Lines 174-189 +- [Application-owned middleware](#application-owned-middleware): Lines 191-202 +- [Built-in middleware](#built-in-middleware): Lines 204-374 +- [Practical patterns](#practical-patterns): Lines 376-451 +- [Migration note](#migration-note): Lines 453-455 +- [See also](#see-also): Lines 457-461 + +--- + ## The mental model A middleware is a config struct that implements two methods: diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/observability.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/observability.md index ab589952..ef37973d 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/observability.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/observability.md @@ -3,6 +3,13 @@ > Pluggable instrumentation, `otelgenai`, and the reflection API settings > require Genkit Go v1.14.0+. +- [How tracing works](#how-tracing-works): Lines 13-26 +- [Export OpenTelemetry GenAI semantic conventions (otelgenai)](#export-opentelemetry-genai-semantic-conventions-otelgenai): Lines 28-70 +- [Custom instrumentation](#custom-instrumentation): Lines 72-92 +- [Reflection API outside dev](#reflection-api-outside-dev): Lines 94-115 + +--- + ## How tracing works - **In dev** (`genkit start`), Genkit posts spans straight to the Developer UI's diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/prompts.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/prompts.md index d11ec0cf..c505f3e1 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/prompts.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/prompts.md @@ -1,5 +1,12 @@ # Prompts +- [DefinePrompt](#defineprompt): Lines 10-53 +- [DefineDataPrompt (Typed Input/Output)](#definedataprompt-typed-inputoutput): Lines 55-94 +- [.prompt Files (Dotprompt)](#prompt-files-dotprompt): Lines 96-276 +- [Schemas](#schemas): Lines 278-320 + +--- + ## DefinePrompt Define a reusable prompt in code with a default model and template. diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/providers.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/providers.md index 4773809c..1ad06c5d 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/providers.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/providers.md @@ -1,5 +1,14 @@ # Model Providers +- [Google AI (Gemini)](#google-ai-gemini): Lines 12-37 +- [Vertex AI](#vertex-ai): Lines 39-55 +- [Anthropic (Claude)](#anthropic-claude): Lines 57-86 +- [OpenAI-Compatible (compat_oai)](#openai-compatible-compat_oai): Lines 88-112 +- [Ollama (Local Models)](#ollama-local-models): Lines 114-141 +- [Multiple Providers](#multiple-providers): Lines 143-166 + +--- + ## Google AI (Gemini) ```go diff --git a/plugins/genkit/.agents/skills/developing-genkit-go/references/tools.md b/plugins/genkit/.agents/skills/developing-genkit-go/references/tools.md index f70cc02f..78b9d00b 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-go/references/tools.md +++ b/plugins/genkit/.agents/skills/developing-genkit-go/references/tools.md @@ -1,5 +1,12 @@ # Tools +- [DefineTool](#definetool): Lines 10-31 +- [Using Tools in Generation](#using-tools-in-generation): Lines 33-61 +- [Tool runtime helpers (ai/tool)](#tool-runtime-helpers-aitool): Lines 63-116 +- [Tool interrupts (DefineResumableTool)](#tool-interrupts-defineresumabletool): Lines 118-257 + +--- + ## DefineTool Define a tool the model can call during generation. diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/SKILL.md b/plugins/genkit/.agents/skills/developing-genkit-js/SKILL.md index 71b410b4..ebb318cf 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/SKILL.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/SKILL.md @@ -1,6 +1,11 @@ --- name: developing-genkit-js -description: Develop AI-powered applications using Genkit in Node.js/TypeScript. Use when the user asks about Genkit, AI agents, flows, or tools in JavaScript/TypeScript, or when encountering Genkit errors, validation issues, type errors, or API problems. +description: >- + Provides guidance on developing AI-powered applications using Genkit in + Node.js/TypeScript. Use when the user asks about Genkit, AI agents, flows, or + tools in JavaScript/TypeScript, or when encountering Genkit errors, validation + issues, type errors, or API problems. Don't use for Genkit in Python, Go, or + Dart. metadata: version: "1.0.0" category: AiAndMachineLearning @@ -195,7 +200,7 @@ See [CLI Reference](references/docs-and-cli.md) for more commands, and `genkit - - [Docs & CLI Reference](references/docs-and-cli.md): Documentation search, CLI tasks, and workflows. - [Common Errors](references/common-errors.md): Critical "gotchas", migration guide, and troubleshooting. - [Setup Guide](references/setup.md): Manual setup instructions for new projects. -- [Examples](references/examples.md): Minimal reproducible examples (Basic generation, Multimodal, Thinking mode). +- [Examples](references/examples.md): Minimal reproducible examples (Basic generation, Multimodal, Thinking mode, Embeddings). - [Agents (Beta)](references/agents.md): Agent basics, serving, and client-managed state. Deeper topics: [sessions](references/agents-sessions.md), [human-in-the-loop](references/agents-human-in-the-loop.md), [branching](references/agents-branching.md), [background agents](references/agents-background.md), [state](references/agents-state.md), [artifacts](references/agents-artifacts.md), [multi-agent](references/agents-multi-agent.md), [custom agents](references/agents-custom.md), [deployment](references/agents-deployment.md). - [Middleware](references/middleware.md): using middleware and the `@genkit-ai/middleware` package. See also [building custom middleware](references/middleware-custom.md). - [Observability](references/observability.md): `@genkit-ai/otel` (OpenTelemetry GenAI semantic conventions), custom `Instrumentation`, and the reflection API outside dev. diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/a2ui.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/a2ui.md index f88a1361..aaffc5af 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/a2ui.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/a2ui.md @@ -13,6 +13,16 @@ An A2UI-enabled agent streams more than prose. It streams interactive UI as the model responds. The entire server-side integration is a single model middleware: add `a2ui()` to an agent's `use` array and nothing else changes. +- [Install](#install): Lines 26-40 +- [Server: add the a2ui() middleware](#server-add-the-a2ui-middleware): Lines 42-105 +- [Client: render surfaces](#client-render-surfaces): Lines 107-160 +- [Handling user actions](#handling-user-actions): Lines 162-220 +- [Custom catalogs](#custom-catalogs): Lines 223-273 +- [Security and the trust boundary](#security-and-the-trust-boundary): Lines 275-294 +- [How it works](#how-it-works): Lines 296-315 + +--- + ## Install ```bash diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-artifacts.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-artifacts.md index 0daedcaa..6db880d0 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-artifacts.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-artifacts.md @@ -9,6 +9,14 @@ session — files, reports, code, etc. They live in the session (deduplicated by name) and are returned in `res.artifacts` / tracked on the client's `chat.artifacts`. +- [Give the model artifact tools](#give-the-model-artifact-tools): Lines 20-52 +- [The Artifact shape](#the-artifact-shape): Lines 54-67 +- [Programmatic access (inside tools / custom agents)](#programmatic-access-inside-tools--custom-agents): Lines 69-83 +- [Reading artifacts on the client](#reading-artifacts-on-the-client): Lines 85-98 +- [Sharing artifacts across agents](#sharing-artifacts-across-agents): Lines 100-113 + +--- + ## Give the model artifact tools The `artifacts()` middleware (see [using middleware](middleware.md)) adds diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-background.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-background.md index 0b30957c..ccf39970 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-background.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-background.md @@ -9,6 +9,14 @@ and returns a `snapshotId` **immediately**, keeps processing, then updates the snapshot to a terminal status (`completed` / `failed` / `aborted` / `expired`). The client polls for completion and can abort. +- [Define the agent (store required)](#define-the-agent-store-required): Lines 20-53 +- [Server-side: detach + wait](#server-side-detach--wait): Lines 55-73 +- [Resume a failed or aborted task](#resume-a-failed-or-aborted-task): Lines 75-89 +- [Client-side: detach + poll + abort](#client-side-detach--poll--abort): Lines 91-131 +- [Status values](#status-values): Lines 133-151 + +--- + ## Define the agent (store required) ```ts diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-custom.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-custom.md index 666e7f13..3a964c42 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-custom.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-custom.md @@ -8,6 +8,13 @@ the turn** — multiple sequential model calls, custom logic between them, manua message/state management, or custom progress streaming — use `ai.defineCustomAgent`. You provide the handler that runs the turn. +- [When to use it](#when-to-use-it): Lines 18-28 +- [Signature](#signature): Lines 30-54 +- [Example: multi-step research agent](#example-multi-step-research-agent): Lines 56-127 +- [Custom status streaming](#custom-status-streaming): Lines 129-148 + +--- + ## When to use it Reach for `defineCustomAgent` when a turn needs to: diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-deployment.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-deployment.md index 9f5d78ce..a5183d55 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-deployment.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-deployment.md @@ -15,6 +15,14 @@ expose two companion actions: These paths match the `remoteAgent` client defaults (`${url}/getSnapshot`, `${url}/abort`), so a client only needs the base `url`. +- [A reusable exposeAgent helper](#a-reusable-exposeagent-helper): Lines 26-75 +- [CORS for browser clients](#cors-for-browser-clients): Lines 77-98 +- [Serving a static web UI (optional)](#serving-a-static-web-ui-optional): Lines 100-118 +- [Registering agents/flows](#registering-agentsflows): Lines 120-137 +- [Other host frameworks](#other-host-frameworks): Lines 139-198 + +--- + ## A reusable `exposeAgent` helper When serving several agents, a small helper keeps registration consistent. diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-human-in-the-loop.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-human-in-the-loop.md index 74e69375..51f2786c 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-human-in-the-loop.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-human-in-the-loop.md @@ -18,6 +18,13 @@ snapshot does it; without one the client round-trips the state blob (the Flow: `chat.send(...)` → response has `res.interrupts` → collect human input → `chat.resume({ respond: [...] })`. +- [Define an interrupt](#define-an-interrupt): Lines 28-72 +- [Detect and resume (server-side)](#detect-and-resume-server-side): Lines 74-122 +- [Client-side (browser) interrupts](#client-side-browser-interrupts): Lines 124-163 +- [Notes & gotchas](#notes--gotchas): Lines 165-182 + +--- + ## Define an interrupt Define it like a tool (with `inputSchema`/`outputSchema`) and add it to the diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-multi-agent.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-multi-agent.md index 7e0dc6cf..afcf630d 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-multi-agent.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-multi-agent.md @@ -10,6 +10,16 @@ appends a `<sub-agents>` block to the orchestrator's system prompt, and — when the model calls a delegation tool — runs the sub-agent and returns its response as the tool result. +- [1. Define the sub-agents](#1-define-the-sub-agents): Lines 23-51 +- [2. Wire up the orchestrator](#2-wire-up-the-orchestrator): Lines 53-95 +- [agents() options](#agents-options): Lines 97-112 +- [Background delegation (async: true)](#background-delegation-async-true): Lines 114-154 +- [Continuing a task (continue_task)](#continuing-a-task-continue_task): Lines 156-173 +- [Sharing artifacts between agents](#sharing-artifacts-between-agents): Lines 175-186 +- [Other middleware](#other-middleware): Lines 188-212 + +--- + ## 1. Define the sub-agents Give each sub-agent a `description` — it's auto-discovered and shown to the diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-sessions.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-sessions.md index 8a170e2c..d011bc97 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-sessions.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-sessions.md @@ -10,6 +10,14 @@ conversation state forward. A store also enables [branching](agents-branching.md and [background execution](agents-background.md). (Interrupts work with or without a store — see [human-in-the-loop](agents-human-in-the-loop.md).) +- [Pick a store](#pick-a-store): Lines 21-66 +- [Typed session state](#typed-session-state): Lines 68-90 +- [Interrupts (human-in-the-loop)](#interrupts-human-in-the-loop): Lines 92-97 +- [Firestore session store (scalable, Beta)](#firestore-session-store-scalable-beta): Lines 99-132 +- [Implementing a custom SessionStore](#implementing-a-custom-sessionstore): Lines 134-155 + +--- + ## Pick a store ```ts diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-state.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-state.md index f75f0e93..02dcb681 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-state.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents-state.md @@ -7,6 +7,13 @@ own structured data (a task list, a workflow status, counters, etc.). Tools read and mutate it during a turn, and it is automatically synced to the [`remoteAgent`](agents.md#consume-an-agent-from-a-client-remoteagent) client. +- [Declare the state shape](#declare-the-state-shape): Lines 17-45 +- [Read & mutate state inside tools](#read--mutate-state-inside-tools): Lines 47-83 +- [Seed and read state (server-side)](#seed-and-read-state-server-side): Lines 85-102 +- [Auto-sync to the remoteAgent client](#auto-sync-to-the-remoteagent-client): Lines 104-140 + +--- + ## Declare the state shape Pass a `stateSchema` (Zod) to `defineAgent`. `State` is inferred from it and diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents.md index 3a821dbc..022d7049 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/agents.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/agents.md @@ -29,6 +29,17 @@ Progressive disclosure — read the file for the level you need: - [Advanced custom agents](agents-custom.md): `defineCustomAgent` for full turn control. - [Deploying agents](agents-deployment.md): serving multiple agents over HTTP, CORS, web UI, other frameworks. +- [Setup](#setup): Lines 43-54 +- [Define an agent](#define-an-agent): Lines 56-93 +- [Agents and middleware go hand in hand](#agents-and-middleware-go-hand-in-hand): Lines 95-134 +- [Chat with an agent (server-side)](#chat-with-an-agent-server-side): Lines 136-158 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 160-176 +- [Serve an agent over HTTP](#serve-an-agent-over-http): Lines 178-210 +- [Consume an agent from a client (remoteAgent)](#consume-an-agent-from-a-client-remoteagent): Lines 212-240 +- [Client-managed state (no server store)](#client-managed-state-no-server-store): Lines 242-289 + +--- + ## Setup ```ts diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/common-errors.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/common-errors.md index c3efd281..0b9c47df 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/common-errors.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/common-errors.md @@ -1,5 +1,15 @@ # Common Errors & Pitfalls +- [When Typecheck Fails](#when-typecheck-fails): Lines 13-15 +- [Genkit v1.x vs Pre-1.0 Migration](#genkit-v1x-vs-pre-10-migration): Lines 17-113 +- [Zod & Schema Errors](#zod--schema-errors): Lines 115-121 +- [Tool Usage](#tool-usage): Lines 123-126 +- [Handling Failures in Tool Loops](#handling-failures-in-tool-loops): Lines 128-159 +- [Multimodal & Image Generation](#multimodal--image-generation): Lines 161-169 +- [Audio & Speech Generation](#audio--speech-generation): Lines 171-176 + +--- + ## When Typecheck Fails **Before searching source code or docs**, check the sections below. Many type errors are caused by deprecated APIs or incorrect imports. @@ -98,6 +108,7 @@ Genkit v1.x introduced significant API changes. This section covers critical syn - **Incorrect (Pre-1.0)**: Importing `defineFlow` globally. ```ts import { defineFlow } from "@genkit-ai/flow"; // INCORRECT + ``` You should never import `@genkit-ai/flow`, `@genkit-ai/ai` or `@genkit-ai/core` packages directly. diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/dotprompt.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/dotprompt.md index 1c728ec4..bfb831a6 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/dotprompt.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/dotprompt.md @@ -1,5 +1,18 @@ # Dotprompt (.prompt files) — Genkit JS +- [What it is](#what-it-is): Lines 16-20 +- [Where files live](#where-files-live): Lines 22-35 +- [File format](#file-format): Lines 37-61 +- [Loading and calling a prompt](#loading-and-calling-a-prompt): Lines 63-100 +- [Registering named schemas](#registering-named-schemas): Lines 102-118 +- [Variants](#variants): Lines 120-127 +- [Partials](#partials): Lines 129-153 +- [Helpers](#helpers): Lines 155-165 +- [Tools, tool-loop control, and middleware](#tools-tool-loop-control-and-middleware): Lines 167-213 +- [Relationship to agents](#relationship-to-agents): Lines 215-219 + +--- + ## What it is `.prompt` files combine YAML frontmatter (model, config, schemas, tools, diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/examples.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/examples.md index 64e0ce88..57f13048 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/examples.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/examples.md @@ -7,6 +7,15 @@ This reference contains minimal, reproducible examples (MREs) for common Genkit > 2. Install and configure the plugin. > 3. Swap the model reference in the code. +- [Basic Text Generation](#basic-text-generation): Lines 19-33 +- [Structured Output](#structured-output): Lines 35-56 +- [Streaming](#streaming): Lines 58-73 +- [Advanced Configuration](#advanced-configuration): Lines 75-112 +- [Multimodal Generation](#multimodal-generation): Lines 114-174 +- [Embeddings](#embeddings): Lines 176-210 + +--- + ## Basic Text Generation ```ts @@ -163,3 +172,39 @@ if (media?.url) { Older preview TTS models (`gemini-2.5-*-preview-tts`, `gemini-3.1-flash-tts-preview`) are deprecated and return raw PCM that needs conversion to WAV. Run `genkit docs:search "speech audio"` for conversion code. + +## Embeddings + +```ts +const embedder = googleAI.embedder('gemini-embedding-001'); + +const embeddings = await ai.embed({ embedder, content: 'The sky is blue.' }); +const vector = embeddings[0].embedding; // number[] + +const batch = await ai.embedMany({ + embedder, + content: ['The sky is blue.', 'Grass is green.'], +}); + +// Optional tuning (default is 3072 dimensions) +await ai.embed({ + embedder, + content: 'The sky is blue.', + options: { outputDimensionality: 768, taskType: 'RETRIEVAL_DOCUMENT' }, +}); +``` + +Common embedders: `gemini-embedding-001` (text), `gemini-embedding-2` +(multimodal: text, image, video). Plain string ids +(`embedder: 'googleai/gemini-embedding-001'`) also work. + +Multimodal input is passed as a document: + +```ts +const imageEmbeddings = await ai.embed({ + embedder: googleAI.embedder('gemini-embedding-2'), + content: { + content: [{ media: { url: 'data:image/png;base64,...', contentType: 'image/png' } }], + }, +}); +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/middleware.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/middleware.md index 1620a4b6..4b85bdad 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/middleware.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/middleware.md @@ -28,6 +28,12 @@ a reference for `use: [...]`. The `@genkit-ai/middleware` package ships a set of ready-made ones (covered below); to write your own see [building custom middleware](middleware-custom.md). +- [Register middleware as a plugin](#register-middleware-as-a-plugin): Lines 37-58 +- [The @genkit-ai/middleware package](#the-genkit-aimiddleware-package): Lines 60-181 +- [Built-in model middleware (in core)](#built-in-model-middleware-in-core): Lines 183-205 + +--- + ## Register middleware as a plugin Register each middleware with Genkit via its `.plugin()` method. This is the diff --git a/plugins/genkit/.agents/skills/developing-genkit-js/references/observability.md b/plugins/genkit/.agents/skills/developing-genkit-js/references/observability.md index 668996d0..ee470d08 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-js/references/observability.md +++ b/plugins/genkit/.agents/skills/developing-genkit-js/references/observability.md @@ -3,6 +3,13 @@ > Pluggable instrumentation, `@genkit-ai/otel`, and the reflection API settings > require `genkit >= 1.43.0`. +- [How tracing works](#how-tracing-works): Lines 13-21 +- [Export OpenTelemetry GenAI semantic conventions (@genkit-ai/otel)](#export-opentelemetry-genai-semantic-conventions-genkit-aiotel): Lines 23-81 +- [Custom instrumentation](#custom-instrumentation): Lines 83-119 +- [Reflection API outside dev](#reflection-api-outside-dev): Lines 121-142 + +--- + ## How tracing works - **In dev** (`genkit start`), Genkit posts spans straight to the Developer UI's diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/SKILL.md b/plugins/genkit/.agents/skills/developing-genkit-python/SKILL.md index 4dc8a3e7..8ab8b3c9 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/SKILL.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/SKILL.md @@ -1,6 +1,11 @@ --- name: developing-genkit-python -description: Develop AI-powered applications using Genkit in Python. Use when the user asks about Genkit, AI agents, flows, or tools in Python, or when encountering Genkit errors, import issues, or API problems. +description: >- + Provides guidance on developing AI-powered applications using Genkit Python + (0.13) and migrating Genkit Python 0.12 projects to 0.13. Use when building AI + agents, flows, tools, structured outputs, or FastAPI routes in Python, + upgrading from Genkit 0.12, or debugging Genkit imports, config errors, or + traces. Don't use for Genkit in JavaScript/TypeScript, Go, or Dart. metadata: version: "1.0.0" category: AiAndMachineLearning @@ -8,16 +13,16 @@ metadata: # Genkit Python -Build AI features in Python — generate, stream, tools, flows, and multi-turn -agents — with one SDK. +Build AI features in Python — generation, structured output, streaming, tools, flows, and multi-turn agents — with Genkit Python 0.13. + +- **New project?** See [Setup](references/setup.md) and [Examples](references/examples.md). +- **Upgrading from 0.12 or fixing broken 0.12 imports/signatures?** Read [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md) first. ## Prerequisites - Python **3.10+** and **`uv`** ([install](https://docs.astral.sh/uv/getting-started/installation/)) - Genkit CLI: `npm install -g genkit-cli` if `genkit --version` is missing -New app? [Setup](references/setup.md). Patterns? [Examples](references/examples.md). - ## Hello World ```python @@ -26,26 +31,50 @@ from genkit_google_genai import GoogleAI ai = Genkit( plugins=[GoogleAI()], - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), ) -async def main(): - response = await ai.generate(prompt='Tell me a joke about Python.') + +async def main() -> None: + response = await ai.generate(prompt='Suggest one seasonal pasta dish.') + if response.error: + raise RuntimeError(response.error.message) print(response.text) + # => Lemony tagliatelle with asparagus, sweet peas, and herb ricotta + if __name__ == '__main__': ai.run_main(main()) ``` -## Agents (Beta) +`ai.generate` does not raise when the provider rejects or fails a call (`INVALID_ARGUMENT`, `UNAVAILABLE`, rate limits). It returns `response.error` (`Failure` with `.status` and `.message`), `finish_reason == 'failed'`, and empty text. Problems caught before the call raise: `GoogleAI()` raises `ValueError` without `GEMINI_API_KEY`; an unknown model, invalid key, or unknown config field raises `GenkitError`. See [Common Errors](references/common-errors.md). -Multi-turn chats with history, typed state, human approval, branching, and -background work. Start here: [Agents](references/agents.md). +## Agents (Experimental) + +Multi-turn chats with history, typed state, human approval, branching, and background work live under `genkit.exp` and `genkit.exp.agent`. Start here: [Agents](references/agents.md). ```python +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore +from genkit_google_genai import GoogleAI + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) +agent = ai.define_agent( + name='concierge', + system='You are a concise hotel concierge.', + store=InMemorySessionStore(), +) + chat = agent.chat() -res = await chat.send('Hello') # AgentResponse -turn = chat.send_stream('Hello') # AgentTurn — .stream / .response +res = await chat.send('Recommend a quiet dinner spot.') # AgentResponse +print(res.text) +# => Try Osteria Lume on 5th; it has booths and low music. + +turn = chat.send_stream('Is it open past 10pm?') # AgentTurn +async for chunk in turn.stream: + if chunk.text: + print(chunk.text, end='') +final = await turn.response # AgentResponse ``` More: [sessions](references/agents-sessions.md) · @@ -59,66 +88,79 @@ More: [sessions](references/agents-sessions.md) · ## Imports -- Google AI: `from genkit_google_genai import GoogleAI` -- Agents: `from genkit.agent import InMemorySessionStore, ...` -- Middleware: `from genkit_middleware import Middleware, ToolApproval, ...` -- FastAPI: `from genkit_fastapi import serve_agent, serve_flow` -- Evals: `from genkit_evaluators import register_genkit_evaluators` +Every symbol lives in one public module (never import private `_` submodules; see [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md) if a 0.12 symbol moved): + +- **Core app**: `from genkit import Genkit, FlowContext, ToolContext, Message, Part, Role, Media, Document, MultipartToolResponse, PublicError, GenkitError, Failure, ErrorReason` +- **Google GenAI & Vertex AI**: `from genkit_google_genai import GoogleAI, VertexAI, GeminiConfig, GeminiImageConfig, GeminiTtsConfig` +- **Other providers**: `from genkit_vertexai import ModelGarden` · `from genkit_anthropic import Anthropic, AnthropicConfig` · `from genkit_openai import OpenAI, OpenAIConfig` · `from genkit_ollama import Ollama, OllamaConfig` · `from genkit_amazon_bedrock import Bedrock, BedrockConfig` +- **Built-in middleware**: `from genkit_middleware import Middleware, Retry, Fallback, ToolApproval, Skills, Filesystem` (plus `from genkit_middleware.exp import Artifacts` for agent sessions) +- **Writing custom middleware**: `from genkit.middleware import Middleware, MiddlewareContext` +- **FastAPI & HTTP**: `from genkit_fastapi import serve_flow, genkit_fastapi_handler, handle_genkit_request` · `from genkit_fastapi.exp import serve_agent` · `from genkit.web import RequestData, to_sse, error_body, error_status` +- **Agents (experimental)**: `from genkit.exp import Genkit, current_session` · `from genkit.exp.agent import Agent, Session, InMemorySessionStore, FileSessionStore, remote_agent` +- **Telemetry**: `from genkit_google_cloud import enable_google_cloud_telemetry` · `from genkit.telemetry import configure_instrumentation` · `from genkit_otel import GenAiInstrumentation` +- **MCP (Client & Host)**: `from genkit_mcp import define_mcp_client, define_mcp_host, McpClient, McpHost` +- **Testing & Mocks**: `from genkit.testing import define_echo_model, define_scripted_model, EchoModel, ScriptedModel` +- **Evaluators (0.13 only; cut from 1.0, additive later)**: `from genkit_evaluators import register_genkit_evaluators` · `from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score, ScoreDetails, ScoreStatus` ## Workflow -1. **Agent or flow?** If the task is conversational, multi-turn, or described as - "an agent", "assistant", or "chatbot", build it with `ai.define_agent` (see - [Agents](references/agents.md)) rather than hand-rolling a `generate` + tools - loop inside a flow. Reach for a plain flow only for single-shot, stateless - generation. -2. Set **`GEMINI_API_KEY`**. Use prefixed model ids (`googleai/gemini-flash-latest`). -3. Enter via **`ai.run_main(main())`** for Genkit apps (especially under - `genkit start`). See [Common Errors](references/common-errors.md). -4. Run with [Dev Workflow](references/dev-workflow.md) (`genkit start` + Dev UI). -5. Verify with traces, not a blind run. Running the app directly (`uv run`) - does **not** capture dev traces. See [Genkit CLI](#genkit-cli-recommended) - for how to run your app and capture traces. -6. Stuck? [Common Errors](references/common-errors.md) first. +1. **Agent or flow?** Use `ai.define_agent` (on `genkit.exp.Genkit`, see [Agents](references/agents.md)) for multi-turn chats, state, or human-in-the-loop approval instead of hand-rolling a message loop inside a flow. Use `@ai.flow()` on `genkit.Genkit` for stateless request/response or streaming pipelines. +2. **Configure credentials & model refs.** Set **`GEMINI_API_KEY`**. Prefer typed refs (`GoogleAI.gemini('gemini-flash-latest')`) and typed flat configs (`GeminiConfig(...)`) so your type checker validates fields before runtime. Prefixed strings (`'googleai/gemini-flash-latest'`) also work. +3. **Enter via `ai.run_main(main())`** for standalone scripts and CLI apps so the process stays up for the Dev UI under `genkit start`. FastAPI apps skip `run_main`; uvicorn keeps the process alive: `genkit start -- uv run uvicorn main:app --reload`. +4. **Run with [Dev Workflow](references/dev-workflow.md)** (`genkit start` + Dev UI) and type-check with `pyright`, `mypy`, or `ty`. +5. **Verify with traces**, not a blind `uv run`. See [Genkit CLI](#genkit-cli-recommended). +6. **Stuck?** Check [Gotchas](#gotchas), [Common Errors](references/common-errors.md), or [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md). + +## Gotchas + +- **Agents need `genkit.exp`**: `from genkit.exp import Genkit`. Stable `genkit.Genkit` has no `define_agent`. +- **Tools are async**: `@ai.tool()` raises `TypeError` on a plain `def`; use `async def`. +- **Check `response.error`**: provider failures come back on the response, not as exceptions (see [Hello World](#hello-world)). +- **Gemini thinking levels**: `GeminiConfig(thinking_level='MINIMAL')` fails on `gemini-flash-latest`; use `'LOW'`, `'MEDIUM'`, or `'HIGH'`. +- **Vertex AI model IDs**: Vertex does not serve `-latest` aliases. Pass a pinned model ID to `VertexAI.gemini(...)`. +- **Logging**: `genkit.get_logger` is structlog-based. `enable_google_cloud_telemetry()` injects trace context for GCP log-trace correlation. +- **Cloud telemetry auth**: call `enable_google_cloud_telemetry()` with Application Default Credentials or explicit `credentials=` (service account dict or `Credentials` instance). ## Genkit CLI (recommended) -`genkit start` unintrusively wraps any Python program that uses the Genkit library, running it unchanged while capturing traces from every Genkit action so you can prove tools were actually called and inspect model I/O from the terminal, even for headless checks. It forwards stdio, so interactive CLI tools that rely on stdin/stdout work without issues. Running the app directly (`uv run`) skips trace capture, so you're debugging blind. +`genkit start` wraps any Python program that uses Genkit, capturing traces from every action while forwarding stdio cleanly. Running `uv run` directly skips trace capture. -**Primary pattern (default):** prefix `genkit start --` to your normal run command. This collects telemetry from any Genkit code your program runs, whether triggered from the dev UI, your own web server/web UI, or a plain script: +**Primary pattern (persistent server + Dev UI):** ```bash genkit start -- uv run src/main.py -genkit start --noui -- uv run src/main.py # same, without the Dev UI (still a persistent server) +genkit start --noui -- uv run src/main.py # persistent runtime without opening the Dev UI ``` -`genkit start` runs until you stop it with Ctrl+C. That is expected and correct for the common cases: a server your web/mobile app calls, or an interactive CLI you exit yourself. `--noui` only drops the Dev UI; it is **not** a one-shot command and will not exit on its own. Do **not** use `genkit start` as a blocking step in automated/non-interactive contexts; use `flow:run` (below) for that. +`genkit start` runs until stopped with Ctrl+C (even with `--noui`). Do **not** run `genkit start` as a blocking step in automated/non-interactive scripts; use `flow:run` instead. -**Non-interactive use (agents/CI):** add the global `--non-interactive` flag before `--` so the CLI uses defaults and never blocks on a prompt (e.g. the first-run analytics notice): `genkit start --non-interactive -- uv run src/main.py` (works with `flow:run` too). +**Non-interactive use (agents/CI):** pass `--non-interactive` before `--` so the CLI never blocks on prompts: +```bash +genkit start --non-interactive -- uv run src/main.py +``` -**Run a flow (`flow:run`):** invoke a specific flow by name from the CLI. Append your run command after `--` to spin up the runtime just for this run (the command runs as-is to register your flows): +**One-shot flow execution (`flow:run`):** spins up the runtime, invokes one `@ai.flow()` by name, prints a `Trace ID`, and exits: ```bash genkit flow:run myFlow '{"data": "input"}' -- uv run src/main.py ``` -This is **self-terminating**: it runs the flow once, prints a `Trace ID`, then exits, so it's the right choice for a quick, non-interactive check (unlike `genkit start`). Note: `flow:run` runs **flows** (`@ai.flow()`), not agents; you can't `flow:run` an agent (`ai.define_agent`) directly. To exercise an agent from the CLI, wrap one turn in a throwaway flow and run that (see [Agents](references/agents.md)). +Note: `flow:run` invokes flows (`@ai.flow()`), not agents (`ai.define_agent`). Wrap a single agent turn in a helper flow to exercise it via `flow:run` (see [Agents](references/agents.md)). -**Debugging with traces:** the fastest way to see prompts, model inputs/outputs, tool calls, latencies, and errors. Inspect from the terminal after any run under `genkit start`: +**Inspecting traces:** ```bash -genkit trace:list # find recent trace IDs -genkit trace:get <traceId> # full trace details (inputs, outputs, tool calls, errors) -genkit trace:get <traceId> --format json # machine-readable JSON, safe to pipe into jq or other parsers +genkit trace:list # list recent trace IDs +genkit trace:get <traceId> # human-readable trace inspection +genkit trace:get <traceId> --format json # machine-readable JSON for jq / scripts ``` +Always pass `--format json` when piping into `jq` or other parsers. -For machine-readable output, pass `--format json` to get clean JSON you can pipe into `jq` or other parsers. The **default** output is human-oriented (banner/log lines, possible truncation on large traces), so don't pipe that form directly; use `--format json`, grep, or the Dev UI trace viewer. - - -See [Dev Workflow](references/dev-workflow.md) for the full checklist and Dev UI walkthrough. +See [Dev Workflow](references/dev-workflow.md) for the full pre-run checklist and Dev UI walkthrough. ## References -- [Examples](references/examples.md): Structured output, streaming, flows, tools, embeddings. -- [Setup](references/setup.md): New project bootstrap and plugins. -- [Common Errors](references/common-errors.md): Read first when something breaks. -- [FastAPI](references/fastapi.md): HTTP, `genkit_fastapi_handler`, parallel flows. -- [Dotprompt](references/dotprompt.md): `.prompt` files and helpers. -- [Evals](references/evals.md): Evaluators and datasets. -- [Dev Workflow](references/dev-workflow.md): `genkit start`, Dev UI, checklist. -- [Agents (Beta)](references/agents.md): Multi-turn API. +- [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md): Deterministic 0.12 -> 0.13 renames, config flattening, `PublicError`, and `exp` moves. +- [Examples](references/examples.md): Structured output, streaming, media parts, flows, tools, interrupts, cancellation, embeddings. +- [Setup](references/setup.md): Project bootstrap with `uv`, version alignment, and provider plugins. +- [Common Errors](references/common-errors.md): Fast fixes for 0.13 validation errors, HTTP 500 redaction, streaming, and sessions. +- [FastAPI](references/fastapi.md): `serve_flow`, `serve_agent`, `PublicError`, SSE streaming, and parallel flows. +- [Dotprompt](references/dotprompt.md): `.prompt` files, schema validation, variants, partials, and helpers. +- [Evals](references/evals.md): 0.13 evaluator syntax (deprecated; removed in 1.0). +- [Dev Workflow](references/dev-workflow.md): `genkit start`, Dev UI, and trace verification checklist. +- [Agents (Experimental)](references/agents.md): Multi-turn stateful agents (`genkit.exp`). diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-artifacts.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-artifacts.md index 3b5e66fa..e0886a6d 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-artifacts.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-artifacts.md @@ -1,64 +1,98 @@ -# Artifacts (Beta) +# Agent Artifacts (Experimental) -> See [agents.md](agents.md). +> See [agents.md](agents.md) · [sessions](agents-sessions.md). -Named deliverables on the session — reports, files, code — available as -`chat.artifacts` and `res.artifacts`. +Artifacts are named session deliverables — markdown reports, SQL migrations, generated files — persisted in `SessionState.artifacts` and exposed as `chat.artifacts`, `res.artifacts`, and streamed `chunk.artifact`. Writing an artifact whose `name` already exists replaces the previous version in place. -## Middleware +## `Artifacts` middleware (`genkit_middleware.exp`) -`Artifacts()` gives the model `write_artifact` and `read_artifact`. Writing the -same name replaces the previous version. +Import `Artifacts` from `genkit_middleware.exp` (`Middleware()` does not register it). On every model call, `Artifacts()` appends an `<artifacts>` listing of the session's current artifacts (name, size, source) to the system prompt, and gives the model `write_artifact` and `read_artifact` tools. Pass `Artifacts(readonly=True)` to expose only `read_artifact`. Outside an agent session the tools return an error result instead of raising. -Store-backed agents cannot seed with `chat(artifacts=...)`. Have the model -write on the first turn, or call `add_artifacts` from a custom agent. +On a store-backed agent, `agent.chat(artifacts=...)` raises `AgentInitError` (`FAILED_PRECONDITION`); have the model write artifacts on a turn, or write them from a tool or [custom agent](agents-custom.md). Store-less agents can seed `artifacts=` ([state](agents-state.md)). ```python from genkit_google_genai import GoogleAI -from genkit_middleware import Artifacts, Middleware +from genkit_middleware.exp import Artifacts -from genkit import Genkit -from genkit.agent import InMemorySessionStore +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore -ai = Genkit(plugins=[GoogleAI(), Middleware()]) +ai = Genkit(plugins=[GoogleAI()]) agent = ai.define_agent( name='workspaceAgent', - model='googleai/gemini-flash-latest', - system='Use write_artifact for files. Use read_artifact to review them.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Use write_artifact to save deliverables and read_artifact to inspect them.', use=[Artifacts()], store=InMemorySessionStore(), ) -chat = agent.chat() -await chat.send('Write poem.txt with a short poem about Python agents.') -print([a.name for a in chat.artifacts]) + +async def main() -> None: + chat = agent.chat() + + # 1. Stream the turn and observe artifacts as they are written + turn = chat.send_stream('Write schema.sql for a multi-tenant invoices table.') + async for chunk in turn.stream: + if chunk.artifact is not None: + print('wrote artifact:', chunk.artifact.name) + # => wrote artifact: schema.sql + await turn.response + + # 2. Inspect persisted session artifacts + print([a.name for a in chat.artifacts]) + # => ['schema.sql'] + + +if __name__ == '__main__': + ai.run_main(main()) ``` -## From a custom agent +## Writing artifacts from tools and custom agents -Pass a list: +Inside an `@ai.tool()`, get the active session with `current_session()` from `genkit.exp` (`None` outside an agent turn). Inside a custom agent, call the same method on the `SessionRunner`. `add_artifacts` takes a `list[Artifact]`, replaces by name, and emits one `chunk.artifact` per artifact to streaming callers: ```python -from genkit import Part, TextPart -from genkit.agent import Artifact +from genkit import Part +from genkit.exp import current_session +from genkit.exp.agent import Artifact + + +@ai.tool(name='exportAuditReport', description='Save an audit report as a session artifact.') +async def export_audit_report(markdown_body: str) -> dict[str, object]: + sess = current_session() + if sess is None: + return {'ok': False, 'error': 'no active agent session'} + + await sess.add_artifacts([ + Artifact( + name='audit-report.md', + parts=[Part.from_text(markdown_body)], + metadata={'source': 'exportAuditReport'}, + ) + ]) + return {'ok': True, 'name': 'audit-report.md'} +``` -await sess.add_artifacts( - [Artifact(name='report.md', parts=[Part(TextPart(text=body))])] -) +From a custom agent's `handle_turn`: + +```python +from genkit import Part +from genkit.exp.agent import Artifact, SessionRunner + + +async def save_report(sess: SessionRunner, body: str) -> None: + await sess.add_artifacts([Artifact(name='report.md', parts=[Part.from_text(body)])]) ``` -## Reading text +## Reading artifact text -Parts use a RootModel — read text from `.root` when it is a `TextPart`: +`Part.text` is `str | None`. This matches what `read_artifact` returns to the model: ```python -from genkit import TextPart - -def artifact_text(artifact) -> str: - return ''.join( - p.root.text - for p in (artifact.parts or []) - if isinstance(p.root, TextPart) - ) +from genkit.exp.agent import Artifact + + +def artifact_text(artifact: Artifact) -> str: + return '\n'.join(p.text for p in artifact.parts if p.text) ``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-background.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-background.md index 5fa59b45..dd4d1840 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-background.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-background.md @@ -1,69 +1,129 @@ -# Background Agents / Detaching (Beta) +# Background Agents & Cancellation (Experimental) -> Needs a [session store](agents-sessions.md). See [agents.md](agents.md). +> Requires a [session store](agents-sessions.md). For abort to cancel a running turn, the store must also implement `SnapshotSubscriber` (`InMemorySessionStore`, `FileSessionStore`, and `FirestoreSessionStore` do). See [agents.md](agents.md). -Hand work to the server and keep going. `detach` returns a task handle with a -`snapshot_id` right away. When you need the reply, load that snapshot. +`await chat.detach(...)` hands a turn to the server, reserves a `snapshot_id` in `SnapshotStatus.PENDING` (with background heartbeat refreshes), and returns a `DetachedTask` handle immediately without waiting for model or tool completion. + +- [Detach, poll, and load results](#detach-poll-and-load-results): Lines 14-56 +- [Client detach vs. server cancellation](#client-detach-vs-server-cancellation): Lines 58-82 +- [Tool cancellation (asyncio.CancelledError)](#tool-cancellation-asynciocancellederror): Lines 84-102 +- [Fan-out parallel research](#fan-out-parallel-research): Lines 104-129 + +--- + +## Detach, poll, and load results + +Because the originating `chat` never streams a detached turn's chunks, rehydrate the finished conversation with `await agent.load_chat(snapshot_id=task.snapshot_id)` once the task settles: ```python -from genkit.agent import InMemorySessionStore +from genkit_google_genai import GoogleAI + +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore, SnapshotStatus + +ai = Genkit(plugins=[GoogleAI()]) agent = ai.define_agent( - name='backgroundAgent', - model='googleai/gemini-flash-latest', - system='Senior research analyst. Produce a comprehensive markdown report.', + name='backgroundAnalyst', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Research analyst. Write concise markdown memos.', store=InMemorySessionStore(), ) -chat = agent.chat() -task = await chat.detach('Write a report on renewable energy trends') -print(task.snapshot_id) -snapshot = await task.wait(interval=2.0) -print(snapshot.status) +async def main() -> None: + # 1. Start the turn on the server and get a snapshot handle immediately + chat = agent.chat() + task = await chat.detach('Summarize grid battery storage economics.') + print(task.snapshot_id) + + # 2. Stream snapshot status updates (or await task.wait(interval=1.0) directly) + snapshot = None + async for snap in task.poll(interval=1.0): + print(snap.status) + snapshot = snap + + # 3. Verify terminal status and load the completed snapshot + assert snapshot is not None and snapshot.status == SnapshotStatus.COMPLETED + done = await agent.load_chat(snapshot_id=task.snapshot_id) + print(done.messages[-1].text) + + +if __name__ == '__main__': + ai.run_main(main()) +``` + +Terminal snapshot statuses on `poll()` / `wait()` are `COMPLETED`, `FAILED`, `ABORTED`, and `EXPIRED` (reported on read when a pending turn's heartbeat goes stale). Always check `snapshot.status == SnapshotStatus.COMPLETED` before reading reply messages. + +## Client detach vs. server cancellation -await task.abort() +Always `await` abort calls. Genkit distinguishes stopping a local listener from cancelling work on the server: -done = await agent.load_chat(snapshot_id=task.snapshot_id) -print(done.messages) +- **Client detach (`await turn.abort()` or `asyncio.timeout` / `asyncio.wait_for`)**: stops streaming locally and resolves `turn.response` as cancelled immediately while the server turn keeps running to completion in the background. Wrapping `turn.stream` or `await turn.response` in `async with asyncio.timeout(...)` detaches the client the same way and re-raises `TimeoutError`. +- **Server cancel (`await task.abort()`, `await chat.abort()`, or `await agent.abort(snapshot_id)`)**: flips a pending snapshot to `SnapshotStatus.ABORTED` in the store and cancels the running server task. Returns the snapshot's **previous** `SnapshotStatus` (`SnapshotStatus.PENDING` if this call stopped a live turn; a terminal status if the turn had already finished). + +`chat.abort()` targets the chat's last recorded snapshot, so calling it while a foreground `send()` is in flight does not stop that turn: it returns the previous turn's status (`SnapshotStatus.COMPLETED`), or raises `ValueError` on a first turn with no snapshot yet. Use `detach()` + `task.abort()` to cancel a running turn. + +```python +chat = agent.chat() +task = await chat.detach('Audit every quarterly filing since 2015.') + +# Cancel server-side execution; returns the status from *before* the abort write +previous_status = await task.abort() +# => SnapshotStatus.PENDING (the snapshot row is now SnapshotStatus.ABORTED) + +# Aborted leaves are not resumable directly; continue from the last completed turn via session_id +recovered = agent.chat(session_id=chat.session_id) +res = await recovered.send('Audit only Q4 2025 instead.') +print(res.text) +# => Q4 2025 audit: ... ``` -## Stopping work +On the very first turn of a session, a client-side `turn.abort()` detaches before the client receives `session_id` or `snapshot_id`. Prefer `await chat.detach(...)` + `await task.abort()` whenever you need a recoverable server-side cancel handle from turn start. -Always `await` aborts — otherwise nothing happens. +## Tool cancellation (`asyncio.CancelledError`) -**Client stop** — `await turn.abort()` (or `asyncio.timeout` around the stream) -stops listening. The server may still finish. Use this mid-stream when you do -not have a snapshot id yet. +When a server turn is aborted (or an in-process caller cancels), Python delivers standard `asyncio.CancelledError` into active model requests, retry sleeps, and running `@ai.tool()` coroutines at the current `await` (there is no abort-signal attribute on `ToolContext`). Clean up partial external work in `try` / `finally` (and always re-raise `CancelledError` if caught): -**Server cancel** — `await chat.abort()` or `await task.abort()`. Needs a store -and an existing snapshot (after a completed turn or after `detach`). +```python +import asyncio -Aborted snapshots are not resume points. Reload the last good leaf. -On the first turn, a client abort can leave this chat without ids even if the -server later saves. Prefer `detach` + `task.abort()` when you need a -recoverable cancel. +@ai.tool(name='runBackfill', description='Run a multi-batch warehouse backfill.') +async def run_backfill(table: str) -> dict[str, object]: + job_id = f'job-{table}' + try: + await asyncio.sleep(10.0) + return {'job_id': job_id, 'status': 'completed'} + finally: + # Runs whether the tool returns normally, fails, or receives CancelledError + print(f'releasing lease for {job_id}') + # => releasing lease for job-orders (also after await task.abort() mid-run) +``` -## Parallel research +## Fan-out parallel research -One `chat` can only host one detach. Fork a leaf from a shared checkpoint for -each branch: +A single `AgentChat` runs one turn at a time. To run background branches concurrently, fork independent chats from a shared checkpoint `snapshot_id`: ```python +import asyncio + root = agent.chat() -await root.send('Context both researchers should know.') +await root.send('Context: B2B SaaS workload, 50k RPS peak, strict multi-region RPO.') checkpoint = root.snapshot_id +assert checkpoint is not None + -async def research(topic: str): - leaf = await agent.load_chat(snapshot_id=checkpoint) - task = await leaf.detach(f'Research {topic} in depth.') - await task.wait(interval=2.0) +async def research(engine: str) -> tuple[str, str]: + branch = await agent.load_chat(snapshot_id=checkpoint) + task = await branch.detach(f'Evaluate {engine} against our requirements.') + snap = await task.wait(interval=1.0) + if snap.status != SnapshotStatus.COMPLETED: + return engine, f'unfinished ({snap.status})' done = await agent.load_chat(snapshot_id=task.snapshot_id) - return topic, done.messages[-1] + return engine, done.messages[-1].text -import asyncio -results = await asyncio.gather(research('Postgres'), research('SQLite')) -``` -Check the loaded messages after wait — status alone is not enough. +results = await asyncio.gather(research('Cloud Spanner'), research('AlloyDB')) +# => [('Cloud Spanner', '...'), ('AlloyDB', '...')] +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-branching.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-branching.md index b17be4f8..cb793284 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-branching.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-branching.md @@ -1,43 +1,73 @@ -# Agent Branching (Beta) +# Agent Branching & Time Travel (Experimental) -> **Beta / preview API.** Needs a [session store](agents-sessions.md). -> See [agents.md](agents.md). +> Requires a [session store](agents-sessions.md). See [agents.md](agents.md). -A `snapshot_id` is an immutable checkpoint. Load it with -`load_chat(snapshot_id=...)` and send again — each turn creates a new leaf; -the original stays put. +Every store-backed turn writes a `SessionSnapshot` keyed by `snapshot_id`, with `parent_id` pointing at the previous turn. Reopening an earlier `snapshot_id` with `await agent.load_chat(snapshot_id=...)` and sending a new message writes a sibling snapshot under the same parent. The original timeline is untouched, and both branches keep the same `session_id`. + +Loading a snapshot restores the **entire** `SessionState` at that point: `messages`, custom `state`, and `artifacts` revert together. `load_chat` only resumes `completed` snapshots (see [sessions](agents-sessions.md) for `load_chat` vs `chat(snapshot_id=...)`). + +## Fork sibling timelines ```python -from genkit.agent import InMemorySessionStore +from genkit_google_genai import GoogleAI + +from genkit import GenkitError +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore -# After a fork, session_id is ambiguous — raise instead of guessing. +ai = Genkit(plugins=[GoogleAI()]) + +# reject_ambiguous_session=True makes session_id lookups fail loudly once a session forks store = InMemorySessionStore(reject_ambiguous_session=True) agent = ai.define_agent( name='designer', - model='googleai/gemini-flash-latest', - system='You help design a product landing page. Reply briefly.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='You design product landing pages. Reply in two short sentences.', store=store, ) -root = agent.chat() -await root.send('Plan a landing page for a note-taking app.') -checkpoint = root.snapshot_id -minimal = await agent.load_chat(snapshot_id=checkpoint) -await minimal.send('Direction: minimal.') +async def main() -> None: + # 1. Shared root turn; bookmark snapshot_id as the fork point + root = agent.chat() + await root.send('Plan a landing page for an AI note-taking app.') + checkpoint = root.snapshot_id + session_id = root.session_id + assert checkpoint and session_id -bold = await agent.load_chat(snapshot_id=checkpoint) -await bold.send('Direction: bold.') + # 2. Fork two independent branches from the same checkpoint + minimal = await agent.load_chat(snapshot_id=checkpoint) + await minimal.send('Direction: minimal with generous whitespace.') -resumed = await agent.load_chat(snapshot_id=bold.snapshot_id) -await resumed.send('Add a pricing section.') -``` + bold = await agent.load_chat(snapshot_id=checkpoint) + await bold.send('Direction: dark mode and high contrast.') + chosen_leaf = bold.snapshot_id + assert chosen_leaf -Same pattern for time travel: reopen an earlier snapshot and send a different -follow-up. After forks, resume by `snapshot_id`, not `session_id`: + print(bold.session_id == minimal.session_id == session_id) + # => True + print([m.text for m in bold.messages if m.role == 'user']) + # => ['Plan a landing page for an AI note-taking app.', 'Direction: dark mode and high contrast.'] -```python -# Ambiguous when reject_ambiguous_session=True — pick a leaf instead: -resumed = await agent.load_chat(snapshot_id=bold.snapshot_id) + # 3. With two leaves, a bare session_id lookup raises FAILED_PRECONDITION + try: + await store.get_snapshot(session_id=session_id) + except GenkitError as err: + print(err.status) + # => FAILED_PRECONDITION + + # 4. Continue linearly from the chosen branch leaf + resumed = await agent.load_chat(snapshot_id=chosen_leaf) + await resumed.send('Add a pricing section.') + + +if __name__ == '__main__': + ai.run_main(main()) ``` + +## Branching rules + +- **Resume by `snapshot_id` after forking**: Once a session has multiple leaf snapshots, "latest turn for `session_id`" is ambiguous. By default the store picks the most recently created leaf. Pass `reject_ambiguous_session=True` on `InMemorySessionStore` or `FileSessionStore` to fail instead: `store.get_snapshot(session_id=...)` and `agent.get_snapshot(session_id=...)` raise `GenkitError` (`FAILED_PRECONDITION`), and a turn sent on `agent.chat(session_id=...)` returns a failed turn with that status on `res.error`. +- **Edit-and-resubmit / regenerate**: Store each turn's `snapshot_id` alongside the message in your UI. When a user edits an earlier prompt or clicks regenerate, load the parent turn's `snapshot_id` and send the revised message. Previous answers stay intact as sibling branches. +- **Artifact time travel**: When an agent edits files via `Artifacts()` (`genkit_middleware.exp`, see [artifacts](agents-artifacts.md)) or a tool, rewinding with `load_chat(snapshot_id=checkpoint)` restores `chat.artifacts` and `chat.state` to their values at that turn. diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-custom.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-custom.md index f93b4dc6..6d054565 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-custom.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-custom.md @@ -1,51 +1,59 @@ -# Advanced Custom Agents — `define_custom_agent` (Beta) +# Custom Agents — `define_custom_agent` (Experimental) -> **Beta / preview API.** See [agents.md](agents.md). +> See [agents.md](agents.md). -`define_agent` runs a fixed prompt + tool loop. For your own orchestration -(multi-step, custom streaming, manual messages/state/artifacts), use -`ai.define_custom_agent`. Callers still use `chat.send` / `send_stream`. +`ai.define_agent` runs a built-in model + tool loop. When you need custom multi-step orchestration, subagent delegation, custom streaming, or manual control over messages, custom state, and artifacts, register a custom turn loop with `ai.define_custom_agent`. Callers still talk to the agent through `chat.send(...)`, `chat.send_stream(...)`, `chat.detach(...)`, and `serve_agent(...)`. + +- [Anatomy](#anatomy): Lines 13-46 +- [Streaming custom agent example](#streaming-custom-agent-example): Lines 48-114 +- [Non-streaming handle_turn with tools](#non-streaming-handle_turn-with-tools): Lines 116-170 + +--- ## Anatomy -Register one async `fn`. Inside it, define `handle_turn`, hand it to -`sess.run`, return `sess.result()`: +Register one async `fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult`. Inside `fn`, define `handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None`, pass it to `await sess.run(handle_turn)`, and return `await sess.result()`: ```python -async def fn(sess: SessionRunner, ctx: ActionRunContext) -> AgentResult: +from genkit.exp.agent import ( + AgentContext, + AgentFinishReason, + AgentInput, + AgentResult, + SessionRunner, + TurnContext, + TurnResult, +) + + +async def custom_fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult: async def handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None: - # user message already in sess.get_messages() - # model / tools / logic… - # ctx.send_chunk(AgentStreamChunk(...)) to stream - # sess.add_messages(...) to persist replies + # 1. inp.message is already appended to sess.get_messages() before handle_turn runs + # 2. Emit live chunks with ctx.send_chunk(AgentStreamChunk(...)) + # 3. Persist assistant/tool messages via sess.add_messages([...]) or sess.set_messages([...]) return TurnResult(finish_reason=AgentFinishReason.STOP) await sess.run(handle_turn) return await sess.result() - - -agent = ai.define_custom_agent(name='customCoder', fn=fn, store=store) ``` -`handle_turn` runs once per user turn. Before it runs, the runtime appends -`inp.message` to history. Return a `TurnResult` (or `None`); common finish -reasons: `STOP`, `INTERRUPTED`, `FAILED`. Raising ends the invocation as -`FAILED`. +Runtime mechanics: -`turn_ctx` (store only): `snapshot_id` (name external worktrees/dirs under -this), `parent_snapshot_id`, `turn_index`. Without a store, `snapshot_id` is -`None`. +- **Automatic user message append**: Before calling `handle_turn`, `sess.run` appends `inp.message` (`AgentInput` carries `message`, `resume`, and `detach`) to session history. `await sess.get_messages()` already returns `list[Message]` including the current user prompt — do not wrap items in `Message(m)`. +- **`TurnContext` (`turn_ctx`)**: Exposes `snapshot_id` (pre-reserved at turn start when a `store` is configured, `None` otherwise), `parent_snapshot_id`, and `turn_index`. Pre-reserving `snapshot_id` lets you name external worktrees or scratch directories after the snapshot before the turn commits. `turn_index` is 0-based within one invocation; each `chat.send` starts a new invocation, so it does not count turns across the session. +- **`SessionRunner` (`sess`) helpers**: `get_messages()` / `add_messages([...])` / `set_messages([...])`, `get_custom()` / `update_custom(sync_fn)`, and `get_artifacts()` / `add_artifacts([...])`. Both `add_messages` and `add_artifacts` take a `list`. +- **Turn completion and failures**: `TurnResult` accepts `(finish_reason=..., error=...)` — it does **not** take `messages=`. Raising an exception inside `handle_turn` ends the turn with `finish_reason=AgentFinishReason.FAILED`, populates `res.error` on the caller's `AgentResponse`, and stops the turn loop. `chat.send` returns that failed turn; it does not raise. +- **Model failures do not raise**: `ai.generate` and `ai.generate_stream` report provider errors on `response.error` with `finish_reason='failed'`. Raise a `GenkitError` from it so the caller's `res.error` keeps the provider status (e.g. `UNAVAILABLE`); otherwise the caller sees a generic `UNKNOWN` "Agent turn failed." (or a silent `STOP` if you hardcode the finish reason). -From `sess`: `get_messages` / `add_messages` / `set_messages`, -`get_custom` / `update_custom`, `get_artifacts` / `add_artifacts`. -`add_messages` and `add_artifacts` each take a **list**. Stream with -`ctx.send_chunk(AgentStreamChunk(...))`. - -## Example +## Streaming custom agent example ```python -from genkit import ActionRunContext, FinishReason, Genkit, Message -from genkit.agent import ( +from genkit_google_genai import GoogleAI + +from genkit import GenkitError +from genkit.exp import Genkit +from genkit.exp.agent import ( + AgentContext, AgentFinishReason, AgentInput, AgentResult, @@ -55,35 +63,36 @@ from genkit.agent import ( TurnContext, TurnResult, ) -from genkit_google_genai import GoogleAI ai = Genkit(plugins=[GoogleAI()]) store = InMemorySessionStore() -async def custom_coder_fn(sess: SessionRunner, ctx: ActionRunContext) -> AgentResult: +async def custom_coder_fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult: async def handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None: + # 1. Read history (already includes inp.message as list[Message]) history = await sess.get_messages() - messages = [Message(m) for m in history] if history else None + # 2. Stream model chunks out through AgentContext stream_resp = ai.generate_stream( - model='googleai/gemini-flash-latest', - system='Concise coding assistant.', - messages=messages, + model=GoogleAI.gemini('gemini-flash-latest'), + system='Concise Python systems assistant.', + messages=history, ) async for chunk in stream_resp.stream: ctx.send_chunk(AgentStreamChunk(model_chunk=chunk)) + # 3. Fail the turn on model errors (generate reports them instead of raising) res = await stream_resp.response + if res.error is not None: + raise GenkitError(status=res.error.status, message=res.error.message) + + # 4. Persist the assistant reply so subsequent turns see it if res.message: await sess.add_messages([res.message]) - fr = ( - AgentFinishReason.STOP - if res.finish_reason == FinishReason.STOP - else AgentFinishReason.UNKNOWN - ) - return TurnResult(finish_reason=fr) + # 5. Every model FinishReason value exists on AgentFinishReason + return TurnResult(finish_reason=AgentFinishReason(res.finish_reason.value)) await sess.run(handle_turn) return await sess.result() @@ -91,26 +100,71 @@ async def custom_coder_fn(sess: SessionRunner, ctx: ActionRunContext) -> AgentRe agent = ai.define_custom_agent(name='customCoder', fn=custom_coder_fn, store=store) -chat = agent.chat() -await chat.send('What is a Python list comprehension?') -``` -Persist model replies with `sess.add_messages` or the next turn won't see them. -Custom state → [state](agents-state.md). Artifacts → [artifacts](agents-artifacts.md). +async def main() -> None: + chat = agent.chat() + res = await chat.send('Show a dict comprehension that inverts keys and values.') + print(res.text) + # => inverted = {v: k for k, v in original.items()} + # (model reply, usually wrapped in a markdown code fence) + +if __name__ == '__main__': + ai.run_main(main()) +``` ## Non-streaming `handle_turn` with tools +When `ai.generate(..., tools=[...])` runs a multi-step tool loop inside `handle_turn`, `res.messages` contains the full conversation transcript (prior history + intermediate tool requests/responses + final model reply, plus a leading `Role.SYSTEM` message when `system=` was passed). Strip `Role.SYSTEM` and save the updated transcript with `await sess.set_messages(...)`: + ```python -async def handle_turn(inp: AgentInput, ctx: TurnContext) -> TurnResult | None: - history = [Message(m) for m in await sess.get_messages()] - # ai.generate runs the tool loop for you - res = await ai.generate( - messages=history + list(inp.messages or []), - tools=[get_time], - system='You are helpful.', - ) - return TurnResult(messages=res.messages, finish_reason=AgentFinishReason.STOP) +from datetime import datetime, timezone + +from genkit_google_genai import GoogleAI + +from genkit import GenkitError, Role +from genkit.exp import Genkit +from genkit.exp.agent import ( + AgentContext, + AgentFinishReason, + AgentInput, + AgentResult, + InMemorySessionStore, + SessionRunner, + TurnContext, + TurnResult, +) + +ai = Genkit(plugins=[GoogleAI()]) + + +@ai.tool(name='utcNow', description='Return the current UTC timestamp.') +async def utc_now() -> str: + return datetime.now(timezone.utc).isoformat() + + +async def ops_agent_fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult: + async def handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None: + # 1. Run the model + tool loop over the session history + history = await sess.get_messages() + res = await ai.generate( + model=GoogleAI.gemini('gemini-flash-latest'), + system='Ops assistant. Use utcNow when asked for the current time.', + messages=history, + tools=[utc_now], + ) + if res.error is not None: + raise GenkitError(status=res.error.status, message=res.error.message) + + # 2. Persist tool-request, tool-response, and final model messages without duplicating system + await sess.set_messages([m for m in res.messages if m.role != Role.SYSTEM]) + return TurnResult(finish_reason=AgentFinishReason(res.finish_reason.value)) + + await sess.run(handle_turn) + return await sess.result() + + +ops_agent = ai.define_custom_agent(name='opsAgent', fn=ops_agent_fn, store=InMemorySessionStore()) ``` -Tool inputs: [agents.md](agents.md#tool-inputs). +Related guides: [state](agents-state.md) · [artifacts](agents-artifacts.md) · [tools](agents.md#tools-progress-chunks-and-multipart-responses) diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-http.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-http.md index fd70c50d..6d690cb2 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-http.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-http.md @@ -1,113 +1,200 @@ -# Agent HTTP (Beta) +# Agent HTTP Serving & Remote Client (Experimental) -> See [agents.md](agents.md). +> See [agents.md](agents.md) · [FastAPI](fastapi.md). -Serve an agent over FastAPI. Talk to it from another process with -`remote_agent`. Flows use `serve_flow` — [FastAPI](fastapi.md). +Mount an agent on FastAPI with `serve_agent` from `genkit_fastapi.exp`, and connect from another Python service or CLI with `remote_agent` from `genkit.exp.agent`. Stable flows use `serve_flow` from `genkit_fastapi` ([FastAPI](fastapi.md)). -## Serve +- [Serve an agent with FastAPI](#serve-an-agent-with-fastapi): Lines 14-49 +- [Connect with remote_agent](#connect-with-remote_agent): Lines 51-87 +- [Auth, context_dependency, and PublicError](#auth-context_dependency-and-publicerror): Lines 89-160 +- [Custom HTTP routes](#custom-http-routes): Lines 162-200 + +--- + +## Serve an agent with FastAPI + +`serve_agent(agent)` mounts `POST /{agent.name}` for turns: one-shot JSON by default, SSE when the caller sends `Accept: text/event-stream` or `?stream=true`. When the agent has a `store`, it also mounts `POST /{agent.name}/getSnapshot` (`404` when the snapshot is missing) and `POST /{agent.name}/abort`. Callers can pass `?session_id=` or `?thread_id=` to continue a session when the body `init` carries no session id. ```python import uvicorn from fastapi import FastAPI -from fastapi.middleware.cors import CORSMiddleware -from genkit import Genkit -from genkit.agent import InMemorySessionStore -from genkit_fastapi import serve_agent +from genkit_fastapi.exp import serve_agent from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore +# 1. Initialize the experimental Genkit instance +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) + +# 2. Define a store-backed agent agent = ai.define_agent( name='weatherAgent', - system='Weather assistant. Be concise.', + system='Weather assistant. Keep answers under two sentences.', store=InMemorySessionStore(), ) +# 3. Mount /api/weatherAgent, /api/weatherAgent/getSnapshot, and /api/weatherAgent/abort app = FastAPI() -app.add_middleware( - CORSMiddleware, - allow_origins=['*'], - allow_methods=['*'], - allow_headers=['*'], - expose_headers=['X-Genkit-Stream-Id'], -) - app.include_router(serve_agent(agent), prefix='/api') if __name__ == '__main__': uvicorn.run(app, host='0.0.0.0', port=8080) ``` -Routes land at `/{agent.name}` by default. Override with `base_path=...`. +Override the route path with `serve_agent(agent, base_path='/chat')`. Add FastAPI's `CORSMiddleware` as usual if browsers call the agent from another origin. Run alongside the Developer UI with: ```bash genkit start -- uv run server.py ``` -## Remote client +## Connect with `remote_agent` -```python -from genkit.agent import remote_agent +`remote_agent` returns an `AgentClient` with the same `chat()`, `load_chat()`, `get_snapshot()`, and `abort()` surface as an in-process `Agent`. `timeout` (default `60.0`) is the longest wait for the next bytes from the server, not a cap on the whole turn. -client = remote_agent( - url='http://127.0.0.1:8080/api/weatherAgent', - state_management='server', # 'client' if the server has no store -) -chat = client.chat() -res = await chat.send('Weather in Tokyo?') -print(res.text, res.snapshot_id) - -turn = chat.send_stream('And humidity?') -async for chunk in turn.stream: - if chunk.text: - print(chunk.text, end='', flush=True) -final = await turn.response - -resumed = await client.load_chat(snapshot_id=final.snapshot_id) -await resumed.send('What city was that?') +```python +from genkit.exp.agent import remote_agent + + +async def run_client() -> None: + client = remote_agent( + url='http://127.0.0.1:8080/api/weatherAgent', + state_management='server', # use 'client' when the server agent has no store + timeout=60.0, + ) + + # 1. One-shot turn + chat = client.chat() + res = await chat.send('Weather in Tokyo?') + print(res.text) + # => I don't have real-time weather data for Tokyo. Check a local forecast for live conditions. + + # 2. Streamed follow-up turn + turn = chat.send_stream('What about humidity?') + async for chunk in turn.stream: + if chunk.text: + print(chunk.text, end='', flush=True) + final = await turn.response + + # 3. Rehydrate from the server store onto a new chat handle + assert final.snapshot_id is not None + resumed = await client.load_chat(snapshot_id=final.snapshot_id) + answer = await resumed.send('Which city were we discussing?') + print(answer.text) + # => We were discussing Tokyo. ``` -Keep one chat for multi-turn. Match `state_management` to the server. Skip the -trailing slash on the URL. +Pass `state_schema=MyStateModel` to `remote_agent(...)` when you want `chat.state`, `res.state`, and `chunk.custom` validated into a Pydantic model on the client. Omit any trailing slash on `url`; the client derives `{url}/getSnapshot` and `{url}/abort` from it. A timeout surfaces as `GenkitError` `DEADLINE_EXCEEDED`, a refused or dropped connection as `UNAVAILABLE`. + +## Auth, `context_dependency`, and `PublicError` -## Auth +Resolve caller identity on the server with `context_dependency`, a FastAPI dependency applied to the turn, `getSnapshot`, and `abort` routes. Return a `dict`; other return values (such as Pydantic models) are dropped. Tools read it from `ctx.context` (or `Genkit.current_context()`), and `SessionStore` methods receive it as `context=`. -Resolve identity on the **server**. Mount the agent with a FastAPI dependency -that returns a plain dict. Tools read it with `Genkit.current_context()` for -the whole turn. +Errors surface at three boundaries: + +- **Inside `context_dependency`**: FastAPI resolves it before Genkit runs, so Genkit's error mapping does not apply. Raise `fastapi.HTTPException(status_code=401, ...)`; a `PublicError` raised here becomes FastAPI's plain `500 Internal Server Error`. +- **Inside Genkit, before or outside a turn**: on `getSnapshot` and `abort`, a `PublicError` from your `SessionStore` keeps its status, message, and details (`PERMISSION_DENIED` -> `403`). `AgentInitError` (an init that does not match the agent's state mode) is a `PublicError` and keeps its status too (`FAILED_PRECONDITION` -> `400`). Any other exception returns a redacted `500`. Full rules: [HTTP Error Redaction](fastapi.md#http-error-redaction--publicerror). +- **Inside a running turn**: tool and store exceptions do not become HTTP errors. The response is `200` with `finishReason: "failed"` and `error`; the client sees `res.finish_reason == AgentFinishReason.FAILED` and `res.error`. ```python from fastapi import FastAPI, Header, HTTPException -from genkit_fastapi import serve_agent +from genkit_fastapi.exp import serve_agent + +from genkit import PublicError, ToolContext +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore, SessionSnapshot, remote_agent +from genkit_google_genai import GoogleAI + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) + + +# 1. Resolve the caller from a header; the dict becomes the action context +async def resolve_user(x_api_key: str | None = Header(default=None)) -> dict[str, object]: + if not x_api_key: + raise HTTPException(status_code=401, detail='Missing X-Api-Key header.') + return {'tenant_id': 'acme', 'role': 'admin'} + + +# 2. Reject snapshot reads that carry no tenant +class TenantScopedStore(InMemorySessionStore): + async def get_snapshot( + self, + *, + snapshot_id: str | None = None, + session_id: str | None = None, + context: dict[str, object] | None = None, + ) -> SessionSnapshot | None: + if not (context or {}).get('tenant_id'): + raise PublicError('PERMISSION_DENIED', 'Tenant context required.') + return await super().get_snapshot(snapshot_id=snapshot_id, session_id=session_id, context=context) + -async def resolve_user(x_user: str | None = Header(default=None)) -> dict: - if not x_user: - raise HTTPException(status_code=401, detail='X-User header required') - return {'sub': x_user, 'role': 'engineer'} +# 3. Read the same context from a tool +@ai.tool(name='whoami', description='Inspect caller tenant context.') +async def whoami(ctx: ToolContext) -> dict[str, object]: + auth = ctx.context or {} + return {'tenant_id': auth.get('tenant_id'), 'role': auth.get('role')} + + +support_agent = ai.define_agent( + name='supportAgent', + system='Support assistant for the caller tenant.', + tools=[whoami], + store=TenantScopedStore(), +) app = FastAPI() app.include_router( - serve_agent(agent, context_dependency=resolve_user), + serve_agent(support_agent, context_dependency=resolve_user), prefix='/api', ) +# POST /api/supportAgent with no X-Api-Key +# => 401 {"detail":"Missing X-Api-Key header."} -@ai.tool() -async def who_am_i() -> str: - ctx = Genkit.current_context() or {} - return f"user={ctx.get('sub')} role={ctx.get('role')}" -``` - -The remote client only forwards credentials: - -```python -client = remote_agent( +# 4. The remote client sends static headers, or a sync/async callable that returns them +authed_client = remote_agent( url='http://127.0.0.1:8080/api/supportAgent', state_management='server', - headers={'X-User': 'alice'}, + headers={'X-Api-Key': 'sk-live-...'}, ) ``` -Return a `dict` from the dependency — Pydantic models are dropped. Custom -routes can pass context through `handle_genkit_request` ([FastAPI](fastapi.md)). -One-shot calls outside chat can use `ai.generate(..., context={...})`. +## Custom HTTP routes + +Use `handle_genkit_request` (`genkit_fastapi`) when a route needs path parameters or extra `Depends(...)` but should keep the Genkit JSON / SSE wire format. Use `to_sse` (`genkit.web`) to frame any stream handle (`chat.send_stream()`, `flow.stream()`) as `data: {"message": ...}` frames followed by `data: {"result": ...}`. Generic flow patterns: [Custom Route Escape Hatch](fastapi.md#3-custom-route-escape-hatch-handle_genkit_request-or-genkitweb). + +```python +from fastapi import FastAPI, Request, Response +from fastapi.responses import StreamingResponse +from genkit_fastapi import handle_genkit_request +from pydantic import BaseModel + +from genkit.web import to_sse + +custom_app = FastAPI() + + +class TurnRequest(BaseModel): + prompt: str + + +# Option A: full Genkit wire contract inside a custom route. +# Body: {"data": {"message": {"role": "user", "content": [{"text": "..."}]}}} +@custom_app.post('/tenants/{tenant_id}/chat') +async def tenant_agent_route(tenant_id: str, request: Request) -> Response: + # Check the caller may act for tenant_id before trusting the path. + return await handle_genkit_request( + request, + action=support_agent, + context={'tenant_id': tenant_id}, + ) + + +# Option B: stream one chat turn as SSE from a plain JSON body. +# agent.chat() takes no request context, so the store and tools see context=None. +@custom_app.post('/sessions/{session_id}/stream') +async def stream_session_turn(session_id: str, body: TurnRequest) -> StreamingResponse: + chat = agent.chat(session_id=session_id) + turn = chat.send_stream(body.prompt) + return StreamingResponse(to_sse(turn), media_type='text/event-stream') +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-human-in-the-loop.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-human-in-the-loop.md index e6a2d411..d2332c34 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-human-in-the-loop.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-human-in-the-loop.md @@ -1,43 +1,48 @@ -# Agent Human-in-the-Loop / Interrupts (Beta) +# Agent Human-in-the-Loop / Interrupts (Experimental) -> **Beta / preview API.** See [agents.md](agents.md). +> See [agents.md](agents.md) · [sessions](agents-sessions.md). -An **interrupt** pauses mid-turn so your code (or a human) can decide, then -**resume**. Works with or without a [store](agents-sessions.md). +An interrupt pauses an agent mid-turn so a human or policy check can inspect the pending tool call, then resume execution. Interrupts work both with a store (across separate HTTP requests) and without a store (in-process or round-tripped client state). -Flow: `send` → `finish_reason == INTERRUPTED` + `res.interrupts` → human input -→ `resume`. +Turn lifecycle: `chat.send(...)` -> `res.finish_reason == AgentFinishReason.INTERRUPTED` with `res.interrupts` -> human decision -> `chat.resume(...)` / `chat.resume_stream(...)`. -## ToolApproval +- [Gate tools with ToolApproval](#gate-tools-with-toolapproval): Lines 15-60 +- [Detect and resume (restart vs respond)](#detect-and-resume-restart-vs-respond): Lines 62-116 +- [Custom tool interrupts and Interrupt (BaseException)](#custom-tool-interrupts-and-interrupt-baseexception): Lines 118-156 -Tools in `allowed_tools` run freely; everything else pauses. Empty list ⇒ every -tool needs approval. When you also mount `Filesystem` / `Artifacts`, remember -to allow-list (or deliberately interrupt on) `write_file` / `edit_file` / -`write_artifact` — the coding-agent example often auto-approves **reads only**. +--- + +## Gate tools with `ToolApproval` + +`ToolApproval(allowed_tools=[...])` lets listed tools execute immediately and interrupts all other tool calls. An empty list requires approval for every tool. When pairing `ToolApproval` with `Filesystem` or `Artifacts`, list read-only tools (`list_files`, `read_file`, `read_artifact`) in `allowed_tools` if only writes (`write_file`, `edit_file`, `write_artifact`) should pause. ```python from uuid import uuid4 - -from pydantic import BaseModel, Field from genkit_google_genai import GoogleAI from genkit_middleware import Middleware, ToolApproval +from pydantic import BaseModel, Field -from genkit import Genkit, ToolRequestPart -from genkit.agent import AgentFinishReason, InMemorySessionStore - -ai = Genkit(plugins=[GoogleAI(), Middleware()]) -tool_approval = ToolApproval(allowed_tools=[]) +from genkit import Part +from genkit.exp import Genkit +from genkit.exp.agent import AgentFinishReason, InMemorySessionStore class TransferInput(BaseModel): amount: float to_account: str = Field(alias='toAccount') + model_config = {'populate_by_name': True} + class TransferOutput(BaseModel): success: bool transaction_id: str = Field(alias='transactionId') + model_config = {'populate_by_name': True} + + +ai = Genkit(plugins=[GoogleAI(), Middleware()]) + @ai.tool(name='transferMoney', description='Transfer money between accounts.') async def transfer_money(input: TransferInput) -> TransferOutput: @@ -46,64 +51,106 @@ async def transfer_money(input: TransferInput) -> TransferOutput: agent = ai.define_agent( name='bankingAgent', - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), system='Banking assistant. Call transferMoney when the user asks to transfer money.', tools=[transfer_money], - use=[tool_approval], + use=[ToolApproval(allowed_tools=[])], store=InMemorySessionStore(), ) ``` -## Detect and resume +## Detect and resume (`restart` vs `respond`) -```python -chat = agent.chat() +Each entry in `res.interrupts` is an `AgentInterrupt` exposing `.name`, `.ref`, and `.input` for rendering your approval UI, plus two methods that build `Part` objects for `chat.resume(...)`: -out1 = await chat.send('Transfer $500 to account 12345 for rent.') -# finish_reason == INTERRUPTED; transferMoney has NOT executed yet. +- **`intr.restart(resumed_metadata={'tool_approved': True})`**: re-runs the interrupted tool on the server (`ctx.is_resumed()` becomes `True`). `ToolApproval` checks `tool_approved` (or `toolApproved`) in `resumed_metadata`. Restart inputs must exactly match the original tool request in session history; a forged or modified restart returns a failed turn (`AgentFinishReason.FAILED`, `res.error.status == 'INVALID_ARGUMENT'`) instead of running the tool. +- **`intr.respond(output)`**: builds a tool response `Part` without executing the tool. Use this to deny an action (e.g. `intr.respond({'ok': False, 'error': 'declined by user'})`) or answer a prompt-style interrupt created with `ai.define_interrupt(...)`. -restart_parts: list[ToolRequestPart] = [ - intr.restart(resumed_metadata={'tool_approved': True}) for intr in out1.interrupts -] -out2 = await chat.resume(restart=restart_parts) -``` +A resumed turn can call more tools that also need approval, so loop until the turn stops interrupting: -Resume builders (return parts; you still call `chat.resume`): +```python +async def main() -> None: + chat = agent.chat() + + # 1. Turn pauses before transferMoney executes + res = await chat.send('Transfer $500 to account 12345 for rent.') + + # 2. Approve pending interrupts until the turn finishes + while res.finish_reason == AgentFinishReason.INTERRUPTED: + for intr in res.interrupts: + print(f'Approving {intr.name}: {intr.input}') + # => Approving transferMoney: {'amount': 500, 'toAccount': '12345'} + restart_parts: list[Part] = [ + intr.restart(resumed_metadata={'tool_approved': True}) + for intr in res.interrupts + ] + res = await chat.resume(restart=restart_parts) + + # 3. Read the final reply (check res.error if the turn failed) + print(res.finish_reason, res.text) + # => stop $500 has been successfully transferred to account 12345. + + +if __name__ == '__main__': + ai.run_main(main()) +``` -- `interrupt.restart(...)` — re-issue the tool request (ToolApproval after OK). - `resumed_metadata={'tool_approved': True}` (camelCase `toolApproved` also ok) -- `interrupt.respond(output)` — supply a tool response without running the tool - (use this to **deny**: e.g. `respond({'ok': False, 'error': 'denied by policy'})`) +Denials and streaming resumes use the same parts (`res` is an interrupted turn): ```python -# Approve (runs the tool): -await chat.resume( - restart=[intr.restart(resumed_metadata={'tool_approved': True}) for intr in out1.interrupts] -) - -# Deny (model sees the respond payload; tool does not execute): -await chat.resume( - respond=[intr.respond({'ok': False, 'error': 'export denied'}) for intr in out1.interrupts] +# Deny without executing the tool: +denied = await chat.resume( + respond=[intr.respond({'ok': False, 'error': 'transfer declined'}) for intr in res.interrupts], ) -await chat.resume( - respond=[a.respond({'approved': True})], - restart=[b.restart(resumed_metadata={'tool_approved': True})], +# Or approve and stream the resumed turn: +turn = chat.resume_stream( + restart=[intr.restart(resumed_metadata={'tool_approved': True}) for intr in res.interrupts], ) - -turn = chat.resume_stream(restart=restart_parts) async for chunk in turn.stream: if chunk.text: print(chunk.text, end='', flush=True) final = await turn.response ``` -Without a store: keep the same `chat`, or round-trip messages/state/artifacts -([client-managed state](agents.md#client-managed-state-no-store)). +Without a store, keep the same `chat` instance or round-trip `messages`, `state`, and `artifacts` ([without a store](agents.md#without-a-store)). + +## Custom tool interrupts and `Interrupt` (`BaseException`) + +For domain-specific thresholds (such as only pausing transfers above `$1,000`), raise `Interrupt` directly inside a tool or register a dedicated prompt tool with `ai.define_interrupt(...)`: + +- **`Interrupt` subclasses `BaseException`**, not `Exception` (like `asyncio.CancelledError`). A `try ... except Exception:` block inside a tool will **not** swallow an `Interrupt` and accidentally let execution proceed without approval. Only catch `Interrupt` explicitly if intended, and always re-raise it. +- Read `ctx.is_resumed()` and `ctx.resumed_metadata` on `ToolContext` when the caller resumes with `intr.restart(resumed_metadata=...)`. The keys are yours: the tool below expects `intr.restart(resumed_metadata={'approved': True})`. + +```python +from genkit import Interrupt, ToolContext + + +@ai.tool(name='wireTransfer', description='Send an external wire transfer.') +async def wire_transfer(input: TransferInput, ctx: ToolContext) -> dict[str, object]: + if input.amount > 1000 and not ctx.is_resumed(): + # BaseException: an `except Exception:` block can't swallow it + raise Interrupt({'reason': 'high_value_transfer', 'amount': input.amount}) -Build resume parts from `res.interrupts` only. Loop until finish reason is no -longer `INTERRUPTED`. Don't treat an interrupted turn as the final reply. + if ctx.is_resumed() and not (ctx.resumed_metadata or {}).get('approved'): + return {'ok': False, 'status': 'rejected'} -For approval UIs, each interrupt exposes `.name`, `.ref`, and `.input` (the -pending tool call). Use those to render the prompt; then `.restart(...)` or -`.respond(...)`. + return {'ok': True, 'amount': input.amount, 'to': input.to_account} +``` + +For pure question/confirmation tools that never run server logic themselves, use `ai.define_interrupt(...)` and resume with `intr.respond(...)`: + +```python +class ConfirmInput(BaseModel): + action: str + summary: str + + +ask_confirmation = ai.define_interrupt( + name='askConfirmation', + description='Ask the user to confirm before irreversible actions.', + input_schema=ConfirmInput, +) +# Add to the agent's tools=[...], then answer with: +# chat.resume(respond=[intr.respond({'confirmed': True}) for intr in res.interrupts]) +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-sessions.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-sessions.md index 16a24151..b4b7c103 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-sessions.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-sessions.md @@ -1,61 +1,78 @@ -# Sessions & Persistence (Beta) +# Sessions & Persistence (Experimental) > See [agents.md](agents.md). -With a store, the server owns history. Each turn writes an immutable snapshot — -the backbone for branching and [background work](agents-background.md). -Approvals work with or without a store ([HITL](agents-human-in-the-loop.md)). +With a `SessionStore`, the server owns conversation history. Every turn writes a `SessionSnapshot` keyed by `snapshot_id`, enabling resume across restarts, [branching](agents-branching.md), and [detached background turns](agents-background.md). [Interrupts](agents-human-in-the-loop.md) work with or without a store. ## Choose a store +Import local stores from `genkit.exp.agent` and the production Firestore store from `genkit_google_cloud.exp`: + ```python -from genkit.agent import InMemorySessionStore, FileSessionStore +from genkit.exp.agent import FileSessionStore, InMemorySessionStore +from genkit_google_cloud.exp import FirestoreSessionStore -mem_store = InMemorySessionStore() # gone on restart -file_store = FileSessionStore('./.snapshots') # one JSON file per snapshot +# 1. Local dev & tests (lost when the process exits) +mem_store = InMemorySessionStore() -# Optional: prune long chains; refuse ambiguous session_id after forks -pruning = FileSessionStore( +# 2. Local disk persistence (one <snapshot_id>.json file per turn) +file_store = FileSessionStore( './.snapshots', - max_persisted_chain_length=3, - reject_ambiguous_session=True, + max_persisted_chain_length=20, # optional: prune oldest ancestors on save + reject_ambiguous_session=True, # optional: raise on session_id lookup after a fork +) + +# 3. Production durable store (JSON Patch diffs + sharded checkpoints, ADC auth) +firestore_store = FirestoreSessionStore( + collection='genkit-sessions', + snapshot_path_prefix=lambda ctx: (ctx or {}).get('tenant_id', 'global'), ) ``` +`FirestoreSessionStore` stores each turn as a JSON Patch diff from its parent with periodic sharded checkpoints so large sessions stay below Firestore's 1 MiB document limit without secondary indexes. Pass `snapshot_path_prefix` to isolate multi-tenant paths from action `context`. + +## Persist and rehydrate a chat + ```python -from genkit import Genkit from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +from genkit.exp import Genkit +from genkit.exp.agent import FileSessionStore + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) agent = ai.define_agent( name='logbookAgent', - system='You are a personal logbook assistant.', - store=file_store, + system='You are a concise engineering logbook assistant.', + store=FileSessionStore('./.snapshots'), ) -chat = agent.chat() -res = await chat.send('Log this: I started studying Genkit today.') -print(res.snapshot_id) -resumed = await agent.load_chat(snapshot_id=res.snapshot_id) -await resumed.send('Add another note.') -``` +async def main() -> None: + # 1. Run turn 1; save res.snapshot_id in your app database + chat = agent.chat() + res = await chat.send('Log: migrated auth service to token exchange.') + checkpoint = res.snapshot_id + assert checkpoint is not None -## Typed state + # 2. Rehydrate full state (messages, state, artifacts) onto a fresh chat + resumed = await agent.load_chat(snapshot_id=checkpoint) + res2 = await resumed.send('Add note: latency p99 stayed at 14ms.') + print(res2.text) -Pass `state_schema`. Seed with `chat(state=...)` only when there is no store. -With a store, update fields from tools via `ai.current_session()` — -[state](agents-state.md). -## What a store unlocks +if __name__ == '__main__': + ai.run_main(main()) +``` -Without: multi-turn on one chat, interrupts. +## Resume and lookup methods -With: durable ids, `load_chat`, branching, detach / background tasks, server -abort. +- **`await agent.load_chat(snapshot_id=...)` / `load_chat(session_id=...)`**: reads the stored snapshot immediately and populates `chat.messages`, `chat.state`, and `chat.artifacts`. Pointing `load_chat` at a failed, aborted, or pending leaf loads that row as-is; calling `send()` on a non-completed snapshot returns `res.error` (`SNAPSHOT_NOT_RESUMABLE`). +- **`agent.chat(session_id=...)`**: attaches a lazy session handle without an upfront read. On the next `send()`, the server automatically walks back past any failed, aborted, or expired leaf to continue from the last completed turn (or starts a fresh session under `session_id` if no turn has completed yet). +- **`agent.chat(snapshot_id=...)`**: attaches a lazy resume handle to a specific completed snapshot. +- **`await agent.get_snapshot(snapshot_id=...)` / `get_snapshot(session_id=...)`**: inspects the raw `SessionSnapshot` without creating a chat. -## Many users, one agent +## Typed state and multi-user isolation -Each `agent.chat()` gets its own session on a shared store. Resume with the -right `session_id` or `snapshot_id` per user. +- **Typed state**: Pass `state_schema=MyModel` on the agent. On a store-backed agent, `agent.chat(state=...)` raises `AgentInitError` immediately; mutate state from tools with `current_session()` from `genkit.exp` (see [state](agents-state.md)). +- **Multi-user sessions**: One `Agent` instance safely serves many users on a shared store. Each `agent.chat()` mints a distinct `session_id`; store `session_id` or `snapshot_id` per user in your application database. diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-state.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-state.md index 04bdb00e..0f194a19 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-state.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents-state.md @@ -1,71 +1,82 @@ -# Agent State (Beta) +# Agent State (Experimental) > See [agents.md](agents.md) · [sessions](agents-sessions.md). -A session carries three layers: messages, custom state (`state_schema`), and -artifacts. With a schema, `chat.state` and streamed `chunk.custom` come back as -that model. +An agent session tracks three layers: `messages`, custom `state` (typed by `state_schema`), and `artifacts`. When `state_schema` (a Pydantic `BaseModel`) is set, `chat.state`, `res.state`, and streamed `chunk.custom` validate into instances of that model instead of bare dicts. -## Two modes +- [Two state-management modes](#two-state-management-modes): Lines 15-20 +- [Client-managed typed state](#client-managed-typed-state): Lines 22-64 +- [Mutating state from tools (current_session)](#mutating-state-from-tools-current_session): Lines 66-115 +- [Live state patches](#live-state-patches): Lines 117-131 +- [Egress redaction (state_transform and chunk_transform)](#egress-redaction-state_transform-and-chunk_transform): Lines 133-163 -**With a store** — Genkit owns history. Resume by `snapshot_id` or -`session_id`. You cannot seed with `chat(state=...)`. +--- -**Without a store** — your app owns history. Seed and round-trip yourself: +## Two state-management modes -```python -chat = agent.chat(state=Profile(name='Ada', tier='pro')) -await chat.send('Hello') -resumed = agent.chat( - messages=chat.messages, state=chat.state, artifacts=chat.artifacts -) -``` +- **Server-managed (`store=...`)**: Genkit owns persistence. Resume with `snapshot_id` or `session_id`. Passing `state=`, `messages=`, or `artifacts=` to `agent.chat(...)` on a store-backed agent raises `AgentInitError` (`FAILED_PRECONDITION`). +- **Client-managed (`store=None`)**: Your caller owns persistence. Seed `agent.chat(state=...)` and pass `messages`, `state`, and `artifacts` back on subsequent chats. -Custom state is for your product (routing, UI). It is not injected into the -model unless you put it in the system prompt or messages. - -`state_schema` alone does not fill `chat.state`. Something in the turn must call -`update_custom`. Prefer tools on a normal agent so you keep middleware. +Custom state holds application data (UI state, workflow counters, active ticket IDs). Genkit does not inject `state` into the model prompt unless your system prompt, messages, or tools expose it. Declaring `state_schema` alone leaves `chat.state` as `None` until client seeding or `session.update_custom(...)` writes a value. ## Client-managed typed state ```python -from pydantic import BaseModel - -from genkit import Genkit from genkit_google_genai import GoogleAI +from pydantic import BaseModel -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +from genkit.exp import Genkit -class Profile(BaseModel): +class DinerProfile(BaseModel): name: str - tier: str = 'free' + allergies: list[str] = [] + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) agent = ai.define_agent( - name='profileAgent', - system='Greet the user by name when you know it.', - state_schema=Profile, + name='conciergeAgent', + system='Restaurant concierge. Keep replies brief.', + state_schema=DinerProfile, ) -chat = agent.chat(state=Profile(name='Ada', tier='pro')) -await chat.send('Hello') -print(chat.state.name) + +async def main() -> None: + # 1. Seed typed state on a storeless agent + chat = agent.chat(state=DinerProfile(name='Ada', allergies=['peanuts'])) + await chat.send('Recommend an appetizer.') + assert chat.state is not None + print(chat.state.name, chat.state.allergies) + # => Ada ['peanuts'] + + # 2. Round-trip messages, state, and artifacts to continue later + resumed = agent.chat( + messages=chat.messages, + state=chat.state, + artifacts=chat.artifacts, + ) + await resumed.send('What about dessert?') + + +if __name__ == '__main__': + ai.run_main(main()) ``` -## Store + middleware +## Mutating state from tools (`current_session`) -Update custom state from tools with `ai.current_session()`. Mutators are -async. Coerce to your model if the callback receives a dict: +Inside an `@ai.tool()`, get the active session with `current_session()` from `genkit.exp` (`None` outside an agent turn) and update custom state there. + +`await sess.update_custom(fn)` is async; `fn` is sync. `fn` receives `None` on the first write, then whatever the session currently holds: the model instance you last returned, or a plain dict once state has round-tripped through JSON (a persistent store, an HTTP client). Normalize it before building the new state: ```python -from pydantic import BaseModel, Field +from typing import Any -from genkit import Genkit -from genkit.agent import InMemorySessionStore from genkit_google_genai import GoogleAI -from genkit_middleware import Middleware, ToolApproval +from pydantic import BaseModel, Field + +from genkit.exp import Genkit, current_session +from genkit.exp.agent import InMemorySessionStore class CaseState(BaseModel): @@ -74,19 +85,19 @@ class CaseState(BaseModel): class OpenCaseInput(BaseModel): - case_id: str = Field(description='Support case id') + case_id: str = Field(description='Support case identifier') -ai = Genkit(plugins=[GoogleAI(), Middleware()], model='googleai/gemini-flash-latest') +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) -@ai.tool(name='openCase', description='Open or update the support case id.') -async def open_case(input: OpenCaseInput) -> dict: - sess = ai.current_session() +@ai.tool(name='openCase', description='Open or switch the active support case.') +async def open_case(input: OpenCaseInput) -> dict[str, Any]: + sess = current_session() if sess is None: - return {'ok': False, 'error': 'no session'} + return {'ok': False, 'error': 'no active agent session'} - async def mutate(c: object) -> CaseState: + def mutate(c: CaseState | dict[str, Any] | None) -> CaseState: base = c if isinstance(c, CaseState) else CaseState.model_validate(c or {}) return CaseState(case_id=input.case_id, status=base.status) @@ -96,40 +107,57 @@ async def open_case(input: OpenCaseInput) -> dict: agent = ai.define_agent( name='supportOps', - system='Support ops. Call openCase when asked. Be brief.', + system='Support ops assistant. Call openCase when given a case ID.', tools=[open_case], state_schema=CaseState, - use=[ToolApproval(allowed_tools=['openCase'])], store=InMemorySessionStore(), ) ``` -## Live patches +## Live state patches -In a [custom agent](agents-custom.md), `update_custom` streams to -`chunk.custom`: +Every `await sess.update_custom(...)` call (from a tool or a [custom agent](agents-custom.md)) emits an RFC 6902 JSON Patch chunk (`chunk.raw.custom_patch`). `AgentTurn.stream` applies the patch and puts the resolved state on `chunk.custom`, typed by `state_schema`. With the `supportOps` agent above: ```python -async def bump(c): - return {'turns': (c or {}).get('turns', 0) + 1} - -await sess.update_custom(bump) +chat = agent.chat() +turn = chat.send_stream('Open case SUP-402.') +async for chunk in turn.stream: + if chunk.custom is not None: + print(chunk.custom.case_id, chunk.custom.status) + # => SUP-402 open +res = await turn.response +print(res.state) +# => case_id='SUP-402' status='open' ``` -## Redacting on the way out +## Egress redaction (`state_transform` and `chunk_transform`) + +`state_transform` and `chunk_transform` filter what leaves the server without altering persisted snapshots: -`state_transform` and `chunk_transform` shape what clients see. They do not -change what the store keeps. Return a full `SessionState` from -`state_transform`. Returning `None` from `chunk_transform` drops that chunk. +- **`state_transform(state: SessionState) -> SessionState`**: shapes snapshot reads, client-managed `AgentOutput.state`, and streamed custom-patch baselines. Must return a `SessionState` (use `state.model_copy(update=...)` so `session_id` is preserved). +- **`chunk_transform(chunk: AgentStreamChunk) -> AgentStreamChunk | None`**: reshapes or drops individual stream chunks; return `None` to suppress a chunk. ```python -from genkit.agent import SessionState +from genkit.exp.agent import AgentStreamChunk, InMemorySessionStore, SessionState -def redact(state: SessionState) -> SessionState: + +def redact_state(state: SessionState) -> SessionState: custom = dict(state.custom or {}) - if 'api_key' in custom: - custom['api_key'] = 'REDACTED' - return SessionState(messages=state.messages, custom=custom, artifacts=state.artifacts) + custom.pop('internal_token', None) + return state.model_copy(update={'custom': custom}) + + +def drop_debug_artifacts(chunk: AgentStreamChunk) -> AgentStreamChunk | None: + if chunk.artifact and chunk.artifact.name == 'debug.log': + return None + return chunk -agent = ai.define_agent(..., state_schema=SecretState, state_transform=redact, store=...) + +guarded_agent = ai.define_agent( + name='guardedAgent', + system='Concise support agent.', + state_transform=redact_state, + chunk_transform=drop_debug_artifacts, + store=InMemorySessionStore(), +) ``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents.md index fd114e91..844a468b 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/agents.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/agents.md @@ -1,26 +1,31 @@ -# Agents (Beta) +# Agents (Experimental) -> Preview API under `genkit.agent`. +> Experimental APIs live under `.exp` modules (`genkit.exp`, `genkit.exp.agent`, `genkit_fastapi.exp`, `genkit_middleware.exp`, `genkit_google_cloud.exp`) and can change between minor releases. -Turn a model + tools into a durable conversation — history, typed state, -approvals, branches, and background work. +Turn a model and tools into a stateful multi-turn conversation with snapshots, typed custom state, human approvals, branching, and detached background runs. -Deeper topics: [sessions](agents-sessions.md) · -[HITL](agents-human-in-the-loop.md) · [branching](agents-branching.md) · -[background](agents-background.md) · [state](agents-state.md) · -[artifacts](agents-artifacts.md) · [custom](agents-custom.md) · -[HTTP](agents-http.md) +Deeper topics: [sessions](agents-sessions.md) · [HITL](agents-human-in-the-loop.md) · [branching](agents-branching.md) · [background](agents-background.md) · [state](agents-state.md) · [artifacts](agents-artifacts.md) · [custom](agents-custom.md) · [HTTP](agents-http.md) + +- [Define an agent](#define-an-agent): Lines 19-69 +- [Tools, progress chunks, and multipart responses](#tools-progress-chunks-and-multipart-responses): Lines 71-99 +- [Middleware](#middleware): Lines 101-136 +- [Chat and streaming](#chat-and-streaming): Lines 138-169 +- [Errors and cancellation](#errors-and-cancellation): Lines 171-194 +- [Without a store](#without-a-store): Lines 196-216 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 218-234 + +--- ## Define an agent +Import `Genkit` from `genkit.exp` to opt into `define_agent`, `define_prompt_agent`, `define_custom_agent`, and `ai.agent(name)`. Stable `from genkit import Genkit` does not expose agent methods. + ```python +from genkit_google_genai import GoogleAI from pydantic import BaseModel -from genkit import Genkit -from genkit_google_genai import GoogleAI -from genkit.agent import InMemorySessionStore - -ai = Genkit(plugins=[GoogleAI()]) +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore class WeatherInput(BaseModel): @@ -32,139 +37,198 @@ class WeatherOutput(BaseModel): temperature: str +# 1. Initialize experimental Genkit instance +ai = Genkit(plugins=[GoogleAI()]) + + +# 2. Define a typed tool @ai.tool(name='getWeather', description='Get weather for a city.') async def get_weather(input: WeatherInput) -> WeatherOutput: - return WeatherOutput(weather='Sunny', temperature='21°C') + return WeatherOutput(weather=f'Sunny in {input.location}', temperature='21°C') +# 3. Register a store-backed agent agent = ai.define_agent( name='weatherAgent', - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), system='Weather assistant. Use getWeather for weather questions.', tools=[get_weather], - store=InMemorySessionStore(), # omit to manage history yourself + store=InMemorySessionStore(), # omit store= for client-managed history ) ``` -Options worth knowing: `use` (middleware), `state_schema`, `max_turns` (tool -loop depth per user message), transforms. Dotprompt agents: -`ai.define_prompt_agent`. Full control: [`define_custom_agent`](agents-custom.md). +Key `define_agent` keyword arguments: + +- **`model`**: `GoogleAI.gemini('gemini-flash-latest')` (or a `'plugin/model'` string) +- **`config`**: typed config such as `GeminiConfig(thinking_level='HIGH')` or `AnthropicConfig(thinking='adaptive', effort='high')` +- **`use`**: middleware instances (`Retry`, `Fallback`, `ToolApproval`, `Filesystem`, `Skills`, `Artifacts`, `Surfaces`) +- **`state_schema`**: Pydantic `BaseModel` typing `chat.state`, `res.state`, and `chunk.custom` ([state](agents-state.md)) +- **`max_turns`**: tool-loop ceiling per user message (default 50) +- **`state_transform` / `chunk_transform`**: egress hooks that redact client views without altering stored state -Tool parameters should be a small Pydantic model — even for one field. Empty -inputs need an empty subclass, not bare `BaseModel`. +For `.prompt` files or `ai.define_prompt(...)`, wire the registered prompt name directly with `ai.define_prompt_agent(name='greeterPrompt', store=...)`. For custom turn loops, use [`ai.define_custom_agent(...)`](agents-custom.md). + +## Tools, progress chunks, and multipart responses + +A tool takes at most one typed input parameter (a Pydantic `BaseModel` for structured arguments or a scalar like `str`, or no input parameter) plus an optional `ToolContext[ChunkT]` parameter. + +```python +import asyncio +from pydantic import BaseModel + +from genkit import MultipartToolResponse, Part, ToolContext + + +class PrepProgress(BaseModel): + stage: str + percent: int + + +@ai.tool(name='prepareOrder', description='Prepare a kitchen order and stream stage progress.') +async def prepare_order(dish: str, ctx: ToolContext[PrepProgress]) -> MultipartToolResponse[dict[str, str]]: + # 1. Stream partial tool progress (arrives on chunk.tool_responses with part.is_partial == True) + ctx.send_chunk(PrepProgress(stage='searing', percent=40)) + await asyncio.sleep(0.2) + ctx.send_chunk(PrepProgress(stage='plating', percent=90)) + + # 2. Return structured output alongside extra content parts (e.g. media or text) + return MultipartToolResponse( + output={'status': 'ready', 'dish': dish}, + content=[Part.from_text(f'Ticket closed for {dish}.')], + ) +``` ## Middleware -Register `Middleware()` once, then pass instances in `use=[...]`. +Import standard middleware from `genkit_middleware`, session-bound `Artifacts` from `genkit_middleware.exp`, and generative UI `Surfaces` from `genkit_a2ui`. Register `Middleware()` in `plugins=[...]` when you want standard middleware listed in the Dev UI or referenced by `.prompt` files; passing instances in `use=[...]` works either way. -- **Filesystem** — `list_files` / `read_file`. Writes stay off until - `allow_write_access=True`. Paths are relative to `root_dir`. -- **ToolApproval** — tools in `allowed_tools` run; everything else pauses for - a human. List write/artifact tools if they should run freely. -- **Skills** — load `skills/<name>/SKILL.md` and expose `use_skill`. -- **Retry** — retries flaky model HTTP calls. Tool failures are different: - return a soft-error payload the model can read. +- **`Filesystem`** (`genkit_middleware`): `list_files` and `read_file` scoped to `root_dir`; pass `allow_write_access=True` to add `write_file` and `edit_file`. +- **`ToolApproval`** (`genkit_middleware`): tools in `allowed_tools` run automatically; all other tools pause the turn with an interrupt ([HITL](agents-human-in-the-loop.md)). It gates every tool call, including the ones `Filesystem` and `Artifacts` add. +- **`Skills`** (`genkit_middleware`): injects `SKILL.md` descriptions and exposes `use_skill`. +- **`Retry` / `Fallback`** (`genkit_middleware`): `Retry` retries transient failures (`UNAVAILABLE`, `DEADLINE_EXCEEDED`, `RESOURCE_EXHAUSTED`, `ABORTED`, `INTERNAL` by default); `Fallback` switches to alternate models. +- **`Artifacts`** (`genkit_middleware.exp`): adds `read_artifact` and `write_artifact` tools plus an active artifact listing in the system prompt ([artifacts](agents-artifacts.md)). +- **`Surfaces`** (`genkit_a2ui`): A2UI surface instructions, validation, and part parsing. ```python -from genkit_middleware import Middleware, ToolApproval, Filesystem, Skills, Retry -from genkit.agent import FileSessionStore +from genkit_google_genai import GoogleAI +from genkit_middleware import Filesystem, Middleware, Retry, Skills, ToolApproval +from genkit_middleware.exp import Artifacts + +from genkit.exp import Genkit +from genkit.exp.agent import FileSessionStore ai = Genkit(plugins=[GoogleAI(), Middleware()]) coding_agent = ai.define_agent( name='codingAgent', - model='googleai/gemini-flash-latest', - system='Coding assistant. Use list_files/read_file/write_file/edit_file.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Coding assistant. Explore with list_files/read_file before editing.', use=[ - ToolApproval(allowed_tools=['list_files', 'read_file', 'use_skill']), + ToolApproval(allowed_tools=['list_files', 'read_file', 'read_artifact', 'use_skill']), Filesystem(root_dir='./workspace', allow_write_access=True), + Artifacts(), Skills(skill_paths=['./skills']), - Retry(), + Retry(max_retries=3), ], store=FileSessionStore('./.snapshots'), max_turns=30, ) ``` -For typed session fields with middleware, update state from tools via -`ai.current_session()` — [state](agents-state.md). Declaring `state_schema` -alone does not fill `chat.state`. +## Chat and streaming -## Chat +Run one turn at a time per `AgentChat`. Use `ai.run_main(main())` for standalone scripts. ```python -chat = agent.chat() +async def main() -> None: + chat = agent.chat() -res = await chat.send('Weather in Tokyo?') -print(res.text, res.snapshot_id) + # 1. One-shot turn + res = await chat.send('Weather in Tokyo?') + print(res.text, res.snapshot_id) + # => It is 21°C and sunny in Tokyo. 8f3c... -turn = chat.send_stream('What about Paris?') -async for chunk in turn.stream: - if chunk.text: - print(chunk.accumulated_text, end='\r', flush=True) -final = await turn.response -``` - -Send returns a response. Stream returns a turn with `.stream` and `.response`. -After an interrupt, use `resume` / `resume_stream`. + # 2. Streamed turn (.stream yields AgentChunk, .response resolves to AgentResponse) + turn = chat.send_stream('What about Paris?') + async for chunk in turn.stream: + if chunk.text: + print(chunk.accumulated_text, end='\r', flush=True) + final = await turn.response -You can await the response without reading every chunk. One send at a time per -`chat`. + # 3. Rehydrate history from the store on a new chat instance + resumed = await agent.load_chat(snapshot_id=final.snapshot_id) + followup = await resumed.send('Which cities did we check?') + print(followup.text) -To reopen a stored conversation, use `load_chat` (not `chat(snapshot_id=...)`, -which only attaches a resume handle): -```python -resumed = await agent.load_chat(snapshot_id=res.snapshot_id) -await resumed.send('What city did I ask about?') +if __name__ == '__main__': + ai.run_main(main()) ``` -## Verify an agent from the CLI (`flow:run`) +- **`await agent.load_chat(snapshot_id=...)`**: fetches the stored snapshot immediately so `chat.messages`, `chat.state`, and `chat.artifacts` are populated locally. +- **`agent.chat(session_id=...)`**: attaches a lazy resume handle that continues from the session's last completed turn, skipping any failed or aborted leaf. -`genkit flow:run` only runs **flows**, not agents, so you can't `flow:run` an -agent directly. To exercise an agent from the CLI (e.g. a quick, self-terminating -check), wrap one turn in a throwaway flow and run that: +## Errors and cancellation -```python -@ai.flow() -async def try_weather_agent(message: str) -> str: - return (await agent.chat().send(message)).text +Genkit separates turn failures from call failures: -# genkit flow:run try_weather_agent '"Weather in Tokyo?"' -- uv run src/main.py +- **Turn failure (`res.error`)**: When an agent turn starts and fails at runtime, `chat.send(...)` does **not** raise. It returns `AgentResponse` with `res.finish_reason == AgentFinishReason.FAILED`, `res.text == ''`, and `res.error` (`Failure` with `status`, `message`, `details`, and `reason: ErrorReason`). The chat drops the unanswered user prompt from `chat.messages`, and the next `send()` continues from the last completed snapshot. +- **No turn result (`GenkitError`)**: When no turn runs at all (dropped HTTP connection, or `AgentInitError` from passing `snapshot_id` to a storeless agent), `chat.send(...)` raises `GenkitError` and restores local chat state. Mismatched init arguments (a `state` blob to a store-backed agent, or `messages`/`state` mixed with a resume id) raise `AgentInitError` from `agent.chat(...)` itself. +- **Cancellation**: native `asyncio.CancelledError`; put tool cleanup in `try` / `finally`. `await turn.abort()` detaches the client stream only. To cancel a running turn server-side, start it with `chat.detach(...)` and call `await task.abort()`; `chat.abort()` during a foreground `send()` does not stop that turn ([background](agents-background.md#client-detach-vs-server-cancellation)). + +```python +from genkit import ErrorReason, GenkitError +from genkit.exp.agent import AgentFinishReason + +try: + res = await chat.send('Run portfolio rebalance.') +except GenkitError as err: + # No turn ran (transport drop or invalid init) + print(err.status, err.message) +else: + if res.error: + # Turn ran and failed; res.finish_reason == AgentFinishReason.FAILED + assert res.finish_reason == AgentFinishReason.FAILED + if res.error.reason == ErrorReason.INVALID_OUTPUT: + print('Truncated structured output:', res.error.message) ``` ## Without a store -Skip `store` when your app owns history. Ids stay `None` — pass messages, -state, and artifacts into the next `chat(...)`: +Omit `store=` when the client owns conversation state. `snapshot_id` stays `None`; round-trip `messages`, `state`, and `artifacts` into the next `agent.chat(...)` call: ```python -agent = ai.define_agent( +stateless = ai.define_agent( name='echoNoStore', - model='googleai/gemini-flash-latest', - system='Echo assistant. Answer briefly and remember context.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Answer briefly and remember context.', ) -chat = agent.chat() -await chat.send('My name is Ada. Remember it.') +chat = stateless.chat() +await chat.send('My name is Ada.') -resumed = agent.chat( - messages=chat.messages, state=chat.state, artifacts=chat.artifacts +resumed = stateless.chat( + messages=chat.messages, + state=chat.state, + artifacts=chat.artifacts, ) await resumed.send('What is my name? One word.') ``` -Add a [store](agents-sessions.md) when the server should own history, or when -you need branching and detach. [Interrupts](agents-human-in-the-loop.md) work -either way. +## Verify an agent from the CLI (`flow:run`) + +`genkit flow:run` invokes flows, not agents directly. Wrap a turn in a small flow and propagate `res.error` so failures surface cleanly: -## Auth +```python +from genkit import GenkitError -Put identity on the server with `serve_agent` and a FastAPI dependency. The -remote client only forwards credentials. See [HTTP](agents-http.md). -## Prompt agents +@ai.flow() +async def try_weather_agent(message: str) -> str: + res = await agent.chat().send(message) + if res.error: + raise GenkitError(status=res.error.status, message=res.error.message, details=res.error.details) + return res.text -`define_prompt_agent(name=...)` uses the same name for the agent and the -`.prompt` file stem. Keep preamble inputs stable for the chat — put dynamic -fields in user messages or tools. +# genkit flow:run try_weather_agent '"Weather in Tokyo?"' -- uv run src/main.py +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/common-errors.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/common-errors.md index fba68af8..f5715cdd 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/common-errors.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/common-errors.md @@ -1,133 +1,323 @@ -# Common Errors - -Quick fixes. Longer recipes live in the linked guides. +# Common Errors — Genkit Python 0.13 + +Find the symptom, apply the fix. Every message below comes from a real 0.13 run. Upgrading a 0.12 codebase? Read [Migrating from 0.12 to 0.13](migrating-from-0.12.md). + +- [ai.generate Returns Empty Text and Does Not Raise](#aigenerate-returns-empty-text-and-does-not-raise): Lines 23-48 +- [Thinking level MINIMAL is not supported for this model](#thinking-level-minimal-is-not-supported-for-this-model): Lines 52-70 +- [ImportError on ActionRunContext, ToolRunContext, MediaPart, TextPart, or genkit.agent](#importerror-on-actionruncontext-toolruncontext-mediapart-textpart-or-genkitagent): Lines 74-106 +- [GenkitError INVALID_ARGUMENT on Config Keys](#genkiterror-invalid_argument-on-config-keys): Lines 110-134 +- [GenkitError INVALID_ARGUMENT: API Key in config](#genkiterror-invalid_argument-api-key-in-config): Lines 138-150 +- [GenkitError NOT_FOUND: Unprefixed Model String](#genkiterror-not_found-unprefixed-model-string): Lines 154-162 +- [Vertex AI Fails on gemini-flash-latest](#vertex-ai-fails-on-gemini-flash-latest): Lines 166-174 +- [TypeError When Defining or Calling Flows and Tools](#typeerror-when-defining-or-calling-flows-and-tools): Lines 178-206 +- [Served Flow Returns 404, 500, or the Wrong Auth Status](#served-flow-returns-404-500-or-the-wrong-auth-status): Lines 210-233 +- [response.output Is None on Structured Output](#responseoutput-is-none-on-structured-output): Lines 237-250 +- [response.json Is Not the Model Output](#responsejson-is-not-the-model-output): Lines 254-262 +- [{{variable}} Sent Literally](#variable-sent-literally): Lines 266-274 +- [Streaming TypeError or a Flow That Keeps Running After break](#streaming-typeerror-or-a-flow-that-keeps-running-after-break): Lines 278-297 +- [Cloud Telemetry Enabled but Spans Do Not Reach Cloud Trace](#cloud-telemetry-enabled-but-spans-do-not-reach-cloud-trace): Lines 301-309 +- [Interrupts and Agent Session Pitfalls](#interrupts-and-agent-session-pitfalls): Lines 313-323 --- -## Wrong Google AI import +## `ai.generate` Returns Empty Text and Does Not Raise + +`ai.generate` boxes model-side failures into the response: provider rejections, `UNAVAILABLE`, rate limits, and model IDs the provider does not serve. You get `response.text == ''`, `response.finish_reason == 'failed'`, and a `Failure` on `response.error`. Code that only reads `response.text` silently continues with an empty string. ```python +from genkit import Genkit from genkit_google_genai import GoogleAI -``` -```bash -uv add genkit genkit-google-genai +ai = Genkit(plugins=[GoogleAI()]) + +# 1. Prefixed but unserved model ID: no exception +res = await ai.generate(model='googleai/gemini-9-ultra', prompt='Suggest a dish.') +print(repr(res.text), res.finish_reason) +# => '' FinishReason.FAILED + +# 2. Check response.error after every call +if res.error: + print(res.error.status, res.error.message) + # => NOT_FOUND models/gemini-9-ultra is not found for API version v1beta, or is not supported for generateContent. ... ``` +Problems caught before the request is sent raise instead: + +- **`ValueError`** from `GoogleAI()` when `GEMINI_API_KEY` is unset (`[Genkit] GEMINI_API_KEY environment variable not set.`). +- **`GenkitError` `UNAUTHENTICATED`** on the first `ai.generate` when the key is invalid (`API key not valid. Please pass a valid API key.`). +- **`GenkitError`** for an unprefixed model string, unknown config keys, and `api_key` in config (sections below). + --- -## Streaming TypeError +## `Thinking level MINIMAL is not supported for this model` -Do not await the stream handle — await the response: +`gemini-flash-latest` rejects `thinking_level='MINIMAL'`. The call returns a failed response (see above), not an exception: ```python -turn = chat.send_stream('hi') -async for chunk in turn.stream: ... -res = await turn.response +from genkit_google_genai import GeminiConfig + +res = await ai.generate(prompt='Name one dessert.', config=GeminiConfig(thinking_level='MINIMAL')) +print(res.error.status, res.error.message) +# => INVALID_ARGUMENT Thinking level MINIMAL is not supported for this model. Please retry with other thinking level. +``` -sr = ai.generate_stream(prompt='...') -async for chunk in sr.stream: ... -final = await sr.response +Use `'LOW'`, `'MEDIUM'`, or `'HIGH'`: + +```python +res = await ai.generate(prompt='Name one dessert.', config=GeminiConfig(thinking_level='LOW')) +print(res.text) +# => Tiramisu. ``` --- -## Tool schema must be an object +## `ImportError` on `ActionRunContext`, `ToolRunContext`, `MediaPart`, `TextPart`, or `genkit.agent` -Wrap tool params in a Pydantic model. Bare `str` / `float` arguments fail on -Gemini. See [agents.md](agents.md). +```text +ImportError: cannot import name 'ActionRunContext' from 'genkit' +ImportError: cannot import name 'MediaPart' from 'genkit' +ModuleNotFoundError: No module named 'genkit.agent' +AttributeError: 'Genkit' object has no attribute 'define_agent' +ValidationError: 1 validation error for Part + Value error, Part(root=...) is gone; use Part.from_text or Part(text=...) +``` -```python -class WeatherInput(BaseModel): - city: str +Each symbol has one public home: -@ai.tool() -async def get_weather(input: WeatherInput) -> str: ... +```python +# Contexts +from genkit import FlowContext, ToolContext +from genkit.model import ModelContext + +# Parts +from genkit import Media, Message, Part, Role +text_part = Part.from_text('Hello') +img_part = Part.from_media('https://example.com/photo.jpg', content_type='image/jpeg') + +# Experimental agents and integrations (define_agent exists only on genkit.exp.Genkit) +from genkit.exp import Genkit, current_session +from genkit.exp.agent import Agent, InMemorySessionStore, Session +from genkit_fastapi.exp import serve_agent +from genkit_middleware.exp import Artifacts ``` +Two different `Middleware` imports: +- `from genkit_middleware import Middleware, Retry, ToolApproval` imports the **`Middleware()` plugin** and built-in middleware. +- `from genkit.middleware import Middleware, MiddlewareContext` imports the **base class** for custom middleware. + --- -## `define_tool` missing +## `GenkitError` `INVALID_ARGUMENT` on Config Keys -Use `@ai.tool()`. +Genkit validates config before calling the model. Typos, nested dicts, and fields that moved off config all raise: ---- +```text +INVALID_ARGUMENT: googleai/gemini-flash-latest: unknown config key 'temprature'; put provider-only settings in config['extra'] +INVALID_ARGUMENT: thinking_config was flattened; use include_thoughts / thinking_budget / thinking_level +INVALID_ARGUMENT: googleai/gemini-flash-latest: unknown config key 'tool_choice'; put provider-only settings in config['extra'] +``` -## Model id without a prefix +Use flat typed config, put provider-native keys in `extra`, and pass `tool_choice` on the call: -Use `googleai/gemini-flash-latest`, not bare `gemini-flash-latest`. +```python +from genkit_google_genai import GeminiConfig + +cfg = GeminiConfig( + temperature=0.2, + thinking_level='HIGH', + include_thoughts=True, + extra={'seed': 7}, +) +res = await ai.generate(prompt='Pick a wine for salmon.', config=cfg, tool_choice='required', tools=[...]) +``` + +`GeminiConfig.extra` is wire-shaped and deep-merged into the request body, so nested provider fields go under their wire key (for example `extra={'generationConfig': {...}}`). --- -## Missing `.json` / `.message` on the response +## `GenkitError` `INVALID_ARGUMENT`: API Key in `config` -Plain text → `response.text`. Structured output → `response.output`. +```text +INVALID_ARGUMENT: API key belongs in context.secrets, not config. Pass the key as context={'secrets': {'api_key': ...}}. +``` + +Set the key on the plugin, or pass a per-request key in `context`: + +```python +# Whole app: GoogleAI(api_key=...), OpenAI(api_key=...) +# Per request: +await ai.generate(prompt='Hello', context={'secrets': {'api_key': user_key}}) +``` --- -## Event loop issues under `genkit start` +## `GenkitError` `NOT_FOUND`: Unprefixed Model String + +```text +NOT_FOUND: Failed to resolve model 'gemini-flash-latest'. Ensure the model name includes the plugin namespace (e.g., 'plugin/model'). +``` + +Plain strings need the `<plugin>/` prefix (`'googleai/gemini-flash-latest'`, `'openai/gpt-4o'`). Model ref helpers (`GoogleAI.gemini('gemini-flash-latest')`, `Anthropic.claude('claude-sonnet-4-6')`, `OpenAI.gpt('gpt-5.2')`) add it for you. -Use `ai.run_main(main())` for long-lived apps. See [fastapi.md](fastapi.md) -for serving under Dev UI. +A prefixed ID the provider does not serve resolves and fails at call time on `response.error` instead (see the first section). --- -## No `snapshot_id` after a turn +## Vertex AI Fails on `gemini-flash-latest` -Attach a `store`. Without one, pass messages / state / artifacts yourself — -[agents.md](agents.md). +Vertex AI does not serve the Gemini API's rotating `-latest` aliases (`gemini-flash-latest`, `gemini-flash-lite-latest`, `gemini-pro-latest`). The Vertex plugin's known-model list omits them. Use a versioned ID: + +```python +from genkit_google_genai import VertexAI + +ai = Genkit(plugins=[VertexAI(location='us-central1')], model=VertexAI.gemini('gemini-3.5-flash')) +``` --- -## Cannot send state to a server-managed agent +## `TypeError` When Defining or Calling Flows and Tools -Store-backed chats resume by id. Update fields inside the turn — -[agents-state.md](agents-state.md). +```text +TypeError: Tool function must be async. Got sync function: lookup_reservation +TypeError: Flow must be async: sync_menu +TypeError: tool 'check_allergens' takes one input, but 'allergen' is a second parameter. Put the fields on one input model, or annotate 'allergen' as ToolContext. +TypeError: Genkit() takes no positional arguments, got 1. Pass keyword arguments instead, e.g. Genkit(plugins=[...], model="..."). +TypeError: Action.run() takes from 1 to 2 positional arguments but 3 were given +``` + +- `@ai.tool()` and `@ai.flow()` functions must be `async def`. +- Each takes at most one input parameter. A context parameter must be annotated (`ctx: FlowContext` or `ctx: ToolContext`); an unannotated `ctx` counts as a second input. Wrap multi-field input in one Pydantic `BaseModel`. +- Constructors (`Genkit`, plugins, `GenkitError`) and every option after the input on `.run(...)` are keyword-only: + +```python +from pydantic import BaseModel +from genkit import ToolContext + +class AllergenCheck(BaseModel): + dish: str + allergen: str + +@ai.tool() +async def check_allergens(check: AllergenCheck, ctx: ToolContext) -> bool: + """Return True if the dish contains the allergen.""" + return check.allergen in MENU[check.dish].allergens + +res = await plan_menu.run(payload, on_chunk=handle_chunk, context={'uid': 'u1'}) +``` --- -## Ambiguous session after a fork +## Served Flow Returns 404, 500, or the Wrong Auth Status -Resume with a leaf `snapshot_id`, not `session_id` — -[agents-branching.md](agents-branching.md). +Over HTTP (`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, `handle_genkit_request`): + +- **`404 {"detail":"Not Found"}`**: the route is the Python function name. `menu_suggestion_flow` mounts at `POST /menu_suggestion_flow`, not `/menuSuggestionFlow`. Override with `serve_flow(flow, base_path=...)`. +- **`500 {"message":"Internal Error","status":"INTERNAL"}`**: the flow raised `GenkitError` or any non-`PublicError` exception. Genkit logs the traceback server-side and redacts the body. Raise `PublicError` for errors the client should see. +- **`400 ... "status":"INVALID_ARGUMENT"`**: the request `data` failed input validation. +- **`500 Internal Server Error` (plain text) from a `context_dependency`**: `serve_flow(..., context_dependency=...)` runs a FastAPI dependency before Genkit's handler, so a `PublicError` raised there is not mapped. Raise `fastapi.HTTPException(status_code=401)` in dependencies; `PublicError('UNAUTHENTICATED', ...)` maps to 401 only inside a `genkit_fastapi_handler` `context_provider` or the flow itself. + +```python +from genkit import PublicError + +@ai.flow() +async def get_invoice(invoice_id: str) -> dict: + inv = await find_invoice(invoice_id) + if inv is None: + raise PublicError('NOT_FOUND', f'Invoice {invoice_id} not found') + return inv + +# POST /get_invoice {"data": "INV-9"} +# => 404 {"message":"Invoice INV-9 not found","status":"NOT_FOUND"} +``` + +Full status mapping: [FastAPI](fastapi.md#http-error-redaction--publicerror). --- -## Approval resume does nothing +## `response.output` Is `None` on Structured Output -Pass approval metadata on the restart part: +When a JSON response is truncated (often by `max_output_tokens`) or does not parse against the schema, `response.output` is `None` and `response.error.reason` is `ErrorReason.INVALID_OUTPUT`: ```python -await chat.resume( - restart=[ - i.restart(resumed_metadata={'tool_approved': True}) - for i in out.interrupts - ] -) +from genkit import ErrorReason + +res = await ai.generate(prompt='Summarize this invoice.', output_schema=InvoiceSummary) +if res.output is None and res.error and res.error.reason == ErrorReason.INVALID_OUTPUT: + print(res.error.message) + # => Model output was not valid JSON for the requested schema: {"vendor": "Acme", "total": 12 ``` -Build from `res.interrupts`. Full flow: -[agents-human-in-the-loop.md](agents-human-in-the-loop.md). +Raise `max_output_tokens` or simplify the schema, then retry. --- -## Abort seems to do nothing +## `response.json` Is Not the Model Output + +`response.json` is Pydantic's deprecated `BaseModel.json` method. Calling it serializes the whole `ModelResponse` and emits `PydanticDeprecatedSince20`. Read: + +- `response.text` for plain text. +- `response.output` for structured output. +- `response.message` for the full message. -Await it. Client stop: `turn.abort()` (or a timeout around the stream). -Server cancel: `chat.abort()` / `task.abort()` after you have a snapshot. -Details: [agents-background.md](agents-background.md). +`GenkitError` exposes `e.message`, `e.status`, `e.details`, `e.reason`, and `e.__cause__`. `e.original_message` raises `AttributeError`. --- -## Tool raise becomes `AgentError` +## `{{variable}}` Sent Literally -A raised tool ends the turn. Catch `AgentError` on `chat.send`. Prefer -returning a soft-error dict so the model can recover. History stays usable — -the next send continues from the last good snapshot. +`ai.generate(prompt=..., system=...)` sends strings verbatim. The model receives `Say hello to {{name}}.` as written. Render templates through `ai.define_prompt` or a `.prompt` file: + +```python +greet = ai.define_prompt(name='greet', prompt='Say hello to {{name}}.') +res = await greet({'name': 'Sam'}) +# Model receives: Say hello to Sam. +``` --- -## Remote client oddities +## Streaming `TypeError` or a Flow That Keeps Running After `break` + +```text +TypeError: object ModelStreamResponse can't be used in 'await' expression +``` + +1. Do not `await` `ai.generate_stream(...)`, `flow.stream(...)`, or `chat.send_stream(...)`. Iterate the stream, then `await` the final response: + ```python + sr = ai.generate_stream(prompt='Describe the tasting menu.') + async for chunk in sr.stream: + print(chunk.text, end='') + final = await sr.response + ``` +2. To stop a flow stream early, use `async with`. Leaving the block cancels the flow: + ```python + async with course_flow.stream(payload) as stream: + async for chunk in stream: + if chunk == 'dessert': + break # course_flow receives CancelledError + ``` + +--- + +## Cloud Telemetry Enabled but Spans Do Not Reach Cloud Trace + +By default in local development (`GENKIT_ENV=dev`), `enable_google_cloud_telemetry()` runs in local-only mode to prevent polluting Cloud Trace with local run data. To export traces to GCP from local development, pass `force_dev_export=True`: + +```python +from genkit_google_cloud import enable_google_cloud_telemetry + +enable_google_cloud_telemetry(project='bistro-prod', force_dev_export=True) +``` + +--- -Match `state_management` to the server. No trailing slash on the URL. Check -server logs. Auth: [agents-http.md](agents-http.md). +## Interrupts and Agent Session Pitfalls + +- **`except Exception:` does not catch `Interrupt`**: `Interrupt` subclasses `BaseException`, so a tool's catch-all cannot swallow approval interrupts. Put side-effect cleanup in `finally`. +- **Approval resume**: build restart parts with `part.restart(resumed_metadata={'tool_approved': True})` on items from `res.interrupts` (see [agents-human-in-the-loop.md](agents-human-in-the-loop.md)). +- **`ImportError: cannot import name 'AgentError'`**: failed turns do not raise. `chat.send()` returns a turn with `finish_reason == AgentFinishReason.FAILED` and `res.error`. `AgentInitError` raises from `agent.chat(...)` on bad init; `send` raises `GenkitError` only when no turn result arrives. +- **`res.snapshot_id` is `None`**: the agent has no store. Pass `store=InMemorySessionStore()` to `define_agent` (see [agents.md](agents.md)). +- **`AgentInitError: FAILED_PRECONDITION: Cannot send 'state' to a server-managed agent (one with a store). Send 'snapshot_id' or 'session_id' instead.`**: store-backed chats resume by ID. Mutate state inside the turn (see [agents-state.md](agents-state.md)). +- **Resume picks the wrong branch after a fork**: `session_id` resumes the most recently created leaf. Pass the branch's `snapshot_id` (see [agents-branching.md](agents-branching.md)). +- **Resume after a failed turn**: only `completed` snapshots resume by `snapshot_id`. Use `agent.chat(session_id=...)` to continue from the last good turn. +- **`chat.abort()` does nothing**: during a foreground `send` it is a no-op. Run the turn with `detach()` and call `await task.abort()` (see [agents-background.md](agents-background.md)). +- **Remote client mismatches**: align `state_management` with the server and omit trailing slashes on the base URL (see [agents-http.md](agents-http.md)). diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/dev-workflow.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/dev-workflow.md index 74681b1e..220d5e38 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/dev-workflow.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/dev-workflow.md @@ -1,127 +1,170 @@ # Dev Workflow — Genkit Python -## Agent responsibility +- [Agent Responsibility](#agent-responsibility): Lines 14-23 +- [Step 1 - Get a Gemini API Key](#step-1--get-a-gemini-api-key): Lines 27-29 +- [Step 2 - Export the API Key](#step-2--export-the-api-key): Lines 33-43 +- [Step 3 - Install Dependencies](#step-3--install-dependencies): Lines 47-56 +- [Step 4 - Start the Dev UI](#step-4--start-the-dev-ui): Lines 60-104 +- [Step 5 - Test in the Dev UI](#step-5--test-in-the-dev-ui): Lines 108-117 +- [CLI Commands](#cli-commands): Lines 121-160 +- [Troubleshooting](#troubleshooting): Lines 164-170 -After generating code, always give the developer: -1. The full pre-run checklist with copy-paste commands using absolute paths -2. The `genkit start` command to run in their terminal (foreground — it's expected to block) -3. Step-by-step Dev UI instructions so they can test without guessing +--- + +## Agent Responsibility -Do not offer to run it for them. Give them the commands and let them run it. +After generating or migrating code: +1. Run the project's type checker (`pyright`, `mypy`, or `ty`) in `.venv` so typed configs (`GeminiConfig`), generic contexts (`FlowContext[T]`, `ToolContext[T]`), keyword-only signatures, and `OutputT | None` checks pass before handing off. +2. Give the developer: + - Copy-paste setup commands with absolute paths + - The `genkit start` command to run in their foreground terminal (expected to block) + - Exact Dev UI steps to exercise the flow or agent and inspect traces + +Do not run `genkit start` as an interactive blocking command on the developer's behalf unless explicitly asked; use `genkit flow:run` for automated one-shot verification. --- -## Step 1 — Get a Gemini API key +## Step 1 — Get a Gemini API Key -If the developer doesn't have one: -> Get a free key at https://aistudio.google.com/apikey — click **"Create API key"**, copy it. +If the developer needs a key, create one at `https://aistudio.google.com/apikey`. --- -## Step 2 — Set the API key +## Step 2 — Export the API Key -Open a terminal and run: ```bash export GEMINI_API_KEY=your-api-key-here ``` -To persist across sessions, add it to your shell profile: +Persist across shell sessions if desired: + ```bash echo 'export GEMINI_API_KEY=your-api-key-here' >> ~/.zshrc && source ~/.zshrc ``` --- -## Step 3 — Install dependencies +## Step 3 — Install Dependencies -Replace `/path/to/your-project` with the actual full path to the project (e.g. `/Users/yourname/projects/my-genkit-app`): +Replace `/path/to/your-project` with the project's absolute path: ```bash cd /path/to/your-project -uv add genkit genkit-google-genai +uv add 'genkit==0.13.*' 'genkit-google-genai==0.13.*' ``` -(Requires a project with `pyproject.toml` — run `uv init` in an empty directory first if needed.) +(Run `uv init` first if `pyproject.toml` does not exist yet — see [Setup](setup.md).) --- ## Step 4 — Start the Dev UI -Run this in your terminal. **It will block — that's expected.** Leave this terminal open while you use the Dev UI. +The examples below use this `src/main.py`: + +```python +from pydantic import BaseModel + +from genkit import Genkit + +ai = Genkit() + + +class OrderInput(BaseModel): + text: str + + +@ai.flow() +async def order_flow(inp: OrderInput) -> str: + return f'order: {inp.text}' + + +async def main() -> None: + print(await order_flow(OrderInput(text='smoked salmon tartine'))) + + +if __name__ == '__main__': + ai.run_main(main()) + # => order: smoked salmon tartine + # Under `genkit start`, the process stays up for the Dev UI until Ctrl+C. +``` + +Run in a foreground terminal (blocks until stopped with `Ctrl+C`): ```bash cd /path/to/your-project GEMINI_API_KEY=your-api-key-here genkit start -- uv run src/main.py ``` -You'll see output like: -``` -Genkit Tools UI: http://localhost:4000 -``` +For FastAPI apps, run uvicorn under `genkit start` instead; the reflection server starts on a background thread (see [FastAPI](fastapi.md)): -The Dev UI is now running at **http://localhost:4000** +```bash +GEMINI_API_KEY=your-api-key-here genkit start -- uv run uvicorn main:app --reload +``` -To stop it: press `Ctrl+C` in the terminal. +Wait for the CLI to print `Genkit Developer UI: http://localhost:4000`. --- ## Step 5 — Test in the Dev UI -1. Open **http://localhost:4000** in your browser -2. Click **"Run"** in the left sidebar -3. Find your flow by name (e.g. `summarize`, `chat`, `joke_generator`) -4. In the input box, paste your input as JSON — e.g: +1. Open **`http://localhost:4000`** in a browser. +2. Pick the registered flow, prompt, or model from the left sidebar. Flows appear under their Python function name (e.g. `order_flow`). +3. Paste JSON matching the flow's single input parameter, e.g. for `order_flow(inp: OrderInput)`: ```json - {"text": "hello world"} + {"text": "smoked salmon tartine"} ``` -5. Click the **"Run"** button — the output appears on the right -6. Click **"Traces"** in the left sidebar to inspect every step, model call, token count, and latency +4. Click **Run** and inspect the validated output on the right. +5. Open **Traces** in the left sidebar to inspect every span, rendered prompt, tool call, token count, latency, and unwrapped error stack trace. --- ## CLI Commands -`genkit start` unintrusively wraps any Python program that uses the Genkit library, running it unchanged while capturing traces from every Genkit action so you can prove tools were actually called and inspect model I/O from the terminal, even for headless checks. It forwards stdio, so interactive CLI tools that rely on stdin/stdout work without issues. Running the app directly (`uv run`) skips trace capture, so you're debugging blind. +`genkit start` sets `GENKIT_ENV=dev`, runs your command, and stores local traces from every Genkit action under `.genkit/` in the project root. The app's stdout and stderr pass through. Running `uv run` directly skips the reflection server and local trace capture. -**Primary pattern (default):** prefix `genkit start --` to your normal run command. This collects telemetry from any Genkit code your program runs, whether triggered from the dev UI, your own web server/web UI, or a plain script: +**Persistent dev runtime:** ```bash genkit start -- uv run src/main.py -genkit start --noui -- uv run src/main.py # same, without the Dev UI (still a persistent server) +genkit start --noui -- uv run src/main.py # no Dev UI; runtime still blocks until Ctrl+C +genkit start --non-interactive -- uv run src/main.py # CLI prompts take the default choice (CI/agents) +genkit start --port 4001 -- uv run src/main.py # Dev UI on another port ``` -`genkit start` runs until you stop it with Ctrl+C. That is expected and correct for the common cases: a server your web/mobile app calls, or an interactive CLI you exit yourself. `--noui` only drops the Dev UI; it is **not** a one-shot command and will not exit on its own. Do **not** use `genkit start` as a blocking step in automated/non-interactive contexts; use `flow:run` (below) for that. -**Non-interactive use (agents/CI):** add the global `--non-interactive` flag before `--` so the CLI uses defaults and never blocks on a prompt (e.g. the first-run analytics notice): `genkit start --non-interactive -- uv run src/main.py` (works with `flow:run` too). - -**Run a flow (`flow:run`):** invoke a specific flow by name from the CLI. Append your run command after `--` to spin up the runtime just for this run (the command runs as-is to register your flows): +**One-shot flow execution (`flow:run`):** with `-- <cmd>`, it starts the app, runs one `@ai.flow()` by its Python function name, prints `Result:` and `Trace ID:`, and exits: ```bash -genkit flow:run myFlow '{"data": "input"}' -- uv run src/main.py +genkit flow:run order_flow '{"text": "smoked salmon tartine"}' -- uv run src/main.py +# => Result: +# order: smoked salmon tartine +# Trace ID: 395f73c3d0d662c28711239a4354bf53 ``` -This is **self-terminating**: it runs the flow once, prints a `Trace ID`, then exits, so it's the right choice for a quick, non-interactive check (unlike `genkit start`). Note: `flow:run` runs **flows** (`@ai.flow()`), not agents; you can't `flow:run` an agent (`ai.define_agent`) directly. To exercise an agent from the CLI, wrap one turn in a throwaway flow and run that (see [Agents](agents.md)). Traces for this run can be inspected using the trace commands below. +Without `-- <cmd>`, `flow:run` targets a runtime already started by `genkit start` in the same directory, and fails with `No runtimes found` if none is running. A `str` input is a JSON string: `'"risotto"'`. + +`flow:run` runs `@ai.flow()` actions, not agents. Wrap one agent turn in a small flow to test it (see [Agents](agents.md#verify-an-agent-from-the-cli-flowrun)). -**Debugging with traces:** the fastest way to see prompts, model inputs/outputs, tool calls, latencies, and errors. Inspect from the terminal after any run under `genkit start`: +**Inspecting traces from the terminal:** ```bash -genkit trace:list # find recent trace IDs -genkit trace:get <traceId> # full trace details (inputs, outputs, tool calls, errors) -genkit trace:get <traceId> --format json # machine-readable JSON, safe to pipe into jq or other parsers +genkit trace:list --limit 5 # recent trace IDs with status, input, output +genkit trace:get <traceId> # full trace as JSON: every span, attributes, input, output +``` +`trace:get` prints a `Telemetry API running on ...` line before the JSON on stdout, so strip it before piping into `jq`: +```bash +genkit trace:get <traceId> | sed 1d | jq -r '.spans[].displayName' +# => order_flow ``` -For machine-readable output, pass `--format json` to get clean JSON you can pipe into `jq` or other parsers. The **default** output is human-oriented (banner/log lines, possible truncation on large traces), so don't pipe that form directly; use `--format json`, grep, or the Dev UI trace viewer. - - -**Documentation:** +**Built-in CLI documentation lookup:** ```bash genkit docs:search "streaming" python genkit docs:list python genkit docs:read python/flows.md ``` +--- ## Troubleshooting - `genkit: command not found` — `npm install -g genkit-cli` -- `GEMINI_API_KEY not set` — `export GEMINI_API_KEY=your-key` -- Port 4000 already in use — - `genkit start --port 4001 -- uv run src/main.py` -- `uv: command not found` — - `curl -LsSf https://astral.sh/uv/install.sh | sh` -- Flow not showing in Dev UI — check `genkit start` output for errors +- `GEMINI_API_KEY` missing — `export GEMINI_API_KEY=your-key` +- Port 4000 busy — `genkit start --port 4001 -- uv run src/main.py` +- `uv: command not found` — `curl -LsSf https://astral.sh/uv/install.sh | sh` +- Action missing in Dev UI — ensure standalone scripts enter via `ai.run_main(main())` and check terminal output for startup import or validation errors (see [Common Errors](common-errors.md)). diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/dotprompt.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/dotprompt.md index 0bc91ad5..0797a0a4 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/dotprompt.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/dotprompt.md @@ -1,166 +1,190 @@ # Dotprompt — Genkit Python -## What it is +- [What It Is](#what-it-is): Lines 15-17 +- [File Format](#file-format): Lines 19-54 +- [Python Setup](#python-setup): Lines 56-77 +- [Calling a Prompt (Prompt)](#calling-a-prompt-prompt): Lines 79-113 +- [Streaming from a Prompt](#streaming-from-a-prompt): Lines 115-132 +- [Render Without Generating](#render-without-generating): Lines 134-146 +- [Helpers](#helpers): Lines 148-168 +- [Variants & Partials](#variants--partials): Lines 170-173 +- [Prompts with Tools & Middleware](#prompts-with-tools--middleware): Lines 175-190 -`.prompt` files combine YAML frontmatter (model config, schemas) with Handlebars -templates. Keeps prompt logic out of Python code and makes variants easy. +--- + +## What It Is + +`.prompt` files pair YAML frontmatter (model, config, input/output schemas, tools, middleware) with Handlebars templates. Handlebars templates **only** render inside `.prompt` files and `ai.define_prompt(...)` — strings passed to `ai.generate(prompt=..., system=...)` are sent to the model verbatim. -## File format +## File Format + +Place `.prompt` files inside `prompts/` and pass `prompt_dir=...` to `Genkit(...)`: ```yaml --- model: googleai/gemini-flash-latest +config: + temperature: 0.4 input: + default: + tone: practical schema: food: string - tone?: string # optional scalar — just `field?: type` - ingredients?(array): string # optional array/object needs the parenthetical + tone?: string # optional scalar — `field?: type` + ingredients?(array): string # optional array/object uses parenthetical syntax output: - schema: Recipe # references a schema registered with ai.define_schema() + schema: Recipe # references a schema registered via ai.define_schema() format: json --- {{role "system"}} -You are a chef. Keep recipes practical. +You are a chef. Keep recipes {{tone}}. {{role "user"}} Generate a recipe for {{food}}. {{#if ingredients}} -Prefer these ingredients: {{ingredients}}. +Prefer these ingredients: +{{list ingredients}} {{/if}} ``` -Use `{{role "system"}}` / `{{role "user"}}` (and friends) so the model sees a -real system instruction. A flat template body becomes a single **user** message. - -Place `.prompt` files in a `prompts/` directory and point `prompt_dir` at it. +Use `{{role "system"}}` and `{{role "user"}}` to emit distinct message roles; a template without `{{role ...}}` becomes a single user message. -**Note:** `input.schema` describes the contract for humans and tooling. Today it -is **not** validated locally before the model call — missing required fields can -still render and hit the API. Validate in your flow if you need hard failures. +**Input & output validation:** +- Prompt inputs **are validated** against `input.schema` (or `input_schema=` on `define_prompt`) before calling the model, and `input.default` values from `.prompt` frontmatter are filled in automatically. +- When `output.schema` names a Pydantic class registered with `ai.define_schema('Recipe', Recipe)`, `response.output` is validated and returned directly as `Recipe | None` (`None` when the model's JSON is truncated or malformed; check `response.finish_reason`). -## Python setup +## Python Setup ```python from pathlib import Path from pydantic import BaseModel -from genkit import Genkit +from genkit import Genkit, Prompt from genkit_google_genai import GoogleAI ai = Genkit( plugins=[GoogleAI()], - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), prompt_dir=Path(__file__).resolve().parent.parent / 'prompts', ) + class Recipe(BaseModel): title: str steps: list[str] + ai.define_schema('Recipe', Recipe) ``` -## Calling a prompt +## Calling a Prompt (`Prompt`) + +`ai.prompt(...)` and `ai.define_prompt(...)` return a `Prompt`. There is no `.execute()` method — call and `await` the `Prompt` directly: + +```python +recipe_prompt: Prompt = ai.prompt('recipe') + +# Call options use the same keyword names as `ai.generate`. +# `config` merges key-by-key over the prompt's frontmatter config. +# `output_schema` is fixed by the prompt and cannot be overridden per call. +response = await recipe_prompt( + {'food': 'banana bread', 'ingredients': ['walnuts', 'cinnamon']}, + config={'temperature': 0.2}, +) +if response.output is not None: + recipe: Recipe = response.output -There is **no** `prompt.execute()`. Non-streaming uses the callable -(`ExecutablePrompt.__call__`): +# Variant (`prompts/recipe.quick.prompt`) +quick_res = await ai.prompt('recipe', variant='quick')({'food': 'banana bread'}) +``` + +### Code-Defined Prompts (`ai.define_prompt`) ```python -# Non-streaming — await the prompt itself (double-call: ai.prompt('name') then (...)) -prompt = ai.prompt('recipe') -response = await prompt(input={'food': 'banana bread'}) -# same as: await ai.prompt('recipe')(input={'food': 'banana bread'}) -result = Recipe.model_validate(response.output) - -# Variant (recipe.robot.prompt file) -response = await ai.prompt('recipe', variant='robot')(input={'food': 'banana bread'}) +class GreetingInput(BaseModel): + name: str + + +greet = ai.define_prompt( + name='greet', + input_schema=GreetingInput, + prompt='Say hello to {{name}}.', +) +res = await greet(GreetingInput(name='Ana')) ``` -## Streaming from a prompt +## Streaming from a Prompt -`.stream(...)` is **not** awaited; then consume `.stream` / `.response`: +`prompt.stream(...)` is synchronous (do not `await` it); iterate `.stream` and `await` `.response`: ```python -from genkit import ActionRunContext +from genkit import FlowContext @ai.flow() -async def tell_story(subject: str, ctx: ActionRunContext) -> str: - result = ai.prompt('story').stream(input={'subject': subject}) - full = '' - async for chunk in result.stream: +async def stream_recipe(food: str, ctx: FlowContext[str]) -> str: + sr = ai.prompt('recipe').stream({'food': food}) + chunks: list[str] = [] + async for chunk in sr.stream: if chunk.text: ctx.send_chunk(chunk.text) - full += chunk.text - final = await result.response # completes even if you stop reading chunks early - return final.text or full + chunks.append(chunk.text) + final = await sr.response + return final.text or ''.join(chunks) ``` -## Render without generating (for LLM-judge evals) +## Render Without Generating + +`render(...)` takes the same arguments as a call and returns the `GenerateActionOptions` (model, config, messages, output schema) without calling a model. Use it to debug templates and input defaults: ```python -rendered = await ai.prompt('my_prompt').render(input={'key': 'value'}) -# Inspect roles/messages before spending tokens: -# print(rendered.messages) -response = await ai.generate(model='googleai/gemini-flash-latest', messages=rendered.messages) +rendered = await ai.prompt('recipe').render({'food': 'focaccia'}) +print(rendered.model, rendered.config) +# => googleai/gemini-flash-latest {'temperature': 0.4} +for message in rendered.messages: + print(message.role, message.text.strip()) +# => system You are a chef. Keep recipes practical. +# => user Generate a recipe for focaccia. ``` ## Helpers -Handlebars helpers receive template args in a packed form. Prefer simple -string/list formatting in the template, or unpack carefully: +Register Handlebars helpers with `ai.define_helper(name, fn)`. Dotprompt calls every helper as `fn(params, options)`: `params` is the list of positional template arguments, `options` carries hash arguments. A one-argument function raises `TypeError` at render time. ```python -def list_helper(data: object, *args, **kwargs) -> str: - # Positional template args arrive packed in the first parameter. - items = data[0] if isinstance(data, (list, tuple)) and data else data +from typing import Any + + +def list_helper(params: list[Any], options: Any) -> str: + items = params[0] if params else None if not isinstance(items, list): return '' return '\n'.join(f'- {item}' for item in items) + ai.define_helper('list', list_helper) ``` -Then `{{list ingredients}}` in the `.prompt`. If output looks like a Python -`repr` of a list, the helper unpacked wrong — fix the helper, not the model. - -## Variants - -Name the file `<name>.<variant>.prompt` — e.g. `recipe.robot.prompt`. -Call with `ai.prompt('recipe', variant='robot')`. - -## Partials +Built-in helpers: `role`, `json`, `media`, `history`, `section`, `ifEquals`, `unlessEquals`. -Use `{{>partial_name param=value}}` in templates. Partial files are named -`_partial_name.prompt`. +Use `{{list ingredients}}` in the `.prompt` body. -## Prompts + tools + structured output - -Naming a tool in the template is not enough — pass tool objects at call time -(or list names in frontmatter `tools:` that match registered tools): - -```python -response = await ai.prompt('insurance_quote')( - input={'age': 35, 'zip': '94105', 'coverage_tier': 'plus'}, - tools=[lookup_rate], -) -quote = QuoteResult.model_validate(response.output) -``` +## Variants & Partials -Register output schemas with `ai.define_schema('QuoteResult', QuoteResult)` -when frontmatter references them by name. +- **Variants**: save `<name>.<variant>.prompt` (e.g. `recipe.vegan.prompt`) and load with `ai.prompt('recipe', variant='vegan')`. +- **Partials**: prefix partial files with `_` (e.g. `prompts/_disclaimer.prompt`) or register with `ai.define_partial('disclaimer', '...')`, then include via `{{>disclaimer}}` or `{{>disclaimer region=region}}`. -### Partials example +## Prompts with Tools & Middleware -``` -prompts/_greeting.prompt # partial body only -prompts/support_reply.prompt -``` +- **Tools**: list registered tool names under `tools:` in YAML frontmatter, or pass `tools=[lookup_rate]` when calling `await prompt(input, tools=[lookup_rate])`. The tool loop stops after `maxTurns` (frontmatter) or `max_turns=` (call), default 50. +- **Middleware**: pass instances at call time with `use=[Retry(max_retries=2)]`. To name middleware under `use:` in frontmatter, register it first: add `Middleware()` from `genkit_middleware` to `Genkit(plugins=[...])` (registers `retry`, `fallback`, `tool_approval`, `skills`, `filesystem`) or define your own with `@ai.middleware`. An unregistered name fails with `NOT_FOUND`. -In `support_reply.prompt`: -``` -{{>greeting}} -... main template ... -{{>disclaimer}} +```yaml +--- +model: googleai/gemini-flash-latest +tools: [lookup_rate] +use: + - name: retry + config: + max_retries: 2 +--- +Convert {{amount}} USD to EUR. ``` - -Call `await ai.prompt('support_reply')(input={...})`. Parent input is visible -inside partials unless you override with `{{>greeting name=name}}`. diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/evals.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/evals.md index 9c1b7a77..22fb8ba9 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/evals.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/evals.md @@ -1,101 +1,117 @@ -# Evals — Genkit Python +# Evals — Genkit Python (0.13 Only; Removed in 1.0) -## Two types of evaluators +> **Warning:** 0.13 is the last release with evaluators (`ai.define_evaluator`, `ai.define_batch_evaluator`, `ai.evaluate`, `genkit.evaluator`). Genkit 1.0 cuts them and the `genkit-evaluators` package from the 1.0 release; evaluation will return as an additive feature in a later release. `genkit-evaluators` 0.13 requires `genkit<1`. +> - **New projects:** do not add `genkit-evaluators` or `define_evaluator` unless you intend to stay pinned to `genkit==0.13.*`. Prefer `pytest` assertions against your flows and prompts until evaluation returns. +> - **Migrating from 0.12:** stop and ask the developer before porting evaluator code (see [Migrating from 0.12 to 0.13](migrating-from-0.12.md)). -1. **Built-in** — ship with `genkit-evaluators`, register with - `register_genkit_evaluators(ai)`. Regex / deep_equal do not need - `GoogleAI()` or `GEMINI_API_KEY`. -2. **BYO** — `ai.define_evaluator()` with your own scoring logic (including - LLM judges). +The patterns below document the `0.13.*` evaluator API for projects staying on the 0.13 release line. -## Install +- [Install (0.13 Only)](#install-013-only): Lines 19-23 +- [Two Evaluator Types](#two-evaluator-types): Lines 25-28 +- [Dataset Format](#dataset-format): Lines 30-41 +- [Built-In Evaluators & Programmatic Execution](#built-in-evaluators--programmatic-execution): Lines 43-68 +- [BYO Evaluator (EvaluatorResponse, Score, ScoreStatus, ScoreDetails)](#byo-evaluator-evaluatorresponse-score-scorestatus-scoredetails): Lines 70-108 +- [LLM-Based Judge](#llm-based-judge): Lines 110-166 +- [CLI Execution](#cli-execution): Lines 168-180 + +--- + +## Install (0.13 Only) ```bash -uv add genkit-evaluators +uv add 'genkit-evaluators==0.13.*' ``` -## Dataset format +## Two Evaluator Types + +1. **Built-in** — provided by `genkit-evaluators`, registered via `register_genkit_evaluators(ai)` (`genkitEval/regex`, `genkitEval/deep_equal`, `genkitEval/jsonata`). They are deterministic and need no model plugin or `GEMINI_API_KEY`. Pass `metrics=['regex']` to register a subset. +2. **BYO** — custom row-by-row (`ai.define_evaluator`) or batch (`ai.define_batch_evaluator`) functions, including LLM judges. + +## Dataset Format + +`genkit eval:run` reads a JSON array: ```json [ - {"testCaseId": "case1", "input": "x", "output": "banana", "reference": "ba?a?a"}, - {"testCaseId": "case2", "input": "x", "output": "apple", "reference": "ba?a?a"} + {"testCaseId": "dessert", "input": "Suggest a dessert.", "output": "Banana bread", "reference": "(?i)banana"}, + {"testCaseId": "starter", "input": "Suggest a starter.", "output": "Tomato soup", "reference": "(?i)banana"} ] ``` -Fields: `testCaseId`, `input`, `output`, `reference` (optional for some -evaluators). +Fields: `testCaseId`, `input`, `output`, `reference` (optional for reference-free evaluators). In Python, the same rows are `BaseDataPoint(test_case_id=..., input=..., output=..., reference=...)`. -## Built-in evaluators +## Built-In Evaluators & Programmatic Execution ```python +from genkit import Genkit +from genkit.evaluator import BaseDataPoint from genkit_evaluators import register_genkit_evaluators + +# 1. Register the built-in evaluators +ai = Genkit() register_genkit_evaluators(ai) -``` -Registered: `genkitEval/regex`, `genkitEval/deep_equal`, `genkitEval/jsonata`. -`eval:run` needs a Genkit runtime: +# 2. Build the dataset +dataset = [ + BaseDataPoint(test_case_id='dessert', input='Suggest a dessert.', output='Banana bread', reference='(?i)banana'), + BaseDataPoint(test_case_id='starter', input='Suggest a starter.', output='Tomato soup', reference='(?i)banana'), +] -```bash -# terminal A -genkit start -- uv run src/main.py -# terminal B -genkit eval:run datasets/my_dataset.json --evaluators=genkitEval/regex +# 3. Run one evaluator per call; returns list[EvaluatorResponse] in dataset order +results = await ai.evaluate(evaluator='genkitEval/regex', dataset=dataset) +for row in results: + print(row.test_case_id, row.evaluation[0].score, row.evaluation[0].status) +# => dessert True PASS +# starter False FAIL ``` -One-shot (starts the app as the runtime): +`config=` is passed to the evaluator as its second argument (`None` when unset). An unregistered evaluator name raises `GenkitError` with `NOT_FOUND`. -```bash -genkit eval:run datasets/my_dataset.json --evaluators=genkitEval/regex -- uv run src/main.py -``` - -`No runtimes found` means nothing is registered — start the app or use `--`. +## BYO Evaluator (`EvaluatorResponse`, `Score`, `ScoreStatus`, `ScoreDetails`) -Programmatic — one evaluator per call (`evaluator=`, singular): +`ai.define_evaluator` takes `(name, fn)` positionally with required keyword-only `display_name=` and `definition=`. `EvaluatorResponse` requires `test_case_id`, and `evaluation` takes a `list[Score]`: ```python -result = await ai.evaluate( - dataset=my_dataset, - evaluator='byo/my_eval', # not evaluators=[...] +from genkit.evaluator import ( + BaseDataPoint, + EvaluatorResponse, + Score, + ScoreDetails, + ScoreStatus, ) -# EvalResponse — use result.root (not .results) -``` -Add `--output=eval_out.json` on CLI for machine-readable pass/fail. -## BYO evaluator - -```python -from genkit.evaluator import BaseDataPoint, Details, EvalFnResponse, EvalStatusEnum, Score - -async def my_eval(datapoint: BaseDataPoint, _options: dict | None = None) -> EvalFnResponse: - output = str(datapoint.output or '') - reference = str(datapoint.reference or '') - passed = output.strip() == reference.strip() - return EvalFnResponse( +async def exact_match(datapoint: BaseDataPoint, _options: object | None = None) -> EvaluatorResponse: + output = str(datapoint.output or '').strip() + reference = str(datapoint.reference or '').strip() + passed = output == reference + return EvaluatorResponse( test_case_id=datapoint.test_case_id or '', - evaluation=Score( - score=1.0 if passed else 0.0, - status=EvalStatusEnum.PASS if passed else EvalStatusEnum.FAIL, - details=Details(reasoning='Exact match check'), - ), + evaluation=[ + Score( + score=1.0 if passed else 0.0, + status=ScoreStatus.PASS if passed else ScoreStatus.FAIL, + details=ScoreDetails(reasoning='Exact match check'), + ) + ], ) + ai.define_evaluator( - name='byo/my_eval', - display_name='My Eval', - definition='Checks exact match of output vs reference.', - fn=my_eval, + 'byo/exact_match', + exact_match, + display_name='Exact Match', + definition='Checks exact match of output against reference.', ) ``` -## LLM-based judge +For batch evaluation across the whole dataset at once, register `async def my_batch_eval(req: EvaluatorRequest) -> list[EvaluatorResponse]:` with `ai.define_batch_evaluator('byo/batch', my_batch_eval, display_name='...', definition='...')` (`from genkit.evaluator import EvaluatorRequest`). The batch function takes exactly one argument; read options from `req.options`. -Keep a judge prompt next to the app and parse a score from the model. +## LLM-Based Judge -`prompts/judge.prompt`: +`prompts/judge.prompt` (load it with `Genkit(prompt_dir='prompts', ...)`; `prompts/` is the default): -``` +```yaml --- model: googleai/gemini-flash-latest input: @@ -111,42 +127,54 @@ Candidate: {{output}} ``` ```python -import re +from genkit import Genkit +from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score, ScoreStatus +from genkit_google_genai import GoogleAI -async def llm_eval(datapoint: BaseDataPoint, _options: dict | None = None) -> EvalFnResponse: - rendered = await ai.prompt('judge').render( - input={'output': str(datapoint.output), 'reference': str(datapoint.reference)} - ) - response = await ai.generate( - model='googleai/gemini-flash-latest', - messages=rendered.messages, +ai = Genkit(plugins=[GoogleAI()]) + + +async def llm_judge(datapoint: BaseDataPoint, _options: object | None = None) -> EvaluatorResponse: + # 1. Run the judge prompt; the model comes from the .prompt frontmatter + response = await ai.prompt('judge')( + {'output': str(datapoint.output), 'reference': str(datapoint.reference)} ) - m = re.search(r'0?\.\d+|1\.0|[01]', (response.text or '').strip()) - score = float(m.group(0)) if m else 0.0 - return EvalFnResponse( + + # 2. Parse the score; treat anything unparseable as 0.0 + try: + score = min(max(float((response.text or '').strip()), 0.0), 1.0) + except ValueError: + score = 0.0 + + return EvaluatorResponse( test_case_id=datapoint.test_case_id or '', - evaluation=Score( - score=score, - status=EvalStatusEnum.PASS if score >= 0.5 else EvalStatusEnum.FAIL, - ), + evaluation=[Score(score=score, status=ScoreStatus.PASS if score >= 0.5 else ScoreStatus.FAIL)], ) + ai.define_evaluator( - name='byo/llm_judge', + 'byo/llm_judge', + llm_judge, display_name='LLM Judge', definition='Scores candidate vs reference with a judge model.', - fn=llm_eval, + is_billed=True, ) ``` -Built-in `genkitEval/regex` scores are `True`/`False`; BYO scores are usually -`0.0`/`1.0` — normalize before aggregating. +`is_billed=True` makes `genkit eval:run` ask for confirmation before calling the model; pass `--force` in CI. -## CLI +Built-in `genkitEval/regex` and `genkitEval/deep_equal` scores are `bool` (`True` / `False`); BYO numeric scores are typically `0.0`–`1.0`. Normalize before aggregating. + +## CLI Execution + +`genkit eval:run` requires an active Genkit runtime (either already running via `genkit start` or launched inline after `--`): ```bash -genkit eval:run datasets/my_dataset.json --evaluators=byo/my_eval -genkit eval:run datasets/my_dataset.json --evaluators=genkitEval/regex,byo/my_eval +# One-shot: launches the app runtime, evaluates, and exits +genkit eval:run datasets/menu_dataset.json --evaluators=genkitEval/regex,byo/exact_match -- uv run src/main.py + +# Write machine-readable results to a file (--output-format json|csv, default json) +genkit eval:run datasets/menu_dataset.json --evaluators=byo/exact_match --output=eval_out.json -- uv run src/main.py ``` -Results appear in the Dev UI under **Evaluate**. +Omitting `--evaluators` runs every registered evaluator. `No runtimes found` means no Genkit runtime is active — pass `-- uv run src/main.py` or run `genkit start` in another terminal. diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/examples.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/examples.md index 13a000fa..13c071fc 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/examples.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/examples.md @@ -1,29 +1,71 @@ # Genkit Python Examples -Minimal patterns for common Genkit APIs. Examples use **Google AI** (`GoogleAI`, `googleai/...`); other providers use the same patterns with the right plugin and model prefix. +Minimal patterns for generation, structured output, streaming, media, flows, tools, interrupts, cancellation, and embeddings. -## Public imports +- [Public Imports & Initialization](#public-imports--initialization): Lines 16-58 +- [Structured Output](#structured-output): Lines 62-95 +- [Streaming (Text & Structured Output)](#streaming-text--structured-output): Lines 99-126 +- [Media Input & Output (Part)](#media-input--output-part): Lines 130-192 +- [Flows & Streaming Flows](#flows--streaming-flows): Lines 196-225 +- [Tools, Multipart Tool Responses, and Interrupts](#tools-multipart-tool-responses-and-interrupts): Lines 229-286 +- [Cancellation](#cancellation): Lines 290-317 +- [Embeddings](#embeddings): Lines 321-347 -Use public packages only — `genkit`, `genkit_google_genai`, `genkit_fastapi`, -`genkit_middleware`, `genkit_evaluators`, `genkit.agent`, `genkit.embedder`, -`genkit.evaluator`, `genkit.model`, etc. Do not import internal modules -(`genkit._core`, …). +--- + +## Public Imports & Initialization + +Import only from public modules (`genkit`, `genkit_google_genai`, `genkit_fastapi`, `genkit_middleware`, `genkit.model`, `genkit.embedder`, `genkit.middleware`, `genkit.web`, `genkit.exp`, `genkit.exp.agent`). Never import private `_` submodules. ```python -from genkit import Genkit, ActionRunContext +from genkit import Genkit from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +ai = Genkit( + plugins=[GoogleAI()], + model=GoogleAI.gemini('gemini-flash-latest'), +) +``` + +For multi-turn stateful agents, import `Genkit` from `genkit.exp` and session types from `genkit.exp.agent` (see [Agents](agents.md)). + +### Provider Model Refs & Typed Config + +Other providers follow the same `Plugin.helper('model-id')` + `<Plugin>Config` shape (import list in [SKILL.md](../SKILL.md#imports)). + +```python +from genkit_google_genai import GeminiConfig, GoogleAI + +# Typed config with flat fields + `extra` escape hatch for unmodeled provider fields +res = await ai.generate( + model=GoogleAI.gemini('gemini-flash-latest'), + prompt='Check recent FDA allergen labeling updates.', + config=GeminiConfig( + thinking_level='HIGH', + include_thoughts=True, + google_search=True, + extra={'seed': 42}, + ), +) + +# Per-request API key (never pass `api_key` inside `config` — that raises GenkitError) +tenant_res = await ai.generate( + prompt='Summarize invoice #4821.', + context={'secrets': {'api_key': tenant_api_key}}, +) ``` -For agents, see [Agents](agents.md) (`from genkit.agent import ...`). +Config defaults merge key-by-key (`ModelRef` < `Prompt` < call); pass `None` on a field to clear an inherited key. Unknown config keys raise `GenkitError` (`INVALID_ARGUMENT`) before calling the provider. --- -## Structured output +## Structured Output + +`response.output` is typed `OutputT | None`. If the model reply is cut off (`max_output_tokens`) or malformed, `response.output` is `None` and `response.error` (`Failure`) has `reason == ErrorReason.INVALID_OUTPUT`. ```python from pydantic import BaseModel, TypeAdapter +from genkit import ErrorReason class CityInfo(BaseModel): name: str @@ -32,181 +74,274 @@ class CityInfo(BaseModel): response = await ai.generate( prompt='Give facts about Tokyo.', - output_format='json', output_schema=CityInfo, ) -city = response.output +if response.output is None: + if response.error and response.error.reason == ErrorReason.INVALID_OUTPUT: + raise RuntimeError(f'Incomplete structured output: {response.error.message}') + raise RuntimeError('No structured output returned') + +city: CityInfo = response.output # Arrays schema = TypeAdapter(list[CityInfo]).json_schema() -response = await ai.generate( - prompt='List 3 cities.', +list_res = await ai.generate( + prompt='List 3 cities in Japan.', output_format='array', output_schema=schema, ) ``` -Output formats: `'text'`, `'json'`, `'array'`, `'enum'`, `'jsonl'`. +Built-in `output_format` values: `'text'`, `'json'`, `'array'`, `'enum'`, `'jsonl'`. --- -## Streaming (text) +## Streaming (Text & Structured Output) ```python -sr = ai.generate_stream(prompt='Tell me a story.') +sr = ai.generate_stream(prompt='Explain sourdough fermentation.') async for chunk in sr.stream: if chunk.text: print(chunk.text, end='', flush=True) -final = await sr.response # final.text +final = await sr.response ``` -You do **not** need to drain `.stream` for the turn to finish — `await sr.response` -completes even if you break out of the loop early (same idea as agent -`send_stream`). - ---- - -## Text and media parts +Streaming structured output yields partial typed objects on `chunk.output` without raising on incomplete JSON: ```python -# Non-streaming -response = await ai.generate(prompt='...') -for media in response.media: - print(media.content_type, (media.url or '')[:80]) - -# Streaming — media usually complete on the final response -from genkit import MediaPart +class RecipeCard(BaseModel): + title: str + prep_minutes: int + steps: list[str] -sr = ai.generate_stream(prompt='...') +sr = ai.generate_stream( + prompt='Create a weeknight shakshuka recipe.', + output_schema=RecipeCard, +) async for chunk in sr.stream: - if chunk.text: - print(chunk.text, end='', flush=True) + if chunk.output: + print(chunk.output) final = await sr.response -for media in final.media: - print(media.content_type, (media.url or '')[:80]) - -if final.message: - for part in final.message.content: - if isinstance(part.root, MediaPart) and part.root.media: - print(part.root.media.content_type) +recipe = final.output # RecipeCard | None ``` --- +## Media Input & Output (`Part`) -## Sending image / media input +Construct parts with `Part.from_text(...)` and `Part.from_media(...)`. Inspect parts via `part.text` and `part.media`. -The section above covers **model-produced** media. To *send* an image to Gemini, -build a user `Message` with text + `MediaPart` (URL or data URI): +### Sending Image / Media Input ```python -from genkit import Media, MediaPart, Message, Part, Role, TextPart +import base64 +from pathlib import Path +from genkit import Message, Part, Role + +# Direct HTTPS URL (< 20 MB) or base64 data URI +b64 = base64.b64encode(Path('receipt.png').read_bytes()).decode() +data_uri = f'data:image/png;base64,{b64}' -image_url = 'https://example.com/cat.jpg' # or data:image/png;base64,... msg = Message( role=Role.USER, content=[ - Part(root=TextPart(text='What is in this image?')), - Part(root=MediaPart(media=Media(url=image_url, content_type='image/jpeg'))), + Part.from_text('Extract the total amount from this receipt.'), + Part.from_media(data_uri, content_type='image/png'), ], ) response = await ai.generate(messages=[msg]) print(response.text) ``` -Prefer a vision-capable model (`googleai/gemini-flash-latest`). URLs are -fetched **server-side** — hotlink-blocked or thumbnail URLs often fail. -Prefer a direct image URL or a data URI: - -```python -import base64 -b64 = base64.b64encode(Path('cat.png').read_bytes()).decode() -url = f'data:image/png;base64,{b64}' -``` - +Media URLs larger than **20 MB** fail instead of downloading into memory, and `Ollama` does not read local filesystem paths for media. -## Streaming + structured output +### Generating Image / Audio / Video Output ```python -class StoryAnalysis(BaseModel): - title: str - genre: str - summary: str +from genkit_google_genai import GeminiImageConfig, GeminiTtsConfig, GoogleAI, VertexAI -sr = ai.generate_stream( - prompt='Write a short story then analyze it.', - output_format='json', - output_schema=StoryAnalysis, +# Image generation (Imagen catalog is retired; use gemini_image) +img_res = await ai.generate( + model=GoogleAI.gemini_image('gemini-2.5-flash-image'), + prompt='Minimalist watercolor illustration of an espresso machine.', + config=GeminiImageConfig(aspect_ratio='1:1'), ) -async for chunk in sr.stream: - if chunk.text: - print(chunk.text, end='', flush=True) -final = await sr.response -analysis = final.output +for media in img_res.media: + print(media.content_type, (media.url or '')[:60]) + +# Inspecting parts directly on a message +if img_res.message: + for part in img_res.message.content: + if part.media: + print(part.media.content_type) + +# Text-to-speech +tts_res = await ai.generate( + model=GoogleAI.gemini_tts('gemini-2.5-flash-preview-tts'), + prompt='Order 42 is ready at the counter.', + config=GeminiTtsConfig(voice='Kore'), +) + +# Background video generation (Veo via Vertex AI) +op = await ai.generate_operation( + model=VertexAI.veo('veo-3.1-generate-preview'), + prompt='Aerial shot of coastal cliffs at sunrise.', +) +while not op.done: + op = await ai.check_operation(op) ``` --- -## Flows +## Flows & Streaming Flows + +Flows validate both input and return values against their type annotations. Calling `await summarize(inp)` or `await summarize.run(inp)` raises body exceptions unwrapped in-process; over HTTP, raise `PublicError` for client-visible 4xx errors (see [FastAPI](fastapi.md)). ```python +from genkit import FlowContext, PublicError + class SummarizeInput(BaseModel): text: str @ai.flow() async def summarize(input: SummarizeInput) -> str: - response = await ai.generate(prompt=f'Summarize: {input.text}') + if not input.text.strip(): + raise PublicError('INVALID_ARGUMENT', 'text must not be empty') + response = await ai.generate(prompt=f'Summarize in two sentences: {input.text}') return response.text -``` - ---- -## Streaming flows - -```python +# Streaming flow: declare the chunk type on FlowContext[ChunkT] @ai.flow() -async def stream_story(subject: str, ctx: ActionRunContext) -> str: - sr = ai.generate_stream(prompt=f'Story about {subject}.') - full = '' +async def stream_story(subject: str, ctx: FlowContext[str]) -> str: + sr = ai.generate_stream(prompt=f'Write a short story about {subject}.') + chunks: list[str] = [] async for chunk in sr.stream: if chunk.text: ctx.send_chunk(chunk.text) - full += chunk.text - return full + chunks.append(chunk.text) + return ''.join(chunks) ``` +Options after the flow's single input on `.run(...)` are keyword-only: `await stream_story.run('lighthouses', on_chunk=print, context={'role': 'reader'})`. + --- -## Tools +## Tools, Multipart Tool Responses, and Interrupts -Parameters must be a **Pydantic `BaseModel`** (bare scalars → 400 from Gemini). Use **`@ai.tool()`**, not `@ai.define_tool()`. +Use `@ai.tool()` (there is no `define_tool`). A tool takes **at most one input parameter** (scalar or Pydantic `BaseModel`) plus an optional `ctx: ToolContext[ChunkT]`. Use a `BaseModel` whenever a tool needs multiple arguments or field descriptions. ```python -class WeatherInput(BaseModel): - city: str +from pydantic import BaseModel, Field +from genkit import MultipartToolResponse, Part, ToolContext + +class OrderLookup(BaseModel): + order_id: str = Field(description='Order identifier, e.g. ORD-104') + include_receipt_image: bool = False + +class ToolProgress(BaseModel): + stage: str @ai.tool() -async def get_weather(input: WeatherInput) -> str: - return f'Sunny in {input.city}' +async def lookup_order(input: OrderLookup, ctx: ToolContext[ToolProgress]) -> str | MultipartToolResponse: + ctx.send_chunk(ToolProgress(stage='fetching_record')) + if not input.include_receipt_image: + return f'Order {input.order_id}: delivered' + receipt_png = Part.from_media('data:image/png;base64,iVBORw0KGgo=', content_type='image/png') + return MultipartToolResponse( + output={'order_id': input.order_id, 'status': 'delivered'}, + content=[receipt_png], + ) + +res = await ai.generate( + prompt='Check status of ORD-104.', + tools=[lookup_order], + tool_choice='required', # 'auto' | 'required' | 'none' +) +``` + +### Tool Interrupts (Human-in-the-Loop) + +Raise `Interrupt(...)` inside a tool (or register a tool via `ai.define_interrupt(...)`) to pause the generate loop. `Interrupt` subclasses `BaseException`, so `except Exception:` inside a tool will not swallow it. Resume using `intr.restart(...)` or `intr.respond(...)`: + +```python +from genkit import Interrupt + +class RefundInput(BaseModel): + order_id: str + amount_cents: int -response = await ai.generate(prompt='Weather in Paris?', tools=[get_weather]) +@ai.tool() +async def issue_refund(input: RefundInput, ctx: ToolContext) -> dict: + if not ctx.is_resumed(): + raise Interrupt({'order_id': input.order_id, 'amount_cents': input.amount_cents}) + return {'refunded': True, 'approved_by': (ctx.resumed_metadata or {}).get('manager')} + +turn1 = await ai.generate(prompt='Refund $25 (2500 cents) on ORD-104.', tools=[issue_refund]) +if turn1.interrupts: + turn2 = await ai.generate( + messages=turn1.messages, + tools=[issue_refund], + resume_restart=[i.restart(resumed_metadata={'manager': 'alex'}) for i in turn1.interrupts], + ) +``` + +--- + +## Cancellation + +Cancellation uses standard `asyncio.CancelledError`; it reaches model calls, retry backoff, and running tools at any `await`: + +```python +import asyncio + +# 1. Cancel an in-flight generate or flow task +task = asyncio.create_task(ai.generate(prompt='Write a detailed migration plan.')) +task.cancel() + +# 2. Stop a flow stream early — use `async with` (or `await stream.aclose()`) +# so breaking out of the loop cancels the background flow immediately. +async with stream_story.stream('glaciers') as stream: + async for chunk in stream: + if len(chunk) > 0: + break + +# 3. Clean up partial side effects inside `finally` in tools +@ai.tool() +async def hold_inventory(sku: str) -> dict: + reservation = await reserve_sku(sku) + try: + return await commit_reservation(reservation) + finally: + if not reservation.committed: + await release_reservation(reservation) ``` --- ## Embeddings +Build embedder refs with `.embedder(...)` and pass provider options in `config=`: + ```python -from genkit_google_genai import GeminiEmbeddingModels +from genkit import Document +from genkit_google_genai import GoogleAI -embedder = f'googleai/{GeminiEmbeddingModels.GEMINI_EMBEDDING_001}' -embeddings = await ai.embed(embedder=embedder, content='The sky is blue.') -vector = embeddings[0].embedding +embedder = GoogleAI.embedder('gemini-embedding-001') +embeddings = await ai.embed( + embedder=embedder, + content='Sourdough starter hydration ratio.', + config={'task_type': 'RETRIEVAL_DOCUMENT'}, +) +vector: list[float] = embeddings[0].embedding -embeddings = await ai.embed_many( +batch = await ai.embed_many( embedder=embedder, - content=['The sky is blue.', 'Grass is green.'], + content=['First paragraph.', 'Second paragraph.'], ) -``` -Common embedders: `googleai/gemini-embedding-001`, `googleai/gemini-embedding-exp-03-07`. +# Multimodal embedding input via Document +image_embeddings = await ai.embed( + embedder=GoogleAI.embedder('gemini-embedding-2'), + content=Document.from_media('data:image/png;base64,iVBORw0KGgo=', 'image/png'), +) +``` diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/fastapi.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/fastapi.md index a2aafd3e..c89f9ad6 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/fastapi.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/fastapi.md @@ -1,344 +1,344 @@ # FastAPI — Genkit Python +- [Install](#install): Lines 14-29 +- [Serve a Flow or Agent](#serve-a-flow-or-agent): Lines 33-87 +- [HTTP Error Redaction & PublicError](#http-error-redaction--publicerror): Lines 91-97 +- [Streaming (SSE)](#streaming-sse): Lines 101-150 +- [Auth & Context Providers](#auth--context-providers): Lines 154-226 +- [Advanced Flow Composition](#advanced-flow-composition): Lines 230-302 +- [Structured Output Endpoint](#structured-output-endpoint): Lines 306-329 +- [Python HTTP Client](#python-http-client): Lines 333-344 + +--- + ## Install ```bash -uv add genkit-fastapi fastapi uvicorn +uv add 'genkit-fastapi==0.13.*' fastapi uvicorn ``` -Import from `genkit_fastapi`: - -- `serve_agent` / `serve_flow` — mount an agent or flow as a router -- `handle_genkit_request` — escape hatch for a custom `@app.post` that needs its own - `Depends` / auth wiring while still speaking the Genkit wire format +Imports are split between stable flow serving and experimental agent serving: -For agents, see also [Agent HTTP](agents-http.md). +- **Stable flows (`genkit_fastapi`)**: + - `serve_flow` — mount a flow as a FastAPI `APIRouter` (supports `base_path=` and `context_dependency=`) + - `genkit_fastapi_handler` — decorator stacking `@app.post('/...')` directly over `@ai.flow()` (supports `context_provider=`) + - `handle_genkit_request` — low-level handler for custom `@app.post` routes with multiple FastAPI `Depends(...)` parameters +- **Experimental agents (`genkit_fastapi.exp`)**: + - `serve_agent` — mount an experimental `Agent` (`genkit.exp.Genkit`) as an `APIRouter` (see [Agent HTTP](agents-http.md)) +- **Framework-agnostic web utilities (`genkit.web`)**: + - `RequestData`, `ContextProvider`, `StreamHandle`, `to_sse`, `read_body`, `wants_stream`, `result_json`, `error_body`, `error_json`, `error_sse`, `error_status` --- -## Serve an agent or flow +## Serve a Flow or Agent ```python +import os import uvicorn from fastapi import FastAPI -from genkit import Genkit -from genkit.agent import InMemorySessionStore -from genkit_fastapi import serve_agent, serve_flow +from genkit import PublicError +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore +from genkit_fastapi import serve_flow +from genkit_fastapi.exp import serve_agent from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) -agent = ai.define_agent( - name='weatherAgent', - system='Weather assistant. Be concise.', +concierge = ai.define_agent( + name='conciergeAgent', + system='Hotel concierge. Keep answers under 40 words.', store=InMemorySessionStore(), ) + @ai.flow() -async def hello(name: str) -> str: - return f'Hello, {name}' +async def lookup_reservation(code: str) -> dict: + if not code.startswith('RES-'): + raise PublicError('NOT_FOUND', f'Reservation {code} not found') # HTTP 404 + return {'code': code, 'status': 'confirmed'} + app = FastAPI() -# Routes use the action name as defined: agent `weatherAgent` → /api/weatherAgent; -# flow `hello` → /api/hello. Override with base_path=... if needed. -app.include_router(serve_agent(agent), prefix='/api') -app.include_router(serve_flow(hello), prefix='/api') +# Paths default to the agent name and the flow's Python function name (override via base_path=...): +# POST /api/conciergeAgent (+ /getSnapshot and /abort, since the agent has a store) +# POST /api/lookup_reservation +app.include_router(serve_agent(concierge), prefix='/api') +app.include_router(serve_flow(lookup_reservation), prefix='/api') + -# Under genkit start, drive uvicorn from ai.run_main (don't nest uvicorn.run inside it). async def main() -> None: - import os config = uvicorn.Config(app, host='0.0.0.0', port=int(os.environ.get('PORT', '8080'))) await uvicorn.Server(config).serve() + if __name__ == '__main__': ai.run_main(main()) ``` +If your app only serves flows (no agents), import `Genkit` from `genkit` instead of `genkit.exp`. + +With `GENKIT_ENV=dev` (which `genkit start` sets automatically), the Dev UI reflection server starts on a background thread, so either entrypoint works: + +```bash +genkit start -- uv run uvicorn main:app --reload # reload-friendly ASGI dev server + Dev UI +genkit start -- uv run src/main.py # driven via ai.run_main(main()) + Dev UI +uvicorn main:app # production (no Dev UI) +``` + +--- + +## HTTP Error Redaction & `PublicError` + +Over HTTP (`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, `handle_genkit_request`), Genkit separates internal errors from caller-visible errors: + +- **`PublicError(status, message, details)`**: returns the corresponding HTTP status (`NOT_FOUND` -> 404, `INVALID_ARGUMENT` -> 400, `UNAUTHENTICATED` -> 401, `PERMISSION_DENIED` -> 403) with `status`, `message`, and `details`. Keep secrets and cross-tenant data out of `PublicError`. +- **Invalid flow input**: returns HTTP 400 with field names and Pydantic's built-in messages (never echoing submitted values; custom validator text shows as `Invalid value` unless the validator raises `PublicError`). +- **Any other `GenkitError`, provider error, or Python exception**: logged server-side with full traceback (`5xx`) and returned to the client as `500 {"status": "INTERNAL", "message": "Internal Error"}`. + --- -## Streaming +## Streaming (SSE) -`serve_agent`, `serve_flow`, and `handle_genkit_request` all stream when the client -sends `Accept: text/event-stream` (same wire path). Otherwise they return a one-shot -`{"result": ...}` JSON body. +`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, and `handle_genkit_request` stream Server-Sent Events when the client sends `Accept: text/event-stream` (or `?stream=true`), and automatically cancel the running flow if the client disconnects. Otherwise they return `{"result": ...}`. **Wire format (SSE):** +```text +data: {"message": "<chunk>"} +data: {"message": "<chunk>"} +data: {"result": "<final output>"} ``` -data: {"message": "<chunk text>"} ← one per ctx.send_chunk() call -data: {"message": "<chunk text>"} -data: {"result": <final output>} ← sent once when the action completes -``` +Errors during streaming emit `data: {"error": {...}}` formatted with the same `PublicError` redaction rules. + +**Minimal streaming FastAPI flow (`FlowContext`):** + +```python +from pydantic import BaseModel +from fastapi import FastAPI +from genkit import FlowContext, Genkit +from genkit_fastapi import serve_flow +from genkit_google_genai import GoogleAI + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) + + +class BriefInput(BaseModel): + topic: str + -**Frontend:** -```js -const res = await fetch('/api/chat', { - method: 'POST', - headers: { 'Content-Type': 'application/json', 'Accept': 'text/event-stream' }, - body: JSON.stringify({ data: { topic: 'quantum computing' } }), -}); -const reader = res.body.getReader(); -// decode and parse each `data: {...}` line +@ai.flow() +async def stream_brief(input: BriefInput, ctx: FlowContext[str]) -> str: + sr = ai.generate_stream(prompt=f'Write a concise brief on {input.topic}.') + parts: list[str] = [] + async for chunk in sr.stream: + if chunk.text: + ctx.send_chunk(chunk.text) # each call emits one SSE frame + parts.append(chunk.text) + return ''.join(parts) + + +app = FastAPI() +app.include_router(serve_flow(stream_brief), prefix='/api') # POST /api/stream_brief ``` -**curl test:** +**Test with `curl`:** ```bash -curl -N -X POST http://localhost:8080/api/chat \ +curl -N -X POST http://localhost:8080/api/stream_brief \ -H 'Content-Type: application/json' \ -H 'Accept: text/event-stream' \ - -d '{"data": {"topic": "quantum computing"}}' + -d '{"data": {"topic": "zero-downtime Postgres migrations"}}' ``` --- -## Minimal streaming FastAPI app +## Auth & Context Providers + +### 1. FastAPI `Depends` via `serve_flow(..., context_dependency=...)` ```python -import uvicorn -from pydantic import BaseModel -from fastapi import FastAPI -from genkit import Genkit, ActionRunContext +from fastapi import Header +from genkit import FlowContext, PublicError from genkit_fastapi import serve_flow -from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +async def require_auth(authorization: str | None = Header(default=None)) -> dict: + if not authorization: + raise PublicError('UNAUTHENTICATED', 'Missing Authorization header') # 401 + return {'auth': {'token': authorization}} -class ChatInput(BaseModel): - topic: str + +@ai.flow() +async def account_summary(account_id: str, ctx: FlowContext) -> dict: + caller = (ctx.context or {}).get('auth') + return {'account_id': account_id, 'caller': caller} +app.include_router( + serve_flow(account_summary, context_dependency=require_auth), + prefix='/api', +) +``` + +### 2. Decorator Style (`@genkit_fastapi_handler`) with `genkit.web.RequestData` + +```python +from genkit import PublicError +from genkit.web import RequestData +from genkit_fastapi import genkit_fastapi_handler + + +async def auth_provider(req: RequestData) -> dict: + token = req.headers.get('authorization') + if not token: + raise PublicError('UNAUTHENTICATED', 'Missing bearer token') # 401 + return {'user_token': token} + + +@app.post('/api/ask') +@genkit_fastapi_handler(ai, context_provider=auth_provider) @ai.flow() -async def chat(input: ChatInput, ctx: ActionRunContext) -> str: - sr = ai.generate_stream(prompt=f'Tell me about {input.topic}.') - full = '' - async for chunk in sr.stream: - if chunk.text: - ctx.send_chunk(chunk.text) # each chunk → SSE event on the wire - full += chunk.text - return full +async def ask(question: str) -> str: + res = await ai.generate(prompt=question) + return res.text +``` +### 3. Custom Route Escape Hatch (`handle_genkit_request` or `genkit.web`) -app = FastAPI() -app.include_router(serve_flow(chat), prefix='/api') # POST /api/chat +When a route needs multiple `Depends(...)` parameters while keeping the Genkit wire format: -async def main() -> None: - import os - config = uvicorn.Config(app, host='0.0.0.0', port=int(os.environ.get('PORT', '8080'))) - await uvicorn.Server(config).serve() +```python +from fastapi import Depends, Request +from genkit_fastapi import handle_genkit_request -if __name__ == '__main__': - ai.run_main(main()) + +@app.post('/api/custom-brief') +async def custom_brief( + request: Request, + auth_ctx: dict = Depends(require_auth), +): + return await handle_genkit_request( + request, + action=stream_brief, + context=auth_ctx, + ) ``` -**Key:** the flow must accept `ctx: ActionRunContext` and call `ctx.send_chunk(text)` -to emit SSE chunks. Without `ctx.send_chunk`, the flow runs but streams nothing — the -client waits for the final result. +If writing a completely custom non-Genkit JSON wrapper around `await my_flow.run(data)`, map exceptions with `error_body(e)` and `error_status(e)` from `genkit.web` so internal errors stay redacted. --- -## Advanced use cases +## Advanced Flow Composition -### Nested flow streaming +### Nested Flow Streaming -Chain flows so a child's chunks surface on the parent's HTTP stream. Call the -child with `.run(..., on_chunk=ctx.send_chunk)` — do **not** pass `ctx` as a -second positional argument to `await child(input, ctx)` (that raises `TypeError`). +Forward a child flow's chunks to the parent's SSE stream via `await child.run(input, on_chunk=ctx.send_chunk)`. All arguments after the input on `.run(...)` are keyword-only: ```python -class ResearchInput(BaseModel): +class SectionInput(BaseModel): topic: str + @ai.flow() -async def research(input: ResearchInput, ctx: ActionRunContext) -> str: - sr = ai.generate_stream(prompt=f'Explain {input.topic} in depth.') - full = '' +async def draft_body(input: SectionInput, ctx: FlowContext[str]) -> str: + sr = ai.generate_stream(prompt=f'Explain {input.topic} with concrete examples.') + chunks: list[str] = [] async for chunk in sr.stream: if chunk.text: ctx.send_chunk(chunk.text) - full += chunk.text - return full - + chunks.append(chunk.text) + return ''.join(chunks) -class HeadlineInput(BaseModel): - text: str @ai.flow() -async def make_headline(input: HeadlineInput) -> str: - response = await ai.generate(prompt=f'One-line headline for: {input.text}') - return response.text.strip() +async def draft_headline(input: SectionInput) -> str: + res = await ai.generate(prompt=f'One-line title for: {input.topic}') + return res.text.strip() -class ReportInput(BaseModel): - topic: str - @ai.flow() -async def report(input: ReportInput, ctx: ActionRunContext) -> str: - headline = await make_headline(HeadlineInput(text=input.topic)) +async def full_report(input: SectionInput, ctx: FlowContext[str]) -> str: + headline = await draft_headline(input) ctx.send_chunk(f'# {headline}\n\n') - - body = ( - await research.run( - ResearchInput(topic=input.topic), - on_chunk=ctx.send_chunk, - ) - ).response - + body = (await draft_body.run(input, on_chunk=ctx.send_chunk)).response return f'# {headline}\n\n{body}' - - -app.include_router(serve_flow(report), prefix='/api') # POST /api/report ``` -**Rules:** -- Streaming children accept `ctx: ActionRunContext` and call `ctx.send_chunk` -- Parents forward chunks with `child.run(input, on_chunk=ctx.send_chunk)` -- Non-streaming children: `await child(input)` is fine - -### Executing flows in parallel - -Use `asyncio.gather` to run multiple flows concurrently. Only makes sense when children don't need to stream. +### Parallel Flows (`asyncio.gather`) ```python import asyncio -class AnalysisInput(BaseModel): - text: str -class CheckResult(BaseModel): - issues: list[str] +class CodeReviewInput(BaseModel): + diff: str -class CombinedAnalysis(BaseModel): - issues: list[str] -@ai.flow() -async def check_security(input: AnalysisInput) -> CheckResult: - r = await ai.generate( - prompt=f'List security concerns as a short comma-separated line (or "none"): {input.text[:2000]}', - ) - raw = (r.text or '').strip() - issues = [s.strip() for s in raw.split(',') if s.strip() and s.strip().lower() != 'none'] - return CheckResult(issues=issues) +class ReviewFindings(BaseModel): + findings: list[str] -@ai.flow() -async def check_bugs(input: AnalysisInput) -> CheckResult: - r = await ai.generate( - prompt=f'List likely bugs or correctness issues as a short comma-separated line (or "none"): {input.text[:2000]}', - ) - raw = (r.text or '').strip() - issues = [s.strip() for s in raw.split(',') if s.strip() and s.strip().lower() != 'none'] - return CheckResult(issues=issues) @ai.flow() -async def check_style(input: AnalysisInput) -> CheckResult: - r = await ai.generate( - prompt=f'List style or clarity issues as a short comma-separated line (or "none"): {input.text[:2000]}', +async def check_security(input: CodeReviewInput) -> ReviewFindings: + res = await ai.generate( + prompt=f'List security issues in this diff: {input.diff[:2000]}', + output_schema=ReviewFindings, ) - raw = (r.text or '').strip() - issues = [s.strip() for s in raw.split(',') if s.strip() and s.strip().lower() != 'none'] - return CheckResult(issues=issues) + return res.output or ReviewFindings(findings=[]) + @ai.flow() -async def analyze(input: AnalysisInput) -> CombinedAnalysis: - security, bugs, style = await asyncio.gather( - check_security(input), - check_bugs(input), - check_style(input), +async def check_correctness(input: CodeReviewInput) -> ReviewFindings: + res = await ai.generate( + prompt=f'List logic bugs in this diff: {input.diff[:2000]}', + output_schema=ReviewFindings, ) - return CombinedAnalysis(issues=security.issues + bugs.issues + style.issues) + return res.output or ReviewFindings(findings=[]) -app.include_router(serve_flow(analyze), prefix='/api') # POST /api/analyze +@ai.flow() +async def review_diff(input: CodeReviewInput) -> ReviewFindings: + sec, bugs = await asyncio.gather(check_security(input), check_correctness(input)) + return ReviewFindings(findings=sec.findings + bugs.findings) ``` --- -## Structured output endpoint (non-streaming) +## Structured Output Endpoint + +Check `response.output is None` before returning so truncated model replies surface cleanly instead of failing output validation: ```python +from genkit import PublicError + + class SentimentResult(BaseModel): - sentiment: str # positive / negative / neutral - confidence: float # 0.0–1.0 + sentiment: str + confidence: float key_phrases: list[str] + @ai.flow() -async def sentiment(input: AnalysisInput) -> SentimentResult: +async def analyze_sentiment(text: str) -> SentimentResult: response = await ai.generate( - prompt=f'Analyze sentiment: {input.text}', - output_format='json', + prompt=f'Analyze customer sentiment: {text}', output_schema=SentimentResult, ) + if response.output is None: + raise RuntimeError(f'Model returned incomplete JSON: {response.error}') return response.output - - -app.include_router(serve_flow(sentiment), prefix='/api') # POST /api/sentiment ``` -Client calls this without `Accept: text/event-stream` — gets `{"result": {...}}` back. - --- -## Custom route with `handle_genkit_request` +## Python HTTP Client -When you need your own FastAPI dependencies (auth, tenancy, etc.) and still want -the Genkit wire format, call `handle_genkit_request` from a hand-written route: +Genkit HTTP endpoints expect `{"data": ...}` on input (`{"data": null}` uses the flow parameter's default) and return `{"result": ...}`: ```python -from fastapi import Depends, Request -from genkit_fastapi import handle_genkit_request +import httpx - -async def current_user(request: Request) -> dict: - # Resolve auth however you like. - return {'uid': request.headers.get('x-user-id', 'anon')} - - -@app.post('/api/secure-chat') -async def secure_chat( - request: Request, - user: dict = Depends(current_user), -): - return await handle_genkit_request( - request, - action=chat, - context={'auth': user}, - ) +async with httpx.AsyncClient(base_url='http://127.0.0.1:8080') as client: + resp = await client.post('/api/analyze_sentiment', json={'data': 'Fast shipping, great packaging.'}) + resp.raise_for_status() + result = resp.json()['result'] ``` - -Prefer `serve_flow` / `serve_agent` with `context_dependency=` when a single -`Depends` is enough — use `handle_genkit_request` when the route shape itself -needs to be custom. - ---- - -## Run with Dev UI - -```bash -GEMINI_API_KEY=your-key genkit start -- uv run src/main.py -``` - -Leave the process running until the CLI prints something like: - -``` -Genkit Developer UI: http://localhost:4000 -``` - -Open that URL. Port may differ if 4000 is busy. - - -## Python HTTP client (mounted flows) - -```python -import httpx, asyncio - -async with httpx.AsyncClient(base_url='http://127.0.0.1:18524') as client: - async def call(path, data): - r = await client.post(path, json={'data': data}) - r.raise_for_status() - return r.json()['result'] - a, b = await asyncio.gather( - call('/api/summarize', {'text': '...'}), - call('/api/sentiment', {'text': '...'}), - ) -``` - -Wire shape is `{"data": ...}` in, `{"result": ...}` out (not raw flow args). diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/migrating-from-0.12.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/migrating-from-0.12.md new file mode 100644 index 00000000..d56e3969 --- /dev/null +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/migrating-from-0.12.md @@ -0,0 +1,467 @@ +# Migrating from Genkit Python 0.12 to 0.13 + +Genkit Python 0.13 aligns every symbol, signature, and error contract ahead of 1.0 without deprecation aliases. Use this reference when upgrading a 0.12 codebase or replacing 0.12 patterns (`ActionRunContext`, `ToolRunContext`, `*_model()`, `GeminiConfigSchema`, `options=`, `abort_signal`, `restart_tool`, `RuntimeErrorReason`, `add_gcp_telemetry`, `MediaPart`, `TextPart`, or `from genkit.exp import Agent`). + +- [Agent Guardrails Before Editing](#agent-guardrails-before-editing): Lines 20-27 +- [1. Upgrade Packages Together](#1-upgrade-packages-together): Lines 31-44 +- [2. Single-Home Namespace Layout](#2-single-home-namespace-layout): Lines 48-107 +- [3. Removed Symbols & Enums Replaced by Strings](#3-removed-symbols--enums-replaced-by-strings): Lines 111-134 +- [4. Model & Embedder Helpers, Constructors, and Plugins](#4-model--embedder-helpers-constructors-and-plugins): Lines 138-194 +- [5. Config Classes, extra, tool_choice, and API Keys](#5-config-classes-extra-tool_choice-and-api-keys): Lines 198-243 +- [6. Typed Contexts, Multipart Tools, Interrupts, and Cancellation](#6-typed-contexts-multipart-tools-interrupts-and-cancellation): Lines 247-323 +- [7. Errors, HTTP Redaction (PublicError), and genkit.web](#7-errors-http-redaction-publicerror-and-genkitweb): Lines 327-366 +- [8. Prompts, Structured Output, Validation, and Telemetry](#8-prompts-structured-output-validation-and-telemetry): Lines 370-391 +- [9. Plugin & Custom Middleware Authors](#9-plugin--custom-middleware-authors): Lines 395-429 +- [10. Evaluators (0.13 Only - Removed in 1.0)](#10-evaluators-013-only--removed-in-10): Lines 433-458 +- [11. Verification Checklist](#11-verification-checklist): Lines 462-467 + +--- + +## Agent Guardrails Before Editing + +1. **Stop before porting evaluators.** Evaluators (`define_evaluator`, `ai.evaluate`, `genkit.evaluator`, `genkit-evaluators`) still run in 0.13, but this is their last release: **Genkit 1.0 cuts them from the 1.0 release** (evaluation will return as an additive feature in a later release; `genkit-evaluators` 0.13 requires `genkit<1`). If the project contains evaluator code, stop and ask the developer before porting it. Recommend deleting evaluator code now; apply the 0.13 renames below only if the developer chooses to stay pinned to `0.13.*`. +2. **Audit HTTP-served error paths.** Over HTTP (`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, `handle_genkit_request`, `genkit_flask_handler`, `genkit_django_handler`, `genkit.web`), `GenkitError` and unhandled exceptions now return `500 {"status": "INTERNAL", "message": "Internal Error"}`. Convert errors meant for callers (404, 401/403, domain validation) to `PublicError(status, message, details)`. Ask the developer before exposing any error message whose contents might include internal data or tenant state. +3. **Audit silent behavior changes.** Type checkers catch symbol renames and signatures, not runtime behavior changes: + - Search for `{{` inside `ai.generate(prompt=..., system=...)` — strings there are now sent literally instead of rendered as Handlebars templates. + - Search for reads of `response.output` — truncated or malformed JSON now returns `None` and sets `response.error` (`Failure` with `ErrorReason.INVALID_OUTPUT`). + - Check `try` / `except GenkitError` blocks around `await my_flow(x)` or `await my_flow.run(x)` — actions now raise body exceptions unwrapped. + +--- + +## 1. Upgrade Packages Together + +Keep `genkit` and every `genkit-*` package on the same `0.13.*` minor line: + +```bash +uv add 'genkit==0.13.*' 'genkit-google-genai==0.13.*' # include every genkit-* package in use +``` + +- **Vertex AI Model Garden extras**: install extras for third-party SDKs: + ```bash + uv add 'genkit-vertexai[anthropic]' # Claude on Model Garden + uv add 'genkit-vertexai[openai]' # Llama, Mistral, and OpenAI-compatible models + ``` +- **Flask**: `genkit-flask` no longer pulls in `genkit-google-genai` transitively. Run `uv add 'genkit-google-genai==0.13.*'` if the app uses Gemini. + +--- + +## 2. Single-Home Namespace Layout + +Every public symbol lives in one module listed in its `__all__`. Plugin submodules (such as `genkit_openai.models` or `genkit_vertexai.model_garden`) are private; import from each plugin's top-level package (`genkit_google_genai`, `genkit_vertexai`, `genkit_openai`, etc.). + +- **`genkit`**: application code (`Genkit`, `Message`, `Role`, `Part`, `Media`, `Document`, `ToolChoice`, `ModelResponse`, `ModelResponseChunk`, `ModelStreamResponse`, `StreamResponse`, `FinishReason`, `Failure`, `ErrorReason`, `GenkitError`, `PublicError`, `Flow`, `FlowContext`, `Tool`, `ToolContext`, `Interrupt`, `MultipartToolResponse`, `Prompt`, `Embedding`, `Operation`, `GenkitLogger`, `get_logger`). +- **`genkit.model`**: custom models and model inspection (`Model`, `ModelRef`, `BackgroundModel`, `ModelContext`, `ModelRequest`, `ModelInfo`, `ModelSupports`, `ModelStage`, `GenerationUsage`, `ConstrainedSupport`, `ModelConfig`). +- **`genkit.embedder`**: custom embedders (`Embedder`, `EmbedderRef`, `EmbedRequest`, `EmbedResponse`, `EmbedderInfo`, `EmbedderSupports`). +- **`genkit.evaluator`**: evaluator types (`Evaluator`, `BatchEvaluator`, `BaseDataPoint`, `EvaluatorResponse`, `Score`, `ScoreDetails`, `ScoreStatus`). +- **`genkit.middleware`**: writing custom middleware (`Middleware`, `MiddlewareContext`, `GenerateHookParams`, `ModelHookParams`, `ToolHookParams`). Note: the `Middleware()` plugin that registers `Retry`, `Fallback`, `ToolApproval`, `Skills`, and `Filesystem` lives in `genkit_middleware`. +- **`genkit.formats`**: custom output formatters (`Formatter`, `FormatterConfig`, `FormatHandler`). +- **`genkit.plugin_api`**: writing plugins (`Plugin`, `Action`, `ActionKind`, `GenkitConfig`, `provider_error`, `loop_local_client`, `GENKIT_USER_AGENT`). +- **`genkit.web`**: HTTP serving helpers (`RequestData`, `ContextProvider`, `to_sse`, `read_body`, `wants_stream`, `result_json`, `error_body`, `error_json`, `error_sse`, `error_status`). +- **`genkit.telemetry`**: custom tracing (`Instrumentation`, `SpanContext`, `configure_instrumentation`). +- **`genkit.testing`**: test doubles (`EchoModel`, `ScriptedModel`, `StaticResponseModel`, `define_echo_model`, `define_scripted_model`, `define_static_response_model`). +- **`genkit.exp` & `genkit.exp.agent`**: experimental agent APIs (`from genkit.exp import Genkit, current_session`; `from genkit.exp.agent import Agent, Session, InMemorySessionStore, FileSessionStore, remote_agent`). + +Look up any moved symbol directly from the installed environment: + +```bash +python - <<'EOF' +import importlib +target = 'ModelSupports' +for mod in [ + 'genkit', 'genkit.model', 'genkit.embedder', 'genkit.evaluator', + 'genkit.middleware', 'genkit.formats', 'genkit.plugin_api', 'genkit.web', + 'genkit.telemetry', 'genkit.testing', 'genkit.exp', 'genkit.exp.agent', +]: + if target in importlib.import_module(mod).__all__: + print(mod) +EOF +``` + +### Renamed While Moving + +```python +# Old (0.12) +from genkit import Constrained, ExecutablePrompt, ModelUsage, RuntimeErrorReason, Stage, Supports + +# New (0.13) +from genkit import ErrorReason, Prompt +from genkit.model import ConstrainedSupport, GenerationUsage, ModelStage, ModelSupports +``` + +### Experimental (`exp`) Moves + +```python +# Old (0.12) +from genkit.exp import Agent, AgentError, InMemorySessionStore, Session +from genkit_fastapi import serve_agent +from genkit_middleware import Artifacts + +# New (0.13) +from genkit.exp import Genkit, current_session +from genkit.exp.agent import Agent, InMemorySessionStore, Session +from genkit_fastapi.exp import serve_agent +from genkit_middleware.exp import Artifacts +from genkit_google_cloud.exp import FirestoreSessionStore +``` + +`define_agent`, `define_custom_agent`, `define_prompt_agent`, and `@ai.agent()` exist on `genkit.exp.Genkit`, not `genkit.Genkit`. `AgentError` is removed: a failed turn sets `res.error` (`Failure`), and a call that produces no turn raises `GenkitError`. Tools in a detached turn are cancelled with `await task.abort()` instead of polling `abort_signal`. + +--- + +## 3. Removed Symbols & Enums Replaced by Strings + +0.12 model and config enums (`GoogleAIGeminiVersion`, `KnownGemini*`, `KnownGpt`, `KnownClaude`, `*Version`, `GeminiEmbeddingModels`, `VertexEmbeddingModels`, `EmbeddingTaskType`, `VertexAIEvaluationMetricType`) are removed. Pass plain string values; helper signatures use `Literal` unions for static checking while still accepting new model IDs as strings. + +```python +# Old (0.12) +from genkit_google_genai import GoogleAI, GoogleAIGeminiVersion + +model = GoogleAI.gemini_model(GoogleAIGeminiVersion.GEMINI_3_FLASH_PREVIEW) + +# New (0.13) +from genkit_google_genai import GeminiConfig, GoogleAI + +model = GoogleAI.gemini('gemini-3-flash-preview') +cfg = GeminiConfig(thinking_level='HIGH') # checked against 'MINIMAL' | 'LOW' | 'MEDIUM' | 'HIGH' +``` + +Other removed items and their 0.13 replacements: +- **`MediaPart` and `TextPart` (`Part(root=...)`)**: removed (`Part(root=...)` raises `ValueError`). Construct parts with `Part.from_text('...')` (or `Part(text='...')`) and `Part.from_media(url, content_type='image/jpeg')` (or `Part(media=Media(url=..., content_type=...))`). Read via `part.text` and `part.media`. +- **Name helpers** (`bedrock_name`, `anthropic_name`, `ollama_name`, `openai_model`, `genkit_eval_name`, `package_name`): pass `'plugin/model'` strings (e.g. `'openai/gpt-4o'`) or typed model refs (`OpenAI.gpt('gpt-4o')`). +- **`GoogleAI.imagen_model(...)`**: Imagen catalog is retired; use `GoogleAI.gemini_image('gemini-2.5-flash-image')`. +- **`Candidate`**: read `response.message` directly. +- **`Document.data`, `Document.data_type`, `Document.from_data()`**: construct with `Document(content=[...], metadata=...)`, `Document.from_text(...)`, or `Document.from_media(...)`. +- **`ai.registry`**: use lookup helpers directly on `Genkit` (e.g. `ai.prompt(...)`). + +--- + +## 4. Model & Embedder Helpers, Constructors, and Plugins + +Model helpers drop `_model`, and `.embedding()` becomes `.embedder()`: + +```python +# Old (0.12) +GoogleAI.gemini_model('gemini-flash-latest') +GoogleAI.gemini_image_model('gemini-2.5-flash-image') +GoogleAI.gemini_tts_model('gemini-2.5-flash-preview-tts') +GoogleAI.embedding('gemini-embedding-001') +VertexAI.veo_model('veo-3.1-generate-001') +Anthropic.claude_model('claude-sonnet-4-6') + +# New (0.13) +GoogleAI.gemini('gemini-flash-latest') +GoogleAI.gemini_image('gemini-2.5-flash-image') +GoogleAI.gemini_tts('gemini-2.5-flash-preview-tts') +GoogleAI.embedder('gemini-embedding-001') +VertexAI.veo('veo-3.1-generate-001') +Anthropic.claude('claude-sonnet-4-6') +OpenAI.gpt('gpt-5.2') +``` + +### Keyword-Only Signatures + +`Genkit(...)`, plugin constructors, `GenkitError(...)`, and options after an action's single input on `.run(...)` are keyword-only: + +```python +# Old (0.12) +ai = Genkit([GoogleAI()], GoogleAI.gemini_model('gemini-flash-latest')) +out = await my_flow.run(data, on_chunk, ctx) + +# New (0.13) +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) +out = await my_flow.run(data, on_chunk=on_chunk, context=ctx) +``` + +### Provider Plugin Updates + +```python +# VertexAI (in genkit_google_genai): project_id= -> project= +from genkit_google_genai import VertexAI +VertexAI(project='my-project', location='us-central1') + +# Vertex AI Model Garden (in genkit_vertexai): ModelGardenPlugin -> ModelGarden +from genkit_vertexai import ModelGarden +ModelGarden(project='my-project', location='us-central1') + +# Ollama: server_address= -> base_url=; pulled models resolve automatically (no ModelDefinition/EmbeddingDefinition) +import os +from genkit_ollama import Ollama +Ollama(base_url=os.getenv('OLLAMA_HOST')) # local file paths for media are no longer read + +# Amazon Bedrock: ModelDefinition removed; BedrockImageConfig -> StabilityImageConfig +from genkit_amazon_bedrock import Bedrock, StabilityImageConfig +Bedrock(region='us-east-1', models=['us.amazon.nova-lite-v1:0']) +``` + +--- + +## 5. Config Classes, `extra`, `tool_choice`, and API Keys + +### Flat Config Classes + +`*ConfigSchema` classes drop the `Schema` suffix and flatten nested provider dicts. Unknown or nested keys (such as `thinking_config` or a typo like `'temprature'`) raise `GenkitError` (`INVALID_ARGUMENT`) before calling the model. + +```python +# Old (0.12) +from genkit_google_genai import GeminiConfigSchema, GeminiImageConfigSchema, GeminiTtsConfigSchema, GemmaConfigSchema +config = {'thinking_config': {'thinking_level': 'HIGH', 'include_thoughts': True}} +config = {'google_search_retrieval': True} +config = {'speech_config': {'voice_config': {'prebuilt_voice_config': {'voice_name': 'Kore'}}}} + +# New (0.13) +from genkit_google_genai import GeminiConfig, GeminiImageConfig, GeminiTtsConfig, GemmaConfig +cfg1 = GeminiConfig(thinking_level='HIGH', include_thoughts=True, google_search=True) +cfg2 = GeminiTtsConfig(voice='Kore') +``` + +```python +# Anthropic flat config (ThinkingConfig, OutputConfig, TaskBudget, RequestMetadata, *ToolChoice removed) +from genkit_anthropic import AnthropicConfig +cfg = AnthropicConfig(thinking='enabled', thinking_budget=1024) +cfg_adaptive = AnthropicConfig(thinking='adaptive', effort='high', task_budget=20000, user_id='diner-42') +``` + +- **Unmodeled provider fields (`extra`)**: pass provider-specific keys not declared on the config class in `extra={...}` (supported across Gemini, Veo, Interactions, OpenAI, Anthropic, Ollama, and Bedrock). `OpenAIConfig.user` is removed; use `extra={'user': '...'}` if needed. + ```python + from genkit_amazon_bedrock import BedrockConfig + cfg = BedrockConfig(extra={'some_provider_option': 1}) + ``` +- **Config layering**: defaults merge key-by-key (`ModelRef` < `Prompt` < call). Pass `None` for a field to clear an inherited key. +- **`tool_choice` on `generate`**: moved out of `config` onto `ai.generate(...)`: + ```python + # Old (0.12): config={'tool_choice': {'type': 'any'}} + await ai.generate(prompt='Check weather in Tokyo.', tools=[get_weather], tool_choice='required') # 'auto' | 'required' | 'none' + ``` +- **`options=` -> `config=` on `embed` and `evaluate`**: + ```python + await ai.embed(embedder=GoogleAI.embedder('gemini-embedding-001'), content='doc', config={'task_type': 'RETRIEVAL_DOCUMENT'}) + ``` +- **API keys rejected in `config`**: passing `api_key` in `config` now raises `GenkitError` (`INVALID_ARGUMENT`) in every plugin. Set the key on the plugin or pass per-request keys in `context`: + ```python + await ai.generate(prompt='Hello', context={'secrets': {'api_key': user_key}}) + ``` + Note: `ModelGarden` uses ADC only and rejects per-request keys; `ai.embed()` has no `context` parameter and always uses the plugin's key. + +--- + +## 6. Typed Contexts, Multipart Tools, Interrupts, and Cancellation + +### Typed Contexts (`FlowContext`, `ToolContext`, `ModelContext`) + +`ActionRunContext` and `ToolRunContext` are removed from `genkit`. Chunk types move from `@ai.flow(chunk_type=...)` into the generic parameter on `FlowContext` / `ToolContext`. Every action accepts at most one input parameter plus an optional typed context; Genkit finds the context by its type annotation. The new contexts subclass `ActionRunContext`, so an old annotation still type-checks once the import is fixed. + +```python +from pydantic import BaseModel +from genkit import FlowContext, ModelResponse, MultipartToolResponse, Part, ToolContext +from genkit.model import ModelContext, ModelRequest + +class Status(BaseModel): + step: str + +# Flow streaming typed chunks +@ai.flow() +async def plan_trip(city: str, ctx: FlowContext[Status]) -> str: + ctx.send_chunk(Status(step='searching flights')) + return f'Planned {city}' + +# Tool streaming typed chunks + returning multipart content +@ai.tool() +async def capture_chart(metric: str, ctx: ToolContext[Status]) -> MultipartToolResponse: + ctx.send_chunk(Status(step='rendering')) + png = Part.from_media('data:image/png;base64,iVBORw0KGgo=', content_type='image/png') + return MultipartToolResponse(output={'ok': True, 'metric': metric}, content=[png]) + +# Custom model implementation +async def my_model_fn(request: ModelRequest, ctx: ModelContext) -> ModelResponse: + ... +``` + +### Multipart Tool Responses + +`from genkit import response` is removed. Replace `response({'ok': True}, parts=[png])` with `MultipartToolResponse(output={'ok': True}, content=[png])` (see `capture_chart` above). + +### Interrupts + +- `restart_tool` and `respond_to_interrupt` are replaced by methods on the interrupt `Part`: + ```python + # Old (0.12): restart_tool(interrupt=intr, resumed_metadata={'approved': True}) + # Old (0.12): respond_to_interrupt({'status': 'declined'}, interrupt=intr) + + # New (0.13) + restart = intr.restart(resumed_metadata={'approved': True}) + reply = intr.respond({'status': 'declined'}) + ``` +- `Interrupt` now subclasses `BaseException` (like `asyncio.CancelledError`) so a tool's `except Exception:` cannot swallow an approval interrupt. Catch `Interrupt` explicitly only if inspecting and re-raising it. + +### Native `asyncio` Cancellation + +`ctx.abort_signal` is removed. Delete `if ctx.abort_signal.is_set(): raise ...` checks; cancellation uses standard `asyncio.CancelledError`, propagating into model requests, retry sleeps, and running tools at any `await`. If a tool catches `asyncio.CancelledError`, it must re-raise it. + +```python +import asyncio + +# Cancel an in-flight call +task = asyncio.create_task(ai.generate(prompt='Draft a long report.')) +task.cancel() + +# Stop a flow stream early — MUST use `async with` (or `await stream.aclose()`), +# otherwise a bare `break` leaves the background flow running. +async with plan_trip.stream('Kyoto') as stream: + async for chunk in stream: + if chunk.step == 'searching flights': + break # cancels plan_trip; stream.response raises CANCELLED + +# Put partial-work cleanup in `finally` inside tools +@ai.tool() +async def reserve_table(party_size: int) -> dict: + hold_id = await acquire_hold(party_size) + try: + return await confirm_hold(hold_id) + finally: + if not await is_confirmed(hold_id): + await release_hold(hold_id) +``` + +--- + +## 7. Errors, HTTP Redaction (`PublicError`), and `genkit.web` + +### Returned vs Raised Errors + +- **Returned errors (`Failure`)**: `res.error`, `operation.error`, and `agent_res.error` are typed `Failure` (`from genkit import ErrorReason, Failure`) with `.status`, `.message`, `.details`, and `.reason`. `RuntimeErrorReason` is renamed to `ErrorReason`. +- **Raised errors (`GenkitError`)**: `e.original_message`, `e.http_code`, `e.trace_id`, and `e.cause` are removed. Read `e.message`, `e.status`, `e.details`, `e.reason`, and `e.__cause__`. +- **Plugin exception classes removed**: classes like `OllamaConnectionError` are gone; inspect `e.status` (e.g. `if e.status == 'UNAVAILABLE':`). +- **Action exceptions are unwrapped**: `await my_flow.run(x)` raises the exception the body raised. Replace `except GenkitError` wrappers that were catching your own exception types with those types directly. `Retry` and `Fallback` only retry failures a plugin classified (such as `UNAVAILABLE`). + +### HTTP Serving & `PublicError` + +In-process calls (`await my_flow(x)`, `await my_flow.run(x)`) see full unwrapped exceptions. Over HTTP, Genkit redacts internal failures: +- **`PublicError(status, message, details)`**: returns the matching HTTP status (e.g. `NOT_FOUND` -> 404, `UNAUTHENTICATED` -> 401, `PERMISSION_DENIED` -> 403) with `status`, `message`, and `details`. +- **Invalid top-level flow input**: returns 400 with field names and Pydantic's built-in messages (custom validator strings are replaced with `Invalid value` unless the validator raises `PublicError`). +- **All other `GenkitError`s, provider errors, and Python exceptions**: logged server-side with traceback and returned to the client as `500 {"status": "INTERNAL", "message": "Internal Error"}`. +- **Streaming**: the same body arrives inside a `data: {"error": {...}}` SSE frame. + +Everything in a `PublicError`'s `message` and `details` reaches the client verbatim; keep secrets and other tenants' IDs out of it. + +**Context providers**: `RequestData` moves from `genkit.plugin_api` to `genkit.web` and exposes `.method`, `.headers`, and `.input` on Flask, FastAPI, and Django. Replace framework-specific reads like `req.request.META.get('HTTP_AUTHORIZATION')` with `req.headers.get('authorization')`. + +```python +from genkit import PublicError +from genkit.web import RequestData, error_body, error_status + +async def auth_context(req: RequestData) -> dict: + token = req.headers.get('authorization') + if not token: + raise PublicError('UNAUTHENTICATED', 'Missing bearer token') # 401 + return {'user': await verify_token(token)} + +@ai.flow() +async def get_order(order_id: str) -> dict: + order = await db_find(order_id) + if order is None: + raise PublicError('NOT_FOUND', f'Order {order_id} not found') # 404 + return order +``` + +For hand-written HTTP routes that call `my_flow.run(...)` directly, serialize errors with `error_body(e)` and `error_status(e)` from `genkit.web`. + +--- + +## 8. Prompts, Structured Output, Validation, and Telemetry + +- **No Handlebars rendering in `ai.generate`**: `ai.generate(prompt='Hi {{name}}')` sends `'Hi {{name}}'` verbatim. Move templates to `ai.define_prompt(name='greet', prompt='Hi {{name}}')` or `.prompt` files. +- **Prompt calls & validation**: `await prompt({'dish': 'pie'}, config={'temperature': 0.2})` validates input against `input.schema`, fills in `.prompt` defaults, merges `config` key-by-key over the prompt's config, and locks `output_schema` at definition time. +- **Truncated structured output**: if a reply hits `max_output_tokens` or produces invalid JSON, `res.output` is `None` and `res.error.reason == ErrorReason.INVALID_OUTPUT`: + ```python + res = await ai.generate(prompt='...', output_schema=City) + if res.output is None: + raise RuntimeError(f'Structured output failed: {res.error}') + ``` +- **Flow I/O validation**: flow inputs and return values are validated against their type annotations, so a flow that returns the wrong shape raises; `{"data": null}` over HTTP uses the parameter default. +- **Media URL limit**: media URLs over 20 MB fail instead of downloading into memory. +- **Telemetry**: + ```python + # Old (0.12): add_gcp_telemetry(project_id='my-project', force_dev_export=True) + # New (0.13): FIREBASE_PROJECT_ID fallback removed; pass project= or set GOOGLE_CLOUD_PROJECT + from genkit_google_cloud import enable_google_cloud_telemetry + enable_google_cloud_telemetry(project='my-project') + + # Custom trace instrumentation (replaces tracer / add_custom_exporter / AdjustingTraceExporter) + from genkit.telemetry import Instrumentation, configure_instrumentation + ``` + +--- + +## 9. Plugin & Custom Middleware Authors + +```python +# Custom middleware: BaseMiddleware -> Middleware, GenerateMiddlewareContext -> MiddlewareContext +from pydantic import BaseModel +from genkit.middleware import Middleware, MiddlewareContext + +class PiiRedactConfig(BaseModel): + mask: str = '[REDACTED]' + +@ai.middleware +class PiiRedact(Middleware[PiiRedactConfig]): + name = 'pii_redact' + + async def model(self, params, ctx: MiddlewareContext, next_fn): + return await next_fn(params, ctx) +``` + +- `GenerateMiddleware`, `MiddlewareRef`, `MiddlewarePlugin`, and `new_middleware` are removed; `Middleware.tools()` returns `Tool` handles. +- Removed `genkit.plugin_api` helpers and their replacements: + ```python + # Old (0.12) + from genkit.plugin_api import model, model_ref, background_model, embedder, embedder_ref, ModelConfig + from genkit.plugin_api import wrap_http_error, from_http_code, parse_retry_after_ms + from genkit.plugin_api import get_cached_client, GENKIT_CLIENT_HEADER, GENKIT_VERSION, tracer, add_custom_exporter + + # New (0.13) + from genkit import get_logger + from genkit.embedder import Embedder, EmbedderRef + from genkit.model import BackgroundModel, Model, ModelConfig, ModelRef + from genkit.plugin_api import GENKIT_USER_AGENT, GenkitConfig, loop_local_client, provider_error + from genkit.telemetry import Instrumentation, configure_instrumentation + ``` +- Custom formatters import `Formatter`, `FormatterConfig`, and `FormatHandler` from `genkit.formats`. +- `Plugin.resolve(kind, name)` now receives `name` **without** the `'myplugin/'` prefix. `Plugin.model()`, `Plugin.embedder()`, and `Supports.context` are removed. `generate_operation` receives only what a background start uses. + +--- + +## 10. Evaluators (0.13 Only — Removed in 1.0) + +Apply only if the developer explicitly confirms keeping evaluators pinned to `genkit==0.13.*`. Renames: `EvalFnResponse` -> `EvaluatorResponse`, `EvalStatusEnum` -> `ScoreStatus`, `Details` -> `ScoreDetails`; `evaluation` is now a `list[Score]`; `ai.define_evaluator(name=..., fn=...)` becomes positional `(name, fn)`; `display_name=` and `definition=` stay required keywords. `EvaluatorResponse` requires `test_case_id`. + +```python +from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score, ScoreDetails, ScoreStatus + +async def exact_match(dp: BaseDataPoint, _options: object = None) -> EvaluatorResponse: + passed = str(dp.output or '').strip() == str(dp.reference or '').strip() + return EvaluatorResponse( + test_case_id=dp.test_case_id or '', + evaluation=[ + Score( + score=1.0 if passed else 0.0, + status=ScoreStatus.PASS if passed else ScoreStatus.FAIL, + details=ScoreDetails(reasoning='Exact string match'), + ) + ], + ) + +# Positional (name, fn); display_name and definition are required keywords +ai.define_evaluator('byo/exact_match', exact_match, display_name='Exact Match', definition='Exact string match.') + +# options= -> config=; returns list[EvaluatorResponse] directly (no .root) +results = await ai.evaluate(evaluator='byo/exact_match', dataset=rows, config={}) +``` + +--- + +## 11. Verification Checklist + +1. Run the project's type checker (`pyright`, `mypy`, or `ty`) — catches moved imports, enum-to-`Literal` arguments, keyword-only calls, `FlowContext`/`ToolContext` generics, and `OutputT | None` checks on `response.output`. +2. Grep for `{{` inside `ai.generate(` calls and move templates to `ai.define_prompt` or `.prompt` files. Grep for reads of `response.output` that assume it is never `None`. +3. Grep for `GenkitError` inside HTTP-served flows and context providers; convert caller-facing errors to `PublicError`. +4. Run the test suite. diff --git a/plugins/genkit/.agents/skills/developing-genkit-python/references/setup.md b/plugins/genkit/.agents/skills/developing-genkit-python/references/setup.md index 136f3869..1043cefd 100644 --- a/plugins/genkit/.agents/skills/developing-genkit-python/references/setup.md +++ b/plugins/genkit/.agents/skills/developing-genkit-python/references/setup.md @@ -2,38 +2,24 @@ ## New project -**Always use a virtual environment** — never install Genkit into the system interpreter. With **uv**, the project’s **`.venv`** is created and used by `uv sync` / `uv run` automatically once you add dependencies. +**Always use a virtual environment** — never install Genkit into the system interpreter. With **uv**, `uv add` and `uv run` manage `.venv` automatically. ```bash mkdir my-app && cd my-app uv init -# uv may set requires-python to a very new floor (e.g. >=3.14). Genkit needs -# 3.10+ — edit pyproject.toml if your interpreter is older. -# uv init may also write .python-version (e.g. 3.14); delete or edit it -# if it fights an explicit `uv venv --python 3.12`. +# Genkit requires Python 3.10+. If `uv init` pins a newer interpreter in +# pyproject.toml or .python-version than you have installed, adjust it first. uv venv --python 3.12 .venv -# Unix: source .venv/bin/activate -# Windows: .venv\Scripts\activate -uv add genkit genkit-google-genai -# Agents with ToolApproval / Filesystem / Artifacts / Retry also need: -# uv add genkit-middleware -# HTTP serve: uv add genkit-fastapi -# Evals: uv add genkit-evaluators -# Also add pydantic if you use BaseModel schemas with Dotprompt. -export GEMINI_API_KEY=your_key_here -``` - -Import the Google AI plugin as: +uv add 'genkit==0.13.*' 'genkit-google-genai==0.13.*' -```python -from genkit_google_genai import GoogleAI +export GEMINI_API_KEY=your_key_here ``` -`uv init` creates `pyproject.toml`. Add your app under something like `src/main.py` (or match whatever layout `uv` generated) and point `genkit start` at that entrypoint. +Keep `genkit` and every `genkit-*` package on the **same `0.13.*` release**. `pydantic` is installed automatically with `genkit`, though adding `pydantic` directly to `pyproject.toml` is fine when defining your own `BaseModel` schemas. -## pyproject.toml +Place your entrypoint at `src/main.py` (or match the file generated by `uv init`) and run it under `genkit start -- uv run src/main.py`. -Minimal `[project]` block with unpinned Genkit deps (resolver picks compatible releases): +## `pyproject.toml` ```toml [project] @@ -41,20 +27,32 @@ name = "my-app" version = "0.1.0" requires-python = ">=3.10" dependencies = [ - "genkit", - "genkit-google-genai", + "genkit==0.13.*", + "genkit-google-genai==0.13.*", ] ``` -## Plugins - -Packages are **`genkit-*`** on PyPI, e.g. `genkit-google-genai`, `genkit-vertexai`, -`genkit-anthropic`, `genkit-fastapi`, `genkit-middleware`, `genkit-evaluators`. -Install with `uv add genkit-<name>`. - -Import modules use underscores (e.g. `genkit_google_genai`, `genkit_middleware`). -See [Examples](examples.md) and [Agents](agents.md). +## Packages & Imports + +PyPI distribution names use hyphens (`genkit-*`); Python import modules use underscores (`genkit_*`). Import everything from each package's top-level module (or `.exp` for experimental agent integrations): + +- **`genkit-google-genai`**: `from genkit_google_genai import GoogleAI, VertexAI, GeminiConfig` (Gemini API via `GEMINI_API_KEY`, or Google Cloud Vertex AI via ADC). +- **`genkit-vertexai`**: `from genkit_vertexai import ModelGarden` (third-party models on Vertex AI Model Garden). Install the extra matching the models you call: + - `uv add 'genkit-vertexai[anthropic]'` for Claude models + - `uv add 'genkit-vertexai[openai]'` for Llama, Mistral, and OpenAI-compatible models +- **`genkit-anthropic`**: `from genkit_anthropic import Anthropic, AnthropicConfig` (`ANTHROPIC_API_KEY`). +- **`genkit-openai`**: `from genkit_openai import OpenAI, OpenAIConfig` (`OPENAI_API_KEY` or custom `base_url=`). +- **`genkit-ollama`**: `from genkit_ollama import Ollama, OllamaConfig` (local Ollama server; auto-discovers pulled models). +- **`genkit-amazon-bedrock`**: `from genkit_amazon_bedrock import Bedrock, BedrockConfig, StabilityImageConfig`. +- **`genkit-middleware`**: `from genkit_middleware import Middleware, Retry, Fallback, ToolApproval, Skills, Filesystem` (plus `from genkit_middleware.exp import Artifacts` for agent sessions). +- **`genkit-fastapi`**: `from genkit_fastapi import serve_flow, genkit_fastapi_handler, handle_genkit_request` (plus `from genkit_fastapi.exp import serve_agent`). +- **`genkit-flask`**: `from genkit_flask import genkit_flask_handler` (requires `flask[async]`; does not install `genkit-google-genai` automatically; add your model plugin explicitly). +- **`genkit-django`**: `from genkit_django import genkit_django_handler`. +- **`genkit-mcp`**: `from genkit_mcp import McpClient, McpHost, define_mcp_client, define_mcp_host` (Model Context Protocol clients and tool hosts). +- **`genkit-google-cloud`**: `from genkit_google_cloud import enable_google_cloud_telemetry` (plus `from genkit_google_cloud.exp import FirestoreSessionStore`). +- **`genkit-otel`**: `from genkit_otel import GenAiInstrumentation, ContentCapturingMode, OtelEventLogger` (OpenTelemetry GenAI spans and metrics). +- **`genkit-evaluators`**: `from genkit_evaluators import register_genkit_evaluators`. Last release with evaluators; Genkit 1.0 cuts evaluators from the 1.0 release (evaluation will return as an additive feature in a later release). Requires `genkit<1` (see [Evals](evals.md)). ## Python version -**3.10+**. Prefer a project venv: `uv venv --python 3.12 .venv` (or newer) before you run commands. +**Python 3.10+**. Upgrading an existing 0.12 app? Follow [Migrating from 0.12 to 0.13](migrating-from-0.12.md). diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/SKILL.md b/plugins/genkit/agent/skills/developing-genkit-dart/SKILL.md index c700c828..c4353e21 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/SKILL.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/SKILL.md @@ -1,8 +1,9 @@ --- name: developing-genkit-dart -description: Generates code and provides documentation for the Genkit Dart SDK. - Use when the user asks to build AI agents in Dart, use Genkit flows, or - integrate LLMs into Dart/Flutter applications. +description: Provides guidance on developing AI-powered applications using the + Genkit Dart SDK. Use when the user asks to build AI agents in Dart, use Genkit + flows, or integrate LLMs into Dart/Flutter applications. Don't use for Genkit + in JavaScript/TypeScript, Go, or Python. metadata: version: 1.0.0 category: AiAndMachineLearning diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/a2ui.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/a2ui.md index cd3a86d7..fbff6124 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/a2ui.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/a2ui.md @@ -13,6 +13,16 @@ An A2UI-enabled agent streams more than prose. It streams interactive UI as the model responds. The entire server-side integration is a single model middleware: add `a2ui()` to an agent's `use` list and nothing else changes. +- [Install](#install): Lines 26-49 +- [Server: add the a2ui() middleware](#server-add-the-a2ui-middleware): Lines 51-124 +- [Client: render surfaces (Flutter + genui)](#client-render-surfaces-flutter--genui): Lines 126-181 +- [Handling user actions](#handling-user-actions): Lines 183-242 +- [Custom catalogs](#custom-catalogs): Lines 245-323 +- [Security and the trust boundary](#security-and-the-trust-boundary): Lines 325-346 +- [How it works](#how-it-works): Lines 348-367 + +--- + ## Install On the **server** (the shelf app that hosts the agent), add `genkit_a2ui` diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-artifacts.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-artifacts.md index ed7eb421..3e792e5e 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-artifacts.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-artifacts.md @@ -26,6 +26,14 @@ name) and are returned in `res.artifacts` / tracked on the client's > session artifacts for conversation-scoped deliverables, `filesystem()` for > persistent on-disk work. +- [Give the model artifact tools](#give-the-model-artifact-tools): Lines 37-99 +- [How artifacts flow](#how-artifacts-flow): Lines 101-113 +- [The Artifact shape](#the-artifact-shape): Lines 115-124 +- [Programmatic access (inside tools / custom agents)](#programmatic-access-inside-tools--custom-agents): Lines 126-142 +- [Reading artifacts on the client](#reading-artifacts-on-the-client): Lines 144-171 + +--- + ## Give the model artifact tools ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-background.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-background.md index 65245fca..fcaa70b9 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-background.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-background.md @@ -9,6 +9,13 @@ and returns a `snapshotId` **immediately**, keeps processing, then updates the snapshot to a terminal status (`completed` / `failed` / `aborted` / `expired`). The client polls for completion and can abort. +- [Define the agent (store required)](#define-the-agent-store-required): Lines 19-46 +- [Client-side: detach + poll + abort](#client-side-detach--poll--abort): Lines 48-87 +- [Server-side: detach + wait](#server-side-detach--wait): Lines 89-101 +- [Status values](#status-values): Lines 103-114 + +--- + ## Define the agent (store required) ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-branching.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-branching.md index 65f1573e..1341a7be 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-branching.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-branching.md @@ -10,6 +10,12 @@ snapshot creates a new, independent snapshot; the original is unchanged. To branch, open a new `chat` attached to an earlier snapshot via `agent.chat(snapshotId: ...)`. +- [Server-side branching](#server-side-branching): Lines 19-46 +- [Client-side branching ("pick a variant")](#client-side-branching-pick-a-variant): Lines 48-79 +- [Restoring history from a snapshot](#restoring-history-from-a-snapshot): Lines 81-109 + +--- + ## Server-side branching ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-custom.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-custom.md index 631fefc3..86ae2ed7 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-custom.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-custom.md @@ -9,6 +9,13 @@ the turn** — multiple sequential model calls, custom logic between them, manua message/state management, or custom progress streaming — use `ai.defineCustomAgent`. You provide the handler that runs the turn. +- [When to use it](#when-to-use-it): Lines 19-29 +- [Signature](#signature): Lines 31-62 +- [Example: multi-step research agent](#example-multi-step-research-agent): Lines 64-174 +- [Custom status streaming](#custom-status-streaming): Lines 176-199 + +--- + ## When to use it Reach for `defineCustomAgent` when a turn needs to: diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-deployment.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-deployment.md index 398b260e..bbfb4d8e 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-deployment.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-deployment.md @@ -19,6 +19,14 @@ match the `remoteAgent` client defaults (`${url}/getSnapshot`, `${url}/abort`), so a client only needs the base `url`. `hideGetSnapshot: true` / `hideAbort: true` remove supported routes; they can't force unsupported ones. +- [Serving several agents (shelf)](#serving-several-agents-shelf): Lines 30-78 +- [Auth](#auth): Lines 80-84 +- [CORS for browser clients](#cors-for-browser-clients): Lines 86-90 +- [Registering agents/flows](#registering-agentsflows): Lines 92-98 +- [Notes](#notes): Lines 100-110 + +--- + ## Serving several agents (shelf) ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-human-in-the-loop.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-human-in-the-loop.md index 5718a0d5..557a58a1 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-human-in-the-loop.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-human-in-the-loop.md @@ -36,6 +36,14 @@ back into the resume). Flow: `chat.send(text: ...)` → response has `res.interrupts` → collect human input → `chat.resume(respond: [...])`. +- [Define an interrupt (a tool that interrupts)](#define-an-interrupt-a-tool-that-interrupts): Lines 47-111 +- [Detect and resume (server-side)](#detect-and-resume-server-side): Lines 113-169 +- [Client-side (browser) interrupts](#client-side-browser-interrupts): Lines 171-202 +- [Interrupts with tool-approval middleware](#interrupts-with-tool-approval-middleware): Lines 204-229 +- [Notes & gotchas](#notes--gotchas): Lines 231-245 + +--- + ## Define an interrupt (a tool that interrupts) The example below uses tools that return `.interrupt(...)`, which pauses the diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-multi-agent.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-multi-agent.md index 416e5c18..249bbd9f 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-multi-agent.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-multi-agent.md @@ -19,6 +19,15 @@ import 'package:genkit_middleware/agents.dart'; final ai = Genkit(plugins: [googleAI(), AgentsPlugin()]); // retry() is built in ``` +- [1. Define the sub-agents](#1-define-the-sub-agents): Lines 31-57 +- [2. Wire up the orchestrator](#2-wire-up-the-orchestrator): Lines 59-101 +- [agents() options](#agents-options): Lines 103-113 +- [Background (async) delegation](#background-async-delegation): Lines 115-143 +- [Sharing artifacts between agents](#sharing-artifacts-between-agents): Lines 145-154 +- [Other middleware](#other-middleware): Lines 156-166 + +--- + ## 1. Define the sub-agents Give each sub-agent a `description` — it's auto-discovered from registry metadata diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-sessions.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-sessions.md index 3b63904c..71368e74 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-sessions.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-sessions.md @@ -12,6 +12,14 @@ conversation state forward. A store also enables [branching](agents-branching.md and [background execution](agents-background.md). (Interrupts work with or without a store — see [human-in-the-loop](agents-human-in-the-loop.md).) +- [Pick a store](#pick-a-store): Lines 23-71 +- [Typed session state](#typed-session-state): Lines 73-112 +- [Interrupts (human-in-the-loop)](#interrupts-human-in-the-loop): Lines 114-119 +- [Firestore session store (scalable)](#firestore-session-store-scalable): Lines 121-161 +- [Implementing a custom SessionStore](#implementing-a-custom-sessionstore): Lines 163-200 + +--- + ## Pick a store ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-state.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-state.md index 7aad1957..ed98f914 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-state.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents-state.md @@ -8,6 +8,13 @@ and mutate it during a turn, and it is automatically synced to the [`remoteAgent`](agents.md#consume-an-agent-from-a-client-remoteagent) client via `customPatch` chunks (so the client's tracked state stays live mid-stream). +- [Declare the state shape](#declare-the-state-shape): Lines 18-55 +- [Read & mutate state inside tools](#read--mutate-state-inside-tools): Lines 57-157 +- [Seed and read state (server-side)](#seed-and-read-state-server-side): Lines 159-182 +- [Auto-sync to the remoteAgent client](#auto-sync-to-the-remoteagent-client): Lines 184-222 + +--- + ## Declare the state shape Pass a `stateSchema` (a schemantic `.$schema`) to `defineAgent`. It's validated diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents.md index c058490f..3ec6b07c 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/agents.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/agents.md @@ -46,6 +46,18 @@ Progressive disclosure — read the file for the level you need: - [Advanced custom agents](agents-custom.md): `defineCustomAgent` for full turn control. - [Deploying agents](agents-deployment.md): serving multiple agents over HTTP with `GenkitRouter.addAgent`, shelf, CORS. +- [Setup](#setup): Lines 61-83 +- [Define an agent](#define-an-agent): Lines 85-153 +- [Agents and middleware go hand in hand](#agents-and-middleware-go-hand-in-hand): Lines 155-196 +- [Chat with an agent (server-side)](#chat-with-an-agent-server-side): Lines 198-240 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 242-255 +- [Serve an agent over HTTP](#serve-an-agent-over-http): Lines 257-278 +- [Consume an agent from a client (remoteAgent)](#consume-an-agent-from-a-client-remoteagent): Lines 280-314 +- [Using from Flutter](#using-from-flutter): Lines 316-390 +- [Client-managed state (no server store)](#client-managed-state-no-server-store): Lines 392-431 + +--- + ## Setup Register the plugins your agents need on a shared `Genkit` instance. `retry` is diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/dotprompt.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/dotprompt.md index 44302ff3..fb23c002 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/dotprompt.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/dotprompt.md @@ -1,5 +1,17 @@ # Dotprompt (.prompt files) — Genkit Dart +- [What it is](#what-it-is): Lines 15-19 +- [Where files live](#where-files-live): Lines 21-34 +- [File format](#file-format): Lines 36-81 +- [Loading and calling a prompt](#loading-and-calling-a-prompt): Lines 83-157 +- [Registering named schemas](#registering-named-schemas): Lines 159-191 +- [Variants](#variants): Lines 193-201 +- [Partials](#partials): Lines 203-213 +- [Tools, tool-loop control, and middleware](#tools-tool-loop-control-and-middleware): Lines 215-266 +- [Backing an agent with a .prompt file](#backing-an-agent-with-a-prompt-file): Lines 268-282 + +--- + ## What it is `.prompt` files combine YAML frontmatter (model, config, schemas, tools, diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit.md index 0575e7fc..99dbe946 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit.md @@ -2,6 +2,23 @@ Genkit Dart is an AI SDK for Dart that provides a unified interface for text generation, structured output, tool calling, and agentic workflows. +- [Initialization](#initialization): Lines 22-33 +- [Reliability: retry transient errors](#reliability-retry-transient-errors): Lines 35-55 +- [Generate Text](#generate-text): Lines 57-67 +- [Stream Responses](#stream-responses): Lines 69-79 +- [Embed Text](#embed-text): Lines 81-95 +- [Define Tools](#define-tools): Lines 97-143 +- [Structured Output](#structured-output): Lines 145-166 +- [Errors and finish reasons](#errors-and-finish-reasons): Lines 168-225 +- [Cancellation](#cancellation): Lines 227-254 +- [Telemetry (instrumentation)](#telemetry-instrumentation): Lines 256-276 +- [Define Flows](#define-flows): Lines 278-326 +- [Calling remote Flows from a dart client](#calling-remote-flows-from-a-dart-client): Lines 328-368 +- [Calling remote Flows from a Javascript client](#calling-remote-flows-from-a-javascript-client): Lines 370-424 +- [Data Models](#data-models): Lines 426-565 + +--- + ## Initialization ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_google_genai.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_google_genai.md index 1828902c..1c724e46 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_google_genai.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_google_genai.md @@ -2,6 +2,15 @@ The Google AI plugin provides an interface against the official Google AI Gemini API. +- [Usage](#usage): Lines 14-34 +- [Model refs and Gemma](#model-refs-and-gemma): Lines 36-48 +- [Embeddings](#embeddings): Lines 50-61 +- [Config types are provider-prefixed](#config-types-are-provider-prefixed): Lines 63-85 +- [Image Generation](#image-generation): Lines 87-112 +- [Text-to-Speech (TTS)](#text-to-speech-tts): Lines 114-146 + +--- + ## Usage ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_mcp.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_mcp.md index 71da3478..12cb885f 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_mcp.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_mcp.md @@ -9,6 +9,12 @@ initialization-based peers automatically. > name is preserved in `metadata.originalName` and tool requests still resolve > correctly. +- [MCP Host (Recommended)](#mcp-host-recommended): Lines 18-73 +- [MCP Client (Advanced / Single Server)](#mcp-client-advanced--single-server): Lines 75-106 +- [MCP Server](#mcp-server): Lines 108-160 + +--- + ## MCP Host (Recommended) Connect to one or more MCP servers and aggregate their capabilities into the Genkit registry automatically. diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_middleware.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_middleware.md index 0d4d4e6f..2f9223bc 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_middleware.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_middleware.md @@ -17,6 +17,13 @@ void main() { } ``` +- [Filesystem Middleware](#filesystem-middleware): Lines 27-40 +- [Skills Middleware](#skills-middleware): Lines 42-55 +- [Tool Approval Middleware](#tool-approval-middleware): Lines 57-99 +- [Custom Middleware (core package:genkit)](#custom-middleware-core-packagegenkit): Lines 101-162 + +--- + ## Filesystem Middleware Allows the agent to list, read, write, and search/replace files within a restricted root directory. diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_shelf.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_shelf.md index c07797e7..54895b27 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_shelf.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/genkit_shelf.md @@ -7,6 +7,16 @@ HTTP serving lives in core `package:genkit/io.dart` (`GenkitRouter`, plain > There is no `startFlowServer` and no `FlowWithContextProvider`. Do not add > `shelf_cors_headers`; CORS is `CorsOptions`. +- [Standalone server (no shelf)](#standalone-server-no-shelf): Lines 20-54 +- [Existing shelf app](#existing-shelf-app): Lines 56-86 +- [Own dart:io server](#own-dartio-server): Lines 88-102 +- [Auth: contextProvider](#auth-contextprovider): Lines 104-119 +- [Agents](#agents): Lines 121-125 +- [CORS](#cors): Lines 127-132 +- [Older clients](#older-clients): Lines 134-142 + +--- + ## Standalone server (no shelf) ```dart diff --git a/plugins/genkit/agent/skills/developing-genkit-dart/references/schemantic.md b/plugins/genkit/agent/skills/developing-genkit-dart/references/schemantic.md index 2d132e97..7907df35 100644 --- a/plugins/genkit/agent/skills/developing-genkit-dart/references/schemantic.md +++ b/plugins/genkit/agent/skills/developing-genkit-dart/references/schemantic.md @@ -2,6 +2,17 @@ Schemantic is a general-purpose Dart library used for defining strongly typed data classes that automatically bind to reusable runtime JSON schemas. It is standard for the `genkit-dart` framework but works independently as well. +- [Core Concepts](#core-concepts): Lines 16-22 +- [Installation](#installation): Lines 24-42 +- [Basic Usage](#basic-usage): Lines 44-97 +- [Primitive Schemas](#primitive-schemas): Lines 99-111 +- [Union Types (AnyOf)](#union-types-anyof): Lines 113-130 +- [Field Annotations](#field-annotations): Lines 132-153 +- [Recursive Schemas](#recursive-schemas): Lines 155-166 +- [Non-nullable getters throw on partial data](#non-nullable-getters-throw-on-partial-data): Lines 168-176 + +--- + ## Core Concepts Always use `schemantic` when strongly typed JSON parsing or programmatic schema validation is required. diff --git a/plugins/genkit/agent/skills/developing-genkit-go/SKILL.md b/plugins/genkit/agent/skills/developing-genkit-go/SKILL.md index 6b198669..015bb542 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/SKILL.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/SKILL.md @@ -1,9 +1,10 @@ --- name: developing-genkit-go -description: Develop AI-powered applications using Genkit in Go. Use when the - user asks to build AI features, agents, flows, or tools in Go using Genkit, or - when working with Genkit Go code involving generation, prompts, streaming, - tool calling, or model providers. +description: Provides guidance on developing AI-powered applications using + Genkit in Go. Use when the user asks to build AI features, agents, flows, or + tools in Go using Genkit, or when working with Genkit Go code involving + generation, prompts, streaming, tool calling, or model providers. Don't use + for Genkit in JavaScript/TypeScript, Python, or Dart. metadata: version: 1.0.0 category: AiAndMachineLearning @@ -55,7 +56,7 @@ Load the appropriate reference based on what you need: | Feature | Reference | When to load | | --- | --- | --- | | Initialization | [references/getting-started.md](references/getting-started.md) | Setting up `genkit.Init`, plugins, the `*Genkit` instance pattern | -| Generation | [references/generation.md](references/generation.md) | `Generate`, `GenerateText`, `GenerateData`, streaming, output formats | +| Generation | [references/generation.md](references/generation.md) | `Generate`, `GenerateText`, `GenerateData`, streaming, output formats, `Embed` | | Prompts | [references/prompts.md](references/prompts.md) | `DefinePrompt`, `DefineDataPrompt`, `.prompt` files, schemas | | Tools | [references/tools.md](references/tools.md) | `DefineTool`, `DefineResumableTool` interrupts (`Interrupted` claim, `Restart`/`Respond`, `WithResume`), `ai/tool` helpers (`tool.Interrupt`, `tool.Fail`, `tool.AttachParts`) | | Middleware | [references/middleware.md](references/middleware.md) | `ai.Middleware`, `ai.WithUse`, `Hooks` (Generate/Model/Tool), built-ins (`Retry`, `Fallback`, `ToolApproval`, `Filesystem`, `Skills`, `SoftToolErrors`, `ContextCompression`, `Budget`) | diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/a2ui.md b/plugins/genkit/agent/skills/developing-genkit-go/references/a2ui.md index 5de54c5b..a005c921 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/a2ui.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/a2ui.md @@ -15,6 +15,15 @@ An A2UI-enabled agent streams more than prose. It streams interactive UI as the model responds. The entire Go integration is a single model middleware: add `&a2uix.Surfaces{}` to a generate/agent's `ai.WithUse` and nothing else changes. +- [Go is server-only: bring your own client](#go-is-server-only-bring-your-own-client): Lines 27-50 +- [Server: add the a2ui middleware](#server-add-the-a2ui-middleware): Lines 52-116 +- [Serve the agent over HTTP](#serve-the-agent-over-http): Lines 118-164 +- [Custom catalogs](#custom-catalogs): Lines 166-220 +- [Security and the trust boundary](#security-and-the-trust-boundary): Lines 222-242 +- [How it works](#how-it-works): Lines 244-263 + +--- + ## Go is server-only: bring your own client **Go has no A2UI client/renderer.** The Go plugin is the server half: it injects diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-artifacts.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-artifacts.md index 843ca999..6ecfaf97 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-artifacts.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-artifacts.md @@ -15,6 +15,13 @@ import ( ) ``` +- [Give the model artifact tools](#give-the-model-artifact-tools): Lines 25-67 +- [The Artifact shape](#the-artifact-shape): Lines 69-80 +- [Programmatic access (inside tools / custom agents)](#programmatic-access-inside-tools--custom-agents): Lines 82-110 +- [Sharing artifacts across agents](#sharing-artifacts-across-agents): Lines 112-128 + +--- + ## Give the model artifact tools The `middlewarex.Artifacts` middleware (see [using middleware](middleware.md)) diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-background.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-background.md index 1f8bb7a6..39851063 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-background.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-background.md @@ -11,6 +11,15 @@ and returns its `SnapshotID` **immediately**, keeps processing, then rewrites th snapshot to a terminal status (`completed` / `failed` / `aborted`). A reader waits on the snapshot (or polls it) and can `Abort` in the meantime. +- [Define the agent (store required)](#define-the-agent-store-required): Lines 23-33 +- [Detach a turn](#detach-a-turn): Lines 35-53 +- [Wait (or poll)](#wait-or-poll): Lines 55-79 +- [Abort and resume](#abort-and-resume): Lines 81-107 +- [Serve detach over HTTP](#serve-detach-over-http): Lines 109-127 +- [Status values (aix.SnapshotStatus)](#status-values-aixsnapshotstatus): Lines 129-142 + +--- + ## Define the agent (store required) ```go diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-branching.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-branching.md index 0d470a30..f68a8229 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-branching.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-branching.md @@ -10,6 +10,13 @@ snapshot creates a new, independent snapshot; the original is unchanged. To branch, run a turn resuming from an earlier snapshot via `aix.WithSnapshotID(id)`. +- [Branch from a checkpoint](#branch-from-a-checkpoint): Lines 20-47 +- ["Pick a variant"](#pick-a-variant): Lines 49-77 +- [Resume by session vs. by snapshot](#resume-by-session-vs-by-snapshot): Lines 79-84 +- [Restore history from a snapshot](#restore-history-from-a-snapshot): Lines 86-109 + +--- + ## Branch from a checkpoint ```go diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-custom.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-custom.md index 7da0384f..d347512f 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-custom.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-custom.md @@ -9,6 +9,14 @@ the turn** — multiple sequential model calls, custom logic between them, manua message/state management, or custom progress streaming — use `genkitx.DefineCustomAgent`. You provide the function that runs the turn loop. +- [When to use it](#when-to-use-it): Lines 20-30 +- [Signature](#signature): Lines 32-66 +- [Example: multi-step research agent](#example-multi-step-research-agent): Lines 68-157 +- [Custom status streaming](#custom-status-streaming): Lines 159-189 +- [Notes](#notes): Lines 191-202 + +--- + ## When to use it Reach for `DefineCustomAgent` when a turn needs to: diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-deployment.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-deployment.md index 03ce103d..ca50878a 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-deployment.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-deployment.md @@ -14,6 +14,15 @@ companion actions: `nil` unless the store implements `aix.SnapshotSubscriber` (both `localstore` stores do). +- [Request/response shape](#requestresponse-shape): Lines 26-41 +- [Serve with the built-in route layout (recommended)](#serve-with-the-built-in-route-layout-recommended): Lines 43-73 +- [A reusable helper for several agents (manual wiring)](#a-reusable-helper-for-several-agents-manual-wiring): Lines 75-121 +- [CORS for browser clients](#cors-for-browser-clients): Lines 123-149 +- [Registering agents](#registering-agents): Lines 151-157 +- [Verifying locally](#verifying-locally): Lines 159-180 + +--- + ## Request/response shape The handler accepts `{"data": <AgentInput>, "init": <AgentInit>}`: diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-human-in-the-loop.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-human-in-the-loop.md index 3389e714..21a7a21b 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-human-in-the-loop.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-human-in-the-loop.md @@ -20,6 +20,13 @@ The tool side is identical to plain `Generate` (see [tools.md](tools.md)): `Interrupted` method. Only the delivery differs: an agent takes the resume parts in an `aix.ToolResume` via `conn.SendResume` or `AgentInput.Resume`. +- [Define a resumable tool](#define-a-resumable-tool): Lines 30-101 +- [Detect, claim, and resume (streaming connection)](#detect-claim-and-resume-streaming-connection): Lines 103-165 +- [Single-turn resume with Run](#single-turn-resume-with-run): Lines 167-190 +- [Notes & gotchas](#notes--gotchas): Lines 192-208 + +--- + ## Define a resumable tool `genkit.DefineResumableTool[In, Out, Res]` defines a tool that can pause by diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-multi-agent.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-multi-agent.md index af07817e..17f6a942 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-multi-agent.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-multi-agent.md @@ -18,6 +18,17 @@ import ( ) ``` +- [1. Define the sub-agents](#1-define-the-sub-agents): Lines 32-55 +- [2. Wire up the orchestrator](#2-wire-up-the-orchestrator): Lines 57-94 +- [middlewarex.Agents fields](#middlewarexagents-fields): Lines 96-114 +- [Background delegation (Async: true)](#background-delegation-async-true): Lines 116-155 +- [Continuing a task (continue_task)](#continuing-a-task-continue_task): Lines 157-173 +- [Delegating to a remote agent](#delegating-to-a-remote-agent): Lines 175-197 +- [Sharing artifacts between agents](#sharing-artifacts-between-agents): Lines 199-211 +- [Combining with retries](#combining-with-retries): Lines 213-235 + +--- + ## 1. Define the sub-agents Give each sub-agent a description via `aix.WithDescription` — it's diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-sessions.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-sessions.md index bc5ca3e8..492da2be 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-sessions.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-sessions.md @@ -10,6 +10,16 @@ snapshot chain carries conversation state forward. A store also enables (Interrupts work with or without a store — see [human-in-the-loop](agents-human-in-the-loop.md).) +- [Pick a store](#pick-a-store): Lines 23-103 +- [Turns persist automatically](#turns-persist-automatically): Lines 105-115 +- [Resume a conversation](#resume-a-conversation): Lines 117-139 +- [Read a snapshot without running a turn](#read-a-snapshot-without-running-a-turn): Lines 141-158 +- [Typed session state](#typed-session-state): Lines 160-188 +- [Interrupts (human-in-the-loop)](#interrupts-human-in-the-loop): Lines 190-195 +- [Implementing a custom SessionStore](#implementing-a-custom-sessionstore): Lines 197-243 + +--- + ## Pick a store `ai/exp/localstore` ships two single-process stores. Both are generic over the diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-state.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-state.md index 66f5e05a..630e3c10 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents-state.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents-state.md @@ -7,6 +7,15 @@ own structured data (a task list, a workflow status, counters, etc.). Tools read and mutate it during a turn, and every mutation is streamed to a connected client as a JSON Patch so the client's view stays live. +- [Declare the state shape](#declare-the-state-shape): Lines 19-44 +- [Read & mutate state inside tools](#read--mutate-state-inside-tools): Lines 46-88 +- [Seed and read state](#seed-and-read-state): Lines 90-107 +- [Live state on a connection](#live-state-on-a-connection): Lines 109-136 +- [Prompt templating with state](#prompt-templating-with-state): Lines 138-145 +- [Custom status streaming from a custom agent](#custom-status-streaming-from-a-custom-agent): Lines 147-152 + +--- + ## Declare the state shape The custom-state type is the `State` type parameter of the agent (and its store). diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/agents.md b/plugins/genkit/agent/skills/developing-genkit-go/references/agents.md index db7a9c4b..bb84ddef 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/agents.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/agents.md @@ -27,6 +27,18 @@ Read the file for the level you need: - [Advanced custom agents](agents-custom.md): `DefineCustomAgent` for full turn control. - [Deploying agents](agents-deployment.md): serving agents over HTTP. +- [Packages and import aliases](#packages-and-import-aliases): Lines 42-59 +- [Setup](#setup): Lines 61-72 +- [Define an agent](#define-an-agent): Lines 74-166 +- [Agents and middleware go hand in hand](#agents-and-middleware-go-hand-in-hand): Lines 168-193 +- [Run an agent (single turn)](#run-an-agent-single-turn): Lines 195-222 +- [Chat with an agent (multi-turn)](#chat-with-an-agent-multi-turn): Lines 224-267 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 269-284 +- [Serve an agent over HTTP](#serve-an-agent-over-http): Lines 286-324 +- [Client-managed vs server-managed state](#client-managed-vs-server-managed-state): Lines 326-366 + +--- + ## Packages and import aliases Agents span a few packages. These aliases are used throughout the agent docs: diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/flows-and-http.md b/plugins/genkit/agent/skills/developing-genkit-go/references/flows-and-http.md index 92f99a22..a52e7d69 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/flows-and-http.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/flows-and-http.md @@ -1,5 +1,12 @@ # Flows & HTTP +- [DefineFlow](#defineflow): Lines 10-29 +- [DefineStreamingFlow](#definestreamingflow): Lines 31-99 +- [Named Sub-Steps](#named-sub-steps): Lines 101-123 +- [HTTP Handlers](#http-handlers): Lines 125-190 + +--- + ## DefineFlow Wrap AI logic in a flow for observability, tracing, and HTTP deployment. diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/generation.md b/plugins/genkit/agent/skills/developing-genkit-go/references/generation.md index 5934575c..b740845f 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/generation.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/generation.md @@ -1,5 +1,15 @@ # Generation +- [GenerateText](#generatetext): Lines 13-22 +- [Generate](#generate): Lines 24-37 +- [GenerateData (Structured Output)](#generatedata-structured-output): Lines 39-54 +- [Streaming](#streaming): Lines 56-114 +- [Common Options](#common-options): Lines 116-132 +- [Output Formats](#output-formats): Lines 134-186 +- [Embeddings](#embeddings): Lines 188-223 + +--- + ## GenerateText Simplest form. Returns a string. @@ -174,3 +184,40 @@ genkit.DefinePrompt(g, "characters", ai.WithOutputFormat(ai.OutputFormatJSONL), ) ``` + +## Embeddings + +```go +res, err := genkit.Embed(ctx, g, + ai.WithEmbedderName("googleai/gemini-embedding-001"), + ai.WithTextDocs("The sky is blue.", "Grass is green."), // one embedding per doc +) +if err != nil { + return err +} +vector := res.Embeddings[0].Embedding // []float32 + +// Optional tuning (default is 3072 dimensions) +res, err = genkit.Embed(ctx, g, + ai.WithEmbedderName("googleai/gemini-embedding-001"), + ai.WithTextDocs("The sky is blue."), + ai.WithConfig(&genai.EmbedContentConfig{ + OutputDimensionality: genai.Ptr[int32](768), + TaskType: "RETRIEVAL_DOCUMENT", + }), +) +``` + +Common embedders: `googleai/gemini-embedding-001` (text), +`googleai/gemini-embedding-2` (multimodal: text, image, video). + +Multimodal input goes in a document's media parts: + +```go +res, err := genkit.Embed(ctx, g, + ai.WithEmbedderName("googleai/gemini-embedding-2"), + ai.WithDocs(&ai.Document{Content: []*ai.Part{ + ai.NewMediaPart("image/png", "data:image/png;base64,..."), + }}), +) +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/getting-started.md b/plugins/genkit/agent/skills/developing-genkit-go/references/getting-started.md index 01e36177..f8d74b8d 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/getting-started.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/getting-started.md @@ -1,5 +1,12 @@ # Getting Started +- [Project Setup](#project-setup): Lines 10-24 +- [Initialization](#initialization): Lines 26-90 +- [Genkit CLI](#genkit-cli): Lines 92-136 +- [Import Paths](#import-paths): Lines 138-147 + +--- + ## Project Setup ```bash diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/middleware.md b/plugins/genkit/agent/skills/developing-genkit-go/references/middleware.md index 027d7bfd..b766d15e 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/middleware.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/middleware.md @@ -2,6 +2,21 @@ Middleware wraps `Generate` calls to add cross-cutting behavior (retries, logging, fallback, gating, sandboxed tools) without touching the flow that uses it. Middleware composes, so a single `Generate` call can stack several behaviors. Built-ins ship in the `plugins/middleware` package; custom middleware is just a Go struct with two methods. +- [The mental model](#the-mental-model): Lines 20-44 +- [When each hook fires](#when-each-hook-fires): Lines 46-56 +- [Composition order](#composition-order): Lines 58-70 +- [Per-call state](#per-call-state): Lines 72-120 +- [Plugin-level state](#plugin-level-state): Lines 122-157 +- [Composition with WithUse](#composition-with-withuse): Lines 159-172 +- [Inline middleware](#inline-middleware): Lines 174-189 +- [Application-owned middleware](#application-owned-middleware): Lines 191-202 +- [Built-in middleware](#built-in-middleware): Lines 204-374 +- [Practical patterns](#practical-patterns): Lines 376-451 +- [Migration note](#migration-note): Lines 453-455 +- [See also](#see-also): Lines 457-461 + +--- + ## The mental model A middleware is a config struct that implements two methods: diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/observability.md b/plugins/genkit/agent/skills/developing-genkit-go/references/observability.md index ab589952..ef37973d 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/observability.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/observability.md @@ -3,6 +3,13 @@ > Pluggable instrumentation, `otelgenai`, and the reflection API settings > require Genkit Go v1.14.0+. +- [How tracing works](#how-tracing-works): Lines 13-26 +- [Export OpenTelemetry GenAI semantic conventions (otelgenai)](#export-opentelemetry-genai-semantic-conventions-otelgenai): Lines 28-70 +- [Custom instrumentation](#custom-instrumentation): Lines 72-92 +- [Reflection API outside dev](#reflection-api-outside-dev): Lines 94-115 + +--- + ## How tracing works - **In dev** (`genkit start`), Genkit posts spans straight to the Developer UI's diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/prompts.md b/plugins/genkit/agent/skills/developing-genkit-go/references/prompts.md index d11ec0cf..c505f3e1 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/prompts.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/prompts.md @@ -1,5 +1,12 @@ # Prompts +- [DefinePrompt](#defineprompt): Lines 10-53 +- [DefineDataPrompt (Typed Input/Output)](#definedataprompt-typed-inputoutput): Lines 55-94 +- [.prompt Files (Dotprompt)](#prompt-files-dotprompt): Lines 96-276 +- [Schemas](#schemas): Lines 278-320 + +--- + ## DefinePrompt Define a reusable prompt in code with a default model and template. diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/providers.md b/plugins/genkit/agent/skills/developing-genkit-go/references/providers.md index 4773809c..1ad06c5d 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/providers.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/providers.md @@ -1,5 +1,14 @@ # Model Providers +- [Google AI (Gemini)](#google-ai-gemini): Lines 12-37 +- [Vertex AI](#vertex-ai): Lines 39-55 +- [Anthropic (Claude)](#anthropic-claude): Lines 57-86 +- [OpenAI-Compatible (compat_oai)](#openai-compatible-compat_oai): Lines 88-112 +- [Ollama (Local Models)](#ollama-local-models): Lines 114-141 +- [Multiple Providers](#multiple-providers): Lines 143-166 + +--- + ## Google AI (Gemini) ```go diff --git a/plugins/genkit/agent/skills/developing-genkit-go/references/tools.md b/plugins/genkit/agent/skills/developing-genkit-go/references/tools.md index f70cc02f..78b9d00b 100644 --- a/plugins/genkit/agent/skills/developing-genkit-go/references/tools.md +++ b/plugins/genkit/agent/skills/developing-genkit-go/references/tools.md @@ -1,5 +1,12 @@ # Tools +- [DefineTool](#definetool): Lines 10-31 +- [Using Tools in Generation](#using-tools-in-generation): Lines 33-61 +- [Tool runtime helpers (ai/tool)](#tool-runtime-helpers-aitool): Lines 63-116 +- [Tool interrupts (DefineResumableTool)](#tool-interrupts-defineresumabletool): Lines 118-257 + +--- + ## DefineTool Define a tool the model can call during generation. diff --git a/plugins/genkit/agent/skills/developing-genkit-js/SKILL.md b/plugins/genkit/agent/skills/developing-genkit-js/SKILL.md index 1ff7a308..a4f9c9f9 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/SKILL.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/SKILL.md @@ -1,9 +1,10 @@ --- name: developing-genkit-js -description: Develop AI-powered applications using Genkit in Node.js/TypeScript. - Use when the user asks about Genkit, AI agents, flows, or tools in - JavaScript/TypeScript, or when encountering Genkit errors, validation issues, - type errors, or API problems. +description: Provides guidance on developing AI-powered applications using + Genkit in Node.js/TypeScript. Use when the user asks about Genkit, AI agents, + flows, or tools in JavaScript/TypeScript, or when encountering Genkit errors, + validation issues, type errors, or API problems. Don't use for Genkit in + Python, Go, or Dart. metadata: version: 1.0.0 category: AiAndMachineLearning @@ -197,7 +198,7 @@ See [CLI Reference](references/docs-and-cli.md) for more commands, and `genkit - - [Docs & CLI Reference](references/docs-and-cli.md): Documentation search, CLI tasks, and workflows. - [Common Errors](references/common-errors.md): Critical "gotchas", migration guide, and troubleshooting. - [Setup Guide](references/setup.md): Manual setup instructions for new projects. -- [Examples](references/examples.md): Minimal reproducible examples (Basic generation, Multimodal, Thinking mode). +- [Examples](references/examples.md): Minimal reproducible examples (Basic generation, Multimodal, Thinking mode, Embeddings). - [Agents (Beta)](references/agents.md): Agent basics, serving, and client-managed state. Deeper topics: [sessions](references/agents-sessions.md), [human-in-the-loop](references/agents-human-in-the-loop.md), [branching](references/agents-branching.md), [background agents](references/agents-background.md), [state](references/agents-state.md), [artifacts](references/agents-artifacts.md), [multi-agent](references/agents-multi-agent.md), [custom agents](references/agents-custom.md), [deployment](references/agents-deployment.md). - [Middleware](references/middleware.md): using middleware and the `@genkit-ai/middleware` package. See also [building custom middleware](references/middleware-custom.md). - [Observability](references/observability.md): `@genkit-ai/otel` (OpenTelemetry GenAI semantic conventions), custom `Instrumentation`, and the reflection API outside dev. diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/a2ui.md b/plugins/genkit/agent/skills/developing-genkit-js/references/a2ui.md index f88a1361..aaffc5af 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/a2ui.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/a2ui.md @@ -13,6 +13,16 @@ An A2UI-enabled agent streams more than prose. It streams interactive UI as the model responds. The entire server-side integration is a single model middleware: add `a2ui()` to an agent's `use` array and nothing else changes. +- [Install](#install): Lines 26-40 +- [Server: add the a2ui() middleware](#server-add-the-a2ui-middleware): Lines 42-105 +- [Client: render surfaces](#client-render-surfaces): Lines 107-160 +- [Handling user actions](#handling-user-actions): Lines 162-220 +- [Custom catalogs](#custom-catalogs): Lines 223-273 +- [Security and the trust boundary](#security-and-the-trust-boundary): Lines 275-294 +- [How it works](#how-it-works): Lines 296-315 + +--- + ## Install ```bash diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-artifacts.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-artifacts.md index 0daedcaa..6db880d0 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-artifacts.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-artifacts.md @@ -9,6 +9,14 @@ session — files, reports, code, etc. They live in the session (deduplicated by name) and are returned in `res.artifacts` / tracked on the client's `chat.artifacts`. +- [Give the model artifact tools](#give-the-model-artifact-tools): Lines 20-52 +- [The Artifact shape](#the-artifact-shape): Lines 54-67 +- [Programmatic access (inside tools / custom agents)](#programmatic-access-inside-tools--custom-agents): Lines 69-83 +- [Reading artifacts on the client](#reading-artifacts-on-the-client): Lines 85-98 +- [Sharing artifacts across agents](#sharing-artifacts-across-agents): Lines 100-113 + +--- + ## Give the model artifact tools The `artifacts()` middleware (see [using middleware](middleware.md)) adds diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-background.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-background.md index 0b30957c..ccf39970 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-background.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-background.md @@ -9,6 +9,14 @@ and returns a `snapshotId` **immediately**, keeps processing, then updates the snapshot to a terminal status (`completed` / `failed` / `aborted` / `expired`). The client polls for completion and can abort. +- [Define the agent (store required)](#define-the-agent-store-required): Lines 20-53 +- [Server-side: detach + wait](#server-side-detach--wait): Lines 55-73 +- [Resume a failed or aborted task](#resume-a-failed-or-aborted-task): Lines 75-89 +- [Client-side: detach + poll + abort](#client-side-detach--poll--abort): Lines 91-131 +- [Status values](#status-values): Lines 133-151 + +--- + ## Define the agent (store required) ```ts diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-custom.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-custom.md index 666e7f13..3a964c42 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-custom.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-custom.md @@ -8,6 +8,13 @@ the turn** — multiple sequential model calls, custom logic between them, manua message/state management, or custom progress streaming — use `ai.defineCustomAgent`. You provide the handler that runs the turn. +- [When to use it](#when-to-use-it): Lines 18-28 +- [Signature](#signature): Lines 30-54 +- [Example: multi-step research agent](#example-multi-step-research-agent): Lines 56-127 +- [Custom status streaming](#custom-status-streaming): Lines 129-148 + +--- + ## When to use it Reach for `defineCustomAgent` when a turn needs to: diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-deployment.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-deployment.md index 9f5d78ce..a5183d55 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-deployment.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-deployment.md @@ -15,6 +15,14 @@ expose two companion actions: These paths match the `remoteAgent` client defaults (`${url}/getSnapshot`, `${url}/abort`), so a client only needs the base `url`. +- [A reusable exposeAgent helper](#a-reusable-exposeagent-helper): Lines 26-75 +- [CORS for browser clients](#cors-for-browser-clients): Lines 77-98 +- [Serving a static web UI (optional)](#serving-a-static-web-ui-optional): Lines 100-118 +- [Registering agents/flows](#registering-agentsflows): Lines 120-137 +- [Other host frameworks](#other-host-frameworks): Lines 139-198 + +--- + ## A reusable `exposeAgent` helper When serving several agents, a small helper keeps registration consistent. diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-human-in-the-loop.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-human-in-the-loop.md index 74e69375..51f2786c 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-human-in-the-loop.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-human-in-the-loop.md @@ -18,6 +18,13 @@ snapshot does it; without one the client round-trips the state blob (the Flow: `chat.send(...)` → response has `res.interrupts` → collect human input → `chat.resume({ respond: [...] })`. +- [Define an interrupt](#define-an-interrupt): Lines 28-72 +- [Detect and resume (server-side)](#detect-and-resume-server-side): Lines 74-122 +- [Client-side (browser) interrupts](#client-side-browser-interrupts): Lines 124-163 +- [Notes & gotchas](#notes--gotchas): Lines 165-182 + +--- + ## Define an interrupt Define it like a tool (with `inputSchema`/`outputSchema`) and add it to the diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-multi-agent.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-multi-agent.md index 7e0dc6cf..afcf630d 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-multi-agent.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-multi-agent.md @@ -10,6 +10,16 @@ appends a `<sub-agents>` block to the orchestrator's system prompt, and — when the model calls a delegation tool — runs the sub-agent and returns its response as the tool result. +- [1. Define the sub-agents](#1-define-the-sub-agents): Lines 23-51 +- [2. Wire up the orchestrator](#2-wire-up-the-orchestrator): Lines 53-95 +- [agents() options](#agents-options): Lines 97-112 +- [Background delegation (async: true)](#background-delegation-async-true): Lines 114-154 +- [Continuing a task (continue_task)](#continuing-a-task-continue_task): Lines 156-173 +- [Sharing artifacts between agents](#sharing-artifacts-between-agents): Lines 175-186 +- [Other middleware](#other-middleware): Lines 188-212 + +--- + ## 1. Define the sub-agents Give each sub-agent a `description` — it's auto-discovered and shown to the diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-sessions.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-sessions.md index 8a170e2c..d011bc97 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-sessions.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-sessions.md @@ -10,6 +10,14 @@ conversation state forward. A store also enables [branching](agents-branching.md and [background execution](agents-background.md). (Interrupts work with or without a store — see [human-in-the-loop](agents-human-in-the-loop.md).) +- [Pick a store](#pick-a-store): Lines 21-66 +- [Typed session state](#typed-session-state): Lines 68-90 +- [Interrupts (human-in-the-loop)](#interrupts-human-in-the-loop): Lines 92-97 +- [Firestore session store (scalable, Beta)](#firestore-session-store-scalable-beta): Lines 99-132 +- [Implementing a custom SessionStore](#implementing-a-custom-sessionstore): Lines 134-155 + +--- + ## Pick a store ```ts diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-state.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-state.md index f75f0e93..02dcb681 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents-state.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents-state.md @@ -7,6 +7,13 @@ own structured data (a task list, a workflow status, counters, etc.). Tools read and mutate it during a turn, and it is automatically synced to the [`remoteAgent`](agents.md#consume-an-agent-from-a-client-remoteagent) client. +- [Declare the state shape](#declare-the-state-shape): Lines 17-45 +- [Read & mutate state inside tools](#read--mutate-state-inside-tools): Lines 47-83 +- [Seed and read state (server-side)](#seed-and-read-state-server-side): Lines 85-102 +- [Auto-sync to the remoteAgent client](#auto-sync-to-the-remoteagent-client): Lines 104-140 + +--- + ## Declare the state shape Pass a `stateSchema` (Zod) to `defineAgent`. `State` is inferred from it and diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/agents.md b/plugins/genkit/agent/skills/developing-genkit-js/references/agents.md index 3a821dbc..022d7049 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/agents.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/agents.md @@ -29,6 +29,17 @@ Progressive disclosure — read the file for the level you need: - [Advanced custom agents](agents-custom.md): `defineCustomAgent` for full turn control. - [Deploying agents](agents-deployment.md): serving multiple agents over HTTP, CORS, web UI, other frameworks. +- [Setup](#setup): Lines 43-54 +- [Define an agent](#define-an-agent): Lines 56-93 +- [Agents and middleware go hand in hand](#agents-and-middleware-go-hand-in-hand): Lines 95-134 +- [Chat with an agent (server-side)](#chat-with-an-agent-server-side): Lines 136-158 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 160-176 +- [Serve an agent over HTTP](#serve-an-agent-over-http): Lines 178-210 +- [Consume an agent from a client (remoteAgent)](#consume-an-agent-from-a-client-remoteagent): Lines 212-240 +- [Client-managed state (no server store)](#client-managed-state-no-server-store): Lines 242-289 + +--- + ## Setup ```ts diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/common-errors.md b/plugins/genkit/agent/skills/developing-genkit-js/references/common-errors.md index c3efd281..0b9c47df 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/common-errors.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/common-errors.md @@ -1,5 +1,15 @@ # Common Errors & Pitfalls +- [When Typecheck Fails](#when-typecheck-fails): Lines 13-15 +- [Genkit v1.x vs Pre-1.0 Migration](#genkit-v1x-vs-pre-10-migration): Lines 17-113 +- [Zod & Schema Errors](#zod--schema-errors): Lines 115-121 +- [Tool Usage](#tool-usage): Lines 123-126 +- [Handling Failures in Tool Loops](#handling-failures-in-tool-loops): Lines 128-159 +- [Multimodal & Image Generation](#multimodal--image-generation): Lines 161-169 +- [Audio & Speech Generation](#audio--speech-generation): Lines 171-176 + +--- + ## When Typecheck Fails **Before searching source code or docs**, check the sections below. Many type errors are caused by deprecated APIs or incorrect imports. @@ -98,6 +108,7 @@ Genkit v1.x introduced significant API changes. This section covers critical syn - **Incorrect (Pre-1.0)**: Importing `defineFlow` globally. ```ts import { defineFlow } from "@genkit-ai/flow"; // INCORRECT + ``` You should never import `@genkit-ai/flow`, `@genkit-ai/ai` or `@genkit-ai/core` packages directly. diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/dotprompt.md b/plugins/genkit/agent/skills/developing-genkit-js/references/dotprompt.md index 1c728ec4..bfb831a6 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/dotprompt.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/dotprompt.md @@ -1,5 +1,18 @@ # Dotprompt (.prompt files) — Genkit JS +- [What it is](#what-it-is): Lines 16-20 +- [Where files live](#where-files-live): Lines 22-35 +- [File format](#file-format): Lines 37-61 +- [Loading and calling a prompt](#loading-and-calling-a-prompt): Lines 63-100 +- [Registering named schemas](#registering-named-schemas): Lines 102-118 +- [Variants](#variants): Lines 120-127 +- [Partials](#partials): Lines 129-153 +- [Helpers](#helpers): Lines 155-165 +- [Tools, tool-loop control, and middleware](#tools-tool-loop-control-and-middleware): Lines 167-213 +- [Relationship to agents](#relationship-to-agents): Lines 215-219 + +--- + ## What it is `.prompt` files combine YAML frontmatter (model, config, schemas, tools, diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/examples.md b/plugins/genkit/agent/skills/developing-genkit-js/references/examples.md index 64e0ce88..57f13048 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/examples.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/examples.md @@ -7,6 +7,15 @@ This reference contains minimal, reproducible examples (MREs) for common Genkit > 2. Install and configure the plugin. > 3. Swap the model reference in the code. +- [Basic Text Generation](#basic-text-generation): Lines 19-33 +- [Structured Output](#structured-output): Lines 35-56 +- [Streaming](#streaming): Lines 58-73 +- [Advanced Configuration](#advanced-configuration): Lines 75-112 +- [Multimodal Generation](#multimodal-generation): Lines 114-174 +- [Embeddings](#embeddings): Lines 176-210 + +--- + ## Basic Text Generation ```ts @@ -163,3 +172,39 @@ if (media?.url) { Older preview TTS models (`gemini-2.5-*-preview-tts`, `gemini-3.1-flash-tts-preview`) are deprecated and return raw PCM that needs conversion to WAV. Run `genkit docs:search "speech audio"` for conversion code. + +## Embeddings + +```ts +const embedder = googleAI.embedder('gemini-embedding-001'); + +const embeddings = await ai.embed({ embedder, content: 'The sky is blue.' }); +const vector = embeddings[0].embedding; // number[] + +const batch = await ai.embedMany({ + embedder, + content: ['The sky is blue.', 'Grass is green.'], +}); + +// Optional tuning (default is 3072 dimensions) +await ai.embed({ + embedder, + content: 'The sky is blue.', + options: { outputDimensionality: 768, taskType: 'RETRIEVAL_DOCUMENT' }, +}); +``` + +Common embedders: `gemini-embedding-001` (text), `gemini-embedding-2` +(multimodal: text, image, video). Plain string ids +(`embedder: 'googleai/gemini-embedding-001'`) also work. + +Multimodal input is passed as a document: + +```ts +const imageEmbeddings = await ai.embed({ + embedder: googleAI.embedder('gemini-embedding-2'), + content: { + content: [{ media: { url: 'data:image/png;base64,...', contentType: 'image/png' } }], + }, +}); +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/middleware.md b/plugins/genkit/agent/skills/developing-genkit-js/references/middleware.md index 1620a4b6..4b85bdad 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/middleware.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/middleware.md @@ -28,6 +28,12 @@ a reference for `use: [...]`. The `@genkit-ai/middleware` package ships a set of ready-made ones (covered below); to write your own see [building custom middleware](middleware-custom.md). +- [Register middleware as a plugin](#register-middleware-as-a-plugin): Lines 37-58 +- [The @genkit-ai/middleware package](#the-genkit-aimiddleware-package): Lines 60-181 +- [Built-in model middleware (in core)](#built-in-model-middleware-in-core): Lines 183-205 + +--- + ## Register middleware as a plugin Register each middleware with Genkit via its `.plugin()` method. This is the diff --git a/plugins/genkit/agent/skills/developing-genkit-js/references/observability.md b/plugins/genkit/agent/skills/developing-genkit-js/references/observability.md index 668996d0..ee470d08 100644 --- a/plugins/genkit/agent/skills/developing-genkit-js/references/observability.md +++ b/plugins/genkit/agent/skills/developing-genkit-js/references/observability.md @@ -3,6 +3,13 @@ > Pluggable instrumentation, `@genkit-ai/otel`, and the reflection API settings > require `genkit >= 1.43.0`. +- [How tracing works](#how-tracing-works): Lines 13-21 +- [Export OpenTelemetry GenAI semantic conventions (@genkit-ai/otel)](#export-opentelemetry-genai-semantic-conventions-genkit-aiotel): Lines 23-81 +- [Custom instrumentation](#custom-instrumentation): Lines 83-119 +- [Reflection API outside dev](#reflection-api-outside-dev): Lines 121-142 + +--- + ## How tracing works - **In dev** (`genkit start`), Genkit posts spans straight to the Developer UI's diff --git a/plugins/genkit/agent/skills/developing-genkit-python/SKILL.md b/plugins/genkit/agent/skills/developing-genkit-python/SKILL.md index 04b66db3..db96ccb3 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/SKILL.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/SKILL.md @@ -1,24 +1,26 @@ --- name: developing-genkit-python -description: Develop AI-powered applications using Genkit in Python. Use when - the user asks about Genkit, AI agents, flows, or tools in Python, or when - encountering Genkit errors, import issues, or API problems. +description: Provides guidance on developing AI-powered applications using + Genkit Python (0.13) and migrating Genkit Python 0.12 projects to 0.13. Use + when building AI agents, flows, tools, structured outputs, or FastAPI routes + in Python, upgrading from Genkit 0.12, or debugging Genkit imports, config + errors, or traces. Don't use for Genkit in JavaScript/TypeScript, Go, or Dart. metadata: version: 1.0.0 category: AiAndMachineLearning --- # Genkit Python -Build AI features in Python — generate, stream, tools, flows, and multi-turn -agents — with one SDK. +Build AI features in Python — generation, structured output, streaming, tools, flows, and multi-turn agents — with Genkit Python 0.13. + +- **New project?** See [Setup](references/setup.md) and [Examples](references/examples.md). +- **Upgrading from 0.12 or fixing broken 0.12 imports/signatures?** Read [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md) first. ## Prerequisites - Python **3.10+** and **`uv`** ([install](https://docs.astral.sh/uv/getting-started/installation/)) - Genkit CLI: `npm install -g genkit-cli` if `genkit --version` is missing -New app? [Setup](references/setup.md). Patterns? [Examples](references/examples.md). - ## Hello World ```python @@ -27,26 +29,50 @@ from genkit_google_genai import GoogleAI ai = Genkit( plugins=[GoogleAI()], - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), ) -async def main(): - response = await ai.generate(prompt='Tell me a joke about Python.') + +async def main() -> None: + response = await ai.generate(prompt='Suggest one seasonal pasta dish.') + if response.error: + raise RuntimeError(response.error.message) print(response.text) + # => Lemony tagliatelle with asparagus, sweet peas, and herb ricotta + if __name__ == '__main__': ai.run_main(main()) ``` -## Agents (Beta) +`ai.generate` does not raise when the provider rejects or fails a call (`INVALID_ARGUMENT`, `UNAVAILABLE`, rate limits). It returns `response.error` (`Failure` with `.status` and `.message`), `finish_reason == 'failed'`, and empty text. Problems caught before the call raise: `GoogleAI()` raises `ValueError` without `GEMINI_API_KEY`; an unknown model, invalid key, or unknown config field raises `GenkitError`. See [Common Errors](references/common-errors.md). -Multi-turn chats with history, typed state, human approval, branching, and -background work. Start here: [Agents](references/agents.md). +## Agents (Experimental) + +Multi-turn chats with history, typed state, human approval, branching, and background work live under `genkit.exp` and `genkit.exp.agent`. Start here: [Agents](references/agents.md). ```python +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore +from genkit_google_genai import GoogleAI + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) +agent = ai.define_agent( + name='concierge', + system='You are a concise hotel concierge.', + store=InMemorySessionStore(), +) + chat = agent.chat() -res = await chat.send('Hello') # AgentResponse -turn = chat.send_stream('Hello') # AgentTurn — .stream / .response +res = await chat.send('Recommend a quiet dinner spot.') # AgentResponse +print(res.text) +# => Try Osteria Lume on 5th; it has booths and low music. + +turn = chat.send_stream('Is it open past 10pm?') # AgentTurn +async for chunk in turn.stream: + if chunk.text: + print(chunk.text, end='') +final = await turn.response # AgentResponse ``` More: [sessions](references/agents-sessions.md) · @@ -60,66 +86,79 @@ More: [sessions](references/agents-sessions.md) · ## Imports -- Google AI: `from genkit_google_genai import GoogleAI` -- Agents: `from genkit.agent import InMemorySessionStore, ...` -- Middleware: `from genkit_middleware import Middleware, ToolApproval, ...` -- FastAPI: `from genkit_fastapi import serve_agent, serve_flow` -- Evals: `from genkit_evaluators import register_genkit_evaluators` +Every symbol lives in one public module (never import private `_` submodules; see [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md) if a 0.12 symbol moved): + +- **Core app**: `from genkit import Genkit, FlowContext, ToolContext, Message, Part, Role, Media, Document, MultipartToolResponse, PublicError, GenkitError, Failure, ErrorReason` +- **Google GenAI & Vertex AI**: `from genkit_google_genai import GoogleAI, VertexAI, GeminiConfig, GeminiImageConfig, GeminiTtsConfig` +- **Other providers**: `from genkit_vertexai import ModelGarden` · `from genkit_anthropic import Anthropic, AnthropicConfig` · `from genkit_openai import OpenAI, OpenAIConfig` · `from genkit_ollama import Ollama, OllamaConfig` · `from genkit_amazon_bedrock import Bedrock, BedrockConfig` +- **Built-in middleware**: `from genkit_middleware import Middleware, Retry, Fallback, ToolApproval, Skills, Filesystem` (plus `from genkit_middleware.exp import Artifacts` for agent sessions) +- **Writing custom middleware**: `from genkit.middleware import Middleware, MiddlewareContext` +- **FastAPI & HTTP**: `from genkit_fastapi import serve_flow, genkit_fastapi_handler, handle_genkit_request` · `from genkit_fastapi.exp import serve_agent` · `from genkit.web import RequestData, to_sse, error_body, error_status` +- **Agents (experimental)**: `from genkit.exp import Genkit, current_session` · `from genkit.exp.agent import Agent, Session, InMemorySessionStore, FileSessionStore, remote_agent` +- **Telemetry**: `from genkit_google_cloud import enable_google_cloud_telemetry` · `from genkit.telemetry import configure_instrumentation` · `from genkit_otel import GenAiInstrumentation` +- **MCP (Client & Host)**: `from genkit_mcp import define_mcp_client, define_mcp_host, McpClient, McpHost` +- **Testing & Mocks**: `from genkit.testing import define_echo_model, define_scripted_model, EchoModel, ScriptedModel` +- **Evaluators (0.13 only; cut from 1.0, additive later)**: `from genkit_evaluators import register_genkit_evaluators` · `from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score, ScoreDetails, ScoreStatus` ## Workflow -1. **Agent or flow?** If the task is conversational, multi-turn, or described as - "an agent", "assistant", or "chatbot", build it with `ai.define_agent` (see - [Agents](references/agents.md)) rather than hand-rolling a `generate` + tools - loop inside a flow. Reach for a plain flow only for single-shot, stateless - generation. -2. Set **`GEMINI_API_KEY`**. Use prefixed model ids (`googleai/gemini-flash-latest`). -3. Enter via **`ai.run_main(main())`** for Genkit apps (especially under - `genkit start`). See [Common Errors](references/common-errors.md). -4. Run with [Dev Workflow](references/dev-workflow.md) (`genkit start` + Dev UI). -5. Verify with traces, not a blind run. Running the app directly (`uv run`) - does **not** capture dev traces. See [Genkit CLI](#genkit-cli-recommended) - for how to run your app and capture traces. -6. Stuck? [Common Errors](references/common-errors.md) first. +1. **Agent or flow?** Use `ai.define_agent` (on `genkit.exp.Genkit`, see [Agents](references/agents.md)) for multi-turn chats, state, or human-in-the-loop approval instead of hand-rolling a message loop inside a flow. Use `@ai.flow()` on `genkit.Genkit` for stateless request/response or streaming pipelines. +2. **Configure credentials & model refs.** Set **`GEMINI_API_KEY`**. Prefer typed refs (`GoogleAI.gemini('gemini-flash-latest')`) and typed flat configs (`GeminiConfig(...)`) so your type checker validates fields before runtime. Prefixed strings (`'googleai/gemini-flash-latest'`) also work. +3. **Enter via `ai.run_main(main())`** for standalone scripts and CLI apps so the process stays up for the Dev UI under `genkit start`. FastAPI apps skip `run_main`; uvicorn keeps the process alive: `genkit start -- uv run uvicorn main:app --reload`. +4. **Run with [Dev Workflow](references/dev-workflow.md)** (`genkit start` + Dev UI) and type-check with `pyright`, `mypy`, or `ty`. +5. **Verify with traces**, not a blind `uv run`. See [Genkit CLI](#genkit-cli-recommended). +6. **Stuck?** Check [Gotchas](#gotchas), [Common Errors](references/common-errors.md), or [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md). + +## Gotchas + +- **Agents need `genkit.exp`**: `from genkit.exp import Genkit`. Stable `genkit.Genkit` has no `define_agent`. +- **Tools are async**: `@ai.tool()` raises `TypeError` on a plain `def`; use `async def`. +- **Check `response.error`**: provider failures come back on the response, not as exceptions (see [Hello World](#hello-world)). +- **Gemini thinking levels**: `GeminiConfig(thinking_level='MINIMAL')` fails on `gemini-flash-latest`; use `'LOW'`, `'MEDIUM'`, or `'HIGH'`. +- **Vertex AI model IDs**: Vertex does not serve `-latest` aliases. Pass a pinned model ID to `VertexAI.gemini(...)`. +- **Logging**: `genkit.get_logger` is structlog-based. `enable_google_cloud_telemetry()` injects trace context for GCP log-trace correlation. +- **Cloud telemetry auth**: call `enable_google_cloud_telemetry()` with Application Default Credentials or explicit `credentials=` (service account dict or `Credentials` instance). ## Genkit CLI (recommended) -`genkit start` unintrusively wraps any Python program that uses the Genkit library, running it unchanged while capturing traces from every Genkit action so you can prove tools were actually called and inspect model I/O from the terminal, even for headless checks. It forwards stdio, so interactive CLI tools that rely on stdin/stdout work without issues. Running the app directly (`uv run`) skips trace capture, so you're debugging blind. +`genkit start` wraps any Python program that uses Genkit, capturing traces from every action while forwarding stdio cleanly. Running `uv run` directly skips trace capture. -**Primary pattern (default):** prefix `genkit start --` to your normal run command. This collects telemetry from any Genkit code your program runs, whether triggered from the dev UI, your own web server/web UI, or a plain script: +**Primary pattern (persistent server + Dev UI):** ```bash genkit start -- uv run src/main.py -genkit start --noui -- uv run src/main.py # same, without the Dev UI (still a persistent server) +genkit start --noui -- uv run src/main.py # persistent runtime without opening the Dev UI ``` -`genkit start` runs until you stop it with Ctrl+C. That is expected and correct for the common cases: a server your web/mobile app calls, or an interactive CLI you exit yourself. `--noui` only drops the Dev UI; it is **not** a one-shot command and will not exit on its own. Do **not** use `genkit start` as a blocking step in automated/non-interactive contexts; use `flow:run` (below) for that. +`genkit start` runs until stopped with Ctrl+C (even with `--noui`). Do **not** run `genkit start` as a blocking step in automated/non-interactive scripts; use `flow:run` instead. -**Non-interactive use (agents/CI):** add the global `--non-interactive` flag before `--` so the CLI uses defaults and never blocks on a prompt (e.g. the first-run analytics notice): `genkit start --non-interactive -- uv run src/main.py` (works with `flow:run` too). +**Non-interactive use (agents/CI):** pass `--non-interactive` before `--` so the CLI never blocks on prompts: +```bash +genkit start --non-interactive -- uv run src/main.py +``` -**Run a flow (`flow:run`):** invoke a specific flow by name from the CLI. Append your run command after `--` to spin up the runtime just for this run (the command runs as-is to register your flows): +**One-shot flow execution (`flow:run`):** spins up the runtime, invokes one `@ai.flow()` by name, prints a `Trace ID`, and exits: ```bash genkit flow:run myFlow '{"data": "input"}' -- uv run src/main.py ``` -This is **self-terminating**: it runs the flow once, prints a `Trace ID`, then exits, so it's the right choice for a quick, non-interactive check (unlike `genkit start`). Note: `flow:run` runs **flows** (`@ai.flow()`), not agents; you can't `flow:run` an agent (`ai.define_agent`) directly. To exercise an agent from the CLI, wrap one turn in a throwaway flow and run that (see [Agents](references/agents.md)). +Note: `flow:run` invokes flows (`@ai.flow()`), not agents (`ai.define_agent`). Wrap a single agent turn in a helper flow to exercise it via `flow:run` (see [Agents](references/agents.md)). -**Debugging with traces:** the fastest way to see prompts, model inputs/outputs, tool calls, latencies, and errors. Inspect from the terminal after any run under `genkit start`: +**Inspecting traces:** ```bash -genkit trace:list # find recent trace IDs -genkit trace:get <traceId> # full trace details (inputs, outputs, tool calls, errors) -genkit trace:get <traceId> --format json # machine-readable JSON, safe to pipe into jq or other parsers +genkit trace:list # list recent trace IDs +genkit trace:get <traceId> # human-readable trace inspection +genkit trace:get <traceId> --format json # machine-readable JSON for jq / scripts ``` +Always pass `--format json` when piping into `jq` or other parsers. -For machine-readable output, pass `--format json` to get clean JSON you can pipe into `jq` or other parsers. The **default** output is human-oriented (banner/log lines, possible truncation on large traces), so don't pipe that form directly; use `--format json`, grep, or the Dev UI trace viewer. - - -See [Dev Workflow](references/dev-workflow.md) for the full checklist and Dev UI walkthrough. +See [Dev Workflow](references/dev-workflow.md) for the full pre-run checklist and Dev UI walkthrough. ## References -- [Examples](references/examples.md): Structured output, streaming, flows, tools, embeddings. -- [Setup](references/setup.md): New project bootstrap and plugins. -- [Common Errors](references/common-errors.md): Read first when something breaks. -- [FastAPI](references/fastapi.md): HTTP, `genkit_fastapi_handler`, parallel flows. -- [Dotprompt](references/dotprompt.md): `.prompt` files and helpers. -- [Evals](references/evals.md): Evaluators and datasets. -- [Dev Workflow](references/dev-workflow.md): `genkit start`, Dev UI, checklist. -- [Agents (Beta)](references/agents.md): Multi-turn API. +- [Migrating from 0.12 to 0.13](references/migrating-from-0.12.md): Deterministic 0.12 -> 0.13 renames, config flattening, `PublicError`, and `exp` moves. +- [Examples](references/examples.md): Structured output, streaming, media parts, flows, tools, interrupts, cancellation, embeddings. +- [Setup](references/setup.md): Project bootstrap with `uv`, version alignment, and provider plugins. +- [Common Errors](references/common-errors.md): Fast fixes for 0.13 validation errors, HTTP 500 redaction, streaming, and sessions. +- [FastAPI](references/fastapi.md): `serve_flow`, `serve_agent`, `PublicError`, SSE streaming, and parallel flows. +- [Dotprompt](references/dotprompt.md): `.prompt` files, schema validation, variants, partials, and helpers. +- [Evals](references/evals.md): 0.13 evaluator syntax (deprecated; removed in 1.0). +- [Dev Workflow](references/dev-workflow.md): `genkit start`, Dev UI, and trace verification checklist. +- [Agents (Experimental)](references/agents.md): Multi-turn stateful agents (`genkit.exp`). diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-artifacts.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-artifacts.md index 3b5e66fa..e0886a6d 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-artifacts.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-artifacts.md @@ -1,64 +1,98 @@ -# Artifacts (Beta) +# Agent Artifacts (Experimental) -> See [agents.md](agents.md). +> See [agents.md](agents.md) · [sessions](agents-sessions.md). -Named deliverables on the session — reports, files, code — available as -`chat.artifacts` and `res.artifacts`. +Artifacts are named session deliverables — markdown reports, SQL migrations, generated files — persisted in `SessionState.artifacts` and exposed as `chat.artifacts`, `res.artifacts`, and streamed `chunk.artifact`. Writing an artifact whose `name` already exists replaces the previous version in place. -## Middleware +## `Artifacts` middleware (`genkit_middleware.exp`) -`Artifacts()` gives the model `write_artifact` and `read_artifact`. Writing the -same name replaces the previous version. +Import `Artifacts` from `genkit_middleware.exp` (`Middleware()` does not register it). On every model call, `Artifacts()` appends an `<artifacts>` listing of the session's current artifacts (name, size, source) to the system prompt, and gives the model `write_artifact` and `read_artifact` tools. Pass `Artifacts(readonly=True)` to expose only `read_artifact`. Outside an agent session the tools return an error result instead of raising. -Store-backed agents cannot seed with `chat(artifacts=...)`. Have the model -write on the first turn, or call `add_artifacts` from a custom agent. +On a store-backed agent, `agent.chat(artifacts=...)` raises `AgentInitError` (`FAILED_PRECONDITION`); have the model write artifacts on a turn, or write them from a tool or [custom agent](agents-custom.md). Store-less agents can seed `artifacts=` ([state](agents-state.md)). ```python from genkit_google_genai import GoogleAI -from genkit_middleware import Artifacts, Middleware +from genkit_middleware.exp import Artifacts -from genkit import Genkit -from genkit.agent import InMemorySessionStore +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore -ai = Genkit(plugins=[GoogleAI(), Middleware()]) +ai = Genkit(plugins=[GoogleAI()]) agent = ai.define_agent( name='workspaceAgent', - model='googleai/gemini-flash-latest', - system='Use write_artifact for files. Use read_artifact to review them.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Use write_artifact to save deliverables and read_artifact to inspect them.', use=[Artifacts()], store=InMemorySessionStore(), ) -chat = agent.chat() -await chat.send('Write poem.txt with a short poem about Python agents.') -print([a.name for a in chat.artifacts]) + +async def main() -> None: + chat = agent.chat() + + # 1. Stream the turn and observe artifacts as they are written + turn = chat.send_stream('Write schema.sql for a multi-tenant invoices table.') + async for chunk in turn.stream: + if chunk.artifact is not None: + print('wrote artifact:', chunk.artifact.name) + # => wrote artifact: schema.sql + await turn.response + + # 2. Inspect persisted session artifacts + print([a.name for a in chat.artifacts]) + # => ['schema.sql'] + + +if __name__ == '__main__': + ai.run_main(main()) ``` -## From a custom agent +## Writing artifacts from tools and custom agents -Pass a list: +Inside an `@ai.tool()`, get the active session with `current_session()` from `genkit.exp` (`None` outside an agent turn). Inside a custom agent, call the same method on the `SessionRunner`. `add_artifacts` takes a `list[Artifact]`, replaces by name, and emits one `chunk.artifact` per artifact to streaming callers: ```python -from genkit import Part, TextPart -from genkit.agent import Artifact +from genkit import Part +from genkit.exp import current_session +from genkit.exp.agent import Artifact + + +@ai.tool(name='exportAuditReport', description='Save an audit report as a session artifact.') +async def export_audit_report(markdown_body: str) -> dict[str, object]: + sess = current_session() + if sess is None: + return {'ok': False, 'error': 'no active agent session'} + + await sess.add_artifacts([ + Artifact( + name='audit-report.md', + parts=[Part.from_text(markdown_body)], + metadata={'source': 'exportAuditReport'}, + ) + ]) + return {'ok': True, 'name': 'audit-report.md'} +``` -await sess.add_artifacts( - [Artifact(name='report.md', parts=[Part(TextPart(text=body))])] -) +From a custom agent's `handle_turn`: + +```python +from genkit import Part +from genkit.exp.agent import Artifact, SessionRunner + + +async def save_report(sess: SessionRunner, body: str) -> None: + await sess.add_artifacts([Artifact(name='report.md', parts=[Part.from_text(body)])]) ``` -## Reading text +## Reading artifact text -Parts use a RootModel — read text from `.root` when it is a `TextPart`: +`Part.text` is `str | None`. This matches what `read_artifact` returns to the model: ```python -from genkit import TextPart - -def artifact_text(artifact) -> str: - return ''.join( - p.root.text - for p in (artifact.parts or []) - if isinstance(p.root, TextPart) - ) +from genkit.exp.agent import Artifact + + +def artifact_text(artifact: Artifact) -> str: + return '\n'.join(p.text for p in artifact.parts if p.text) ``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-background.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-background.md index 5fa59b45..dd4d1840 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-background.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-background.md @@ -1,69 +1,129 @@ -# Background Agents / Detaching (Beta) +# Background Agents & Cancellation (Experimental) -> Needs a [session store](agents-sessions.md). See [agents.md](agents.md). +> Requires a [session store](agents-sessions.md). For abort to cancel a running turn, the store must also implement `SnapshotSubscriber` (`InMemorySessionStore`, `FileSessionStore`, and `FirestoreSessionStore` do). See [agents.md](agents.md). -Hand work to the server and keep going. `detach` returns a task handle with a -`snapshot_id` right away. When you need the reply, load that snapshot. +`await chat.detach(...)` hands a turn to the server, reserves a `snapshot_id` in `SnapshotStatus.PENDING` (with background heartbeat refreshes), and returns a `DetachedTask` handle immediately without waiting for model or tool completion. + +- [Detach, poll, and load results](#detach-poll-and-load-results): Lines 14-56 +- [Client detach vs. server cancellation](#client-detach-vs-server-cancellation): Lines 58-82 +- [Tool cancellation (asyncio.CancelledError)](#tool-cancellation-asynciocancellederror): Lines 84-102 +- [Fan-out parallel research](#fan-out-parallel-research): Lines 104-129 + +--- + +## Detach, poll, and load results + +Because the originating `chat` never streams a detached turn's chunks, rehydrate the finished conversation with `await agent.load_chat(snapshot_id=task.snapshot_id)` once the task settles: ```python -from genkit.agent import InMemorySessionStore +from genkit_google_genai import GoogleAI + +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore, SnapshotStatus + +ai = Genkit(plugins=[GoogleAI()]) agent = ai.define_agent( - name='backgroundAgent', - model='googleai/gemini-flash-latest', - system='Senior research analyst. Produce a comprehensive markdown report.', + name='backgroundAnalyst', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Research analyst. Write concise markdown memos.', store=InMemorySessionStore(), ) -chat = agent.chat() -task = await chat.detach('Write a report on renewable energy trends') -print(task.snapshot_id) -snapshot = await task.wait(interval=2.0) -print(snapshot.status) +async def main() -> None: + # 1. Start the turn on the server and get a snapshot handle immediately + chat = agent.chat() + task = await chat.detach('Summarize grid battery storage economics.') + print(task.snapshot_id) + + # 2. Stream snapshot status updates (or await task.wait(interval=1.0) directly) + snapshot = None + async for snap in task.poll(interval=1.0): + print(snap.status) + snapshot = snap + + # 3. Verify terminal status and load the completed snapshot + assert snapshot is not None and snapshot.status == SnapshotStatus.COMPLETED + done = await agent.load_chat(snapshot_id=task.snapshot_id) + print(done.messages[-1].text) + + +if __name__ == '__main__': + ai.run_main(main()) +``` + +Terminal snapshot statuses on `poll()` / `wait()` are `COMPLETED`, `FAILED`, `ABORTED`, and `EXPIRED` (reported on read when a pending turn's heartbeat goes stale). Always check `snapshot.status == SnapshotStatus.COMPLETED` before reading reply messages. + +## Client detach vs. server cancellation -await task.abort() +Always `await` abort calls. Genkit distinguishes stopping a local listener from cancelling work on the server: -done = await agent.load_chat(snapshot_id=task.snapshot_id) -print(done.messages) +- **Client detach (`await turn.abort()` or `asyncio.timeout` / `asyncio.wait_for`)**: stops streaming locally and resolves `turn.response` as cancelled immediately while the server turn keeps running to completion in the background. Wrapping `turn.stream` or `await turn.response` in `async with asyncio.timeout(...)` detaches the client the same way and re-raises `TimeoutError`. +- **Server cancel (`await task.abort()`, `await chat.abort()`, or `await agent.abort(snapshot_id)`)**: flips a pending snapshot to `SnapshotStatus.ABORTED` in the store and cancels the running server task. Returns the snapshot's **previous** `SnapshotStatus` (`SnapshotStatus.PENDING` if this call stopped a live turn; a terminal status if the turn had already finished). + +`chat.abort()` targets the chat's last recorded snapshot, so calling it while a foreground `send()` is in flight does not stop that turn: it returns the previous turn's status (`SnapshotStatus.COMPLETED`), or raises `ValueError` on a first turn with no snapshot yet. Use `detach()` + `task.abort()` to cancel a running turn. + +```python +chat = agent.chat() +task = await chat.detach('Audit every quarterly filing since 2015.') + +# Cancel server-side execution; returns the status from *before* the abort write +previous_status = await task.abort() +# => SnapshotStatus.PENDING (the snapshot row is now SnapshotStatus.ABORTED) + +# Aborted leaves are not resumable directly; continue from the last completed turn via session_id +recovered = agent.chat(session_id=chat.session_id) +res = await recovered.send('Audit only Q4 2025 instead.') +print(res.text) +# => Q4 2025 audit: ... ``` -## Stopping work +On the very first turn of a session, a client-side `turn.abort()` detaches before the client receives `session_id` or `snapshot_id`. Prefer `await chat.detach(...)` + `await task.abort()` whenever you need a recoverable server-side cancel handle from turn start. -Always `await` aborts — otherwise nothing happens. +## Tool cancellation (`asyncio.CancelledError`) -**Client stop** — `await turn.abort()` (or `asyncio.timeout` around the stream) -stops listening. The server may still finish. Use this mid-stream when you do -not have a snapshot id yet. +When a server turn is aborted (or an in-process caller cancels), Python delivers standard `asyncio.CancelledError` into active model requests, retry sleeps, and running `@ai.tool()` coroutines at the current `await` (there is no abort-signal attribute on `ToolContext`). Clean up partial external work in `try` / `finally` (and always re-raise `CancelledError` if caught): -**Server cancel** — `await chat.abort()` or `await task.abort()`. Needs a store -and an existing snapshot (after a completed turn or after `detach`). +```python +import asyncio -Aborted snapshots are not resume points. Reload the last good leaf. -On the first turn, a client abort can leave this chat without ids even if the -server later saves. Prefer `detach` + `task.abort()` when you need a -recoverable cancel. +@ai.tool(name='runBackfill', description='Run a multi-batch warehouse backfill.') +async def run_backfill(table: str) -> dict[str, object]: + job_id = f'job-{table}' + try: + await asyncio.sleep(10.0) + return {'job_id': job_id, 'status': 'completed'} + finally: + # Runs whether the tool returns normally, fails, or receives CancelledError + print(f'releasing lease for {job_id}') + # => releasing lease for job-orders (also after await task.abort() mid-run) +``` -## Parallel research +## Fan-out parallel research -One `chat` can only host one detach. Fork a leaf from a shared checkpoint for -each branch: +A single `AgentChat` runs one turn at a time. To run background branches concurrently, fork independent chats from a shared checkpoint `snapshot_id`: ```python +import asyncio + root = agent.chat() -await root.send('Context both researchers should know.') +await root.send('Context: B2B SaaS workload, 50k RPS peak, strict multi-region RPO.') checkpoint = root.snapshot_id +assert checkpoint is not None + -async def research(topic: str): - leaf = await agent.load_chat(snapshot_id=checkpoint) - task = await leaf.detach(f'Research {topic} in depth.') - await task.wait(interval=2.0) +async def research(engine: str) -> tuple[str, str]: + branch = await agent.load_chat(snapshot_id=checkpoint) + task = await branch.detach(f'Evaluate {engine} against our requirements.') + snap = await task.wait(interval=1.0) + if snap.status != SnapshotStatus.COMPLETED: + return engine, f'unfinished ({snap.status})' done = await agent.load_chat(snapshot_id=task.snapshot_id) - return topic, done.messages[-1] + return engine, done.messages[-1].text -import asyncio -results = await asyncio.gather(research('Postgres'), research('SQLite')) -``` -Check the loaded messages after wait — status alone is not enough. +results = await asyncio.gather(research('Cloud Spanner'), research('AlloyDB')) +# => [('Cloud Spanner', '...'), ('AlloyDB', '...')] +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-branching.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-branching.md index b17be4f8..cb793284 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-branching.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-branching.md @@ -1,43 +1,73 @@ -# Agent Branching (Beta) +# Agent Branching & Time Travel (Experimental) -> **Beta / preview API.** Needs a [session store](agents-sessions.md). -> See [agents.md](agents.md). +> Requires a [session store](agents-sessions.md). See [agents.md](agents.md). -A `snapshot_id` is an immutable checkpoint. Load it with -`load_chat(snapshot_id=...)` and send again — each turn creates a new leaf; -the original stays put. +Every store-backed turn writes a `SessionSnapshot` keyed by `snapshot_id`, with `parent_id` pointing at the previous turn. Reopening an earlier `snapshot_id` with `await agent.load_chat(snapshot_id=...)` and sending a new message writes a sibling snapshot under the same parent. The original timeline is untouched, and both branches keep the same `session_id`. + +Loading a snapshot restores the **entire** `SessionState` at that point: `messages`, custom `state`, and `artifacts` revert together. `load_chat` only resumes `completed` snapshots (see [sessions](agents-sessions.md) for `load_chat` vs `chat(snapshot_id=...)`). + +## Fork sibling timelines ```python -from genkit.agent import InMemorySessionStore +from genkit_google_genai import GoogleAI + +from genkit import GenkitError +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore -# After a fork, session_id is ambiguous — raise instead of guessing. +ai = Genkit(plugins=[GoogleAI()]) + +# reject_ambiguous_session=True makes session_id lookups fail loudly once a session forks store = InMemorySessionStore(reject_ambiguous_session=True) agent = ai.define_agent( name='designer', - model='googleai/gemini-flash-latest', - system='You help design a product landing page. Reply briefly.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='You design product landing pages. Reply in two short sentences.', store=store, ) -root = agent.chat() -await root.send('Plan a landing page for a note-taking app.') -checkpoint = root.snapshot_id -minimal = await agent.load_chat(snapshot_id=checkpoint) -await minimal.send('Direction: minimal.') +async def main() -> None: + # 1. Shared root turn; bookmark snapshot_id as the fork point + root = agent.chat() + await root.send('Plan a landing page for an AI note-taking app.') + checkpoint = root.snapshot_id + session_id = root.session_id + assert checkpoint and session_id -bold = await agent.load_chat(snapshot_id=checkpoint) -await bold.send('Direction: bold.') + # 2. Fork two independent branches from the same checkpoint + minimal = await agent.load_chat(snapshot_id=checkpoint) + await minimal.send('Direction: minimal with generous whitespace.') -resumed = await agent.load_chat(snapshot_id=bold.snapshot_id) -await resumed.send('Add a pricing section.') -``` + bold = await agent.load_chat(snapshot_id=checkpoint) + await bold.send('Direction: dark mode and high contrast.') + chosen_leaf = bold.snapshot_id + assert chosen_leaf -Same pattern for time travel: reopen an earlier snapshot and send a different -follow-up. After forks, resume by `snapshot_id`, not `session_id`: + print(bold.session_id == minimal.session_id == session_id) + # => True + print([m.text for m in bold.messages if m.role == 'user']) + # => ['Plan a landing page for an AI note-taking app.', 'Direction: dark mode and high contrast.'] -```python -# Ambiguous when reject_ambiguous_session=True — pick a leaf instead: -resumed = await agent.load_chat(snapshot_id=bold.snapshot_id) + # 3. With two leaves, a bare session_id lookup raises FAILED_PRECONDITION + try: + await store.get_snapshot(session_id=session_id) + except GenkitError as err: + print(err.status) + # => FAILED_PRECONDITION + + # 4. Continue linearly from the chosen branch leaf + resumed = await agent.load_chat(snapshot_id=chosen_leaf) + await resumed.send('Add a pricing section.') + + +if __name__ == '__main__': + ai.run_main(main()) ``` + +## Branching rules + +- **Resume by `snapshot_id` after forking**: Once a session has multiple leaf snapshots, "latest turn for `session_id`" is ambiguous. By default the store picks the most recently created leaf. Pass `reject_ambiguous_session=True` on `InMemorySessionStore` or `FileSessionStore` to fail instead: `store.get_snapshot(session_id=...)` and `agent.get_snapshot(session_id=...)` raise `GenkitError` (`FAILED_PRECONDITION`), and a turn sent on `agent.chat(session_id=...)` returns a failed turn with that status on `res.error`. +- **Edit-and-resubmit / regenerate**: Store each turn's `snapshot_id` alongside the message in your UI. When a user edits an earlier prompt or clicks regenerate, load the parent turn's `snapshot_id` and send the revised message. Previous answers stay intact as sibling branches. +- **Artifact time travel**: When an agent edits files via `Artifacts()` (`genkit_middleware.exp`, see [artifacts](agents-artifacts.md)) or a tool, rewinding with `load_chat(snapshot_id=checkpoint)` restores `chat.artifacts` and `chat.state` to their values at that turn. diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-custom.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-custom.md index f93b4dc6..6d054565 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-custom.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-custom.md @@ -1,51 +1,59 @@ -# Advanced Custom Agents — `define_custom_agent` (Beta) +# Custom Agents — `define_custom_agent` (Experimental) -> **Beta / preview API.** See [agents.md](agents.md). +> See [agents.md](agents.md). -`define_agent` runs a fixed prompt + tool loop. For your own orchestration -(multi-step, custom streaming, manual messages/state/artifacts), use -`ai.define_custom_agent`. Callers still use `chat.send` / `send_stream`. +`ai.define_agent` runs a built-in model + tool loop. When you need custom multi-step orchestration, subagent delegation, custom streaming, or manual control over messages, custom state, and artifacts, register a custom turn loop with `ai.define_custom_agent`. Callers still talk to the agent through `chat.send(...)`, `chat.send_stream(...)`, `chat.detach(...)`, and `serve_agent(...)`. + +- [Anatomy](#anatomy): Lines 13-46 +- [Streaming custom agent example](#streaming-custom-agent-example): Lines 48-114 +- [Non-streaming handle_turn with tools](#non-streaming-handle_turn-with-tools): Lines 116-170 + +--- ## Anatomy -Register one async `fn`. Inside it, define `handle_turn`, hand it to -`sess.run`, return `sess.result()`: +Register one async `fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult`. Inside `fn`, define `handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None`, pass it to `await sess.run(handle_turn)`, and return `await sess.result()`: ```python -async def fn(sess: SessionRunner, ctx: ActionRunContext) -> AgentResult: +from genkit.exp.agent import ( + AgentContext, + AgentFinishReason, + AgentInput, + AgentResult, + SessionRunner, + TurnContext, + TurnResult, +) + + +async def custom_fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult: async def handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None: - # user message already in sess.get_messages() - # model / tools / logic… - # ctx.send_chunk(AgentStreamChunk(...)) to stream - # sess.add_messages(...) to persist replies + # 1. inp.message is already appended to sess.get_messages() before handle_turn runs + # 2. Emit live chunks with ctx.send_chunk(AgentStreamChunk(...)) + # 3. Persist assistant/tool messages via sess.add_messages([...]) or sess.set_messages([...]) return TurnResult(finish_reason=AgentFinishReason.STOP) await sess.run(handle_turn) return await sess.result() - - -agent = ai.define_custom_agent(name='customCoder', fn=fn, store=store) ``` -`handle_turn` runs once per user turn. Before it runs, the runtime appends -`inp.message` to history. Return a `TurnResult` (or `None`); common finish -reasons: `STOP`, `INTERRUPTED`, `FAILED`. Raising ends the invocation as -`FAILED`. +Runtime mechanics: -`turn_ctx` (store only): `snapshot_id` (name external worktrees/dirs under -this), `parent_snapshot_id`, `turn_index`. Without a store, `snapshot_id` is -`None`. +- **Automatic user message append**: Before calling `handle_turn`, `sess.run` appends `inp.message` (`AgentInput` carries `message`, `resume`, and `detach`) to session history. `await sess.get_messages()` already returns `list[Message]` including the current user prompt — do not wrap items in `Message(m)`. +- **`TurnContext` (`turn_ctx`)**: Exposes `snapshot_id` (pre-reserved at turn start when a `store` is configured, `None` otherwise), `parent_snapshot_id`, and `turn_index`. Pre-reserving `snapshot_id` lets you name external worktrees or scratch directories after the snapshot before the turn commits. `turn_index` is 0-based within one invocation; each `chat.send` starts a new invocation, so it does not count turns across the session. +- **`SessionRunner` (`sess`) helpers**: `get_messages()` / `add_messages([...])` / `set_messages([...])`, `get_custom()` / `update_custom(sync_fn)`, and `get_artifacts()` / `add_artifacts([...])`. Both `add_messages` and `add_artifacts` take a `list`. +- **Turn completion and failures**: `TurnResult` accepts `(finish_reason=..., error=...)` — it does **not** take `messages=`. Raising an exception inside `handle_turn` ends the turn with `finish_reason=AgentFinishReason.FAILED`, populates `res.error` on the caller's `AgentResponse`, and stops the turn loop. `chat.send` returns that failed turn; it does not raise. +- **Model failures do not raise**: `ai.generate` and `ai.generate_stream` report provider errors on `response.error` with `finish_reason='failed'`. Raise a `GenkitError` from it so the caller's `res.error` keeps the provider status (e.g. `UNAVAILABLE`); otherwise the caller sees a generic `UNKNOWN` "Agent turn failed." (or a silent `STOP` if you hardcode the finish reason). -From `sess`: `get_messages` / `add_messages` / `set_messages`, -`get_custom` / `update_custom`, `get_artifacts` / `add_artifacts`. -`add_messages` and `add_artifacts` each take a **list**. Stream with -`ctx.send_chunk(AgentStreamChunk(...))`. - -## Example +## Streaming custom agent example ```python -from genkit import ActionRunContext, FinishReason, Genkit, Message -from genkit.agent import ( +from genkit_google_genai import GoogleAI + +from genkit import GenkitError +from genkit.exp import Genkit +from genkit.exp.agent import ( + AgentContext, AgentFinishReason, AgentInput, AgentResult, @@ -55,35 +63,36 @@ from genkit.agent import ( TurnContext, TurnResult, ) -from genkit_google_genai import GoogleAI ai = Genkit(plugins=[GoogleAI()]) store = InMemorySessionStore() -async def custom_coder_fn(sess: SessionRunner, ctx: ActionRunContext) -> AgentResult: +async def custom_coder_fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult: async def handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None: + # 1. Read history (already includes inp.message as list[Message]) history = await sess.get_messages() - messages = [Message(m) for m in history] if history else None + # 2. Stream model chunks out through AgentContext stream_resp = ai.generate_stream( - model='googleai/gemini-flash-latest', - system='Concise coding assistant.', - messages=messages, + model=GoogleAI.gemini('gemini-flash-latest'), + system='Concise Python systems assistant.', + messages=history, ) async for chunk in stream_resp.stream: ctx.send_chunk(AgentStreamChunk(model_chunk=chunk)) + # 3. Fail the turn on model errors (generate reports them instead of raising) res = await stream_resp.response + if res.error is not None: + raise GenkitError(status=res.error.status, message=res.error.message) + + # 4. Persist the assistant reply so subsequent turns see it if res.message: await sess.add_messages([res.message]) - fr = ( - AgentFinishReason.STOP - if res.finish_reason == FinishReason.STOP - else AgentFinishReason.UNKNOWN - ) - return TurnResult(finish_reason=fr) + # 5. Every model FinishReason value exists on AgentFinishReason + return TurnResult(finish_reason=AgentFinishReason(res.finish_reason.value)) await sess.run(handle_turn) return await sess.result() @@ -91,26 +100,71 @@ async def custom_coder_fn(sess: SessionRunner, ctx: ActionRunContext) -> AgentRe agent = ai.define_custom_agent(name='customCoder', fn=custom_coder_fn, store=store) -chat = agent.chat() -await chat.send('What is a Python list comprehension?') -``` -Persist model replies with `sess.add_messages` or the next turn won't see them. -Custom state → [state](agents-state.md). Artifacts → [artifacts](agents-artifacts.md). +async def main() -> None: + chat = agent.chat() + res = await chat.send('Show a dict comprehension that inverts keys and values.') + print(res.text) + # => inverted = {v: k for k, v in original.items()} + # (model reply, usually wrapped in a markdown code fence) + +if __name__ == '__main__': + ai.run_main(main()) +``` ## Non-streaming `handle_turn` with tools +When `ai.generate(..., tools=[...])` runs a multi-step tool loop inside `handle_turn`, `res.messages` contains the full conversation transcript (prior history + intermediate tool requests/responses + final model reply, plus a leading `Role.SYSTEM` message when `system=` was passed). Strip `Role.SYSTEM` and save the updated transcript with `await sess.set_messages(...)`: + ```python -async def handle_turn(inp: AgentInput, ctx: TurnContext) -> TurnResult | None: - history = [Message(m) for m in await sess.get_messages()] - # ai.generate runs the tool loop for you - res = await ai.generate( - messages=history + list(inp.messages or []), - tools=[get_time], - system='You are helpful.', - ) - return TurnResult(messages=res.messages, finish_reason=AgentFinishReason.STOP) +from datetime import datetime, timezone + +from genkit_google_genai import GoogleAI + +from genkit import GenkitError, Role +from genkit.exp import Genkit +from genkit.exp.agent import ( + AgentContext, + AgentFinishReason, + AgentInput, + AgentResult, + InMemorySessionStore, + SessionRunner, + TurnContext, + TurnResult, +) + +ai = Genkit(plugins=[GoogleAI()]) + + +@ai.tool(name='utcNow', description='Return the current UTC timestamp.') +async def utc_now() -> str: + return datetime.now(timezone.utc).isoformat() + + +async def ops_agent_fn(sess: SessionRunner, ctx: AgentContext) -> AgentResult: + async def handle_turn(inp: AgentInput, turn_ctx: TurnContext) -> TurnResult | None: + # 1. Run the model + tool loop over the session history + history = await sess.get_messages() + res = await ai.generate( + model=GoogleAI.gemini('gemini-flash-latest'), + system='Ops assistant. Use utcNow when asked for the current time.', + messages=history, + tools=[utc_now], + ) + if res.error is not None: + raise GenkitError(status=res.error.status, message=res.error.message) + + # 2. Persist tool-request, tool-response, and final model messages without duplicating system + await sess.set_messages([m for m in res.messages if m.role != Role.SYSTEM]) + return TurnResult(finish_reason=AgentFinishReason(res.finish_reason.value)) + + await sess.run(handle_turn) + return await sess.result() + + +ops_agent = ai.define_custom_agent(name='opsAgent', fn=ops_agent_fn, store=InMemorySessionStore()) ``` -Tool inputs: [agents.md](agents.md#tool-inputs). +Related guides: [state](agents-state.md) · [artifacts](agents-artifacts.md) · [tools](agents.md#tools-progress-chunks-and-multipart-responses) diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-http.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-http.md index fd70c50d..6d690cb2 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-http.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-http.md @@ -1,113 +1,200 @@ -# Agent HTTP (Beta) +# Agent HTTP Serving & Remote Client (Experimental) -> See [agents.md](agents.md). +> See [agents.md](agents.md) · [FastAPI](fastapi.md). -Serve an agent over FastAPI. Talk to it from another process with -`remote_agent`. Flows use `serve_flow` — [FastAPI](fastapi.md). +Mount an agent on FastAPI with `serve_agent` from `genkit_fastapi.exp`, and connect from another Python service or CLI with `remote_agent` from `genkit.exp.agent`. Stable flows use `serve_flow` from `genkit_fastapi` ([FastAPI](fastapi.md)). -## Serve +- [Serve an agent with FastAPI](#serve-an-agent-with-fastapi): Lines 14-49 +- [Connect with remote_agent](#connect-with-remote_agent): Lines 51-87 +- [Auth, context_dependency, and PublicError](#auth-context_dependency-and-publicerror): Lines 89-160 +- [Custom HTTP routes](#custom-http-routes): Lines 162-200 + +--- + +## Serve an agent with FastAPI + +`serve_agent(agent)` mounts `POST /{agent.name}` for turns: one-shot JSON by default, SSE when the caller sends `Accept: text/event-stream` or `?stream=true`. When the agent has a `store`, it also mounts `POST /{agent.name}/getSnapshot` (`404` when the snapshot is missing) and `POST /{agent.name}/abort`. Callers can pass `?session_id=` or `?thread_id=` to continue a session when the body `init` carries no session id. ```python import uvicorn from fastapi import FastAPI -from fastapi.middleware.cors import CORSMiddleware -from genkit import Genkit -from genkit.agent import InMemorySessionStore -from genkit_fastapi import serve_agent +from genkit_fastapi.exp import serve_agent from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore +# 1. Initialize the experimental Genkit instance +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) + +# 2. Define a store-backed agent agent = ai.define_agent( name='weatherAgent', - system='Weather assistant. Be concise.', + system='Weather assistant. Keep answers under two sentences.', store=InMemorySessionStore(), ) +# 3. Mount /api/weatherAgent, /api/weatherAgent/getSnapshot, and /api/weatherAgent/abort app = FastAPI() -app.add_middleware( - CORSMiddleware, - allow_origins=['*'], - allow_methods=['*'], - allow_headers=['*'], - expose_headers=['X-Genkit-Stream-Id'], -) - app.include_router(serve_agent(agent), prefix='/api') if __name__ == '__main__': uvicorn.run(app, host='0.0.0.0', port=8080) ``` -Routes land at `/{agent.name}` by default. Override with `base_path=...`. +Override the route path with `serve_agent(agent, base_path='/chat')`. Add FastAPI's `CORSMiddleware` as usual if browsers call the agent from another origin. Run alongside the Developer UI with: ```bash genkit start -- uv run server.py ``` -## Remote client +## Connect with `remote_agent` -```python -from genkit.agent import remote_agent +`remote_agent` returns an `AgentClient` with the same `chat()`, `load_chat()`, `get_snapshot()`, and `abort()` surface as an in-process `Agent`. `timeout` (default `60.0`) is the longest wait for the next bytes from the server, not a cap on the whole turn. -client = remote_agent( - url='http://127.0.0.1:8080/api/weatherAgent', - state_management='server', # 'client' if the server has no store -) -chat = client.chat() -res = await chat.send('Weather in Tokyo?') -print(res.text, res.snapshot_id) - -turn = chat.send_stream('And humidity?') -async for chunk in turn.stream: - if chunk.text: - print(chunk.text, end='', flush=True) -final = await turn.response - -resumed = await client.load_chat(snapshot_id=final.snapshot_id) -await resumed.send('What city was that?') +```python +from genkit.exp.agent import remote_agent + + +async def run_client() -> None: + client = remote_agent( + url='http://127.0.0.1:8080/api/weatherAgent', + state_management='server', # use 'client' when the server agent has no store + timeout=60.0, + ) + + # 1. One-shot turn + chat = client.chat() + res = await chat.send('Weather in Tokyo?') + print(res.text) + # => I don't have real-time weather data for Tokyo. Check a local forecast for live conditions. + + # 2. Streamed follow-up turn + turn = chat.send_stream('What about humidity?') + async for chunk in turn.stream: + if chunk.text: + print(chunk.text, end='', flush=True) + final = await turn.response + + # 3. Rehydrate from the server store onto a new chat handle + assert final.snapshot_id is not None + resumed = await client.load_chat(snapshot_id=final.snapshot_id) + answer = await resumed.send('Which city were we discussing?') + print(answer.text) + # => We were discussing Tokyo. ``` -Keep one chat for multi-turn. Match `state_management` to the server. Skip the -trailing slash on the URL. +Pass `state_schema=MyStateModel` to `remote_agent(...)` when you want `chat.state`, `res.state`, and `chunk.custom` validated into a Pydantic model on the client. Omit any trailing slash on `url`; the client derives `{url}/getSnapshot` and `{url}/abort` from it. A timeout surfaces as `GenkitError` `DEADLINE_EXCEEDED`, a refused or dropped connection as `UNAVAILABLE`. + +## Auth, `context_dependency`, and `PublicError` -## Auth +Resolve caller identity on the server with `context_dependency`, a FastAPI dependency applied to the turn, `getSnapshot`, and `abort` routes. Return a `dict`; other return values (such as Pydantic models) are dropped. Tools read it from `ctx.context` (or `Genkit.current_context()`), and `SessionStore` methods receive it as `context=`. -Resolve identity on the **server**. Mount the agent with a FastAPI dependency -that returns a plain dict. Tools read it with `Genkit.current_context()` for -the whole turn. +Errors surface at three boundaries: + +- **Inside `context_dependency`**: FastAPI resolves it before Genkit runs, so Genkit's error mapping does not apply. Raise `fastapi.HTTPException(status_code=401, ...)`; a `PublicError` raised here becomes FastAPI's plain `500 Internal Server Error`. +- **Inside Genkit, before or outside a turn**: on `getSnapshot` and `abort`, a `PublicError` from your `SessionStore` keeps its status, message, and details (`PERMISSION_DENIED` -> `403`). `AgentInitError` (an init that does not match the agent's state mode) is a `PublicError` and keeps its status too (`FAILED_PRECONDITION` -> `400`). Any other exception returns a redacted `500`. Full rules: [HTTP Error Redaction](fastapi.md#http-error-redaction--publicerror). +- **Inside a running turn**: tool and store exceptions do not become HTTP errors. The response is `200` with `finishReason: "failed"` and `error`; the client sees `res.finish_reason == AgentFinishReason.FAILED` and `res.error`. ```python from fastapi import FastAPI, Header, HTTPException -from genkit_fastapi import serve_agent +from genkit_fastapi.exp import serve_agent + +from genkit import PublicError, ToolContext +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore, SessionSnapshot, remote_agent +from genkit_google_genai import GoogleAI + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) + + +# 1. Resolve the caller from a header; the dict becomes the action context +async def resolve_user(x_api_key: str | None = Header(default=None)) -> dict[str, object]: + if not x_api_key: + raise HTTPException(status_code=401, detail='Missing X-Api-Key header.') + return {'tenant_id': 'acme', 'role': 'admin'} + + +# 2. Reject snapshot reads that carry no tenant +class TenantScopedStore(InMemorySessionStore): + async def get_snapshot( + self, + *, + snapshot_id: str | None = None, + session_id: str | None = None, + context: dict[str, object] | None = None, + ) -> SessionSnapshot | None: + if not (context or {}).get('tenant_id'): + raise PublicError('PERMISSION_DENIED', 'Tenant context required.') + return await super().get_snapshot(snapshot_id=snapshot_id, session_id=session_id, context=context) + -async def resolve_user(x_user: str | None = Header(default=None)) -> dict: - if not x_user: - raise HTTPException(status_code=401, detail='X-User header required') - return {'sub': x_user, 'role': 'engineer'} +# 3. Read the same context from a tool +@ai.tool(name='whoami', description='Inspect caller tenant context.') +async def whoami(ctx: ToolContext) -> dict[str, object]: + auth = ctx.context or {} + return {'tenant_id': auth.get('tenant_id'), 'role': auth.get('role')} + + +support_agent = ai.define_agent( + name='supportAgent', + system='Support assistant for the caller tenant.', + tools=[whoami], + store=TenantScopedStore(), +) app = FastAPI() app.include_router( - serve_agent(agent, context_dependency=resolve_user), + serve_agent(support_agent, context_dependency=resolve_user), prefix='/api', ) +# POST /api/supportAgent with no X-Api-Key +# => 401 {"detail":"Missing X-Api-Key header."} -@ai.tool() -async def who_am_i() -> str: - ctx = Genkit.current_context() or {} - return f"user={ctx.get('sub')} role={ctx.get('role')}" -``` - -The remote client only forwards credentials: - -```python -client = remote_agent( +# 4. The remote client sends static headers, or a sync/async callable that returns them +authed_client = remote_agent( url='http://127.0.0.1:8080/api/supportAgent', state_management='server', - headers={'X-User': 'alice'}, + headers={'X-Api-Key': 'sk-live-...'}, ) ``` -Return a `dict` from the dependency — Pydantic models are dropped. Custom -routes can pass context through `handle_genkit_request` ([FastAPI](fastapi.md)). -One-shot calls outside chat can use `ai.generate(..., context={...})`. +## Custom HTTP routes + +Use `handle_genkit_request` (`genkit_fastapi`) when a route needs path parameters or extra `Depends(...)` but should keep the Genkit JSON / SSE wire format. Use `to_sse` (`genkit.web`) to frame any stream handle (`chat.send_stream()`, `flow.stream()`) as `data: {"message": ...}` frames followed by `data: {"result": ...}`. Generic flow patterns: [Custom Route Escape Hatch](fastapi.md#3-custom-route-escape-hatch-handle_genkit_request-or-genkitweb). + +```python +from fastapi import FastAPI, Request, Response +from fastapi.responses import StreamingResponse +from genkit_fastapi import handle_genkit_request +from pydantic import BaseModel + +from genkit.web import to_sse + +custom_app = FastAPI() + + +class TurnRequest(BaseModel): + prompt: str + + +# Option A: full Genkit wire contract inside a custom route. +# Body: {"data": {"message": {"role": "user", "content": [{"text": "..."}]}}} +@custom_app.post('/tenants/{tenant_id}/chat') +async def tenant_agent_route(tenant_id: str, request: Request) -> Response: + # Check the caller may act for tenant_id before trusting the path. + return await handle_genkit_request( + request, + action=support_agent, + context={'tenant_id': tenant_id}, + ) + + +# Option B: stream one chat turn as SSE from a plain JSON body. +# agent.chat() takes no request context, so the store and tools see context=None. +@custom_app.post('/sessions/{session_id}/stream') +async def stream_session_turn(session_id: str, body: TurnRequest) -> StreamingResponse: + chat = agent.chat(session_id=session_id) + turn = chat.send_stream(body.prompt) + return StreamingResponse(to_sse(turn), media_type='text/event-stream') +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-human-in-the-loop.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-human-in-the-loop.md index e6a2d411..d2332c34 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-human-in-the-loop.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-human-in-the-loop.md @@ -1,43 +1,48 @@ -# Agent Human-in-the-Loop / Interrupts (Beta) +# Agent Human-in-the-Loop / Interrupts (Experimental) -> **Beta / preview API.** See [agents.md](agents.md). +> See [agents.md](agents.md) · [sessions](agents-sessions.md). -An **interrupt** pauses mid-turn so your code (or a human) can decide, then -**resume**. Works with or without a [store](agents-sessions.md). +An interrupt pauses an agent mid-turn so a human or policy check can inspect the pending tool call, then resume execution. Interrupts work both with a store (across separate HTTP requests) and without a store (in-process or round-tripped client state). -Flow: `send` → `finish_reason == INTERRUPTED` + `res.interrupts` → human input -→ `resume`. +Turn lifecycle: `chat.send(...)` -> `res.finish_reason == AgentFinishReason.INTERRUPTED` with `res.interrupts` -> human decision -> `chat.resume(...)` / `chat.resume_stream(...)`. -## ToolApproval +- [Gate tools with ToolApproval](#gate-tools-with-toolapproval): Lines 15-60 +- [Detect and resume (restart vs respond)](#detect-and-resume-restart-vs-respond): Lines 62-116 +- [Custom tool interrupts and Interrupt (BaseException)](#custom-tool-interrupts-and-interrupt-baseexception): Lines 118-156 -Tools in `allowed_tools` run freely; everything else pauses. Empty list ⇒ every -tool needs approval. When you also mount `Filesystem` / `Artifacts`, remember -to allow-list (or deliberately interrupt on) `write_file` / `edit_file` / -`write_artifact` — the coding-agent example often auto-approves **reads only**. +--- + +## Gate tools with `ToolApproval` + +`ToolApproval(allowed_tools=[...])` lets listed tools execute immediately and interrupts all other tool calls. An empty list requires approval for every tool. When pairing `ToolApproval` with `Filesystem` or `Artifacts`, list read-only tools (`list_files`, `read_file`, `read_artifact`) in `allowed_tools` if only writes (`write_file`, `edit_file`, `write_artifact`) should pause. ```python from uuid import uuid4 - -from pydantic import BaseModel, Field from genkit_google_genai import GoogleAI from genkit_middleware import Middleware, ToolApproval +from pydantic import BaseModel, Field -from genkit import Genkit, ToolRequestPart -from genkit.agent import AgentFinishReason, InMemorySessionStore - -ai = Genkit(plugins=[GoogleAI(), Middleware()]) -tool_approval = ToolApproval(allowed_tools=[]) +from genkit import Part +from genkit.exp import Genkit +from genkit.exp.agent import AgentFinishReason, InMemorySessionStore class TransferInput(BaseModel): amount: float to_account: str = Field(alias='toAccount') + model_config = {'populate_by_name': True} + class TransferOutput(BaseModel): success: bool transaction_id: str = Field(alias='transactionId') + model_config = {'populate_by_name': True} + + +ai = Genkit(plugins=[GoogleAI(), Middleware()]) + @ai.tool(name='transferMoney', description='Transfer money between accounts.') async def transfer_money(input: TransferInput) -> TransferOutput: @@ -46,64 +51,106 @@ async def transfer_money(input: TransferInput) -> TransferOutput: agent = ai.define_agent( name='bankingAgent', - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), system='Banking assistant. Call transferMoney when the user asks to transfer money.', tools=[transfer_money], - use=[tool_approval], + use=[ToolApproval(allowed_tools=[])], store=InMemorySessionStore(), ) ``` -## Detect and resume +## Detect and resume (`restart` vs `respond`) -```python -chat = agent.chat() +Each entry in `res.interrupts` is an `AgentInterrupt` exposing `.name`, `.ref`, and `.input` for rendering your approval UI, plus two methods that build `Part` objects for `chat.resume(...)`: -out1 = await chat.send('Transfer $500 to account 12345 for rent.') -# finish_reason == INTERRUPTED; transferMoney has NOT executed yet. +- **`intr.restart(resumed_metadata={'tool_approved': True})`**: re-runs the interrupted tool on the server (`ctx.is_resumed()` becomes `True`). `ToolApproval` checks `tool_approved` (or `toolApproved`) in `resumed_metadata`. Restart inputs must exactly match the original tool request in session history; a forged or modified restart returns a failed turn (`AgentFinishReason.FAILED`, `res.error.status == 'INVALID_ARGUMENT'`) instead of running the tool. +- **`intr.respond(output)`**: builds a tool response `Part` without executing the tool. Use this to deny an action (e.g. `intr.respond({'ok': False, 'error': 'declined by user'})`) or answer a prompt-style interrupt created with `ai.define_interrupt(...)`. -restart_parts: list[ToolRequestPart] = [ - intr.restart(resumed_metadata={'tool_approved': True}) for intr in out1.interrupts -] -out2 = await chat.resume(restart=restart_parts) -``` +A resumed turn can call more tools that also need approval, so loop until the turn stops interrupting: -Resume builders (return parts; you still call `chat.resume`): +```python +async def main() -> None: + chat = agent.chat() + + # 1. Turn pauses before transferMoney executes + res = await chat.send('Transfer $500 to account 12345 for rent.') + + # 2. Approve pending interrupts until the turn finishes + while res.finish_reason == AgentFinishReason.INTERRUPTED: + for intr in res.interrupts: + print(f'Approving {intr.name}: {intr.input}') + # => Approving transferMoney: {'amount': 500, 'toAccount': '12345'} + restart_parts: list[Part] = [ + intr.restart(resumed_metadata={'tool_approved': True}) + for intr in res.interrupts + ] + res = await chat.resume(restart=restart_parts) + + # 3. Read the final reply (check res.error if the turn failed) + print(res.finish_reason, res.text) + # => stop $500 has been successfully transferred to account 12345. + + +if __name__ == '__main__': + ai.run_main(main()) +``` -- `interrupt.restart(...)` — re-issue the tool request (ToolApproval after OK). - `resumed_metadata={'tool_approved': True}` (camelCase `toolApproved` also ok) -- `interrupt.respond(output)` — supply a tool response without running the tool - (use this to **deny**: e.g. `respond({'ok': False, 'error': 'denied by policy'})`) +Denials and streaming resumes use the same parts (`res` is an interrupted turn): ```python -# Approve (runs the tool): -await chat.resume( - restart=[intr.restart(resumed_metadata={'tool_approved': True}) for intr in out1.interrupts] -) - -# Deny (model sees the respond payload; tool does not execute): -await chat.resume( - respond=[intr.respond({'ok': False, 'error': 'export denied'}) for intr in out1.interrupts] +# Deny without executing the tool: +denied = await chat.resume( + respond=[intr.respond({'ok': False, 'error': 'transfer declined'}) for intr in res.interrupts], ) -await chat.resume( - respond=[a.respond({'approved': True})], - restart=[b.restart(resumed_metadata={'tool_approved': True})], +# Or approve and stream the resumed turn: +turn = chat.resume_stream( + restart=[intr.restart(resumed_metadata={'tool_approved': True}) for intr in res.interrupts], ) - -turn = chat.resume_stream(restart=restart_parts) async for chunk in turn.stream: if chunk.text: print(chunk.text, end='', flush=True) final = await turn.response ``` -Without a store: keep the same `chat`, or round-trip messages/state/artifacts -([client-managed state](agents.md#client-managed-state-no-store)). +Without a store, keep the same `chat` instance or round-trip `messages`, `state`, and `artifacts` ([without a store](agents.md#without-a-store)). + +## Custom tool interrupts and `Interrupt` (`BaseException`) + +For domain-specific thresholds (such as only pausing transfers above `$1,000`), raise `Interrupt` directly inside a tool or register a dedicated prompt tool with `ai.define_interrupt(...)`: + +- **`Interrupt` subclasses `BaseException`**, not `Exception` (like `asyncio.CancelledError`). A `try ... except Exception:` block inside a tool will **not** swallow an `Interrupt` and accidentally let execution proceed without approval. Only catch `Interrupt` explicitly if intended, and always re-raise it. +- Read `ctx.is_resumed()` and `ctx.resumed_metadata` on `ToolContext` when the caller resumes with `intr.restart(resumed_metadata=...)`. The keys are yours: the tool below expects `intr.restart(resumed_metadata={'approved': True})`. + +```python +from genkit import Interrupt, ToolContext + + +@ai.tool(name='wireTransfer', description='Send an external wire transfer.') +async def wire_transfer(input: TransferInput, ctx: ToolContext) -> dict[str, object]: + if input.amount > 1000 and not ctx.is_resumed(): + # BaseException: an `except Exception:` block can't swallow it + raise Interrupt({'reason': 'high_value_transfer', 'amount': input.amount}) -Build resume parts from `res.interrupts` only. Loop until finish reason is no -longer `INTERRUPTED`. Don't treat an interrupted turn as the final reply. + if ctx.is_resumed() and not (ctx.resumed_metadata or {}).get('approved'): + return {'ok': False, 'status': 'rejected'} -For approval UIs, each interrupt exposes `.name`, `.ref`, and `.input` (the -pending tool call). Use those to render the prompt; then `.restart(...)` or -`.respond(...)`. + return {'ok': True, 'amount': input.amount, 'to': input.to_account} +``` + +For pure question/confirmation tools that never run server logic themselves, use `ai.define_interrupt(...)` and resume with `intr.respond(...)`: + +```python +class ConfirmInput(BaseModel): + action: str + summary: str + + +ask_confirmation = ai.define_interrupt( + name='askConfirmation', + description='Ask the user to confirm before irreversible actions.', + input_schema=ConfirmInput, +) +# Add to the agent's tools=[...], then answer with: +# chat.resume(respond=[intr.respond({'confirmed': True}) for intr in res.interrupts]) +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-sessions.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-sessions.md index 16a24151..b4b7c103 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-sessions.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-sessions.md @@ -1,61 +1,78 @@ -# Sessions & Persistence (Beta) +# Sessions & Persistence (Experimental) > See [agents.md](agents.md). -With a store, the server owns history. Each turn writes an immutable snapshot — -the backbone for branching and [background work](agents-background.md). -Approvals work with or without a store ([HITL](agents-human-in-the-loop.md)). +With a `SessionStore`, the server owns conversation history. Every turn writes a `SessionSnapshot` keyed by `snapshot_id`, enabling resume across restarts, [branching](agents-branching.md), and [detached background turns](agents-background.md). [Interrupts](agents-human-in-the-loop.md) work with or without a store. ## Choose a store +Import local stores from `genkit.exp.agent` and the production Firestore store from `genkit_google_cloud.exp`: + ```python -from genkit.agent import InMemorySessionStore, FileSessionStore +from genkit.exp.agent import FileSessionStore, InMemorySessionStore +from genkit_google_cloud.exp import FirestoreSessionStore -mem_store = InMemorySessionStore() # gone on restart -file_store = FileSessionStore('./.snapshots') # one JSON file per snapshot +# 1. Local dev & tests (lost when the process exits) +mem_store = InMemorySessionStore() -# Optional: prune long chains; refuse ambiguous session_id after forks -pruning = FileSessionStore( +# 2. Local disk persistence (one <snapshot_id>.json file per turn) +file_store = FileSessionStore( './.snapshots', - max_persisted_chain_length=3, - reject_ambiguous_session=True, + max_persisted_chain_length=20, # optional: prune oldest ancestors on save + reject_ambiguous_session=True, # optional: raise on session_id lookup after a fork +) + +# 3. Production durable store (JSON Patch diffs + sharded checkpoints, ADC auth) +firestore_store = FirestoreSessionStore( + collection='genkit-sessions', + snapshot_path_prefix=lambda ctx: (ctx or {}).get('tenant_id', 'global'), ) ``` +`FirestoreSessionStore` stores each turn as a JSON Patch diff from its parent with periodic sharded checkpoints so large sessions stay below Firestore's 1 MiB document limit without secondary indexes. Pass `snapshot_path_prefix` to isolate multi-tenant paths from action `context`. + +## Persist and rehydrate a chat + ```python -from genkit import Genkit from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +from genkit.exp import Genkit +from genkit.exp.agent import FileSessionStore + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) agent = ai.define_agent( name='logbookAgent', - system='You are a personal logbook assistant.', - store=file_store, + system='You are a concise engineering logbook assistant.', + store=FileSessionStore('./.snapshots'), ) -chat = agent.chat() -res = await chat.send('Log this: I started studying Genkit today.') -print(res.snapshot_id) -resumed = await agent.load_chat(snapshot_id=res.snapshot_id) -await resumed.send('Add another note.') -``` +async def main() -> None: + # 1. Run turn 1; save res.snapshot_id in your app database + chat = agent.chat() + res = await chat.send('Log: migrated auth service to token exchange.') + checkpoint = res.snapshot_id + assert checkpoint is not None -## Typed state + # 2. Rehydrate full state (messages, state, artifacts) onto a fresh chat + resumed = await agent.load_chat(snapshot_id=checkpoint) + res2 = await resumed.send('Add note: latency p99 stayed at 14ms.') + print(res2.text) -Pass `state_schema`. Seed with `chat(state=...)` only when there is no store. -With a store, update fields from tools via `ai.current_session()` — -[state](agents-state.md). -## What a store unlocks +if __name__ == '__main__': + ai.run_main(main()) +``` -Without: multi-turn on one chat, interrupts. +## Resume and lookup methods -With: durable ids, `load_chat`, branching, detach / background tasks, server -abort. +- **`await agent.load_chat(snapshot_id=...)` / `load_chat(session_id=...)`**: reads the stored snapshot immediately and populates `chat.messages`, `chat.state`, and `chat.artifacts`. Pointing `load_chat` at a failed, aborted, or pending leaf loads that row as-is; calling `send()` on a non-completed snapshot returns `res.error` (`SNAPSHOT_NOT_RESUMABLE`). +- **`agent.chat(session_id=...)`**: attaches a lazy session handle without an upfront read. On the next `send()`, the server automatically walks back past any failed, aborted, or expired leaf to continue from the last completed turn (or starts a fresh session under `session_id` if no turn has completed yet). +- **`agent.chat(snapshot_id=...)`**: attaches a lazy resume handle to a specific completed snapshot. +- **`await agent.get_snapshot(snapshot_id=...)` / `get_snapshot(session_id=...)`**: inspects the raw `SessionSnapshot` without creating a chat. -## Many users, one agent +## Typed state and multi-user isolation -Each `agent.chat()` gets its own session on a shared store. Resume with the -right `session_id` or `snapshot_id` per user. +- **Typed state**: Pass `state_schema=MyModel` on the agent. On a store-backed agent, `agent.chat(state=...)` raises `AgentInitError` immediately; mutate state from tools with `current_session()` from `genkit.exp` (see [state](agents-state.md)). +- **Multi-user sessions**: One `Agent` instance safely serves many users on a shared store. Each `agent.chat()` mints a distinct `session_id`; store `session_id` or `snapshot_id` per user in your application database. diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-state.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-state.md index 04bdb00e..0f194a19 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents-state.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents-state.md @@ -1,71 +1,82 @@ -# Agent State (Beta) +# Agent State (Experimental) > See [agents.md](agents.md) · [sessions](agents-sessions.md). -A session carries three layers: messages, custom state (`state_schema`), and -artifacts. With a schema, `chat.state` and streamed `chunk.custom` come back as -that model. +An agent session tracks three layers: `messages`, custom `state` (typed by `state_schema`), and `artifacts`. When `state_schema` (a Pydantic `BaseModel`) is set, `chat.state`, `res.state`, and streamed `chunk.custom` validate into instances of that model instead of bare dicts. -## Two modes +- [Two state-management modes](#two-state-management-modes): Lines 15-20 +- [Client-managed typed state](#client-managed-typed-state): Lines 22-64 +- [Mutating state from tools (current_session)](#mutating-state-from-tools-current_session): Lines 66-115 +- [Live state patches](#live-state-patches): Lines 117-131 +- [Egress redaction (state_transform and chunk_transform)](#egress-redaction-state_transform-and-chunk_transform): Lines 133-163 -**With a store** — Genkit owns history. Resume by `snapshot_id` or -`session_id`. You cannot seed with `chat(state=...)`. +--- -**Without a store** — your app owns history. Seed and round-trip yourself: +## Two state-management modes -```python -chat = agent.chat(state=Profile(name='Ada', tier='pro')) -await chat.send('Hello') -resumed = agent.chat( - messages=chat.messages, state=chat.state, artifacts=chat.artifacts -) -``` +- **Server-managed (`store=...`)**: Genkit owns persistence. Resume with `snapshot_id` or `session_id`. Passing `state=`, `messages=`, or `artifacts=` to `agent.chat(...)` on a store-backed agent raises `AgentInitError` (`FAILED_PRECONDITION`). +- **Client-managed (`store=None`)**: Your caller owns persistence. Seed `agent.chat(state=...)` and pass `messages`, `state`, and `artifacts` back on subsequent chats. -Custom state is for your product (routing, UI). It is not injected into the -model unless you put it in the system prompt or messages. - -`state_schema` alone does not fill `chat.state`. Something in the turn must call -`update_custom`. Prefer tools on a normal agent so you keep middleware. +Custom state holds application data (UI state, workflow counters, active ticket IDs). Genkit does not inject `state` into the model prompt unless your system prompt, messages, or tools expose it. Declaring `state_schema` alone leaves `chat.state` as `None` until client seeding or `session.update_custom(...)` writes a value. ## Client-managed typed state ```python -from pydantic import BaseModel - -from genkit import Genkit from genkit_google_genai import GoogleAI +from pydantic import BaseModel -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +from genkit.exp import Genkit -class Profile(BaseModel): +class DinerProfile(BaseModel): name: str - tier: str = 'free' + allergies: list[str] = [] + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) agent = ai.define_agent( - name='profileAgent', - system='Greet the user by name when you know it.', - state_schema=Profile, + name='conciergeAgent', + system='Restaurant concierge. Keep replies brief.', + state_schema=DinerProfile, ) -chat = agent.chat(state=Profile(name='Ada', tier='pro')) -await chat.send('Hello') -print(chat.state.name) + +async def main() -> None: + # 1. Seed typed state on a storeless agent + chat = agent.chat(state=DinerProfile(name='Ada', allergies=['peanuts'])) + await chat.send('Recommend an appetizer.') + assert chat.state is not None + print(chat.state.name, chat.state.allergies) + # => Ada ['peanuts'] + + # 2. Round-trip messages, state, and artifacts to continue later + resumed = agent.chat( + messages=chat.messages, + state=chat.state, + artifacts=chat.artifacts, + ) + await resumed.send('What about dessert?') + + +if __name__ == '__main__': + ai.run_main(main()) ``` -## Store + middleware +## Mutating state from tools (`current_session`) -Update custom state from tools with `ai.current_session()`. Mutators are -async. Coerce to your model if the callback receives a dict: +Inside an `@ai.tool()`, get the active session with `current_session()` from `genkit.exp` (`None` outside an agent turn) and update custom state there. + +`await sess.update_custom(fn)` is async; `fn` is sync. `fn` receives `None` on the first write, then whatever the session currently holds: the model instance you last returned, or a plain dict once state has round-tripped through JSON (a persistent store, an HTTP client). Normalize it before building the new state: ```python -from pydantic import BaseModel, Field +from typing import Any -from genkit import Genkit -from genkit.agent import InMemorySessionStore from genkit_google_genai import GoogleAI -from genkit_middleware import Middleware, ToolApproval +from pydantic import BaseModel, Field + +from genkit.exp import Genkit, current_session +from genkit.exp.agent import InMemorySessionStore class CaseState(BaseModel): @@ -74,19 +85,19 @@ class CaseState(BaseModel): class OpenCaseInput(BaseModel): - case_id: str = Field(description='Support case id') + case_id: str = Field(description='Support case identifier') -ai = Genkit(plugins=[GoogleAI(), Middleware()], model='googleai/gemini-flash-latest') +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) -@ai.tool(name='openCase', description='Open or update the support case id.') -async def open_case(input: OpenCaseInput) -> dict: - sess = ai.current_session() +@ai.tool(name='openCase', description='Open or switch the active support case.') +async def open_case(input: OpenCaseInput) -> dict[str, Any]: + sess = current_session() if sess is None: - return {'ok': False, 'error': 'no session'} + return {'ok': False, 'error': 'no active agent session'} - async def mutate(c: object) -> CaseState: + def mutate(c: CaseState | dict[str, Any] | None) -> CaseState: base = c if isinstance(c, CaseState) else CaseState.model_validate(c or {}) return CaseState(case_id=input.case_id, status=base.status) @@ -96,40 +107,57 @@ async def open_case(input: OpenCaseInput) -> dict: agent = ai.define_agent( name='supportOps', - system='Support ops. Call openCase when asked. Be brief.', + system='Support ops assistant. Call openCase when given a case ID.', tools=[open_case], state_schema=CaseState, - use=[ToolApproval(allowed_tools=['openCase'])], store=InMemorySessionStore(), ) ``` -## Live patches +## Live state patches -In a [custom agent](agents-custom.md), `update_custom` streams to -`chunk.custom`: +Every `await sess.update_custom(...)` call (from a tool or a [custom agent](agents-custom.md)) emits an RFC 6902 JSON Patch chunk (`chunk.raw.custom_patch`). `AgentTurn.stream` applies the patch and puts the resolved state on `chunk.custom`, typed by `state_schema`. With the `supportOps` agent above: ```python -async def bump(c): - return {'turns': (c or {}).get('turns', 0) + 1} - -await sess.update_custom(bump) +chat = agent.chat() +turn = chat.send_stream('Open case SUP-402.') +async for chunk in turn.stream: + if chunk.custom is not None: + print(chunk.custom.case_id, chunk.custom.status) + # => SUP-402 open +res = await turn.response +print(res.state) +# => case_id='SUP-402' status='open' ``` -## Redacting on the way out +## Egress redaction (`state_transform` and `chunk_transform`) + +`state_transform` and `chunk_transform` filter what leaves the server without altering persisted snapshots: -`state_transform` and `chunk_transform` shape what clients see. They do not -change what the store keeps. Return a full `SessionState` from -`state_transform`. Returning `None` from `chunk_transform` drops that chunk. +- **`state_transform(state: SessionState) -> SessionState`**: shapes snapshot reads, client-managed `AgentOutput.state`, and streamed custom-patch baselines. Must return a `SessionState` (use `state.model_copy(update=...)` so `session_id` is preserved). +- **`chunk_transform(chunk: AgentStreamChunk) -> AgentStreamChunk | None`**: reshapes or drops individual stream chunks; return `None` to suppress a chunk. ```python -from genkit.agent import SessionState +from genkit.exp.agent import AgentStreamChunk, InMemorySessionStore, SessionState -def redact(state: SessionState) -> SessionState: + +def redact_state(state: SessionState) -> SessionState: custom = dict(state.custom or {}) - if 'api_key' in custom: - custom['api_key'] = 'REDACTED' - return SessionState(messages=state.messages, custom=custom, artifacts=state.artifacts) + custom.pop('internal_token', None) + return state.model_copy(update={'custom': custom}) + + +def drop_debug_artifacts(chunk: AgentStreamChunk) -> AgentStreamChunk | None: + if chunk.artifact and chunk.artifact.name == 'debug.log': + return None + return chunk -agent = ai.define_agent(..., state_schema=SecretState, state_transform=redact, store=...) + +guarded_agent = ai.define_agent( + name='guardedAgent', + system='Concise support agent.', + state_transform=redact_state, + chunk_transform=drop_debug_artifacts, + store=InMemorySessionStore(), +) ``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/agents.md b/plugins/genkit/agent/skills/developing-genkit-python/references/agents.md index fd114e91..844a468b 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/agents.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/agents.md @@ -1,26 +1,31 @@ -# Agents (Beta) +# Agents (Experimental) -> Preview API under `genkit.agent`. +> Experimental APIs live under `.exp` modules (`genkit.exp`, `genkit.exp.agent`, `genkit_fastapi.exp`, `genkit_middleware.exp`, `genkit_google_cloud.exp`) and can change between minor releases. -Turn a model + tools into a durable conversation — history, typed state, -approvals, branches, and background work. +Turn a model and tools into a stateful multi-turn conversation with snapshots, typed custom state, human approvals, branching, and detached background runs. -Deeper topics: [sessions](agents-sessions.md) · -[HITL](agents-human-in-the-loop.md) · [branching](agents-branching.md) · -[background](agents-background.md) · [state](agents-state.md) · -[artifacts](agents-artifacts.md) · [custom](agents-custom.md) · -[HTTP](agents-http.md) +Deeper topics: [sessions](agents-sessions.md) · [HITL](agents-human-in-the-loop.md) · [branching](agents-branching.md) · [background](agents-background.md) · [state](agents-state.md) · [artifacts](agents-artifacts.md) · [custom](agents-custom.md) · [HTTP](agents-http.md) + +- [Define an agent](#define-an-agent): Lines 19-69 +- [Tools, progress chunks, and multipart responses](#tools-progress-chunks-and-multipart-responses): Lines 71-99 +- [Middleware](#middleware): Lines 101-136 +- [Chat and streaming](#chat-and-streaming): Lines 138-169 +- [Errors and cancellation](#errors-and-cancellation): Lines 171-194 +- [Without a store](#without-a-store): Lines 196-216 +- [Verify an agent from the CLI (flow:run)](#verify-an-agent-from-the-cli-flowrun): Lines 218-234 + +--- ## Define an agent +Import `Genkit` from `genkit.exp` to opt into `define_agent`, `define_prompt_agent`, `define_custom_agent`, and `ai.agent(name)`. Stable `from genkit import Genkit` does not expose agent methods. + ```python +from genkit_google_genai import GoogleAI from pydantic import BaseModel -from genkit import Genkit -from genkit_google_genai import GoogleAI -from genkit.agent import InMemorySessionStore - -ai = Genkit(plugins=[GoogleAI()]) +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore class WeatherInput(BaseModel): @@ -32,139 +37,198 @@ class WeatherOutput(BaseModel): temperature: str +# 1. Initialize experimental Genkit instance +ai = Genkit(plugins=[GoogleAI()]) + + +# 2. Define a typed tool @ai.tool(name='getWeather', description='Get weather for a city.') async def get_weather(input: WeatherInput) -> WeatherOutput: - return WeatherOutput(weather='Sunny', temperature='21°C') + return WeatherOutput(weather=f'Sunny in {input.location}', temperature='21°C') +# 3. Register a store-backed agent agent = ai.define_agent( name='weatherAgent', - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), system='Weather assistant. Use getWeather for weather questions.', tools=[get_weather], - store=InMemorySessionStore(), # omit to manage history yourself + store=InMemorySessionStore(), # omit store= for client-managed history ) ``` -Options worth knowing: `use` (middleware), `state_schema`, `max_turns` (tool -loop depth per user message), transforms. Dotprompt agents: -`ai.define_prompt_agent`. Full control: [`define_custom_agent`](agents-custom.md). +Key `define_agent` keyword arguments: + +- **`model`**: `GoogleAI.gemini('gemini-flash-latest')` (or a `'plugin/model'` string) +- **`config`**: typed config such as `GeminiConfig(thinking_level='HIGH')` or `AnthropicConfig(thinking='adaptive', effort='high')` +- **`use`**: middleware instances (`Retry`, `Fallback`, `ToolApproval`, `Filesystem`, `Skills`, `Artifacts`, `Surfaces`) +- **`state_schema`**: Pydantic `BaseModel` typing `chat.state`, `res.state`, and `chunk.custom` ([state](agents-state.md)) +- **`max_turns`**: tool-loop ceiling per user message (default 50) +- **`state_transform` / `chunk_transform`**: egress hooks that redact client views without altering stored state -Tool parameters should be a small Pydantic model — even for one field. Empty -inputs need an empty subclass, not bare `BaseModel`. +For `.prompt` files or `ai.define_prompt(...)`, wire the registered prompt name directly with `ai.define_prompt_agent(name='greeterPrompt', store=...)`. For custom turn loops, use [`ai.define_custom_agent(...)`](agents-custom.md). + +## Tools, progress chunks, and multipart responses + +A tool takes at most one typed input parameter (a Pydantic `BaseModel` for structured arguments or a scalar like `str`, or no input parameter) plus an optional `ToolContext[ChunkT]` parameter. + +```python +import asyncio +from pydantic import BaseModel + +from genkit import MultipartToolResponse, Part, ToolContext + + +class PrepProgress(BaseModel): + stage: str + percent: int + + +@ai.tool(name='prepareOrder', description='Prepare a kitchen order and stream stage progress.') +async def prepare_order(dish: str, ctx: ToolContext[PrepProgress]) -> MultipartToolResponse[dict[str, str]]: + # 1. Stream partial tool progress (arrives on chunk.tool_responses with part.is_partial == True) + ctx.send_chunk(PrepProgress(stage='searing', percent=40)) + await asyncio.sleep(0.2) + ctx.send_chunk(PrepProgress(stage='plating', percent=90)) + + # 2. Return structured output alongside extra content parts (e.g. media or text) + return MultipartToolResponse( + output={'status': 'ready', 'dish': dish}, + content=[Part.from_text(f'Ticket closed for {dish}.')], + ) +``` ## Middleware -Register `Middleware()` once, then pass instances in `use=[...]`. +Import standard middleware from `genkit_middleware`, session-bound `Artifacts` from `genkit_middleware.exp`, and generative UI `Surfaces` from `genkit_a2ui`. Register `Middleware()` in `plugins=[...]` when you want standard middleware listed in the Dev UI or referenced by `.prompt` files; passing instances in `use=[...]` works either way. -- **Filesystem** — `list_files` / `read_file`. Writes stay off until - `allow_write_access=True`. Paths are relative to `root_dir`. -- **ToolApproval** — tools in `allowed_tools` run; everything else pauses for - a human. List write/artifact tools if they should run freely. -- **Skills** — load `skills/<name>/SKILL.md` and expose `use_skill`. -- **Retry** — retries flaky model HTTP calls. Tool failures are different: - return a soft-error payload the model can read. +- **`Filesystem`** (`genkit_middleware`): `list_files` and `read_file` scoped to `root_dir`; pass `allow_write_access=True` to add `write_file` and `edit_file`. +- **`ToolApproval`** (`genkit_middleware`): tools in `allowed_tools` run automatically; all other tools pause the turn with an interrupt ([HITL](agents-human-in-the-loop.md)). It gates every tool call, including the ones `Filesystem` and `Artifacts` add. +- **`Skills`** (`genkit_middleware`): injects `SKILL.md` descriptions and exposes `use_skill`. +- **`Retry` / `Fallback`** (`genkit_middleware`): `Retry` retries transient failures (`UNAVAILABLE`, `DEADLINE_EXCEEDED`, `RESOURCE_EXHAUSTED`, `ABORTED`, `INTERNAL` by default); `Fallback` switches to alternate models. +- **`Artifacts`** (`genkit_middleware.exp`): adds `read_artifact` and `write_artifact` tools plus an active artifact listing in the system prompt ([artifacts](agents-artifacts.md)). +- **`Surfaces`** (`genkit_a2ui`): A2UI surface instructions, validation, and part parsing. ```python -from genkit_middleware import Middleware, ToolApproval, Filesystem, Skills, Retry -from genkit.agent import FileSessionStore +from genkit_google_genai import GoogleAI +from genkit_middleware import Filesystem, Middleware, Retry, Skills, ToolApproval +from genkit_middleware.exp import Artifacts + +from genkit.exp import Genkit +from genkit.exp.agent import FileSessionStore ai = Genkit(plugins=[GoogleAI(), Middleware()]) coding_agent = ai.define_agent( name='codingAgent', - model='googleai/gemini-flash-latest', - system='Coding assistant. Use list_files/read_file/write_file/edit_file.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Coding assistant. Explore with list_files/read_file before editing.', use=[ - ToolApproval(allowed_tools=['list_files', 'read_file', 'use_skill']), + ToolApproval(allowed_tools=['list_files', 'read_file', 'read_artifact', 'use_skill']), Filesystem(root_dir='./workspace', allow_write_access=True), + Artifacts(), Skills(skill_paths=['./skills']), - Retry(), + Retry(max_retries=3), ], store=FileSessionStore('./.snapshots'), max_turns=30, ) ``` -For typed session fields with middleware, update state from tools via -`ai.current_session()` — [state](agents-state.md). Declaring `state_schema` -alone does not fill `chat.state`. +## Chat and streaming -## Chat +Run one turn at a time per `AgentChat`. Use `ai.run_main(main())` for standalone scripts. ```python -chat = agent.chat() +async def main() -> None: + chat = agent.chat() -res = await chat.send('Weather in Tokyo?') -print(res.text, res.snapshot_id) + # 1. One-shot turn + res = await chat.send('Weather in Tokyo?') + print(res.text, res.snapshot_id) + # => It is 21°C and sunny in Tokyo. 8f3c... -turn = chat.send_stream('What about Paris?') -async for chunk in turn.stream: - if chunk.text: - print(chunk.accumulated_text, end='\r', flush=True) -final = await turn.response -``` - -Send returns a response. Stream returns a turn with `.stream` and `.response`. -After an interrupt, use `resume` / `resume_stream`. + # 2. Streamed turn (.stream yields AgentChunk, .response resolves to AgentResponse) + turn = chat.send_stream('What about Paris?') + async for chunk in turn.stream: + if chunk.text: + print(chunk.accumulated_text, end='\r', flush=True) + final = await turn.response -You can await the response without reading every chunk. One send at a time per -`chat`. + # 3. Rehydrate history from the store on a new chat instance + resumed = await agent.load_chat(snapshot_id=final.snapshot_id) + followup = await resumed.send('Which cities did we check?') + print(followup.text) -To reopen a stored conversation, use `load_chat` (not `chat(snapshot_id=...)`, -which only attaches a resume handle): -```python -resumed = await agent.load_chat(snapshot_id=res.snapshot_id) -await resumed.send('What city did I ask about?') +if __name__ == '__main__': + ai.run_main(main()) ``` -## Verify an agent from the CLI (`flow:run`) +- **`await agent.load_chat(snapshot_id=...)`**: fetches the stored snapshot immediately so `chat.messages`, `chat.state`, and `chat.artifacts` are populated locally. +- **`agent.chat(session_id=...)`**: attaches a lazy resume handle that continues from the session's last completed turn, skipping any failed or aborted leaf. -`genkit flow:run` only runs **flows**, not agents, so you can't `flow:run` an -agent directly. To exercise an agent from the CLI (e.g. a quick, self-terminating -check), wrap one turn in a throwaway flow and run that: +## Errors and cancellation -```python -@ai.flow() -async def try_weather_agent(message: str) -> str: - return (await agent.chat().send(message)).text +Genkit separates turn failures from call failures: -# genkit flow:run try_weather_agent '"Weather in Tokyo?"' -- uv run src/main.py +- **Turn failure (`res.error`)**: When an agent turn starts and fails at runtime, `chat.send(...)` does **not** raise. It returns `AgentResponse` with `res.finish_reason == AgentFinishReason.FAILED`, `res.text == ''`, and `res.error` (`Failure` with `status`, `message`, `details`, and `reason: ErrorReason`). The chat drops the unanswered user prompt from `chat.messages`, and the next `send()` continues from the last completed snapshot. +- **No turn result (`GenkitError`)**: When no turn runs at all (dropped HTTP connection, or `AgentInitError` from passing `snapshot_id` to a storeless agent), `chat.send(...)` raises `GenkitError` and restores local chat state. Mismatched init arguments (a `state` blob to a store-backed agent, or `messages`/`state` mixed with a resume id) raise `AgentInitError` from `agent.chat(...)` itself. +- **Cancellation**: native `asyncio.CancelledError`; put tool cleanup in `try` / `finally`. `await turn.abort()` detaches the client stream only. To cancel a running turn server-side, start it with `chat.detach(...)` and call `await task.abort()`; `chat.abort()` during a foreground `send()` does not stop that turn ([background](agents-background.md#client-detach-vs-server-cancellation)). + +```python +from genkit import ErrorReason, GenkitError +from genkit.exp.agent import AgentFinishReason + +try: + res = await chat.send('Run portfolio rebalance.') +except GenkitError as err: + # No turn ran (transport drop or invalid init) + print(err.status, err.message) +else: + if res.error: + # Turn ran and failed; res.finish_reason == AgentFinishReason.FAILED + assert res.finish_reason == AgentFinishReason.FAILED + if res.error.reason == ErrorReason.INVALID_OUTPUT: + print('Truncated structured output:', res.error.message) ``` ## Without a store -Skip `store` when your app owns history. Ids stay `None` — pass messages, -state, and artifacts into the next `chat(...)`: +Omit `store=` when the client owns conversation state. `snapshot_id` stays `None`; round-trip `messages`, `state`, and `artifacts` into the next `agent.chat(...)` call: ```python -agent = ai.define_agent( +stateless = ai.define_agent( name='echoNoStore', - model='googleai/gemini-flash-latest', - system='Echo assistant. Answer briefly and remember context.', + model=GoogleAI.gemini('gemini-flash-latest'), + system='Answer briefly and remember context.', ) -chat = agent.chat() -await chat.send('My name is Ada. Remember it.') +chat = stateless.chat() +await chat.send('My name is Ada.') -resumed = agent.chat( - messages=chat.messages, state=chat.state, artifacts=chat.artifacts +resumed = stateless.chat( + messages=chat.messages, + state=chat.state, + artifacts=chat.artifacts, ) await resumed.send('What is my name? One word.') ``` -Add a [store](agents-sessions.md) when the server should own history, or when -you need branching and detach. [Interrupts](agents-human-in-the-loop.md) work -either way. +## Verify an agent from the CLI (`flow:run`) + +`genkit flow:run` invokes flows, not agents directly. Wrap a turn in a small flow and propagate `res.error` so failures surface cleanly: -## Auth +```python +from genkit import GenkitError -Put identity on the server with `serve_agent` and a FastAPI dependency. The -remote client only forwards credentials. See [HTTP](agents-http.md). -## Prompt agents +@ai.flow() +async def try_weather_agent(message: str) -> str: + res = await agent.chat().send(message) + if res.error: + raise GenkitError(status=res.error.status, message=res.error.message, details=res.error.details) + return res.text -`define_prompt_agent(name=...)` uses the same name for the agent and the -`.prompt` file stem. Keep preamble inputs stable for the chat — put dynamic -fields in user messages or tools. +# genkit flow:run try_weather_agent '"Weather in Tokyo?"' -- uv run src/main.py +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/common-errors.md b/plugins/genkit/agent/skills/developing-genkit-python/references/common-errors.md index fba68af8..f5715cdd 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/common-errors.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/common-errors.md @@ -1,133 +1,323 @@ -# Common Errors - -Quick fixes. Longer recipes live in the linked guides. +# Common Errors — Genkit Python 0.13 + +Find the symptom, apply the fix. Every message below comes from a real 0.13 run. Upgrading a 0.12 codebase? Read [Migrating from 0.12 to 0.13](migrating-from-0.12.md). + +- [ai.generate Returns Empty Text and Does Not Raise](#aigenerate-returns-empty-text-and-does-not-raise): Lines 23-48 +- [Thinking level MINIMAL is not supported for this model](#thinking-level-minimal-is-not-supported-for-this-model): Lines 52-70 +- [ImportError on ActionRunContext, ToolRunContext, MediaPart, TextPart, or genkit.agent](#importerror-on-actionruncontext-toolruncontext-mediapart-textpart-or-genkitagent): Lines 74-106 +- [GenkitError INVALID_ARGUMENT on Config Keys](#genkiterror-invalid_argument-on-config-keys): Lines 110-134 +- [GenkitError INVALID_ARGUMENT: API Key in config](#genkiterror-invalid_argument-api-key-in-config): Lines 138-150 +- [GenkitError NOT_FOUND: Unprefixed Model String](#genkiterror-not_found-unprefixed-model-string): Lines 154-162 +- [Vertex AI Fails on gemini-flash-latest](#vertex-ai-fails-on-gemini-flash-latest): Lines 166-174 +- [TypeError When Defining or Calling Flows and Tools](#typeerror-when-defining-or-calling-flows-and-tools): Lines 178-206 +- [Served Flow Returns 404, 500, or the Wrong Auth Status](#served-flow-returns-404-500-or-the-wrong-auth-status): Lines 210-233 +- [response.output Is None on Structured Output](#responseoutput-is-none-on-structured-output): Lines 237-250 +- [response.json Is Not the Model Output](#responsejson-is-not-the-model-output): Lines 254-262 +- [{{variable}} Sent Literally](#variable-sent-literally): Lines 266-274 +- [Streaming TypeError or a Flow That Keeps Running After break](#streaming-typeerror-or-a-flow-that-keeps-running-after-break): Lines 278-297 +- [Cloud Telemetry Enabled but Spans Do Not Reach Cloud Trace](#cloud-telemetry-enabled-but-spans-do-not-reach-cloud-trace): Lines 301-309 +- [Interrupts and Agent Session Pitfalls](#interrupts-and-agent-session-pitfalls): Lines 313-323 --- -## Wrong Google AI import +## `ai.generate` Returns Empty Text and Does Not Raise + +`ai.generate` boxes model-side failures into the response: provider rejections, `UNAVAILABLE`, rate limits, and model IDs the provider does not serve. You get `response.text == ''`, `response.finish_reason == 'failed'`, and a `Failure` on `response.error`. Code that only reads `response.text` silently continues with an empty string. ```python +from genkit import Genkit from genkit_google_genai import GoogleAI -``` -```bash -uv add genkit genkit-google-genai +ai = Genkit(plugins=[GoogleAI()]) + +# 1. Prefixed but unserved model ID: no exception +res = await ai.generate(model='googleai/gemini-9-ultra', prompt='Suggest a dish.') +print(repr(res.text), res.finish_reason) +# => '' FinishReason.FAILED + +# 2. Check response.error after every call +if res.error: + print(res.error.status, res.error.message) + # => NOT_FOUND models/gemini-9-ultra is not found for API version v1beta, or is not supported for generateContent. ... ``` +Problems caught before the request is sent raise instead: + +- **`ValueError`** from `GoogleAI()` when `GEMINI_API_KEY` is unset (`[Genkit] GEMINI_API_KEY environment variable not set.`). +- **`GenkitError` `UNAUTHENTICATED`** on the first `ai.generate` when the key is invalid (`API key not valid. Please pass a valid API key.`). +- **`GenkitError`** for an unprefixed model string, unknown config keys, and `api_key` in config (sections below). + --- -## Streaming TypeError +## `Thinking level MINIMAL is not supported for this model` -Do not await the stream handle — await the response: +`gemini-flash-latest` rejects `thinking_level='MINIMAL'`. The call returns a failed response (see above), not an exception: ```python -turn = chat.send_stream('hi') -async for chunk in turn.stream: ... -res = await turn.response +from genkit_google_genai import GeminiConfig + +res = await ai.generate(prompt='Name one dessert.', config=GeminiConfig(thinking_level='MINIMAL')) +print(res.error.status, res.error.message) +# => INVALID_ARGUMENT Thinking level MINIMAL is not supported for this model. Please retry with other thinking level. +``` -sr = ai.generate_stream(prompt='...') -async for chunk in sr.stream: ... -final = await sr.response +Use `'LOW'`, `'MEDIUM'`, or `'HIGH'`: + +```python +res = await ai.generate(prompt='Name one dessert.', config=GeminiConfig(thinking_level='LOW')) +print(res.text) +# => Tiramisu. ``` --- -## Tool schema must be an object +## `ImportError` on `ActionRunContext`, `ToolRunContext`, `MediaPart`, `TextPart`, or `genkit.agent` -Wrap tool params in a Pydantic model. Bare `str` / `float` arguments fail on -Gemini. See [agents.md](agents.md). +```text +ImportError: cannot import name 'ActionRunContext' from 'genkit' +ImportError: cannot import name 'MediaPart' from 'genkit' +ModuleNotFoundError: No module named 'genkit.agent' +AttributeError: 'Genkit' object has no attribute 'define_agent' +ValidationError: 1 validation error for Part + Value error, Part(root=...) is gone; use Part.from_text or Part(text=...) +``` -```python -class WeatherInput(BaseModel): - city: str +Each symbol has one public home: -@ai.tool() -async def get_weather(input: WeatherInput) -> str: ... +```python +# Contexts +from genkit import FlowContext, ToolContext +from genkit.model import ModelContext + +# Parts +from genkit import Media, Message, Part, Role +text_part = Part.from_text('Hello') +img_part = Part.from_media('https://example.com/photo.jpg', content_type='image/jpeg') + +# Experimental agents and integrations (define_agent exists only on genkit.exp.Genkit) +from genkit.exp import Genkit, current_session +from genkit.exp.agent import Agent, InMemorySessionStore, Session +from genkit_fastapi.exp import serve_agent +from genkit_middleware.exp import Artifacts ``` +Two different `Middleware` imports: +- `from genkit_middleware import Middleware, Retry, ToolApproval` imports the **`Middleware()` plugin** and built-in middleware. +- `from genkit.middleware import Middleware, MiddlewareContext` imports the **base class** for custom middleware. + --- -## `define_tool` missing +## `GenkitError` `INVALID_ARGUMENT` on Config Keys -Use `@ai.tool()`. +Genkit validates config before calling the model. Typos, nested dicts, and fields that moved off config all raise: ---- +```text +INVALID_ARGUMENT: googleai/gemini-flash-latest: unknown config key 'temprature'; put provider-only settings in config['extra'] +INVALID_ARGUMENT: thinking_config was flattened; use include_thoughts / thinking_budget / thinking_level +INVALID_ARGUMENT: googleai/gemini-flash-latest: unknown config key 'tool_choice'; put provider-only settings in config['extra'] +``` -## Model id without a prefix +Use flat typed config, put provider-native keys in `extra`, and pass `tool_choice` on the call: -Use `googleai/gemini-flash-latest`, not bare `gemini-flash-latest`. +```python +from genkit_google_genai import GeminiConfig + +cfg = GeminiConfig( + temperature=0.2, + thinking_level='HIGH', + include_thoughts=True, + extra={'seed': 7}, +) +res = await ai.generate(prompt='Pick a wine for salmon.', config=cfg, tool_choice='required', tools=[...]) +``` + +`GeminiConfig.extra` is wire-shaped and deep-merged into the request body, so nested provider fields go under their wire key (for example `extra={'generationConfig': {...}}`). --- -## Missing `.json` / `.message` on the response +## `GenkitError` `INVALID_ARGUMENT`: API Key in `config` -Plain text → `response.text`. Structured output → `response.output`. +```text +INVALID_ARGUMENT: API key belongs in context.secrets, not config. Pass the key as context={'secrets': {'api_key': ...}}. +``` + +Set the key on the plugin, or pass a per-request key in `context`: + +```python +# Whole app: GoogleAI(api_key=...), OpenAI(api_key=...) +# Per request: +await ai.generate(prompt='Hello', context={'secrets': {'api_key': user_key}}) +``` --- -## Event loop issues under `genkit start` +## `GenkitError` `NOT_FOUND`: Unprefixed Model String + +```text +NOT_FOUND: Failed to resolve model 'gemini-flash-latest'. Ensure the model name includes the plugin namespace (e.g., 'plugin/model'). +``` + +Plain strings need the `<plugin>/` prefix (`'googleai/gemini-flash-latest'`, `'openai/gpt-4o'`). Model ref helpers (`GoogleAI.gemini('gemini-flash-latest')`, `Anthropic.claude('claude-sonnet-4-6')`, `OpenAI.gpt('gpt-5.2')`) add it for you. -Use `ai.run_main(main())` for long-lived apps. See [fastapi.md](fastapi.md) -for serving under Dev UI. +A prefixed ID the provider does not serve resolves and fails at call time on `response.error` instead (see the first section). --- -## No `snapshot_id` after a turn +## Vertex AI Fails on `gemini-flash-latest` -Attach a `store`. Without one, pass messages / state / artifacts yourself — -[agents.md](agents.md). +Vertex AI does not serve the Gemini API's rotating `-latest` aliases (`gemini-flash-latest`, `gemini-flash-lite-latest`, `gemini-pro-latest`). The Vertex plugin's known-model list omits them. Use a versioned ID: + +```python +from genkit_google_genai import VertexAI + +ai = Genkit(plugins=[VertexAI(location='us-central1')], model=VertexAI.gemini('gemini-3.5-flash')) +``` --- -## Cannot send state to a server-managed agent +## `TypeError` When Defining or Calling Flows and Tools -Store-backed chats resume by id. Update fields inside the turn — -[agents-state.md](agents-state.md). +```text +TypeError: Tool function must be async. Got sync function: lookup_reservation +TypeError: Flow must be async: sync_menu +TypeError: tool 'check_allergens' takes one input, but 'allergen' is a second parameter. Put the fields on one input model, or annotate 'allergen' as ToolContext. +TypeError: Genkit() takes no positional arguments, got 1. Pass keyword arguments instead, e.g. Genkit(plugins=[...], model="..."). +TypeError: Action.run() takes from 1 to 2 positional arguments but 3 were given +``` + +- `@ai.tool()` and `@ai.flow()` functions must be `async def`. +- Each takes at most one input parameter. A context parameter must be annotated (`ctx: FlowContext` or `ctx: ToolContext`); an unannotated `ctx` counts as a second input. Wrap multi-field input in one Pydantic `BaseModel`. +- Constructors (`Genkit`, plugins, `GenkitError`) and every option after the input on `.run(...)` are keyword-only: + +```python +from pydantic import BaseModel +from genkit import ToolContext + +class AllergenCheck(BaseModel): + dish: str + allergen: str + +@ai.tool() +async def check_allergens(check: AllergenCheck, ctx: ToolContext) -> bool: + """Return True if the dish contains the allergen.""" + return check.allergen in MENU[check.dish].allergens + +res = await plan_menu.run(payload, on_chunk=handle_chunk, context={'uid': 'u1'}) +``` --- -## Ambiguous session after a fork +## Served Flow Returns 404, 500, or the Wrong Auth Status -Resume with a leaf `snapshot_id`, not `session_id` — -[agents-branching.md](agents-branching.md). +Over HTTP (`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, `handle_genkit_request`): + +- **`404 {"detail":"Not Found"}`**: the route is the Python function name. `menu_suggestion_flow` mounts at `POST /menu_suggestion_flow`, not `/menuSuggestionFlow`. Override with `serve_flow(flow, base_path=...)`. +- **`500 {"message":"Internal Error","status":"INTERNAL"}`**: the flow raised `GenkitError` or any non-`PublicError` exception. Genkit logs the traceback server-side and redacts the body. Raise `PublicError` for errors the client should see. +- **`400 ... "status":"INVALID_ARGUMENT"`**: the request `data` failed input validation. +- **`500 Internal Server Error` (plain text) from a `context_dependency`**: `serve_flow(..., context_dependency=...)` runs a FastAPI dependency before Genkit's handler, so a `PublicError` raised there is not mapped. Raise `fastapi.HTTPException(status_code=401)` in dependencies; `PublicError('UNAUTHENTICATED', ...)` maps to 401 only inside a `genkit_fastapi_handler` `context_provider` or the flow itself. + +```python +from genkit import PublicError + +@ai.flow() +async def get_invoice(invoice_id: str) -> dict: + inv = await find_invoice(invoice_id) + if inv is None: + raise PublicError('NOT_FOUND', f'Invoice {invoice_id} not found') + return inv + +# POST /get_invoice {"data": "INV-9"} +# => 404 {"message":"Invoice INV-9 not found","status":"NOT_FOUND"} +``` + +Full status mapping: [FastAPI](fastapi.md#http-error-redaction--publicerror). --- -## Approval resume does nothing +## `response.output` Is `None` on Structured Output -Pass approval metadata on the restart part: +When a JSON response is truncated (often by `max_output_tokens`) or does not parse against the schema, `response.output` is `None` and `response.error.reason` is `ErrorReason.INVALID_OUTPUT`: ```python -await chat.resume( - restart=[ - i.restart(resumed_metadata={'tool_approved': True}) - for i in out.interrupts - ] -) +from genkit import ErrorReason + +res = await ai.generate(prompt='Summarize this invoice.', output_schema=InvoiceSummary) +if res.output is None and res.error and res.error.reason == ErrorReason.INVALID_OUTPUT: + print(res.error.message) + # => Model output was not valid JSON for the requested schema: {"vendor": "Acme", "total": 12 ``` -Build from `res.interrupts`. Full flow: -[agents-human-in-the-loop.md](agents-human-in-the-loop.md). +Raise `max_output_tokens` or simplify the schema, then retry. --- -## Abort seems to do nothing +## `response.json` Is Not the Model Output + +`response.json` is Pydantic's deprecated `BaseModel.json` method. Calling it serializes the whole `ModelResponse` and emits `PydanticDeprecatedSince20`. Read: + +- `response.text` for plain text. +- `response.output` for structured output. +- `response.message` for the full message. -Await it. Client stop: `turn.abort()` (or a timeout around the stream). -Server cancel: `chat.abort()` / `task.abort()` after you have a snapshot. -Details: [agents-background.md](agents-background.md). +`GenkitError` exposes `e.message`, `e.status`, `e.details`, `e.reason`, and `e.__cause__`. `e.original_message` raises `AttributeError`. --- -## Tool raise becomes `AgentError` +## `{{variable}}` Sent Literally -A raised tool ends the turn. Catch `AgentError` on `chat.send`. Prefer -returning a soft-error dict so the model can recover. History stays usable — -the next send continues from the last good snapshot. +`ai.generate(prompt=..., system=...)` sends strings verbatim. The model receives `Say hello to {{name}}.` as written. Render templates through `ai.define_prompt` or a `.prompt` file: + +```python +greet = ai.define_prompt(name='greet', prompt='Say hello to {{name}}.') +res = await greet({'name': 'Sam'}) +# Model receives: Say hello to Sam. +``` --- -## Remote client oddities +## Streaming `TypeError` or a Flow That Keeps Running After `break` + +```text +TypeError: object ModelStreamResponse can't be used in 'await' expression +``` + +1. Do not `await` `ai.generate_stream(...)`, `flow.stream(...)`, or `chat.send_stream(...)`. Iterate the stream, then `await` the final response: + ```python + sr = ai.generate_stream(prompt='Describe the tasting menu.') + async for chunk in sr.stream: + print(chunk.text, end='') + final = await sr.response + ``` +2. To stop a flow stream early, use `async with`. Leaving the block cancels the flow: + ```python + async with course_flow.stream(payload) as stream: + async for chunk in stream: + if chunk == 'dessert': + break # course_flow receives CancelledError + ``` + +--- + +## Cloud Telemetry Enabled but Spans Do Not Reach Cloud Trace + +By default in local development (`GENKIT_ENV=dev`), `enable_google_cloud_telemetry()` runs in local-only mode to prevent polluting Cloud Trace with local run data. To export traces to GCP from local development, pass `force_dev_export=True`: + +```python +from genkit_google_cloud import enable_google_cloud_telemetry + +enable_google_cloud_telemetry(project='bistro-prod', force_dev_export=True) +``` + +--- -Match `state_management` to the server. No trailing slash on the URL. Check -server logs. Auth: [agents-http.md](agents-http.md). +## Interrupts and Agent Session Pitfalls + +- **`except Exception:` does not catch `Interrupt`**: `Interrupt` subclasses `BaseException`, so a tool's catch-all cannot swallow approval interrupts. Put side-effect cleanup in `finally`. +- **Approval resume**: build restart parts with `part.restart(resumed_metadata={'tool_approved': True})` on items from `res.interrupts` (see [agents-human-in-the-loop.md](agents-human-in-the-loop.md)). +- **`ImportError: cannot import name 'AgentError'`**: failed turns do not raise. `chat.send()` returns a turn with `finish_reason == AgentFinishReason.FAILED` and `res.error`. `AgentInitError` raises from `agent.chat(...)` on bad init; `send` raises `GenkitError` only when no turn result arrives. +- **`res.snapshot_id` is `None`**: the agent has no store. Pass `store=InMemorySessionStore()` to `define_agent` (see [agents.md](agents.md)). +- **`AgentInitError: FAILED_PRECONDITION: Cannot send 'state' to a server-managed agent (one with a store). Send 'snapshot_id' or 'session_id' instead.`**: store-backed chats resume by ID. Mutate state inside the turn (see [agents-state.md](agents-state.md)). +- **Resume picks the wrong branch after a fork**: `session_id` resumes the most recently created leaf. Pass the branch's `snapshot_id` (see [agents-branching.md](agents-branching.md)). +- **Resume after a failed turn**: only `completed` snapshots resume by `snapshot_id`. Use `agent.chat(session_id=...)` to continue from the last good turn. +- **`chat.abort()` does nothing**: during a foreground `send` it is a no-op. Run the turn with `detach()` and call `await task.abort()` (see [agents-background.md](agents-background.md)). +- **Remote client mismatches**: align `state_management` with the server and omit trailing slashes on the base URL (see [agents-http.md](agents-http.md)). diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/dev-workflow.md b/plugins/genkit/agent/skills/developing-genkit-python/references/dev-workflow.md index 74681b1e..220d5e38 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/dev-workflow.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/dev-workflow.md @@ -1,127 +1,170 @@ # Dev Workflow — Genkit Python -## Agent responsibility +- [Agent Responsibility](#agent-responsibility): Lines 14-23 +- [Step 1 - Get a Gemini API Key](#step-1--get-a-gemini-api-key): Lines 27-29 +- [Step 2 - Export the API Key](#step-2--export-the-api-key): Lines 33-43 +- [Step 3 - Install Dependencies](#step-3--install-dependencies): Lines 47-56 +- [Step 4 - Start the Dev UI](#step-4--start-the-dev-ui): Lines 60-104 +- [Step 5 - Test in the Dev UI](#step-5--test-in-the-dev-ui): Lines 108-117 +- [CLI Commands](#cli-commands): Lines 121-160 +- [Troubleshooting](#troubleshooting): Lines 164-170 -After generating code, always give the developer: -1. The full pre-run checklist with copy-paste commands using absolute paths -2. The `genkit start` command to run in their terminal (foreground — it's expected to block) -3. Step-by-step Dev UI instructions so they can test without guessing +--- + +## Agent Responsibility -Do not offer to run it for them. Give them the commands and let them run it. +After generating or migrating code: +1. Run the project's type checker (`pyright`, `mypy`, or `ty`) in `.venv` so typed configs (`GeminiConfig`), generic contexts (`FlowContext[T]`, `ToolContext[T]`), keyword-only signatures, and `OutputT | None` checks pass before handing off. +2. Give the developer: + - Copy-paste setup commands with absolute paths + - The `genkit start` command to run in their foreground terminal (expected to block) + - Exact Dev UI steps to exercise the flow or agent and inspect traces + +Do not run `genkit start` as an interactive blocking command on the developer's behalf unless explicitly asked; use `genkit flow:run` for automated one-shot verification. --- -## Step 1 — Get a Gemini API key +## Step 1 — Get a Gemini API Key -If the developer doesn't have one: -> Get a free key at https://aistudio.google.com/apikey — click **"Create API key"**, copy it. +If the developer needs a key, create one at `https://aistudio.google.com/apikey`. --- -## Step 2 — Set the API key +## Step 2 — Export the API Key -Open a terminal and run: ```bash export GEMINI_API_KEY=your-api-key-here ``` -To persist across sessions, add it to your shell profile: +Persist across shell sessions if desired: + ```bash echo 'export GEMINI_API_KEY=your-api-key-here' >> ~/.zshrc && source ~/.zshrc ``` --- -## Step 3 — Install dependencies +## Step 3 — Install Dependencies -Replace `/path/to/your-project` with the actual full path to the project (e.g. `/Users/yourname/projects/my-genkit-app`): +Replace `/path/to/your-project` with the project's absolute path: ```bash cd /path/to/your-project -uv add genkit genkit-google-genai +uv add 'genkit==0.13.*' 'genkit-google-genai==0.13.*' ``` -(Requires a project with `pyproject.toml` — run `uv init` in an empty directory first if needed.) +(Run `uv init` first if `pyproject.toml` does not exist yet — see [Setup](setup.md).) --- ## Step 4 — Start the Dev UI -Run this in your terminal. **It will block — that's expected.** Leave this terminal open while you use the Dev UI. +The examples below use this `src/main.py`: + +```python +from pydantic import BaseModel + +from genkit import Genkit + +ai = Genkit() + + +class OrderInput(BaseModel): + text: str + + +@ai.flow() +async def order_flow(inp: OrderInput) -> str: + return f'order: {inp.text}' + + +async def main() -> None: + print(await order_flow(OrderInput(text='smoked salmon tartine'))) + + +if __name__ == '__main__': + ai.run_main(main()) + # => order: smoked salmon tartine + # Under `genkit start`, the process stays up for the Dev UI until Ctrl+C. +``` + +Run in a foreground terminal (blocks until stopped with `Ctrl+C`): ```bash cd /path/to/your-project GEMINI_API_KEY=your-api-key-here genkit start -- uv run src/main.py ``` -You'll see output like: -``` -Genkit Tools UI: http://localhost:4000 -``` +For FastAPI apps, run uvicorn under `genkit start` instead; the reflection server starts on a background thread (see [FastAPI](fastapi.md)): -The Dev UI is now running at **http://localhost:4000** +```bash +GEMINI_API_KEY=your-api-key-here genkit start -- uv run uvicorn main:app --reload +``` -To stop it: press `Ctrl+C` in the terminal. +Wait for the CLI to print `Genkit Developer UI: http://localhost:4000`. --- ## Step 5 — Test in the Dev UI -1. Open **http://localhost:4000** in your browser -2. Click **"Run"** in the left sidebar -3. Find your flow by name (e.g. `summarize`, `chat`, `joke_generator`) -4. In the input box, paste your input as JSON — e.g: +1. Open **`http://localhost:4000`** in a browser. +2. Pick the registered flow, prompt, or model from the left sidebar. Flows appear under their Python function name (e.g. `order_flow`). +3. Paste JSON matching the flow's single input parameter, e.g. for `order_flow(inp: OrderInput)`: ```json - {"text": "hello world"} + {"text": "smoked salmon tartine"} ``` -5. Click the **"Run"** button — the output appears on the right -6. Click **"Traces"** in the left sidebar to inspect every step, model call, token count, and latency +4. Click **Run** and inspect the validated output on the right. +5. Open **Traces** in the left sidebar to inspect every span, rendered prompt, tool call, token count, latency, and unwrapped error stack trace. --- ## CLI Commands -`genkit start` unintrusively wraps any Python program that uses the Genkit library, running it unchanged while capturing traces from every Genkit action so you can prove tools were actually called and inspect model I/O from the terminal, even for headless checks. It forwards stdio, so interactive CLI tools that rely on stdin/stdout work without issues. Running the app directly (`uv run`) skips trace capture, so you're debugging blind. +`genkit start` sets `GENKIT_ENV=dev`, runs your command, and stores local traces from every Genkit action under `.genkit/` in the project root. The app's stdout and stderr pass through. Running `uv run` directly skips the reflection server and local trace capture. -**Primary pattern (default):** prefix `genkit start --` to your normal run command. This collects telemetry from any Genkit code your program runs, whether triggered from the dev UI, your own web server/web UI, or a plain script: +**Persistent dev runtime:** ```bash genkit start -- uv run src/main.py -genkit start --noui -- uv run src/main.py # same, without the Dev UI (still a persistent server) +genkit start --noui -- uv run src/main.py # no Dev UI; runtime still blocks until Ctrl+C +genkit start --non-interactive -- uv run src/main.py # CLI prompts take the default choice (CI/agents) +genkit start --port 4001 -- uv run src/main.py # Dev UI on another port ``` -`genkit start` runs until you stop it with Ctrl+C. That is expected and correct for the common cases: a server your web/mobile app calls, or an interactive CLI you exit yourself. `--noui` only drops the Dev UI; it is **not** a one-shot command and will not exit on its own. Do **not** use `genkit start` as a blocking step in automated/non-interactive contexts; use `flow:run` (below) for that. -**Non-interactive use (agents/CI):** add the global `--non-interactive` flag before `--` so the CLI uses defaults and never blocks on a prompt (e.g. the first-run analytics notice): `genkit start --non-interactive -- uv run src/main.py` (works with `flow:run` too). - -**Run a flow (`flow:run`):** invoke a specific flow by name from the CLI. Append your run command after `--` to spin up the runtime just for this run (the command runs as-is to register your flows): +**One-shot flow execution (`flow:run`):** with `-- <cmd>`, it starts the app, runs one `@ai.flow()` by its Python function name, prints `Result:` and `Trace ID:`, and exits: ```bash -genkit flow:run myFlow '{"data": "input"}' -- uv run src/main.py +genkit flow:run order_flow '{"text": "smoked salmon tartine"}' -- uv run src/main.py +# => Result: +# order: smoked salmon tartine +# Trace ID: 395f73c3d0d662c28711239a4354bf53 ``` -This is **self-terminating**: it runs the flow once, prints a `Trace ID`, then exits, so it's the right choice for a quick, non-interactive check (unlike `genkit start`). Note: `flow:run` runs **flows** (`@ai.flow()`), not agents; you can't `flow:run` an agent (`ai.define_agent`) directly. To exercise an agent from the CLI, wrap one turn in a throwaway flow and run that (see [Agents](agents.md)). Traces for this run can be inspected using the trace commands below. +Without `-- <cmd>`, `flow:run` targets a runtime already started by `genkit start` in the same directory, and fails with `No runtimes found` if none is running. A `str` input is a JSON string: `'"risotto"'`. + +`flow:run` runs `@ai.flow()` actions, not agents. Wrap one agent turn in a small flow to test it (see [Agents](agents.md#verify-an-agent-from-the-cli-flowrun)). -**Debugging with traces:** the fastest way to see prompts, model inputs/outputs, tool calls, latencies, and errors. Inspect from the terminal after any run under `genkit start`: +**Inspecting traces from the terminal:** ```bash -genkit trace:list # find recent trace IDs -genkit trace:get <traceId> # full trace details (inputs, outputs, tool calls, errors) -genkit trace:get <traceId> --format json # machine-readable JSON, safe to pipe into jq or other parsers +genkit trace:list --limit 5 # recent trace IDs with status, input, output +genkit trace:get <traceId> # full trace as JSON: every span, attributes, input, output +``` +`trace:get` prints a `Telemetry API running on ...` line before the JSON on stdout, so strip it before piping into `jq`: +```bash +genkit trace:get <traceId> | sed 1d | jq -r '.spans[].displayName' +# => order_flow ``` -For machine-readable output, pass `--format json` to get clean JSON you can pipe into `jq` or other parsers. The **default** output is human-oriented (banner/log lines, possible truncation on large traces), so don't pipe that form directly; use `--format json`, grep, or the Dev UI trace viewer. - - -**Documentation:** +**Built-in CLI documentation lookup:** ```bash genkit docs:search "streaming" python genkit docs:list python genkit docs:read python/flows.md ``` +--- ## Troubleshooting - `genkit: command not found` — `npm install -g genkit-cli` -- `GEMINI_API_KEY not set` — `export GEMINI_API_KEY=your-key` -- Port 4000 already in use — - `genkit start --port 4001 -- uv run src/main.py` -- `uv: command not found` — - `curl -LsSf https://astral.sh/uv/install.sh | sh` -- Flow not showing in Dev UI — check `genkit start` output for errors +- `GEMINI_API_KEY` missing — `export GEMINI_API_KEY=your-key` +- Port 4000 busy — `genkit start --port 4001 -- uv run src/main.py` +- `uv: command not found` — `curl -LsSf https://astral.sh/uv/install.sh | sh` +- Action missing in Dev UI — ensure standalone scripts enter via `ai.run_main(main())` and check terminal output for startup import or validation errors (see [Common Errors](common-errors.md)). diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/dotprompt.md b/plugins/genkit/agent/skills/developing-genkit-python/references/dotprompt.md index 0bc91ad5..0797a0a4 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/dotprompt.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/dotprompt.md @@ -1,166 +1,190 @@ # Dotprompt — Genkit Python -## What it is +- [What It Is](#what-it-is): Lines 15-17 +- [File Format](#file-format): Lines 19-54 +- [Python Setup](#python-setup): Lines 56-77 +- [Calling a Prompt (Prompt)](#calling-a-prompt-prompt): Lines 79-113 +- [Streaming from a Prompt](#streaming-from-a-prompt): Lines 115-132 +- [Render Without Generating](#render-without-generating): Lines 134-146 +- [Helpers](#helpers): Lines 148-168 +- [Variants & Partials](#variants--partials): Lines 170-173 +- [Prompts with Tools & Middleware](#prompts-with-tools--middleware): Lines 175-190 -`.prompt` files combine YAML frontmatter (model config, schemas) with Handlebars -templates. Keeps prompt logic out of Python code and makes variants easy. +--- + +## What It Is + +`.prompt` files pair YAML frontmatter (model, config, input/output schemas, tools, middleware) with Handlebars templates. Handlebars templates **only** render inside `.prompt` files and `ai.define_prompt(...)` — strings passed to `ai.generate(prompt=..., system=...)` are sent to the model verbatim. -## File format +## File Format + +Place `.prompt` files inside `prompts/` and pass `prompt_dir=...` to `Genkit(...)`: ```yaml --- model: googleai/gemini-flash-latest +config: + temperature: 0.4 input: + default: + tone: practical schema: food: string - tone?: string # optional scalar — just `field?: type` - ingredients?(array): string # optional array/object needs the parenthetical + tone?: string # optional scalar — `field?: type` + ingredients?(array): string # optional array/object uses parenthetical syntax output: - schema: Recipe # references a schema registered with ai.define_schema() + schema: Recipe # references a schema registered via ai.define_schema() format: json --- {{role "system"}} -You are a chef. Keep recipes practical. +You are a chef. Keep recipes {{tone}}. {{role "user"}} Generate a recipe for {{food}}. {{#if ingredients}} -Prefer these ingredients: {{ingredients}}. +Prefer these ingredients: +{{list ingredients}} {{/if}} ``` -Use `{{role "system"}}` / `{{role "user"}}` (and friends) so the model sees a -real system instruction. A flat template body becomes a single **user** message. - -Place `.prompt` files in a `prompts/` directory and point `prompt_dir` at it. +Use `{{role "system"}}` and `{{role "user"}}` to emit distinct message roles; a template without `{{role ...}}` becomes a single user message. -**Note:** `input.schema` describes the contract for humans and tooling. Today it -is **not** validated locally before the model call — missing required fields can -still render and hit the API. Validate in your flow if you need hard failures. +**Input & output validation:** +- Prompt inputs **are validated** against `input.schema` (or `input_schema=` on `define_prompt`) before calling the model, and `input.default` values from `.prompt` frontmatter are filled in automatically. +- When `output.schema` names a Pydantic class registered with `ai.define_schema('Recipe', Recipe)`, `response.output` is validated and returned directly as `Recipe | None` (`None` when the model's JSON is truncated or malformed; check `response.finish_reason`). -## Python setup +## Python Setup ```python from pathlib import Path from pydantic import BaseModel -from genkit import Genkit +from genkit import Genkit, Prompt from genkit_google_genai import GoogleAI ai = Genkit( plugins=[GoogleAI()], - model='googleai/gemini-flash-latest', + model=GoogleAI.gemini('gemini-flash-latest'), prompt_dir=Path(__file__).resolve().parent.parent / 'prompts', ) + class Recipe(BaseModel): title: str steps: list[str] + ai.define_schema('Recipe', Recipe) ``` -## Calling a prompt +## Calling a Prompt (`Prompt`) + +`ai.prompt(...)` and `ai.define_prompt(...)` return a `Prompt`. There is no `.execute()` method — call and `await` the `Prompt` directly: + +```python +recipe_prompt: Prompt = ai.prompt('recipe') + +# Call options use the same keyword names as `ai.generate`. +# `config` merges key-by-key over the prompt's frontmatter config. +# `output_schema` is fixed by the prompt and cannot be overridden per call. +response = await recipe_prompt( + {'food': 'banana bread', 'ingredients': ['walnuts', 'cinnamon']}, + config={'temperature': 0.2}, +) +if response.output is not None: + recipe: Recipe = response.output -There is **no** `prompt.execute()`. Non-streaming uses the callable -(`ExecutablePrompt.__call__`): +# Variant (`prompts/recipe.quick.prompt`) +quick_res = await ai.prompt('recipe', variant='quick')({'food': 'banana bread'}) +``` + +### Code-Defined Prompts (`ai.define_prompt`) ```python -# Non-streaming — await the prompt itself (double-call: ai.prompt('name') then (...)) -prompt = ai.prompt('recipe') -response = await prompt(input={'food': 'banana bread'}) -# same as: await ai.prompt('recipe')(input={'food': 'banana bread'}) -result = Recipe.model_validate(response.output) - -# Variant (recipe.robot.prompt file) -response = await ai.prompt('recipe', variant='robot')(input={'food': 'banana bread'}) +class GreetingInput(BaseModel): + name: str + + +greet = ai.define_prompt( + name='greet', + input_schema=GreetingInput, + prompt='Say hello to {{name}}.', +) +res = await greet(GreetingInput(name='Ana')) ``` -## Streaming from a prompt +## Streaming from a Prompt -`.stream(...)` is **not** awaited; then consume `.stream` / `.response`: +`prompt.stream(...)` is synchronous (do not `await` it); iterate `.stream` and `await` `.response`: ```python -from genkit import ActionRunContext +from genkit import FlowContext @ai.flow() -async def tell_story(subject: str, ctx: ActionRunContext) -> str: - result = ai.prompt('story').stream(input={'subject': subject}) - full = '' - async for chunk in result.stream: +async def stream_recipe(food: str, ctx: FlowContext[str]) -> str: + sr = ai.prompt('recipe').stream({'food': food}) + chunks: list[str] = [] + async for chunk in sr.stream: if chunk.text: ctx.send_chunk(chunk.text) - full += chunk.text - final = await result.response # completes even if you stop reading chunks early - return final.text or full + chunks.append(chunk.text) + final = await sr.response + return final.text or ''.join(chunks) ``` -## Render without generating (for LLM-judge evals) +## Render Without Generating + +`render(...)` takes the same arguments as a call and returns the `GenerateActionOptions` (model, config, messages, output schema) without calling a model. Use it to debug templates and input defaults: ```python -rendered = await ai.prompt('my_prompt').render(input={'key': 'value'}) -# Inspect roles/messages before spending tokens: -# print(rendered.messages) -response = await ai.generate(model='googleai/gemini-flash-latest', messages=rendered.messages) +rendered = await ai.prompt('recipe').render({'food': 'focaccia'}) +print(rendered.model, rendered.config) +# => googleai/gemini-flash-latest {'temperature': 0.4} +for message in rendered.messages: + print(message.role, message.text.strip()) +# => system You are a chef. Keep recipes practical. +# => user Generate a recipe for focaccia. ``` ## Helpers -Handlebars helpers receive template args in a packed form. Prefer simple -string/list formatting in the template, or unpack carefully: +Register Handlebars helpers with `ai.define_helper(name, fn)`. Dotprompt calls every helper as `fn(params, options)`: `params` is the list of positional template arguments, `options` carries hash arguments. A one-argument function raises `TypeError` at render time. ```python -def list_helper(data: object, *args, **kwargs) -> str: - # Positional template args arrive packed in the first parameter. - items = data[0] if isinstance(data, (list, tuple)) and data else data +from typing import Any + + +def list_helper(params: list[Any], options: Any) -> str: + items = params[0] if params else None if not isinstance(items, list): return '' return '\n'.join(f'- {item}' for item in items) + ai.define_helper('list', list_helper) ``` -Then `{{list ingredients}}` in the `.prompt`. If output looks like a Python -`repr` of a list, the helper unpacked wrong — fix the helper, not the model. - -## Variants - -Name the file `<name>.<variant>.prompt` — e.g. `recipe.robot.prompt`. -Call with `ai.prompt('recipe', variant='robot')`. - -## Partials +Built-in helpers: `role`, `json`, `media`, `history`, `section`, `ifEquals`, `unlessEquals`. -Use `{{>partial_name param=value}}` in templates. Partial files are named -`_partial_name.prompt`. +Use `{{list ingredients}}` in the `.prompt` body. -## Prompts + tools + structured output - -Naming a tool in the template is not enough — pass tool objects at call time -(or list names in frontmatter `tools:` that match registered tools): - -```python -response = await ai.prompt('insurance_quote')( - input={'age': 35, 'zip': '94105', 'coverage_tier': 'plus'}, - tools=[lookup_rate], -) -quote = QuoteResult.model_validate(response.output) -``` +## Variants & Partials -Register output schemas with `ai.define_schema('QuoteResult', QuoteResult)` -when frontmatter references them by name. +- **Variants**: save `<name>.<variant>.prompt` (e.g. `recipe.vegan.prompt`) and load with `ai.prompt('recipe', variant='vegan')`. +- **Partials**: prefix partial files with `_` (e.g. `prompts/_disclaimer.prompt`) or register with `ai.define_partial('disclaimer', '...')`, then include via `{{>disclaimer}}` or `{{>disclaimer region=region}}`. -### Partials example +## Prompts with Tools & Middleware -``` -prompts/_greeting.prompt # partial body only -prompts/support_reply.prompt -``` +- **Tools**: list registered tool names under `tools:` in YAML frontmatter, or pass `tools=[lookup_rate]` when calling `await prompt(input, tools=[lookup_rate])`. The tool loop stops after `maxTurns` (frontmatter) or `max_turns=` (call), default 50. +- **Middleware**: pass instances at call time with `use=[Retry(max_retries=2)]`. To name middleware under `use:` in frontmatter, register it first: add `Middleware()` from `genkit_middleware` to `Genkit(plugins=[...])` (registers `retry`, `fallback`, `tool_approval`, `skills`, `filesystem`) or define your own with `@ai.middleware`. An unregistered name fails with `NOT_FOUND`. -In `support_reply.prompt`: -``` -{{>greeting}} -... main template ... -{{>disclaimer}} +```yaml +--- +model: googleai/gemini-flash-latest +tools: [lookup_rate] +use: + - name: retry + config: + max_retries: 2 +--- +Convert {{amount}} USD to EUR. ``` - -Call `await ai.prompt('support_reply')(input={...})`. Parent input is visible -inside partials unless you override with `{{>greeting name=name}}`. diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/evals.md b/plugins/genkit/agent/skills/developing-genkit-python/references/evals.md index 9c1b7a77..22fb8ba9 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/evals.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/evals.md @@ -1,101 +1,117 @@ -# Evals — Genkit Python +# Evals — Genkit Python (0.13 Only; Removed in 1.0) -## Two types of evaluators +> **Warning:** 0.13 is the last release with evaluators (`ai.define_evaluator`, `ai.define_batch_evaluator`, `ai.evaluate`, `genkit.evaluator`). Genkit 1.0 cuts them and the `genkit-evaluators` package from the 1.0 release; evaluation will return as an additive feature in a later release. `genkit-evaluators` 0.13 requires `genkit<1`. +> - **New projects:** do not add `genkit-evaluators` or `define_evaluator` unless you intend to stay pinned to `genkit==0.13.*`. Prefer `pytest` assertions against your flows and prompts until evaluation returns. +> - **Migrating from 0.12:** stop and ask the developer before porting evaluator code (see [Migrating from 0.12 to 0.13](migrating-from-0.12.md)). -1. **Built-in** — ship with `genkit-evaluators`, register with - `register_genkit_evaluators(ai)`. Regex / deep_equal do not need - `GoogleAI()` or `GEMINI_API_KEY`. -2. **BYO** — `ai.define_evaluator()` with your own scoring logic (including - LLM judges). +The patterns below document the `0.13.*` evaluator API for projects staying on the 0.13 release line. -## Install +- [Install (0.13 Only)](#install-013-only): Lines 19-23 +- [Two Evaluator Types](#two-evaluator-types): Lines 25-28 +- [Dataset Format](#dataset-format): Lines 30-41 +- [Built-In Evaluators & Programmatic Execution](#built-in-evaluators--programmatic-execution): Lines 43-68 +- [BYO Evaluator (EvaluatorResponse, Score, ScoreStatus, ScoreDetails)](#byo-evaluator-evaluatorresponse-score-scorestatus-scoredetails): Lines 70-108 +- [LLM-Based Judge](#llm-based-judge): Lines 110-166 +- [CLI Execution](#cli-execution): Lines 168-180 + +--- + +## Install (0.13 Only) ```bash -uv add genkit-evaluators +uv add 'genkit-evaluators==0.13.*' ``` -## Dataset format +## Two Evaluator Types + +1. **Built-in** — provided by `genkit-evaluators`, registered via `register_genkit_evaluators(ai)` (`genkitEval/regex`, `genkitEval/deep_equal`, `genkitEval/jsonata`). They are deterministic and need no model plugin or `GEMINI_API_KEY`. Pass `metrics=['regex']` to register a subset. +2. **BYO** — custom row-by-row (`ai.define_evaluator`) or batch (`ai.define_batch_evaluator`) functions, including LLM judges. + +## Dataset Format + +`genkit eval:run` reads a JSON array: ```json [ - {"testCaseId": "case1", "input": "x", "output": "banana", "reference": "ba?a?a"}, - {"testCaseId": "case2", "input": "x", "output": "apple", "reference": "ba?a?a"} + {"testCaseId": "dessert", "input": "Suggest a dessert.", "output": "Banana bread", "reference": "(?i)banana"}, + {"testCaseId": "starter", "input": "Suggest a starter.", "output": "Tomato soup", "reference": "(?i)banana"} ] ``` -Fields: `testCaseId`, `input`, `output`, `reference` (optional for some -evaluators). +Fields: `testCaseId`, `input`, `output`, `reference` (optional for reference-free evaluators). In Python, the same rows are `BaseDataPoint(test_case_id=..., input=..., output=..., reference=...)`. -## Built-in evaluators +## Built-In Evaluators & Programmatic Execution ```python +from genkit import Genkit +from genkit.evaluator import BaseDataPoint from genkit_evaluators import register_genkit_evaluators + +# 1. Register the built-in evaluators +ai = Genkit() register_genkit_evaluators(ai) -``` -Registered: `genkitEval/regex`, `genkitEval/deep_equal`, `genkitEval/jsonata`. -`eval:run` needs a Genkit runtime: +# 2. Build the dataset +dataset = [ + BaseDataPoint(test_case_id='dessert', input='Suggest a dessert.', output='Banana bread', reference='(?i)banana'), + BaseDataPoint(test_case_id='starter', input='Suggest a starter.', output='Tomato soup', reference='(?i)banana'), +] -```bash -# terminal A -genkit start -- uv run src/main.py -# terminal B -genkit eval:run datasets/my_dataset.json --evaluators=genkitEval/regex +# 3. Run one evaluator per call; returns list[EvaluatorResponse] in dataset order +results = await ai.evaluate(evaluator='genkitEval/regex', dataset=dataset) +for row in results: + print(row.test_case_id, row.evaluation[0].score, row.evaluation[0].status) +# => dessert True PASS +# starter False FAIL ``` -One-shot (starts the app as the runtime): +`config=` is passed to the evaluator as its second argument (`None` when unset). An unregistered evaluator name raises `GenkitError` with `NOT_FOUND`. -```bash -genkit eval:run datasets/my_dataset.json --evaluators=genkitEval/regex -- uv run src/main.py -``` - -`No runtimes found` means nothing is registered — start the app or use `--`. +## BYO Evaluator (`EvaluatorResponse`, `Score`, `ScoreStatus`, `ScoreDetails`) -Programmatic — one evaluator per call (`evaluator=`, singular): +`ai.define_evaluator` takes `(name, fn)` positionally with required keyword-only `display_name=` and `definition=`. `EvaluatorResponse` requires `test_case_id`, and `evaluation` takes a `list[Score]`: ```python -result = await ai.evaluate( - dataset=my_dataset, - evaluator='byo/my_eval', # not evaluators=[...] +from genkit.evaluator import ( + BaseDataPoint, + EvaluatorResponse, + Score, + ScoreDetails, + ScoreStatus, ) -# EvalResponse — use result.root (not .results) -``` -Add `--output=eval_out.json` on CLI for machine-readable pass/fail. -## BYO evaluator - -```python -from genkit.evaluator import BaseDataPoint, Details, EvalFnResponse, EvalStatusEnum, Score - -async def my_eval(datapoint: BaseDataPoint, _options: dict | None = None) -> EvalFnResponse: - output = str(datapoint.output or '') - reference = str(datapoint.reference or '') - passed = output.strip() == reference.strip() - return EvalFnResponse( +async def exact_match(datapoint: BaseDataPoint, _options: object | None = None) -> EvaluatorResponse: + output = str(datapoint.output or '').strip() + reference = str(datapoint.reference or '').strip() + passed = output == reference + return EvaluatorResponse( test_case_id=datapoint.test_case_id or '', - evaluation=Score( - score=1.0 if passed else 0.0, - status=EvalStatusEnum.PASS if passed else EvalStatusEnum.FAIL, - details=Details(reasoning='Exact match check'), - ), + evaluation=[ + Score( + score=1.0 if passed else 0.0, + status=ScoreStatus.PASS if passed else ScoreStatus.FAIL, + details=ScoreDetails(reasoning='Exact match check'), + ) + ], ) + ai.define_evaluator( - name='byo/my_eval', - display_name='My Eval', - definition='Checks exact match of output vs reference.', - fn=my_eval, + 'byo/exact_match', + exact_match, + display_name='Exact Match', + definition='Checks exact match of output against reference.', ) ``` -## LLM-based judge +For batch evaluation across the whole dataset at once, register `async def my_batch_eval(req: EvaluatorRequest) -> list[EvaluatorResponse]:` with `ai.define_batch_evaluator('byo/batch', my_batch_eval, display_name='...', definition='...')` (`from genkit.evaluator import EvaluatorRequest`). The batch function takes exactly one argument; read options from `req.options`. -Keep a judge prompt next to the app and parse a score from the model. +## LLM-Based Judge -`prompts/judge.prompt`: +`prompts/judge.prompt` (load it with `Genkit(prompt_dir='prompts', ...)`; `prompts/` is the default): -``` +```yaml --- model: googleai/gemini-flash-latest input: @@ -111,42 +127,54 @@ Candidate: {{output}} ``` ```python -import re +from genkit import Genkit +from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score, ScoreStatus +from genkit_google_genai import GoogleAI -async def llm_eval(datapoint: BaseDataPoint, _options: dict | None = None) -> EvalFnResponse: - rendered = await ai.prompt('judge').render( - input={'output': str(datapoint.output), 'reference': str(datapoint.reference)} - ) - response = await ai.generate( - model='googleai/gemini-flash-latest', - messages=rendered.messages, +ai = Genkit(plugins=[GoogleAI()]) + + +async def llm_judge(datapoint: BaseDataPoint, _options: object | None = None) -> EvaluatorResponse: + # 1. Run the judge prompt; the model comes from the .prompt frontmatter + response = await ai.prompt('judge')( + {'output': str(datapoint.output), 'reference': str(datapoint.reference)} ) - m = re.search(r'0?\.\d+|1\.0|[01]', (response.text or '').strip()) - score = float(m.group(0)) if m else 0.0 - return EvalFnResponse( + + # 2. Parse the score; treat anything unparseable as 0.0 + try: + score = min(max(float((response.text or '').strip()), 0.0), 1.0) + except ValueError: + score = 0.0 + + return EvaluatorResponse( test_case_id=datapoint.test_case_id or '', - evaluation=Score( - score=score, - status=EvalStatusEnum.PASS if score >= 0.5 else EvalStatusEnum.FAIL, - ), + evaluation=[Score(score=score, status=ScoreStatus.PASS if score >= 0.5 else ScoreStatus.FAIL)], ) + ai.define_evaluator( - name='byo/llm_judge', + 'byo/llm_judge', + llm_judge, display_name='LLM Judge', definition='Scores candidate vs reference with a judge model.', - fn=llm_eval, + is_billed=True, ) ``` -Built-in `genkitEval/regex` scores are `True`/`False`; BYO scores are usually -`0.0`/`1.0` — normalize before aggregating. +`is_billed=True` makes `genkit eval:run` ask for confirmation before calling the model; pass `--force` in CI. -## CLI +Built-in `genkitEval/regex` and `genkitEval/deep_equal` scores are `bool` (`True` / `False`); BYO numeric scores are typically `0.0`–`1.0`. Normalize before aggregating. + +## CLI Execution + +`genkit eval:run` requires an active Genkit runtime (either already running via `genkit start` or launched inline after `--`): ```bash -genkit eval:run datasets/my_dataset.json --evaluators=byo/my_eval -genkit eval:run datasets/my_dataset.json --evaluators=genkitEval/regex,byo/my_eval +# One-shot: launches the app runtime, evaluates, and exits +genkit eval:run datasets/menu_dataset.json --evaluators=genkitEval/regex,byo/exact_match -- uv run src/main.py + +# Write machine-readable results to a file (--output-format json|csv, default json) +genkit eval:run datasets/menu_dataset.json --evaluators=byo/exact_match --output=eval_out.json -- uv run src/main.py ``` -Results appear in the Dev UI under **Evaluate**. +Omitting `--evaluators` runs every registered evaluator. `No runtimes found` means no Genkit runtime is active — pass `-- uv run src/main.py` or run `genkit start` in another terminal. diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/examples.md b/plugins/genkit/agent/skills/developing-genkit-python/references/examples.md index 13a000fa..13c071fc 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/examples.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/examples.md @@ -1,29 +1,71 @@ # Genkit Python Examples -Minimal patterns for common Genkit APIs. Examples use **Google AI** (`GoogleAI`, `googleai/...`); other providers use the same patterns with the right plugin and model prefix. +Minimal patterns for generation, structured output, streaming, media, flows, tools, interrupts, cancellation, and embeddings. -## Public imports +- [Public Imports & Initialization](#public-imports--initialization): Lines 16-58 +- [Structured Output](#structured-output): Lines 62-95 +- [Streaming (Text & Structured Output)](#streaming-text--structured-output): Lines 99-126 +- [Media Input & Output (Part)](#media-input--output-part): Lines 130-192 +- [Flows & Streaming Flows](#flows--streaming-flows): Lines 196-225 +- [Tools, Multipart Tool Responses, and Interrupts](#tools-multipart-tool-responses-and-interrupts): Lines 229-286 +- [Cancellation](#cancellation): Lines 290-317 +- [Embeddings](#embeddings): Lines 321-347 -Use public packages only — `genkit`, `genkit_google_genai`, `genkit_fastapi`, -`genkit_middleware`, `genkit_evaluators`, `genkit.agent`, `genkit.embedder`, -`genkit.evaluator`, `genkit.model`, etc. Do not import internal modules -(`genkit._core`, …). +--- + +## Public Imports & Initialization + +Import only from public modules (`genkit`, `genkit_google_genai`, `genkit_fastapi`, `genkit_middleware`, `genkit.model`, `genkit.embedder`, `genkit.middleware`, `genkit.web`, `genkit.exp`, `genkit.exp.agent`). Never import private `_` submodules. ```python -from genkit import Genkit, ActionRunContext +from genkit import Genkit from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +ai = Genkit( + plugins=[GoogleAI()], + model=GoogleAI.gemini('gemini-flash-latest'), +) +``` + +For multi-turn stateful agents, import `Genkit` from `genkit.exp` and session types from `genkit.exp.agent` (see [Agents](agents.md)). + +### Provider Model Refs & Typed Config + +Other providers follow the same `Plugin.helper('model-id')` + `<Plugin>Config` shape (import list in [SKILL.md](../SKILL.md#imports)). + +```python +from genkit_google_genai import GeminiConfig, GoogleAI + +# Typed config with flat fields + `extra` escape hatch for unmodeled provider fields +res = await ai.generate( + model=GoogleAI.gemini('gemini-flash-latest'), + prompt='Check recent FDA allergen labeling updates.', + config=GeminiConfig( + thinking_level='HIGH', + include_thoughts=True, + google_search=True, + extra={'seed': 42}, + ), +) + +# Per-request API key (never pass `api_key` inside `config` — that raises GenkitError) +tenant_res = await ai.generate( + prompt='Summarize invoice #4821.', + context={'secrets': {'api_key': tenant_api_key}}, +) ``` -For agents, see [Agents](agents.md) (`from genkit.agent import ...`). +Config defaults merge key-by-key (`ModelRef` < `Prompt` < call); pass `None` on a field to clear an inherited key. Unknown config keys raise `GenkitError` (`INVALID_ARGUMENT`) before calling the provider. --- -## Structured output +## Structured Output + +`response.output` is typed `OutputT | None`. If the model reply is cut off (`max_output_tokens`) or malformed, `response.output` is `None` and `response.error` (`Failure`) has `reason == ErrorReason.INVALID_OUTPUT`. ```python from pydantic import BaseModel, TypeAdapter +from genkit import ErrorReason class CityInfo(BaseModel): name: str @@ -32,181 +74,274 @@ class CityInfo(BaseModel): response = await ai.generate( prompt='Give facts about Tokyo.', - output_format='json', output_schema=CityInfo, ) -city = response.output +if response.output is None: + if response.error and response.error.reason == ErrorReason.INVALID_OUTPUT: + raise RuntimeError(f'Incomplete structured output: {response.error.message}') + raise RuntimeError('No structured output returned') + +city: CityInfo = response.output # Arrays schema = TypeAdapter(list[CityInfo]).json_schema() -response = await ai.generate( - prompt='List 3 cities.', +list_res = await ai.generate( + prompt='List 3 cities in Japan.', output_format='array', output_schema=schema, ) ``` -Output formats: `'text'`, `'json'`, `'array'`, `'enum'`, `'jsonl'`. +Built-in `output_format` values: `'text'`, `'json'`, `'array'`, `'enum'`, `'jsonl'`. --- -## Streaming (text) +## Streaming (Text & Structured Output) ```python -sr = ai.generate_stream(prompt='Tell me a story.') +sr = ai.generate_stream(prompt='Explain sourdough fermentation.') async for chunk in sr.stream: if chunk.text: print(chunk.text, end='', flush=True) -final = await sr.response # final.text +final = await sr.response ``` -You do **not** need to drain `.stream` for the turn to finish — `await sr.response` -completes even if you break out of the loop early (same idea as agent -`send_stream`). - ---- - -## Text and media parts +Streaming structured output yields partial typed objects on `chunk.output` without raising on incomplete JSON: ```python -# Non-streaming -response = await ai.generate(prompt='...') -for media in response.media: - print(media.content_type, (media.url or '')[:80]) - -# Streaming — media usually complete on the final response -from genkit import MediaPart +class RecipeCard(BaseModel): + title: str + prep_minutes: int + steps: list[str] -sr = ai.generate_stream(prompt='...') +sr = ai.generate_stream( + prompt='Create a weeknight shakshuka recipe.', + output_schema=RecipeCard, +) async for chunk in sr.stream: - if chunk.text: - print(chunk.text, end='', flush=True) + if chunk.output: + print(chunk.output) final = await sr.response -for media in final.media: - print(media.content_type, (media.url or '')[:80]) - -if final.message: - for part in final.message.content: - if isinstance(part.root, MediaPart) and part.root.media: - print(part.root.media.content_type) +recipe = final.output # RecipeCard | None ``` --- +## Media Input & Output (`Part`) -## Sending image / media input +Construct parts with `Part.from_text(...)` and `Part.from_media(...)`. Inspect parts via `part.text` and `part.media`. -The section above covers **model-produced** media. To *send* an image to Gemini, -build a user `Message` with text + `MediaPart` (URL or data URI): +### Sending Image / Media Input ```python -from genkit import Media, MediaPart, Message, Part, Role, TextPart +import base64 +from pathlib import Path +from genkit import Message, Part, Role + +# Direct HTTPS URL (< 20 MB) or base64 data URI +b64 = base64.b64encode(Path('receipt.png').read_bytes()).decode() +data_uri = f'data:image/png;base64,{b64}' -image_url = 'https://example.com/cat.jpg' # or data:image/png;base64,... msg = Message( role=Role.USER, content=[ - Part(root=TextPart(text='What is in this image?')), - Part(root=MediaPart(media=Media(url=image_url, content_type='image/jpeg'))), + Part.from_text('Extract the total amount from this receipt.'), + Part.from_media(data_uri, content_type='image/png'), ], ) response = await ai.generate(messages=[msg]) print(response.text) ``` -Prefer a vision-capable model (`googleai/gemini-flash-latest`). URLs are -fetched **server-side** — hotlink-blocked or thumbnail URLs often fail. -Prefer a direct image URL or a data URI: - -```python -import base64 -b64 = base64.b64encode(Path('cat.png').read_bytes()).decode() -url = f'data:image/png;base64,{b64}' -``` - +Media URLs larger than **20 MB** fail instead of downloading into memory, and `Ollama` does not read local filesystem paths for media. -## Streaming + structured output +### Generating Image / Audio / Video Output ```python -class StoryAnalysis(BaseModel): - title: str - genre: str - summary: str +from genkit_google_genai import GeminiImageConfig, GeminiTtsConfig, GoogleAI, VertexAI -sr = ai.generate_stream( - prompt='Write a short story then analyze it.', - output_format='json', - output_schema=StoryAnalysis, +# Image generation (Imagen catalog is retired; use gemini_image) +img_res = await ai.generate( + model=GoogleAI.gemini_image('gemini-2.5-flash-image'), + prompt='Minimalist watercolor illustration of an espresso machine.', + config=GeminiImageConfig(aspect_ratio='1:1'), ) -async for chunk in sr.stream: - if chunk.text: - print(chunk.text, end='', flush=True) -final = await sr.response -analysis = final.output +for media in img_res.media: + print(media.content_type, (media.url or '')[:60]) + +# Inspecting parts directly on a message +if img_res.message: + for part in img_res.message.content: + if part.media: + print(part.media.content_type) + +# Text-to-speech +tts_res = await ai.generate( + model=GoogleAI.gemini_tts('gemini-2.5-flash-preview-tts'), + prompt='Order 42 is ready at the counter.', + config=GeminiTtsConfig(voice='Kore'), +) + +# Background video generation (Veo via Vertex AI) +op = await ai.generate_operation( + model=VertexAI.veo('veo-3.1-generate-preview'), + prompt='Aerial shot of coastal cliffs at sunrise.', +) +while not op.done: + op = await ai.check_operation(op) ``` --- -## Flows +## Flows & Streaming Flows + +Flows validate both input and return values against their type annotations. Calling `await summarize(inp)` or `await summarize.run(inp)` raises body exceptions unwrapped in-process; over HTTP, raise `PublicError` for client-visible 4xx errors (see [FastAPI](fastapi.md)). ```python +from genkit import FlowContext, PublicError + class SummarizeInput(BaseModel): text: str @ai.flow() async def summarize(input: SummarizeInput) -> str: - response = await ai.generate(prompt=f'Summarize: {input.text}') + if not input.text.strip(): + raise PublicError('INVALID_ARGUMENT', 'text must not be empty') + response = await ai.generate(prompt=f'Summarize in two sentences: {input.text}') return response.text -``` - ---- -## Streaming flows - -```python +# Streaming flow: declare the chunk type on FlowContext[ChunkT] @ai.flow() -async def stream_story(subject: str, ctx: ActionRunContext) -> str: - sr = ai.generate_stream(prompt=f'Story about {subject}.') - full = '' +async def stream_story(subject: str, ctx: FlowContext[str]) -> str: + sr = ai.generate_stream(prompt=f'Write a short story about {subject}.') + chunks: list[str] = [] async for chunk in sr.stream: if chunk.text: ctx.send_chunk(chunk.text) - full += chunk.text - return full + chunks.append(chunk.text) + return ''.join(chunks) ``` +Options after the flow's single input on `.run(...)` are keyword-only: `await stream_story.run('lighthouses', on_chunk=print, context={'role': 'reader'})`. + --- -## Tools +## Tools, Multipart Tool Responses, and Interrupts -Parameters must be a **Pydantic `BaseModel`** (bare scalars → 400 from Gemini). Use **`@ai.tool()`**, not `@ai.define_tool()`. +Use `@ai.tool()` (there is no `define_tool`). A tool takes **at most one input parameter** (scalar or Pydantic `BaseModel`) plus an optional `ctx: ToolContext[ChunkT]`. Use a `BaseModel` whenever a tool needs multiple arguments or field descriptions. ```python -class WeatherInput(BaseModel): - city: str +from pydantic import BaseModel, Field +from genkit import MultipartToolResponse, Part, ToolContext + +class OrderLookup(BaseModel): + order_id: str = Field(description='Order identifier, e.g. ORD-104') + include_receipt_image: bool = False + +class ToolProgress(BaseModel): + stage: str @ai.tool() -async def get_weather(input: WeatherInput) -> str: - return f'Sunny in {input.city}' +async def lookup_order(input: OrderLookup, ctx: ToolContext[ToolProgress]) -> str | MultipartToolResponse: + ctx.send_chunk(ToolProgress(stage='fetching_record')) + if not input.include_receipt_image: + return f'Order {input.order_id}: delivered' + receipt_png = Part.from_media('data:image/png;base64,iVBORw0KGgo=', content_type='image/png') + return MultipartToolResponse( + output={'order_id': input.order_id, 'status': 'delivered'}, + content=[receipt_png], + ) + +res = await ai.generate( + prompt='Check status of ORD-104.', + tools=[lookup_order], + tool_choice='required', # 'auto' | 'required' | 'none' +) +``` + +### Tool Interrupts (Human-in-the-Loop) + +Raise `Interrupt(...)` inside a tool (or register a tool via `ai.define_interrupt(...)`) to pause the generate loop. `Interrupt` subclasses `BaseException`, so `except Exception:` inside a tool will not swallow it. Resume using `intr.restart(...)` or `intr.respond(...)`: + +```python +from genkit import Interrupt + +class RefundInput(BaseModel): + order_id: str + amount_cents: int -response = await ai.generate(prompt='Weather in Paris?', tools=[get_weather]) +@ai.tool() +async def issue_refund(input: RefundInput, ctx: ToolContext) -> dict: + if not ctx.is_resumed(): + raise Interrupt({'order_id': input.order_id, 'amount_cents': input.amount_cents}) + return {'refunded': True, 'approved_by': (ctx.resumed_metadata or {}).get('manager')} + +turn1 = await ai.generate(prompt='Refund $25 (2500 cents) on ORD-104.', tools=[issue_refund]) +if turn1.interrupts: + turn2 = await ai.generate( + messages=turn1.messages, + tools=[issue_refund], + resume_restart=[i.restart(resumed_metadata={'manager': 'alex'}) for i in turn1.interrupts], + ) +``` + +--- + +## Cancellation + +Cancellation uses standard `asyncio.CancelledError`; it reaches model calls, retry backoff, and running tools at any `await`: + +```python +import asyncio + +# 1. Cancel an in-flight generate or flow task +task = asyncio.create_task(ai.generate(prompt='Write a detailed migration plan.')) +task.cancel() + +# 2. Stop a flow stream early — use `async with` (or `await stream.aclose()`) +# so breaking out of the loop cancels the background flow immediately. +async with stream_story.stream('glaciers') as stream: + async for chunk in stream: + if len(chunk) > 0: + break + +# 3. Clean up partial side effects inside `finally` in tools +@ai.tool() +async def hold_inventory(sku: str) -> dict: + reservation = await reserve_sku(sku) + try: + return await commit_reservation(reservation) + finally: + if not reservation.committed: + await release_reservation(reservation) ``` --- ## Embeddings +Build embedder refs with `.embedder(...)` and pass provider options in `config=`: + ```python -from genkit_google_genai import GeminiEmbeddingModels +from genkit import Document +from genkit_google_genai import GoogleAI -embedder = f'googleai/{GeminiEmbeddingModels.GEMINI_EMBEDDING_001}' -embeddings = await ai.embed(embedder=embedder, content='The sky is blue.') -vector = embeddings[0].embedding +embedder = GoogleAI.embedder('gemini-embedding-001') +embeddings = await ai.embed( + embedder=embedder, + content='Sourdough starter hydration ratio.', + config={'task_type': 'RETRIEVAL_DOCUMENT'}, +) +vector: list[float] = embeddings[0].embedding -embeddings = await ai.embed_many( +batch = await ai.embed_many( embedder=embedder, - content=['The sky is blue.', 'Grass is green.'], + content=['First paragraph.', 'Second paragraph.'], ) -``` -Common embedders: `googleai/gemini-embedding-001`, `googleai/gemini-embedding-exp-03-07`. +# Multimodal embedding input via Document +image_embeddings = await ai.embed( + embedder=GoogleAI.embedder('gemini-embedding-2'), + content=Document.from_media('data:image/png;base64,iVBORw0KGgo=', 'image/png'), +) +``` diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/fastapi.md b/plugins/genkit/agent/skills/developing-genkit-python/references/fastapi.md index a2aafd3e..c89f9ad6 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/fastapi.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/fastapi.md @@ -1,344 +1,344 @@ # FastAPI — Genkit Python +- [Install](#install): Lines 14-29 +- [Serve a Flow or Agent](#serve-a-flow-or-agent): Lines 33-87 +- [HTTP Error Redaction & PublicError](#http-error-redaction--publicerror): Lines 91-97 +- [Streaming (SSE)](#streaming-sse): Lines 101-150 +- [Auth & Context Providers](#auth--context-providers): Lines 154-226 +- [Advanced Flow Composition](#advanced-flow-composition): Lines 230-302 +- [Structured Output Endpoint](#structured-output-endpoint): Lines 306-329 +- [Python HTTP Client](#python-http-client): Lines 333-344 + +--- + ## Install ```bash -uv add genkit-fastapi fastapi uvicorn +uv add 'genkit-fastapi==0.13.*' fastapi uvicorn ``` -Import from `genkit_fastapi`: - -- `serve_agent` / `serve_flow` — mount an agent or flow as a router -- `handle_genkit_request` — escape hatch for a custom `@app.post` that needs its own - `Depends` / auth wiring while still speaking the Genkit wire format +Imports are split between stable flow serving and experimental agent serving: -For agents, see also [Agent HTTP](agents-http.md). +- **Stable flows (`genkit_fastapi`)**: + - `serve_flow` — mount a flow as a FastAPI `APIRouter` (supports `base_path=` and `context_dependency=`) + - `genkit_fastapi_handler` — decorator stacking `@app.post('/...')` directly over `@ai.flow()` (supports `context_provider=`) + - `handle_genkit_request` — low-level handler for custom `@app.post` routes with multiple FastAPI `Depends(...)` parameters +- **Experimental agents (`genkit_fastapi.exp`)**: + - `serve_agent` — mount an experimental `Agent` (`genkit.exp.Genkit`) as an `APIRouter` (see [Agent HTTP](agents-http.md)) +- **Framework-agnostic web utilities (`genkit.web`)**: + - `RequestData`, `ContextProvider`, `StreamHandle`, `to_sse`, `read_body`, `wants_stream`, `result_json`, `error_body`, `error_json`, `error_sse`, `error_status` --- -## Serve an agent or flow +## Serve a Flow or Agent ```python +import os import uvicorn from fastapi import FastAPI -from genkit import Genkit -from genkit.agent import InMemorySessionStore -from genkit_fastapi import serve_agent, serve_flow +from genkit import PublicError +from genkit.exp import Genkit +from genkit.exp.agent import InMemorySessionStore +from genkit_fastapi import serve_flow +from genkit_fastapi.exp import serve_agent from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) -agent = ai.define_agent( - name='weatherAgent', - system='Weather assistant. Be concise.', +concierge = ai.define_agent( + name='conciergeAgent', + system='Hotel concierge. Keep answers under 40 words.', store=InMemorySessionStore(), ) + @ai.flow() -async def hello(name: str) -> str: - return f'Hello, {name}' +async def lookup_reservation(code: str) -> dict: + if not code.startswith('RES-'): + raise PublicError('NOT_FOUND', f'Reservation {code} not found') # HTTP 404 + return {'code': code, 'status': 'confirmed'} + app = FastAPI() -# Routes use the action name as defined: agent `weatherAgent` → /api/weatherAgent; -# flow `hello` → /api/hello. Override with base_path=... if needed. -app.include_router(serve_agent(agent), prefix='/api') -app.include_router(serve_flow(hello), prefix='/api') +# Paths default to the agent name and the flow's Python function name (override via base_path=...): +# POST /api/conciergeAgent (+ /getSnapshot and /abort, since the agent has a store) +# POST /api/lookup_reservation +app.include_router(serve_agent(concierge), prefix='/api') +app.include_router(serve_flow(lookup_reservation), prefix='/api') + -# Under genkit start, drive uvicorn from ai.run_main (don't nest uvicorn.run inside it). async def main() -> None: - import os config = uvicorn.Config(app, host='0.0.0.0', port=int(os.environ.get('PORT', '8080'))) await uvicorn.Server(config).serve() + if __name__ == '__main__': ai.run_main(main()) ``` +If your app only serves flows (no agents), import `Genkit` from `genkit` instead of `genkit.exp`. + +With `GENKIT_ENV=dev` (which `genkit start` sets automatically), the Dev UI reflection server starts on a background thread, so either entrypoint works: + +```bash +genkit start -- uv run uvicorn main:app --reload # reload-friendly ASGI dev server + Dev UI +genkit start -- uv run src/main.py # driven via ai.run_main(main()) + Dev UI +uvicorn main:app # production (no Dev UI) +``` + +--- + +## HTTP Error Redaction & `PublicError` + +Over HTTP (`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, `handle_genkit_request`), Genkit separates internal errors from caller-visible errors: + +- **`PublicError(status, message, details)`**: returns the corresponding HTTP status (`NOT_FOUND` -> 404, `INVALID_ARGUMENT` -> 400, `UNAUTHENTICATED` -> 401, `PERMISSION_DENIED` -> 403) with `status`, `message`, and `details`. Keep secrets and cross-tenant data out of `PublicError`. +- **Invalid flow input**: returns HTTP 400 with field names and Pydantic's built-in messages (never echoing submitted values; custom validator text shows as `Invalid value` unless the validator raises `PublicError`). +- **Any other `GenkitError`, provider error, or Python exception**: logged server-side with full traceback (`5xx`) and returned to the client as `500 {"status": "INTERNAL", "message": "Internal Error"}`. + --- -## Streaming +## Streaming (SSE) -`serve_agent`, `serve_flow`, and `handle_genkit_request` all stream when the client -sends `Accept: text/event-stream` (same wire path). Otherwise they return a one-shot -`{"result": ...}` JSON body. +`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, and `handle_genkit_request` stream Server-Sent Events when the client sends `Accept: text/event-stream` (or `?stream=true`), and automatically cancel the running flow if the client disconnects. Otherwise they return `{"result": ...}`. **Wire format (SSE):** +```text +data: {"message": "<chunk>"} +data: {"message": "<chunk>"} +data: {"result": "<final output>"} ``` -data: {"message": "<chunk text>"} ← one per ctx.send_chunk() call -data: {"message": "<chunk text>"} -data: {"result": <final output>} ← sent once when the action completes -``` +Errors during streaming emit `data: {"error": {...}}` formatted with the same `PublicError` redaction rules. + +**Minimal streaming FastAPI flow (`FlowContext`):** + +```python +from pydantic import BaseModel +from fastapi import FastAPI +from genkit import FlowContext, Genkit +from genkit_fastapi import serve_flow +from genkit_google_genai import GoogleAI + +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) + + +class BriefInput(BaseModel): + topic: str + -**Frontend:** -```js -const res = await fetch('/api/chat', { - method: 'POST', - headers: { 'Content-Type': 'application/json', 'Accept': 'text/event-stream' }, - body: JSON.stringify({ data: { topic: 'quantum computing' } }), -}); -const reader = res.body.getReader(); -// decode and parse each `data: {...}` line +@ai.flow() +async def stream_brief(input: BriefInput, ctx: FlowContext[str]) -> str: + sr = ai.generate_stream(prompt=f'Write a concise brief on {input.topic}.') + parts: list[str] = [] + async for chunk in sr.stream: + if chunk.text: + ctx.send_chunk(chunk.text) # each call emits one SSE frame + parts.append(chunk.text) + return ''.join(parts) + + +app = FastAPI() +app.include_router(serve_flow(stream_brief), prefix='/api') # POST /api/stream_brief ``` -**curl test:** +**Test with `curl`:** ```bash -curl -N -X POST http://localhost:8080/api/chat \ +curl -N -X POST http://localhost:8080/api/stream_brief \ -H 'Content-Type: application/json' \ -H 'Accept: text/event-stream' \ - -d '{"data": {"topic": "quantum computing"}}' + -d '{"data": {"topic": "zero-downtime Postgres migrations"}}' ``` --- -## Minimal streaming FastAPI app +## Auth & Context Providers + +### 1. FastAPI `Depends` via `serve_flow(..., context_dependency=...)` ```python -import uvicorn -from pydantic import BaseModel -from fastapi import FastAPI -from genkit import Genkit, ActionRunContext +from fastapi import Header +from genkit import FlowContext, PublicError from genkit_fastapi import serve_flow -from genkit_google_genai import GoogleAI -ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest') +async def require_auth(authorization: str | None = Header(default=None)) -> dict: + if not authorization: + raise PublicError('UNAUTHENTICATED', 'Missing Authorization header') # 401 + return {'auth': {'token': authorization}} -class ChatInput(BaseModel): - topic: str + +@ai.flow() +async def account_summary(account_id: str, ctx: FlowContext) -> dict: + caller = (ctx.context or {}).get('auth') + return {'account_id': account_id, 'caller': caller} +app.include_router( + serve_flow(account_summary, context_dependency=require_auth), + prefix='/api', +) +``` + +### 2. Decorator Style (`@genkit_fastapi_handler`) with `genkit.web.RequestData` + +```python +from genkit import PublicError +from genkit.web import RequestData +from genkit_fastapi import genkit_fastapi_handler + + +async def auth_provider(req: RequestData) -> dict: + token = req.headers.get('authorization') + if not token: + raise PublicError('UNAUTHENTICATED', 'Missing bearer token') # 401 + return {'user_token': token} + + +@app.post('/api/ask') +@genkit_fastapi_handler(ai, context_provider=auth_provider) @ai.flow() -async def chat(input: ChatInput, ctx: ActionRunContext) -> str: - sr = ai.generate_stream(prompt=f'Tell me about {input.topic}.') - full = '' - async for chunk in sr.stream: - if chunk.text: - ctx.send_chunk(chunk.text) # each chunk → SSE event on the wire - full += chunk.text - return full +async def ask(question: str) -> str: + res = await ai.generate(prompt=question) + return res.text +``` +### 3. Custom Route Escape Hatch (`handle_genkit_request` or `genkit.web`) -app = FastAPI() -app.include_router(serve_flow(chat), prefix='/api') # POST /api/chat +When a route needs multiple `Depends(...)` parameters while keeping the Genkit wire format: -async def main() -> None: - import os - config = uvicorn.Config(app, host='0.0.0.0', port=int(os.environ.get('PORT', '8080'))) - await uvicorn.Server(config).serve() +```python +from fastapi import Depends, Request +from genkit_fastapi import handle_genkit_request -if __name__ == '__main__': - ai.run_main(main()) + +@app.post('/api/custom-brief') +async def custom_brief( + request: Request, + auth_ctx: dict = Depends(require_auth), +): + return await handle_genkit_request( + request, + action=stream_brief, + context=auth_ctx, + ) ``` -**Key:** the flow must accept `ctx: ActionRunContext` and call `ctx.send_chunk(text)` -to emit SSE chunks. Without `ctx.send_chunk`, the flow runs but streams nothing — the -client waits for the final result. +If writing a completely custom non-Genkit JSON wrapper around `await my_flow.run(data)`, map exceptions with `error_body(e)` and `error_status(e)` from `genkit.web` so internal errors stay redacted. --- -## Advanced use cases +## Advanced Flow Composition -### Nested flow streaming +### Nested Flow Streaming -Chain flows so a child's chunks surface on the parent's HTTP stream. Call the -child with `.run(..., on_chunk=ctx.send_chunk)` — do **not** pass `ctx` as a -second positional argument to `await child(input, ctx)` (that raises `TypeError`). +Forward a child flow's chunks to the parent's SSE stream via `await child.run(input, on_chunk=ctx.send_chunk)`. All arguments after the input on `.run(...)` are keyword-only: ```python -class ResearchInput(BaseModel): +class SectionInput(BaseModel): topic: str + @ai.flow() -async def research(input: ResearchInput, ctx: ActionRunContext) -> str: - sr = ai.generate_stream(prompt=f'Explain {input.topic} in depth.') - full = '' +async def draft_body(input: SectionInput, ctx: FlowContext[str]) -> str: + sr = ai.generate_stream(prompt=f'Explain {input.topic} with concrete examples.') + chunks: list[str] = [] async for chunk in sr.stream: if chunk.text: ctx.send_chunk(chunk.text) - full += chunk.text - return full - + chunks.append(chunk.text) + return ''.join(chunks) -class HeadlineInput(BaseModel): - text: str @ai.flow() -async def make_headline(input: HeadlineInput) -> str: - response = await ai.generate(prompt=f'One-line headline for: {input.text}') - return response.text.strip() +async def draft_headline(input: SectionInput) -> str: + res = await ai.generate(prompt=f'One-line title for: {input.topic}') + return res.text.strip() -class ReportInput(BaseModel): - topic: str - @ai.flow() -async def report(input: ReportInput, ctx: ActionRunContext) -> str: - headline = await make_headline(HeadlineInput(text=input.topic)) +async def full_report(input: SectionInput, ctx: FlowContext[str]) -> str: + headline = await draft_headline(input) ctx.send_chunk(f'# {headline}\n\n') - - body = ( - await research.run( - ResearchInput(topic=input.topic), - on_chunk=ctx.send_chunk, - ) - ).response - + body = (await draft_body.run(input, on_chunk=ctx.send_chunk)).response return f'# {headline}\n\n{body}' - - -app.include_router(serve_flow(report), prefix='/api') # POST /api/report ``` -**Rules:** -- Streaming children accept `ctx: ActionRunContext` and call `ctx.send_chunk` -- Parents forward chunks with `child.run(input, on_chunk=ctx.send_chunk)` -- Non-streaming children: `await child(input)` is fine - -### Executing flows in parallel - -Use `asyncio.gather` to run multiple flows concurrently. Only makes sense when children don't need to stream. +### Parallel Flows (`asyncio.gather`) ```python import asyncio -class AnalysisInput(BaseModel): - text: str -class CheckResult(BaseModel): - issues: list[str] +class CodeReviewInput(BaseModel): + diff: str -class CombinedAnalysis(BaseModel): - issues: list[str] -@ai.flow() -async def check_security(input: AnalysisInput) -> CheckResult: - r = await ai.generate( - prompt=f'List security concerns as a short comma-separated line (or "none"): {input.text[:2000]}', - ) - raw = (r.text or '').strip() - issues = [s.strip() for s in raw.split(',') if s.strip() and s.strip().lower() != 'none'] - return CheckResult(issues=issues) +class ReviewFindings(BaseModel): + findings: list[str] -@ai.flow() -async def check_bugs(input: AnalysisInput) -> CheckResult: - r = await ai.generate( - prompt=f'List likely bugs or correctness issues as a short comma-separated line (or "none"): {input.text[:2000]}', - ) - raw = (r.text or '').strip() - issues = [s.strip() for s in raw.split(',') if s.strip() and s.strip().lower() != 'none'] - return CheckResult(issues=issues) @ai.flow() -async def check_style(input: AnalysisInput) -> CheckResult: - r = await ai.generate( - prompt=f'List style or clarity issues as a short comma-separated line (or "none"): {input.text[:2000]}', +async def check_security(input: CodeReviewInput) -> ReviewFindings: + res = await ai.generate( + prompt=f'List security issues in this diff: {input.diff[:2000]}', + output_schema=ReviewFindings, ) - raw = (r.text or '').strip() - issues = [s.strip() for s in raw.split(',') if s.strip() and s.strip().lower() != 'none'] - return CheckResult(issues=issues) + return res.output or ReviewFindings(findings=[]) + @ai.flow() -async def analyze(input: AnalysisInput) -> CombinedAnalysis: - security, bugs, style = await asyncio.gather( - check_security(input), - check_bugs(input), - check_style(input), +async def check_correctness(input: CodeReviewInput) -> ReviewFindings: + res = await ai.generate( + prompt=f'List logic bugs in this diff: {input.diff[:2000]}', + output_schema=ReviewFindings, ) - return CombinedAnalysis(issues=security.issues + bugs.issues + style.issues) + return res.output or ReviewFindings(findings=[]) -app.include_router(serve_flow(analyze), prefix='/api') # POST /api/analyze +@ai.flow() +async def review_diff(input: CodeReviewInput) -> ReviewFindings: + sec, bugs = await asyncio.gather(check_security(input), check_correctness(input)) + return ReviewFindings(findings=sec.findings + bugs.findings) ``` --- -## Structured output endpoint (non-streaming) +## Structured Output Endpoint + +Check `response.output is None` before returning so truncated model replies surface cleanly instead of failing output validation: ```python +from genkit import PublicError + + class SentimentResult(BaseModel): - sentiment: str # positive / negative / neutral - confidence: float # 0.0–1.0 + sentiment: str + confidence: float key_phrases: list[str] + @ai.flow() -async def sentiment(input: AnalysisInput) -> SentimentResult: +async def analyze_sentiment(text: str) -> SentimentResult: response = await ai.generate( - prompt=f'Analyze sentiment: {input.text}', - output_format='json', + prompt=f'Analyze customer sentiment: {text}', output_schema=SentimentResult, ) + if response.output is None: + raise RuntimeError(f'Model returned incomplete JSON: {response.error}') return response.output - - -app.include_router(serve_flow(sentiment), prefix='/api') # POST /api/sentiment ``` -Client calls this without `Accept: text/event-stream` — gets `{"result": {...}}` back. - --- -## Custom route with `handle_genkit_request` +## Python HTTP Client -When you need your own FastAPI dependencies (auth, tenancy, etc.) and still want -the Genkit wire format, call `handle_genkit_request` from a hand-written route: +Genkit HTTP endpoints expect `{"data": ...}` on input (`{"data": null}` uses the flow parameter's default) and return `{"result": ...}`: ```python -from fastapi import Depends, Request -from genkit_fastapi import handle_genkit_request +import httpx - -async def current_user(request: Request) -> dict: - # Resolve auth however you like. - return {'uid': request.headers.get('x-user-id', 'anon')} - - -@app.post('/api/secure-chat') -async def secure_chat( - request: Request, - user: dict = Depends(current_user), -): - return await handle_genkit_request( - request, - action=chat, - context={'auth': user}, - ) +async with httpx.AsyncClient(base_url='http://127.0.0.1:8080') as client: + resp = await client.post('/api/analyze_sentiment', json={'data': 'Fast shipping, great packaging.'}) + resp.raise_for_status() + result = resp.json()['result'] ``` - -Prefer `serve_flow` / `serve_agent` with `context_dependency=` when a single -`Depends` is enough — use `handle_genkit_request` when the route shape itself -needs to be custom. - ---- - -## Run with Dev UI - -```bash -GEMINI_API_KEY=your-key genkit start -- uv run src/main.py -``` - -Leave the process running until the CLI prints something like: - -``` -Genkit Developer UI: http://localhost:4000 -``` - -Open that URL. Port may differ if 4000 is busy. - - -## Python HTTP client (mounted flows) - -```python -import httpx, asyncio - -async with httpx.AsyncClient(base_url='http://127.0.0.1:18524') as client: - async def call(path, data): - r = await client.post(path, json={'data': data}) - r.raise_for_status() - return r.json()['result'] - a, b = await asyncio.gather( - call('/api/summarize', {'text': '...'}), - call('/api/sentiment', {'text': '...'}), - ) -``` - -Wire shape is `{"data": ...}` in, `{"result": ...}` out (not raw flow args). diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/migrating-from-0.12.md b/plugins/genkit/agent/skills/developing-genkit-python/references/migrating-from-0.12.md new file mode 100644 index 00000000..d56e3969 --- /dev/null +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/migrating-from-0.12.md @@ -0,0 +1,467 @@ +# Migrating from Genkit Python 0.12 to 0.13 + +Genkit Python 0.13 aligns every symbol, signature, and error contract ahead of 1.0 without deprecation aliases. Use this reference when upgrading a 0.12 codebase or replacing 0.12 patterns (`ActionRunContext`, `ToolRunContext`, `*_model()`, `GeminiConfigSchema`, `options=`, `abort_signal`, `restart_tool`, `RuntimeErrorReason`, `add_gcp_telemetry`, `MediaPart`, `TextPart`, or `from genkit.exp import Agent`). + +- [Agent Guardrails Before Editing](#agent-guardrails-before-editing): Lines 20-27 +- [1. Upgrade Packages Together](#1-upgrade-packages-together): Lines 31-44 +- [2. Single-Home Namespace Layout](#2-single-home-namespace-layout): Lines 48-107 +- [3. Removed Symbols & Enums Replaced by Strings](#3-removed-symbols--enums-replaced-by-strings): Lines 111-134 +- [4. Model & Embedder Helpers, Constructors, and Plugins](#4-model--embedder-helpers-constructors-and-plugins): Lines 138-194 +- [5. Config Classes, extra, tool_choice, and API Keys](#5-config-classes-extra-tool_choice-and-api-keys): Lines 198-243 +- [6. Typed Contexts, Multipart Tools, Interrupts, and Cancellation](#6-typed-contexts-multipart-tools-interrupts-and-cancellation): Lines 247-323 +- [7. Errors, HTTP Redaction (PublicError), and genkit.web](#7-errors-http-redaction-publicerror-and-genkitweb): Lines 327-366 +- [8. Prompts, Structured Output, Validation, and Telemetry](#8-prompts-structured-output-validation-and-telemetry): Lines 370-391 +- [9. Plugin & Custom Middleware Authors](#9-plugin--custom-middleware-authors): Lines 395-429 +- [10. Evaluators (0.13 Only - Removed in 1.0)](#10-evaluators-013-only--removed-in-10): Lines 433-458 +- [11. Verification Checklist](#11-verification-checklist): Lines 462-467 + +--- + +## Agent Guardrails Before Editing + +1. **Stop before porting evaluators.** Evaluators (`define_evaluator`, `ai.evaluate`, `genkit.evaluator`, `genkit-evaluators`) still run in 0.13, but this is their last release: **Genkit 1.0 cuts them from the 1.0 release** (evaluation will return as an additive feature in a later release; `genkit-evaluators` 0.13 requires `genkit<1`). If the project contains evaluator code, stop and ask the developer before porting it. Recommend deleting evaluator code now; apply the 0.13 renames below only if the developer chooses to stay pinned to `0.13.*`. +2. **Audit HTTP-served error paths.** Over HTTP (`serve_flow`, `serve_agent`, `genkit_fastapi_handler`, `handle_genkit_request`, `genkit_flask_handler`, `genkit_django_handler`, `genkit.web`), `GenkitError` and unhandled exceptions now return `500 {"status": "INTERNAL", "message": "Internal Error"}`. Convert errors meant for callers (404, 401/403, domain validation) to `PublicError(status, message, details)`. Ask the developer before exposing any error message whose contents might include internal data or tenant state. +3. **Audit silent behavior changes.** Type checkers catch symbol renames and signatures, not runtime behavior changes: + - Search for `{{` inside `ai.generate(prompt=..., system=...)` — strings there are now sent literally instead of rendered as Handlebars templates. + - Search for reads of `response.output` — truncated or malformed JSON now returns `None` and sets `response.error` (`Failure` with `ErrorReason.INVALID_OUTPUT`). + - Check `try` / `except GenkitError` blocks around `await my_flow(x)` or `await my_flow.run(x)` — actions now raise body exceptions unwrapped. + +--- + +## 1. Upgrade Packages Together + +Keep `genkit` and every `genkit-*` package on the same `0.13.*` minor line: + +```bash +uv add 'genkit==0.13.*' 'genkit-google-genai==0.13.*' # include every genkit-* package in use +``` + +- **Vertex AI Model Garden extras**: install extras for third-party SDKs: + ```bash + uv add 'genkit-vertexai[anthropic]' # Claude on Model Garden + uv add 'genkit-vertexai[openai]' # Llama, Mistral, and OpenAI-compatible models + ``` +- **Flask**: `genkit-flask` no longer pulls in `genkit-google-genai` transitively. Run `uv add 'genkit-google-genai==0.13.*'` if the app uses Gemini. + +--- + +## 2. Single-Home Namespace Layout + +Every public symbol lives in one module listed in its `__all__`. Plugin submodules (such as `genkit_openai.models` or `genkit_vertexai.model_garden`) are private; import from each plugin's top-level package (`genkit_google_genai`, `genkit_vertexai`, `genkit_openai`, etc.). + +- **`genkit`**: application code (`Genkit`, `Message`, `Role`, `Part`, `Media`, `Document`, `ToolChoice`, `ModelResponse`, `ModelResponseChunk`, `ModelStreamResponse`, `StreamResponse`, `FinishReason`, `Failure`, `ErrorReason`, `GenkitError`, `PublicError`, `Flow`, `FlowContext`, `Tool`, `ToolContext`, `Interrupt`, `MultipartToolResponse`, `Prompt`, `Embedding`, `Operation`, `GenkitLogger`, `get_logger`). +- **`genkit.model`**: custom models and model inspection (`Model`, `ModelRef`, `BackgroundModel`, `ModelContext`, `ModelRequest`, `ModelInfo`, `ModelSupports`, `ModelStage`, `GenerationUsage`, `ConstrainedSupport`, `ModelConfig`). +- **`genkit.embedder`**: custom embedders (`Embedder`, `EmbedderRef`, `EmbedRequest`, `EmbedResponse`, `EmbedderInfo`, `EmbedderSupports`). +- **`genkit.evaluator`**: evaluator types (`Evaluator`, `BatchEvaluator`, `BaseDataPoint`, `EvaluatorResponse`, `Score`, `ScoreDetails`, `ScoreStatus`). +- **`genkit.middleware`**: writing custom middleware (`Middleware`, `MiddlewareContext`, `GenerateHookParams`, `ModelHookParams`, `ToolHookParams`). Note: the `Middleware()` plugin that registers `Retry`, `Fallback`, `ToolApproval`, `Skills`, and `Filesystem` lives in `genkit_middleware`. +- **`genkit.formats`**: custom output formatters (`Formatter`, `FormatterConfig`, `FormatHandler`). +- **`genkit.plugin_api`**: writing plugins (`Plugin`, `Action`, `ActionKind`, `GenkitConfig`, `provider_error`, `loop_local_client`, `GENKIT_USER_AGENT`). +- **`genkit.web`**: HTTP serving helpers (`RequestData`, `ContextProvider`, `to_sse`, `read_body`, `wants_stream`, `result_json`, `error_body`, `error_json`, `error_sse`, `error_status`). +- **`genkit.telemetry`**: custom tracing (`Instrumentation`, `SpanContext`, `configure_instrumentation`). +- **`genkit.testing`**: test doubles (`EchoModel`, `ScriptedModel`, `StaticResponseModel`, `define_echo_model`, `define_scripted_model`, `define_static_response_model`). +- **`genkit.exp` & `genkit.exp.agent`**: experimental agent APIs (`from genkit.exp import Genkit, current_session`; `from genkit.exp.agent import Agent, Session, InMemorySessionStore, FileSessionStore, remote_agent`). + +Look up any moved symbol directly from the installed environment: + +```bash +python - <<'EOF' +import importlib +target = 'ModelSupports' +for mod in [ + 'genkit', 'genkit.model', 'genkit.embedder', 'genkit.evaluator', + 'genkit.middleware', 'genkit.formats', 'genkit.plugin_api', 'genkit.web', + 'genkit.telemetry', 'genkit.testing', 'genkit.exp', 'genkit.exp.agent', +]: + if target in importlib.import_module(mod).__all__: + print(mod) +EOF +``` + +### Renamed While Moving + +```python +# Old (0.12) +from genkit import Constrained, ExecutablePrompt, ModelUsage, RuntimeErrorReason, Stage, Supports + +# New (0.13) +from genkit import ErrorReason, Prompt +from genkit.model import ConstrainedSupport, GenerationUsage, ModelStage, ModelSupports +``` + +### Experimental (`exp`) Moves + +```python +# Old (0.12) +from genkit.exp import Agent, AgentError, InMemorySessionStore, Session +from genkit_fastapi import serve_agent +from genkit_middleware import Artifacts + +# New (0.13) +from genkit.exp import Genkit, current_session +from genkit.exp.agent import Agent, InMemorySessionStore, Session +from genkit_fastapi.exp import serve_agent +from genkit_middleware.exp import Artifacts +from genkit_google_cloud.exp import FirestoreSessionStore +``` + +`define_agent`, `define_custom_agent`, `define_prompt_agent`, and `@ai.agent()` exist on `genkit.exp.Genkit`, not `genkit.Genkit`. `AgentError` is removed: a failed turn sets `res.error` (`Failure`), and a call that produces no turn raises `GenkitError`. Tools in a detached turn are cancelled with `await task.abort()` instead of polling `abort_signal`. + +--- + +## 3. Removed Symbols & Enums Replaced by Strings + +0.12 model and config enums (`GoogleAIGeminiVersion`, `KnownGemini*`, `KnownGpt`, `KnownClaude`, `*Version`, `GeminiEmbeddingModels`, `VertexEmbeddingModels`, `EmbeddingTaskType`, `VertexAIEvaluationMetricType`) are removed. Pass plain string values; helper signatures use `Literal` unions for static checking while still accepting new model IDs as strings. + +```python +# Old (0.12) +from genkit_google_genai import GoogleAI, GoogleAIGeminiVersion + +model = GoogleAI.gemini_model(GoogleAIGeminiVersion.GEMINI_3_FLASH_PREVIEW) + +# New (0.13) +from genkit_google_genai import GeminiConfig, GoogleAI + +model = GoogleAI.gemini('gemini-3-flash-preview') +cfg = GeminiConfig(thinking_level='HIGH') # checked against 'MINIMAL' | 'LOW' | 'MEDIUM' | 'HIGH' +``` + +Other removed items and their 0.13 replacements: +- **`MediaPart` and `TextPart` (`Part(root=...)`)**: removed (`Part(root=...)` raises `ValueError`). Construct parts with `Part.from_text('...')` (or `Part(text='...')`) and `Part.from_media(url, content_type='image/jpeg')` (or `Part(media=Media(url=..., content_type=...))`). Read via `part.text` and `part.media`. +- **Name helpers** (`bedrock_name`, `anthropic_name`, `ollama_name`, `openai_model`, `genkit_eval_name`, `package_name`): pass `'plugin/model'` strings (e.g. `'openai/gpt-4o'`) or typed model refs (`OpenAI.gpt('gpt-4o')`). +- **`GoogleAI.imagen_model(...)`**: Imagen catalog is retired; use `GoogleAI.gemini_image('gemini-2.5-flash-image')`. +- **`Candidate`**: read `response.message` directly. +- **`Document.data`, `Document.data_type`, `Document.from_data()`**: construct with `Document(content=[...], metadata=...)`, `Document.from_text(...)`, or `Document.from_media(...)`. +- **`ai.registry`**: use lookup helpers directly on `Genkit` (e.g. `ai.prompt(...)`). + +--- + +## 4. Model & Embedder Helpers, Constructors, and Plugins + +Model helpers drop `_model`, and `.embedding()` becomes `.embedder()`: + +```python +# Old (0.12) +GoogleAI.gemini_model('gemini-flash-latest') +GoogleAI.gemini_image_model('gemini-2.5-flash-image') +GoogleAI.gemini_tts_model('gemini-2.5-flash-preview-tts') +GoogleAI.embedding('gemini-embedding-001') +VertexAI.veo_model('veo-3.1-generate-001') +Anthropic.claude_model('claude-sonnet-4-6') + +# New (0.13) +GoogleAI.gemini('gemini-flash-latest') +GoogleAI.gemini_image('gemini-2.5-flash-image') +GoogleAI.gemini_tts('gemini-2.5-flash-preview-tts') +GoogleAI.embedder('gemini-embedding-001') +VertexAI.veo('veo-3.1-generate-001') +Anthropic.claude('claude-sonnet-4-6') +OpenAI.gpt('gpt-5.2') +``` + +### Keyword-Only Signatures + +`Genkit(...)`, plugin constructors, `GenkitError(...)`, and options after an action's single input on `.run(...)` are keyword-only: + +```python +# Old (0.12) +ai = Genkit([GoogleAI()], GoogleAI.gemini_model('gemini-flash-latest')) +out = await my_flow.run(data, on_chunk, ctx) + +# New (0.13) +ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest')) +out = await my_flow.run(data, on_chunk=on_chunk, context=ctx) +``` + +### Provider Plugin Updates + +```python +# VertexAI (in genkit_google_genai): project_id= -> project= +from genkit_google_genai import VertexAI +VertexAI(project='my-project', location='us-central1') + +# Vertex AI Model Garden (in genkit_vertexai): ModelGardenPlugin -> ModelGarden +from genkit_vertexai import ModelGarden +ModelGarden(project='my-project', location='us-central1') + +# Ollama: server_address= -> base_url=; pulled models resolve automatically (no ModelDefinition/EmbeddingDefinition) +import os +from genkit_ollama import Ollama +Ollama(base_url=os.getenv('OLLAMA_HOST')) # local file paths for media are no longer read + +# Amazon Bedrock: ModelDefinition removed; BedrockImageConfig -> StabilityImageConfig +from genkit_amazon_bedrock import Bedrock, StabilityImageConfig +Bedrock(region='us-east-1', models=['us.amazon.nova-lite-v1:0']) +``` + +--- + +## 5. Config Classes, `extra`, `tool_choice`, and API Keys + +### Flat Config Classes + +`*ConfigSchema` classes drop the `Schema` suffix and flatten nested provider dicts. Unknown or nested keys (such as `thinking_config` or a typo like `'temprature'`) raise `GenkitError` (`INVALID_ARGUMENT`) before calling the model. + +```python +# Old (0.12) +from genkit_google_genai import GeminiConfigSchema, GeminiImageConfigSchema, GeminiTtsConfigSchema, GemmaConfigSchema +config = {'thinking_config': {'thinking_level': 'HIGH', 'include_thoughts': True}} +config = {'google_search_retrieval': True} +config = {'speech_config': {'voice_config': {'prebuilt_voice_config': {'voice_name': 'Kore'}}}} + +# New (0.13) +from genkit_google_genai import GeminiConfig, GeminiImageConfig, GeminiTtsConfig, GemmaConfig +cfg1 = GeminiConfig(thinking_level='HIGH', include_thoughts=True, google_search=True) +cfg2 = GeminiTtsConfig(voice='Kore') +``` + +```python +# Anthropic flat config (ThinkingConfig, OutputConfig, TaskBudget, RequestMetadata, *ToolChoice removed) +from genkit_anthropic import AnthropicConfig +cfg = AnthropicConfig(thinking='enabled', thinking_budget=1024) +cfg_adaptive = AnthropicConfig(thinking='adaptive', effort='high', task_budget=20000, user_id='diner-42') +``` + +- **Unmodeled provider fields (`extra`)**: pass provider-specific keys not declared on the config class in `extra={...}` (supported across Gemini, Veo, Interactions, OpenAI, Anthropic, Ollama, and Bedrock). `OpenAIConfig.user` is removed; use `extra={'user': '...'}` if needed. + ```python + from genkit_amazon_bedrock import BedrockConfig + cfg = BedrockConfig(extra={'some_provider_option': 1}) + ``` +- **Config layering**: defaults merge key-by-key (`ModelRef` < `Prompt` < call). Pass `None` for a field to clear an inherited key. +- **`tool_choice` on `generate`**: moved out of `config` onto `ai.generate(...)`: + ```python + # Old (0.12): config={'tool_choice': {'type': 'any'}} + await ai.generate(prompt='Check weather in Tokyo.', tools=[get_weather], tool_choice='required') # 'auto' | 'required' | 'none' + ``` +- **`options=` -> `config=` on `embed` and `evaluate`**: + ```python + await ai.embed(embedder=GoogleAI.embedder('gemini-embedding-001'), content='doc', config={'task_type': 'RETRIEVAL_DOCUMENT'}) + ``` +- **API keys rejected in `config`**: passing `api_key` in `config` now raises `GenkitError` (`INVALID_ARGUMENT`) in every plugin. Set the key on the plugin or pass per-request keys in `context`: + ```python + await ai.generate(prompt='Hello', context={'secrets': {'api_key': user_key}}) + ``` + Note: `ModelGarden` uses ADC only and rejects per-request keys; `ai.embed()` has no `context` parameter and always uses the plugin's key. + +--- + +## 6. Typed Contexts, Multipart Tools, Interrupts, and Cancellation + +### Typed Contexts (`FlowContext`, `ToolContext`, `ModelContext`) + +`ActionRunContext` and `ToolRunContext` are removed from `genkit`. Chunk types move from `@ai.flow(chunk_type=...)` into the generic parameter on `FlowContext` / `ToolContext`. Every action accepts at most one input parameter plus an optional typed context; Genkit finds the context by its type annotation. The new contexts subclass `ActionRunContext`, so an old annotation still type-checks once the import is fixed. + +```python +from pydantic import BaseModel +from genkit import FlowContext, ModelResponse, MultipartToolResponse, Part, ToolContext +from genkit.model import ModelContext, ModelRequest + +class Status(BaseModel): + step: str + +# Flow streaming typed chunks +@ai.flow() +async def plan_trip(city: str, ctx: FlowContext[Status]) -> str: + ctx.send_chunk(Status(step='searching flights')) + return f'Planned {city}' + +# Tool streaming typed chunks + returning multipart content +@ai.tool() +async def capture_chart(metric: str, ctx: ToolContext[Status]) -> MultipartToolResponse: + ctx.send_chunk(Status(step='rendering')) + png = Part.from_media('data:image/png;base64,iVBORw0KGgo=', content_type='image/png') + return MultipartToolResponse(output={'ok': True, 'metric': metric}, content=[png]) + +# Custom model implementation +async def my_model_fn(request: ModelRequest, ctx: ModelContext) -> ModelResponse: + ... +``` + +### Multipart Tool Responses + +`from genkit import response` is removed. Replace `response({'ok': True}, parts=[png])` with `MultipartToolResponse(output={'ok': True}, content=[png])` (see `capture_chart` above). + +### Interrupts + +- `restart_tool` and `respond_to_interrupt` are replaced by methods on the interrupt `Part`: + ```python + # Old (0.12): restart_tool(interrupt=intr, resumed_metadata={'approved': True}) + # Old (0.12): respond_to_interrupt({'status': 'declined'}, interrupt=intr) + + # New (0.13) + restart = intr.restart(resumed_metadata={'approved': True}) + reply = intr.respond({'status': 'declined'}) + ``` +- `Interrupt` now subclasses `BaseException` (like `asyncio.CancelledError`) so a tool's `except Exception:` cannot swallow an approval interrupt. Catch `Interrupt` explicitly only if inspecting and re-raising it. + +### Native `asyncio` Cancellation + +`ctx.abort_signal` is removed. Delete `if ctx.abort_signal.is_set(): raise ...` checks; cancellation uses standard `asyncio.CancelledError`, propagating into model requests, retry sleeps, and running tools at any `await`. If a tool catches `asyncio.CancelledError`, it must re-raise it. + +```python +import asyncio + +# Cancel an in-flight call +task = asyncio.create_task(ai.generate(prompt='Draft a long report.')) +task.cancel() + +# Stop a flow stream early — MUST use `async with` (or `await stream.aclose()`), +# otherwise a bare `break` leaves the background flow running. +async with plan_trip.stream('Kyoto') as stream: + async for chunk in stream: + if chunk.step == 'searching flights': + break # cancels plan_trip; stream.response raises CANCELLED + +# Put partial-work cleanup in `finally` inside tools +@ai.tool() +async def reserve_table(party_size: int) -> dict: + hold_id = await acquire_hold(party_size) + try: + return await confirm_hold(hold_id) + finally: + if not await is_confirmed(hold_id): + await release_hold(hold_id) +``` + +--- + +## 7. Errors, HTTP Redaction (`PublicError`), and `genkit.web` + +### Returned vs Raised Errors + +- **Returned errors (`Failure`)**: `res.error`, `operation.error`, and `agent_res.error` are typed `Failure` (`from genkit import ErrorReason, Failure`) with `.status`, `.message`, `.details`, and `.reason`. `RuntimeErrorReason` is renamed to `ErrorReason`. +- **Raised errors (`GenkitError`)**: `e.original_message`, `e.http_code`, `e.trace_id`, and `e.cause` are removed. Read `e.message`, `e.status`, `e.details`, `e.reason`, and `e.__cause__`. +- **Plugin exception classes removed**: classes like `OllamaConnectionError` are gone; inspect `e.status` (e.g. `if e.status == 'UNAVAILABLE':`). +- **Action exceptions are unwrapped**: `await my_flow.run(x)` raises the exception the body raised. Replace `except GenkitError` wrappers that were catching your own exception types with those types directly. `Retry` and `Fallback` only retry failures a plugin classified (such as `UNAVAILABLE`). + +### HTTP Serving & `PublicError` + +In-process calls (`await my_flow(x)`, `await my_flow.run(x)`) see full unwrapped exceptions. Over HTTP, Genkit redacts internal failures: +- **`PublicError(status, message, details)`**: returns the matching HTTP status (e.g. `NOT_FOUND` -> 404, `UNAUTHENTICATED` -> 401, `PERMISSION_DENIED` -> 403) with `status`, `message`, and `details`. +- **Invalid top-level flow input**: returns 400 with field names and Pydantic's built-in messages (custom validator strings are replaced with `Invalid value` unless the validator raises `PublicError`). +- **All other `GenkitError`s, provider errors, and Python exceptions**: logged server-side with traceback and returned to the client as `500 {"status": "INTERNAL", "message": "Internal Error"}`. +- **Streaming**: the same body arrives inside a `data: {"error": {...}}` SSE frame. + +Everything in a `PublicError`'s `message` and `details` reaches the client verbatim; keep secrets and other tenants' IDs out of it. + +**Context providers**: `RequestData` moves from `genkit.plugin_api` to `genkit.web` and exposes `.method`, `.headers`, and `.input` on Flask, FastAPI, and Django. Replace framework-specific reads like `req.request.META.get('HTTP_AUTHORIZATION')` with `req.headers.get('authorization')`. + +```python +from genkit import PublicError +from genkit.web import RequestData, error_body, error_status + +async def auth_context(req: RequestData) -> dict: + token = req.headers.get('authorization') + if not token: + raise PublicError('UNAUTHENTICATED', 'Missing bearer token') # 401 + return {'user': await verify_token(token)} + +@ai.flow() +async def get_order(order_id: str) -> dict: + order = await db_find(order_id) + if order is None: + raise PublicError('NOT_FOUND', f'Order {order_id} not found') # 404 + return order +``` + +For hand-written HTTP routes that call `my_flow.run(...)` directly, serialize errors with `error_body(e)` and `error_status(e)` from `genkit.web`. + +--- + +## 8. Prompts, Structured Output, Validation, and Telemetry + +- **No Handlebars rendering in `ai.generate`**: `ai.generate(prompt='Hi {{name}}')` sends `'Hi {{name}}'` verbatim. Move templates to `ai.define_prompt(name='greet', prompt='Hi {{name}}')` or `.prompt` files. +- **Prompt calls & validation**: `await prompt({'dish': 'pie'}, config={'temperature': 0.2})` validates input against `input.schema`, fills in `.prompt` defaults, merges `config` key-by-key over the prompt's config, and locks `output_schema` at definition time. +- **Truncated structured output**: if a reply hits `max_output_tokens` or produces invalid JSON, `res.output` is `None` and `res.error.reason == ErrorReason.INVALID_OUTPUT`: + ```python + res = await ai.generate(prompt='...', output_schema=City) + if res.output is None: + raise RuntimeError(f'Structured output failed: {res.error}') + ``` +- **Flow I/O validation**: flow inputs and return values are validated against their type annotations, so a flow that returns the wrong shape raises; `{"data": null}` over HTTP uses the parameter default. +- **Media URL limit**: media URLs over 20 MB fail instead of downloading into memory. +- **Telemetry**: + ```python + # Old (0.12): add_gcp_telemetry(project_id='my-project', force_dev_export=True) + # New (0.13): FIREBASE_PROJECT_ID fallback removed; pass project= or set GOOGLE_CLOUD_PROJECT + from genkit_google_cloud import enable_google_cloud_telemetry + enable_google_cloud_telemetry(project='my-project') + + # Custom trace instrumentation (replaces tracer / add_custom_exporter / AdjustingTraceExporter) + from genkit.telemetry import Instrumentation, configure_instrumentation + ``` + +--- + +## 9. Plugin & Custom Middleware Authors + +```python +# Custom middleware: BaseMiddleware -> Middleware, GenerateMiddlewareContext -> MiddlewareContext +from pydantic import BaseModel +from genkit.middleware import Middleware, MiddlewareContext + +class PiiRedactConfig(BaseModel): + mask: str = '[REDACTED]' + +@ai.middleware +class PiiRedact(Middleware[PiiRedactConfig]): + name = 'pii_redact' + + async def model(self, params, ctx: MiddlewareContext, next_fn): + return await next_fn(params, ctx) +``` + +- `GenerateMiddleware`, `MiddlewareRef`, `MiddlewarePlugin`, and `new_middleware` are removed; `Middleware.tools()` returns `Tool` handles. +- Removed `genkit.plugin_api` helpers and their replacements: + ```python + # Old (0.12) + from genkit.plugin_api import model, model_ref, background_model, embedder, embedder_ref, ModelConfig + from genkit.plugin_api import wrap_http_error, from_http_code, parse_retry_after_ms + from genkit.plugin_api import get_cached_client, GENKIT_CLIENT_HEADER, GENKIT_VERSION, tracer, add_custom_exporter + + # New (0.13) + from genkit import get_logger + from genkit.embedder import Embedder, EmbedderRef + from genkit.model import BackgroundModel, Model, ModelConfig, ModelRef + from genkit.plugin_api import GENKIT_USER_AGENT, GenkitConfig, loop_local_client, provider_error + from genkit.telemetry import Instrumentation, configure_instrumentation + ``` +- Custom formatters import `Formatter`, `FormatterConfig`, and `FormatHandler` from `genkit.formats`. +- `Plugin.resolve(kind, name)` now receives `name` **without** the `'myplugin/'` prefix. `Plugin.model()`, `Plugin.embedder()`, and `Supports.context` are removed. `generate_operation` receives only what a background start uses. + +--- + +## 10. Evaluators (0.13 Only — Removed in 1.0) + +Apply only if the developer explicitly confirms keeping evaluators pinned to `genkit==0.13.*`. Renames: `EvalFnResponse` -> `EvaluatorResponse`, `EvalStatusEnum` -> `ScoreStatus`, `Details` -> `ScoreDetails`; `evaluation` is now a `list[Score]`; `ai.define_evaluator(name=..., fn=...)` becomes positional `(name, fn)`; `display_name=` and `definition=` stay required keywords. `EvaluatorResponse` requires `test_case_id`. + +```python +from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score, ScoreDetails, ScoreStatus + +async def exact_match(dp: BaseDataPoint, _options: object = None) -> EvaluatorResponse: + passed = str(dp.output or '').strip() == str(dp.reference or '').strip() + return EvaluatorResponse( + test_case_id=dp.test_case_id or '', + evaluation=[ + Score( + score=1.0 if passed else 0.0, + status=ScoreStatus.PASS if passed else ScoreStatus.FAIL, + details=ScoreDetails(reasoning='Exact string match'), + ) + ], + ) + +# Positional (name, fn); display_name and definition are required keywords +ai.define_evaluator('byo/exact_match', exact_match, display_name='Exact Match', definition='Exact string match.') + +# options= -> config=; returns list[EvaluatorResponse] directly (no .root) +results = await ai.evaluate(evaluator='byo/exact_match', dataset=rows, config={}) +``` + +--- + +## 11. Verification Checklist + +1. Run the project's type checker (`pyright`, `mypy`, or `ty`) — catches moved imports, enum-to-`Literal` arguments, keyword-only calls, `FlowContext`/`ToolContext` generics, and `OutputT | None` checks on `response.output`. +2. Grep for `{{` inside `ai.generate(` calls and move templates to `ai.define_prompt` or `.prompt` files. Grep for reads of `response.output` that assume it is never `None`. +3. Grep for `GenkitError` inside HTTP-served flows and context providers; convert caller-facing errors to `PublicError`. +4. Run the test suite. diff --git a/plugins/genkit/agent/skills/developing-genkit-python/references/setup.md b/plugins/genkit/agent/skills/developing-genkit-python/references/setup.md index 136f3869..1043cefd 100644 --- a/plugins/genkit/agent/skills/developing-genkit-python/references/setup.md +++ b/plugins/genkit/agent/skills/developing-genkit-python/references/setup.md @@ -2,38 +2,24 @@ ## New project -**Always use a virtual environment** — never install Genkit into the system interpreter. With **uv**, the project’s **`.venv`** is created and used by `uv sync` / `uv run` automatically once you add dependencies. +**Always use a virtual environment** — never install Genkit into the system interpreter. With **uv**, `uv add` and `uv run` manage `.venv` automatically. ```bash mkdir my-app && cd my-app uv init -# uv may set requires-python to a very new floor (e.g. >=3.14). Genkit needs -# 3.10+ — edit pyproject.toml if your interpreter is older. -# uv init may also write .python-version (e.g. 3.14); delete or edit it -# if it fights an explicit `uv venv --python 3.12`. +# Genkit requires Python 3.10+. If `uv init` pins a newer interpreter in +# pyproject.toml or .python-version than you have installed, adjust it first. uv venv --python 3.12 .venv -# Unix: source .venv/bin/activate -# Windows: .venv\Scripts\activate -uv add genkit genkit-google-genai -# Agents with ToolApproval / Filesystem / Artifacts / Retry also need: -# uv add genkit-middleware -# HTTP serve: uv add genkit-fastapi -# Evals: uv add genkit-evaluators -# Also add pydantic if you use BaseModel schemas with Dotprompt. -export GEMINI_API_KEY=your_key_here -``` - -Import the Google AI plugin as: +uv add 'genkit==0.13.*' 'genkit-google-genai==0.13.*' -```python -from genkit_google_genai import GoogleAI +export GEMINI_API_KEY=your_key_here ``` -`uv init` creates `pyproject.toml`. Add your app under something like `src/main.py` (or match whatever layout `uv` generated) and point `genkit start` at that entrypoint. +Keep `genkit` and every `genkit-*` package on the **same `0.13.*` release**. `pydantic` is installed automatically with `genkit`, though adding `pydantic` directly to `pyproject.toml` is fine when defining your own `BaseModel` schemas. -## pyproject.toml +Place your entrypoint at `src/main.py` (or match the file generated by `uv init`) and run it under `genkit start -- uv run src/main.py`. -Minimal `[project]` block with unpinned Genkit deps (resolver picks compatible releases): +## `pyproject.toml` ```toml [project] @@ -41,20 +27,32 @@ name = "my-app" version = "0.1.0" requires-python = ">=3.10" dependencies = [ - "genkit", - "genkit-google-genai", + "genkit==0.13.*", + "genkit-google-genai==0.13.*", ] ``` -## Plugins - -Packages are **`genkit-*`** on PyPI, e.g. `genkit-google-genai`, `genkit-vertexai`, -`genkit-anthropic`, `genkit-fastapi`, `genkit-middleware`, `genkit-evaluators`. -Install with `uv add genkit-<name>`. - -Import modules use underscores (e.g. `genkit_google_genai`, `genkit_middleware`). -See [Examples](examples.md) and [Agents](agents.md). +## Packages & Imports + +PyPI distribution names use hyphens (`genkit-*`); Python import modules use underscores (`genkit_*`). Import everything from each package's top-level module (or `.exp` for experimental agent integrations): + +- **`genkit-google-genai`**: `from genkit_google_genai import GoogleAI, VertexAI, GeminiConfig` (Gemini API via `GEMINI_API_KEY`, or Google Cloud Vertex AI via ADC). +- **`genkit-vertexai`**: `from genkit_vertexai import ModelGarden` (third-party models on Vertex AI Model Garden). Install the extra matching the models you call: + - `uv add 'genkit-vertexai[anthropic]'` for Claude models + - `uv add 'genkit-vertexai[openai]'` for Llama, Mistral, and OpenAI-compatible models +- **`genkit-anthropic`**: `from genkit_anthropic import Anthropic, AnthropicConfig` (`ANTHROPIC_API_KEY`). +- **`genkit-openai`**: `from genkit_openai import OpenAI, OpenAIConfig` (`OPENAI_API_KEY` or custom `base_url=`). +- **`genkit-ollama`**: `from genkit_ollama import Ollama, OllamaConfig` (local Ollama server; auto-discovers pulled models). +- **`genkit-amazon-bedrock`**: `from genkit_amazon_bedrock import Bedrock, BedrockConfig, StabilityImageConfig`. +- **`genkit-middleware`**: `from genkit_middleware import Middleware, Retry, Fallback, ToolApproval, Skills, Filesystem` (plus `from genkit_middleware.exp import Artifacts` for agent sessions). +- **`genkit-fastapi`**: `from genkit_fastapi import serve_flow, genkit_fastapi_handler, handle_genkit_request` (plus `from genkit_fastapi.exp import serve_agent`). +- **`genkit-flask`**: `from genkit_flask import genkit_flask_handler` (requires `flask[async]`; does not install `genkit-google-genai` automatically; add your model plugin explicitly). +- **`genkit-django`**: `from genkit_django import genkit_django_handler`. +- **`genkit-mcp`**: `from genkit_mcp import McpClient, McpHost, define_mcp_client, define_mcp_host` (Model Context Protocol clients and tool hosts). +- **`genkit-google-cloud`**: `from genkit_google_cloud import enable_google_cloud_telemetry` (plus `from genkit_google_cloud.exp import FirestoreSessionStore`). +- **`genkit-otel`**: `from genkit_otel import GenAiInstrumentation, ContentCapturingMode, OtelEventLogger` (OpenTelemetry GenAI spans and metrics). +- **`genkit-evaluators`**: `from genkit_evaluators import register_genkit_evaluators`. Last release with evaluators; Genkit 1.0 cuts evaluators from the 1.0 release (evaluation will return as an additive feature in a later release). Requires `genkit<1` (see [Evals](evals.md)). ## Python version -**3.10+**. Prefer a project venv: `uv venv --python 3.12 .venv` (or newer) before you run commands. +**Python 3.10+**. Upgrading an existing 0.12 app? Follow [Migrating from 0.12 to 0.13](migrating-from-0.12.md). diff --git a/plugins/genkit/skills-lock.json b/plugins/genkit/skills-lock.json index 27aa8c51..183424a7 100644 --- a/plugins/genkit/skills-lock.json +++ b/plugins/genkit/skills-lock.json @@ -5,25 +5,25 @@ "source": "genkit-ai/skills", "sourceType": "github", "skillPath": "skills/developing-genkit-dart/SKILL.md", - "computedHash": "f926f5759c9c181c41fde7b467e54ac80cdbfbc77bdc3d7e9b23670a110166bc" + "computedHash": "e4b28b8204c27ccf241f4b6789abc8667793f97d0e3464e9ab2ce06c173ad5f3" }, "developing-genkit-go": { "source": "genkit-ai/skills", "sourceType": "github", "skillPath": "skills/developing-genkit-go/SKILL.md", - "computedHash": "bada73fe5e169f18af415c3b23c597b8d56c4425520d00402da0d1250170a595" + "computedHash": "a6e8c1fae5faa586d32a8804baae5987e05f05b068390fe0995e23c4fbacc143" }, "developing-genkit-js": { "source": "genkit-ai/skills", "sourceType": "github", "skillPath": "skills/developing-genkit-js/SKILL.md", - "computedHash": "36a18619777e1355afd74a21f3eacbff19539f6605889f0d6ec8ae0c70faf3b3" + "computedHash": "de3b0d682be3a54521211c26be8bc9a569790d77a6c24314fc9b9b0a254683fe" }, "developing-genkit-python": { "source": "genkit-ai/skills", "sourceType": "github", "skillPath": "skills/developing-genkit-python/SKILL.md", - "computedHash": "1af3c74cb881ccf0edfdf406fc629413dc749ddd5f6bf67fffce802be879ef0f" + "computedHash": "07bd4272ec6e9133d7ad01b301f2aa54a55814450320023434473b6b1fb7a1d2" } } } diff --git a/plugins/mastra/.agents/skills/mastra/SKILL.md b/plugins/mastra/.agents/skills/mastra/SKILL.md index cf7e734c..173a67f9 100644 --- a/plugins/mastra/.agents/skills/mastra/SKILL.md +++ b/plugins/mastra/.agents/skills/mastra/SKILL.md @@ -1,10 +1,10 @@ --- name: mastra -description: "Comprehensive Mastra framework guide for building agents, workflows, tools, memory, workspaces, and storage with current APIs. Use for native capability discovery, logging, tracing, metrics/cost estimates, dataset experiments, documentation lookup, API verification, TypeScript setup, common errors, migrations, and `mastra api` CLI tasks: inspect or call resources on local, Mastra platform, or remote servers. For Mastra Factory operations, discover and activate the companion mastra-factory skill. Also report verified bugs and concrete documentation defects to the Mastra team automatically via the feedback endpoint." +description: "Comprehensive Mastra framework guide for building agents, workflows, tools, memory, workspaces, and storage with current APIs. Use for native capability discovery, logging, tracing, metrics/cost estimates, dataset experiments, documentation lookup, API verification, TypeScript setup, common errors, migrations, building and deploying apps to Mastra Platform via Platform APIs and `mastra deploy`, and `mastra api` CLI tasks: inspect or call resources on local, Mastra platform, or remote servers. For Mastra Factory operations, discover and activate the companion mastra-factory skill. Also report verified bugs and concrete documentation defects to the Mastra team automatically via the feedback endpoint." license: Apache-2.0 metadata: author: Mastra - version: "2.4.0" + version: "2.5.0" repository: https://github.com/mastra-ai/skills --- @@ -46,7 +46,8 @@ ls node_modules/@mastra/ | I'm getting an error... | [`references/common-errors.md`](references/common-errors.md) | Common errors and solutions | | Upgrade from v0.x to v1.x | [`references/migration-guide.md`](references/migration-guide.md) | Version upgrade workflows | | Inspect/call server resources via CLI | [`references/mastra-api.md`](references/mastra-api.md) | `mastra api` CLI for local, Mastra platform, or remote servers | -| Find exact traces with complex predicates | [`references/trace-query.md`](references/trace-query.md) | Query completed traces by trace fields or related spans, scores, and feedback | +| Deploy to Mastra Platform, provision projects, or use Platform sandbox/filesystem | [`references/platform.md`](references/platform.md) | Platform REST APIs, credentials, and `mastra deploy` without browser automation | +| Find exact traces with complex predicates | [`references/trace-query.md`](references/trace-query.md) | Query completed traces by trace fields or related spans | | Count traces or compute grouped measures (error rate, latency, tokens, cost) | [`references/trace-query.md`](references/trace-query.md#aggregate-traces) | Aggregate completed traces with `mastra api trace aggregate` over the same selection | | List individual spans (failed tool calls, nested agent runs) | [`references/trace-query.md`](references/trace-query.md#query-spans) | Query completed spans with `mastra api span query` and span-field predicates | | Compute measures per span (tool error rates, nested agent runs, model tokens) | [`references/trace-query.md`](references/trace-query.md#aggregate-spans) | Aggregate spans with `mastra api span aggregate`, grouped by span fields | @@ -95,7 +96,11 @@ Then open `http://localhost:4111` in a browser to show Mastra Studio to your hum Use `mastra api` to inspect or call resources on local dev servers, Mastra platform deployments, or remote Mastra endpoints. It is useful for agent-readable state, execution, traces, logs, scores, threads, and workflow operations. See [`references/mastra-api.md`](references/mastra-api.md) for usage patterns. -For exact trace selection that needs recursive predicates or conditions over related spans, scores, or feedback, read [`references/trace-query.md`](references/trace-query.md). Before using `mastra api trace query`, confirm that the installed CLI exposes the command. Use `--schema` for the target's request/response shape and structural constraints, and use the canonical documentation found through [`references/remote-docs.md`](references/remote-docs.md) for supported fields, operators, and semantics. Preserve opaque pagination cursors and fetch trace or span details only after selecting candidates. +For exact trace selection that needs recursive predicates or conditions over related spans, read [`references/trace-query.md`](references/trace-query.md). Before using `mastra api trace query`, confirm that the installed CLI exposes the command. Use `--schema` for the target's request/response shape and structural constraints, and use the canonical documentation found through [`references/remote-docs.md`](references/remote-docs.md) for supported fields, operators, and semantics. Preserve opaque pagination cursors and fetch trace or span details only after selecting candidates. + +## Deploy to Mastra Platform + +When the user asks to deploy to Mastra, whether to build a new app and deploy it or to deploy an existing Mastra project, read [`references/platform.md`](references/platform.md) before running any deploy or Platform API command. It covers which steps need the human (sign-up and sign-in only), credentials, REST discovery and project creation, `PlatformFilesystem` and `PlatformSandbox`, `mastra deploy`, and verification. Build or locate the app first with the references above, then follow `platform.md` to deploy. ## Mastra Factory diff --git a/plugins/mastra/.agents/skills/mastra/references/create-mastra.md b/plugins/mastra/.agents/skills/mastra/references/create-mastra.md index a09d8d3b..d3cfbd06 100644 --- a/plugins/mastra/.agents/skills/mastra/references/create-mastra.md +++ b/plugins/mastra/.agents/skills/mastra/references/create-mastra.md @@ -44,12 +44,6 @@ npm create mastra@latest my-project --empty `--empty` and `--template` are separate creation modes. The `--llm` (`-l`) and `--llm-api-key` options apply only to the default starter. -**Skip the example agent:** - -```bash -npm create mastra@latest --no-example -``` - **Use a specific template:** ```bash @@ -228,3 +222,5 @@ After creating your project with `create mastra`: - [Agents](https://mastra.ai/docs/agents/overview) - [Workflows](https://mastra.ai/docs/workflows/overview) - [GitHub](https://github.com/mastra-ai/mastra) + +To deploy the project to Mastra Platform, see [`platform.md`](platform.md). diff --git a/plugins/mastra/.agents/skills/mastra/references/mastra-api.md b/plugins/mastra/.agents/skills/mastra/references/mastra-api.md index 2fcc70ad..a54c6374 100644 --- a/plugins/mastra/.agents/skills/mastra/references/mastra-api.md +++ b/plugins/mastra/.agents/skills/mastra/references/mastra-api.md @@ -4,6 +4,8 @@ How to use the `mastra api` CLI to interact with Mastra servers. Prefer fast, fo Use this reference when the user asks to inspect or call agents, workflows, tools, MCP servers, memory threads, traces, logs, metrics, scores, datasets, experiments, or to debug/test `mastra api` commands. +This reference does not cover the Mastra Platform control plane (projects, environments, deploys, tokens, workspace provisioning); for those, see [`platform.md`](platform.md). + ## Setup The CLI can interact with any reachable Mastra server: @@ -11,7 +13,7 @@ The CLI can interact with any reachable Mastra server: - Local dev server: `http://localhost:4111` from `npm run dev` - Mastra platform deployment: Use the deployment URL - Remote/self-hosted server: Use the server URL -- Hosted Mastra Platform Observability: `https://observability.mastra.ai` (auto-targeted by `trace`, `log`, `score`, and `metric` commands) +- Hosted Mastra Platform Observability: `https://observability.mastra.ai` (auto-targeted by `trace`, `span`, `log`, `score`, and `metric` commands) For local servers, `mastra api` defaults to `http://localhost:4111`: @@ -50,7 +52,7 @@ Runtime commands (`agent`, `workflow`, `tool`, `mcp`, `thread`, `memory`, `datas 2. `http://localhost:4111` for a local `mastra dev` server. 3. `.mastra-project.json` for a Mastra platform project. -Observability commands (`trace`, `log`, `score`, `metric`) target `https://observability.mastra.ai` by default instead of a project deployment URL. EU projects use `https://observability.eu.mastra.ai`: the CLI picks it automatically from `mastra@1.34.2`. On older versions, EU requests time out with `REQUEST_TIMEOUT`; pass `--url https://observability.eu.mastra.ai`. The CLI resolves credentials in this order: +Observability commands (`trace`, `span`, `log`, `score`, `metric`) target `https://observability.mastra.ai` by default instead of a project deployment URL. EU projects use `https://observability.eu.mastra.ai`: the CLI picks it automatically from `mastra@1.34.2`. On older versions, EU requests time out with `REQUEST_TIMEOUT`; pass `--url https://observability.eu.mastra.ai`. The CLI resolves credentials in this order: 1. Explicit `Authorization` and `X-Mastra-Project-Id` headers passed with `--header`. 2. `MASTRA_PLATFORM_ACCESS_TOKEN` and `MASTRA_PROJECT_ID` from the environment. @@ -76,7 +78,7 @@ Pass `--url` and `--header` only when overriding the hosted observability target Start with these command groups when present; verify with `mastra api --help` if the group fails. ```text -agent workflow tool mcp thread memory trace log metric score dataset experiment +agent workflow tool mcp thread memory trace span log metric score dataset experiment ``` ## Fast path for read-only requests @@ -146,7 +148,7 @@ npx mastra api span aggregate --help npx mastra api span aggregate --schema ``` -Use `trace query` instead of `trace list` when selection requires recursive predicates, metadata filters, or conditions over related spans, scores, or feedback. First use `trace query --help` to confirm that the installed CLI exposes the command. The inline JSON query is required, and its cursor-bearing response stays nested under `data`. Read [`trace-query.md`](trace-query.md) for availability checks, the division between CLI schema discovery and canonical documentation, query construction, and pagination. +Use `trace query` instead of `trace list` when selection requires recursive predicates, metadata filters, or conditions over related spans. First use `trace query --help` to confirm that the installed CLI exposes the command. The inline JSON query is required, and its cursor-bearing response stays nested under `data`. Read [`trace-query.md`](trace-query.md) for availability checks, the division between CLI schema discovery and canonical documentation, query construction, and pagination. Use `trace aggregate` when the answer is a count or measure (error rate, latency percentiles, tokens, cost) grouped by up to two dimensions or time buckets rather than a list of traces. It takes the same `timeRange` and `where` as `trace query`. See [`trace-query.md`](trace-query.md#aggregate-traces). @@ -215,6 +217,6 @@ curl -fsS "$MASTRA_URL/api/system/api-schema" \ - Working memory update requires the agent's memory to have working memory enabled. - Empty lists may simply mean the server has no matching stored data yet. - `trace list` and `trace get` return lightweight payloads by default (no span input, output, attributes, or metadata). Pass `--verbose` to fetch full span records, or use `trace span <traceId> <spanId>` to fetch one specific span in full. -- `trace query` requires inline JSON, queries completed traces, and preserves its opaque cursor at `data.page.next`. Pass that value unchanged as `page.after` with the same query shape. +- `trace query` requires inline JSON, queries completed traces, and defaults to keyset pagination: pass `data.page.next` unchanged as `page.after` with the same query shape. Numbered pages and delta polling use different response metadata; see [`trace-query.md`](trace-query.md#pagination). - `trace aggregate` requires inline JSON and isn't paginated; when `truncated` is `true`, narrow the selection. - `span query` requires inline JSON and pages like `trace query`: pass `data.page.next` unchanged as `page.after`. `span aggregate` isn't paginated, like `trace aggregate`. diff --git a/plugins/mastra/.agents/skills/mastra/references/platform.md b/plugins/mastra/.agents/skills/mastra/references/platform.md new file mode 100644 index 00000000..758cf5cd --- /dev/null +++ b/plugins/mastra/.agents/skills/mastra/references/platform.md @@ -0,0 +1,164 @@ +# Mastra Platform + +## When to use + +Use this reference to create Mastra Platform projects, deploy a Mastra app (new or existing), and use Platform workspaces (`PlatformFilesystem`, `PlatformSandbox`) in generated code. For calls to a running Mastra server (agents, workflows, traces), use [`mastra-api.md`](mastra-api.md); it does not cover the Platform control plane. + +The Platform API changes often. Treat the live spec `https://platform.mastra.ai/openapi.json` and `mastra <command> --help` as the source of truth, and check them before using anything not listed here. + +## Human steps vs agent steps + +The browser is only for the human: signing up, creating an organization, and `mastra auth login`. Never automate the browser or scrape the dashboard. + +Everything else is agent work: scaffolding, REST discovery and provisioning, `mastra deploy`, and verification. When credentials are missing, stop and ask the user to do one of: + +- Run `mastra auth login` in their own terminal (browser sign-in). The CLI saves the session, and later CLI commands reuse it without a TTY. +- Create an API token themselves (`mastra auth tokens create <name>` in their own terminal, or the dashboard) and put it, with `MASTRA_ORG_ID`, in the environment or an ignored env file. + +Do not run `mastra auth tokens create` yourself: it prints the secret to stdout. Never ask for secret values in chat, never print or log them, and never commit them. + +If you need anything you were not given — a credential, a model-provider key, which org or project to use — stop and ask the user. Do not take secrets from your own shell environment, other projects, or files the user did not point you to. + +## Credentials + +Two credential models exist. Do not mix them. + +| Variable | Used by | Notes | +| - | - | - | +| `MASTRA_API_TOKEN` | CLI and control-plane REST | Wins over saved `mastra auth login` credentials. Setting it puts `mastra deploy` in headless mode. | +| `MASTRA_ORG_ID` | CLI and control-plane REST | Sent as `x-organization-id`. Required with `MASTRA_API_TOKEN`. | +| `MASTRA_PROJECT_ID` | CLI headless deploy | Must be a project ID (not a slug or name) in headless mode. | +| `MASTRA_PLATFORM_API_URL` | CLI | Optional override. Default `https://platform.mastra.ai`. | +| `MASTRA_PLATFORM_ACCESS_TOKEN` | App runtime (workspaces, observability) | Not a deploy credential. Injected on deploy; locally, an org API token the user creates. | +| `MASTRA_ENVIRONMENT_ID`, `MASTRA_PLATFORM_BUCKET_NAME` | App runtime (workspaces) | Injected on deploy; set manually only for local runs. | + +Check for a saved session with `mastra auth whoami` before asking for credentials. With a saved session, CLI commands (including `mastra deploy`) work without a TTY and use the current org (`mastra auth orgs`; override with `--org <id>`). Only signing in, or signing in again after the session expires, needs a TTY. Do not read the saved credentials file; in session mode use CLI commands instead of the REST calls below, which need `MASTRA_API_TOKEN`. + +Keep control-plane credentials (`MASTRA_API_TOKEN`, `MASTRA_ORG_ID`) in the shell environment, not in the app's `.env`: deploy uploads app env files as-is and does not filter them. + +## Discover and provision over REST + +`mastra deploy` and `mastra auth` have no JSON output, so use REST when you need machine-readable state. Every control-plane call uses the same headers: + +```bash +curl -fsS https://platform.mastra.ai/v1/auth/orgs \ + -H "Authorization: Bearer $MASTRA_API_TOKEN" \ + -H "x-organization-id: $MASTRA_ORG_ID" +``` + +| Route | Purpose | +| - | - | +| `GET /v1/auth/orgs` | `{ organizations: [{ id, slug, name, role, isCurrent }] }`. Pick `MASTRA_ORG_ID` here (or `mastra auth whoami`). Org API keys (`sk_...`) get an empty list; ask the user for the org ID. | +| `GET /v1/projects` | List projects in the org. | +| `POST /v1/studio/projects` | Create a project. Body `{"name": "..."}` (1–64 chars); response `{ project: { id, slug, workspacesEnabled, ... } }`. Viewer/read-only roles get 403. | +| `GET /v1/projects/{id}` | Project details, including `workspacesEnabled`. | +| `GET /v1/projects/{projectId}/environments` | `{ environments: [{ id, name, slug, type, region, instanceUrl, customServerUrl, ... }] }`. Not in the public spec; called by the CLI. | +| `POST /v1/projects/{projectId}/environments` | Create an environment. Body `{"name": "production", "type": "production", "region": "us"}` (`region` optional). Not in the public spec; called by the CLI. | +| `GET /v1/projects/{projectId}/environment-deploys` | `{ deploys: [{ id, environmentName, status, error, errorCode, createdAt, ... }] }`. Not in the public spec; called by the CLI. | + +`/v1/projects` itself is list-only; project creation is under `/v1/studio/projects`. The spec also lists `POST /v1/auth/orgs` (create organization), but creating or choosing an organization is the user's decision: ask the user, do not create one. + +Before calling a route not listed here, check it exists with the method you need: + +```bash +curl -fsS https://platform.mastra.ai/openapi.json | jq '.paths["/v1/studio/projects"] | keys' +``` + +## Build with Platform workspaces + +Use `@mastra/platform-workspace` when the app needs durable file storage or remote command execution on the Platform. New projects have workspaces enabled and get one bucket per environment when the environment is created. + +```bash +npm install @mastra/platform-workspace +``` + +```typescript title="src/mastra/workspace.ts" +import { Workspace } from '@mastra/core/workspace'; +import { PlatformFilesystem, PlatformSandbox } from '@mastra/platform-workspace'; + +// Reads MASTRA_PLATFORM_ACCESS_TOKEN, MASTRA_PROJECT_ID, MASTRA_ENVIRONMENT_ID, +// and MASTRA_PLATFORM_BUCKET_NAME from the environment. The Platform injects them on deploy. +export const workspace = new Workspace({ + filesystem: new PlatformFilesystem(), + sandbox: new PlatformSandbox(), +}); +``` + +```typescript title="src/mastra/index.ts" +import { Mastra } from '@mastra/core'; +import { workspace } from './workspace'; + +export const mastra = new Mastra({ + // TODO: register your agents, workflows, and storage + workspace, +}); +``` + +The bucket is shared by every request in the environment. A statically configured `PlatformSandbox` is one shared sandbox; construct a new `PlatformSandbox()` (call `start()`, then `destroy()`) for per-task isolation. The sandbox does not mount the bucket. Check the installed package's types or embedded docs for sandbox options before using them. + +Local runs need all four runtime variables in the app's `.env`. Get `MASTRA_ENVIRONMENT_ID` from the environments route and the bucket name from `GET /v1/server/projects/{id}/environments/{envId}/bucket` (`envId` is the environment ID; response includes `status`: `provisioning`, `ready`, `failed`, or `destroyed`). The user creates the local `MASTRA_PLATFORM_ACCESS_TOKEN` (an org API token from the dashboard). For an offline loop, swap in `LocalFilesystem` and `LocalSandbox`. + +Projects created before workspaces were on by default may have `workspacesEnabled: false`. Enable with `POST /v1/server/projects/{id}/workspaces/enable`: it requires an admin role (an org API key may not qualify), returns 409 when the project has no environment yet, is idempotent, and cannot be undone. Ask the user before calling it. + +## Deploy + +`mastra deploy` builds the app, runs preflight checks, uploads `.mastra/output`, and follows the deploy until it is running or fails. The first deploy creates the environment (`--env`, default `production`) and writes `.mastra-project.json` (`projectId`, `projectName`, `projectSlug`, `organizationId`, optional `disablePlatformObservability`). It holds no secrets and links the directory to the project for later deploys. + +| Situation | Command | +| - | - | +| Saved `mastra auth login` session (no `MASTRA_API_TOKEN` set) | `mastra deploy --project <name> --yes` (creates the project in the current org if not found) | +| Headless, token in env | Export `MASTRA_API_TOKEN`, `MASTRA_ORG_ID`, `MASTRA_PROJECT_ID`, then `mastra deploy --yes` | +| Directory already linked | `mastra deploy --yes` (reads `.mastra-project.json`) | + +`--project <value>` deploys into an existing project whose ID, slug, or name matches, replacing whatever is running there; it creates a project only when nothing matches. When the directory is not linked yet, check for an existing project with that name first (`GET /v1/projects` with a token; otherwise ask the user). If one exists and you did not create it in this session, ask the user whether to deploy into it or pick a different name. + +In headless mode the project must already exist and `MASTRA_PROJECT_ID` must be its ID. For a new app, create it first with `POST /v1/studio/projects`, read `project.id`, and export it. Headless mode auto-accepts prompts. + +Other useful flags: `--env <name>`, `--region us|eu` (new environments only), `--env-file <file>`, `--workers dedicated|in-process`, `--skip-build`. Avoid `--skip-preflight`; it skips the checks that prevent broken deploys. + +**App environment variables.** Variables stored on the Platform environment are used at deploy time. `--env-file <file>` layers a local file on top (overriding stored keys); with several `.env*` files present, headless mode requires an explicit `--env-file`. A missing model-provider key fails at runtime, not at deploy. Before deploying, check which keys the app needs (for example `OPENAI_API_KEY`); if they are not already stored on the environment, ask the user to put them in a gitignored env file in the project (for example `.env.production`) and deploy with `--env-file <file>` (path relative to the project). Deploy saves those variables on the environment, so later deploys reuse them. Never fill that file from your own shell (for example `printf 'KEY=%s' "$KEY" > .env.production`): the value persists on the Platform environment. If a key was deployed that the user did not provide, tell the user so they can remove or rotate it; do not change stored environment credentials yourself. + +**Storage.** Preflight blocks file-backed storage (`file:./mastra.db`) because the Platform filesystem is ephemeral. Guard the path with an environment variable: + +```typescript +import { LibSQLStore } from '@mastra/libsql'; + +export const storage = new LibSQLStore({ + id: 'mastra-storage', + // Hosted database when deployed, local file during development + url: process.env.TURSO_DATABASE_URL ?? 'file:./mastra.db', + authToken: process.env.TURSO_AUTH_TOKEN, +}); +``` + +The deploy-time offer to create a managed database is interactive only; in headless or `--yes` runs the CLI prints the command instead. Provision it explicitly once the environment exists (create the environment first with `mastra env create production --type production --project "$MASTRA_PROJECT_ID"` or the REST route above): + +```bash +mastra env db create production --kind turso --project "$MASTRA_PROJECT_ID" +``` + +The database's connection variables are injected into deploys automatically. + +## Verify + +1. `mastra deploy` exits non-zero on failure and prints the Server URL on success. The CLI waits until the deploy status is `running` (success), `failed`, or `stopped`; any other status is still in progress. +2. For machine-readable status, read the newest entry from `GET /v1/projects/{projectId}/environment-deploys`, or poll `GET /v1/projects/{projectId}/environments/{environmentId}/deploys/{deployId}`. +3. On failure, run `mastra env diagnosis <deploy-id>` for fix suggestions. +4. Call the deployed server with `mastra api --url <server-url> agent list` (see [`mastra-api.md`](mastra-api.md)), then exercise one agent, including a workspace tool if the app uses one. + +## Clean up + +Only delete resources you created in this session, and only with the user's consent. `DELETE /v1/projects/{id}` is a soft delete: `GET /v1/projects/recently-deleted` lists deleted projects and `POST /v1/projects/{id}/undelete` restores one. `mastra env delete <environment>` removes an environment. + +## Failures + +| Symptom | Cause and fix | +| - | - | +| 401 | Missing, expired, or wrong credential type. `MASTRA_PLATFORM_ACCESS_TOKEN` cannot replace `MASTRA_API_TOKEN`. Ask the user for a new token. | +| 403 | Role lacks permission (viewer on project create, non-admin on workspace enable). Ask the user for a token with a sufficient role. | +| Headless deploy asks for `MASTRA_ORG_ID` / `MASTRA_PROJECT_ID` | Both are required with `MASTRA_API_TOKEN`; create the project over REST first. | +| `No organizations found` from `mastra env ...` with `MASTRA_API_TOKEN` | Known CLI issue: org resolution can fail before `MASTRA_ORG_ID` is read. Use the REST route with `x-organization-id` instead. | +| 409 on workspace enable | The project has no environment; deploy or create one first. | +| Generic 500 from `PlatformFilesystem` / `PlatformSandbox` | Upstream storage or sandbox error, not auth. Capture the request ID and report it; do not retry in a loop. | +| Preflight blocks a local storage path | Guard the path with an env var and attach a managed database (see Storage). | +| Deploy succeeds but agent calls fail | Model-provider key missing from the environment. Ask the user to add it to a gitignored env file, then redeploy with `--env-file <file>`. | diff --git a/plugins/mastra/.agents/skills/mastra/references/trace-query.md b/plugins/mastra/.agents/skills/mastra/references/trace-query.md index 66090c45..f58ebfdf 100644 --- a/plugins/mastra/.agents/skills/mastra/references/trace-query.md +++ b/plugins/mastra/.agents/skills/mastra/references/trace-query.md @@ -1,6 +1,6 @@ -# Advanced Trace Query Reference +# Query Traces and Spans -Use `mastra api trace query` when a normal trace list cannot express the required filters: recursive boolean predicates, related span, score, or feedback conditions, metadata filters, ordering, or opaque cursor pagination. +Use `mastra api trace query` when a normal trace list cannot express the required filters: recursive boolean predicates, related span conditions, metadata filters, ordering, or opaque cursor pagination. ## Command contract @@ -18,7 +18,9 @@ After confirming availability, inspect the target server's current contract: npx mastra api trace query --schema ``` -Use `--schema` to confirm that the target supports the route and to inspect the current request and response shape and structural constraints. Predicate paths are generic strings in the schema, so it does not provide the context-specific field/operator matrix. If you need more information, read the [reference docs online](https://mastra.ai/reference/observability/tracing/advanced-queries). Do not infer unsupported predicates from storage columns or older documentation. The server remains the ultimate validation authority. +Use `--schema` to inspect the route's request/response shape and structural constraints. A route in the schema does not prove that the configured store supports every feature. Predicate paths are generic strings, so the schema also does not provide the context-specific field/operator matrix. Use the [canonical query reference](https://mastra.ai/reference/observability/tracing/advanced-queries) for fields, operators, optional capabilities, and limits. Do not infer valid predicates from storage columns. The target server's capabilities and validation determine what is available; a newer local CLI or Core checkout does not prove a deployed Platform server supports the same features. + +Check the target's capabilities using the client SDK's `getObservabilityCapabilities()`: `traceQuery`, `spanQuery`, `traceAggregate`, and `spanAggregate` enable the four operations. These examples require a CLI exposing the relevant command and a server/store advertising its capability. Replace the example timestamps with a recent range containing the user's data. For Platform retention, follow the public [pricing page](https://mastra.ai/pricing); plan retention and API request validation limits are separate. ## Query workflow @@ -45,17 +47,9 @@ npx mastra api trace query \ | jq '{traces: .data.traces, next: .data.page.next}' ``` -Find traces with a low score from one scorer. Conditions inside one `scores.some` clause must match the same score record: - -```bash -npx mastra api trace query \ - '{"timeRange":{"from":"2026-08-01T00:00:00.000Z","to":"2026-08-08T00:00:00.000Z"},"where":{"scores":{"some":{"op":"and","args":[{"op":"eq","left":{"path":"scorerId"},"right":{"literal":"factuality"}},{"op":"lt","left":{"path":"score"},"right":{"literal":0.6}}]}}},"page":{"limit":25}}' \ - | jq '{traces: .data.traces, next: .data.page.next}' -``` - ## Pagination -The query response remains nested under `data` so the pagination cursor is preserved: +Trace queries default to keyset pagination. Their CLI response remains nested under `data` so the pagination cursor is preserved: ```json { @@ -75,21 +69,23 @@ npx mastra api trace query \ A cursor is bound to the accepted query and ordering. Do not change the time range, predicates, or ordering between pages. A mismatched cursor returns `409`; a malformed cursor returns `400`. -A `422` with code `query_too_expensive` means the query hit a resource limit; retrying the same request fails again. Narrow `timeRange` (start with the last 24 hours) or add filters. +Trace queries also support numbered pages (`pagination`) and delta polling (`mode: "delta"`). These return `data.pagination` or `data.deltaCursor`/`data.delta.hasMore`, respectively. Do not combine pagination modes or interchange their cursors. Span queries support keyset pagination only. Follow the [pagination reference](https://mastra.ai/reference/observability/tracing/advanced-queries#pagination-and-errors) when choosing numbered pages or polling. + +For errors, inspect the CLI error details. `422 TRACE_QUERY_INVALID` identifies invalid requests through issue codes and paths. `503 TRACE_QUERY_RESOURCE_LIMIT` or `504 TRACE_QUERY_EXECUTION_TIMEOUT` calls for a narrower time range or more selective filters; increasing the CLI timeout does not raise the database execution budget. See the canonical reference for operation-specific errors. ## Predicate semantics - Compose predicates with `and`, `or`, and `not`. - Use `some` when one related record must satisfy the complete nested predicate. - Use `none` when no related record may satisfy the complete nested predicate. Traces with no related records also match `none`. -- Related span, score, and feedback clauses correlate records by trace. Conditions in separate related clauses do not imply that they refer to one shared related record. +- Related span clauses correlate records by trace. Conditions in separate `spans.some` clauses can match different spans; put conditions in one clause when they must match the same span. - String comparisons are exact and case-sensitive unless the canonical documentation states otherwise. - The root `timeRange` filters the selected root span's start time; it does not independently constrain related-record timestamps. -- Query results are lightweight and do not embed matching spans, scores, or feedback. Fetch trace or span details after selecting candidates. +- Query results are lightweight and do not embed matching spans. Fetch trace or span details after selecting candidates. ## Query spans -Use `mastra api span query` (`POST /api/observability/spans/query`) when the answer is a list of spans rather than traces: every failed tool call, or every run of one agent including runs nested under other agents or workflows. Trace query returns one row per trace and matches spans only through `spans.some`; span query returns one row per completed span, and its `where` filters span fields directly (`spanType`, `name`, `status`, `entityName`, `durationMs`, `error`, …) with `and`/`or`/`not`. It has no `spans.some`, `scores`, or `feedback` clauses. +Use `mastra api span query` (`POST /api/observability/spans/query`) when the answer is a list of spans rather than traces: every failed tool call, or every run of one agent including runs nested under other agents or workflows. Trace query returns one row per trace and tests related spans through `spans.some` or `spans.none`; span query returns one row per completed span, and its `where` filters span fields directly (`spanType`, `name`, `status`, `entityName`, `durationMs`, `error`, …) with `and`/`or`/`not`. Do not wrap span predicates in `spans.some`. ```bash npx mastra api span query --help @@ -99,9 +95,9 @@ npx mastra api span query \ | jq '{spans: [.data.spans[] | {traceId, spanId, entityName, rootEntityName, errorPreview}], next: .data.page.next}' ``` -`page.limit` defaults to 100 (max 1000); `orderBy` takes one of `startedAt`, `endedAt`, or `timeToFirstChunkMs` (default `startedAt desc`). Paginate exactly like `trace query`: pass a non-null `data.page.next` unchanged as `page.after` with the same query. There is no total count. +For ordering, use a one-item array, such as `"orderBy":[{"field":"startedAt","direction":"desc"}]`. Paginate exactly like a keyset trace query: pass a non-null `data.page.next` unchanged as `page.after` with the same query. Span queries do not return a total count. Read the [span query reference](https://mastra.ai/reference/observability/tracing/advanced-queries#query-spans) for supported sort fields and response fields. -Check capabilities before relying on span query features instead of assuming what a store supports: `getObservabilityCapabilities()` reports `spanQuery`, and `spanQueryContextFields` for filters on identity, context, and parent/root entity fields (`traceId`, `threadId`, `resourceId`, `environment`, `rootEntityName`, …). Treat a missing value as unavailable. Unsupported stores return `501 SPAN_QUERY_UNSUPPORTED`; unsupported context-field filters return `422` with a `field_not_allowed` issue. Per-field capability reporting is still evolving, so re-check capabilities rather than hardcoding which store supports which field. +Check `capabilities.spanQuery` from the client SDK's `getObservabilityCapabilities()` before relying on span queries. Optional fields have separate capabilities, including `spanQueryContextFields`, `spanQueryModelFields`, and `spanQueryOutcomeFields`; consult the canonical reference for the exact fields each gates. Treat missing or false capabilities as unavailable. Missing values in a result may mean a field wasn't recorded or isn't supported, not that a run succeeded. ## Aggregate traces @@ -116,18 +112,18 @@ npx mastra api trace aggregate --schema From the client SDK, check `capabilities.traceAggregate` from `getObservabilityCapabilities()` before calling `aggregateTraces()`. A missing value means the server doesn't support aggregation; stores without it return `501 TRACE_AGGREGATE_UNSUPPORTED`. -Request shape: `timeRange` (required, up to 365 days), optional `where`, `measures` (required), `groupBy` (up to two dimensions such as `entityName`, `environment`, `userId`, or `metadata.<key>`), optional `interval` (`1m`, `5m`, `15m`, `1h`, `1d`), `having` over measures, `orderBy` (`{ field, direction }`, default `count desc`), and `limit` (groups, default 100, max 1000). Measures include `count`, `countDistinct.<field>`, `errorCount`, `errorRate`, `duration.{avg,min,max,p50,p90,p95,p99}`, `tokens.{input,output,total,reasoning,cached}.{sum,avg}`, and `cost.{sum,avg}`. +Supply `timeRange` and `measures`; add `where`, `groupBy`, `interval`, `having`, `orderBy`, and `limit` as needed. Aggregate `orderBy` is one object, unlike the one-item array used by trace and span queries. Consult the [aggregate reference](https://mastra.ai/reference/observability/tracing/advanced-queries#aggregate-traces) for available dimensions, measures, and request limits. ```bash npx mastra api trace aggregate \ '{"timeRange":{"from":"2026-08-01T00:00:00.000Z","to":"2026-08-08T00:00:00.000Z"},"where":{"op":"eq","left":{"path":"entityType"},"right":{"literal":"agent"}},"groupBy":["entityName"],"measures":["count","errorRate","duration.p95","cost.sum"],"orderBy":{"field":"errorRate","direction":"desc"},"limit":10}' ``` -Each row has `dimensions`, `measures`, a `bucket` when `interval` is set, and a `cost` object (`coverage`, `unit`) when a cost measure is requested. Report low `cost.coverage` as understated spend; `cost.sum` is `null` when `cost.unit` is `'mixed'`. Aggregation isn't paginated: when `truncated` is `true`, narrow the selection instead of paging. Empty time buckets are omitted, not zero-filled. To see the traces behind a group, rerun `trace query` with the same selection plus the group's dimension values in `where`. +Rows contain `dimensions` and `measures`, plus bucket and cost information when applicable. Report low `cost.coverage` as incomplete spend; don't present a `null` cost as zero. Aggregation isn't paginated: when `truncated` is `true`, narrow the selection instead of paging. Empty time buckets are omitted, not zero-filled. To see the traces behind a group, rerun `trace query` with the same selection plus the group's dimension values in `where`. For a null dimension, use the documented missing-value predicate rather than `eq` with a null literal. ## Aggregate spans -Use `mastra api span aggregate` (`POST /api/observability/spans/aggregate`) when measures should be computed per span rather than per trace. Trace aggregation rolls each trace up and groups by root fields (e.g. the top-level agent); span aggregation counts any span, including nested agent runs and tool calls, grouped by span fields: `name`, `spanType`, `status`, `entityType`/`entityId`/`entityName`, `parentEntity*`/`rootEntity*` type and name, `environment`, `serviceName`, `executionSource`, `threadId`, `resourceId`, `userId`, `sessionId`, `organizationId`, `experimentId`. It selects exactly the spans `span query` would list for the same `timeRange` and `where`, and takes the same measures, `interval`, `having`, `orderBy`, and `limit` as `trace aggregate`. +Use `mastra api span aggregate` (`POST /api/observability/spans/aggregate`) when measures should be computed per span rather than per trace. Trace aggregation rolls up each trace; span aggregation measures completed spans, including nested agent runs and tool calls. It selects the same spans as `span query` for the same `timeRange` and `where`. Its request structure follows trace aggregation, but predicate and dimension fields follow the [span aggregation contract](https://mastra.ai/reference/observability/tracing/advanced-queries#aggregate-spans). ```bash npx mastra api span aggregate --help @@ -136,13 +132,12 @@ npx mastra api span aggregate \ '{"timeRange":{"from":"2026-08-01T00:00:00.000Z","to":"2026-08-08T00:00:00.000Z"},"where":{"op":"eq","left":{"path":"spanType"},"right":{"literal":"tool_call"}},"groupBy":["entityName"],"measures":["count","errorRate","duration.p95"],"orderBy":{"field":"errorRate","direction":"desc"},"limit":10}' ``` -Check `capabilities.spanAggregate` first; treat a missing value as unavailable. Unsupported servers return `501` (on Mastra platform, `404` while the feature is off). Rows and `truncated` match `trace aggregate`. Limits: +Check `capabilities.spanAggregate` first; treat a missing or false value as unavailable. Unsupported stores return `501 SPAN_AGGREGATE_UNSUPPORTED`; a server without the route returns `404`. Like trace aggregation, results are not paginated: check `truncated` and narrow the selection when needed. -- Tokens and cost belong to the span that recorded them (usually `model_generation`) and are never rolled up to ancestors, so an `agent_run` group normally shows no usage. For spend per agent, filter `spanType` to `model_generation` and group by `rootEntityName`, or use `trace aggregate`. -- No `model` or `provider` dimension yet. -- On Mastra platform, organization-wide requests (no project) over 14 days return `422` with a `time_range_too_large` issue unless `where` has a top-level `spanType` `eq`/`in` limited to branch span types: `agent_run`, `workflow_run`, `tool_call`, `mcp_tool_call`, `mcp_server_request`, `processor_run`, `scorer_run`, `rag_ingestion`, `provider_tool_call`. Narrow the window, add that filter, or aggregate one project. +- Tokens and cost belong to the span that recorded them and are not rolled up to ancestors. Use trace aggregation for totals per trace; for span aggregation, select the spans recording usage before grouping by agent identity. +- Grouping or counting distinct values by `model` or `provider` requires `spanAggregateModelDimensions`. These dimensions read dedicated columns, not older attributes; older spans can group under `null`. Check the capability instead of assuming support from the CLI version. -## Choosing the trace command +## Choosing the command - `trace list`: browse recent traces with simple list filters. - `trace query`: select completed traces using complex, explicit predicates. @@ -152,4 +147,4 @@ Check `capabilities.spanAggregate` first; treat a missing value as unavailable. - `trace get <traceId>`: inspect one selected trace. - `trace span <traceId> <spanId>`: fetch one span in full. -Advanced queries only return completed traces. The configured observability store must support the query API. If the server rejects a query that follows the current structural schema and canonical documentation as unsupported, report that limitation rather than falling back to an inaccurate client-side approximation. +These query and aggregate APIs return completed traces or spans only. The configured observability store must support the query API. If the server rejects a query that follows the current structural schema and canonical documentation as unsupported, report that limitation rather than falling back to an inaccurate client-side approximation. diff --git a/plugins/mastra/agent/skills/mastra/SKILL.md b/plugins/mastra/agent/skills/mastra/SKILL.md index 2541efb8..ff47c4cb 100644 --- a/plugins/mastra/agent/skills/mastra/SKILL.md +++ b/plugins/mastra/agent/skills/mastra/SKILL.md @@ -4,15 +4,16 @@ description: "Comprehensive Mastra framework guide for building agents, workflows, tools, memory, workspaces, and storage with current APIs. Use for native capability discovery, logging, tracing, metrics/cost estimates, dataset experiments, documentation lookup, API verification, TypeScript setup, common - errors, migrations, and `mastra api` CLI tasks: inspect or call resources on - local, Mastra platform, or remote servers. For Mastra Factory operations, - discover and activate the companion mastra-factory skill. Also report verified - bugs and concrete documentation defects to the Mastra team automatically via - the feedback endpoint." + errors, migrations, building and deploying apps to Mastra Platform via + Platform APIs and `mastra deploy`, and `mastra api` CLI tasks: inspect or call + resources on local, Mastra platform, or remote servers. For Mastra Factory + operations, discover and activate the companion mastra-factory skill. Also + report verified bugs and concrete documentation defects to the Mastra team + automatically via the feedback endpoint." license: Apache-2.0 metadata: author: Mastra - version: 2.4.0 + version: 2.5.0 repository: https://github.com/mastra-ai/skills --- # Mastra Framework Guide @@ -53,7 +54,8 @@ ls node_modules/@mastra/ | I'm getting an error... | [`references/common-errors.md`](references/common-errors.md) | Common errors and solutions | | Upgrade from v0.x to v1.x | [`references/migration-guide.md`](references/migration-guide.md) | Version upgrade workflows | | Inspect/call server resources via CLI | [`references/mastra-api.md`](references/mastra-api.md) | `mastra api` CLI for local, Mastra platform, or remote servers | -| Find exact traces with complex predicates | [`references/trace-query.md`](references/trace-query.md) | Query completed traces by trace fields or related spans, scores, and feedback | +| Deploy to Mastra Platform, provision projects, or use Platform sandbox/filesystem | [`references/platform.md`](references/platform.md) | Platform REST APIs, credentials, and `mastra deploy` without browser automation | +| Find exact traces with complex predicates | [`references/trace-query.md`](references/trace-query.md) | Query completed traces by trace fields or related spans | | Count traces or compute grouped measures (error rate, latency, tokens, cost) | [`references/trace-query.md`](references/trace-query.md#aggregate-traces) | Aggregate completed traces with `mastra api trace aggregate` over the same selection | | List individual spans (failed tool calls, nested agent runs) | [`references/trace-query.md`](references/trace-query.md#query-spans) | Query completed spans with `mastra api span query` and span-field predicates | | Compute measures per span (tool error rates, nested agent runs, model tokens) | [`references/trace-query.md`](references/trace-query.md#aggregate-spans) | Aggregate spans with `mastra api span aggregate`, grouped by span fields | @@ -102,7 +104,11 @@ Then open `http://localhost:4111` in a browser to show Mastra Studio to your hum Use `mastra api` to inspect or call resources on local dev servers, Mastra platform deployments, or remote Mastra endpoints. It is useful for agent-readable state, execution, traces, logs, scores, threads, and workflow operations. See [`references/mastra-api.md`](references/mastra-api.md) for usage patterns. -For exact trace selection that needs recursive predicates or conditions over related spans, scores, or feedback, read [`references/trace-query.md`](references/trace-query.md). Before using `mastra api trace query`, confirm that the installed CLI exposes the command. Use `--schema` for the target's request/response shape and structural constraints, and use the canonical documentation found through [`references/remote-docs.md`](references/remote-docs.md) for supported fields, operators, and semantics. Preserve opaque pagination cursors and fetch trace or span details only after selecting candidates. +For exact trace selection that needs recursive predicates or conditions over related spans, read [`references/trace-query.md`](references/trace-query.md). Before using `mastra api trace query`, confirm that the installed CLI exposes the command. Use `--schema` for the target's request/response shape and structural constraints, and use the canonical documentation found through [`references/remote-docs.md`](references/remote-docs.md) for supported fields, operators, and semantics. Preserve opaque pagination cursors and fetch trace or span details only after selecting candidates. + +## Deploy to Mastra Platform + +When the user asks to deploy to Mastra, whether to build a new app and deploy it or to deploy an existing Mastra project, read [`references/platform.md`](references/platform.md) before running any deploy or Platform API command. It covers which steps need the human (sign-up and sign-in only), credentials, REST discovery and project creation, `PlatformFilesystem` and `PlatformSandbox`, `mastra deploy`, and verification. Build or locate the app first with the references above, then follow `platform.md` to deploy. ## Mastra Factory diff --git a/plugins/mastra/agent/skills/mastra/references/create-mastra.md b/plugins/mastra/agent/skills/mastra/references/create-mastra.md index a09d8d3b..d3cfbd06 100644 --- a/plugins/mastra/agent/skills/mastra/references/create-mastra.md +++ b/plugins/mastra/agent/skills/mastra/references/create-mastra.md @@ -44,12 +44,6 @@ npm create mastra@latest my-project --empty `--empty` and `--template` are separate creation modes. The `--llm` (`-l`) and `--llm-api-key` options apply only to the default starter. -**Skip the example agent:** - -```bash -npm create mastra@latest --no-example -``` - **Use a specific template:** ```bash @@ -228,3 +222,5 @@ After creating your project with `create mastra`: - [Agents](https://mastra.ai/docs/agents/overview) - [Workflows](https://mastra.ai/docs/workflows/overview) - [GitHub](https://github.com/mastra-ai/mastra) + +To deploy the project to Mastra Platform, see [`platform.md`](platform.md). diff --git a/plugins/mastra/agent/skills/mastra/references/mastra-api.md b/plugins/mastra/agent/skills/mastra/references/mastra-api.md index 2fcc70ad..a54c6374 100644 --- a/plugins/mastra/agent/skills/mastra/references/mastra-api.md +++ b/plugins/mastra/agent/skills/mastra/references/mastra-api.md @@ -4,6 +4,8 @@ How to use the `mastra api` CLI to interact with Mastra servers. Prefer fast, fo Use this reference when the user asks to inspect or call agents, workflows, tools, MCP servers, memory threads, traces, logs, metrics, scores, datasets, experiments, or to debug/test `mastra api` commands. +This reference does not cover the Mastra Platform control plane (projects, environments, deploys, tokens, workspace provisioning); for those, see [`platform.md`](platform.md). + ## Setup The CLI can interact with any reachable Mastra server: @@ -11,7 +13,7 @@ The CLI can interact with any reachable Mastra server: - Local dev server: `http://localhost:4111` from `npm run dev` - Mastra platform deployment: Use the deployment URL - Remote/self-hosted server: Use the server URL -- Hosted Mastra Platform Observability: `https://observability.mastra.ai` (auto-targeted by `trace`, `log`, `score`, and `metric` commands) +- Hosted Mastra Platform Observability: `https://observability.mastra.ai` (auto-targeted by `trace`, `span`, `log`, `score`, and `metric` commands) For local servers, `mastra api` defaults to `http://localhost:4111`: @@ -50,7 +52,7 @@ Runtime commands (`agent`, `workflow`, `tool`, `mcp`, `thread`, `memory`, `datas 2. `http://localhost:4111` for a local `mastra dev` server. 3. `.mastra-project.json` for a Mastra platform project. -Observability commands (`trace`, `log`, `score`, `metric`) target `https://observability.mastra.ai` by default instead of a project deployment URL. EU projects use `https://observability.eu.mastra.ai`: the CLI picks it automatically from `mastra@1.34.2`. On older versions, EU requests time out with `REQUEST_TIMEOUT`; pass `--url https://observability.eu.mastra.ai`. The CLI resolves credentials in this order: +Observability commands (`trace`, `span`, `log`, `score`, `metric`) target `https://observability.mastra.ai` by default instead of a project deployment URL. EU projects use `https://observability.eu.mastra.ai`: the CLI picks it automatically from `mastra@1.34.2`. On older versions, EU requests time out with `REQUEST_TIMEOUT`; pass `--url https://observability.eu.mastra.ai`. The CLI resolves credentials in this order: 1. Explicit `Authorization` and `X-Mastra-Project-Id` headers passed with `--header`. 2. `MASTRA_PLATFORM_ACCESS_TOKEN` and `MASTRA_PROJECT_ID` from the environment. @@ -76,7 +78,7 @@ Pass `--url` and `--header` only when overriding the hosted observability target Start with these command groups when present; verify with `mastra api --help` if the group fails. ```text -agent workflow tool mcp thread memory trace log metric score dataset experiment +agent workflow tool mcp thread memory trace span log metric score dataset experiment ``` ## Fast path for read-only requests @@ -146,7 +148,7 @@ npx mastra api span aggregate --help npx mastra api span aggregate --schema ``` -Use `trace query` instead of `trace list` when selection requires recursive predicates, metadata filters, or conditions over related spans, scores, or feedback. First use `trace query --help` to confirm that the installed CLI exposes the command. The inline JSON query is required, and its cursor-bearing response stays nested under `data`. Read [`trace-query.md`](trace-query.md) for availability checks, the division between CLI schema discovery and canonical documentation, query construction, and pagination. +Use `trace query` instead of `trace list` when selection requires recursive predicates, metadata filters, or conditions over related spans. First use `trace query --help` to confirm that the installed CLI exposes the command. The inline JSON query is required, and its cursor-bearing response stays nested under `data`. Read [`trace-query.md`](trace-query.md) for availability checks, the division between CLI schema discovery and canonical documentation, query construction, and pagination. Use `trace aggregate` when the answer is a count or measure (error rate, latency percentiles, tokens, cost) grouped by up to two dimensions or time buckets rather than a list of traces. It takes the same `timeRange` and `where` as `trace query`. See [`trace-query.md`](trace-query.md#aggregate-traces). @@ -215,6 +217,6 @@ curl -fsS "$MASTRA_URL/api/system/api-schema" \ - Working memory update requires the agent's memory to have working memory enabled. - Empty lists may simply mean the server has no matching stored data yet. - `trace list` and `trace get` return lightweight payloads by default (no span input, output, attributes, or metadata). Pass `--verbose` to fetch full span records, or use `trace span <traceId> <spanId>` to fetch one specific span in full. -- `trace query` requires inline JSON, queries completed traces, and preserves its opaque cursor at `data.page.next`. Pass that value unchanged as `page.after` with the same query shape. +- `trace query` requires inline JSON, queries completed traces, and defaults to keyset pagination: pass `data.page.next` unchanged as `page.after` with the same query shape. Numbered pages and delta polling use different response metadata; see [`trace-query.md`](trace-query.md#pagination). - `trace aggregate` requires inline JSON and isn't paginated; when `truncated` is `true`, narrow the selection. - `span query` requires inline JSON and pages like `trace query`: pass `data.page.next` unchanged as `page.after`. `span aggregate` isn't paginated, like `trace aggregate`. diff --git a/plugins/mastra/agent/skills/mastra/references/platform.md b/plugins/mastra/agent/skills/mastra/references/platform.md new file mode 100644 index 00000000..758cf5cd --- /dev/null +++ b/plugins/mastra/agent/skills/mastra/references/platform.md @@ -0,0 +1,164 @@ +# Mastra Platform + +## When to use + +Use this reference to create Mastra Platform projects, deploy a Mastra app (new or existing), and use Platform workspaces (`PlatformFilesystem`, `PlatformSandbox`) in generated code. For calls to a running Mastra server (agents, workflows, traces), use [`mastra-api.md`](mastra-api.md); it does not cover the Platform control plane. + +The Platform API changes often. Treat the live spec `https://platform.mastra.ai/openapi.json` and `mastra <command> --help` as the source of truth, and check them before using anything not listed here. + +## Human steps vs agent steps + +The browser is only for the human: signing up, creating an organization, and `mastra auth login`. Never automate the browser or scrape the dashboard. + +Everything else is agent work: scaffolding, REST discovery and provisioning, `mastra deploy`, and verification. When credentials are missing, stop and ask the user to do one of: + +- Run `mastra auth login` in their own terminal (browser sign-in). The CLI saves the session, and later CLI commands reuse it without a TTY. +- Create an API token themselves (`mastra auth tokens create <name>` in their own terminal, or the dashboard) and put it, with `MASTRA_ORG_ID`, in the environment or an ignored env file. + +Do not run `mastra auth tokens create` yourself: it prints the secret to stdout. Never ask for secret values in chat, never print or log them, and never commit them. + +If you need anything you were not given — a credential, a model-provider key, which org or project to use — stop and ask the user. Do not take secrets from your own shell environment, other projects, or files the user did not point you to. + +## Credentials + +Two credential models exist. Do not mix them. + +| Variable | Used by | Notes | +| - | - | - | +| `MASTRA_API_TOKEN` | CLI and control-plane REST | Wins over saved `mastra auth login` credentials. Setting it puts `mastra deploy` in headless mode. | +| `MASTRA_ORG_ID` | CLI and control-plane REST | Sent as `x-organization-id`. Required with `MASTRA_API_TOKEN`. | +| `MASTRA_PROJECT_ID` | CLI headless deploy | Must be a project ID (not a slug or name) in headless mode. | +| `MASTRA_PLATFORM_API_URL` | CLI | Optional override. Default `https://platform.mastra.ai`. | +| `MASTRA_PLATFORM_ACCESS_TOKEN` | App runtime (workspaces, observability) | Not a deploy credential. Injected on deploy; locally, an org API token the user creates. | +| `MASTRA_ENVIRONMENT_ID`, `MASTRA_PLATFORM_BUCKET_NAME` | App runtime (workspaces) | Injected on deploy; set manually only for local runs. | + +Check for a saved session with `mastra auth whoami` before asking for credentials. With a saved session, CLI commands (including `mastra deploy`) work without a TTY and use the current org (`mastra auth orgs`; override with `--org <id>`). Only signing in, or signing in again after the session expires, needs a TTY. Do not read the saved credentials file; in session mode use CLI commands instead of the REST calls below, which need `MASTRA_API_TOKEN`. + +Keep control-plane credentials (`MASTRA_API_TOKEN`, `MASTRA_ORG_ID`) in the shell environment, not in the app's `.env`: deploy uploads app env files as-is and does not filter them. + +## Discover and provision over REST + +`mastra deploy` and `mastra auth` have no JSON output, so use REST when you need machine-readable state. Every control-plane call uses the same headers: + +```bash +curl -fsS https://platform.mastra.ai/v1/auth/orgs \ + -H "Authorization: Bearer $MASTRA_API_TOKEN" \ + -H "x-organization-id: $MASTRA_ORG_ID" +``` + +| Route | Purpose | +| - | - | +| `GET /v1/auth/orgs` | `{ organizations: [{ id, slug, name, role, isCurrent }] }`. Pick `MASTRA_ORG_ID` here (or `mastra auth whoami`). Org API keys (`sk_...`) get an empty list; ask the user for the org ID. | +| `GET /v1/projects` | List projects in the org. | +| `POST /v1/studio/projects` | Create a project. Body `{"name": "..."}` (1–64 chars); response `{ project: { id, slug, workspacesEnabled, ... } }`. Viewer/read-only roles get 403. | +| `GET /v1/projects/{id}` | Project details, including `workspacesEnabled`. | +| `GET /v1/projects/{projectId}/environments` | `{ environments: [{ id, name, slug, type, region, instanceUrl, customServerUrl, ... }] }`. Not in the public spec; called by the CLI. | +| `POST /v1/projects/{projectId}/environments` | Create an environment. Body `{"name": "production", "type": "production", "region": "us"}` (`region` optional). Not in the public spec; called by the CLI. | +| `GET /v1/projects/{projectId}/environment-deploys` | `{ deploys: [{ id, environmentName, status, error, errorCode, createdAt, ... }] }`. Not in the public spec; called by the CLI. | + +`/v1/projects` itself is list-only; project creation is under `/v1/studio/projects`. The spec also lists `POST /v1/auth/orgs` (create organization), but creating or choosing an organization is the user's decision: ask the user, do not create one. + +Before calling a route not listed here, check it exists with the method you need: + +```bash +curl -fsS https://platform.mastra.ai/openapi.json | jq '.paths["/v1/studio/projects"] | keys' +``` + +## Build with Platform workspaces + +Use `@mastra/platform-workspace` when the app needs durable file storage or remote command execution on the Platform. New projects have workspaces enabled and get one bucket per environment when the environment is created. + +```bash +npm install @mastra/platform-workspace +``` + +```typescript title="src/mastra/workspace.ts" +import { Workspace } from '@mastra/core/workspace'; +import { PlatformFilesystem, PlatformSandbox } from '@mastra/platform-workspace'; + +// Reads MASTRA_PLATFORM_ACCESS_TOKEN, MASTRA_PROJECT_ID, MASTRA_ENVIRONMENT_ID, +// and MASTRA_PLATFORM_BUCKET_NAME from the environment. The Platform injects them on deploy. +export const workspace = new Workspace({ + filesystem: new PlatformFilesystem(), + sandbox: new PlatformSandbox(), +}); +``` + +```typescript title="src/mastra/index.ts" +import { Mastra } from '@mastra/core'; +import { workspace } from './workspace'; + +export const mastra = new Mastra({ + // TODO: register your agents, workflows, and storage + workspace, +}); +``` + +The bucket is shared by every request in the environment. A statically configured `PlatformSandbox` is one shared sandbox; construct a new `PlatformSandbox()` (call `start()`, then `destroy()`) for per-task isolation. The sandbox does not mount the bucket. Check the installed package's types or embedded docs for sandbox options before using them. + +Local runs need all four runtime variables in the app's `.env`. Get `MASTRA_ENVIRONMENT_ID` from the environments route and the bucket name from `GET /v1/server/projects/{id}/environments/{envId}/bucket` (`envId` is the environment ID; response includes `status`: `provisioning`, `ready`, `failed`, or `destroyed`). The user creates the local `MASTRA_PLATFORM_ACCESS_TOKEN` (an org API token from the dashboard). For an offline loop, swap in `LocalFilesystem` and `LocalSandbox`. + +Projects created before workspaces were on by default may have `workspacesEnabled: false`. Enable with `POST /v1/server/projects/{id}/workspaces/enable`: it requires an admin role (an org API key may not qualify), returns 409 when the project has no environment yet, is idempotent, and cannot be undone. Ask the user before calling it. + +## Deploy + +`mastra deploy` builds the app, runs preflight checks, uploads `.mastra/output`, and follows the deploy until it is running or fails. The first deploy creates the environment (`--env`, default `production`) and writes `.mastra-project.json` (`projectId`, `projectName`, `projectSlug`, `organizationId`, optional `disablePlatformObservability`). It holds no secrets and links the directory to the project for later deploys. + +| Situation | Command | +| - | - | +| Saved `mastra auth login` session (no `MASTRA_API_TOKEN` set) | `mastra deploy --project <name> --yes` (creates the project in the current org if not found) | +| Headless, token in env | Export `MASTRA_API_TOKEN`, `MASTRA_ORG_ID`, `MASTRA_PROJECT_ID`, then `mastra deploy --yes` | +| Directory already linked | `mastra deploy --yes` (reads `.mastra-project.json`) | + +`--project <value>` deploys into an existing project whose ID, slug, or name matches, replacing whatever is running there; it creates a project only when nothing matches. When the directory is not linked yet, check for an existing project with that name first (`GET /v1/projects` with a token; otherwise ask the user). If one exists and you did not create it in this session, ask the user whether to deploy into it or pick a different name. + +In headless mode the project must already exist and `MASTRA_PROJECT_ID` must be its ID. For a new app, create it first with `POST /v1/studio/projects`, read `project.id`, and export it. Headless mode auto-accepts prompts. + +Other useful flags: `--env <name>`, `--region us|eu` (new environments only), `--env-file <file>`, `--workers dedicated|in-process`, `--skip-build`. Avoid `--skip-preflight`; it skips the checks that prevent broken deploys. + +**App environment variables.** Variables stored on the Platform environment are used at deploy time. `--env-file <file>` layers a local file on top (overriding stored keys); with several `.env*` files present, headless mode requires an explicit `--env-file`. A missing model-provider key fails at runtime, not at deploy. Before deploying, check which keys the app needs (for example `OPENAI_API_KEY`); if they are not already stored on the environment, ask the user to put them in a gitignored env file in the project (for example `.env.production`) and deploy with `--env-file <file>` (path relative to the project). Deploy saves those variables on the environment, so later deploys reuse them. Never fill that file from your own shell (for example `printf 'KEY=%s' "$KEY" > .env.production`): the value persists on the Platform environment. If a key was deployed that the user did not provide, tell the user so they can remove or rotate it; do not change stored environment credentials yourself. + +**Storage.** Preflight blocks file-backed storage (`file:./mastra.db`) because the Platform filesystem is ephemeral. Guard the path with an environment variable: + +```typescript +import { LibSQLStore } from '@mastra/libsql'; + +export const storage = new LibSQLStore({ + id: 'mastra-storage', + // Hosted database when deployed, local file during development + url: process.env.TURSO_DATABASE_URL ?? 'file:./mastra.db', + authToken: process.env.TURSO_AUTH_TOKEN, +}); +``` + +The deploy-time offer to create a managed database is interactive only; in headless or `--yes` runs the CLI prints the command instead. Provision it explicitly once the environment exists (create the environment first with `mastra env create production --type production --project "$MASTRA_PROJECT_ID"` or the REST route above): + +```bash +mastra env db create production --kind turso --project "$MASTRA_PROJECT_ID" +``` + +The database's connection variables are injected into deploys automatically. + +## Verify + +1. `mastra deploy` exits non-zero on failure and prints the Server URL on success. The CLI waits until the deploy status is `running` (success), `failed`, or `stopped`; any other status is still in progress. +2. For machine-readable status, read the newest entry from `GET /v1/projects/{projectId}/environment-deploys`, or poll `GET /v1/projects/{projectId}/environments/{environmentId}/deploys/{deployId}`. +3. On failure, run `mastra env diagnosis <deploy-id>` for fix suggestions. +4. Call the deployed server with `mastra api --url <server-url> agent list` (see [`mastra-api.md`](mastra-api.md)), then exercise one agent, including a workspace tool if the app uses one. + +## Clean up + +Only delete resources you created in this session, and only with the user's consent. `DELETE /v1/projects/{id}` is a soft delete: `GET /v1/projects/recently-deleted` lists deleted projects and `POST /v1/projects/{id}/undelete` restores one. `mastra env delete <environment>` removes an environment. + +## Failures + +| Symptom | Cause and fix | +| - | - | +| 401 | Missing, expired, or wrong credential type. `MASTRA_PLATFORM_ACCESS_TOKEN` cannot replace `MASTRA_API_TOKEN`. Ask the user for a new token. | +| 403 | Role lacks permission (viewer on project create, non-admin on workspace enable). Ask the user for a token with a sufficient role. | +| Headless deploy asks for `MASTRA_ORG_ID` / `MASTRA_PROJECT_ID` | Both are required with `MASTRA_API_TOKEN`; create the project over REST first. | +| `No organizations found` from `mastra env ...` with `MASTRA_API_TOKEN` | Known CLI issue: org resolution can fail before `MASTRA_ORG_ID` is read. Use the REST route with `x-organization-id` instead. | +| 409 on workspace enable | The project has no environment; deploy or create one first. | +| Generic 500 from `PlatformFilesystem` / `PlatformSandbox` | Upstream storage or sandbox error, not auth. Capture the request ID and report it; do not retry in a loop. | +| Preflight blocks a local storage path | Guard the path with an env var and attach a managed database (see Storage). | +| Deploy succeeds but agent calls fail | Model-provider key missing from the environment. Ask the user to add it to a gitignored env file, then redeploy with `--env-file <file>`. | diff --git a/plugins/mastra/agent/skills/mastra/references/trace-query.md b/plugins/mastra/agent/skills/mastra/references/trace-query.md index 66090c45..f58ebfdf 100644 --- a/plugins/mastra/agent/skills/mastra/references/trace-query.md +++ b/plugins/mastra/agent/skills/mastra/references/trace-query.md @@ -1,6 +1,6 @@ -# Advanced Trace Query Reference +# Query Traces and Spans -Use `mastra api trace query` when a normal trace list cannot express the required filters: recursive boolean predicates, related span, score, or feedback conditions, metadata filters, ordering, or opaque cursor pagination. +Use `mastra api trace query` when a normal trace list cannot express the required filters: recursive boolean predicates, related span conditions, metadata filters, ordering, or opaque cursor pagination. ## Command contract @@ -18,7 +18,9 @@ After confirming availability, inspect the target server's current contract: npx mastra api trace query --schema ``` -Use `--schema` to confirm that the target supports the route and to inspect the current request and response shape and structural constraints. Predicate paths are generic strings in the schema, so it does not provide the context-specific field/operator matrix. If you need more information, read the [reference docs online](https://mastra.ai/reference/observability/tracing/advanced-queries). Do not infer unsupported predicates from storage columns or older documentation. The server remains the ultimate validation authority. +Use `--schema` to inspect the route's request/response shape and structural constraints. A route in the schema does not prove that the configured store supports every feature. Predicate paths are generic strings, so the schema also does not provide the context-specific field/operator matrix. Use the [canonical query reference](https://mastra.ai/reference/observability/tracing/advanced-queries) for fields, operators, optional capabilities, and limits. Do not infer valid predicates from storage columns. The target server's capabilities and validation determine what is available; a newer local CLI or Core checkout does not prove a deployed Platform server supports the same features. + +Check the target's capabilities using the client SDK's `getObservabilityCapabilities()`: `traceQuery`, `spanQuery`, `traceAggregate`, and `spanAggregate` enable the four operations. These examples require a CLI exposing the relevant command and a server/store advertising its capability. Replace the example timestamps with a recent range containing the user's data. For Platform retention, follow the public [pricing page](https://mastra.ai/pricing); plan retention and API request validation limits are separate. ## Query workflow @@ -45,17 +47,9 @@ npx mastra api trace query \ | jq '{traces: .data.traces, next: .data.page.next}' ``` -Find traces with a low score from one scorer. Conditions inside one `scores.some` clause must match the same score record: - -```bash -npx mastra api trace query \ - '{"timeRange":{"from":"2026-08-01T00:00:00.000Z","to":"2026-08-08T00:00:00.000Z"},"where":{"scores":{"some":{"op":"and","args":[{"op":"eq","left":{"path":"scorerId"},"right":{"literal":"factuality"}},{"op":"lt","left":{"path":"score"},"right":{"literal":0.6}}]}}},"page":{"limit":25}}' \ - | jq '{traces: .data.traces, next: .data.page.next}' -``` - ## Pagination -The query response remains nested under `data` so the pagination cursor is preserved: +Trace queries default to keyset pagination. Their CLI response remains nested under `data` so the pagination cursor is preserved: ```json { @@ -75,21 +69,23 @@ npx mastra api trace query \ A cursor is bound to the accepted query and ordering. Do not change the time range, predicates, or ordering between pages. A mismatched cursor returns `409`; a malformed cursor returns `400`. -A `422` with code `query_too_expensive` means the query hit a resource limit; retrying the same request fails again. Narrow `timeRange` (start with the last 24 hours) or add filters. +Trace queries also support numbered pages (`pagination`) and delta polling (`mode: "delta"`). These return `data.pagination` or `data.deltaCursor`/`data.delta.hasMore`, respectively. Do not combine pagination modes or interchange their cursors. Span queries support keyset pagination only. Follow the [pagination reference](https://mastra.ai/reference/observability/tracing/advanced-queries#pagination-and-errors) when choosing numbered pages or polling. + +For errors, inspect the CLI error details. `422 TRACE_QUERY_INVALID` identifies invalid requests through issue codes and paths. `503 TRACE_QUERY_RESOURCE_LIMIT` or `504 TRACE_QUERY_EXECUTION_TIMEOUT` calls for a narrower time range or more selective filters; increasing the CLI timeout does not raise the database execution budget. See the canonical reference for operation-specific errors. ## Predicate semantics - Compose predicates with `and`, `or`, and `not`. - Use `some` when one related record must satisfy the complete nested predicate. - Use `none` when no related record may satisfy the complete nested predicate. Traces with no related records also match `none`. -- Related span, score, and feedback clauses correlate records by trace. Conditions in separate related clauses do not imply that they refer to one shared related record. +- Related span clauses correlate records by trace. Conditions in separate `spans.some` clauses can match different spans; put conditions in one clause when they must match the same span. - String comparisons are exact and case-sensitive unless the canonical documentation states otherwise. - The root `timeRange` filters the selected root span's start time; it does not independently constrain related-record timestamps. -- Query results are lightweight and do not embed matching spans, scores, or feedback. Fetch trace or span details after selecting candidates. +- Query results are lightweight and do not embed matching spans. Fetch trace or span details after selecting candidates. ## Query spans -Use `mastra api span query` (`POST /api/observability/spans/query`) when the answer is a list of spans rather than traces: every failed tool call, or every run of one agent including runs nested under other agents or workflows. Trace query returns one row per trace and matches spans only through `spans.some`; span query returns one row per completed span, and its `where` filters span fields directly (`spanType`, `name`, `status`, `entityName`, `durationMs`, `error`, …) with `and`/`or`/`not`. It has no `spans.some`, `scores`, or `feedback` clauses. +Use `mastra api span query` (`POST /api/observability/spans/query`) when the answer is a list of spans rather than traces: every failed tool call, or every run of one agent including runs nested under other agents or workflows. Trace query returns one row per trace and tests related spans through `spans.some` or `spans.none`; span query returns one row per completed span, and its `where` filters span fields directly (`spanType`, `name`, `status`, `entityName`, `durationMs`, `error`, …) with `and`/`or`/`not`. Do not wrap span predicates in `spans.some`. ```bash npx mastra api span query --help @@ -99,9 +95,9 @@ npx mastra api span query \ | jq '{spans: [.data.spans[] | {traceId, spanId, entityName, rootEntityName, errorPreview}], next: .data.page.next}' ``` -`page.limit` defaults to 100 (max 1000); `orderBy` takes one of `startedAt`, `endedAt`, or `timeToFirstChunkMs` (default `startedAt desc`). Paginate exactly like `trace query`: pass a non-null `data.page.next` unchanged as `page.after` with the same query. There is no total count. +For ordering, use a one-item array, such as `"orderBy":[{"field":"startedAt","direction":"desc"}]`. Paginate exactly like a keyset trace query: pass a non-null `data.page.next` unchanged as `page.after` with the same query. Span queries do not return a total count. Read the [span query reference](https://mastra.ai/reference/observability/tracing/advanced-queries#query-spans) for supported sort fields and response fields. -Check capabilities before relying on span query features instead of assuming what a store supports: `getObservabilityCapabilities()` reports `spanQuery`, and `spanQueryContextFields` for filters on identity, context, and parent/root entity fields (`traceId`, `threadId`, `resourceId`, `environment`, `rootEntityName`, …). Treat a missing value as unavailable. Unsupported stores return `501 SPAN_QUERY_UNSUPPORTED`; unsupported context-field filters return `422` with a `field_not_allowed` issue. Per-field capability reporting is still evolving, so re-check capabilities rather than hardcoding which store supports which field. +Check `capabilities.spanQuery` from the client SDK's `getObservabilityCapabilities()` before relying on span queries. Optional fields have separate capabilities, including `spanQueryContextFields`, `spanQueryModelFields`, and `spanQueryOutcomeFields`; consult the canonical reference for the exact fields each gates. Treat missing or false capabilities as unavailable. Missing values in a result may mean a field wasn't recorded or isn't supported, not that a run succeeded. ## Aggregate traces @@ -116,18 +112,18 @@ npx mastra api trace aggregate --schema From the client SDK, check `capabilities.traceAggregate` from `getObservabilityCapabilities()` before calling `aggregateTraces()`. A missing value means the server doesn't support aggregation; stores without it return `501 TRACE_AGGREGATE_UNSUPPORTED`. -Request shape: `timeRange` (required, up to 365 days), optional `where`, `measures` (required), `groupBy` (up to two dimensions such as `entityName`, `environment`, `userId`, or `metadata.<key>`), optional `interval` (`1m`, `5m`, `15m`, `1h`, `1d`), `having` over measures, `orderBy` (`{ field, direction }`, default `count desc`), and `limit` (groups, default 100, max 1000). Measures include `count`, `countDistinct.<field>`, `errorCount`, `errorRate`, `duration.{avg,min,max,p50,p90,p95,p99}`, `tokens.{input,output,total,reasoning,cached}.{sum,avg}`, and `cost.{sum,avg}`. +Supply `timeRange` and `measures`; add `where`, `groupBy`, `interval`, `having`, `orderBy`, and `limit` as needed. Aggregate `orderBy` is one object, unlike the one-item array used by trace and span queries. Consult the [aggregate reference](https://mastra.ai/reference/observability/tracing/advanced-queries#aggregate-traces) for available dimensions, measures, and request limits. ```bash npx mastra api trace aggregate \ '{"timeRange":{"from":"2026-08-01T00:00:00.000Z","to":"2026-08-08T00:00:00.000Z"},"where":{"op":"eq","left":{"path":"entityType"},"right":{"literal":"agent"}},"groupBy":["entityName"],"measures":["count","errorRate","duration.p95","cost.sum"],"orderBy":{"field":"errorRate","direction":"desc"},"limit":10}' ``` -Each row has `dimensions`, `measures`, a `bucket` when `interval` is set, and a `cost` object (`coverage`, `unit`) when a cost measure is requested. Report low `cost.coverage` as understated spend; `cost.sum` is `null` when `cost.unit` is `'mixed'`. Aggregation isn't paginated: when `truncated` is `true`, narrow the selection instead of paging. Empty time buckets are omitted, not zero-filled. To see the traces behind a group, rerun `trace query` with the same selection plus the group's dimension values in `where`. +Rows contain `dimensions` and `measures`, plus bucket and cost information when applicable. Report low `cost.coverage` as incomplete spend; don't present a `null` cost as zero. Aggregation isn't paginated: when `truncated` is `true`, narrow the selection instead of paging. Empty time buckets are omitted, not zero-filled. To see the traces behind a group, rerun `trace query` with the same selection plus the group's dimension values in `where`. For a null dimension, use the documented missing-value predicate rather than `eq` with a null literal. ## Aggregate spans -Use `mastra api span aggregate` (`POST /api/observability/spans/aggregate`) when measures should be computed per span rather than per trace. Trace aggregation rolls each trace up and groups by root fields (e.g. the top-level agent); span aggregation counts any span, including nested agent runs and tool calls, grouped by span fields: `name`, `spanType`, `status`, `entityType`/`entityId`/`entityName`, `parentEntity*`/`rootEntity*` type and name, `environment`, `serviceName`, `executionSource`, `threadId`, `resourceId`, `userId`, `sessionId`, `organizationId`, `experimentId`. It selects exactly the spans `span query` would list for the same `timeRange` and `where`, and takes the same measures, `interval`, `having`, `orderBy`, and `limit` as `trace aggregate`. +Use `mastra api span aggregate` (`POST /api/observability/spans/aggregate`) when measures should be computed per span rather than per trace. Trace aggregation rolls up each trace; span aggregation measures completed spans, including nested agent runs and tool calls. It selects the same spans as `span query` for the same `timeRange` and `where`. Its request structure follows trace aggregation, but predicate and dimension fields follow the [span aggregation contract](https://mastra.ai/reference/observability/tracing/advanced-queries#aggregate-spans). ```bash npx mastra api span aggregate --help @@ -136,13 +132,12 @@ npx mastra api span aggregate \ '{"timeRange":{"from":"2026-08-01T00:00:00.000Z","to":"2026-08-08T00:00:00.000Z"},"where":{"op":"eq","left":{"path":"spanType"},"right":{"literal":"tool_call"}},"groupBy":["entityName"],"measures":["count","errorRate","duration.p95"],"orderBy":{"field":"errorRate","direction":"desc"},"limit":10}' ``` -Check `capabilities.spanAggregate` first; treat a missing value as unavailable. Unsupported servers return `501` (on Mastra platform, `404` while the feature is off). Rows and `truncated` match `trace aggregate`. Limits: +Check `capabilities.spanAggregate` first; treat a missing or false value as unavailable. Unsupported stores return `501 SPAN_AGGREGATE_UNSUPPORTED`; a server without the route returns `404`. Like trace aggregation, results are not paginated: check `truncated` and narrow the selection when needed. -- Tokens and cost belong to the span that recorded them (usually `model_generation`) and are never rolled up to ancestors, so an `agent_run` group normally shows no usage. For spend per agent, filter `spanType` to `model_generation` and group by `rootEntityName`, or use `trace aggregate`. -- No `model` or `provider` dimension yet. -- On Mastra platform, organization-wide requests (no project) over 14 days return `422` with a `time_range_too_large` issue unless `where` has a top-level `spanType` `eq`/`in` limited to branch span types: `agent_run`, `workflow_run`, `tool_call`, `mcp_tool_call`, `mcp_server_request`, `processor_run`, `scorer_run`, `rag_ingestion`, `provider_tool_call`. Narrow the window, add that filter, or aggregate one project. +- Tokens and cost belong to the span that recorded them and are not rolled up to ancestors. Use trace aggregation for totals per trace; for span aggregation, select the spans recording usage before grouping by agent identity. +- Grouping or counting distinct values by `model` or `provider` requires `spanAggregateModelDimensions`. These dimensions read dedicated columns, not older attributes; older spans can group under `null`. Check the capability instead of assuming support from the CLI version. -## Choosing the trace command +## Choosing the command - `trace list`: browse recent traces with simple list filters. - `trace query`: select completed traces using complex, explicit predicates. @@ -152,4 +147,4 @@ Check `capabilities.spanAggregate` first; treat a missing value as unavailable. - `trace get <traceId>`: inspect one selected trace. - `trace span <traceId> <spanId>`: fetch one span in full. -Advanced queries only return completed traces. The configured observability store must support the query API. If the server rejects a query that follows the current structural schema and canonical documentation as unsupported, report that limitation rather than falling back to an inaccurate client-side approximation. +These query and aggregate APIs return completed traces or spans only. The configured observability store must support the query API. If the server rejects a query that follows the current structural schema and canonical documentation as unsupported, report that limitation rather than falling back to an inaccurate client-side approximation. diff --git a/plugins/mastra/skills-lock.json b/plugins/mastra/skills-lock.json index 338f7fff..ef0df82e 100644 --- a/plugins/mastra/skills-lock.json +++ b/plugins/mastra/skills-lock.json @@ -5,7 +5,7 @@ "source": "mastra-ai/skills", "sourceType": "github", "skillPath": "skills/mastra/SKILL.md", - "computedHash": "a857fcd05f042b20ad3561df502073c43d2eb4b50b0795ee45ee7ce305104a97" + "computedHash": "82c644898a7b5c61cdac9641f38b50cdc70604104ef11c64623e16a5f2465ad6" } } } diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/SKILL.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/SKILL.md index f4e0dbce..6aabbe77 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/SKILL.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/SKILL.md @@ -2,31 +2,13 @@ name: modern-web-guidance description: | Search tool for modern web development best practices. MANDATORY: Execute FIRST for all HTML/CSS and clientside JS tasks. Do NOT skip — web APIs evolve rapidly and training weights contain obsolete patterns. - - Trigger immediately for: - - UI/Layout: Modals/dialogs/popovers, tables, dark mode, anchor positioning, container queries, `:has()`, etc. - - Scroll/Motion: View Transitions, Scroll-driven animations, parallax/reveals, etc. - - Performance: CWV (LCP, INP), content-visibility, Fetch Priority, image optimization, etc. - - System/APIs: Security, passkeys, HTML sanitization, Web Components, dates/timezones, Built-in AI, WebAssembly, etc. - - Frameworks: Standard HTML/CSS/DOM in React, Vue, Angular, etc. - - General Frontend: Forms, validation (`:user-valid`), autofill, accessibility, custom selects, scrollbars, etc. - - DO NOT trigger for: - - Backend: SQL, ORMs, API routes, etc. - - Infra/Scripts: CI/CD, Docker, Actions, Python/Go, Git, etc. - - Tooling/Trivial: Vite, tsconfig, ESLint, tests, copy/renames, 3P docs, etc. + Trigger immediately for: modals/dialogs/popovers, tables, dark mode, anchor positioning, container queries, view transitions, scroll-driven animations, parallax/reveals, forms, validation, autofill, accessibility, custom selects, scrollbars, CWV (LCP, INP, CLS), content-visibility, fetch priority, image optimization, security, passkeys, HTML sanitization, web components, dates/timezones, built-in AI, WebAssembly, standard HTML/CSS/DOM in frameworks, etc. + DO NOT trigger for: backend (SQL, ORMs, API routes), infra/scripts (CI/CD, Docker, Actions, Python/Go, Git), or tooling/trivial (Vite, tsconfig, ESLint, tests, copy/renames, 3P docs), etc. --- # Modern Web Guidance -A skill to search for specific web development use cases and retrieve their corresponding best practice guides. - -## When to use - -Must use this skill: -- At the **start** of implementing any web UI, CSS layout/animation, or client-side JS logic (dates/timezones, sanitization, performance, accessibility)—including when fixing UX symptoms (e.g., janky interactions, layout shifts). -- Before creating or updating a component, to check if a standardized modern web pattern already exists. -- To avoid implementing ad-hoc solutions or loading large dependencies unnecessarily. +Search and retrieve standardized modern web best practices at the **start** of implementing or updating any web UI, CSS layout/animation, or client-side JS logic (dates/timezones, sanitization, performance, accessibility, UX fixes) before writing ad-hoc solutions or adding dependencies. ## Usage Instructions @@ -37,38 +19,14 @@ Run `modern-web-guidance` directly with `npx` using a **2–5 word action-orient - **Example**: Search `"light dismiss dialog"` or `"responsive data table"`, **not** `"dialog"` (too terse), `"responsive table dark mode"` (compound — split into two searches), or `"close React settings modal on outside click"` (framework/app noise). ```sh -npx -y modern-web-guidance@latest search "<query>" --skill-version 2026_10_16-6654652b -``` - -**Example Output**: -```json -[ - { - "id": "optimize-image-priority", - "description": "Optimize the loading priority of Largest Contentful Paint (LCP) candidate images.", - "category": "performance", - "featuresUsed": [ "Fetch priority" ], - "tokenCount": 985, - "similarity": 0.7289 - }, - { - "id": "defer-rendering-heavy-content", - "description": "Reduce rendering times in content-heavy web pages by deferring rendering for offscreen content.", - "category": "performance", - "featuresUsed": [ "content-visibility", "hidden=\"until-found\"" ], - "tokenCount": 1250, - "similarity": 0.6961 - } -] +npx -y modern-web-guidance@latest search "<query>" --skill-version 2026_10_30-58d055df ``` -> **Note**: Judge results by their `description` fields (not raw `similarity` scores): +> **Note**: Returns `[{ "id", "description" }, ...]`. Judge results by their `description` fields: > - **Match found**: Call `retrieve` directly—do not also run `list`. > - **No match on a web UI/CSS/DOM behavior (or broad UI audit)**: If `search` returns `[]` or unrelated descriptions because your phrasing differed from the catalog (or your task spans multiple categories), run `npx -y modern-web-guidance@latest list` once to check if the pattern is covered under more general terminology. > - **Topic not covered**: If no guide `description` directly matches your task, proceed with your implementation without retry loops or retrieving unrelated guides. ---- - ### Step 2. Retrieve Best Practices Once you have relevant `id`(s) from the search results, call `retrieve` to get the full guide(s)—including both specific use-case guides that match your task and any broader discipline guide (e.g., `html`, `css`, `forms`, `accessibility`, `color`, `performance`, `security`) that applies. You can pass multiple IDs separated by commas. @@ -77,12 +35,7 @@ Once you have relevant `id`(s) from the search results, call `retrieve` to get t npx -y modern-web-guidance@latest retrieve "<id>" ``` -If the output is truncated, you must repeat the command but redirect to a file and read that file. - -**Example Output**: -`The markdown content of the guide describing implementation steps...` - ---- +If `retrieve` saves large output (> 9.5 KB) to a temporary file to prevent truncation, read that file path in full using your file-reading tool (reading remaining line ranges if needed)—do not use `cat` or `head`. ### Step 3. Verify Guidance Compliance @@ -100,37 +53,20 @@ When you consult `modern-web-guidance` for a task, end the response where you de ## Using npx / pnpx -- Prefer `pnpx` over `npx` if `pnpm` is available (note: `pnpx` does not use the `-y` flag). -- When requesting tool permissions, allowlist `npx -y modern-web-guidance@latest *` specifically (or `pnpx modern-web-guidance@latest *`), never bare `npx *` or `pnpx *`. -- IMPORTANT: on Windows, using `npx` may fail. Use `npx.cmd ...` instead. -- Fetching and running `modern-web-guidance` requires outbound network access. If running in a sandboxed, permission-gated, or approval-based environment (e.g., Codex, Claude Code), **proactively request approval/allowlisting for the command with network access BEFORE executing it the first time**, avoiding sandbox network timeouts. -- In sandboxed environments where `~/.npm` is read-only or restricted, set `NPM_CONFIG_CACHE=/tmp/npm-cache`. -- If the command hangs due to being offline, try running again in offline mode: `npx --offline …`. -- The `--skill-version` flag is used to determine if this SKILL.md is out of date. If it is, a warning message is logged to stderr. +- **Runner & Permissions**: Prefer `pnpx` over `npx` if `pnpm` is available (`pnpx` does not use `-y`); on Windows, use `npx.cmd`. When requesting tool permissions, allowlist `npx -y modern-web-guidance@latest *` (or `pnpx modern-web-guidance@latest *`) specifically, never bare `npx *` or `pnpx *`. +- **Sandboxes & Network**: Fetching `modern-web-guidance` requires outbound network access. In sandboxed or approval-gated environments (e.g., Codex, Claude Code), **proactively request approval/allowlisting with network access BEFORE the first execution** to avoid timeouts, and set `NPM_CONFIG_CACHE=/tmp/npm-cache` if `~/.npm` is read-only. +- **Offline**: If the command hangs due to being offline, retry with `npx --offline …`. ## Guidelines -- Always search **first** to find the most relevant guides. +- Always search **first** to find the most relevant guides. Do not hallucinate guides or ignore them; they represent the preferred local standard for the user's project. - These guides use vanilla HTML/CSS/JS; adapt them to your framework without dropping the modern feature: - **Types**: Spread/cast newly shipped HTML attributes (`{...({ popover: 'hint', interestfor: id, commandfor: id, switch: '' } as any)}`, `[attr.*]`) rather than removing them or merging `@types/react`. - **SSR & Bundlers**: Guard `window`/`HTMLElement` in SSR and load polyfills via bare `import('<pkg>')` in client effects (or set Vite `build.target: 'esnext'` for top-level `await`)—never copy files from `node_modules`. - **DOM, IDs & CSS**: Call `showModal()`/`moveBefore()` via refs on non-VDOM-managed nodes, generate per-instance IDs/`anchor-name`s (`useId()`), and keep `:root` and `@starting-style` top-level/unscoped. -- Do not hallucinate guides or ignore them; they represent the preferred local standard for the user's project. ## Interpreting Browser Support & Fallbacks * **Default Behavior**: All guides assume **Baseline Widely available** features are safe to use without fallbacks. For features that are not Baseline widely available, you **MUST** follow the fallback recommendations in the guide, unless the user has specified a custom browser support policy. -* **Custom Policies**: If the user has already defined explicit browser support requirements, use the browser compatibility data in the guide to determine if a fallback can be safely ignored. - - For Baseline YYYY targets, a feature satisfies this target if its "Baseline since" date is <= YYYY. - - **Policy Examples**: - - _"Do not implement feature fallbacks."_ (for exploratory prototypes of the cutting-edge web) - - _"Safari 17.4+"_ (for internal tools targeting macOS or Tauri-based desktop apps) - - _"Never recommend or implement polyfills; if a Baseline Newly Available feature is required for core functionality, provide a lightweight custom fallback or redesign the approach."_ (to minimize bundle size and avoid technical debt) - - _"Assume a modern execution environment where Baseline Newly Available features can be used natively, provided they are strictly feature-detected and degrade gracefully."_ (for progressive enhancement strategies) -* **Reactive Policy Discovery**: Watch for environmental cues to suggest documenting a policy in CLAUDE.md or AGENTS.md. Suggest this if the developer: - - Mentions building for a restricted runtime (e.g., Electron or Tauri). - - Explicitly excludes specific targets (e.g., "we don't support Desktop Chrome"). - - Expresses hesitation about polyfill complexity, bundle size, or performance cost. - - Questions if a feature is safe to use without fallbacks. - - No defined policy format. This is an example: `**Browser Support:** Allow Newly Available features, but only adopt custom fallback code that adds <= 20 lines and does not require external dependencies.` +* **Custom Policies**: If the user defines explicit browser support requirements, use the guide's compatibility data to determine if a fallback can be ignored (for `Baseline YYYY` targets, a feature satisfies the target if its "Baseline since" date is `<= YYYY`). Examples: _"Do not implement feature fallbacks"_, _"Safari 17.4+"_, _"Never implement polyfills; use a lightweight custom fallback or redesign"_, or _"Allow Newly Available features if feature-detected with graceful degradation"_. +* **Reactive Policy Discovery**: Suggest documenting a policy in `CLAUDE.md` or `AGENTS.md` (e.g., `**Browser Support:** Allow Newly Available features, but only adopt custom fallback code that adds <= 20 lines and no external dependencies`) if the developer targets a restricted runtime (Electron/Tauri), excludes specific browsers, hesitates over polyfill size/complexity, or asks if a feature is safe without fallbacks. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/accessibility/accessible-generated-content.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/accessibility/accessible-generated-content.md index e1dcf23c..299c9a93 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/accessibility/accessible-generated-content.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/accessibility/accessible-generated-content.md @@ -21,3 +21,21 @@ button.save::before { ``` A screen reader would read it out as "Save save". + +## Fallback strategies + +Baseline status for Alt text for generated content: Newly available. It's been Baseline since 2024-07-09. +Supported by: Chrome 77 (Sep 2019), Edge 79 (Jan 2020), Firefox 128 (Jul 2024), and Safari 17.4 (Mar 2024). + +If your Baseline target includes browsers that do not support the `/` alternative text syntax, precede the declaration with a single-value `content` fallback so unsupported browsers do not drop the entire declaration as invalid CSS: + +```css +.more-link::after { + /* Fallback: render the glyph in browsers without `/` support + (screen readers there will still announce it) */ + content: " →"; + content: " →" / ""; +} +``` + +For non-empty alternative text (`content: url(...) / "..."`), a single-value `content: url(...)` fallback keeps the image visible in older browsers but drops its alt text. If that label is required in older browsers, put it in the DOM instead (e.g., via a visually hidden `<span>` or `aria-label`) and use a plain `content: url(...)` declaration without `/`. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/language-model.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/language-model.md index 88ec832b..cca31d0a 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/language-model.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/language-model.md @@ -183,7 +183,7 @@ Try to be one step ahead by starting the next most likely AI task as soon as you **Don't:** Unless necessary, don't wait for the user to click "Generate" to initialize the session. This leads to a cold start delay, because the model must first load into memory and prepare its execution pipeline. > [!CAUTION] -> **Caution:** For the Prompt API, wait until you have the `initialPrompts` ready before calling `create()`, because these can only be set during session creation. +> For the Prompt API, wait until you have the `initialPrompts` ready before calling `create()`, because these can only be set during session creation. #### Set initial prompts during creation diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/prompt-api.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/prompt-api.md index 88ec832b..cca31d0a 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/prompt-api.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/built-in-ai/prompt-api.md @@ -183,7 +183,7 @@ Try to be one step ahead by starting the next most likely AI task as soon as you **Don't:** Unless necessary, don't wait for the user to click "Generate" to initialize the session. This leads to a cold start delay, because the model must first load into memory and prepare its execution pipeline. > [!CAUTION] -> **Caution:** For the Prompt API, wait until you have the `initialPrompts` ready before calling `create()`, because these can only be set during session creation. +> For the Prompt API, wait until you have the `initialPrompts` ready before calling `create()`, because these can only be set during session creation. #### Set initial prompts during creation diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/animated-select-picker.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/animated-select-picker.md index f66fd413..6976832c 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/animated-select-picker.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/animated-select-picker.md @@ -157,7 +157,7 @@ The following example demonstrates a custom select styled with standard page ani ## Fallback strategies -### Fallbacks & browser support for Customizable <select> +### Fallbacks & browser support for Customizable `<select>` Browser support for Customizable <select>: Limited availability. Supported by: Chrome 135 (Apr 2025), Edge 135 (Apr 2025), and Safari 27 (Sep 2026). diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/autofill-highlight-inputs.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/autofill-highlight-inputs.md index 586192e5..f3fe6f20 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/autofill-highlight-inputs.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/autofill-highlight-inputs.md @@ -1,4 +1,4 @@ -# Use the CSS :autofill pseudo-class to highlight form fields that have been autofilled by the browser and not edited by the user +# Highlight browser-autofilled fields the user hasn't edited with CSS :autofill Use the CSS `:autofill` to highlight fields that have (or have not been) autofilled, to help guide the user to successful form completion. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/branded-select-styling.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/branded-select-styling.md index 52be5a7b..ec7533fa 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/branded-select-styling.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/branded-select-styling.md @@ -93,7 +93,7 @@ The following example demonstrates a custom select styled with a monospace font ## Fallback strategies -### Fallbacks & browser support for Customizable <select> +### Fallbacks & browser support for Customizable `<select>` Browser support for Customizable <select>: Limited availability. Supported by: Chrome 135 (Apr 2025), Edge 135 (Apr 2025), and Safari 27 (Sep 2026). diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/custom-select-picker-layouts.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/custom-select-picker-layouts.md index 65cb46c6..9dcbc649 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/custom-select-picker-layouts.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/custom-select-picker-layouts.md @@ -144,7 +144,7 @@ Otherwise, it may be ultimately necessary to switch to a custom solution using t ## Fallback Strategy -### Fallbacks & browser support for Customizable <select> +### Fallbacks & browser support for Customizable `<select>` Browser support for Customizable <select>: Limited availability. Supported by: Chrome 135 (Apr 2025), Edge 135 (Apr 2025), and Safari 27 (Sep 2026). diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/ime-safe-enter-submit.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/ime-safe-enter-submit.md index a263a8e2..add93ece 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/ime-safe-enter-submit.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/ime-safe-enter-submit.md @@ -60,7 +60,7 @@ If you need to support cross-browser compatibility across older versions of Safa ### Strategy 1: Add a `keyCode === 229` check (recommended) -By pairing `event.isComposing` with a check for `event.keyCode === 229`, you can reliably catch Safari's out-of-order confirming `Enter` keydown. Because this is nested under the `event.key === 'Enter'` gate, it is safe from mobile virtual keyboards that might use `229` for normal character layout entry. +During IME composition, browsers report `keyCode: 229` (`VK_PROCESSKEY`) instead of `13` on `keydown`, even while setting `event.key === 'Enter'` on the confirming keystroke. Pairing `event.isComposing` with `event.keyCode === 229` reliably catches Safari's out-of-order confirming `Enter` keydown. (This also means legacy `event.keyCode === 13` handlers are already immune to IME confirmation, whereas migrating them to `event.key === 'Enter'` without `event.isComposing` and `event.keyCode === 229` will introduce premature submission.) Because this check is nested under the `event.key === 'Enter'` gate, it is safe from mobile virtual keyboards that might use `229` for normal character layout entry. ```js textarea.addEventListener('keydown', (event) => { diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/rich-media-picker.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/rich-media-picker.md index 63c51f3a..78e087ff 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/rich-media-picker.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/rich-media-picker.md @@ -111,7 +111,7 @@ select.custom-select option:checked .option-title { ## Fallback Strategies -### Fallbacks & browser support for Customizable <select> +### Fallbacks & browser support for Customizable `<select>` Browser support for Customizable <select>: Limited availability. Supported by: Chrome 135 (Apr 2025), Edge 135 (Apr 2025), and Safari 27 (Sep 2026). diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/slider.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/slider.md index 1e3cba1b..2e66a71c 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/slider.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/forms/slider.md @@ -9,7 +9,7 @@ Use a labeled `<input type="range">` rather than recreating a slider with `<div> <input type="range" id="range-slider" value="70"/> ``` -If the value requires a formatted label or unit, expose it in ordinary DOM content and update `aria-valuetext` when changed. Do not use `::before` or `::after` on slider pseudo-elements, as generated content on form controls is not interoperable. If a tooltip that follows the thumb is required, use the `slider-tooltip` (via `npx -y modern-web-guidance@latest retrieve "slider-tooltip"`) guide. +If the value requires a formatted label or unit, expose it in ordinary DOM content and update `aria-valuetext` when changed. Do not use `::before` or `::after` on slider pseudo-elements, as generated content on form controls is not interoperable. If a tooltip that follows the thumb is required, anchor it to the thumb with CSS anchor positioning; see `position-aware-tooltips` (via `npx -y modern-web-guidance@latest retrieve "position-aware-tooltips"`) for tooltips that stay correctly oriented when they flip. ## Choose the styling approach diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/efficient-background-processing.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/efficient-background-processing.md index f78873d5..fe59af2f 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/efficient-background-processing.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/efficient-background-processing.md @@ -44,7 +44,9 @@ Set `content-visibility: auto` on the heavy container and provide a placeholder Add an event listener for `contentvisibilityautostatechange` to pause or resume background tasks. -> **Important:** The `contentvisibilityautostatechange` event does not bubble in some browser implementations. To handle this event reliably, you must either: +> [!IMPORTANT] +> The `contentvisibilityautostatechange` event does not bubble in some browser implementations. To handle this event reliably, you must either: +> > - Attach the event listener directly to the element that has `content-visibility: auto` applied. > - Use a capturing event listener (`{ capture: true }`) if you are delegating events to a parent container. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/faster-spa-view-transitions.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/faster-spa-view-transitions.md index e04cd285..f2b08890 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/faster-spa-view-transitions.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/faster-spa-view-transitions.md @@ -27,7 +27,8 @@ While this approach offers massive performance benefits, it introduces a specifi Set `content-visibility: hidden` on views that are not currently active. -> **Note:** The `content-visibility: hidden` property hides an element's *contents*, but the element itself remains styled and visible. Its background, borders, padding, and margins will still be painted by the browser. +> [!NOTE] +> The `content-visibility: hidden` property hides an element's *contents*, but the element itself remains styled and visible. Its background, borders, padding, and margins will still be painted by the browser. ```css .spa-view.inactive { @@ -41,7 +42,7 @@ Set `content-visibility: hidden` on views that are not currently active. ### 2. Manage Focus -When swapping views, ensure you manage keyboard focus correctly to preserve accessibility. +When swapping views, ensure you manage keyboard focus correctly to preserve accessibility. Use the `inert` attribute to ensure interface elements in the hidden content are not included in the keyboard tab order. ```javascript function switchToView(viewId) { @@ -49,12 +50,14 @@ function switchToView(viewId) { document.querySelectorAll('.spa-view').forEach(view => { view.classList.add('inactive'); view.setAttribute('aria-hidden', 'true'); + view.setAttribute('inert', 'inert'); }); // Show the target view const activeView = document.getElementById(viewId); activeView.classList.remove('inactive'); activeView.setAttribute('aria-hidden', 'false'); + activeView.removeAttribute('inert'); // MANDATORY: Move focus to the new view to ensure a logical tab-order activeView.focus(); diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/performance.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/performance.md index 32c6eccb..2a77100d 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/performance.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/performance/performance.md @@ -142,7 +142,7 @@ Third-party scripts (analytics, ads, chat widgets) are the primary source of mai **HTML: Third-Party Script Execution** ```html <!-- 1. Place third-party scripts near the end of the page with the defer attribute --> -<script defer src="http://www.example.com/third-party.js"></script> +<script defer src="https://www.example.com/third-party.js"></script> ``` ## CSS Rendering & Containment Optimization diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/local-network-access.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/local-network-access.md index 262a8374..59c5d041 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/local-network-access.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/local-network-access.md @@ -162,7 +162,7 @@ Permissions-Policy: local-network=(self "https://setup.partner.example.com"), lo ## Fallback Strategies Browser support for Local network access: Limited availability. -Supported by: Chrome 142 (Oct 2025) and Edge 142 (Oct 2025). +Supported by: Chrome 145 (Feb 2026) and Edge 145 (Feb 2026). Unsupported in: Firefox and Safari. If your Baseline target does not support Local Network Access permissions and `targetAddressSpace` options, implement **feature detection with progressive fallback**: diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/passkey-management.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/passkey-management.md index f258c26f..1375dc26 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/passkey-management.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/passkey-management.md @@ -221,6 +221,7 @@ import 'webauthn-polyfills'; Browser support for Web authentication signal methods: Limited availability. Supported by: Chrome 132 (Jan 2025), Edge 132 (Jan 2025), and Safari 26 (Sep 2025). Unsupported in: Firefox. + If the browser does not support `PublicKeyCredential.parseRequestOptionsFromJSON`, use the 'webauthn-polyfills': ```html @@ -229,7 +230,7 @@ If the browser does not support `PublicKeyCredential.parseRequestOptionsFromJSON await import('https://unpkg.com/webauthn-polyfills'); } </script> - ``` +``` This will also add support for `PublicKeyCredential.prototype.toJSON`. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/security.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/security.md index 4b6f839b..364926e4 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/security.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/security.md @@ -40,7 +40,7 @@ Guidelines for implementing preventative security measures on the web safely and The right starting point depends on the application and whether you can update the codebase directly: -- **Auditing and hardening a codebase directly**: Apply Phase 1 code hygiene, wire up Phase 2 reporting (`Reporting-Endpoints` and `report-to`), and enforce Phase 3 headers (`Content-Security-Policy` with nonces/hashes, `'strict-dynamic'`, `'report-sample'`, `object-src 'none'`, `base-uri 'none'`, `frame-ancestors 'self'`, and `require-trusted-types-for 'script'`, plus COOP, CORP, Fetch Metadata, HSTS, `X-Content-Type-Options`, `Referrer-Policy`, and `Permissions-Policy`) directly by updating `<script>` tags, cookies, and server middleware in the same pass. +- **Auditing and hardening a codebase directly**: Apply Phase 1 code hygiene, wire up Phase 2 reporting (`Reporting-Endpoints` and `report-to`), and enforce Phase 3 headers (`Content-Security-Policy` with nonces/hashes, `'strict-dynamic'`, `'report-sample'`, `object-src 'none'`, `base-uri 'none'`, `frame-ancestors 'none'` (or `'self'`), and `require-trusted-types-for 'script'`, plus COOP, CORP, Fetch Metadata, HSTS, `X-Content-Type-Options`, `Referrer-Policy`, and `Permissions-Policy`) directly by updating `<script>` tags, cookies, and server middleware in the same pass. - **Staged rollout on a large production site with unknown third-party dependencies**: Start with Phase 1 quick wins and deploy Phase 2 report-only headers alongside enforced companion policies while collecting violation telemetry, then promote the report-only policies to Phase 3 enforcement. - **Greenfield app or SaaS template / framework defaults**: Ship Phase 1 hygiene and Phase 3 enforced policies by default, with Phase 2 reporting configured so regressions are captured immediately. @@ -88,12 +88,22 @@ Set-Cookie: third_party_var=value; SameSite=None; Secure; Partitioned ``` ### 1.4 Clickjacking Protection (Frame-Ancestors & X-Frame-Options) -Clickjacking protection is easy to deploy, carries extremely low risk of breaking legitimate functionality, and provides immediate defense against malicious UI redressing. -- **DO**: Set the `X-Frame-Options: SAMEORIGIN` header to prevent other sites from embedding your pages in an iframe (or use `DENY` if you should never be embedded). -- **DO**: For fine-grained control, use `frame-ancestors 'self'` (or specified trusted domains) in your CSP header. +Clickjacking protection is easy to deploy, carries extremely low risk of breaking legitimate functionality, and provides immediate defense against malicious UI redressing. Choose the policy tier that matches your framing requirements: +- **DO (Default — no framing needed)**: Set `Content-Security-Policy: frame-ancestors 'none'` alongside `X-Frame-Options: DENY` when your pages never need to be embedded in an iframe. +- **DO (Same-origin framing only)**: Set `Content-Security-Policy: frame-ancestors 'self'` alongside `X-Frame-Options: SAMEORIGIN` when your application embeds its own pages in same-origin iframes. +- **DO (Specific cross-origin framing required)**: Set `Content-Security-Policy: frame-ancestors 'self' https://trusted-partner.com` and **omit `X-Frame-Options`** (`X-Frame-Options` cannot express cross-origin allowlists—`ALLOW-FROM` is obsolete and unsupported—and `DENY` or `SAMEORIGIN` would block the embed in any client that evaluates `X-Frame-Options` without `frame-ancestors`). +- **DO (Arbitrary framing explicitly required)**: Only omit both `frame-ancestors` and `X-Frame-Options` on routes designed as publicly embeddable widgets. ```http +# Default (never framed): +Content-Security-Policy: frame-ancestors 'none'; +X-Frame-Options: DENY + +# Same-origin framing allowed: +Content-Security-Policy: frame-ancestors 'self'; X-Frame-Options: SAMEORIGIN + +# Specific cross-origin framing allowed (omit X-Frame-Options): Content-Security-Policy: frame-ancestors 'self' https://trusted-partner.com; ``` @@ -145,7 +155,7 @@ Use "Report-Only" headers to identify potential breakages before they happen. **Example headers:** ```http Reporting-Endpoints: default="https://reports.example/default", main-endpoint="https://reports.example/main" -Content-Security-Policy-Report-Only: script-src 'nonce-{RANDOM}' 'strict-dynamic' 'report-sample'; object-src 'none'; base-uri 'none'; frame-ancestors 'self'; require-trusted-types-for 'script'; report-to main-endpoint; +Content-Security-Policy-Report-Only: script-src 'nonce-{RANDOM}' 'strict-dynamic' 'report-sample'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'; require-trusted-types-for 'script'; report-to main-endpoint; ``` The `'strict-dynamic'`, `https:`, and `'unsafe-inline'` tokens together form a backwards-compatibility ladder: modern browsers honor `'strict-dynamic'` (nonce-propagating) and ignore the others; older browsers fall back to `https:`; very old browsers fall back to `'unsafe-inline'`. The fallbacks are harmless on any browser that supports a stricter token. @@ -196,7 +206,7 @@ When enforcing `Content-Security-Policy`, always keep `Reporting-Endpoints` and - `script-src 'nonce-{RANDOM}' 'strict-dynamic' 'report-sample'` (or SHA-256/384/512 script hashes for static HTML) — the core directive of any CSP and the primary mechanism to prevent XSS. - `object-src 'none'` — blocks plugin-based code execution (`<object>`, `<embed>`). - `base-uri 'none'` — blocks `<base>` tag hijacking of relative URLs. -- `frame-ancestors 'self'` — prevents clickjacking alongside `X-Frame-Options: SAMEORIGIN` (§1.4). +- `frame-ancestors 'none'` (or `'self'` when same-origin framing is needed) — prevents clickjacking alongside `X-Frame-Options: DENY` (or `SAMEORIGIN`; see §1.4). - `require-trusted-types-for 'script'` — enforces Trusted Types runtime checks on DOM XSS sinks (§3.3). - `report-to <endpoint>` — sends violation reports to the endpoint defined in `Reporting-Endpoints`. - *Optional*: `form-action 'self'` prevents form submissions to attacker-controlled origins. @@ -204,7 +214,7 @@ When enforcing `Content-Security-Policy`, always keep `Reporting-Endpoints` and **Enforced Header Example (CSP with reporting):** ```http Reporting-Endpoints: main-endpoint="https://reports.example/main" -Content-Security-Policy: script-src 'nonce-{RANDOM}' 'strict-dynamic' 'report-sample'; object-src 'none'; base-uri 'none'; frame-ancestors 'self'; require-trusted-types-for 'script'; report-to main-endpoint; +Content-Security-Policy: script-src 'nonce-{RANDOM}' 'strict-dynamic' 'report-sample'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'; require-trusted-types-for 'script'; report-to main-endpoint; ``` HTML for nonce-based CSP: @@ -224,7 +234,7 @@ const csp = [ `script-src 'nonce-${nonce}' 'strict-dynamic' 'report-sample'`, "object-src 'none'", "base-uri 'none'", - "frame-ancestors 'self'", + "frame-ancestors 'none'", // or "frame-ancestors 'self'" if same-origin framing is needed "require-trusted-types-for 'script'", 'report-to main-endpoint', ].join('; '); diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/trusted-types.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/trusted-types.md index 41e6364d..121c918b 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/trusted-types.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/security/trusted-types.md @@ -39,7 +39,8 @@ const escapePolicy = trustedTypes.createPolicy('escape-html', { }); ``` -> **Warning:** Character-level HTML escaping (`escapePolicy`) converts all markup into plain text and is **not** a context-aware HTML sanitizer. If you only need plain text, prefer `element.textContent` (which is not a Trusted Types sink). Never interpolate `escapePolicy.createHTML()` output into another HTML template string—such as an unquoted attribute (`<img src=${escaped}>`) or a URL attribute (`<a href="${escaped}">`, where `javascript:` URLs are not neutralized by entity escaping). Use a full HTML sanitizer (`DOMPurify` or `element.setHTML()`, covered in `sanitize-untrusted-html` (via `npx -y modern-web-guidance@latest retrieve "sanitize-untrusted-html"`)) whenever accepting HTML markup or attributes. +> [!WARNING] +> Character-level HTML escaping (`escapePolicy`) converts all markup into plain text and is **not** a context-aware HTML sanitizer. If you only need plain text, prefer `element.textContent` (which is not a Trusted Types sink). Never interpolate `escapePolicy.createHTML()` output into another HTML template string—such as an unquoted attribute (`<img src=${escaped}>`) or a URL attribute (`<a href="${escaped}">`, where `javascript:` URLs are not neutralized by entity escaping). Use a full HTML sanitizer (`DOMPurify` or `element.setHTML()`, covered in `sanitize-untrusted-html` (via `npx -y modern-web-guidance@latest retrieve "sanitize-untrusted-html"`)) whenever accepting HTML markup or attributes. ### 2. Using the Policy diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-atoms/scroll-position-aware-elements.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-atoms/scroll-position-aware-elements.md index 44b9dbb9..18e405cd 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-atoms/scroll-position-aware-elements.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-atoms/scroll-position-aware-elements.md @@ -36,7 +36,8 @@ Place the element inside the container and style it. By default, it should be hi } ``` -> **Important:** Sticky or floating elements hover above the scrollable content. Ensure that the main content has sufficient bottom padding or margin so that the last few elements are not permanently covered by the button when the user scrolls completely to the bottom. +> [!IMPORTANT] +> Sticky or floating elements hover above the scrollable content. Ensure that the main content has sufficient bottom padding or margin so that the last few elements are not permanently covered by the button when the user scrolls completely to the bottom. ### 3. Respond to the Scroll State diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/same-document-transitions.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/same-document-transitions.md index 07308918..281acc65 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/same-document-transitions.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/same-document-transitions.md @@ -27,7 +27,7 @@ Use the `view-transition-name` CSS property to tell the browser which elements s You can use multiple `view-transition-name`s to morph multiple pairs of elements. For example, you may want to transition both the product image and title with separate transitions. -Because there are multiple items on the list view, you can not give the all of them the same `view-transition-name`. This can be solved in two ways in a SPA. +Because there are multiple items on the list view, you cannot give all of them the same `view-transition-name`. This can be solved in two ways in a SPA. 1. **Dynamic detail page:** Assign each item on the list page a unique `view-transition-name`, and then dynamically apply that name to the matching element on the detail page when the list item is selected, as shown here. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/scoped-component-transitions.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/scoped-component-transitions.md index 5b33125f..98e4f838 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/scoped-component-transitions.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/scoped-component-transitions.md @@ -22,7 +22,8 @@ The scope element automatically participates in its own view transition because When the scope clips its overflow (`overflow: hidden`, `scroll`, or `clip`), `::view-transition-group-children(root)` automatically applies `overflow: clip`, preventing transitioning children from bleeding outside the component bounds. -> **Note:** Automatic overflow clipping only applies to the scope element itself (`root`). If a captured element inside the scope clips its own overflow and contains other nested captured elements (for example, a scrollable or clipped `<ul>` inside a `.card`), use Nested View Transition Groups so those nested elements stay clipped to their outer captured container: +> [!NOTE] +> Automatic overflow clipping only applies to the scope element itself (`root`). If a captured element inside the scope clips its own overflow and contains other nested captured elements (for example, a scrollable or clipped `<ul>` inside a `.card`), use Nested View Transition Groups so those nested elements stay clipped to their outer captured container: > > ```css > .card ul { diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/swipe-to-remove.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/swipe-to-remove.md index bfab2040..97f21965 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/swipe-to-remove.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-behaviors/swipe-to-remove.md @@ -132,7 +132,8 @@ The track configuration is gated behind an `.is-initialized` class on the list i The action color lives on the spacers (Step 2). The action **icon** lives on the list item itself, anchored to the row's left and right edges. -> **Note:** Throughout this guide, "left" refers to the *left side of the row* (which is revealed by swiping right), and "right" refers to the *right side of the row* (revealed by swiping left). +> [!NOTE] +> Throughout this guide, "left" refers to the *left side of the row* (which is revealed by swiping right), and "right" refers to the *right side of the row* (revealed by swiping left). The placement, sizing, and motion below are a **starting suggestion**, not a requirement. Adjust the icon size, edge insets, threshold-pop scale, transition duration, and even the choice of pseudo-elements vs. real DOM nodes to match your design. The only mechanical requirement is that the icon sits behind the content (so the content can cover it pre-swipe) and inside the list item (so it doesn't scroll with the spacer). Everything else is taste. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/navigation-drawer.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/navigation-drawer.md index 6b17dc03..54f2d9d3 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/navigation-drawer.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/navigation-drawer.md @@ -357,7 +357,7 @@ Supported by: Chrome 85 (Aug 2020), Edge 85 (Aug 2020), Firefox 128 (Jul 2024), #### Popover API fallback (no `popover` attribute support): -Baseline status for the api.HTMLElement.showPopover capability: Newly available. It's been Baseline since 2024-04-16. +Baseline status for the api.HTMLElement.showPopover capability: Widely available. It's been Baseline since 2024-04-16. Supported by: Chrome 114 (May 2023), Edge 114 (Jun 2023), Firefox 125 (Apr 2024), and Safari 17 (Sep 2023). Because this component uses `popover="manual"` and implements dismissal entirely from JavaScript, it does not depend on the popover API's defining behaviors — light-dismiss, the `popovertarget` attribute, top-layer-managed Escape handling, or focus management. The only popover features it actually uses are top-layer promotion (via `showPopover()`) and the `::backdrop` pseudo-element. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/scrollspy.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/scrollspy.md index 6924a2cb..d8b15952 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/scrollspy.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/ui-components/scrollspy.md @@ -104,7 +104,7 @@ For browsers that do not support `scroll-target-group`, you should use a fallbac navLinks.forEach(link => { const isActive = link.getAttribute('href') === `#${id}`; // Use the escaped colon class for consistent styling - link.classList.toggle('\:target-current', isActive); + link.classList.toggle('target-current', isActive); link.setAttribute('aria-current', isActive); }); } @@ -117,8 +117,8 @@ For browsers that do not support `scroll-target-group`, you should use a fallbac <style> /* Fallback style if :target-current is not supported. - Use :is() to combine the native pseudo-class with a fallback class for older browsers. DO NOT use `nav a:target-current, nav a.\:target-current, as the unrecognized pseudoselector invalidates the entire rule in older browsers. */ - nav a:is(:target-current, .\:target-current) { + Use :is() to combine the native pseudo-class with a fallback class for older browsers. DO NOT use `nav a:target-current, nav a.target-current, as the unrecognized pseudoselector invalidates the entire rule in older browsers. */ + nav a:is(:target-current, .target-current) { color: var(--accent, #007bff); border-bottom-color: var(--accent, #007bff); font-weight: bold; diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/user-experience/content-first-noise-reduction.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/user-experience/content-first-noise-reduction.md new file mode 100644 index 00000000..ef13d0f3 --- /dev/null +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/user-experience/content-first-noise-reduction.md @@ -0,0 +1,155 @@ +# Maximize content and reduce noise + +Users come for the content. Keep the page frame small, move it out of the way while users read, and show secondary content only when asked. + +## Implementation + +### Don't block or interrupt the user + +- DO NOT open newsletter modals, interstitials, or full-screen promos when the page loads. Where possible, remove banners and unrelated content that compete with the main content. +- If a notice is legally required (for example, cookie consent), show it as a small, non-modal bar that covers well under 15% of the viewport, with a visible dismiss button labeled with text such as "Accept" or "Close". +- Match the interruption to the stakes: use a blocking dialog only for consequential actions, such as deleting an account, paying, or editing personal data. Use non-blocking messages for everything else. +- Don't move content while the user reads. Reserve space for images, ads, and embeds before they load (for example, with `width` and `height` attributes), don't insert content above what's on screen, and don't open popovers the user didn't ask for. + +### Keep persistent bars compact + +Keep every fixed or sticky bar under 12% of the viewport height. Use a single row, and don't stack several sticky bars (announcement, header, and sub-navigation). Move extra items into a popover menu or drawer instead (see below). + +### Hide persistent bars while the user scrolls down + +Hide the top bar (the site header or navigation bar) while the user scrolls down and bring it back as soon as they scroll up, so it's out of the way while reading but one flick away. A scroll-state container query tracks the scroll direction in CSS, with no script. Hide bottom bars, such as a tab bar, the same way, sliding them down instead of up. + +```css +/* The root scroller remembers the direction of the most recent scroll. */ +html { + container-type: scroll-state; +} +.top-bar { + position: sticky; + top: 0; + z-index: 1; /* Example value: keeps positioned content from painting over the bar. */ +} +/* Scrolling down: move the bar out of view. transform doesn't change layout, so content doesn't jump. + (scrollable: top) shows the bar at the very top, even after jumps such as the Home key, which don't update "scrolled". */ +@container scroll-state((scrolled: bottom) and (scrollable: top)) { + .top-bar { + transform: translateY(-100%); + } +} +/* Keep the bar visible while it contains keyboard focus. */ +.top-bar:focus-within { + transform: none; +} +/* Example value: at least the bar's height, so in-page link targets don't land under the bar. */ +:target { + scroll-margin-top: 4rem; +} +``` + +Alternatively, shrink the bar to under 80% of its height as the user scrolls; see `shrinking-header-on-scroll` (via `npx -y modern-web-guidance@latest retrieve "shrinking-header-on-scroll"`). + +### Put secondary UI in popovers and drawers + +Use the `popover` attribute for menus, overflow actions, tooltips, hover cards, and onboarding tips instead of permanent toolbars or custom JavaScript overlays. A button with `popovertarget` opens and closes the popover on click, <kbd>Enter</kbd>, or <kbd>Space</kbd>. The default `popover="auto"` also closes it on <kbd>Esc</kbd> or a click outside. + +```html +<button popovertarget="more-links">More</button> +<!-- Rendered in the top layer, so no z-index management is needed. --> +<nav id="more-links" popover aria-label="More links"> + <a href="/about">About</a> + <a href="/contact">Contact</a> +</nav> +``` + +- Popovers open in the center of the screen by default. To place a popover next to its button, see `viewport-aware-dropdowns` (via `npx -y modern-web-guidance@latest retrieve "viewport-aware-dropdowns"`). +- Buttons can also open popovers with `commandfor` and `command="toggle-popover"`; see `declarative-dialog-popover-control` (via `npx -y modern-web-guidance@latest retrieve "declarative-dialog-popover-control"`). +- For larger sets of options, use an off-canvas drawer: a `popover` panel pinned to one side of the screen that slides in over the page (see `navigation-drawer` (via `npx -y modern-web-guidance@latest retrieve "navigation-drawer"`) for swipe-to-close and animation). Show rows of related items as a horizontally scrolling carousel instead of a long stacked list. + +### Collapse secondary content with `details` + +Put long or rarely needed content (specs, FAQs, fine print) in a `details` element. It starts closed, and its `summary` works with the keyboard and screen readers without JavaScript. + +```html +<details> + <summary>Shipping and returns</summary> + <p>Free returns within 30 days.</p> +</details> +``` + +## Fallback strategies + +### Scroll-state queries + +Browser support for Container scroll-state queries: Limited availability. +Supported by: Chrome 133 (Feb 2025) and Edge 133 (Feb 2025). +Unsupported in: Firefox and Safari. + +Without scroll-state queries, the bar never hides, but it stays compact and usable. If your Baseline target includes those browsers, add a small script so the bar hides there too. Browsers that support other scroll-state queries but not `scrolled` also keep the bar visible. + +```css +/* Don't hide the bar while it contains keyboard focus. */ +.top-bar.is-hidden:not(:focus-within) { + transform: translateY(-100%); +} +``` + +```js +// Runs only where scroll-state queries aren't supported. Run it after .top-bar is parsed (for example, in a module script). +if (!CSS.supports('container-type', 'scroll-state')) { + const bar = document.querySelector('.top-bar'); + let lastY = scrollY; + addEventListener('scroll', () => { + // Clamp so rubber-band overscroll at the top or bottom doesn't flip the scroll direction. + const y = Math.max(0, Math.min(scrollY, document.documentElement.scrollHeight - innerHeight)); + if (y === lastY) return; + // Hide only after scrolling down past the bar's own height, so it doesn't flicker near the top. + bar?.classList.toggle('is-hidden', y > lastY && y > bar.offsetHeight); + lastY = y; + }, { passive: true }); +} +``` + +### Fallbacks & browser support for Popover + +Baseline status for Popover: Newly available. It's been Baseline since 2025-01-27. +Supported by: Chrome 116 (Aug 2023), Edge 116 (Aug 2023), Firefox 125 (Apr 2024), Safari 17 (Sep 2023), and Safari iOS 18.3 (Jan 2025). + +The Popover API is mostly **progressive enhancement**, but its defining behaviors — top-layer promotion, light-dismiss, and `popovertarget` invocation — have no CSS-only equivalent. Older browsers need a polyfill, or a manual fallback if you would rather not ship one. + +**Polyfill:** To support the `popover` attribute in older browsers, conditionally load [`@oddbird/popover-polyfill`](https://github.com/oddbird/popover-polyfill). **MANDATORY:** Feature detect by checking for the `popover` property on `HTMLElement.prototype`, and load the polyfill **only** when native support is missing — do NOT load it unconditionally. + +With a bundler or import map: + +```js +// MANDATORY: Feature detect 'popover' on HTMLElement.prototype. +if (!("popover" in HTMLElement.prototype)) { + import("@oddbird/popover-polyfill"); +} +``` + +Without a bundler, import from a CDN inside a `<script type="module">`: + +```html +<script type="module"> + if (!("popover" in HTMLElement.prototype)) { + import("https://unpkg.com/@oddbird/popover-polyfill@latest/dist/popover.min.js"); + } +</script> +``` + +**Styling caveat:** The polyfill cannot define the real `:popover-open` pseudo-class, so it applies a `.\:popover-open` class instead. **MANDATORY:** Combine the two with `:is()` or `:where()`, otherwise browsers that lack `:popover-open` discard the entire rule: + +```css +[popover]:is(:popover-open, .\:popover-open) { + display: block; +} +``` + +Alternatively, for a legacy fallback without a polyfill, use `position: fixed` and manually calculate coordinates via `getBoundingClientRect()` or rely on default positioning with `inset: auto` if that's acceptable for the use case. + +### Invoker commands + +Baseline status for Invoker commands: Newly available. It's been Baseline since 2025-12-12. +Supported by: Chrome 135 (Apr 2025), Edge 135 (Apr 2025), Firefox 144 (Oct 2025), and Safari 26.2 (Dec 2025). + +Browsers without invoker commands ignore `commandfor`, so the button does nothing there. Prefer `popovertarget`, which works wherever popovers do. If you use `commandfor`, dynamically import `invokers-polyfill` (through your bundler, or from a CDN if you don't use one) only when `'commandForElement' in HTMLButtonElement.prototype` is false. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/custom-text-decorations.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/custom-text-decorations.md index f255eb73..20401499 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/custom-text-decorations.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/custom-text-decorations.md @@ -162,8 +162,6 @@ Use a pseudo-element when the line must be a shape `text-decoration` cannot draw } ``` -For a freeform path or a draw-on that follows the stroke, see `line-drawing-animation` (via `npx -y modern-web-guidance@latest retrieve "line-drawing-animation"`). - ## Outline text with a box instead Two box properties decorate text by outlining it rather than running a line along it. Give the text padding so the outline does not touch it. @@ -206,10 +204,10 @@ When only the corners need a different shape, such as a beveled, scooped, or not ## Choose a technique * **`text-decoration`** (line, ink skipping, spaces, inset): the line should run along the text, wrap with it, and follow its font metrics. This is the default choice. -* **Pseudo-element with `border-shape` and `clip-path`**: the line is a custom shape or needs a draw-on animation. Choose it only when `text-decoration-style` cannot produce the look, because it needs extra positioning. For vector path animations, see `line-drawing-animation` (via `npx -y modern-web-guidance@latest retrieve "line-drawing-animation"`). -* **`background-clip: border-area`**: the text needs a gradient outline. Use plain `border-color` when a solid outline is enough. For complete gradient border styling, see `gradient-borders` (via `npx -y modern-web-guidance@latest retrieve "gradient-borders"`). +* **Pseudo-element with `border-shape` and `clip-path`**: the line is a custom shape or needs a draw-on animation. Choose it only when `text-decoration-style` cannot produce the look, because it needs extra positioning. +* **`background-clip: border-area`**: the text needs a gradient outline. Use plain `border-color` when a solid outline is enough. * **`corner-shape`**: only the corners need a different shape, such as beveled, scooped, or notched. Pair it with `border-radius`, which sets the corner size. -* **`border-shape`**: the outline needs a shape `border-radius` cannot draw, such as a tag or arrow. Use `border-radius` when rounded corners are enough. For shaped element styling, see `element-shape` (via `npx -y modern-web-guidance@latest retrieve "element-shape"`). +* **`border-shape`**: the outline needs a shape `border-radius` cannot draw, such as a tag or arrow. Use `border-radius` when rounded corners are enough. * **Inset hover reveal**: worth adding when a link needs a hover affordance beyond a color change. Skip it for text that is not interactive. ## Fallback Strategies diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/equal-width-text-lines.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/equal-width-text-lines.md index 99733d13..8ac42299 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/equal-width-text-lines.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/equal-width-text-lines.md @@ -3,7 +3,7 @@ Poster-style lockups where each line is scaled so it spans the container's full width can create visually striking headings. Using `text-fit`, the browser scales each line during layout, so it stays correct on resize, copy changes, and font load. -For a single line filling its container (e.g. a heading or pull quote), see `fit-text-to-container` (via `npx -y modern-web-guidance@latest retrieve "fit-text-to-container"`) instead. +For a single line filling its container (e.g. a heading or pull quote), apply `text-fit` to that element directly; this guide covers multi-line lockups. ## Basic implementation diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/visually-texture-content.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/visually-texture-content.md index 92eeffad..4bbe1d26 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/visually-texture-content.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/visual-design/visually-texture-content.md @@ -55,7 +55,8 @@ By default, CSS masks use `mask-mode: match-source`. This means the browser auto | **Direct Image File** (e.g. PNG, SVG file) | `alpha` | Opacity is determined by transparency. **Opaque** parts reveal content, and **transparent** parts hide it. | | **CSS Gradient** | `alpha` | Opacity is determined by transparency. **Opaque** colors (like `black`) reveal content, and **transparent** colors hide it. | -> **Note:** You can explicitly override the default mask mode using the `mask-mode` CSS property (e.g., `mask-mode: luminance;` or `mask-mode: alpha;`). +> [!NOTE] +> You can explicitly override the default mask mode using the `mask-mode` CSS property (e.g., `mask-mode: luminance;` or `mask-mode: alpha;`). ## Fallback strategies Baseline status for Masks: Widely available. It's been Baseline since 2023-12-07. diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/wasm/cpp-on-the-web.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/wasm/cpp-on-the-web.md index ebb90f01..b42460a7 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/wasm/cpp-on-the-web.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/wasm/cpp-on-the-web.md @@ -1,73 +1,186 @@ # Compiling C and C++ to WebAssembly with Emscripten -Use Emscripten to compile C and C++ into high-performance WebAssembly (Wasm) -modules for the browser. +Use Emscripten (`emcc` and `em++`) to compile C and C++ into high-performance +WebAssembly (Wasm) modules for the browser. + +## 1. Recommended Emscripten Compilation Flags + +When compiling and linking with `emcc` or `em++`, use modern modular flags: + +- `-sSTRICT`: Opt into strict modern Emscripten behavior and disable deprecated + or legacy compatibility features. +- `-sEXPORT_ES6`: Emit a standard ES6 module (`import`/`export`). This option + automatically implies `-sMODULARIZE`, encapsulating generated code so it does + not pollute the global namespace. +- `-sENVIRONMENT=web` (or `-sENVIRONMENT=worker` when running solely inside a + Web Worker, or `-sENVIRONMENT=web,worker` for both): Strip Node.js and shell + runtime support code to minimize bundle size. +- `-sALLOW_MEMORY_GROWTH`: Allow the WebAssembly linear memory heap to expand + dynamically when allocations exceed the initial heap size. +- `-Werror -Wall`: Treat compiler warnings as errors to catch C/C++ bugs and + invalid compiler settings early. +- `-O3`, `-Os`, or `-Oz` with `-flto`: In release builds, choose `-O3` when + runtime performance is most critical, or `-Os` / `-Oz` when minimizing binary + payload size is the priority. Pass `-flto` during both compilation and + linking for Link-Time Optimization. + +### Command-Line Style + +Keep `emcc` and `em++` invocations clean, minimal, and consistent: + +- **Omit settings that are enabled by default**: Do not pass redundant flags + that are already default in modern Emscripten, such as `-sWASM`, or + `-sMODULARIZE` when `-sEXPORT_ES6` is already specified. +- **Omit the `=1` suffix on boolean `-s` flags**: Write `-sSTRICT`, + `-sEXPORT_ES6`, and `-sALLOW_MEMORY_GROWTH` rather than `-sSTRICT=1`, + `-sEXPORT_ES6=1`, or `-sALLOW_MEMORY_GROWTH=1`. +- **Prefer standard compiler flags over Emscripten-specific ones**: Use + `-pthread` instead of `-sUSE_PTHREADS`, and `-m64` instead of `-sMEMORY64`. +- **Use simple comma-separated lists**: For list-based settings, pass + comma-separated values without brackets or quotes (for example, + `-sEXPORTED_FUNCTIONS=_render,_malloc,_free` and + `-sEXPORTED_RUNTIME_METHODS=HEAPU8`) rather than JSON arrays like + `-sEXPORTED_FUNCTIONS=['_render','_malloc','_free']`. +- **Use response files for long lists**: If an export list is large, pass a + response file (`@filename`) instead of a long inline list (for example, + `-sEXPORTED_FUNCTIONS=@exported_funcs.txt`). +- **Use separate compilation for multi-file projects**: Compile `.c` / `.cpp` + sources to `.o` object files (`em++ -c`) before linking them into the final + module rather than combining everything into a single monolithic invocation. -## 1. Modern Emscripten Compilation Flags - -When compiling and linking with `emcc` or `em++`, always use modern modular -flags: - -- `-sEXPORT_ES6`: Emits standard ES6 module syntax (`import`/`export`) rather - than legacy global objects. This automatically sets `-sMODULARIZE`. -- `-sENVIRONMENT=web`: Strips Node.js/shell runtime code to minimize bundle - size. -- `-sSTRICT`: Enforces modern Emscripten defaults and fails on deprecated - flags. -- `-sALLOW_MEMORY_GROWTH`: Allows the WebAssembly linear memory heap to expand - dynamically when allocating memory. -- `-Oz` (or `-O3`) + `-flto`: Optimizes for minimal payload size in release builds. - -## 2. Avoid blocking the main thread - -If a call into WebAssembly is likely to take a significant amount of time (i.e. -more than a few milliseconds) it should **not** be called on the main browser -thread. - -To avoid running such code on the main browser thread there are several possible -approaches: - -1. Run the whole WebAssembly program in a Web Worker. This is often the - simplest solution since it doesn't involve setting up SharedArrayBuffer - access, or changing the program source. - -2. Use pthreads or Wasm Workers. This allows the work to be shared across - many workers but requires SharedArrayBuffer access which can involve adding - COOP/COEP HTTP headers. - -3. Refactor the C/C++ code to avoid long running calls. For example slicing - the work into small pieces that can be called from `requestIdleCallback`. - -## 3. Pay attention for runtime warnings and errors - -Make sure you actually test the resulting WebAssembly program and that no -warnings or errors are generated on the console. - -Build and run the programs with `-sASSERTIONS` enabled and fix any runtime -failures that are produced. - -## 4. Avoid dynamic linking if possible - -Emscripten works best with static libraries (`.a`). Shared libraries (`.so`, -`.dylib`, `.dll`) are less useful in browser deployment and can complicate -things unnecessarily. +```bash +# Compile C++ source to an object file +em++ -c renderer.cpp -o renderer.o -O3 -flto -Werror -Wall + +# Link into an ES6 WebAssembly module targeting a Web Worker +em++ renderer.o -o renderer.js \ + -O3 -flto \ + -sSTRICT \ + -sEXPORT_ES6 \ + -sENVIRONMENT=worker \ + -sALLOW_MEMORY_GROWTH \ + -sEXPORTED_FUNCTIONS=_render_frame,_malloc,_free \ + -sEXPORTED_RUNTIME_METHODS=HEAPU8 +``` -## 5. Porting Existing C/C++ Libraries +### Debug vs. Release Profiles + +Keep compile and link flags consistent within each build profile, and never +ship debug information in production: + +- **Release builds**: Use `-O3`, `-Os`, or `-Oz` combined with `-flto` on both + compile and link steps, and omit `-g` and `-sASSERTIONS`. Add `--closure=1` + when linking to minify the generated JavaScript glue code unless an external + bundler already minifies it. +- **Debug builds**: Use `-g` when compiling and `-g`, `-gline-tables-only`, or + `-gsource-map` when linking, alongside `-sASSERTIONS`. Avoid optimization + flags (`-O2`, `-O3`, `-Os`, `-Oz`) and `-flto` in debug builds for faster + compilation and accurate source-level debugging. + +## 2. JavaScript-WebAssembly Boundary Efficiency + +Avoid high-frequency calls across the JavaScript-WebAssembly boundary. Calling +an exported Wasm function per pixel, per audio sample, or per vertex incurs +boundary overhead that negates Wasm performance gains. + +Instead, design C/C++ entry points to process entire buffers or memory regions +in a single call and read or write the contiguous buffer in linear memory: + +```js +import initRenderer from './renderer.js'; + +const wasm = await initRenderer(); +const byteLength = width * height * 4; +const ptr = wasm._malloc(byteLength); + +try { + wasm._render_frame(ptr, width, height, samples); + const pixels = new Uint8ClampedArray( + wasm.HEAPU8.buffer, + ptr, + byteLength, + ).slice(); + self.postMessage({ pixels, width, height }, [pixels.buffer]); +} finally { + wasm._free(ptr); +} +``` -When integrating existing C/C++ libraries with standard build systems: +Note: Accessing `wasm.HEAPU8.buffer` directly after calls that may grow memory +requires reading `wasm.HEAPU8` fresh after the Wasm call returns, because +`-sALLOW_MEMORY_GROWTH` detaches the previous `ArrayBuffer` when resizing. + +## 3. Avoid Blocking the Main Thread + +The browser relies on cooperative multitasking on the main UI thread. Any +WebAssembly call that takes more than a few milliseconds (such as image +rendering, ray tracing, physics simulation, or heavy parsing) **must not** run +synchronously on the main thread, or it will freeze page rendering and input. + +Use one of these approaches to keep the UI responsive: + +1. **Run the WebAssembly module in a Web Worker**: Instantiate and call the + ES6 Wasm module inside a dedicated module Worker + (`new Worker(new URL('./worker.js', import.meta.url), { type: 'module' })`), + and pass rendered buffers back via `postMessage` with transferable + `ArrayBuffer` objects. +2. **Use multithreading (`-pthread`)**: Offload heavy compute across background + workers using standard POSIX threads. Note that `-pthread` requires + `SharedArrayBuffer`, which requires serving the page in a cross-origin + isolated context (`Cross-Origin-Opener-Policy: same-origin` and + `Cross-Origin-Embedder-Policy: require-corp`). +3. **Yield to the browser event loop**: Restructure continuous loops in C/C++ + to run one frame at a time via `emscripten_set_main_loop`, or use + JavaScript Promise Integration (`-sJSPI`) / Asyncify (`-sASYNCIFY`) when + synchronous C/C++ code must pause for asynchronous JavaScript APIs. + +## 4. Test Runtime Execution and Memory Limits + +Always run and verify the compiled WebAssembly module in a real browser +environment and confirm that no warnings or unhandled errors appear in the +console: + +- **Validate with `-sASSERTIONS` during development**: Build and test with + `-sASSERTIONS` enabled to catch invalid memory accesses, missing exports, or + ABI mismatches early, then remove `-sASSERTIONS` for the final release build. +- **Watch for stack overflows in recursive C/C++ code**: Emscripten's default + execution stack size (`-sSTACK_SIZE`) is 64 KB, which is much smaller than + native OS stacks. Deeply recursive algorithms (such as recursive ray tracers + or tree traversals) can overflow the stack and corrupt linear memory unless + converted to an iterative loop or linked with a larger stack (for example, + `-sSTACK_SIZE=1MB`). + +## 5. Avoid Dynamic Linking if Possible + +Emscripten works best with static linking and static archive libraries (`.a`). +Shared libraries (`.so`, `.dylib`, `.dll`) and dynamic linking (`-sMAIN_MODULE` +/ `-sSIDE_MODULE`) add code-size and runtime overhead and should be avoided +unless runtime plugin loading is strictly required. + +## 6. Porting Existing C/C++ Libraries and Build Integration + +When integrating C/C++ compilation into a web application project, wire the +`emcc` / `em++` build into the project's build pipeline (such as the `"build"` +script in `package.json`) so that building the project compiles the WebAssembly +artifacts before bundling. + +When building third-party C/C++ libraries with existing build systems: ### CMake Projects -Wrap CMake commands with `emcmake` and `emmake`: +Wrap `cmake` and `make` with `emcmake` and `emmake`: + ```bash -mkdir build && cd build +mkdir -p build && cd build emcmake cmake .. -DCMAKE_BUILD_TYPE=Release emmake make ``` ### Autoconf / Make Projects -Use `emconfigure` to set cross-compilation toolchain targets: +Use `emconfigure` to configure cross-compilation toolchain targets and disable +shared libraries: + ```bash emconfigure ./configure --host=wasm32-unknown-emscripten --disable-shared emmake make diff --git a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/webmcp/webmcp.md b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/webmcp/webmcp.md index 93f36785..f8e7c70e 100644 --- a/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/webmcp/webmcp.md +++ b/plugins/modern-web-guidance/.agents/skills/modern-web-guidance/guides/webmcp/webmcp.md @@ -1,4 +1,4 @@ -# WebMCP (Web Model Context Protocol) +# WebMCP WebMCP is a browser-native JavaScript API that allows web pages to expose their client-side functionality as structured "tools" to AI agents, browser assistants, and assistive technologies. diff --git a/plugins/modern-web-guidance/.claude/skills/modern-web-guidance b/plugins/modern-web-guidance/.claude/skills/modern-web-guidance new file mode 120000 index 00000000..5c255488 --- /dev/null +++ b/plugins/modern-web-guidance/.claude/skills/modern-web-guidance @@ -0,0 +1 @@ +../../.agents/skills/modern-web-guidance \ No newline at end of file diff --git a/plugins/modern-web-guidance/agent/skills/modern-web-guidance/SKILL.md b/plugins/modern-web-guidance/agent/skills/modern-web-guidance/SKILL.md new file mode 100644 index 00000000..e08e53a1 --- /dev/null +++ b/plugins/modern-web-guidance/agent/skills/modern-web-guidance/SKILL.md @@ -0,0 +1,83 @@ +--- +name: modern-web-guidance +description: > + Search tool for modern web development best practices. MANDATORY: Execute + FIRST for all HTML/CSS and clientside JS tasks. Do NOT skip — web APIs evolve + rapidly and training weights contain obsolete patterns. + + Trigger immediately for: modals/dialogs/popovers, tables, dark mode, anchor + positioning, container queries, view transitions, scroll-driven animations, + parallax/reveals, forms, validation, autofill, accessibility, custom selects, + scrollbars, CWV (LCP, INP, CLS), content-visibility, fetch priority, image + optimization, security, passkeys, HTML sanitization, web components, + dates/timezones, built-in AI, WebAssembly, standard HTML/CSS/DOM in + frameworks, etc. + + DO NOT trigger for: backend (SQL, ORMs, API routes), infra/scripts (CI/CD, + Docker, Actions, Python/Go, Git), or tooling/trivial (Vite, tsconfig, ESLint, + tests, copy/renames, 3P docs), etc. +--- +# Modern Web Guidance + +Search and retrieve standardized modern web best practices at the **start** of implementing or updating any web UI, CSS layout/animation, or client-side JS logic (dates/timezones, sanitization, performance, accessibility, UX fixes) before writing ad-hoc solutions or adding dependencies. + +## Usage Instructions + +### Step 1. Search Use Cases + +Run `modern-web-guidance` directly with `npx` using a **2–5 word action-oriented query** for a **single web platform capability or UI pattern**: +- **Keep queries focused and generic**: Describe the user-visible capability (`<verb> + <UI/platform object>`, optionally paired with the target HTML/CSS/JS feature name if known). Omit framework/library names (`React`, `Vue`, `Tailwind`, etc.) and app/domain nouns (product, route, or component/file names). Split multi-feature tasks into separate `search` calls. +- **Example**: Search `"light dismiss dialog"` or `"responsive data table"`, **not** `"dialog"` (too terse), `"responsive table dark mode"` (compound — split into two searches), or `"close React settings modal on outside click"` (framework/app noise). + +```sh +npx -y modern-web-guidance@latest search "<query>" --skill-version 2026_10_30-58d055df +``` + +> **Note**: Returns `[{ "id", "description" }, ...]`. Judge results by their `description` fields: +> - **Match found**: Call `retrieve` directly—do not also run `list`. +> - **No match on a web UI/CSS/DOM behavior (or broad UI audit)**: If `search` returns `[]` or unrelated descriptions because your phrasing differed from the catalog (or your task spans multiple categories), run `npx -y modern-web-guidance@latest list` once to check if the pattern is covered under more general terminology. +> - **Topic not covered**: If no guide `description` directly matches your task, proceed with your implementation without retry loops or retrieving unrelated guides. + +### Step 2. Retrieve Best Practices + +Once you have relevant `id`(s) from the search results, call `retrieve` to get the full guide(s)—including both specific use-case guides that match your task and any broader discipline guide (e.g., `html`, `css`, `forms`, `accessibility`, `color`, `performance`, `security`) that applies. You can pass multiple IDs separated by commas. + +```sh +npx -y modern-web-guidance@latest retrieve "<id>" +``` + +If `retrieve` saves large output (> 9.5 KB) to a temporary file to prevent truncation, read that file path in full using your file-reading tool (reading remaining line ranges if needed)—do not use `cat` or `head`. + +### Step 3. Verify Guidance Compliance + +When generating or modifying code, cross-check the implementation against the retrieved guide before concluding: +- **Applicable Guidance & Fallbacks**: Ensure the relevant modern patterns and necessary fallback strategies from the guide are correctly applied, without forcing unrequested features. +- **Task Fulfillment**: Confirm that the implementation fully satisfies the user's request. + +### Step 4. Disclose Skill Usage + +When you consult `modern-web-guidance` for a task, end the response where you deliver those changes with a `### Modern Web Guidance` section: +- **When retrieved guides shaped your code**: Write **one** bullet (1–2 sentences, ≤75 words) per newly applied `<id>` in first-person voice explaining the non-obvious implementation choices, CSS/API overrides, or fallbacks `<id>` required and the pitfall it prevented, anchored by a short verbatim quote of that specific rule or gotcha (never the guide's intro overview): + `- **\`<id>\`**: <Non-obvious APIs, overrides, or fallbacks applied and the pitfall prevented> ("<short verbatim quote of the specific rule or gotcha>").` + - **Never** guess `<id>`s or quotes from memory after context compaction (re-run `search`/`retrieve` if needed), and credit **only** guide-backed changes from the current turn. +- **When searched/listed in this turn but zero guides applied**: Write one first-person sentence naming which topics or guides you checked in `modern-web-guidance` and why none applied (omit if any guide applied). + +## Using npx / pnpx + +- **Runner & Permissions**: Prefer `pnpx` over `npx` if `pnpm` is available (`pnpx` does not use `-y`); on Windows, use `npx.cmd`. When requesting tool permissions, allowlist `npx -y modern-web-guidance@latest *` (or `pnpx modern-web-guidance@latest *`) specifically, never bare `npx *` or `pnpx *`. +- **Sandboxes & Network**: Fetching `modern-web-guidance` requires outbound network access. In sandboxed or approval-gated environments (e.g., Codex, Claude Code), **proactively request approval/allowlisting with network access BEFORE the first execution** to avoid timeouts, and set `NPM_CONFIG_CACHE=/tmp/npm-cache` if `~/.npm` is read-only. +- **Offline**: If the command hangs due to being offline, retry with `npx --offline …`. + +## Guidelines + +- Always search **first** to find the most relevant guides. Do not hallucinate guides or ignore them; they represent the preferred local standard for the user's project. +- These guides use vanilla HTML/CSS/JS; adapt them to your framework without dropping the modern feature: + - **Types**: Spread/cast newly shipped HTML attributes (`{...({ popover: 'hint', interestfor: id, commandfor: id, switch: '' } as any)}`, `[attr.*]`) rather than removing them or merging `@types/react`. + - **SSR & Bundlers**: Guard `window`/`HTMLElement` in SSR and load polyfills via bare `import('<pkg>')` in client effects (or set Vite `build.target: 'esnext'` for top-level `await`)—never copy files from `node_modules`. + - **DOM, IDs & CSS**: Call `showModal()`/`moveBefore()` via refs on non-VDOM-managed nodes, generate per-instance IDs/`anchor-name`s (`useId()`), and keep `:root` and `@starting-style` top-level/unscoped. + +## Interpreting Browser Support & Fallbacks + +* **Default Behavior**: All guides assume **Baseline Widely available** features are safe to use without fallbacks. For features that are not Baseline widely available, you **MUST** follow the fallback recommendations in the guide, unless the user has specified a custom browser support policy. +* **Custom Policies**: If the user defines explicit browser support requirements, use the guide's compatibility data to determine if a fallback can be ignored (for `Baseline YYYY` targets, a feature satisfies the target if its "Baseline since" date is `<= YYYY`). Examples: _"Do not implement feature fallbacks"_, _"Safari 17.4+"_, _"Never implement polyfills; use a lightweight custom fallback or redesign"_, or _"Allow Newly Available features if feature-detected with graceful degradation"_. +* **Reactive Policy Discovery**: Suggest documenting a policy in `CLAUDE.md` or `AGENTS.md` (e.g., `**Browser Support:** Allow Newly Available features, but only adopt custom fallback code that adds <= 20 lines and no external dependencies`) if the developer targets a restricted runtime (Electron/Tauri), excludes specific browsers, hesitates over polyfill size/complexity, or asks if a feature is safe without fallbacks. diff --git a/plugins/modern-web-guidance/agent/skills/modern-web-guidance/guides/accessibility/accessibility.md b/plugins/modern-web-guidance/agent/skills/modern-web-guidance/guides/accessibility/accessibility.md new file mode 100644 index 00000000..00cb44a2 --- /dev/null +++ b/plugins/modern-web-guidance/agent/skills/modern-web-guidance/guides/accessibility/accessibility.md @@ -0,0 +1,585 @@ +# Accessibility Coding Guidelines + +This guide provides actionable DOs and DON'Ts for AI coding agents to ensure web applications are accessible to all users, including those using assistive technologies. + +Keep these principles in mind throughout: + +- **Accessibility is the minimum, not the ceiling.** Conformance to standards is the floor; aim for genuine usability. +- **Patterns are use-case specific.** No checklist replaces real testing — including testing with disabled users — to confirm a given implementation is actually accessible in context. + +Contents: + +- [1. Content Navigability and Structure](#1-content-navigability-and-structure) + - [Landmarks](#landmarks) + - [Headings](#headings) + - [Lists](#lists) + - [Tables](#tables) + - [Skip links](#skip-links) + - [Code example: good use of landmarks, headings, lists, tables, and a skip link](#code-example-good-use-of-landmarks-headings-lists-tables-and-a-skip-link) +- [2. Semantic HTML and ARIA](#2-semantic-html-and-aria) + - [Prefer native HTML](#prefer-native-html) + - [ARIA roles and states](#aria-roles-and-states) +- [3. Accessible Names and Descriptions](#3-accessible-names-and-descriptions) + - [Naming mechanisms](#naming-mechanisms) + - [Choosing the name](#choosing-the-name) + - [Descriptions](#descriptions) + - [Code Example: Visually Hidden Utility](#code-example-visually-hidden-utility) +- [4. Document Metadata and Language](#4-document-metadata-and-language) + - [Language](#language) + - [Titles](#titles) + - [Iframes](#iframes) + - [Code Example: using the `<title>` and `<iframe>` elements](#code-example-using-the-title-and-iframe-elements) +- [5. Keyboard and Focus Management](#5-keyboard-and-focus-management) + - [Tab order and focusability](#tab-order-and-focusability) + - [Keyboard handlers](#keyboard-handlers) + - [Focus indicators](#focus-indicators) + - [Target sizes](#target-sizes) +- [6. Alternate Text and Media](#6-alternate-text-and-media) + - [Images](#images) + - [Inline SVGs](#inline-svgs) + - [Complex images and charts](#complex-images-and-charts) + - [Audio and video](#audio-and-video) + - [CSS generated content](#css-generated-content) + - [Content Visibility Decision Matrix](#content-visibility-decision-matrix) +- [7. Forms and Input Controls](#7-forms-and-input-controls) + - [Labels and hints](#labels-and-hints) + - [Input assistance and validation](#input-assistance-and-validation) +- [8. Live Regions](#8-live-regions) + - [Live Region Urgency Table](#live-region-urgency-table) + - [When to announce](#when-to-announce-live-regions) + - [Managing announcements](#managing-announcements) +- [9. Color](#9-color) +- [10. Typography](#10-typography) +- [11. Motion and Preferences](#11-motion-and-preferences) +- [12. Modals and Native Dialogs](#12-modals-and-native-dialogs) + - [Native `<dialog>`](#native-dialog) + - [Custom overlays](#custom-overlays) +- [13. Testing Validations](#13-testing-validations) + - [Automated checks](#automated-checks) + - [Manual testing](#manual-testing) + +## 1. Content Navigability and Structure + +### Landmarks + +- **Place all content within landmarks:** Wrap each relevant part of the page in `<header>`, `<nav>`, `<main>`, `<aside>`, and `<footer>` elements so assistive-tech users can jump between regions. (1.3.6 Identify Purpose (Level AAA)) +- **Use the correct landmark:** Use the `<header>` element as a container for introductory content, the `<nav>` element as a container for site-wide or in-page navigation links, the `<main>` element as a container for the main content of the page, the `<aside>` element as a container for content that is tangentially related to the main content, and the `<footer>` element as a container for information about the related content, such as the author, copyright information, and links to related documents. (1.3.1 Info and Relationships (Level A)) +- **Use one `<main>` landmark per page:** Avoid using multiple `<main>` elements on a single page. (1.3.1 Info and Relationships (Level A)) +- **Provide distinguishable accessible names for navigation regions:** If a page contains multiple `<nav>` elements, apply the `aria-label` attribute to each element and set its value to a distinguishable accessible name for the region. For example: `<nav aria-label="Primary">`, `<nav aria-label="Secondary">`, and so on. (Best practice) +- **Don't overuse landmarks:** Too many landmarks dilute their value. In particular, avoid labeling a `<section>` (which turns it into a `region` landmark) — `region` should be a last resort when no other landmark fits. (Best practice) + +### Headings + +- **Structure main content with headings:** Use `<h1>`–`<h6>` elements sequentially so screen reader users get a navigable outline. (1.3.1 Info and Relationships (Level A)) +- **Mandatory - Don't use "fake" headings:** Never use `<div>` or `<span>` elements to look like headings without standard `<h1>`–`<h6>` elements. (1.3.1 Info and Relationships (Level A)) +- **Avoid nesting headings incorrectly:** Make sure that, if a sub-heading is visually less important than its parent heading, its heading level is lower in the heading hierarchy. (1.3.1 Info and Relationships (Level A)) +- **Use one `<h1>` element per page:** Use the `<h1>` element for the page's main heading. Having more than one `<h1>` element on a page makes it difficult for screen reader users to identify the actual main heading. (Best Practice) + - **Caveat:** If a page includes a modal dialog that is marked up using the `<dialog>` element, `role="dialog"`, or `role="alertdialog"`, it is fine to mark up its main heading using the `<h1>` element because the dialog is a new window that is separate from the page. +- **Avoid "skipped" heading levels:** Avoid jumping directly from (for example) `<h1>` → `<h4>` to avoid breaking the logical hierarchy. (Best Practice) +- **Avoid placing headings inside the `<summary>` element:** Some browsers still expose this element as a button, which removes all roles from its children including headings. If a heading must act as a disclosure trigger, use an accordion or a disclosure implemented with ARIA where the heading element wraps the button, rather than use the `<details>` and `<summary>` elements. (Best Practice) + +### Lists + +- **Use lists for repeated, contiguous content:** `<ul>`/`<ol>` give assistive tech a count of listitems up front and let users skip the entire list. (1.3.1 Info and Relationships (Level A)) +- **Use the correct list element:** Use the `<ul>` element for unordered lists where the order of the items is **not** important (such as a list of related products), and use the `<ol>` element for ordered lists where the order of the content **is** important (such as a list of instructions). (1.3.1 Info and Relationships (Level A)) +- **Don't use lists for non-list content:** Do not use list markup to mark up separate paragraphs of text, as screen readers will announce list information when they encounter lists, therefore leading to a screen reader hearing potentially confusing and redundant information about the related content. (1.3.1 Info and Relationships (Level A)) +- **Don't use lists for checkboxes and radio buttons:** When marked up correctly, screen readers will already announce the number of checkboxes or radio buttons, as well as the number of the currently focused checkbox / radio button in the set, even though there is no list markup. Additional list markup will lead to duplication. (1.3.1 Info and Relationships (Level A)) + +### Tables + +- **Do use the correct table elements:** Use the `<table>` element to identify the table, the `<caption>` element to provide a title and accessible name for the description, the `<tr>` element to identify individual rows in the table, and the `<th>` and `<td>` elements to identify header and data cells respectively. (1.3.1 Info and Relationships (Level A)) +- **Do use the `scope` attribute on `<th>` elements:** The `scope` attribute indicates whether the `<th>` element represents a row header (`<th scope="row">`) or a column header (`<th scope="col">`). (1.3.1 Info and Relationships (Level A)) +- **Mandatory - Don't use tables for layout:** Screen readers will announce redundant information about the layout, which may be confusing for screen reader users, while the use of tabular markup has an impact on the reading and keyboard focus order of content. Use CSS Grid/Flexbox for visual layouts instead. (1.3.1 Info and Relationships (Level A)) +- **Avoid overly-complex tables:** Spanned rows, spanned columns, nested tables, and expandable / collapsible rows can make sense visually and meet the related WCAG success criterion, but the overall structure can be disorienting and difficult to navigate for screen reader users. In such cases, break tables up into smaller tables that are easier and more efficient to navigate. (Best Practice) + +### Skip links + +- **Provide skip links** prior to repeated content like site headers with navigation or long/infinite lists, so that keyboard users can easily bypass them. (2.4.1 Bypass Blocks (Level A)) + - **Caveat:** Sometimes screen magnifiers do not follow the focus when a skip link’s target is not normally focusable. + The recommended workaround is to make the target programmatically focusable, but not a part of the tab order, using `tabindex="-1"`. + If the target contains focusable elements, this creates another usability issue as clicking within the target will always set the **sequential focus navigation start point** to the target rather than at a point between focusable elements. + A solution that satisfies both user expectations is to make the first meaningful element within the intended target the actual target of the skip link with `tabindex="-1"`, for example a heading (`<h1 id="content" tabindex="-1">`). + Beware that any content prior to such an element will be skipped, including other relevant content within the targeted section, and this creates usability gaps for other assistive tech users. +- **Avoid using skip links where they are not needed:** A set of pages whose only common element is a single link to the Home page does not need a skip link. (Best Practice) +- **Avoid overusing skip links:** A single skip link, or two skip links to cover the primary and secondary sets of common navigation links, is usually sufficient on a single page. Multiple unnecessary skip links add to the number of <kbd>Tab</kbd> stops to a page, which causes frustration for people who use the keyboard alone for navigation. (Best Practice) + +```html +<!-- Good: Skip to main content --> +<a href="#content" class="skip-link">Skip to main content</a> +<main id="content">...</main> +``` + +### Code Example: good use of landmarks, headings, lists, tables, and a skip link + +```html +<!-- Good: Semantic landmarks, heading hierarchy, skip link --> +<header> + <a href="#platform-dashboard" class="skip-link visually-hidden">Skip to content</a> + <nav aria-label="Primary"> + <ul> + <li><a href="/">Home</a></li> + </ul> + </nav> +</header> +<main> + <h1 id="platform-dashboard" tabindex="-1">Platform Dashboard</h1> + <section> + <h2>User Statistics</h2> + <table> + <caption>Monthly active users</caption> + <thead> + <tr> + <th scope="col">Month</th> + <th scope="col">Users</th> + </tr> + </thead> + <tbody> + <tr> + <td>January</td> + <td>12,000</td> + </tr> + </tbody> + </table> + <h3>Notes</h3> + <ul> + <li>Results do not include users without active subscriptions</li> + <li>Results are accurate as of 31 December 2025</li> + </ul> + </section> +</main> +``` + +## 2. Semantic HTML and ARIA + +### Prefer native HTML + +- **Prefer HTML elements and attributes to ARIA:** Avoid `<div role="button">` or `<a role="button">` and use `<button>` instead. Native HTML elements and attributes already have the correct role and behavior, including keyboard support, built-in by default. `<button>` already implies `role="button"`, `required` already implies `aria-required`, and no additional JavaScript is required to ensure the correct role and behavior is supported. (Best Practice) + - **Caveat:** ARIA is required when the related feature is not available in HTML, for example there are no HTML elements explicitly intended for building a tabbed navigation widget, but ARIA includes appropriate role and state properties for this purpose. +- **Don't add redundant ARIA roles or properties:** Avoid `<ul role="list">`, `<nav role="navigation">`, and `<input required aria-required="true">`. (Best Practice) + - **Caveat:** Safari removes list semantics from `<ul>`/`<ol>` outside `<nav>` when `list-style: none` or `display: flex`/`grid` is applied. In that case `role="list"` is required to restore them. +- **Don't apply interactive ARIA roles to structural HTML elements:** Use `<li><span role="button">Button in a list</span></li>`, not `<li role="button">Button in a list</li>`, to avoid breaking the container list semantics. (4.1.2 Name, Role, Value (Level A)) + - **Caveat:** It is acceptable to apply ARIA roles to structural HTML elements that have similar semantics, for example a tree view: `<ul role="tree"><li role="treeitem">…</li>…</ul>`. + +### ARIA roles and states + +- **Match ARIA implementations to actual behavior:** When applying `role="tab"` to an element, it must then behave like a tab — including keyboard interactions. Many ARIA patterns can't be implemented in CSS alone and need JavaScript. (4.1.2 Name, Role, Value (Level A)) +- **Manage toggle states:** Utilize `aria-expanded` and `aria-pressed` to communicate toggle states for custom controls. (4.1.2 Name, Role, Value (Level A)) +- **Be deliberate about `disabled` vs `aria-disabled`:** `disabled` removes the element from the focus order entirely (and `tabindex="0"` won't bring it back), which is often wrong for toolbar buttons or links. `aria-disabled="true"` keeps the element focusable so users can land on it and learn it's disabled, but because it only communicates state to assistive technologies, you must also prevent activation (such as pointer clicks and keyboard triggers) in JavaScript. (4.1.2 Name, Role, Value (Level A)) +- **Mandatory - Don't hide interactive elements from screen readers:** Do not apply `aria-hidden="true"` to interactive elements that receive focus, or to their parent elements. (4.1.2 Name, Role, Value (Level A)) +- **Mandatory - Don't suppress the roles of interactive elements:** Applying `role="presentation"` or `role="none"` to an interactive element strips its semantics. The element will still be exposed to assistive technologies and announced by screen readers, but its role and any state information will not be, which results in confusion and potentially unexpected behavior. (4.1.2 Name, Role, Value (Level A)) +- **Don't assume custom elements have no ARIA:** Custom elements can attach ARIA via `ElementInternals`, which some automated test tools can't see — so the absence of `role`/`aria-*` attributes in markup doesn't prove the element has no semantics. Verify with the browser's accessibility-tree inspector. (Best Practice) + +## 3. Accessible Names and Descriptions + +Every interactive element and some landmarks need an accessible name, and many benefit from an accessible description. Names are short and identify the element; descriptions add context. + +### Naming mechanisms + +- **Prefer native naming mechanisms:** `<label>` for form controls, `<caption>` for `<table>`, `<legend>` for `<fieldset>`, `<figcaption>` for `<figure>`. (Best Practice) +- **Explicitly associate `<label>` with its control via `for`/`id`:** As well as ensuring the control is appropriately labeled by the contents of the `<label>` element, this results in a larger target area for people to activate the label _or_ its related control to move focus to the control, which isn't globally supported with other forms of labeling (such as using the `aria-labelledby` attribute). Additionally, some assistive technologies, particularly voice recognition software, require the `for`/`id` relationship to be explicitly set, even if an equivalent accessibility-supported approach (such as nesting an `<input>` inside a `<label>`) is taken. (4.1.2 Name, Role, Value (Level A)) +- **Prefer `aria-labelledby` over `aria-label` when a visible label exists:** This avoids duplication, improves maintainability, and translates better. (Best Practice) +- **Don't use `title` as a naming mechanism:** The value of this attribute is only displayed on mouse hover, and cannot be displayed or otherwise accessed using the keyboard in any browser. (Best Practice) + - **Caveat:** `<iframe>` is an exception, as the `title` attribute is the standard way of providing an accessible name for this element. +- **Don't use `placeholder` as a naming mechanism:** While the value of this attribute is initially displayed inside an `<input>` or `<textarea>`, it disappears as soon as the user enters text into the field, meaning that there is no longer a visual cue as to what the field represents. The contrast between `placeholder` text and its background in most browsers is very low, and therefore is difficult for people to perceive. (3.3.2 Labels or Instructions (Level A)) +- **Don't put `aria-label`/`aria-labelledby` on elements that shouldn't be named:** Examples include plain `<div>`, `<span>`, and other custom elements without a role. These attributes are only intended for providing accessible names for elements that have explicit or implicit widget (for example, button, link, and tab), landmark, image, or iframe roles. (1.3.1 Info and Relationships (Level A)) + - **Caveat:** Custom elements may have an implicit role set via `ElementInternals`, so the absence of a `role` attribute isn't conclusive. + +### Choosing the name + +- **Avoid inconsistent link text:** If multiple hyperlinks direct users to the same target page, make sure each link has the same accessible name, which should typically be the same as its visible link text. For example, if the target page for a link is `profile.html`, make sure that all links have an accessible name of "My Profile" **or** "Profile"; do not use "My Profile" in one instance, and "Profile" in the other. (Best Practice) +- **Disambiguate controls that look visually identical but which perform different actions by their accessible name:** For example, a page may include multiple "Edit" buttons, each of which triggers functionality to perform a different action, such as edit the user's name, edit the user's password, and edit the user's address. While it may be visually obvious from the surrounding content what the purpose of these buttons might be, screen reader users may only hear "Edit, button". Techniques to resolve this include the following: (4.1.2 Name, Role, Value (Level A)) + - **Use visually hidden text:** The text is included in the DOM but is visually hidden using CSS. For example `<button>Edit <span class="visually-hidden">first name</span></button>`. Refer to the [code example: visually hidden utility](#code-example-visually-hidden-utility) for the full CSS. + - **Apply the `aria-label` attribute to the control:** The value of this attribute should then include the visible text as well as important contextual information, for example `<button aria-label="Edit name">Edit</button>`, `<button aria-label="Edit address">Edit</button>`, and so on. + - **Use the `aria-labelledby` or `aria-describedby` attribute to associate the control with the content it relates to:** Chaining IDs in `aria-labelledby` appends the related text to the control's accessible name (for example, `<p id="firstName">First name</p> <button id="editFirstName" aria-labelledby="editFirstName firstName">Edit</button>`), whereas `aria-describedby` provides the context as an accessible description (`<p id="firstName">First name</p> <button aria-describedby="firstName">Edit</button>`). +- **Do include a control's visible text in its accessible name:** Any visible text used to label a control must be included exactly as written in the control's accessible name. The accessible name can include additional information before or after the visible text, but the visible text itself must not differ. For example, the accessible name for a link containing the link phrase "Read more" can be "Read more about [subject]", because the visible text is included without modification. However, the accessible name cannot be "Read some more about [subject]", because the term "some" appears between the words "Read" and "more", which is not reflected in the visible text. (2.5.3 Label in Name (Level A)) +- **Don't reuse an accessible name across controls with different effects in the same view:** Make sure that each accessible name is disambiguated. (4.1.2 Name, Role, Value (Level A)) + - **Caveat:** Having two separate buttons with the accessible name of "Close" in two separate dialog boxes is fine, because only one dialog box is reachable at a time; however, multiple "Edit" buttons for different effects in the same dialog box is not. +- **Don't reuse an accessible name across hyperlinks that point to different `href` targets:** For example, avoid `<a href="annualreport2025.html">Annual Report</a>` and `<a href="annualreport2026.html">Annual Report</a>`. Use `<a href="annualreport2025.html">Annual Report 2025</a>` and `<a href="annualreport2026.html">Annual Report 2026</a>` (or an equivalent technique for disambiguating controls) instead. +- **Don't pack descriptions, error messages, or instructions into the label:** Verbose labels confuse and frustrate people who use screen readers. Instead, use the `aria-describedby` attribute to programmatically associate any additional content with the field. For example: `<label for="yourComments">Your comments</label><p id="yourCommentsWarning">Please do not include any personal information in this field.</p><textarea id="yourComments" aria-describedby="yourCommentsWarning"></textarea>`. Screen readers will then announce the label first, followed by the control type, and only then announce additional information that a user can skip if they are already aware of it. (Best Practice) +- **Don't repeat state information already exposed by ARIA inside the label:** Examples include information exposed via the `aria-expanded`, `aria-checked`, `aria-selected`, `aria-pressed`, and similar attributes. This creates duplication, redundancy, and ambiguity. (Best Practice) +- **Don't include the role name in the label:** This will lead to duplication. For example, `<nav aria-label="Primary navigation">` will be read by screen readers as "Primary navigation, navigation". (Best Practice) + +### Descriptions + +- **Identify instructions, error messages, and any other important information explicitly related to a control as a description:** Examples of descriptions include: (1.3.1 Info and Relationships (Level A)) + - A list of password requirements when creating an account + - An error message related to a field in a previously submitted form + - Additional instructions, such as "You must agree to the terms and conditions before you can submit the form" +- **Make sure the description is available to assistive technologies when the related element receives focus:** Take one of the following approaches (1.3.1 Info and Relationships (Level A)): + - If the description is already visible on the page, programmatically associate it with the element using the `aria-describedby` attribute. + - If the description is not visible on the page and is explicitly for screen reader users (in other words, when providing a visible description is not appropriate or useful), apply the `aria-description` attribute to the element and use it to provide an appropriate description. +- **Avoid including a description for every element:** if the primary label already provides sufficient information (such as "First name"), any additional description is likely to be redundant. (Best Practice) + - **Caveat:** Inline error messages associated with controls, such as error messages below / above text fields, as well as secondary labels that should be announced when the element receives focus, are exceptions. +- **Avoid including information in the description that is already provided by attributes applied to the element:** For example, including "This is a required field" as a description for an `<input>` element that already includes the `required` attribute will lead to duplicate announcements. (Best Practice) + - **Caveat:** `aria-invalid="true"` is an exception; many screen readers announce "invalid" when they reach an `<input>` that includes this attribute set to `true`, which indicates that the field is in error. However, it does not indicate what the error is. In such cases, a description (the actual human readable error message) is required. This attribute should only be applied and set to true when the error message is present. +- **Avoid including interactive elements in an `aria-describedby` target** unless the content of the element reads sensibly as a description on its own. For example: (Best Practice) + - This is acceptable, as the entire text string makes sense when announced: `<p id="description">I understand the <a href="tandc.html">terms and conditions</a>.</p>` + - This is not appropriate, as the description will be announced as "Please don't include personal information in this field. Learn more.", where "Learn more" will be announced out of its intended context: `<p id="description">Please don't include personal information in this field. <a href="…">Learn more</a></p>`. Instead, only include the plain text in the referenced description: `<p><span id="description">Please don't include personal information in this field.</span> <a href="…">Learn more</a></p>`. + +### Code Example: Visually Hidden Utility + +A `.visually-hidden` utility provides text for screen readers without rendering it visually. It's commonly used for skip links, additional context on icon-only buttons, and supplementary labels. + +```css +/* Hides content visually but keeps it in the accessibility tree. + :focus-within / :active opt elements out — useful for skip links and + any focusable content wrapped in this class. */ +.visually-hidden:where(:not(:focus-within, :active)) { + position: absolute !important; + clip-path: inset(50%) !important; + overflow: hidden !important; + width: 1px !important; + height: 1px !important; + margin: -1px !important; + padding: 0 !important; + border: 0 !important; + white-space: nowrap !important; +} +``` + +When the hidden content is focusable (skip links, focus-receiving wrappers), the `:focus-within`/`:active` exception lets it become visible. Style the visible state per situation, e.g. a skip link to the main content typically wants fixed positioning at the top-left of the viewport so the rest of the page doesn't shift. + +## 4. Document Metadata and Language + +### Language + +- **Do declare the page's human language:** Always apply the `lang` attribute to the page's `<html>` element, and always set it to the primary human language of the page's content. For example, `<html lang="en">`. (3.1.1 Language of Page (Level A)) +- **Do declare changes in the page's human language:** If there is a language change on the page, apply the `lang` attribute to the parent/container element for the affected content, and set the value of this attribute to the shortcode for the language of the affected content. For example: `<p>In Germany, you may be greeted by someone saying <span lang="de">Guten Morgen</span>.</p>`. (3.1.2 Language of Parts (Level AA)) +- **Don't set the value of the `lang` attribute to a value that doesn't match the human language of the content:** This causes screen readers and other text-to-speech software to announce the content in a non-understandable manner. (3.1.1 Language of Page (Level A), 3.1.2 Language of Parts (Level AA)) + +### Titles + +- **Do include the `<title>` element in the `<head>` section for all pages:** This includes pages embedded into a document using an `<iframe>`. (2.4.2 Page Titled (Level A)) +- **Provide unique page titles:** Every page should have its own title that unambiguously reflects the page's purpose or content. (2.4.2 Page Titled (Level A)) +- **Front-load the page's `<title>` with unique content:** For example, `<title>Page Topic | Site Name`. (Best Practice) +- **Update single-page application (SPA) titles during page transitions:** For example, if the page title is `Contact Us` when a contact form is displayed, update the title to `Thank you for contacting us!` when the form is successfully submitted. (2.4.2 Page Titled (Level A)) +- **Don't include an empty `` element in the `<head>` section of a page:** No matter how short or inconsequential a page appears, do not use `<title>`. Page titles appear in the browser tab, so pages are distinguishable when users have multiple tabs open, and are typically announced by screen readers when the page is opened. (2.4.2 Page Titled (Level A)) +- **Avoid using the same title for multiple pages, or using ambiguous page titles:** In such cases, pages are indistinguishable without reviewing the actual page content itself. (2.4.2 Page Titled (Level A)) + +### Iframes + +- **Do provide a descriptive `title` for ` + + +``` + +## 5. Keyboard and Focus Management + +### Tab order and focusability + +- **Provide a logical tab order:** Make sure that the order in which elements receive focus follows an appropriate sequence. (2.4.3 Focus Order (Level A)) + - **Caveat:** In left-to-right reading languages, the tab order is logical when focus moves from top-to-bottom and left-to-right. However, in right-to-left reading languages, the tab order is logical when focus moves from top-to-bottom and right-to-left. +- **Use `tabindex` only when necessary:** Apply `tabindex="0"` to elements requiring focus where an appropriate interactive role has been applied but the element isn't focusable by default; for example, `Print` (2.4.3 Focus Order (Level A)) +- **Manage focus correctly to and from updated content:** For example, if a dialog box appears on the page, make sure that focus immediately moves to the dialog box, is restricted to the contents of the dialog box, and (when dismissed) returns to the element that originally triggered the dialog box. (2.4.3 Focus Order (Level A)) +- **Don't apply `tabindex="0"` to interactive elements that are already focusable by default:** Native elements like the `