Skip to content

docs(bb-agent, bb-knowledge-base): errors and options as tables, one createChat wiring, corrected error cases - #1637

Draft
xyos wants to merge 6 commits into
aws:mainfrom
xyos:docs/readme-agent-knowledge-base
Draft

xyos wants to merge 6 commits into
aws:mainfrom
xyos:docs/readme-agent-knowledge-base

Conversation

@xyos

@xyos xyos commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem

  • The Agent README was 1,390 lines with 28 prose lines over 400 characters; the constructor options were 174 lines into ## API and the errors 408.
  • The createChat example called six api.agent* methods that no backend in the README defined, and its subscribe callback passed a value that is a handler or an options object straight to channel.subscribe(handler).
  • The InvalidModelConfig row listed "Missing modelId, apiKey, unknown provider" with no condition; two of those never surface as InvalidModelConfig, and four other cases were missing.
  • Two of five searchDocs examples returned retrieved content without untrustedOutput: true and without a filter from the tool context.
  • The KnowledgeBase README gave chunkOverlap as "0–100" in one place and "1–99" in another.

Issue #, if available:

Changes

This builds on commit 91c0004a1, which moves the seven Model Configuration subsections verbatim into packages/bb-agent/MODELS.md (already listed in files of package.json), names createChat for subscribe-before-send ordering, and adds trustable with interrupt and guardrail on a non-bedrock provider to the InvalidModelConfig row. It also made the useChat samples compile under strict; they are removed here.

packages/bb-agent/README.md

Below, src/ is packages/bb-agent/src/. Corrections:

  • The backend ApiNamespace is shown with the method names the client calls (agentCreateConversationId, agentGetChannel, agentStream, agentResume, agentGetConversation, agentGetPendingInterrupts). Shapes: src/index.chat.ts:58-79, src/transport.ts:181-198. The subscribe callback narrows with typeof (src/transport.ts:164-167, packages/bb-realtime/src/types.ts:79-80).
  • The backend sample defines agentGetRawDescriptor, the method the refresh rule names, with the same auth.requireAuth(context) as agentGetChannel; the frontend sample passes it to refresh, as test-apps/comprehensive/src/index.ts:356 does. Shapes: src/index.chat.ts:113, packages/bb-realtime/src/types.ts:82, :89-93.
  • Both samples name their file on the first line (aws-blocks/index.ts, src/chat.ts), and the frontend sample declares the three UI functions it calls. packages/blocks doc-snippets therefore typechecks them together as one app (packages/blocks/src/test-support/doc-snippets.ts:19-23) instead of checking only their imports.
  • InvalidModelConfig is a case list: thrown by the constructor, thrown by stream()/resume(), or failing the turn, where complete() rejects with StreamFailed (src/agent.ts:77-82, :244-258, :639, :1035, src/model-factory.ts:219-303, src/providers/canned.ts:77-89). A bedrock model without modelId and an unknown provider fail the health check and end in ModelUnavailable (src/agent.ts:693-702, src/model-factory.ts:51-83, :184, src/agent.mock.ts:11-13).
  • PersistenceRequired also covers stream()/resume() without options.userId (src/agent.ts:795, :877); InterruptRequired also covers resume() with empty responses or no conversationId (:854, :875); InvalidUsage also covers an invalid channel name and chat.subscribe() without channelId (:922-937, src/index.chat.ts:763-770).
  • The createChat intro no longer says "Lambda + Realtime today, others later" (src/agent.aws.ts:126, src/index.cdk.ts:67); model is { deployed?, local? } (src/types.ts:129-134); the fallback example sets apiKey: 'ollama' (src/model-factory.ts:174-176, :281-284); reconnect recovery states the 11-minute timeout (src/index.chat.ts:191); the message-roles note names createChat (src/index.hooks.ts:237).

Tables added: ### Options with a Default column and removalPolicy/logger rows (src/types.ts:95-227); error.name column (src/errors.ts:23-34); createChat options and controller methods (src/index.chat.ts:82-167); ConversationManagerConfig (src/types.ts:235-237); InterruptResponse (src/types.ts:258-271).

Moved inside the file: ### Options and ## Errors follow the method table; Authorization, Model Configuration and Limiting runaway cost are ## sections; a new ## Conversation history holds the history subsections.

Removed: the useChat call sample, "Full Examples" and the "Key points" lists (the useChat section is a comparison table with createChat); two searchDocs definitions (the complete example is "Use as an Agent tool" in the KnowledgeBase README); "Declaring tools", "Streaming Mode", "Reading conversation history", "Use Local LLM", "Best Practices" and the headless example without approval, whose facts are in the tables and neighbouring sentences.

packages/bb-agent/MODELS.md

## AWS Credentials (Bedrock) is ## Bedrock model access and leads with a table of error message, cause and fix; the wordings match packages/bb-agent/src/errors.ts:166-196. README links point to the new anchor.

packages/bb-knowledge-base/README.md

Below, src/ is packages/bb-knowledge-base/src/.

  • chunkOverlap: 1–99 on AWS, the mock also accepts 0 (src/types.ts:54-60, src/index.mock.ts:80-84, src/index.cdk.ts:59).
  • maxResults: a value outside 1–100 is clamped; a non-integer throws ValidationError (src/normalize.ts:28-37).
  • waitUntilSynced() has an options table and a retry table (src/types.ts:242-292, src/index.aws.ts:212-222, :565-636), replacing three paragraphs.
  • ## Error Handling is ## Errors, after ## API. Formats and maximum document size moved to "Source Configuration"; "Deploy Behavior" is merged into "Sync".
File Lines Words Prose lines over 400 characters
packages/bb-agent/README.md 1390 → 937 11179 → 7844 28 → 0
packages/bb-agent/MODELS.md (new) 0 → 166 0 → 1388 0 → 0
packages/bb-knowledge-base/README.md 274 → 282 2698 → 2358 8 → 2

Look hardest at:

  • The backend ApiNamespace and the createChat example: same method names, refresh included, and typechecked together by doc-snippets.
  • The InvalidModelConfig cases, in particular the two that end in ModelUnavailable because the health check fails first (for an undefined modelId on bedrock, read from the source, not run).
  • The 16-line useChat section: the hook is deprecated but still exported, and its samples are gone.
  • Which searchDocs examples were removed (each remaining one sets untrustedOutput: true and a filter), and the new createChat options and controller-method tables.

Not changed on purpose:

  • Examples import from @aws-blocks/core and @aws-blocks/bb-*, not from @aws-blocks/blocks.
  • The JSDoc in src/transport.ts:9-10, :145-148 still says "Lambda + Realtime"; src/types.ts:206-207 still says an omitted removalPolicy leaves the CDK default, while FileBucket falls back to the stack defaults (packages/bb-file-bucket/src/index.cdk.ts:92-97), which the README states.
  • Two "Trust Boundary" items in the KnowledgeBase README stay over 400 characters.

Open questions:

  • Should block READMEs import from @aws-blocks/blocks?
  • An InvalidModelConfig raised inside a turn reaches the caller as StreamFailed with the original message. Is that the intended contract?
  • "Max document size: 50 MB per file", the KnowledgeBase cost figures and the "Access denied / Legacy model" troubleshooting entry have no source in this repository and are kept as written.

Changed after review:

  • agentGetRawDescriptor is defined in the backend sample and used by refresh in the frontend sample; the refresh rules name only methods the samples define.
  • The backend and createChat samples are under the shipped-docs compile check. No test file was added: the existing check takes a sample whose first line names its file. Proof that it compiles them: with agentGetRawDescriptor misspelled in the frontend sample, doc-snippets fails with "Property 'agentGetRawDescriptorX' does not exist on type 'AsyncAPI<…>'" at packages/bb-agent/README.md:729. I found no compile check of the useChat README samples to copy; packages/bb-agent/src/index.test.ts tests the hook at runtime.

Related pull requests: This supersedes the bb-agent part of #1496 (the MODELS.md move and three README fixes, taken over unchanged as commit 91c0004a1). Open pull requests on the same files; whichever merges second rebases:

Part of a set of README revisions: #1635 (core), #1636 (hosting and pipeline), #1638 (bb-auth and auth-common), #1639 (the storage blocks), #1640 (compute, jobs, realtime, email and observability), #1641 (the guide, the scaffolder and its templates), #1642 (the native SDKs), #1643 (contributor docs). Each merges on its own.

Validation

Run on this branch alone at 667df3803, on main 8940cfd2f, with Node 22. All pass:

  • npm run build
  • the shipped-docs tests in packages/blocks (readme-links, docs-pointers, doc-snippets, imports-reference)
  • packages/blocks sandbox-dir-coverage (no tracked file names the old directory .blocks-sandbox)
  • packages/core readme-policies (the IAM policy JSON)
  • packages/bb-auth readme-recipes, first-admin, multi-block-snippets, users-cli, gate-diagnostics
  • packages/auth-common ui.react-types
  • packages/bb-kv-store readme.generated (every README snippet compiled) and legacy/runtime
  • packages/create-blocks-app npm test
  • npm run sync-docs:check
  • npm run check:imports
  • changeset-guard.ts validate-structure
  • changeset-guard.ts verify-coverage
  • changeset-guard.ts verify-umbrella
  • changeset-guard.ts block-major
  • every relative link and #anchor in the changed files, and every link into them from another Markdown file, resolves (checked with a script)

Run once on all the README branches of this set combined, as they were on an earlier main commit (e1e484465), and not run again after the rebases: packages/core unit suite, packages/blocks unit suite, packages/bb-kv-store unit suite, packages/bb-auth unit suite, packages/auth-common unit suite, npm run lint, npm run lint:deps and npm run check:api. All passed.

Not run locally: the full npm test across all workspaces, npm run test:e2e:local and the sandbox e2e suites. CI runs them.

No test was added: the tests listed above already read these READMEs, and the remaining statements are checked against the source by review.

Checklist

  • PR description included
  • Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@changeset-bot

changeset-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 8e077ef

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@aws-blocks/bb-agent Patch
@aws-blocks/bb-knowledge-base Patch
@aws-blocks/blocks Patch
@aws-blocks/create-blocks-app Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Comment thread packages/bb-agent/README.md Outdated
`refresh` must return the raw channel descriptor, not a hydrated channel:

- A backend method returns `channel.toJSON()` without its `__blocks` field. That method issues tokens, so give it the same authorization as the method that returns the channel.
- The callback adds the field back: `async (channelId) => ({ ...(await api.agentGetRawDescriptor(channelId)), __blocks: 'realtime/channel' })`, where `agentGetRawDescriptor` is that backend method.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refresh example calls api.agentGetRawDescriptor, a backend method the README never defines.

Either add an agentGetRawDescriptor method to the backend ApiNamespace example (returning channel.toJSON() minus __blocks, with the same auth as agentGetChannel), or point the reference at a method the example does define. At minimum, we can say explicitly that the reader must add this backend method.


const agent = new Agent(scope, 'chat', {
systemPrompt: 'You are a helpful assistant.',
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The backend and createChat wiring examples are not compile-checked, so a type error would ship silently.

Consider bringing the backend + createChat wiring under the doc-snippets/readme compile harness, as the useChat samples were, even as a type-only fixture, so these examples cannot drift out of compilation.

xyos added 6 commits November 19, 2026 05:47
The seven Model Configuration subsections move verbatim into MODELS.md, which package.json already lists in files.
The README keeps the model option table, the deployed/local/fallback examples and the facts needed before deploying.
Also: recommend createChat, not the deprecated useChat, for subscribe-before-send ordering; list the two missing cases that throw InvalidModelConfig; and make the useChat samples compile under tsc --strict.
…correct the error cases

The createChat example called six api.agent* methods that no backend in the README defined, and its subscribe callback passed a handler-or-options argument straight to channel.subscribe(handler); the backend ApiNamespace is now shown with the same method names and the callback narrows with typeof.
The Errors table gains an error.name column and case lists: InvalidModelConfig says which cases the constructor and stream()/resume() throw, which fail the turn with StreamFailed, and that a bedrock model without modelId or an unknown provider fails the health check and ends in ModelUnavailable; PersistenceRequired, InterruptRequired and InvalidUsage list their missing cases.
The createChat intro no longer says the runtime is Lambda + Realtime, the model option is typed { deployed?, local? }, the fallback example sets apiKey on its openai-api candidate, and reconnect recovery states the 11-minute timeout.
Options (with a Default column) and Errors follow the method table; createChat has an options table and a controller-method table; ConversationManagerConfig and InterruptResponse have field tables.
The useChat section is a comparison table with createChat; the useChat samples and the Full Examples section are removed.
Three searchDocs definitions remain of five, each with untrustedOutput: true and a filter from the tool context.
The section AWS Credentials (Bedrock) is renamed Bedrock model access and leads with a table: the error message, its cause and the fix (use case details form, AWS Marketplace subscription, account availability, global versus regional inference profile).
The example that only set deployed: BedrockModels.BALANCED is one sentence, and the preset notes state that BALANCED is the default when model.deployed is omitted.
The Ollama preset example, the custom-endpoint snippet with its apiKey and the logger example keep their imports.
…waitUntilSynced tables

chunkOverlap was documented as 0-100 in the options table and 1-99 under Local development; it is 1-99 on AWS and the mock also accepts 0, stated once.
maxResults said Range: 1-100; a value outside the range is clamped and a non-integer throws ValidationError.
waitUntilSynced() has an options table and a table of which poll failures are retried and which are thrown at once, replacing three paragraphs.
Errors follows API, the asAgentTool() filter rules and tool properties are two lists, supported formats and the maximum document size are under Source Configuration, and Deploy Behavior is merged into Sync.
…t` samples and compile both

The `refresh` rule under the `createChat` options table called `api.agentGetRawDescriptor`, a backend method no sample defined. The backend sample now defines it, with the same `auth.requireAuth(context)` as `agentGetChannel`, and the frontend sample passes it to `refresh`.
Both samples name their file on the first line (`aws-blocks/index.ts`, `src/chat.ts`) and the frontend sample declares the three UI functions it calls, so the shipped-docs code-sample check (`packages/blocks` `doc-snippets`) typechecks them together as one app instead of checking only their imports.
The changeset gains one line for the two changes.
@xyos
xyos force-pushed the docs/readme-agent-knowledge-base branch from 08227de to 667df38 Compare November 19, 2026 06:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants