Skip to content

docs(repo, bench, foundations, admin-ui): correct stale statements in contributor READMEs - #1643

Draft
xyos wants to merge 7 commits into
aws:mainfrom
xyos:docs/readme-contributor-docs
Draft

xyos wants to merge 7 commits into
aws:mainfrom
xyos:docs/readme-contributor-docs

Conversation

@xyos

@xyos xyos commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Ten READMEs that contributors and maintainers read (1,653 lines, 13,773 words) contained statements the source contradicts, and their run commands sat far from the top.

  • packages/foundations/README.md documented 197 lines of API (SQLTable, LLM, getValue, enqueue) for a package whose src/index.ts is export default {}.
  • packages/admin-ui/README.md said npm run dashboard starts the dashboard; no template defines that script.
  • test-apps/native-bindings/README.md described the upload handle as { url, fields } for a multipart upload; the code returns a tagged handle and uploads with PUT.
  • scripts/agent-bench/README.md placed the task matrix in the wrong workflow file, counted 16 Building Blocks where the matrix names 14, and gave its local run example at the end of a 490-line file.

Issue #, if available:

Changes

README.md. Listed 11 of 14 templates; now points to npx @aws-blocks/create-blocks-app --help (14 directories in packages/create-blocks-app/templates/). The Blocks table lists Compute. The layout root is aws-blocks/, not blocks/, and adds test-infra/, tasks/, docs/. Contributing links CONTRIBUTING.md and AGENTS.md.

docs/README.md. The Design list adds design/compute-selection-api.md and design/scaffold-new-bb.md.

scripts/agent-bench/README.md.

  • The matrix is the include: list of the bench job in agent-bench.yml (.github/workflows/agent-bench.yml:90-101), not in pr-agent-bench.yml; it names 14 Building Blocks, not 16.
  • lib/... paths read steps/lib/...; scripts/agent-bench/ has no lib/.
  • "No label gate" and "No microVM, no S3 transport" are removed; the triggers are stated as they are (.github/workflows/pr-agent-bench.yml:38-54).
  • Added: a "Run it" section (workflows, triggers, path filter, dispatch command, npm run test:ci -w scripts/agent-bench, the local example moved from the end) and a "Configuration" table derived from agent-bench.yml, steps/2-agent-run.ts, steps/4-judge.ts and steps/lib/isolation.mjs.
  • Indicators and noise bands are tables; scoring topics are ### headings; the file table adds six files it omitted.

scripts/harbor-bench/README.md. The dispatch command moves from "Repository setup" to a new "Run the check" section, with the trials range and same-PR cancellation (.github/workflows/harbor-agent-check.yml:10-21). "Approving a run" is its own section. Two one-off measurements and the planned cross-version replay sentence are removed.

test-infra/README.md. Said "VPC quota is 5 per region" while step 4 of "More E2E accounts" requests 20; now 5 by default, 20 requested (test-infra/vpc-test-stack.ts:7 for the default). One title with two sections instead of two level-1 headings. The blocks-github-e2e permissions are a list and the bench follow-up is five numbered steps.

tasks/README.md. "Every spec re-declares the same small helpers" became "A spec declares the helpers it needs" (tasks/observability-api/test.spec.ts has no rpc wrapper).

test-apps/native-bindings/README.md. Upload: fileCreateUploadHandle(path, contentType?) (test-apps/native-bindings/aws-blocks/index.ts:585) returns { __blocks: 'file-bucket/upload', url, contentType }, uploaded with PUT (packages/bb-file-bucket/src/index.aws.ts:221-231). Download: { __blocks: 'file-bucket/download', url } (index.aws.ts:217). "Sandbox/Production" became "Deployed sandbox" (package.json:22-24: deploy is deploy:sandbox). Sign-up codes, OIDC, the second federated block (auth-sso) and the bearer-token note are lists; general stub IdP rules point to the Auth README. "All four auth blocks are Auth" is removed; the table has the four Auth rows.

test-apps/telemetry/README.md. "Running" is the first section. The sentence naming aws-actions/configure-aws-credentials is removed; the job uses ./.github/actions/e2e-aws-credentials (.github/workflows/pr-checks.yml:1426).

packages/foundations/README.md. Now states the package is private and exports nothing, points to @aws-blocks/blocks and the packages/bb-* READMEs, and keeps the two-Auth-block snippet unchanged.

packages/admin-ui/README.md. Entry point is dashboard() (packages/core/src/scripts/dashboard.ts:364, exported through ./scripts at packages/core/package.json:67). Install offer: dashboard.ts:259-272. Port: dashboard.ts:17,33.

File Lines Words Prose lines over 400 chars
README.md 91 → 90 657 → 628 2 → 2
docs/README.md 25 → 27 144 → 151 0 → 0
packages/admin-ui/README.md 43 → 42 304 → 320 0 → 0
packages/foundations/README.md 197 → 40 861 → 253 1 → 0
scripts/agent-bench/README.md 490 → 562 5,121 → 4,833 0 → 0
scripts/harbor-bench/README.md 311 → 333 2,971 → 2,859 0 → 0
tasks/README.md 43 → 40 317 → 273 0 → 0
test-apps/native-bindings/README.md 129 → 149 1,272 → 1,274 4 → 0
test-apps/telemetry/README.md 111 → 111 754 → 723 0 → 0
test-infra/README.md 210 → 205 1,344 → 1,378 0 → 0
Total 1,658 → 1,607 13,807 → 12,754 7 → 2

The two bench READMEs gain lines because paragraphs became lists and tables.

Look hardest at:

  • packages/foundations/README.md, 197 → 40 lines. A maintainer must confirm that the removed API was never shipped under these names. The kept snippet is unchanged because packages/bb-auth/src/multi-block-snippets.test.ts reads this README, compiles the snippet and expects three such snippets under packages/.
  • The corrected statements in test-infra (VPC quota), test-apps/native-bindings (handle shapes, "Deployed sandbox") and scripts/agent-bench (matrix location, 14 Building Blocks, the "Configuration" table defaults).

Not changed on purpose: .changeset/README.md (stock text of the changesets tool). The root README's opening paragraph, which stays one paragraph over 400 characters. .github/workflows/agent-bench.yml:89, whose comment still says "10 cells cover 16 Blocks".

Open questions:

  • foundations: keep the package with this README, or remove it? Removal also changes the file list and the count of 3 in multi-block-snippets.test.ts.
  • The Admin UI still tells users to run npm run dashboard (packages/admin-ui/app/page.tsx:300, app/playground/page.tsx:247, app/deployments/[id]/page.tsx:195, src/components/inventory-nav.tsx:194), and packages/admin-ui/src/lib/commands.ts:36 says the default template defines it. No template's package.json has a dashboard script. Should a template define it, or should those strings change?
  • test-infra: the README says pool accounts request a VPC quota of 20; whether each increase was granted is not visible in the repository.
  • scripts/agent-bench: BLOCKS_TELEMETRY_CANARY_ID is called a "workflow secret"; repository or environment secret? steps/finalize-result.mjs also stamps status: cancelled, which the failure-handling paragraph does not list.

Carried from main: two changes merged after this pull request was opened are applied to the revised text. The directory rename to .cdk-blocks-sandbox (#1644): one line each in scripts/agent-bench/README.md and test-apps/telemetry/README.md. The repo slug aws/cdk-blocks (#1647): the dispatch command in scripts/harbor-bench/README.md, and in test-infra/README.md the npm test paragraph and the new paragraph on renaming or moving the repo.

Related pull requests: supersedes none. Open pull requests that touch the same files:

Whichever merges second rebases.

Part of a set of README revisions: #1635 (core), #1636 (hosting and pipeline), #1637 (bb-agent and bb-knowledge-base), #1638 (bb-auth and auth-common), #1639 (the storage blocks), #1640 (compute, jobs, realtime, email and observability), #1641 (the guide, the scaffolder and its templates), #1642 (the native SDKs). Each merges on its own.

Validation

Run on this branch alone at 0aebdb3ba, on main 8940cfd2f, with Node 22. All pass:

  • npm run build
  • the shipped-docs tests in packages/blocks (readme-links, docs-pointers, doc-snippets, imports-reference)
  • packages/blocks sandbox-dir-coverage (no tracked file names the old directory .blocks-sandbox)
  • packages/core readme-policies (the IAM policy JSON)
  • packages/bb-auth readme-recipes, first-admin, multi-block-snippets, users-cli, gate-diagnostics
  • packages/auth-common ui.react-types
  • packages/bb-kv-store readme.generated (every README snippet compiled) and legacy/runtime
  • packages/create-blocks-app npm test
  • npm run sync-docs:check
  • npm run check:imports
  • changeset-guard.ts validate-structure
  • changeset-guard.ts verify-coverage
  • changeset-guard.ts verify-umbrella
  • changeset-guard.ts block-major
  • every relative link and #anchor in the changed files, and every link into them from another Markdown file, resolves (checked with a script)

Run once on all the README branches of this set combined, as they were on an earlier main commit (e1e484465), and not run again after the rebases: packages/core unit suite, packages/blocks unit suite, packages/bb-kv-store unit suite, packages/bb-auth unit suite, packages/auth-common unit suite, npm run lint, npm run lint:deps and npm run check:api. All passed.

Not run locally: the full npm test across all workspaces, npm run test:e2e:local and the sandbox e2e suites. CI runs them.

No test was added: the tests listed above already read these READMEs, and the remaining statements are checked against the source by review.

Checklist

  • PR description included
  • Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@changeset-bot

changeset-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 8eca810

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@aws-blocks/admin-ui Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

xyos added 7 commits November 19, 2026 14:23
…design docs to the index

The root README listed 11 of the 14 `create-blocks-app` templates and omitted `Compute` from the Blocks table; it now points to `npx @aws-blocks/create-blocks-app --help`, which lists every template, and the table lists `Compute`.
The repository layout named the root `blocks/` and omitted three directories; it now shows `aws-blocks/` with `test-infra/`, `tasks/` and `docs/`.
The Contributing section links `CONTRIBUTING.md` and `AGENTS.md`.
Statements made twice (no AWS account needed, no code changes, the definition of a Block, the Node.js requirement, the Developer Guide link) are made once.
`docs/README.md` adds `design/compute-selection-api.md` and `design/scaffold-new-bb.md` to the Design list and drops bold from its links.
…b` paths; add a config table

The README placed the task matrix in `pr-agent-bench.yml` and said it covers 16 Building Blocks; the matrix is the `include:` list of the `bench` job in `agent-bench.yml` and names 14.
It wrote helper paths as `lib/...`; the files are in `steps/lib/`.
A "Run it" section now follows the introduction: the two workflows, the automatic triggers and their path filter, the manual dispatch command, the harness test command and the local single-step example.
A "Configuration" table lists each variable, where it is set and its default.
Indicators and noise bands are tables, the scoring topics are `###` headings, and the file table adds six files it omitted.
Sentences about designs the bench does not use (microVM, label gate) are removed.
…val its own section

The dispatch command sat unlabelled under "Repository setup"; it is now in a "Run the check" section after the introduction, with the `trials` range (1 to 50, default 1) and the fact that a new dispatch for the same pull request cancels the running one.
"Approving a run" is its own section, as a list.
The cost metrics, the session invariants and the calibration outputs are lists.
Two one-off measurements and the sentence about a planned cross-version replay are removed, and security facts already stated in "Private traces", "Package provenance" and "Accounting" are not repeated under "Security boundaries".
…e shapes and CI credentials

`test-infra/README.md` said the VPC quota is 5 per Region while its own account setup steps request 20; it now says 5 is the default and 20 is what the E2E pool accounts request.
`test-apps/native-bindings/README.md` described the upload handle as `{ url, fields }` for a multipart upload; it is `{ __blocks: 'file-bucket/upload', url, contentType }`, sent with `PUT`, and the download handle carries the `__blocks` tag too.
`test-apps/telemetry/README.md` said CI provides credentials through `aws-actions/configure-aws-credentials`; the job uses the repository's `e2e-aws-credentials` action, and the sentence is removed.
`tasks/README.md` no longer says every spec declares the same three helpers.
`test-infra/README.md` has one title with two sections, the `blocks-github-e2e` permissions as a list and the bench follow-up as numbered steps; `test-apps/telemetry/README.md` starts with "Running".
The README documented an API that no package exports: `SQLTable`, `LLM`, `getValue`, `enqueue`, string scopes such as `'app'`.
`packages/foundations/src/index.ts` is `export default {}` and `package.json` sets `"private": true`.
The README now says so, points to `@aws-blocks/blocks` and the `packages/bb-*` READMEs for the Building Blocks that ship, and keeps one snippet.
The two-`Auth`-block snippet is unchanged because `packages/bb-auth/src/multi-block-snippets.test.ts` compiles and counts it.
… run dashboard`

The README said `npm run dashboard` in an app starts the dashboard; no `create-blocks-app` template defines that script.
It now names `dashboard()` from `@aws-blocks/core/scripts`.
The install offer states its conditions (interactive terminal, `npm install --save-dev`) and the port is given as the first free port from 4400, or `--port <n>`.
The line about a future provider is removed.
@xyos
xyos force-pushed the docs/readme-contributor-docs branch from 0aebdb3 to 4625767 Compare November 19, 2026 14:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant