Skip to content

[AI-3330] Usage tracking in the router - #2063

Draft
kanat wants to merge 9 commits into
mainfrom
feature/ai-3330-router-usage-tracking
Draft

kanat wants to merge 9 commits into
mainfrom
feature/ai-3330-router-usage-tracking

Conversation

@kanat

@kanat kanat commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Do not merge: ship-feature in progress.

This is the feature branch for Linear AI-3330 "Usage tracking in the router".

What the branch adds to the router:

  • counts inbound hooks, the router's own background work and socket refusals per customer
  • names the app on the log lines where a customer gets stuck
  • makes the SDKs send their language and version
  • posts a daily Slack digest of Agents activity

How it lands: sub-PRs merge into this branch, one wave at a time. After all waves, the branch is checked on staging against the epic's acceptance criteria.

Gates for the merge into main

  • To be filled at the end of the run.

🤖 Generated with Claude Code

kanat and others added 2 commits November 18, 2026 11:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## What

The Go and JavaScript SDKs now say which SDK and which release made each
request. The router already logs `user_agent` and `x_stream_client` on
every request and counts `X-Stream-Client: sdk` under client `sdk`, so
the router is not changed.

| SDK | Where | `X-Stream-Client` | `User-Agent` |
| --- | --- | --- | --- |
| Go | `Backend.Credentials()` (every REST call and every socket
handshake) | `sdk` (as before) | `stream-agents-go/0.3.0` (new) |
| JavaScript, Node | `Backend.headers()` (every request, anonymous guest
request included) | `sdk` (new) | `stream-agents-js/0.4.3` (new) |
| JavaScript, browser | `Backend.headers()` | `sdk` (new) | not set: the
browser's own |

```
Go app ──Credentials()──► X-Stream-Client: sdk, User-Agent: stream-agents-go/0.3.0 ──► router
Node app ──headers()────► X-Stream-Client: sdk, User-Agent: stream-agents-js/0.4.3 ──► router
page ──────headers()────► X-Stream-Client: sdk ──► proxy / router (both CORS lists allow it)
                                                    router log: user_agent, x_stream_client
                                                    customer_activity: client = sdk (JS was api)
```

- `Version` (Go, `sdks/go/stream/backend.go`) is the latest tag
`sdks/go/v0.3.0` without the v. It is a literal: build info reads
`(devel)` for a branch build. The `sdk-go` skill's release step 1 now
bumps it.
- `VERSION` (JS, `sdks/js/src/backend.ts`, exported) is `package.json`'s
version. A test compares the `User-Agent` a request sends with
`package.json`, so a version bump that misses the constant fails `npm
test`.
- A page sends no `User-Agent`: neither the router's CORS list
(`acceleration/internal/api/server.go` `corsRequestHeaders`) nor the
proxy's allows it, so the preflight would refuse the request.
- JS calls with `X-Stream-Client: sdk` that change an agent config are
now recorded in the audit log with source `sdk` instead of `api`, as
Go's and Python's already are.
- `.claude/skills/sdk/SKILL.md` gets the note that the other SDKs need
the same change.

Ticket: AI-3327 (serves AC-5 of AI-3330).

## Tests

- Go: `TestEveryRequestAndSocketNamesTheGoSDKAndItsRelease` (REST and
socket handshake, local router and proxy paths),
`TestTheVersionIsAReleaseNumber`. `cd sdks/go && go vet ./... && go test
./...` pass.
- JS: "names the SDK and its release on every request from Node"
(against the test router, `User-Agent` checked against `package.json`;
anonymous path too), "names the SDK alone from a page" (no `process`,
and a polyfilled `process` with empty `versions`), "names the SDK on the
upgrade request of a socket opened from Node". Existing exact-header
tests updated. `cd sdks/js && npm test`: 712 pass.
- Mutation checks: 13 mutants plus the bare `process` check and the
socket headers (header dropped, wrong prefix, `v`-prefixed version, Node
check forced or inverted, browser header dropped, anonymous path
skipped, version drift), all killed.

## Not in this PR

- Python, .NET, Ruby, Rust, PHP, Swift, Kotlin and Dart (AGENTS.md: Go
first). Noted in the sdk skill.
- A browser socket cannot send headers and authenticates in its query
string; the router reads `X-Stream-Client` only from the header, so
browser sockets are still counted as `api` (ticket: the router needs to
read a query value). Node sockets send the headers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Walkthrough

The Go and JavaScript SDKs add X-Stream-Client: sdk to requests. Go requests include a versioned User-Agent. JavaScript requests include that header in Node and omit it in other runtimes. The changes also update WebSocket header handling, tests, SDK release documentation, and the Go release workflow’s version check. The acceleration service records activity for verified hooks, phone/SMS work, outgoing webhook forwards, and refused operations. It also adds a daily UTC activity digest and activity stages for workers, voice/video, and phone/SMS.

Merge Risk: Moderate · up to 7f87f

This change adds per-customer activity counting, a daily Slack digest, and SDK identification headers. Several counting paths can under- or over-count work (queued forwards, deployment-mode message hooks, scheduler takeovers, organization attribution), and the digest can miss a day or exceed its deadline. These affect reporting accuracy rather than customer traffic. Resolve the attribution and digest issues, and fix the SDK release-version checks, before merging.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

golangci-lint (2.15.0)

golangci-lint (2.15.0) exceeded its 180-second time budget. Findings from completed work are retained; remaining checks are incomplete.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Minor · Distinguish an explicit WebSocket option from the default. · backend.ts:695

sdks/js/src/backend.ts:695
Minor · Functional Correctness · Quick win

Distinguish an explicit WebSocket option from the default.

On Node, webSocket: globalThis.WebSocket passes the identity check and gets the headers options object. That contradicts the URL-only behavior documented and tested for supplied constructors. Record whether the constructor came from BackendOptions. Add headers only when the SDK selected the default. Keep a wire test for both the omitted option and the explicitly supplied global constructor. Node v22.0.0 accepts this options object, so the difference is in the handshake headers, not whether the constructor throws. (raw.githubusercontent.com)

Nitpick comments: 1

sdks/go/stream/backend_test.go: 1

619-619: Trivial · Maintainability & Code Quality · Low value

Group the wire and version tests in a testify suite, one test per behaviour.

The new tests are top-level functions. The go-testing skill requires tests in sdks/go to be grouped in a suite.Suite. The wire test also iterates a map of named backends, which is a table test. The skill says to avoid table tests.

Do not split the wire test into separate local-router and proxy tests. Credentials() in sdks/go/stream/backend.go:269-272 sets X-Stream-Client and User-Agent before it branches on Authenticate. Both configurations send the same headers on the request and the socket handshake. The proxy credential path is already covered by TestASocketIsAuthenticatedOnTheHandshake.

In the suite, write one test per behaviour, each named for what it proves. For example: the request identifies the Go SDK and its release, and the socket handshake identifies the Go SDK and its release. Move the shared header check into a helper. Keep the version check as a suite method as well.

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 31ba2e0 and ee84a68.

Files selected for processing: 13

  • .claude/skills/sdk-go/SKILL.md
  • .claude/skills/sdk-js/SKILL.md
  • .claude/skills/sdk/SKILL.md
  • .github/workflows/release-sdk-go.yml
  • CHANGELOG.md
  • sdks/go/stream/backend.go
  • sdks/go/stream/backend_test.go
  • sdks/js/README.md
  • sdks/js/src/backend.ts
  • sdks/js/src/index.ts
  • sdks/js/tests/backend.test.ts
  • sdks/js/tests/router.ts
  • sdks/js/tests/spend-limits.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 3 remain after this review.

Comment thread .claude/skills/sdk-go/SKILL.md Outdated
Comment on lines +115 to +116
(`stream-agents-go/X.Y.Z`), and nothing else sets it. `release-sdk-go.yml` fails a tag
whose `Version` differs.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Check stream.Version before git push creates the Go release tag.

The release steps push the tag first. release-sdk-go.yml checks Version only after the push. The Go module proxy can serve the tag to a consumer in that window, so a mismatched version can be published. A failed workflow reports the error. It does not remove the pushed tag. The documented recovery is retract vX.Y.Z and a new patch release.

Add the check before git tag:

  • Read the declaration from the merge commit: git show "$MERGED:sdks/go/stream/backend.go".
  • Compare the Version value with X.Y.Z as a literal string.
  • Read the declaration as Go source. A text match can accept a comment line that holds the same assignment, such as // Version = "X.Y.Z" or a line inside a /* */ block.
  • Stop the release when the value differs.

Keep the workflow check as a second check.

Tools

SkillSpector (2.12.0)

[error] 124: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))

A running session's caller gives it tools and takes them back (no socket change): `POST /v1/agents/sessions/{id}/tools` (`addSessionTools`, body `AddSessionToolsRequest` `{tools: SessionTool[]}`, at least one) and `DELETE /v1/agents/sessions/{id}/tools/{name}` (`removeSessionTool`), both client-accessible and 204. A tool named as one the caller already has is a 409: to change one, remove it, then add it again, so no call runs half under each declaration. A change waits for the turn in progress and every tool call in flight, one answered as still running included, so a call is only checked against the tools it was made with, the model is offered the change from the next turn, and once a removal answers no call of the tool can still arrive: a handler goes then, and must not wait for a change from inside a call. Only the kind of caller that opened the session may: a device its own session's, a backend a backend's (403 otherwise). A name the session offers from elsewhere (built in, a connector's, a worker's, another agent's) is a 409; withdrawing a name the caller has no tool by, or changing a session that is not running, a 404; a speech-to-speech model whose tools are fixed when it opens (Gemini Live) a 400; a speech-to-speech session that drops while the change waits a 503, to ask again. A native model's call made while the tools change, or by a reply that opened while they did, is refused rather than run, and so is one of a tool it is not offered. A watcher that names the tools it runs (`?tools=`) is also asked to run the ones its side added, and leaves a call it does not hold to one that does, as every SDK's socket already does. Go has `stream.Pipeline.AddTools(ctx, *tools.Registry)` and `RemoveTool(ctx, name)`, with `client.Session` wrappers; the added functions are the session's alone, not its agent's registry, and are forgotten when it is left. Swift has `AgentSession.add(tools:)` and `remove(toolNamed:)`, and `swift_assistant`'s seat selector brings `reserve_table` that way. Every SDK has them now, the added functions kept per session: JavaScript `Session.addTools`/`removeTool`, Python (`plugins/stream`) `Session.add_tools`/`remove_tool` and `Accelerated.add_tools`/`remove_tool`, .NET `Session.AddToolsAsync`/`RemoveToolAsync`, Ruby `Session#add_tools`/`remove_tool`, Rust `Session::add_tools`/`remove_tool`, PHP `Session::addTools`/`removeTool`, and Kotlin and Dart `AgentSession.addTools`/`removeTool`, whose generators list the two operations. Each puts a handler in place before the request for a name it does not have yet, never replaces one it has, takes it back only when the router refused, marks a name whose adding got no answer unsettled and withdraws it (a 404 being as good) before the name is added again, lets a name withdrawn from its agent-wide registry be added again with another function, and makes one change at a time.
A removal whose answer is lost is unsettled as an add's is, in every SDK, Go and Swift included: the handler stays until the next remove (a 404 accepted) or add (which withdraws first) of the name settles it. A change given up on while its request is in flight is unsettled the same way; Rust marks the name before the request, since a dropped future runs nothing after it. Only a 4xx is a refusal (an add's handlers taken back, a removal changing nothing): a 5xx, a 2xx that cannot be read, or no answer leaves the name unsettled, since a gateway can answer 502 or 504 after the router applied the change. The SDK's own HTTP stack resends neither request (a client the app hands in, with retries of its own, is the app's), so a lost answer never comes back as a replay's 404 or 409: JavaScript sends the DELETE as not idempotent, Ruby turns off Net::HTTP's one resend of it, PHP sends it on an Amp client without `RetryRequests`, .NET sends it with an empty body, since its handler resends a bodyless request whose connection closed before any answer, and Kotlin sends both as a one-shot body OkHttp never replays; Go, Python, Rust, Swift and Dart's stacks resend neither. A fork of a running session is offered the tools the parent has now (`forkSpec` takes the live spec, which a change of tools updates; `TestAForkOfARunningSessionIsOfferedTheToolsItHasNow`), so a fork watched with handlers gets the session's current ones (opening minus withdrawn, plus added), copied with the session's unsettled names and asked for under the change lock: Go `Session.Fork`, JavaScript, Python, .NET, Ruby, Rust, PHP and Dart. Such a fork waits for a change in flight, so it is not awaited from inside one of the session's tools either. Swift's and Kotlin's `fork` return a session the app attaches with its own tools. Python, .NET, Ruby, Rust and PHP keep a copy of the added functions, since a caller's registry there could later hold another function under the name; Go's and JavaScript's refuse a second one, and Swift's, Kotlin's and Dart's tools are values.

Every SDK names itself and its release on each request, so the router's request log and its activity count can tell which SDK, and which release, made a call (AI-3327). Go sets `User-Agent: stream-agents-go/<Version>` beside `X-Stream-Client: sdk` in `Backend.Credentials()` (`sdks/go/stream/backend.go`), with `Version` a literal bumped at each release, never read from build info (a branch build reads `(devel)`). JavaScript sends `X-Stream-Client: sdk` from `Backend.headers()`, and from Node also `User-Agent: stream-agents-js/<VERSION>`; a page sends `X-Stream-Client` alone, since neither the router's nor the proxy's CORS list allows `User-Agent`. The router needs no change: it logs `user_agent` and `x_stream_client` on every request and counts `X-Stream-Client: sdk` under client `sdk`. Other SDKs need to update: Python (`plugins/stream`) sends `X-Stream-Client: sdk` with no `User-Agent` of its own; .NET, Ruby, Rust, PHP, Swift, Kotlin and Dart send neither yet. Each sends `stream-agents-<language>/<version>` from a literal version constant, and only a server-side SDK or one off the browser sets `User-Agent`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Limit the attribution claim to requests that carry SDK identifiers.

Browser HTTP requests send the SDK marker but not the SDK release. Browser WebSocket upgrades send neither SDK identification header and remain counted as api. State both exceptions so operators do not interpret missing browser attribution as a telemetry fault.

Tools

LanguageTool

[grammar] ~907-~907: Use a hyphen to join words.
Context: ...stant, and only a server-side SDK or one off the browser sets User-Agent.

(QB_NEW_EN_HYPHEN)

SkillSpector (2.12.0)

[error] 37: [PE3] Credential Access: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Remediation: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.

(Privilege Escalation (PE3))


[error] 404: [MP3] Memory Manipulation: Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Remediation: Protect agent memory and state from modification by untrusted content. Use read-only memory for critical instructions and validate all state changes.

(Memory Poisoning (MP3))


[error] 404: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 404: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 406: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 408: [MP3] Memory Manipulation: Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Remediation: Protect agent memory and state from modification by untrusted content. Use read-only memory for critical instructions and validate all state changes.

(Memory Poisoning (MP3))


[error] 471: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 519: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 544: [PE3] Credential Access: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Remediation: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.

(Privilege Escalation (PE3))


[error] 544: [PE3] Credential Access: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Remediation: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.

(Privilege Escalation (PE3))


[error] 562: [PE3] Credential Access: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Remediation: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.

(Privilege Escalation (PE3))


[error] 579: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 740: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 763: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 765: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 765: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 899: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))


[error] 904: [TM1] Tool Parameter Abuse: Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).

Remediation: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.

(Tool Misuse (TM1))

exit 1
fi
git merge-base --is-ancestor HEAD origin/main || { echo "$TAG is not on main"; exit 1; }
grep -q "Version = \"${TAG#sdks/go/v}\"" sdks/go/stream/backend.go || { echo "stream.Version is not ${TAG#sdks/go/v}"; exit 1; }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Compare the declared Go version, not a substring.

For tag sdks/go/v0.3.1, a comment containing Version = "0.3.1" satisfies this check even if stream.Version is still "0.3.0". The format test also passes with "0.3.0". Extract the version from the Version declaration and compare it literally with the tag suffix. A matching declaration should pass even if comments mention other versions.

Comment thread CHANGELOG.md

### The Go and JavaScript SDKs name themselves and their release

Every Go SDK request sends `User-Agent: stream-agents-go/<version>` beside `X-Stream-Client: sdk`, with the new `stream.Version`. The JavaScript SDK now sends `X-Stream-Client: sdk`, so its calls are counted as the SDK's rather than the API's, and from Node also `User-Agent: stream-agents-js/<version>` (`VERSION`); a page sends `X-Stream-Client` alone (#2065).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Distinguish browser HTTP requests from WebSocket upgrades.

Browser HTTP requests send X-Stream-Client: sdk. Browser WebSocket upgrades do not send it and remain counted as api. Qualify the attribution claim in the release note.

seen := make(chan http.Header, 2)
upgrader := websocket.Upgrader{}
router := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
seen <- r.Header.Clone()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Stability & Availability · Quick win

Do not block the test server on header capture.

If the client sends an unexpected third request before the assertions, the handler blocks on the full seen channel. The test waits for the handler, so it never reaches the channel reads. Capture the requests without blocking the handler, then assert the count and headers. The expected REST request and upgrade should still pass.

if got := header.Get("User-Agent"); got != want {
t.Errorf("%s: the %s says User-Agent %q, want %q", name, call, got, want)
}
if got := header.Get(ClientHeader); got != ClientSDK {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Assert the literal SDK header on the wire.

If ClientSDK changes to "api" or ClientHeader changes to another name, Credentials() and this assertion change together. Both transports can pass while sending the wrong identification. Assert header.Get("X-Stream-Client") == "sdk" so the test checks the wire contract independently of the implementation. As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

Comment thread sdks/js/README.md
Comment on lines +981 to +982
A version bump committed to `package.json` also sets `VERSION` in `src/backend.ts`, the release
every request from Node names in its `User-Agent`; `npm test` fails while the two differ.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

The published Node User-Agent can report the previous version.

On the documented release path, semantic-release sets the version in the published package.json. Nothing updates VERSION in src/backend.ts, and prepublishOnly only rebuilds dist. A release published as 0.4.4 therefore sends stream-agents-js/0.4.3 in every Node request. This defeats the SDK version identification this PR adds for usage tracking.

The published build must report the version it is published under. The README at lines 981-982 documents a manual commit as the only sync path. That path does not apply to this release flow, so the README must not rely on it.

const opening = router.socket();

const socket = backend.openSocket(router.url.replace("http", "ws") + "/v1/anything");
const connection = await opening;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Stability & Availability · Quick win

Reject the test promptly when the client socket fails during the handshake.

router.socket() resolves only after a successful upgrade, so a refused upgrade leaves opening pending until the suite timeout. Race opening against the client socket's error and close events, and reject with the socket failure when either fires first. Remove the listeners once opening settles. This matches the failure handling in Socket.open (sdks/js/src/socket.ts:89-114).

Close the client socket in a finally block, so a failed assertion or a rejected handshake does not leave it open. The success path still runs the wire-header assertions after the upgrade.

Comment on lines +114 to +116
class Recording {
constructor(...args: unknown[]) {
seen.push(args);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Heavy lift

Replace the two Recording WebSocket doubles with real upgrades through TestRouter.

Both tests replace the WebSocket constructor with Recording, which records constructor arguments only. Neither test opens a socket, so neither shows that the connection happens or what reaches the server. The JavaScript SDK conventions prohibit mocks and require assertions on what reached the far end.

  • Simulated-browser test (lines 114-129): keep the URL-only expectation, but run it with no process global, as a page does. Assert the upgrade that reaches router. Assert that it has no x-stream-client: sdk header and no user-agent equal to stream-agents-js/${VERSION}. A real transport can send its own User-Agent, so check for the SDK value rather than header absence.
  • Caller-supplied socket test (lines 134-147): pass a real WebSocket implementation as the webSocket option. Assert the upgrade received by router with the same header checks.

Both tests must assert on the upgrade received by router, not on the constructor call. The Node upgrade test at lines 97-108 already covers SDK headers on Node sockets, so these tests need to cover only the URL and the absence of SDK identification.

kanat and others added 2 commits November 18, 2026 12:38
#2066)

## What

AI-2380, part A: hooks, forwards to customer endpoints and socket frame
refusals are now counted in `customer_activity` under the app they
belong to. Serves AC-1, AC-2 and AC-3 of AI-3330.

```
vendor / Stream ──hook──▶ withRequestLog ─ activity.WithNote(ctx)
                             │
                             ▼ handler checks signature / token
                         activity.NoteCustomer(ctx, app, area)
                             │
                             ▼ recordActivity: no credential, not a peer's replay
                         Recorder.Record{app, area, client: api}
                             │  (once a minute)
                             ▼
              store.RecordActivity: an empty org is read from app_organizations
```

| Caller | Area | Where the app is noted |
|---|---|---|
| Stream message hook, call hook, Stream Chat delivery | `hooks_in` |
after `verifyHook` / `chatInboundOrigin` (`noteHook`, app mode). In
deployment mode, the call hook uses the rung number's app |
| Telnyx call hook | `phone_sms` | `phone.InboundBridge`, after the
number resolves (the handler checks the signature first) |
| Phone plan fetch, status post, machine report | `phone_sms` |
`phone.Answer`, `callOfToken`, after the token resolves |
| Channel line delivery (SMS, WhatsApp) | `phone_sms` |
`channels.Receive`, after `provider.Verify` |
| Provider-app connector events | `hooks_in` | `receiveEvent`, after the
verifier. A connector's own route names no customer |
| `eventforward.Forwarder.attempt` | `webhooks_out` | once per finished
forward. Retries count nothing. A give-up is refused with
`destination_refused`, `destination_unavailable` or `forward_failed` |
| `/v1/agents/socket` frame refusals | `conversations` | `refuse` /
`refuseWith` call `noteRefusal`. `Refused = status >= 400 or a noted
code` |

## Decisions

- **The slot lives in `internal/activity`**, not `internal/api`. This
lets `phone` and `channels` note the app where they resolve it, with no
change to their signatures.
- **Organization:** one scalar subquery on `app_organizations` (the
table `Enforcer.Join` writes) inside the existing upsert, only when the
delta names no organization. It runs once a minute per row and needs no
cache. An org the request names wins over the mapping.
- **Client for router-started work:** `activity.ClientRouter = api`. A
distinct `router` value would change the ops endpoint's documented
client list, so it would need an openapi change and regenerated clients.
See Questions.
- **Socket `refuse(msg)` paths** (bad first frame, config lookup,
session start) note the code that `POST /v1/agents/sessions` answers the
same failure with: `invalid_request`, the config error's code,
`conflict`. The frames are unchanged: no `code` field is added, which a
test asserts.
- **A hook replayed to a peer** (`node.Forwarded`) is counted only by
the node it arrived at.

## Not in this PR (part B, size stop)

Scheduled task firing (`scheduling`), campaign dial (`phone_sms`) and
simulation runs (`evals`). These are
`internal/api/scheduled_delivery.go`, `internal/campaign/campaign.go`
and the simulation-run finish in `internal/store/models.go`. Also MCP
connection events (`mcpevents.Receive`) and 10DLC reports (`dlc.Hook`).

## Tests

Each test names the defect it catches.
- store: an empty org is resolved, a named org is kept, and an unmapped
app is written with `''`.
- eventforward: a 410 gives refused=1 with code and org (AC-2). Retried
then sent counts 1 request and 0 refused. Given up counts refused once.
Unsigned counts `forward_failed`.
- api: Stream message, call and chat-inbound hooks. A bad signature
counts nothing. A hook carried to a peer is counted once. A Telnyx call
counts as `phone_sms`, not `hooks_in`. Channel line, status post, plan
fetch and provider-app event are counted. Socket refusals: bad first
frame, unknown config, open session, failed start, and a spent limit
(AC-3).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (3)

Major · Keep an explicit organization when folding an unnamed delta. · activity.go:137

acceleration/internal/store/activity.go:137
Major · Data Integrity & Integration · Quick win

Keep an explicit organization when folding an unnamed delta.

If a named phone API request and an unnamed phone/SMS hook share an app, area, client, and day, ActivityDelta.Add replaces the named organization with "". If the membership join has not completed, the flush inserts both counts without their organization. This also affects a fresh row, before ON CONFLICT can help.

Only replace OrganizationID when the incoming delta names one. Keep the membership lookup for batches where no delta names an organization. Add a same-batch regression test without a membership row.

Major · Do not clear a stored organization when the lookup finds none. · activity.go:109

acceleration/internal/store/activity.go:109
Major · Data Integrity & Integration · Quick win

Do not clear a stored organization when the lookup finds none.

If a named delta has already persisted a row and the membership join is still pending, a later unnamed delta for the same key resolves to "". The conflict update then writes EXCLUDED.organization_id over the known organization. The row keeps both counts, but the organization digest can omit them. (postgresql.org)

Preserve customer_activity.organization_id when the incoming organization is empty. Still apply a nonempty organization to the row. Test this with two writes and no membership row; a change to ActivityDelta.Add alone cannot fix separate flushes.

Minor · Do not discard a completed send when shutdown starts. · forwarder.go:510

acceleration/internal/eventforward/forwarder.go:510
Minor · Data Integrity & Integration · Quick win

Do not discard a completed send when shutdown starts.

If send returns nil for a 204 and Close cancels f.ctx before this check, attempt skips both counting and finalization. The next worker sends the same webhook again after the lease expires. If the destination rejects the duplicate with 409, activity reports a refusal instead of the successful delivery. Waiting for the worker before closing the recorder does not recover that outcome. (raw.githubusercontent.com)

Use the send result to distinguish a completed response from a transport operation interrupted by shutdown. Finalize a known 2xx with the existing bounded write context. Leave genuinely interrupted sends available for retry.

Nitpick comments: 1

acceleration/internal/api/hook_activity_test.go: 1

79-79: Trivial · Maintainability & Code Quality · Quick win

Test a held-number call that reaches a worker.

This test sends the same no-worker call as TestACallNoWorkerCanTakeIsCountedRefused. It verifies ownership, but it would still pass if every held-number call were marked refused. Register a worker, verify that it receives the call, and assert row.refused == 0. Keep the separate no-worker test for the refusal case. (raw.githubusercontent.com)

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between ee84a68 and b9b22a4.

Files selected for processing: 24

  • CHANGELOG.md
  • acceleration/README.md
  • acceleration/cmd/router/main.go
  • acceleration/internal/activity/activity.go
  • acceleration/internal/api/activity.go
  • acceleration/internal/api/activity_test.go
  • acceleration/internal/api/activity_unit_test.go
  • acceleration/internal/api/admission_test.go
  • acceleration/internal/api/callhooks.go
  • acceleration/internal/api/chatinbound.go
  • acceleration/internal/api/connector_events.go
  • acceleration/internal/api/hook_activity_test.go
  • acceleration/internal/api/hookorigin.go
  • acceleration/internal/api/messagehooks.go
  • acceleration/internal/api/server.go
  • acceleration/internal/api/socketws.go
  • acceleration/internal/api/telnyxhook.go
  • acceleration/internal/channels/service.go
  • acceleration/internal/eventforward/AGENTS.md
  • acceleration/internal/eventforward/forwarder.go
  • acceleration/internal/eventforward/forwarder_activity_test.go
  • acceleration/internal/phone/service.go
  • acceleration/internal/store/activity.go
  • acceleration/internal/store/activity_test.go
Files skipped from review as they are similar to previous changes: 1

  • CHANGELOG.md

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

func (s *ActivitySuite) refusedSocket(start map[string]any) map[string]any {
connection := s.serverClient.with(clientHeader, store.AuditSourceSDK).opens("/v1/agents/socket")
s.Require().NoError(connection.WriteJSON(start))
return s.await(connection, "error")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Assert the client-visible error frame.

All four tests pass if the server sends only {"type":"error"} and writes the correct activity row. await checks type, but none of the tests checks that the error message is present. Check that error is a nonempty string and that code and error_type are absent for these message-only refusals. Keep the activity assertions separate. (raw.githubusercontent.com)

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

// dispatchArrivingCall works out whose call it is and hands it to one of their workers.
func (s *Server) dispatchArrivingCall(r *http.Request, origin hookOrigin, event callEvent, payload []byte) {
// dispatchArrivingCall works out whose call it is and hands it to one of their workers. A call
// it refuses is still answered 200, so the refusal's code is noted on w for the activity record.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Heavy lift

Record refusals from answerWithAgent in customer_activity.

dispatchArrivingCall starts answerWithAgent in a goroutine (callhooks.go:253) and returns, so the hook answers 200 first. Two refusals inside answerWithAgent do not reach the activity record. One is the config lookup failure at callhooks.go:303, which includes a number whose ConfigID no longer resolves. The other is the session open failure at callhooks.go:342. Both call refuseArrivingCall, which notes no refusal code. The request stays recorded under hooks_in as a success, with no refusal and no code.

These refusals need to reach customer_activity with their code. The socket path already counts refusals after its 101 this way. Do not write to the response writer after the response is sent. Do not write the request's activity note from the goroutine without synchronization with the request's own record write.

// A suspension that cannot be read refuses the call too: it is there to stop fraud that
// costs money.
if _, err := s.store.TelephonySuspension(r.Context(), number.CustomerID); !errors.Is(err, store.ErrNotSuspended) {
noteRefusal(w, codeTelephonySuspended)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Use a different refusal code when suspension status cannot be read.

If TelephonySuspension returns a database timeout, this branch correctly refuses the call, but records codeTelephonySuspended. The activity row then reports a confirmed suspension when the router could not determine the suspension state. Keep the refusal and the HTTP 200 acknowledgement. Record an internal error for a lookup failure and reserve codeTelephonySuspended for a confirmed suspension.

s.logger.Error("nobody could answer an arriving call",
"call", event.CallCid, "session_id", event.SessionID, "customer", number.CustomerID,
"number", number.E164, "error", err)
noteRefusal(w, unavailable("").Code)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Heavy lift

Record a peer's refusal of a forwarded call in the ingress activity.

When the ingress node hands a call to a peer, it treats any 200 response as handled and returns without noting a refusal. If the peer refuses the call, for example when its worker assignment fails at callhooks.go:282, the peer still answers 200. The peer records the refusal code only on its own writer, and it does not count forwarded hooks. The refusal therefore appears in no activity row. The ingress row for hooks_in shows the call as answered.

The ingress activity record for a forwarded call must reflect the refusal the peer returned. The peer's refusal code must reach the ingress node's record. Refusal counts for customers are then correct across nodes.

if !ok {
return
}
s.noteHook(r.Context(), origin)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Data Integrity & Integration · Heavy lift

Count deployment-mode message hooks after resolving the customer.

A signed message.new in deployment mode has no customer when noteHook runs. If ownerOf later resolves the channel's stored config to app A, the router can dispatch the message to A's worker. The activity note remains empty, so the recorder drops the hook. App A's activity report misses the request. (raw.githubusercontent.com)

Keep the early note for gateway-vouched hooks. For shared deployment hooks, set the note when routing resolves a trusted owner, before any peer handoff. Leave unresolved or ambiguous hooks uncounted. Use the resolved owner, not an app ID taken directly from the payload.

Comment on lines +126 to +128
s.Equal(int32(0), hits.Load())
s.Equal(int64(1), row.refused)
s.Equal(codeNotSent, row.code.String)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Assert one total count for a forward that was not sent.

This test passes if the router writes both a forward_failed refusal and an incorrect success for the same forward. The destination still sees zero requests, and row.refused remains 1. Assert row.requests == 1 as the other outcome tests do.

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

attempts := delivery.Attempts + 1
switch {
case err == nil:
f.count(delivery, "")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Data Integrity & Integration · Heavy lift

Commit one activity outcome per queued forward.

If the destination returns 204 but FinishEventDelivery fails before deleting the row, the first count remains buffered. After the lease expires, a worker can send and count the same row again. The give-up branch has the same problem. The daily activity total can then show two completions for one queued forward. (raw.githubusercontent.com)

Commit one durable activity outcome in the same transaction that finalizes the claimed row. If aggregation stays asynchronous, consume that outcome idempotently. Add a regression test that rejects the first finalization, lets the lease expire, and verifies that recovery adds only one count. Moving count after the delete alone leaves a gap if the process exits between the two operations.

Also applies to: 529-529

return Bridge{}, err
}
// The vendor's hook asking is the number's app's, however it is answered below.
activity.NoteCustomer(ctx, stored.CustomerID, activity.AreaPhoneSMS)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Heavy lift

Count Telnyx refusals when the attached number's trunk cannot be opened.

A signed call to an attached number gets no activity row if the router has no keyring or the attachment predates stored trunk credentials. Both paths return before this note. receiveTelnyxCall then sends a 200 hangup before reaching its refusal markers. The existing Telnyx tests exercise both hangups. (raw.githubusercontent.com)

Resolve and note the attached number's owner independently of opening its trunk. Mark both hangup branches as refused without changing the TeXML response. Unknown, released, or unattached numbers should still produce no customer row.

Comment thread acceleration/README.md
`areas` is one item per area and client, `meters` one per modality (requests, tokens, audio,
characters, time and cost), and `first_seen` and `last_seen` the first and last request of any
area. `furthest_step` is the furthest of `dashboard_reads`, `agent_setup` and `conversations`
area. A hook counts under the app its signature or token was checked for, from client `api`:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Reconcile the client recorded for verified hooks with the README.

The README says verified hooks count from client api. recordActivity records every request under the client named in X-Stream-Client, including hooks, and the hook branch does not replace it. A hook that sends X-Stream-Client: sdk is counted under sdk, and the OpenAPI description of api does not cover it either.

Decide which behavior is intended, then make the README, the OpenAPI description, and recordActivity agree. If hook rows must always be api, the hook branch must set the client. If the header applies to hooks, the README and the OpenAPI description must say so.

Tools

LanguageTool

[grammar] ~2156-~2156: Ensure spelling is correct
Context: ...token was checked for, from client api: phone_sms for a phone or SMS vendor's, else `hooks_in...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

kanat and others added 2 commits November 18, 2026 14:03
…2073)

## What

Every router log line where a customer's caller or writer gets stuck now
names the app (`customer=<app id>`), so a line in the log store says
whose number or channel it was. Logging only: no status, body or frame
changes.

```
Telnyx ──signed call──▶ telnyxhook ──InboundBridge──▶ number → app noted (activity note, #2066)
                              │                                   │
                              │◀──── activity.NotedCustomer ──────┘
                              ▼
   level=WARN msg="rejected a telnyx call to a number of an app whose telephony is suspended" customer=<app id> to=+1•••42
```

| Path | Change |
| -- | -- |
| Telnyx inbound call (`internal/api/telnyxhook.go`) | `customer` on
every line after the number resolves; the two rejections (suspended, may
not pay) Info → Warn |
| Outbound answer callback (`internal/api/phone.go` answerPhoneCall) |
`customer` on both Error lines |
| Stream arriving call, no number (`internal/api/callhooks.go`) | Debug
→ Warn, keeps `stream_app`; a video call (no number carried, not named
`phone-…`) stays Debug |
| SMS/WhatsApp channel (`internal/channels/service.go` answer) |
`customer` on every line of the answer path |
| Chat message not answered (`internal/api/messagehooks.go`) |
`customer` from the session's spec on the three lines |
| Campaign dial (`internal/campaign/campaign.go`) | `customer` from the
campaign |
| Connector refresh warning (`oauth2code/credential.go`) | `customer`
from the connection ref |

### How the app is found

- Telnyx and the answer callback: `phone.InboundBridge` and
`phone.Answer` already note the number's app on the request for the
activity record, before the suspension and payment checks.
`activity.NotedCustomer(ctx)` reads it back; there is no second lookup.
- Everything else already holds the app (account, session spec,
campaign, connection ref).

### Why the Stream call line is not Warn for every call

Every `call.session_started` in the app reaches `dispatchArrivingCall`,
video calls included (`callhooks.go` comment; store test
`some-video-call`). A Warn on each would flood the log.
`store.NumberByCallInApp` now matches `store.ErrNoNumber` only for a
call that is not named for a number, and only that case stays Debug. A
call that carries a number, or is named `phone-…`, warns.

## Tests

`internal/api/stuck_logs_test.go` (integration): suspended Telnyx app
(AC-4), may-not-pay Telnyx app, answer callback, channel line with no
agent, Stream call that rang a number / named for a number / video call.
Existing tests strengthened: trunk-password Telnyx test,
speech-to-speech message test, oauth2code refresh warning, store
`ErrNoNumber`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
## What

Every morning one router posts yesterday's (UTC) Agents activity to the
money notices' Slack channel: the new organizations and how far each
got, the organizations whose every request was refused, and the
active-organization count against the day before.

```
[STAGING] Agents yesterday (13 Nov, staging)        <- the tag is empty on prod
New orgs (first request yesterday): 2
  • `org-dental`   reached: conversations   agent_setup (cli) → workers (sdk) → conversations (api) + model_api (sdk)
  • `org-pizza`   reached: —   all 14 requests refused (invalid_api_key)
All requests refused: 1
  • `org-pizza`   14 requests (invalid_api_key)
Active orgs: 41 (day before: 38)
```

(The org ids above are made up.)

## How

```
cmd/router/limits.go  costAnomalyOptions(..., Digest: true)          <- production caller
        │
limitnotice.Anomalies.check   (hourly tick, holds the cost anomaly lock)
        ├─ postDigest(now)      (new)
        │    ├─ before 00:05 UTC (digestSettle)? → wait for the next tick
        │    ├─ store.ClaimActivityDigest(yesterday)   INSERT … ON CONFLICT (day) DO UPDATE … WHERE not posted AND stale
        │    │      lost → return (another router or an earlier check has it)
        │    ├─ store.DayActivity(yesterday)           customer_activity by org, area, client; org '' skipped
        │    ├─ store.ActiveOrganizations(day before)
        │    ├─ Poster.Post(digestMessage)             same Slack client as the anomaly notice
        │    └─ store.MarkActivityDigestPosted
        └─ note(...)            (unchanged cost anomaly check)
```

```mermaid
sequenceDiagram
    participant R1 as Router pod 1 (holds lock)
    participant R2 as Router pod 2
    participant PG as Postgres activity_digests
    participant S as Slack
    R1->>PG: claim 2026-11-13 (insert if absent)
    PG-->>R1: won
    R1->>S: post digest
    R1->>PG: posted_at = now()
    Note over R2: restarts and takes the lock later the same day
    R2->>PG: claim 2026-11-13
    PG-->>R2: lost (already posted)
```

| Piece | File |
|---|---|
| Marker table `activity_digests (day PK, claimed_at, posted_at)` |
`acceleration/migrations/20270102090000_activity_digests.sql` (additive;
slot to be re-slotted by ship-feature) |
| Claim, mark, day read, active count |
`acceleration/internal/store/activity_digest.go` |
| Digest build + message | `acceleration/internal/limitnotice/digest.go`
|
| Hook into the hourly check |
`acceleration/internal/limitnotice/anomaly.go` (`AnomalyOptions.Digest`)
|
| Shared ladder `activity.Steps` / `FurthestStep` |
`acceleration/internal/activity/steps.go`, used by
`internal/api/org_activity.go` too |

- **Once only.** The day row decides who posts, not the lock alone: a
restart, or a lock handed to another pod, finds the day claimed. If a
post fails, the claim stays unposted. The first check after `stale` (5
min, the limit notices' value) claims it again, so a failed post is
retried.
- **"Reached" ladder.** `dashboard_reads → agent_setup → workers →
conversations → voice_video → phone_sms`. The ladder now lives in
`internal/activity`, so the ops endpoint `GET
/v1/ops/orgs/{org_id}/activity` and the digest give the same answer.
That endpoint's `ActivityStep` enum gains `workers`, `voice_video` and
`phone_sms`. `openapi.yaml` and the clients are regenerated: Go, JS,
Python, Rust and PHP changed. Ruby, .NET, Swift and Kotlin were
regenerated or checked with no diff. Dart is not affected: it generates
only client-accessible operations.
- **First reader of `customer_activity_organization_day`.** The "new
org" check is `NOT EXISTS (… organization_id = ca.organization_id AND
day < yesterday)`.
- **AC-6 check.** The message's active count is the same query as
`select count(distinct organization_id) from customer_activity where day
= <yesterday> and organization_id <> ''`.

## Tests

- `internal/limitnotice/digest_unit_test.go` (no DB): the full message;
14/14 refused flagged vs 13/14 not; the most common refusal code and its
tie-break; the list cap with "… and N more"; the environment tag; the
Staff link; grouping rows by org.
- `internal/limitnotice/digest_test.go` (integration, own DB): first-day
org vs one active the day before; org `''` skipped; active count = the
SQL distinct count, with the day before counted separately. Also: two
routers at once, a restarted router, and a re-check after the claim ages
each post once; 8 claims at once give one winner; a failed post is
retried once; no post before 00:05 UTC; no digest without the option.
- `internal/api/org_activity_unit_test.go`: workers, voice_video and
phone_sms in ladder order.
- `cmd/router/limits_test.go`: the production options turn the digest
on.
- Mutations: 45 mutants, all killed (script and log in the author
notes).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between b9b22a4 and dd9de7c.

Files selected for processing: 15

  • CHANGELOG.md
  • acceleration/internal/activity/activity.go
  • acceleration/internal/api/callhooks.go
  • acceleration/internal/api/hook_activity_test.go
  • acceleration/internal/api/messagehooks.go
  • acceleration/internal/api/messagehooks_recordings_router_test.go
  • acceleration/internal/api/phone.go
  • acceleration/internal/api/stuck_logs_test.go
  • acceleration/internal/api/telnyxhook.go
  • acceleration/internal/campaign/campaign.go
  • acceleration/internal/channels/service.go
  • acceleration/internal/connectors/schemes/oauth2code/credential.go
  • acceleration/internal/connectors/schemes/oauth2code/credential_test.go
  • acceleration/internal/store/hooks.go
  • acceleration/internal/store/store_test.go
Files skipped from review as they are similar to previous changes: 3

  • CHANGELOG.md
  • acceleration/internal/api/messagehooks.go
  • acceleration/internal/channels/service.go

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

Comment on lines +301 to +302
return strings.Contains(s.logged.String(), `msg="could not answer a message" channel=`+s.channelID+" session=") &&
loggedOneLine("ERROR", "could not answer a message", s.customerID()).MatchString(s.logged.String())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Match the channel, session, and customer on the same log line.

The first predicate can match this channel's failure without a customer field. The second can match another channel's failure with the expected customer. The test then passes even though this channel's failure lacks attribution.

Read the log once per poll and require one line to contain the error level, message, channel, opened.Id, and expected customer. As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

// Defect: a call that rang an app's number, which no attached number reaches, logged at Debug,
// which production does not keep.
func (s *CallHooksSuite) TestACallThatRangANumberNoNumberReachesIsWarnedAbout() {
s.Require().Equal(http.StatusOK, s.arriveRingingNumber())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Quick win

Use a call ID that does not encode the number.

arriveRingingNumber() sends custom.phone_number, but the suite also gives the call a phone- ID. If the router ignores custom.phone_number, the legacy lookup still warns for that ID, so this test passes. Call s.answerable() before delivery and match the resulting call ID in the warning. Keep the phone-prefixed ID in the separate legacy test. (raw.githubusercontent.com)

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

renewed := s.accessCredentialDue(scheme, stored, resolved)
s.Contains(log.String(), "the refresh token expires before the next refresh")
s.Contains(log.String(), `"connection":"conn-1"`)
s.Contains(log.String(), `"customer":"`+s.ref.CustomerID+`"`, "the warning names the app that has to reconnect")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Quick win

Check customer on the expiry warning itself.

If the expiry warning omits customer but a separate refresh-success record includes it, all three substring checks pass. Decode the JSON records and assert that the expiry WARN contains both the expected connection and customer. Keep the secret checks over the whole buffer. (raw.githubusercontent.com)

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

Nitpick comments: 1

acceleration/internal/activity/steps.go: 1

12-12: Trivial · Maintainability & Code Quality · Quick win

Use one read-only source for the activity ladder.

FurthestStep and the digest read activity.Steps, but ActivityStep.Schema lists the same steps separately. If a future step is added only to Steps, the API can emit it without declaring it in OpenAPI. Keep the ladder private and expose a copy for the digest and schema. The schema can add none and steady_use around that copy. (github.com)

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between dd9de7c and 625d45f.

Files ignored due to path filters: 4

  • plugins/stream/vision_agents/plugins/stream/_generated/models/activity_step.py is excluded by !**/_generated/**
  • plugins/stream/vision_agents/plugins/stream/_generated/models/organization_activity.py is excluded by !**/_generated/**
  • sdks/js/src/generated/api.ts is excluded by !**/generated/**
  • sdks/php/src/Generated/ActivityStep.php is excluded by !**/generated/**
Files selected for processing: 17

  • CHANGELOG.md
  • acceleration/README.md
  • acceleration/api/openapi.yaml
  • acceleration/cmd/router/limits.go
  • acceleration/cmd/router/limits_test.go
  • acceleration/cmd/router/main.go
  • acceleration/internal/activity/steps.go
  • acceleration/internal/api/org_activity.go
  • acceleration/internal/api/org_activity_unit_test.go
  • acceleration/internal/limitnotice/anomaly.go
  • acceleration/internal/limitnotice/digest.go
  • acceleration/internal/limitnotice/digest_test.go
  • acceleration/internal/limitnotice/digest_unit_test.go
  • acceleration/internal/store/activity_digest.go
  • acceleration/migrations/20270102090000_activity_digests.sql
  • sdks/go/acceleration/generated.go
  • sdks/rust/src/types.rs
Files skipped from review as they are similar to previous changes: 2

  • CHANGELOG.md
  • acceleration/cmd/router/main.go

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 2 remain after this review.

Comment on lines +10 to +11
// it reads the dashboard, sets an agent up, runs a worker, then holds conversations, by voice
// or video, then on a phone line or SMS. The other areas are used along the way and are no step.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Quick win

Describe these stages as answered requests, not completed conversations.

An app can get phone_sms by listing its phone numbers, even when the list is empty. The router counts the successful request in that area. It does not require a call or SMS. State that distinction so readers do not interpret phone_sms as evidence that a conversation occurred. (raw.githubusercontent.com)

areaWith(activity.AreaVoiceVideo, 1), areaWith(activity.AreaConversations, 1),
}, 0))
s.Equal(activityStepPhoneSMS, furthestStep([]store.ActivityArea{
areaWith(activity.AreaVoiceVideo, 1), areaWith(activity.AreaPhoneSMS, 1),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Test that refused phone activity does not advance the stage.

All new fixtures have Refused == 0. The existing refusal test uses only conversations. This suite would still pass if the classifier skipped the refusal check for the new steps. Add a test where an answered dashboard_reads row outranks a phone_sms row with one request and one refusal. Keep the answered phone_sms case as the control.

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

Comment on lines +56 to +57
string(activityStepWorkers), string(activityStepConversations), string(activityStepVoiceVideo),
string(activityStepPhoneSMS), string(activityStepSteadyUse))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Document the Rust SDK source break.

The new schema values become named variants in the public Rust ActivityStep enum. A caller that matched every previous variant, including Unlisted(_), no longer compiles after upgrading. Unlisted protects deserialization, not exhaustive matches. Record this under Breaking Changes and tell callers to add the new arms or use a wildcard. (github.com)

As per path instructions, “The changelog is for user-facing changes, organized under version headings and Breaking Changes, New Features, and Bug Fixes.”

Source: Path instructions

return
}
if a.digest {
a.postDigest(checking, now)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Stability & Availability · Quick win

Give spend alerts a deadline independent of the digest.

If the digest query uses the full two-minute check budget, note receives an expired context. The router then skips both detecting new spend anomalies and retrying pending alerts. Before this change, the same spend check had the full budget.

Run the existing spend check with its own budget. Give digest work a separate bounded context so a slow digest cannot consume that budget, and an early return from the spend check cannot suppress the digest.

// checked waits until the checks running have read the clock past reads more than once, so
// at least one whole check ran after them.
func (s *DigestSuite) checked(reads int64) {
s.Require().Eventually(func() bool { return s.reads.Load() >= reads+2 }, settleFor, time.Millisecond)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Wait for the digest post before asserting its count.

checked waits for two clock reads, but each router reads the clock before it tries the lock. The non-holding router keeps ticking while the holder's postDigest is still running, so checked can return before the post exists. The s.Len(s.digests(), 1) assertion then reads zero digests and fails. The test fails only when the post runs slower than about two non-holder ticks, so it is intermittent.

Wait for the post to appear before the first count assertion:

Suggested fix

 	s.checked(0)
+	s.Require().Eventually(func() bool { return len(s.digests()) == 1 }, settleFor, time.Millisecond,
+		"the holder posts the digest")
 	s.checked(s.reads.Load())
 	s.Len(s.digests(), 1, "two routers, a check each every tick, one post")

s.Equal(2, strings.Count(message, "\n … and 2 more"), "once under each list")
s.NotContains(message, fmt.Sprintf("org-%02d", digestListMax))
s.Contains(message, "Active orgs: 17 ", "the count is of every organization, not the ones listed")
s.Equal(15, digestListMax, "two lists of 15 lines of up to 1,000 characters stay under Slack's 40,000")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Test the rendered message against Slack's text limit.

This assertion checks that digestListMax is 15. It does not check the length of the message. Longer rows can pass every assertion here while putting the totals past Slack's truncation point. Keep the row-count checks. Add a fixture with long IDs and Staff links, then check that the rendered text stays within 40,000 characters and includes the totals. (docs.slack.dev)

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

// writeList writes a line for each of the first digestListMax organizations, and how many more
// there are.
func writeList(b *strings.Builder, organizations []OrganizationDay, line func(OrganizationDay) string) {
for _, organization := range organizations[:min(len(organizations), digestListMax)] {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Bound the Staff link length, or cap the rendered message.

The size bound in digestListMax assumes a Staff link of about 1,000 characters or less. StaffURL has no length check, and link applies no cap. Each of up to 30 rows repeats the link. A template of roughly 1,300 characters or more can push the message past Slack's 40,000-character text limit. Slack then cuts the message off, and the Active orgs totals line is lost.

Enforce a maximum length on limits.staff_url in config validation, or cap the total rendered message length before posting. Choose one. The chosen bound must hold for the worst case of 30 rows.

if now.Before(today.Add(digestSettle)) {
return
}
day := today.AddDate(0, 0, -1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Stability & Availability · Heavy lift

Retry unposted digests after midnight UTC.

If Slack rejects the March 9 digest at 23:30 UTC on March 10, the next hourly check selects March 10. No later check selects March 9, even though its claim is stale. The operator never receives that digest.

In addition to yesterday's digest, process a bounded batch of stale, unposted days. Keep the atomic per-day claim. Use the digest date instead of “yesterday” for a backfilled message.

a.logger.Error("could not post the activity digest; it is retried", "day", day.Format(time.DateOnly), "error", err)
return
}
if err := a.store.MarkActivityDigestPosted(ctx, day); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Document that a successful Slack post can be repeated.

If Slack accepts a post but the posted_at write fails before committing, the next check can reclaim the day and send the same digest again. This contradicts the once-per-day guarantee in the posting and option comments. Describe this as best-effort deduplication. Give the final write a short, separate deadline so an expired check context does not create an avoidable duplicate.

);

-- +goose Down
DROP TABLE activity_digests;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Preserve digest delivery state across rollback.

If an operator rolls back after Slack receives the digest, DROP TABLE deletes the posted marker. If the operator redeploys on the same UTC day, the router claims the day again and sends a duplicate. Use an application-only rollback that retains this table, and document why the database migration must remain applied.

As per coding guidelines, flag “a down migration that loses data.” The lost data here is the delivery state.

Tools

Squawk (2.70.0)

[warning] 15-15: Dropping a table may break existing clients.

(ban-drop-table)

Source: Coding guidelines

if _, err := found.Ask(ctx, text, images...); err != nil {
s.logger.Error("could not answer a message",
"channel", event.ChannelID, "session", found.ID(), "error", err)
"channel", event.ChannelID, "session", found.ID(), "customer", found.Spec().CustomerID, "error", err)
if err != nil {
r.logger.Warn("a campaign dial failed", "campaign", campaign.ID, "contact", contact.ID, "error", err)
r.logger.Warn("a campaign dial failed", "campaign", campaign.ID, "contact", contact.ID,
"customer", campaign.CustomerID, "error", err)
kanat and others added 2 commits November 18, 2026 14:25
… events and 10DLC reports per customer (#2071)

## What

AI-3334, part B of AI-2380 (#2066): the work the router does for an app
on its own now leaves a row in `customer_activity`, under the app the
job holds (its organization is read from `app_organizations` by the
existing upsert), with client `api` (`activity.ClientRouter`). Serves
AC-1 of AI-3330.

```
scheduler.run ──deliverWithRetries (1..3 attempts)──▶ finish ──stored──▶ count ─┐
campaign.Runner.ring ──Dialer.Dial──▶ countDial ────────────────────────────────┤
simulation.Runner.run ──cases──▶ settle ────────────────────────────────────────┼─▶ activity.Recorder.Record
MCP server ──signed delivery──▶ mcpevents.Receive ─ NoteCustomer ─┐             │      (once a minute)
registrar ──signed report────▶ dlc.Hook ───────── NoteCustomer ─┴▶ withRequestLog ┘
```

| Work | Where it is counted | Area | Refused when (code) |
|---|---|---|---|
| Scheduled task occurrence | `scheduling.Scheduler.finish`, once the
run is stored | `scheduling` | failed (the run's reason, e.g.
`no_phone_number`), missed (`missed`), schedule unreadable |
| Campaign dial | `campaign.Runner.ring`, after the dial | `phone_sms` |
the dial errored (`dial_failed`) or placed no call (its reason, else its
result: `insufficient_credits`, `invalid_number`) |
| Simulation run | `simulation.Runner.settle` | `evals` | the run
errored (`errored`) |
| MCP connection event | `mcpevents.Receive`, after `webhook.Verify` |
`hooks_in` | any 4xx/5xx after the signature check |
| 10DLC campaign report | `dlc.Hook`, once the report names a use case |
`phone_sms` | any 4xx/5xx after that |

## Decisions

- **Scheduled tasks are counted where the scheduler settles the
occurrence, not in `deliverChain`/`deliverOn`.** `deliverWithRetries`
calls the delivery once per attempt, a hand-off runs `deliverChain` on
the peer, and a thread delivery never reaches `deliverChain`. At
`finish` every path counts once, as the forwarder counts a forward once
and not per attempt.
- Not counted until the occurrence settles: a quiet-hours hold, a retry
left to its lease by a drain, a claim lost to an edit, a failed run
write (the next claim settles and counts it). A run settled before its
advance was lost is not counted again, except one the claim itself
failed because its workers were lost.
- A dial a shutdown cut off (`not_placed`) rang nobody and the contact
is rung again, so it is not counted. A voicemail is a placed call.
- A run the judge failed, or a cancelled one, is counted and not refused
(see Questions).
- A 10DLC brand report names no app the hook reads, so it is not
counted.
- `Recorder.Record` on a nil recorder does nothing, so the three runners
need no nil check of their own.

## Tests

Each test names the defect it catches.
- scheduling: delivered once; failed after 3 attempts counts 1, refused
under its reason; missed counts refused `missed`; a hold then delivery
counts 1; a run settled before its advance counts 0; a workers-lost run
counts refused.
- campaign: both dials of a double dial count, not refused; voicemail
not refused; money refusal under its reason; invalid number under its
result; dialer error `dial_failed`; a dial a stopping router did not
place counts 0.
- simulation: errored refused; judge-failed and cancelled not refused.
- api: a signed MCP event counts once and a forged one not; a signed MCP
event the router turns down (400) counts refused; a signed 10DLC report
counts as `phone_sms` (not `hooks_in`) and a forged one not.

23 mutants (every guard, code fallback and note placement), all killed;
review round 1 added 4 more tests (fenced settle, unwritten run,
not-JSON MCP delivery, failed 10DLC report). A task run by hand ("Run
now") is counted under `scheduling` too, refused unless it delivered.
Untested: the `cmd/router/main.go` wiring.

## Questions

- Simulations: should a run the judge failed, or one cancelled by its 30
min timeout, count as refused? Now only `errored` does.
- 10DLC brand reports: count them under the brand's app (needs a lookup
by vendor brand id)?

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Nitpick comments: 2

acceleration/internal/campaign/runner_activity_test.go: 1

66-66: Trivial · Maintainability & Code Quality · Quick win

Assert the two persisted dial attempts.

Len(s.dialer.dialed(), 2) checks calls to the stub, not what the runner recorded. Read the campaign’s persisted attempts and check that they contain dial numbers 1 and 2. Keep the activity assertion for two requests and zero refusals.

As per coding guidelines, “Assert on outputs and state, never on which method was called.”

Source: Coding guidelines

acceleration/internal/scheduling/scheduler_activity_test.go: 1

78-78: Trivial · Maintainability & Code Quality · Quick win

Assert delivery outcomes instead of callback counts.

Use observable delivery outcomes to prove that retries took effect, a missed occurrence did not deliver, and a held occurrence delivered after the hold. Keep the exact activity-count assertions so the tests still catch double counting. As per coding guidelines, “Assert on outputs and state, never on which method was called.”

Also applies to: 92-92, 108-108

Source: Coding guidelines

Additional comments: 1

acceleration/internal/campaign/runner_activity_test.go: 1

34-34: Trivial · Maintainability & Code Quality · Heavy lift

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Inject the failure at the provider boundary.

The error case replaces the entire campaign Dialer. It can pass even if the production phoneDialer maps the same provider failure to a result that is skipped or misclassified. Keep the production dialer and make the registered test provider return the error. Assert on the same stored activity row.

As per coding guidelines, “A provider is a stub registered in its router’s registry, or the real one behind an integration build tag; everything else is real.”

Source: Coding guidelines

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 625d45f and 7f87f43.

Files selected for processing: 16

  • CHANGELOG.md
  • acceleration/README.md
  • acceleration/cmd/router/main.go
  • acceleration/internal/activity/activity.go
  • acceleration/internal/api/dlc_test.go
  • acceleration/internal/api/hook_activity_test.go
  • acceleration/internal/api/scheduled_tasks.go
  • acceleration/internal/api/scheduled_tasks_test.go
  • acceleration/internal/campaign/campaign.go
  • acceleration/internal/campaign/runner_activity_test.go
  • acceleration/internal/dlc/service.go
  • acceleration/internal/mcpevents/mcpevents.go
  • acceleration/internal/scheduling/scheduler.go
  • acceleration/internal/scheduling/scheduler_activity_test.go
  • acceleration/internal/simulation/activity_test.go
  • acceleration/internal/simulation/simulation.go

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 2 remain after this review.

Comment on lines +296 to +301
before, after := s.hooksAfter(func() {
s.Require().Equal(http.StatusUnauthorized, s.provider.DeliverEvent(subs[0].URL, subs[1].Secret, subs[0].ID,
"evt_"+s.utils.uuid(), subs[0].Name, map[string]any{}))
s.Require().Equal(http.StatusAccepted, s.provider.DeliverEvent(subs[0].URL, subs[0].Secret, subs[0].ID,
"evt_"+s.utils.uuid(), subs[0].Name, map[string]any{}))
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Quick win

Check attribution after each delivery.

Both tests check one combined delta for the unauthorized and authorized deliveries. They still pass if the recorder counts only the unauthorized delivery as a non-refused request. Flush and assert no change after the unauthorized delivery. Then send the authorized delivery and assert that it adds one request.

As per path instructions, flag “tests that would still pass on a broken implementation.”

Also applies to: 331-333

Source: Path instructions

s.activityWritten()
before := s.counted(activity.AreaScheduling, activity.ClientRouter)

s.Require().Equal(http.StatusAccepted, s.serverClient.do(http.MethodPost, "/v1/agents/scheduled-tasks/"+created.Id+"/run", nil, nil))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Quick win

Verify that the accepted run reaches delivered.

The test discards the run ID. It can pass if the router records successful activity at dispatch but never delivers the run. Decode the 202 response and wait for that run ID to reach delivered before checking the activity totals.

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

Comment on lines +3500 to +3505
s.Require().Eventually(func() bool {
return s.counted(activity.AreaScheduling, activity.ClientRouter).refused == before.refused+1
}, settleFor, 10*time.Millisecond, "the failed run by hand was not counted refused")
after := s.counted(activity.AreaScheduling, activity.ClientRouter)
// The request that set it going, and the run.
s.Equal(before.requests+2, after.requests)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Stability & Availability · Quick win

Wait for both activity increments before checking the failure reason.

The manual run can record its refusal before the HTTP middleware records the accepted request. If the recorder flushes between those events, the refusal poll succeeds while the total is still before.requests+1. Poll until a single counted snapshot contains both expected deltas. Check the refusal code on that snapshot.

// Hook takes a vendor's report on a brand or a campaign, verified, and moves whichever use
// cases it is about.
// cases it is about. A report on a campaign is counted as its use case's app's phone work. One on
// a brand is not: the hook reads no app for it, and has every waiting use case registered.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Maintainability & Code Quality · Quick win

Describe brand registration as a poll, not a guarantee.

A brand report calls Poll once. Poll reads one page and logs refresh errors without returning them to the hook. The hook can acknowledge the report while use cases are still waiting. Say that a brand report triggers a poll of pending use cases. (raw.githubusercontent.com)

task := s.task(time.Now().UTC().Add(-time.Hour).Truncate(time.Second), nil)
stop := s.counting((&deliverer{}).deliver, nil)

s.Require().Equal(store.ScheduledTaskRunDelivered, s.settled(task)[0].State)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Keep fatal assertions on the test goroutine.

s.Require() calls FailNow, which must run on the test goroutine. Two paths in this file break that rule:

  • The fence and lose deliver callbacks call s.Require().NoError on a scheduler worker goroutine.
  • The Eventually conditions call s.read and s.runs, and those helpers in scheduler_test.go call s.Require().NoError. The settled helper, used on lines 60, 75 and 89, polls through the same path.

A failed FailNow stops only that goroutine. The worker never returns its result, so the test waits for its deadline and reports a timeout instead of the query error.

Have the read, runs and settled helpers return errors. Record callback errors in a test-owned variable and assert it on the test goroutine after the wait. Poll with EventuallyWithT, using its collector for assertions.

// run this claim failed itself, its workers lost, is not running: the next claim
// finds it settled by somebody else, and it is never counted.
log.Error("could not record a scheduled task run", "error", err)
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Quick win

Keep the attempt-cap refusal count when task advancement fails.

The claim transaction can commit a failed run at the attempt cap before finish runs. If finish then hits a transient database error, it returns without counting. After lease expiry, the next claim advances the task but skips counting because the run is already terminal and WorkersLost is now false. The refusal is lost even though the run is durably failed. (raw.githubusercontent.com)

Count the cap transition from the committed claim, independently of task advancement. Keep the ordinary finish-error path uncounted when the run has not settled.

case !applied:
log.Info("a scheduled task changed under its run; the schedule was left alone")
}
s.count(claim, outcome)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Heavy lift

Count the run settlement, not each successful finish call.

If worker A loses its lease, worker B can reclaim the same run, finish it, and count it. If A later returns success, FinishScheduledTaskRun still writes the run and returns (false, nil) because A cannot advance the task. Line 622 counts the same run again. A's AlreadySettled flag describes the run when A claimed it, not when A finishes. (raw.githubusercontent.com)

Have the store distinguish a newly settled run from a run another worker already settled. Preserve the count when a fenced finish is the first to settle the run. Do not use applied alone; it only reports whether the task advanced.

record, err := activity.New(activity.Options{Store: s.db, Interval: time.Hour, Logger: slog.New(slog.DiscardHandler)})
s.Require().NoError(err)
runner := &Runner{store: s.db, activity: record, logger: slog.New(slog.DiscardHandler)}
run := store.SimulationRun{ID: "run-" + rand.Text(), CustomerID: "app-" + rand.Text(), State: state, Cases: 1}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Create a stored run before testing settlement.

settled generates a run ID but never inserts the run. FinishSimulationRun updates zero rows, and the tests still pass because they read only customer_activity. They would also pass if the finish call were removed. Create a running row through the store, then assert both its persisted terminal state and its activity row. (raw.githubusercontent.com)

As per path instructions, flag “tests that would still pass on a broken implementation.”

Source: Path instructions

if run.State == store.SimulationErrored {
request.Refused, request.Code = true, store.SimulationErrored
}
r.activity.Record(request)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Data Integrity & Integration · Heavy lift

Count simulation runs that recovery marks as errored.

If a router dies before settle, another router marks the expired run SimulationErrored through AbandonSimulationRuns. That path never records activity. The run history shows an error, but the daily digest omits both the run and the refusal. Record the recovery transition for each run that recovery actually changes. A second recovery pass must not count it again. (raw.githubusercontent.com)

Staging applies goose in strict order; the next-day 09:00 slot stays free
for the next migration.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants