Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ dist/
.DS_Store
*.log
bun.lock
package-lock.json
.opencode/node_modules/
.opencode/bun.lock
.opencode/*.md
.opencode/*.md
1 change: 0 additions & 1 deletion .opencode/plugins/throughput.ts

This file was deleted.

1 change: 1 addition & 0 deletions .opencode/plugins/throughput/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { ThroughputPlugin as default, ThroughputPlugin, PLUGIN_ID } from "../../../src/index.ts"
1 change: 1 addition & 0 deletions .opencode/plugins/throughput/tui.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { ThroughputTuiPlugin as default, ThroughputTuiPlugin } from "../../../src/tui.ts"
64 changes: 49 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,49 @@
# opencode-throughput

Real-time LLM performance monitoring plugin for [OpenCode](https://opencode.ai). Tracks latency, throughput, token usage, and cost per model with toast notifications and JSONL logging.
Real-time LLM performance monitoring plugin for [OpenCode](https://opencode.ai) V2. Tracks latency, throughput, token usage, and cost per model with toast notifications and JSONL logging.

## Features

- **TTFT** (Time To First Token) — measures how fast the model starts generating
- **TPS** (Tokens Per Second) — generation throughput during streaming
- **Total Latency** — end-to-end request time
- **Total Latency** — end-to-end time for each model step
- **Token Usage** — input, output, reasoning, cache read/write
- **Cost Tracking** — per-request and cumulative cost
- **Toast Notifications** — real-time performance summary after each LLM response
- **Toast Notifications** — real-time performance summary after each model step
- **JSONL Logging** — persistent logs at `~/.opencode/throughput.jsonl`
- **Benchmark Tool** — query historical stats on demand from the AI agent

Requires OpenCode V2 (`@opencode/plugin` 2.x). The plugin uses the V2 plugin API:
`Plugin.define({ id, setup })`, `ctx.event.subscribe`, `ctx.tool.transform`, and a
TUI part for the toast.

## Installation

Add to your `opencode.json`:
Add to your `opencode.json(c)`:

```json
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["opencode-throughput"]
"plugins": ["opencode-throughput"]
}
```

OpenCode will automatically install the plugin from npm at startup.
OpenCode installs the package from npm at startup and loads every part it exports:
the server part (`.`), which records metrics and registers the `benchmark` tool, and
the TUI part (`./tui`), which shows the toast.

## Usage

Once installed, the plugin runs automatically:

1. **After each LLM response**, a toast notification appears with metrics:
1. **After each model step**, a toast notification appears with metrics:

```
claude-sonnet-4 | 2.1s TTFT | 58.3 tok/s | 4.2s | ↑1.2k ↓892
```

2. **Logs are written** to `~/.opencode/throughput.jsonl` (one JSON object per line)
and to `<project>/.opencode/throughput.md` (one readable line per step)

3. **Query benchmarks** — ask the AI agent to use the `benchmark` tool:

Expand Down Expand Up @@ -76,28 +83,55 @@ Each line in `~/.opencode/throughput.jsonl`:
}
```

Metrics are recorded per **step** (one model call), so a turn with tool calls
produces one entry per request. `messageID` is the assistant message the step
belongs to.

## How It Works

The plugin hooks into OpenCode's event system:
The plugin subscribes to OpenCode V2 session events:

| Event | Used for |
|-------|----------|
| `session.step.started` | Model reference and step start time |
| `session.text.started` / `session.reasoning.started` / `session.step.streamed` | First streamed output (TTFT) |
| `session.step.ended` | Latency, output tokens, `finish`, `cost`, token usage |
| `session.step.failed` | Failed steps, logged with `finish: "error"` |
| `session.execution.failed` | Drops steps that never completed |

## Project layout

- `message.updated` — captures `AssistantMessage.time.completed`, `tokens`, and `cost`
- `message.part.updated` — captures the first `TextPart.time.start` for TTFT calculation
```
src/index.ts server part: event subscription, log writers, benchmark tool
src/tui.ts TUI part: toast notifications
src/metrics.ts step tracker and formatting helpers
src/report.ts log file IO and benchmark aggregation
src/types.ts shared types
```

## Development

### Local development
This repo is also its own local plugin. OpenCode auto-discovers
`.opencode/plugins/throughput/`, whose `index.ts` and `tui.ts` re-export the
implementation from `src/`, so edits take effect on save.

To point another project at a checkout, configure a plugin directory (a
directory, not a file):

```json
{
"plugin": ["file:///absolute/path/to/opencode-throughput/src/index.ts"]
"plugins": [{ "package": "file:///absolute/path/to/opencode-throughput" }]
}
```

### Build
Or copy the `.opencode/plugins/throughput/` directory into that project.

### Typecheck and test

```bash
bun install
bun run build
npm install
npm run typecheck
npm test
```

### Publish
Expand Down
19 changes: 11 additions & 8 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,20 +1,24 @@
{
"name": "opencode-throughput",
"version": "1.2.0",
"version": "2.0.0",
"description": "Real-time LLM performance monitoring plugin for OpenCode. Tracks TTFT, TPS, latency, token usage and cost per model with toast notifications and JSONL logging.",
"type": "module",
"main": "./src/index.ts",
"exports": {
".": "./src/index.ts",
"./plugins/*": "./src/plugins/*"
"./tui": "./src/tui.ts",
"./metrics": "./src/metrics.ts",
"./report": "./src/report.ts",
"./types": "./src/types.ts"
},
"files": [
"src"
],
"scripts": {
"build": "bun build src/index.ts --outdir dist --target node",
"prepublishOnly": "bun run build",
"typecheck": "tsc --noEmit"
"typecheck": "tsc --noEmit",
"test": "node --test --experimental-strip-types test/*.test.ts",
"build": "tsc --noEmit",
"prepublishOnly": "npm run typecheck"
},
"keywords": [
"opencode",
Expand All @@ -33,11 +37,10 @@
"type": "git",
"url": "git+https://github.com/Howardzhangdqs/opencode-throughput.git"
},
"peerDependencies": {
"@opencode-ai/plugin": ">=1.0.0"
"dependencies": {
"@opencode/plugin": "^2.2.5"
},
"devDependencies": {
"@opencode-ai/plugin": "^1.2.27",
"@types/node": "^25.5.0",
"typescript": "^5.7.0"
}
Expand Down
80 changes: 77 additions & 3 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,77 @@
export { ThroughputPlugin } from "./plugins/throughput.js"
export { default } from "./plugins/throughput.js"
export type { LogEntry, ModelStats } from "./types.js"
import { Plugin } from "@opencode/plugin"
import { buildLogMsg, createStepTracker, TRACKED_EVENTS } from "./metrics.ts"
import { appendLog, buildBenchmarkReport, readLogs, writePerfFile } from "./report.ts"
import type { LogEntry, TrackedEvent } from "./types.ts"

export const PLUGIN_ID = "throughput"

const BENCHMARK_TOOL = {
name: "benchmark",
description:
"Query model performance benchmarks including latency, throughput (TTFT, TPS), token usage, and cost. Shows stats aggregated by model.",
input: {
type: "object",
properties: {
model: {
type: "string",
description:
"Filter by model name (partial match, e.g. 'claude' or 'gpt'). Omit for all models.",
},
last: {
type: "integer",
minimum: 1,
description: "Only show the last N entries (default: show all)",
},
},
additionalProperties: false,
},
} as const

export const ThroughputPlugin = Plugin.define({
id: PLUGIN_ID,
async setup(ctx) {
await ctx.tool.transform((editor) => {
editor.add({
...BENCHMARK_TOOL,
async execute(input) {
const args = (input ?? {}) as { model?: string; last?: number }
return { content: buildBenchmarkReport(readLogs(), args) }
},
})
})

const directory = ctx.location.directory
const tracker = createStepTracker()
const controller = new AbortController()

function record(entry: LogEntry) {
try {
appendLog(entry)
} catch (error) {
console.error(`[${PLUGIN_ID}] failed to append throughput log`, error)
}
writePerfFile(directory, entry)
console.log(`[${PLUGIN_ID}] ${buildLogMsg(entry)}`)
}

void (async () => {
try {
for await (const event of ctx.event.subscribe({
types: [...TRACKED_EVENTS],
signal: controller.signal,
})) {
const entry = tracker.handle(event as TrackedEvent)
if (entry) record(entry)
}
} catch (error) {
if (!controller.signal.aborted) {
console.error(`[${PLUGIN_ID}] event stream failed`, error)
}
}
})()

return () => controller.abort()
},
})

export default ThroughputPlugin
Loading