Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## [Unreleased]

### Features

* `modelNameTemplate` setting for discovered model display names, with `{name}`, `{id}`, `{provider_id}` and `{provider_name}` placeholders ([#52](https://github.com/yuseferi/opencode-litellm/issues/52))

# [1.9.0](https://github.com/yuseferi/opencode-litellm/compare/v1.8.1...v1.9.0) (2026-11-03)


Expand Down
45 changes: 45 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,51 @@ By default the plugin prettifies each discovered id into a display name — `ant
- Like the other options above, `formatModelNames` is part of the cache identity, so toggling it triggers a fresh discovery on the next start instead of serving previously cached names.
- To rename just a handful of models while keeping smart formatting for the rest, use the per-model `name` override described in [Overriding or curating individual models](#overriding-or-curating-individual-models-optional).

### Naming models per provider (`modelNameTemplate`)

Several providers can point at the same LiteLLM proxy with different keys. Every key then exposes the same model ids, so the picker shows identical names under each provider and there is no way to tell which key a model will use. `modelNameTemplate` wraps every discovered model's display name with that provider's context:

```jsonc
{
"providers": {
"litellm-team-a": {
"name": "Team A",
"settings": {
"baseURL": "https://litellm.example.com/v1",
"apiKey": "{env:KEY_A}",
"modelNameTemplate": "{name} ({provider_name})"
}
},
"litellm-team-b": {
"name": "Team B",
"settings": {
"baseURL": "https://litellm.example.com/v1",
"apiKey": "{env:KEY_B}",
"modelNameTemplate": "[B] {name}"
}
}
}
}
```

`Claude 3.5 Sonnet` shows up as `Claude 3.5 Sonnet (Team A)` under the first provider and `[B] Claude 3.5 Sonnet` under the second.

| Placeholder | Value |
| --- | --- |
| `{name}` | The display name the plugin would produce anyway — formatted, or the raw id when `formatModelNames: false` |
| `{id}` | The model id exactly as `/v1/models` returned it |
| `{provider_id}` | The OpenCode provider id, for example `litellm-team-a` |
| `{provider_name}` | The provider's configured `name`, or its id when no name is set |

- The default is `{name}`, so omitting the setting changes nothing.
- The template runs after name formatting, so it composes with `formatModelNames: false`.
- An explicit `models.<id>.name` still wins and is never templated.
- Unknown placeholders are left as written, and substitution is single-pass: a placeholder that appears inside a resolved value stays literal.
- Non-string and empty values are ignored.
- On OpenCode 1, models that need the Responses API are served by a `<provider>-responses` sibling, so `{provider_id}` and `{provider_name}` resolve to that sibling for those models (for example `litellm-team-a-responses`).
- Like `formatModelNames`, the template is part of the cache identity — and so is the resolved provider name — so changing either starts a fresh discovery on the next start instead of serving previously cached names.
- Model ids, capability flags, pricing and filtering are unaffected.

### Prompt caching (`modelCompatibility`)

On OpenCode 2.0.28+ (earlier 2.x releases ignore the setting), the OpenAI-compatible Chat Completions runtime only places Anthropic-style `cache_control` breakpoints when a model declares `compatibility.cacheFormat: "anthropic"`. The plugin sets that for every discovered Chat Completions model whose `/v1/model/info` entry lists `cache_control` under `supported_openai_params` — which LiteLLM reports for Anthropic and Vertex AI Anthropic deployments — so prompt caching keeps working for Claude behind the proxy. Only the Chat Completions runtimes read this setting: the Anthropic Messages runtime adds its own cache breakpoints, and the Responses runtimes do not use it, so models on those packages are left alone.
Expand Down
159 changes: 138 additions & 21 deletions src/plugin/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@ import {
normalizeBaseURL,
} from '../utils/litellm-api'
import {
applyModelNameTemplate,
DEFAULT_MODEL_NAME_TEMPLATE,
formatModelName,
categorizeModel,
type ModelNaming,
} from '../utils/format-model-name'
import { requiresResponsesAPI } from '../utils/transport'
import type { LiteLLMModel, LiteLLMModelInfo } from '../types'
Expand Down Expand Up @@ -114,6 +117,8 @@ interface RefreshContext {
filters: ModelFilters
capabilities: ModelCapabilities
formatModelNames: boolean
modelNameTemplate?: string
providerName: string
providerId: string
}
const refreshContexts = new Map<string, RefreshContext>()
Expand Down Expand Up @@ -231,6 +236,20 @@ export function readFormatModelNames(options: Record<string, unknown>): boolean
return options.formatModelNames !== false
}

/**
* Read the optional display-name template. Non-strings are ignored. An empty
* string, or the `"{name}"` default spelled out, counts as unset so those
* configs keep the same cache identity (and the same warm cache) as before
* the setting existed.
*/
export function readModelNameTemplate(options: Record<string, unknown>): string | undefined {
const value = options.modelNameTemplate
if (typeof value !== 'string') return undefined
const trimmed = value.trim()
if (trimmed === '' || trimmed === DEFAULT_MODEL_NAME_TEMPLATE) return undefined
return value
}

/**
* Overlay metadata onto a `/v1/models` entry in three tiers: the entry's
* own fields win, `/v1/model/info` fills gaps (notably `mode`, which
Expand Down Expand Up @@ -276,27 +295,44 @@ function enrichModel(
*/
const USD_PER_TOKEN_TO_PER_MILLION = 1_000_000

/**
* The display name a model gets before any provider template is applied —
* the plugin's default formatting, or the raw `/v1/models` id.
*/
function baseDisplayName(model: { id: string }, formatModelNames: boolean): string {
return formatModelNames ? formatModelName(model) : model.id
}

/**
* Convert a discovered LiteLLM model into an OpenCode config-level
* model entry (the shape used in `provider.*.models` inside
* `opencode.json`). Returns `null` for non-chat models (embedding,
* image, audio) — they can't be used as primary chat models and would
* clutter the picker.
*
* Exported for tests: `formatModelNames` decides whether the display
* name is the prettified id or the raw `/v1/models` id verbatim.
* Exported for tests: the `naming` options decide the display name — the
* prettified id or the raw `/v1/models` id, optionally wrapped in the
* provider's `modelNameTemplate`.
*/
export function toConfigModel(
model: LiteLLMModel,
info?: LiteLLMModelInfo,
formatModelNames = true,
naming: ModelNaming = {},
): Record<string, unknown> | null {
const type = categorizeModel(model)
if (type === 'embedding' || type === 'image' || type === 'audio') {
return null
}
const providerId = naming.providerId ?? ''
const entry: Record<string, unknown> = {
name: formatModelNames ? formatModelName(model) : model.id,
name: applyModelNameTemplate(naming.modelNameTemplate, {
name: baseDisplayName(model, naming.formatModelNames !== false),
id: model.id,
providerId,
// Without a configured display name the picker falls back to the id,
// so the placeholder resolves to the same thing the user sees.
providerName: naming.providerName ?? providerId,
}),
}
// Some deployments only report the OpenAI-style `max_tokens` total;
// use it as the context limit when `max_input_tokens` is absent.
Expand Down Expand Up @@ -389,9 +425,9 @@ export interface DiscoveredModels {
* classifies + formats each model, and returns the built entries plus the
* ids that require the Responses API and per-model hints. The provider's
* `includeModels`/`excludeModels` filters, `modelCapabilities` overrides and
* `formatModelNames` choice are applied here (not at merge time) so every
* path that persists or serves a cache — cold discovery and background
* refresh — writes the same adjusted view.
* naming choices (`formatModelNames`, `modelNameTemplate`) are applied here
* (not at merge time) so every path that persists or serves a cache — cold
* discovery and background refresh — writes the same adjusted view.
*
* Returns `null` when the proxy is unreachable/unauthorized or exposes
* no models, so callers can distinguish "no data" from "empty result".
Expand All @@ -403,7 +439,7 @@ export async function discoverModels(
providerId: string,
filters: ModelFilters = {},
capabilities: ModelCapabilities = {},
formatModelNames = true,
naming: ModelNaming = {},
): Promise<DiscoveredModels | null> {
if (!(await checkLiteLLMHealth(baseURL, apiKey, customHeaders))) {
log(
Expand Down Expand Up @@ -478,7 +514,7 @@ export async function discoverModels(
const info = infoByName?.get(model.id)
if (infoByName && !info) unmatched.push(model.id)
const enriched = enrichModel(model, info, capabilities[model.id])
const entry = toConfigModel(enriched, info, formatModelNames)
const entry = toConfigModel(enriched, info, { ...naming, providerId })
Comment thread
yuseferi marked this conversation as resolved.
Resolved
if (!entry) {
skipped++
continue
Expand Down Expand Up @@ -573,10 +609,14 @@ function ensureResponsesProvider(
const provider = config.provider[id] as Record<string, unknown>
if (!provider.npm) provider.npm = RESPONSES_NPM
if (typeof provider.name !== 'string' || !provider.name) {
// Mirror how the base provider is labelled: its configured name, or its
// id when none is set (which is what the picker falls back to). This keeps
// `{provider_name}` in the sibling's model names equal to the sibling's
// visible label.
const baseName =
typeof baseProvider.name === 'string' && baseProvider.name
? baseProvider.name
: 'LiteLLM (proxy)'
: baseProviderId
provider.name = `${baseName} Responses`
}
if (!provider.options || typeof provider.options !== 'object') {
Expand Down Expand Up @@ -626,6 +666,35 @@ function readHeaderMap(value: unknown): Record<string, string> {
return out
}

/**
* Re-render one discovered entry's name for the provider it is about to be
* injected into.
*
* OpenCode 1 moves Responses models into a `${baseProviderId}-responses`
* sibling, so a template that uses `{provider_id}`/`{provider_name}` must
* resolve to the provider the user actually selects. The entry in the cache
* stays as discovered — OpenCode 2 reads the same cache with a single
* provider — so this returns a copy instead of mutating it.
*/
function renameForProvider(
entry: Record<string, unknown>,
modelId: string,
naming: ModelNaming,
providerId: string,
providerName: string,
): Record<string, unknown> {
if (naming.modelNameTemplate === undefined) return entry
return {
...entry,
name: applyModelNameTemplate(naming.modelNameTemplate, {
name: baseDisplayName({ id: modelId }, naming.formatModelNames !== false),
id: modelId,
providerId,
providerName,
}),
}
}

/**
* Split freshly discovered entries into the chat provider and the Responses
* sibling, injecting each side without clobbering curated entries. Returns
Expand All @@ -642,35 +711,46 @@ function injectDiscoveredModels(
baseURL: string,
apiKey: string | undefined,
customHeaders: Record<string, string> | undefined,
runtimeFetch?: typeof fetch,
runtimeFetch: typeof fetch | undefined,
naming: ModelNaming,
): string[] {
const responsesSet = new Set(responsesIds)
const existingSibling = config.provider[`${baseProviderId}${RESPONSES_PROVIDER_SUFFIX}`]
const responsesProviderId = `${baseProviderId}${RESPONSES_PROVIDER_SUFFIX}`
const existingSibling =
config.provider[responsesProviderId] &&
typeof config.provider[responsesProviderId] === 'object'
? (config.provider[responsesProviderId] as Record<string, unknown>)
: undefined
// A name the user (or an earlier run) already gave the sibling, captured
// before `ensureResponsesProvider` generates one.
const configuredSiblingName =
typeof existingSibling?.name === 'string' && existingSibling.name
? existingSibling.name
: undefined
const siblingModelsExisting: Record<string, unknown> =
existingSibling &&
typeof existingSibling === 'object' &&
(existingSibling as Record<string, unknown>).models &&
typeof (existingSibling as Record<string, unknown>).models === 'object'
? ((existingSibling as Record<string, unknown>).models as Record<string, unknown>)
existingSibling.models &&
typeof existingSibling.models === 'object'
? (existingSibling.models as Record<string, unknown>)
: {}

const chatEntries: Record<string, unknown> = {}
const responsesEntries: Record<string, unknown> = {}
const responsesIdsToInject: string[] = []
for (const [id, entry] of Object.entries(entries)) {
// A model already present under the other provider was curated there (or
// injected by an earlier run); leave it alone instead of duplicating it
// across both providers.
if (responsesSet.has(id)) {
if (Object.hasOwn(baseModels, id)) continue
responsesEntries[id] = entry
responsesIdsToInject.push(id)
} else {
if (Object.hasOwn(siblingModelsExisting, id)) continue
chatEntries[id] = entry
}
}

const added = mergeModels(baseModels, chatEntries)
if (Object.keys(responsesEntries).length > 0) {
if (responsesIdsToInject.length > 0) {
const siblingModels = ensureResponsesProvider(
config,
baseProviderId,
Expand All @@ -680,6 +760,26 @@ function injectDiscoveredModels(
customHeaders,
runtimeFetch,
)
// The sibling generates its name from the base provider's, so read it
// back; a user-configured name (from this run or an earlier one) wins.
// `{provider_name}` must equal the label the picker shows for the sibling
// because that is the provider the model is selected through.
const sibling = config.provider[responsesProviderId] as Record<string, unknown> | undefined
const siblingName =
configuredSiblingName ??
(sibling && typeof sibling.name === 'string' && sibling.name
? sibling.name
: responsesProviderId)
const responsesEntries: Record<string, unknown> = {}
for (const id of responsesIdsToInject) {
responsesEntries[id] = renameForProvider(
entries[id] as Record<string, unknown>,
id,
naming,
responsesProviderId,
siblingName,
)
}
added.push(...mergeModels(siblingModels, responsesEntries))
}
return added
Expand Down Expand Up @@ -712,7 +812,12 @@ async function backgroundRefresh(cacheKey: string): Promise<void> {
ctx.providerId,
ctx.filters,
ctx.capabilities,
ctx.formatModelNames,
{
formatModelNames: ctx.formatModelNames,
modelNameTemplate: ctx.modelNameTemplate,
providerId: ctx.providerId,
providerName: ctx.providerName,
},
),
DISCOVERY_TIMEOUT_MS,
)
Expand Down Expand Up @@ -870,6 +975,12 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
const filters = readModelFilters(options)
const capabilities = parseModelCapabilities(options.modelCapabilities)
const formatModelNames = readFormatModelNames(options)
const modelNameTemplate = readModelNameTemplate(options)
// `{provider_name}` resolves to what the picker labels the provider
// with; OpenCode falls back to the id when no name is configured.
const providerName =
typeof provider.name === 'string' && provider.name ? provider.name : providerId
const naming = { formatModelNames, modelNameTemplate, providerId, providerName }

// Resolve base URL
let baseURL: string | null = null
Expand Down Expand Up @@ -944,6 +1055,8 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
// the provider keeps its models and can be reconnected from `/connect`.
const cacheKey = buildCacheKey(providerId, baseURL, filters, capabilities, {
formatModelNames,
modelNameTemplate,
providerName,
}, sso ? ssoIdentity(sso) : apiKey)

// Remember how to reach this proxy so the `event` hook can
Expand All @@ -960,6 +1073,8 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
filters,
capabilities,
formatModelNames,
modelNameTemplate,
providerName,
providerId,
})

Expand Down Expand Up @@ -1006,6 +1121,7 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
runtimeKey,
customHeaders,
runtimeFetch,
naming,
)
injectedModelIds.set(cacheKey, new Set(added))
log(
Expand All @@ -1026,7 +1142,7 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
providerId,
filters,
capabilities,
formatModelNames,
{ formatModelNames, modelNameTemplate, providerId, providerName },
),
DISCOVERY_TIMEOUT_MS,
)
Expand All @@ -1042,6 +1158,7 @@ export const LiteLLMPlugin: Plugin = async (input: PluginInput) => {
runtimeKey,
customHeaders,
runtimeFetch,
naming,
)
injectedModelIds.set(cacheKey, new Set(added))
writeModelCache(cacheKey, discovered.models, discovered.responsesIds, discovered.hints)
Expand Down
Loading
Loading