From 8d7085a9e05e9c8fbebdd8655b188efc780c1eff Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sat, 24 Oct 2026 22:49:01 +0400 Subject: [PATCH 01/18] docs(adk): align readme and guides with apple and ai sdk 7 - trim the readme to the apple shape and link to the docs site - fix the ml kit feature status table (1 is downloadable, 3 is available) - state that cloud gemini needs an explicit modelName - use ai sdk 7 apis: sync streamText, flat usage fields, Output.json, stopWhen - explain that tools only run when registered via availableTools - drop duplicated examples and the untyped updateTools section Co-Authored-By: Claude Opus 5.5 --- packages/adk/README.md | 202 +++-------------------- website/src/docs/adk/generating.md | 143 +++++----------- website/src/docs/adk/getting-started.mdx | 198 ++++++++-------------- 3 files changed, 134 insertions(+), 409 deletions(-) diff --git a/packages/adk/README.md b/packages/adk/README.md index ac720cbd4..6653ed9a1 100644 --- a/packages/adk/README.md +++ b/packages/adk/README.md @@ -1,206 +1,46 @@ # ADK Provider for Vercel AI SDK -A Vercel AI SDK provider for [Google's Agent Development Kit (ADK)](https://developer.android.com/ai/adk) on Android. Use on-device Gemini Nano or cloud Gemini with tool calling and multi-turn sessions. +A Vercel AI SDK provider for [Google's Agent Development Kit (ADK)](https://developer.android.com/ai/adk), enabling on-device Gemini Nano and cloud Gemini in React Native applications on Android. **Requirements:** -- Android `minSdkVersion` 26 or greater (required by ML Kit GenAI / Gemini Nano; set in your app, not only in this package) -- React Native New Architecture +- Android only, with `minSdkVersion` 26+ set in your app +- A device that supports Gemini Nano through AICore for on-device inference (see [ML Kit GenAI device support](https://developers.google.com/ml-kit/genai#feature-device)) +- A Google AI API key for cloud Gemini - Vercel AI SDK v7 -- If using on-device models: a device capable of running Gemini Nano (Android 14+ with AICore); you can consult the [ML Kit GenAI documentation](https://developers.google.com/ml-kit/genai#feature-device) for more details. - -Consuming apps must set `minSdkVersion` to at least 26. ADK pulls in Google GenAI libraries that duplicate `META-INF/INDEX.LIST`; exclude it in your app packaging. For Expo: - -```json -[ - "expo-build-properties", - { - "android": { - "minSdkVersion": 26, - "packagingOptions": { - "exclude": ["META-INF/INDEX.LIST", "META-INF/DEPENDENCIES"] - } - } - } -] -``` - -For bare React Native, add to `android/app/build.gradle`: - -```gradle -android { - packagingOptions { - excludes += ["META-INF/INDEX.LIST", "META-INF/DEPENDENCIES"] - } -} -``` +- React Native New Architecture -Then run `npx expo prebuild --clean` so the native Android project picks up the change. +Your app also has to exclude duplicate `META-INF` files from Android packaging. See [Getting Started](https://react-native-ai.com/docs/adk/getting-started) for the Expo and bare React Native setup. ```ts import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -const { text } = await generateText({ - model: adk(), - prompt: 'What time is it in New York?', -}) -``` - -## Features - -- Cloud Gemini model via ADK `LlmAgent` and `InMemoryRunner` -- On-device Gemini Nano via ML Kit GenAI (`genai-nano` model type) -- Tool calling bridged to JavaScript executors -- Streaming responses with tool-call stream parts -- Post-generation token usage via ADK `UsageMetadata` -- Multimodal user prompts (text + inline images) -- Structured JSON output via `responseMimeType` (schema constraints not yet supported) -- Vercel AI SDK v7 `LanguageModelV4` provider - -## Usage metadata - -ADK returns token usage in response events (`promptTokenCount`, `candidatesTokenCount`, `totalTokenCount`). The provider maps this into AI SDK `usage` on both `generateText` and streaming `finish` events. - -## Multimodal input - -Pass file parts in user messages using the standard AI SDK prompt format: - -```ts -import { generateText } from 'ai' - -const { text } = await generateText({ - model: adk(), - messages: [ - { - role: 'user', - content: [ - { type: 'text', text: 'What is in this image?' }, - { - type: 'file', - mediaType: 'image/jpeg', - data: base64Image, - }, - ], - }, - ], -}) -``` - -## Structured output - -Streaming structured JSON is not supported by ADK yet. - -## Cloud Gemini - -```ts -import { createAdkProvider } from '@react-native-ai/adk' -import { generateText } from 'ai' - -const adk = createAdkProvider({ - apiKey: process.env.GOOGLE_API_KEY, - modelType: 'gemini', - modelName: 'gemini-2.5-flash', - instruction: 'You are a helpful assistant.', -}) - const { text } = await generateText({ model: adk(), prompt: 'Summarize on-device AI in one sentence.', }) ``` -> Do not embed API keys in production client apps. Prefer a backend proxy or secure runtime configuration. - -## On-device Gemini Nano - -Gemini Nano has two separate availability checks: - -| API | Label | Question | -| ------------------------------- | --------------------- | ------------------------------------------ | -| `adk.isNanoSupported()` | **Device capability** | Can this device ever run Nano? | -| `adk.isAvailable('genai-nano')` | **Runtime readiness** | Can I call `prepareNano()` / generate now? | - -If `isNanoSupported()` is `false`, `isAvailable('genai-nano')` is also `false`. - -### ML Kit status codes - -Both checks use ML Kit `Generation.getClient().checkStatus()`: - -| Status | `isNanoSupported()` | `isAvailable('genai-nano')` | Suggested UX | -| -------------- | ------------------- | --------------------------- | ---------------------------------------- | -| `0` | `false` | `false` | Hide or disable — not supported | -| `1` | `true` | `true` | Ready — call `prepareNano()` | -| `3` | `true` | `true` | Ready to download — call `prepareNano()` | -| Other non-zero | `true` | `false` | Show disabled — not ready yet | - -See [ML Kit GenAI Prompt API](https://developers.google.com/ml-kit/genai) for details. - -### Recommended flow - -```ts -import { createAdkProvider } from '@react-native-ai/adk' - -const adk = createAdkProvider({ - modelType: 'genai-nano', - modelName: 'gemini-nano', -}) - -const supported = await adk.isNanoSupported() -if (!supported) { - // Device lacks Gemini Nano / AICore support — hide from model picker - return -} - -const ready = await adk.isAvailable('genai-nano') -if (!ready) { - // Device supports Nano but ML Kit is not ready yet (e.g. downloading) - // Show disabled — do not mark as "Ready" - return -} - -await adk.prepareNano() -const model = adk() -``` - -Both checks are cheap native calls (no model download). Safe to cache at app startup and re-check after resume. - -> **Note:** `model.prepare()`, `generateText()`, and `streamText()` auto-call `prepareNano()` for `genai-nano` models, but only gate on `isNanoSupported()`. For UI gating, always use `isAvailable('genai-nano')` and handle prepare/generation errors in your chat flow. - -## Tool calling - -```ts -import { createAdkProvider } from '@react-native-ai/adk' -import { generateText, tool } from 'ai' -import { z } from 'zod' - -const getCurrentTime = tool({ - description: 'Get the current time for a city', - inputSchema: z.object({ - city: z.string(), - }), - execute: async ({ city }) => ({ - city, - time: new Date().toLocaleTimeString(), - }), -}) - -const adk = createAdkProvider({ - apiKey: process.env.GOOGLE_API_KEY, - availableTools: { getCurrentTime }, -}) +## Features -const { text } = await generateText({ - model: adk(), - tools: { getCurrentTime }, - prompt: 'What time is it in Warsaw?', -}) -``` +- ✅ On-device Gemini Nano through ML Kit GenAI +- ✅ Cloud Gemini through ADK `LlmAgent` +- ✅ Device capability and readiness checks for Gemini Nano +- ✅ Streaming +- ✅ Tool calling, with the agent loop run natively by ADK +- ✅ Image prompts +- ✅ JSON output (without schema constraints) +- ✅ Token usage -During streaming, the provider emits AI SDK `tool-input-*` and `tool-call` stream parts when ADK surfaces function calls from the model. +## Documentation -Pass tools through the AI SDK `tools` option as usual; the provider bridges execution to JavaScript while ADK orchestrates the agent loop natively. +For complete installation instructions and API documentation, visit our [documentation site](https://react-native-ai.com/docs/adk). ## License MIT + +--- + +Made with ❤️ and [create-react-native-library](https://github.com/callstack/react-native-builder-bob) diff --git a/website/src/docs/adk/generating.md b/website/src/docs/adk/generating.md index fc1a95b49..71176aa68 100644 --- a/website/src/docs/adk/generating.md +++ b/website/src/docs/adk/generating.md @@ -1,12 +1,11 @@ # Generating -You can generate responses using ADK with the Vercel AI SDK's `generateText` or `streamText` functions. ADK orchestrates the agent loop natively on Android while the provider bridges tool execution and streaming back to JavaScript. Import the default provider, call `adk()` to construct a language model, and pass it to the AI SDK — the default export targets on-device Gemini Nano. +You can generate responses using ADK with the Vercel AI SDK's `generateText` or `streamText` functions. ADK runs the agent loop natively on Android, and the provider bridges tool execution and streaming back to JavaScript. The default `adk` provider targets on-device Gemini Nano. ## Requirements -- **Physical Android device with AICore** - Required for Gemini Nano; the device must be capable of running Gemini Nano. Please consult the [ML Kit GenAI documentation](https://developers.google.com/ml-kit/genai#feature-device). We provide runtime checks for this capability - please refer to [Getting Started - On-device Gemini Nano](./getting-started.mdx#on-device-gemini-nano). +- **Physical Android device with AICore** - Required for Gemini Nano. See [Getting Started - Availability](./getting-started.mdx#availability) for the runtime checks - **Polyfills** - Streaming requires `ReadableStream`; see [Polyfills](../polyfills.md) -- **Prepare Nano** - Call `prepareNano()` or `model.prepare()` before first on-device use (see [Getting Started - On-device Gemini Nano](./getting-started.mdx#on-device-gemini-nano)) ## Text Generation @@ -14,17 +13,17 @@ You can generate responses using ADK with the Vercel AI SDK's `generateText` or import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -await adk.prepareNano() - const { text } = await generateText({ model: adk(), prompt: 'Explain quantum computing in simple terms', }) ``` +The first call prepares Gemini Nano. To prepare it ahead of time, call `adk.prepareNano()` (see [Getting Started - Preparation](./getting-started.mdx#preparation)). + ### Cloud Gemini -Cloud models use ADK's `LlmAgent` with Google's Gemini API. This is useful when Gemini Nano is unavailable, you need a more capable model, or you want to develop on an emulator without AICore. Create a provider with `modelType: 'gemini'` and an API key: +Cloud models use ADK's `LlmAgent` with the Gemini API. They are useful when Gemini Nano is unavailable, when you need a more capable model, or when you develop on an emulator without AICore. Create a provider with `modelType: 'gemini'`, a `modelName`, and an API key: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -42,8 +41,6 @@ const { text } = await generateText({ }) ``` -Cloud models do not require a separate download or prepare step. - > Do not embed API keys in production client apps. Prefer a backend proxy or secure runtime configuration. ## Streaming @@ -54,9 +51,7 @@ Stream responses for real-time output: import { adk } from '@react-native-ai/adk' import { streamText } from 'ai' -await adk.prepareNano() - -const { textStream } = await streamText({ +const { textStream } = streamText({ model: adk(), prompt: 'Write a short story about a robot learning to paint', }) @@ -66,43 +61,42 @@ for await (const delta of textStream) { } ``` -During streaming, the provider emits standard AI SDK stream parts: `text-start`, `text-delta`, `text-end`, and `finish` with usage metadata. +The provider emits standard AI SDK stream parts: `text-start`, `text-delta`, `text-end`, and `finish` with token usage. -## Usage Metadata +## Token Usage -ADK returns token usage in response events (`promptTokenCount`, `candidatesTokenCount`, `totalTokenCount`). The provider maps this into AI SDK `usage` on both `generateText` results and streaming `finish` events: +ADK reports token usage as `promptTokenCount`, `candidatesTokenCount`, and `totalTokenCount`. The provider maps these to AI SDK `usage` on `generateText` results and on the streaming `finish` part: ```typescript import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -await adk.prepareNano() - const { usage } = await generateText({ model: adk(), prompt: 'Count to five.', }) -console.log(usage.inputTokens.total) // promptTokenCount -console.log(usage.outputTokens.total) // candidatesTokenCount +console.log(usage.inputTokens) // promptTokenCount +console.log(usage.outputTokens) // candidatesTokenCount console.log(usage.raw?.totalTokenCount) ``` ## Tool Calling -Enable ADK agents to call JavaScript tools during generation. Works on both on-device Nano and cloud Gemini. +ADK agents can call JavaScript tools during generation, on both Gemini Nano and cloud Gemini. -### Important ADK-Specific Behavior +### ADK-Specific Behavior -Tools are orchestrated by ADK natively, which means: +ADK runs tools natively, which differs from other providers: -- **Pre-register executors**: Pass tools to `createAdkProvider` via `availableTools` so ADK can invoke their `execute` handlers -- **Provider-executed**: Streamed tool calls are marked with `providerExecuted: true` -- **Native agent loop**: ADK runs the multi-turn tool loop; AI SDK `maxSteps` does not control ADK's internal iteration +- **Register executors on the provider** - Pass tools to `createAdkProvider` via `availableTools` so ADK can call their `execute` handlers. Tools passed only to `generateText` or `streamText` are described to the model but never executed +- **Provider-executed** - Streamed tool calls are marked with `providerExecuted: true` +- **Native agent loop** - ADK runs the multi-step tool loop, so AI SDK `stopWhen` does not limit ADK's internal steps +- **Gemini Nano** - Gemini Nano has no native function calling. The provider lists the tools in the system instruction and converts the model's `tool_code` blocks into ADK function calls ### Setup -ADK needs tool executors registered both in the AI SDK and on the provider. Use `createAdkProvider` with `availableTools`, then pass tools to the AI SDK and call `adk()` as usual. Example: +Register the tools on the provider with `availableTools`, and pass the same tools to the AI SDK: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -124,8 +118,6 @@ const adk = createAdkProvider({ availableTools: { getCurrentTime }, }) -await adk.prepareNano() - const { text } = await generateText({ model: adk(), tools: { getCurrentTime }, @@ -135,60 +127,14 @@ const { text } = await generateText({ During streaming, the provider emits `tool-input-start`, `tool-input-delta`, `tool-input-end`, and `tool-call` stream parts when ADK surfaces function calls from the model. -Pass tools through the AI SDK `tools` option as usual; the provider bridges execution to JavaScript while ADK orchestrates the agent loop natively. +## Image Prompts -### Updating Tools at Runtime - -Register tools when creating the provider so ADK can execute them from JavaScript: - -```typescript -import { adk } from '@react-native-ai/adk' - -const model = adk() - -// Add or replace tools on an existing model instance -model.updateTools({ - getCurrentTime, -}) -``` - -To register executors at provider creation time: - -```typescript -import { createAdkProvider } from '@react-native-ai/adk' -import { tool } from 'ai' -import { z } from 'zod' - -const getCurrentTime = tool({ - description: 'Get the current time for a city', - inputSchema: z.object({ city: z.string() }), - execute: async ({ city }) => ({ - city, - time: new Date().toLocaleTimeString(), - }), -}) - -const adk = createAdkProvider({ - availableTools: { getCurrentTime }, -}) - -const model = adk() - -model.updateTools({ - getCurrentTime, -}) -``` - -## Multimodal Input - -Pass file parts in user messages using the standard AI SDK prompt format: +Pass image file parts in user messages using the standard AI SDK prompt format: ```typescript import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -await adk.prepareNano() - const { text } = await generateText({ model: adk(), messages: [ @@ -214,37 +160,33 @@ Supported file data formats: > **Note**: File URLs (`file://` or HTTP) are not supported yet. Pass base64 or `Uint8Array` data directly. -## Structured Output +## JSON Output -Generate JSON responses using AI SDK JSON mode: +Generate JSON with `Output.json()`: ```typescript import { adk } from '@react-native-ai/adk' -import { generateText } from 'ai' - -await adk.prepareNano() +import { generateText, Output } from 'ai' -const { text } = await generateText({ +const { output } = await generateText({ model: adk(), + output: Output.json(), prompt: 'Return a JSON object with name and age fields.', - responseFormat: { type: 'json' }, }) ``` -> **Note**: JSON schema constraints (`responseFormat.schema`) are not supported yet. Use JSON mode without a schema, or parse and validate the response in your app. - -Streaming structured JSON is not supported by ADK yet. +> **Note**: Schema constraints are not supported yet, so `Output.object()` and other schema-based outputs throw. Use `Output.json()` and validate the result in your app. JSON output is not supported with `streamText`. ## Available Options Configure model behavior with generation options: -| Option | Type | Description | -| ------------- | ------ | -------------------------------------------------------- | -| `temperature` | number | Controls randomness | -| `maxTokens` | number | Maximum tokens to generate (`maxOutputTokens` in AI SDK) | -| `topP` | number | Nucleus sampling threshold | -| `topK` | number | Top-K sampling parameter | +| Option | Type | Description | +| ----------------- | ------ | -------------------------- | +| `temperature` | number | Controls randomness | +| `maxOutputTokens` | number | Maximum tokens to generate | +| `topP` | number | Nucleus sampling threshold | +| `topK` | number | Top-K sampling parameter | Example: @@ -252,8 +194,6 @@ Example: import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -await adk.prepareNano() - const { text } = await generateText({ model: adk(), prompt: 'Write a creative story', @@ -266,12 +206,13 @@ const { text } = await generateText({ ## Limitations -The following features are not yet supported: +The following features are not supported yet: -| Feature | Status | -| ---------------------------------- | -------------------------------------------- | -| `responseFormat.schema` | Not supported - use JSON mode without schema | -| Streaming JSON | Not supported | -| `toolChoice: { type: 'required' }` | Ignored with a warning - defaults to auto | -| File URLs in multimodal prompts | Not supported - use base64 or `Uint8Array` | -| iOS / web | Android only | +| Feature | Status | +| ------------------------------------------ | ------------------------------------------- | +| Schema-constrained output | Throws - use `Output.json()` without schema | +| JSON output with `streamText` | Throws | +| `toolChoice` `required` or a specific tool | Ignored with an AI SDK warning - uses auto | +| `reasoning` | Ignored with an AI SDK warning | +| File URLs in prompts | Throws - use base64 or `Uint8Array` | +| iOS / web | Android only | diff --git a/website/src/docs/adk/getting-started.mdx b/website/src/docs/adk/getting-started.mdx index 1107b7b69..fe79c5844 100644 --- a/website/src/docs/adk/getting-started.mdx +++ b/website/src/docs/adk/getting-started.mdx @@ -2,7 +2,7 @@ import { PackageManagerTabs } from '@theme' # Getting Started -The ADK provider brings on-device Gemini Nano to Android React Native apps through [Google's Agent Development Kit (ADK)](https://developer.android.com/ai/adk) and the Vercel AI SDK. Gemini Nano runs locally via ML Kit GenAI - provisioned by the Android system through AICore, with no API key and no model files to bundle. You also get cloud Gemini as an option when you need a larger model or broader device support. +The ADK provider brings on-device Gemini Nano to Android React Native apps through [Google's Agent Development Kit (ADK)](https://developer.android.com/ai/adk) and the Vercel AI SDK. Gemini Nano runs locally via ML Kit GenAI - provisioned by the Android system through AICore, with no API key and no model files to bundle. You can also use cloud Gemini when you need a larger model or broader device support. ## Installation @@ -17,22 +17,23 @@ You can use the ADK provider standalone, but we recommend the Vercel AI SDK for ## Requirements - **Android only** - The provider is not available on iOS or web -- **Android `minSdkVersion` 26 or greater** - Required by ML Kit GenAI / Gemini Nano +- **Android `minSdkVersion` 26 or greater** - Required by ML Kit GenAI - **React Native New Architecture** - Required for native module functionality -- **React Native >= 0.76.0** -- **Gemini Nano** - A physical Android device with AICore support (Android 14+). See the [ML Kit GenAI documentation](https://developers.google.com/ml-kit/genai#feature-device) for device support details. +- **React Native 0.76 or later** :::danger Gemini Nano - physical supported device required -Gemini Nano requires a physical Android device with AICore support and support for the Gemini Nano models. Please consult the [ML Kit GenAI documentation](https://developers.google.com/ml-kit/genai#feature-device). +Gemini Nano requires a physical Android device with AICore and Gemini Nano support. Please consult the [ML Kit GenAI documentation](https://developers.google.com/ml-kit/genai#feature-device). -Cloud Gemini works on all devices supported by ADK & MLKit when an API key is configured. +Cloud Gemini works on any Android device or emulator once you configure an API key. ::: -## Integration with your app +## App Configuration -Consuming apps must set `minSdkVersion` to at least 26. ADK pulls in Google GenAI libraries that duplicate `META-INF/INDEX.LIST`; exclude it in your app packaging via `expo-build-properties`: +Consuming apps must set `minSdkVersion` to at least 26. ADK pulls in Google GenAI libraries that duplicate `META-INF/INDEX.LIST` and `META-INF/DEPENDENCIES`, so exclude them from your app packaging. -### Expo Setup +### Expo + +Configure `expo-build-properties` in your app config: ```json { @@ -54,7 +55,7 @@ Consuming apps must set `minSdkVersion` to at least 26. ADK pulls in Google GenA } ``` -After changing CNG config, run: +Then regenerate the native project: ```bash npx expo prebuild --clean @@ -62,7 +63,7 @@ npx expo prebuild --clean ### Bare React Native -Add the following to `android/app/build.gradle`: +Add the following to `android/app/build.gradle` and make sure `minSdkVersion` is at least 26: ```groovy android { @@ -72,130 +73,93 @@ android { } ``` -Ensure your app's `minSdkVersion` is at least 26. - ## Available Model Types -| Model Type | `modelType` | Default Name | Use Case | -| --------------------- | ------------ | ------------------ | ------------------------------------------------------------------ | -| On-device Gemini Nano | `genai-nano` | `gemini-nano` | Private, offline-capable inference - system-provisioned via AICore | -| Cloud Gemini | `gemini` | `gemini-2.5-flash` | Cloud inference via Google AI API | +| Model Type | `modelType` | `modelName` | Use Case | +| --------------------- | ------------ | --------------------------------- | ------------------------------------------------------------------ | +| On-device Gemini Nano | `genai-nano` | `gemini-nano` (default) | Private, offline-capable inference - system-provisioned via AICore | +| Cloud Gemini | `gemini` | Required, e.g. `gemini-2.5-flash` | Cloud inference via the Gemini API | ## Basic Usage -Import the default ADK provider and call `adk()` to construct a language model, then pass it to the AI SDK. The default export targets on-device Gemini Nano (`gemini-nano`): +Import the default ADK provider and call `adk()` to construct a language model, then pass it to the AI SDK. The default provider targets on-device Gemini Nano: ```typescript import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -const supported = await adk.isNanoSupported() -if (!supported) throw new Error('Gemini Nano not supported on this device') - -const ready = await adk.isAvailable() -if (!ready) throw new Error('Gemini Nano not ready yet') - -await adk.prepareNano() - const { text } = await generateText({ model: adk(), prompt: 'Summarize on-device AI in one sentence.', }) ``` -See [Cloud Gemini](#cloud-gemini) when you need cloud inference instead. +Before you offer Gemini Nano in your UI, check that the device supports it and that it is ready. See [Availability](#availability) below. -### On-device Gemini Nano +## On-device Gemini Nano -Gemini Nano runs locally via ML Kit GenAI and is provisioned by the Android system (AICore). The default `adk()` export already targets Nano — check availability and prepare before generating: - -```typescript -import { adk } from '@react-native-ai/adk' -import { generateText } from 'ai' - -const supported = await adk.isNanoSupported() -if (!supported) throw new Error('Gemini Nano not supported on this device') - -const ready = await adk.isAvailable('genai-nano') -if (!ready) throw new Error('Gemini Nano not ready yet') - -await adk.prepareNano() - -const { text } = await generateText({ - model: adk(), - prompt: 'Summarize on-device AI in one sentence.', -}) -``` - -To customize the system instruction or register tool executors, create a provider with `createAdkProvider`: +Gemini Nano runs locally via ML Kit GenAI and is provisioned by the Android system through AICore. To set a system instruction, create a provider with `createAdkProvider`: ```typescript import { createAdkProvider } from '@react-native-ai/adk' import { generateText } from 'ai' const adk = createAdkProvider({ - modelType: 'genai-nano', - modelName: 'gemini-nano', instruction: 'You are a helpful assistant.', }) -await adk.prepareNano() - const { text } = await generateText({ model: adk(), prompt: 'Summarize on-device AI in one sentence.', }) ``` -#### Availability Check - -Nano has two separate checks: +### Availability -| API | Label | Question | -| ------------------------------- | --------------------- | ------------------------------------------ | -| `adk.isNanoSupported()` | **Device capability** | Can this device ever run Nano? | -| `adk.isAvailable('genai-nano')` | **Runtime readiness** | Can I call `prepareNano()` / generate now? | +The provider exposes two checks for Gemini Nano: -If `isNanoSupported()` is `false`, `isAvailable('genai-nano')` is also `false`. +| API | Checks | Resolves to `true` when | +| ------------------------------- | ----------------- | ------------------------------------------------------------ | +| `adk.isNanoSupported()` | Device capability | The device can run Gemini Nano, even if it is not downloaded | +| `adk.isAvailable('genai-nano')` | Runtime readiness | Gemini Nano is downloaded, or can be downloaded now | -Both checks are cheap native calls (no model download), safe to cache at app startup and re-check after resume. +`isAvailable()` defaults to `'genai-nano'` and always resolves to `true` for `'gemini'`. When `isNanoSupported()` is `false`, `isAvailable('genai-nano')` is `false` too. -Both checks use ML Kit `Generation.getClient().checkStatus()`: +Both checks are cheap native calls that do not download the model. You can cache them at app startup and check again when the app resumes. -| Status | `isNanoSupported()` | `isAvailable('genai-nano')` | Suggested UX | -| -------------- | ------------------- | --------------------------- | ---------------------------------------- | -| `0` | `false` | `false` | Hide or disable - not supported | -| `1` | `true` | `true` | Ready - call `prepareNano()` | -| `3` | `true` | `true` | Ready to download - call `prepareNano()` | -| Other non-zero | `true` | `false` | Show disabled - not ready yet | +Both checks read the ML Kit [`FeatureStatus`](https://developers.google.com/android/reference/com/google/mlkit/genai/common/FeatureStatus) returned by `Generation.getClient().checkStatus()`: -If needed, consult the [ML Kit GenAI Prompt API](https://developers.google.com/ml-kit/genai) for details. +| `FeatureStatus` | `isNanoSupported()` | `isAvailable('genai-nano')` | Suggested UX | +| ------------------ | ------------------- | --------------------------- | ------------------------------------------------ | +| `UNAVAILABLE` (0) | `false` | `false` | Hide Gemini Nano | +| `DOWNLOADABLE` (1) | `true` | `true` | Offer Gemini Nano - `prepareNano()` downloads it | +| `DOWNLOADING` (2) | `true` | `false` | Show as unavailable until the download finishes | +| `AVAILABLE` (3) | `true` | `true` | Ready | -#### Recommended Flow +A typical availability flow: ```typescript import { adk } from '@react-native-ai/adk' -const supported = await adk.isNanoSupported() -if (!supported) { - // Device lacks Gemini Nano / AICore support - hide from model picker - return -} +async function loadGeminiNano() { + if (!(await adk.isNanoSupported())) { + // The device cannot run Gemini Nano - hide it from the model picker + return null + } -const ready = await adk.isAvailable('genai-nano') -if (!ready) { - // Device supports Nano but ML Kit is not ready yet (e.g. downloading) - // Show disabled - do not mark as "Ready" - return -} + if (!(await adk.isAvailable('genai-nano'))) { + // Gemini Nano is still downloading - show it as unavailable + return null + } -await adk.prepareNano() -const model = adk() + await adk.prepareNano() + return adk() +} ``` -#### Automatic Preparation +### Preparation -`model.prepare()`, `generateText()`, and `streamText()` auto-call `prepareNano()` for `genai-nano` models, but only gate on `isNanoSupported()`. For UI gating, always use `isAvailable('genai-nano')` and handle prepare/generation errors in your chat flow. +`adk.prepareNano()` downloads and initializes Gemini Nano. Calling `model.prepare()` on a model does the same thing. `generateText()` and `streamText()` prepare the model automatically on first use: ```typescript import { adk } from '@react-native-ai/adk' @@ -203,34 +167,18 @@ import { generateText } from 'ai' const model = adk() -// Explicit prepare (initializes Nano via ML Kit) +// Prepare explicitly, for example while showing a loading state await model.prepare() -// Or let generation trigger prepare automatically +// Or let the first generation prepare the model await generateText({ model, prompt: 'Hello!' }) ``` -### Cloud Gemini - -When you need cloud inference - for example on emulators or devices without AICore - create a provider with `modelType: 'gemini'` and an API key: - -```typescript -import { createAdkProvider } from '@react-native-ai/adk' -import { generateText } from 'ai' - -const adk = createAdkProvider({ - apiKey: process.env.GOOGLE_API_KEY, - modelType: 'gemini', - modelName: 'gemini-2.5-flash', -}) +Automatic preparation only checks `isNanoSupported()`. Use `isAvailable('genai-nano')` for UI gating, and handle preparation and generation errors in your app. -const { text } = await generateText({ - model: adk(), - prompt: 'What time is it in New York?', -}) -``` +## Cloud Gemini -To customize the system instruction or cloud model name: +When you need cloud inference - for example on emulators or devices without AICore - create a provider with `modelType: 'gemini'`, a `modelName`, and an API key: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -249,40 +197,36 @@ const { text } = await generateText({ }) ``` -> Do not embed API keys in production client apps. Prefer a backend proxy or secure runtime configuration. +Cloud models do not need a download or prepare step. -Cloud models do not require a separate download or prepare step. +> Do not embed API keys in production client apps. Prefer a backend proxy or secure runtime configuration. -### Custom provider (advanced) +## Provider Options -Use `createAdkProvider` when you need to change the model type, agent name, system instruction, API key, or registered tool executors. It returns the same callable interface as the default `adk` export - call it to construct a language model: +`createAdkProvider` returns the same callable interface as the default `adk` export. Call `adk()` or `adk.languageModel()` to construct a language model: ```typescript +import { createAdkProvider } from '@react-native-ai/adk' + const adk = createAdkProvider({ name: 'my_agent', description: 'A helpful assistant', instruction: 'You are a concise, friendly assistant.', - modelType: 'genai-nano', - modelName: 'gemini-nano', }) -const model = adk() -// or const model = adk.languageModel() ``` -#### Provider Options - -| Option | Type | Description | -| ---------------- | -------------------------- | --------------------------------------------------------------------------- | -| `name` | `string` | ADK agent name (default: `react_native_adk_agent`) | -| `description` | `string` | Agent description shown to ADK | -| `instruction` | `string` | System instruction for the agent | -| `modelType` | `'genai-nano' \| 'gemini'` | On-device or cloud backend (default: `genai-nano`) | -| `modelName` | `string` | Model identifier (default: `gemini-nano`; use `gemini-2.5-flash` for cloud) | -| `apiKey` | `string` | Google AI API key - required for cloud `gemini` only | -| `availableTools` | `Record` | Tools whose `execute` handlers ADK can call from JavaScript | +| Option | Type | Description | +| ---------------- | -------------------------- | --------------------------------------------------------------------------------------------- | +| `name` | `string` | ADK agent name (default: `react_native_adk_agent`) | +| `description` | `string` | Agent description passed to ADK | +| `instruction` | `string` | System instruction for the agent | +| `modelType` | `'genai-nano' \| 'gemini'` | On-device or cloud backend (default: `genai-nano`) | +| `modelName` | `string` | Model identifier (default: `gemini-nano`; required for `gemini`) | +| `apiKey` | `string` | Gemini API key - required for `gemini` only | +| `availableTools` | `Record` | Tools whose `execute` handlers ADK can call. See [Tool Calling](./generating.md#tool-calling) | ## Next Steps -See **[Generating](./generating.md)** for text generation, streaming, tool calling and multimodal input. +See **[Generating](./generating.md)** for text generation, streaming, tool calling, image prompts, and JSON output. From 2fcc0b1b0b33bd33c1c8f1e5d76be8a0c44c4c7f Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sun, 25 Oct 2026 02:55:59 +0400 Subject: [PATCH 02/18] chore(adk): upgrade adk kotlin to 1.4.0 and ml kit genai-prompt to beta4 - depend on google-adk-kotlin-core 1.4.0 and the new google-adk-kotlin-mlkit-android module - import GenaiPrompt and GenerativeModelHelpers from com.google.adk.kt.mlkit - match the nullable FunctionTool.execute args - pin genai-prompt beta4 for gemini nano v4 status and streaming fixes - default compile and target sdk to 36 Co-Authored-By: Claude Opus 5.5 --- packages/adk/android/build.gradle | 7 ++++--- packages/adk/android/gradle.properties | 4 ++-- .../src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt | 4 ++-- .../java/com/callstack/ai/adk/NanoFunctionCallingModel.kt | 2 +- .../java/com/callstack/ai/adk/ReactNativeFunctionTool.kt | 2 +- 5 files changed, 10 insertions(+), 9 deletions(-) diff --git a/packages/adk/android/build.gradle b/packages/adk/android/build.gradle index 458bb0e57..0f5c53f72 100644 --- a/packages/adk/android/build.gradle +++ b/packages/adk/android/build.gradle @@ -92,9 +92,10 @@ def kotlin_version = getExtOrDefault("kotlinVersion") dependencies { implementation "com.facebook.react:react-native:+" implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version" - implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2" - implementation "com.google.adk:google-adk-kotlin-core-android:0.2.0" - implementation "com.google.mlkit:genai-prompt:1.0.0-beta2" + implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.11.0" + implementation "com.google.adk:google-adk-kotlin-core:1.4.0" + implementation "com.google.adk:google-adk-kotlin-mlkit-android:1.4.0-beta" + implementation "com.google.mlkit:genai-prompt:1.0.0-beta4" } react { diff --git a/packages/adk/android/gradle.properties b/packages/adk/android/gradle.properties index cc84bcaa4..96ed9f69a 100644 --- a/packages/adk/android/gradle.properties +++ b/packages/adk/android/gradle.properties @@ -1,4 +1,4 @@ ADK_kotlinVersion=2.1.20 ADK_minSdkVersion=26 -ADK_targetSdkVersion=34 -ADK_compileSdkVersion=34 +ADK_targetSdkVersion=36 +ADK_compileSdkVersion=36 diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index a82bb2bdf..30cb78f99 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -8,7 +8,7 @@ import com.google.adk.kt.agents.LlmAgent import com.google.adk.kt.events.Event import com.google.adk.kt.models.Gemini import com.google.adk.kt.models.Model -import com.google.adk.kt.models.mlkit.GenaiPrompt +import com.google.adk.kt.mlkit.GenaiPrompt import com.google.adk.kt.runners.InMemoryRunner import com.google.adk.kt.sessions.InMemorySessionService import com.google.adk.kt.sessions.SessionKey @@ -19,7 +19,7 @@ import com.google.adk.kt.types.GenerateContentConfig import com.google.adk.kt.types.Part import com.google.adk.kt.types.Role import com.google.adk.kt.types.UsageMetadata -import com.google.adk.kt.utils.mlkit.GenerativeModelHelpers +import com.google.adk.kt.mlkit.GenerativeModelHelpers import com.google.mlkit.genai.prompt.Generation import com.google.mlkit.genai.prompt.GenerativeModel import kotlinx.coroutines.flow.Flow diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt index 97e838b2b..7ae69bf2f 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt @@ -3,7 +3,7 @@ package com.callstack.ai.adk import com.google.adk.kt.models.LlmRequest import com.google.adk.kt.models.LlmResponse import com.google.adk.kt.models.Model -import com.google.adk.kt.models.mlkit.GenaiPrompt +import com.google.adk.kt.mlkit.GenaiPrompt import com.google.adk.kt.types.Content import com.google.adk.kt.types.FunctionCall import com.google.adk.kt.types.Part diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt index e9ab8367c..734c968f9 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt @@ -85,7 +85,7 @@ class ReactNativeFunctionTool( ) } - override suspend fun execute(context: ToolContext, args: Map): Any { + override suspend fun execute(context: ToolContext, args: Map): Any { val toolCallId = UUID.randomUUID().toString() val argumentsJson = AdkJson.encode(args) onToolCall(toolCallId, toolId, argumentsJson, toolCallScope) From 122ccd76593dd1160d650fea7e9d93f947ca8b21 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sun, 25 Oct 2026 03:25:40 +0400 Subject: [PATCH 03/18] feat(adk): align provider with apple and support ai sdk 7 features - take per-model options in adk() and export AdkLanguageModel - add getModelInfo, Gemini Nano download progress, model selection and countTokens - add AdkErrorCodes mapped from ML Kit and Gemini API errors - send no tools for toolChoice none and only the chosen tool for a specific one - warn and return an error when a tool has no registered executor - default cloud Gemini to gemini-2.5-flash and infer it from the model name - support schemas for structured output, including streaming - map reasoning to Gemini thinking and return reasoning parts - return provider-executed tool calls and results, and stream tool results - pass seed, stop sequences and penalties, and warn when Nano ignores them - cancel native runs on abort and read file:// and content:// images - report reasoning and cached tokens and map content filter finish reasons Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 315 ++++-- .../java/com/callstack/ai/adk/AdkErrors.kt | 57 ++ .../java/com/callstack/ai/adk/AdkSchema.kt | 57 +- .../callstack/ai/adk/AdkStreamToolCalls.kt | 16 + .../java/com/callstack/ai/adk/AdkUsage.kt | 2 + .../callstack/ai/adk/NativeAdkEngineModule.kt | 203 +++- packages/adk/src/NativeAdkEngine.ts | 77 +- packages/adk/src/__tests__/ai-sdk.test.ts | 541 +++++++++- packages/adk/src/ai-sdk.ts | 941 +++++++++++++----- packages/adk/src/errors.ts | 49 + packages/adk/src/index.ts | 20 +- 11 files changed, 1873 insertions(+), 405 deletions(-) create mode 100644 packages/adk/android/src/main/java/com/callstack/ai/adk/AdkErrors.kt create mode 100644 packages/adk/src/errors.ts diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 30cb78f99..75d4ec2dc 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -1,14 +1,18 @@ package com.callstack.ai.adk +import android.content.Context +import android.net.Uri import android.util.Base64 import com.facebook.react.bridge.ReadableArray import com.facebook.react.bridge.ReadableMap import com.google.adk.kt.agents.Instruction import com.google.adk.kt.agents.LlmAgent +import com.google.adk.kt.agents.RunConfig +import com.google.adk.kt.agents.StreamingMode import com.google.adk.kt.events.Event +import com.google.adk.kt.mlkit.GenaiPrompt import com.google.adk.kt.models.Gemini import com.google.adk.kt.models.Model -import com.google.adk.kt.mlkit.GenaiPrompt import com.google.adk.kt.runners.InMemoryRunner import com.google.adk.kt.sessions.InMemorySessionService import com.google.adk.kt.sessions.SessionKey @@ -18,85 +22,204 @@ import com.google.adk.kt.types.Content import com.google.adk.kt.types.GenerateContentConfig import com.google.adk.kt.types.Part import com.google.adk.kt.types.Role +import com.google.adk.kt.types.ThinkingConfig import com.google.adk.kt.types.UsageMetadata -import com.google.adk.kt.mlkit.GenerativeModelHelpers +import com.google.mlkit.genai.common.DownloadStatus +import com.google.mlkit.genai.common.FeatureStatus +import com.google.mlkit.genai.common.GenAiException import com.google.mlkit.genai.prompt.Generation import com.google.mlkit.genai.prompt.GenerativeModel +import com.google.mlkit.genai.prompt.ModelPreference +import com.google.mlkit.genai.prompt.ModelReleaseStage +import com.google.mlkit.genai.prompt.TextPart +import com.google.mlkit.genai.prompt.generateContentRequest +import com.google.mlkit.genai.prompt.generationConfig +import com.google.mlkit.genai.prompt.modelConfig import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.toList import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.withLock import java.util.UUID +import java.util.concurrent.ConcurrentHashMap + +data class AdkToolCallRecord(val toolCallId: String, val toolName: String, val input: String) + +data class AdkToolResultRecord(val toolCallId: String, val toolName: String, val output: String) data class AdkRunResult( val content: String, + val reasoning: String?, + val toolCalls: List, + val toolResults: List, val finishReason: String?, val usage: UsageMetadata?, ) +data class AdkModelInfo( + val availability: String, + val baseModelName: String? = null, + val contextSize: Int? = null, +) + +/** Gemini Nano model selection, see ML Kit `ModelConfig`. */ +data class NanoConfig( + val releaseStage: String? = null, + val preference: String? = null, +) { + companion object { + fun from(map: ReadableMap?): NanoConfig = + NanoConfig( + releaseStage = map?.getString("releaseStage"), + preference = map?.getString("preference"), + ) + } +} + +fun Part.isThought(): Boolean = thought == true + +fun Event.textParts(thoughts: Boolean): String = + content?.parts?.filter { it.isThought() == thoughts }?.mapNotNull { it.text }?.joinToString("") + ?: "" + class AdkAgentRunner( + private val context: Context, private val onToolCall: ( toolCallId: String, toolId: String, arguments: String, scope: ToolCallScope?, ) -> Unit, + private val onDownloadProgress: ( + phase: String, + bytesDownloaded: Long?, + bytesToDownload: Long?, + ) -> Unit, ) { - private var generativeModel: GenerativeModel? = null + private val nanoModels = ConcurrentHashMap() private val nanoPrepareMutex = Mutex() - suspend fun prepareNano() { - if (!isNanoSupported()) { - throw IllegalStateException( - "Gemini Nano is not supported on this device or device hasn't fetched the latest configuration to support it", - ) - } - nanoPrepareMutex.withLock { - generativeModel = GenerativeModelHelpers.initGenerativeModel() + private fun nanoClient(config: NanoConfig): GenerativeModel { + if (config.releaseStage == null && config.preference == null) { + return Generation.getClient() } + + return Generation.getClient( + generationConfig { + modelConfig = modelConfig { + config.releaseStage?.let { + releaseStage = + if (it == "preview") ModelReleaseStage.PREVIEW else ModelReleaseStage.STABLE + } + config.preference?.let { + preference = if (it == "fast") ModelPreference.FAST else ModelPreference.FULL + } + } + }, + ) } - private suspend fun ensureNanoPrepared() { - if (generativeModel != null) { - return + private suspend fun nanoStatus(config: NanoConfig): Int = + try { + nanoClient(config).checkStatus() + } catch (_: Exception) { + FeatureStatus.UNAVAILABLE + } + + /** Downloads Gemini Nano when needed, reporting progress, then warms it up. */ + suspend fun prepareNano(config: NanoConfig) { + nanoPrepareMutex.withLock { + if (nanoModels.containsKey(config)) return + + val model = nanoClient(config) + when (val status = model.checkStatus()) { + FeatureStatus.UNAVAILABLE -> + throw AdkErrors.AdkException( + AdkErrors.MODEL_UNAVAILABLE, + "Gemini Nano is not supported on this device or the device has not fetched the latest configuration to support it", + ) + FeatureStatus.DOWNLOADABLE, FeatureStatus.DOWNLOADING -> download(model, status) + } + model.warmup() + nanoModels[config] = model } - prepareNano() } - /** - * Whether this device supports on-device Gemini Nano at all (ML Kit checkStatus != 0). - * Does not download models or run full initialization. - */ - suspend fun isNanoSupported(): Boolean { - return try { - when (Generation.getClient().checkStatus()) { - 0 -> false - else -> true + private suspend fun download(model: GenerativeModel, status: Int) { + try { + model.download().collect { downloadStatus -> + when (downloadStatus) { + is DownloadStatus.DownloadStarted -> + onDownloadProgress("started", 0, downloadStatus.bytesToDownload) + is DownloadStatus.DownloadProgress -> + onDownloadProgress("progress", downloadStatus.totalBytesDownloaded, null) + is DownloadStatus.DownloadCompleted -> onDownloadProgress("completed", null, null) + is DownloadStatus.DownloadFailed -> + throw AdkErrors.AdkException( + AdkErrors.DOWNLOAD_FAILED, + "Gemini Nano download failed: ${downloadStatus.e.message}", + downloadStatus.e, + ) + } } - } catch (_: Exception) { - false + } catch (error: GenAiException) { + if (status == FeatureStatus.DOWNLOADING) { + throw AdkErrors.AdkException( + AdkErrors.MODEL_DOWNLOADING, + "Gemini Nano is still downloading", + error, + ) + } + throw error } } - /** - * Whether Gemini Nano is ready to use now (downloaded or ready to download). - * Returns false when [isNanoSupported] is false. - */ - suspend fun isNanoAvailable(): Boolean { - if (!isNanoSupported()) { - return false + private suspend fun preparedNano(config: NanoConfig): GenerativeModel { + nanoModels[config]?.let { return it } + prepareNano(config) + return nanoModels.getValue(config) + } + + /** Whether this device supports Gemini Nano at all. Does not download the model. */ + suspend fun isNanoSupported(config: NanoConfig): Boolean = + nanoStatus(config) != FeatureStatus.UNAVAILABLE + + /** Whether Gemini Nano is downloaded or can be downloaded now. */ + suspend fun isNanoAvailable(config: NanoConfig): Boolean = + when (nanoStatus(config)) { + FeatureStatus.DOWNLOADABLE, FeatureStatus.AVAILABLE -> true + else -> false + } + + suspend fun getModelInfo(modelType: String, config: NanoConfig): AdkModelInfo { + if (modelType != "genai-nano") { + return AdkModelInfo(availability = "available") } - return try { - when (Generation.getClient().checkStatus()) { - 1, 3 -> true - else -> false + val model = nanoClient(config) + val status = + try { + model.checkStatus() + } catch (_: Exception) { + return AdkModelInfo(availability = "unknown") } - } catch (_: Exception) { - false + + return when (status) { + FeatureStatus.UNAVAILABLE -> AdkModelInfo(availability = "unavailable") + FeatureStatus.DOWNLOADABLE -> AdkModelInfo(availability = "downloadable") + FeatureStatus.DOWNLOADING -> AdkModelInfo(availability = "downloading") + FeatureStatus.AVAILABLE -> + AdkModelInfo( + availability = "available", + baseModelName = runCatching { model.getBaseModelName() }.getOrNull(), + contextSize = runCatching { model.getTokenLimit() }.getOrNull(), + ) + else -> AdkModelInfo(availability = "unknown") } } + suspend fun countTokens(text: String, config: NanoConfig): Int = + preparedNano(config).countTokens(generateContentRequest(TextPart(text)) {}).totalTokens + suspend fun generateText( messages: ReadableArray, config: ReadableMap, @@ -105,21 +228,49 @@ class AdkAgentRunner( runId: String, ): AdkRunResult { val events = runAgent(messages, config, options, tools, stream = false, runId = runId).toList() + val completeEvents = events.filterNot { it.partial } val text = - events - .lastOrNull { it.isFinalResponse && !it.partial } - ?.content - ?.parts - ?.mapNotNull { it.text } - ?.joinToString("") + completeEvents + .lastOrNull { it.isFinalResponse } + ?.textParts(thoughts = false) ?.takeIf { it.isNotEmpty() } - ?: events - .flatMap { event -> event.content?.parts?.mapNotNull { it.text } ?: emptyList() } - .joinToString("") + ?: completeEvents.joinToString("") { it.textParts(thoughts = false) } + val reasoning = + completeEvents + .map { it.textParts(thoughts = true) } + .filter { it.isNotEmpty() } + .joinToString("\n\n") + .ifEmpty { null } + + val toolCalls = + completeEvents.flatMap { event -> + event.functionCalls().map { call -> + AdkToolCallRecord( + toolCallId = call.id ?: UUID.randomUUID().toString(), + toolName = call.name, + input = AdkJson.encode(call.args), + ) + } + } + val toolResults = + completeEvents.flatMap { event -> + event.functionResponses().map { response -> + AdkToolResultRecord( + toolCallId = response.id ?: "", + toolName = response.name, + output = AdkJson.encode(response.response), + ) + } + } - val finishReason = events.lastOrNull()?.finishReason?.name - val usage = AdkUsage.latestFromEvents(events) - return AdkRunResult(content = text, finishReason = finishReason, usage = usage) + return AdkRunResult( + content = text, + reasoning = reasoning, + toolCalls = toolCalls, + toolResults = toolResults, + finishReason = events.lastOrNull()?.finishReason?.name, + usage = AdkUsage.latestFromEvents(events), + ) } suspend fun streamText( @@ -198,22 +349,14 @@ class AdkAgentRunner( userId = userId, sessionId = sessionId, newMessage = lastMessage.content, - runConfig = com.google.adk.kt.agents.RunConfig(streamingMode = if (stream) { - com.google.adk.kt.agents.StreamingMode.SSE - } else { - com.google.adk.kt.agents.StreamingMode.NONE - }), + runConfig = RunConfig(streamingMode = if (stream) StreamingMode.SSE else StreamingMode.NONE), ) } private suspend fun createModel(config: AgentConfig): Model { return when (config.modelType) { - "genai-nano" -> { - ensureNanoPrepared() - val model = generativeModel - ?: throw IllegalStateException("Failed to initialize Gemini Nano") - NanoFunctionCallingModel(GenaiPrompt.create(model, config.modelName)) - } + "genai-nano" -> + NanoFunctionCallingModel(GenaiPrompt.create(preparedNano(config.nano), config.modelName)) else -> Gemini(name = config.modelName, apiKey = config.apiKey) } } @@ -225,23 +368,50 @@ class AdkAgentRunner( if (options == null) return null val responseFormat = options.getMap("responseFormat") - if (responseFormat?.hasKey("schema") == true) { - throw IllegalArgumentException( - "ADK GenerateContentConfig does not support responseFormat.schema yet", - ) - } + val responseSchema = + responseFormat?.getMap("schema")?.let { schema -> + try { + AdkSchema.fromReadableMap(schema) + } catch (error: Exception) { + throw AdkErrors.AdkException( + AdkErrors.INVALID_SCHEMA, + "Unsupported response schema: ${error.message}", + error, + ) + } + } val responseMimeType = when { responseFormat?.hasKey("mimeType") == true -> responseFormat.getString("mimeType") responseFormat?.getString("type") == "json" -> "application/json" else -> null } + + val thinkingConfig = + options.getMap("thinking")?.let { thinking -> + ThinkingConfig( + includeThoughts = thinking.hasKey("includeThoughts") && thinking.getBoolean("includeThoughts"), + thinkingBudget = thinking.takeIf { it.hasKey("budget") }?.getDouble("budget")?.toInt(), + ) + } + return GenerateContentConfig( temperature = options.takeIf { it.hasKey("temperature") }?.getDouble("temperature")?.toFloat(), maxOutputTokens = options.takeIf { it.hasKey("maxTokens") }?.getInt("maxTokens"), topP = options.takeIf { it.hasKey("topP") }?.getDouble("topP")?.toFloat(), topK = options.takeIf { it.hasKey("topK") }?.getDouble("topK")?.toFloat()?.toInt(), + seed = options.takeIf { it.hasKey("seed") }?.getDouble("seed")?.toInt(), + stopSequences = + options.getArray("stopSequences")?.let { array -> + (0 until array.size()).mapNotNull { index -> array.getString(index) } + }, + presencePenalty = + options.takeIf { it.hasKey("presencePenalty") }?.getDouble("presencePenalty")?.toFloat(), + frequencyPenalty = + options.takeIf { it.hasKey("frequencyPenalty") }?.getDouble("frequencyPenalty")?.toFloat(), responseMimeType = responseMimeType, + responseSchema = responseSchema, + thinkingConfig = thinkingConfig, ) } @@ -281,9 +451,10 @@ class AdkAgentRunner( when (part.getString("type")) { "text" -> part.getString("text")?.let { parts.add(Part(text = it)) } "file" -> { - val data = part.getString("data") ?: continue val mimeType = part.getString("mimeType") ?: "application/octet-stream" - val bytes = Base64.decode(data, Base64.DEFAULT) + val bytes = part.getString("data")?.let { Base64.decode(it, Base64.DEFAULT) } + ?: part.getString("uri")?.let { readUri(it) } + ?: continue parts.add(Part(inlineData = Blob(mimeType = mimeType, data = bytes))) } } @@ -296,6 +467,10 @@ class AdkAgentRunner( return Content(role = role, parts = parts) } + private fun readUri(uri: String): ByteArray = + context.contentResolver.openInputStream(Uri.parse(uri))?.use { it.readBytes() } + ?: throw IllegalArgumentException("Cannot read file: $uri") + private fun parseTools( tools: ReadableArray?, toolCallScope: ToolCallScope?, @@ -396,6 +571,7 @@ class AdkAgentRunner( modelType = model.getString("type") ?: "genai-nano", modelName = model.getString("name") ?: "gemini-nano", apiKey = model.getString("apiKey"), + nano = NanoConfig.from(model.getMap("nano")), ) } } @@ -408,6 +584,7 @@ data class AgentConfig( val modelType: String, val modelName: String, val apiKey: String?, + val nano: NanoConfig, ) /** ADK content role — one of [Role.USER], [Role.MODEL], or [Role.SYSTEM]. */ diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkErrors.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkErrors.kt new file mode 100644 index 000000000..107478c33 --- /dev/null +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkErrors.kt @@ -0,0 +1,57 @@ +package com.callstack.ai.adk + +import com.google.mlkit.genai.common.GenAiException +import com.google.mlkit.genai.common.GenAiException.ErrorCode + +/** Error codes shared with `AdkErrorCodes` in `src/errors.ts`. */ +object AdkErrors { + const val MODEL_UNAVAILABLE = "MODEL_UNAVAILABLE" + const val MODEL_DOWNLOADING = "MODEL_DOWNLOADING" + const val DOWNLOAD_FAILED = "DOWNLOAD_FAILED" + const val GENERATION_ERROR = "GENERATION_ERROR" + const val INVALID_MESSAGE = "INVALID_MESSAGE" + const val INVALID_SCHEMA = "INVALID_SCHEMA" + const val CONTEXT_WINDOW_EXCEEDED = "CONTEXT_WINDOW_EXCEEDED" + const val RATE_LIMITED = "RATE_LIMITED" + const val BACKGROUND_USE_BLOCKED = "BACKGROUND_USE_BLOCKED" + + /** Exception that already knows its error code. */ + class AdkException(val code: String, message: String, cause: Throwable? = null) : + Exception(message, cause) + + fun codeFor(error: Throwable): String { + var current: Throwable? = error + while (current != null) { + when (current) { + is AdkException -> return current.code + is GenAiException -> return codeFor(current) + is IllegalArgumentException -> return INVALID_MESSAGE + } + current = current.cause + } + + // Cloud Gemini surfaces HTTP status codes in the message + val message = error.message.orEmpty() + return when { + message.contains("RESOURCE_EXHAUSTED") || message.contains("429") -> RATE_LIMITED + message.contains("exceeds the maximum number of tokens") -> CONTEXT_WINDOW_EXCEEDED + else -> GENERATION_ERROR + } + } + + private fun codeFor(error: GenAiException): String = + when (error.errorCode) { + ErrorCode.NOT_AVAILABLE, + ErrorCode.NOT_SUPPORTED, + ErrorCode.AICORE_INCOMPATIBLE, + ErrorCode.NEEDS_SYSTEM_UPDATE, + ErrorCode.NOT_ENOUGH_DISK_SPACE -> MODEL_UNAVAILABLE + ErrorCode.BUSY, + ErrorCode.PER_APP_BATTERY_USE_QUOTA_EXCEEDED -> RATE_LIMITED + ErrorCode.BACKGROUND_USE_BLOCKED -> BACKGROUND_USE_BLOCKED + ErrorCode.REQUEST_TOO_LARGE -> CONTEXT_WINDOW_EXCEEDED + ErrorCode.INVALID_INPUT_IMAGE, + ErrorCode.REQUEST_TOO_SMALL -> INVALID_MESSAGE + else -> GENERATION_ERROR + } +} diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkSchema.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkSchema.kt index 00c935126..a0c45b10e 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkSchema.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkSchema.kt @@ -6,6 +6,10 @@ import com.facebook.react.bridge.ReadableType import com.google.adk.kt.types.Schema import com.google.adk.kt.types.Type +/** + * Converts a JSON Schema from the AI SDK to an ADK [Schema]. Keywords that Gemini does not + * support, such as `$ref` or `additionalProperties`, are dropped. + */ object AdkSchema { fun fromReadableMap(map: ReadableMap): Schema { val properties = map.getMap("properties")?.let { props -> @@ -33,20 +37,69 @@ object AdkSchema { } } + // `type: ["string", "null"]` and `anyOf: [..., { type: "null" }]` both mean nullable + val typeNames = readTypeNames(map) + var nullable = typeNames.contains("null") || map.getBooleanOrNull("nullable") == true + val type = typeNames.firstOrNull { it != "null" }?.let { parseType(it) } + + val anyOf = (map.getArray("anyOf") ?: map.getArray("oneOf"))?.let { array -> + val schemas = (0 until array.size()).mapNotNull { index -> array.getMap(index) } + val nonNull = schemas.filterNot { readTypeNames(it) == listOf("null") } + if (nonNull.size < schemas.size) nullable = true + nonNull.map { fromReadableMap(it) } + } + + // A nullable value with a single option is that option, marked nullable + if (type == null && anyOf?.size == 1) { + return anyOf.first().copy(nullable = if (nullable) true else anyOf.first().nullable) + } + return Schema( - type = parseType(map.getString("type")), + type = type, properties = properties, items = map.getMap("items")?.let { fromReadableMap(it) }, required = required, description = map.getString("description"), enum = enumValues, + format = map.getString("format"), + nullable = if (nullable) true else null, + anyOf = anyOf?.takeIf { it.isNotEmpty() }, + title = map.getString("title"), + pattern = map.getString("pattern"), + minimum = map.getDoubleOrNull("minimum"), + maximum = map.getDoubleOrNull("maximum"), + minLength = map.getDoubleOrNull("minLength")?.toLong(), + maxLength = map.getDoubleOrNull("maxLength")?.toLong(), + minItems = map.getDoubleOrNull("minItems")?.toLong(), + maxItems = map.getDoubleOrNull("maxItems")?.toLong(), + minProperties = map.getDoubleOrNull("minProperties")?.toLong(), + maxProperties = map.getDoubleOrNull("maxProperties")?.toLong(), ) } + private fun readTypeNames(map: ReadableMap): List { + if (!map.hasKey("type")) return emptyList() + return when (map.getType("type")) { + ReadableType.String -> listOfNotNull(map.getString("type")) + ReadableType.Array -> map.getArray("type")?.strings() ?: emptyList() + else -> emptyList() + } + } + + private fun ReadableArray.strings(): List = + (0 until size()).mapNotNull { index -> getString(index) } + + private fun ReadableMap.getDoubleOrNull(key: String): Double? = + if (hasKey(key) && getType(key) == ReadableType.Number) getDouble(key) else null + + private fun ReadableMap.getBooleanOrNull(key: String): Boolean? = + if (hasKey(key) && getType(key) == ReadableType.Boolean) getBoolean(key) else null + private fun parseType(type: String?): Type? { return when (type) { "string" -> Type.STRING - "number", "integer" -> Type.NUMBER + "number" -> Type.NUMBER + "integer" -> Type.INTEGER "boolean" -> Type.BOOLEAN "array" -> Type.ARRAY "object" -> Type.OBJECT diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt index 928907bc4..705ad4543 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt @@ -83,6 +83,22 @@ class AdkStreamToolCallEmitter( } } + /** Emits the result of each tool call that ADK ran in this event. */ + fun handleResults(event: Event) { + for (functionResponse in event.functionResponses()) { + val toolCallId = functionResponse.id ?: continue + emit( + Arguments.createMap().apply { + putString("streamId", streamId) + putString("phase", "result") + putString("toolCallId", toolCallId) + putString("toolName", functionResponse.name) + putString("output", AdkJson.encode(functionResponse.response)) + }, + ) + } + } + private fun emitEnd(toolCallId: String, toolName: String, input: String) { emit( Arguments.createMap().apply { diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt index 76b926a98..480319291 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt @@ -13,6 +13,8 @@ object AdkUsage { usage.promptTokenCount?.let { putInt("promptTokenCount", it) } usage.candidatesTokenCount?.let { putInt("candidatesTokenCount", it) } usage.totalTokenCount?.let { putInt("totalTokenCount", it) } + usage.thoughtsTokenCount?.let { putInt("thoughtsTokenCount", it) } + usage.cachedContentTokenCount?.let { putInt("cachedContentTokenCount", it) } } } diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt index 372a2349c..416c031f3 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt @@ -5,6 +5,7 @@ import com.facebook.react.bridge.Promise import com.facebook.react.bridge.ReactApplicationContext import com.facebook.react.bridge.ReadableArray import com.facebook.react.bridge.ReadableMap +import com.google.adk.kt.events.Event import com.google.adk.kt.types.UsageMetadata import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers @@ -22,51 +23,96 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : override fun getName(): String = NAME private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO) - private val runner = AdkAgentRunner { toolCallId, toolId, arguments, toolCallScope -> - toolCallScope?.streamId?.let { AdkToolBridge.registerToolCall(it, toolCallId) } - toolCallScope?.runId?.let { AdkToolBridge.registerToolCall(it, toolCallId) } - val args = Arguments.createMap().apply { - putString("toolCallId", toolCallId) - putString("toolId", toolId) - putString("arguments", arguments) - toolCallScope?.streamId?.let { putString("streamId", it) } - toolCallScope?.runId?.let { putString("runId", it) } - } - emitOnToolCall(args) + private val runner = AdkAgentRunner( + context = reactContext, + onToolCall = { toolCallId, toolId, arguments, toolCallScope -> + toolCallScope?.streamId?.let { AdkToolBridge.registerToolCall(it, toolCallId) } + toolCallScope?.runId?.let { AdkToolBridge.registerToolCall(it, toolCallId) } + val args = Arguments.createMap().apply { + putString("toolCallId", toolCallId) + putString("toolId", toolId) + putString("arguments", arguments) + toolCallScope?.streamId?.let { putString("streamId", it) } + toolCallScope?.runId?.let { putString("runId", it) } + } + emitOnToolCall(args) + }, + onDownloadProgress = { phase, bytesDownloaded, bytesToDownload -> + emitOnNanoDownloadProgress( + Arguments.createMap().apply { + putString("phase", phase) + bytesDownloaded?.let { putDouble("bytesDownloaded", it.toDouble()) } + bytesToDownload?.let { putDouble("bytesToDownload", it.toDouble()) } + }, + ) + }, + ) + + // Streams and generateText runs, by id, so both can be cancelled + private val activeRuns = ConcurrentHashMap() + + private fun Promise.rejectWithCode(error: Exception) { + reject(AdkErrors.codeFor(error), error.message ?: "Unknown ADK error", error) } - private val activeStreams = ConcurrentHashMap() - override fun isAvailable(modelType: String, promise: Promise) { + override fun isAvailable(modelType: String, nano: ReadableMap?, promise: Promise) { scope.launch { try { val available = when (modelType) { - "genai-nano" -> runner.isNanoAvailable() + "genai-nano" -> runner.isNanoAvailable(NanoConfig.from(nano)) else -> true } promise.resolve(available) - } catch (error: Exception) { + } catch (_: Exception) { promise.resolve(false) } } } - override fun isNanoSupported(promise: Promise) { + override fun isNanoSupported(nano: ReadableMap?, promise: Promise) { scope.launch { try { - promise.resolve(runner.isNanoSupported()) + promise.resolve(runner.isNanoSupported(NanoConfig.from(nano))) } catch (_: Exception) { promise.resolve(false) } } } - override fun prepareNano(promise: Promise) { + override fun getModelInfo(modelType: String, nano: ReadableMap?, promise: Promise) { scope.launch { try { - runner.prepareNano() + val info = runner.getModelInfo(modelType, NanoConfig.from(nano)) + promise.resolve( + Arguments.createMap().apply { + putString("availability", info.availability) + info.baseModelName?.let { putString("baseModelName", it) } + info.contextSize?.let { putInt("contextSize", it) } + }, + ) + } catch (error: Exception) { + promise.rejectWithCode(error) + } + } + } + + override fun prepareNano(nano: ReadableMap?, promise: Promise) { + scope.launch { + try { + runner.prepareNano(NanoConfig.from(nano)) promise.resolve(null) } catch (error: Exception) { - promise.reject("ADK_PREPARE_ERROR", error.message, error) + promise.rejectWithCode(error) + } + } + } + + override fun countTokens(text: String, nano: ReadableMap?, promise: Promise) { + scope.launch { + try { + promise.resolve(runner.countTokens(text, NanoConfig.from(nano)).toDouble()) + } catch (error: Exception) { + promise.rejectWithCode(error) } } } @@ -79,22 +125,53 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : tools: ReadableArray?, promise: Promise, ) { - scope.launch { + val job = scope.launch { try { val result = runner.generateText(messages, config, options, tools, runId) val response = Arguments.createMap().apply { putString("role", "assistant") putString("content", result.content) + result.reasoning?.let { putString("reasoning", it) } + putArray( + "toolCalls", + Arguments.createArray().apply { + result.toolCalls.forEach { call -> + pushMap( + Arguments.createMap().apply { + putString("toolCallId", call.toolCallId) + putString("toolName", call.toolName) + putString("input", call.input) + }, + ) + } + }, + ) + putArray( + "toolResults", + Arguments.createArray().apply { + result.toolResults.forEach { toolResult -> + pushMap( + Arguments.createMap().apply { + putString("toolCallId", toolResult.toolCallId) + putString("toolName", toolResult.toolName) + putString("output", toolResult.output) + }, + ) + } + }, + ) result.finishReason?.let { putString("finishReason", it) } AdkUsage.toWritableMap(result.usage)?.let { putMap("usage", it) } } promise.resolve(response) } catch (error: Exception) { - promise.reject("ADK_GENERATION_ERROR", error.message, error) + promise.rejectWithCode(error) } finally { + activeRuns.remove(runId) AdkToolBridge.cancelForStream(runId) } } + activeRuns[runId] = job } override fun streamText( @@ -105,8 +182,8 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : tools: ReadableArray?, promise: Promise, ) { - if (activeStreams.containsKey(streamId)) { - promise.reject("ADK_STREAM_ERROR", "Stream ID already in use: $streamId") + if (activeRuns.containsKey(streamId)) { + promise.reject(AdkErrors.GENERATION_ERROR, "Stream ID already in use: $streamId") return } @@ -124,9 +201,12 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : // currentTurnText: full reconstructed text of the current agent turn // emittedText: the prefix of currentTurnText already sent to JS // suppressCurrentTurn: true once we confirm this turn is a tool-code turn + // currentTurnReasoning / emittedReasoning: the same for thought parts var currentTurnText = "" var emittedText = "" var suppressCurrentTurn = false + var currentTurnReasoning = "" + var emittedReasoning = "" val toolCallEmitter = AdkStreamToolCallEmitter(streamId) { args -> emitOnStreamToolCall(args) } @@ -136,8 +216,22 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : event.usageMetadata?.let { latestUsage = it } toolCallEmitter.handleEvent(event) + toolCallEmitter.handleResults(event) - val text = event.content?.parts?.mapNotNull { it.text }?.joinToString("") ?: "" + val reasoning = event.textParts(thoughts = true) + if (reasoning.isNotEmpty()) { + currentTurnReasoning = + if (reasoning.startsWith(currentTurnReasoning)) reasoning + else currentTurnReasoning + reasoning + if (currentTurnReasoning.length > emittedReasoning.length && + currentTurnReasoning.startsWith(emittedReasoning) + ) { + emitDelta(streamId, currentTurnReasoning.substring(emittedReasoning.length), "reasoning") + emittedReasoning = currentTurnReasoning + } + } + + val text = event.textParts(thoughts = false) if (text.isNotEmpty()) { // Reconstruct the full turn text. If the event repeats what we already have @@ -158,6 +252,8 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : currentTurnText = "" emittedText = "" suppressCurrentTurn = false + currentTurnReasoning = "" + emittedReasoning = "" } if (!suppressCurrentTurn && !(event.isFinalResponse && !event.partial)) { @@ -165,14 +261,8 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : // so we never emit a partial "```", "```tool", etc. that later completes. val safeEnd = currentTurnText.length - pendingMarkerPrefixLength(currentTurnText) if (safeEnd > emittedText.length) { - val delta = currentTurnText.substring(emittedText.length, safeEnd) + emitDelta(streamId, currentTurnText.substring(emittedText.length, safeEnd), "text") emittedText = currentTurnText.substring(0, safeEnd) - - val updateArgs = Arguments.createMap().apply { - putString("streamId", streamId) - putString("delta", delta) - } - emitOnStreamUpdate(updateArgs) } } @@ -180,42 +270,53 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : // Flush any held-back text for a normal turn (e.g. a response legitimately // ending in backticks that turned out not to be a tool-code marker). if (!suppressCurrentTurn && currentTurnText.length > emittedText.length) { - val delta = currentTurnText.substring(emittedText.length) + emitDelta(streamId, currentTurnText.substring(emittedText.length), "text") emittedText = currentTurnText - val updateArgs = Arguments.createMap().apply { - putString("streamId", streamId) - putString("delta", delta) - } - emitOnStreamUpdate(updateArgs) } - val completeArgs = Arguments.createMap().apply { - putString("streamId", streamId) - putString("finishReason", event.finishReason?.name) - AdkUsage.toWritableMap(latestUsage ?: event.usageMetadata)?.let { - putMap("usage", it) - } - } - emitOnStreamComplete(completeArgs) + emitComplete(streamId, event, latestUsage) } } } catch (error: Exception) { val errorArgs = Arguments.createMap().apply { putString("streamId", streamId) putString("error", error.message ?: "Unknown ADK stream error") + putString("code", AdkErrors.codeFor(error)) } emitOnStreamError(errorArgs) } finally { - activeStreams.remove(streamId) + activeRuns.remove(streamId) } } - activeStreams[streamId] = job + activeRuns[streamId] = job promise.resolve(null) } + private fun emitDelta(streamId: String, delta: String, kind: String) { + emitOnStreamUpdate( + Arguments.createMap().apply { + putString("streamId", streamId) + putString("delta", delta) + putString("kind", kind) + }, + ) + } + + private fun emitComplete(streamId: String, event: Event, latestUsage: UsageMetadata?) { + emitOnStreamComplete( + Arguments.createMap().apply { + putString("streamId", streamId) + event.finishReason?.name?.let { putString("finishReason", it) } + AdkUsage.toWritableMap(latestUsage ?: event.usageMetadata)?.let { + putMap("usage", it) + } + }, + ) + } + override fun cancelStream(streamId: String, promise: Promise) { - activeStreams.remove(streamId)?.cancel() + activeRuns.remove(streamId)?.cancel() AdkToolBridge.cancelForStream(streamId) promise.resolve(null) } @@ -225,8 +326,8 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : } override fun invalidate() { - activeStreams.values.forEach { it.cancel() } - activeStreams.clear() + activeRuns.values.forEach { it.cancel() } + activeRuns.clear() AdkToolBridge.cancelAll() scope.cancel() super.invalidate() diff --git a/packages/adk/src/NativeAdkEngine.ts b/packages/adk/src/NativeAdkEngine.ts index 4862243cc..9c5fffc5a 100644 --- a/packages/adk/src/NativeAdkEngine.ts +++ b/packages/adk/src/NativeAdkEngine.ts @@ -5,7 +5,7 @@ export type AdkMessageRole = 'assistant' | 'system' | 'user' export type AdkMessagePart = | { type: 'text'; text: string } - | { type: 'file'; mimeType: string; data: string } + | { type: 'file'; mimeType: string; data?: string; uri?: string } export interface AdkMessage { role: AdkMessageRole @@ -15,6 +15,13 @@ export interface AdkMessage { export type AdkModelType = 'gemini' | 'genai-nano' +export interface AdkNanoModelConfig { + /** `stable` (default) or `preview`, which needs the AICore developer preview */ + releaseStage?: 'stable' | 'preview' + /** `fast` favors latency, `full` favors quality */ + preference?: 'fast' | 'full' +} + export interface AdkAgentConfig { name: string description?: string @@ -23,6 +30,7 @@ export interface AdkAgentConfig { type: AdkModelType name: string apiKey?: string + nano?: AdkNanoModelConfig } } @@ -52,25 +60,59 @@ export interface AdkGenerationOptions { maxTokens?: number topP?: number topK?: number + seed?: number + stopSequences?: string[] + presencePenalty?: number + frequencyPenalty?: number responseFormat?: AdkResponseFormat + /** Thinking on/off with an optional budget. Gemini Nano only supports on/off */ + thinking?: { + includeThoughts: boolean + budget?: number + } } export interface AdkUsageMetadata { promptTokenCount?: number candidatesTokenCount?: number totalTokenCount?: number + thoughtsTokenCount?: number + cachedContentTokenCount?: number +} + +export interface AdkToolCallRecord { + toolCallId: string + toolName: string + input: string +} + +export interface AdkToolResultRecord { + toolCallId: string + toolName: string + output: string } export interface AdkGeneratedMessage { role: AdkMessageRole content: string + reasoning?: string + toolCalls?: AdkToolCallRecord[] + toolResults?: AdkToolResultRecord[] finishReason?: string usage?: AdkUsageMetadata } +export interface AdkNativeModelInfo { + availability: string + baseModelName?: string + contextSize?: number +} + export interface StreamUpdateEvent { streamId: string delta: string + /** `text` (default) or `reasoning` */ + kind?: string } export interface StreamCompleteEvent { @@ -82,15 +124,17 @@ export interface StreamCompleteEvent { export interface StreamErrorEvent { streamId: string error: string + code?: string } export interface StreamToolCallEvent { streamId: string - phase: 'start' | 'delta' | 'end' + phase: 'start' | 'delta' | 'end' | 'result' toolCallId: string toolName?: string inputDelta?: string input?: string + output?: string } export interface ToolCallEvent { @@ -101,10 +145,25 @@ export interface ToolCallEvent { runId?: string } +export interface NanoDownloadProgressEvent { + /** `started`, `progress` or `completed` */ + phase: string + bytesDownloaded?: number + bytesToDownload?: number +} + export interface Spec extends TurboModule { - isAvailable(modelType: AdkModelType): Promise - isNanoSupported(): Promise - prepareNano(): Promise + isAvailable( + modelType: AdkModelType, + nano?: AdkNanoModelConfig + ): Promise + isNanoSupported(nano?: AdkNanoModelConfig): Promise + getModelInfo( + modelType: AdkModelType, + nano?: AdkNanoModelConfig + ): Promise + prepareNano(nano?: AdkNanoModelConfig): Promise + countTokens(text: string, nano?: AdkNanoModelConfig): Promise generateText( runId: string, @@ -122,6 +181,7 @@ export interface Spec extends TurboModule { tools?: AdkTool[] ): Promise + /** Cancels a stream or a `generateText` run by its id */ cancelStream(streamId: string): Promise submitToolResult(toolCallId: string, result: string): void @@ -130,6 +190,7 @@ export interface Spec extends TurboModule { onStreamError: CodegenTypes.EventEmitter onStreamToolCall: CodegenTypes.EventEmitter onToolCall: CodegenTypes.EventEmitter + onNanoDownloadProgress: CodegenTypes.EventEmitter } declare global { @@ -148,9 +209,11 @@ export default NativeAdkEngine export function getNativeAdkEngine(): Spec { if (!NativeAdkEngine) { - throw new Error( + const error = new Error( '@react-native-ai/adk is only available on Android. See https://developer.android.com/ai/adk' - ) + ) as Error & { code: string } + error.code = 'UNSUPPORTED_PLATFORM' + throw error } return NativeAdkEngine } diff --git a/packages/adk/src/__tests__/ai-sdk.test.ts b/packages/adk/src/__tests__/ai-sdk.test.ts index 17490adc3..654eb6a6a 100644 --- a/packages/adk/src/__tests__/ai-sdk.test.ts +++ b/packages/adk/src/__tests__/ai-sdk.test.ts @@ -4,6 +4,31 @@ import type { } from '@ai-sdk/provider' import { beforeEach, describe, expect, mock, test } from 'bun:test' +type Listener = (event: any) => void + +function createEmitter() { + const listeners = new Set() + const subscribe = (listener: Listener) => { + listeners.add(listener) + return { remove: () => listeners.delete(listener) } + } + subscribe.emit = (event: unknown) => { + for (const listener of listeners) listener(event) + } + return subscribe +} + +const onStreamUpdate = createEmitter() +const onStreamComplete = createEmitter() +const onStreamError = createEmitter() +const onStreamToolCall = createEmitter() +const onNanoDownloadProgress = createEmitter() + +let generateTextResult: Record = { + content: 'ok', + finishReason: 'STOP', +} + const generateText = mock( async ( _runId: string, @@ -11,27 +36,48 @@ const generateText = mock( _config: unknown, _options: unknown, _tools: unknown - ) => ({ content: 'ok', finishReason: 'STOP' }) + ): Promise> => generateTextResult ) +const streamText = mock( + async ( + _streamId: string, + _messages: unknown[], + _config: unknown, + _options: unknown, + _tools: unknown + ) => {} +) +const prepareNano = mock(async (_nano?: unknown) => {}) +const cancelStream = mock(async (_id: string) => {}) +const getModelInfo = mock(async (_modelType: string, _nano?: unknown) => ({ + availability: 'downloadable', +})) mock.module('react-native', () => ({ Platform: { OS: 'android' }, TurboModuleRegistry: { getEnforcing: () => ({ generateText, - streamText: async () => {}, + streamText, + prepareNano, + cancelStream, + getModelInfo, + isNanoSupported: async () => true, + isAvailable: async () => true, onToolCall: () => ({ remove: () => {} }), - onStreamUpdate: () => ({ remove: () => {} }), - onStreamToolCall: () => ({ remove: () => {} }), - onStreamComplete: () => ({ remove: () => {} }), - onStreamError: () => ({ remove: () => {} }), + onStreamUpdate, + onStreamToolCall, + onStreamComplete, + onStreamError, + onNanoDownloadProgress, }), }, })) const { createAdkProvider } = await import('../ai-sdk') -const model = createAdkProvider({ modelType: 'gemini' })() +const gemini = createAdkProvider({ modelType: 'gemini' })() +const nano = createAdkProvider()() // PNG signature followed by padding bytes const pngBytes = new Uint8Array([ @@ -47,13 +93,79 @@ function createCallOptions( } } +const lastGenerateCall = () => generateText.mock.calls.at(-1)! + +async function readAll(stream: ReadableStream) { + const parts: LanguageModelV4StreamPart[] = [] + const reader = stream.getReader() + while (true) { + const { done, value } = await reader.read() + if (done) return parts + parts.push(value) + } +} + +// Starts a stream, then plays native events once the native call is made +async function runStream( + options: LanguageModelV4CallOptions, + play: (streamId: string) => void, + model = gemini +) { + streamText.mockImplementationOnce(async (streamId: string) => { + queueMicrotask(() => play(streamId)) + }) + const { stream } = await model.doStream(options) + return readAll(stream) +} + beforeEach(() => { generateText.mockClear() + streamText.mockClear() + prepareNano.mockClear() + cancelStream.mockClear() + generateTextResult = { content: 'ok', finishReason: 'STOP' } }) -describe('adk prompt conversion', () => { +describe('model resolution', () => { + test('defaults to on-device Gemini Nano', () => { + expect(nano.modelId).toBe('gemini-nano') + }) + + test('defaults cloud Gemini to gemini-2.5-flash', () => { + expect(gemini.modelId).toBe('gemini-2.5-flash') + }) + + test('infers cloud Gemini from a model name', async () => { + const model = createAdkProvider()('gemini-2.5-pro') + await model.doGenerate(createCallOptions()) + + expect(lastGenerateCall()[2]).toMatchObject({ + model: { type: 'gemini', name: 'gemini-2.5-pro' }, + }) + }) + + test('per-model options override provider defaults', () => { + const provider = createAdkProvider({ instruction: 'Be brief.' }) + expect(provider({ modelName: 'gemini-2.5-pro' }).modelId).toBe( + 'gemini-2.5-pro' + ) + expect(provider.languageModel().modelId).toBe('gemini-nano') + }) + + test('reports model availability', async () => { + expect(await createAdkProvider().getModelInfo()).toEqual({ + modelType: 'genai-nano', + modelName: 'gemini-nano', + availability: 'downloadable', + baseModelName: undefined, + contextSize: undefined, + }) + }) +}) + +describe('prompt conversion', () => { test('resolves top-level file media types to a full mime type', async () => { - await model.doGenerate( + await gemini.doGenerate( createCallOptions({ prompt: [ { @@ -71,7 +183,7 @@ describe('adk prompt conversion', () => { }) ) - expect(generateText.mock.calls.at(-1)![1]).toEqual([ + expect(lastGenerateCall()[1]).toEqual([ { role: 'user', parts: [ @@ -85,70 +197,405 @@ describe('adk prompt conversion', () => { }, ]) }) + + test('passes local image URLs to native code', async () => { + await gemini.doGenerate( + createCallOptions({ + prompt: [ + { + role: 'user', + content: [ + { + type: 'file', + mediaType: 'image/jpeg', + data: { type: 'url', url: new URL('file:///tmp/cat.jpg') }, + }, + ], + }, + ], + }) + ) + + expect(lastGenerateCall()[1]).toEqual([ + { + role: 'user', + parts: [ + { type: 'file', mimeType: 'image/jpeg', uri: 'file:///tmp/cat.jpg' }, + ], + }, + ]) + }) + + test('replays earlier tool steps as assistant text', async () => { + await gemini.doGenerate( + createCallOptions({ + prompt: [ + { role: 'user', content: [{ type: 'text', text: 'time?' }] }, + { + role: 'assistant', + content: [ + { + type: 'tool-call', + toolCallId: '1', + toolName: 'getTime', + input: { city: 'Warsaw' }, + }, + { + type: 'tool-result', + toolCallId: '1', + toolName: 'getTime', + output: { type: 'json', value: { time: '10:00' } }, + }, + { type: 'text', text: 'It is 10:00.' }, + ], + }, + { role: 'user', content: [{ type: 'text', text: 'thanks' }] }, + ], + }) + ) + + expect(lastGenerateCall()[1][1]).toEqual({ + role: 'assistant', + content: + '[Tool call: getTime({"city":"Warsaw"})]\n[Tool result for getTime: {"time":"10:00"}]\nIt is 10:00.', + }) + }) }) -const reasoningWarning = { - type: 'unsupported', - feature: 'reasoning', - details: 'ADK models do not support configuring reasoning.', +const weatherTool = { + type: 'function' as const, + name: 'getWeather', + inputSchema: { + type: 'object' as const, + properties: { city: { type: 'string' } }, + required: ['city'], + }, } -async function readFirstPart( - stream: ReadableStream -) { - const { value } = await stream.getReader().read() - return value -} +describe('tool choice', () => { + test('sends no tools for toolChoice none', async () => { + await gemini.doGenerate( + createCallOptions({ tools: [weatherTool], toolChoice: { type: 'none' } }) + ) + + expect(lastGenerateCall()[4]).toEqual([]) + }) -describe('adk language model warnings', () => { - test('warns that reasoning is unsupported in doGenerate', async () => { - const result = await model.doGenerate( - createCallOptions({ reasoning: 'high' }) + test('warns that required tool choice is unsupported', async () => { + const result = await gemini.doGenerate( + createCallOptions({ toolChoice: { type: 'required' } }) ) - expect(result.warnings).toEqual([reasoningWarning]) + expect(result.warnings).toEqual([ + { + type: 'unsupported', + feature: 'toolChoice', + details: + 'ADK runs tools natively and cannot be required to call a tool.', + }, + ]) + }) + + test('limits tools to the chosen tool', async () => { + await gemini.doGenerate( + createCallOptions({ + tools: [weatherTool, { ...weatherTool, name: 'getTime' }], + toolChoice: { type: 'tool', toolName: 'getTime' }, + }) + ) + + expect( + (lastGenerateCall()[4] as { name: string }[]).map((tool) => tool.name) + ).toEqual(['getTime']) }) - test('warns that reasoning is unsupported in doStream', async () => { - const { stream } = await model.doStream( - createCallOptions({ reasoning: 'high' }) + test('warns when a tool has no registered executor', async () => { + const result = await gemini.doGenerate( + createCallOptions({ tools: [weatherTool] }) ) - expect(await readFirstPart(stream)).toEqual({ - type: 'stream-start', - warnings: [reasoningWarning], + expect(result.warnings).toContainEqual({ + type: 'other', + message: + 'Tool "getWeather" has no executor. Pass it to createAdkProvider via availableTools so ADK can run it.', + }) + }) +}) + +describe('generation options', () => { + test('maps reasoning levels to a thinking budget', async () => { + await gemini.doGenerate(createCallOptions({ reasoning: 'high' })) + + expect(lastGenerateCall()[3]).toMatchObject({ + thinking: { includeThoughts: true, budget: 24576 }, }) }) - test('does not warn for provider-default reasoning', async () => { - const result = await model.doGenerate( - createCallOptions({ reasoning: 'provider-default' }) + test('disables thinking for reasoning none', async () => { + await gemini.doGenerate(createCallOptions({ reasoning: 'none' })) + + expect(lastGenerateCall()[3]).toMatchObject({ + thinking: { includeThoughts: false, budget: 0 }, + }) + }) + + test('leaves thinking to the model by default', async () => { + await gemini.doGenerate(createCallOptions()) + + expect((lastGenerateCall()[3] as { thinking?: unknown }).thinking).toBe( + undefined ) + }) - expect(result.warnings).toEqual([]) + test('passes a response schema to cloud Gemini', async () => { + const schema = { type: 'object', properties: { name: { type: 'string' } } } + await gemini.doGenerate( + createCallOptions({ responseFormat: { type: 'json', schema } }) + ) + + expect(lastGenerateCall()[3]).toMatchObject({ + responseFormat: { type: 'json', schema }, + }) + }) + + test('puts the response schema in the Gemini Nano instruction', async () => { + const schema = { type: 'object', properties: { name: { type: 'string' } } } + await nano.doGenerate( + createCallOptions({ responseFormat: { type: 'json', schema } }) + ) + + const [, , config, options] = lastGenerateCall() as [ + string, + unknown, + { instruction: string }, + { responseFormat: { schema?: unknown } }, + ] + expect(config.instruction).toContain(JSON.stringify(schema)) + expect(options.responseFormat.schema).toBe(undefined) + }) + + test('warns about settings Gemini Nano ignores', async () => { + const result = await nano.doGenerate( + createCallOptions({ topP: 0.5, stopSequences: ['END'] }) + ) + + expect( + result.warnings.map((warning) => 'feature' in warning && warning.feature) + ).toEqual(['topP', 'stopSequences']) }) }) -describe('reasoning none and tool choice warnings', () => { - test('does not warn for reasoning none', async () => { - const result = await model.doGenerate( - createCallOptions({ reasoning: 'none' }) +describe('doGenerate results', () => { + test('returns reasoning, provider-executed tool steps and text', async () => { + generateTextResult = { + content: 'It is sunny.', + reasoning: 'Check the weather.', + toolCalls: [ + { toolCallId: '1', toolName: 'getWeather', input: '{"city":"Oslo"}' }, + ], + toolResults: [ + { toolCallId: '1', toolName: 'getWeather', output: '{"sky":"clear"}' }, + ], + finishReason: 'STOP', + usage: { + promptTokenCount: 10, + candidatesTokenCount: 5, + thoughtsTokenCount: 3, + cachedContentTokenCount: 4, + }, + } + + const result = await gemini.doGenerate(createCallOptions()) + + expect(result.content).toEqual([ + { type: 'reasoning', text: 'Check the weather.' }, + { + type: 'tool-call', + toolCallId: '1', + toolName: 'getWeather', + input: '{"city":"Oslo"}', + providerExecuted: true, + }, + { + type: 'tool-result', + toolCallId: '1', + toolName: 'getWeather', + result: { sky: 'clear' }, + }, + { type: 'text', text: 'It is sunny.' }, + ]) + expect(result.usage.inputTokens).toEqual({ + total: 10, + noCache: 6, + cacheRead: 4, + cacheWrite: undefined, + }) + expect(result.usage.outputTokens).toEqual({ + total: 8, + text: 5, + reasoning: 3, + }) + }) + + test('strips Markdown fences from JSON output', async () => { + generateTextResult = { content: '```json\n{"a":1}\n```' } + + const result = await nano.doGenerate( + createCallOptions({ responseFormat: { type: 'json' } }) ) - expect(result.warnings).toEqual([]) + expect(result.content).toEqual([{ type: 'text', text: '{"a":1}' }]) }) - test('warns that required tool choice is unsupported', async () => { - const result = await model.doGenerate( - createCallOptions({ toolChoice: { type: 'required' } }) + test('maps safety finish reasons to content-filter', async () => { + generateTextResult = { content: '', finishReason: 'SAFETY' } + + const result = await gemini.doGenerate(createCallOptions()) + + expect(result.finishReason).toEqual({ + unified: 'content-filter', + raw: 'SAFETY', + }) + }) + + test('attaches error codes to native rejections', async () => { + generateText.mockImplementationOnce(async () => { + throw Object.assign(new Error('busy'), { code: 'RATE_LIMITED' }) + }) + + expect(gemini.doGenerate(createCallOptions())).rejects.toMatchObject({ + message: 'busy', + code: 'RATE_LIMITED', + }) + }) + + test('cancels the native run when aborted', async () => { + const controller = new AbortController() + generateText.mockImplementationOnce(async () => { + controller.abort() + throw new Error('cancelled') + }) + + const error = await gemini + .doGenerate(createCallOptions({ abortSignal: controller.signal })) + .catch((caught: unknown) => caught) + + expect(error).toBe(controller.signal.reason) + expect(cancelStream).toHaveBeenCalledTimes(1) + }) +}) + +describe('doStream', () => { + test('streams reasoning, tool steps and text', async () => { + const parts = await runStream(createCallOptions(), (streamId) => { + onStreamUpdate.emit({ streamId, delta: 'Thinking', kind: 'reasoning' }) + onStreamToolCall.emit({ + streamId, + phase: 'start', + toolCallId: '1', + toolName: 'getWeather', + }) + onStreamToolCall.emit({ + streamId, + phase: 'end', + toolCallId: '1', + toolName: 'getWeather', + input: '{}', + }) + onStreamToolCall.emit({ + streamId, + phase: 'result', + toolCallId: '1', + toolName: 'getWeather', + output: '{"sky":"clear"}', + }) + onStreamUpdate.emit({ streamId, delta: 'Sunny', kind: 'text' }) + onStreamComplete.emit({ streamId, finishReason: 'STOP' }) + }) + + expect(parts.map((part) => part.type)).toEqual([ + 'stream-start', + 'reasoning-start', + 'reasoning-delta', + 'reasoning-end', + 'tool-input-start', + 'tool-input-end', + 'tool-call', + 'tool-result', + 'text-start', + 'text-delta', + 'text-end', + 'finish', + ]) + expect(parts).toContainEqual({ + type: 'tool-result', + toolCallId: '1', + toolName: 'getWeather', + result: { sky: 'clear' }, + }) + }) + + test('streams JSON without Markdown fences', async () => { + const parts = await runStream( + createCallOptions({ responseFormat: { type: 'json' } }), + (streamId) => { + for (const delta of ['``', '`json\n{"a"', ':1}', '\n`', '``']) { + onStreamUpdate.emit({ streamId, delta }) + } + onStreamComplete.emit({ streamId }) + }, + nano ) - expect(result.warnings).toEqual([ + const text = parts + .flatMap((part) => (part.type === 'text-delta' ? [part.delta] : [])) + .join('') + expect(text).toBe('{"a":1}') + }) + + test('surfaces native stream errors with their code', async () => { + const parts = await runStream(createCallOptions(), (streamId) => { + onStreamError.emit({ + streamId, + error: 'Too long', + code: 'CONTEXT_WINDOW_EXCEEDED', + }) + }) + + expect(parts.at(-1)).toMatchObject({ + type: 'error', + error: { message: 'Too long', code: 'CONTEXT_WINDOW_EXCEEDED' }, + }) + }) +}) + +describe('Gemini Nano preparation', () => { + test('reports download progress', async () => { + const provider = createAdkProvider() + const progress: unknown[] = [] + prepareNano.mockImplementationOnce(async () => { + onNanoDownloadProgress.emit({ phase: 'started', bytesToDownload: 100 }) + onNanoDownloadProgress.emit({ phase: 'completed' }) + }) + + await provider.prepareNano({ onProgress: (event) => progress.push(event) }) + + expect(progress).toEqual([ + { phase: 'started', bytesDownloaded: undefined, bytesToDownload: 100 }, { - type: 'unsupported', - feature: 'toolChoice', - details: 'ADK models only support auto and none tool choices.', + phase: 'completed', + bytesDownloaded: undefined, + bytesToDownload: undefined, }, ]) }) + + test('passes the model selection to native code', async () => { + const provider = createAdkProvider({ nano: { releaseStage: 'preview' } }) + await provider().prepare() + + expect(prepareNano.mock.calls.at(-1)![0]).toEqual({ + releaseStage: 'preview', + }) + }) }) diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index 079e1505c..e0ea49dc1 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -1,6 +1,8 @@ import type { + JSONValue, LanguageModelV4, LanguageModelV4CallOptions, + LanguageModelV4Content, LanguageModelV4FilePart, LanguageModelV4FinishReason, LanguageModelV4FunctionTool, @@ -12,7 +14,6 @@ import type { } from '@ai-sdk/provider' import { generateId, - isCustomReasoning, jsonSchema, parseJSON, resolveFullMediaType, @@ -21,133 +22,362 @@ import { } from '@ai-sdk/provider-utils' import { Platform } from 'react-native' +import { AdkErrorCodes, createAdkError, toAdkError } from './errors' import { type AdkAgentConfig, type AdkGenerationOptions, type AdkMessage, type AdkMessagePart, type AdkModelType, + type AdkNanoModelConfig, type AdkTool, type AdkToolParameter, type AdkUsageMetadata, getNativeAdkEngine, + type NanoDownloadProgressEvent, type StreamToolCallEvent, type ToolCallEvent, } from './NativeAdkEngine' -export interface AdkProviderOptions { +const DEFAULT_NANO_MODEL = 'gemini-nano' +const DEFAULT_GEMINI_MODEL = 'gemini-2.5-flash' + +export type AdkToolDefinitionSet = Record + +/** `gemini-nano` runs on device, any other name is a cloud Gemini model */ +export type AdkLanguageModelId = 'gemini-nano' | (string & {}) + +export interface AdkLanguageModelOptions { + /** Defaults to `genai-nano`, or `gemini` when `modelName` names a cloud model */ + modelType?: AdkModelType + /** Defaults to `gemini-nano` on device and `gemini-2.5-flash` in the cloud */ + modelName?: AdkLanguageModelId + /** ADK agent name */ name?: string + /** ADK agent description */ description?: string + /** System instruction for the agent */ instruction?: string - modelType?: AdkModelType - modelName?: string + /** Gemini API key, required for `gemini` */ apiKey?: string - availableTools?: Record + /** Tools whose `execute` handlers ADK can call */ + availableTools?: AdkToolDefinitionSet + /** Gemini Nano model selection */ + nano?: AdkNanoModelConfig } -/** - * Whether the device can support on-device Gemini Nano at all - * (ML Kit `checkStatus !== 0`). Does not download models. - */ -async function checkNanoSupported(): Promise { - if (Platform.OS !== 'android') { - return false - } +/** Per-call options passed as `providerOptions.adk` */ +export interface AdkProviderOptions { + /** Thinking token budget for cloud Gemini. `0` disables thinking */ + thinkingBudget?: number + /** Whether the model's thoughts come back as reasoning parts */ + includeThoughts?: boolean +} + +export type AdkModelAvailability = + | 'available' + | 'downloadable' + | 'downloading' + | 'unavailable' + | 'unknown' + +export interface AdkModelInfo { + modelType: AdkModelType + modelName: string + availability: AdkModelAvailability + /** Name of the Gemini Nano base model, when ML Kit reports it */ + baseModelName?: string + /** Gemini Nano input token limit, when ML Kit reports it */ + contextSize?: number +} - return getNativeAdkEngine().isNanoSupported() +export interface AdkDownloadProgress { + phase: 'started' | 'progress' | 'completed' + bytesDownloaded?: number + bytesToDownload?: number +} + +export interface AdkPrepareOptions { + onProgress?: (progress: AdkDownloadProgress) => void +} + +export interface AdkLanguageModel extends LanguageModelV4 { + /** Downloads and initializes Gemini Nano. No-op for cloud models */ + prepare: () => Promise + updateTools: (tools: AdkToolDefinitionSet) => void + /** Counts tokens with the Gemini Nano tokenizer. Not available for cloud models */ + countTokens: (text: string) => Promise } export interface AdkProvider { - (): LanguageModelV4 + (options?: AdkLanguageModelId | AdkLanguageModelOptions): AdkLanguageModel readonly specificationVersion: 'v4' - languageModel: () => LanguageModelV4 - /** Device capability check (ML Kit `checkStatus !== 0`). Returns `false` on non-Android. */ + languageModel: ( + options?: AdkLanguageModelId | AdkLanguageModelOptions + ) => AdkLanguageModel + /** Whether the device can run Gemini Nano at all. Returns `false` on non-Android */ isNanoSupported: () => Promise /** - * Whether `modelType` can be used now. For `genai-nano`, checks ML Kit readiness - * (status 1 or 3). Returns `false` when {@link isNanoSupported} is `false`. - * Cloud `gemini` always resolves to `true` on Android. + * Whether `modelType` can be used now. For `genai-nano`, Gemini Nano must be + * available or downloadable. Cloud `gemini` resolves to `true` on Android */ isAvailable: (modelType?: AdkModelType) => Promise - /** Downloads and initializes on-device Gemini Nano. No-op for cloud models. */ - prepareNano: () => Promise + /** Availability details for a model type */ + getModelInfo: (options?: { + modelType?: AdkModelType + nano?: AdkNanoModelConfig + }) => Promise + /** Downloads and initializes Gemini Nano, reporting download progress */ + prepareNano: (options?: AdkPrepareOptions) => Promise +} + +type ResolvedModelOptions = Omit< + AdkLanguageModelOptions, + 'modelType' | 'modelName' +> & { + modelType: AdkModelType + modelName: string +} + +function resolveModelOptions( + defaults: AdkLanguageModelOptions, + options: AdkLanguageModelId | AdkLanguageModelOptions = {} +): ResolvedModelOptions { + const overrides = + typeof options === 'string' ? { modelName: options } : options + const merged = { ...defaults, ...overrides } + const modelType = + merged.modelType ?? + (merged.modelName && merged.modelName !== DEFAULT_NANO_MODEL + ? 'gemini' + : 'genai-nano') + const modelName = + merged.modelName ?? + (modelType === 'gemini' ? DEFAULT_GEMINI_MODEL : DEFAULT_NANO_MODEL) + + return { ...merged, modelType, modelName } +} + +const toModelAvailability = (value: string): AdkModelAvailability => { + switch (value) { + case 'available': + case 'downloadable': + case 'downloading': + case 'unavailable': + return value + default: + return 'unknown' + } +} + +async function prepareNano( + nano: AdkNanoModelConfig | undefined, + options: AdkPrepareOptions = {} +) { + const native = getNativeAdkEngine() + const listener = options.onProgress + ? native.onNanoDownloadProgress((event: NanoDownloadProgressEvent) => { + options.onProgress?.({ + phase: event.phase as AdkDownloadProgress['phase'], + bytesDownloaded: event.bytesDownloaded, + bytesToDownload: event.bytesToDownload, + }) + }) + : null + + try { + await native.prepareNano(nano) + } catch (error) { + throw toAdkError(error) + } finally { + listener?.remove() + } } export function createAdkProvider( - options: AdkProviderOptions = {} + defaults: AdkLanguageModelOptions = {} ): AdkProvider { - const createLanguageModel = () => new AdkChatLanguageModel(options) - const provider = function () { - return createLanguageModel() + const createLanguageModel = ( + options?: AdkLanguageModelId | AdkLanguageModelOptions + ): AdkLanguageModel => + new AdkChatLanguageModel(resolveModelOptions(defaults, options)) + + const provider = function ( + options?: AdkLanguageModelId | AdkLanguageModelOptions + ) { + return createLanguageModel(options) } provider.specificationVersion = 'v4' as const provider.languageModel = createLanguageModel - provider.isNanoSupported = () => checkNanoSupported() + provider.isNanoSupported = async () => { + if (Platform.OS !== 'android') { + return false + } + return getNativeAdkEngine().isNanoSupported(defaults.nano) + } provider.isAvailable = async (modelType: AdkModelType = 'genai-nano') => { - if (modelType === 'genai-nano' && !(await checkNanoSupported())) { + if (Platform.OS !== 'android') { return false } - return getNativeAdkEngine().isAvailable(modelType).then(Boolean) + return getNativeAdkEngine() + .isAvailable(modelType, defaults.nano) + .then(Boolean) + } + provider.getModelInfo = async ( + options: { modelType?: AdkModelType; nano?: AdkNanoModelConfig } = {} + ): Promise => { + const resolved = resolveModelOptions(defaults) + const { modelType, modelName } = + options.modelType && options.modelType !== resolved.modelType + ? resolveModelOptions({ modelType: options.modelType }) + : resolved + const nano = options.nano ?? defaults.nano + + if (Platform.OS !== 'android') { + return { modelType, modelName, availability: 'unavailable' } + } + + const info = await getNativeAdkEngine() + .getModelInfo(modelType, nano) + .catch((error: unknown) => { + throw toAdkError(error) + }) + return { + modelType, + modelName, + availability: toModelAvailability(info.availability), + baseModelName: info.baseModelName, + contextSize: info.contextSize, + } } - provider.prepareNano = () => getNativeAdkEngine().prepareNano() + provider.prepareNano = (options?: AdkPrepareOptions) => + prepareNano(defaults.nano, options) return provider } export const adk = createAdkProvider() type Tool = LanguageModelV4FunctionTool | LanguageModelV4ProviderTool -type ToolDefinitionSet = Record - -const createEmptyUsage = (): LanguageModelV4Usage => ({ - inputTokens: { - total: undefined, - noCache: undefined, - cacheRead: undefined, - cacheWrite: undefined, - }, - outputTokens: { - total: undefined, - text: undefined, - reasoning: undefined, - }, -}) -const mapUsage = (usage?: AdkUsageMetadata): LanguageModelV4Usage => { - if (!usage) { - return createEmptyUsage() +const toolChoiceWarning: SharedV4Warning = { + type: 'unsupported', + feature: 'toolChoice', + details: 'ADK runs tools natively and cannot be required to call a tool.', +} + +function prepareToolChoice( + tools: Tool[] = [], + toolChoice: LanguageModelV4CallOptions['toolChoice'] +): { tools: Tool[]; warnings: SharedV4Warning[] } { + switch (toolChoice?.type) { + case 'none': + return { tools: [], warnings: [] } + case 'required': + return { tools, warnings: [toolChoiceWarning] } + case 'tool': + return { + tools: tools.filter((tool) => tool.name === toolChoice.toolName), + warnings: [toolChoiceWarning], + } + default: + return { tools, warnings: [] } + } +} + +// Gemini thinking budgets for the AI SDK reasoning levels +const thinkingBudgets = { + minimal: 512, + low: 2048, + medium: 8192, + high: 24576, + xhigh: 24576, + max: 24576, +} as const + +function prepareThinking( + reasoning: LanguageModelV4CallOptions['reasoning'], + adkOptions: AdkProviderOptions +): AdkGenerationOptions['thinking'] { + if (adkOptions.thinkingBudget !== undefined) { + return { + includeThoughts: + adkOptions.includeThoughts ?? adkOptions.thinkingBudget !== 0, + budget: adkOptions.thinkingBudget, + } + } + + switch (reasoning) { + case undefined: + case 'provider-default': + return adkOptions.includeThoughts === undefined + ? undefined + : { includeThoughts: adkOptions.includeThoughts } + case 'none': + return { includeThoughts: false, budget: 0 } + default: + return { + includeThoughts: adkOptions.includeThoughts ?? true, + budget: thinkingBudgets[reasoning], + } } +} + +const mapUsage = (usage?: AdkUsageMetadata): LanguageModelV4Usage => { + const promptTokens = usage?.promptTokenCount + const cachedTokens = usage?.cachedContentTokenCount + const textTokens = usage?.candidatesTokenCount + const reasoningTokens = usage?.thoughtsTokenCount return { inputTokens: { - total: usage.promptTokenCount, - noCache: undefined, - cacheRead: undefined, + total: promptTokens, + noCache: + promptTokens !== undefined && cachedTokens !== undefined + ? promptTokens - cachedTokens + : undefined, + cacheRead: cachedTokens, cacheWrite: undefined, }, outputTokens: { - total: usage.candidatesTokenCount, - text: undefined, - reasoning: undefined, - }, - raw: { - promptTokenCount: usage.promptTokenCount, - candidatesTokenCount: usage.candidatesTokenCount, - totalTokenCount: usage.totalTokenCount, + // Gemini counts thoughts separately from candidates + total: + textTokens === undefined && reasoningTokens === undefined + ? undefined + : (textTokens ?? 0) + (reasoningTokens ?? 0), + text: textTokens, + reasoning: reasoningTokens, }, + raw: usage ? { ...usage } : undefined, } } +const contentFilterReasons = new Set([ + 'SAFETY', + 'RECITATION', + 'BLOCKLIST', + 'PROHIBITED_CONTENT', + 'SPII', + 'IMAGE_SAFETY', + 'IMAGE_PROHIBITED_CONTENT', + 'IMAGE_RECITATION', +]) + const convertFinishReason = ( finishReason?: string ): LanguageModelV4FinishReason => { let unified: LanguageModelV4FinishReason['unified'] = 'other' - if (finishReason === 'STOP' || finishReason === 'stop') { + if (finishReason === undefined || finishReason === 'STOP') { unified = 'stop' - } else if (finishReason === 'MAX_TOKENS' || finishReason === 'length') { + } else if (finishReason === 'MAX_TOKENS') { unified = 'length' - } else if (finishReason === 'tool_calls' || finishReason === 'TOOL_CALLS') { - unified = 'tool-calls' + } else if (contentFilterReasons.has(finishReason)) { + unified = 'content-filter' + } else if ( + finishReason === 'MALFORMED_FUNCTION_CALL' || + finishReason === 'UNEXPECTED_TOOL_CALL' + ) { + unified = 'error' } return { @@ -174,34 +404,6 @@ const schemaPropertyToAdkParameter = ( } } -const getCallWarnings = ( - options: LanguageModelV4CallOptions -): SharedV4Warning[] => { - const warnings: SharedV4Warning[] = [] - - // `none` is already what a model without reasoning control does - if (isCustomReasoning(options.reasoning) && options.reasoning !== 'none') { - warnings.push({ - type: 'unsupported', - feature: 'reasoning', - details: 'ADK models do not support configuring reasoning.', - }) - } - - if ( - options.toolChoice?.type === 'required' || - options.toolChoice?.type === 'tool' - ) { - warnings.push({ - type: 'unsupported', - feature: 'toolChoice', - details: 'ADK models only support auto and none tool choices.', - }) - } - - return warnings -} - const uint8ArrayToBase64 = (bytes: Uint8Array): string => { let binary = '' for (let index = 0; index < bytes.length; index++) { @@ -210,97 +412,151 @@ const uint8ArrayToBase64 = (bytes: Uint8Array): string => { return btoa(binary) } -const normalizeFilePart = ( - part: LanguageModelV4FilePart -): { data: string; mediaType: string } => { - if (part.data.type !== 'data') { - throw new Error( - 'ADK wrapper does not support file URLs or references yet. Pass base64 or Uint8Array data.' - ) +const localUrlProtocols = new Set(['file:', 'content:']) + +const toFileMessagePart = (part: LanguageModelV4FilePart): AdkMessagePart => { + // v4 allows top-level media types such as `image`, which Gemini rejects + const mimeType = resolveFullMediaType({ part }) + + switch (part.data.type) { + case 'data': + return { + type: 'file', + mimeType, + data: + typeof part.data.data === 'string' + ? part.data.data.replace(/^data:[^;]+;base64,/, '') + : uint8ArrayToBase64(part.data.data), + } + case 'url': + // The AI SDK downloads other URLs because they are not in `supportedUrls` + if (!localUrlProtocols.has(part.data.url.protocol)) { + throw createAdkError( + `ADK only reads file:// and content:// URLs. Got ${part.data.url.protocol}`, + AdkErrorCodes.InvalidMessage + ) + } + return { type: 'file', mimeType, uri: part.data.url.toString() } + case 'text': + return { type: 'text', text: part.data.text } + default: + throw createAdkError( + 'ADK does not support provider file references.', + AdkErrorCodes.InvalidMessage + ) } +} - const data = - typeof part.data.data === 'string' - ? part.data.data.replace(/^data:[^;]+;base64,/, '') - : uint8ArrayToBase64(part.data.data) +const getAbortReason = (signal: AbortSignal) => { + if (signal.reason !== undefined) { + return signal.reason + } + const error = new Error('The operation was aborted.') + error.name = 'AbortError' + return error +} - return { - data, - // v4 allows top-level media types such as `image`, which Gemini rejects - mediaType: resolveFullMediaType({ part }), +const stringifyToolOutput = (value: unknown) => + typeof value === 'string' ? value : JSON.stringify(value ?? null) + +// Gemini Nano often wraps JSON in a Markdown code fence +const jsonFencePrefix = /^\s*```(?:json)?\s*\n?/ +const stripJsonFences = (text: string) => + text.replace(jsonFencePrefix, '').replace(/\s*```\s*$/, '') + +const parseToolResult = (output: string): NonNullable => { + try { + return (JSON.parse(output) as JSONValue) ?? {} + } catch { + return output } } -class AdkChatLanguageModel implements LanguageModelV4 { +const createNanoJsonInstruction = (schema: unknown) => + schema + ? `Respond only with JSON that matches this JSON schema, without Markdown or other text:\n${JSON.stringify(schema)}` + : 'Respond only with valid JSON, without Markdown or other text.' + +class AdkChatLanguageModel implements AdkLanguageModel { readonly specificationVersion = 'v4' - readonly supportedUrls = {} + readonly supportedUrls = { + 'image/*': [/^file:\/\//, /^content:\/\//], + } readonly provider = 'adk' readonly modelId: string private readonly agentConfig: AdkAgentConfig - private tools: ToolDefinitionSet = {} + private tools: AdkToolDefinitionSet = {} private nanoPreparePromise: Promise | null = null - constructor(options: AdkProviderOptions) { - const modelType = options.modelType ?? 'genai-nano' - this.modelId = options.modelName ?? 'gemini-nano' + constructor(options: ResolvedModelOptions) { + this.modelId = options.modelName this.agentConfig = { name: options.name ?? 'react_native_adk_agent', description: options.description ?? 'React Native ADK agent', instruction: options.instruction, model: { - type: modelType, - name: this.modelId, + type: options.modelType, + name: options.modelName, apiKey: options.apiKey, + nano: options.nano, }, } this.updateTools(options.availableTools ?? {}) } - updateTools(tools: ToolDefinitionSet) { - this.tools = tools + private get isNano() { + return this.agentConfig.model.type === 'genai-nano' } - public isAvailable() { - return getNativeAdkEngine() - .isAvailable(this.agentConfig.model.type) - .then(Boolean) - } - - public prepareNano() { - return getNativeAdkEngine().prepareNano() + updateTools(tools: AdkToolDefinitionSet) { + this.tools = tools } async prepare(): Promise { await this.ensureNanoPrepared() } + async countTokens(text: string): Promise { + if (!this.isNano) { + throw createAdkError( + 'Token counting is only available for Gemini Nano.', + AdkErrorCodes.ModelUnavailable + ) + } + + try { + return await getNativeAdkEngine().countTokens( + text, + this.agentConfig.model.nano + ) + } catch (error) { + throw toAdkError(error) + } + } + private ensureNanoPrepared(): Promise { - if (this.agentConfig.model.type !== 'genai-nano') { + if (!this.isNano) { return Promise.resolve() } if (!this.nanoPreparePromise) { - this.nanoPreparePromise = checkNanoSupported() - .then((supported) => { - if (!supported) { - throw new Error( - 'Gemini Nano is not supported on this device or device has not fetched the latest configuration to support it' - ) - } - return getNativeAdkEngine().prepareNano() - }) - .catch((error) => { + this.nanoPreparePromise = prepareNano(this.agentConfig.model.nano).catch( + (error) => { this.nanoPreparePromise = null throw error - }) + } + ) } return this.nanoPreparePromise } - private prepareMessages(messages: LanguageModelV4Prompt): AdkMessage[] { + private prepareMessages( + messages: LanguageModelV4Prompt, + warnings: SharedV4Warning[] + ): AdkMessage[] { return messages.map((message): AdkMessage => { if (message.role === 'system') { return { @@ -310,19 +566,12 @@ class AdkChatLanguageModel implements LanguageModelV4 { } if (message.role === 'user') { - const parts = message.content - .map((part): AdkMessagePart | null => { - if (part.type === 'text') { - return { type: 'text', text: part.text } - } - if (part.type === 'file') { - const { data, mediaType } = normalizeFilePart(part) - return { type: 'file', mimeType: mediaType, data } - } - console.warn('Unsupported user message content type:', part) - return null - }) - .filter((part): part is AdkMessagePart => part !== null) + const parts = message.content.map((part): AdkMessagePart => { + if (part.type === 'text') { + return { type: 'text', text: part.text } + } + return toFileMessagePart(part) + }) if (parts.length === 1 && parts[0]?.type === 'text') { return { @@ -338,13 +587,34 @@ class AdkChatLanguageModel implements LanguageModelV4 { } if (message.role === 'assistant') { - const content = message.content.reduce((acc, part) => { - if (part.type === 'text') { - return acc + part.text - } - console.warn('Unsupported assistant message content type:', part) - return acc - }, '') + // ADK replays history as text, so earlier tool steps become text too + const content = message.content + .flatMap((part) => { + switch (part.type) { + case 'text': + return [part.text] + case 'tool-call': + return [ + `[Tool call: ${part.toolName}(${stringifyToolOutput(part.input)})]`, + ] + case 'tool-result': + return [ + `[Tool result for ${part.toolName}: ${stringifyToolOutput( + 'value' in part.output ? part.output.value : part.output + )}]`, + ] + case 'reasoning': + return [] + default: + warnings.push({ + type: 'unsupported', + feature: `assistant ${part.type} parts`, + details: 'ADK replays assistant messages as text only.', + }) + return [] + } + }) + .join('\n') return { role: 'assistant', @@ -359,9 +629,7 @@ class AdkChatLanguageModel implements LanguageModelV4 { if (part.output.type === 'execution-denied') { return part.output.reason ?? 'Execution denied' } - return typeof part.output.value === 'string' - ? part.output.value - : JSON.stringify(part.output.value) + return stringifyToolOutput(part.output.value) }) .join('\n') @@ -371,20 +639,54 @@ class AdkChatLanguageModel implements LanguageModelV4 { } } - throw new Error(`Unsupported message role for ADK provider`) + throw createAdkError( + 'Unsupported message role for ADK provider', + AdkErrorCodes.InvalidMessage + ) }) } - private prepareGenerationOptions( + private prepareAgentConfig( options: LanguageModelV4CallOptions + ): AdkAgentConfig { + // Gemini Nano has no response schema, so the schema goes in the instruction + if (!this.isNano || options.responseFormat?.type !== 'json') { + return this.agentConfig + } + + const instruction = [ + this.agentConfig.instruction, + createNanoJsonInstruction(options.responseFormat.schema), + ] + .filter(Boolean) + .join('\n\n') + + return { ...this.agentConfig, instruction } + } + + private prepareGenerationOptions( + options: LanguageModelV4CallOptions, + warnings: SharedV4Warning[] ): AdkGenerationOptions { - if ( - options.responseFormat?.type === 'json' && - options.responseFormat.schema - ) { - throw new Error( - 'ADK does not support responseFormat.schema yet. Use JSON mode without a schema or omit responseFormat.' - ) + const adkOptions = (options.providerOptions?.adk ?? + {}) as AdkProviderOptions + + if (this.isNano) { + const unsupported = { + topP: options.topP, + stopSequences: options.stopSequences, + presencePenalty: options.presencePenalty, + frequencyPenalty: options.frequencyPenalty, + } + for (const [feature, value] of Object.entries(unsupported)) { + if (value !== undefined) { + warnings.push({ + type: 'unsupported', + feature, + details: 'Gemini Nano does not support this setting.', + }) + } + } } return { @@ -392,19 +694,34 @@ class AdkChatLanguageModel implements LanguageModelV4 { maxTokens: options.maxOutputTokens, topP: options.topP, topK: options.topK, + seed: options.seed, + stopSequences: options.stopSequences, + presencePenalty: options.presencePenalty, + frequencyPenalty: options.frequencyPenalty, responseFormat: options.responseFormat?.type === 'json' ? { type: 'json', mimeType: 'application/json', + schema: this.isNano ? undefined : options.responseFormat.schema, } : undefined, + thinking: prepareThinking(options.reasoning, adkOptions), } } - private prepareTools(tools: Tool[] = []) { + private prepareTools(tools: Tool[], warnings: SharedV4Warning[]) { return tools - .filter((tool) => tool.type === 'function') + .filter((tool): tool is LanguageModelV4FunctionTool => { + if (tool.type === 'function') { + return true + } + warnings.push({ + type: 'unsupported', + feature: `provider-defined tool ${tool.name}`, + }) + return false + }) .map((tool) => { const required = Array.isArray(tool.inputSchema.required) ? tool.inputSchema.required @@ -417,6 +734,13 @@ class AdkChatLanguageModel implements LanguageModelV4 { const schema = jsonSchema(tool.inputSchema) + if (!this.tools[tool.name]?.execute) { + warnings.push({ + type: 'other', + message: `Tool "${tool.name}" has no executor. Pass it to createAdkProvider via availableTools so ADK can run it.`, + }) + } + return { native: { id: generateId(), @@ -426,6 +750,13 @@ class AdkChatLanguageModel implements LanguageModelV4 { parameters, } satisfies AdkTool, execute: async (modelInput: unknown) => { + const execute = this.tools[tool.name]?.execute + if (!execute) { + return { + error: `Tool "${tool.name}" is not available in this app.`, + } + } + const text = typeof modelInput === 'string' ? modelInput @@ -439,12 +770,32 @@ class AdkChatLanguageModel implements LanguageModelV4 { messages: [], context: undefined, } - return this.tools[tool.name]?.execute?.(toolCallArguments, opts) + return execute(toolCallArguments, opts) }, } }) } + private prepareCall(options: LanguageModelV4CallOptions) { + const { tools, warnings } = prepareToolChoice( + options.tools, + options.toolChoice + ) + const messages = this.prepareMessages(options.prompt, warnings) + const generationOptions = this.prepareGenerationOptions(options, warnings) + const preparedTools = this.prepareTools(tools, warnings) + + return { + messages, + generationOptions, + preparedTools, + nativeTools: preparedTools.map((tool) => tool.native), + ownedToolIds: new Set(preparedTools.map((tool) => tool.native.id)), + agentConfig: this.prepareAgentConfig(options), + warnings, + } + } + private registerTools( preparedTools: ReturnType ) { @@ -508,52 +859,105 @@ class AdkChatLanguageModel implements LanguageModelV4 { } async doGenerate(options: LanguageModelV4CallOptions) { - const messages = this.prepareMessages(options.prompt) - const generationOptions = this.prepareGenerationOptions(options) - const preparedTools = this.prepareTools(options.tools) - const nativeTools = preparedTools.map((tool) => tool.native) - const ownedToolIds = new Set(preparedTools.map((tool) => tool.native.id)) + const { + messages, + generationOptions, + preparedTools, + nativeTools, + ownedToolIds, + agentConfig, + warnings, + } = this.prepareCall(options) const runId = generateId() + const native = getNativeAdkEngine() + const abortSignal = options.abortSignal + + if (abortSignal?.aborted) { + throw getAbortReason(abortSignal) + } + + const onAbort = () => { + void native + .cancelStream(runId) + .catch((error) => console.warn('[adk] cancel failed:', error)) + } this.registerTools(preparedTools) const toolListener = this.createToolCallListener({ runId, ownedToolIds }) + abortSignal?.addEventListener('abort', onAbort, { once: true }) try { await this.ensureNanoPrepared() - const response = await getNativeAdkEngine().generateText( + const response = await native.generateText( runId, messages, - this.agentConfig, + agentConfig, generationOptions, nativeTools ) + const content: LanguageModelV4Content[] = [] + if (response.reasoning) { + content.push({ type: 'reasoning', text: response.reasoning }) + } + for (const toolCall of response.toolCalls ?? []) { + content.push({ + type: 'tool-call', + toolCallId: toolCall.toolCallId, + toolName: toolCall.toolName, + input: toolCall.input, + providerExecuted: true, + }) + } + for (const toolResult of response.toolResults ?? []) { + content.push({ + type: 'tool-result', + toolCallId: toolResult.toolCallId, + toolName: toolResult.toolName, + result: parseToolResult(toolResult.output), + }) + } + content.push({ + type: 'text', + text: + options.responseFormat?.type === 'json' + ? stripJsonFences(response.content) + : response.content, + }) + return { - content: [{ type: 'text' as const, text: response.content }], + content, finishReason: convertFinishReason(response.finishReason), usage: mapUsage(response.usage), - warnings: getCallWarnings(options), + warnings, + } + } catch (error) { + if (abortSignal?.aborted) { + throw getAbortReason(abortSignal) } + throw toAdkError(error) } finally { + abortSignal?.removeEventListener('abort', onAbort) toolListener.remove() this.unregisterTools(preparedTools) } } async doStream(options: LanguageModelV4CallOptions) { - const messages = this.prepareMessages(options.prompt) - const generationOptions = this.prepareGenerationOptions(options) - const preparedTools = this.prepareTools(options.tools) - const nativeTools = preparedTools.map((tool) => tool.native) - const agentConfig = this.agentConfig + const { + messages, + generationOptions, + preparedTools, + nativeTools, + ownedToolIds, + agentConfig, + warnings, + } = this.prepareCall(options) const streamId = generateId() - const ownedToolIds = new Set(preparedTools.map((tool) => tool.native.id)) - const warnings = getCallWarnings(options) - - if (options.responseFormat?.type === 'json') { - throw new Error('Streaming JSON responses is not yet supported by ADK.') - } + const native = getNativeAdkEngine() + const abortSignal = options.abortSignal + const isJson = options.responseFormat?.type === 'json' if (typeof ReadableStream === 'undefined') { throw new Error( @@ -561,11 +965,18 @@ class AdkChatLanguageModel implements LanguageModelV4 { ) } + if (abortSignal?.aborted) { + throw getAbortReason(abortSignal) + } + this.registerTools(preparedTools) let listeners: { remove(): void }[] = [] let cleanedUp = false const toolListener = this.createToolCallListener({ streamId, ownedToolIds }) + const toolNames = new Map() + + let onAbort = () => {} const cleanup = () => { if (cleanedUp) { @@ -576,37 +987,112 @@ class AdkChatLanguageModel implements LanguageModelV4 { listener.remove() }) listeners = [] + abortSignal?.removeEventListener('abort', onAbort) toolListener.remove() this.unregisterTools(preparedTools) } + const cancelNative = () => { + void native + .cancelStream(streamId) + .catch((error) => console.warn('[adk] cancelStream failed:', error)) + } + const stream = new ReadableStream({ start: async (controller) => { - controller.enqueue({ type: 'stream-start', warnings }) + const textId = `${streamId}-text` + let textOpen = false + let reasoningId: string | undefined + let reasoningCount = 0 + // JSON mode keeps the full text so code fences can be stripped + let rawText = '' + let emittedText = '' const addListener = (listener: { remove(): void }) => { listeners.push(listener) } + const closeReasoning = () => { + if (reasoningId) { + controller.enqueue({ type: 'reasoning-end', id: reasoningId }) + reasoningId = undefined + } + } + + const emitText = (delta: string) => { + if (!delta) { + return + } + closeReasoning() + if (!textOpen) { + controller.enqueue({ type: 'text-start', id: textId }) + textOpen = true + } + controller.enqueue({ type: 'text-delta', id: textId, delta }) + } + + const emitJsonText = (final: boolean) => { + const stripped = final + ? stripJsonFences(rawText) + : // Hold back trailing backticks that could start a closing fence + rawText.replace(jsonFencePrefix, '').replace(/\s*`{1,3}$/, '') + // Only emit once the opening fence can no longer change the prefix + if (!final && /^\s*`{0,3}(j(s(o(n)?)?)?)?$/.test(rawText)) { + return + } + if (stripped.startsWith(emittedText)) { + emitText(stripped.slice(emittedText.length)) + emittedText = stripped + } + } + + controller.enqueue({ type: 'stream-start', warnings }) + + onAbort = () => { + cleanup() + cancelNative() + controller.error(abortSignal && getAbortReason(abortSignal)) + } + abortSignal?.addEventListener('abort', onAbort, { once: true }) + addListener( - getNativeAdkEngine().onStreamUpdate((data) => { - if (data.streamId === streamId && data.delta) { + native.onStreamUpdate((data) => { + if (data.streamId !== streamId || !data.delta) { + return + } + + if (data.kind === 'reasoning') { + if (!reasoningId) { + reasoningId = `${streamId}-reasoning-${reasoningCount++}` + controller.enqueue({ type: 'reasoning-start', id: reasoningId }) + } controller.enqueue({ - type: 'text-delta', + type: 'reasoning-delta', + id: reasoningId, delta: data.delta, - id: streamId, }) + return } + + if (isJson) { + rawText += data.delta + emitJsonText(false) + return + } + + emitText(data.delta) }) ) addListener( - getNativeAdkEngine().onStreamToolCall((data: StreamToolCallEvent) => { + native.onStreamToolCall((data: StreamToolCallEvent) => { if (data.streamId !== streamId) { return } if (data.phase === 'start' && data.toolName) { + closeReasoning() + toolNames.set(data.toolCallId, data.toolName) controller.enqueue({ type: 'tool-input-start', id: data.toolCallId, @@ -637,34 +1123,49 @@ class AdkChatLanguageModel implements LanguageModelV4 { input: data.input, providerExecuted: true, }) + return + } + + if (data.phase === 'result' && data.output !== undefined) { + controller.enqueue({ + type: 'tool-result', + toolCallId: data.toolCallId, + toolName: + data.toolName ?? toolNames.get(data.toolCallId) ?? 'unknown', + result: parseToolResult(data.output), + }) } }) ) addListener( - getNativeAdkEngine().onStreamComplete((data) => { - if (data.streamId === streamId) { - controller.enqueue({ - type: 'text-end', - id: streamId, - }) - controller.enqueue({ - type: 'finish', - finishReason: convertFinishReason(data.finishReason), - usage: mapUsage(data.usage), - }) - cleanup() - controller.close() + native.onStreamComplete((data) => { + if (data.streamId !== streamId) { + return + } + if (isJson) { + emitJsonText(true) + } + closeReasoning() + if (textOpen) { + controller.enqueue({ type: 'text-end', id: textId }) } + controller.enqueue({ + type: 'finish', + finishReason: convertFinishReason(data.finishReason), + usage: mapUsage(data.usage), + }) + cleanup() + controller.close() }) ) addListener( - getNativeAdkEngine().onStreamError((data) => { + native.onStreamError((data) => { if (data.streamId === streamId) { controller.enqueue({ type: 'error', - error: new Error(data.error), + error: toAdkError(data.error, data.code), }) cleanup() controller.close() @@ -675,12 +1176,7 @@ class AdkChatLanguageModel implements LanguageModelV4 { try { await this.ensureNanoPrepared() - controller.enqueue({ - type: 'text-start', - id: streamId, - }) - - await getNativeAdkEngine().streamText( + await native.streamText( streamId, messages, agentConfig, @@ -689,15 +1185,12 @@ class AdkChatLanguageModel implements LanguageModelV4 { ) } catch (error) { cleanup() - const message = error instanceof Error ? error.message : String(error) - controller.error(new Error(`ADK stream failed: ${message}`)) + controller.error(toAdkError(error)) } }, cancel: () => { cleanup() - void getNativeAdkEngine() - .cancelStream(streamId) - .catch((error) => console.warn('[adk] cancelStream failed:', error)) + cancelNative() }, }) diff --git a/packages/adk/src/errors.ts b/packages/adk/src/errors.ts new file mode 100644 index 000000000..7b76f30d7 --- /dev/null +++ b/packages/adk/src/errors.ts @@ -0,0 +1,49 @@ +export type AdkError = Error & { + code: AdkErrorCode +} + +export const AdkErrorCodes = { + UnsupportedPlatform: 'UNSUPPORTED_PLATFORM', + ModelUnavailable: 'MODEL_UNAVAILABLE', + ModelDownloading: 'MODEL_DOWNLOADING', + DownloadFailed: 'DOWNLOAD_FAILED', + GenerationError: 'GENERATION_ERROR', + InvalidMessage: 'INVALID_MESSAGE', + InvalidSchema: 'INVALID_SCHEMA', + ToolCallError: 'TOOL_CALL_ERROR', + ContextWindowExceeded: 'CONTEXT_WINDOW_EXCEEDED', + RateLimited: 'RATE_LIMITED', + BackgroundUseBlocked: 'BACKGROUND_USE_BLOCKED', +} as const + +export type AdkErrorCode = (typeof AdkErrorCodes)[keyof typeof AdkErrorCodes] + +const adkErrorCodes = new Set(Object.values(AdkErrorCodes)) + +export function isAdkErrorCode(value: unknown): value is AdkErrorCode { + return typeof value === 'string' && adkErrorCodes.has(value) +} + +export function createAdkError(message: string, code: AdkErrorCode): AdkError { + const error = new Error(message) as AdkError + error.code = code + return error +} + +/** + * Wraps a native rejection or stream error so it carries an `AdkErrorCode`. + * Errors without a known code become `GENERATION_ERROR`. + */ +export function toAdkError(error: unknown, code?: string): AdkError { + const nativeCode = + code ?? + (typeof error === 'object' && error !== null && 'code' in error + ? String((error as { code: unknown }).code) + : undefined) + const message = error instanceof Error ? error.message : String(error) + + return createAdkError( + message, + isAdkErrorCode(nativeCode) ? nativeCode : AdkErrorCodes.GenerationError + ) +} diff --git a/packages/adk/src/index.ts b/packages/adk/src/index.ts index 405983830..64353ce0b 100644 --- a/packages/adk/src/index.ts +++ b/packages/adk/src/index.ts @@ -1,14 +1,24 @@ -export { - adk, - type AdkProvider, - type AdkProviderOptions, - createAdkProvider, +export type { + AdkDownloadProgress, + AdkLanguageModel, + AdkLanguageModelId, + AdkLanguageModelOptions, + AdkModelAvailability, + AdkModelInfo, + AdkPrepareOptions, + AdkProvider, + AdkProviderOptions, + AdkToolDefinitionSet, } from './ai-sdk' +export { adk, createAdkProvider } from './ai-sdk' +export type { AdkError, AdkErrorCode } from './errors' +export { AdkErrorCodes } from './errors' export { type AdkAgentConfig, default as AdkEngine, type AdkMessage, type AdkModelType, + type AdkNanoModelConfig, type AdkTool, getNativeAdkEngine, } from './NativeAdkEngine' From 68a284451c21cddad6c1170fc1a3a87d14d25304 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sun, 25 Oct 2026 03:25:51 +0400 Subject: [PATCH 04/18] feat(expo-example): show gemini nano download progress and image input Co-Authored-By: Claude Opus 5.5 --- .../components/adapters/adkSetupAdapter.ts | 21 ++++++++++++--- apps/expo-example/src/store/providerStore.ts | 27 +++++++++---------- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/apps/expo-example/src/components/adapters/adkSetupAdapter.ts b/apps/expo-example/src/components/adapters/adkSetupAdapter.ts index 408523f07..815e5e9e4 100644 --- a/apps/expo-example/src/components/adapters/adkSetupAdapter.ts +++ b/apps/expo-example/src/components/adapters/adkSetupAdapter.ts @@ -38,19 +38,32 @@ export const createAdkLanguageSetupAdapter = ( icon: options.icon, }, builtIn: options.builtIn, + imageInput: true, isAvailable(): Availability { if (options.modelType === 'gemini' && !options.apiKey) { return 'no' } return 'yes' }, - async download() {}, + async download(onProgress) { + let bytesToDownload: number | undefined + await provider.prepareNano({ + onProgress: (progress) => { + bytesToDownload = progress.bytesToDownload ?? bytesToDownload + if (progress.phase === 'completed') { + onProgress(100) + } else if (bytesToDownload && progress.bytesDownloaded) { + onProgress( + Math.round((progress.bytesDownloaded / bytesToDownload) * 100) + ) + } + }, + }) + }, async delete() {}, async unload() {}, async prepare() { - if ('prepare' in model && typeof model.prepare === 'function') { - await model.prepare() - } + await model.prepare() }, } } diff --git a/apps/expo-example/src/store/providerStore.ts b/apps/expo-example/src/store/providerStore.ts index 09d18497d..35b869e92 100644 --- a/apps/expo-example/src/store/providerStore.ts +++ b/apps/expo-example/src/store/providerStore.ts @@ -2,7 +2,6 @@ import { adk } from '@react-native-ai/adk' import { generateId } from 'ai' import { atom, useAtomValue, useSetAtom } from 'jotai' import { atomWithRefresh, loadable } from 'jotai/utils' -import { Platform } from 'react-native' import { createLlamaLanguageSetupAdapter } from '../components/adapters/llamaModelSetupAdapter' import { languageAdapters } from '../config/providers' @@ -43,21 +42,21 @@ const availabilityAtom = atomWithRefresh(async (get) => { availability.set(adapter.modelId, adapter.isAvailable()) } - if (Platform.OS === 'android') { - const supported = await adk.isNanoSupported() - if (!supported) { + const nano = await adk.getModelInfo() + switch (nano.availability) { + case 'available': + availability.set('adk-gemini-nano', 'yes') + break + case 'downloadable': + availability.set('adk-gemini-nano', 'availableForDownload') + break + case 'downloading': + availability.set('adk-gemini-nano', 'no') + nanoUnavailableReason = 'not-ready' + break + default: availability.set('adk-gemini-nano', 'no') nanoUnavailableReason = 'unsupported' - } else { - const ready = await adk.isAvailable('genai-nano') - availability.set('adk-gemini-nano', ready ? 'yes' : 'no') - if (!ready) { - nanoUnavailableReason = 'not-ready' - } - } - } else { - availability.set('adk-gemini-nano', 'no') - nanoUnavailableReason = 'unsupported' } return { availability, nanoUnavailableReason } satisfies AvailabilityState From a2f80eda1fa44462ae9a79d5c2e413d23f27543f Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sun, 25 Oct 2026 03:25:54 +0400 Subject: [PATCH 05/18] docs(adk): document model info, errors, reasoning and structured output Co-Authored-By: Claude Opus 5.5 --- packages/adk/README.md | 8 +- website/src/docs/adk/generating.md | 129 ++++++++++++----- website/src/docs/adk/getting-started.mdx | 173 +++++++++++++++++------ 3 files changed, 230 insertions(+), 80 deletions(-) diff --git a/packages/adk/README.md b/packages/adk/README.md index 6653ed9a1..2ad9b3fd3 100644 --- a/packages/adk/README.md +++ b/packages/adk/README.md @@ -24,14 +24,16 @@ const { text } = await generateText({ ## Features -- ✅ On-device Gemini Nano through ML Kit GenAI +- ✅ On-device Gemini Nano through ML Kit GenAI, with model selection - ✅ Cloud Gemini through ADK `LlmAgent` -- ✅ Device capability and readiness checks for Gemini Nano +- ✅ Availability, download progress and token counting for Gemini Nano - ✅ Streaming - ✅ Tool calling, with the agent loop run natively by ADK +- ✅ Structured outputs +- ✅ Reasoning - ✅ Image prompts -- ✅ JSON output (without schema constraints) - ✅ Token usage +- ✅ Error codes ## Documentation diff --git a/website/src/docs/adk/generating.md b/website/src/docs/adk/generating.md index 71176aa68..3f3ba6384 100644 --- a/website/src/docs/adk/generating.md +++ b/website/src/docs/adk/generating.md @@ -61,11 +61,11 @@ for await (const delta of textStream) { } ``` -The provider emits standard AI SDK stream parts: `text-start`, `text-delta`, `text-end`, and `finish` with token usage. +The provider emits standard AI SDK stream parts: `text-*`, `reasoning-*`, `tool-*`, and `finish` with token usage. Pass an `abortSignal` to `generateText` or `streamText` to cancel the native run. ## Token Usage -ADK reports token usage as `promptTokenCount`, `candidatesTokenCount`, and `totalTokenCount`. The provider maps these to AI SDK `usage` on `generateText` results and on the streaming `finish` part: +ADK reports token usage as `promptTokenCount`, `candidatesTokenCount`, `thoughtsTokenCount`, `cachedContentTokenCount` and `totalTokenCount`. The provider maps these to AI SDK `usage` on `generateText` results and on the streaming `finish` part: ```typescript import { adk } from '@react-native-ai/adk' @@ -77,10 +77,50 @@ const { usage } = await generateText({ }) console.log(usage.inputTokens) // promptTokenCount -console.log(usage.outputTokens) // candidatesTokenCount +console.log(usage.inputTokenDetails.cacheReadTokens) // cachedContentTokenCount +console.log(usage.outputTokens) // candidatesTokenCount + thoughtsTokenCount +console.log(usage.outputTokenDetails.reasoningTokens) // thoughtsTokenCount console.log(usage.raw?.totalTokenCount) ``` +## Reasoning + +Set the AI SDK `reasoning` option to turn on the model's thinking. The thoughts come back as reasoning parts: + +```typescript +import { createAdkProvider } from '@react-native-ai/adk' +import { generateText } from 'ai' + +const adk = createAdkProvider({ modelName: 'gemini-2.5-flash', apiKey }) + +const { reasoningText, text } = await generateText({ + model: adk(), + reasoning: 'medium', + prompt: 'How many weekdays are there in March 2027?', +}) +``` + +| `reasoning` | Cloud Gemini | Gemini Nano | +| --------------------------- | -------------------------------- | ----------- | +| `provider-default` or unset | The model's default | Off | +| `none` | Thinking budget `0` | Off | +| `minimal`, `low`, `medium` | Budget of 512, 2048, 8192 tokens | On | +| `high`, `xhigh`, `max` | Budget of 24576 tokens | On | + +Gemini Nano only supports thinking on or off, and only on devices whose AICore and model support it. Elsewhere the request still succeeds without thoughts. Some cloud models cannot turn thinking off, so `none` may fail for them. + +To set an exact budget, or to hide the thoughts while the model still thinks, use `providerOptions.adk`: + +```typescript +await generateText({ + model: adk(), + prompt, + providerOptions: { + adk: { thinkingBudget: 4096, includeThoughts: false }, + }, +}) +``` + ## Tool Calling ADK agents can call JavaScript tools during generation, on both Gemini Nano and cloud Gemini. @@ -89,8 +129,8 @@ ADK agents can call JavaScript tools during generation, on both Gemini Nano and ADK runs tools natively, which differs from other providers: -- **Register executors on the provider** - Pass tools to `createAdkProvider` via `availableTools` so ADK can call their `execute` handlers. Tools passed only to `generateText` or `streamText` are described to the model but never executed -- **Provider-executed** - Streamed tool calls are marked with `providerExecuted: true` +- **Register executors on the provider** - Pass tools to `createAdkProvider` via `availableTools` so ADK can call their `execute` handlers. A tool passed only to `generateText` or `streamText` is described to the model, but calling it returns an error to the model and the call result has a warning +- **Provider-executed** - Tool calls and their results are part of the result content, and tool calls are marked with `providerExecuted: true` - **Native agent loop** - ADK runs the multi-step tool loop, so AI SDK `stopWhen` does not limit ADK's internal steps - **Gemini Nano** - Gemini Nano has no native function calling. The provider lists the tools in the system instruction and converts the model's `tool_code` blocks into ADK function calls @@ -118,18 +158,34 @@ const adk = createAdkProvider({ availableTools: { getCurrentTime }, }) -const { text } = await generateText({ +const { text, toolCalls, toolResults } = await generateText({ model: adk(), tools: { getCurrentTime }, prompt: 'What time is it in Warsaw?', }) ``` -During streaming, the provider emits `tool-input-start`, `tool-input-delta`, `tool-input-end`, and `tool-call` stream parts when ADK surfaces function calls from the model. +To change the executors later, call `updateTools` on the model: + +```typescript +const model = adk() +model.updateTools({ getCurrentTime }) +``` + +During streaming, the provider emits `tool-input-start`, `tool-input-delta`, `tool-input-end`, `tool-call` and `tool-result` stream parts as ADK runs the tools. + +### Tool Choice + +| `toolChoice` | Behavior | +| ----------------------- | ------------------------------------------------------------- | +| `auto` (default) | The model decides | +| `none` | No tools are sent to the model | +| `required` | Sent as `auto`, with a warning - ADK cannot force a tool call | +| `{ type: 'tool', ... }` | Only that tool is sent, as `auto`, with a warning | ## Image Prompts -Pass image file parts in user messages using the standard AI SDK prompt format: +Pass image file parts in user messages using the standard AI SDK prompt format. You can send several images in one message: ```typescript import { adk } from '@react-native-ai/adk' @@ -157,36 +213,48 @@ Supported file data formats: - **Base64 strings** - Raw base64 or `data:image/jpeg;base64,...` data URLs - **`Uint8Array`** - Binary image data +- **`file://` and `content://` URLs** - Read natively, for example from an image picker +- **HTTP URLs** - Downloaded by the AI SDK before the call -> **Note**: File URLs (`file://` or HTTP) are not supported yet. Pass base64 or `Uint8Array` data directly. +## Structured Output -## JSON Output - -Generate JSON with `Output.json()`: +Generate objects that match a schema with `Output.object()`: ```typescript import { adk } from '@react-native-ai/adk' import { generateText, Output } from 'ai' +import { z } from 'zod' const { output } = await generateText({ model: adk(), - output: Output.json(), - prompt: 'Return a JSON object with name and age fields.', + output: Output.object({ + schema: z.object({ name: z.string(), age: z.number() }), + }), + prompt: 'Invent a character.', }) ``` -> **Note**: Schema constraints are not supported yet, so `Output.object()` and other schema-based outputs throw. Use `Output.json()` and validate the result in your app. JSON output is not supported with `streamText`. +Structured output also works with `streamText` and `Output.json()`. + +- **Cloud Gemini** - The schema is converted to a Gemini response schema, so the model can only produce matching JSON. Keywords that Gemini does not support, such as `$ref`, are dropped +- **Gemini Nano** - ML Kit has no runtime schema API, so the provider puts the schema in the system instruction and strips Markdown code fences from the reply. The AI SDK validates the result, so check for `NoObjectGeneratedError` ## Available Options Configure model behavior with generation options: -| Option | Type | Description | -| ----------------- | ------ | -------------------------- | -| `temperature` | number | Controls randomness | -| `maxOutputTokens` | number | Maximum tokens to generate | -| `topP` | number | Nucleus sampling threshold | -| `topK` | number | Top-K sampling parameter | +| Option | Type | Description | Gemini Nano | +| ------------------ | -------- | ------------------------------ | ----------- | +| `temperature` | number | Controls randomness | Yes | +| `maxOutputTokens` | number | Maximum tokens to generate | Yes | +| `topK` | number | Top-K sampling parameter | Yes | +| `seed` | number | Seed for repeatable sampling | Yes | +| `topP` | number | Nucleus sampling threshold | No | +| `stopSequences` | string[] | Stop generating at these texts | No | +| `presencePenalty` | number | Penalize repeated topics | No | +| `frequencyPenalty` | number | Penalize repeated tokens | No | + +Settings that Gemini Nano does not support are ignored with an AI SDK warning. Example: @@ -199,20 +267,17 @@ const { text } = await generateText({ prompt: 'Write a creative story', temperature: 0.8, maxOutputTokens: 500, - topP: 0.9, topK: 40, + seed: 42, }) ``` ## Limitations -The following features are not supported yet: - -| Feature | Status | -| ------------------------------------------ | ------------------------------------------- | -| Schema-constrained output | Throws - use `Output.json()` without schema | -| JSON output with `streamText` | Throws | -| `toolChoice` `required` or a specific tool | Ignored with an AI SDK warning - uses auto | -| `reasoning` | Ignored with an AI SDK warning | -| File URLs in prompts | Throws - use base64 or `Uint8Array` | -| iOS / web | Android only | +| Feature | Status | +| --------------------------- | ----------------------------------------------------------- | +| Native tool calling on Nano | Not in ML Kit - the provider parses `tool_code` blocks | +| Forcing a tool call | `required` and specific tools run as `auto`, with a warning | +| Schema enforcement on Nano | Prompted, then validated by the AI SDK | +| Provider file references | Throws - pass data or a URL | +| iOS / web | Android only | diff --git a/website/src/docs/adk/getting-started.mdx b/website/src/docs/adk/getting-started.mdx index fe79c5844..f820797b3 100644 --- a/website/src/docs/adk/getting-started.mdx +++ b/website/src/docs/adk/getting-started.mdx @@ -75,10 +75,12 @@ android { ## Available Model Types -| Model Type | `modelType` | `modelName` | Use Case | -| --------------------- | ------------ | --------------------------------- | ------------------------------------------------------------------ | -| On-device Gemini Nano | `genai-nano` | `gemini-nano` (default) | Private, offline-capable inference - system-provisioned via AICore | -| Cloud Gemini | `gemini` | Required, e.g. `gemini-2.5-flash` | Cloud inference via the Gemini API | +| Model Type | `modelType` | Default `modelName` | Use Case | +| --------------------- | ------------ | ------------------- | ------------------------------------------------------------------ | +| On-device Gemini Nano | `genai-nano` | `gemini-nano` | Private, offline-capable inference - system-provisioned via AICore | +| Cloud Gemini | `gemini` | `gemini-2.5-flash` | Cloud inference via the Gemini API | + +When you set `modelName` to anything other than `gemini-nano`, `modelType` defaults to `gemini`. ## Basic Usage @@ -94,11 +96,18 @@ const { text } = await generateText({ }) ``` +Pass a model name or options to `adk()` to pick a model per call site, like `apple()`: + +```typescript +const nano = adk() +const flash = adk({ modelName: 'gemini-2.5-flash', apiKey }) +``` + Before you offer Gemini Nano in your UI, check that the device supports it and that it is ready. See [Availability](#availability) below. ## On-device Gemini Nano -Gemini Nano runs locally via ML Kit GenAI and is provisioned by the Android system through AICore. To set a system instruction, create a provider with `createAdkProvider`: +Gemini Nano runs locally via ML Kit GenAI and is provisioned by the Android system through AICore. To set a system instruction, create a provider with `createAdkProvider`. ADK passes it to ML Kit as a native system instruction: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -116,50 +125,64 @@ const { text } = await generateText({ ### Availability -The provider exposes two checks for Gemini Nano: +`adk.getModelInfo()` reports whether Gemini Nano can be used, based on the ML Kit [`FeatureStatus`](https://developers.google.com/android/reference/com/google/mlkit/genai/common/FeatureStatus): -| API | Checks | Resolves to `true` when | -| ------------------------------- | ----------------- | ------------------------------------------------------------ | -| `adk.isNanoSupported()` | Device capability | The device can run Gemini Nano, even if it is not downloaded | -| `adk.isAvailable('genai-nano')` | Runtime readiness | Gemini Nano is downloaded, or can be downloaded now | +```typescript +import { adk } from '@react-native-ai/adk' -`isAvailable()` defaults to `'genai-nano'` and always resolves to `true` for `'gemini'`. When `isNanoSupported()` is `false`, `isAvailable('genai-nano')` is `false` too. +const info = await adk.getModelInfo() + +switch (info.availability) { + case 'available': + // Ready to generate + break + case 'downloadable': + // Offer a download - adk.prepareNano() downloads it + break + case 'downloading': + // Show as unavailable until the download finishes + break + case 'unavailable': + case 'unknown': + // The device cannot run Gemini Nano - hide it from the model picker + break +} +``` -Both checks are cheap native calls that do not download the model. You can cache them at app startup and check again when the app resumes. +| `availability` | `FeatureStatus` | Meaning | +| -------------- | ------------------ | -------------------------------------------- | +| `available` | `AVAILABLE` (3) | Downloaded and ready | +| `downloadable` | `DOWNLOADABLE` (1) | Supported, `prepareNano()` downloads it | +| `downloading` | `DOWNLOADING` (2) | The system is downloading it | +| `unavailable` | `UNAVAILABLE` (0) | Not supported on this device, or not Android | +| `unknown` | - | ML Kit could not report a status | -Both checks read the ML Kit [`FeatureStatus`](https://developers.google.com/android/reference/com/google/mlkit/genai/common/FeatureStatus) returned by `Generation.getClient().checkStatus()`: +When Gemini Nano is `available`, `getModelInfo()` also returns `baseModelName` and `contextSize` (the input token limit). For cloud Gemini, `getModelInfo({ modelType: 'gemini' })` always reports `available` on Android. -| `FeatureStatus` | `isNanoSupported()` | `isAvailable('genai-nano')` | Suggested UX | -| ------------------ | ------------------- | --------------------------- | ------------------------------------------------ | -| `UNAVAILABLE` (0) | `false` | `false` | Hide Gemini Nano | -| `DOWNLOADABLE` (1) | `true` | `true` | Offer Gemini Nano - `prepareNano()` downloads it | -| `DOWNLOADING` (2) | `true` | `false` | Show as unavailable until the download finishes | -| `AVAILABLE` (3) | `true` | `true` | Ready | +The provider also has two yes/no shortcuts: -A typical availability flow: +| API | Resolves to `true` when | +| ------------------------------- | ------------------------------------------------------------ | +| `adk.isNanoSupported()` | The device can run Gemini Nano, even if it is not downloaded | +| `adk.isAvailable('genai-nano')` | Gemini Nano is `available` or `downloadable` | -```typescript -import { adk } from '@react-native-ai/adk' +All of these are cheap native calls that do not download the model. You can cache them at app startup and check again when the app resumes. -async function loadGeminiNano() { - if (!(await adk.isNanoSupported())) { - // The device cannot run Gemini Nano - hide it from the model picker - return null - } +### Preparation - if (!(await adk.isAvailable('genai-nano'))) { - // Gemini Nano is still downloading - show it as unavailable - return null - } +`adk.prepareNano()` downloads and initializes Gemini Nano, and can report download progress: - await adk.prepareNano() - return adk() -} -``` +```typescript +import { adk } from '@react-native-ai/adk' -### Preparation +await adk.prepareNano({ + onProgress: ({ phase, bytesDownloaded, bytesToDownload }) => { + // phase is 'started', 'progress' or 'completed' + }, +}) +``` -`adk.prepareNano()` downloads and initializes Gemini Nano. Calling `model.prepare()` on a model does the same thing. `generateText()` and `streamText()` prepare the model automatically on first use: +Calling `model.prepare()` on a model prepares it too, and `generateText()` and `streamText()` prepare the model automatically on first use: ```typescript import { adk } from '@react-native-ai/adk' @@ -174,11 +197,41 @@ await model.prepare() await generateText({ model, prompt: 'Hello!' }) ``` -Automatic preparation only checks `isNanoSupported()`. Use `isAvailable('genai-nano')` for UI gating, and handle preparation and generation errors in your app. +Preparation fails with a `MODEL_UNAVAILABLE` error on unsupported devices and `MODEL_DOWNLOADING` when the system is still downloading. See [Errors](#errors). + +### Model Selection + +ML Kit can run a preview Gemini Nano model or favor speed over quality. Set `nano` on the provider or the model: + +```typescript +import { createAdkProvider } from '@react-native-ai/adk' + +const adk = createAdkProvider({ + nano: { releaseStage: 'preview', preference: 'full' }, +}) +``` + +| Option | Values | Description | +| -------------- | --------------------------------- | ------------------------------------------------------------- | +| `releaseStage` | `'stable'` (default), `'preview'` | Preview models need the AICore developer preview | +| `preference` | `'fast'`, `'full'` | `fast` favors latency, `full` favors quality and capabilities | + +Availability depends on the selected model, so `getModelInfo()`, `isAvailable()` and `prepareNano()` use the provider's `nano` setting. + +### Token Counting + +`model.countTokens(text)` counts tokens with the Gemini Nano tokenizer. Compare it with `contextSize` from `getModelInfo()` before you send a long prompt: + +```typescript +const model = adk() +const tokens = await model.countTokens(longText) +``` + +Cloud models do not support token counting. ## Cloud Gemini -When you need cloud inference - for example on emulators or devices without AICore - create a provider with `modelType: 'gemini'`, a `modelName`, and an API key: +When you need cloud inference - for example on emulators or devices without AICore - create a provider with `modelType: 'gemini'` and an API key: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -197,13 +250,13 @@ const { text } = await generateText({ }) ``` -Cloud models do not need a download or prepare step. +Cloud models do not need a download or prepare step. ADK retries transient Gemini API failures. > Do not embed API keys in production client apps. Prefer a backend proxy or secure runtime configuration. ## Provider Options -`createAdkProvider` returns the same callable interface as the default `adk` export. Call `adk()` or `adk.languageModel()` to construct a language model: +`createAdkProvider` sets defaults for every model it creates. Call `adk()` or `adk.languageModel()` to construct a language model, optionally overriding any option: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -214,7 +267,7 @@ const adk = createAdkProvider({ instruction: 'You are a concise, friendly assistant.', }) -const model = adk.languageModel() +const model = adk.languageModel({ instruction: 'Answer in one word.' }) ``` | Option | Type | Description | @@ -222,11 +275,41 @@ const model = adk.languageModel() | `name` | `string` | ADK agent name (default: `react_native_adk_agent`) | | `description` | `string` | Agent description passed to ADK | | `instruction` | `string` | System instruction for the agent | -| `modelType` | `'genai-nano' \| 'gemini'` | On-device or cloud backend (default: `genai-nano`) | -| `modelName` | `string` | Model identifier (default: `gemini-nano`; required for `gemini`) | +| `modelType` | `'genai-nano' \| 'gemini'` | On-device or cloud backend (default: `genai-nano`, or `gemini` for other model names) | +| `modelName` | `string` | Model identifier (default: `gemini-nano` on device, `gemini-2.5-flash` in the cloud) | | `apiKey` | `string` | Gemini API key - required for `gemini` only | | `availableTools` | `Record` | Tools whose `execute` handlers ADK can call. See [Tool Calling](./generating.md#tool-calling) | +| `nano` | `AdkNanoModelConfig` | Gemini Nano model selection. See [Model Selection](#model-selection) | + +## Errors + +Errors thrown by the provider have a `code` from `AdkErrorCodes`: + +```typescript +import { AdkErrorCodes, type AdkError } from '@react-native-ai/adk' + +try { + await adk.prepareNano() +} catch (error) { + if ((error as AdkError).code === AdkErrorCodes.ModelDownloading) { + // Try again later + } +} +``` + +| Code | Meaning | +| ------------------------- | ----------------------------------------------------------------------- | +| `UNSUPPORTED_PLATFORM` | Called on iOS or web | +| `MODEL_UNAVAILABLE` | Gemini Nano is not supported, or AICore needs an update | +| `MODEL_DOWNLOADING` | The system is still downloading Gemini Nano | +| `DOWNLOAD_FAILED` | The Gemini Nano download failed | +| `INVALID_MESSAGE` | The prompt has an unsupported part, such as a remote file URL | +| `INVALID_SCHEMA` | The response schema cannot be converted for Gemini | +| `CONTEXT_WINDOW_EXCEEDED` | The prompt is too long for the model | +| `RATE_LIMITED` | AICore is busy, the app's quota ran out, or the Gemini API returned 429 | +| `BACKGROUND_USE_BLOCKED` | Gemini Nano was called while the app was in the background | +| `GENERATION_ERROR` | Any other generation failure | ## Next Steps -See **[Generating](./generating.md)** for text generation, streaming, tool calling, image prompts, and JSON output. +See **[Generating](./generating.md)** for text generation, streaming, tool calling, image prompts, reasoning and structured output. From f2c1363b05a4afb915087bfb8d79fc2473cd15b1 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:42:50 +0400 Subject: [PATCH 06/18] fix(adk): pass system messages as the agent instruction ADK replayed system messages as history turns, which sends a system role in the Gemini request contents and a [system] turn to Gemini Nano. Builds on #268. Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 3 +- packages/adk/src/NativeAdkEngine.ts | 2 +- packages/adk/src/__tests__/ai-sdk.test.ts | 21 ++++++++++ packages/adk/src/ai-sdk.ts | 42 +++++++++---------- 4 files changed, 42 insertions(+), 26 deletions(-) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 75d4ec2dc..41a87d6e9 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -425,8 +425,7 @@ class AdkAgentRunner( val message = messages.getMap(index) ?: continue val role = when (message.getString("role")) { "assistant" -> Role.MODEL - "system" -> Role.SYSTEM - "user", "tool" -> Role.USER + "user" -> Role.USER else -> throw IllegalArgumentException( "Unsupported message role: ${message.getString("role")}", ) diff --git a/packages/adk/src/NativeAdkEngine.ts b/packages/adk/src/NativeAdkEngine.ts index 9c5fffc5a..4cb1b15bf 100644 --- a/packages/adk/src/NativeAdkEngine.ts +++ b/packages/adk/src/NativeAdkEngine.ts @@ -1,7 +1,7 @@ import type { CodegenTypes, TurboModule } from 'react-native' import { Platform, TurboModuleRegistry } from 'react-native' -export type AdkMessageRole = 'assistant' | 'system' | 'user' +export type AdkMessageRole = 'assistant' | 'user' export type AdkMessagePart = | { type: 'text'; text: string } diff --git a/packages/adk/src/__tests__/ai-sdk.test.ts b/packages/adk/src/__tests__/ai-sdk.test.ts index 654eb6a6a..4cb2316d5 100644 --- a/packages/adk/src/__tests__/ai-sdk.test.ts +++ b/packages/adk/src/__tests__/ai-sdk.test.ts @@ -164,6 +164,27 @@ describe('model resolution', () => { }) describe('prompt conversion', () => { + test('moves system messages into the agent instruction', async () => { + const model = createAdkProvider({ + modelType: 'gemini', + instruction: 'Be brief.', + })() + await model.doGenerate( + createCallOptions({ + prompt: [ + { role: 'system', content: 'Answer in Polish.' }, + { role: 'user', content: [{ type: 'text', text: 'hello' }] }, + ], + }) + ) + + const [, messages, config] = lastGenerateCall() + expect(messages).toEqual([{ role: 'user', content: 'hello' }]) + expect(config).toMatchObject({ + instruction: 'Be brief.\n\nAnswer in Polish.', + }) + }) + test('resolves top-level file media types to a full mime type', async () => { await gemini.doGenerate( createCallOptions({ diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index e0ea49dc1..8c8717ff0 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -557,12 +557,10 @@ class AdkChatLanguageModel implements AdkLanguageModel { messages: LanguageModelV4Prompt, warnings: SharedV4Warning[] ): AdkMessage[] { - return messages.map((message): AdkMessage => { + // System messages become part of the agent instruction instead + return messages.flatMap((message): AdkMessage[] => { if (message.role === 'system') { - return { - role: 'system', - content: message.content, - } + return [] } if (message.role === 'user') { @@ -574,16 +572,10 @@ class AdkChatLanguageModel implements AdkLanguageModel { }) if (parts.length === 1 && parts[0]?.type === 'text') { - return { - role: 'user', - content: parts[0].text, - } + return [{ role: 'user', content: parts[0].text }] } - return { - role: 'user', - parts, - } + return [{ role: 'user', parts }] } if (message.role === 'assistant') { @@ -616,10 +608,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { }) .join('\n') - return { - role: 'assistant', - content, - } + return [{ role: 'assistant', content }] } if (message.role === 'tool') { @@ -633,10 +622,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { }) .join('\n') - return { - role: 'user', - content, - } + return [{ role: 'user', content }] } throw createAdkError( @@ -649,14 +635,24 @@ class AdkChatLanguageModel implements AdkLanguageModel { private prepareAgentConfig( options: LanguageModelV4CallOptions ): AdkAgentConfig { + // ADK only takes a system prompt as the agent instruction + const systemPrompts = options.prompt.flatMap((message) => + message.role === 'system' ? [message.content] : [] + ) // Gemini Nano has no response schema, so the schema goes in the instruction - if (!this.isNano || options.responseFormat?.type !== 'json') { + const jsonInstruction = + this.isNano && options.responseFormat?.type === 'json' + ? createNanoJsonInstruction(options.responseFormat.schema) + : undefined + + if (systemPrompts.length === 0 && !jsonInstruction) { return this.agentConfig } const instruction = [ this.agentConfig.instruction, - createNanoJsonInstruction(options.responseFormat.schema), + ...systemPrompts, + jsonInstruction, ] .filter(Boolean) .join('\n\n') From 84967ade458b96a1f44ef5d46ee71d76dd09b6ed Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:49:22 +0400 Subject: [PATCH 07/18] fix(adk): report streamed tool calls without arguments The stream only ended a tool call once it had arguments, so a call to a tool without parameters had no tool-call part before its result. Co-Authored-By: Claude Opus 5.5 --- .../java/com/callstack/ai/adk/AdkStreamToolCalls.kt | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt index 705ad4543..d5623ce59 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkStreamToolCalls.kt @@ -60,8 +60,14 @@ class AdkStreamToolCallEmitter( } } - if (functionCall.args.isNotEmpty() && !state.completed) { - val input = AdkJson.encode(functionCall.args) + // A complete event carries the final arguments, which are empty for tools without parameters + if ((functionCall.args.isNotEmpty() || !event.partial) && state.started && !state.completed) { + val input = + if (functionCall.args.isEmpty() && state.inputBuilder.isNotEmpty()) { + state.inputBuilder.toString() + } else { + AdkJson.encode(functionCall.args) + } if (state.inputBuilder.isEmpty()) { emit( Arguments.createMap().apply { From c4e214170d6576e569d158d0e514e3568f561a0d Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:49:28 +0400 Subject: [PATCH 08/18] fix(adk): route tool calls by run and fix tool, usage and seed handling - run tools with ADK's function call id, so execute sees the reported id - route tool calls by run id and tool name instead of a global registry - always send tool results as JSON, so strings stay strings natively - describe each Gemini Nano tool once in the instruction - drop the flat parameters list and fail on unsupported input schemas - sum token usage over every model call in a tool loop - warn that Gemini Nano ignores seed, and fix the docs table - track a run before starting it, so a run that ends at once is removed Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 94 +++----- .../main/java/com/callstack/ai/adk/AdkJson.kt | 15 +- .../com/callstack/ai/adk/AdkToolBridge.kt | 52 ++--- .../java/com/callstack/ai/adk/AdkUsage.kt | 27 ++- .../callstack/ai/adk/NativeAdkEngineModule.kt | 50 +++-- .../ai/adk/ReactNativeFunctionTool.kt | 96 +++------ packages/adk/src/NativeAdkEngine.ts | 25 +-- packages/adk/src/__tests__/ai-sdk.test.ts | 74 ++++++- packages/adk/src/ai-sdk.ts | 201 ++++++------------ website/src/docs/adk/generating.md | 3 +- 10 files changed, 275 insertions(+), 362 deletions(-) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 41a87d6e9..91215ae87 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -16,7 +16,6 @@ import com.google.adk.kt.models.Model import com.google.adk.kt.runners.InMemoryRunner import com.google.adk.kt.sessions.InMemorySessionService import com.google.adk.kt.sessions.SessionKey -import com.google.adk.kt.tools.BaseTool import com.google.adk.kt.types.Blob import com.google.adk.kt.types.Content import com.google.adk.kt.types.GenerateContentConfig @@ -84,10 +83,10 @@ fun Event.textParts(thoughts: Boolean): String = class AdkAgentRunner( private val context: Context, private val onToolCall: ( + runId: String, toolCallId: String, - toolId: String, + toolName: String, arguments: String, - scope: ToolCallScope?, ) -> Unit, private val onDownloadProgress: ( phase: String, @@ -269,7 +268,7 @@ class AdkAgentRunner( toolCalls = toolCalls, toolResults = toolResults, finishReason = events.lastOrNull()?.finishReason?.name, - usage = AdkUsage.latestFromEvents(events), + usage = AdkUsage.fromEvents(events), ) } @@ -278,9 +277,9 @@ class AdkAgentRunner( config: ReadableMap, options: ReadableMap?, tools: ReadableArray?, - streamId: String, + runId: String, ): Flow { - return runAgent(messages, config, options, tools, stream = true, streamId = streamId) + return runAgent(messages, config, options, tools, stream = true, runId = runId) } private suspend fun runAgent( @@ -289,17 +288,11 @@ class AdkAgentRunner( options: ReadableMap?, tools: ReadableArray?, stream: Boolean, - streamId: String? = null, - runId: String? = null, + runId: String, ): Flow { val parsedMessages = parseMessages(messages) val agentConfig = parseAgentConfig(config) - val toolCallScope = when { - stream && streamId != null -> ToolCallScope(streamId = streamId) - !stream && runId != null -> ToolCallScope(runId = runId) - else -> null - } - val agentTools = parseTools(tools, toolCallScope, agentConfig.modelType) + val agentTools = parseTools(tools, runId) val model = createModel(agentConfig) val generateContentConfig = parseGenerationConfig(options) val instructionText = @@ -470,48 +463,32 @@ class AdkAgentRunner( context.contentResolver.openInputStream(Uri.parse(uri))?.use { it.readBytes() } ?: throw IllegalArgumentException("Cannot read file: $uri") - private fun parseTools( - tools: ReadableArray?, - toolCallScope: ToolCallScope?, - modelType: String, - ): List { + private fun parseTools(tools: ReadableArray?, runId: String): List { if (tools == null) return emptyList() - val injectPromptDescription = modelType == "genai-nano" - val parsed = mutableListOf() + val parsed = mutableListOf() for (index in 0 until tools.size()) { val tool = tools.getMap(index) ?: continue - val parametersArray = tool.getArray("parameters") - val parameters = mutableListOf() - - if (parametersArray != null) { - for (paramIndex in 0 until parametersArray.size()) { - val parameter = parametersArray.getMap(paramIndex) ?: continue - parameters.add( - ToolParameterSpec( - name = parameter.getString("name") ?: continue, - description = parameter.getString("description"), - type = parseParameterType(parameter.getString("type")), - required = parameter.hasKey("required") && parameter.getBoolean("required"), - ) - ) - } - } - + val name = tool.getString("name") ?: continue val parameterSchema = tool.getMap("inputSchema")?.let { schemaMap -> - runCatching { AdkSchema.fromReadableMap(schemaMap) }.getOrNull() + try { + AdkSchema.fromReadableMap(schemaMap) + } catch (error: Exception) { + throw AdkErrors.AdkException( + AdkErrors.INVALID_SCHEMA, + "Unsupported input schema for tool $name: ${error.message}", + error, + ) + } } parsed.add( ReactNativeFunctionTool( - toolId = tool.getString("id") ?: UUID.randomUUID().toString(), - name = tool.getString("name") ?: continue, + name = name, description = tool.getString("description") ?: "", parameterSchema = parameterSchema, - parameters = parameters, - injectPromptDescription = injectPromptDescription, - toolCallScope = toolCallScope, + runId = runId, onToolCall = onToolCall, ) ) @@ -520,24 +497,18 @@ class AdkAgentRunner( return parsed } + /** Gemini Nano has no native function calling, so its tools are described in the instruction. */ private fun buildInstruction( baseInstruction: String?, - tools: List, + tools: List, ): String? { if (tools.isEmpty()) { return baseInstruction } - val toolSummary = - tools.joinToString("\n") { tool -> - "- ${tool.name}: ${tool.description}" - } - val toolGuidance = - """ - You have access to the following tools. Use them when they help answer the user: - $toolSummary - """.trimIndent() + "You have access to the following tools. Use them when they help answer the user:\n" + + tools.joinToString("\n") { it.promptDescription() } return if (baseInstruction.isNullOrBlank()) { toolGuidance @@ -546,16 +517,6 @@ class AdkAgentRunner( } } - private fun parseParameterType(type: String?): com.google.adk.kt.types.Type { - return when (type) { - "number", "integer" -> com.google.adk.kt.types.Type.NUMBER - "boolean" -> com.google.adk.kt.types.Type.BOOLEAN - "array" -> com.google.adk.kt.types.Type.ARRAY - "object" -> com.google.adk.kt.types.Type.OBJECT - else -> com.google.adk.kt.types.Type.STRING - } - } - companion object { fun parseAgentConfig(config: ReadableMap): AgentConfig = parseAgentConfigStatic(config) @@ -593,8 +554,3 @@ data class ParsedMessage( val role: ContentRole, val content: Content, ) - -data class ToolCallScope( - val streamId: String? = null, - val runId: String? = null, -) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkJson.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkJson.kt index da43e387e..91cd71b0a 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkJson.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkJson.kt @@ -2,24 +2,15 @@ package com.callstack.ai.adk import org.json.JSONArray import org.json.JSONObject +import org.json.JSONTokener object AdkJson { fun encode(value: Map): String { return mapToJsonObject(value).toString() } - fun decodeResult(value: String): Any? { - val trimmed = value.trim() - return when { - trimmed.startsWith("{") -> jsonObjectToMap(JSONObject(trimmed)) - trimmed.startsWith("[") -> jsonArrayToList(JSONArray(trimmed)) - trimmed == "true" -> true - trimmed == "false" -> false - trimmed == "null" -> null - trimmed.toDoubleOrNull() != null -> trimmed.toDouble() - else -> trimmed.removeSurrounding("\"") - } - } + /** Decodes a tool result, which JS always sends as JSON. */ + fun decodeResult(value: String): Any? = jsonValue(JSONTokener(value).nextValue()) private fun mapToJsonObject(value: Map): JSONObject { val json = JSONObject() diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkToolBridge.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkToolBridge.kt index 7b99145c4..734f0c81a 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkToolBridge.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkToolBridge.kt @@ -3,43 +3,37 @@ package com.callstack.ai.adk import kotlinx.coroutines.CompletableDeferred import java.util.concurrent.ConcurrentHashMap +/** Waits for tool results that JavaScript submits, grouped by run. */ object AdkToolBridge { - private val pendingResults = ConcurrentHashMap>() - private val streamToolCalls = ConcurrentHashMap>() + private val pendingResults = + ConcurrentHashMap>>() - suspend fun awaitResult(toolCallId: String): String { - val deferred = pendingResults.computeIfAbsent(toolCallId) { CompletableDeferred() } - val result = deferred.await() - pendingResults.remove(toolCallId, deferred) - return result - } - - fun registerToolCall(streamId: String, toolCallId: String) { - streamToolCalls.computeIfAbsent(streamId) { ConcurrentHashMap.newKeySet() }.add(toolCallId) + /** Registers the call before [request] asks JS to run it, then waits for the result. */ + suspend fun call(runId: String, toolCallId: String, request: () -> Unit): String { + val calls = pendingResults.computeIfAbsent(runId) { ConcurrentHashMap() } + val deferred = CompletableDeferred() + calls[toolCallId] = deferred + try { + request() + return deferred.await() + } finally { + calls.remove(toolCallId, deferred) + // Drop the run once it has no pending calls + pendingResults.computeIfPresent(runId) { _, runCalls -> runCalls.takeIf { it.isNotEmpty() } } + } } - fun submitResult(toolCallId: String, result: String) { - pendingResults.computeIfAbsent(toolCallId) { CompletableDeferred() }.complete(result) - streamToolCalls.values.forEach { toolCallIds -> toolCallIds.remove(toolCallId) } + fun submitResult(runId: String, toolCallId: String, result: String) { + pendingResults[runId]?.get(toolCallId)?.complete(result) } - fun cancelForStream(streamId: String) { - streamToolCalls.remove(streamId)?.forEach { toolCallId -> - pendingResults.remove(toolCallId)?.let { deferred -> - if (!deferred.isCompleted) { - deferred.complete("{\"error\":\"Tool call cancelled\"}") - } - } - } + fun cancel(runId: String) { + pendingResults.remove(runId)?.values?.forEach { it.complete(CANCELLED_RESULT) } } fun cancelAll() { - pendingResults.values.forEach { deferred -> - if (!deferred.isCompleted) { - deferred.complete("{\"error\":\"Tool call cancelled\"}") - } - } - pendingResults.clear() - streamToolCalls.clear() + pendingResults.keys.toList().forEach(::cancel) } + + private const val CANCELLED_RESULT = "{\"error\":\"Tool call cancelled\"}" } diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt index 480319291..4494e4c01 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkUsage.kt @@ -18,6 +18,29 @@ object AdkUsage { } } - fun latestFromEvents(events: List): UsageMetadata? = - events.mapNotNull { it.usageMetadata }.lastOrNull() + /** + * Usage of a whole run. A tool loop makes one model call per step, and each complete event + * carries the usage of its call, so the run's usage is their sum. + */ + fun fromEvents(events: List): UsageMetadata? = + events.filterNot { it.partial }.fold(null as UsageMetadata?) { total, event -> add(total, event) } + ?: events.mapNotNull { it.usageMetadata }.lastOrNull() + + /** Adds a complete event's usage to [total]. Partial events repeat their call's usage. */ + fun add(total: UsageMetadata?, event: Event): UsageMetadata? { + val usage = event.usageMetadata + if (event.partial || usage == null) return total + if (total == null) return usage + + return UsageMetadata( + promptTokenCount = sum(total.promptTokenCount, usage.promptTokenCount), + candidatesTokenCount = sum(total.candidatesTokenCount, usage.candidatesTokenCount), + totalTokenCount = sum(total.totalTokenCount, usage.totalTokenCount), + thoughtsTokenCount = sum(total.thoughtsTokenCount, usage.thoughtsTokenCount), + toolUsePromptTokenCount = sum(total.toolUsePromptTokenCount, usage.toolUsePromptTokenCount), + cachedContentTokenCount = sum(total.cachedContentTokenCount, usage.cachedContentTokenCount), + ) + } + + private fun sum(a: Int?, b: Int?): Int? = if (a == null && b == null) null else (a ?: 0) + (b ?: 0) } diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt index 416c031f3..53a546deb 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt @@ -8,6 +8,7 @@ import com.facebook.react.bridge.ReadableMap import com.google.adk.kt.events.Event import com.google.adk.kt.types.UsageMetadata import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.CoroutineStart import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Job import kotlinx.coroutines.SupervisorJob @@ -25,17 +26,15 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO) private val runner = AdkAgentRunner( context = reactContext, - onToolCall = { toolCallId, toolId, arguments, toolCallScope -> - toolCallScope?.streamId?.let { AdkToolBridge.registerToolCall(it, toolCallId) } - toolCallScope?.runId?.let { AdkToolBridge.registerToolCall(it, toolCallId) } - val args = Arguments.createMap().apply { - putString("toolCallId", toolCallId) - putString("toolId", toolId) - putString("arguments", arguments) - toolCallScope?.streamId?.let { putString("streamId", it) } - toolCallScope?.runId?.let { putString("runId", it) } - } - emitOnToolCall(args) + onToolCall = { runId, toolCallId, toolName, arguments -> + emitOnToolCall( + Arguments.createMap().apply { + putString("runId", runId) + putString("toolCallId", toolCallId) + putString("toolName", toolName) + putString("arguments", arguments) + }, + ) }, onDownloadProgress = { phase, bytesDownloaded, bytesToDownload -> emitOnNanoDownloadProgress( @@ -125,7 +124,8 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : tools: ReadableArray?, promise: Promise, ) { - val job = scope.launch { + // Started after it is tracked, so a run that ends at once is still removed + val job = scope.launch(start = CoroutineStart.LAZY) { try { val result = runner.generateText(messages, config, options, tools, runId) val response = Arguments.createMap().apply { @@ -168,10 +168,11 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : promise.rejectWithCode(error) } finally { activeRuns.remove(runId) - AdkToolBridge.cancelForStream(runId) + AdkToolBridge.cancel(runId) } } activeRuns[runId] = job + job.start() } override fun streamText( @@ -187,9 +188,11 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : return } - val job = scope.launch { + val job = scope.launch(start = CoroutineStart.LAZY) { try { - var latestUsage: UsageMetadata? = null + // Summed over complete events, with the latest partial usage as a fallback + var totalUsage: UsageMetadata? = null + var partialUsage: UsageMetadata? = null // Per-turn state. Gemini Nano emits ```tool_code blocks (as streamed text) instead // of native function calls, and the marker can be split across stream chunks. // @@ -214,7 +217,8 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : runner.streamText(messages, config, options, tools, streamId).collect { event -> if (!isActive) return@collect - event.usageMetadata?.let { latestUsage = it } + totalUsage = AdkUsage.add(totalUsage, event) + if (event.partial) event.usageMetadata?.let { partialUsage = it } toolCallEmitter.handleEvent(event) toolCallEmitter.handleResults(event) @@ -274,7 +278,7 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : emittedText = currentTurnText } - emitComplete(streamId, event, latestUsage) + emitComplete(streamId, event, totalUsage ?: partialUsage) } } } catch (error: Exception) { @@ -286,10 +290,12 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : emitOnStreamError(errorArgs) } finally { activeRuns.remove(streamId) + AdkToolBridge.cancel(streamId) } } activeRuns[streamId] = job + job.start() promise.resolve(null) } @@ -303,12 +309,12 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : ) } - private fun emitComplete(streamId: String, event: Event, latestUsage: UsageMetadata?) { + private fun emitComplete(streamId: String, event: Event, usage: UsageMetadata?) { emitOnStreamComplete( Arguments.createMap().apply { putString("streamId", streamId) event.finishReason?.name?.let { putString("finishReason", it) } - AdkUsage.toWritableMap(latestUsage ?: event.usageMetadata)?.let { + AdkUsage.toWritableMap(usage)?.let { putMap("usage", it) } }, @@ -317,12 +323,12 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : override fun cancelStream(streamId: String, promise: Promise) { activeRuns.remove(streamId)?.cancel() - AdkToolBridge.cancelForStream(streamId) + AdkToolBridge.cancel(streamId) promise.resolve(null) } - override fun submitToolResult(toolCallId: String, result: String) { - AdkToolBridge.submitResult(toolCallId, result) + override fun submitToolResult(runId: String, toolCallId: String, result: String) { + AdkToolBridge.submitResult(runId, toolCallId, result) } override fun invalidate() { diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt index 734c968f9..fe9289b5e 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/ReactNativeFunctionTool.kt @@ -1,103 +1,63 @@ package com.callstack.ai.adk -import com.google.adk.kt.models.LlmRequest +import com.google.adk.kt.tools.BaseTool import com.google.adk.kt.tools.FunctionTool import com.google.adk.kt.tools.ToolContext -import com.google.adk.kt.types.Content import com.google.adk.kt.types.FunctionDeclaration -import com.google.adk.kt.types.Part import com.google.adk.kt.types.Schema import com.google.adk.kt.types.Type import java.util.UUID +/** An ADK tool whose `execute` runs in JavaScript. */ class ReactNativeFunctionTool( - private val toolId: String, name: String, description: String, private val parameterSchema: Schema?, - private val parameters: List, - private val injectPromptDescription: Boolean, - private val toolCallScope: ToolCallScope?, + private val runId: String, private val onToolCall: ( + runId: String, toolCallId: String, - toolId: String, + toolName: String, arguments: String, - scope: ToolCallScope?, ) -> Unit, ) : FunctionTool(name = name, description = description) { - override fun declaration(): FunctionDeclaration { - val schema = - parameterSchema - ?: Schema( - type = Type.OBJECT, - properties = - parameters.associate { parameter -> - parameter.name to - Schema( - type = parameter.type, - description = parameter.description, - ) - }, - required = parameters.filter { it.required }.map { it.name }.ifEmpty { null }, - ) - - return FunctionDeclaration( + override fun declaration(): FunctionDeclaration = + FunctionDeclaration( name = name, description = description, - parameters = schema, + parameters = parameterSchema ?: Schema(type = Type.OBJECT), ) - } - - override suspend fun processLlmRequest( - toolContext: ToolContext, - llmRequest: LlmRequest, - ): LlmRequest { - val requestWithTool = super.processLlmRequest(toolContext, llmRequest) - if (!injectPromptDescription) { - return requestWithTool - } - - val declaration = declaration() + /** Describes the tool for models without native function calling. */ + fun promptDescription(): String { + val parameters = declaration().parameters val parameterLines = - declaration.parameters?.properties?.entries?.joinToString("\n") { (paramName, paramSchema) -> + parameters?.properties?.entries?.joinToString("\n") { (paramName, paramSchema) -> val paramType = paramSchema.type?.name?.lowercase() ?: "string" val paramDescription = paramSchema.description?.let { " — $it" } ?: "" - val required = - declaration.parameters?.required?.contains(paramName) == true + val required = parameters.required?.contains(paramName) == true " - $paramName ($paramType, required=$required)$paramDescription" } ?: "" - val toolDescription = - buildString { - append("Tool: ${declaration.name}") - if (declaration.description.isNotBlank()) { - append(" — ${declaration.description}") - } - if (parameterLines.isNotBlank()) { - append("\n Parameters:\n$parameterLines") - } + return buildString { + append("- $name") + if (description.isNotBlank()) { + append(": $description") } - - return requestWithTool.appendInstructions( - Content(parts = listOf(Part(text = toolDescription))), - ) + if (parameterLines.isNotBlank()) { + append("\n Parameters:\n$parameterLines") + } + } } override suspend fun execute(context: ToolContext, args: Map): Any { - val toolCallId = UUID.randomUUID().toString() - val argumentsJson = AdkJson.encode(args) - onToolCall(toolCallId, toolId, argumentsJson, toolCallScope) - val result = AdkToolBridge.awaitResult(toolCallId) - return AdkJson.decodeResult(result) - ?: throw IllegalArgumentException("Failed to decode result in ReactNativeFunctionTool::execute") + // Use ADK's id so JS sees the same id as the reported tool call + val toolCallId = context.functionCallId ?: UUID.randomUUID().toString() + val result = AdkToolBridge.call(runId, toolCallId) { + onToolCall(runId, toolCallId, name, AdkJson.encode(args)) + } + // ADK wraps values that are not maps under `result` + return AdkJson.decodeResult(result) ?: mapOf(BaseTool.RESULT_KEY to null) } } - -data class ToolParameterSpec( - val name: String, - val description: String?, - val type: Type, - val required: Boolean, -) diff --git a/packages/adk/src/NativeAdkEngine.ts b/packages/adk/src/NativeAdkEngine.ts index 4cb1b15bf..9d6c611b7 100644 --- a/packages/adk/src/NativeAdkEngine.ts +++ b/packages/adk/src/NativeAdkEngine.ts @@ -34,19 +34,10 @@ export interface AdkAgentConfig { } } -export interface AdkToolParameter { - name: string - description?: string - type?: 'string' | 'number' | 'boolean' | 'object' | 'array' - required?: boolean -} - export interface AdkTool { - id: string name: string description: string inputSchema?: CodegenTypes.UnsafeObject - parameters?: AdkToolParameter[] } export interface AdkResponseFormat { @@ -138,11 +129,11 @@ export interface StreamToolCallEvent { } export interface ToolCallEvent { + /** The `generateText` or `streamText` run that called the tool */ + runId: string toolCallId: string - toolId: string + toolName: string arguments: string - streamId?: string - runId?: string } export interface NanoDownloadProgressEvent { @@ -183,7 +174,8 @@ export interface Spec extends TurboModule { /** Cancels a stream or a `generateText` run by its id */ cancelStream(streamId: string): Promise - submitToolResult(toolCallId: string, result: string): void + /** `result` is JSON */ + submitToolResult(runId: string, toolCallId: string, result: string): void onStreamUpdate: CodegenTypes.EventEmitter onStreamComplete: CodegenTypes.EventEmitter @@ -193,13 +185,6 @@ export interface Spec extends TurboModule { onNanoDownloadProgress: CodegenTypes.EventEmitter } -declare global { - var __ADK_TOOLS__: Record< - string, - (args: string) => unknown | Promise - > -} - const NativeAdkEngine: Spec | null = Platform.OS === 'android' ? TurboModuleRegistry.getEnforcing('AdkEngine') diff --git a/packages/adk/src/__tests__/ai-sdk.test.ts b/packages/adk/src/__tests__/ai-sdk.test.ts index 4cb2316d5..0f4afbc94 100644 --- a/packages/adk/src/__tests__/ai-sdk.test.ts +++ b/packages/adk/src/__tests__/ai-sdk.test.ts @@ -23,6 +23,7 @@ const onStreamComplete = createEmitter() const onStreamError = createEmitter() const onStreamToolCall = createEmitter() const onNanoDownloadProgress = createEmitter() +const onToolCall = createEmitter() let generateTextResult: Record = { content: 'ok', @@ -49,6 +50,9 @@ const streamText = mock( ) const prepareNano = mock(async (_nano?: unknown) => {}) const cancelStream = mock(async (_id: string) => {}) +const submitToolResult = mock( + (_runId: string, _toolCallId: string, _result: string) => {} +) const getModelInfo = mock(async (_modelType: string, _nano?: unknown) => ({ availability: 'downloadable', })) @@ -64,7 +68,8 @@ mock.module('react-native', () => ({ getModelInfo, isNanoSupported: async () => true, isAvailable: async () => true, - onToolCall: () => ({ remove: () => {} }), + onToolCall, + submitToolResult, onStreamUpdate, onStreamToolCall, onStreamComplete, @@ -123,6 +128,7 @@ beforeEach(() => { streamText.mockClear() prepareNano.mockClear() cancelStream.mockClear() + submitToolResult.mockClear() generateTextResult = { content: 'ok', finishReason: 'STOP' } }) @@ -343,6 +349,68 @@ describe('tool choice', () => { }) }) +describe('tool execution', () => { + // Calls the tool from native code during the run, then waits for its result + async function callTool( + execute: (input: unknown, options: { toolCallId: string }) => unknown, + eventRunId = (runId: string) => runId + ) { + const model = createAdkProvider({ + modelType: 'gemini', + availableTools: { + getWeather: { inputSchema: weatherTool.inputSchema, execute }, + } as never, + })() + generateText.mockImplementationOnce(async (runId: string) => { + onToolCall.emit({ + runId: eventRunId(runId), + toolCallId: 'adk-call-1', + toolName: 'getWeather', + arguments: '{"city":"Oslo"}', + }) + await new Promise((resolve) => setTimeout(resolve, 0)) + return generateTextResult + }) + + await model.doGenerate(createCallOptions({ tools: [weatherTool] })) + return submitToolResult.mock.calls.at(-1) + } + + test('runs the tool with the ADK tool call id', async () => { + const execute = mock(async () => ({ sky: 'clear' })) + const [, toolCallId, result] = (await callTool(execute))! + + expect(execute.mock.calls[0]).toMatchObject([ + { city: 'Oslo' }, + { toolCallId: 'adk-call-1' }, + ]) + expect(toolCallId).toBe('adk-call-1') + expect(result).toBe('{"sky":"clear"}') + }) + + test('sends string results as JSON', async () => { + const [, , result] = (await callTool(async () => '42'))! + + expect(result).toBe('"42"') + }) + + test('sends tool errors back to the model', async () => { + const [, , result] = (await callTool(async () => { + throw new Error('offline') + }))! + + expect(result).toBe('{"error":"offline"}') + }) + + test('ignores tool calls from other runs', async () => { + const execute = mock(async () => 'ok') + await callTool(execute, () => 'other') + + expect(execute).not.toHaveBeenCalled() + expect(submitToolResult).not.toHaveBeenCalled() + }) +}) + describe('generation options', () => { test('maps reasoning levels to a thinking budget', async () => { await gemini.doGenerate(createCallOptions({ reasoning: 'high' })) @@ -397,12 +465,12 @@ describe('generation options', () => { test('warns about settings Gemini Nano ignores', async () => { const result = await nano.doGenerate( - createCallOptions({ topP: 0.5, stopSequences: ['END'] }) + createCallOptions({ topP: 0.5, seed: 1, stopSequences: ['END'] }) ) expect( result.warnings.map((warning) => 'feature' in warning && warning.feature) - ).toEqual(['topP', 'stopSequences']) + ).toEqual(['topP', 'seed', 'stopSequences']) }) }) diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index 8c8717ff0..142b00104 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -31,7 +31,6 @@ import { type AdkModelType, type AdkNanoModelConfig, type AdkTool, - type AdkToolParameter, type AdkUsageMetadata, getNativeAdkEngine, type NanoDownloadProgressEvent, @@ -386,24 +385,6 @@ const convertFinishReason = ( } } -const schemaPropertyToAdkParameter = ( - name: string, - value: unknown, - required: boolean -): AdkToolParameter => { - const schema = value as { - type?: string - description?: string - } - - return { - name, - description: schema?.description, - type: (schema?.type as AdkToolParameter['type']) ?? 'string', - required, - } -} - const uint8ArrayToBase64 = (bytes: Uint8Array): string => { let binary = '' for (let index = 0; index < bytes.length; index++) { @@ -668,8 +649,10 @@ class AdkChatLanguageModel implements AdkLanguageModel { {}) as AdkProviderOptions if (this.isNano) { + // ADK's ML Kit model only passes on temperature, topK and maxOutputTokens const unsupported = { topP: options.topP, + seed: options.seed, stopSequences: options.stopSequences, presencePenalty: options.presencePenalty, frequencyPenalty: options.frequencyPenalty, @@ -707,8 +690,8 @@ class AdkChatLanguageModel implements AdkLanguageModel { } private prepareTools(tools: Tool[], warnings: SharedV4Warning[]) { - return tools - .filter((tool): tool is LanguageModelV4FunctionTool => { + const functionTools = tools.filter( + (tool): tool is LanguageModelV4FunctionTool => { if (tool.type === 'function') { return true } @@ -717,59 +700,46 @@ class AdkChatLanguageModel implements AdkLanguageModel { feature: `provider-defined tool ${tool.name}`, }) return false - }) - .map((tool) => { - const required = Array.isArray(tool.inputSchema.required) - ? tool.inputSchema.required - : [] - const properties = tool.inputSchema.properties ?? {} - - const parameters = Object.entries(properties).map(([name, value]) => - schemaPropertyToAdkParameter(name, value, required.includes(name)) - ) + } + ) - const schema = jsonSchema(tool.inputSchema) + for (const tool of functionTools) { + if (!this.tools[tool.name]?.execute) { + warnings.push({ + type: 'other', + message: `Tool "${tool.name}" has no executor. Pass it to createAdkProvider via availableTools so ADK can run it.`, + }) + } + } - if (!this.tools[tool.name]?.execute) { - warnings.push({ - type: 'other', - message: `Tool "${tool.name}" has no executor. Pass it to createAdkProvider via availableTools so ADK can run it.`, - }) - } + return functionTools.map( + (tool): AdkTool => ({ + name: tool.name, + description: tool.description ?? '', + inputSchema: tool.inputSchema, + }) + ) + } - return { - native: { - id: generateId(), - name: tool.name, - description: tool.description ?? '', - inputSchema: tool.inputSchema, - parameters, - } satisfies AdkTool, - execute: async (modelInput: unknown) => { - const execute = this.tools[tool.name]?.execute - if (!execute) { - return { - error: `Tool "${tool.name}" is not available in this app.`, - } - } + private async executeTool( + tool: LanguageModelV4FunctionTool, + event: ToolCallEvent + ) { + const execute = this.tools[tool.name]?.execute + if (!execute) { + return { error: `Tool "${tool.name}" is not available in this app.` } + } - const text = - typeof modelInput === 'string' - ? modelInput - : JSON.stringify(modelInput ?? {}) - const toolCallArguments = await parseJSON({ - text, - schema, - }) - const opts: ToolExecutionOptions = { - toolCallId: generateId(), - messages: [], - context: undefined, - } - return execute(toolCallArguments, opts) - }, - } - }) + const input = await parseJSON({ + text: event.arguments, + schema: jsonSchema(tool.inputSchema), + }) + const options: ToolExecutionOptions = { + toolCallId: event.toolCallId, + messages: [], + context: undefined, + } + return execute(input, options) } private prepareCall(options: LanguageModelV4CallOptions) { @@ -779,78 +749,46 @@ class AdkChatLanguageModel implements AdkLanguageModel { ) const messages = this.prepareMessages(options.prompt, warnings) const generationOptions = this.prepareGenerationOptions(options, warnings) - const preparedTools = this.prepareTools(tools, warnings) return { messages, generationOptions, - preparedTools, - nativeTools: preparedTools.map((tool) => tool.native), - ownedToolIds: new Set(preparedTools.map((tool) => tool.native.id)), + tools, + nativeTools: this.prepareTools(tools, warnings), agentConfig: this.prepareAgentConfig(options), warnings, } } - private registerTools( - preparedTools: ReturnType - ) { - globalThis.__ADK_TOOLS__ = globalThis.__ADK_TOOLS__ ?? {} - - for (const tool of preparedTools) { - globalThis.__ADK_TOOLS__[tool.native.id] = tool.execute - } - } - - private unregisterTools( - preparedTools: ReturnType - ) { - for (const tool of preparedTools) { - delete globalThis.__ADK_TOOLS__?.[tool.native.id] - } - } - - private createToolCallListener(scope: { - streamId?: string - runId?: string - ownedToolIds: Set - }) { + // Runs the tools that native code calls during `runId` + private listenForToolCalls(runId: string, tools: Tool[]) { const native = getNativeAdkEngine() return native.onToolCall(async (event: ToolCallEvent) => { - if (scope.streamId !== undefined && event.streamId !== scope.streamId) { - return - } - if (scope.runId !== undefined && event.runId !== scope.runId) { - return - } - if (!scope.ownedToolIds.has(event.toolId)) { - return - } - - const executor = globalThis.__ADK_TOOLS__?.[event.toolId] - if (!executor) { - native.submitToolResult( - event.toolCallId, - JSON.stringify({ error: `Tool not found: ${event.toolId}` }) - ) + if (event.runId !== runId) { return } + const tool = tools.find( + (candidate): candidate is LanguageModelV4FunctionTool => + candidate.type === 'function' && candidate.name === event.toolName + ) + let result: unknown try { - const result = await executor(event.arguments) - native.submitToolResult( - event.toolCallId, - typeof result === 'string' ? result : JSON.stringify(result ?? {}) - ) + result = tool + ? await this.executeTool(tool, event) + : { error: `Tool not found: ${event.toolName}` } } catch (error) { - native.submitToolResult( - event.toolCallId, - JSON.stringify({ - error: error instanceof Error ? error.message : String(error), - }) - ) + result = { + error: error instanceof Error ? error.message : String(error), + } } + // Always JSON, so native code can tell strings from other values + native.submitToolResult( + runId, + event.toolCallId, + JSON.stringify(result ?? null) + ) }) } @@ -858,9 +796,8 @@ class AdkChatLanguageModel implements AdkLanguageModel { const { messages, generationOptions, - preparedTools, + tools, nativeTools, - ownedToolIds, agentConfig, warnings, } = this.prepareCall(options) @@ -878,8 +815,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { .catch((error) => console.warn('[adk] cancel failed:', error)) } - this.registerTools(preparedTools) - const toolListener = this.createToolCallListener({ runId, ownedToolIds }) + const toolListener = this.listenForToolCalls(runId, tools) abortSignal?.addEventListener('abort', onAbort, { once: true }) try { @@ -936,7 +872,6 @@ class AdkChatLanguageModel implements AdkLanguageModel { } finally { abortSignal?.removeEventListener('abort', onAbort) toolListener.remove() - this.unregisterTools(preparedTools) } } @@ -944,9 +879,8 @@ class AdkChatLanguageModel implements AdkLanguageModel { const { messages, generationOptions, - preparedTools, + tools, nativeTools, - ownedToolIds, agentConfig, warnings, } = this.prepareCall(options) @@ -965,11 +899,9 @@ class AdkChatLanguageModel implements AdkLanguageModel { throw getAbortReason(abortSignal) } - this.registerTools(preparedTools) - let listeners: { remove(): void }[] = [] let cleanedUp = false - const toolListener = this.createToolCallListener({ streamId, ownedToolIds }) + const toolListener = this.listenForToolCalls(streamId, tools) const toolNames = new Map() let onAbort = () => {} @@ -985,7 +917,6 @@ class AdkChatLanguageModel implements AdkLanguageModel { listeners = [] abortSignal?.removeEventListener('abort', onAbort) toolListener.remove() - this.unregisterTools(preparedTools) } const cancelNative = () => { diff --git a/website/src/docs/adk/generating.md b/website/src/docs/adk/generating.md index 3f3ba6384..eb35f630a 100644 --- a/website/src/docs/adk/generating.md +++ b/website/src/docs/adk/generating.md @@ -248,7 +248,7 @@ Configure model behavior with generation options: | `temperature` | number | Controls randomness | Yes | | `maxOutputTokens` | number | Maximum tokens to generate | Yes | | `topK` | number | Top-K sampling parameter | Yes | -| `seed` | number | Seed for repeatable sampling | Yes | +| `seed` | number | Seed for repeatable sampling | No | | `topP` | number | Nucleus sampling threshold | No | | `stopSequences` | string[] | Stop generating at these texts | No | | `presencePenalty` | number | Penalize repeated topics | No | @@ -268,7 +268,6 @@ const { text } = await generateText({ temperature: 0.8, maxOutputTokens: 500, topK: 40, - seed: 42, }) ``` From 4cd07ec98a4dd931625eb5b51d8cc362e2d930a0 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:51:47 +0400 Subject: [PATCH 09/18] refactor(adk): hide gemini nano tool_code text in one place Kotlin held back tool_code markers and JS held back JSON fences. Both now live in a tested StreamTextFilter, and tool_code text is only hidden for Gemini Nano. Co-Authored-By: Claude Opus 5.5 --- .../callstack/ai/adk/NativeAdkEngineModule.kt | 69 ++------------ packages/adk/src/__tests__/ai-sdk.test.ts | 25 +++++ packages/adk/src/__tests__/streamText.test.ts | 56 +++++++++++ packages/adk/src/ai-sdk.ts | 41 ++------- packages/adk/src/streamText.ts | 92 +++++++++++++++++++ 5 files changed, 190 insertions(+), 93 deletions(-) create mode 100644 packages/adk/src/__tests__/streamText.test.ts create mode 100644 packages/adk/src/streamText.ts diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt index 53a546deb..aca9c398a 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt @@ -193,21 +193,11 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : // Summed over complete events, with the latest partial usage as a fallback var totalUsage: UsageMetadata? = null var partialUsage: UsageMetadata? = null - // Per-turn state. Gemini Nano emits ```tool_code blocks (as streamed text) instead - // of native function calls, and the marker can be split across stream chunks. - // - // ADK/GenaiPrompt may deliver text either cumulatively (each event repeats the full - // text) or incrementally (each event carries only the new delta). We can't rely on - // either, so we reconstruct the full turn text ourselves by appending the delta of - // each event, then run marker detection and hold-back on that reconstruction. - // - // currentTurnText: full reconstructed text of the current agent turn - // emittedText: the prefix of currentTurnText already sent to JS - // suppressCurrentTurn: true once we confirm this turn is a tool-code turn - // currentTurnReasoning / emittedReasoning: the same for thought parts + // ADK may deliver text cumulatively (each event repeats the turn so far) or as + // deltas, so the turn text is rebuilt here and only the new part is sent to JS. + // JS hides Gemini Nano tool_code blocks. Reasoning is tracked the same way. var currentTurnText = "" var emittedText = "" - var suppressCurrentTurn = false var currentTurnReasoning = "" var emittedReasoning = "" val toolCallEmitter = AdkStreamToolCallEmitter(streamId) { args -> @@ -236,48 +226,25 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : } val text = event.textParts(thoughts = false) - if (text.isNotEmpty()) { - // Reconstruct the full turn text. If the event repeats what we already have - // (cumulative mode) take it as-is; otherwise treat it as an incremental delta - // and append it. This keeps a split marker (e.g. "```tool_" + "code") intact. currentTurnText = if (text.startsWith(currentTurnText)) text else currentTurnText + text - - // Once the full tool-code marker appears, suppress the rest of this turn. - if (currentTurnText.contains(TOOL_CODE_MARKER)) { - suppressCurrentTurn = true - } } - // When a tool-calling turn ends, reset per-turn state so the next turn - // (the actual answer) starts with a clean slate and streams normally. if (!event.partial && event.functionCalls().isNotEmpty()) { + // A tool call ends the turn, and the next turn starts with new text currentTurnText = "" emittedText = "" - suppressCurrentTurn = false currentTurnReasoning = "" emittedReasoning = "" - } - - if (!suppressCurrentTurn && !(event.isFinalResponse && !event.partial)) { - // Hold back any trailing substring that could be the beginning of the marker, - // so we never emit a partial "```", "```tool", etc. that later completes. - val safeEnd = currentTurnText.length - pendingMarkerPrefixLength(currentTurnText) - if (safeEnd > emittedText.length) { - emitDelta(streamId, currentTurnText.substring(emittedText.length, safeEnd), "text") - emittedText = currentTurnText.substring(0, safeEnd) - } + } else if (currentTurnText.length > emittedText.length && + currentTurnText.startsWith(emittedText) + ) { + emitDelta(streamId, currentTurnText.substring(emittedText.length), "text") + emittedText = currentTurnText } if (event.isFinalResponse && !event.partial) { - // Flush any held-back text for a normal turn (e.g. a response legitimately - // ending in backticks that turned out not to be a tool-code marker). - if (!suppressCurrentTurn && currentTurnText.length > emittedText.length) { - emitDelta(streamId, currentTurnText.substring(emittedText.length), "text") - emittedText = currentTurnText - } - emitComplete(streamId, event, totalUsage ?: partialUsage) } } @@ -339,25 +306,7 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : super.invalidate() } - /** - * Returns the length of the trailing suffix of [text] that is a proper prefix of - * [TOOL_CODE_MARKER] (e.g. "`", "``", "```", "```tool"). This portion is held back - * from streaming because a subsequent token could complete the marker, at which point - * the whole turn is suppressed. If no suffix could begin the marker, returns 0. - */ - private fun pendingMarkerPrefixLength(text: String): Int { - val maxLen = minOf(text.length, TOOL_CODE_MARKER.length - 1) - for (len in maxLen downTo 1) { - val suffix = text.substring(text.length - len) - if (TOOL_CODE_MARKER.startsWith(suffix)) { - return len - } - } - return 0 - } - companion object { const val NAME = "AdkEngine" - private const val TOOL_CODE_MARKER = "```tool_code" } } diff --git a/packages/adk/src/__tests__/ai-sdk.test.ts b/packages/adk/src/__tests__/ai-sdk.test.ts index 0f4afbc94..4ec320d58 100644 --- a/packages/adk/src/__tests__/ai-sdk.test.ts +++ b/packages/adk/src/__tests__/ai-sdk.test.ts @@ -642,6 +642,31 @@ describe('doStream', () => { expect(text).toBe('{"a":1}') }) + test('hides Gemini Nano tool_code text and streams the next turn', async () => { + const parts = await runStream( + createCallOptions(), + (streamId) => { + for (const delta of ['Checking.\n``', '`tool_code\nf()\n```']) { + onStreamUpdate.emit({ streamId, delta }) + } + onStreamToolCall.emit({ + streamId, + phase: 'start', + toolCallId: '1', + toolName: 'f', + }) + onStreamUpdate.emit({ streamId, delta: 'Done.' }) + onStreamComplete.emit({ streamId }) + }, + nano + ) + + const text = parts + .flatMap((part) => (part.type === 'text-delta' ? [part.delta] : [])) + .join('') + expect(text).toBe('Checking.\nDone.') + }) + test('surfaces native stream errors with their code', async () => { const parts = await runStream(createCallOptions(), (streamId) => { onStreamError.emit({ diff --git a/packages/adk/src/__tests__/streamText.test.ts b/packages/adk/src/__tests__/streamText.test.ts new file mode 100644 index 000000000..6714bc811 --- /dev/null +++ b/packages/adk/src/__tests__/streamText.test.ts @@ -0,0 +1,56 @@ +import { describe, expect, test } from 'bun:test' + +import { StreamTextFilter } from '../streamText' + +function run( + deltas: string[], + options: { json?: boolean; toolCode?: boolean } = {} +) { + const filter = new StreamTextFilter({ + json: options.json ?? false, + toolCode: options.toolCode ?? false, + }) + const shown = deltas.map((delta) => filter.push(delta)) + return { shown, text: shown.join('') + filter.flush() } +} + +describe('StreamTextFilter', () => { + test('passes plain text through', () => { + expect(run(['Hel', 'lo']).shown).toEqual(['Hel', 'lo']) + }) + + test('hides a tool_code block split across deltas', () => { + const { shown, text } = run( + ['Let me check.\n``', '`tool_', 'code\ngetWeather()\n```'], + { toolCode: true } + ) + + expect(shown).toEqual(['Let me check.\n', '', '']) + expect(text).toBe('Let me check.\n') + }) + + test('shows held back backticks that are not a tool_code block', () => { + const { shown, text } = run(['Use ``', 'x`` here'], { toolCode: true }) + + expect(shown).toEqual(['Use ', '``x`` here']) + expect(text).toBe('Use ``x`` here') + }) + + test('starts a new turn after a tool call', () => { + const filter = new StreamTextFilter({ json: false, toolCode: true }) + filter.push('```tool_code\nf()\n```') + filter.endTurn() + + expect(filter.push('Done.')).toBe('Done.') + }) + + test('leaves tool_code text alone for other models', () => { + expect(run(['```tool_code\nf()\n```']).text).toBe('```tool_code\nf()\n```') + }) + + test('strips JSON code fences', () => { + expect( + run(['``', '`json\n{"a"', ':1}', '\n`', '``'], { json: true }).text + ).toBe('{"a":1}') + }) +}) diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index 142b00104..431f305cc 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -37,6 +37,7 @@ import { type StreamToolCallEvent, type ToolCallEvent, } from './NativeAdkEngine' +import { StreamTextFilter, stripJsonFences } from './streamText' const DEFAULT_NANO_MODEL = 'gemini-nano' const DEFAULT_GEMINI_MODEL = 'gemini-2.5-flash' @@ -440,11 +441,6 @@ const getAbortReason = (signal: AbortSignal) => { const stringifyToolOutput = (value: unknown) => typeof value === 'string' ? value : JSON.stringify(value ?? null) -// Gemini Nano often wraps JSON in a Markdown code fence -const jsonFencePrefix = /^\s*```(?:json)?\s*\n?/ -const stripJsonFences = (text: string) => - text.replace(jsonFencePrefix, '').replace(/\s*```\s*$/, '') - const parseToolResult = (output: string): NonNullable => { try { return (JSON.parse(output) as JSONValue) ?? {} @@ -931,9 +927,10 @@ class AdkChatLanguageModel implements AdkLanguageModel { let textOpen = false let reasoningId: string | undefined let reasoningCount = 0 - // JSON mode keeps the full text so code fences can be stripped - let rawText = '' - let emittedText = '' + const textFilter = new StreamTextFilter({ + json: isJson, + toolCode: this.isNano, + }) const addListener = (listener: { remove(): void }) => { listeners.push(listener) @@ -958,21 +955,6 @@ class AdkChatLanguageModel implements AdkLanguageModel { controller.enqueue({ type: 'text-delta', id: textId, delta }) } - const emitJsonText = (final: boolean) => { - const stripped = final - ? stripJsonFences(rawText) - : // Hold back trailing backticks that could start a closing fence - rawText.replace(jsonFencePrefix, '').replace(/\s*`{1,3}$/, '') - // Only emit once the opening fence can no longer change the prefix - if (!final && /^\s*`{0,3}(j(s(o(n)?)?)?)?$/.test(rawText)) { - return - } - if (stripped.startsWith(emittedText)) { - emitText(stripped.slice(emittedText.length)) - emittedText = stripped - } - } - controller.enqueue({ type: 'stream-start', warnings }) onAbort = () => { @@ -1001,13 +983,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { return } - if (isJson) { - rawText += data.delta - emitJsonText(false) - return - } - - emitText(data.delta) + emitText(textFilter.push(data.delta)) }) ) @@ -1019,6 +995,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { if (data.phase === 'start' && data.toolName) { closeReasoning() + textFilter.endTurn() toolNames.set(data.toolCallId, data.toolName) controller.enqueue({ type: 'tool-input-start', @@ -1070,9 +1047,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { if (data.streamId !== streamId) { return } - if (isJson) { - emitJsonText(true) - } + emitText(textFilter.flush()) closeReasoning() if (textOpen) { controller.enqueue({ type: 'text-end', id: textId }) diff --git a/packages/adk/src/streamText.ts b/packages/adk/src/streamText.ts new file mode 100644 index 000000000..1dc20fcf6 --- /dev/null +++ b/packages/adk/src/streamText.ts @@ -0,0 +1,92 @@ +// Gemini Nano writes tool calls as text, which ADK turns into function calls +const toolCodeMarker = '```tool_code' + +// Gemini Nano often wraps JSON in a Markdown code fence +const jsonFencePrefix = /^\s*```(?:json)?\s*\n?/ +// Text that could still grow into the opening fence +const partialJsonFence = /^\s*`{0,3}(j(s(o(n)?)?)?)?$/ + +export const stripJsonFences = (text: string) => + text.replace(jsonFencePrefix, '').replace(/\s*```\s*$/, '') + +// Length of the end of `text` that could be the start of `marker` +function pendingPrefixLength(text: string, marker: string) { + for ( + let length = Math.min(text.length, marker.length - 1); + length > 0; + length-- + ) { + if (marker.startsWith(text.slice(-length))) { + return length + } + } + return 0 +} + +/** + * Filters streamed model text. It hides Gemini Nano `tool_code` blocks and, in + * JSON mode, Markdown code fences, holding back text until it cannot become one. + */ +export class StreamTextFilter { + private turnText = '' + private emitted = '' + + constructor(private readonly options: { json: boolean; toolCode: boolean }) {} + + /** Adds a text delta and returns the text that is now safe to show */ + push(delta: string) { + this.turnText += delta + return this.take(false) + } + + /** Returns the rest of the text at the end of the stream */ + flush() { + return this.take(true) + } + + /** A tool call ended the model turn, so the next turn starts empty */ + endTurn() { + this.turnText = '' + this.emitted = '' + } + + private take(final: boolean) { + const visible = this.visibleText(final) + if (visible === undefined || !visible.startsWith(this.emitted)) { + return '' + } + const delta = visible.slice(this.emitted.length) + this.emitted = visible + return delta + } + + private visibleText(final: boolean): string | undefined { + let text = this.turnText + const markerIndex = this.options.toolCode + ? text.indexOf(toolCodeMarker) + : -1 + if (markerIndex !== -1) { + text = text.slice(0, markerIndex) + } + + if (this.options.json) { + if (final) { + return stripJsonFences(text) + } + if (partialJsonFence.test(text)) { + return undefined + } + // Hold back trailing backticks that could start a closing fence + text = text.replace(jsonFencePrefix, '').replace(/\s*`{1,3}$/, '') + } + + // Hold back the end of the text while it could still become the marker + if (this.options.toolCode && markerIndex === -1 && !final) { + text = text.slice( + 0, + text.length - pendingPrefixLength(text, toolCodeMarker) + ) + } + return text + } +} From 7b7292c5e6d3dafebb37320143aed84d4ab274f9 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:52:47 +0400 Subject: [PATCH 10/18] refactor(adk): drop duplicate native defaults and unused error code - Kotlin no longer repeats the agent and model defaults that JS sends - remove TOOL_CALL_ERROR, which native code never reports - build the unsupported platform error with createAdkError Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 39 +++++++++---------- packages/adk/src/NativeAdkEngine.ts | 13 ++++--- packages/adk/src/errors.ts | 1 - 3 files changed, 25 insertions(+), 28 deletions(-) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 91215ae87..ae390e70c 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -354,9 +354,6 @@ class AdkAgentRunner( } } - private fun parseAgentConfig(config: ReadableMap): AgentConfig = - parseAgentConfigStatic(config) - private fun parseGenerationConfig(options: ReadableMap?): GenerateContentConfig? { if (options == null) return null @@ -517,23 +514,23 @@ class AdkAgentRunner( } } - companion object { - fun parseAgentConfig(config: ReadableMap): AgentConfig = parseAgentConfigStatic(config) - - private fun parseAgentConfigStatic(config: ReadableMap): AgentConfig { - val model = config.getMap("model") - ?: throw IllegalArgumentException("Agent config requires a model") - - return AgentConfig( - name = config.getString("name") ?: "react_native_adk_agent", - description = config.getString("description") ?: "", - instruction = config.getString("instruction"), - modelType = model.getString("type") ?: "genai-nano", - modelName = model.getString("name") ?: "gemini-nano", - apiKey = model.getString("apiKey"), - nano = NanoConfig.from(model.getMap("nano")), - ) - } + // JS always sends the name and model, with defaults applied + private fun parseAgentConfig(config: ReadableMap): AgentConfig { + fun ReadableMap.require(key: String) = + getString(key) ?: throw IllegalArgumentException("Agent config requires $key") + + val model = config.getMap("model") + ?: throw IllegalArgumentException("Agent config requires a model") + + return AgentConfig( + name = config.require("name"), + description = config.getString("description") ?: "", + instruction = config.getString("instruction"), + modelType = model.require("type"), + modelName = model.require("name"), + apiKey = model.getString("apiKey"), + nano = NanoConfig.from(model.getMap("nano")), + ) } } @@ -547,7 +544,7 @@ data class AgentConfig( val nano: NanoConfig, ) -/** ADK content role — one of [Role.USER], [Role.MODEL], or [Role.SYSTEM]. */ +/** ADK content role, [Role.USER] or [Role.MODEL]. */ private typealias ContentRole = String data class ParsedMessage( diff --git a/packages/adk/src/NativeAdkEngine.ts b/packages/adk/src/NativeAdkEngine.ts index 9d6c611b7..fa8e5f479 100644 --- a/packages/adk/src/NativeAdkEngine.ts +++ b/packages/adk/src/NativeAdkEngine.ts @@ -1,6 +1,8 @@ import type { CodegenTypes, TurboModule } from 'react-native' import { Platform, TurboModuleRegistry } from 'react-native' +import { AdkErrorCodes, createAdkError } from './errors' + export type AdkMessageRole = 'assistant' | 'user' export type AdkMessagePart = @@ -172,7 +174,7 @@ export interface Spec extends TurboModule { tools?: AdkTool[] ): Promise - /** Cancels a stream or a `generateText` run by its id */ + /** Cancels a `streamText` or `generateText` run by its id */ cancelStream(streamId: string): Promise /** `result` is JSON */ submitToolResult(runId: string, toolCallId: string, result: string): void @@ -194,11 +196,10 @@ export default NativeAdkEngine export function getNativeAdkEngine(): Spec { if (!NativeAdkEngine) { - const error = new Error( - '@react-native-ai/adk is only available on Android. See https://developer.android.com/ai/adk' - ) as Error & { code: string } - error.code = 'UNSUPPORTED_PLATFORM' - throw error + throw createAdkError( + '@react-native-ai/adk is only available on Android. See https://developer.android.com/ai/adk', + AdkErrorCodes.UnsupportedPlatform + ) } return NativeAdkEngine } diff --git a/packages/adk/src/errors.ts b/packages/adk/src/errors.ts index 7b76f30d7..89fd2e654 100644 --- a/packages/adk/src/errors.ts +++ b/packages/adk/src/errors.ts @@ -10,7 +10,6 @@ export const AdkErrorCodes = { GenerationError: 'GENERATION_ERROR', InvalidMessage: 'INVALID_MESSAGE', InvalidSchema: 'INVALID_SCHEMA', - ToolCallError: 'TOOL_CALL_ERROR', ContextWindowExceeded: 'CONTEXT_WINDOW_EXCEEDED', RateLimited: 'RATE_LIMITED', BackgroundUseBlocked: 'BACKGROUND_USE_BLOCKED', From f2050af9c6daf5de765f36e7b94332cd8a7a5d24 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:56:29 +0400 Subject: [PATCH 11/18] refactor(adk): align model, availability and prepare apis with apple - take the model id as a model option, like apple; modelType and modelName still work but are deprecated - getModelInfo takes a model and reports model and supportsTokenCounting - derive isAvailable from getModelInfo and deprecate isNanoSupported, removing the two native availability methods - report download progress from model.prepare({ onProgress }) and deprecate provider.prepareNano - default cloud Gemini to gemini-3.8-flash, since 2.5 models are limited to existing users Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 18 -- .../callstack/ai/adk/NativeAdkEngineModule.kt | 24 -- packages/adk/src/NativeAdkEngine.ts | 5 - packages/adk/src/__tests__/ai-sdk.test.ts | 54 ++++- packages/adk/src/ai-sdk.ts | 214 ++++++++++-------- 5 files changed, 163 insertions(+), 152 deletions(-) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index ae390e70c..91589ab8c 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -117,13 +117,6 @@ class AdkAgentRunner( ) } - private suspend fun nanoStatus(config: NanoConfig): Int = - try { - nanoClient(config).checkStatus() - } catch (_: Exception) { - FeatureStatus.UNAVAILABLE - } - /** Downloads Gemini Nano when needed, reporting progress, then warms it up. */ suspend fun prepareNano(config: NanoConfig) { nanoPrepareMutex.withLock { @@ -178,17 +171,6 @@ class AdkAgentRunner( return nanoModels.getValue(config) } - /** Whether this device supports Gemini Nano at all. Does not download the model. */ - suspend fun isNanoSupported(config: NanoConfig): Boolean = - nanoStatus(config) != FeatureStatus.UNAVAILABLE - - /** Whether Gemini Nano is downloaded or can be downloaded now. */ - suspend fun isNanoAvailable(config: NanoConfig): Boolean = - when (nanoStatus(config)) { - FeatureStatus.DOWNLOADABLE, FeatureStatus.AVAILABLE -> true - else -> false - } - suspend fun getModelInfo(modelType: String, config: NanoConfig): AdkModelInfo { if (modelType != "genai-nano") { return AdkModelInfo(availability = "available") diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt index aca9c398a..715a36e4d 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt @@ -54,30 +54,6 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : reject(AdkErrors.codeFor(error), error.message ?: "Unknown ADK error", error) } - override fun isAvailable(modelType: String, nano: ReadableMap?, promise: Promise) { - scope.launch { - try { - val available = when (modelType) { - "genai-nano" -> runner.isNanoAvailable(NanoConfig.from(nano)) - else -> true - } - promise.resolve(available) - } catch (_: Exception) { - promise.resolve(false) - } - } - } - - override fun isNanoSupported(nano: ReadableMap?, promise: Promise) { - scope.launch { - try { - promise.resolve(runner.isNanoSupported(NanoConfig.from(nano))) - } catch (_: Exception) { - promise.resolve(false) - } - } - } - override fun getModelInfo(modelType: String, nano: ReadableMap?, promise: Promise) { scope.launch { try { diff --git a/packages/adk/src/NativeAdkEngine.ts b/packages/adk/src/NativeAdkEngine.ts index fa8e5f479..ba7129242 100644 --- a/packages/adk/src/NativeAdkEngine.ts +++ b/packages/adk/src/NativeAdkEngine.ts @@ -146,11 +146,6 @@ export interface NanoDownloadProgressEvent { } export interface Spec extends TurboModule { - isAvailable( - modelType: AdkModelType, - nano?: AdkNanoModelConfig - ): Promise - isNanoSupported(nano?: AdkNanoModelConfig): Promise getModelInfo( modelType: AdkModelType, nano?: AdkNanoModelConfig diff --git a/packages/adk/src/__tests__/ai-sdk.test.ts b/packages/adk/src/__tests__/ai-sdk.test.ts index 4ec320d58..27875bf26 100644 --- a/packages/adk/src/__tests__/ai-sdk.test.ts +++ b/packages/adk/src/__tests__/ai-sdk.test.ts @@ -66,8 +66,6 @@ mock.module('react-native', () => ({ prepareNano, cancelStream, getModelInfo, - isNanoSupported: async () => true, - isAvailable: async () => true, onToolCall, submitToolResult, onStreamUpdate, @@ -137,36 +135,58 @@ describe('model resolution', () => { expect(nano.modelId).toBe('gemini-nano') }) - test('defaults cloud Gemini to gemini-2.5-flash', () => { - expect(gemini.modelId).toBe('gemini-2.5-flash') + test('defaults cloud Gemini to gemini-3.8-flash', () => { + expect(gemini.modelId).toBe('gemini-3.8-flash') + }) + + test('takes the model id as a model option', () => { + const provider = createAdkProvider({ model: 'gemini-3.5-flash-lite' }) + + expect(provider().modelId).toBe('gemini-3.5-flash-lite') + expect(provider({ modelName: 'gemini-nano' }).modelId).toBe('gemini-nano') }) test('infers cloud Gemini from a model name', async () => { - const model = createAdkProvider()('gemini-2.5-pro') + const model = createAdkProvider()('gemini-4-argon') await model.doGenerate(createCallOptions()) expect(lastGenerateCall()[2]).toMatchObject({ - model: { type: 'gemini', name: 'gemini-2.5-pro' }, + model: { type: 'gemini', name: 'gemini-4-argon' }, }) }) test('per-model options override provider defaults', () => { const provider = createAdkProvider({ instruction: 'Be brief.' }) - expect(provider({ modelName: 'gemini-2.5-pro' }).modelId).toBe( - 'gemini-2.5-pro' - ) + expect(provider({ model: 'gemini-4-argon' }).modelId).toBe('gemini-4-argon') expect(provider.languageModel().modelId).toBe('gemini-nano') }) test('reports model availability', async () => { expect(await createAdkProvider().getModelInfo()).toEqual({ + model: 'gemini-nano', modelType: 'genai-nano', - modelName: 'gemini-nano', availability: 'downloadable', + supportsTokenCounting: true, baseModelName: undefined, contextSize: undefined, }) }) + + test('reports availability for another model', async () => { + await createAdkProvider().getModelInfo({ model: 'gemini-3.8-flash' }) + + expect(getModelInfo.mock.calls.at(-1)![0]).toBe('gemini') + }) + + test('derives isAvailable from the model info', async () => { + const provider = createAdkProvider() + expect(await provider.isAvailable()).toBe(true) + + getModelInfo.mockImplementationOnce(async () => ({ + availability: 'downloading', + })) + expect(await provider.isAvailable()).toBe(false) + }) }) describe('prompt conversion', () => { @@ -692,7 +712,7 @@ describe('Gemini Nano preparation', () => { onNanoDownloadProgress.emit({ phase: 'completed' }) }) - await provider.prepareNano({ onProgress: (event) => progress.push(event) }) + await provider().prepare({ onProgress: (event) => progress.push(event) }) expect(progress).toEqual([ { phase: 'started', bytesDownloaded: undefined, bytesToDownload: 100 }, @@ -704,6 +724,18 @@ describe('Gemini Nano preparation', () => { ]) }) + test('keeps the deprecated prepareNano', async () => { + await createAdkProvider({ model: 'gemini-3.8-flash' }).prepareNano() + + expect(prepareNano).toHaveBeenCalledTimes(1) + }) + + test('does not prepare cloud models', async () => { + await gemini.prepare() + + expect(prepareNano).not.toHaveBeenCalled() + }) + test('passes the model selection to native code', async () => { const provider = createAdkProvider({ nano: { releaseStage: 'preview' } }) await provider().prepare() diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index 431f305cc..b32e26138 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -40,7 +40,7 @@ import { import { StreamTextFilter, stripJsonFences } from './streamText' const DEFAULT_NANO_MODEL = 'gemini-nano' -const DEFAULT_GEMINI_MODEL = 'gemini-2.5-flash' +const DEFAULT_GEMINI_MODEL = 'gemini-3.8-flash' export type AdkToolDefinitionSet = Record @@ -48,9 +48,11 @@ export type AdkToolDefinitionSet = Record export type AdkLanguageModelId = 'gemini-nano' | (string & {}) export interface AdkLanguageModelOptions { - /** Defaults to `genai-nano`, or `gemini` when `modelName` names a cloud model */ + /** `gemini-nano` (default) runs on device, any other id is a cloud Gemini model */ + model?: AdkLanguageModelId + /** @deprecated Use `model`. Without a model, `gemini` uses `gemini-3.8-flash` */ modelType?: AdkModelType - /** Defaults to `gemini-nano` on device and `gemini-2.5-flash` in the cloud */ + /** @deprecated Use `model` */ modelName?: AdkLanguageModelId /** ADK agent name */ name?: string @@ -82,9 +84,11 @@ export type AdkModelAvailability = | 'unknown' export interface AdkModelInfo { + model: AdkLanguageModelId modelType: AdkModelType - modelName: string availability: AdkModelAvailability + /** Whether `countTokens` works, which only Gemini Nano supports */ + supportsTokenCounting: boolean /** Name of the Gemini Nano base model, when ML Kit reports it */ baseModelName?: string /** Gemini Nano input token limit, when ML Kit reports it */ @@ -102,8 +106,8 @@ export interface AdkPrepareOptions { } export interface AdkLanguageModel extends LanguageModelV4 { - /** Downloads and initializes Gemini Nano. No-op for cloud models */ - prepare: () => Promise + /** Downloads and initializes Gemini Nano, reporting progress. No-op for cloud models */ + prepare: (options?: AdkPrepareOptions) => Promise updateTools: (tools: AdkToolDefinitionSet) => void /** Counts tokens with the Gemini Nano tokenizer. Not available for cloud models */ countTokens: (text: string) => Promise @@ -115,47 +119,54 @@ export interface AdkProvider { languageModel: ( options?: AdkLanguageModelId | AdkLanguageModelOptions ) => AdkLanguageModel - /** Whether the device can run Gemini Nano at all. Returns `false` on non-Android */ - isNanoSupported: () => Promise /** - * Whether `modelType` can be used now. For `genai-nano`, Gemini Nano must be - * available or downloadable. Cloud `gemini` resolves to `true` on Android + * Whether the provider's model, or the default model of `modelType`, can be + * used now: its availability is `available` or `downloadable` */ isAvailable: (modelType?: AdkModelType) => Promise - /** Availability details for a model type */ + /** Availability details for the provider's model, or for `model` */ getModelInfo: (options?: { - modelType?: AdkModelType + model?: AdkLanguageModelId nano?: AdkNanoModelConfig }) => Promise - /** Downloads and initializes Gemini Nano, reporting download progress */ + /** @deprecated Check `(await getModelInfo()).availability` instead */ + isNanoSupported: () => Promise + /** @deprecated Use `prepare()` on a Gemini Nano model */ prepareNano: (options?: AdkPrepareOptions) => Promise } type ResolvedModelOptions = Omit< AdkLanguageModelOptions, - 'modelType' | 'modelName' + 'model' | 'modelType' | 'modelName' > & { + model: string modelType: AdkModelType - modelName: string } function resolveModelOptions( defaults: AdkLanguageModelOptions, options: AdkLanguageModelId | AdkLanguageModelOptions = {} ): ResolvedModelOptions { - const overrides = - typeof options === 'string' ? { modelName: options } : options - const merged = { ...defaults, ...overrides } + const overrides = typeof options === 'string' ? { model: options } : options + const { + model: defaultModel, + modelName: defaultModelName, + ...defaultRest + } = defaults + const { model, modelName, ...rest } = overrides + const merged = { ...defaultRest, ...rest } + const id = model ?? modelName ?? defaultModel ?? defaultModelName const modelType = merged.modelType ?? - (merged.modelName && merged.modelName !== DEFAULT_NANO_MODEL - ? 'gemini' - : 'genai-nano') - const modelName = - merged.modelName ?? - (modelType === 'gemini' ? DEFAULT_GEMINI_MODEL : DEFAULT_NANO_MODEL) - - return { ...merged, modelType, modelName } + (id && id !== DEFAULT_NANO_MODEL ? 'gemini' : 'genai-nano') + + return { + ...merged, + modelType, + model: + id ?? + (modelType === 'gemini' ? DEFAULT_GEMINI_MODEL : DEFAULT_NANO_MODEL), + } } const toModelAvailability = (value: string): AdkModelAvailability => { @@ -170,27 +181,48 @@ const toModelAvailability = (value: string): AdkModelAvailability => { } } -async function prepareNano( - nano: AdkNanoModelConfig | undefined, - options: AdkPrepareOptions = {} -) { - const native = getNativeAdkEngine() - const listener = options.onProgress - ? native.onNanoDownloadProgress((event: NanoDownloadProgressEvent) => { - options.onProgress?.({ - phase: event.phase as AdkDownloadProgress['phase'], - bytesDownloaded: event.bytesDownloaded, - bytesToDownload: event.bytesToDownload, - }) - }) +const listenForDownloadProgress = ( + onProgress: AdkPrepareOptions['onProgress'] +) => + onProgress + ? getNativeAdkEngine().onNanoDownloadProgress( + (event: NanoDownloadProgressEvent) => { + onProgress({ + phase: event.phase as AdkDownloadProgress['phase'], + bytesDownloaded: event.bytesDownloaded, + bytesToDownload: event.bytesToDownload, + }) + } + ) : null - try { - await native.prepareNano(nano) - } catch (error) { - throw toAdkError(error) - } finally { - listener?.remove() +async function getModelInfo( + { model, modelType }: ResolvedModelOptions, + nano: AdkNanoModelConfig | undefined +): Promise { + const supportsTokenCounting = modelType === 'genai-nano' + + if (Platform.OS !== 'android') { + return { + model, + modelType, + availability: 'unavailable', + supportsTokenCounting, + } + } + + const info = await getNativeAdkEngine() + .getModelInfo(modelType, nano) + .catch((error: unknown) => { + throw toAdkError(error) + }) + return { + model, + modelType, + availability: toModelAvailability(info.availability), + supportsTokenCounting, + baseModelName: info.baseModelName, + contextSize: info.contextSize, } } @@ -209,49 +241,34 @@ export function createAdkProvider( } provider.specificationVersion = 'v4' as const provider.languageModel = createLanguageModel - provider.isNanoSupported = async () => { - if (Platform.OS !== 'android') { - return false - } - return getNativeAdkEngine().isNanoSupported(defaults.nano) + provider.isAvailable = async (modelType?: AdkModelType) => { + const resolved = modelType + ? resolveModelOptions({}, { modelType }) + : resolveModelOptions(defaults) + const { availability } = await getModelInfo(resolved, defaults.nano) + return availability === 'available' || availability === 'downloadable' } - provider.isAvailable = async (modelType: AdkModelType = 'genai-nano') => { - if (Platform.OS !== 'android') { - return false - } - return getNativeAdkEngine() - .isAvailable(modelType, defaults.nano) - .then(Boolean) - } - provider.getModelInfo = async ( - options: { modelType?: AdkModelType; nano?: AdkNanoModelConfig } = {} - ): Promise => { - const resolved = resolveModelOptions(defaults) - const { modelType, modelName } = - options.modelType && options.modelType !== resolved.modelType - ? resolveModelOptions({ modelType: options.modelType }) - : resolved - const nano = options.nano ?? defaults.nano - - if (Platform.OS !== 'android') { - return { modelType, modelName, availability: 'unavailable' } - } - - const info = await getNativeAdkEngine() - .getModelInfo(modelType, nano) - .catch((error: unknown) => { - throw toAdkError(error) - }) - return { - modelType, - modelName, - availability: toModelAvailability(info.availability), - baseModelName: info.baseModelName, - contextSize: info.contextSize, - } + provider.getModelInfo = ( + options: { model?: AdkLanguageModelId; nano?: AdkNanoModelConfig } = {} + ) => + getModelInfo( + options.model + ? resolveModelOptions({}, options.model) + : resolveModelOptions(defaults), + options.nano ?? defaults.nano + ) + provider.isNanoSupported = async () => { + const { availability } = await getModelInfo( + resolveModelOptions({}, DEFAULT_NANO_MODEL), + defaults.nano + ) + return availability !== 'unavailable' && availability !== 'unknown' } provider.prepareNano = (options?: AdkPrepareOptions) => - prepareNano(defaults.nano, options) + createLanguageModel({ + model: DEFAULT_NANO_MODEL, + modelType: 'genai-nano', + }).prepare(options) return provider } @@ -468,14 +485,14 @@ class AdkChatLanguageModel implements AdkLanguageModel { private nanoPreparePromise: Promise | null = null constructor(options: ResolvedModelOptions) { - this.modelId = options.modelName + this.modelId = options.model this.agentConfig = { name: options.name ?? 'react_native_adk_agent', description: options.description ?? 'React Native ADK agent', instruction: options.instruction, model: { type: options.modelType, - name: options.modelName, + name: options.model, apiKey: options.apiKey, nano: options.nano, }, @@ -491,8 +508,17 @@ class AdkChatLanguageModel implements AdkLanguageModel { this.tools = tools } - async prepare(): Promise { - await this.ensureNanoPrepared() + async prepare(options: AdkPrepareOptions = {}): Promise { + if (!this.isNano) { + return + } + + const listener = listenForDownloadProgress(options.onProgress) + try { + await this.ensureNanoPrepared() + } finally { + listener?.remove() + } } async countTokens(text: string): Promise { @@ -519,12 +545,12 @@ class AdkChatLanguageModel implements AdkLanguageModel { } if (!this.nanoPreparePromise) { - this.nanoPreparePromise = prepareNano(this.agentConfig.model.nano).catch( - (error) => { + this.nanoPreparePromise = getNativeAdkEngine() + .prepareNano(this.agentConfig.model.nano) + .catch((error: unknown) => { this.nanoPreparePromise = null - throw error - } - ) + throw toAdkError(error) + }) } return this.nanoPreparePromise From 4c0ca370ef06438dd5de2e678b6e89954c7e496c Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:56:33 +0400 Subject: [PATCH 12/18] refactor(expo-example): use the adk model option and model.prepare Co-Authored-By: Claude Opus 5.5 --- .../components/adapters/adkSetupAdapter.ts | 23 +++++++++---------- .../src/screens/ChatScreen/ModelItem.tsx | 2 +- 2 files changed, 12 insertions(+), 13 deletions(-) diff --git a/apps/expo-example/src/components/adapters/adkSetupAdapter.ts b/apps/expo-example/src/components/adapters/adkSetupAdapter.ts index 815e5e9e4..13c0ee4c2 100644 --- a/apps/expo-example/src/components/adapters/adkSetupAdapter.ts +++ b/apps/expo-example/src/components/adapters/adkSetupAdapter.ts @@ -1,13 +1,15 @@ import type { LanguageModelV4 } from '@ai-sdk/provider' import type { Tool } from '@ai-sdk/provider-utils' -import { type AdkModelType, createAdkProvider } from '@react-native-ai/adk' +import { + type AdkLanguageModelId, + createAdkProvider, +} from '@react-native-ai/adk' import { getGoogleApiKey } from '../../config/adk' import type { Availability, SetupAdapter } from '../../config/providers.common' type AdkAdapterOptions = { - modelType: AdkModelType - modelName: string + model: AdkLanguageModelId label: string accentColor: string icon: string @@ -20,8 +22,7 @@ export const createAdkLanguageSetupAdapter = ( tools: Record ): SetupAdapter => { const provider = createAdkProvider({ - modelType: options.modelType, - modelName: options.modelName, + model: options.model, apiKey: options.apiKey, instruction: 'You are a helpful assistant in the Callstack React Native AI demo app.', @@ -31,7 +32,7 @@ export const createAdkLanguageSetupAdapter = ( return { model, - modelId: `adk-${options.modelName}`, + modelId: `adk-${options.model}`, display: { label: options.label, accentColor: options.accentColor, @@ -40,14 +41,14 @@ export const createAdkLanguageSetupAdapter = ( builtIn: options.builtIn, imageInput: true, isAvailable(): Availability { - if (options.modelType === 'gemini' && !options.apiKey) { + if (options.model !== 'gemini-nano' && !options.apiKey) { return 'no' } return 'yes' }, async download(onProgress) { let bytesToDownload: number | undefined - await provider.prepareNano({ + await model.prepare({ onProgress: (progress) => { bytesToDownload = progress.bytesToDownload ?? bytesToDownload if (progress.phase === 'completed') { @@ -71,8 +72,7 @@ export const createAdkLanguageSetupAdapter = ( export const createAdkCloudSetupAdapter = (tools: Record) => createAdkLanguageSetupAdapter( { - modelType: 'gemini', - modelName: 'gemini-2.5-flash', + model: 'gemini-3.8-flash', label: 'ADK Gemini Flash', accentColor: '#4285F4', icon: 'psychology', @@ -85,8 +85,7 @@ export const createAdkCloudSetupAdapter = (tools: Record) => export const createAdkNanoSetupAdapter = (tools: Record) => createAdkLanguageSetupAdapter( { - modelType: 'genai-nano', - modelName: 'gemini-nano', + model: 'gemini-nano', label: 'ADK Gemini Nano', accentColor: '#34A853', icon: 'memory', diff --git a/apps/expo-example/src/screens/ChatScreen/ModelItem.tsx b/apps/expo-example/src/screens/ChatScreen/ModelItem.tsx index 91db2e78b..3f50cbe55 100644 --- a/apps/expo-example/src/screens/ChatScreen/ModelItem.tsx +++ b/apps/expo-example/src/screens/ChatScreen/ModelItem.tsx @@ -118,7 +118,7 @@ export function ModelItem({ : 'Downloaded' : selectedModelAvailability === 'availableForDownload' ? 'Tap to download' - : adapter.modelId === 'adk-gemini-2.5-flash' && + : adapter.modelId === 'adk-gemini-3.8-flash' && !hasGoogleApiKey() ? 'Set EXPO_PUBLIC_GOOGLE_API_KEY' : adapter.modelId === 'adk-gemini-nano' From 59fdd8f21254a153aaa19bde1255cc3280434b8d Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Mon, 26 Oct 2026 00:57:20 +0400 Subject: [PATCH 13/18] docs(adk): document the model option, prepare progress and availability Co-Authored-By: Claude Opus 5.5 --- website/src/docs/adk/generating.md | 9 ++- website/src/docs/adk/getting-started.mdx | 87 +++++++++++------------- 2 files changed, 43 insertions(+), 53 deletions(-) diff --git a/website/src/docs/adk/generating.md b/website/src/docs/adk/generating.md index eb35f630a..4032afbd0 100644 --- a/website/src/docs/adk/generating.md +++ b/website/src/docs/adk/generating.md @@ -19,19 +19,18 @@ const { text } = await generateText({ }) ``` -The first call prepares Gemini Nano. To prepare it ahead of time, call `adk.prepareNano()` (see [Getting Started - Preparation](./getting-started.mdx#preparation)). +The first call prepares Gemini Nano. To prepare it ahead of time, call `model.prepare()` (see [Getting Started - Preparation](./getting-started.mdx#preparation)). ### Cloud Gemini -Cloud models use ADK's `LlmAgent` with the Gemini API. They are useful when Gemini Nano is unavailable, when you need a more capable model, or when you develop on an emulator without AICore. Create a provider with `modelType: 'gemini'`, a `modelName`, and an API key: +Cloud models use ADK's `LlmAgent` with the Gemini API. They are useful when Gemini Nano is unavailable, when you need a more capable model, or when you develop on an emulator without AICore. Create a provider with a Gemini API model id and an API key: ```typescript import { createAdkProvider } from '@react-native-ai/adk' import { generateText } from 'ai' const adk = createAdkProvider({ - modelType: 'gemini', - modelName: 'gemini-2.5-flash', + model: 'gemini-3.8-flash', apiKey: process.env.GOOGLE_API_KEY, }) @@ -91,7 +90,7 @@ Set the AI SDK `reasoning` option to turn on the model's thinking. The thoughts import { createAdkProvider } from '@react-native-ai/adk' import { generateText } from 'ai' -const adk = createAdkProvider({ modelName: 'gemini-2.5-flash', apiKey }) +const adk = createAdkProvider({ model: 'gemini-3.8-flash', apiKey }) const { reasoningText, text } = await generateText({ model: adk(), diff --git a/website/src/docs/adk/getting-started.mdx b/website/src/docs/adk/getting-started.mdx index f820797b3..2f3653a1e 100644 --- a/website/src/docs/adk/getting-started.mdx +++ b/website/src/docs/adk/getting-started.mdx @@ -73,14 +73,14 @@ android { } ``` -## Available Model Types +## Available Models -| Model Type | `modelType` | Default `modelName` | Use Case | -| --------------------- | ------------ | ------------------- | ------------------------------------------------------------------ | -| On-device Gemini Nano | `genai-nano` | `gemini-nano` | Private, offline-capable inference - system-provisioned via AICore | -| Cloud Gemini | `gemini` | `gemini-2.5-flash` | Cloud inference via the Gemini API | +| Model | `model` | Use Case | +| --------------------- | ----------------------- | ------------------------------------------------------------------ | +| On-device Gemini Nano | `gemini-nano` (default) | Private, offline-capable inference - system-provisioned via AICore | +| Cloud Gemini | Any Gemini API model id | Cloud inference via the Gemini API, such as `gemini-3.8-flash` | -When you set `modelName` to anything other than `gemini-nano`, `modelType` defaults to `gemini`. +Any model id other than `gemini-nano` runs in the cloud. ## Basic Usage @@ -96,18 +96,18 @@ const { text } = await generateText({ }) ``` -Pass a model name or options to `adk()` to pick a model per call site, like `apple()`: +Pass a model id or options to `adk()` to pick a model per call site, like `apple()`: ```typescript const nano = adk() -const flash = adk({ modelName: 'gemini-2.5-flash', apiKey }) +const flash = adk({ model: 'gemini-3.8-flash', apiKey }) ``` Before you offer Gemini Nano in your UI, check that the device supports it and that it is ready. See [Availability](#availability) below. ## On-device Gemini Nano -Gemini Nano runs locally via ML Kit GenAI and is provisioned by the Android system through AICore. To set a system instruction, create a provider with `createAdkProvider`. ADK passes it to ML Kit as a native system instruction: +Gemini Nano runs locally via ML Kit GenAI and is provisioned by the Android system through AICore. To set a system instruction, create a provider with `createAdkProvider`. ADK passes it to ML Kit as a native system instruction. A `system` prompt passed to `generateText()` or `streamText()` is added to it: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -137,7 +137,7 @@ switch (info.availability) { // Ready to generate break case 'downloadable': - // Offer a download - adk.prepareNano() downloads it + // Offer a download - adk().prepare() downloads it break case 'downloading': // Show as unavailable until the download finishes @@ -152,51 +152,44 @@ switch (info.availability) { | `availability` | `FeatureStatus` | Meaning | | -------------- | ------------------ | -------------------------------------------- | | `available` | `AVAILABLE` (3) | Downloaded and ready | -| `downloadable` | `DOWNLOADABLE` (1) | Supported, `prepareNano()` downloads it | +| `downloadable` | `DOWNLOADABLE` (1) | Supported, `prepare()` downloads it | | `downloading` | `DOWNLOADING` (2) | The system is downloading it | | `unavailable` | `UNAVAILABLE` (0) | Not supported on this device, or not Android | | `unknown` | - | ML Kit could not report a status | -When Gemini Nano is `available`, `getModelInfo()` also returns `baseModelName` and `contextSize` (the input token limit). For cloud Gemini, `getModelInfo({ modelType: 'gemini' })` always reports `available` on Android. +`getModelInfo()` also returns the `model`, and `supportsTokenCounting`, which is only `true` for Gemini Nano. When Gemini Nano is `available`, it returns `baseModelName` and `contextSize` (the input token limit) too. Pass a `model` to check another model. Cloud Gemini always reports `available` on Android: -The provider also has two yes/no shortcuts: +```typescript +const info = await adk.getModelInfo({ model: 'gemini-3.8-flash' }) +``` -| API | Resolves to `true` when | -| ------------------------------- | ------------------------------------------------------------ | -| `adk.isNanoSupported()` | The device can run Gemini Nano, even if it is not downloaded | -| `adk.isAvailable('genai-nano')` | Gemini Nano is `available` or `downloadable` | +For a yes/no answer, `adk.isAvailable()` resolves to `true` when the provider's model is `available` or `downloadable`, like `apple.isAvailable()`. -All of these are cheap native calls that do not download the model. You can cache them at app startup and check again when the app resumes. +These are cheap native calls that do not download the model. You can cache them at app startup and check again when the app resumes. ### Preparation -`adk.prepareNano()` downloads and initializes Gemini Nano, and can report download progress: +`model.prepare()` downloads and initializes Gemini Nano, and can report download progress. `generateText()` and `streamText()` prepare the model automatically on first use: ```typescript import { adk } from '@react-native-ai/adk' +import { generateText } from 'ai' + +const model = adk() -await adk.prepareNano({ +// Prepare explicitly, for example while showing a download progress bar +await model.prepare({ onProgress: ({ phase, bytesDownloaded, bytesToDownload }) => { // phase is 'started', 'progress' or 'completed' }, }) -``` - -Calling `model.prepare()` on a model prepares it too, and `generateText()` and `streamText()` prepare the model automatically on first use: - -```typescript -import { adk } from '@react-native-ai/adk' -import { generateText } from 'ai' - -const model = adk() - -// Prepare explicitly, for example while showing a loading state -await model.prepare() // Or let the first generation prepare the model await generateText({ model, prompt: 'Hello!' }) ``` +Cloud models need no preparation, so `prepare()` returns at once. + Preparation fails with a `MODEL_UNAVAILABLE` error on unsupported devices and `MODEL_DOWNLOADING` when the system is still downloading. See [Errors](#errors). ### Model Selection @@ -216,7 +209,7 @@ const adk = createAdkProvider({ | `releaseStage` | `'stable'` (default), `'preview'` | Preview models need the AICore developer preview | | `preference` | `'fast'`, `'full'` | `fast` favors latency, `full` favors quality and capabilities | -Availability depends on the selected model, so `getModelInfo()`, `isAvailable()` and `prepareNano()` use the provider's `nano` setting. +Availability depends on the selected model, so `getModelInfo()` and `isAvailable()` use the provider's `nano` setting, and `prepare()` uses the model's. ### Token Counting @@ -231,7 +224,7 @@ Cloud models do not support token counting. ## Cloud Gemini -When you need cloud inference - for example on emulators or devices without AICore - create a provider with `modelType: 'gemini'` and an API key: +When you need cloud inference - for example on emulators or devices without AICore - create a provider with a Gemini API model id and an API key: ```typescript import { createAdkProvider } from '@react-native-ai/adk' @@ -239,8 +232,7 @@ import { generateText } from 'ai' const adk = createAdkProvider({ apiKey: process.env.GOOGLE_API_KEY, - modelType: 'gemini', - modelName: 'gemini-2.5-flash', + model: 'gemini-3.8-flash', instruction: 'You are a helpful assistant.', }) @@ -252,7 +244,7 @@ const { text } = await generateText({ Cloud models do not need a download or prepare step. ADK retries transient Gemini API failures. -> Do not embed API keys in production client apps. Prefer a backend proxy or secure runtime configuration. +> Do not embed API keys in production client apps. Every user of the app can read a key bundled with it. Prefer a backend proxy or secure runtime configuration. ## Provider Options @@ -270,16 +262,15 @@ const adk = createAdkProvider({ const model = adk.languageModel({ instruction: 'Answer in one word.' }) ``` -| Option | Type | Description | -| ---------------- | -------------------------- | --------------------------------------------------------------------------------------------- | -| `name` | `string` | ADK agent name (default: `react_native_adk_agent`) | -| `description` | `string` | Agent description passed to ADK | -| `instruction` | `string` | System instruction for the agent | -| `modelType` | `'genai-nano' \| 'gemini'` | On-device or cloud backend (default: `genai-nano`, or `gemini` for other model names) | -| `modelName` | `string` | Model identifier (default: `gemini-nano` on device, `gemini-2.5-flash` in the cloud) | -| `apiKey` | `string` | Gemini API key - required for `gemini` only | -| `availableTools` | `Record` | Tools whose `execute` handlers ADK can call. See [Tool Calling](./generating.md#tool-calling) | -| `nano` | `AdkNanoModelConfig` | Gemini Nano model selection. See [Model Selection](#model-selection) | +| Option | Type | Description | +| ---------------- | ---------------------- | --------------------------------------------------------------------------------------------- | +| `name` | `string` | ADK agent name (default: `react_native_adk_agent`) | +| `description` | `string` | Agent description passed to ADK | +| `instruction` | `string` | System instruction for the agent | +| `model` | `string` | Model id (default: `gemini-nano`). Any other id is a cloud Gemini model | +| `apiKey` | `string` | Gemini API key - required for cloud models only | +| `availableTools` | `Record` | Tools whose `execute` handlers ADK can call. See [Tool Calling](./generating.md#tool-calling) | +| `nano` | `AdkNanoModelConfig` | Gemini Nano model selection. See [Model Selection](#model-selection) | ## Errors @@ -289,7 +280,7 @@ Errors thrown by the provider have a `code` from `AdkErrorCodes`: import { AdkErrorCodes, type AdkError } from '@react-native-ai/adk' try { - await adk.prepareNano() + await adk().prepare() } catch (error) { if ((error as AdkError).code === AdkErrorCodes.ModelDownloading) { // Try again later From cfe0f09ac46804422d58b56f3fe9a518ab4a92e7 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Thu, 29 Oct 2026 21:28:09 +0100 Subject: [PATCH 14/18] fix(adk): send gemini nano the whole prompt as one text part Gemini Nano reads only the first text part of an ML Kit request. ADK's GenaiPrompt sends one content per turn, and a separate system instruction that devices without system prompt support put first, so Nano answered the agent instruction and never saw the user's message. Merge every turn into one text part, after the system instruction when the device does not support one. Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 4 +- .../callstack/ai/adk/NanoSingleTextPrompt.kt | 64 +++++++++++++++++++ 2 files changed, 67 insertions(+), 1 deletion(-) create mode 100644 packages/adk/android/src/main/java/com/callstack/ai/adk/NanoSingleTextPrompt.kt diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 91589ab8c..8df653880 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -331,7 +331,9 @@ class AdkAgentRunner( private suspend fun createModel(config: AgentConfig): Model { return when (config.modelType) { "genai-nano" -> - NanoFunctionCallingModel(GenaiPrompt.create(preparedNano(config.nano), config.modelName)) + NanoFunctionCallingModel( + GenaiPrompt.create(NanoSingleTextPrompt(preparedNano(config.nano)), config.modelName), + ) else -> Gemini(name = config.modelName, apiKey = config.apiKey) } } diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoSingleTextPrompt.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoSingleTextPrompt.kt new file mode 100644 index 000000000..271f4541f --- /dev/null +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoSingleTextPrompt.kt @@ -0,0 +1,64 @@ +package com.callstack.ai.adk + +import com.google.mlkit.genai.prompt.GenerateContentRequest +import com.google.mlkit.genai.prompt.GenerateContentResponse +import com.google.mlkit.genai.prompt.GenerativeModel +import com.google.mlkit.genai.prompt.ImagePart +import com.google.mlkit.genai.prompt.TextPart +import com.google.mlkit.genai.prompt.content +import kotlinx.coroutines.flow.Flow +import kotlinx.coroutines.flow.emitAll +import kotlinx.coroutines.flow.flow + +/** + * Wraps ML Kit's [GenerativeModel] so Gemini Nano sees the whole prompt. + * + * Gemini Nano reads only the first text part of a request. ADK's `GenaiPrompt` sends one content + * per turn, and a system instruction where the device has no system prompt support, so Nano + * answers the instruction and never sees the user's message. This merges the text of every turn + * into one part, after the system instruction when the device does not support one. Images stay + * as their own parts, before the text. + */ +internal class NanoSingleTextPrompt( + private val delegate: GenerativeModel, +) : GenerativeModel by delegate { + + @Volatile private var systemPromptAvailable: Boolean? = null + + override suspend fun generateContent(request: GenerateContentRequest): GenerateContentResponse = + delegate.generateContent(merge(request)) + + override fun generateContentStream(request: GenerateContentRequest): Flow = + flow { emitAll(delegate.generateContentStream(merge(request))) } + + private suspend fun merge(request: GenerateContentRequest): GenerateContentRequest { + val parts = request.contents.flatMap { it.parts } + val systemInstruction = request.systemInstruction?.takeUnless { supportsSystemPrompt() } + if (parts.size <= 1 && systemInstruction == null) { + return request + } + + val text = parts.filterIsInstance().joinToString("\n") { it.textString } + val images = parts.filterIsInstance() + val merged = TextPart(systemInstruction?.let { "${it.textString}\n\n$text" } ?: text) + return GenerateContentRequest.Builder(content { parts(*images.toTypedArray(), merged) }) + .apply { + if (systemInstruction == null) this.systemInstruction = request.systemInstruction + temperature = request.temperature + seed = request.seed + topK = request.topK + candidateCount = request.candidateCount + maxOutputTokens = request.maxOutputTokens + enableThinking = request.enableThinking + promptPrefix = request.promptPrefix + cachedContextName = request.cachedContextName + } + .build() + } + + private suspend fun supportsSystemPrompt(): Boolean = + systemPromptAvailable + ?: runCatching { delegate.isSystemPromptAvailable() } + .getOrDefault(false) + .also { systemPromptAvailable = it } +} From 5df379176f442e4af874186716148efdee0ec86c Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Thu, 29 Oct 2026 21:28:10 +0100 Subject: [PATCH 15/18] fix(adk): tell gemini nano how to call tools and accept positional arguments The instruction listed the tools but not the call format, so Nano wrote free-form Python the parser could not read. Describe the tool_code format, and map positional arguments, which Nano still often uses, to the parameters in the order the tool declares them. Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 6 +- .../ai/adk/NanoFunctionCallingModel.kt | 67 +++++++++++++------ 2 files changed, 51 insertions(+), 22 deletions(-) diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 8df653880..790565389 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -489,7 +489,11 @@ class AdkAgentRunner( val toolGuidance = "You have access to the following tools. Use them when they help answer the user:\n" + - tools.joinToString("\n") { it.promptDescription() } + tools.joinToString("\n") { it.promptDescription() } + + "\n\nTo call a tool, reply with only a tool_code block with one call per line. " + + "Pass the arguments as a JSON object, and call a tool without parameters with " + + "empty parentheses. For example:\n" + + "```tool_code\ntool_name({\"parameter\": \"value\"})\n```" return if (baseInstruction.isNullOrBlank()) { toolGuidance diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt index 7ae69bf2f..1df9205ec 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt @@ -10,6 +10,7 @@ import com.google.adk.kt.types.Part import com.google.adk.kt.types.Role import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.map +import org.json.JSONArray import org.json.JSONException import org.json.JSONObject import java.util.UUID @@ -30,10 +31,18 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod override val name: String get() = delegate.name - override fun generateContent(request: LlmRequest, stream: Boolean): Flow = - delegate.generateContent(prepareRequest(request), stream).map { response -> - if (response.partial) response else parseToolCodeBlocks(response) + override fun generateContent(request: LlmRequest, stream: Boolean): Flow { + val parameterNames = parameterNames(request) + return delegate.generateContent(prepareRequest(request), stream).map { response -> + if (response.partial) response else parseToolCodeBlocks(response, parameterNames) } + } + + /** Parameter names of each tool, in declaration order, to map positional arguments */ + private fun parameterNames(request: LlmRequest): Map> = + request.config?.tools.orEmpty() + .flatMap { it.functionDeclarations.orEmpty() } + .associate { it.name to it.parameters?.properties?.keys?.toList().orEmpty() } /** * Request preparation: convert FunctionCall/FunctionResponse parts to text @@ -72,7 +81,10 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod /** * Response post-processing: convert ```tool_code blocks to FunctionCall parts */ - private fun parseToolCodeBlocks(response: LlmResponse): LlmResponse { + private fun parseToolCodeBlocks( + response: LlmResponse, + parameterNames: Map>, + ): LlmResponse { val content = response.content ?: return response val newParts = mutableListOf() var hasToolCalls = false @@ -84,7 +96,7 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod continue } - val calls = extractToolCalls(text) + val calls = extractToolCalls(text, parameterNames) if (calls.isEmpty()) { newParts.add(part) continue @@ -104,7 +116,10 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod return response.copy(content = Content(role = Role.MODEL, parts = newParts)) } - private fun extractToolCalls(text: String): List { + private fun extractToolCalls( + text: String, + parameterNames: Map>, + ): List { val calls = mutableListOf() val blockRegex = Regex("```tool_code\\s*\\n([\\s\\S]*?)\\n?```") for (block in blockRegex.findAll(text)) { @@ -112,7 +127,7 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod for (line in code.lines()) { val trimmed = line.trim() if (trimmed.isNotEmpty()) { - parseFunctionCall(trimmed)?.let { calls.add(it) } + parseFunctionCall(trimmed, parameterNames)?.let { calls.add(it) } } } } @@ -126,9 +141,13 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod * Parses a single call line such as: * - `functionName()` * - `functionName({"key": "value"})` + * - `functionName("value")` (positional, in the order the tool declares its parameters) * - `print(functionName(...))` (some models wrap in print) */ - private fun parseFunctionCall(code: String): FunctionCall? { + private fun parseFunctionCall( + code: String, + parameterNames: Map>, + ): FunctionCall? { // Regex: outer function name + everything between the outermost parens (greedy) val callRegex = Regex("""^([A-Za-z_][A-Za-z0-9_]*)\s*\(([\s\S]*)\)\s*$""") val outer = callRegex.find(code) ?: return null @@ -145,19 +164,25 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod if (name.isBlank()) return null - val args: Map = - if (argsStr.isBlank()) { - emptyMap() - } else { - val jsonStr = argsStr.trim().let { if (it.startsWith("{")) it else "{$it}" } - try { - val jsonObj = JSONObject(jsonStr) - jsonObj.keys().asSequence().associateWith { key -> jsonObj.get(key) as Any } - } catch (_: JSONException) { - emptyMap() - } - } - + val args = parseArguments(argsStr.trim(), parameterNames[name].orEmpty()) return FunctionCall(name = name, args = args, id = UUID.randomUUID().toString()) } + + private fun parseArguments(argsStr: String, parameterNames: List): Map { + if (argsStr.isEmpty()) return emptyMap() + + val jsonStr = if (argsStr.startsWith("{")) argsStr else "{$argsStr}" + try { + val jsonObj = JSONObject(jsonStr) + return jsonObj.keys().asSequence().associateWith { key -> jsonObj.get(key) as Any } + } catch (_: JSONException) {} + + // Gemini Nano often passes the arguments positionally + return try { + val values = JSONArray("[$argsStr]") + parameterNames.zip((0 until values.length()).map { values.get(it) }).toMap() + } catch (_: JSONException) { + emptyMap() + } + } } From 2e11fb9c45e4d9295e2087d35f12e90d2fe4a273 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Thu, 29 Oct 2026 21:28:11 +0100 Subject: [PATCH 16/18] fix(adk): strip the role markers gemini nano repeats ADK labels Nano's history with [user]: and [model]: markers. Nano can start its reply with one, or write the user's next turn, so strip the leading marker and end the text at a user marker, in generated and streamed text. Co-Authored-By: Claude Opus 5.5 --- packages/adk/src/__tests__/streamText.test.ts | 30 ++++++++++++- packages/adk/src/ai-sdk.ts | 14 ++++-- packages/adk/src/streamText.ts | 44 +++++++++++++++++-- 3 files changed, 80 insertions(+), 8 deletions(-) diff --git a/packages/adk/src/__tests__/streamText.test.ts b/packages/adk/src/__tests__/streamText.test.ts index 6714bc811..96055894c 100644 --- a/packages/adk/src/__tests__/streamText.test.ts +++ b/packages/adk/src/__tests__/streamText.test.ts @@ -1,14 +1,15 @@ import { describe, expect, test } from 'bun:test' -import { StreamTextFilter } from '../streamText' +import { StreamTextFilter, stripRoleMarkers } from '../streamText' function run( deltas: string[], - options: { json?: boolean; toolCode?: boolean } = {} + options: { json?: boolean; toolCode?: boolean; roleMarkers?: boolean } = {} ) { const filter = new StreamTextFilter({ json: options.json ?? false, toolCode: options.toolCode ?? false, + roleMarkers: options.roleMarkers ?? false, }) const shown = deltas.map((delta) => filter.push(delta)) return { shown, text: shown.join('') + filter.flush() } @@ -53,4 +54,29 @@ describe('StreamTextFilter', () => { run(['``', '`json\n{"a"', ':1}', '\n`', '``'], { json: true }).text ).toBe('{"a":1}') }) + + test('strips a model marker split across deltas', () => { + const { shown, text } = run(['[mo', 'del]: \nTeal', '.'], { + roleMarkers: true, + }) + + expect(shown).toEqual(['', 'Teal', '.']) + expect(text).toBe('Teal.') + }) + + test('ends the reply where the model writes the next user turn', () => { + const { text } = run(['Done.\n[us', 'er]: Thanks'], { roleMarkers: true }) + + expect(text).toBe('Done.') + }) + + test('shows text that only starts like a marker', () => { + expect(run(['[mod', 'e] on'], { roleMarkers: true }).text).toBe('[mode] on') + }) +}) + +describe('stripRoleMarkers', () => { + test('strips the model marker and later turns', () => { + expect(stripRoleMarkers('[model]: Teal\n[user]: Thanks')).toBe('Teal') + }) }) diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index b32e26138..b80f4ec43 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -37,7 +37,11 @@ import { type StreamToolCallEvent, type ToolCallEvent, } from './NativeAdkEngine' -import { StreamTextFilter, stripJsonFences } from './streamText' +import { + StreamTextFilter, + stripJsonFences, + stripRoleMarkers, +} from './streamText' const DEFAULT_NANO_MODEL = 'gemini-nano' const DEFAULT_GEMINI_MODEL = 'gemini-3.8-flash' @@ -872,12 +876,15 @@ class AdkChatLanguageModel implements AdkLanguageModel { result: parseToolResult(toolResult.output), }) } + const text = this.isNano + ? stripRoleMarkers(response.content) + : response.content content.push({ type: 'text', text: options.responseFormat?.type === 'json' - ? stripJsonFences(response.content) - : response.content, + ? stripJsonFences(text) + : text, }) return { @@ -956,6 +963,7 @@ class AdkChatLanguageModel implements AdkLanguageModel { const textFilter = new StreamTextFilter({ json: isJson, toolCode: this.isNano, + roleMarkers: this.isNano, }) const addListener = (listener: { remove(): void }) => { diff --git a/packages/adk/src/streamText.ts b/packages/adk/src/streamText.ts index 1dc20fcf6..9d781b0c9 100644 --- a/packages/adk/src/streamText.ts +++ b/packages/adk/src/streamText.ts @@ -9,6 +9,21 @@ const partialJsonFence = /^\s*`{0,3}(j(s(o(n)?)?)?)?$/ export const stripJsonFences = (text: string) => text.replace(jsonFencePrefix, '').replace(/\s*```\s*$/, '') +// ADK labels the turns of Gemini Nano's history with role markers. Nano can +// start its reply with one, or go on to write the user's next turn +const modelMarkerPrefix = /^\s*\[model\]:\s*/ +// Text that could still grow into the model marker +const partialModelMarker = /^\s*(\[(m(o(d(e(l(\](:)?)?)?)?)?)?)?)?$/ +const userMarker = '\n[user]:' + +export const stripRoleMarkers = (text: string) => { + const userIndex = text.indexOf(userMarker) + return (userIndex === -1 ? text : text.slice(0, userIndex)).replace( + modelMarkerPrefix, + '' + ) +} + // Length of the end of `text` that could be the start of `marker` function pendingPrefixLength(text: string, marker: string) { for ( @@ -24,14 +39,21 @@ function pendingPrefixLength(text: string, marker: string) { } /** - * Filters streamed model text. It hides Gemini Nano `tool_code` blocks and, in - * JSON mode, Markdown code fences, holding back text until it cannot become one. + * Filters streamed model text. It hides Gemini Nano `tool_code` blocks and + * role markers and, in JSON mode, Markdown code fences, holding back text + * until it cannot become one. */ export class StreamTextFilter { private turnText = '' private emitted = '' - constructor(private readonly options: { json: boolean; toolCode: boolean }) {} + constructor( + private readonly options: { + json: boolean + toolCode: boolean + roleMarkers?: boolean + } + ) {} /** Adds a text delta and returns the text that is now safe to show */ push(delta: string) { @@ -69,6 +91,14 @@ export class StreamTextFilter { text = text.slice(0, markerIndex) } + const userIndex = this.options.roleMarkers ? text.indexOf(userMarker) : -1 + if (this.options.roleMarkers) { + if (!final && partialModelMarker.test(text)) { + return undefined + } + text = stripRoleMarkers(text) + } + if (this.options.json) { if (final) { return stripJsonFences(text) @@ -87,6 +117,14 @@ export class StreamTextFilter { text.length - pendingPrefixLength(text, toolCodeMarker) ) } + if ( + this.options.roleMarkers && + markerIndex === -1 && + userIndex === -1 && + !final + ) { + text = text.slice(0, text.length - pendingPrefixLength(text, userMarker)) + } return text } } From 42ef0cd2e8d58130ea36ce5c76dcef0d862d633e Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sun, 25 Oct 2026 04:20:59 +0400 Subject: [PATCH 17/18] feat(adk): cache gemini nano prompt prefixes with ml kit ADK's GenaiPrompt builds ML Kit requests without a prompt prefix or cache name, so wrap ML Kit's GenerativeModel and rewrite each request: - providerOptions.adk.cacheSystemPrompt sends the system instruction as an implicitly cached PromptPrefix and folds leading system messages into the instruction - providerOptions.adk.cachedContent names an explicit ML Kit cache, or a cloud Gemini cachedContents resource through ADK - adk.caches creates, lists and deletes explicit caches and clears implicit ones ML Kit reports no usage, so cached prompt tokens are estimated from the prefix token count and reported as inputTokens.cacheRead. Builds on #268. Co-Authored-By: Claude Opus 5.5 --- .../com/callstack/ai/adk/AdkAgentRunner.kt | 48 ++++++- .../ai/adk/NanoFunctionCallingModel.kt | 16 ++- .../com/callstack/ai/adk/NanoPromptCache.kt | 112 ++++++++++++++++ .../callstack/ai/adk/NativeAdkEngineModule.kt | 43 ++++++ packages/adk/src/NativeAdkEngine.ts | 12 ++ packages/adk/src/__tests__/ai-sdk.test.ts | 126 ++++++++++++++++++ packages/adk/src/ai-sdk.ts | 65 +++++++++ 7 files changed, 414 insertions(+), 8 deletions(-) create mode 100644 packages/adk/android/src/main/java/com/callstack/ai/adk/NanoPromptCache.kt diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt index 790565389..966ef2ca4 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/AdkAgentRunner.kt @@ -30,7 +30,9 @@ import com.google.mlkit.genai.prompt.Generation import com.google.mlkit.genai.prompt.GenerativeModel import com.google.mlkit.genai.prompt.ModelPreference import com.google.mlkit.genai.prompt.ModelReleaseStage +import com.google.mlkit.genai.prompt.PromptPrefix import com.google.mlkit.genai.prompt.TextPart +import com.google.mlkit.genai.prompt.createCachedContextRequest import com.google.mlkit.genai.prompt.generateContentRequest import com.google.mlkit.genai.prompt.generationConfig import com.google.mlkit.genai.prompt.modelConfig @@ -96,6 +98,10 @@ class AdkAgentRunner( ) { private val nanoModels = ConcurrentHashMap() private val nanoPrepareMutex = Mutex() + private val nanoCacheStates = ConcurrentHashMap() + + private fun nanoCacheState(config: NanoConfig): NanoCacheState = + nanoCacheStates.getOrPut(config) { NanoCacheState() } private fun nanoClient(config: NanoConfig): GenerativeModel { if (config.releaseStage == null && config.preference == null) { @@ -201,6 +207,28 @@ class AdkAgentRunner( suspend fun countTokens(text: String, config: NanoConfig): Int = preparedNano(config).countTokens(generateContentRequest(TextPart(text)) {}).totalTokens + /** Processes [prefix] once and stores it as an explicit ML Kit cache named [name]. */ + suspend fun createCache(name: String, prefix: String, config: NanoConfig) { + val model = preparedNano(config) + val cache = model.caches.create(createCachedContextRequest(name, PromptPrefix(prefix))) + runCatching { countTokens(prefix, config) } + .getOrNull() + ?.let { nanoCacheState(config).cacheTokens[cache.name] = it } + } + + suspend fun listCaches(config: NanoConfig): List = + preparedNano(config).caches.list().map { it.name } + + suspend fun deleteCache(name: String, config: NanoConfig): Boolean { + nanoCacheState(config).cacheTokens.remove(name) + return preparedNano(config).caches.delete(name) + } + + suspend fun clearImplicitCaches(config: NanoConfig) { + preparedNano(config).clearImplicitCaches() + nanoCacheState(config).prefixTokens.clear() + } + suspend fun generateText( messages: ReadableArray, config: ReadableMap, @@ -275,7 +303,7 @@ class AdkAgentRunner( val parsedMessages = parseMessages(messages) val agentConfig = parseAgentConfig(config) val agentTools = parseTools(tools, runId) - val model = createModel(agentConfig) + val model = createModel(agentConfig, options) val generateContentConfig = parseGenerationConfig(options) val instructionText = if (agentConfig.modelType == "genai-nano") { @@ -328,12 +356,19 @@ class AdkAgentRunner( ) } - private suspend fun createModel(config: AgentConfig): Model { + private suspend fun createModel(config: AgentConfig, options: ReadableMap?): Model { return when (config.modelType) { - "genai-nano" -> - NanoFunctionCallingModel( - GenaiPrompt.create(NanoSingleTextPrompt(preparedNano(config.nano)), config.modelName), - ) + "genai-nano" -> { + val promptCache = + NanoPromptCache( + delegate = NanoSingleTextPrompt(preparedNano(config.nano)), + state = nanoCacheState(config.nano), + cachedContent = options?.getString("cachedContent"), + cacheSystemPrompt = + options?.hasKey("cacheSystemPrompt") == true && options.getBoolean("cacheSystemPrompt"), + ) + NanoFunctionCallingModel(GenaiPrompt.create(promptCache, config.modelName), promptCache) + } else -> Gemini(name = config.modelName, apiKey = config.apiKey) } } @@ -386,6 +421,7 @@ class AdkAgentRunner( responseMimeType = responseMimeType, responseSchema = responseSchema, thinkingConfig = thinkingConfig, + cachedContent = options.getString("cachedContent"), ) } diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt index 1df9205ec..a4522fdde 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoFunctionCallingModel.kt @@ -8,6 +8,7 @@ import com.google.adk.kt.types.Content import com.google.adk.kt.types.FunctionCall import com.google.adk.kt.types.Part import com.google.adk.kt.types.Role +import com.google.adk.kt.types.UsageMetadata import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.map import org.json.JSONArray @@ -26,15 +27,20 @@ import java.util.UUID * 2. After the final (non-partial) response: detects ```tool_code blocks in the text and converts * each invocation to a proper [FunctionCall] part, allowing ADK's agent loop to execute the * tools and continue the conversation. + * + * 3. Reports prompt tokens served from [promptCache] as usage, since ML Kit reports none. */ -internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Model { +internal class NanoFunctionCallingModel( + private val delegate: GenaiPrompt, + private val promptCache: NanoPromptCache, +) : Model { override val name: String get() = delegate.name override fun generateContent(request: LlmRequest, stream: Boolean): Flow { val parameterNames = parameterNames(request) return delegate.generateContent(prepareRequest(request), stream).map { response -> - if (response.partial) response else parseToolCodeBlocks(response, parameterNames) + if (response.partial) response else withCachedUsage(parseToolCodeBlocks(response, parameterNames)) } } @@ -44,6 +50,12 @@ internal class NanoFunctionCallingModel(private val delegate: GenaiPrompt) : Mod .flatMap { it.functionDeclarations.orEmpty() } .associate { it.name to it.parameters?.properties?.keys?.toList().orEmpty() } + private fun withCachedUsage(response: LlmResponse): LlmResponse { + val cachedTokens = promptCache.cachedTokenCount ?: return response + val usage = response.usageMetadata ?: UsageMetadata() + return response.copy(usageMetadata = usage.copy(cachedContentTokenCount = cachedTokens)) + } + /** * Request preparation: convert FunctionCall/FunctionResponse parts to text */ diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoPromptCache.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoPromptCache.kt new file mode 100644 index 000000000..51890bbbf --- /dev/null +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NanoPromptCache.kt @@ -0,0 +1,112 @@ +package com.callstack.ai.adk + +import com.google.mlkit.genai.prompt.GenerateContentRequest +import com.google.mlkit.genai.prompt.GenerateContentResponse +import com.google.mlkit.genai.prompt.GenerativeModel +import com.google.mlkit.genai.prompt.ImagePart +import com.google.mlkit.genai.prompt.PromptPrefix +import com.google.mlkit.genai.prompt.TextPart +import com.google.mlkit.genai.prompt.generateContentRequest +import kotlinx.coroutines.flow.Flow +import kotlinx.coroutines.flow.emitAll +import kotlinx.coroutines.flow.flow +import java.util.concurrent.ConcurrentHashMap + +/** Prompt prefix state that outlives a single run, kept per Gemini Nano client. */ +internal class NanoCacheState { + /** Token counts of prefixes ML Kit has already processed in this app session */ + val prefixTokens = ConcurrentHashMap() + + /** Token counts of explicit caches created in this app session */ + val cacheTokens = ConcurrentHashMap() + + @Volatile var cachingAvailable: Boolean? = null +} + +/** + * Wraps ML Kit's [GenerativeModel] so Gemini Nano reuses an already processed prompt prefix. + * + * ADK's `GenaiPrompt` builds ML Kit requests without a prompt prefix or cache name, so this + * rewrites each request before it reaches ML Kit: + * + * - [cachedContent] names an explicit cache created with `caches.create`. + * - [cacheSystemPrompt] sends the system instruction as a [PromptPrefix], which ML Kit caches + * implicitly. Prefix caching only supports text, so requests with images stay unchanged. + * + * ML Kit does not report usage, so [cachedTokenCount] is an estimate: the prefix token count + * when the prefix was processed before in this app session, or when this app session created + * the explicit cache. + */ +internal class NanoPromptCache( + private val delegate: GenerativeModel, + private val state: NanoCacheState, + private val cachedContent: String?, + private val cacheSystemPrompt: Boolean, +) : GenerativeModel by delegate { + + /** Prompt tokens likely served from the cache in the latest request */ + @Volatile var cachedTokenCount: Int? = null + private set + + override suspend fun generateContent(request: GenerateContentRequest): GenerateContentResponse = + delegate.generateContent(prepareRequest(request)) + + override fun generateContentStream(request: GenerateContentRequest): Flow = + flow { emitAll(delegate.generateContentStream(prepareRequest(request))) } + + private suspend fun prepareRequest(request: GenerateContentRequest): GenerateContentRequest { + cachedTokenCount = null + + if (cachedContent != null) { + cachedTokenCount = state.cacheTokens[cachedContent] + return request.rebuild(keepSystemInstruction = true) { cachedContextName = cachedContent } + } + + val prefix = request.systemInstruction?.textString + if ( + !cacheSystemPrompt || + prefix.isNullOrEmpty() || + request.contents.any { content -> content.parts.any { it is ImagePart } } || + !isCachingAvailable() + ) { + return request + } + + // The first request with a prefix is a cache miss + cachedTokenCount = state.prefixTokens[prefix] + if (cachedTokenCount == null) { + runCatching { countTokens(prefix) }.getOrNull()?.let { state.prefixTokens[prefix] = it } + } + + return request.rebuild(keepSystemInstruction = false) { promptPrefix = PromptPrefix(prefix) } + } + + private suspend fun isCachingAvailable(): Boolean = + state.cachingAvailable + ?: runCatching { delegate.isCachingFeatureAvailable() } + .getOrDefault(false) + .also { state.cachingAvailable = it } + + private suspend fun countTokens(text: String): Int = + delegate.countTokens(generateContentRequest(TextPart(text)) {}).totalTokens + + /** ML Kit requests have no copy builder, so copy every setting ADK may have set */ + private fun GenerateContentRequest.rebuild( + keepSystemInstruction: Boolean, + configure: GenerateContentRequest.Builder.() -> Unit, + ): GenerateContentRequest { + val source = this + return GenerateContentRequest.Builder(source.contents) + .apply { + if (keepSystemInstruction) systemInstruction = source.systemInstruction + temperature = source.temperature + seed = source.seed + topK = source.topK + candidateCount = source.candidateCount + maxOutputTokens = source.maxOutputTokens + enableThinking = source.enableThinking + configure() + } + .build() + } +} diff --git a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt index 715a36e4d..bdb7a7892 100644 --- a/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt +++ b/packages/adk/android/src/main/java/com/callstack/ai/adk/NativeAdkEngineModule.kt @@ -92,6 +92,49 @@ class NativeAdkEngineModule(reactContext: ReactApplicationContext) : } } + override fun createCache(name: String, prefix: String, nano: ReadableMap?, promise: Promise) { + scope.launch { + try { + runner.createCache(name, prefix, NanoConfig.from(nano)) + promise.resolve(null) + } catch (error: Exception) { + promise.rejectWithCode(error) + } + } + } + + override fun listCaches(nano: ReadableMap?, promise: Promise) { + scope.launch { + try { + val names = runner.listCaches(NanoConfig.from(nano)) + promise.resolve(Arguments.createArray().apply { names.forEach { pushString(it) } }) + } catch (error: Exception) { + promise.rejectWithCode(error) + } + } + } + + override fun deleteCache(name: String, nano: ReadableMap?, promise: Promise) { + scope.launch { + try { + promise.resolve(runner.deleteCache(name, NanoConfig.from(nano))) + } catch (error: Exception) { + promise.rejectWithCode(error) + } + } + } + + override fun clearImplicitCaches(nano: ReadableMap?, promise: Promise) { + scope.launch { + try { + runner.clearImplicitCaches(NanoConfig.from(nano)) + promise.resolve(null) + } catch (error: Exception) { + promise.rejectWithCode(error) + } + } + } + override fun generateText( runId: String, messages: ReadableArray, diff --git a/packages/adk/src/NativeAdkEngine.ts b/packages/adk/src/NativeAdkEngine.ts index ba7129242..1e52b2bc2 100644 --- a/packages/adk/src/NativeAdkEngine.ts +++ b/packages/adk/src/NativeAdkEngine.ts @@ -63,6 +63,10 @@ export interface AdkGenerationOptions { includeThoughts: boolean budget?: number } + /** Context cache to prefix the prompt with */ + cachedContent?: string + /** Gemini Nano: send the system instruction as an implicitly cached prompt prefix */ + cacheSystemPrompt?: boolean } export interface AdkUsageMetadata { @@ -152,6 +156,14 @@ export interface Spec extends TurboModule { ): Promise prepareNano(nano?: AdkNanoModelConfig): Promise countTokens(text: string, nano?: AdkNanoModelConfig): Promise + createCache( + name: string, + prefix: string, + nano?: AdkNanoModelConfig + ): Promise + listCaches(nano?: AdkNanoModelConfig): Promise + deleteCache(name: string, nano?: AdkNanoModelConfig): Promise + clearImplicitCaches(nano?: AdkNanoModelConfig): Promise generateText( runId: string, diff --git a/packages/adk/src/__tests__/ai-sdk.test.ts b/packages/adk/src/__tests__/ai-sdk.test.ts index 27875bf26..9f49c6922 100644 --- a/packages/adk/src/__tests__/ai-sdk.test.ts +++ b/packages/adk/src/__tests__/ai-sdk.test.ts @@ -56,6 +56,12 @@ const submitToolResult = mock( const getModelInfo = mock(async (_modelType: string, _nano?: unknown) => ({ availability: 'downloadable', })) +const createCache = mock( + async (_name: string, _prefix: string, _nano?: unknown) => {} +) +const listCaches = mock(async (_nano?: unknown) => ['faq']) +const deleteCache = mock(async (_name: string, _nano?: unknown) => true) +const clearImplicitCaches = mock(async (_nano?: unknown) => {}) mock.module('react-native', () => ({ Platform: { OS: 'android' }, @@ -68,6 +74,10 @@ mock.module('react-native', () => ({ getModelInfo, onToolCall, submitToolResult, + createCache, + listCaches, + deleteCache, + clearImplicitCaches, onStreamUpdate, onStreamToolCall, onStreamComplete, @@ -494,6 +504,122 @@ describe('generation options', () => { }) }) +describe('context caching', () => { + const systemPrompt = createCallOptions({ + prompt: [ + { role: 'system', content: 'You are a pirate.' }, + { role: 'user', content: [{ type: 'text', text: 'hello' }] }, + ], + providerOptions: { adk: { cacheSystemPrompt: true } }, + }) + + test('caches the Gemini Nano instruction with the system prompt', async () => { + const provider = createAdkProvider({ instruction: 'Be brief.' }) + await provider().doGenerate(systemPrompt) + + const [, messages, config, options] = lastGenerateCall() as [ + string, + { role: string }[], + { instruction: string }, + { cacheSystemPrompt?: boolean }, + ] + expect(config.instruction).toBe('Be brief.\n\nYou are a pirate.') + expect(messages.map((message) => message.role)).toEqual(['user']) + expect(options.cacheSystemPrompt).toBe(true) + }) + + test('passes cachedContent and drops cacheSystemPrompt with a warning', async () => { + const result = await nano.doGenerate({ + ...systemPrompt, + providerOptions: { + adk: { cachedContent: 'faq', cacheSystemPrompt: true }, + }, + }) + + const [, messages, , options] = lastGenerateCall() as [ + string, + unknown[], + unknown, + { cachedContent?: string; cacheSystemPrompt?: boolean }, + ] + expect(options.cachedContent).toBe('faq') + expect(options.cacheSystemPrompt).toBe(undefined) + expect(messages).toHaveLength(1) + expect(result.warnings).toEqual([ + { + type: 'other', + message: 'cacheSystemPrompt is ignored because cachedContent is set.', + }, + ]) + }) + + test('warns that cloud Gemini does not support cacheSystemPrompt', async () => { + const result = await gemini.doGenerate(systemPrompt) + + expect( + (lastGenerateCall()[3] as { cacheSystemPrompt?: boolean }) + .cacheSystemPrompt + ).toBe(undefined) + expect( + result.warnings.map((warning) => 'feature' in warning && warning.feature) + ).toEqual(['cacheSystemPrompt']) + }) + + test('reports cached prompt tokens as cache reads', async () => { + generateTextResult = { + content: 'Arr.', + finishReason: 'STOP', + usage: { cachedContentTokenCount: 40 }, + } + + const result = await nano.doGenerate(systemPrompt) + + expect(result.usage.inputTokens).toEqual({ + total: undefined, + noCache: undefined, + cacheRead: 40, + cacheWrite: undefined, + }) + }) + + test('manages Gemini Nano caches with the model selection', async () => { + const provider = createAdkProvider({ nano: { preference: 'fast' } }) + + await provider.caches.create('faq', 'Answer from this FAQ: ...') + expect(await provider.caches.list()).toEqual(['faq']) + expect(await provider.caches.delete('faq')).toBe(true) + await provider.caches.clearImplicit() + + expect(createCache.mock.calls.at(-1)).toEqual([ + 'faq', + 'Answer from this FAQ: ...', + { preference: 'fast' }, + ]) + expect(deleteCache.mock.calls.at(-1)).toEqual([ + 'faq', + { preference: 'fast' }, + ]) + expect(clearImplicitCaches).toHaveBeenCalledWith({ preference: 'fast' }) + }) + + test('wraps native cache errors', async () => { + createCache.mockImplementationOnce(async () => { + throw Object.assign( + new Error('Prefix Caching feature is not available.'), + { + code: 'MODEL_UNAVAILABLE', + } + ) + }) + + await expect( + createAdkProvider().caches.create('faq', 'prefix') + ).rejects.toMatchObject({ + code: 'MODEL_UNAVAILABLE', + }) + }) +}) + describe('doGenerate results', () => { test('returns reasoning, provider-executed tool steps and text', async () => { generateTextResult = { diff --git a/packages/adk/src/ai-sdk.ts b/packages/adk/src/ai-sdk.ts index b80f4ec43..561a1f766 100644 --- a/packages/adk/src/ai-sdk.ts +++ b/packages/adk/src/ai-sdk.ts @@ -78,6 +78,18 @@ export interface AdkProviderOptions { thinkingBudget?: number /** Whether the model's thoughts come back as reasoning parts */ includeThoughts?: boolean + /** + * Context cache that prefixes the prompt: a Gemini Nano cache created with + * `adk.caches.create`, or a cloud Gemini `cachedContents/...` resource + */ + cachedContent?: string + /** + * Gemini Nano: sends the system prompt as a prompt prefix that ML Kit caches, + * so later calls with the same system prompt skip reprocessing it. The + * prefix is the instruction, which includes `system` messages. Ignored for + * prompts with images + */ + cacheSystemPrompt?: boolean } export type AdkModelAvailability = @@ -137,6 +149,22 @@ export interface AdkProvider { isNanoSupported: () => Promise /** @deprecated Use `prepare()` on a Gemini Nano model */ prepareNano: (options?: AdkPrepareOptions) => Promise + /** Explicit Gemini Nano context caches, stored by ML Kit across app launches */ + caches: AdkNanoCaches +} + +export interface AdkNanoCaches { + /** + * Processes `prefix` once and stores it as `name`. Pass `name` as + * `providerOptions.adk.cachedContent` to put the prefix before the prompt + */ + create: (name: string, prefix: string) => Promise + /** Names of the stored caches */ + list: () => Promise + /** Resolves to `false` when no cache has this name */ + delete: (name: string) => Promise + /** Deletes the prefixes ML Kit cached for `cacheSystemPrompt` */ + clearImplicit: () => Promise } type ResolvedModelOptions = Omit< @@ -185,6 +213,11 @@ const toModelAvailability = (value: string): AdkModelAvailability => { } } +const withAdkErrors = (promise: Promise) => + promise.catch((error: unknown) => { + throw toAdkError(error) + }) + const listenForDownloadProgress = ( onProgress: AdkPrepareOptions['onProgress'] ) => @@ -273,6 +306,18 @@ export function createAdkProvider( model: DEFAULT_NANO_MODEL, modelType: 'genai-nano', }).prepare(options) + provider.caches = { + create: async (name: string, prefix: string) => + withAdkErrors( + getNativeAdkEngine().createCache(name, prefix, defaults.nano) + ), + list: async () => + withAdkErrors(getNativeAdkEngine().listCaches(defaults.nano)), + delete: async (name: string) => + withAdkErrors(getNativeAdkEngine().deleteCache(name, defaults.nano)), + clearImplicit: async () => + withAdkErrors(getNativeAdkEngine().clearImplicitCaches(defaults.nano)), + } satisfies AdkNanoCaches return provider } @@ -673,6 +718,24 @@ class AdkChatLanguageModel implements AdkLanguageModel { ): AdkGenerationOptions { const adkOptions = (options.providerOptions?.adk ?? {}) as AdkProviderOptions + let cacheSystemPrompt = adkOptions.cacheSystemPrompt + + if (cacheSystemPrompt && !this.isNano) { + warnings.push({ + type: 'unsupported', + feature: 'cacheSystemPrompt', + details: + 'Only Gemini Nano supports it. Gemini caches repeated prefixes implicitly.', + }) + cacheSystemPrompt = undefined + } + if (cacheSystemPrompt && adkOptions.cachedContent) { + warnings.push({ + type: 'other', + message: 'cacheSystemPrompt is ignored because cachedContent is set.', + }) + cacheSystemPrompt = undefined + } if (this.isNano) { // ADK's ML Kit model only passes on temperature, topK and maxOutputTokens @@ -712,6 +775,8 @@ class AdkChatLanguageModel implements AdkLanguageModel { } : undefined, thinking: prepareThinking(options.reasoning, adkOptions), + cachedContent: adkOptions.cachedContent, + cacheSystemPrompt, } } From 94564fa98a068b28f1e76491e688cc1e9e2c7b6d Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Sun, 25 Oct 2026 04:21:03 +0400 Subject: [PATCH 18/18] docs(adk): document gemini nano context caching Co-Authored-By: Claude Opus 5.5 --- website/src/docs/adk/generating.md | 62 ++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/website/src/docs/adk/generating.md b/website/src/docs/adk/generating.md index 4032afbd0..a8adcb19f 100644 --- a/website/src/docs/adk/generating.md +++ b/website/src/docs/adk/generating.md @@ -82,6 +82,67 @@ console.log(usage.outputTokenDetails.reasoningTokens) // thoughtsTokenCount console.log(usage.raw?.totalTokenCount) ``` +ML Kit does not report token usage for Gemini Nano, so Nano usage is empty apart from the cached prompt tokens described in [Context Caching](#context-caching). + +## Context Caching + +Chat apps send the same system prompt with every turn, and Gemini Nano processes it again each time. ML Kit can cache the processed prompt prefix and reuse it. Google's ML Kit docs report these Pixel 9 timings: + +| Prompt | Without cache | Cache hit | +| ------------------------------------ | ------------- | --------- | +| 300-token prefix, 50-token suffix | 0.82 s | 0.45 s | +| 1,000-token prefix, 100-token suffix | 2.11 s | 0.5 s | + +Prefix caching is experimental in ML Kit, only works on some devices and only with text prompts. Where it is not available, requests run without a cache. + +### Caching the System Prompt + +Set `cacheSystemPrompt` to send the system prompt to Gemini Nano as a prompt prefix that ML Kit caches by itself: + +```typescript +import { createAdkProvider } from '@react-native-ai/adk' +import { streamText } from 'ai' + +const adk = createAdkProvider({ instruction: 'You are a cooking assistant.' }) + +const result = streamText({ + model: adk(), + system: 'Answer in two sentences.', + messages, + providerOptions: { adk: { cacheSystemPrompt: true } }, +}) +``` + +The prefix is the agent `instruction`, the prompt's `system` messages, and the tool and JSON guidance the provider adds for Gemini Nano. Keep it the same across calls: any change is a new prefix, and the first call with a prefix is a cache miss. ADK adds a note about role markers to the prefix once a conversation has more than one message, so the second turn of a chat is a miss too. Prompts with images skip the cache. ML Kit stores the caches in app storage and evicts the least used ones. Call `adk.caches.clearImplicit()` to delete them. + +### Explicit Caches + +To process a long, fixed prefix ahead of time, such as a document to answer questions about, create a named cache and pass its name as `cachedContent`: + +```typescript +import { adk } from '@react-native-ai/adk' +import { generateText } from 'ai' + +await adk.caches.create( + 'manual', + `Answer questions about this manual:\n${manual}` +) + +const { text } = await generateText({ + model: adk(), + prompt: 'How do I reset the device?', + providerOptions: { adk: { cachedContent: 'manual' } }, +}) +``` + +ML Kit keeps explicit caches across app launches until you delete them. Use `adk.caches.list()` and `adk.caches.delete(name)` to manage them. The cache comes before the agent instruction and the messages, so leave out a `system` prompt that repeats it. `cacheSystemPrompt` is ignored when `cachedContent` is set. + +For cloud Gemini, `cachedContent` takes the name of a Gemini API cache, such as `cachedContents/abc123`. Cloud Gemini models also cache repeated prefixes on their own, so `cacheSystemPrompt` only applies to Gemini Nano. + +### Cached Tokens in Usage + +With Gemini Nano, `usage.inputTokenDetails.cacheReadTokens` is the token count of the cached prefix. Because ML Kit does not report cache hits, this is an estimate. It is set when the prefix was already used in this app session, or when the explicit cache was created in this app session. + ## Reasoning Set the AI SDK `reasoning` option to turn on the model's thinking. The thoughts come back as reasoning parts: @@ -277,5 +338,6 @@ const { text } = await generateText({ | Native tool calling on Nano | Not in ML Kit - the provider parses `tool_code` blocks | | Forcing a tool call | `required` and specific tools run as `auto`, with a warning | | Schema enforcement on Nano | Prompted, then validated by the AI SDK | +| Token usage on Nano | Not in ML Kit - only cached prompt tokens are estimated | | Provider file references | Throws - pass data or a URL | | iOS / web | Android only |