Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
299 changes: 83 additions & 216 deletions README.md

Large diffs are not rendered by default.

16 changes: 14 additions & 2 deletions packages/adk/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
# ADK Provider for Vercel AI SDK
# @react-native-ai/adk

A Vercel AI SDK provider for [Google's Agent Development Kit (ADK)](https://developer.android.com/ai/adk) on Android. Use on-device Gemini Nano or cloud Gemini with tool calling and multi-turn sessions.
Add Gemini to your Android app without shipping a model or an API key. Gemini
Nano runs on-device through Google's
[Agent Development Kit (ADK)](https://developer.android.com/ai/adk) and
AICore, for chat, summaries and tool-using assistants that keep data on the
device. AICore downloads Nano the first time `prepareNano()` runs, if the
device doesn't have it yet. When you need a larger model, switch to cloud
Gemini.

It also comes as a [Vercel AI SDK](https://ai-sdk.dev) provider, so it drops
into code that already uses the AI SDK.

To run models you choose and ship yourself on Android, see
[`@react-native-ai/llama`](../llama).

**Requirements:**

Expand Down
134 changes: 92 additions & 42 deletions packages/apple-llm/README.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,118 @@
# Apple Provider for Vercel AI SDK
# @react-native-ai/apple

A Vercel AI SDK provider for Apple Foundation Models, enabling access to Apple Intelligence in React Native applications.
Add AI features to your iOS app without shipping a model. Chat, summaries,
structured data extraction and tool-using assistants run on Apple Intelligence,
and semantic search, voice input and spoken output run on system models. Data
stays on the device, there are no API costs, and your app stays small, since
iOS provides the models.

**Requirements:**
When a task needs more than the on-device model, iOS 27 can send it to Apple's
Private Cloud Compute instead. It also comes as a
[Vercel AI SDK](https://ai-sdk.dev) provider, so it drops into code that
already uses the AI SDK.

- iOS 26+ for text generation
- iOS 26.4+ for token counting
- iOS 26.4 for Image Playground generation, which Apple removed in iOS 27
- iOS 27+ for image prompts, Private Cloud Compute, and Vision built-in tools
- Apple Intelligence enabled device
- Vercel AI SDK v7
To run models you choose and ship yourself, see
[`@react-native-ai/core-ai`](../core-ai) or
[`@react-native-ai/llama`](../llama).

## Features

| Capability | Apple framework | AI SDK function | iOS |
| ---------------- | -------------------------------------------------------------------------------------------------------- | --------------------------- | ---- |
| Text generation | [Foundation Models](https://developer.apple.com/documentation/FoundationModels) | `generateText`/`streamText` | 26+ |
| Embeddings | [NLContextualEmbedding](https://developer.apple.com/documentation/naturallanguage/nlcontextualembedding) | `embed`/`embedMany` | 17+ |
| Transcription | [SpeechAnalyzer](https://developer.apple.com/documentation/speech/speechanalyzer) | `transcribe` | 26+ |
| Speech synthesis | [AVSpeechSynthesizer](https://developer.apple.com/documentation/avfaudio/avspeechsynthesizer) | `generateSpeech` | 13+ |
| Image generation | Image Playground | `generateImage` | 26.4 |

Text generation supports streaming, structured output, tool calling, image
prompts, reasoning levels and Private Cloud Compute. Each capability also has a
native API (`AppleFoundationModels`, `AppleEmbeddings`, `AppleTranscription`,
`AppleSpeech`) for features outside the AI SDK, such as live microphone
transcription on iOS 27 and voice listing.

## Requirements

- iOS 26+ and a device with Apple Intelligence enabled, for text generation
- React Native New Architecture
- Vercel AI SDK v7

## Installation

```bash
npm install @react-native-ai/apple ai
```

The package is autolinked, so there is no native setup. The one exception is
live microphone transcription (`AppleTranscription.transcribeLive`), which
needs a microphone usage description in `Info.plist`:

```xml
<key>NSMicrophoneUsageDescription</key>
<string>Transcribe what you say.</string>
```

## Usage

```ts
import { apple } from '@react-native-ai/apple'
import { generateText } from 'ai'
import { embed, generateSpeech, generateText, transcribe } from 'ai'

// Recorded audio, for example a WAV file read into a Uint8Array
declare const audioBuffer: Uint8Array

const answer = await generateText({
const { text } = await generateText({
model: apple(),
prompt: 'What is the meaning of life?',
prompt: 'Explain quantum computing in one sentence.',
})

const { embedding } = await embed({
model: apple.embeddingModel(),
value: 'Hello world',
})

const { text: transcript } = await transcribe({
model: apple.transcriptionModel(),
audio: audioBuffer,
Comment thread
grabbou marked this conversation as resolved.
Resolved
})

const { audio } = await generateSpeech({
model: apple.speechModel(),
text: 'Hello from Apple!',
})
```

For reusable history utilities such as summarization, rolling windows, and
completed tool-call pruning, use `@react-native-ai/utils`.
Check `await apple.isAvailable()` before generating text, since Apple
Intelligence can be off or still downloading. For long chats on the small
on-device context window, wrap the model with
[`@react-native-ai/utils`](../utils) (not published to npm yet).

## iOS 26 and iOS 27

The provider runs on both. iOS 26.4 and iOS 27 features also need an app built
with an SDK that has them (Xcode 26 with Swift 6.3, and Xcode 27).

| Feature | iOS 26 | iOS 26.4 | iOS 27 |
| ------------------------------------ | ------ | -------- | ----------------------- |
| Text, streaming, structured output | ✅ | ✅ | ✅ |
| JavaScript tools | ✅ | ✅ | ✅ |
| Tool calls while streaming | ❌ | ❌ | ✅ |
| `toolChoice: 'auto'` and `'none'` | ✅ | ✅ | ✅ |
| `toolChoice: 'required'`, named tool | ❌ | ❌ | ✅ |
| Token counting and context size | ❌ | ✅ | ✅ |
| Image Playground generation | ❌ | ✅ | ❌ Removed by Apple |
| Reasoning level and reasoning parts | ❌ | ❌ | ✅ |
| Image prompts | ❌ | ❌ | ✅ |
| Private Cloud Compute | ❌ | ❌ | ✅ Physical device only |
| Vision OCR and barcode tools | ❌ | ❌ | ✅ Physical device only |
| Token usage | ❌ | ❌ | ✅ |
| Feature | iOS 26 | iOS 26.4 | iOS 27 |
| --------------------------------------- | ------ | -------- | ----------------------- |
| Text, streaming, structured output | ✅ | ✅ | ✅ |
| JavaScript tools | ✅ | ✅ | ✅ |
| Tool calls while streaming | ❌ | ❌ | ✅ |
| `toolChoice: 'auto'` and `'none'` | ✅ | ✅ | ✅ |
| `toolChoice: 'required'`, named tool | ❌ | ❌ | ✅ |
| Token counting and context size | ❌ | ✅ | ✅ |
| Image Playground generation | ❌ | ✅ | ❌ Removed by Apple |
| Reasoning level and reasoning parts | ❌ | ❌ | ✅ |
| Image prompts | ❌ | ❌ | ✅ |
| Private Cloud Compute | ❌ | ❌ | ✅ Physical device only |
| Vision OCR, barcode and Spotlight tools | ❌ | ❌ | ✅ Physical device only |
| Token usage | ❌ | ❌ | ✅ |
| Live microphone transcription | ❌ | ❌ | ✅ Physical device only |

See [iOS Versions](https://react-native-ai.dev/docs/apple/ios-versions) for details.

## Features

- ✅ Text generation with Apple Foundation Models
- ✅ Image prompts on iOS 27+
- ✅ Runtime model info and context-size metadata
- ✅ Private Cloud Compute language model selection on iOS 27+
- ✅ Reasoning level control on iOS 27+
- ✅ Structured outputs
- ✅ Tool calling
- ✅ Vision OCR and barcode built-in tools on iOS 27+
- ✅ Streaming
- ✅ Image Playground generation through the AI SDK image API on iOS 26.4 (deprecated)

## Documentation

For complete installation instructions and API documentation, visit our [documentation site](https://react-native-ai.dev/docs/apple/getting-started).
Guides and API reference are on the [documentation site](https://react-native-ai.dev/docs/apple/getting-started).

## License

Expand Down
12 changes: 10 additions & 2 deletions packages/core-ai/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
# @react-native-ai/core-ai

Apple Core AI provider for React Native AI. It runs model bundles exported with [`apple/coreai-models`](https://github.com/apple/coreai-models) on Apple's Core AI runtime, with the Vercel AI SDK and direct native calls.
Bring your own model to iPhone and run it on Apple's Core AI framework. Start
from a [hosted export](https://react-native-ai.dev/docs/core-ai/models), such
as Qwen3 for chat or FLUX.2 for images, or export your own with
[`apple/coreai-models`](https://github.com/apple/coreai-models), and run it
on-device from React Native. Use it for text, image and video generation, transcription, embeddings,
and vision tasks such as segmentation and object detection.

For Apple's built-in system models (Foundation Models, NLContextualEmbedding, SpeechAnalyzer, AVSpeechSynthesizer), use `@react-native-ai/apple`.
It also comes as a [Vercel AI SDK](https://ai-sdk.dev) provider, so it drops
into code that already uses the AI SDK.

For Apple's built-in system models (Foundation Models, NLContextualEmbedding, SpeechAnalyzer, AVSpeechSynthesizer), use [`@react-native-ai/apple`](../apple-llm).

| Task | API |
| ---------------- | ------------------------------------------------------------------------------------------------------------------ |
Expand Down
30 changes: 14 additions & 16 deletions packages/dev-tools/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
![rozenite-banner](https://www.rozenite.dev/rozenite-banner.jpg)

### A Rozenite plugin that captures Vercel AI SDK spans in Rozenite
# @react-native-ai/dev-tools

The AI SDK Profiler plugin captures OpenTelemetry spans emitted by the Vercel AI
SDK and shows them in Rozenite DevTools. It helps you inspect requests,
inputs, outputs, provider metadata, and latency.
See every Vercel AI SDK call your React Native app makes. The AI SDK Profiler
is a [Rozenite](https://www.rozenite.dev) DevTools plugin that captures the
OpenTelemetry spans the AI SDK emits and shows their inputs, outputs, tool
calls, provider metadata and latency.

![AI SDK Profiler preview](../../website/public/dev-tools-preview.png)

Expand All @@ -27,39 +28,36 @@ https://www.rozenite.dev/docs/getting-started
### 1. Initialize the tracer and DevTools hook

```tsx
import {
getAiSdkTracer,
useAiSdkDevTools,
} from '@react-native-ai/dev-tools';
import { getAiSdkTracer, useAiSdkDevTools } from '@react-native-ai/dev-tools'

export function App() {
useAiSdkDevTools();
return <RootNavigator />;
useAiSdkDevTools()
return <RootNavigator />
}
```

### 2. Enable AI SDK telemetry on requests

```ts
import { OpenTelemetry } from '@ai-sdk/otel';
import { generateText, registerTelemetry } from 'ai';
import { openai } from '@ai-sdk/openai';
import { getAiSdkTracer } from '@react-native-ai/dev-tools';
import { OpenTelemetry } from '@ai-sdk/otel'
import { generateText, registerTelemetry } from 'ai'
import { openai } from '@ai-sdk/openai'
import { getAiSdkTracer } from '@react-native-ai/dev-tools'

// Register once, at app startup
registerTelemetry(
new OpenTelemetry({
tracer: getAiSdkTracer({ serviceName: 'my-app' }),
})
);
)

await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Write a short story about a cat.',
telemetry: {
functionId: 'unique-identifier-where-the-call-comes-from',
},
});
})
```

### 3. Open DevTools
Expand Down
7 changes: 5 additions & 2 deletions packages/json-ui/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# `@react-native-ai/json-ui`
# @react-native-ai/json-ui

Lightweight JSON UI tooling for React Native + Vercel AI SDK tool calling.
Let a language model build React Native UI. The model calls tools that add,
update and remove nodes in a JSON spec, and `GenerativeUIView` renders it. It
plugs into Vercel AI SDK tool calling, and is designed for small on-device
models with limited context.

**This package provides:**

Expand Down
76 changes: 55 additions & 21 deletions packages/llama/README.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,79 @@
# Llama Provider for Vercel AI SDK
# @react-native-ai/llama

A Vercel AI SDK provider for llama.rn, enabling on-device inference with GGUF models in React Native applications.
Pick an open-weight model and run it on iOS and Android. Download a GGUF model
such as Qwen, Llama or Gemma from Hugging Face and run it fully on-device with
[llama.rn](https://github.com/mybigday/llama.rn), for chat, semantic search,
reranking or speech, with the same model on both platforms.

**Requirements:**
- React Native >= 0.76.0
- llama.rn >= 0.13.0-rc.8 (required for React Native 0.88)
It also comes as a [Vercel AI SDK](https://ai-sdk.dev) provider, so it drops
into code that already uses the AI SDK.

Use it when you need the same models on both platforms, or a model the system
doesn't provide. For models built into the OS, see
[`@react-native-ai/apple`](../apple-llm) (iOS) and
[`@react-native-ai/adk`](../adk) (Android).

## Features

| Capability | Method | AI SDK function | Model |
| --------------- | ------------------------ | --------------------------- | --------------------------------------- |
| Text generation | `llama.languageModel()` | `generateText`/`streamText` | Chat or instruct model |
| Embeddings | `llama.embeddingModel()` | `embed`/`embedMany` | Embedding model |
| Reranking | `llama.rerankingModel()` | `rerank` | Reranker model |
| Speech | `llama.speechModel()` | `generateSpeech` | TTS model and a vocoder (`vocoderPath`) |

Language models support:

- Streaming
- Reasoning, controlled with the AI SDK `reasoning` option
- Structured output (`Output.object`), enforced with a JSON schema grammar
- Tool calling
- Vision and audio input with a multimodal projector (`projectorPath`)
- GPU acceleration

Model download management (`downloadModel`, `getDownloadedModels`,
`removeModel`) is included.

## Requirements

- React Native 0.76+
- llama.rn 0.13.0-rc.8+ (required for React Native 0.88)
- Vercel AI SDK v7

## Installation

```bash
npm install @react-native-ai/llama llama.rn react-native-blob-util ai
```

## Usage

```ts
import { downloadModel, llama } from '@react-native-ai/llama'
import { generateText } from 'ai'

// Download (format: "owner/repo/filename.gguf") and prepare
// Model ID format: "owner/repo/filename.gguf"
const modelPath = await downloadModel(
'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf'
'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf',
(progress) => console.log(`Downloading: ${progress.percentage}%`)
)

const model = llama.languageModel(modelPath)
await model.prepare()

// Generate
const { text } = await generateText({
model,
prompt: 'Write a haiku about coding.'
prompt: 'Write a haiku about coding.',
})
```

## Features
await model.unload()
```

- ✅ Text generation with GGUF models
- ✅ Streaming
- ✅ Reasoning, controlled with the AI SDK `reasoning` option
- ✅ Structured output (`Output.object`), enforced with a JSON schema grammar
- ✅ Tool calling
- ✅ Vision and audio input with a multimodal projector (`projectorPath`)
- ✅ On-device inference (iOS & Android)
- ✅ Model download management
- ✅ GPU acceleration support
Any GGUF model from Hugging Face whose architecture llama.cpp supports works.

## Documentation

For complete installation instructions and API documentation, visit our [documentation site](https://react-native-ai.dev/docs/llama/getting-started).
Guides and API reference are on the [documentation site](https://react-native-ai.dev/docs/llama/getting-started).

## License

Expand Down
7 changes: 5 additions & 2 deletions packages/utils/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# React Native AI Utils
# @react-native-ai/utils

Shared utilities for React Native AI SDK providers.
Keep long chats working on small on-device context windows.
`wrapLanguageModelWithHistory` wraps any Vercel AI SDK language model,
including the Apple, Llama and ADK providers, and compacts the conversation
before each request.

> Not published to npm yet. Use it from this monorepo, or copy
> `src/index.ts` into your app. Its only dependency is `@ai-sdk/provider`.
Expand Down
Loading