From bec55d2a772c09ba8b9f3fd3f8b59f4736eca4b1 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 12:07:52 +0400 Subject: [PATCH 01/11] docs: restructure the readme and describe each package on its own Co-Authored-By: Claude Opus 5.5 --- README.md | 247 ++++++++++++++--------------------- packages/adk/README.md | 15 ++- packages/apple-llm/README.md | 119 +++++++++++------ packages/dev-tools/README.md | 30 ++--- packages/json-ui/README.md | 7 +- packages/llama/README.md | 75 ++++++++--- packages/utils/README.md | 6 +- 7 files changed, 267 insertions(+), 232 deletions(-) diff --git a/README.md b/README.md index 72261b5bd..319dce1a0 100644 --- a/README.md +++ b/README.md @@ -2,132 +2,62 @@ # React Native AI -A collection of on-device AI primitives for React Native with first-class Vercel AI SDK support. Run AI models directly on users' devices for privacy-preserving, low-latency inference without server costs. +Run AI models on users' devices from React Native, with first-class +[Vercel AI SDK](https://ai-sdk.dev) support. Each provider wraps an on-device +runtime and exposes it as an AI SDK provider, so `generateText`, `streamText`, +`embed` and the other AI SDK functions work with local models the same way they +work with cloud ones. -## Features - -- ๐Ÿš€ **Instant AI** - Use built-in system models immediately without downloads +- ๐Ÿš€ **Instant AI** - Use the models built into iOS and Android, with nothing to bundle - ๐Ÿ”’ **Privacy-first** - Inference runs on-device, so data stays local. Cloud models are opt-in -- ๐ŸŽฏ **Vercel AI SDK compatible** - Use `generateText`, `streamText`, `embed` and other AI SDK functions with local models -- ๐ŸŽจ **Beyond text** - Embeddings, transcription, speech synthesis and image generation - -## AI SDK Compatibility - -| React Native AI | AI SDK | -| --------------- | ------ | -| 0.11 and below | v5 | -| 0.12 | v6 | -| 0.13 and above | v7 | - -## DevTools - -![AI SDK Profiler preview](website/src/public/dev-tools-preview.png) - -The AI SDK Profiler plugin captures OpenTelemetry spans from Vercel AI SDK -requests and surfaces them in Rozenite DevTools. DevTools are runtime -agnostic, so they work with on-device and remote runtimes. - -```bash -npm install @react-native-ai/dev-tools -``` - -Rozenite must be installed and enabled in your app. See the -[Rozenite getting started guide](https://www.rozenite.dev/docs/getting-started). - -The Expo demo app in [apps/expo-example](./apps/expo-example) -includes native-development Rozenite wiring for this plugin. Run the example -app, open React Native DevTools, and select `AI SDK Profiler` to inspect chat -spans emitted by the Vercel AI SDK. +- ๐ŸŽฏ **Vercel AI SDK compatible** - Swap local and cloud models without changing your code +- ๐ŸŽจ **Beyond text** - Embeddings, reranking, transcription, speech and image generation -If the `AI SDK Profiler` panel stays empty after you send a message, close the -React Native DevTools window and open a new one. A stale debugger session can -keep the panel mounted without receiving telemetry. - -## Available Providers +## Providers -| Provider | Built-in | Platforms | Runtime | Description | -| ------------------- | -------- | ------------ | ------------------------------------------------------------------- | ---------------------------------------------------------- | -| [Apple](#apple) | โœ… Yes | iOS | [Apple](https://developer.apple.com/documentation/FoundationModels) | Apple Foundation Models, embeddings, transcription, speech | -| [ADK](#google-adk) | โœ… Yes | Android | [Google ADK](https://developer.android.com/ai/adk) | On-device Gemini Nano, with optional cloud Gemini | -| [Llama](#llama) | โŒ No | iOS, Android | [llama.rn](https://github.com/mybigday/llama.rn) | Run GGUF models via llama.rn | - ---- +| Provider | Package | Platforms | Models | Download needed | +| ---------------------------------- | ------------------------------------------------ | ------------ | ------------------------------------------ | --------------- | +| [Apple](#apple) | [`@react-native-ai/apple`](./packages/apple-llm) | iOS | Apple Intelligence and system models | No | +| [Google ADK](#google-adk) | [`@react-native-ai/adk`](./packages/adk) | Android | Gemini Nano, optional cloud Gemini | No | +| [Llama](#llama) | [`@react-native-ai/llama`](./packages/llama) | iOS, Android | Any GGUF model, via llama.rn | Yes | +| [Core AI](#core-ai) (not released) | [`@react-native-ai/core-ai`](./packages/core-ai) | iOS 27+ | Core AI bundles from `apple/coreai-models` | Yes | ### Apple -Native integration with Apple's on-device AI capabilities. **Built-in** - no model downloads required, uses system models. - -- **Text Generation** - Apple Foundation Models for chat and completion -- **Embeddings** - NLContextualEmbedding for 512-dimensional semantic vectors -- **Transcription** - SpeechAnalyzer for fast, accurate speech-to-text -- **Speech Synthesis** - AVSpeechSynthesizer for natural text-to-speech with system voices - -#### Installation +Use the AI built into iOS: Apple Intelligence for text generation, +NLContextualEmbedding for embeddings, SpeechAnalyzer for transcription and +AVSpeechSynthesizer for speech. Nothing to download, the models ship with the +system. ```bash npm install @react-native-ai/apple ``` -No additional linking needed, works immediately on iOS devices (autolinked). - -#### Usage - -```typescript +```ts import { apple } from '@react-native-ai/apple' -import { - generateText, - embed, - transcribe, - generateSpeech as speech, -} from 'ai' - -// Text generation with Apple Intelligence +import { generateText } from 'ai' + const { text } = await generateText({ model: apple(), prompt: 'Explain quantum computing', }) - -// Generate embeddings -const { embedding } = await embed({ - model: apple.embeddingModel(), - value: 'Hello world', -}) - -// Transcribe audio -const { text: transcript } = await transcribe({ - model: apple.transcriptionModel(), - audio: audioBuffer, -}) - -// Text-to-speech -const { audio } = await speech({ - model: apple.speechModel(), - text: 'Hello from Apple!', -}) ``` -#### Availability +| Feature | iOS | Additional requirements | +| ---------------- | ------- | -------------------------- | +| Text generation | iOS 26+ | Apple Intelligence device | +| Embeddings | iOS 17+ | - | +| Transcription | iOS 26+ | - | +| Speech synthesis | iOS 13+ | iOS 17+ for Personal Voice | -| Feature | iOS Version | Additional Requirements | -| ---------------- | ----------- | -------------------------- | -| Text Generation | iOS 26+ | Apple Intelligence device | -| Embeddings | iOS 17+ | - | -| Transcription | iOS 26+ | - | -| Speech Synthesis | iOS 13+ | iOS 17+ for Personal Voice | - -See the [Apple documentation](https://react-native-ai.dev/docs/apple/getting-started) for detailed setup and usage guides. - ---- +[Package README](./packages/apple-llm) ยท [Documentation](https://react-native-ai.dev/docs/apple/getting-started) ### Google ADK -Run Gemini Nano on-device on Android through Google's [Agent Development Kit](https://developer.android.com/ai/adk). **Built-in** - Gemini Nano is provisioned by the system via AICore, so there are no model files to bundle and no API key. Cloud Gemini is available as an option. - -- **Text Generation** - `generateText` and `streamText` with on-device Gemini Nano -- **Tool Calling** - Native ADK agent loop with JavaScript tool executors -- **Cloud Gemini** - Optional cloud inference via ADK `LlmAgent` when you need a larger model - -#### Installation +Use Gemini on Android through Google's +[Agent Development Kit](https://developer.android.com/ai/adk). Gemini Nano runs +on-device and is provided by the system through AICore, so there are no model +files and no API key. Cloud Gemini is available when you need a larger model. ```bash npm install @react-native-ai/adk @@ -136,9 +66,7 @@ npm install @react-native-ai/adk > [!IMPORTANT] > Requires Android `minSdkVersion` 26 and a physical device with AICore support (Android 14+) for Gemini Nano. Your app also needs packaging excludes for duplicate `META-INF` files, see the [getting started guide](https://react-native-ai.dev/docs/adk/getting-started). -#### Usage - -```typescript +```ts import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' @@ -154,78 +82,105 @@ const { text } = await generateText({ }) ``` -See the [ADK documentation](https://react-native-ai.dev/docs/adk/getting-started) for device checks, cloud Gemini, and tool calling. - ---- +[Package README](./packages/adk) ยท [Documentation](https://react-native-ai.dev/docs/adk/getting-started) ### Llama -Run any GGUF model on-device using [llama.rn](https://github.com/mybigday/llama.rn). **Requires download** - models are downloaded from HuggingFace. - -#### Supported Features - -| Feature | Method | Description | -| --------------- | ------------------------ | --------------------------------------------- | -| Text Generation | `llama.languageModel()` | Chat, completion, streaming, reasoning models | -| Embeddings | `llama.embeddingModel()` | Text embeddings for RAG and similarity search | -| Speech | `llama.speechModel()` | Text-to-speech with vocoder models | - -#### Installation +Run any GGUF model from Hugging Face on iOS and Android with +[llama.rn](https://github.com/mybigday/llama.rn). Supports text generation, +embeddings, reranking and speech. ```bash npm install @react-native-ai/llama llama.rn react-native-blob-util ``` -#### Usage - -```typescript +```ts import { downloadModel, llama } from '@react-native-ai/llama' import { generateText } from 'ai' -// Download from HuggingFace (Model ID format: "owner/repo/filename.gguf") +// Model ID format: "owner/repo/filename.gguf" const modelPath = await downloadModel( - 'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf', - (progress) => console.log(`Downloading: ${progress.percentage}%`) + 'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf' ) -// Create model instance const model = llama.languageModel(modelPath) - -// Initialize model (loads into memory) await model.prepare() -// Generate text const { text } = await generateText({ model, - messages: [ - { role: 'system', content: 'You are a helpful assistant.' }, - { role: 'user', content: 'Write a haiku about coding.' }, - ], + prompt: 'Write a haiku about coding.', }) +``` + +[Package README](./packages/llama) ยท [Documentation](https://react-native-ai.dev/docs/llama/getting-started) + +### Core AI + +> [!WARNING] +> Work in progress. This package is not published to npm yet. -// Cleanup when done -await model.unload() +Run model bundles exported with +[`apple/coreai-models`](https://github.com/apple/coreai-models), such as Qwen3 +or FLUX.2, on iOS 27 with Apple's Core AI framework. Supports text and image +generation through the AI SDK, plus native APIs for segmentation and object +detection. + +```ts +import { coreAI } from '@react-native-ai/core-ai' +import { streamText } from 'ai' + +const result = streamText({ + model: coreAI.languageModel({ + id: 'qwen3-0.6b', + source: { type: 'file', uri: qwen3ModelDirectory }, + variant: 'iOS', + }), + prompt: 'Explain Core AI in one sentence.', +}) ``` -#### Model Compatibility +[Package README](./packages/core-ai) -Any GGUF model from HuggingFace can be used. Use the format `owner/repo/filename.gguf` as the model ID. Popular choices include: +## Tools -- `bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf` -- `bartowski/Llama-3.2-3B-Instruct-GGUF/Llama-3.2-3B-Instruct-Q4_K_M.gguf` -- `Qwen/Qwen2.5-1.5B-Instruct-GGUF/qwen2.5-1.5b-instruct-q4_k_m.gguf` +These work with any AI SDK provider, on-device or cloud. -๐Ÿ“š **[View full Llama documentation โ†’](https://react-native-ai.dev/docs/llama/getting-started)** +| Package | Description | +| ---------------------------------------------------- | ------------------------------------------------------------------------- | +| [`@react-native-ai/json-ui`](./packages/json-ui) | Let a tool-calling model build React Native UI from a JSON spec | +| [`@react-native-ai/dev-tools`](./packages/dev-tools) | Inspect AI SDK calls in Rozenite DevTools | +| [`@react-native-ai/utils`](./packages/utils) | Compact conversation history for small context windows (not released yet) | -## JSON UI +### DevTools -`@react-native-ai/json-ui` lets a tool-calling model build React Native UI. The model calls tools that add, update, and delete nodes in a JSON spec, and `GenerativeUIView` renders it. It's designed for small on-device models with limited context. +![AI SDK Profiler preview](website/src/public/dev-tools-preview.png) + +The AI SDK Profiler plugin captures OpenTelemetry spans from Vercel AI SDK +requests and surfaces them in Rozenite DevTools. ```bash -npm install @react-native-ai/json-ui +npm install @react-native-ai/dev-tools ``` -See the [JSON UI documentation](https://react-native-ai.dev/docs/json-ui/getting-started). +Rozenite must be installed and enabled in your app. See the +[Rozenite getting started guide](https://www.rozenite.dev/docs/getting-started). + +The Expo demo app in [apps/expo-example](./apps/expo-example) +includes native-development Rozenite wiring for this plugin. Run the example +app, open React Native DevTools, and select `AI SDK Profiler` to inspect chat +spans emitted by the Vercel AI SDK. + +If the `AI SDK Profiler` panel stays empty after you send a message, close the +React Native DevTools window and open a new one. A stale debugger session can +keep the panel mounted without receiving telemetry. + +## AI SDK Compatibility + +| React Native AI | AI SDK | +| --------------- | ------ | +| 0.11 and below | v5 | +| 0.12 | v6 | +| 0.13 and above | v7 | ## Documentation diff --git a/packages/adk/README.md b/packages/adk/README.md index ac720cbd4..174bc95ba 100644 --- a/packages/adk/README.md +++ b/packages/adk/README.md @@ -1,6 +1,15 @@ -# ADK Provider for Vercel AI SDK - -A Vercel AI SDK provider for [Google's Agent Development Kit (ADK)](https://developer.android.com/ai/adk) on Android. Use on-device Gemini Nano or cloud Gemini with tool calling and multi-turn sessions. +# @react-native-ai/adk + +Use Gemini on Android from React Native, through Google's +[Agent Development Kit (ADK)](https://developer.android.com/ai/adk). Gemini +Nano runs on-device and is provided by the system through AICore, so there is +no model to download or bundle and no API key. Cloud Gemini is available when +you need a larger model. The package comes with a +[Vercel AI SDK](https://ai-sdk.dev) provider, so you can call both with +`generateText` and `streamText`, including tool calling and image input. + +To run models you choose and ship yourself on Android, see +[`@react-native-ai/llama`](../llama). **Requirements:** diff --git a/packages/apple-llm/README.md b/packages/apple-llm/README.md index ad5cd8f73..f1ba99fcc 100644 --- a/packages/apple-llm/README.md +++ b/packages/apple-llm/README.md @@ -1,68 +1,103 @@ -# Apple Provider for Vercel AI SDK +# @react-native-ai/apple -A Vercel AI SDK provider for Apple Foundation Models, enabling access to Apple Intelligence in React Native applications. +Use the AI built into iOS from React Native. Apple Intelligence (Foundation +Models), text embeddings, speech-to-text and text-to-speech run on-device with +the models that ship with the system, so there is nothing to download or +bundle. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, +so you can call these models with `generateText`, `streamText`, `embed`, +`transcribe` and `generateSpeech`. -**Requirements:** +To run models you choose and ship yourself, see +[`@react-native-ai/core-ai`](../core-ai) or +[`@react-native-ai/llama`](../llama). -- iOS 26+ for text generation -- iOS 26.4+ for token counting -- iOS 26.4 for Image Playground generation, which Apple removed in iOS 27 -- iOS 27+ for image prompts, Private Cloud Compute, and Vision built-in tools -- Apple Intelligence enabled device -- Vercel AI SDK v7 +## Features + +| Capability | Apple framework | AI SDK function | iOS | +| ---------------- | -------------------------------------------------------------------------------------------------------- | --------------------------- | ---- | +| Text generation | [Foundation Models](https://developer.apple.com/documentation/FoundationModels) | `generateText`/`streamText` | 26+ | +| Embeddings | [NLContextualEmbedding](https://developer.apple.com/documentation/naturallanguage/nlcontextualembedding) | `embed`/`embedMany` | 17+ | +| Transcription | [SpeechAnalyzer](https://developer.apple.com/documentation/speech/speechanalyzer) | `transcribe` | 26+ | +| Speech synthesis | [AVSpeechSynthesizer](https://developer.apple.com/documentation/avfaudio/avspeechsynthesizer) | `generateSpeech` | 13+ | +| Image generation | Image Playground | `generateImage` | 26.4 | + +Text generation supports streaming, structured output, tool calling, image +prompts, reasoning levels and Private Cloud Compute. Each capability also has a +native API (`AppleFoundationModels`, `AppleEmbeddings`, `AppleTranscription`, +`AppleSpeech`) for features outside the AI SDK, such as live microphone +transcription on iOS 27 and voice listing. + +## Requirements + +- iOS 26+ and a device with Apple Intelligence enabled, for text generation - React Native New Architecture +- Vercel AI SDK v7 + +## Installation + +```bash +npm install @react-native-ai/apple ai +``` + +The package is autolinked, so there is no native setup. + +## Usage ```ts import { apple } from '@react-native-ai/apple' -import { generateText } from 'ai' +import { embed, generateSpeech, generateText, transcribe } from 'ai' -const answer = await generateText({ +const { text } = await generateText({ model: apple(), - prompt: 'What is the meaning of life?', + prompt: 'Explain quantum computing in one sentence.', +}) + +const { embedding } = await embed({ + model: apple.embeddingModel(), + value: 'Hello world', +}) + +const { text: transcript } = await transcribe({ + model: apple.transcriptionModel(), + audio: audioBuffer, +}) + +const { audio } = await generateSpeech({ + model: apple.speechModel(), + text: 'Hello from Apple!', }) ``` -For reusable history utilities such as summarization, rolling windows, and -completed tool-call pruning, use `@react-native-ai/utils`. +Check `await apple.isAvailable()` before generating text, since Apple +Intelligence can be off or still downloading. For long chats on the small +on-device context window, wrap the model with `@react-native-ai/utils`. ## iOS 26 and iOS 27 The provider runs on both. iOS 26.4 and iOS 27 features also need an app built with an SDK that has them (Xcode 26 with Swift 6.3, and Xcode 27). -| Feature | iOS 26 | iOS 26.4 | iOS 27 | -| ------------------------------------ | ------ | -------- | ----------------------- | -| Text, streaming, structured output | โœ… | โœ… | โœ… | -| JavaScript tools | โœ… | โœ… | โœ… | -| Tool calls while streaming | โŒ | โŒ | โœ… | -| `toolChoice: 'auto'` and `'none'` | โœ… | โœ… | โœ… | -| `toolChoice: 'required'`, named tool | โŒ | โŒ | โœ… | -| Token counting and context size | โŒ | โœ… | โœ… | -| Image Playground generation | โŒ | โœ… | โŒ Removed by Apple | -| Reasoning level and reasoning parts | โŒ | โŒ | โœ… | -| Image prompts | โŒ | โŒ | โœ… | -| Private Cloud Compute | โŒ | โŒ | โœ… Physical device only | -| Vision OCR and barcode tools | โŒ | โŒ | โœ… Physical device only | -| Token usage | โŒ | โŒ | โœ… | +| Feature | iOS 26 | iOS 26.4 | iOS 27 | +| --------------------------------------- | ------ | -------- | ----------------------- | +| Text, streaming, structured output | โœ… | โœ… | โœ… | +| JavaScript tools | โœ… | โœ… | โœ… | +| Tool calls while streaming | โŒ | โŒ | โœ… | +| `toolChoice: 'auto'` and `'none'` | โœ… | โœ… | โœ… | +| `toolChoice: 'required'`, named tool | โŒ | โŒ | โœ… | +| Token counting and context size | โŒ | โœ… | โœ… | +| Image Playground generation | โŒ | โœ… | โŒ Removed by Apple | +| Reasoning level and reasoning parts | โŒ | โŒ | โœ… | +| Image prompts | โŒ | โŒ | โœ… | +| Private Cloud Compute | โŒ | โŒ | โœ… Physical device only | +| Vision OCR, barcode and Spotlight tools | โŒ | โŒ | โœ… Physical device only | +| Token usage | โŒ | โŒ | โœ… | +| Live microphone transcription | โŒ | โŒ | โœ… Physical device only | See [iOS Versions](https://react-native-ai.dev/docs/apple/ios-versions) for details. -## Features - -- โœ… Text generation with Apple Foundation Models -- โœ… Image prompts on iOS 27+ -- โœ… Runtime model info and context-size metadata -- โœ… Private Cloud Compute language model selection on iOS 27+ -- โœ… Reasoning level control on iOS 27+ -- โœ… Structured outputs -- โœ… Tool calling -- โœ… Vision OCR and barcode built-in tools on iOS 27+ -- โœ… Streaming -- โœ… Image Playground generation through the AI SDK image API on iOS 26.4 (deprecated) - ## Documentation -For complete installation instructions and API documentation, visit our [documentation site](https://react-native-ai.com/docs/apple). +Guides and API reference are on the [documentation site](https://react-native-ai.dev/docs/apple/getting-started). ## License diff --git a/packages/dev-tools/README.md b/packages/dev-tools/README.md index 2b5498a87..af6e67d1a 100644 --- a/packages/dev-tools/README.md +++ b/packages/dev-tools/README.md @@ -1,10 +1,11 @@ ![rozenite-banner](https://www.rozenite.dev/rozenite-banner.jpg) -### A Rozenite plugin that captures Vercel AI SDK spans in Rozenite +# @react-native-ai/dev-tools -The AI SDK Profiler plugin captures OpenTelemetry spans emitted by the Vercel AI -SDK and shows them in Rozenite DevTools. It helps you inspect requests, -inputs, outputs, provider metadata, and latency. +See every Vercel AI SDK call your React Native app makes. The AI SDK Profiler +is a [Rozenite](https://www.rozenite.dev) DevTools plugin that captures the +OpenTelemetry spans the AI SDK emits and shows their inputs, outputs, tool +calls, provider metadata and latency. ![AI SDK Profiler preview](../../website/src/public/dev-tools-preview.png) @@ -27,31 +28,28 @@ https://www.rozenite.dev/docs/getting-started ### 1. Initialize the tracer and DevTools hook ```tsx -import { - getAiSdkTracer, - useAiSdkDevTools, -} from '@react-native-ai/dev-tools'; +import { getAiSdkTracer, useAiSdkDevTools } from '@react-native-ai/dev-tools' export function App() { - useAiSdkDevTools(); - return ; + useAiSdkDevTools() + return } ``` ### 2. Enable AI SDK telemetry on requests ```ts -import { OpenTelemetry } from '@ai-sdk/otel'; -import { generateText, registerTelemetry } from 'ai'; -import { openai } from '@ai-sdk/openai'; -import { getAiSdkTracer } from '@react-native-ai/dev-tools'; +import { OpenTelemetry } from '@ai-sdk/otel' +import { generateText, registerTelemetry } from 'ai' +import { openai } from '@ai-sdk/openai' +import { getAiSdkTracer } from '@react-native-ai/dev-tools' // Register once, at app startup registerTelemetry( new OpenTelemetry({ tracer: getAiSdkTracer({ serviceName: 'my-app' }), }) -); +) await generateText({ model: openai('gpt-4o-mini'), @@ -59,7 +57,7 @@ await generateText({ telemetry: { functionId: 'unique-identifier-where-the-call-comes-from', }, -}); +}) ``` ### 3. Open DevTools diff --git a/packages/json-ui/README.md b/packages/json-ui/README.md index a55bd291a..3f38559e6 100644 --- a/packages/json-ui/README.md +++ b/packages/json-ui/README.md @@ -1,6 +1,9 @@ -# `@react-native-ai/json-ui` +# @react-native-ai/json-ui -Lightweight JSON UI tooling for React Native + Vercel AI SDK tool calling. +Let a language model build React Native UI. The model calls tools that add, +update and remove nodes in a JSON spec, and `GenerativeUIView` renders it. It +plugs into Vercel AI SDK tool calling, and is designed for small on-device +models with limited context. **This package provides:** diff --git a/packages/llama/README.md b/packages/llama/README.md index d6d648d13..79fb31107 100644 --- a/packages/llama/README.md +++ b/packages/llama/README.md @@ -1,45 +1,78 @@ -# Llama Provider for Vercel AI SDK +# @react-native-ai/llama -A Vercel AI SDK provider for llama.rn, enabling on-device inference with GGUF models in React Native applications. +Run open-weight GGUF models on iOS and Android with +[llama.rn](https://github.com/mybigday/llama.rn). Download a model such as +Qwen, Llama or Gemma from Hugging Face, load it into memory and run it fully +on-device. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) +provider, so the same models work with `generateText`, `streamText`, `embed`, +`rerank` and `generateSpeech`. -**Requirements:** -- React Native >= 0.76.0 -- llama.rn >= 0.13.0-rc.8 (required for React Native 0.88) +Use it when you need the same models on both platforms, or a model the system +doesn't provide. For models built into the OS, see +[`@react-native-ai/apple`](../apple-llm) (iOS) and +[`@react-native-ai/adk`](../adk) (Android). + +## Features + +| Capability | Method | AI SDK function | +| --------------- | ------------------------ | --------------------------- | +| Text generation | `llama.languageModel()` | `generateText`/`streamText` | +| Embeddings | `llama.embeddingModel()` | `embed`/`embedMany` | +| Reranking | `llama.rerankingModel()` | `rerank` | +| Speech | `llama.speechModel()` | `generateSpeech` | + +Language models support: + +- Streaming +- Reasoning, controlled with the AI SDK `reasoning` option +- Structured output (`Output.object`), enforced with a JSON schema grammar +- Tool calling +- Vision and audio input with a multimodal projector (`projectorPath`) +- GPU acceleration + +Model download management (`downloadModel`, `getDownloadedModels`, +`removeModel`) is included. + +## Requirements + +- React Native 0.76+ +- llama.rn 0.13.0-rc.8+ (required for React Native 0.88) - Vercel AI SDK v7 +## Installation + +```bash +npm install @react-native-ai/llama llama.rn react-native-blob-util ai +``` + +## Usage + ```ts import { downloadModel, llama } from '@react-native-ai/llama' import { generateText } from 'ai' -// Download (format: "owner/repo/filename.gguf") and prepare +// Model ID format: "owner/repo/filename.gguf" const modelPath = await downloadModel( - 'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf' + 'bartowski/Qwen_Qwen3.5-2B-GGUF/Qwen_Qwen3.5-2B-Q4_K_M.gguf', + (progress) => console.log(`Downloading: ${progress.percentage}%`) ) + const model = llama.languageModel(modelPath) await model.prepare() -// Generate const { text } = await generateText({ model, - prompt: 'Write a haiku about coding.' + prompt: 'Write a haiku about coding.', }) -``` -## Features +await model.unload() +``` -- โœ… Text generation with GGUF models -- โœ… Streaming -- โœ… Reasoning, controlled with the AI SDK `reasoning` option -- โœ… Structured output (`Output.object`), enforced with a JSON schema grammar -- โœ… Tool calling -- โœ… Vision and audio input with a multimodal projector (`projectorPath`) -- โœ… On-device inference (iOS & Android) -- โœ… Model download management -- โœ… GPU acceleration support +Any GGUF model from Hugging Face works. ## Documentation -For complete installation instructions and API documentation, visit our [documentation site](https://react-native-ai.com/docs/llama). +Guides and API reference are on the [documentation site](https://react-native-ai.dev/docs/llama/getting-started). ## License diff --git a/packages/utils/README.md b/packages/utils/README.md index 0a21144b0..935526585 100644 --- a/packages/utils/README.md +++ b/packages/utils/README.md @@ -1,6 +1,8 @@ -# React Native AI Utils +# @react-native-ai/utils -Shared utilities for React Native AI SDK providers. +Utilities for running Vercel AI SDK language models on small on-device context +windows. It works with any AI SDK language model, including the Apple, Llama +and ADK providers. > Not published to npm yet. Use it from this monorepo, or copy > `src/index.ts` into your app. Its only dependency is `@ai-sdk/provider`. From 6099055b81658240a1dc688d4199d0fc65876586 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 12:07:21 +0400 Subject: [PATCH 02/11] docs(core-ai): describe the package on its own in the readme Co-Authored-By: Claude Opus 5.5 --- packages/core-ai/README.md | 61 ++++++++++++++++++++++---------------- 1 file changed, 36 insertions(+), 25 deletions(-) diff --git a/packages/core-ai/README.md b/packages/core-ai/README.md index 18a4a02d3..a66547be4 100644 --- a/packages/core-ai/README.md +++ b/packages/core-ai/README.md @@ -1,30 +1,39 @@ # @react-native-ai/core-ai -Apple Core AI provider for React Native AI. +Run your own models on iPhone with Apple's Core AI framework. Bring a model +bundle exported with [`apple/coreai-models`](https://github.com/apple/coreai-models), +such as Qwen3 or FLUX.2, and run it on-device from React Native. The package +comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, so the same models +work with `generateText`, `streamText`, `generateImage` and the other AI SDK +functions. > [!WARNING] > Work in progress. This package is not published to npm yet. > Its docs live in [`docs/`](./docs) until it is released. -This package is for app-provided Core AI model bundles exported from -[`apple/coreai-models`](https://github.com/apple/coreai-models). It is separate -from `@react-native-ai/apple`, which wraps Apple system capabilities such as -Foundation Models, NLContextualEmbedding, SpeechAnalyzer, and AVSpeechSynthesizer. +Use it when you want to choose and ship the model yourself. For the models +built into iOS (Apple Intelligence, system speech and embeddings), use +[`@react-native-ai/apple`](../apple-llm) instead. -## API Layers +## Features -Core AI exposes two layers: +As a Vercel AI SDK provider: -- AI SDK adapters: language models, image generation, transcription and - text embeddings (CLIP). -- Direct native APIs: model loading, persistent language sessions, model - inspection, segmentation, video segmentation, object detection, - super-resolution, classification, depth estimation, and raw `.aimodel` - function calls. `specializeModel` returns metadata only. +- Text generation and streaming with language models +- Image generation +- Video generation with `experimental_generateVideo` +- Transcription +- Text embeddings (CLIP) -Video generation runs through `coreAI.videoModel(...)` and AI SDK -`experimental_generateVideo`. Reranking and speech generation have no adapters, because -`coreai-models` has no runtime for them yet. +As native APIs: + +- Model loading, inspection and persistent language sessions +- Image and video segmentation, object detection, super-resolution, + classification and depth estimation +- Raw `.aimodel` function calls with tensors + +`specializeModel` returns metadata only. Reranking and speech generation are +not available, because `coreai-models` has no runtime for them yet. ## Installation @@ -69,15 +78,15 @@ Set `platform :ios, '27.0'` in `ios/Podfile` and the deployment target of the ap By default the pod links native support for every task the module implements. Each task links one `coreai-models` product: -| Task | Used by | Linked product | -| ------------------ | ----------------------------------------- | ----------------------- | -| `language` | `coreAI.languageModel(...)` and sessions | `CoreAILM` | -| `diffusion` | `coreAI.imageModel(...)` | `CoreAIDiffusion` | -| `segmentation` | `CoreAI.runTask('segmentation', ...)` | `CoreAISegmentation` | -| `object-detection` | `CoreAI.runTask('object-detection', ...)` | `CoreAIObjectDetection` | -| `asr` | `coreAI.transcriptionModel(...)` | `CoreAISpeech` | +| Task | Used by | Linked product | +| -------------------- | ------------------------------------------- | ------------------------- | +| `language` | `coreAI.languageModel(...)` and sessions | `CoreAILM` | +| `diffusion` | `coreAI.imageModel(...)` | `CoreAIDiffusion` | +| `segmentation` | `CoreAI.runTask('segmentation', ...)` | `CoreAISegmentation` | +| `object-detection` | `CoreAI.runTask('object-detection', ...)` | `CoreAIObjectDetection` | +| `asr` | `coreAI.transcriptionModel(...)` | `CoreAISpeech` | | `video-segmentation` | `CoreAI.runTask('video-segmentation', ...)` | `CoreAIVideoSegmentation` | -| `video-diffusion` | `coreAI.videoModel(...)` | `CoreAIVideoDiffusion` | +| `video-diffusion` | `coreAI.videoModel(...)` | `CoreAIVideoDiffusion` | To link fewer, list the tasks you use. With Expo, pass them to the plugin: @@ -155,7 +164,9 @@ await CoreAI.inspectModel(modelConfig) Core AI loads models from local `.aimodel` / `.aimodelc` files and model folders. `source` is a path or `file://` URL in app storage, or `bundle://` and a path in the app bundle for a model shipped with the app: ```ts -const downloaded = coreAI.languageModel({ source: `${modelsDirectory}/qwen3-0.6b` }) +const downloaded = coreAI.languageModel({ + source: `${modelsDirectory}/qwen3-0.6b`, +}) const bundled = coreAI.languageModel({ source: 'bundle://models/qwen3-0.6b' }) ``` From 5c5515f1b5fdfac3aa7be6c1f0baf3a60ddbc50b Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 12:10:55 +0400 Subject: [PATCH 03/11] docs: match the readmes to the core ai api on main Co-Authored-By: Claude Opus 5.5 --- README.md | 10 +++------- packages/core-ai/README.md | 2 +- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 319dce1a0..ab35dbb46 100644 --- a/README.md +++ b/README.md @@ -122,19 +122,15 @@ const { text } = await generateText({ Run model bundles exported with [`apple/coreai-models`](https://github.com/apple/coreai-models), such as Qwen3 or FLUX.2, on iOS 27 with Apple's Core AI framework. Supports text and image -generation through the AI SDK, plus native APIs for segmentation and object -detection. +generation, transcription and embeddings through the AI SDK, plus native APIs +for segmentation, object detection and other vision tasks. ```ts import { coreAI } from '@react-native-ai/core-ai' import { streamText } from 'ai' const result = streamText({ - model: coreAI.languageModel({ - id: 'qwen3-0.6b', - source: { type: 'file', uri: qwen3ModelDirectory }, - variant: 'iOS', - }), + model: coreAI.languageModel({ source: qwen3ModelDirectory }), prompt: 'Explain Core AI in one sentence.', }) ``` diff --git a/packages/core-ai/README.md b/packages/core-ai/README.md index a66547be4..e73e0ef5f 100644 --- a/packages/core-ai/README.md +++ b/packages/core-ai/README.md @@ -19,7 +19,7 @@ built into iOS (Apple Intelligence, system speech and embeddings), use As a Vercel AI SDK provider: -- Text generation and streaming with language models +- Text generation, streaming and tool calling with language models - Image generation - Video generation with `experimental_generateVideo` - Transcription From 6ee312f45f1119ad940e1d20be3e4bc5056567cf Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 12:25:37 +0400 Subject: [PATCH 04/11] docs: address review feedback on the readmes Co-Authored-By: Claude Opus 5.5 --- README.md | 24 +++++++++++++----------- packages/adk/README.md | 5 +++-- packages/apple-llm/README.md | 19 +++++++++++++++---- packages/core-ai/README.md | 4 ++-- packages/llama/README.md | 18 +++++++++--------- 5 files changed, 42 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index ab35dbb46..2ccf03e67 100644 --- a/README.md +++ b/README.md @@ -15,19 +15,19 @@ work with cloud ones. ## Providers -| Provider | Package | Platforms | Models | Download needed | -| ---------------------------------- | ------------------------------------------------ | ------------ | ------------------------------------------ | --------------- | -| [Apple](#apple) | [`@react-native-ai/apple`](./packages/apple-llm) | iOS | Apple Intelligence and system models | No | -| [Google ADK](#google-adk) | [`@react-native-ai/adk`](./packages/adk) | Android | Gemini Nano, optional cloud Gemini | No | -| [Llama](#llama) | [`@react-native-ai/llama`](./packages/llama) | iOS, Android | Any GGUF model, via llama.rn | Yes | -| [Core AI](#core-ai) (not released) | [`@react-native-ai/core-ai`](./packages/core-ai) | iOS 27+ | Core AI bundles from `apple/coreai-models` | Yes | +| Provider | Package | Platforms | Models | App downloads the model | +| ---------------------------------- | ------------------------------------------------ | ------------ | ------------------------------------------ | ----------------------- | +| [Apple](#apple) | [`@react-native-ai/apple`](./packages/apple-llm) | iOS | Apple Intelligence and system models | No | +| [Google ADK](#google-adk) | [`@react-native-ai/adk`](./packages/adk) | Android | Gemini Nano, optional cloud Gemini | No | +| [Llama](#llama) | [`@react-native-ai/llama`](./packages/llama) | iOS, Android | llama.cpp-compatible GGUF models | Yes | +| [Core AI](#core-ai) (not released) | [`@react-native-ai/core-ai`](./packages/core-ai) | iOS 27+ | Core AI bundles from `apple/coreai-models` | Yes | ### Apple Use the AI built into iOS: Apple Intelligence for text generation, NLContextualEmbedding for embeddings, SpeechAnalyzer for transcription and -AVSpeechSynthesizer for speech. Nothing to download, the models ship with the -system. +AVSpeechSynthesizer for speech. Your app has nothing to bundle or download, the +models ship with the system. ```bash npm install @react-native-ai/apple @@ -86,7 +86,7 @@ const { text } = await generateText({ ### Llama -Run any GGUF model from Hugging Face on iOS and Android with +Run llama.cpp-compatible GGUF models from Hugging Face on iOS and Android with [llama.rn](https://github.com/mybigday/llama.rn). Supports text generation, embeddings, reranking and speech. @@ -155,10 +155,12 @@ The AI SDK Profiler plugin captures OpenTelemetry spans from Vercel AI SDK requests and surfaces them in Rozenite DevTools. ```bash -npm install @react-native-ai/dev-tools +npm install @react-native-ai/dev-tools @ai-sdk/otel ``` -Rozenite must be installed and enabled in your app. See the +Register the profiler's tracer with `registerTelemetry` at app startup, see the +[package README](./packages/dev-tools) for the setup. Rozenite must be +installed and enabled in your app. See the [Rozenite getting started guide](https://www.rozenite.dev/docs/getting-started). The Expo demo app in [apps/expo-example](./apps/expo-example) diff --git a/packages/adk/README.md b/packages/adk/README.md index 174bc95ba..2c37a61e1 100644 --- a/packages/adk/README.md +++ b/packages/adk/README.md @@ -2,8 +2,9 @@ Use Gemini on Android from React Native, through Google's [Agent Development Kit (ADK)](https://developer.android.com/ai/adk). Gemini -Nano runs on-device and is provided by the system through AICore, so there is -no model to download or bundle and no API key. Cloud Gemini is available when +Nano runs on-device and is provided by the system through AICore, so your app +has no model to bundle and needs no API key. AICore downloads Nano when +`prepareNano()` is first called, if the device doesn't have it yet. Cloud Gemini is available when you need a larger model. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, so you can call both with `generateText` and `streamText`, including tool calling and image input. diff --git a/packages/apple-llm/README.md b/packages/apple-llm/README.md index f1ba99fcc..07ed8ea07 100644 --- a/packages/apple-llm/README.md +++ b/packages/apple-llm/README.md @@ -2,8 +2,8 @@ Use the AI built into iOS from React Native. Apple Intelligence (Foundation Models), text embeddings, speech-to-text and text-to-speech run on-device with -the models that ship with the system, so there is nothing to download or -bundle. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, +the models that ship with the system, so your app has no model to bundle or +download. The OS may still download model assets the first time they are used. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, so you can call these models with `generateText`, `streamText`, `embed`, `transcribe` and `generateSpeech`. @@ -39,7 +39,14 @@ transcription on iOS 27 and voice listing. npm install @react-native-ai/apple ai ``` -The package is autolinked, so there is no native setup. +The package is autolinked, so there is no native setup. The one exception is +live microphone transcription (`AppleTranscription.transcribeLive`), which +needs a microphone usage description in `Info.plist`: + +```xml +NSMicrophoneUsageDescription +Transcribe what you say. +``` ## Usage @@ -47,6 +54,9 @@ The package is autolinked, so there is no native setup. import { apple } from '@react-native-ai/apple' import { embed, generateSpeech, generateText, transcribe } from 'ai' +// Recorded audio, for example a WAV file read into a Uint8Array +declare const audioBuffer: Uint8Array + const { text } = await generateText({ model: apple(), prompt: 'Explain quantum computing in one sentence.', @@ -70,7 +80,8 @@ const { audio } = await generateSpeech({ Check `await apple.isAvailable()` before generating text, since Apple Intelligence can be off or still downloading. For long chats on the small -on-device context window, wrap the model with `@react-native-ai/utils`. +on-device context window, wrap the model with +[`@react-native-ai/utils`](../utils) (not published to npm yet). ## iOS 26 and iOS 27 diff --git a/packages/core-ai/README.md b/packages/core-ai/README.md index e73e0ef5f..ee2941818 100644 --- a/packages/core-ai/README.md +++ b/packages/core-ai/README.md @@ -4,8 +4,8 @@ Run your own models on iPhone with Apple's Core AI framework. Bring a model bundle exported with [`apple/coreai-models`](https://github.com/apple/coreai-models), such as Qwen3 or FLUX.2, and run it on-device from React Native. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, so the same models -work with `generateText`, `streamText`, `generateImage` and the other AI SDK -functions. +work with `generateText`, `streamText`, `generateImage`, `transcribe`, `embed` +and `experimental_generateVideo`. > [!WARNING] > Work in progress. This package is not published to npm yet. diff --git a/packages/llama/README.md b/packages/llama/README.md index 79fb31107..6cdd6b96d 100644 --- a/packages/llama/README.md +++ b/packages/llama/README.md @@ -4,8 +4,8 @@ Run open-weight GGUF models on iOS and Android with [llama.rn](https://github.com/mybigday/llama.rn). Download a model such as Qwen, Llama or Gemma from Hugging Face, load it into memory and run it fully on-device. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) -provider, so the same models work with `generateText`, `streamText`, `embed`, -`rerank` and `generateSpeech`. +provider for text generation, embeddings, reranking and speech, each with a +model trained for that task. Use it when you need the same models on both platforms, or a model the system doesn't provide. For models built into the OS, see @@ -14,12 +14,12 @@ doesn't provide. For models built into the OS, see ## Features -| Capability | Method | AI SDK function | -| --------------- | ------------------------ | --------------------------- | -| Text generation | `llama.languageModel()` | `generateText`/`streamText` | -| Embeddings | `llama.embeddingModel()` | `embed`/`embedMany` | -| Reranking | `llama.rerankingModel()` | `rerank` | -| Speech | `llama.speechModel()` | `generateSpeech` | +| Capability | Method | AI SDK function | Model | +| --------------- | ------------------------ | --------------------------- | --------------------------------------- | +| Text generation | `llama.languageModel()` | `generateText`/`streamText` | Chat or instruct model | +| Embeddings | `llama.embeddingModel()` | `embed`/`embedMany` | Embedding model | +| Reranking | `llama.rerankingModel()` | `rerank` | Reranker model | +| Speech | `llama.speechModel()` | `generateSpeech` | TTS model and a vocoder (`vocoderPath`) | Language models support: @@ -68,7 +68,7 @@ const { text } = await generateText({ await model.unload() ``` -Any GGUF model from Hugging Face works. +Any GGUF model from Hugging Face whose architecture llama.cpp supports works. ## Documentation From 6474c4867b7b41b7b5df3f8c36db9a0c0023bf6b Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 12:35:03 +0400 Subject: [PATCH 05/11] docs: install ai and unload the llama model in the readme Co-Authored-By: Claude Opus 5.5 --- README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 2ccf03e67..f01d189a7 100644 --- a/README.md +++ b/README.md @@ -26,11 +26,12 @@ work with cloud ones. Use the AI built into iOS: Apple Intelligence for text generation, NLContextualEmbedding for embeddings, SpeechAnalyzer for transcription and -AVSpeechSynthesizer for speech. Your app has nothing to bundle or download, the -models ship with the system. +AVSpeechSynthesizer for speech. The models ship with the system, so your app +has nothing to bundle, although iOS may download model assets the first time +they are used. ```bash -npm install @react-native-ai/apple +npm install @react-native-ai/apple ai ``` ```ts @@ -60,7 +61,7 @@ on-device and is provided by the system through AICore, so there are no model files and no API key. Cloud Gemini is available when you need a larger model. ```bash -npm install @react-native-ai/adk +npm install @react-native-ai/adk ai ``` > [!IMPORTANT] @@ -91,7 +92,7 @@ Run llama.cpp-compatible GGUF models from Hugging Face on iOS and Android with embeddings, reranking and speech. ```bash -npm install @react-native-ai/llama llama.rn react-native-blob-util +npm install @react-native-ai/llama llama.rn react-native-blob-util ai ``` ```ts @@ -110,6 +111,9 @@ const { text } = await generateText({ model, prompt: 'Write a haiku about coding.', }) + +// Release the native context when you no longer need the model +await model.unload() ``` [Package README](./packages/llama) ยท [Documentation](https://react-native-ai.dev/docs/llama/getting-started) From f778ba0cab26df2f7ec33d26f8fc3eb6dd5599f8 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 12:44:46 +0400 Subject: [PATCH 06/11] docs: load the core ai example model from the app bundle Co-Authored-By: Claude Opus 5.5 --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f01d189a7..fd5611843 100644 --- a/README.md +++ b/README.md @@ -134,7 +134,8 @@ import { coreAI } from '@react-native-ai/core-ai' import { streamText } from 'ai' const result = streamText({ - model: coreAI.languageModel({ source: qwen3ModelDirectory }), + // A model folder shipped in the app bundle, or a path in app storage + model: coreAI.languageModel({ source: 'bundle://models/qwen3-0.6b' }), prompt: 'Explain Core AI in one sentence.', }) ``` From 0fdeb68306ed886d8ecb78380fd93f78825e198d Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 13:13:53 +0400 Subject: [PATCH 07/11] docs: open the readme with a one-line summary Co-Authored-By: Claude Opus 5.5 --- README.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index fd5611843..6d65b0de0 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,16 @@ # React Native AI -Run AI models on users' devices from React Native, with first-class -[Vercel AI SDK](https://ai-sdk.dev) support. Each provider wraps an on-device -runtime and exposes it as an AI SDK provider, so `generateText`, `streamText`, -`embed` and the other AI SDK functions work with local models the same way they -work with cloud ones. +**On-device AI for React Native, built on the [Vercel AI SDK](https://ai-sdk.dev).** + +React Native AI is a set of packages that run AI models on your users' phones. +Each provider package wraps one on-device runtime, such as Apple Intelligence on iOS, +Gemini Nano on Android or llama.cpp on both, and exposes it as an AI SDK +provider. + +You write the same `generateText`, `streamText` or `embed` call you would for a +cloud model, and it runs locally: no server, no per-request cost, and the data +never leaves the device. - ๐Ÿš€ **Instant AI** - Use the models built into iOS and Android, with nothing to bundle - ๐Ÿ”’ **Privacy-first** - Inference runs on-device, so data stays local. Cloud models are opt-in From 05eb232b6547b8deb5c42c3d3312644ce4a2dc10 Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 13:14:35 +0400 Subject: [PATCH 08/11] docs: rewrap the readme intro Co-Authored-By: Claude Opus 5.5 --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 6d65b0de0..f4fd46196 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,9 @@ **On-device AI for React Native, built on the [Vercel AI SDK](https://ai-sdk.dev).** React Native AI is a set of packages that run AI models on your users' phones. -Each provider package wraps one on-device runtime, such as Apple Intelligence on iOS, -Gemini Nano on Android or llama.cpp on both, and exposes it as an AI SDK -provider. +Each provider package wraps one on-device runtime, such as Apple Intelligence +on iOS, Gemini Nano on Android or llama.cpp on both, and exposes it as an AI +SDK provider. You write the same `generateText`, `streamText` or `embed` call you would for a cloud model, and it runs locally: no server, no per-request cost, and the data From a3963a44e6810fb304765bd3d530d7f7d5cb07fa Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 13:15:45 +0400 Subject: [PATCH 09/11] docs: open each package readme with a one-line summary Co-Authored-By: Claude Opus 5.5 --- packages/adk/README.md | 15 ++++++++------- packages/apple-llm/README.md | 16 ++++++++++------ packages/core-ai/README.md | 15 +++++++++------ packages/dev-tools/README.md | 9 +++++---- packages/json-ui/README.md | 9 ++++++--- packages/llama/README.md | 14 ++++++++------ packages/utils/README.md | 8 +++++--- 7 files changed, 51 insertions(+), 35 deletions(-) diff --git a/packages/adk/README.md b/packages/adk/README.md index 2c37a61e1..76764d636 100644 --- a/packages/adk/README.md +++ b/packages/adk/README.md @@ -1,12 +1,13 @@ # @react-native-ai/adk -Use Gemini on Android from React Native, through Google's -[Agent Development Kit (ADK)](https://developer.android.com/ai/adk). Gemini -Nano runs on-device and is provided by the system through AICore, so your app -has no model to bundle and needs no API key. AICore downloads Nano when -`prepareNano()` is first called, if the device doesn't have it yet. Cloud Gemini is available when -you need a larger model. The package comes with a -[Vercel AI SDK](https://ai-sdk.dev) provider, so you can call both with +**Gemini on Android, as a Vercel AI SDK provider for React Native.** + +Gemini Nano runs on-device through Google's +[Agent Development Kit (ADK)](https://developer.android.com/ai/adk) and +AICore. Your app has no model to bundle and needs no API key. AICore downloads +Nano the first time `prepareNano()` runs, if the device doesn't have it yet. + +Cloud Gemini is available when you need a larger model. Both work with `generateText` and `streamText`, including tool calling and image input. To run models you choose and ship yourself on Android, see diff --git a/packages/apple-llm/README.md b/packages/apple-llm/README.md index 07ed8ea07..caf0a072b 100644 --- a/packages/apple-llm/README.md +++ b/packages/apple-llm/README.md @@ -1,11 +1,15 @@ # @react-native-ai/apple -Use the AI built into iOS from React Native. Apple Intelligence (Foundation -Models), text embeddings, speech-to-text and text-to-speech run on-device with -the models that ship with the system, so your app has no model to bundle or -download. The OS may still download model assets the first time they are used. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, -so you can call these models with `generateText`, `streamText`, `embed`, -`transcribe` and `generateSpeech`. +**Apple Intelligence and the other built-in iOS models, as a Vercel AI SDK provider for React Native.** + +Text generation (Foundation Models), embeddings, speech-to-text and +text-to-speech all run on-device with the models that ship with iOS. + +Your app has no model to bundle or download, although iOS may download model +assets the first time they are used. + +Call them with the usual AI SDK functions: `generateText`, `streamText`, +`embed`, `transcribe` and `generateSpeech`. To run models you choose and ship yourself, see [`@react-native-ai/core-ai`](../core-ai) or diff --git a/packages/core-ai/README.md b/packages/core-ai/README.md index ee2941818..353bc6b90 100644 --- a/packages/core-ai/README.md +++ b/packages/core-ai/README.md @@ -1,11 +1,14 @@ # @react-native-ai/core-ai -Run your own models on iPhone with Apple's Core AI framework. Bring a model -bundle exported with [`apple/coreai-models`](https://github.com/apple/coreai-models), -such as Qwen3 or FLUX.2, and run it on-device from React Native. The package -comes with a [Vercel AI SDK](https://ai-sdk.dev) provider, so the same models -work with `generateText`, `streamText`, `generateImage`, `transcribe`, `embed` -and `experimental_generateVideo`. +**Your own Core AI models on iPhone, as a Vercel AI SDK provider for React Native.** + +Bring a model bundle exported with +[`apple/coreai-models`](https://github.com/apple/coreai-models), such as Qwen3 +or FLUX.2, and run it on-device with Apple's Core AI framework. + +Call it with `generateText`, `streamText`, `generateImage`, `transcribe`, +`embed` and `experimental_generateVideo`, or use the native APIs for vision +tasks. > [!WARNING] > Work in progress. This package is not published to npm yet. diff --git a/packages/dev-tools/README.md b/packages/dev-tools/README.md index af6e67d1a..66b9aae95 100644 --- a/packages/dev-tools/README.md +++ b/packages/dev-tools/README.md @@ -2,10 +2,11 @@ # @react-native-ai/dev-tools -See every Vercel AI SDK call your React Native app makes. The AI SDK Profiler -is a [Rozenite](https://www.rozenite.dev) DevTools plugin that captures the -OpenTelemetry spans the AI SDK emits and shows their inputs, outputs, tool -calls, provider metadata and latency. +**See every Vercel AI SDK call your React Native app makes.** + +The AI SDK Profiler is a [Rozenite](https://www.rozenite.dev) DevTools plugin. +It captures the OpenTelemetry spans the AI SDK emits and shows their inputs, +outputs, tool calls, provider metadata and latency. ![AI SDK Profiler preview](../../website/src/public/dev-tools-preview.png) diff --git a/packages/json-ui/README.md b/packages/json-ui/README.md index 3f38559e6..c2cde67dd 100644 --- a/packages/json-ui/README.md +++ b/packages/json-ui/README.md @@ -1,8 +1,11 @@ # @react-native-ai/json-ui -Let a language model build React Native UI. The model calls tools that add, -update and remove nodes in a JSON spec, and `GenerativeUIView` renders it. It -plugs into Vercel AI SDK tool calling, and is designed for small on-device +**Let a language model build React Native UI.** + +The model calls tools that add, update and remove nodes in a JSON spec, and +`GenerativeUIView` renders it. + +It plugs into Vercel AI SDK tool calling and is designed for small on-device models with limited context. **This package provides:** diff --git a/packages/llama/README.md b/packages/llama/README.md index 6cdd6b96d..eddf7794c 100644 --- a/packages/llama/README.md +++ b/packages/llama/README.md @@ -1,11 +1,13 @@ # @react-native-ai/llama -Run open-weight GGUF models on iOS and Android with -[llama.rn](https://github.com/mybigday/llama.rn). Download a model such as -Qwen, Llama or Gemma from Hugging Face, load it into memory and run it fully -on-device. The package comes with a [Vercel AI SDK](https://ai-sdk.dev) -provider for text generation, embeddings, reranking and speech, each with a -model trained for that task. +**Open-weight GGUF models on iOS and Android, as a Vercel AI SDK provider for React Native.** + +Download a model such as Qwen, Llama or Gemma from Hugging Face, load it into +memory and run it fully on-device with +[llama.rn](https://github.com/mybigday/llama.rn). + +The provider covers text generation, embeddings, reranking and speech, each +with a model trained for that task. Use it when you need the same models on both platforms, or a model the system doesn't provide. For models built into the OS, see diff --git a/packages/utils/README.md b/packages/utils/README.md index 935526585..cb9d946cb 100644 --- a/packages/utils/README.md +++ b/packages/utils/README.md @@ -1,8 +1,10 @@ # @react-native-ai/utils -Utilities for running Vercel AI SDK language models on small on-device context -windows. It works with any AI SDK language model, including the Apple, Llama -and ADK providers. +**Fit long chats into small on-device context windows.** + +`wrapLanguageModelWithHistory` wraps any Vercel AI SDK language model, +including the Apple, Llama and ADK providers, and compacts the conversation +before each request. > Not published to npm yet. Use it from this monorepo, or copy > `src/index.ts` into your app. Its only dependency is `@ai-sdk/provider`. From 5dc17ae7a1744cdf6b5d94f85f4856d1babce8cf Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 13:20:40 +0400 Subject: [PATCH 10/11] docs: group readme providers into built-in and custom models Co-Authored-By: Claude Opus 5.5 --- README.md | 58 +++++++++++++++++++++++++------------------------------ 1 file changed, 26 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index f4fd46196..39daeeb15 100644 --- a/README.md +++ b/README.md @@ -20,20 +20,33 @@ never leaves the device. ## Providers -| Provider | Package | Platforms | Models | App downloads the model | -| ---------------------------------- | ------------------------------------------------ | ------------ | ------------------------------------------ | ----------------------- | -| [Apple](#apple) | [`@react-native-ai/apple`](./packages/apple-llm) | iOS | Apple Intelligence and system models | No | -| [Google ADK](#google-adk) | [`@react-native-ai/adk`](./packages/adk) | Android | Gemini Nano, optional cloud Gemini | No | -| [Llama](#llama) | [`@react-native-ai/llama`](./packages/llama) | iOS, Android | llama.cpp-compatible GGUF models | Yes | -| [Core AI](#core-ai) (not released) | [`@react-native-ai/core-ai`](./packages/core-ai) | iOS 27+ | Core AI bundles from `apple/coreai-models` | Yes | +Providers come in two kinds: built-in models that ship with the operating +system, and custom models that your app downloads or bundles. -### Apple +**Built-in models** -Use the AI built into iOS: Apple Intelligence for text generation, -NLContextualEmbedding for embeddings, SpeechAnalyzer for transcription and -AVSpeechSynthesizer for speech. The models ship with the system, so your app -has nothing to bundle, although iOS may download model assets the first time -they are used. +| Provider | Package | Platform | Models | +| ----------------------------------------- | ------------------------------------------------ | -------- | ----------------------------------------------------------------------------------------------------- | +| [Apple Intelligence](#apple-intelligence) | [`@react-native-ai/apple`](./packages/apple-llm) | iOS | Foundation Models on-device and on Private Cloud Compute, system embeddings, transcription and speech | +| [Google ADK](#google-adk) | [`@react-native-ai/adk`](./packages/adk) | Android | Gemini Nano on-device, optional cloud Gemini | + +**Custom models** + +| Provider | Package | Platform | Models | +| ---------------------------------- | ------------------------------------------------ | ------------ | ------------------------------------------------- | +| [Llama](#llama) | [`@react-native-ai/llama`](./packages/llama) | iOS, Android | llama.cpp-compatible GGUF models | +| [Core AI](#core-ai) (not released) | [`@react-native-ai/core-ai`](./packages/core-ai) | iOS 27+ | Model bundles exported with `apple/coreai-models` | + +**Llama or Core AI?** Use Llama to run the same model on iOS and Android, or +a GGUF model from Hugging Face. Use Core AI when your app targets iOS 27+ and +you want Apple's own runtime, or need what Llama doesn't cover: image and video +generation, and vision tasks such as segmentation and object detection. + +### Apple Intelligence + +Use the models built into iOS: Foundation Models for text generation, on-device +or on Private Cloud Compute, plus system models for embeddings, transcription +and speech. Your app has nothing to bundle. ```bash npm install @react-native-ai/apple ai @@ -49,13 +62,6 @@ const { text } = await generateText({ }) ``` -| Feature | iOS | Additional requirements | -| ---------------- | ------- | -------------------------- | -| Text generation | iOS 26+ | Apple Intelligence device | -| Embeddings | iOS 17+ | - | -| Transcription | iOS 26+ | - | -| Speech synthesis | iOS 13+ | iOS 17+ for Personal Voice | - [Package README](./packages/apple-llm) ยท [Documentation](https://react-native-ai.dev/docs/apple/getting-started) ### Google ADK @@ -69,19 +75,10 @@ files and no API key. Cloud Gemini is available when you need a larger model. npm install @react-native-ai/adk ai ``` -> [!IMPORTANT] -> Requires Android `minSdkVersion` 26 and a physical device with AICore support (Android 14+) for Gemini Nano. Your app also needs packaging excludes for duplicate `META-INF` files, see the [getting started guide](https://react-native-ai.dev/docs/adk/getting-started). - ```ts import { adk } from '@react-native-ai/adk' import { generateText } from 'ai' -if (!(await adk.isAvailable())) { - throw new Error('Gemini Nano is not available on this device') -} - -await adk.prepareNano() - const { text } = await generateText({ model: adk(), prompt: 'Explain quantum computing', @@ -125,10 +122,7 @@ await model.unload() ### Core AI -> [!WARNING] -> Work in progress. This package is not published to npm yet. - -Run model bundles exported with +Not published to npm yet. Run model bundles exported with [`apple/coreai-models`](https://github.com/apple/coreai-models), such as Qwen3 or FLUX.2, on iOS 27 with Apple's Core AI framework. Supports text and image generation, transcription and embeddings through the AI SDK, plus native APIs From e73ff3c4e38b305527ec6232c2d42f2b2cd30c1f Mon Sep 17 00:00:00 2001 From: Mike Grabowski Date: Tue, 27 Oct 2026 13:30:47 +0400 Subject: [PATCH 11/11] docs: lead package readmes with what they enable and trim the main readme Co-Authored-By: Claude Opus 5.5 --- README.md | 43 +++++------------------------------- packages/adk/README.md | 15 +++++++------ packages/apple-llm/README.md | 20 ++++++++--------- packages/core-ai/README.md | 17 +++++++------- packages/dev-tools/README.md | 9 ++++---- packages/json-ui/README.md | 9 +++----- packages/llama/README.md | 13 +++++------ packages/utils/README.md | 3 +-- 8 files changed, 45 insertions(+), 84 deletions(-) diff --git a/README.md b/README.md index 39daeeb15..7d4a59572 100644 --- a/README.md +++ b/README.md @@ -9,15 +9,6 @@ Each provider package wraps one on-device runtime, such as Apple Intelligence on iOS, Gemini Nano on Android or llama.cpp on both, and exposes it as an AI SDK provider. -You write the same `generateText`, `streamText` or `embed` call you would for a -cloud model, and it runs locally: no server, no per-request cost, and the data -never leaves the device. - -- ๐Ÿš€ **Instant AI** - Use the models built into iOS and Android, with nothing to bundle -- ๐Ÿ”’ **Privacy-first** - Inference runs on-device, so data stays local. Cloud models are opt-in -- ๐ŸŽฏ **Vercel AI SDK compatible** - Swap local and cloud models without changing your code -- ๐ŸŽจ **Beyond text** - Embeddings, reranking, transcription, speech and image generation - ## Providers Providers come in two kinds: built-in models that ship with the operating @@ -37,10 +28,11 @@ system, and custom models that your app downloads or bundles. | [Llama](#llama) | [`@react-native-ai/llama`](./packages/llama) | iOS, Android | llama.cpp-compatible GGUF models | | [Core AI](#core-ai) (not released) | [`@react-native-ai/core-ai`](./packages/core-ai) | iOS 27+ | Model bundles exported with `apple/coreai-models` | -**Llama or Core AI?** Use Llama to run the same model on iOS and Android, or -a GGUF model from Hugging Face. Use Core AI when your app targets iOS 27+ and -you want Apple's own runtime, or need what Llama doesn't cover: image and video -generation, and vision tasks such as segmentation and object detection. +> [!TIP] +> **Llama or Core AI?** Use Llama to run the same model on iOS and Android, or +> a GGUF model from Hugging Face. Use Core AI when your app targets iOS 27+ and +> you want Apple's own runtime, or need what Llama doesn't cover: image and +> video generation, and vision tasks such as segmentation and object detection. ### Apple Intelligence @@ -151,31 +143,6 @@ These work with any AI SDK provider, on-device or cloud. | [`@react-native-ai/dev-tools`](./packages/dev-tools) | Inspect AI SDK calls in Rozenite DevTools | | [`@react-native-ai/utils`](./packages/utils) | Compact conversation history for small context windows (not released yet) | -### DevTools - -![AI SDK Profiler preview](website/src/public/dev-tools-preview.png) - -The AI SDK Profiler plugin captures OpenTelemetry spans from Vercel AI SDK -requests and surfaces them in Rozenite DevTools. - -```bash -npm install @react-native-ai/dev-tools @ai-sdk/otel -``` - -Register the profiler's tracer with `registerTelemetry` at app startup, see the -[package README](./packages/dev-tools) for the setup. Rozenite must be -installed and enabled in your app. See the -[Rozenite getting started guide](https://www.rozenite.dev/docs/getting-started). - -The Expo demo app in [apps/expo-example](./apps/expo-example) -includes native-development Rozenite wiring for this plugin. Run the example -app, open React Native DevTools, and select `AI SDK Profiler` to inspect chat -spans emitted by the Vercel AI SDK. - -If the `AI SDK Profiler` panel stays empty after you send a message, close the -React Native DevTools window and open a new one. A stale debugger session can -keep the panel mounted without receiving telemetry. - ## AI SDK Compatibility | React Native AI | AI SDK | diff --git a/packages/adk/README.md b/packages/adk/README.md index 76764d636..8973a0e1a 100644 --- a/packages/adk/README.md +++ b/packages/adk/README.md @@ -1,14 +1,15 @@ # @react-native-ai/adk -**Gemini on Android, as a Vercel AI SDK provider for React Native.** - -Gemini Nano runs on-device through Google's +Add Gemini to your Android app without shipping a model or an API key. Gemini +Nano runs on-device through Google's [Agent Development Kit (ADK)](https://developer.android.com/ai/adk) and -AICore. Your app has no model to bundle and needs no API key. AICore downloads -Nano the first time `prepareNano()` runs, if the device doesn't have it yet. +AICore, for chat, summaries and tool-using assistants that keep data on the +device. AICore downloads Nano the first time `prepareNano()` runs, if the +device doesn't have it yet. When you need a larger model, switch to cloud +Gemini. -Cloud Gemini is available when you need a larger model. Both work with -`generateText` and `streamText`, including tool calling and image input. +It also comes as a [Vercel AI SDK](https://ai-sdk.dev) provider, so it drops +into code that already uses the AI SDK. To run models you choose and ship yourself on Android, see [`@react-native-ai/llama`](../llama). diff --git a/packages/apple-llm/README.md b/packages/apple-llm/README.md index caf0a072b..075f28ad4 100644 --- a/packages/apple-llm/README.md +++ b/packages/apple-llm/README.md @@ -1,15 +1,15 @@ # @react-native-ai/apple -**Apple Intelligence and the other built-in iOS models, as a Vercel AI SDK provider for React Native.** - -Text generation (Foundation Models), embeddings, speech-to-text and -text-to-speech all run on-device with the models that ship with iOS. - -Your app has no model to bundle or download, although iOS may download model -assets the first time they are used. - -Call them with the usual AI SDK functions: `generateText`, `streamText`, -`embed`, `transcribe` and `generateSpeech`. +Add AI features to your iOS app without shipping a model. Chat, summaries, +structured data extraction and tool-using assistants run on Apple Intelligence, +and semantic search, voice input and spoken output run on system models. Data +stays on the device, there are no API costs, and your app stays small, since +iOS provides the models. + +When a task needs more than the on-device model, iOS 27 can send it to Apple's +Private Cloud Compute instead. It also comes as a +[Vercel AI SDK](https://ai-sdk.dev) provider, so it drops into code that +already uses the AI SDK. To run models you choose and ship yourself, see [`@react-native-ai/core-ai`](../core-ai) or diff --git a/packages/core-ai/README.md b/packages/core-ai/README.md index 353bc6b90..4b860a61e 100644 --- a/packages/core-ai/README.md +++ b/packages/core-ai/README.md @@ -1,14 +1,13 @@ # @react-native-ai/core-ai -**Your own Core AI models on iPhone, as a Vercel AI SDK provider for React Native.** - -Bring a model bundle exported with -[`apple/coreai-models`](https://github.com/apple/coreai-models), such as Qwen3 -or FLUX.2, and run it on-device with Apple's Core AI framework. - -Call it with `generateText`, `streamText`, `generateImage`, `transcribe`, -`embed` and `experimental_generateVideo`, or use the native APIs for vision -tasks. +Bring your own model to iPhone and run it on Apple's Core AI framework. Export +a model with [`apple/coreai-models`](https://github.com/apple/coreai-models), +such as Qwen3 for chat or FLUX.2 for images, and run it on-device from React +Native. Use it for text, image and video generation, transcription, embeddings, +and vision tasks such as segmentation and object detection. + +It also comes as a [Vercel AI SDK](https://ai-sdk.dev) provider, so it drops +into code that already uses the AI SDK. > [!WARNING] > Work in progress. This package is not published to npm yet. diff --git a/packages/dev-tools/README.md b/packages/dev-tools/README.md index 66b9aae95..af6e67d1a 100644 --- a/packages/dev-tools/README.md +++ b/packages/dev-tools/README.md @@ -2,11 +2,10 @@ # @react-native-ai/dev-tools -**See every Vercel AI SDK call your React Native app makes.** - -The AI SDK Profiler is a [Rozenite](https://www.rozenite.dev) DevTools plugin. -It captures the OpenTelemetry spans the AI SDK emits and shows their inputs, -outputs, tool calls, provider metadata and latency. +See every Vercel AI SDK call your React Native app makes. The AI SDK Profiler +is a [Rozenite](https://www.rozenite.dev) DevTools plugin that captures the +OpenTelemetry spans the AI SDK emits and shows their inputs, outputs, tool +calls, provider metadata and latency. ![AI SDK Profiler preview](../../website/src/public/dev-tools-preview.png) diff --git a/packages/json-ui/README.md b/packages/json-ui/README.md index c2cde67dd..3f38559e6 100644 --- a/packages/json-ui/README.md +++ b/packages/json-ui/README.md @@ -1,11 +1,8 @@ # @react-native-ai/json-ui -**Let a language model build React Native UI.** - -The model calls tools that add, update and remove nodes in a JSON spec, and -`GenerativeUIView` renders it. - -It plugs into Vercel AI SDK tool calling and is designed for small on-device +Let a language model build React Native UI. The model calls tools that add, +update and remove nodes in a JSON spec, and `GenerativeUIView` renders it. It +plugs into Vercel AI SDK tool calling, and is designed for small on-device models with limited context. **This package provides:** diff --git a/packages/llama/README.md b/packages/llama/README.md index eddf7794c..ed7876079 100644 --- a/packages/llama/README.md +++ b/packages/llama/README.md @@ -1,13 +1,12 @@ # @react-native-ai/llama -**Open-weight GGUF models on iOS and Android, as a Vercel AI SDK provider for React Native.** +Pick an open-weight model and run it on iOS and Android. Download a GGUF model +such as Qwen, Llama or Gemma from Hugging Face and run it fully on-device with +[llama.rn](https://github.com/mybigday/llama.rn), for chat, semantic search, +reranking or speech, with the same model on both platforms. -Download a model such as Qwen, Llama or Gemma from Hugging Face, load it into -memory and run it fully on-device with -[llama.rn](https://github.com/mybigday/llama.rn). - -The provider covers text generation, embeddings, reranking and speech, each -with a model trained for that task. +It also comes as a [Vercel AI SDK](https://ai-sdk.dev) provider, so it drops +into code that already uses the AI SDK. Use it when you need the same models on both platforms, or a model the system doesn't provide. For models built into the OS, see diff --git a/packages/utils/README.md b/packages/utils/README.md index cb9d946cb..e544cca72 100644 --- a/packages/utils/README.md +++ b/packages/utils/README.md @@ -1,7 +1,6 @@ # @react-native-ai/utils -**Fit long chats into small on-device context windows.** - +Keep long chats working on small on-device context windows. `wrapLanguageModelWithHistory` wraps any Vercel AI SDK language model, including the Apple, Llama and ADK providers, and compacts the conversation before each request.