Skip to content

feat(ai)!: Add explicit initializeDeviceModel() method when using hybrid#9800

Draft
hsubox76 wants to merge 7 commits intomainfrom
ch-initialize-device
Draft

feat(ai)!: Add explicit initializeDeviceModel() method when using hybrid#9800
hsubox76 wants to merge 7 commits intomainfrom
ch-initialize-device

Conversation

@hsubox76
Copy link
Copy Markdown
Contributor

@hsubox76 hsubox76 commented Apr 1, 2026

Chrome needs to download the on-device model before it can run on-device inference. This download is triggered when Chrome's LanguageModel.create() is called. Previously, the SDK made this call automatically when an inference method like generateContent() was called in hybrid mode (if mode is set to PREFER_ON_DEVICE or PREFER_IN_CLOUD).

Since the download (1) takes several minutes and (2) will error if there has been no user interaction with the page, we thought it better to make an explicit method to initialize on-device mode and kick off the download. This method is called initializeDeviceModel() and resolves when the download is complete. Developers can gate this to ensure it happens after a user interaction, and handle the wait time by having the user wait, or simply allow the cloud fallback in the meantime.

This PR also adds a way to provide a download monitor callback, to track download progress.

This is a breaking change and will go out with the GA release of the hybrid feature.

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 1, 2026

🦋 Changeset detected

Latest commit: 615612e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@firebase/ai Minor
firebase Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 1, 2026

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v16.2

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the initializeDeviceModel method to the GenerativeModel class, enabling explicit initialization and progress tracking for on-device models. The implementation updates the ChromeAdapter to handle download progress via a new monitor property in the creation options. Feedback highlights several areas for improvement: the downloadIfAvailable logic should account for the DOWNLOADING state to avoid premature resolution; initializeDeviceModel should throw an error when the model is unavailable in ONLY_ON_DEVICE mode for consistency; type assertions should be replaced by proper interface definitions; and shared configuration objects should not be mutated to avoid side effects across sessions.

Comment thread packages/ai/src/methods/chrome-adapter.ts Outdated
Comment thread packages/ai/src/models/generative-model.ts
Comment thread packages/ai/src/models/generative-model.ts
Comment thread packages/ai/src/methods/chrome-adapter.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant