feat(ai)!: Add explicit initializeDeviceModel() method when using hybrid#9800
feat(ai)!: Add explicit initializeDeviceModel() method when using hybrid#9800
initializeDeviceModel() method when using hybrid#9800Conversation
🦋 Changeset detectedLatest commit: 615612e The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Vertex AI Mock Responses Check
|
There was a problem hiding this comment.
Code Review
This pull request introduces the initializeDeviceModel method to the GenerativeModel class, enabling explicit initialization and progress tracking for on-device models. The implementation updates the ChromeAdapter to handle download progress via a new monitor property in the creation options. Feedback highlights several areas for improvement: the downloadIfAvailable logic should account for the DOWNLOADING state to avoid premature resolution; initializeDeviceModel should throw an error when the model is unavailable in ONLY_ON_DEVICE mode for consistency; type assertions should be replaced by proper interface definitions; and shared configuration objects should not be mutated to avoid side effects across sessions.
Chrome needs to download the on-device model before it can run on-device inference. This download is triggered when Chrome's LanguageModel.create() is called. Previously, the SDK made this call automatically when an inference method like generateContent() was called in hybrid mode (if mode is set to PREFER_ON_DEVICE or PREFER_IN_CLOUD).
Since the download (1) takes several minutes and (2) will error if there has been no user interaction with the page, we thought it better to make an explicit method to initialize on-device mode and kick off the download. This method is called
initializeDeviceModel()and resolves when the download is complete. Developers can gate this to ensure it happens after a user interaction, and handle the wait time by having the user wait, or simply allow the cloud fallback in the meantime.This PR also adds a way to provide a download monitor callback, to track download progress.
This is a breaking change and will go out with the GA release of the hybrid feature.