Add Genie NPU backend support (Qualcomm Snapdragon)#446
Merged
shubhammalhotra28 merged 28 commits intomainfrom Mar 20, 2026
Merged
Add Genie NPU backend support (Qualcomm Snapdragon)#446shubhammalhotra28 merged 28 commits intomainfrom
shubhammalhotra28 merged 28 commits intomainfrom
Conversation
- Add RAC_FRAMEWORK_GENIE (=10) and RAC_MODEL_FORMAT_QNN_CONTEXT (=5) across C++ commons, Kotlin SDK, and bridge layers - Register Genie as LLM backend with priority 200 (preferred over CPU) - Example app: add 3 Genie NPU models (Qwen 2.5 7B, Llama 3.2 3B/1B) - Example app: prefer Genie models in ChatViewModel model selection - AndroidManifest: add libcdsprpc.so uses-native-library for QNN HTP - Consume Genie AAR from private repo via Maven coordinate - Build scripts: stage Genie pre-built .so from local/sibling paths - Add ALOGD debug logging in LLM service + service registry for NPU debugging - Flutter: add .gitkeep placeholders for jniLibs/Frameworks directories Genie C++ backend and Kotlin wrapper live in separate private repo: https://github.com/RunanywhereAI/runanywhere-genie Made-with: Cursor
- ModelList.kt: Replace empty URLs with HuggingFace download URLs for Llama 3.2 1B and Qwen 2.5 7B Genie NPU models hosted at runanywhere/genie-npu-models - ModelList.kt: Remove genieLocalPaths map, CppBridgeModelRegistry local path logic, and Llama 3.2 3B entry (no compiled binaries) - build-android.sh: Remove Genie .so staging block (belongs in private repo, not monorepo) - build-kotlin.sh: Remove GENIE_JNILIBS_DIR and Genie .so copy logic Models uploaded to: https://huggingface.co/runanywhere/genie-npu-models Made-with: Cursor
Resolves 4 merge conflicts (all in Gradle/settings files) caused by RAG being folded into the core SDK on main. Keeps Genie additions alongside the updated RAG architecture. Made-with: Cursor
- Add RAC_MODEL_FORMAT_QNN_CONTEXT case to rac_model_detect_framework_from_format() - Add "qnn_context"/"qnn" format string parsing in model_assignment.cpp - Add QNN_CONTEXT to Kotlin CppBridgeModelRegistry.ModelFormat and ModelFormat enum - Wire QNN_CONTEXT through all bridge mapping when blocks (ModelManagement, Storage) - Fix ALOGD fprintf fallback to append newline on non-Android builds - Update Genie AAR dependency version to 0.2.0-SNAPSHOT Made-with: Cursor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds Qualcomm Genie (GENIE/QNN) support across samples and SDK: new Android dependency and manifest entries, public enum values for GENIE/QNN in C++ and Kotlin, model-management parsing/mappings, Genie-first model selection in apps, and added diagnostic logging in service/LLM flows. Changes
Sequence Diagram(s)sequenceDiagram
participant App as Sample App
participant Kotlin as Kotlin SDK
participant Cpp as C++ Model Registry
participant Service as LLM Service Layer
App->>Kotlin: requestAvailableModels()
Kotlin->>Cpp: queryDiscoveredModels()
Cpp-->>Kotlin: return(models including GENIE)
Kotlin->>App: recommendModel(prefer=GENIE)
App->>Service: createServiceRequest(framework=GENIE, model_path)
Service->>Service: evaluateProviders()
Service-->>App: serviceCreated / error
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt (1)
197-212:⚠️ Potential issue | 🟠 MajorTreat Genie as optional in both setup and catalog registration.
Genie.register()now shares the same failure path as the required backends, so one Genie init error abortssetupModels()before the existing LlamaCPP/ONNX models are registered.genieModelsare also always appended toallModels, which means unsupported devices can still see NPU-only entries. Split Genie into its own guarded path and only add that group when Genie is actually available.Proposed fix
fun setupModels() { Timber.i("Registering backends and models...") + var genieRegistered = false try { LlamaCPP.register(priority = 100) ONNX.register(priority = 100) - Genie.register(priority = 200) Timber.i("Backends registered") } catch (e: Exception) { Timber.e(e, "Failed to register backends") return } + + try { + Genie.register(priority = 200) + genieRegistered = true + } catch (e: Exception) { + Timber.w(e, "Genie backend unavailable; continuing without NPU models") + } - val allModels = listOf( - "LLM/STT/TTS" to (llmModels + sttModels + ttsModels), - "Genie NPU" to genieModels, - "Embedding" to embeddingModels, - "VLM" to vlmModels, - ) + val allModels = buildList { + add("LLM/STT/TTS" to (llmModels + sttModels + ttsModels)) + if (genieRegistered) add("Genie NPU" to genieModels) + add("Embedding" to embeddingModels) + add("VLM" to vlmModels) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt` around lines 197 - 212, The current setup calls Genie.register() inside the same try/catch as LlamaCPP.register and ONNX.register, so a Genie init failure aborts setupModels() and the "Genie NPU" group is always added; change this by keeping LlamaCPP.register and ONNX.register in the existing try/catch but move Genie.register into its own guarded try/catch (or use a boolean genieAvailable flag set only if Genie.register() succeeds), log any Genie error without returning, and only append the "Genie NPU" entry to allModels when genieAvailable is true (use the existing symbols LlamaCPP.register, ONNX.register, Genie.register, genieModels, allModels, setupModels to locate and update the code).sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt (1)
1347-1361:⚠️ Potential issue | 🔴 CriticalUse bridge constants here; the current framework ids are wrong.
This parser does not match
CppBridgeModelRegistry.Framework: Line 1357 maps2toONNX, but the bridge defines2asFOUNDATION_MODELSandONNXas0. That will misparse assignment payloads and can send models to the wrong backend. Please decode throughCppBridgeModelRegistry.Framework.*/ModelFormat.*instead of duplicating raw ints.Suggested fix
format = when (formatInt) { - 1 -> ModelFormat.GGUF - 2 -> ModelFormat.ONNX - 3 -> ModelFormat.ORT - 5 -> ModelFormat.QNN_CONTEXT + CppBridgeModelRegistry.ModelFormat.GGUF -> ModelFormat.GGUF + CppBridgeModelRegistry.ModelFormat.ONNX -> ModelFormat.ONNX + CppBridgeModelRegistry.ModelFormat.ORT -> ModelFormat.ORT + CppBridgeModelRegistry.ModelFormat.BIN -> ModelFormat.BIN + CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT -> ModelFormat.QNN_CONTEXT else -> ModelFormat.UNKNOWN }, framework = when (frameworkInt) { - 1 -> InferenceFramework.LLAMA_CPP - 2 -> InferenceFramework.ONNX - 3 -> InferenceFramework.FOUNDATION_MODELS - 4 -> InferenceFramework.SYSTEM_TTS - 10 -> InferenceFramework.GENIE + CppBridgeModelRegistry.Framework.LLAMACPP -> InferenceFramework.LLAMA_CPP + CppBridgeModelRegistry.Framework.ONNX -> InferenceFramework.ONNX + CppBridgeModelRegistry.Framework.FOUNDATION_MODELS -> InferenceFramework.FOUNDATION_MODELS + CppBridgeModelRegistry.Framework.SYSTEM_TTS -> InferenceFramework.SYSTEM_TTS + CppBridgeModelRegistry.Framework.FLUID_AUDIO -> InferenceFramework.FLUID_AUDIO + CppBridgeModelRegistry.Framework.BUILTIN -> InferenceFramework.BUILT_IN + CppBridgeModelRegistry.Framework.NONE -> InferenceFramework.NONE + CppBridgeModelRegistry.Framework.GENIE -> InferenceFramework.GENIE else -> InferenceFramework.UNKNOWN },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt around lines 1347 - 1361, The switch/mapping for formatInt and frameworkInt is using hard-coded numeric IDs that do not match the bridge enums; update the mapping logic to decode using the bridge constants (CppBridgeModelRegistry.Framework.* and CppBridgeModelRegistry.ModelFormat.*) instead of raw ints so ModelFormat and InferenceFramework are derived from the bridge definitions; locate the mappings around the formatInt/frameworkInt handling (the when blocks producing ModelFormat and InferenceFramework) and replace the numeric-based branches with lookups that translate the incoming int via the bridge enum/constants (e.g., compare to or convert from CppBridgeModelRegistry.Framework/ModelFormat values) so the returned ModelFormat and InferenceFramework align with the bridge.
🧹 Nitpick comments (1)
examples/android/RunAnywhereAI/app/src/main/AndroidManifest.xml (1)
30-30: RedundantextractNativeLibssetting.The
android:extractNativeLibs="true"attribute is redundant becauseuseLegacyPackaging = trueis already set inapp/build.gradle.kts(lines 139-145). Both achieve the same effect—extracting native libraries to the filesystem. The Gradle DSL setting is the preferred approach and takes precedence.Consider removing this manifest attribute to avoid confusion about which setting is authoritative.
🧹 Suggested removal
android:largeHeap="true" - android:extractNativeLibs="true" android:usesCleartextTraffic="false"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/android/RunAnywhereAI/app/src/main/AndroidManifest.xml` at line 30, Remove the redundant android:extractNativeLibs="true" attribute from the AndroidManifest.xml entry because the project already sets useLegacyPackaging = true in app/build.gradle.kts (which handles native lib extraction and is the preferred, authoritative setting); delete the android:extractNativeLibs attribute so only useLegacyPackaging controls native library extraction and avoid conflicting/duplicative configuration.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@sdk/runanywhere-commons/include/rac/infrastructure/model_management/rac_model_types.h`:
- Around line 190-192: Add the new enum variants from the C++ header into the
Dart public types by updating ModelFormat and InferenceFramework in
model_types.dart (add the missing cases that correspond to RAC_FRAMEWORK_GENIE,
RAC_FRAMEWORK_WHISPERKIT_COREML, RAC_FRAMEWORK_MLX, RAC_FRAMEWORK_COREML and the
new QNN format), then update the bridge mapping functions in
dart_bridge_model_registry.dart—specifically _frameworkToFfi(), _formatToFfi()
and their reverse mapping functions—to include switch branches that map each new
Dart enum case to the correct C++ integer value and vice‑versa; ensure all new
cases are covered and add a default/fallback branch for unknown values.
In `@sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp`:
- Around line 22-27: The non-Android ALOGD macro uses fprintf and stderr but the
translation unit doesn't include <cstdio>, causing build failures; add an
`#include` <cstdio> at the top of service_registry.cpp (before the ALOGD macro
definition) so the symbols fprintf and stderr are declared; ensure the ALOGD
macro block (the else branch defining ALOGD) remains unchanged and references
fprintf and stderr as before.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt`:
- Around line 80-83: The constants in CppBridgeModelRegistry.kt currently assign
the same integer to COREML and QNN_CONTEXT, making raw format 5 ambiguous;
update the constants (BIN, COREML, QNN_CONTEXT, TFLITE) so each maps 1:1 to the
native C++ enum values (use the native enum as the source of truth) and ensure
COREML and QNN_CONTEXT have distinct integers; adjust the inline comment(s) to
reflect the correct C++ values and verify any when/compare logic that consumes
these constants still matches the native enum ordering.
---
Outside diff comments:
In
`@examples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.kt`:
- Around line 197-212: The current setup calls Genie.register() inside the same
try/catch as LlamaCPP.register and ONNX.register, so a Genie init failure aborts
setupModels() and the "Genie NPU" group is always added; change this by keeping
LlamaCPP.register and ONNX.register in the existing try/catch but move
Genie.register into its own guarded try/catch (or use a boolean genieAvailable
flag set only if Genie.register() succeeds), log any Genie error without
returning, and only append the "Genie NPU" entry to allModels when
genieAvailable is true (use the existing symbols LlamaCPP.register,
ONNX.register, Genie.register, genieModels, allModels, setupModels to locate and
update the code).
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt:
- Around line 1347-1361: The switch/mapping for formatInt and frameworkInt is
using hard-coded numeric IDs that do not match the bridge enums; update the
mapping logic to decode using the bridge constants
(CppBridgeModelRegistry.Framework.* and CppBridgeModelRegistry.ModelFormat.*)
instead of raw ints so ModelFormat and InferenceFramework are derived from the
bridge definitions; locate the mappings around the formatInt/frameworkInt
handling (the when blocks producing ModelFormat and InferenceFramework) and
replace the numeric-based branches with lookups that translate the incoming int
via the bridge enum/constants (e.g., compare to or convert from
CppBridgeModelRegistry.Framework/ModelFormat values) so the returned ModelFormat
and InferenceFramework align with the bridge.
---
Nitpick comments:
In `@examples/android/RunAnywhereAI/app/src/main/AndroidManifest.xml`:
- Line 30: Remove the redundant android:extractNativeLibs="true" attribute from
the AndroidManifest.xml entry because the project already sets
useLegacyPackaging = true in app/build.gradle.kts (which handles native lib
extraction and is the preferred, authoritative setting); delete the
android:extractNativeLibs attribute so only useLegacyPackaging controls native
library extraction and avoid conflicting/duplicative configuration.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 68ebd083-4269-43bf-a3aa-6d3bd6a563a9
📒 Files selected for processing (28)
examples/android/RunAnywhereAI/app/build.gradle.ktsexamples/android/RunAnywhereAI/app/src/main/AndroidManifest.xmlexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/data/ModelList.ktexamples/android/RunAnywhereAI/app/src/main/java/com/runanywhere/runanywhereai/presentation/chat/ChatViewModel.ktexamples/android/RunAnywhereAI/settings.gradle.ktssdk/runanywhere-commons/include/rac/infrastructure/model_management/rac_model_types.hsdk/runanywhere-commons/src/features/diffusion/diffusion_json.cppsdk/runanywhere-commons/src/features/llm/rac_llm_service.cppsdk/runanywhere-commons/src/infrastructure/model_management/model_assignment.cppsdk/runanywhere-commons/src/infrastructure/model_management/model_paths.cppsdk/runanywhere-commons/src/infrastructure/model_management/model_registry.cppsdk/runanywhere-commons/src/infrastructure/model_management/model_types.cppsdk/runanywhere-commons/src/infrastructure/registry/service_registry.cppsdk/runanywhere-commons/src/infrastructure/telemetry/telemetry_manager.cppsdk/runanywhere-flutter/packages/runanywhere/android/src/main/jniLibs/.gitkeepsdk/runanywhere-flutter/packages/runanywhere/ios/Frameworks/.gitkeepsdk/runanywhere-flutter/packages/runanywhere_llamacpp/android/src/main/jniLibs/.gitkeepsdk/runanywhere-flutter/packages/runanywhere_llamacpp/ios/Frameworks/.gitkeepsdk/runanywhere-flutter/packages/runanywhere_onnx/android/src/main/jniLibs/.gitkeepsdk/runanywhere-flutter/packages/runanywhere_onnx/ios/Frameworks/.gitkeepsdk/runanywhere-kotlin/settings.gradle.ktssdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/core/types/ComponentTypes.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/foundation/SDKLogger.ktsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/public/extensions/Models/ModelTypes.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeEvents.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+Storage.jvmAndroid.kt
Comment on lines
+190
to
192
| RAC_FRAMEWORK_WHISPERKIT_COREML = 9, /**< WhisperKit CoreML (Apple Neural Engine STT) */ | ||
| RAC_FRAMEWORK_GENIE = 10, /**< Qualcomm Genie (Hexagon NPU LLM) */ | ||
| RAC_FRAMEWORK_UNKNOWN = 99 /**< Unknown framework */ |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify Flutter SDK bridge files exist and check for missing enum cases
echo "=== Checking Flutter bridge for framework/format mappings ==="
# Find the dart bridge file
fd "dart_bridge_model_registry.dart" --type f
# Check current framework handling
echo -e "\n=== Current _frameworkToFfi implementation ==="
rg -n "_frameworkToFfi" -A 25 --type dart
# Check current format handling
echo -e "\n=== Current _formatToFfi implementation ==="
rg -n "_formatToFfi" -A 20 --type dart
# Search for any existing GENIE references in Flutter code
echo -e "\n=== Existing GENIE references in Flutter ==="
rg -i "genie" --type dart || echo "No GENIE references found in Flutter"Repository: RunanywhereAI/runanywhere-sdks
Length of output: 15943
🏁 Script executed:
# Find the public types file to check if new enum values exist
fd "public_types.dart" --type f
# Get the complete _formatToFfi and _frameworkToFfi implementations
rg -n "_formatToFfi|_frameworkToFfi" -A 30 --type dart sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_model_registry.dart
# Check for reverse conversion functions (_fromFfi)
rg -n "_frameworkFromFfi|_formatFromFfi" -A 30 --type dart sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_model_registry.dart
# Check the ModelFormat and InferenceFramework enum definitions
rg -n "enum ModelFormat|enum InferenceFramework" -A 15 --type dartRepository: RunanywhereAI/runanywhere-sdks
Length of output: 12564
🏁 Script executed:
# Check the C++ header file to confirm the new enum values mentioned
cat -n sdk/runanywhere-commons/include/rac/infrastructure/model_management/rac_model_types.h | grep -A 5 -B 5 "RAC_FRAMEWORK_GENIE\|RAC_FRAMEWORK_WHISPERKIT_COREML\|RAC_MODEL_FORMAT_QNN_CONTEXT"
# Also search for the full ModelFormat and InferenceFramework enum definitions in C++
rg "typedef enum" -A 20 sdk/runanywhere-commons/include/rac/infrastructure/model_management/rac_model_types.h | grep -A 20 "RAC_MODEL_FORMAT\|RAC_FRAMEWORK"Repository: RunanywhereAI/runanywhere-sdks
Length of output: 3994
🏁 Script executed:
# Verify that model_types.dart is the source of truth for Flutter public enums
head -50 sdk/runanywhere-flutter/packages/runanywhere/lib/core/types/model_types.dart
# Check if there are any other files where these enums might be defined
fd "model_types\|inference_framework\|model_format" --type f sdk/runanywhere-flutter/Repository: RunanywhereAI/runanywhere-sdks
Length of output: 1229
🏁 Script executed:
# Final verification - check if the review is about the bridge needing to handle public enum values
# or if it's saying public enums need to be extended first
rg "RAC_MODEL_FORMAT_QNN_CONTEXT|RAC_FRAMEWORK_GENIE|RAC_FRAMEWORK_WHISPERKIT_COREML" -A 2 -B 2 sdk/runanywhere-flutter/ || echo "No references to new C++ values in Flutter SDK"
# Check the full InferenceFramework enum definition to ensure we have all values
cat -n sdk/runanywhere-flutter/packages/runanywhere/lib/core/types/model_types.dart | grep -A 20 "enum InferenceFramework"Repository: RunanywhereAI/runanywhere-sdks
Length of output: 975
Flutter SDK public types must be updated to match new C++ framework and format enums.
The C++ header rac_model_types.h adds new enum values (RAC_FRAMEWORK_GENIE, RAC_FRAMEWORK_WHISPERKIT_COREML, RAC_FRAMEWORK_MLX, RAC_FRAMEWORK_COREML, and RAC_MODEL_FORMAT_QNN_CONTEXT), but the Flutter public API in sdk/runanywhere-flutter/packages/runanywhere/lib/core/types/model_types.dart does not include corresponding enum cases.
The Flutter bridge code (dart_bridge_model_registry.dart) is currently correct for the existing enum values, but once the public type enums are extended, the bridge mapping functions _frameworkToFfi(), _formatToFfi(), and their reverse counterparts must be updated to handle the new cases.
Update priority: (1) Add missing enum values to ModelFormat and InferenceFramework in model_types.dart, (2) Then update the bridge switch statements to map the new enum cases to their C++ integer values.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@sdk/runanywhere-commons/include/rac/infrastructure/model_management/rac_model_types.h`
around lines 190 - 192, Add the new enum variants from the C++ header into the
Dart public types by updating ModelFormat and InferenceFramework in
model_types.dart (add the missing cases that correspond to RAC_FRAMEWORK_GENIE,
RAC_FRAMEWORK_WHISPERKIT_COREML, RAC_FRAMEWORK_MLX, RAC_FRAMEWORK_COREML and the
new QNN format), then update the bridge mapping functions in
dart_bridge_model_registry.dart—specifically _frameworkToFfi(), _formatToFfi()
and their reverse mapping functions—to include switch branches that map each new
Dart enum case to the correct C++ integer value and vice‑versa; ensure all new
cases are covered and add a default/fallback branch for unknown values.
Comment on lines
+22
to
+27
| #ifdef __ANDROID__ | ||
| #include <android/log.h> | ||
| #define ALOGD(...) __android_log_print(ANDROID_LOG_DEBUG, "RAC_SVC_REG", __VA_ARGS__) | ||
| #else | ||
| #define ALOGD(...) do { fprintf(stderr, __VA_ARGS__); fprintf(stderr, "\n"); } while(0) | ||
| #endif |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check the includes and ALOGD macro in service_registry.cpp
echo "=== Lines 15-30 of service_registry.cpp ==="
sed -n '15,30p' sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp
echo -e "\n=== Searching for stdio-related includes and fprintf/stderr usage ==="
rg -n '#include <cstdio>|#include <stdio.h>|fprintf|stderr' \
sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cppRepository: RunanywhereAI/runanywhere-sdks
Length of output: 693
Add <cstdio> for the non-Android ALOGD fallback.
Line 26 expands the non-Android ALOGD macro to use fprintf(stderr, ...), but this translation unit does not include <cstdio>. This breaks non-Android builds due to missing declarations for fprintf and stderr.
Suggested fix
`#include` <algorithm>
+#include <cstdio>
`#include` <cstring>
`#include` <mutex>
`#include` <string>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| #ifdef __ANDROID__ | |
| #include <android/log.h> | |
| #define ALOGD(...) __android_log_print(ANDROID_LOG_DEBUG, "RAC_SVC_REG", __VA_ARGS__) | |
| #else | |
| #define ALOGD(...) do { fprintf(stderr, __VA_ARGS__); fprintf(stderr, "\n"); } while(0) | |
| #endif | |
| `#include` <algorithm> | |
| `#include` <cstdio> | |
| `#include` <cstring> | |
| `#include` <mutex> | |
| `#include` <string> |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp`
around lines 22 - 27, The non-Android ALOGD macro uses fprintf and stderr but
the translation unit doesn't include <cstdio>, causing build failures; add an
`#include` <cstdio> at the top of service_registry.cpp (before the ALOGD macro
definition) so the symbols fprintf and stderr are declared; ensure the ALOGD
macro block (the else branch defining ALOGD) remains unchanged and references
fprintf and stderr as before.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+81
to
+82
| const val COREML = 5 // RAC_MODEL_FORMAT_COREML | ||
| const val QNN_CONTEXT = 5 // RAC_MODEL_FORMAT_QNN_CONTEXT (same C++ value; COREML is 4 in C++) |
Contributor
There was a problem hiding this comment.
Constant value mismatch causes silent CoreML model misrouting
Both COREML and QNN_CONTEXT are assigned the value 5, but the C++ header defines RAC_MODEL_FORMAT_COREML = 4 and RAC_MODEL_FORMAT_QNN_CONTEXT = 5.
This causes CoreML models with format 4 from C++ to fall through when expressions in the Kotlin bridge to UNKNOWN, silently misidentifying the format. The comment on line 82 even acknowledges this mismatch ("COREML is 4 in C++"), but the fix was not applied.
Correct the constant:
const val BIN = 3 // RAC_MODEL_FORMAT_BIN
const val COREML = 4 // RAC_MODEL_FORMAT_COREML
const val QNN_CONTEXT = 5 // RAC_MODEL_FORMAT_QNN_CONTEXT
This requires updating when expressions and format mapping throughout the bridge.
Prompt To Fix With AI
This is a comment left during a code review.
Path: sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt
Line: 81-82
Comment:
**Constant value mismatch causes silent CoreML model misrouting**
Both `COREML` and `QNN_CONTEXT` are assigned the value `5`, but the C++ header defines `RAC_MODEL_FORMAT_COREML = 4` and `RAC_MODEL_FORMAT_QNN_CONTEXT = 5`.
This causes CoreML models with format `4` from C++ to fall through `when` expressions in the Kotlin bridge to `UNKNOWN`, silently misidentifying the format. The comment on line 82 even acknowledges this mismatch ("COREML is 4 in C++"), but the fix was not applied.
Correct the constant:
```
const val BIN = 3 // RAC_MODEL_FORMAT_BIN
const val COREML = 4 // RAC_MODEL_FORMAT_COREML
const val QNN_CONTEXT = 5 // RAC_MODEL_FORMAT_QNN_CONTEXT
```
This requires updating `when` expressions and format mapping throughout the bridge.
How can I resolve this? If you propose a fix, please make it concise.
Comment on lines
42
to
+113
| @@ -57,10 +68,15 @@ rac_result_t rac_llm_create(const char* model_id, rac_handle_t* out_handle) { | |||
| } else { | |||
| model_path = reg_path; | |||
| } | |||
| ALOGD("Found in registry: id=%s, framework=%d, local_path=%s", | |||
| model_info->id ? model_info->id : "NULL", | |||
| static_cast<int>(framework), model_path ? model_path : "NULL"); | |||
| RAC_LOG_INFO(LOG_CAT, "Found model in registry: id=%s, framework=%d, local_path=%s", | |||
| model_info->id ? model_info->id : "NULL", | |||
| static_cast<int>(framework), model_path ? model_path : "NULL"); | |||
| } else { | |||
| ALOGD("NOT found in registry (result=%d), default framework=%d", | |||
| result, static_cast<int>(framework)); | |||
| RAC_LOG_WARNING(LOG_CAT, | |||
| "Model NOT found in registry (result=%d), using default framework=%d", | |||
| result, static_cast<int>(framework)); | |||
| @@ -73,22 +89,28 @@ rac_result_t rac_llm_create(const char* model_id, rac_handle_t* out_handle) { | |||
| request.framework = framework; | |||
| request.model_path = model_path; | |||
|
|
|||
| ALOGD("Service request: framework=%d, model_path=%s", | |||
| static_cast<int>(request.framework), | |||
| request.model_path ? request.model_path : "NULL"); | |||
| RAC_LOG_INFO(LOG_CAT, "Service request: framework=%d, model_path=%s", | |||
| static_cast<int>(request.framework), | |||
| request.model_path ? request.model_path : "NULL"); | |||
|
|
|||
| // Service registry returns an rac_llm_service_t* with vtable already set | |||
| result = rac_service_create(RAC_CAPABILITY_TEXT_GENERATION, &request, out_handle); | |||
| ALOGD("rac_service_create result=%d", result); | |||
|
|
|||
| if (model_info) { | |||
| rac_model_info_free(model_info); | |||
| } | |||
|
|
|||
| if (result != RAC_SUCCESS) { | |||
| ALOGD("Failed to create service: %d", result); | |||
| RAC_LOG_ERROR(LOG_CAT, "Failed to create service via registry: %d", result); | |||
| return result; | |||
| } | |||
|
|
|||
| ALOGD("LLM service created successfully"); | |||
Contributor
There was a problem hiding this comment.
Unconditional debug ALOGD calls left in production code
Multiple ALOGD() macros (lines 42, 48, 52, 55, 71–73, 78–79, 92–94, 101, 108, 113) fire unconditionally on every LLM creation, bypassing the existing RAC_LOG_* infrastructure.
Each ALOGD call is immediately followed by an equivalent RAC_LOG_INFO/DEBUG/ERROR call, making the ALOGD calls redundant. For example:
ALOGD("rac_llm_create: model_id=%s", model_id);
RAC_LOG_INFO(LOG_CAT, "Creating LLM service for: %s", model_id);These are development artifacts and should be removed before merging. The same pattern appears in sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp (lines 187, 192, 197, 201, 204, 210, 217).
Prompt To Fix With AI
This is a comment left during a code review.
Path: sdk/runanywhere-commons/src/features/llm/rac_llm_service.cpp
Line: 42-113
Comment:
**Unconditional debug ALOGD calls left in production code**
Multiple `ALOGD()` macros (lines 42, 48, 52, 55, 71–73, 78–79, 92–94, 101, 108, 113) fire unconditionally on every LLM creation, bypassing the existing `RAC_LOG_*` infrastructure.
Each `ALOGD` call is immediately followed by an equivalent `RAC_LOG_INFO/DEBUG/ERROR` call, making the `ALOGD` calls redundant. For example:
```cpp
ALOGD("rac_llm_create: model_id=%s", model_id);
RAC_LOG_INFO(LOG_CAT, "Creating LLM service for: %s", model_id);
```
These are development artifacts and should be removed before merging. The same pattern appears in `sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp` (lines 187, 192, 197, 201, 204, 210, 217).
How can I resolve this? If you propose a fix, please make it concise.…integration" This reverts commit c30b3cd.
Bumps runanywhere-genie-android from 0.1.5-SNAPSHOT to 0.2.0-SNAPSHOT which includes the fixed libQnnHtpV79Stub.so (v2.42.0) for Snapdragon 8 Elite (Galaxy S25) NPU inference via Qualcomm Genie. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
sdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/public/extensions/Models/ModelTypes.kt (1)
114-119:⚠️ Potential issue | 🔴 CriticalAdd
QNN_CONTEXTto the public model types before surfacingGENIEhere.This makes Genie models eligible in
LLM, butModelFormatin this file still has noQNN_CONTEXT, and the provided bridge code insdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt:83-89currently degrades unknown formats toUNKNOWN. As a result, Genie models can become selectable here while still losing their real format before registration.🔧 Minimal follow-up
enum class ModelFormat( val value: String, ) { ONNX("onnx"), ORT("ort"), GGUF("gguf"), + QNN_CONTEXT("qnn_context"), BIN("bin"), UNKNOWN("unknown"), }// sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt format = when (modelInfo.format) { ModelFormat.GGUF -> CppBridgeModelRegistry.ModelFormat.GGUF ModelFormat.ONNX -> CppBridgeModelRegistry.ModelFormat.ONNX ModelFormat.ORT -> CppBridgeModelRegistry.ModelFormat.ORT ModelFormat.QNN_CONTEXT -> CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT else -> CppBridgeModelRegistry.ModelFormat.UNKNOWN }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/public/extensions/Models/ModelTypes.kt` around lines 114 - 119, The public model types are missing QNN_CONTEXT, so GENIE becomes selectable in isFrameworkRelevant while the JVM/Android bridge still maps unknown formats to UNKNOWN; add QNN_CONTEXT to the public ModelFormat enum (the symbol ModelFormat) used by isFrameworkRelevant and ensure the JVM/Android bridge mapping in RunAnywhere+ModelManagement.jvmAndroid.kt maps ModelFormat.QNN_CONTEXT to CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT instead of falling through to UNKNOWN, and verify isFrameworkRelevant (the function isFrameworkRelevant) continues to include GENIE for LLM.
♻️ Duplicate comments (1)
sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp (1)
22-27:⚠️ Potential issue | 🔴 CriticalNon-Android
ALOGDstill needs<cstdio>and newline termination.Line 26 uses
fprintf(stderr, ...), but this TU still does not include<cstdio>, so non-Androidg++builds are relying on undeclared stdio symbols. The fallback also no longer appends a newline, so consecutive debug records get merged onstderr.Proposed fix
`#include` <algorithm> +#include <cstdio> `#include` <cstring> `#include` <mutex> `#include` <string> `#include` <unordered_map> `#include` <vector> @@ `#ifdef` __ANDROID__ `#include` <android/log.h> `#define` ALOGD(...) __android_log_print(ANDROID_LOG_DEBUG, "RAC_SVC_REG", __VA_ARGS__) `#else` -#define ALOGD(...) fprintf(stderr, __VA_ARGS__) +#define ALOGD(...) \ + do { \ + std::fprintf(stderr, __VA_ARGS__); \ + std::fputc('\n', stderr); \ + } while (0) `#endif`🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp` around lines 22 - 27, The non-Android debug macro ALOGD uses fprintf but the TU lacks <cstdio> and the fallback output omits a newline; fix by adding `#include` <cstdio> to the top of service_registry.cpp and change the non-Android macro definition (ALOGD) to append a terminating "\n" to its fprintf call so messages don't run together (refer to the ALOGD macro and the existing `#ifdef` __ANDROID__ block).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp`:
- Around line 187-217: The ALOGD calls in rac_service_create are always-on and
duplicate RAC_LOG_* messages, bypassing logger-level control; remove or gate all
direct ALOGD probes around provider checking and creation (the ALOGD lines that
log provider.name, can_handle result, "Calling create", create return value,
nullptr case, and final "No provider could handle the request") and rely on the
existing RAC_LOG_INFO/ERROR calls instead, or wrap those ALOGD calls behind the
same log-level guard used by RAC_LOG_* so provider.can_handle, provider.create,
out_handle assignment and RAC_SUCCESS return behavior remain unchanged while
avoiding unconditional Logcat/stderr output.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt:
- Line 107: The mappings for GENIE were added on the framework side but the
Kotlin conversion paths still collapse QNN_CONTEXT to UNKNOWN; update the Kotlin
mapping code in registerModelInternal, bridgeModelToPublic, and
parseModelAssignmentsJson to preserve QNN_CONTEXT by mapping it to
InferenceFramework.GENIE / CppBridgeModelRegistry.Framework.GENIE (and
vice‑versa) instead of UNKNOWN so Genie models round‑trip correctly through
Kotlin; locate the conversion switch/case or when branches that reference
QNN_CONTEXT/UNKNOWN and add the symmetric mapping entries for GENIE.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+Storage.jvmAndroid.kt:
- Line 209: convertToModelStorageMetrics() currently maps
CppBridgeModelRegistry.Framework.GENIE to InferenceFramework.GENIE but the
adjacent format mapping still converts
CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT to ModelFormat.UNKNOWN, causing
storageInfo() to report inconsistent metadata; update the format mapping in the
same conversion function to map CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT
to the appropriate ModelFormat (QNN_CONTEXT) so downloaded Genie models keep
consistent format and framework metadata.
---
Outside diff comments:
In
`@sdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/public/extensions/Models/ModelTypes.kt`:
- Around line 114-119: The public model types are missing QNN_CONTEXT, so GENIE
becomes selectable in isFrameworkRelevant while the JVM/Android bridge still
maps unknown formats to UNKNOWN; add QNN_CONTEXT to the public ModelFormat enum
(the symbol ModelFormat) used by isFrameworkRelevant and ensure the JVM/Android
bridge mapping in RunAnywhere+ModelManagement.jvmAndroid.kt maps
ModelFormat.QNN_CONTEXT to CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT
instead of falling through to UNKNOWN, and verify isFrameworkRelevant (the
function isFrameworkRelevant) continues to include GENIE for LLM.
---
Duplicate comments:
In `@sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp`:
- Around line 22-27: The non-Android debug macro ALOGD uses fprintf but the TU
lacks <cstdio> and the fallback output omits a newline; fix by adding `#include`
<cstdio> to the top of service_registry.cpp and change the non-Android macro
definition (ALOGD) to append a terminating "\n" to its fprintf call so messages
don't run together (refer to the ALOGD macro and the existing `#ifdef` __ANDROID__
block).
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9924b320-b4d5-4e49-afd5-fdc4879435bb
📒 Files selected for processing (8)
sdk/runanywhere-commons/src/features/llm/rac_llm_service.cppsdk/runanywhere-commons/src/infrastructure/model_management/model_assignment.cppsdk/runanywhere-commons/src/infrastructure/model_management/model_types.cppsdk/runanywhere-commons/src/infrastructure/registry/service_registry.cppsdk/runanywhere-kotlin/src/commonMain/kotlin/com/runanywhere/sdk/public/extensions/Models/ModelTypes.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+Storage.jvmAndroid.kt
🚧 Files skipped from review as they are similar to previous changes (4)
- sdk/runanywhere-commons/src/infrastructure/model_management/model_assignment.cpp
- sdk/runanywhere-commons/src/infrastructure/model_management/model_types.cpp
- sdk/runanywhere-commons/src/features/llm/rac_llm_service.cpp
- sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt
Comment on lines
+187
to
+217
| ALOGD("Checking provider '%s' (priority=%d)", provider.name.c_str(), provider.priority); | ||
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Checking provider '%s' (priority=%d)", | ||
| provider.name.c_str(), provider.priority); | ||
|
|
||
| bool can_handle = provider.can_handle(request, provider.user_data); | ||
| ALOGD("Provider '%s' can_handle=%s", provider.name.c_str(), can_handle ? "TRUE" : "FALSE"); | ||
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Provider '%s' can_handle=%s", | ||
| provider.name.c_str(), can_handle ? "TRUE" : "FALSE"); | ||
|
|
||
| if (can_handle) { | ||
| ALOGD("Calling create for provider '%s'", provider.name.c_str()); | ||
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Calling create for provider '%s'", | ||
| provider.name.c_str()); | ||
| rac_handle_t handle = provider.create(request, provider.user_data); | ||
| ALOGD("Provider '%s' create returned handle=%p", provider.name.c_str(), handle); | ||
| if (handle != nullptr) { | ||
| *out_handle = handle; | ||
| ALOGD("Service created by provider '%s'", provider.name.c_str()); | ||
| RAC_LOG_INFO(LOG_CAT, | ||
| "rac_service_create: Service created by provider '%s', handle=%p", | ||
| provider.name.c_str(), handle); | ||
| return RAC_SUCCESS; | ||
| } else { | ||
| ALOGD("Provider '%s' create returned nullptr!", provider.name.c_str()); | ||
| RAC_LOG_ERROR(LOG_CAT, "rac_service_create: Provider '%s' create returned nullptr", | ||
| provider.name.c_str()); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| ALOGD("No provider could handle the request"); |
There was a problem hiding this comment.
Please gate or remove the always-on ALOGD probes.
These calls duplicate the existing RAC_LOG_* messages and bypass logger-level control by writing straight to Logcat/stderr on every provider probe. Since rac_service_create is on the backend-selection path, this turns routine capability checks into noisy production logging.
Proposed fix
- ALOGD("Checking provider '%s' (priority=%d)", provider.name.c_str(), provider.priority);
RAC_LOG_INFO(LOG_CAT, "rac_service_create: Checking provider '%s' (priority=%d)",
provider.name.c_str(), provider.priority);
@@
- ALOGD("Provider '%s' can_handle=%s", provider.name.c_str(), can_handle ? "TRUE" : "FALSE");
RAC_LOG_INFO(LOG_CAT, "rac_service_create: Provider '%s' can_handle=%s",
provider.name.c_str(), can_handle ? "TRUE" : "FALSE");
@@
- ALOGD("Calling create for provider '%s'", provider.name.c_str());
RAC_LOG_INFO(LOG_CAT, "rac_service_create: Calling create for provider '%s'",
provider.name.c_str());
rac_handle_t handle = provider.create(request, provider.user_data);
- ALOGD("Provider '%s' create returned handle=%p", provider.name.c_str(), handle);
if (handle != nullptr) {
*out_handle = handle;
- ALOGD("Service created by provider '%s'", provider.name.c_str());
RAC_LOG_INFO(LOG_CAT,
"rac_service_create: Service created by provider '%s', handle=%p",
provider.name.c_str(), handle);
return RAC_SUCCESS;
} else {
- ALOGD("Provider '%s' create returned nullptr!", provider.name.c_str());
RAC_LOG_ERROR(LOG_CAT, "rac_service_create: Provider '%s' create returned nullptr",
provider.name.c_str());
}
@@
- ALOGD("No provider could handle the request");
RAC_LOG_ERROR(LOG_CAT, "rac_service_create: No provider could handle the request");Based on learnings Applies to sdk/runanywhere-commons/src/infrastructure/registry/*.{cpp,h} : Service registry must support priority-based provider selection with canHandle pattern for capability matching.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ALOGD("Checking provider '%s' (priority=%d)", provider.name.c_str(), provider.priority); | |
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Checking provider '%s' (priority=%d)", | |
| provider.name.c_str(), provider.priority); | |
| bool can_handle = provider.can_handle(request, provider.user_data); | |
| ALOGD("Provider '%s' can_handle=%s", provider.name.c_str(), can_handle ? "TRUE" : "FALSE"); | |
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Provider '%s' can_handle=%s", | |
| provider.name.c_str(), can_handle ? "TRUE" : "FALSE"); | |
| if (can_handle) { | |
| ALOGD("Calling create for provider '%s'", provider.name.c_str()); | |
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Calling create for provider '%s'", | |
| provider.name.c_str()); | |
| rac_handle_t handle = provider.create(request, provider.user_data); | |
| ALOGD("Provider '%s' create returned handle=%p", provider.name.c_str(), handle); | |
| if (handle != nullptr) { | |
| *out_handle = handle; | |
| ALOGD("Service created by provider '%s'", provider.name.c_str()); | |
| RAC_LOG_INFO(LOG_CAT, | |
| "rac_service_create: Service created by provider '%s', handle=%p", | |
| provider.name.c_str(), handle); | |
| return RAC_SUCCESS; | |
| } else { | |
| ALOGD("Provider '%s' create returned nullptr!", provider.name.c_str()); | |
| RAC_LOG_ERROR(LOG_CAT, "rac_service_create: Provider '%s' create returned nullptr", | |
| provider.name.c_str()); | |
| } | |
| } | |
| } | |
| ALOGD("No provider could handle the request"); | |
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Checking provider '%s' (priority=%d)", | |
| provider.name.c_str(), provider.priority); | |
| bool can_handle = provider.can_handle(request, provider.user_data); | |
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Provider '%s' can_handle=%s", | |
| provider.name.c_str(), can_handle ? "TRUE" : "FALSE"); | |
| if (can_handle) { | |
| RAC_LOG_INFO(LOG_CAT, "rac_service_create: Calling create for provider '%s'", | |
| provider.name.c_str()); | |
| rac_handle_t handle = provider.create(request, provider.user_data); | |
| if (handle != nullptr) { | |
| *out_handle = handle; | |
| RAC_LOG_INFO(LOG_CAT, | |
| "rac_service_create: Service created by provider '%s', handle=%p", | |
| provider.name.c_str(), handle); | |
| return RAC_SUCCESS; | |
| } else { | |
| RAC_LOG_ERROR(LOG_CAT, "rac_service_create: Provider '%s' create returned nullptr", | |
| provider.name.c_str()); | |
| } | |
| } | |
| } | |
| RAC_LOG_ERROR(LOG_CAT, "rac_service_create: No provider could handle the request"); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@sdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp`
around lines 187 - 217, The ALOGD calls in rac_service_create are always-on and
duplicate RAC_LOG_* messages, bypassing logger-level control; remove or gate all
direct ALOGD probes around provider checking and creation (the ALOGD lines that
log provider.name, can_handle result, "Calling create", create return value,
nullptr case, and final "No provider could handle the request") and rely on the
existing RAC_LOG_INFO/ERROR calls instead, or wrap those ALOGD calls behind the
same log-level guard used by RAC_LOG_* so provider.can_handle, provider.create,
out_handle assignment and RAC_SUCCESS return behavior remain unchanged while
avoiding unconditional Logcat/stderr output.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt (3)
294-300:⚠️ Potential issue | 🟠 MajorQNN_CONTEXT format reverse mapping missing.
Similar to the forward mapping, the reverse conversion from bridge to public
ModelFormatdoesn't handleQNN_CONTEXT.Proposed fix
format = when (bridge.format) { CppBridgeModelRegistry.ModelFormat.GGUF -> ModelFormat.GGUF CppBridgeModelRegistry.ModelFormat.ONNX -> ModelFormat.ONNX CppBridgeModelRegistry.ModelFormat.ORT -> ModelFormat.ORT + CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT -> ModelFormat.QNN_CONTEXT else -> ModelFormat.UNKNOWN },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt around lines 294 - 300, The reverse mapping of bridge.format to the public ModelFormat misses QNN_CONTEXT; update the when expression that currently maps CppBridgeModelRegistry.ModelFormat.GGUF/ONNX/ORT to include a case for CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT -> ModelFormat.QNN_CONTEXT so the bridge-to-public conversion (the when on bridge.format) correctly returns ModelFormat.QNN_CONTEXT instead of falling through to ModelFormat.UNKNOWN.
91-97:⚠️ Potential issue | 🟠 MajorQNN_CONTEXT format still maps to UNKNOWN.
The format conversion doesn't include
ModelFormat.QNN_CONTEXT -> CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT, so Genie models registered with QNN_CONTEXT format will lose their format information.Proposed fix
format = when (modelInfo.format) { ModelFormat.GGUF -> CppBridgeModelRegistry.ModelFormat.GGUF ModelFormat.ONNX -> CppBridgeModelRegistry.ModelFormat.ONNX ModelFormat.ORT -> CppBridgeModelRegistry.ModelFormat.ORT + ModelFormat.QNN_CONTEXT -> CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT else -> CppBridgeModelRegistry.ModelFormat.UNKNOWN },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt around lines 91 - 97, The format mapping in the when expression for modelInfo.format currently falls through QNN_CONTEXT to UNKNOWN; update the conversion in RunAnywhere+ModelManagement (the when block that maps ModelFormat -> CppBridgeModelRegistry.ModelFormat) to add a branch mapping ModelFormat.QNN_CONTEXT to CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT so QNN_CONTEXT formats are preserved when registering models (keep the existing GGUF, ONNX, ORT branches and the final UNKNOWN fallback).
1366-1371:⚠️ Potential issue | 🟠 MajorQNN_CONTEXT format parsing missing.
The format integer-to-enum mapping doesn't handle
5 -> ModelFormat.QNN_CONTEXT, so Genie models from assignments will haveUNKNOWNformat.Proposed fix
format = when (formatInt) { 1 -> ModelFormat.GGUF 2 -> ModelFormat.ONNX 3 -> ModelFormat.ORT + 5 -> ModelFormat.QNN_CONTEXT else -> ModelFormat.UNKNOWN },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt around lines 1366 - 1371, The when mapping for formatInt in the Model parsing code (in RunAnywhere+ModelManagement.jvmAndroid.kt) is missing the QNN_CONTEXT case; update the when expression that maps formatInt to ModelFormat (the block that currently returns ModelFormat.GGUF for 1, ONNX for 2, ORT for 3, else UNKNOWN) to include the case `5 -> ModelFormat.QNN_CONTEXT`, keeping the else branch as ModelFormat.UNKNOWN.sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt (1)
76-84:⚠️ Potential issue | 🟠 MajorQNN_CONTEXT format constant is still missing.
The
ModelFormatobject lacksQNN_CONTEXT, which is required for Genie NPU models to be properly identified. Per the PR objectives,RAC_MODEL_FORMAT_QNN_CONTEXT = 5was added in C++.Since COREML = 5 already exists here, you need to verify the actual C++ enum values and ensure this table matches 1:1. If COREML is being reassigned in C++ to make room for QNN_CONTEXT, update accordingly.
Proposed fix (assuming C++ assigns QNN_CONTEXT = 5, COREML = 6, TFLITE = 7)
object ModelFormat { const val UNKNOWN = 0 // RAC_MODEL_FORMAT_UNKNOWN const val GGUF = 1 // RAC_MODEL_FORMAT_GGUF const val ONNX = 2 // RAC_MODEL_FORMAT_ONNX const val ORT = 3 // RAC_MODEL_FORMAT_ORT const val BIN = 4 // RAC_MODEL_FORMAT_BIN - const val COREML = 5 // RAC_MODEL_FORMAT_COREML - const val TFLITE = 6 // RAC_MODEL_FORMAT_TFLITE + const val QNN_CONTEXT = 5 // RAC_MODEL_FORMAT_QNN_CONTEXT + const val COREML = 6 // RAC_MODEL_FORMAT_COREML + const val TFLITE = 7 // RAC_MODEL_FORMAT_TFLITE }#!/bin/bash # Verify the actual C++ enum values for model formats ast-grep --pattern 'RAC_MODEL_FORMAT_$_' # Also check the header file directly fd "rac_model_types.h" --exec cat {}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt` around lines 76 - 84, ModelFormat is missing the QNN_CONTEXT constant and the numeric values must match the C++ enum 1:1; inspect the C++ enum (RAC_MODEL_FORMAT_*) to confirm each value and then add/update entries in the ModelFormat object (e.g., QNN_CONTEXT and the shifted COREML/TFLITE values if C++ moved them) so the Kotlin constants exactly mirror the C++ enum names and integer values referenced by the native bridge.
🧹 Nitpick comments (1)
sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt (1)
879-884: Fallback directory matching is loose and may match unrelated models.The logic
it.name.contains(modelId.substringBefore("-"))could match directories from other models sharing the same prefix. For example,"llama-3.2-1b"would match any directory containing"llama".Consider using a more specific match, such as checking if the directory name starts with the modelId or matches more characters.
More specific matching suggestion
- val matchingDir = parentDir.listFiles() - ?.firstOrNull { it.isDirectory && it.name.contains(modelId.substringBefore("-")) } + // Prefer exact match first, then startsWith, then contains as last resort + val matchingDir = parentDir.listFiles() + ?.filter { it.isDirectory && it !in itemsBeforeExtraction.mapNotNull { name -> File(parentDir, name).takeIf { f -> f.isDirectory } } } + ?.firstOrNull { it.name == modelId || it.name.startsWith(modelId) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt around lines 879 - 884, The fallback directory match in RunAnywhere+ModelManagement.jvmAndroid.kt is too loose: replace the contains-based check on matchingDir (it.name.contains(modelId.substringBefore("-"))) with a stricter comparison against modelId (or a well-defined prefix plus separator) so unrelated directories aren't matched; for example, compute modelPrefix = modelId.substringBefore("-") and require either it.name.startsWith(modelId) or it.name.startsWith(modelPrefix + "-") or use a regex like ^modelPrefix(-|\\d|$) to ensure the directory truly corresponds to the same model when resolving matchingDir.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt:
- Around line 886-896: The code currently ignores failures from File.renameTo
when moving flat-extracted files into expectedModelDir (see expectedModelDir,
parentDir.listFiles(), and the forEach { file -> file.renameTo(...) } block);
update this to detect and handle failures by either using
java.nio.file.Files.move (which throws on error) or by checking renameTo's
boolean result and logging and throwing an IOException if any move fails,
ensuring the operation does not return expectedModelDir.absolutePath when the
directory is incomplete.
---
Duplicate comments:
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt`:
- Around line 76-84: ModelFormat is missing the QNN_CONTEXT constant and the
numeric values must match the C++ enum 1:1; inspect the C++ enum
(RAC_MODEL_FORMAT_*) to confirm each value and then add/update entries in the
ModelFormat object (e.g., QNN_CONTEXT and the shifted COREML/TFLITE values if
C++ moved them) so the Kotlin constants exactly mirror the C++ enum names and
integer values referenced by the native bridge.
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt:
- Around line 294-300: The reverse mapping of bridge.format to the public
ModelFormat misses QNN_CONTEXT; update the when expression that currently maps
CppBridgeModelRegistry.ModelFormat.GGUF/ONNX/ORT to include a case for
CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT -> ModelFormat.QNN_CONTEXT so the
bridge-to-public conversion (the when on bridge.format) correctly returns
ModelFormat.QNN_CONTEXT instead of falling through to ModelFormat.UNKNOWN.
- Around line 91-97: The format mapping in the when expression for
modelInfo.format currently falls through QNN_CONTEXT to UNKNOWN; update the
conversion in RunAnywhere+ModelManagement (the when block that maps ModelFormat
-> CppBridgeModelRegistry.ModelFormat) to add a branch mapping
ModelFormat.QNN_CONTEXT to CppBridgeModelRegistry.ModelFormat.QNN_CONTEXT so
QNN_CONTEXT formats are preserved when registering models (keep the existing
GGUF, ONNX, ORT branches and the final UNKNOWN fallback).
- Around line 1366-1371: The when mapping for formatInt in the Model parsing
code (in RunAnywhere+ModelManagement.jvmAndroid.kt) is missing the QNN_CONTEXT
case; update the when expression that maps formatInt to ModelFormat (the block
that currently returns ModelFormat.GGUF for 1, ONNX for 2, ORT for 3, else
UNKNOWN) to include the case `5 -> ModelFormat.QNN_CONTEXT`, keeping the else
branch as ModelFormat.UNKNOWN.
---
Nitpick comments:
In
`@sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere`+ModelManagement.jvmAndroid.kt:
- Around line 879-884: The fallback directory match in
RunAnywhere+ModelManagement.jvmAndroid.kt is too loose: replace the
contains-based check on matchingDir
(it.name.contains(modelId.substringBefore("-"))) with a stricter comparison
against modelId (or a well-defined prefix plus separator) so unrelated
directories aren't matched; for example, compute modelPrefix =
modelId.substringBefore("-") and require either it.name.startsWith(modelId) or
it.name.startsWith(modelPrefix + "-") or use a regex like ^modelPrefix(-|\\d|$)
to ensure the directory truly corresponds to the same model when resolving
matchingDir.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c6def322-db9a-4f46-b0e7-d30fbac6ade2
📒 Files selected for processing (2)
sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.ktsdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/public/extensions/RunAnywhere+ModelManagement.jvmAndroid.kt
Switch from SNAPSHOT/mavenLocal to published Maven Central artifact: io.github.sanchitmonga22:runanywhere-genie-android:0.2.1 Made-with: Cursor
…alidation - Switch Android app to qwen2.5-0.5b-abliterated-lora-f16.gguf (17.6MB F16) - Add C++ validation: scale check, GGUF magic verify, tensor match via ab_map - Add pre-generation adapter state verification - Update API from deprecated llama_set_adapter_lora to llama_set_adapters_lora (b8201) - Add GGUF validation after download in Kotlin SDK - Update docs for llama.cpp b8201 API changes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Introduced the Genie NPU backend for Android/Snapdragon devices, including model registration and framework integration. - Updated pubspec.yaml to include the runanywhere_genie dependency. - Enhanced AndroidManifest.xml to declare native library requirements for Genie. - Modified GeneratedPluginRegistrant to register Genie and related plugins. - Updated model handling to support Genie in the app's logic and UI. - Improved documentation and changelog to reflect these changes. This commit enhances the SDK's capabilities by integrating support for the Genie NPU, allowing for optimized inference on compatible devices.
…m/RunanywhereAI/runanywhere-sdks into smonga/genie_support
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp (1)
294-301:⚠️ Potential issue | 🟡 MinorAdd
QNN_CONTEXTformat mapping to prevent silent failures if TypeScript passes Genie model formats.The
RAC_MODEL_FORMAT_QNN_CONTEXTenum exists in the C++ codebase for Genie NPU models, butformatFromString()lacks a handler for it. If the TypeScript layer ever passes"QNN_CONTEXT"or"qnn_context"as a format string (which currently doesn't happen, but could in future integrations), it will silently map toRAC_MODEL_FORMAT_UNKNOWN.While Genie models are presently registered through a separate SDK path, adding this mapping ensures defensive completeness:
Suggested fix
if (format == "BIN" || format == "bin") return RAC_MODEL_FORMAT_BIN; + if (format == "QNN_CONTEXT" || format == "qnn_context") return RAC_MODEL_FORMAT_QNN_CONTEXT; return RAC_MODEL_FORMAT_UNKNOWN;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp` around lines 294 - 301, The formatFromString function currently maps several string formats to rac_model_format_t but omits handling for the Genie NPU format; add checks in rac_model_format_t formatFromString(const std::string& format) to return RAC_MODEL_FORMAT_QNN_CONTEXT when format equals "QNN_CONTEXT" or "qnn_context" (similar to the other comparisons), ensuring Genie model strings don't fall through to RAC_MODEL_FORMAT_UNKNOWN.
🧹 Nitpick comments (3)
sdk/runanywhere-react-native/packages/core/cpp/bridges/RAGBridge.hpp (1)
36-39: Unusedmutex_member.With the pipeline member removed and all methods now being stateless stubs,
mutex_is no longer used. Consider removing it to avoid confusion.♻️ Optional cleanup
private: RAGBridge() = default; - std::mutex mutex_; };If you remove
mutex_, you can also drop#include <mutex>at line 15.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-react-native/packages/core/cpp/bridges/RAGBridge.hpp` around lines 36 - 39, Remove the now-unused std::mutex member from RAGBridge: delete the private field named mutex_ from the RAGBridge class declaration (constructor RAGBridge() remains), and also remove the `#include` <mutex> directive if nothing else in this header needs it; ensure no other code references RAGBridge::mutex_ before committing.sdk/runanywhere-flutter/packages/runanywhere/lib/infrastructure/download/download_service.dart (1)
334-344: Consider case-insensitive format detection and additional supported formats.The
endsWithchecks are case-sensitive, so archives with uppercase extensions (e.g.,.ZIP,.TAR.GZ) would be rejected. Additionally, thearchivepackage'sextractFileToDiskalso supports.tar.xzand.txzformats.Suggested improvement
- final supported = archivePath.endsWith('.tar.gz') || - archivePath.endsWith('.tgz') || - archivePath.endsWith('.tar.bz2') || - archivePath.endsWith('.tbz2') || - archivePath.endsWith('.zip') || - archivePath.endsWith('.tar'); + final lowerPath = archivePath.toLowerCase(); + final supported = lowerPath.endsWith('.tar.gz') || + lowerPath.endsWith('.tgz') || + lowerPath.endsWith('.tar.bz2') || + lowerPath.endsWith('.tbz2') || + lowerPath.endsWith('.tar.xz') || + lowerPath.endsWith('.txz') || + lowerPath.endsWith('.zip') || + lowerPath.endsWith('.tar');🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-flutter/packages/runanywhere/lib/infrastructure/download/download_service.dart` around lines 334 - 344, The supported-detection is case-sensitive and missing .tar.xz/.txz; convert archivePath to lowercase before testing (e.g., use final lc = archivePath.toLowerCase()) and update the supported boolean (used in the same function) to include lc.endsWith('.tar.xz') and lc.endsWith('.txz') alongside the existing checks; keep the existing _logger.warning('Unknown archive format: $archivePath') and return behavior unchanged.sdk/runanywhere-react-native/packages/core/cpp/bridges/CompatibilityBridge.hpp (1)
14-22: Fragile macro guard pattern for typedef fallback.The fallback relies on the exact include guard macro name
RAC_MODEL_REGISTRY_Hfromrac_model_registry.h. If the include guard is ever renamed or the header is refactored, this condition silently breaks and could cause ODR (One Definition Rule) violations or pointer-type mismatches between translation units.Consider a more robust approach:
♻️ Proposed fix using forward declaration
-// rac_model_registry_handle_t is defined in rac_model_registry.h. -// In a stub context where the header may not be on the search path, -// fall back to void* — the real type is struct rac_model_registry*. -// (ModelRegistryBridge.hpp includes rac_model_registry.h first in practice.) -#ifdef RAC_MODEL_REGISTRY_H -// Already included via ModelRegistryBridge.hpp — type is already defined -#else -typedef void* rac_model_registry_handle_t; -#endif +// Forward declaration to avoid header dependency. +// rac_model_registry_handle_t is defined as struct rac_model_registry* in rac_model_registry.h. +#ifndef RAC_MODEL_REGISTRY_HANDLE_DEFINED +struct rac_model_registry; +typedef struct rac_model_registry* rac_model_registry_handle_t; +#define RAC_MODEL_REGISTRY_HANDLE_DEFINED +#endifThis uses a dedicated macro for the typedef itself and forward-declares the struct, preserving type safety while avoiding header coupling.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sdk/runanywhere-react-native/packages/core/cpp/bridges/CompatibilityBridge.hpp` around lines 14 - 22, The current fallback typedef for rac_model_registry_handle_t depends on the external include guard macro RAC_MODEL_REGISTRY_H which is fragile; instead forward-declare struct rac_model_registry and introduce a dedicated guard macro (e.g. RAC_MODEL_REGISTRY_HANDLE_T_DEFINED) to protect the typedef so all translation units agree on the pointer type; update CompatibilityBridge.hpp to remove the RAC_MODEL_REGISTRY_H conditional, add a forward declaration of struct rac_model_registry, and typedef rac_model_registry_handle_t = struct rac_model_registry* (protected by the new guard) and note that ModelRegistryBridge.hpp still includes the real header so there is no ODR mismatch.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@examples/react-native/RunAnywhereAI/metro.config.js`:
- Around line 12-14: metro.config currently unconditionally sets geniePkgPath
and always watches/aliases '@runanywhere/genie', which breaks when the package
is absent; change it to attempt resolving the package with require.resolve
inside a try/catch (or use try { const p =
require.resolve('@runanywhere/genie'); geniePkgPath =
path.resolve(path.dirname(p)); } catch (e) { geniePkgPath = null; }) and only
push geniePkgPath into watchFolders and add the resolver.alias entry when
geniePkgPath is non-null so Metro mirrors the App.tsx optional-load behavior for
the '@runanywhere/genie' package.
In
`@sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_model_registry.dart`:
- Around line 275-276: The reverse mapping in _frameworkFromFfi is missing the
case for value 10 (RAC_FRAMEWORK_GENIE), causing integer 10 to be decoded to
InferenceFramework.unknown; update the _frameworkFromFfi switch to add a case
for 10 that returns public_types.InferenceFramework.genie so it mirrors
_frameworkToFfi (which maps InferenceFramework.genie to 10) and preserves
symmetry when converting between FFI ints and InferenceFramework.
In `@sdk/runanywhere-flutter/packages/runanywhere/lib/runanywhere.dart`:
- Line 30: The barrel export was narrowed to only export DartBridgeRAG, removing
the RAG type symbols and breaking consumers; restore public RAG types by either
adding an export for the type definitions (export
'public/types/rag_types.dart';) to the main barrel (runanywhere.dart) so
RAGConfiguration, RAGQueryOptions, RAGSearchResult, RAGResult are publicly
available, or alternatively add explicit re-exports of those types from
runanywhere_rag.dart (e.g., re-export RAGConfiguration, RAGQueryOptions,
RAGSearchResult, RAGResult) and then export runanywhere_rag.dart from the main
barrel alongside DartBridgeRAG.
In
`@sdk/runanywhere-react-native/packages/core/cpp/bridges/CompatibilityBridge.hpp`:
- Around line 31-33: The permissive defaults (isCompatible, canRun, canFit) in
CompatibilityBridge.hpp are masking real compatibility checks; update the
CompatibilityBridge implementation (in the checkCompatibility method) to emit a
diagnostic warning whenever the stub path is taken so developers/telemetry can
detect bypassed checks—log the model identifier and which flags are being
stubbed (reference isCompatible, canRun, canFit and the checkCompatibility
function) using the platform logger (e.g., ALOGW or the project’s logging
utility) with a clear message like "CompatibilityBridge: returning stub result
for model <modelId> - isCompatible=<>, canRun=<>, canFit=<>". Ensure the log is
executed only when the stub branch is used.
In `@sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp`:
- Line 909: Replace the magic numeric literal "case 10:" with the named enum
constant RAC_FRAMEWORK_GENIE in both switch statements (the ones inside
frameworkFromString() and the corresponding switch that maps framework enums to
strings) so the case labels use RAC_FRAMEWORK_GENIE instead of 10, keeping the
existing break and comment (// RAC_FRAMEWORK_GENIE) intact to maintain
consistency with the other case labels.
- Line 274: Replace the hardcoded cast (rac_inference_framework_t)10 used for
the "Genie"/"genie" branch with the enum constant RAC_FRAMEWORK_GENIE from
rac_model_types.h; locate the string-checking branch (if (framework == "Genie"
|| framework == "genie")) in HybridRunAnywhereCore.cpp and return
RAC_FRAMEWORK_GENIE instead of the magic number so the mapping stays correct if
the enum values change.
In `@sdk/runanywhere-react-native/packages/core/src/Public/RunAnywhere.ts`:
- Line 19: The import of SecureStorageService is currently at module top-level
which eagerly instantiates the service for all RunAnywhere consumers; change
this to a dynamic import inside the authentication flow so the service is
lazy-loaded only when needed. Specifically, remove the top-level import and
perform an await import('../Foundation/Security/SecureStorageService') inside
the _authenticateWithBackend() method (just before persisting tokens), then call
SecureStorageService.storeAuthTokens(authResponse.access_token,
authResponse.refresh_token, authResponse.expires_in). Ensure any TypeScript
types are adjusted for the dynamic import result.
---
Outside diff comments:
In `@sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp`:
- Around line 294-301: The formatFromString function currently maps several
string formats to rac_model_format_t but omits handling for the Genie NPU
format; add checks in rac_model_format_t formatFromString(const std::string&
format) to return RAC_MODEL_FORMAT_QNN_CONTEXT when format equals "QNN_CONTEXT"
or "qnn_context" (similar to the other comparisons), ensuring Genie model
strings don't fall through to RAC_MODEL_FORMAT_UNKNOWN.
---
Nitpick comments:
In
`@sdk/runanywhere-flutter/packages/runanywhere/lib/infrastructure/download/download_service.dart`:
- Around line 334-344: The supported-detection is case-sensitive and missing
.tar.xz/.txz; convert archivePath to lowercase before testing (e.g., use final
lc = archivePath.toLowerCase()) and update the supported boolean (used in the
same function) to include lc.endsWith('.tar.xz') and lc.endsWith('.txz')
alongside the existing checks; keep the existing _logger.warning('Unknown
archive format: $archivePath') and return behavior unchanged.
In
`@sdk/runanywhere-react-native/packages/core/cpp/bridges/CompatibilityBridge.hpp`:
- Around line 14-22: The current fallback typedef for
rac_model_registry_handle_t depends on the external include guard macro
RAC_MODEL_REGISTRY_H which is fragile; instead forward-declare struct
rac_model_registry and introduce a dedicated guard macro (e.g.
RAC_MODEL_REGISTRY_HANDLE_T_DEFINED) to protect the typedef so all translation
units agree on the pointer type; update CompatibilityBridge.hpp to remove the
RAC_MODEL_REGISTRY_H conditional, add a forward declaration of struct
rac_model_registry, and typedef rac_model_registry_handle_t = struct
rac_model_registry* (protected by the new guard) and note that
ModelRegistryBridge.hpp still includes the real header so there is no ODR
mismatch.
In `@sdk/runanywhere-react-native/packages/core/cpp/bridges/RAGBridge.hpp`:
- Around line 36-39: Remove the now-unused std::mutex member from RAGBridge:
delete the private field named mutex_ from the RAGBridge class declaration
(constructor RAGBridge() remains), and also remove the `#include` <mutex>
directive if nothing else in this header needs it; ensure no other code
references RAGBridge::mutex_ before committing.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ee26b73b-8b45-4828-bf5e-750f6a226d47
⛔ Files ignored due to path filters (2)
examples/react-native/RunAnywhereAI/package-lock.jsonis excluded by!**/package-lock.jsonexamples/react-native/RunAnywhereAI/yarn.lockis excluded by!**/yarn.lock,!**/*.lock
📒 Files selected for processing (37)
examples/flutter/RunAnywhereAI/android/app/src/main/AndroidManifest.xmlexamples/flutter/RunAnywhereAI/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.javaexamples/flutter/RunAnywhereAI/ios/Runner/GeneratedPluginRegistrant.mexamples/flutter/RunAnywhereAI/lib/app/runanywhere_ai_app.dartexamples/flutter/RunAnywhereAI/lib/features/chat/chat_interface_view.dartexamples/flutter/RunAnywhereAI/lib/features/models/model_list_view_model.dartexamples/flutter/RunAnywhereAI/lib/features/models/model_types.dartexamples/flutter/RunAnywhereAI/pubspec.yamlexamples/react-native/RunAnywhereAI/App.tsxexamples/react-native/RunAnywhereAI/android/app/build.gradleexamples/react-native/RunAnywhereAI/android/app/src/main/AndroidManifest.xmlexamples/react-native/RunAnywhereAI/android/app/src/main/java/com/runanywhereaI/MainApplication.ktexamples/react-native/RunAnywhereAI/android/settings.gradleexamples/react-native/RunAnywhereAI/metro.config.jsexamples/react-native/RunAnywhereAI/package.jsonexamples/react-native/RunAnywhereAI/src/components/model/ModelSelectionSheet.tsxexamples/react-native/RunAnywhereAI/src/screens/ChatScreen.tsxexamples/react-native/RunAnywhereAI/src/types/model.tssdk/runanywhere-flutter/packages/runanywhere/CHANGELOG.mdsdk/runanywhere-flutter/packages/runanywhere/lib/core/types/model_types.dartsdk/runanywhere-flutter/packages/runanywhere/lib/infrastructure/download/download_service.dartsdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_model_paths.dartsdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_model_registry.dartsdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_rag.dartsdk/runanywhere-flutter/packages/runanywhere/lib/native/type_conversions/model_types_cpp_bridge.dartsdk/runanywhere-flutter/packages/runanywhere/lib/public/runanywhere.dartsdk/runanywhere-flutter/packages/runanywhere/lib/public/types/rag_types.dartsdk/runanywhere-flutter/packages/runanywhere/lib/runanywhere.dartsdk/runanywhere-flutter/packages/runanywhere/pubspec.yamlsdk/runanywhere-react-native/packages/core/android/CMakeLists.txtsdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cppsdk/runanywhere-react-native/packages/core/cpp/bridges/CompatibilityBridge.hppsdk/runanywhere-react-native/packages/core/cpp/bridges/RAGBridge.hppsdk/runanywhere-react-native/packages/core/src/Public/Extensions/RunAnywhere+Models.tssdk/runanywhere-react-native/packages/core/src/Public/RunAnywhere.tssdk/runanywhere-react-native/packages/core/src/types/enums.tssdk/runanywhere-react-native/packages/llamacpp/android/CMakeLists.txt
✅ Files skipped from review due to trivial changes (1)
- sdk/runanywhere-flutter/packages/runanywhere/pubspec.yaml
Comment on lines
+12
to
+14
| // Genie package — consumed from npm (@runanywhere/genie) | ||
| const geniePkgPath = path.resolve(__dirname, 'node_modules/@runanywhere/genie'); | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
React Native Metro documentation for monorepos/workspaces: how should watchFoldersandresolver.extraNodeModuleslocate an installed package, and is hard-coding/node_modules/ supported when dependencies may be hoisted or missing?
💡 Result:
-
watchFolders: In a workspace/monorepo, Metro expects you to add the workspace root (and/or linked package source directories) towatchFoldersso it can see and resolve code outside the app folder (including hoistednode_modules). [1] -
How Metro finds an installed package (high level):
- Normal Node-style lookup: try
node_modulesfrom the importing file’s directory upward. [2] - Then check any configured
resolver.nodeModulesPaths. [2] - Then (optionally) apply
resolver.extraNodeModules: it’s an alias map from package name → directory, and Metro constructs the candidate path by replacing the package-name portion with the mapped directory. [1][2]
- Normal Node-style lookup: try
-
What
resolver.extraNodeModulesshould point at in monorepos:- It should resolve to where the package actually lives (often the workspace root
node_modules/<name>when deps are hoisted), commonly implemented as a Proxy fallback toworkspaceRoot/node_modules/<name>in monorepos. [3]
- It should resolve to where the package actually lives (often the workspace root
-
Is hard-coding
<app>/node_modules/<package>“supported” when deps may be hoisted/missing?- It’s not robust: if the dependency is hoisted (so it isn’t under
<app>/node_modules) or you’re using an install strategy where it’s not physically there, that hard-coded path simply won’t exist, and Metro will fail to resolve it. Metro/Expo guidance for monorepos is to include the workspace root in resolution (viawatchFolders+nodeModulesPaths, and optionallyextraNodeModules), not to assume a flat, app-localnode_modules. [1][3]
- It’s not robust: if the dependency is hoisted (so it isn’t under
Sources: Metro configuration docs [1], Metro resolution algorithm docs [2], Expo monorepo Metro config examples (including Proxy extraNodeModules / nodeModulesPaths) [3].
🏁 Script executed:
# Check if the file exists and view the relevant lines
if [ -f "examples/react-native/RunAnywhereAI/metro.config.js" ]; then
echo "=== File exists, showing lines 10-35 ==="
cat -n "examples/react-native/RunAnywhereAI/metro.config.js" | sed -n '10,35p'
else
echo "File not found at expected path"
fiRepository: RunanywhereAI/runanywhere-sdks
Length of output: 1340
🏁 Script executed:
# Search for other Metro config files in the repo
fd -e "metro.config.js" -o -e "metro.config.ts"Repository: RunanywhereAI/runanywhere-sdks
Length of output: 190
🏁 Script executed:
# Search for require.resolve patterns in Metro configs to see if this approach is used elsewhere
rg "require\.resolve" --type js --type ts -l | head -20Repository: RunanywhereAI/runanywhere-sdks
Length of output: 56
🏁 Script executed:
# Check the package.json structure to understand dependencies and workspace setup
head -100 "examples/react-native/RunAnywhereAI/package.json"Repository: RunanywhereAI/runanywhere-sdks
Length of output: 3656
🏁 Script executed:
# Check what sdkPackagesPath resolves to (from metro.config.js lines before 12)
cat -n "examples/react-native/RunAnywhereAI/metro.config.js" | sed -n '1,15p'Repository: RunanywhereAI/runanywhere-sdks
Length of output: 856
🏁 Script executed:
# Search for any conditional logic around genie package in the codebase
rg "@runanywhere/genie" -C 3Repository: RunanywhereAI/runanywhere-sdks
Length of output: 5899
🏁 Script executed:
# Check if there's a workspace root package.json
cat -n "package.json" 2>/dev/null | head -50 || echo "No package.json at root"Repository: RunanywhereAI/runanywhere-sdks
Length of output: 56
Make Metro config match app-level Genie optionality.
The app in App.tsx already treats @runanywhere/genie as optional (try-catch with fallback), but metro.config.js unconditionally watches and aliases it. If the package is missing from node_modules (e.g., in environments without access to the private package or in different install strategies), Metro will fail before the app can gracefully degrade.
Use require.resolve() with error handling to conditionally include Genie only when present, matching the app's existing optional-load pattern:
Suggested fix
-// Genie package — consumed from npm (`@runanywhere/genie`)
-const geniePkgPath = path.resolve(__dirname, 'node_modules/@runanywhere/genie');
+// Genie package — consumed from npm (`@runanywhere/genie`)
+let geniePkgPath = null;
+try {
+ geniePkgPath = path.dirname(require.resolve('@runanywhere/genie/package.json'));
+} catch {
+ geniePkgPath = null;
+}
@@
- watchFolders: [sdkPackagesPath, geniePkgPath],
+ watchFolders: [sdkPackagesPath, ...(geniePkgPath ? [geniePkgPath] : [])],
resolver: {
@@
- '@runanywhere/genie': geniePkgPath,
+ ...(geniePkgPath ? { '@runanywhere/genie': geniePkgPath } : {}),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // Genie package — consumed from npm (@runanywhere/genie) | |
| const geniePkgPath = path.resolve(__dirname, 'node_modules/@runanywhere/genie'); | |
| // Genie package — consumed from npm (`@runanywhere/genie`) | |
| let geniePkgPath = null; | |
| try { | |
| geniePkgPath = path.dirname(require.resolve('@runanywhere/genie/package.json')); | |
| } catch { | |
| geniePkgPath = null; | |
| } | |
| // ... (other config code) | |
| watchFolders: [sdkPackagesPath, ...(geniePkgPath ? [geniePkgPath] : [])], | |
| resolver: { | |
| extraNodeModules: { | |
| ...(geniePkgPath ? { '@runanywhere/genie': geniePkgPath } : {}), | |
| // ... other aliases | |
| }, | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@examples/react-native/RunAnywhereAI/metro.config.js` around lines 12 - 14,
metro.config currently unconditionally sets geniePkgPath and always
watches/aliases '@runanywhere/genie', which breaks when the package is absent;
change it to attempt resolving the package with require.resolve inside a
try/catch (or use try { const p = require.resolve('@runanywhere/genie');
geniePkgPath = path.resolve(path.dirname(p)); } catch (e) { geniePkgPath = null;
}) and only push geniePkgPath into watchFolders and add the resolver.alias entry
when geniePkgPath is non-null so Metro mirrors the App.tsx optional-load
behavior for the '@runanywhere/genie' package.
Comment on lines
+275
to
+276
| case public_types.InferenceFramework.genie: | ||
| return 10; // RAC_FRAMEWORK_GENIE |
There was a problem hiding this comment.
Missing reverse mapping for Genie framework.
The _frameworkToFfi correctly maps genie to 10, but the corresponding _frameworkFromFfi method (lines 342-360) is missing a case for value 10. This asymmetry means models with framework=10 retrieved from the C++ registry will be incorrectly converted to InferenceFramework.unknown.
🐛 Proposed fix to add reverse mapping
Add the missing case in _frameworkFromFfi around line 358:
case 6:
return public_types.InferenceFramework.none;
+ case 10:
+ return public_types.InferenceFramework.genie;
default:
return public_types.InferenceFramework.unknown;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@sdk/runanywhere-flutter/packages/runanywhere/lib/native/dart_bridge_model_registry.dart`
around lines 275 - 276, The reverse mapping in _frameworkFromFfi is missing the
case for value 10 (RAC_FRAMEWORK_GENIE), causing integer 10 to be decoded to
InferenceFramework.unknown; update the _frameworkFromFfi switch to add a case
for 10 that returns public_types.InferenceFramework.genie so it mirrors
_frameworkToFfi (which maps InferenceFramework.genie to 10) and preserves
symmetry when converting between FFI ints and InferenceFramework.
| export 'public/extensions/runanywhere_storage.dart'; | ||
| export 'native/dart_bridge_rag.dart' | ||
| show RAGConfiguration, RAGQueryOptions, RAGSearchResult, RAGResult; | ||
| export 'native/dart_bridge_rag.dart' show DartBridgeRAG; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify that RAG types are exported from the extension file
rg -n 'export.*rag_types' sdk/runanywhere-flutter/packages/runanywhere/lib/public/extensions/
rg -n 'RAGConfiguration|RAGQueryOptions|RAGSearchResult|RAGResult' sdk/runanywhere-flutter/packages/runanywhere/lib/public/extensions/runanywhere_rag.dartRepository: RunanywhereAI/runanywhere-sdks
Length of output: 374
🏁 Script executed:
# Check the actual contents of runanywhere_rag.dart to see if types are exported
cat -n sdk/runanywhere-flutter/packages/runanywhere/lib/public/extensions/runanywhere_rag.dart | head -50
# Check if rag_types.dart exists and see its contents
cat -n sdk/runanywhere-flutter/packages/runanywhere/lib/public/types/rag_types.dart 2>/dev/null || echo "File not found"
# Check the main runanywhere.dart file to see all current exports
cat -n sdk/runanywhere-flutter/packages/runanywhere/lib/runanywhere.dartRepository: RunanywhereAI/runanywhere-sdks
Length of output: 11752
Breaking API change: RAG types are inaccessible to consumers.
The narrowed export at line 30 removes all RAG type re-exports, but the promised "migration path" does not exist. Verification shows:
rag_types.dartdefines the types (RAGConfiguration,RAGQueryOptions,RAGSearchResult,RAGResult) but does not export them.runanywhere_rag.dartimports these types for internal use but does not re-export them.- The main
runanywhere.dartbarrel file does not exportrag_types.dartor therunanywhere_rag.dartextension.
Consumers who previously imported RAG types from this barrel file will now get a breaking error. Either:
- Add
export 'public/types/rag_types.dart';to the main barrel file, or - Add explicit re-exports in
runanywhere_rag.dartand export that extension from the barrel file.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@sdk/runanywhere-flutter/packages/runanywhere/lib/runanywhere.dart` at line
30, The barrel export was narrowed to only export DartBridgeRAG, removing the
RAG type symbols and breaking consumers; restore public RAG types by either
adding an export for the type definitions (export
'public/types/rag_types.dart';) to the main barrel (runanywhere.dart) so
RAGConfiguration, RAGQueryOptions, RAGSearchResult, RAGResult are publicly
available, or alternatively add explicit re-exports of those types from
runanywhere_rag.dart (e.g., re-export RAGConfiguration, RAGQueryOptions,
RAGSearchResult, RAGResult) and then export runanywhere_rag.dart from the main
barrel alongside DartBridgeRAG.
Comment on lines
+31
to
+33
| bool isCompatible = true; // Default permissive — function not yet in librac_commons | ||
| bool canRun = true; | ||
| bool canFit = true; |
There was a problem hiding this comment.
Permissive defaults may mask compatibility failures at runtime.
Setting isCompatible, canRun, and canFit to true by default means that when the stub is active, all models will appear compatible regardless of actual device capabilities. This could lead to:
- Runtime crashes when loading incompatible models
- Poor user experience on unsupported devices
- Difficulty debugging compatibility issues in production
While the stub behavior is documented, consider adding runtime logging when the stub path is taken, so developers and telemetry can identify when real compatibility checks are being bypassed.
🛡️ Proposed fix: Add diagnostic logging
struct CompatibilityResult {
bool isCompatible = true; // Default permissive — function not yet in librac_commons
bool canRun = true;
bool canFit = true;
+ bool isStubResult = true; // Indicates result came from stub, not real check
int64_t requiredMemory = 0;
int64_t availableMemory = 0;
int64_t requiredStorage = 0;
int64_t availableStorage = 0;
};Then in checkCompatibility:
// Consider logging: ALOGW("CompatibilityBridge: returning stub result for model %s", modelId.c_str());🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@sdk/runanywhere-react-native/packages/core/cpp/bridges/CompatibilityBridge.hpp`
around lines 31 - 33, The permissive defaults (isCompatible, canRun, canFit) in
CompatibilityBridge.hpp are masking real compatibility checks; update the
CompatibilityBridge implementation (in the checkCompatibility method) to emit a
diagnostic warning whenever the stub path is taken so developers/telemetry can
detect bypassed checks—log the model identifier and which flags are being
stubbed (reference isCompatible, canRun, canFit and the checkCompatibility
function) using the platform logger (e.g., ALOGW or the project’s logging
utility) with a clear message like "CompatibilityBridge: returning stub result
for model <modelId> - isCompatible=<>, canRun=<>, canFit=<>". Ensure the log is
executed only when the stub branch is used.
| #endif | ||
| if (framework == "FoundationModels") return RAC_FRAMEWORK_FOUNDATION_MODELS; | ||
| if (framework == "SystemTTS") return RAC_FRAMEWORK_SYSTEM_TTS; | ||
| if (framework == "Genie" || framework == "genie") return (rac_inference_framework_t)10; // RAC_FRAMEWORK_GENIE |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Use the RAC_FRAMEWORK_GENIE constant instead of hardcoded magic number.
The hardcoded cast (rac_inference_framework_t)10 is fragile and inconsistent with how other frameworks are handled in this function (e.g., RAC_FRAMEWORK_LLAMACPP, RAC_FRAMEWORK_ONNX). If the enum value changes in rac_model_types.h, this code will silently break.
Proposed fix
- if (framework == "Genie" || framework == "genie") return (rac_inference_framework_t)10; // RAC_FRAMEWORK_GENIE
+ if (framework == "Genie" || framework == "genie") return RAC_FRAMEWORK_GENIE;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if (framework == "Genie" || framework == "genie") return (rac_inference_framework_t)10; // RAC_FRAMEWORK_GENIE | |
| if (framework == "Genie" || framework == "genie") return RAC_FRAMEWORK_GENIE; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp` at
line 274, Replace the hardcoded cast (rac_inference_framework_t)10 used for the
"Genie"/"genie" branch with the enum constant RAC_FRAMEWORK_GENIE from
rac_model_types.h; locate the string-checking branch (if (framework == "Genie"
|| framework == "genie")) in HybridRunAnywhereCore.cpp and return
RAC_FRAMEWORK_GENIE instead of the magic number so the mapping stays correct if
the enum values change.
| #endif | ||
| case RAC_FRAMEWORK_FOUNDATION_MODELS: frameworkStr = "FoundationModels"; break; | ||
| case RAC_FRAMEWORK_SYSTEM_TTS: frameworkStr = "SystemTTS"; break; | ||
| case 10: frameworkStr = "Genie"; break; // RAC_FRAMEWORK_GENIE |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Replace magic number case 10: with RAC_FRAMEWORK_GENIE in both switch statements.
Same issue as frameworkFromString() — using a hardcoded 10 instead of the named constant creates maintenance risk and is inconsistent with the other case labels in these switches.
Proposed fix for both locations
Line 909:
- case 10: frameworkStr = "Genie"; break; // RAC_FRAMEWORK_GENIE
+ case RAC_FRAMEWORK_GENIE: frameworkStr = "Genie"; break;Line 983:
- case 10: frameworkStr = "Genie"; break; // RAC_FRAMEWORK_GENIE
+ case RAC_FRAMEWORK_GENIE: frameworkStr = "Genie"; break;Also applies to: 983-983
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp` at
line 909, Replace the magic numeric literal "case 10:" with the named enum
constant RAC_FRAMEWORK_GENIE in both switch statements (the ones inside
frameworkFromString() and the corresponding switch that maps framework enums to
strings) so the case labels use RAC_FRAMEWORK_GENIE instead of 10, keeping the
existing break and comment (// RAC_FRAMEWORK_GENIE) intact to maintain
consistency with the other case labels.
| import { SDKLogger } from '../Foundation/Logging/Logger/SDKLogger'; | ||
| import { SDKConstants } from '../Foundation/Constants'; | ||
| import { FileSystem } from '../services/FileSystem'; | ||
| import { SecureStorageService } from '../Foundation/Security/SecureStorageService'; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
secure_file="$(fd '^SecureStorageService\.(ts|tsx|js|jsx)$' sdk/runanywhere-react-native/packages/core/src | head -n 1)"
echo "== SecureStorageService file =="
echo "$secure_file"
echo
echo "== Top of SecureStorageService =="
sed -n '1,220p' "$secure_file"
echo
echo "== Top-level native/platform-related references in SecureStorageService =="
rg -n -C3 'requireNativeModule|NativeModules|expo-secure-store|react-native-keychain|MMKV|AsyncStorage|Platform\.' "$secure_file"
echo
echo "== All imports of SecureStorageService in core package =="
rg -n -C2 "SecureStorageService" sdk/runanywhere-react-native/packages/core/srcRepository: RunanywhereAI/runanywhere-sdks
Length of output: 12662
Keep SecureStorageService lazy-loaded in the auth path.
This shifts a previously on-demand dependency to eager instantiation at module load. While native modules are not accessed at module scope, the service instantiation itself becomes an upfront cost for all consumers of RunAnywhere, even those that never authenticate. Deferring the import to the auth path aligns with SOLID principles by keeping dependencies tight and reducing unnecessary module-load coupling.
Suggested change
-import { SecureStorageService } from '../Foundation/Security/SecureStorageService';// inside _authenticateWithBackend(), just before persisting tokens
const { SecureStorageService } = await import('../Foundation/Security/SecureStorageService');
await SecureStorageService.storeAuthTokens(
authResponse.access_token,
authResponse.refresh_token,
authResponse.expires_in
);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@sdk/runanywhere-react-native/packages/core/src/Public/RunAnywhere.ts` at line
19, The import of SecureStorageService is currently at module top-level which
eagerly instantiates the service for all RunAnywhere consumers; change this to a
dynamic import inside the authentication flow so the service is lazy-loaded only
when needed. Specifically, remove the top-level import and perform an await
import('../Foundation/Security/SecureStorageService') inside the
_authenticateWithBackend() method (just before persisting tokens), then call
SecureStorageService.storeAuthTokens(authResponse.access_token,
authResponse.refresh_token, authResponse.expires_in). Ensure any TypeScript
types are adjusted for the dynamic import result.
… in model picker UI
…tive SDKs Adds public getChip() API that detects Qualcomm NPU chipsets (Snapdragon 8 Elite / 8 Elite Gen 5) and returns an identifier used to construct dynamic HuggingFace download URLs for Genie NPU models. Android app ModelList now uses getChip() for dynamic model registration instead of hardcoded URLs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update NPUChip: new base URL (runanywhere/genie-npu-models), new naming
convention ({slug}-genie-w4a16-{chipSuffix}.tar.gz)
- Add Llama 3.2 1B Instruct (both 8 Elite and 8 Elite Gen 5)
- Add Qwen3 4B (8 Elite Gen 5 only)
- Extensible GenieModelDef pattern for easy addition of future models
- Update NPUChip in Flutter SDK: new base URL, npuSuffix field, tar.gz format - Update NPUChip in React Native SDK: same changes as Flutter - Update Flutter example: chip-based dynamic model registration with per-chip filtering (Qwen3 4B gen5-only, Llama 3.2 1B both chips) - Update React Native example: same chip-based registration pattern - Update doc comments to reflect new URL format
Genie NPU fixes (all 3 SDKs): - downloadUrl() now accepts quant parameter (default w4a16) for w8a16 models - Fixed model slug mismatch (llama-v3.2 → llama3.2) - Added missing models: SEA-LION v3.5 8B, Qwen 2.5 7B (w8a16) Kotlin SDK: - SecureStorageFactory.create() race condition → @synchronized - SDKLogger destination iteration thread safety → .toList() snapshot - isNativeInitialized() now calls RunAnywhereBridge.racIsInitialized() - Removed runBlocking ANR risk → suspend chain with Mutex React Native SDK: - Memory leak: added rac_llm_result_free() in generate() - cancelDownload() now actually stops RNFS download via jobId tracking - CMakeLists RAC_INCLUDE_DIR fixed to jniLibs/include/ (3 files) - PlatformAdapterBridge.getChipName() uses Build.SOC_MODEL (API 31+) - extractIntValue/extractFloatValue crash guard (try/catch) - transcribeFile returns JSON matching transcribe() + fixed JSON escaping - WAV parser: memcpy instead of reinterpret_cast (ARM safe) - lastError_ protected with errorMutex_ - deleteModel handles all file extensions - getModelInfo handles empty object as null - Fixed VLM cross-package import error Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oss SDKs - Create runanywhere_genie Flutter package (thin FFI wrapper for rac_backend_genie_register) with Genie class: register/unregister/addModel/isAvailable/canHandle (Android/Snapdragon only) - Fix Kotlin DeviceInfoService.getChipName() to return Build.SOC_MODEL instead of ABI string - Standardize Genie display name to "Qualcomm Genie" across Flutter, RN (matching C++ source) - Fix Flutter example app to use local path dependency for runanywhere_genie Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix Flutter FFI struct mismatch: add missing supports_lora field to RacModelInfoCStruct causing SIGSEGV crash on model registration - Fix Flutter model extraction double-nesting: flatten single extracted subdirectory contents into destDir instead of renaming - Fix Kotlin model extraction: snapshot-based logic to handle archives with non-matching root directories (Genie NPU tar.gz) - Fix Kotlin moveDownloadToFinal: use deleteRecursively() for directory cleanup instead of delete() which silently fails on non-empty dirs - Fix Kotlin ModelFormat enum: align integer values with C++ rac_model_format_t and add QNN_CONTEXT format for Genie models - Add error dialog to Android example app for model load failures Made-with: Cursor
…AI/runanywhere-sdks into smonga/genie_support
shubhammalhotra28
added a commit
that referenced
this pull request
Mar 20, 2026
…ragon)" This reverts commit 7be8565.
4 tasks
5 tasks
sanchitmonga22
added a commit
that referenced
this pull request
Apr 14, 2026
Adds Qualcomm Genie NPU backend (SM8750 S25 + SM8850 S26) with full support in Flutter/Kotlin/RN. Critical enum renumbering: - #476 MetalRT claimed slot 10 earlier in this integration branch. - Genie renumbered from RAC_FRAMEWORK_GENIE=10 to =11 in rac_model_types.h. - Reverse-lookup integer literals updated from 10 → 11 across: - sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/ foundation/bridge/extensions/CppBridgeModelRegistry.kt (added MLX/COREML/ WHISPERKIT_COREML/METALRT constants too, matching C++) - sdk/runanywhere-react-native/packages/core/cpp/HybridRunAnywhereCore.cpp (3 sites: frameworkFromString + 2 reverse lookups) - sdk/runanywhere-flutter/packages/runanywhere/lib/native/ type_conversions/model_types_cpp_bridge.dart (added mlx/coreml/ whisperkitCoreml/metalrt constants) - sdk/runanywhere-flutter/packages/runanywhere/lib/native/ dart_bridge_model_paths.dart + dart_bridge_model_registry.dart Conflict resolution: - llamacpp_backend.cpp: combined <climits> + <cstdint> (for UTF-8 safety casts from #472) with <cmath> (pr-462's math helpers). - model_paths.cpp: kept both METALRT and GENIE in framework name LUT and in framework-directory validation list. - model_registry.cpp: added both METALRT and GENIE to scan list for downloaded models. - model_types.cpp: both METALRT and GENIE are directory-based frameworks. Re-creation of reverted #446. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
23 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runanywhere-commons) and Kotlin (runanywhere-kotlin)runanywhere-genierepo), keeping the public SDK cleanChanges in
runanywhere-commons(C++)RAC_FRAMEWORK_GENIE = 10andRAC_MODEL_FORMAT_QNN_CONTEXT = 5enumsrac_model_detect_framework_from_format()missing QNN_CONTEXT caseChanges in
runanywhere-kotlin(Kotlin Multiplatform)InferenceFramework.GENIEandSDKLogger.genieModelFormat.QNN_CONTEXTenum entry and bridge constantChanges in example app
com.runanywhere.sdk:runanywhere-genie-android:0.2.0-SNAPSHOT)extractNativeLibs=true+libcdsprpc.sonative library declarationTest plan
publishToMavenLocalsucceeds)assembleDebugsucceeds with Genie dependencySummary by CodeRabbit
New Features
Chores
Greptile Summary
This PR adds end-to-end type-system support for Qualcomm's Genie NPU backend across the C++ commons layer, the Kotlin Multiplatform SDK, and the example Android app. The new
RAC_FRAMEWORK_GENIE = 10andRAC_MODEL_FORMAT_QNN_CONTEXT = 5enums are wired through model registration, path detection, registry scanning, framework routing, and telemetry — mirroring the pattern already used for LlamaCPP and CoreML. Genie is distributed as a closed-source AAR and registered at priority 200, so NPU inference is preferred over CPU (LlamaCPP at 100) on supported Snapdragon devices.Critical issues found:
Constant value mismatch in Kotlin bridge:
COREMLis set to5inCppBridgeModelRegistry.kt, butRAC_MODEL_FORMAT_COREML = 4in C++. This creates a collision with the newly addedQNN_CONTEXT = 5, causing CoreML models (format4from C++) to be silently misidentified asUNKNOWNin Kotlin bridges — an acknowledged mismatch that must be fixed in this PR.Unconditional debug logging in production paths: Multiple
ALOGD()calls inrac_llm_service.cppandservice_registry.cppemit to Android logcat unconditionally on every LLM creation and service provider selection, bypassing theRAC_LOG_*infrastructure. These are development artifacts and should be removed before merge.Confidence Score: 2/5
COREML = 5vsQNN_CONTEXT = 5collision inCppBridgeModelRegistry.ktcauses silent misidentification of CoreML format4from C++. This is particularly dangerous because the code comment acknowledges the mismatch but doesn't fix it. (2) UnconditionalALOGD()debug calls throughout LLM service creation and provider selection are clearly development artifacts that should be removed. The C++ enum additions and Kotlin/example app integrations are otherwise correct.sdk/runanywhere-kotlin/src/jvmAndroidMain/kotlin/com/runanywhere/sdk/foundation/bridge/extensions/CppBridgeModelRegistry.kt(constant collision must be fixed)sdk/runanywhere-commons/src/features/llm/rac_llm_service.cppandsdk/runanywhere-commons/src/infrastructure/registry/service_registry.cpp(debug logging must be removed)Last reviewed commit: c30b3cd
Context used:
dashboard- CLAUDE.md (source)