make whisper-cli a selectable backend

README.md

@@ -30,8 +30,9 @@ separators too.
 The Rust application is a control plane with seven direct dependencies. It does
 not compile or vendor whisper.cpp, llama.cpp, CUDA, an async runtime, an audio
 stack, or a GUI toolkit. `whispers setup` downloads checksum-pinned sherpa-onnx,
-Nemotron, Parakeet, llama.cpp, and Qwen artifacts. Inference runtimes can
-therefore be
+Nemotron, Parakeet, llama.cpp, and Qwen artifacts; `whispers setup --backend
+whisper` installs the pinned GGML model and uses the external whisper.cpp CLI
+instead of the managed recognizer. Inference runtimes can therefore be
 upgraded independently without turning every application build into an ML
 toolchain build.
 

config.example.toml

@@ -12,13 +12,13 @@ feedback = true
 typing_delay_ms = 0
 
 [recognition]
-# The managed runtime and model installed by `whispers setup`.
+# Managed recognizer: "sherpa-onnx" (default) or "whisper-cli".
 runtime = "sherpa-onnx"
 # Low-latency text shown in the OSD while speaking.
 model = "nemotron-3.5-asr-streaming-0.6b-560ms-int8"
 # Multilingual full-recording pass used for delivered text.
 final_model = "nemo-parakeet-tdt-0.6b-v3-int8"
-# Existing whisper.cpp model used only if managed ASR is unavailable.
+# whisper.cpp model used by "whisper-cli" and as the managed ASR fallback.
 fallback_model = "ggml-large-v3-turbo.bin"
 
 [enhancement]

docs/architecture.md

@@ -49,10 +49,14 @@ limited to terminal surfaces.
 ## Privacy and failure policy
 
 Audio is held in memory for the active dictation and is not retained. Temporary
-fallback WAV files are deleted immediately after `whisper-cli` exits.
+WAV files used by `whisper-cli` are deleted immediately after it exits.
 Transcripts are not logged. Setup downloads public artifacts; the dictation
 pipeline makes no network requests.
 
+Setting `[recognition].runtime = "whisper-cli"` skips the managed sherpa-onnx
+stage entirely: the recording is transcribed by the external whisper.cpp CLI
+after it stops, so there are no streaming previews.
+
 Enhancement output is rejected if it is empty, invalid, excessively expanded,
 or removes protected symbols or numbers. Final ASR falls back first to the
 streaming result, then to an existing local whisper.cpp model; enhancement falls

docs/cli.md

@@ -50,8 +50,12 @@ from the delivered text.
 ## Setup
 
 ```sh
-whispers setup [--runtime-pack DIR] [--no-service]
+whispers setup [--runtime-pack DIR] [--no-service] [--backend sherpa|whisper]
 ```
 
 Network setup verifies SHA-256 before atomically installing each asset. An
 offline pack is copied without following symbolic links or special files.
+
+`--backend whisper` sets `[recognition].runtime = "whisper-cli"`, skips the
+sherpa-onnx downloads and installs the pinned GGML model. The default
+`--backend sherpa` installs the managed runtime.

docs/install.md

@@ -45,6 +45,48 @@ separate process, so the interface adds no GUI toolkit to the Rust dependency
 graph. The OSD follows dictation through listening, transcription, enhancement,
 and delivery without taking keyboard focus.
 
+## Whisper.cpp backend
+
+Recognition can run on the external whisper.cpp CLI instead of the managed
+sherpa-onnx runtime. Select it during installation:
+
+```sh
+whispers setup --backend whisper
+```
+
+Setup sets `[recognition].runtime = "whisper-cli"`, skips the sherpa-onnx
+downloads, and installs the pinned `ggml-large-v3-turbo.bin` into the data
+directory. The equivalent configuration is:
+
+```toml
+[recognition]
+runtime = "whisper-cli"
+fallback_model = "ggml-large-v3-turbo.bin"
+```
+
+`fallback_model` is resolved relative to the data directory
+(`${XDG_DATA_HOME:-~/.local/share}/whispers`); use an absolute path only for a
+model stored elsewhere. The CLI transcribes the finished recording, so this mode
+has no partial previews.
+
+Setup requires `whisper-cli` in PATH. The distro `whisper-cpp` package ships only
+the CPU `ggml` backend, which takes seconds per sentence with larger models.
+Install a GPU backend as well. On Arch Linux:
+
+```sh
+sudo pacman -S whisper-cpp ggml-vulkan
+```
+
+`ggml-vulkan` provides `/usr/lib/ggml/libggml-vulkan.so`, which `whisper-cli`
+loads automatically. Verify on one short recording:
+
+```sh
+whisper-cli -m /path/to/ggml-large-v3-turbo.bin -f sample.wav -nt
+```
+
+The log must contain `using Vulkan0 backend`. Without a GPU backend the same
+model is roughly an order of magnitude slower.
+
 ## Setup choices
 
 Normal network setup:
@@ -53,6 +95,16 @@ Normal network setup:
 whispers setup
 ```
 
+Pick the recognition backend during installation:
+
+```sh
+whispers setup --backend whisper
+```
+
+`--backend sherpa` is the default and installs the managed runtime. Choosing
+whisper requires `whisper-cli` in PATH and installs the GGML model instead; see
+[Whisper.cpp backend](#whispercpp-backend).
+
 Install without enabling the user service:
 
 ```sh
@@ -77,6 +129,9 @@ pack/
 whispers setup --runtime-pack ./pack
 ```
 
+`whispers setup --backend whisper --runtime-pack` uses an existing model and
+fails when it is missing, because runtime packs do not carry GGML models.
+
 Setup writes configuration version 2, imports the old dictionary and technical
 glossary when present, and copies the old configuration to a timestamped backup.
 It never deletes legacy models or configuration.

docs/troubleshooting.md

@@ -26,6 +26,25 @@ pw-cat --record --raw --format f32 --rate 16000 --channels 1 >/dev/null
 Set `[audio].device` only when the default PipeWire source is not the microphone
 you want.
 
+## Whisper.cpp shows no partial text
+
+Expected when `[recognition].runtime = "whisper-cli"`. The external CLI runs
+after the recording stops, so the OSD only shows listening and transcribing.
+Switch to `sherpa-onnx` to get the live preview.
+
+## Whisper.cpp is slow or the GPU stays idle
+
+The backend package, not whispers, selects the device:
+
+```sh
+pacman -Q whisper-cpp ggml-vulkan
+whisper-cli -m /path/to/ggml-large-v3-turbo.bin -f sample.wav -nt
+```
+
+The log must print `using Vulkan0 backend`. With only `ggml` installed the CLI
+falls back to CPU and a large model takes seconds per sentence. `whispers
+doctor` reports the configured CLI and model path.
+
 ## Text was copied instead of typed
 
 Whispers refuses to type into a different window than the one where dictation

src/cli.rs

@@ -3,6 +3,7 @@ use std::path::PathBuf;
 
 use lexopt::prelude::*;
 
+use crate::config::Backend;
 use crate::{Error, Result};
 
 #[derive(Clone, Debug, Eq, PartialEq)]
@@ -13,6 +14,7 @@ pub enum Command {
     Setup {
         offline_pack: Option<PathBuf>,
         no_service: bool,
+        backend: Option<Backend>,
     },
     Doctor,
     Vocabulary(VocabularyCommand),
@@ -55,16 +57,26 @@ where
         Value(value) if value == "setup" => {
             let mut offline_pack = None;
             let mut no_service = false;
+            let mut backend = None;
             while let Some(argument) = parser.next()? {
                 match argument {
                     Long("runtime-pack") => offline_pack = Some(parser.value()?.into()),
                     Long("no-service") => no_service = true,
+                    Long("backend") => {
+                        let value = parser.value()?.string()?;
+                        backend = Some(Backend::parse(&value).ok_or_else(|| {
+                            Error::message(format!(
+                                "unknown backend: {value} (use sherpa or whisper)"
+                            ))
+                        })?);
+                    }
                     _ => return Err(argument.unexpected().into()),
                 }
             }
             Command::Setup {
                 offline_pack,
                 no_service,
+                backend,
             }
         }
         Value(value) if value == "vocabulary" => parse_vocabulary(&mut parser)?,
@@ -121,7 +133,7 @@ Usage:
   whispers toggle                  Start or stop dictation
   whispers cancel                  Cancel the active dictation
   whispers status                  Show service state
-  whispers setup [--runtime-pack DIR] [--no-service]
+  whispers setup [--runtime-pack DIR] [--no-service] [--backend sherpa|whisper]
   whispers doctor
   whispers vocabulary list
   whispers vocabulary add TERM --alias SPOKEN [--alias SPOKEN ...]
@@ -137,6 +149,27 @@ mod tests {
         assert_eq!(parse(Vec::<String>::new()).ok(), Some(Command::Toggle));
     }
 
+    #[test]
+    fn parses_setup_backend() {
+        assert_eq!(
+            parse(["setup", "--backend", "whisper"]).ok(),
+            Some(Command::Setup {
+                offline_pack: None,
+                no_service: false,
+                backend: Some(Backend::WhisperCli),
+            })
+        );
+        assert_eq!(
+            parse(["setup", "--no-service", "--backend", "sherpa"]).ok(),
+            Some(Command::Setup {
+                offline_pack: None,
+                no_service: true,
+                backend: Some(Backend::SherpaOnnx),
+            })
+        );
+        assert!(parse(["setup", "--backend", "cloud"]).is_err());
+    }
+
     #[test]
     fn parses_vocabulary_aliases() {
         let command = parse([

src/config.rs

@@ -33,12 +33,38 @@ pub struct DeliveryConfig {
     pub typing_delay_ms: u64,
 }
 
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum Backend {
+    SherpaOnnx,
+    WhisperCli,
+}
+
+impl Backend {
+    pub fn parse(value: &str) -> Option<Self> {
+        match value.trim().to_ascii_lowercase().as_str() {
+            "sherpa" | "sherpa-onnx" | "sherpa_onnx" => Some(Self::SherpaOnnx),
+            "whisper" | "whisper-cli" | "whisper_cli" => Some(Self::WhisperCli),
+            _ => None,
+        }
+    }
+
+    pub const fn runtime(self) -> &'static str {
+        match self {
+            Self::SherpaOnnx => "sherpa-onnx",
+            Self::WhisperCli => "whisper-cli",
+        }
+    }
+}
+
 #[derive(Clone, Debug, Deserialize, Serialize)]
 #[serde(default)]
 pub struct RecognitionConfig {
+    /// `sherpa-onnx` uses the managed streaming engine. `whisper-cli` runs the
+    /// external whisper.cpp binary on the completed recording instead.
     pub runtime: String,
     pub model: String,
     pub final_model: String,
+    /// whisper.cpp model used by `whisper-cli` and as the managed fallback.
     pub fallback_model: String,
 }
 
@@ -84,6 +110,14 @@ impl Default for RecognitionConfig {
     }
 }
 
+impl RecognitionConfig {
+    /// True when recognition runs through the external `whisper-cli` binary.
+    /// That path transcribes the finished recording and has no partial results.
+    pub fn uses_whisper_cli(&self) -> bool {
+        Backend::parse(&self.runtime) == Some(Backend::WhisperCli)
+    }
+}
+
 impl Default for EnhancementConfig {
     fn default() -> Self {
         Self {
@@ -143,6 +177,31 @@ impl Config {
 mod tests {
     use super::*;
 
+    #[test]
+    fn backend_names_are_parsed() {
+        assert_eq!(Backend::parse("sherpa"), Some(Backend::SherpaOnnx));
+        assert_eq!(Backend::parse(" Sherpa_Onnx "), Some(Backend::SherpaOnnx));
+        assert_eq!(Backend::parse("whisper"), Some(Backend::WhisperCli));
+        assert_eq!(Backend::parse("Whisper_CLI"), Some(Backend::WhisperCli));
+        assert_eq!(Backend::parse("cloud"), None);
+        assert_eq!(Backend::WhisperCli.runtime(), "whisper-cli");
+    }
+
+    #[test]
+    fn whisper_cli_runtime_is_recognized() {
+        let mut config = Config::default();
+        assert!(!config.recognition.uses_whisper_cli());
+
+        config.recognition.runtime = "whisper-cli".into();
+        assert!(config.recognition.uses_whisper_cli());
+
+        config.recognition.runtime = " Whisper_CLI ".into();
+        assert!(config.recognition.uses_whisper_cli());
+
+        config.recognition.runtime = "sherpa-onnx".into();
+        assert!(!config.recognition.uses_whisper_cli());
+    }
+
     #[test]
     fn existing_unversioned_configuration_requires_migration() {
         let path = std::env::temp_dir().join(format!(