Repository navigation
Let people download a Whisper model, or not - #37
savvasioakeim wants to merge 2 commits into
Conversation
Dictation's Web Speech path is gone on Windows (WebView2 Runtime 153+), and it is not ours to fix, so recognition has to run on this machine. That means a model has to get here somehow, and the decision this commit encodes is that it is never the installer's call. Nothing is bundled. A model arrives only when someone presses a button, and every row states its size and its trade-off before the download rather than after. Backend (whisper.rs): the four sizes OpenAI publishes as safetensors, with commands to list, download and delete. A few things were deliberate. Each file lands as `<name>.part` and is renamed only when its stream ends, and "installed" means all three real files exist. An interrupted download therefore reads as absent rather than as a model that would fail later at load time, when the cause would be far less obvious. Progress is measured against sizes fetched by HEAD rather than against the catalogue figure, so the bar cannot sit at 100% with bytes still arriving. Events are throttled to roughly one per megabyte; per chunk would be thousands for a 1GB file. A second download of the same model is refused rather than allowed to interleave two streams into the same .part files, and deletion is refused while a download is running. The claim is released by a Drop guard so an early return cannot strand it. Sizes in the catalogue were measured against the real URLs, not estimated: 153/292/969/3058 MB for tiny/base/small/medium, all three files included. Frontend: the picker lives under Settings › Speech-to-text and only appears once Whisper is the selected engine. Delete asks first and names the size it will reclaim. The progress listener is torn down on unmount, and a row's busy state follows the download promise rather than the event stream, which can stop arriving without the download being done. Recognition itself is not here yet. The settings copy says so rather than implying the engine works. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Completes the engine behind the model picker: capture, mel, encoder, language detection and a greedy decode, all local. Audio never leaves the machine and no service can withdraw it the way the webview's did. mic.rs takes audio through cpal rather than getUserMedia. The webview has already proved unreliable for speech, and a native device also avoids a second permission prompt per window, since the QA window and the main window are separate webviews asked separately. Channels are averaged in the callback — keeping interleaved stereo would double the buffer for nothing, Whisper being mono. A recording is capped at five minutes so a forgotten button cannot grow the buffer until memory runs out. Resampling to 16kHz is linear, which is enough: these devices run at 44.1k or 48k, so it is always downsampling, and Whisper's mel front end bins to 80 bands under 8kHz, far above any interpolation error. It is covered by tests because an off-by-one would pitch-shift the audio and wreck every transcript with nothing in the output pointing at the cause. whisper.rs decodes greedily rather than running candle's temperature fallback: dictation is someone speaking deliberately into a microphone, and the fallback costs up to six decodes per window for cases that barely arise here. Audio longer than 30 seconds is walked window by window, with the last one zero-padded as the model was trained to expect. Language is detected once on the first window and held, because re-detecting per window lets a quiet stretch flip mid-sentence into another language. One bug fixed before it ever ran. Language detection masked the logits from <|startoftranscript|> to <|endoftext|>, but in Whisper's vocabulary eot is 50257 and sot is 50258 — the end sits BELOW the start, so the range was always empty and detection would have failed every time. The bound is <|translate|>, which sits immediately after the 99 language tokens. Deriving the range from those two tokens also keeps this file from carrying a list of 99 language codes that could drift out of step with the vocabulary. The model stays loaded between dictations: reading `small` off disk costs seconds that would otherwise be paid per phrase. One model is cached, and switching size evicts the previous rather than holding both, since the whole point of choosing a size is a machine that cannot afford both. Recognition runs on a blocking thread; on the async runtime it would stall every other command. Not yet verified: the decode loop has never run against real weights. The test that does it exists and is ignored (OCTOSHELL_WHISPER_DIR), but the run was killed when the machine ran out of memory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recognition is now in this branch too (
|
|
Closing this unmerged. The engine works — the decode path was finished and fixed today, and it does transcribe real speech — but it is not worth shipping. What it took to get it running. Two bugs, both invisible to the compiler:
Why it is still not worth it. Measured on an i7-8750H, after moving the inference crates to
Greek only becomes usable around What replaced it. #38: the real bug was that we asked the speech service for The branch stays if local recognition is ever worth revisiting — quantized weights, or CUDA, would change the arithmetic. The measurements above are the bar it would have to clear. |
Dictation's Web Speech path is gone on Windows (WebView2 Runtime 153+) and it is not ours to fix, so recognition has to run on this machine. That means a model has to get here somehow, and the decision this PR encodes is that it is never the installer's call. Nothing is bundled. A model arrives only when someone presses a button, and every row states its size and its trade-off before the download rather than after.
Backend —
src-tauri/src/whisper.rsThe four sizes OpenAI publishes as safetensors, with
whisper_models,whisper_downloadandwhisper_delete.Deliberate choices worth reviewing:
<name>.partand is renamed only when its stream ends; "installed" means all three real files exist. An interrupted download reads as absent rather than as a model that fails later at load time, where the cause would be far less obvious.HEADper file, not from the catalogue figure, so the bar cannot sit at 100% with bytes still arriving, or stop at 94%..partfiles; it is rejected. Deletion is refused while a download runs. The in-flight claim is released by aDropguard, so an early?return cannot strand it.Catalogue sizes are measured, not estimated
HEADagainst the real URLs, all three files counted:These are fp32 weights, which is why they are several times the quantised GGUF files people may have seen. Quantised builds exist, but only
tinyis published in a form candle reads today, andtinyis where multilingual accuracy collapses. The compression is available exactly where it is useless.Frontend
The picker lives under Settings › Speech-to-text and appears only once Whisper is the selected engine. Delete asks first and names the size it reclaims. The progress listener is torn down on unmount, and a row's busy state follows the download promise rather than the event stream, which can stop arriving without the download being finished.
Not in this PR
Recognition itself: microphone capture and the candle decode loop. The settings copy says so plainly rather than implying the engine works, and the engine dropdown still marks Whisper as unavailable.
Verification
cargo checkandcargo clippyclean, no warnings fromwhisper.rs.npx tsc --noEmitandnpm run buildpass..partrename are reasoned rather than observed.🤖 Generated with Claude Code