Repository navigation
Let a QA reviewer point and talk instead of typing - #34
Merged
Merged
Conversation
Writing a QA note meant leaving the app under test, describing a spot in words, and typing. Two ways in, both landing on the same item's result. Point and capture. Ctrl+Alt+Q grabs the monitor under the pointer, rings where you were pointing, and attaches a full frame plus a detail crop. The trigger is an OS-level shortcut rather than a DOM keydown because by definition the app under test has focus, not ours. Alt+Shift was avoided on purpose: it switches keyboard layout on Windows. Two things about the capture were not obvious. Pointer-to-pixel mapping is done by the ratio of captured pixels to reported monitor size rather than by subtracting an origin, so it stays correct under per-monitor DPI scaling. And xcap needs the `wgc` feature: without it the backend is GDI BitBlt, which returns an all-black frame for anything the compositor draws on the GPU. The geometry is identical either way, so that failure is silent, and the ignored test prints the geometry and asserts a real marker pixel for exactly that reason. Previews come back as base64 thumbnails beside each path. The asset protocol is not enabled in this app, and turning it on for a thumbnail would mean a new protocol plus scope management. Talk instead. The dictation that existed only in the chat input is now a shared hook in src/voice, with a button under the QA notes. Pulling it out of InputBar also fixed two things it had been hiding: a failure wrote "(speech-to-text is not supported in this WebView)" into the user's own input box, and `onerror` discarded the error code entirely. It now says which failure happened, which is how we learned the Web Speech API has been failing with `network` since WebView2 Runtime 153 (it worked in 152, WebView2Feedback#5724). That is not fixed here. The settings copy says so plainly rather than claiming the engine works, and Whisper stays listed as not available yet. One latent bug came out of this. `resultsRef` was assigned during render, so handlers read it one render stale. Screenshots got away with it by arriving rarely; dictated phrases arrive about a second apart, and two close together would both read the same old notes, the second overwriting the first. `commitResult` now writes ref, state and event together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Writing a QA note meant leaving the app under test, describing a spot in words, and typing it out. This adds two ways in, both landing on the same item's result.
Point and capture
Ctrl+Alt+Qgrabs the monitor under the pointer, rings where you were pointing, and attaches a full frame plus a detail crop.The trigger is an OS-level shortcut rather than a DOM keydown because by definition the app under test has focus, not ours.
Alt+Shiftwas avoided deliberately: it switches keyboard layout on Windows.Two non-obvious points:
xcapneeds thewgcfeature. Without it the backend is GDI BitBlt, which returns an all-black frame for anything the compositor draws on the GPU. The geometry is identical either way, so the failure is silent: a correctly marked screenshot of nothing. The ignored test prints the geometry and asserts a real marker pixel for exactly that reason.Previews come back as base64 thumbnails beside each path. The asset protocol is not enabled in this app, and turning it on just for a thumbnail would mean a new protocol plus scope management.
Talk instead
The dictation that existed only in the chat input is now a shared hook in
src/voice, with a button under the QA notes.Pulling it out of
InputBarsurfaced two things it had been hiding:# (speech-to-text is not supported in this WebView)into the user's own input box.onerrordiscarded the error code entirely.With the code surfaced, we learned the Web Speech API has been failing with
networksince WebView2 Runtime 153 (it worked in 152, WebView2Feedback#5724). Confirmed here on Runtime 155. That is not fixed in this PR and is not ours to fix; the settings copy now says so plainly instead of claiming the engine works, and Whisper is listed as not available yet. A local Whisper engine is the follow-up.One latent bug fixed along the way
resultsRefwas assigned during render, so handlers read it one render stale. Screenshots got away with it by arriving rarely. Dictated phrases arrive about a second apart, and two close together would both read the same old notes, with the second overwriting the first.commitResultnow writes ref, state and event together.Verification
cargo checkandcargo clippyclean — no new warnings fromcapture.rs; the ~20 existing ones are pre-existing elsewhere.npx tsc --noEmitandnpm run buildpass.shoots_where_the_pointer_istest (needs a desktop session) confirmed the ring lands on the pointer with real, non-black pixels.🤖 Generated with Claude Code