Repository navigation
Turn-by-turn live text on this Mac, with the measurements behind it - #479
Merged
Merged
Conversation
…peaker The live transcript preview transcribes windows of 4 to 12 seconds, so a turn's last words wait up to 4 s for their window to fill, and windows cut mid-sentence. Meta's fast live text was the only way to get text right after each turn. A new setting, Turn-by-turn live text (Settings → Meetings → Processing, off by default), does it with the Transcribe model on this Mac: - LiveTurnCutter closes a turn after two quiet 256 ms chunks of speech detection, after one once the turn is 4 s long, or at 15 s at the quietest point of its last 1.5 s. - LiveMeetingTranscriber.runTurns hears both live files every 100 ms, and transcribes closed turns one at a time. A failed turn is retried and survives a pause of the preview. - While the model answers within 0.25 s and runs on this Mac, the turn still being spoken is transcribed again every 2 s and shown as a partial line. A transcription server hears each turn once, when it is finished. - deliversTurns is true in this mode, so Live Radar reads after each turn. Muse's fast live text keeps precedence and is unchanged. One read of the live file can stop short of the frames asked for; span reads now loop until they have them. The setting's two strings are in all 24 languages, and the Windows and Linux catalogs are regenerated from them.
LiveTurnLatencyBenchmarkTests replays the bundled sample call, in English and Serbian, through the app's engines as if it were being recorded: turn by turn with LiveTurnCutter, in the preview's windows, through FluidAudio's streaming recognizer, and in real time through LiveMeetingTranscriber itself. Benchmarks/LiveTurns has the scripts, the method and the results of 10 November 2026. On an M4 Max with one track per side, a turn's text followed its last word by a median 0.5 to 0.7 s (0.9 s at worst) with Parakeet, Granite fast and Qwen3-ASR 1.7B, and 1.3 to 1.4 s with Whisper, against 1.7 to 3.2 s (3.8 to 5.0 s at worst) for windows, with fewer word errors and no more CPU time. The three tests are opt-in and listed in unit-skips.json.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
lokalbot | 725e265 | Commit Preview URL Branch Preview URL |
Nov 10 2026, 09:25 PM |
stevyhacker
marked this pull request as ready for review
November 11, 2026 05:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fast live text was the last feature that needed Meta's API. This adds the same thing with the Transcribe model on this Mac: Turn-by-turn live text (Settings → Meetings → Processing, off by default). The live preview cuts each track at pauses and transcribes one turn at a time, so a turn's text follows its last word by about half a second, where the preview's 4 to 12 second windows take about two seconds and up to five. Live Radar then reads after each turn. Muse's fast live text is unchanged and keeps precedence when it is on.
It was measured before it was built (report). The sample call as a microphone track and a call track, on an M4 Max, from a turn's last word to its text:
Muse Voice Transcribe measured 0.74 s (median) on other audio. Turns took no more CPU time than windows.
How it works:
LiveTurnCutterreads speech detection's verdict on each 256 ms chunk. A turn closes after two quiet chunks, after one once it is 4 s long (this catches 23 and 24 of 25 speaker changes when two voices share a track, against 18 and 15), or at 15 s at a quiet point.LiveMeetingTranscriber.runTurnshears both live files every 100 ms and transcribes closed turns one at a time. A failed turn is retried, and a turn waiting for the model survives a pause of the preview.The report also covers a streaming recognizer the app already links (words 0.34 s behind the speaker and 4.7% word error in English, but 43.9% on the Serbian recording), and what the sherpa-onnx and Apple routes would take.
Validation
Mac, on this branch's files (identical to the tree tested, apart from the merge commit of #478):
deliversTurnsput back, the server guard removed and a turn taken off the queue before it is transcribed, 5 of the 12LiveTurnTextTestsfail (the ones for those three things) and pass again with the code restored. The span-read test failed first on its own: one read of the live file returned 15,552 of 16,000 frames, which is why span reads now loop.LiveTurnLatencyBenchmarkTests/testPreviewReplayInRealTimewrote 90 s of the English call to disk at recording speed and ranLiveMeetingTranscriberon it with the real speech detection and model: text 0.62 s (median) and 0.94 s (worst) after a speaker's last word with Parakeet v3, 0.70 and 0.96 s with Qwen3-ASR 1.7B, with the turn in progress shown 31 times.Benchmarks/LiveTurns/README.md.Not verified:
RadarScreenAnswersTests/testAModelThatReadsPicturesIsSentTheCropAndItsCostCountsfailed once in a combined run (it timed out after 10 s) and passed in three others, twice alone. It does not touch this code.Changed test expectations
None. Every test in this PR is new.
Linked issues
None.
Risk / rollout notes
run()now shares its cursor setup and failure handling with the new mode, and the 24 existingLiveMeetingTranscriberTestspass unchanged.srhint it writes Cyrillic and scored 81 to 84% word errors on the Serbian recording, against 26 to 30% with a Croatian hint. Not fixed here.Scripts/ci/unit-skips.jsonlists the three opt-in measurements.