feat(meetings): label Zoom speakers from Accessibility - #833
Open
Rahulk644 wants to merge 1 commit into
Open
Conversation
Add an optional, bounded macOS helper that reads only native Zoom Accessibility evidence while Rowboat is already recording. Align active-speaker observations with system-audio transcript intervals, preserve generic diarization on stale, self, overlap, or conflicting evidence, and emit a conservative meeting-end hint after a proven surface disappears. The helper is adapted from Anarlog meeting_ax at revision 609ee772801f29292e4edee453f269089ebf0e8b under its retained MIT notice. It is fail-open for recording, uses an explicit System Settings action instead of a surprise permission prompt, and makes no Chrome, Teams, Windows, or full physical-matrix claim.
Rahulk644
force-pushed
the
codex/zoom-accessibility-evidence
branch
from
August 6, 2026 07:07
355a7ec to
cccc737
Compare
Rahulk644
marked this pull request as ready for review
August 6, 2026 11:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Rowboat captures the local microphone and system audio separately, but diarization cannot map system-audio clusters to the participant display names shown by native Zoom. Meeting end also needs a stronger signal than a quiet audio track.
What this adds
us.zoom.xos).The microphone remains You. Missing, stale, self, overlapping, or conflicting evidence keeps the existing generic
Speaker Nlabel instead of guessing. Capture and transcription remain usable when Accessibility or the optional helper is unavailable.Privacy, provenance, and bounds
The helper inspects at most two native Zoom processes, eight windows per process, 1,800 AX nodes total, 18 levels, 128 children per node, and one 600 ms budget per observation. It does not read editable/secure-field values or emit raw AX nodes, arbitrary labels, window titles, screenshots, transcript text, process paths, or participant rosters. It exits when Electron closes its stdin lifetime pipe.
Traversal and explicit-label parsing are adapted from Anarlog's MIT-licensed
meeting_axat revision609ee772801f29292e4edee453f269089ebf0e8b. The exact revision, retained copyright/license, and Rowboat-specific changes are documented beside the helper.Scope and composition
This PR supports native Zoom on macOS only. The documentation gives concrete extension paths for Chrome/Meet via a metadata-only extension/native host when AX is insufficient, Teams via its own bundle-scoped AX parser, and Windows via UI Automation. Unsupported platforms keep generic diarization and never block recording.
The evidence resolver is independent of the transcription vendor. #835 now exposes provider-neutral stable-audio intervals; the two PRs have been exercised together on a combined integration branch without creating a source dependency between them.
Verification
Passed on the PR head:
git diff --checkThe broader core suite passed 585/586 tests; the unchanged ChatGPT token-revocation test timed out while attempting its network path in the restricted test environment. This branch has no core-package diff.
Remaining qualification
This PR is ready for code review, but unit/parser/build checks are not physical Zoom or macOS TCC evidence. A signed packaged build still needs the documented matrix: local-only speech, remote-only speech, overlap, mute/unmute, minimization, screen sharing, identical display names, permission revoke/re-grant, and meeting end while Zoom remains open.