Repository navigation
Bring autocomplete to other apps on Wayland as an input method - #432
Merged
Merged
Conversation
Ports what the Mac has and the Rust app lacked: selection actions; the dictation and autocomplete gaps (delivery guard, Compose context, live transcript, accept keys, macros, statistics, learning, an autocomplete server); spoken summaries; remembered speakers; Files; screen search by meaning; the overnight review; Personal Context; coding-agent sessions; note and daily exports; routines; library health; retention review; the Actions, People and Projects workspaces; agent handoff; the command palette; Settings Overview and search; agents changing settings, behind the Mac's approval; and hosting paired phones and tablets. Also fixes bugs that reached users. Saving settings dropped keys this port didn't model, so a library from a Mac lost them. Microphone pickers were always empty. Closing the window during a recording lost its end time and job. Mic transcript rows were deleted on word overlap alone. Debug logs held typed text. Think accepted plain HTTP to a server off this computer, where the Mac requires HTTPS. Picture vectors outlived their pixels, and deleted sources left generated journals and work memory behind. Coding-agent sessions need an explicit choice here, unlike on the Mac: installs already saved that key as true without asking, so an update must not start reading them silently. Remembered speakers only suggest names, because the Mac's thresholds aren't calibrated for this voice model on real calls. Not ported: Gmail in meeting briefs, and Mac-only speech runtimes. All Windows code is only type-checked, and Wayland, real phones and real calls were never run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
589 strings into the 23 other interface languages, machine-assisted like the rest of the desktop catalog and in need of native-speaker review. The two hints that show the "Remember" trigger keep it in English or Serbian, the only words the parser recognises. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moves features out of the Mac-only lists, and says what differs where it does. Also removes a paragraph in PRIVACY.md that an old merge had duplicated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and Windows Follows #424 and #426 on the Mac: every pass walks the chosen folders again, so a change the crawl would leave out no longer starts one. It leaves out tmp and temp folders too, while a folder chosen on its own is still read whole. Unlike FSEvents, the watchers here often don't say whether a path is a file or a folder, so the path answers where it still exists. When nothing can tell, the change counts, unless it's in a hidden folder. Also restores the test of clean_other_names, lost when that function moved to lokalbot-core. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ocrs, the only engine so far, reads unaccented Latin letters, and the Mac's Vision reads the rest with nothing to download. Windows now uses Windows.Media.Ocr with the language packs installed there. Linux runs the system's Tesseract for the app and system language, over a pipe, with no image file at any point. It uses the language data it has, or pinned tessdata_fast files the person chooses to download. ocrs stays as the fallback, and stays the engine for English where its models are installed. Settings and onboarding now offer whichever download helps, right under the choice that needs it, and say what is read and what is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The folder picker saves a rule under the home folder as `~\Code\client`, but only `~/` was expanded, so on Windows that exclusion matched nothing and sessions from an excluded folder were still read. A home path with backslashes was also left unnormalized. Found by the first real Windows CI run, which failed on a test that fed Unix paths to `Path::join`; that test now uses each platform's separator. The Windows job now runs every crate, so one run shows every failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wayland lets no ordinary app take another app's keys, show text at its caret or type into it, so autocomplete worked only in the practice editor there. An input method is the supported way in: it sees the text around the caret, shows the suggestion inline as preedit and commits text without the clipboard. Sway and Hyprland use zwp_input_method_v2, with the keyboard grabbed only while a suggestion shows. GNOME uses an IBus engine the user chooses as an input source. Both name the field's owner before using any of its text, refuse password fields, terminals, excluded apps and apps they cannot name, and stay out of the way of another input method. X11 and Windows are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
lokalbot | 40b877e | Commit Preview URL Branch Preview URL |
Nov 03 2026, 01:36 AM |
Stopping a pipe server woke its accept with a connection it closed at once. A Windows pipe listener skips a client that has already left, so if the thread hadn't reached ConnectNamedPipe yet, accept kept waiting and join waited on it forever. That could freeze the app on quit, or when agent settings control is turned off. The wake-up connection now stays open until the thread has gone. IpcServer, which already shipped, had the same bug. Found by the first full Windows CI run, where the test hung until the job timed out. Also fixes two tests that run failed: one fed a Unix path to Path::join, and one compared days across midnight on the runner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both failed on and off in xcodebuild test on master and on pull requests. Turning Files off deletes the index on the reader's and worker's queues, and the test gave that 300 ms. It now waits up to 20 s for the folder to go. FSEvents can report notes.txt, written just before its stream began, and that starts a second pass during the wait that should show none. The test now outlasts the watcher's latency before it counts, and again after the writes that can't alter the index: with a 2 s latency, the old 1 s wait could end before those writes had been reported at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Windows and Linux checks photographed the app's older pages only, so Files, Devices and Personal Context had never been seen on a real desktop. On Windows the palette is opened with Ctrl+K, as a person would. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The update's end-to-end check on a real Windows desktop failed with "could not replace lokalbot.exe: The process cannot access the file because it is being used by another process", and the whole update gave up on the first try. Master passed that check on the same step, so nothing shows this had reached users, but anything that opens the file without allowing its rename can cause it: antivirus after a new file, or reading a running executable's version details, as LokalBot does to name the focused app. Each rename on Windows now keeps trying for up to 30 s on a sharing violation, and says when it waited. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release notes for the next version link these. The screenshots are the demo library on a Windows CI desktop, and each explainer says what hasn't been run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR merges the whole stack. It contains #430 (Windows and Linux parity) and #431 (screen text in more scripts), so the required suite runs once on all of it. Those two have their own descriptions, including their changed test expectations. It also adds the two explainers the next release's notes link:
web/explainers/windows-linux-app.pdfandweb/explainers/wayland-autocomplete.pdf.Autocomplete now works in other apps on Sway, Hyprland and GNOME, as an input method. Before this, every Wayland session got only the practice editor in Settings, because X11's approach (key grabs, an overlay placed by coordinates, XTest typing) isn't allowed there.
zwp_input_method_v2, pluszwp_virtual_keyboard_v1to give keys back. The keyboard is grabbed only while a suggestion shows. Nothing to select..tar.gzgets a command to copy in Settings.input-method-v1), usually Fcitx 5, whose engines are C++ plugins, andkwin_waylandisn't installed here. Other compositors have no way to name the focused app.Reuses the existing suggestion, accept-key and learning logic; only the ghost's drawing changes. X11 and Windows code paths are unchanged.
Validation
Everything ran inside
isolatedwith a private bus, runtime folder andWAYLAND_DISPLAY. OnlyLOKALBOT_FAKE_AUTOCOMPLETEsupplied suggestions in app runs, so no real model produced a suggestion on these routes; the engine takes the sameFieldContextit always did.Real LokalBot app, end to end (type, ghost, Tab, full accept, typing through, Esc):
autocomplete_watch --ghostexample: GTK 3 and GTK 4 entries and text views on both desktops, including pass-through of a turned-down key; Firefox on GNOME with the same stand-in. Chrome with--enable-wayland-imeand--wayland-text-input-version=3on Sway behaves the same as with no flags.Refused, with no text reported: GTK 3 and 4, Firefox and Chrome password fields on Sway; GTK 3 and 4 on GNOME, where GNOME also swaps the engine out, so keys never reach it, and it comes back afterwards. foot refused on both.
Conflicts on Sway: a process owning
org.fcitx.Fcitx5before and after LokalBot starts, and a second copy of the route holding the seat. Typing reached the field throughout. Real Fcitx 5 isn't installed here.Others: Kate (Qt 5) never activated the input method. With the source chosen and autocomplete off or LokalBot not running, typing on GNOME is untouched. Visible text over AT-SPI reached the snapshots on Sway. The app log held no typed text in any app run with autocomplete debug logging on. The Settings row was checked in a screenshot on GNOME.
Not run:
sudostep and the built packages: tested throughIBUS_COMPONENT_PATH; a test checks the packaged file matches--ibus-component.Changed test expectations
This PR's own changes: none. No existing test referred to the old Wayland message. From the stack: see the two PRs above, which include the two
FileSearchServiceTeststiming changes.Linked issues
Refs #430, #431
Risk / rollout notes
wayland-protocols-misc0.3.12 (MIT), from the same project aswayland-client, for the two protocols.xkbcommon0.9 was already in the tree and is now a direct dependency.LICENSES/rust-crates.mdis regenerated.PRIVACY.mdsection for Linux and Windows has the wording. On GNOME the desktop asks the engine about every key in a text field while it's the chosen source; on Sway and Hyprland keys pass through only while a suggestion shows.🤖 Generated with Claude Code