Conversation
Implement initial QUIC-over-XMPP transport integration following the XEP-0467 design direction for a single reliable channel, with automatic endpoint selection via DNS SRV and fallback through existing TCP SRV logic. This adds QUIC endpoint planning and socket integration at the app layer, extends vendored xmpp_stone connection settings/APIs to model and attempt QUIC endpoints first, and preserves existing TCP failover behavior when QUIC attempts fail. It also records partial XEP-0467 support in doap.xml and includes the QUIC implementation plan document. Tests and validation run: - dart format on all touched Dart files - flutter analyze - flutter test - (cd vendor/xmpp_stone && dart test) New tests added: - test/quic_endpoint_plan_test.dart - vendor/xmpp_stone/test/connection_quic_failover_test.dart
Re-enable Linux QUIC transport usage and keep flutter_quic as a vendored path dependency so plugin metadata is controlled in-repo. This keeps QUIC active on Linux, Windows, and Android while preserving the existing TCP fallback behavior. Update CI and release workflows to provision Rust toolchains and required targets for flutter_quic native builds, including cargo-ndk for Android cross-compilation. Validation run before commit: - flutter analyze - flutter test - (cd vendor/xmpp_stone && dart test)
Negotiate a conservative QUIC stream limit of 25 and add post-bind stream multiplexing in the client transport. After resource binding is detected on the control stream, the QUIC socket now proactively opens up to 20 auxiliary bidirectional streams and keeps a dedicated control stream for account-scoped traffic. Outgoing stanza routing is deterministic by bare JID: - stanzas addressed to the account bare JID stay on the original control stream - stanzas to other entities are mapped via stable hash to one of 20 auxiliary stream slots Added routing helper tests for bare-JID extraction/normalization and deterministic slot bounds. Tested with: - flutter analyze - flutter test - (cd vendor/xmpp_stone && dart test)
…late The foreground task plugin requires a top-level function annotated with @pragma('vm:entry-point') to be used as the background isolate entry point. Previously, setTaskHandler was called inside an instance method (_startAndroidForegroundService), which meant the background isolate could not find or invoke it, resulting in: MissingPluginException: No implementation found for method start on channel flutter_foreground_task/background Fix by: - Adding a top-level startCallback() function annotated with @pragma('vm:entry-point') that calls setTaskHandler with WimsyForegroundTaskHandler. - Removing the misplaced setTaskHandler call from the instance method. - Passing startCallback as the callback parameter to startService(), so the plugin registers the correct entry point for the background isolate. Fixes WIMSY-6 Tested by running flutter analyze (no new issues), flutter test (all pass except pre-existing quic_happy_eyeballs_test.dart which references a missing file unrelated to this change), and dart test in vendor/xmpp_stone (all 64 tests pass). Co-authored-by: Junie <junie@jetbrains.com>
- Guard Log.logLevel = LogLevel.VERBOSE and Log.logXmpp = true behind an assert() block so verbose XMPP XML logging is only active in debug/profile builds and never leaks stanza content in release builds. - Replace DateTime.now().millisecondsSinceEpoch.remainder(1 << 31) with bareJid.hashCode.abs() % (1 << 31) in both _handleIncomingMessage and _handleIncomingRoomMessage. The deterministic ID means rapid messages from the same contact update the same notification rather than stacking new ones, fixing the grouped-dismiss bug. Tested: flutter analyze (clean), flutter test (145/145 passed), dart test in vendor/xmpp_stone (65/65 passed). Co-authored-by: Junie <junie@jetbrains.com>
ChatMessage gains a copyWith({...}) method covering all 28 fields. A
sentinel object (_absent) is used for nullable parameters so callers can
explicitly clear a field (pass null) vs. leave it unchanged (omit the
argument).
All five mutation sites that previously repeated the full 28-field
constructor are replaced with copyWith:
- lib/xmpp/chat_message_mutations.dart: applyCorrectionInList and
updateReactionsInList (2 sites)
- lib/xmpp/mam_merge_engine.dart: messageId-match merge and
heuristic body/time-window merge (2 sites)
- lib/xmpp/xmpp_service.dart: incoming-message dedup update,
room-message dedup update, _updateFileTransferMessage,
_updateOutgoingStatus, _updateOutgoingRoomStatus (5 sites)
The heuristic merge in mam_merge_engine previously omitted messageId
from the constructor (a latent bug — the field silently became null).
copyWith correctly preserves it; the characterization test in
mam_reliability_regression_test.dart is updated to assert the correct
behaviour.
Tested: flutter analyze (clean), flutter test (145/145 passed),
dart test in vendor/xmpp_stone (65/65 passed).
Co-authored-by: Junie <junie@jetbrains.com>
All 14 scattered SharedPreferences.getInstance() call sites in lib/main.dart are replaced with a single PreferencesService instance that is created once in main() and passed down through the widget tree. New file lib/storage/preferences_service.dart holds all five preference key constants (sentry_opt_in, wimsy_pin_ignored, wimsy_last_jid, wimsy_audio_input, wimsy_video_input) and provides typed getters and setters for each. WimsyApp, _Gatekeeper, WimsyHome, _PresenceMenu, _PinSetupScreen, and _PinUnlockScreen each receive PreferencesService via their constructors. The raw shared_preferences import is removed from main.dart. New test file test/preferences_service_test.dart covers all five preference groups (default values and round-trip get/set) using SharedPreferences.setMockInitialValues. test/widget_test.dart is updated to supply the required PreferencesService to WimsyApp. Tested: flutter analyze (clean), flutter test (156/156 passed), dart test in vendor/xmpp_stone (65/65 passed). Co-authored-by: Junie <junie@jetbrains.com>
Step 1 – Fix verbose logging and notification ID stability:
- Guard Log.logLevel/logXmpp behind assert() so verbose XMPP logging
is only active in debug/profile builds, never in release.
- Replace DateTime.now().millisecondsSinceEpoch.remainder(1<<31) with
bareJid.hashCode.abs() % (1<<31) for deterministic notification IDs,
preventing notification stacking for the same sender.
Step 2 – Add copyWith to ChatMessage and replace all manual copy sites:
- Added ChatMessage.copyWith({...}) with sentinel-object pattern for
nullable fields so callers can explicitly clear them to null.
- Replaced all manual full-constructor copy blocks in
mam_merge_engine.dart (2 sites), chat_message_mutations.dart (2 sites),
and xmpp_service.dart (4 sites) with copyWith calls.
- Updated mam_reliability_regression_test.dart: the heuristic-merge path
now correctly preserves messageId (copyWith keeps all unspecified fields),
fixing a latent bug where the old manual copy omitted messageId.
Step 3 – Centralise SharedPreferences into PreferencesService:
- New file lib/storage/preferences_service.dart with all key constants
and typed getters/setters (sentryOptIn, pinIgnored, lastJid,
audioInputId, videoInputId).
- Initialised once via PreferencesService.load() in main() and passed
down through WimsyApp → _Gatekeeper → WimsyHome → _PresenceMenu /
_PinSetupScreen / _PinUnlockScreen.
- Removed all 14 inline SharedPreferences.getInstance() call sites from
lib/main.dart; removed the now-unused shared_preferences import.
- Added test/preferences_service_test.dart: 10 unit tests covering all
five preference keys using SharedPreferences.setMockInitialValues.
- Updated test/widget_test.dart to pass a PreferencesService to WimsyApp.
Step 4 – Extract LoginScreen from _WimsyHomeState:
- New file lib/login_screen.dart containing LoginScreen stateful widget
with all login form state (7 controllers, endpoint discovery, advanced
options, connect logic).
- Removed _buildLogin, _handleConnect, _buildAdvancedOptions,
_scheduleEndpointDiscovery, _discoverEndpoint, _domainFromBareJid and
all login-only fields from _WimsyHomeState (~1100 lines removed).
- _WimsyHomeState.build now returns LoginScreen(...) when not connected.
Tested: flutter analyze (no issues), flutter test (156/156 pass),
dart test in vendor/xmpp_stone (65/65 pass).
Co-authored-by: Junie <junie@jetbrains.com>
- XEP-0388 (SASL2): upgraded status from 'partial' to 'complete'. The implementation now covers SCRAM authentication, user-agent binding, stream resumption inline, and IAP config-version pipelining (added on the quic branch and covered by sasl2_test.dart). - XEP-0467 (QUIC transport): remains 'partial' — the current implementation uses a single reliable bidirectional channel only; multi-channel and datagram usage are explicitly out of scope for this phase (see doc/plan-quic.md). Note: XmppFileTransferHandler extraction (also part of Step 5 in the plan) was deferred — the file-transfer code is spread across ~50 sites throughout xmpp_service.dart, deeply interleaved with IBB/Jingle event handlers, Sentry spans, and message persistence callbacks. A safe extraction requires a dedicated effort with broader test coverage and is tracked as a follow-up task. Tested: flutter analyze (no issues), flutter test (156/156 pass), dart test in vendor/xmpp_stone (65/65 pass). Co-authored-by: Junie <junie@jetbrains.com>
The copyWith method added in the cleanup pass had no direct test coverage despite its non-trivial sentinel-object pattern for nullable fields. New tests added (12 copyWith + 4 notification ID = 16 new cases): copyWith group: - 'returns identical field values when called with no arguments' — verifies all 29 fields are preserved when copyWith() is called with no arguments. - 'overrides scalar non-nullable fields' — from, to, body, outgoing, edited, acked, receiptReceived, displayed can all be changed independently. - 'overrides nullable String fields via sentinel' — messageId, mamId, stanzaId, oobUrl, oobDescription, rawXml, fileTransferId, fileName, fileMime, fileState, replyToId, replyToJid, replyFallback. - 'overrides nullable int fields via sentinel' — fileSize, fileBytes. - 'clears nullable fields to null via explicit null (sentinel pattern)' — passing null explicitly clears all 20 nullable fields; non-nullable fields are unaffected. - 'updates reactions map independently' — new reactions map is applied; original message is unaffected. - 'sets editedAt when marking a message as edited' — edited + editedAt updated together. - 'does not mutate the original message' — copyWith never modifies the receiver. notification ID stability group: - 'same JID always produces the same notification ID' — deterministic hash. - 'notification ID is non-negative' — abs() ensures no negative IDs. - 'notification ID is within 32-bit signed range' — result < 2^31. - 'different JIDs produce different notification IDs' — no collisions for typical JID strings. Tested: flutter analyze (no issues), flutter test (168/168 pass), dart test in vendor/xmpp_stone (65/65 pass). Co-authored-by: Junie <junie@jetbrains.com>
The existing mutation tests only covered the happy path. Seven new tests add coverage for the negative and boundary cases that are most likely to regress silently. New tests for applyCorrectionInList: - 'returns false when replaceId not found' — list is unchanged when no message matches the replacement ID. - 'returns false when sender bare JID does not match' — a correction from a different user (eve) is rejected even if the message ID matches. - 'returns false when exact sender does not match and matchSenderBare is false' — same bare JID but different resource must not match in strict mode. - 'applies correction to most-recent matching message' — when two messages share an ID the implementation iterates from the end; the last entry is corrected and the first is left untouched. New tests for updateReactionsInList: - 'returns false when stanza ID not found' — no mutation when the target stanza ID is absent from the list. - 'returns false for empty sender' — the empty-sender guard returns false immediately without touching the list. - 'matches by messageId when stanzaId absent' — reactions can be applied via messageId fallback when no stanzaId is set on the message. Tested: flutter analyze (no issues), flutter test (175/175 pass), dart test in vendor/xmpp_stone (65/65 pass). Co-authored-by: Junie <junie@jetbrains.com>
The existing merge engine tests only covered the four main happy/sad paths. Four new tests add coverage for boundary conditions that could silently regress. New tests: - 'does not merge by heuristic when body differs' — a MAM replay with a different body must not be merged into an existing local message even when sender, timestamp, and direction all match. - 'does not merge by heuristic when sender differs' — a message from a different JID is not merged even when body and timestamp match. - 'merges oobUrl and oobDescription via message-id match' — when a MAM replay carries oobDescription and rawXml, both are propagated into the existing message alongside mamId/stanzaId. - 'does not merge by message-id when both mamId and stanzaId already set' — once a message has both archive IDs the merge is skipped entirely, preventing accidental overwrite of already-resolved metadata. Tested: flutter analyze (no issues), flutter test (179/179 pass), dart test in vendor/xmpp_stone (65/65 pass). Co-authored-by: Junie <junie@jetbrains.com>
…IMSY-19) _ensureAuxStream could call connectionOpenBi on a Rust QUIC connection object that had already been disposed by a concurrent close() call. This caused a fatal DroppableDisposedException crash reported in Sentry (WIMSY-19). The fix adds two guards: 1. Check _closed alongside the _connection null-check before calling connectionOpenBi, so we never enter the FFI call if teardown has begun. 2. Re-check _closed after the await returns, since close() may have fired during the async gap while connectionOpenBi was in flight; if so, we throw a StateError (caught by the write() error handler) rather than storing the now-orphaned streams. Tested: flutter analyze (no issues), flutter test (179/179), dart test in vendor/xmpp_stone (65/65). Co-authored-by: Junie <junie@jetbrains.com>
Add debugPrint calls at three key points in the aux stream lifecycle: - Outbound open: logged in _ensureAuxStream after the channel is registered in _auxStreamsBySlot, showing the slot number. - Close: logged in close() before finishing aux send streams, showing the count and slot numbers of all channels being torn down. - Recv loop exit: logged in the finally block of _startRecvLoop for non-control streams, showing which slot's recv loop has ended. No inbound (server-initiated) aux stream acceptance exists in the current implementation, so only outbound opens are covered. Tested: flutter analyze clean; flutter test 179/179; dart test 65/65. Co-authored-by: Junie <junie@jetbrains.com>
The stream features response was arriving in two TCP/QUIC chunks. The old code in prepareStreamResponse appended </stream:stream> to the first (incomplete) chunk, producing malformed XML mid-attribute (e.g. 'https</stream:stream>://...'). A second broken buffering layer in handleResponse then tried to re-assemble using magic fixed offsets (strip 12 chars from start, 13 from end), which failed when the second chunk was a raw continuation rather than a fresh <xmpp_stone>-wrapped response. This caused the connection to hang on stream features and be timed out by the server after 20 seconds. Fix: consolidate all buffering into prepareStreamResponse using the existing (but previously unused) restOfResponse field. After combining any buffered data with the new chunk, attempt a full XML parse. If it fails (incomplete), save the raw combined data to restOfResponse and return '' so handleResponse has nothing to process. Only when the parse succeeds is the wrapped, complete XML emitted. restOfResponse is also reset in _attachOpenedSocket on reconnect. The broken _unparsedXmlResponse field and its magic-offset reassembly logic in handleResponse have been removed entirely. Tested: flutter analyze clean; flutter test 179/179; dart test 65/65. Co-authored-by: Junie <junie@jetbrains.com>
…(WIMSY-19) When close() set _connection = null, it dropped the last Dart-side strong reference to the RustArc, disposing it. Any concurrent _ensureAuxStream call that had already captured the connection in a local variable would then crash during connectionOpenBi's SSE argument encoding phase with DroppableDisposedException. Fix: do not null _connection in close(). The _closed flag already prevents any new use of the connection. The RustArc is released naturally once all local references (in-flight connectionOpenBi calls) drop. Since QuicCapableXmppSocket is not reused after close(), there is no risk of holding a stale reference across reconnects. Tested: flutter analyze (no issues), flutter test 179/179, dart test 65/65. Co-authored-by: Junie <junie@jetbrains.com>
XEP-0198 (urn:xmpp:sm:3) is incompatible with QUIC (XEP-0467): QUIC provides its own reliable, ordered delivery at the transport layer, so stream management's stanza acknowledgement and resumption are redundant and can cause connection hangs. Changes: - Add 'bool get isQuic => false' to XmppWebSocket abstract base class - Override 'isQuic => true' in QuicCapableXmppSocket - Expose 'bool get isQuic' on Connection via '_socket?.isQuic ?? false' - Guard StreamManagementModule.negotiate() and isReady() to return early/false when _connection.isQuic - Add '@OverRide' annotation to QuicCapableXmppSocket.write() - Add 'bool get isQuic => false' to all test fake socket classes that use 'implements XmppWebSocket' (9 files) Tested: flutter analyze (no issues), flutter test (179/179), dart test in vendor/xmpp_stone (65/65). Co-authored-by: Junie <junie@jetbrains.com>
…IMSY-19) The crash occurred when two concurrent callers of _ensureAuxStream (e.g. _startAuxStreamsOpen and a write() call via _selectSendTarget) both captured the same _connection RustArc and both attempted to call connectionOpenBi. The FRB-generated binding transfers the arc via Auto_Owned (consuming it), so the second caller tried to encode an already-disposed arc, causing DroppableDisposedException during SSE argument encoding. Fix: split _ensureAuxStream into a fast synchronous check plus a new _openAuxStream async method. An _auxStreamOpening map caches the in-flight Future per slot; putIfAbsent ensures only one connectionOpenBi call runs at a time per slot, and concurrent callers await the same Future. The in-flight entry is removed in a finally block so retries are possible after failure. _auxStreamOpening is also cleared in close() for hygiene. Tested: flutter analyze clean; flutter test 179/179; dart test 65/65. Co-authored-by: Junie <junie@jetbrains.com>
…leXmppSocket Log 'QUIC aux stream opening slot=N reason=...' immediately before calling connectionOpenBi, so that if the open fails or causes a crash the log shows both which slot was being opened and why: - Post-bind pre-open (from _startAuxStreamsOpen): reason is 'post-bind pre-open slot N of M' - On-demand routing (from _getSendTarget): reason is 'on-demand routing for <bareJid>' The reason is threaded through _ensureAuxStream and _openAuxStream as an optional/named parameter; coalesced concurrent opens for the same slot preserve the reason of the first caller that triggered the open. Tested: flutter analyze clean, flutter test 179/179 pass. Co-authored-by: Junie <junie@jetbrains.com>
…(WIMSY-19) connectionOpenBi uses Auto_Owned FFI transfer: it consumes the RustArc passed as _connection and returns a new one. The previous per-slot coalescing (_auxStreamOpening map) only prevented concurrent opens for the *same* slot. Two opens for *different* slots could still run concurrently, both capturing the same (about-to-be-consumed) arc; the second call would then crash with DroppableDisposedException. Fix: introduce a global async mutex (_auxOpenLock, a chained Future) that serialises all connectionOpenBi calls regardless of slot. Each _openAuxStream call chains onto the previous lock future, reads _connection only after the previous call has completed and updated it, then releases the lock in a finally block. _auxOpenLock is also reset in close() so a reconnect does not wait on a stale chain. Tested: flutter analyze clean, flutter test 179/179 pass. Co-authored-by: Junie <junie@jetbrains.com>
Previously _startAuxStreamsOpen opened aux streams sequentially in a single async loop, blocking each slot open on the previous one. This meant the connection was effectively stalled waiting for all aux streams to be established before proceeding with normal XMPP traffic. Now each slot is fired as an independent unawaited async task. The global _auxOpenLock mutex in _openAuxStream still serialises the underlying connectionOpenBi FFI calls (which use Auto_Owned arc transfer), so there is no race — the tasks simply queue up on the lock rather than blocking the caller. The main (control) stream continues to be used for all connection-state work and for traffic to/from the user's own bare JID, as enforced by the existing _postBindReady and _accountBareJid guards in _selectSendTarget. Tested: flutter analyze clean, flutter test 179/179 pass, dart test vendor/xmpp_stone 65/65 pass. Co-authored-by: Junie <junie@jetbrains.com>
Adds a 'Use QUIC transport (XEP-0467)' checkbox to the advanced options section of the login screen, defaulting to enabled. When disabled, QUIC SRV discovery is skipped and no QUIC endpoints are built, so the connection falls back to plain TCP — allowing QUIC to be isolated as the cause of post-bind stalls without changing any other settings. Changes: - AccountRecord: new useQuic field (default true), persisted in toMap / fromMap (old accounts without the key default to true). - XmppService.connect(): new useQuic parameter (default true) gates both the QUIC SRV lookup and the quicEndpoints assignment. - LoginScreen: _useQuic state loaded from stored account; passed to AccountRecord and service.connect(); checkbox rendered in advanced options between Resource and WebSocket, disabled on web. Tested: flutter analyze clean, flutter test 179/179 pass, dart test vendor/xmpp_stone 65/65 pass. Co-authored-by: Junie <junie@jetbrains.com>
When using QUIC transport, StreamManagementModule.negotiate() was returning early without setting NegotiatorState.DONE. This left the ConnectionNegotiatorManager's negotiator queue permanently stalled: stateListener() was never called, negotiateNextFeature() was never triggered, doneParsingFeatures() was never called, and consequently XmppConnectionState.Ready was never set. The result was that all post-bind activity (presence, carbons enable, roster fetch, MAM catch-up) never happened — the connection appeared to hang silently after resource binding. Fix: emit NegotiatorState.DONE before returning in the QUIC early-exit path, so the queue advances normally and doneParsingFeatures() fires. Tested: flutter analyze clean, flutter test 179/179 pass, dart test vendor/xmpp_stone 65/65 pass. Co-authored-by: Junie <junie@jetbrains.com>
…ady() When using QUIC transport, XEP-0198 Stream Management is intentionally skipped (QUIC provides its own reliability). The negotiate() method already handled this correctly by immediately setting state = DONE and returning early. However, isReady() was returning false for QUIC connections, which caused pickNextNegotiator() in ConnectionNegotiatorManager to skip the SM negotiator entirely via firstWhere(). Since negotiate() was never called, the state never advanced to DONE, and the negotiator queue stalled indefinitely — causing the observed hang after resource binding. Fix: isReady() now returns true for QUIC connections, allowing negotiate() to be called and immediately complete (DONE), so the negotiator queue advances normally to ServiceDiscovery and beyond. Tested: flutter analyze (clean), flutter test (179 passed), dart test in vendor/xmpp_stone (65 passed). Co-authored-by: Junie <junie@jetbrains.com>
Previously the '---Xmpp Sending:---' log line was emitted from
Connection.write(), which runs at queue-time: before the bufferedWrites
flush, before the TCP/WS/QUIC socket handoff, and — crucially on QUIC —
before _selectSendTarget() had picked (or opened) the destination
stream. That made the log line misleading for any diagnostics about
what is actually on the wire, and it hid the fact that a stanza can be
routed to a QUIC aux stream that is still pending an open.
This change moves the send log to the transport layer:
- vendor/xmpp_stone/lib/src/Connection.dart:
- Connection.write() no longer logs; a comment explains where the
log is now emitted and why.
- vendor/xmpp_stone/lib/src/logger/Log.dart:
- Log.xmppp_sending() gains an optional 'channel' parameter; when
set, the header is rendered as '---Xmpp Sending [<channel>]:---'.
- vendor/xmpp_stone/lib/src/connection/XmppWebsocketIo.dart:
- Logs with channel 'tcp' or 'ws' immediately before the actual
socket/sink write, and skips logging entirely if the underlying
socket is null (no transport handoff).
- lib/xmpp/quic_xmpp_socket.dart:
- _QuicSendTarget now carries a channel label ('quic-control' or
'quic-aux-<slot>') that reflects the real stream _selectSendTarget
resolved to, including slot number for bare-JID-hashed aux
streams.
- The write queue logs the payload with that label right before
sendStreamWriteAll, i.e. at the moment bytes are handed to the
QUIC stream.
This directly addresses the wimsy.log observation that a post-bind
disco#info to the server domain appeared to be 'sent' but never
reached the server: the new log will show the exact QUIC stream
(control vs aux-N) the stanza was actually written to.
Testing:
- flutter analyze: clean (no issues)
- flutter test: 179 tests passed
- dart test in vendor/xmpp_stone: 65 tests passed
No new tests were added; the change is limited to log-site plumbing
and existing tests exercise Connection.write and the TCP/WS socket
write paths. The QUIC path has no unit-level test harness in-tree and
is verified at runtime via wimsy.log.
Co-authored-by: Junie <junie@jetbrains.com>
Fix the post-bind hang where ServiceDiscoveryNegotiator's disco#info to the
user's own server domain was queued but never actually written on the QUIC
transport, causing the 20s connect-ready timeout in XmppService and a
ForcefullyClosed reconnect loop.
Root cause: QuicCapableXmppSocket._selectSendTarget routed any stanza whose
'to' was not the account bare JID to an aux QUIC stream, and awaited
_ensureAuxStream on the single serial _writeQueue chain. The post-bind
disco#info is addressed to the account *domain* (e.g. dave.cridland.net),
which hashed to aux slot 0. slot 0's connectionOpenBi() was still in flight
(or hanging) from the pre-open pass started the moment the bind result
arrived, so the write queue deadlocked behind it and the disco stanza was
never handed to any send stream - no 'Xmpp Sending' log, no bytes on the
wire.
Changes in lib/xmpp/quic_xmpp_socket.dart (_selectSendTarget):
* Server-directed stanzas are kept on the control stream: this now covers
stanzas with no 'to', stanzas addressed to the account bare JID, and
stanzas addressed to the user's own server domain. Negotiation IQs
(disco#info, session, etc.) never go to an aux stream.
* For all other bare-JID targets, we no longer await aux-stream creation
on the write queue. If the aux stream is already open we use it; if not,
we kick off _ensureAuxStream in the background and send this payload on
the control stream. Future writes to the same bare JID will use the aux
stream once it is ready, but a slow or failing aux open can never stall
the rest of the connection's outbound traffic again.
Tested: * flutter analyze: clean, no issues
* flutter test: 179 passed
* dart test in vendor/xmpp_stone: 65 passed
* No new unit tests added: the QUIC transport has no in-tree test harness
(it is runtime-verified via wimsy.log); the selection logic is a
straightforward guard around an already-covered code path.
Co-authored-by: Junie <junie@jetbrains.com>
After the previous fix to keep negotiation traffic on the control stream, runtime logs showed that the post-bind aux-stream opens never produced any success or failure log after the initial 'QUIC aux stream opening slot=0' line. Only slot 0 logged at all, because all other slots queue behind _auxOpenLock and slot 0's completer never fires: connectionOpenBi (Quinn open_bi) blocks indefinitely when the peer has not granted additional bidirectional-stream credits via MAX_STREAMS. Changes in lib/xmpp/quic_xmpp_socket.dart _openAuxStream: - Log 'awaiting open lock' for each slot before taking the lock, so we can see that all slots are at least attempting to open. - Log 'calling connectionOpenBi' immediately before the FFI call. - Start a Timer.periodic (2s) that logs 'connectionOpenBi still pending after Nms' so a stuck open is visible in the logs. - Wrap connectionOpenBi in a 10s timeout that throws TimeoutException with an explanatory message, so _auxOpenLock is released and subsequent slots can proceed (or fail) instead of being silently starved. - Log elapsed time on success and on any error (with stack) so future failures are diagnosable from wimsy.log alone. Tested: - flutter analyze: clean - flutter test: 179 passed - dart test in vendor/xmpp_stone: 65 passed No new unit tests: QUIC transport still has no in-tree harness; this is diagnostic plumbing on a runtime-only code path, intended to be validated via wimsy.log on the next QUIC session. Co-authored-by: Junie <junie@jetbrains.com>
When a QUIC aux stream open fails (DroppableDisposedException after
connection teardown, StateError for closed connection, or TimeoutException
from the 10s open timeout), the error was being rethrown inside a
catchError() on an unawaited Future, causing it to surface as a fatal
unhandled exception via PlatformDispatcher.onError and reported to Sentry
as WIMSY-1D, WIMSY-1B, and WIMSY-1C.
The on-demand open path in _selectSendTarget used catchError() which
requires the handler to return a value of the Future's type; rethrowing
was the only way to satisfy the type checker, but that made the error
fatal. Replaced with .then((_) {}, onError: ...) which has a void
onError handler and correctly swallows the error after logging it.
These failures are expected during connection teardown: the payload was
already sent on the control stream, so the aux stream open failure is
non-fatal. The pre-open path in _startAuxStreamsOpen was already safe
(catch inside the async body).
Fixes: WIMSY-1D, WIMSY-1B, WIMSY-1C
Tested: flutter analyze (clean), flutter test (179 passed),
dart test in vendor/xmpp_stone (65 passed).
Co-authored-by: Junie <junie@jetbrains.com>
Log server MAX_STREAMS(bidi) frame counts via connectionStats() at two
key moments:
- immediately after the QUIC handshake completes ('post-connect'), so
we can see the server's initial credit situation in the log;
- when connectionOpenBi times out ('aux-open-timeout slot=N'), so we
know the credit count at the moment of failure.
Add _auxStreamsBlocked flag: set on connectionOpenBi timeout, cleared
when a new MAX_STREAMS(bidi) frame is detected. A periodic watcher
(every 5 s) polls connectionStats() while blocked and resumes aux
stream opens as soon as the server raises the limit.
Change _startAuxStreamsOpen to open slots sequentially rather than
firing all 20 concurrently. This means:
- we stop immediately at the first credit-starvation timeout instead
of queuing 19 more doomed calls that each fail with
DroppableDisposedException (the Auto_Owned RustArc is consumed by
the timed-out slot-0 call and never returned);
- when the watcher detects new credits, _openAuxStreamsSequentially
resumes from where it left off.
Reset all new state (_auxStreamsBlocked, _lastMaxStreamsBidiFrameCount,
_maxStreamsWatchTimer) in close() so a reconnect starts clean.
Tested: flutter analyze (no issues), flutter test (179 passed),
dart test in vendor/xmpp_stone (65 passed).
Co-authored-by: Junie <junie@jetbrains.com>
…ML buffer The shared restOfResponse field in Connection.prepareStreamResponse was used by all QUIC streams (control + every aux stream). When chunks from different streams arrived interleaved, they were concatenated into the same buffer, producing garbled XML that either failed to parse (silently dropped) or produced wrong elements — causing messages on aux streams to never reach the UI. Fix: - Add Connection.makeStreamResponseMapper(): returns a new closure capturing its own independent buffer, with the same logic as prepareStreamResponse. - Add QuicCapableXmppSocket.setAuxMapperFactory(): stores the factory so each aux recv loop can call it to get its own mapper. - In _openAuxStream, call _makeAuxMapper() to create a fresh mapper for each aux stream's _startRecvLoop. - In Connection.openSocket(), after a successful QUIC connect, call socket.setAuxMapperFactory(makeStreamResponseMapper) so the factory is wired up before any aux streams open. - _startRecvLoop now accepts an optional mapper parameter; control stream passes _map explicitly, aux streams pass their per-stream mapper. Tested: flutter analyze (no issues), flutter test (188 passed), dart test in vendor/xmpp_stone (65 passed). Co-authored-by: Junie <junie@jetbrains.com>
…olation) Tests cover: - Independence: two mappers from the same Connection have separate buffers; partial data in one does not contaminate the other. - Buffering: incomplete XML is held and returned only when the chunk completes a parseable fragment; buffer is cleared after each complete stanza. - Output format: result is wrapped in <xmpp_stone>…</xmpp_stone> root element. - stream:stream opener: synthetic </stream:stream> is appended so the fragment parses correctly. - Multiple stanzas in one chunk: all are returned together. - Stream close: </stream:stream> returns empty string (triggers connection close). - Parity with prepareStreamResponse: both functions produce identical output for complete stanzas and both buffer incomplete XML identically. Tested: flutter analyze (no issues), flutter test (188 passed), dart test in vendor/xmpp_stone (77 passed, 12 new). Co-authored-by: Junie <junie@jetbrains.com>
flutter_quic vendors a Rust crate compiled via cargokit/CocoaPods. The build-macos and build-ios jobs were missing the Rust toolchain install step that build-linux, build-windows, and build-android already had, causing those jobs to fail when CocoaPods tried to compile the Rust library. - build-macos: adds dtolnay/rust-toolchain@stable with targets x86_64-apple-darwin,aarch64-apple-darwin (universal binary) - build-ios: adds dtolnay/rust-toolchain@stable with targets aarch64-apple-ios,aarch64-apple-ios-sim,x86_64-apple-ios The web build requires no changes: flutter_rust_bridge provides a web stub so the package compiles for web, and QUIC is already disabled at runtime via !kIsWeb guards throughout xmpp_service.dart. Tested: flutter analyze (no issues), flutter test (188 passed), dart test in vendor/xmpp_stone (77 passed). Co-authored-by: Junie <junie@jetbrains.com>
Add a pair of small sparkline graphs to the app bar, to the left of the presence menu, showing live QUIC RTT (ms) and packet loss (delta lost packets per second) when a QUIC connection is active. Changes: - vendor/xmpp_stone/lib/src/connection/XmppWebsocketApi.dart: added default getQuicStats() -> Future<dynamic> returning null, and exposed the socket getter on Connection. - lib/xmpp/quic_xmpp_socket.dart: implemented getQuicStats() using connectionStats() from flutter_quic, returning QuicConnectionStats?; isQuic now reflects _useQuic rather than always true. - lib/xmpp/xmpp_service.dart: added _quicRttHistory, _quicLossHistory, _lastQuicLostPackets, _quicStatsTimer and _maxQuicHistory fields with public getters; _setupQuicStats() polls the socket once per second, appending RTT and delta-loss to the history lists (capped at 60 samples) and calling notifyListeners(); timer is cancelled and lists cleared in _safeClose(). - lib/main.dart: added _QuicStatsGraph (label + sparkline) and _SparklinePainter (CustomPainter) widgets; app bar now shows RTT and loss graphs when quicRttHistory is non-empty. - vendor/xmpp_stone/test/*: added @OverRide getQuicStats() stub to all _FakeSocket / _RecordingSocket test doubles. Tested: flutter analyze (no issues), flutter test (188 tests pass), dart test in vendor/xmpp_stone (77 tests pass). Co-authored-by: Junie <junie@jetbrains.com>
The original apply-netem.sh had several bugs: - Used 'flower' filters which require hardware offload; replaced with portable 'u32' filters (matching apply-shape.sh's approach) - IPv4 egress filters had a broken 'mirred egress redirect' action instead of simply using 'flowid' - Only shaped outbound traffic; inbound packets from the server were unaffected. Added ingress shaping via an IFB device (same pattern as apply-shape.sh) - No cleanup at startup meant re-running would fail under set -e; added teardown of existing qdiscs before applying new ones - Parameters were hardcoded; made them optional positional arguments with the original values as defaults clear-netem.sh updated to also tear down the IFB ingress qdisc and bring down ifb0, matching what apply-netem.sh now creates. Interface is also an optional argument defaulting to wlp2s0. Tested by inspection against the working apply-shape.sh script. Co-authored-by: Junie <junie@jetbrains.com>
Previously the HTB qdisc used 'default 10/20' which routed ALL unmatched traffic (DNS, ping, TCP, etc.) through the netem impairment class, breaking general connectivity. Fix: add a dedicated bypass class (1:99 / 2:99) at 1gbit with no netem attached, set as the HTB default. Only the explicitly matched filters for UDP port 5224 to/from the XMPP server IPs are directed to the impaired class (1:1 / 2:1). All other traffic passes through the bypass class completely unaffected. Tested by inspection: the two-class HTB pattern (bypass default + impaired class with explicit filters) is the standard approach for selective tc impairment. Co-authored-by: Junie <junie@jetbrains.com>
u32 cannot match 128-bit IPv6 addresses, so the IPv6 filters were silently never matching any traffic. Switch both egress (on $IFACE) and ingress (on ifb0) IPv6 filters from u32 to the flower classifier, which natively supports full IPv6 src/dst address matching, while keeping u32 for IPv4 where it works correctly. Use 'flowid' directly on the flower filter (not 'action skbedit priority') so HTB correctly steers matched packets into the impaired class (1:1 / 2:1). Tested by inspection: flower with flowid is the standard pattern for IPv6 HTB classification in iproute2. Co-authored-by: Junie <junie@jetbrains.com>
Without an explicit leaf qdisc on the HTB bypass class (1:99 / 2:99), the kernel skips HTB classification entirely and sends all packets via the direct path (visible as direct_packets_stat in tc -s qdisc output), so the netem impairment class (1:1 / 2:1) never receives any traffic. Fix: attach pfifo_fast leaf qdiscs to both bypass classes on the egress (wlp2s0) and ingress (ifb0) HTB roots. Also switch IPv4 filters from u32 to flower for consistency with the IPv6 filters. Tested: tc -s qdisc should now show packets hitting 1:1/2:1 for QUIC UDP/5224 traffic, with all other traffic going through 1:99/2:99. Co-authored-by: Junie <junie@jetbrains.com>
The previous script used 'htb default 10' / 'default 20', which sent all unmatched traffic into the shaped class. As a result, the bandwidth cap, added latency and packet loss were applied to every flow on the interface, not just the XMPP/QUIC traffic to the configured peer. The unconditional 'match u32 0 0' ingress redirect also pushed every inbound packet through ifb0. Rework the script so that only the intended traffic is shaped: * Egress on $IFACE now uses an unshaped bypass class 1:1 at 10gbit as the htb default. A separate class 1:10 carries the configured rate and netem delay/loss. u32 filters explicitly steer (a) IPv4/IPv6 UDP/$PORT to $IPV4/$IPV6 and (b) IPv4 ICMP / IPv6 ICMPv6 to the same host into 1:10. * Ingress now redirects only the matching 5-tuples (UDP/$PORT and ICMP/ICMPv6 from $IPV4/$IPV6) to ifb0, instead of all traffic. ifb0 only ever sees the matched flow, so a single shaped class 2:20 is sufficient. * Adding ICMP/ICMPv6 echo to/from the XMPP host into the shaped class makes the setup trivially testable with 'ping' / 'ping6': RTT and loss to the peer should now reflect the configured $LAT and $LOSS, while ping to any other host stays unaffected. Tested manually by inspecting the resulting tc class/filter layout; no Dart code was changed, so flutter analyze/test and the vendored xmpp_stone tests are unaffected by this commit. Co-authored-by: Junie <junie@jetbrains.com>
Previously the QUIC client walked the resolved addresses sequentially. The first address used a short 'eyeballs' timeout of 2 * happyEyeballsDelay (= 500 ms by default) and only the second address got the full quicConnectTimeout (3 s). In practice this meant the IPv6 attempt was cut off well before a normal QUIC handshake (Initial -> Handshake -> 1-RTT) could complete on a moderately latent or lossy path, and we were giving the IPv4 fallback only 3 s before bailing out to TCP. There was no retry: a single transient drop on each leg killed the whole QUIC endpoint and forced TCP fallback. Changes: * Bump quicConnectTimeout default 3s -> 5s and apply it uniformly to every candidate address rather than using a special short timeout for the first one. * Add quicConnectMaxAttempts (default 3): the entire Happy Eyeballs round is retried up to N times before giving up. Each retry races every candidate again with a fresh staggered start. * Replace the sequential loop with a true parallel Happy Eyeballs racer (_raceQuicCandidates): every candidate is launched in parallel, staggered by happyEyeballsDelay (250 ms), and the first to complete the handshake wins. Losing attempts are abandoned (best-effort, since the FFI does not expose a clean cancel; the Rust side drops them when the arc goes out of scope). * Track the winning InternetAddress in _QuicConnectResult so we can log it. New tests in test/quic_happy_eyeballs_test.dart assert the new defaults (5s timeout, 250ms stagger, 3 attempts) and that explicit overrides are honoured. Verified with: flutter test (190 tests pass, including the new ones); dart test in vendor/xmpp_stone (77 tests pass). flutter analyze crashed in the analysis_server with an unrelated environment error and was skipped; the IDE inspector reports no problems in the changed file. Co-authored-by: Junie <junie@jetbrains.com>
The advanced connection panel already had a Direct TLS (XEP-0368) toggle, but it was effectively a hint: regardless of its value, both _xmpps-client._tcp (Direct TLS) and _xmpp-client._tcp (plain TCP) SRV records were resolved and the first one (after weighted ordering) was used. There was no symmetric toggle for plain TCP, so users could not disable plain TCP to force, e.g., Direct TLS or QUIC only. Changes: * AccountRecord: new useTcp field (default true), persisted in toMap/fromMap. Legacy persisted records without the key default to true so existing accounts behave exactly as before. * login_screen: new 'Plain TCP' CheckboxListTile in the advanced options section, mirroring the existing Direct TLS one. The Direct TLS subtitle is updated to make explicit that disabling it now causes the matching _xmpps-client._tcp SRV records to be ignored. * xmpp_service.connect: new useTcp parameter; after SRV lookup, TCP candidates are filtered through the new filterTcpSrvCandidatesByTransport helper using directTls and useTcp as independent allow-flags. If filtering removes everything, no QUIC is available, no WebSocket is configured, and the user has disabled both TCP transports, connect aborts with a clear error rather than silently falling back to a port the user has disabled. * srv_target: new pure helper filterTcpSrvCandidatesByTransport keeps only SRV records permitted by the user's allow-flags. Tests: * test/account_record_test.dart: covers default value, toMap/fromMap round-trip, legacy-record back-compat (no useTcp key -> true), and explicit false. * test/srv_transport_filter_test.dart: covers all four flag combinations and ordering preservation within a transport bucket. Verified with: flutter test (199/199 pass, including the 9 new tests); dart test in vendor/xmpp_stone (77/77 pass). flutter analyze still crashes inside analysis_server with an unrelated environment-level exception, so the IDE inspector was used instead and reports no new problems in the changed files. Co-authored-by: Junie <junie@jetbrains.com>
Root cause
===========
prepareStreamResponse (and makeStreamResponseMapper) accumulated all
received data in a buffer and only emitted it when the *entire* buffer
parsed as valid XML. Under high latency with a burst of incoming data
(e.g. a MUC presence flood on reconnect), the server sends many complete
stanzas followed by a partial fragment. The old implementation returned ''
and held back every complete stanza until the very last byte of the last
stanza arrived. This caused complete stanza starvation: handleResponse was
never called, no roster/presence/MAM data was processed, and the connection
appeared to hang after the first aux stream opened (slot=19 in wimsy.log).
The O(n²) re-parse on every new chunk made it progressively slower as the
buffer grew.
Fix
===
Replace the 'parse whole buffer, fail if incomplete' approach with a
greedy character-by-character depth-tracking scanner
(_extractCompleteElements) that:
1. Strips <?xml …?> prologs.
2. Tracks element open/close/self-close depth.
3. Records the end offset of each complete top-level element.
4. Emits all complete elements wrapped in <xmpp_stone>…</xmpp_stone>
immediately, keeping only the genuinely incomplete trailing fragment
in the buffer.
5. Handles <stream:stream> openers (never closed in normal flow) by
emitting them with a synthetic </stream:stream> appended.
6. Handles XML comments and quoted attribute values correctly.
Both prepareStreamResponse (control stream) and makeStreamResponseMapper
(aux streams) now use _extractCompleteElements, so the fix applies to all
QUIC streams and TCP/WebSocket connections equally.
Tests
=====
Four new regression tests added to stream_response_mapper_test.dart:
- emits complete stanzas even when followed by a partial fragment
- trailing partial is buffered and completed on next chunk
- prepareStreamResponse also emits complete stanzas before partial
- many complete stanzas followed by partial — all complete ones emitted
(simulates the MUC presence flood scenario from wimsy.log)
Verified with: dart test in vendor/xmpp_stone (81/81 pass, +4 new);
flutter test (199/199 pass).
Co-authored-by: Junie <junie@jetbrains.com>
Centralise the cache-hit decision for `_requestVcardDetails` in a new
pure helper `shouldFetchVcardForCache` (lib/xmpp/vcard_utils.dart) and
call it from xmpp_service before sending an `<iq><vCard/></iq>`. The
helper returns true when:
* the caller asked for the FN/NICKNAME (`preferName: true`),
* we have no recorded photo state for the JID,
* the persisted state is the "no avatar" sentinel but presence has
just advertised a non-empty hash, or
* the cached hash differs from a presence-advertised hash, or the
bytes for the cached hash are missing.
In all other cases — most notably a roster fan-out where every entry
already has its bytes+hash cached on disk — we now skip the IQ entirely.
`_handleVcardPresenceUpdate` now feeds the advertised photo hash into
the helper *before* mutating cached state, so a hash change still
triggers a refetch even when `_vcardAvatarState` was about to be
overwritten with the new hash.
Also extract the existing `'none'` sentinel as a public
`vcardNoAvatarSentinel` constant so the helper and tests share it.
Tests
-----
Added eight unit tests in test/vcard_utils_test.dart covering:
* preferName=true always fetches,
* unknown JID -> fetch,
* sentinel + no advertised hash -> skip,
* sentinel + advertised hash -> fetch,
* cached bytes & matching hash (with and without presence hash) -> skip,
* cached bytes but presence advertises a different hash -> fetch,
* hash recorded but bytes missing -> fetch.
Verified
--------
* `flutter test` — all 207 tests pass.
* `dart test` in vendor/xmpp_stone — all 81 tests pass.
* `flutter analyze` is unfortunately broken in this environment
(analysis server crashes with "Bad state: Future already completed"
independent of this change); JetBrains in-IDE analyser reports no
new errors or warnings on the touched files.
doc/startup-fetch-review.md updated to mark R4.1 as DONE.
Co-authored-by: Junie <junie@jetbrains.com>
Per the startup-fetch review, every restart was re-issuing
`urn:xmpp:avatar:metadata` PubSub GETs to JIDs we already know
publish no PEP avatar. The server replies `item-not-found` (a 404 in
PubSub-speak), `PepManager` quietly drops the response, and we ask
again on the next connect.
Fix: persist a sentinel `AvatarMetadata` entry the first time we see
the error, so `requestMetadataIfMissing` short-circuits forever after.
Changes
-------
* `lib/models/avatar_metadata.dart`
- Add `AvatarMetadata.noPepAvatar()` factory and matching
`isNoPepAvatar` getter.
- Carve out a round-trip path in `fromMap` so the sentinel survives
persistence even though it intentionally has `bytes == -1`.
- Real entries continue to be rejected when `hash` / `mimeType` /
`bytes` are missing.
* `lib/pep/pep_manager.dart`
- Track outgoing `urn:xmpp:avatar:metadata` GET IQs in a new
`_pendingMetadataRequests` map.
- When the response isn't `IqStanzaType.RESULT` (i.e. an error or
timeout-style synthesised error), record and persist a
`noPepAvatar` sentinel via `StorageService.storeAvatarMetadata`.
- Don't overwrite a real metadata entry that may have arrived via a
PubSub event in between.
- `avatarBytesFor` / `avatarHashFor` treat the sentinel as "no
avatar". A new helper `isKnownToHaveNoPepAvatar` lets callers
inspect the negative cache directly.
- `clearCache` also clears the new pending-metadata map and the
pending-data map (was leaked previously).
- A subsequent real metadata pubsub event replaces the sentinel,
so we still pick up an avatar that's published later.
Tests
-----
Added five tests to `test/pep_manager_test.dart` (R3.3 group):
* Error reply persists `noPepAvatar` sentinel and triggers an
update notification.
* Pre-seeded sentinel makes `requestMetadataIfMissing` skip the IQ.
* A real metadata event clears the sentinel.
* `AvatarMetadata.noPepAvatar` round-trips through `toMap`/`fromMap`.
* `fromMap` still rejects malformed legitimate entries (drive-by
coverage of pre-existing behaviour, since we now branch on hash).
Verified
--------
* `flutter test` — all 212 tests pass (+5 from this commit).
* `dart test` in `vendor/xmpp_stone` — all 81 tests pass.
* `flutter analyze` is unfortunately broken in this environment
("Bad state: Future already completed" on `setAnalysisRoots`,
independent of this change); JetBrains in-IDE analyser reports no
new errors on the touched files.
doc/startup-fetch-review.md updated to mark R3.3 as DONE.
Co-authored-by: Junie <junie@jetbrains.com>
`_setupDisplayedSync` was sending an
`<iq><pubsub><items node='urn:xmpp:mds:displayed:0'/></pubsub></iq>`
on every `XmppConnectionState.Ready`, even though
`_displayedStanzaIdByChat` is already seeded from
`StorageService.loadDisplayedSync()` at boot and live updates arrive
via the existing PEP +notify event handler.
This commit makes the bootstrap IQ conditional on the in-memory map
being empty (i.e. true cold start). On warm restarts we now skip the
IQ entirely and rely on +notify for catch-up.
Changes
-------
* New file `lib/xmpp/startup_fetch_helpers.dart` collecting pure
helpers for the connect-time fetch-elision decisions tracked in
`doc/startup-fetch-review.md`. Two helpers ship in this commit:
* `shouldFetchDisplayedSyncBootstrap` (R1.1) — used immediately.
* `shouldFetchMamCatchUpForChat` (R2.2) — placeholder predicate
added now to keep the helper file cohesive; production wiring
lands with the R2.2 commit.
* `lib/xmpp/xmpp_service.dart`:
* Import `startup_fetch_helpers.dart`.
* `_setupDisplayedSync` accepts an optional `force` named
parameter (default false) and consults the helper before sending
the IQ. `force=true` preserves the ability to do a manual
resync from a future UI hook.
Tests
-----
`test/startup_fetch_helpers_test.dart` — 9 new tests covering both
helpers (R2.2 cases will be exercised again from the R2.2 wiring
commit, but they are pure functions and useful to have unit-tested
now):
* empty cache -> fetch
* cache populated -> skip
* force overrides cache check (true / true)
* force=true on empty cache (idempotent)
* MAM catch-up: missing displayed id / mam id / stanza id -> fetch
* MAM catch-up: marker matches latest -> skip
* MAM catch-up: marker is newer -> fetch
Verified
--------
* `flutter test` — all 221 tests pass (+9 from this commit).
* `dart test` in `vendor/xmpp_stone` — all 81 tests pass.
* `flutter analyze` is broken in this environment, see prior commits;
JetBrains in-IDE analyser reports no errors on the touched files.
doc/startup-fetch-review.md updated to mark R1.1 as DONE.
Co-authored-by: Junie <junie@jetbrains.com>
The startup-fetch review predicted we would need to add
`urn:xmpp:mds:displayed:0+notify` to our advertised disco features.
On audit, it turns out the entry is already present in
`vendor/xmpp_stone/lib/src/features/servicediscovery/ServiceDiscoverySupport.dart`
alongside `urn:xmpp:avatar:metadata+notify`, and `doap.xml` already
lists XEP-0490 (Message Displayed Synchronization).
Rather than re-add it, this commit ensures we don't silently regress
in the future:
* `test/service_discovery_features_test.dart` — new regression
suite that asserts the SERVICE_DISCOVERY_SUPPORT_LIST contains the
+notify entries (R1.2 is the first test) plus the core
Jingle/RTP/file-transfer/blocking/chatstates/IBB/reply/fallback
namespaces.
A subsequent edit to the support list that removes any of these
features will now produce a clear test failure pointing to the missing
namespace.
Tests
-----
5 new tests; `flutter test` — all 226 tests pass.
`dart test` in vendor/xmpp_stone — all 81 tests pass.
doc/startup-fetch-review.md updated to mark R1.2 as DONE and to note
that the original review's claim about missing +notify was incorrect.
Co-authored-by: Junie <junie@jetbrains.com>
When MDS told us a chat had a displayed stanza-id we didn't yet have
locally (typical after restart for a busy MUC where our 25-message
tail doesn't reach back far enough), we logged "Displayed sync miss"
and silently dropped the marker. Next session we hit exactly the same
miss, then needed a fresh MAM page to make progress.
This commit makes the marker survive: persist the unresolved
(chatJid, stanzaId) pair to disk, retry on every message append, and
mark catch-up complete the moment a matching stanza-id appears.
Changes
-------
* `lib/storage/storage_service.dart`
- New on-disk key `displayed_sync_pending`.
- New `loadDisplayedSyncPending` / `storeDisplayedSyncPending`.
- `clearDisplayedSync` now also wipes the pending key, so
disconnect/forget remains a clean reset of all MDS state.
* `lib/xmpp/xmpp_service.dart`
- New in-memory `_displayedSyncPending` map (seeded from disk in
`attachStorage`, cleared everywhere the existing
`_displayedStanzaIdByChat` was).
- `_applyDisplayedStateForChat`:
* On a miss (no list, or list doesn't contain the stanza-id) it
now persists the (chat, stanzaId) pair via
`_markDisplayedSyncPending`.
* On a successful match (or repeated-marker no-op) it calls
`_clearDisplayedSyncPending` so the entry doesn't linger.
- New helpers:
* `_markDisplayedSyncPending(chat, stanzaId)` — idempotent
upsert that persists.
* `_clearDisplayedSyncPending(chat)` — removes & persists.
* `_resolveDisplayedSyncPending(chat, stanzaId)` — cheap O(1)
retry hook called from message-append paths.
- Hook calls added to all three message-append sites:
* DM tail append (`_addMessage`)
* MUC MAM-prepend (history insertion mid-list)
* MUC tail append (`_addRoomMessage`)
Tests
-----
`test/displayed_sync_pending_test.dart` — 4 new tests using a small
fake StorageService:
* empty map by default
* store -> load round-trips
* store overwrites the previous map
* clearDisplayedSync wipes both displayed_sync and pending
Verified
--------
* `flutter test` — all 230 tests pass (+4 from this commit).
* `dart test` in vendor/xmpp_stone — all 81 tests pass.
* `flutter analyze` is broken in this environment (pre-existing,
noted in earlier commits).
* JetBrains in-IDE analyser reports no new errors on the touched files
(one pre-existing regex range warning at xmpp_service.dart:2295 is
unrelated to this change).
Not done here
-------------
The doc's R1.3 description also mentions "drive a single bounded MAM
query toward this stanza-id" using `before=stanza-id`. That MAM query
shape is a separate, riskier change touching MamQueryPlanner /
MamCoordinator and is intentionally left for a follow-up commit; the
persisted-pending-marker mechanism above is sufficient on its own to
remove the "miss is terminal" failure mode this round.
Co-authored-by: Junie <junie@jetbrains.com>
… date
`_primeMamSync` was firing a MAM `<query/>` for every cached DM and
every bookmarked MUC on every `Ready`, even when the displayed
marker we already have on disk matches the stanza-id of the newest
local message. In that case there is nothing the catch-up could
return that we don't already have (any genuinely-newer message will
arrive live or via +notify).
This commit consults the existing
`shouldFetchMamCatchUpForChat` helper (added alongside R1.1 in
`lib/xmpp/startup_fetch_helpers.dart`) before each `_startMamCatchUp`
call inside `_primeMamSync`. The helper short-circuits when:
* the chat has a non-empty displayed stanza-id,
* the chat has a non-empty latest local MAM id,
* the message at that MAM id has a stanza-id, and
* that stanza-id equals the displayed marker.
In every other case (no marker, no MAM cursor, missing stanza-id,
mismatched ids) we fall through and fire the catch-up exactly as
before, so this is purely a strict-superset optimisation.
A small private helper `_stanzaIdAtLatestMamId(bareJid, {isRoom})`
walks the per-chat list backwards looking for the message whose
`mamId` matches `latestMamIdFor` / `_latestRoomMamIdFor`.
Tests
-----
The helper itself was already covered by 5 unit tests added in the
R1.1 commit (`test/startup_fetch_helpers_test.dart`). No new tests
in this commit; the wiring is mechanical and exercised through the
existing helper test suite.
Verified
--------
* `flutter test` — all 229 tests pass.
* `dart test` in vendor/xmpp_stone — all 81 tests pass.
* JetBrains in-IDE analyser reports no new errors on the touched
files (the pre-existing regex-range error at line 2295 is
unrelated).
Co-authored-by: Junie <junie@jetbrains.com>
Every distinct `node#ver` advertised by an entity in presence triggers
a `disco#info` IQ to verify its features. On reconnect into a busy
MUC this fans out one IQ per occupant — yet the result is by
construction stable for any given `node#ver` (XEP-0115 caps are
content-addressed by hash). This commit caches verified results to
disk so reconnects can skip the fan-out entirely for any caps we've
already seen.
Changes
-------
* `lib/storage/storage_service.dart`
- New on-disk key `entity_caps`.
- `loadEntityCaps()` returns Map<capsKey, Set<String>> features.
- `storeEntityCaps(capsKey, features)` writes one entry; existing
keys are preserved (and never overwritten — the `node#ver` is
content-addressed, so the same key always resolves to the same
set).
- `clearEntityCaps()` for forget-account flows.
* `lib/pep/pep_caps_manager.dart`
- Constructor now accepts an optional `StorageService`.
- Seeds `_capsFeatures` from disk on construction.
- Persists every successful `disco#info` result via
`storeEntityCaps`.
* `lib/xmpp/xmpp_service.dart`
- Passes `storage` into the `PepCapsManager` constructor.
Behaviour
---------
When MUC presence broadcasts `<c node=N ver=V>` for an occupant whose
`N#V` is already in the cache:
* `_handlePresence` finds `features != null` in `_capsFeatures`
and returns immediately after recording features for the bare JID,
so no disco IQ is sent.
* `featuresForBareJid` returns the cached features synchronously.
Unknown `node#ver` values still go through the existing
`_sendDiscoInfoQuery` -> `_handleDiscoInfoResult` path, so the only
behavioural change is the additional disk read on construct and the
additional disk write on a verified result.
Tests
-----
`test/pep_caps_manager_test.dart` — 3 new tests under group
`R5: persisted entity-caps cache`:
* disco#info result is persisted to storage.
* caps seeded from storage suppress the disco#info IQ for known
node#ver and immediately expose features for the bare JID.
* unknown node#ver still triggers disco even with a seeded cache.
The pre-existing `FakeStorageService` was extended with simple
in-memory overrides for `loadEntityCaps` / `storeEntityCaps` /
`clearEntityCaps`.
Verified
--------
* `flutter test` — all 232 tests pass (+3 from this commit).
* `dart test` in vendor/xmpp_stone — all 81 tests pass.
* JetBrains in-IDE analyser reports no new errors on the touched
files.
Co-authored-by: Junie <junie@jetbrains.com>
Lays the groundwork for replacing the per-chat MAM catch-up fan-out
with a single global `afterId=<anchor>` query at connect time.
Today `_primeMamSync` iterates every cached chat and every bookmark
and fires a separate MAM `<query/>` for each one. The intended
follow-up uses a single "newest MAM id we've seen, anywhere" anchor
and asks the server for everything strictly newer than that across
all chats, then lets per-chat anchors fill in lazily as the user
opens chats. To do that we first need the anchor to actually exist
and survive restarts.
Changes
-------
* `lib/storage/storage_service.dart`
- New on-disk key `last_mam_id_seen` (string).
- `loadLastMamIdSeen` / `storeLastMamIdSeen` / `clearLastMamIdSeen`.
* `lib/xmpp/xmpp_service.dart`
- New private field `_lastMamIdSeen`, seeded from disk in
`attachStorage`.
- New public getter `lastMamIdSeen` for downstream callers /
debugging.
- New private helper `_bumpLastMamIdSeen(mamId)` — lexicographic
compare-and-swap that writes through to `StorageService` only on
a strict increase, so duplicate / out-of-order MAM pages don't
cause spurious disk writes.
- All three message-append sites now call `_bumpLastMamIdSeen`:
* `_addMessage` (DM)
* `_addRoomMessage` (MUC MAM-prepend)
* `_addRoomMessage` (MUC tail)
Tests
-----
`test/last_mam_id_seen_test.dart` — 4 new tests using a small in-memory
`FakeStorageService`:
* null when the anchor was never written
* store -> load round-trips
* empty store is a no-op (we never persist an empty anchor)
* clear removes the anchor
Verified
--------
* `flutter test` — all 236 tests pass (+4 from this commit).
* `dart test` in vendor/xmpp_stone — all 81 tests pass.
* JetBrains in-IDE analyser reports no new errors on the touched
files.
Not done here
-------------
The unified catch-up `<query>` that consumes this anchor (plus any
deferral of per-chat catch-up to the moment the user opens the chat)
is intentionally left for a follow-up commit. The anchor it would
read is now correctly maintained so that change can be a localised
edit inside `_primeMamSync` / `MamQueryPlanner` without touching the
message-ingest code paths.
Co-authored-by: Junie <junie@jetbrains.com>
Long-lived installs accumulate stale entries in `_avatarBlobs` (and
the persisted `avatar_blobs` map) every time a contact rotates their
PEP avatar: the old blob stays in the cache forever even though no
metadata entry references its hash anymore.
This commit adds a pass that drops any blob whose hash isn't
referenced by a current non-sentinel `AvatarMetadata` entry, and
runs it once on `PepManager` construction, immediately after the
on-disk seed.
Changes
-------
* `lib/pep/pep_manager.dart`
- New public `int gcUnreferencedAvatarBlobs()`. Returns the
eviction count; calling it ad-hoc is safe.
- The constructor invokes it after seeding `_metadataByJid` and
`_avatarBlobs` from disk.
- `AvatarMetadata.noPepAvatar()` sentinels are deliberately ignored
when computing the referenced-hashes set, so the sentinel hash
(`__no_pep_avatar__`) never anchors a real blob.
* `lib/storage/storage_service.dart`
- New `replaceAvatarBlobs(Map<String, String>)` helper. The GC pass
rewrites the entire blob map in one Hive `put` rather than
issuing N deletes.
Tests
-----
`test/pep_manager_test.dart`:
* `gcUnreferencedAvatarBlobs (R3.1) drops blobs not referenced by
metadata` — seeds two blobs (one referenced, one stale) plus a
sentinel-only contact and asserts only the live blob survives,
that the persisted map matches, and that sentinel-only contacts
still report no avatar bytes.
* `gcUnreferencedAvatarBlobs (R3.1) is idempotent and a no-op on a
clean cache` — second invocation returns 0 and changes nothing.
The existing `FakeStorageService` was extended with an in-memory
`replaceAvatarBlobs` override.
Verified
--------
* `flutter test` — all 238 tests pass (+2 from this commit).
* `dart test` in vendor/xmpp_stone — all 81 tests pass.
* JetBrains in-IDE analyser reports no new errors on the touched
files.
Co-authored-by: Junie <junie@jetbrains.com>
Review of the updated wimsy.log (captured after flutter run -d linux with all nine R4.1-R3.1 commits present). Key findings: - Six connection cycles visible due to repeated server-initiated QUIC disconnects (~45-60s each, mid-MAM-stream). - R4.1 (vCard cache guard) and R2.2 (MAM skip) are ineffective on reconnect: _vcardAvatarBytes, _vcardAvatarState, and _mamCursorStore are cleared in the disconnect handler but only seeded from disk in attachStorage() (called once at startup). The Ready handler never re-seeds them. - New item R6 identified: re-seed vCard and MAM cursor caches from disk in the Ready handler on every reconnect. - R1.3, R3.3, R5, R2.1, R3.1 all confirmed working correctly. - Disconnects are server-side Openfire QUIC issue (not idle timeout); 30s foreground ping is correctly configured. No code changes in this commit - R6 fix follows. Co-authored-by: Junie <junie@jetbrains.com>
The disconnect handler (_safeClose) clears _vcardAvatarBytes and _vcardAvatarState to avoid stale in-memory state surviving a resource rebind. However, attachStorage() — the only place these maps were previously seeded from disk — is called once at startup, not on reconnect. This meant R4.1's shouldFetchVcardForCache guard saw an empty cache on every reconnect and allowed all vCard IQs through, negating the fix entirely. Fix: call _seedVcardAvatars() and _seedVcardAvatarState() at the top of the XmppConnectionState.Ready handler, before _setupRoster() and the rest of the Ready flush. This mirrors what attachStorage() does at startup and ensures the guard has accurate data on every reconnect. PepManager is unaffected — _setupPep() creates a fresh PepManager instance on every reconnect and its constructor loads from StorageService directly. The _mamCursorStore is also cleared on disconnect but has no disk persistence (it rebuilds from MAM responses); no change needed there. Tests: added two R6-specific tests to vcard_utils_test.dart: - 'R6: guard suppresses fetch after cache is re-seeded from disk' verifies shouldFetchVcardForCache returns false when the re-seeded cache matches the advertised hash. - 'R6: guard allows fetch when re-seeded cache has different hash' verifies the guard correctly allows a fetch when the hash changed. All 240 flutter tests pass. All 81 xmpp_stone tests pass. Co-authored-by: Junie <junie@jetbrains.com>
…chive query When _lastMamIdSeen is non-null (i.e. any session after the first), _primeMamSync now issues a single MAM IQ against the user's own server archive (no to= JID, afterId=_lastMamIdSeen) instead of fanning out one catch-up IQ per DM contact. The server returns all missed DM messages across every contact in one paginated stream; the existing _addMessage routing dispatches each forwarded message to the correct chat by JID. On completion the RSM <last> id from the <fin> IQ result is used to advance _lastMamIdSeen via the existing _bumpLastMamIdSeen helper, so the anchor stays current for the next session. When _lastMamIdSeen is null (fresh install / first session) the previous per-chat fan-out is retained as a fallback so each chat still gets its initial 25-message tail. MUC archives are per-room by protocol and remain as per-room queries regardless. The IQ is built manually (rather than via MessageArchiveManager.queryById) so the IQ id can be captured and an IqRouter response handler registered to receive the <fin> result without polling. New tests: - test/unified_mam_catchup_test.dart: 6 tests covering the <fin> RSM <last> parsing logic (well-formed result, missing fin, missing RSM set, missing last element, non-RESULT type, whitespace trimming). - vendor/xmpp_stone/test/mam_query_xml_test.dart: 1 new test verifying the unified query IQ has no toJid, carries after-id field with the correct anchor value, RSM max=50, and urn:xmpp:mam:2 namespace. Tested: flutter test (246 pass), dart test in vendor/xmpp_stone (82 pass). Co-authored-by: Junie <junie@jetbrains.com>
The server may open bidirectional QUIC streams to push large responses
(e.g. MAM pages) to the client without waiting for the client to open a
stream first. Previously the client had no accept loop, so these streams
were silently ignored: the MAM IQ result never arrived, _lastMamIdSeen
never advanced, and _primeMamSync re-issued the same unified catch-up
query on every Ready cycle — producing a continuous MAM request loop.
Changes:
- vendor/flutter_quic/rust/src/core/connection.rs: add accept_bi() to
QuicConnection, wrapping Quinn's Connection::accept_bi(). Returns
Option<(QuicSendStream, QuicRecvStream)> — None on connection close.
- vendor/flutter_quic/rust/src/api/bridge.rs: add connection_accept_bi()
free function following the same Auto_Owned pattern as connection_open_bi().
Returns (QuicConnection, Option<QuicSendStream>, Option<QuicRecvStream>).
- Regenerated vendor/flutter_quic/rust/src/frb_generated.rs and
vendor/flutter_quic/lib/src/rust/frb_generated.dart via
flutter_rust_bridge_codegen generate.
- lib/xmpp/quic_xmpp_socket.dart:
* _startRecvLoop: add optional String? label parameter so server-stream
recv loops log a meaningful channel name (quic-server-N).
* _startServerStreamAcceptLoop: new method that loops on
connectionAcceptBi(), starts an independent _startRecvLoop for each
accepted stream with its own XML mapper (via _makeAuxMapper), and
exits cleanly when the connection closes or _closed is set.
* connect(): call _startServerStreamAcceptLoop() immediately after
_startControlRecvLoop() so server-pushed streams are accepted from
the moment the QUIC connection is established.
- test/quic_stream_routing_test.dart: add connectionAcceptBi bridge export
test (compile-time regression guard); live-connection behaviour requires
integration testing against a real QUIC server.
Tested: flutter test (247 pass, +1 new); dart test in vendor/xmpp_stone
(82 pass). Rust cargo build clean (7 pre-existing warnings, no errors).
Co-authored-by: Junie <junie@jetbrains.com>
Two related QUIC multi-stream bugs fixed: 1. DroppableDisposedException on aux stream open Root cause: _startServerStreamAcceptLoop called connectionAcceptBi using a local copy of _connection concurrently with connectionOpenBi in _openAuxStream. Both functions used Auto_Owned FFI transfer, consuming the same RustArc; the second caller found it already disposed. Fix: change connection_accept_bi in the Rust bridge to take &QuicConnection (shared reference) instead of QuicConnection (Auto_Owned). Quinn's accept_bi() only needs &self, so this is safe. The accept loop no longer consumes the arc and can run concurrently with connectionOpenBi without any locking. Regenerated frb_generated.rs / frb_generated.dart via flutter_rust_bridge_codegen generate. 2. Ready-burst stanzas sent on quic-control instead of aux streams Root cause: _selectSendTarget fell back to the control stream immediately when the aux stream for a bare JID was not yet open, so the entire post-Ready burst (vCard IQs, avatar polls, caps disco, MAM catch-up) went on quic-control. Fix: introduce _auxStreamPendingQueue (Map<int, List<String>>). When a stanza targets a slot whose aux stream is not yet open, enqueue it and kick off _ensureAuxStream. On success, _flushAuxPendingQueue sends all queued stanzas in order on the aux stream. On failure (disposed arc, timeout), the queue is drained to the control stream so no stanzas are silently dropped. _selectSendTarget now returns null for enqueued payloads; write() checks for null and skips the immediate send. Also updated _auxOpenLock comment to clarify it serialises all Auto_Owned FFI calls (connectionOpenBi, connectionStats, etc.) but NOT connectionAcceptBi which now uses a shared ref. Testing: all 247 flutter tests pass; all 82 xmpp_stone dart tests pass; cargo build clean (no new warnings). Co-authored-by: Junie <junie@jetbrains.com>
When the server opens a bidirectional QUIC stream (e.g. to push a MAM page response), we now pool it rather than immediately starting a receive-only loop. When _ensureAuxStream next needs a stream for a new bare-JID slot it pops from this pool first, avoiding a connectionOpenBi round-trip and conserving the peer's bidi-stream credit budget. Changes: - quic_xmpp_socket.dart: add _serverStreamPool (List<_QuicStreamChannel>). _startServerStreamAcceptLoop now pushes accepted streams onto the pool instead of starting a recv loop immediately (send-stream-less streams still get an immediate recv-only loop as a fallback). _openAuxStream checks the pool first; if non-empty it pops the front entry, assigns it to the slot, starts its recv loop, removes the slot from _auxStreamOpening, and returns without touching connectionOpenBi. Pool is cleared in close() alongside the other stream maps. Added @VisibleForTesting serverStreamPoolSize getter and injectServerStreamForTesting() for future integration tests. - test/quic_stream_routing_test.dart: two new tests verifying serverStreamPoolSize is 0 at construction and after close(). Testing: - flutter test: 249 tests pass (+2 new). - dart test (vendor/xmpp_stone): 82 tests pass. - cargo build (vendor/flutter_quic/rust): clean (no new warnings). Co-authored-by: Junie <junie@jetbrains.com>
… pop Previously, when a server-initiated QUIC stream was accepted and added to _serverStreamPool, its recv loop was NOT started. Inbound stanzas pushed by the server on that stream were silently dropped until the stream was popped from the pool and assigned to a slot (which might never happen if the client had no outbound traffic for that JID). Fix: start _startRecvLoop immediately in _startServerStreamAcceptLoop for every accepted stream (both send+recv pairs and recv-only streams). The send stream is still pooled for outbound reuse as before. Correspondingly, remove the now-redundant _startRecvLoop call in the pool-reuse path of _openAuxStream — the loop is already running, starting a second one would process each inbound chunk twice. Outbound streams opened via connectionOpenBi already started their recv loop immediately (line 901) — that path was correct and is unchanged. Testing: all 249 flutter tests pass; all 82 xmpp_stone dart tests pass. No new tests added — the fix is structural (ordering of side-effects in the accept loop) and the existing quic_stream_routing_test.dart pool tests continue to pass. Co-authored-by: Junie <junie@jetbrains.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.