Skip to content

Add FetchKeyCache.invalidate for swappable database wrappers.#455

Open
no33jou wants to merge 1 commit intopointfreeco:mainfrom
no33jou:fix/fetchkey-cache-invalidate
Open

Add FetchKeyCache.invalidate for swappable database wrappers.#455
no33jou wants to merge 1 commit intopointfreeco:mainfrom
no33jou:fix/fetchkey-cache-invalidate

Conversation

@no33jou
Copy link
Copy Markdown

@no33jou no33jou commented Apr 29, 2026

Linked discussion: #454

Problem

A DatabaseWriter wrapper that keeps its own object identity stable across inner-connection swaps causes @FetchOne push to stop. FetchKeyID uses ObjectIdentifier(database); swift-sharing's persistent-reference cache hands back the reference whose ValueObservation is bound to the closed previous connection. @FetchAll only escapes by accident (different generic-cast path inside PersistentReferences).

Repro (5 sec, runs swift run Repro / swift run Repro -- --fix):
https://github.com/no33jou/fetchone-mutable-db-repro

Change

  • New file Sources/SQLiteData/FetchKeyCache.swiftpublic enum FetchKeyCache { public static func invalidate(database:) }. Internally an _FetchKeyRegistry (lock-isolated) keys reattach closures by ObjectIdentifier(database).
  • Sources/SQLiteData/Internal/FetchKey.swiftsubscribe(...) now registers a reattach closure on each subscribe; invalidate cancels the existing GRDB cancellable and starts a fresh one against the wrapper's current inner connection. The SharedSubscription's cancel closure unregisters from the registry.

invalidate does not emit a new value to existing observers. Callers refresh state via the standard .load(...) flow after invalidating; this is the same call shape they already use after a connection swap, and it keeps the reattach safely outside of any in-progress observer callback (see code comments for the deadlock scenario this avoids).

Additive: no public API surface change for callers that don't swap inner connections. No behavior change unless FetchKeyCache.invalidate(database:) is called.

Tests

Tests/SQLiteDataTests/FetchKeyCacheTests.swift adds two cases against a minimal SwappableWriter:

  1. pushDropsAfterInnerSwap_withoutInvalidate — without invalidate, writes to the new connection do not reach @FetchOne. Locks the regression in.
  2. pushRestoredAfterInvalidate — with invalidate, push resumes against the new connection.
swift test --filter FetchKeyCacheTests   # 2/2 pass
swift test                               # 236/236 pass (no regressions)

I also verified the tests catch the bug: temporarily reverting FetchKey.swift while keeping the test file and FetchKeyCache.swift makes test (2) fail with unread → 0 ≠ 1. Test (1) keeps passing — it's the regression sanity, not the fix verification.

Compatibility

  • Additive public enum FetchKeyCache.invalidate(database:).
  • Internal _FetchKeyRegistry is @unchecked Sendable guarded by LockIsolated. Process-wide lifetime, matching the swift-sharing cache it shadows.
  • No change to the existing FetchKey.subscribe return value or the public FetchKeyRequest protocol.

Open questions for review

  • Naming: FetchKeyCache.invalidate(database:) vs SQLiteData.invalidateFetchKeys(for:) vs something nested under FetchKey. Happy to follow your preference.
  • Whether to add a brief <doc:SwappableDatabase> article in Documentation.docc — can do as a follow-up if useful.

@no33jou no33jou force-pushed the fix/fetchkey-cache-invalidate branch from b143a2a to 181f781 Compare April 29, 2026 05:18
@mbrandonw
Copy link
Copy Markdown
Member

Hi @no33jou, it seems to me that you are only able to reproduce this "bug" by making a custom conformance to DatabaseReader and DatabaseWriter, which GRDB's docs specifically say is not valid to do.

Further, there are other ways to accomplish this "swappable" database idea that are not literally changing the database connection. You can attach/detach databases too. I would encourage you to explore that option as an alternative, or you will need to open a discussion on GRDB to get a better understanding of why custom conformances to DatabaseReader/DatabaseWriter are not allowed.

`FetchKeyID` keyed on `ObjectIdentifier(database)` goes stale when a
transparent wrapper swaps its inner pool while keeping its own object
identity. Add a `PersistentDatabaseIdentity` hook so wrappers can
forward identity to the current inner reader; plain `DatabasePool` /
`DatabaseQueue` keep the existing `ObjectIdentifier` fallback.
@no33jou no33jou force-pushed the fix/fetchkey-cache-invalidate branch from 181f781 to a805ee6 Compare April 30, 2026 09:36
@no33jou
Copy link
Copy Markdown
Author

no33jou commented Apr 30, 2026

Thanks for the response. One thing the contract argument doesn't explain: @FetchAll and @FetchOne diverge on the same wrapper, same swap, same write. Probe on _add(observation:) after mutable.database = poolB:

[_add] Reducer.Value=Result<[Item], Error>  poolPath=B   ← FetchAll re-subscribes
(no _add for FetchOne)                                   ← FetchOne reuses stale ref

Why they diverge — swift-sharing init/load asymmetry:

// SharedReader.init(wrappedValue:_:)         — unwraps .Default
self.init(wrappedValue: wrappedValue(), key.base)

// SharedReader.load(_:)                      — does not unwrap
projectedValue = SharedReader(
  reference: persistentReferences.value(forKey: key, ...)
)

// PersistentReferences.value(forKey:)        — matches by exact type
storage[key.id] as? Weak<Key>

Combined with sqlite-data's overload choice (.Default for collections, none for single-value), FetchAll lands in different cache buckets at init vs load → miss → reattach. FetchOne lands in the same bucket → hit → stale. FetchAll "working" is a side-effect, not a feature.

Fix (PR ready, ~30 lines + tests, 236 tests pass, no API change for DatabasePool / DatabaseQueue users):

public protocol PersistentDatabaseIdentity {
  var persistentIdentity: AnyHashable { get }
}

// FetchKeyID derives databaseID from this hook when available,
// else falls back to ObjectIdentifier(database).
// Wrappers forward to current inner reader → swap invalidates cache.

Decoupled from swift-sharing's asymmetry, so future symmetrization won't regress FetchAll.

Branch: https://github.com/no33jou/sqlite-data/tree/fix/fetchkey-id-from-inner-pool
Repro: standalone Swift package, three modes (bug / reset / protocol) demonstrating the divergence and the fix — happy to share on request.

(ATTACH/DETACH is a non-starter for per-user DB isolation — multi-week rearchitecture of StructuredQueries + region tracking. But that's secondary; the divergence reproduces regardless of swap implementation.)

@stephencelis
Copy link
Copy Markdown
Member

@no33jou Your branch still depends on subclassing DatabaseReader/DatabaseWriter, which Brandon mentioned above is not a GRDB-supported thing to do.

@rcarver
Copy link
Copy Markdown
Contributor

rcarver commented Apr 30, 2026

fwiw I'm using ATTACH to support on-disk file moves against the connected database (document-based app using sqlite in document package). It's a bit tricky but works great. Can package up some code if that's helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants