Skip to content

Commit d6c179c

Browse files
authored
feat: make D1 replicas opt-in as safer default and add support for Sessions API to mitigate stale data returned from replicas (#813)
This commit closes #783. It adds two new primitives into D1 `void/db` that utilize [D1 Session API](https://developers.cloudflare.com/d1/best-practices/read-replication/#use-sessions-api): - `getBookmark` - returns the current session [bookmark](https://developers.cloudflare.com/d1/reference/time-travel/#bookmarks) - `setSession` - starts a new session from a previous bookmark. Using these allow to solve the D1 Replicas problem in the scenario `read OLD -> write NEW -> read OLD` and ensure sequential consistency. It also makes replicas opt-in and adds a new config `database.readReplication` to enable/disable the feature: ```ts import { defineConfig } from 'void/config'; export default defineConfig({ database: { readReplication: true }, }); ``` --- See guides for more information and usage examples: - [See your writes in subsequent requests](https://void.cloud/guide/database/d1#cross-request-bookmarks) - [API requests with concurrent clients](https://void.cloud/guide/database/d1#api-requests-with-concurrent-clients) - [Supporting both transports](https://void.cloud/guide/database/d1#supporting-both-transports) TL;DR; To utilize Sessions API the user has to create a middleware: ```ts // middleware/01.database.ts import { getCookie, setCookie } from 'hono/cookie'; import { defineMiddleware } from 'void'; import { getBookmark, setSession } from 'void/db'; export default defineMiddleware(async (c, next) => { const incoming = c.req.header('X-DB-Bookmark'); setSession(!incoming ? 'first-unconstrained' : { bookmark: incoming }); await next(); const bookmark = getBookmark(); if (bookmark) { setCookie(c, 'db-bookmark', bookmark, { httpOnly: true, secure: new URL(c.req.url).protocol === 'https:', sameSite: 'Lax', path: '/', }); } c.header('Cache-Control', 'private, no-store'); }); ``` To make it work with client requests, instead of the cookie (or along with the cookie transport), the devs should use a header, e.g., `X-DB-Bookmark` and make requests through a wrapper that sets the bookmark.
1 parent 2ae2da7 commit d6c179c

44 files changed

Lines changed: 1952 additions & 193 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/guide/database/d1.md‎

Lines changed: 283 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,47 @@ await db.batch([
4141

4242
## Read replication
4343

44-
Void enables [D1 read replication](https://developers.cloudflare.com/d1/best-practices/read-replication/) when it provisions a database. To enable it on an existing database, run `void deploy --platform cloudflare --provision`. Queries through `db` from `void/db` use a D1 session for each Void request, so reads can run near the request and later queries in that request see at least the same data. Framework handlers without Void's request scope start each database operation on the primary. `createDb` starts a replica-capable session for the binding you pass it.
44+
::: warning Previously deployed databases
45+
Void **0.21.3** introduced automatic D1 read replication. If you deployed with that version or a later version before replication became opt-in, your database may already have replicas enabled. Replicas can serve stale data between requests.
4546

46-
Replicas can lag between requests. Authentication checks use the primary so a lagging replica can't accept a revoked session. For other reads that need the latest primary data, create a client inside the request:
47+
Unless your app intentionally uses replicas with the consistency handling described below, we recommend upgrading to a Void version that supports `database.readReplication`, setting it to `false` in `void.config.ts`, and running `void deploy`:
48+
49+
```ts
50+
import { defineConfig } from 'void/config';
51+
52+
export default defineConfig({
53+
database: { readReplication: false },
54+
});
55+
```
56+
57+
If you self-host the Void platform, upgrade it as well before deploying. Upgrading alone or omitting the option does **not** disable existing replication.
58+
:::
59+
60+
[D1 read replication](https://developers.cloudflare.com/d1/best-practices/read-replication/) is opt-in. Enable it in `void.config.ts`, then run `void deploy`:
61+
62+
```ts
63+
import { defineConfig } from 'void/config';
64+
65+
export default defineConfig({
66+
database: { readReplication: true },
67+
});
68+
```
69+
70+
This enables replication for new and existing application D1 databases on both direct Cloudflare and Void platform deployments. Set `readReplication: false` and deploy again to disable replication.
71+
72+
When you omit `readReplication`, deployment leaves the database's replication setting unchanged. New databases default to replication off. Removing the option from a database that already has replication enabled keeps it enabled.
73+
74+
Queries through `db` from `void/db` use a D1 session for each Void request. With replication enabled, reads can run near the request and later queries in that request see at least the same data. Framework handlers without Void's request scope start each database operation on the primary. `createDb` starts a replica-capable session for the binding you pass it.
75+
76+
::: warning
77+
Replicas can lag between requests. [Carry a bookmark](#cross-request-bookmarks) to preserve previously observed database state, or use a [primary-first session](#reading-from-the-primary) when you need the latest primary data.
78+
:::
79+
80+
For raw D1 calls or another query library, use `withSession()` to read from replicas. A bare D1 binding reads from the primary.
81+
82+
### Avoid stale reads with a primary-first session {#reading-from-the-primary}
83+
84+
Authentication checks use the primary so a lagging replica can't accept a revoked session. For other reads that need the latest primary data, create a client inside the request:
4785

4886
```ts
4987
import { users } from '@schema';
@@ -59,7 +97,249 @@ export const GET = defineHandler(async () => {
5997

6098
With `session: 'first-primary'`, the first query reads from the primary and later queries see at least the same data. The default, `'first-unconstrained'`, allows the first query to use a replica.
6199

62-
For raw D1 calls or another query library, use `withSession()` to read from replicas. A bare D1 binding reads from the primary.
100+
### See your writes in subsequent requests {#cross-request-bookmarks}
101+
102+
To make a read after a save see that save in a later request, carry a [D1 bookmark](https://developers.cloudflare.com/d1/best-practices/read-replication/#start-a-session-from-previous-context-bookmark) between requests. A bookmark allows D1 to use a replica that has caught up to at least the previously observed database state.
103+
104+
Keep saved bookmarks scoped to the database that issued them. Use separate cookie names or browser-storage keys for different databases and environments, and clear saved bookmarks when switching to a different database. Clearing a bookmark starts a new consistency chain; subsequent requests no longer carry the previous bookmark's guarantee. Treat bookmarks as opaque strings rather than parsing or constructing them.
105+
106+
::: warning
107+
Bookmarks constrain database queries, not cached responses. A cached response can predate your save without running a query. Disable ISR for participating pages (`export const revalidate = 0` in the page's `.server.ts` file) and bypass any existing response caches for these routes. Setting response headers alone does not bypass an ISR cache hit.
108+
:::
109+
110+
Use middleware to set the default `db` session before your handlers or Pages loaders run, and return its bookmark after their queries. For a browser app, a cookie carries it through redirects and navigation:
111+
112+
This example opts `/profile` and its subpaths into bookmark handling and `Cache-Control: private, no-store`, preventing browser and shared response-cache storage. Change the path to cover your participating reads, saves, and redirect destinations. Routes outside this scope keep their existing cache policies.
113+
114+
Local server-side requests through `void/client` share that session. If middleware runs again, repeating an already satisfied session requirement preserves the session and its database progress. For an independent primary-first read or a newer bookmark after using `db`, use `createDb` instead.
115+
116+
Await consistency-critical database work before returning from your handler or loader. Deferred or background queries that finish after middleware captures the response bookmark are not guaranteed to be reflected in it.
117+
118+
::: warning
119+
Concurrent responses can overwrite this cookie with an older bookmark. The [header-based recipe](#api-requests-with-concurrent-clients) retains the newest bookmark across concurrent API responses. Clearing or expiring bookmark storage starts a new bookmark chain.
120+
:::
121+
122+
::: note
123+
To use cookies and API headers together, see [Supporting both transports](#supporting-both-transports).
124+
:::
125+
126+
```ts
127+
// middleware/01.database.ts
128+
import { getCookie, setCookie } from 'hono/cookie';
129+
import { defineMiddleware } from 'void';
130+
import { getBookmark, setSession } from 'void/db';
131+
132+
export default defineMiddleware(async (c, next) => {
133+
const path = c.req.path;
134+
if (path !== '/profile' && !path.startsWith('/profile/')) {
135+
await next();
136+
return;
137+
}
138+
139+
const incoming = getCookie(c, 'db-bookmark');
140+
setSession(incoming === undefined ? 'first-unconstrained' : { bookmark: incoming });
141+
await next();
142+
143+
const bookmark = getBookmark();
144+
if (bookmark) {
145+
setCookie(c, 'db-bookmark', bookmark, {
146+
httpOnly: true,
147+
secure: new URL(c.req.url).protocol === 'https:',
148+
sameSite: 'Lax',
149+
path: '/',
150+
});
151+
}
152+
c.header('Cache-Control', 'private, no-store');
153+
});
154+
```
155+
156+
#### API requests with concurrent clients
157+
158+
For browser API requests, exchange an `X-DB-Bookmark` header and retain the newest bookmark. [D1 bookmarks are lexicographically sortable](https://developers.cloudflare.com/d1/reference/time-travel/#bookmarks), so an older response must not replace a newer bookmark from the same database.
159+
160+
For requests made through the browser fetch wrapper below, exchange bookmarks using this middleware. It opts only `/api/profile` and its subpaths into bookmark handling and uncached responses; adjust the path to match your participating API routes:
161+
162+
::: warning
163+
This recipe carries bookmarks only through the fetch wrapper below, not ordinary navigation, form submissions, or Pages requests. It does not share bookmarks with the cookie-based recipe.
164+
:::
165+
166+
::: note
167+
To use cookies and API headers together, see [Supporting both transports](#supporting-both-transports).
168+
:::
169+
170+
```ts
171+
// middleware/01.database.ts
172+
import { defineMiddleware } from 'void';
173+
import { getBookmark, setSession } from 'void/db';
174+
175+
export default defineMiddleware(async (c, next) => {
176+
const path = c.req.path;
177+
if (path !== '/api/profile' && !path.startsWith('/api/profile/')) {
178+
await next();
179+
return;
180+
}
181+
182+
const incoming = c.req.header('X-DB-Bookmark');
183+
setSession(incoming === undefined ? 'first-unconstrained' : { bookmark: incoming });
184+
await next();
185+
const bookmark = getBookmark();
186+
if (bookmark) {
187+
c.header('X-DB-Bookmark', bookmark);
188+
}
189+
c.header('Cache-Control', 'private, no-store');
190+
});
191+
```
192+
193+
On the browser, send the saved bookmark and atomically retain the newest returned value using [Web Locks](https://developer.mozilla.org/en-US/docs/Web/API/Web_Locks_API). This requires HTTPS (or localhost), browser storage, and Web Locks support. Use a scope unique to the database and signed-in account; same-origin tabs with the same scope share progress.
194+
195+
```ts
196+
// src/bookmark-fetch.ts
197+
export function createBookmarkFetch(scope: string) {
198+
const key = `d1-bookmark:${scope}`;
199+
200+
return async function bookmarkFetch(path: string, init: RequestInit = {}): Promise<Response> {
201+
const url = new URL(path, location.origin);
202+
if (url.origin !== location.origin || !url.pathname.startsWith('/api/')) {
203+
throw new Error('bookmarkFetch requires a same-origin /api/ URL.');
204+
}
205+
206+
const bookmark = localStorage.getItem(key);
207+
const headers = new Headers(init.headers);
208+
headers.delete('X-DB-Bookmark');
209+
if (bookmark) headers.set('X-DB-Bookmark', bookmark);
210+
211+
const response = await fetch(url, {
212+
...init,
213+
headers,
214+
credentials: 'same-origin',
215+
cache: 'no-store',
216+
redirect: 'error',
217+
});
218+
219+
const returned = response.headers.get('X-DB-Bookmark');
220+
if (returned) {
221+
await navigator.locks.request(key, () => {
222+
const current = localStorage.getItem(key);
223+
if (current === null || returned > current) {
224+
localStorage.setItem(key, returned);
225+
}
226+
});
227+
}
228+
229+
return response;
230+
};
231+
}
232+
```
233+
234+
Use the wrapper for each participating API request. For example, with an existing profile API:
235+
236+
```ts
237+
import { createBookmarkFetch } from './bookmark-fetch';
238+
239+
export async function saveAndReadProfile(accountId: string, name: string) {
240+
const api = createBookmarkFetch(`app-db:${accountId}`);
241+
const saved = await api('/api/profile', {
242+
method: 'POST',
243+
headers: { 'Content-Type': 'text/plain' },
244+
body: name,
245+
});
246+
if (!saved.ok) throw new Error(`Save failed: ${saved.status}`);
247+
248+
// Sends at least the completed save's bookmark.
249+
const response = await api('/api/profile');
250+
if (!response.ok) throw new Error(`Read failed: ${response.status}`);
251+
return response.json();
252+
}
253+
```
254+
255+
The wrapper saves the newest bookmark before resolving, so subsequent requests inherit its progress even if concurrent responses arrive out of order.
256+
257+
Use API endpoints that return data, not redirects.
258+
259+
#### Supporting both transports
260+
261+
Use this single middleware with the fetch wrapper above to support API requests alongside browser navigation and forms. It selects the newest incoming bookmark for the same database and returns it through both transports.
262+
263+
Only `/profile`, `/api/profile`, and their subpaths participate and receive `Cache-Control: private, no-store`. Adjust this list for your app, disable ISR on participating pages, and leave unrelated routes' cache policies unchanged.
264+
265+
::: warning
266+
The fetch wrapper retains the newest bookmark across concurrent responses, but cookies can still be overwritten out of order. For a guaranteed fresh read on a redirect's destination page, use a primary-first session.
267+
:::
268+
269+
```ts
270+
// middleware/01.database.ts
271+
import { getCookie, setCookie } from 'hono/cookie';
272+
import { defineMiddleware } from 'void';
273+
import { getBookmark, setSession } from 'void/db';
274+
275+
export default defineMiddleware(async (c, next) => {
276+
const path = c.req.path;
277+
const participates = ['/profile', '/api/profile'].some(
278+
(base) => path === base || path.startsWith(`${base}/`),
279+
);
280+
if (!participates) {
281+
await next();
282+
return;
283+
}
284+
285+
const incoming: string[] = [];
286+
287+
// Cookie input: navigation, forms, and same-origin fetch requests.
288+
// Remove for header-only use.
289+
const cookieBookmark = getCookie(c, 'db-bookmark');
290+
if (cookieBookmark) incoming.push(cookieBookmark);
291+
292+
// Header input: the fetch wrapper's saved progress.
293+
// Remove for cookie-only use.
294+
const headerBookmark = c.req.header('X-DB-Bookmark');
295+
if (headerBookmark) incoming.push(headerBookmark);
296+
297+
// Bookmarks from the same database sort oldest to newest.
298+
const newest = incoming.sort().at(-1);
299+
setSession(newest === undefined ? 'first-unconstrained' : { bookmark: newest });
300+
await next();
301+
302+
const bookmark = getBookmark();
303+
if (bookmark) {
304+
// Cookie output: progress for subsequent navigation and forms.
305+
// Remove for header-only use.
306+
setCookie(c, 'db-bookmark', bookmark, {
307+
httpOnly: true,
308+
secure: new URL(c.req.url).protocol === 'https:',
309+
sameSite: 'Lax',
310+
path: '/',
311+
});
312+
313+
// Header output: progress for the fetch wrapper.
314+
// Remove for cookie-only use.
315+
c.header('X-DB-Bookmark', bookmark);
316+
}
317+
318+
c.header('Cache-Control', 'private, no-store');
319+
});
320+
```
321+
322+
#### Using `createDb`
323+
324+
For another D1 binding or a framework-owned endpoint, such as an Astro API route, create a client with the incoming bookmark. After querying, return `getBookmark(client)` in the response header:
325+
326+
```ts
327+
import { users } from '@schema';
328+
import { createDb, getBookmark } from 'void/db';
329+
330+
export async function readUsers(request: Request, binding: D1Database) {
331+
const incoming = request.headers.get('X-DB-Bookmark');
332+
const client = createDb(binding, {
333+
session: incoming === null ? 'first-unconstrained' : { bookmark: incoming },
334+
});
335+
const result = await client.select().from(users).all();
336+
337+
const headers = new Headers({ 'Cache-Control': 'private, no-store' });
338+
const bookmark = getBookmark(client);
339+
if (bookmark) headers.set('X-DB-Bookmark', bookmark);
340+
return Response.json(result, { headers });
341+
}
342+
```
63343

64344
## Alternative Query Layers
65345

‎docs/reference/api.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ Import Void's app-facing APIs from these paths:
5858
| `void/schema-pg` | Re-exports [`drizzle-orm/pg-core`](https://orm.drizzle.team/docs/column-types/pg) for PostgreSQL table and column builders |
5959
| `void/schema-mysql` | Re-exports `drizzle-orm/mysql-core` for MySQL table and column builders |
6060
| `void/database-provider` | `defineDatabaseProvider` and adapter types for `void db connect --provider` |
61-
| `void/db` | `db` (Drizzle D1, PostgreSQL, or MySQL instance for the active dialect), `createDb`, query operators (`eq`, `and`, `or`, `desc`, `like`, `inArray`, etc.) |
61+
| `void/db` | `db` (Drizzle D1, PostgreSQL, or MySQL instance for the active dialect), `createDb`, D1 session helpers (`setSession`, `getBookmark`), query operators (`eq`, `and`, `or`, `desc`, `like`, `inArray`, etc.) |
6262
| `void/seed` | `defineSeed`, `SeedContext`, `SeedFn` for programmatic `void db seed` modules |
6363
| `void/kv` | `kv`, `createKV`, typed KV client/map types |
6464
| `void/storage` | `storage`, `createStorage` for the inferred R2 binding or a specific `R2Bucket` |

0 commit comments

Comments
 (0)