fix(sdk,server,docs): forward maxTokens in middleware, fix bench error message, add recall timeout note

docs/sdk/api-reference.md

@@ -84,6 +84,8 @@ HTTP 426 throws `MemWalCompatibilityError`.
 
 Submit one memory and poll until the background job completes. Accepts `{ idempotencyKey?: string; dedupe?: boolean; pollIntervalMs?: number; timeoutMs?: number }` in `opts` (or in the `namespace` position when using the client default namespace).
 
+> **Retry and idempotency collapse**: When `idempotencyKey` is omitted, the SDK derives a deterministic key (`r1-<sha256>`) from `(30-minute UTC bucket, resolvedNamespace, text)` (or a bucket-less `dedupe:<sha256>` key when `dedupe: true`) and caches it in the instance's in-memory `pendingRememberKeys` map until `waitForRememberJob()` completes with terminal success. If `rememberAndWait()` times out or hits a transport error during polling, retrying `rememberAndWait(sameText, sameNamespace)` on the same client instance reuses that cached key (even across a 30-minute bucket boundary), and the relayer collapses the submission onto the existing job (`job_id`) without appending a duplicate memory. Because `pendingRememberKeys` is only cleared upon terminal success, terminal failures (`MemWalRememberJobFailedError` or `MemWalRememberJobNotFoundError`) leave the key cached on the client instance; to execute a fresh submission after a terminal failure on the live instance, pass an explicit fresh `idempotencyKey` in call options, or explicitly destroy and create a new client with credentials and a fresh operation key (recreating alone can still derive the same bucket key). After a process restart, `pendingRememberKeys` is empty, so retries within the same 30-minute UTC bucket still collapse onto the existing job, while retries across a bucket boundary require an explicit `idempotencyKey` (or `dedupe: true`).
+
 **Returns:**
 
 ```ts
@@ -136,6 +138,8 @@ Search for memories matching a natural language query, scoped to `owner + namesp
   recall throws `MemWalRequestTimeout` (status 504). `isTimeoutError(err)` is true for that error
   and for a preflight `TimeoutError` whose `phase` names the stalled round-trip.
 
+> **Timeout and retry behaviour**: `recall()` times out after `recallTimeoutMs` milliseconds (default: 15 000 ms, configurable in `MemWalConfig` and overridable per call via `timeoutMs`). On network errors, the call throws immediately without internal retries; callers that need retry logic should wrap `recall()` in their own retry loop or configure a higher `recallTimeoutMs` for slow networks.
+
 **Returns:**
 
 ```ts
@@ -460,6 +464,7 @@ Wraps a Vercel AI SDK model with automatic memory recall and save.
 | Option | Default | Description |
 |--------|---------|-------------|
 | `maxMemories` | `5` | Max memories to inject per request |
+| `maxTokens` | `undefined` | Optional token budget for recalled memories injected per request (forwarded to `recall({ maxTokens })`) |
 | `autoSave` | `true` | Auto-save new facts from conversation |
 | `minRelevance` | `0.3` | Minimum similarity score (0–1) to include a memory |
 | `debug` | `false` | Enable debug logging |

docs/sdk/changelog.mdx

@@ -28,9 +28,15 @@ questions:
   - When was bulk remember added to the Walrus Memory SDK?
   - What security improvements have been made to the MemWal SDK?
 answer: >-
-  Version 0.1.12 bounds withMemWal recall queries (preserving both head context and tail question/instructions via strategy: "head-tail") to 16,384 UTF-8 bytes and auto-save analyze inputs to 65,536 UTF-8 bytes without splitting multi-byte UTF-8 codepoints across boundaries, preventing HTTP 400 errors on long turns. Version 0.1.11 adds opt-in content-hash deduplication (dedupe: true) for remember(), copies caller-supplied Uint8Array keys on client creation, hardens MemWalManual.destroy() secret cleanup and client invalidation, validates and clamps recalled memory distance before similarity relevance filtering and formatting, validates listNamespaces and listMemories limits before signing, adds AbortSignal cancellation support to analyze() via AnalyzeOptions.signal, and validates manual-mode vector and encryptedData inputs before signing.
+  Version 0.1.13 updates withMemWal to accept maxTokens in WithMemWalOptions and forward it to recall({ maxTokens }) before calling the underlying language model. Version 0.1.12 bounds withMemWal recall queries (preserving both head context and tail question/instructions via strategy: "head-tail") to 16,384 UTF-8 bytes and auto-save analyze inputs to 65,536 UTF-8 bytes without splitting multi-byte UTF-8 codepoints across boundaries, preventing HTTP 400 errors on long turns. Version 0.1.11 adds opt-in content-hash deduplication (dedupe: true) for remember(), copies caller-supplied Uint8Array keys on client creation, hardens MemWalManual.destroy() secret cleanup and client invalidation, validates and clamps recalled memory distance before similarity relevance filtering and formatting, validates listNamespaces and listMemories limits before signing, adds AbortSignal cancellation support to analyze() via AnalyzeOptions.signal, and validates manual-mode vector and encryptedData inputs before signing.
 ---
 
+## 0.1.13
+
+### Fixed
+
+- `withMemWal` accepts `maxTokens` in `WithMemWalOptions` and forwards it to `recall({ maxTokens })` before calling the underlying language model, capping the token budget of recalled memories injected into each prompt.
+
 ## 0.1.12
 
 ### Fixed

docs/sdk/production-readiness.md

@@ -48,6 +48,8 @@ Upgrade from 0.1.8 for bulk idempotency; its bulk methods send no key even if a
 
 The bucket is not a rolling 30-minute guarantee: 2 attempts a few seconds apart can cross its boundary. Changing batch order, content, or namespaces also changes the automatic key.
 
+Within a single `MemWal` instance, `rememberAndWait()` also retains the generated key in an in-memory `pendingRememberKeys` map until the job reaches terminal success (`done`). If `rememberAndWait(sameText, sameNamespace)` times out while polling or encounters transient network errors, an immediate retry on the same client instance reuses the retained key from `pendingRememberKeys` (even across a 30-minute bucket boundary), so the relayer collapses the retry onto the original `job_id` instead of appending a second fact. On terminal failure (`failed` or `not_found`), the key remains cached on the instance; callers recovering from terminal failures on the live instance should provide an explicit fresh `idempotencyKey` in call options, or explicitly destroy and create a new client with credentials and a fresh operation key (recreating alone can still derive the same bucket key). A process restart clears `pendingRememberKeys` and falls back to the 30-minute time-bucket calculation.
+
 For a durable queue, or any retry that can cross a bucket boundary or a process restart, persist one operation key with the payload. Reuse it on every submission attempt.
 
 ```ts

docs/sdk/usage/with-memwal.md

@@ -58,15 +58,25 @@ const result = await generateText({
 Before generation:
 
 - Reads the last user message
-- Runs `recall()` against Walrus Memory
-- Filters by relevance
+- Runs `recall()` against Walrus Memory (bounded by `maxMemories` and optional `maxTokens`)
+- Filters by minimum similarity (`minRelevance`, default `0.3`)
 - Injects memory context into the prompt
 
 After generation:
 
-- Optionally runs `analyze()` on the user message
+- Optionally runs `analyze()` on the user message (`autoSave`, default `true`)
 - Saves extracted facts asynchronously
 
+## Options
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `maxMemories` | `5` | Maximum number of memories to recall and inject per request (top-K limit) |
+| `maxTokens` | `undefined` | Optional token budget cap on recalled memories injected into the prompt (forwarded to `recall({ maxTokens })`) |
+| `minRelevance` | `0.3` | Minimum similarity score (`0`–`1`) required for a recalled memory to be injected |
+| `autoSave` | `true` | Automatically run `analyze()` after generation to extract and store new facts |
+| `debug` | `false` | Log recall and auto-save activity to `console.warn` |
+
 Set a namespace explicitly for each product surface that uses the middleware. Otherwise recalled
 and auto-saved memories fall back to `"default"`.
 

packages/sdk/CHANGELOG.md

@@ -1,5 +1,11 @@
 # @mysten-incubation/memwal
 
+## 0.1.13
+
+### Fixed
+
+- `withMemWal` accepts `maxTokens` in `WithMemWalOptions` and forwards it to `recall({ maxTokens })` before calling the underlying language model, capping the token budget of recalled memories injected into each prompt.
+
 ## 0.1.12
 
 ### Fixed

packages/sdk/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@mysten-incubation/memwal",
-    "version": "0.1.12",
+    "version": "0.1.13",
     "description": "Walrus Memory \u2014 Privacy-first AI memory SDK with Ed25519 delegate key auth",
     "type": "module",
     "engines": {

packages/sdk/src/ai/middleware.ts

@@ -47,6 +47,8 @@ type AnyLanguageModel = any;
 export interface WithMemWalOptions extends MemWalConfig {
     /** Max memories to inject per request (default: 5) */
     maxMemories?: number;
+    /** Token budget for recalled memories injected per request. Maps to recall({ maxTokens }). */
+    maxTokens?: number;
     /** Auto-save new facts from conversation (default: true) */
     autoSave?: boolean;
     /** Minimum similarity score to include a memory (0-1, default: 0.3) */
@@ -183,6 +185,7 @@ export function withMemWal(
 ) {
     const memwal = MemWal.create(options);
     const maxMemories = options.maxMemories ?? 5;
+    const maxTokens = options.maxTokens;
     const autoSave = options.autoSave ?? true;
     const minRelevance = options.minRelevance ?? 0.3;
     const debug = options.debug ?? false;
@@ -258,7 +261,10 @@ export function withMemWal(
                     const recallQuery = truncateUtf8Bytes(lastUserMessage, MAX_EMBED_INPUT_BYTES, {
                         strategy: "head-tail",
                     });
-                    const recallResult = await memwal.recall(recallQuery, maxMemories);
+                    const recallResult =
+                        maxTokens !== undefined
+                            ? await memwal.recall({ query: recallQuery, limit: maxMemories, maxTokens })
+                            : await memwal.recall(recallQuery, maxMemories);
                     const relevant = recallResult.results.filter((m: RecallMemory) => {
                         const relevance = memoryRelevance(m?.distance);
                         return relevance !== null && relevance >= minRelevance;

packages/sdk/src/memwal.ts

@@ -788,6 +788,16 @@ export class MemWal {
 
     /**
      * Remember something and wait for the background job to complete.
+     *
+     * When `idempotencyKey` is omitted, derives a deterministic key (`r1-<sha256>`)
+     * from `(30-minute UTC bucket, resolvedNamespace, text)` (or `dedupe:<sha256>`
+     * when `dedupe: true`) and retains it in `pendingRememberKeys` until the job
+     * succeeds terminally (`done`). Retrying `rememberAndWait` with the same
+     * `(text, namespace)` after a polling timeout reuses the key so the relayer
+     * collapses the retry onto the existing `job_id` without appending a
+     * duplicate memory. After a terminal failure, provide an explicit fresh
+     * `idempotencyKey` on the live instance, or explicitly destroy and create
+     * a new client with credentials and a fresh operation key.
      */
     async rememberAndWait(
         text: string,

packages/sdk/test/withMemWal-recall-error.test.mjs

@@ -136,3 +136,37 @@ test("if onRecallError throws, generation still proceeds", async () => {
         console.warn = originalWarn;
     }
 });
+
+test("withMemWal forwards maxTokens to recall() when configured", async () => {
+    const originalRecall = MemWal.prototype.recall;
+    const recallCalls = [];
+
+    MemWal.prototype.recall = async (params) => {
+        recallCalls.push(params);
+        return { results: [], total: 0 };
+    };
+
+    try {
+        const model = withMemWal(fakeLanguageModel(), {
+            key: TEST_KEY,
+            accountId: TEST_ACCOUNT_ID,
+            autoSave: false,
+            maxMemories: 7,
+            maxTokens: 256,
+        });
+
+        await generateText({
+            model,
+            prompt: "Summarize my preferences",
+        });
+
+        assert.equal(recallCalls.length, 1);
+        assert.deepEqual(recallCalls[0], {
+            query: "Summarize my preferences",
+            limit: 7,
+            maxTokens: 256,
+        });
+    } finally {
+        MemWal.prototype.recall = originalRecall;
+    }
+});

services/server/scripts/bench-recall-latency.ts

@@ -420,7 +420,7 @@ async function runBatch(
       console.log(`ok ${ms(result!.latencyMs)} ${formatOk(result!)}`);
     } else {
       failCount++;
-      const errMsg = result!.error ?? `HTTP ${result!.statusCode}`;
+      const errMsg = result!.error?.trim() || `HTTP ${result!.statusCode}`;
       errors.push(errMsg);
       console.log(`FAILED: ${errMsg.slice(0, 120)}`);
     }