diff --git a/docs/sdk/api-reference.md b/docs/sdk/api-reference.md index 7557cea2e..8b30d02c5 100644 --- a/docs/sdk/api-reference.md +++ b/docs/sdk/api-reference.md @@ -84,6 +84,8 @@ HTTP 426 throws `MemWalCompatibilityError`. Submit one memory and poll until the background job completes. Accepts `{ idempotencyKey?: string; dedupe?: boolean; pollIntervalMs?: number; timeoutMs?: number }` in `opts` (or in the `namespace` position when using the client default namespace). +> **Retry and idempotency collapse**: When `idempotencyKey` is omitted, the SDK derives a deterministic key (`r1-`) from `(30-minute UTC bucket, resolvedNamespace, text)` (or a bucket-less `dedupe:` key when `dedupe: true`) and caches it in the instance's in-memory `pendingRememberKeys` map until `waitForRememberJob()` completes with terminal success. If `rememberAndWait()` times out or hits a transport error during polling, retrying `rememberAndWait(sameText, sameNamespace)` on the same client instance reuses that cached key (even across a 30-minute bucket boundary), and the relayer collapses the submission onto the existing job (`job_id`) without appending a duplicate memory. Because `pendingRememberKeys` is only cleared upon terminal success, terminal failures (`MemWalRememberJobFailedError` or `MemWalRememberJobNotFoundError`) leave the key cached on the client instance; to execute a fresh submission after a terminal failure on the live instance, pass an explicit fresh `idempotencyKey` in call options, or explicitly destroy and create a new client with credentials and a fresh operation key (recreating alone can still derive the same bucket key). After a process restart, `pendingRememberKeys` is empty, so retries within the same 30-minute UTC bucket still collapse onto the existing job, while retries across a bucket boundary require an explicit `idempotencyKey` (or `dedupe: true`). + **Returns:** ```ts @@ -137,6 +139,8 @@ Search for memories matching a natural language query, scoped to `owner + namesp recall throws `MemWalRequestTimeout` (status 504). `isTimeoutError(err)` is true for that error and for a preflight `TimeoutError` whose `phase` names the stalled round-trip. +> **Timeout and retry behaviour**: `recall()` times out after `recallTimeoutMs` milliseconds (default: 15 000 ms, configurable in `MemWalConfig` and overridable per call via `timeoutMs`). On network errors, the call throws immediately without internal retries; callers that need retry logic should wrap `recall()` in their own retry loop or configure a higher `recallTimeoutMs` for slow networks. + **Returns:** ```ts @@ -463,6 +467,7 @@ Wraps a Vercel AI SDK model with automatic memory recall and save. | Option | Default | Description | |--------|---------|-------------| | `maxMemories` | `5` | Max memories to inject per request | +| `maxTokens` | `undefined` | Optional token budget for recalled memories injected per request (forwarded to `recall({ maxTokens })`) | | `autoSave` | `true` | Auto-save new facts from conversation | | `minRelevance` | `0.3` | Minimum similarity score (0–1) to include a memory | | `debug` | `false` | Enable debug logging | diff --git a/docs/sdk/changelog.mdx b/docs/sdk/changelog.mdx index cd0f29734..91259c521 100644 --- a/docs/sdk/changelog.mdx +++ b/docs/sdk/changelog.mdx @@ -28,7 +28,7 @@ questions: - When was bulk remember added to the Walrus Memory SDK? - What security improvements have been made to the MemWal SDK? answer: >- - Version 0.1.13 adds degraded recall flags (degraded and dropped_count on RecallResult, returned by default via onDropped: "ignore"), opt-in MemWalRecallAllDroppedError when onDropped: "throw" is passed, aligns MemWal and MemWalMock input validation, rejects invalid idempotencyKey values before signing and non-positive limits consistently, and preserves access to whitespace namespaces on read paths while rejecting empty namespace strings. Version 0.1.12 bounds withMemWal recall queries (preserving both head context and tail question/instructions via strategy: "head-tail") to 16,384 UTF-8 bytes and auto-save analyze inputs to 65,536 UTF-8 bytes without splitting multi-byte UTF-8 codepoints across boundaries, preventing HTTP 400 errors on long turns. Version 0.1.11 adds opt-in content-hash deduplication (dedupe: true) for remember(), copies caller-supplied Uint8Array keys on client creation, hardens MemWalManual.destroy() secret cleanup and client invalidation, validates and clamps recalled memory distance before similarity relevance filtering and formatting, validates listNamespaces and listMemories limits before signing, adds AbortSignal cancellation support to analyze() via AnalyzeOptions.signal, and validates manual-mode vector and encryptedData inputs before signing. + Version 0.1.13 adds degraded recall flags (degraded and dropped_count on RecallResult, returned by default via onDropped: "ignore"), opt-in MemWalRecallAllDroppedError when onDropped: "throw" is passed, updates withMemWal to accept maxTokens in WithMemWalOptions and forward it to recall({ maxTokens }) before calling the underlying language model, aligns MemWal and MemWalMock input validation, rejects invalid idempotencyKey values before signing and non-positive limits consistently, and preserves access to whitespace namespaces on read paths while rejecting empty namespace strings. Version 0.1.12 bounds withMemWal recall queries (preserving both head context and tail question/instructions via strategy: "head-tail") to 16,384 UTF-8 bytes and auto-save analyze inputs to 65,536 UTF-8 bytes without splitting multi-byte UTF-8 codepoints across boundaries, preventing HTTP 400 errors on long turns. Version 0.1.11 adds opt-in content-hash deduplication (dedupe: true) for remember(), copies caller-supplied Uint8Array keys on client creation, hardens MemWalManual.destroy() secret cleanup and client invalidation, validates and clamps recalled memory distance before similarity relevance filtering and formatting, validates listNamespaces and listMemories limits before signing, adds AbortSignal cancellation support to analyze() via AnalyzeOptions.signal, and validates manual-mode vector and encryptedData inputs before signing. --- ## 0.1.13 @@ -41,6 +41,7 @@ answer: >- ### Fixed - `MemWal` and `MemWalMock` share input validation on `remember*`, `rememberBulk*`, `analyze*`, `embed`, `recall`, `restore`, and `listNamespaces`, rejecting invalid `idempotencyKey` values before signing and rejecting `limit: 0` consistently. On read paths (`recall`, `recallManual`, `restore`, and `listMemories`), empty string `""` is rejected while whitespace namespaces remain readable to match the server's namespace validation. (Refs #978) +- `withMemWal` accepts `maxTokens` in `WithMemWalOptions` and forwards it to `recall({ maxTokens })` before calling the underlying language model, capping the token budget of recalled memories injected into each prompt. ## 0.1.12 diff --git a/docs/sdk/production-readiness.md b/docs/sdk/production-readiness.md index 8444f61e6..911e50ed3 100644 --- a/docs/sdk/production-readiness.md +++ b/docs/sdk/production-readiness.md @@ -48,6 +48,8 @@ Upgrade from 0.1.8 for bulk idempotency; its bulk methods send no key even if a The bucket is not a rolling 30-minute guarantee: 2 attempts a few seconds apart can cross its boundary. Changing batch order, content, or namespaces also changes the automatic key. +Within a single `MemWal` instance, `rememberAndWait()` also retains the generated key in an in-memory `pendingRememberKeys` map until the job reaches terminal success (`done`). If `rememberAndWait(sameText, sameNamespace)` times out while polling or encounters transient network errors, an immediate retry on the same client instance reuses the retained key from `pendingRememberKeys` (even across a 30-minute bucket boundary), so the relayer collapses the retry onto the original `job_id` instead of appending a second fact. On terminal failure (`failed` or `not_found`), the key remains cached on the instance; callers recovering from terminal failures on the live instance should provide an explicit fresh `idempotencyKey` in call options, or explicitly destroy and create a new client with credentials and a fresh operation key (recreating alone can still derive the same bucket key). A process restart clears `pendingRememberKeys` and falls back to the 30-minute time-bucket calculation. + For a durable queue, or any retry that can cross a bucket boundary or a process restart, persist one operation key with the payload. Reuse it on every submission attempt. ```ts diff --git a/docs/sdk/usage/with-memwal.md b/docs/sdk/usage/with-memwal.md index edbf577c2..f5c784300 100644 --- a/docs/sdk/usage/with-memwal.md +++ b/docs/sdk/usage/with-memwal.md @@ -58,15 +58,25 @@ const result = await generateText({ Before generation: - Reads the last user message -- Runs `recall()` against Walrus Memory -- Filters by relevance +- Runs `recall()` against Walrus Memory (bounded by `maxMemories` and optional `maxTokens`) +- Filters by minimum similarity (`minRelevance`, default `0.3`) - Injects memory context into the prompt After generation: -- Optionally runs `analyze()` on the user message +- Optionally runs `analyze()` on the user message (`autoSave`, default `true`) - Saves extracted facts asynchronously +## Options + +| Option | Default | Description | +| --- | --- | --- | +| `maxMemories` | `5` | Maximum number of memories to recall and inject per request (top-K limit) | +| `maxTokens` | `undefined` | Optional token budget cap on recalled memories injected into the prompt (forwarded to `recall({ maxTokens })`) | +| `minRelevance` | `0.3` | Minimum similarity score (`0`–`1`) required for a recalled memory to be injected | +| `autoSave` | `true` | Automatically run `analyze()` after generation to extract and store new facts | +| `debug` | `false` | Log recall and auto-save activity to `console.warn` | + Set a namespace explicitly for each product surface that uses the middleware. Otherwise recalled and auto-saved memories fall back to `"default"`. diff --git a/packages/sdk/CHANGELOG.md b/packages/sdk/CHANGELOG.md index 6ea9721b5..5a8d83fa4 100644 --- a/packages/sdk/CHANGELOG.md +++ b/packages/sdk/CHANGELOG.md @@ -10,6 +10,7 @@ ### Fixed - `MemWal` and `MemWalMock` share input validation on `remember*`, `rememberBulk*`, `analyze*`, `embed`, `recall`, `restore`, and `listNamespaces`, rejecting invalid `idempotencyKey` values before signing and rejecting `limit: 0` consistently. On read paths (`recall`, `recallManual`, `restore`, and `listMemories`), empty string `""` is rejected while whitespace namespaces remain readable to match the server's namespace validation. (Refs #978) +- `withMemWal` accepts `maxTokens` in `WithMemWalOptions` and forwards it to `recall({ maxTokens })` before calling the underlying language model, capping the token budget of recalled memories injected into each prompt. ## 0.1.12 diff --git a/packages/sdk/src/ai/middleware.ts b/packages/sdk/src/ai/middleware.ts index fb4c48c1b..1ee76d2ac 100644 --- a/packages/sdk/src/ai/middleware.ts +++ b/packages/sdk/src/ai/middleware.ts @@ -48,6 +48,8 @@ type AnyLanguageModel = any; export interface WithMemWalOptions extends MemWalConfig { /** Max memories to inject per request (default: 5) */ maxMemories?: number; + /** Token budget for recalled memories injected per request. Maps to recall({ maxTokens }). */ + maxTokens?: number; /** Auto-save new facts from conversation (default: true) */ autoSave?: boolean; /** Minimum similarity score to include a memory (0-1, default: 0.3) */ @@ -184,6 +186,7 @@ export function withMemWal( ) { const memwal = MemWal.create(options); const maxMemories = options.maxMemories ?? 5; + const maxTokens = options.maxTokens; const autoSave = options.autoSave ?? true; const minRelevance = options.minRelevance ?? 0.3; const debug = options.debug ?? false; @@ -259,7 +262,10 @@ export function withMemWal( const recallQuery = truncateUtf8Bytes(lastUserMessage, MAX_EMBED_INPUT_BYTES, { strategy: "head-tail", }); - const recallResult = await memwal.recall(recallQuery, maxMemories); + const recallResult = + maxTokens !== undefined + ? await memwal.recall({ query: recallQuery, limit: maxMemories, maxTokens }) + : await memwal.recall(recallQuery, maxMemories); if (recallResult.results.length === 0 && recallResult.degraded) { const droppedCount = typeof recallResult.dropped_count === "number" && recallResult.dropped_count > 0 diff --git a/packages/sdk/src/memwal.ts b/packages/sdk/src/memwal.ts index 0af191968..d5aeec400 100644 --- a/packages/sdk/src/memwal.ts +++ b/packages/sdk/src/memwal.ts @@ -783,6 +783,16 @@ export class MemWal { /** * Remember something and wait for the background job to complete. + * + * When `idempotencyKey` is omitted, derives a deterministic key (`r1-`) + * from `(30-minute UTC bucket, resolvedNamespace, text)` (or `dedupe:` + * when `dedupe: true`) and retains it in `pendingRememberKeys` until the job + * succeeds terminally (`done`). Retrying `rememberAndWait` with the same + * `(text, namespace)` after a polling timeout reuses the key so the relayer + * collapses the retry onto the existing `job_id` without appending a + * duplicate memory. After a terminal failure, provide an explicit fresh + * `idempotencyKey` on the live instance, or explicitly destroy and create + * a new client with credentials and a fresh operation key. */ async rememberAndWait( text: string, diff --git a/packages/sdk/test/withMemWal-recall-error.test.mjs b/packages/sdk/test/withMemWal-recall-error.test.mjs index e2e0efb4d..2de1d92cb 100644 --- a/packages/sdk/test/withMemWal-recall-error.test.mjs +++ b/packages/sdk/test/withMemWal-recall-error.test.mjs @@ -194,3 +194,37 @@ test("all-dropped recall invokes onRecallError and does not cache an empty memor console.warn = originalWarn; } }); + +test("withMemWal forwards maxTokens to recall() when configured", async () => { + const originalRecall = MemWal.prototype.recall; + const recallCalls = []; + + MemWal.prototype.recall = async (params) => { + recallCalls.push(params); + return { results: [], total: 0 }; + }; + + try { + const model = withMemWal(fakeLanguageModel(), { + key: TEST_KEY, + accountId: TEST_ACCOUNT_ID, + autoSave: false, + maxMemories: 7, + maxTokens: 256, + }); + + await generateText({ + model, + prompt: "Summarize my preferences", + }); + + assert.equal(recallCalls.length, 1); + assert.deepEqual(recallCalls[0], { + query: "Summarize my preferences", + limit: 7, + maxTokens: 256, + }); + } finally { + MemWal.prototype.recall = originalRecall; + } +}); diff --git a/services/server/scripts/bench-recall-latency.ts b/services/server/scripts/bench-recall-latency.ts index 65906eb4d..037ca6c8c 100644 --- a/services/server/scripts/bench-recall-latency.ts +++ b/services/server/scripts/bench-recall-latency.ts @@ -420,7 +420,7 @@ async function runBatch( console.log(`ok ${ms(result!.latencyMs)} ${formatOk(result!)}`); } else { failCount++; - const errMsg = result!.error ?? `HTTP ${result!.statusCode}`; + const errMsg = result!.error?.trim() || `HTTP ${result!.statusCode}`; errors.push(errMsg); console.log(`FAILED: ${errMsg.slice(0, 120)}`); }