Skip to content

fix(sdk,server,docs): forward maxTokens in middleware, fix bench error message, add recall timeout note - #1338

Open
harrymove-ctrl wants to merge 8 commits into
devfrom
fix/sdk-server-docs-small-improvements
Open

harrymove-ctrl wants to merge 8 commits into
devfrom
fix/sdk-server-docs-small-improvements

Conversation

@harrymove-ctrl

Copy link
Copy Markdown
Collaborator

Summary

This PR groups three small, independent fixes across the SDK middleware, server benchmark script, and SDK docs.


Fix 1 — bench-recall-latency: correct empty-string error fallback

File: services/server/scripts/bench-recall-latency.ts

The result.error field can be an empty string "". The nullish coalescing operator (??) treats "" as a defined (truthy-empty) value and does not fall through to the HTTP status fallback, resulting in a blank error message being logged and stored.

- const errMsg = result!.error ?? `HTTP ${result!.statusCode}`;
+ const errMsg = result!.error?.trim() || `HTTP ${result!.statusCode}`;

.trim() || correctly falls through to the HTTP status code when the error string is blank or whitespace-only.


Fix 2 — SDK middleware: forward maxTokens from WithMemWalOptions to recall()

File: packages/sdk/src/ai/middleware.ts

The WithMemWalOptions interface accepted maxMemories but not maxTokens, even though the underlying recall() SDK method supports a maxTokens budget in RecallOptions. Callers using the AI middleware had no way to pass a token budget through to the recall call.

Changes:

  1. Added maxTokens?: number to WithMemWalOptions with JSDoc.
  2. Extracted const maxTokens = options.maxTokens in the middleware body.
  3. Changed the recall() call from positional args to the object form, spreading maxTokens when defined:
    - const recallResult = await memwal.recall(lastUserMessage, maxMemories);
    + const recallResult = await memwal.recall({ query: lastUserMessage, limit: maxMemories, ...(maxTokens !== undefined ? { maxTokens } : {}) });

Fix 3 — docs: add timeout/retry behaviour note to recall() API reference

File: docs/sdk/api-reference.md

The recall() section described the timeoutMs parameter but did not explain the overall timeout/retry behaviour — specifically that recall() does not retry internally and callers must implement their own retry logic if needed.

Added a blockquote note immediately after the parameter list:

Timeout and retry behaviour: recall() times out after recallTimeoutMs milliseconds (default: 5 000 ms, configurable in MemWalConfig). On network errors, the call throws immediately without internal retries — callers that need retry logic should wrap recall() in their own retry loop or configure a higher recallTimeoutMs for slow networks.


Checklist

  • No breaking changes — all fixes are backwards-compatible
  • No new dependencies
  • Targets dev branch

@harrymove-ctrl
harrymove-ctrl requested a review from ducnmm October 28, 2026 07:17

@ducnmm ducnmm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the issues below before merging. Findings are based on the current diff and repository source; no live production writes were run.

Comment thread docs/sdk/api-reference.md Outdated
recall throws `MemWalRequestTimeout` (status 504). `isTimeoutError(err)` is true for that error
and for a preflight `TimeoutError` whose `phase` names the stalled round-trip.

> **Timeout and retry behaviour**: `recall()` times out after `recallTimeoutMs` milliseconds (default: 5 000 ms, configurable in `MemWalConfig`). On network errors, the call throws immediately without internal retries — callers that need retry logic should wrap `recall()` in their own retry loop or configure a higher `recallTimeoutMs` for slow networks.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Match the documented recall timeout to the SDK default

DEFAULT_RECALL_TIMEOUT_MS is 15_000 in packages/sdk/src/utils.ts, and the constructor uses that constant when recallTimeoutMs is omitted. This new paragraph promises a 5-second bound although default calls can wait 15 seconds, contradicting the configuration reference. Document 15,000 ms and preserve the per-call timeoutMs override.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated docs/sdk/api-reference.md to match the 15 000 ms SDK default recall timeout (configurable via recallTimeoutMs and timeoutMs) and bumped the TypeScript SDK to 0.1.13 in commit c3f7f31.

@harrymove-ctrl harrymove-ctrl left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7a2fc84.

P2 — Recall timeout: Corrected the documented default from 5 000 ms to 15 000 ms, matching DEFAULT_RECALL_TIMEOUT_MS in packages/sdk/src/utils.ts. The per-call timeoutMs override is preserved in the description.

@harrymove-ctrl

Copy link
Copy Markdown
Collaborator Author

Updated in 9280e77a to address all review feedback:

  1. Documented maxTokens in docs/sdk/usage/with-memwal.md and docs/sdk/api-reference.md and added unit test withMemWal forwards maxTokens to recall() when configured in packages/sdk/test/withMemWal-recall-error.test.mjs.
  2. Added rememberAndWait retry/idempotency collapse documentation (pendingRememberKeys + 30-minute time bucket) to docs/sdk/api-reference.md, docs/sdk/production-readiness.md, and JSDoc in packages/sdk/src/memwal.ts.

All 24 CI checks are green. Ready for re-review @ducnmm.

@harrymove-ctrl
harrymove-ctrl requested a review from ducnmm October 28, 2026 14:16

@ducnmm ducnmm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeout correction addresses the prior finding. The new retry documentation has the separate behavior mismatch below.

Comment thread docs/sdk/api-reference.md Outdated

Submit one memory and poll until the background job completes. Accepts `{ idempotencyKey?: string; dedupe?: boolean; pollIntervalMs?: number; timeoutMs?: number }` in `opts` (or in the `namespace` position when using the client default namespace).

> **Retry and idempotency collapse**: When `idempotencyKey` is omitted, the SDK derives a deterministic key (`r1-<sha256>`) from `(30-minute UTC bucket, resolvedNamespace, text)` (or a bucket-less `dedupe:<sha256>` key when `dedupe: true`) and caches it in the instance's in-memory `pendingRememberKeys` map until `waitForRememberJob()` completes terminally or `destroy()` is called. If `rememberAndWait()` times out or hits a transport error during polling, retrying `rememberAndWait(sameText, sameNamespace)` on the same client instance reuses that cached key (even across a 30-minute bucket boundary), and the relayer collapses the submission onto the existing job (`job_id`) without appending a duplicate memory. After a process restart, `pendingRememberKeys` is empty, so retries within the same 30-minute UTC bucket still collapse onto the existing job, while retries across a bucket boundary require an explicit `idempotencyKey` (or `dedupe: true`).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Distinguish terminal success from terminal failure in key retention

The new guidance says the pending key is cleared when polling completes terminally; production-readiness.md explicitly includes terminal failure. On this head, rememberAndWait awaits waitForRememberJob and only deletes pendingRememberKeys after it returns successfully. Failed/not_found exceptions bypass deletion, so retrying the same text on this instance reuses the failed job key rather than starting a fresh operation. Document terminal success as the cleanup case and explain terminal-failure recovery, or implement and test the broader cleanup guarantee before promising it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated docs/sdk/api-reference.md, docs/sdk/production-readiness.md, and packages/sdk/src/memwal.ts to document that pendingRememberKeys is cleared only upon terminal success (done). Clarified that on terminal failures (failed or not_found), the key remains cached on the client instance, and callers recovering from terminal failures on that client should pass an explicit idempotencyKey in call options or invoke destroy().

@harrymove-ctrl
harrymove-ctrl requested a review from ducnmm October 29, 2026 01:19

@ducnmm ducnmm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous finding has been addressed; the follow-up issue below remains on this head. Validation was offline; no production writes were run.

Comment thread docs/sdk/api-reference.md Outdated

Submit one memory and poll until the background job completes. Accepts `{ idempotencyKey?: string; dedupe?: boolean; pollIntervalMs?: number; timeoutMs?: number }` in `opts` (or in the `namespace` position when using the client default namespace).

> **Retry and idempotency collapse**: When `idempotencyKey` is omitted, the SDK derives a deterministic key (`r1-<sha256>`) from `(30-minute UTC bucket, resolvedNamespace, text)` (or a bucket-less `dedupe:<sha256>` key when `dedupe: true`) and caches it in the instance's in-memory `pendingRememberKeys` map until `waitForRememberJob()` completes with terminal success or `destroy()` is called. If `rememberAndWait()` times out or hits a transport error during polling, retrying `rememberAndWait(sameText, sameNamespace)` on the same client instance reuses that cached key (even across a 30-minute bucket boundary), and the relayer collapses the submission onto the existing job (`job_id`) without appending a duplicate memory. Because `pendingRememberKeys` is only cleared upon terminal success (or via `destroy()`), terminal failures (`MemWalRememberJobFailedError` or `MemWalRememberJobNotFoundError`) leave the key cached on the client instance; to execute a fresh submission after a terminal failure on the same instance, pass an explicit `idempotencyKey` in call options or call `destroy()`. After a process restart, `pendingRememberKeys` is empty, so retries within the same 30-minute UTC bucket still collapse onto the existing job, while retries across a bucket boundary require an explicit `idempotencyKey` (or `dedupe: true`).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Recreate the client after destroy instead of retrying that instance

destroy() permanently sets destroyed=true and zeroes the signing key; rememberAndWait/signedRequest then reject that client as destroyed. The new recovery guidance and method docstring say calling destroy triggers a fresh operation on the same instance, which cannot work. Recommend a fresh explicit idempotency key on the live instance, or explicitly destroy and create a new client with credentials and a fresh operation key (recreating alone can still derive the same bucket key).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API reference and method JSDoc now explain fresh keys and client recreation correctly. Please update docs/sdk/production-readiness.md:51 with the same recovery wording: it still advises callers recovering on that client to call destroy() without creating a new client, and omits the fresh-key requirement. This is the remaining occurrence of the existing finding.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in commit 1ac597a. Updated docs/sdk/production-readiness.md to align with the recovery wording in the API reference and JSDoc: clarified that callers recovering from terminal failures on the live instance should provide an explicit fresh idempotencyKey in call options, or explicitly destroy and create a new client with credentials and a fresh operation key.

@harrymove-ctrl

Copy link
Copy Markdown
Collaborator Author

@ducnmm Updated in commit 0392256 to address review feedback:

  1. Clarified in docs/sdk/api-reference.md and packages/sdk/src/memwal.ts that calling destroy permanently invalidates the client instance. For retrying after a terminal failure, caller should provide an explicit fresh idempotencyKey on the live instance, or explicitly destroy and create a new client with credentials and a fresh operation key.
  2. Resolved the OpenClaw unit test timing issue in packages/openclaw-memory-memwal/test/plugin.test.mjs by priming relayer compatibility before the timed test turn and providing sufficient timeout headroom for the socket abort event.

Ready for re-review.

@harrymove-ctrl
harrymove-ctrl requested a review from ducnmm October 29, 2026 04:14
@harrymove-ctrl
harrymove-ctrl force-pushed the fix/sdk-server-docs-small-improvements branch from 0392256 to a67982f Compare October 29, 2026 05:57
@harrymove-ctrl

Copy link
Copy Markdown
Collaborator Author

Updated docs/sdk/production-readiness.md (line 51) to match the recovery wording in docs/sdk/api-reference.md and packages/sdk/src/memwal.ts: callers recovering from a terminal failure should provide an explicit fresh idempotencyKey on the live instance, or explicitly destroy and create a new client with credentials and a fresh operation key. Ready for re-review @ducnmm.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants