Repository navigation
fix(sdk,server,docs): forward maxTokens in middleware, fix bench error message, add recall timeout note - #1338
harrymove-ctrl wants to merge 8 commits into
Conversation
ducnmm
left a comment
There was a problem hiding this comment.
Please address the issues below before merging. Findings are based on the current diff and repository source; no live production writes were run.
| recall throws `MemWalRequestTimeout` (status 504). `isTimeoutError(err)` is true for that error | ||
| and for a preflight `TimeoutError` whose `phase` names the stalled round-trip. | ||
|
|
||
| > **Timeout and retry behaviour**: `recall()` times out after `recallTimeoutMs` milliseconds (default: 5 000 ms, configurable in `MemWalConfig`). On network errors, the call throws immediately without internal retries — callers that need retry logic should wrap `recall()` in their own retry loop or configure a higher `recallTimeoutMs` for slow networks. |
There was a problem hiding this comment.
[P2] Match the documented recall timeout to the SDK default
DEFAULT_RECALL_TIMEOUT_MS is 15_000 in packages/sdk/src/utils.ts, and the constructor uses that constant when recallTimeoutMs is omitted. This new paragraph promises a 5-second bound although default calls can wait 15 seconds, contradicting the configuration reference. Document 15,000 ms and preserve the per-call timeoutMs override.
There was a problem hiding this comment.
Updated docs/sdk/api-reference.md to match the 15 000 ms SDK default recall timeout (configurable via recallTimeoutMs and timeoutMs) and bumped the TypeScript SDK to 0.1.13 in commit c3f7f31.
harrymove-ctrl
left a comment
There was a problem hiding this comment.
Fixed in 7a2fc84.
P2 — Recall timeout: Corrected the documented default from 5 000 ms to 15 000 ms, matching DEFAULT_RECALL_TIMEOUT_MS in packages/sdk/src/utils.ts. The per-call timeoutMs override is preserved in the description.
|
Updated in
All 24 CI checks are green. Ready for re-review @ducnmm. |
ducnmm
left a comment
There was a problem hiding this comment.
The timeout correction addresses the prior finding. The new retry documentation has the separate behavior mismatch below.
|
|
||
| Submit one memory and poll until the background job completes. Accepts `{ idempotencyKey?: string; dedupe?: boolean; pollIntervalMs?: number; timeoutMs?: number }` in `opts` (or in the `namespace` position when using the client default namespace). | ||
|
|
||
| > **Retry and idempotency collapse**: When `idempotencyKey` is omitted, the SDK derives a deterministic key (`r1-<sha256>`) from `(30-minute UTC bucket, resolvedNamespace, text)` (or a bucket-less `dedupe:<sha256>` key when `dedupe: true`) and caches it in the instance's in-memory `pendingRememberKeys` map until `waitForRememberJob()` completes terminally or `destroy()` is called. If `rememberAndWait()` times out or hits a transport error during polling, retrying `rememberAndWait(sameText, sameNamespace)` on the same client instance reuses that cached key (even across a 30-minute bucket boundary), and the relayer collapses the submission onto the existing job (`job_id`) without appending a duplicate memory. After a process restart, `pendingRememberKeys` is empty, so retries within the same 30-minute UTC bucket still collapse onto the existing job, while retries across a bucket boundary require an explicit `idempotencyKey` (or `dedupe: true`). |
There was a problem hiding this comment.
[P2] Distinguish terminal success from terminal failure in key retention
The new guidance says the pending key is cleared when polling completes terminally; production-readiness.md explicitly includes terminal failure. On this head, rememberAndWait awaits waitForRememberJob and only deletes pendingRememberKeys after it returns successfully. Failed/not_found exceptions bypass deletion, so retrying the same text on this instance reuses the failed job key rather than starting a fresh operation. Document terminal success as the cleanup case and explain terminal-failure recovery, or implement and test the broader cleanup guarantee before promising it.
There was a problem hiding this comment.
Updated docs/sdk/api-reference.md, docs/sdk/production-readiness.md, and packages/sdk/src/memwal.ts to document that pendingRememberKeys is cleared only upon terminal success (done). Clarified that on terminal failures (failed or not_found), the key remains cached on the client instance, and callers recovering from terminal failures on that client should pass an explicit idempotencyKey in call options or invoke destroy().
ducnmm
left a comment
There was a problem hiding this comment.
The previous finding has been addressed; the follow-up issue below remains on this head. Validation was offline; no production writes were run.
|
|
||
| Submit one memory and poll until the background job completes. Accepts `{ idempotencyKey?: string; dedupe?: boolean; pollIntervalMs?: number; timeoutMs?: number }` in `opts` (or in the `namespace` position when using the client default namespace). | ||
|
|
||
| > **Retry and idempotency collapse**: When `idempotencyKey` is omitted, the SDK derives a deterministic key (`r1-<sha256>`) from `(30-minute UTC bucket, resolvedNamespace, text)` (or a bucket-less `dedupe:<sha256>` key when `dedupe: true`) and caches it in the instance's in-memory `pendingRememberKeys` map until `waitForRememberJob()` completes with terminal success or `destroy()` is called. If `rememberAndWait()` times out or hits a transport error during polling, retrying `rememberAndWait(sameText, sameNamespace)` on the same client instance reuses that cached key (even across a 30-minute bucket boundary), and the relayer collapses the submission onto the existing job (`job_id`) without appending a duplicate memory. Because `pendingRememberKeys` is only cleared upon terminal success (or via `destroy()`), terminal failures (`MemWalRememberJobFailedError` or `MemWalRememberJobNotFoundError`) leave the key cached on the client instance; to execute a fresh submission after a terminal failure on the same instance, pass an explicit `idempotencyKey` in call options or call `destroy()`. After a process restart, `pendingRememberKeys` is empty, so retries within the same 30-minute UTC bucket still collapse onto the existing job, while retries across a bucket boundary require an explicit `idempotencyKey` (or `dedupe: true`). |
There was a problem hiding this comment.
[P2] Recreate the client after destroy instead of retrying that instance
destroy() permanently sets destroyed=true and zeroes the signing key; rememberAndWait/signedRequest then reject that client as destroyed. The new recovery guidance and method docstring say calling destroy triggers a fresh operation on the same instance, which cannot work. Recommend a fresh explicit idempotency key on the live instance, or explicitly destroy and create a new client with credentials and a fresh operation key (recreating alone can still derive the same bucket key).
There was a problem hiding this comment.
The API reference and method JSDoc now explain fresh keys and client recreation correctly. Please update docs/sdk/production-readiness.md:51 with the same recovery wording: it still advises callers recovering on that client to call destroy() without creating a new client, and omits the fresh-key requirement. This is the remaining occurrence of the existing finding.
There was a problem hiding this comment.
Addressed in commit 1ac597a. Updated docs/sdk/production-readiness.md to align with the recovery wording in the API reference and JSDoc: clarified that callers recovering from terminal failures on the live instance should provide an explicit fresh idempotencyKey in call options, or explicitly destroy and create a new client with credentials and a fresh operation key.
|
@ducnmm Updated in commit 0392256 to address review feedback:
Ready for re-review. |
…r msg, add recall timeout note
…pendingRememberKeys
0392256 to
a67982f
Compare
|
Updated |
Summary
This PR groups three small, independent fixes across the SDK middleware, server benchmark script, and SDK docs.
Fix 1 — bench-recall-latency: correct empty-string error fallback
File:
services/server/scripts/bench-recall-latency.tsThe
result.errorfield can be an empty string"". The nullish coalescing operator (??) treats""as a defined (truthy-empty) value and does not fall through to the HTTP status fallback, resulting in a blank error message being logged and stored..trim() ||correctly falls through to the HTTP status code when the error string is blank or whitespace-only.Fix 2 — SDK middleware: forward
maxTokensfromWithMemWalOptionstorecall()File:
packages/sdk/src/ai/middleware.tsThe
WithMemWalOptionsinterface acceptedmaxMemoriesbut notmaxTokens, even though the underlyingrecall()SDK method supports amaxTokensbudget inRecallOptions. Callers using the AI middleware had no way to pass a token budget through to the recall call.Changes:
maxTokens?: numbertoWithMemWalOptionswith JSDoc.const maxTokens = options.maxTokensin the middleware body.recall()call from positional args to the object form, spreadingmaxTokenswhen defined:Fix 3 — docs: add timeout/retry behaviour note to
recall()API referenceFile:
docs/sdk/api-reference.mdThe
recall()section described thetimeoutMsparameter but did not explain the overall timeout/retry behaviour — specifically thatrecall()does not retry internally and callers must implement their own retry logic if needed.Added a blockquote note immediately after the parameter list:
Checklist
devbranch