Skip to content

Thread lock silently expires after 30s mid-handler; queue/burst/debounce lose serialization for long-running handlers #685

Description

@spookyuser

Bug Description

The per-thread lock that backs every concurrency strategy is acquired with a hardcoded 30-second TTL (DEFAULT_LOCK_TTL_MS = 30_000) and is never extended while a handler is running. extendLock exists on the StateAdapter interface and the core does call it — but only between debounce sleeps and queue drains, never around dispatchToHandlers.

The result: any handler that takes longer than 30s silently loses its lock mid-run. The next inbound message on the same thread doesn't queue behind it — it acquires a fresh lock and dispatches concurrently, on the same thread, with no error and no log line. The serialization guarantee that queue/burst/debounce advertise quietly breaks exactly in the case those strategies exist for: slow, AI-agent-style handlers.

There is also no way to configure the TTL — it's a module-level constant, not a ChatConfig option — so there's no supported escape hatch short of wrapping the state adapter.

Verified present in 4.31.0 and in 4.33.0 (dist/index.js: var DEFAULT_LOCK_TTL_MS = 3e4; acquisitions pass it verbatim; the only extendLock call sites are the post-debounce sleep, debounceLoop, and drainQueue).

Steps to Reproduce

  1. Create a Chat instance with a Redis state adapter and concurrency: "queue" (or "burst").
  2. Register a DM handler that takes ~45s (e.g. an AI agent loop, or just await sleep(45_000)).
  3. Send message A at t=0 — handler starts, lock acquired with 30s TTL.
  4. Send message B on the same thread at t=35s.

Expected Behavior

B is enqueued behind A's in-flight handler and dispatched afterwards (with context.skipped semantics), per the Overlapping Messages docs: "Messages that arrive while a handler is running are enqueued."

Actual Behavior

A's lock expired at t=30s, so B's webhook acquires a brand-new lock and dispatches immediately. Two handlers now run concurrently on the same thread. Neither the A-run nor the B-run has any indication this happened.

Note this is different from onLockConflict: 'force', where the docs explicitly warn "two handlers may briefly run concurrently on the same thread" — here the same outcome occurs without opting into anything, under the default-safe strategies.

Code Sample

import { Chat } from "chat";
import { createRedisState } from "@chat-adapter/state-redis";

const bot = new Chat({
  adapters: { whatsapp: createWhatsAppAdapter() },
  state: createRedisState({ url: process.env.REDIS_URL }),
  concurrency: "queue",
});

bot.onDirectMessage(async (thread, message) => {
  // Anything slower than 30s: agent loop, tool calls, slow API...
  await runAgent(message); // ~45s
  await thread.post("done");
});
// Send two messages 35s apart on one thread -> both handlers run at once.

Chat SDK Version

4.31.0 (reproduced against 4.33.0 dist as well)

Node.js Version

20.x

Platform Adapter

WhatsApp

Operating System

macOS

Additional Context

Real-world impact. We run a WhatsApp grocery-shopping agent where handler runs routinely take 40–90s.

Workaround — a state-adapter wrapper that heartbeats the lock while it's held, using the extendLock primitive that already exists on every built-in adapter:

export function withLockKeepalive(state: StateAdapter, intervalMs = 10_000): StateAdapter {
  const heartbeats = new Map<string, ReturnType<typeof setInterval>>();
  const stop = (key: string) => { clearInterval(heartbeats.get(key)!); heartbeats.delete(key); };
  return new Proxy(state, {
    get(target, prop) {
      if (prop === "acquireLock") {
        return async (threadId: string, ttlMs: number) => {
          const lock = await target.acquireLock(threadId, ttlMs);
          if (lock) {
            const key = `${threadId}:${lock.token}`;
            heartbeats.set(key, setInterval(() => {
              void target.extendLock(lock, ttlMs).then((ok) => { if (!ok) stop(key); });
            }, intervalMs));
          }
          return lock;
        };
      }
      if (prop === "releaseLock") {
        return async (lock: Lock) => { stop(`${lock.threadId}:${lock.token}`); await target.releaseLock(lock); };
      }
      if (prop === "forceReleaseLock") {
        return async (threadId: string) => {
          for (const key of heartbeats.keys()) if (key.startsWith(`${threadId}:`)) stop(key);
          await target.forceReleaseLock(threadId);
        };
      }
      const value = Reflect.get(target, prop, target);
      return typeof value === "function" ? value.bind(target) : value;
    },
  });
}

If the process dies, the interval dies with it and the lock expires within the normal 30s TTL, so crash recovery is preserved.

Suggested fix (either or both):

  1. Heartbeat during dispatch in core: wrap dispatchToHandlers with a setInterval(() => stateAdapter.extendLock(lock, DEFAULT_LOCK_TTL_MS), DEFAULT_LOCK_TTL_MS / 3), cleared in a finally. ~15 lines, no API change, keeps the short TTL as the crash-recovery bound while making the hold effectively handler-scoped.
  2. Expose lockTtlMs on ChatConfig for consumers who'd rather size the TTL to their handler budget.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions