Skip to content

fix(search): guard CrossEncoderReranker.ensure_initialized with asyncio.Lock#2445

Open
Srujan-rai wants to merge 1 commit into
vectorize-io:mainfrom
Srujan-rai:fix/reranker-init-lock
Open

fix(search): guard CrossEncoderReranker.ensure_initialized with asyncio.Lock#2445
Srujan-rai wants to merge 1 commit into
vectorize-io:mainfrom
Srujan-rai:fix/reranker-init-lock

Conversation

@Srujan-rai

Copy link
Copy Markdown

Summary

  • Two concurrent requests that both observed _initialized=False would both
    call initialize(), double-loading the local model and causing a memory spike
  • Added a lazily-created asyncio.Lock with double-checked locking pattern so
    only the first caller runs initialization; subsequent callers wait and return
    immediately once _initialized=True

Tests

Added tests/test_reranker_init_lock.py:

  • test_concurrent_ensure_initialized_calls_initialize_once — fires 3 concurrent
    ensure_initialized() calls via asyncio.gather, asserts initialize called once
  • test_second_call_after_init_skips_initialize sequential second call is a no-op

Test plan

  • Concurrent recall requests no longer double-initialize the reranker model
  • Memory usage stable under concurrent load with local cross-encoder

…io.Lock

Two concurrent requests that both observed _initialized=False would both
call initialize(), double-loading the local model and causing a memory
spike. Added a lazily-created asyncio.Lock with double-checked locking so
only the first caller runs initialization; subsequent callers wait and
return immediately once _initialized=True.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant