Skip to content

⏱️ feat: Configurable Timeouts for the Web-Search Reranker and Image Generation - #16849

Open
kevinpark1217 wants to merge 1 commit into
LibreChat-AI:devfrom
kevinpark1217:feat/tool-timeouts
Open

kevinpark1217 wants to merge 1 commit into
LibreChat-AI:devfrom
kevinpark1217:feat/tool-timeouts

Conversation

@kevinpark1217

@kevinpark1217 kevinpark1217 commented Oct 7, 2026 •

Copy link
Copy Markdown

Summary

Two tool timeouts are fixed today, and slower self-hosted backends exceed them: the web-search rerank (10 s per page in @librechat/agents) and the image_gen_oai OpenAI client. This PR makes both configurable:

  • webSearch.rerankerTimeout (ms), passed to createSearchTool, which already accepts it.
  • IMAGE_GEN_OAI_TIMEOUT_MS and IMAGE_GEN_OAI_MAX_RETRIES, passed to the image_gen_oai client as timeout and maxRetries. These are env vars because the tool's other settings are too.

The image timeout is also passed to the request's undici dispatcher as headersTimeout/bodyTimeout, the same way the chat endpoints use transportTimeouts. Without it, undici's 300 s defaults end the request first, and openai-node reports that as Request timed out.

When unset, behavior is unchanged.

Related to #16494
Related to LibreChat-AI/agents#290

Type of change

  • Feature

Testing

Added tests for schema validation, env parsing, dispatcher selection (direct and proxy), the authResult passthrough, and the options the OpenAI client receives. I also checked both settings against stub endpoints. With rerankerTimeout: 500, a 1.5 s rerank is aborted at ~500 ms. With IMAGE_GEN_OAI_TIMEOUT_MS=1800000, an image response held for 330 s succeeds; with only openai's timeout set, the same request fails at ~300 s.

Screenshots / recordings

No user-facing change.

Checklist

  • I reviewed my own changes
  • Relevant tests have been added or updated
  • Existing relevant tests pass
  • The change does not introduce new warnings or errors
  • User-facing or complex behavior is documented where necessary
  • Required dependency changes have been merged/published
  • Required documentation PR: N/A

…Generation

- Add `webSearch.rerankerTimeout` and pass it to `createSearchTool`
- Add `IMAGE_GEN_OAI_TIMEOUT_MS` and `IMAGE_GEN_OAI_MAX_RETRIES` for the
  `image_gen_oai` client, applying the timeout to the undici dispatcher too
@codegraph-librechat codegraph-librechat Bot added the 🗺️ Backend Infra codegraph: the taxonomy area this belongs to (classifier, confidence ≥ 0.9) label Oct 9, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🗺️ Backend Infra codegraph: the taxonomy area this belongs to (classifier, confidence ≥ 0.9)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant