feat: add max concurrency limit for guard() chunk/PDF-page analysis - #1164
Open
serhiy-bzhezytskyy wants to merge 2 commits into
Open
serhiy-bzhezytskyy wants to merge 2 commits into
serhiy-bzhezytskyy wants to merge 2 commits into
Conversation
|
@serhiy-bzhezytskyy is attempting to deploy a commit to the Superagent Team on Vercel. A member of the Team first needs to authorize it. |
Contributor License AgreementAll contributors are covered by a CLA. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional
maxConcurrency(TypeScript) /max_concurrency(Python) setting toguard()that bounds how many provider requests run at once when analyzing multiple text chunks or PDF pages. Unset, behavior is unchanged (unbounded, same as today).Fixes #1161
Problem
guard()currently starts one provider request per chunk/page via an unboundedPromise.all()(TypeScript) /asyncio.gather()(Python). A large input can create a large burst of concurrent provider requests, increasing the likelihood of rate limits, connection/memory pressure, and fallback bursts, with no per-call way to apply backpressure.Solution
mapWithConcurrencyin TS,_gather_with_concurrencyin Python) that runs at most N callbacks at once, preserving result order and existing error-propagation/aggregation semantics.chunkSize/chunk_sizevalidation pattern.Test results
Note
Low Risk
Opt-in SDK change with backward-compatible defaults; only throttles parallel provider calls without altering guard classification or aggregation semantics.
Overview
Adds an optional
maxConcurrency/max_concurrencysetting onguard()in the Python and TypeScript SDKs so callers can cap how many provider requests run at once when large text is chunked or PDF pages are analyzed in parallel.Each SDK introduces a bounded-concurrency helper (
mapWithConcurrencyin TS,_gather_with_concurrencyin Python) and wires it into the existing text-chunk and PDF-page paths instead of unboundedPromise.all/asyncio.gather. Unset (or a value ≥ item count) keeps today’s unbounded behavior; invalid values are rejected with the same positive-integer validation pattern aschunkSize.New unit tests assert in-flight call limits, default full concurrency, and validation errors.
Reviewed by Cursor Bugbot for commit 04c693b. Configure here.