fix(service): run compressed upload/download streams off the blocking pool - #3
Closed
walter-zeromatter wants to merge 3 commits into
Closed
fix(service): run compressed upload/download streams off the blocking pool#3walter-zeromatter wants to merge 3 commits into
walter-zeromatter wants to merge 3 commits into
Conversation
… blocking pool A compressed-download encode stream drains at the gRPC client's pace. The current implementation holds one tokio blocking-pool thread per stream for the stream's entire lifetime (BufChannelReader blocks waiting for input and blocking_send parks until the client consumes), so N concurrent downloads with slow consumers occupy N pool threads and every unrelated spawn_blocking user queues behind them. This test wires encode streams the way ByteStreamServer::inner_read_compressed does, holds more of them than the pool has threads, and asserts a trivial unrelated spawn_blocking closure still runs. It fails against the current thread-per-stream implementation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…cking pool stream_encode_compressed_download previously ran under spawn_blocking with a blocking reader over raw_rx and blocking_send into the output channel, so one tokio blocking-pool thread was parked per compressed download for the whole stream at the client's drain rate. Concurrent downloads with slow consumers could exhaust the pool (default cap 512), starving every other spawn_blocking user — filesystem store I/O, upload decode, credential resolution — and the per-thread zstd contexts, stacks, and buffers inflated memory accordingly. The encoder is now an async fn driving zstd's raw streaming API (ZSTD_compressStream via zstd::stream::raw::Encoder): async recv, inline per-chunk encode (CPU bounded by the small channel chunk size), async send for backpressure. No thread is held while waiting on either channel. The wire format is unchanged: one well-formed zstd frame, identical to what zstd::stream::read::Encoder emitted. The bytestream_server caller drops its spawn_blocking wrapper; dropping the read stream still cancels the encode because the future itself is dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ing pool Same defect shape as the download encode path: stream_decode_compressed_upload ran under spawn_blocking, blocking on compressed_rx for client bytes and parking in blocking_send while the store drained, holding one blocking-pool thread per compressed upload for the stream's lifetime. The decoder is now an async fn driving zstd's raw streaming API (ZSTD_decompressStream via zstd::stream::raw::Decoder) with async channel sends. Validation is unchanged: the per-chunk decoded-size cap (bomb rejection), the exact final-size check, and the digest check are preserved, and a stream that ends mid-frame is rejected as InvalidArgument (previously surfaced as a read error from the blocking decoder). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Author
|
Fixed by a separate PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
cargo test -p nativelink-service --test wire_compression_test— 10 passed, including the new hang regression testcargo check -p nativelink-service --tests🤖 Generated with Claude Code