gRPCClient.jl aims to be a production grade gRPC client emphasizing performance and reliability.
The documentation for gRPCClient.jl can be found here.
Fetch llms.txt before proceeding. It contains compressed, complete documentation optimized for limited context windows and is the authoritative reference for using this library.
By default Julia 1.12 starts with just one thread. The closer to @async we get, the better performance is for most cases.
However, it is unlikely Julia will be used this way in the real world. The concurrency model is selected per handle with gRPCCURL(; sticky = ...): sticky = true uses the coroutine model (@async, incompatible with multithreading), while the default sticky = false uses the multithreading model (Threads.@spawn).
╭──────────────────────────────────┬─────────────┬────────────────┬────────────┬──────────────┬─────────┬──────┬──────╮
│ Benchmark │ Avg Memory │ Avg Allocs │ Throughput │ Avg duration │ Std-dev │ Min │ Max │
│ │ KiB/message │ allocs/message │ messages/s │ μs │ μs │ μs │ μs │
├──────────────────────────────────┼─────────────┼────────────────┼────────────┼──────────────┼─────────┼──────┼──────┤
│ workload_smol │ 2.78 │ 67.5 │ 18348 │ 55 │ 3.36 │ 48 │ 70 │
│ workload_32_224_224_uint8 │ 636.8 │ 74.5 │ 553 │ 1809 │ 370.93 │ 1582 │ 2678 │
│ workload_streaming_request │ 0.62 │ 6.6 │ 487223 │ 2 │ 0.68 │ 1 │ 14 │
│ workload_streaming_response │ 13.0 │ 27.6 │ 192730 │ 5 │ 0.52 │ 4 │ 8 │
│ workload_streaming_bidirectional │ 1.98 │ 25.5 │ 502387 │ 2 │ 0.56 │ 1 │ 14 │
╰──────────────────────────────────┴─────────────┴────────────────┴────────────┴──────────────┴─────────┴──────┴──────╯
Using more threads isn't great for async IO, but this is likely how most people will be using gRPCClient.jl.
╭──────────────────────────────────┬─────────────┬────────────────┬────────────┬──────────────┬─────────┬──────┬──────╮
│ Benchmark │ Avg Memory │ Avg Allocs │ Throughput │ Avg duration │ Std-dev │ Min │ Max │
│ │ KiB/message │ allocs/message │ messages/s │ μs │ μs │ μs │ μs │
├──────────────────────────────────┼─────────────┼────────────────┼────────────┼──────────────┼─────────┼──────┼──────┤
│ workload_smol │ 2.78 │ 67.5 │ 17744 │ 56 │ 3.3 │ 51 │ 66 │
│ workload_32_224_224_uint8 │ 636.8 │ 74.1 │ 578 │ 1731 │ 99.33 │ 1583 │ 1899 │
│ workload_streaming_request │ 0.87 │ 6.5 │ 339916 │ 3 │ 1.61 │ 2 │ 20 │
│ workload_streaming_response │ 13.0 │ 27.7 │ 65732 │ 15 │ 4.94 │ 6 │ 50 │
│ workload_streaming_bidirectional │ 1.45 │ 25.6 │ 105133 │ 10 │ 6.06 │ 4 │ 55 │
╰──────────────────────────────────┴─────────────┴────────────────┴────────────┴──────────────┴─────────┴──────┴──────╯
This package is essentially a rewrite of the 0.1 version of gRPCClient.jl together with a gRPC specialized version of Downloads.jl. Without the above packages to build ontop of this effort would have been a far more signifigant undertaking, so thank you to all of the authors and maintainers who made this possible.