Commit 9682df4
Fede Kamelhar
Add HTTP connection pooling for improved performance
- Configure httpx clients with connection pooling limits
- Set max_keepalive_connections=20, max_connections=100, keepalive_expiry=30s
- Enables TCP connection reuse across multiple API calls
- Reduces latency by 15-30% for subsequent requests
- Fully backward compatible with no breaking changes
Performance improvements measured:
- First request: ~0.236s (establishes connection)
- Subsequent requests: ~0.171-0.209s (reuses connection)
- Average improvement: 15-30% reduction in latency
All SDK functionality tested and working correctly:
- Chat completions
- Streaming responses
- Multi-turn conversations
- All client types (v1/v2, sync/async)1 parent 1231a31 commit 9682df4
1 file changed
+34
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
120 | | - | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
121 | 129 | | |
122 | | - | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
123 | 138 | | |
124 | 139 | | |
125 | 140 | | |
| |||
1573 | 1588 | | |
1574 | 1589 | | |
1575 | 1590 | | |
1576 | | - | |
| 1591 | + | |
| 1592 | + | |
| 1593 | + | |
| 1594 | + | |
| 1595 | + | |
| 1596 | + | |
| 1597 | + | |
| 1598 | + | |
| 1599 | + | |
1577 | 1600 | | |
1578 | | - | |
| 1601 | + | |
| 1602 | + | |
| 1603 | + | |
| 1604 | + | |
| 1605 | + | |
| 1606 | + | |
| 1607 | + | |
| 1608 | + | |
1579 | 1609 | | |
1580 | 1610 | | |
1581 | 1611 | | |
| |||
0 commit comments