perf(timing): high-resolution rate-loop pacing for exact rate delivery - #1185
perf(timing): high-resolution rate-loop pacing for exact rate delivery#1185ajcasagrande wants to merge 13 commits into
Conversation
The request-rate loop slept via the event loop's timer wheel, which quantizes sub-millisecond waits to ~1ms under uvloop/libuv. At high rates (e.g. 200us intervals at 5,000 req/s) every sleep overshoots, and the unconditional reset-to-now policy forfeited the overshot schedule on every tick -> silent under-delivery (measured ~3,569 against a 5,000 target). - Add high_res_timer.py: TimerFdPacer (Linux timerfd, kernel hrtimer, observed via loop.add_reader) and ThreadPacer (cross-platform dedicated sleep thread waking the loop via call_soon_threadsafe). Both expose async sleep_until(deadline) with ~50us precision, gated on AIPERF_TIMING_HIGH_RES_TIMER (default on). - Bounded catch-up window (AIPERF_TIMING_MAX_CATCHUP_SECONDS, default 0.01): re-anchor to now only once the backlog exceeds the window, so sub-ms oversleeps stay on the original schedule while a genuine multi-second stall still re-anchors instead of firing a burst storm. - Use the pacer only for short sleeps (<= RATE_RAMP_UPDATE_INTERVAL); longer sleeps keep the event-wait so mid-sleep rate updates (ramping, set_request_rate) still wake promptly, where ~1ms precision is moot. Ported verbatim from prior 5k-QPS work; re-implemented against main's request-rate loop. Env-var docs regenerated. Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>
Try out this PRQuick install: pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@0d28a334bb73d2da01249d83c37f54fe9d732b11Recommended with virtual environment (using uv): uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@0d28a334bb73d2da01249d83c37f54fe9d732b11Last updated for commit: |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe request-rate strategy now supports configurable high-resolution pacing and bounded schedule catch-up. Linux ChangesHigh-Resolution Request Pacing
Estimated code review effort: 4 (Complex) | ~45 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
tests/unit/timing/test_high_res_timer.py (1)
13-13: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMove pacer imports to module top.
Each test imports
ThreadPacer/TimerFdPacerlocally (Lines 13, 31, 42, 55, 73, 84).high_res_timeris pure stdlib/ctypesand imports cleanly on every platform, so these can be hoisted to the top of the file.As per coding guidelines: "keep imports at the top".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/unit/timing/test_high_res_timer.py` at line 13, Move the local ThreadPacer and TimerFdPacer imports out of the individual tests and place them with the module-level imports at the top of the test file. Remove the now-redundant in-test imports while preserving each test’s existing behavior.Source: Coding guidelines
src/aiperf/timing/strategies/request_rate.py (1)
215-235: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueOptional: pacer
sleep_untilfailures aren't recovered at runtime.Creation falls back gracefully, but
TimerFdPacer.sleep_untilcan still raiseOSError(fromtimerfd_settime) mid-loop, which propagates out and aborts the phase with no fallback to event-loop waiting. It's a low-probability path once the fd exists, but if you want the phase to survive a transient timer failure, catch it here, dropself._pacertoNone, and continue with the event-wait branch.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiperf/timing/strategies/request_rate.py` around lines 215 - 235, Update the `_pacer.sleep_until` path in the request-rate loop to catch `OSError` from runtime timer failures, set `self._pacer` to `None`, and continue into the existing `_rate_update_event` waiting branch so the phase can proceed without the pacer.src/aiperf/timing/high_res_timer.py (1)
65-97: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueDeclare ctypes signatures for the timerfd calls
argtypes/restypewould make thelibccontract explicit and catch mismatched arguments sooner, especially for thetimerfd_settimepointer parameter.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiperf/timing/high_res_timer.py` around lines 65 - 97, Define explicit ctypes argtypes and restype for libc.timerfd_create and libc.timerfd_settime immediately after loading libc in the timer initialization code. Ensure the declarations match each function’s integer arguments, pointer parameter, and return type, while leaving _on_readable and sleep_until behavior unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/unit/timing/test_high_res_timer.py`:
- Line 26: Make the median wake-error assertions in the high-resolution timer
tests deterministic by removing or gating the host-load-sensitive thresholds at
the assertions for errors_us and TimerFdPacer. Preserve the min(errors_us) >=
0.0 invariants, and use a wider bound or the repository’s existing opt-in
performance-test mechanism rather than enforcing fixed latency limits in normal
CI.
---
Nitpick comments:
In `@src/aiperf/timing/high_res_timer.py`:
- Around line 65-97: Define explicit ctypes argtypes and restype for
libc.timerfd_create and libc.timerfd_settime immediately after loading libc in
the timer initialization code. Ensure the declarations match each function’s
integer arguments, pointer parameter, and return type, while leaving
_on_readable and sleep_until behavior unchanged.
In `@src/aiperf/timing/strategies/request_rate.py`:
- Around line 215-235: Update the `_pacer.sleep_until` path in the request-rate
loop to catch `OSError` from runtime timer failures, set `self._pacer` to
`None`, and continue into the existing `_rate_update_event` waiting branch so
the phase can proceed without the pacer.
In `@tests/unit/timing/test_high_res_timer.py`:
- Line 13: Move the local ThreadPacer and TimerFdPacer imports out of the
individual tests and place them with the module-level imports at the top of the
test file. Remove the now-redundant in-test imports while preserving each test’s
existing behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e52213b2-a2d6-436c-8496-435c8d267474
📒 Files selected for processing (6)
docs/environment-variables.mdsrc/aiperf/common/environment.pysrc/aiperf/timing/high_res_timer.pysrc/aiperf/timing/strategies/request_rate.pytests/unit/timing/strategies/test_request_rate.pytests/unit/timing/test_high_res_timer.py
Cross-platform pacing (macOS / Windows)The high-res pacer is not Linux-only. The selection chain is timerfd (Linux) →
Cross-OS validationThe portability of the underlying technique (absolute deadlines beat relative sleeps regardless of timer quality) was validated in a standalone 3-target CI harness across Linux, macOS, and Windows. Cumulative schedule drift at 50k req/s, constant arrivals:
Takeaway: the schedule model (absolute vs relative) dominates timer quality. Windows' high-res timer is ~18× better than the wheel, yet relative gaps still drift to ~1.1s because they accumulate; switching the same timer to absolute deadlines holds 0.5ms. Absolute deadlines also tame the pathological macOS-VM timer for free (~1.6ms total drift over the whole run, never accumulating, zero extra CPU). This is exactly what Harness + full results: https://github.com/ajcasagrande/timer-lab |
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/aiperf/timing/strategies/request_rate.py`:
- Around line 170-190: Update ThreadPacer’s worker loop, including _run and
sleep_until, so cancellation interrupts any active wait and stale deadlines
cannot signal _tick after the pacer is reused; preserve correct signaling for
the current deadline. Add a regression test that cancels/replaces a pending wait
and verifies the subsequent sleep_until does not return before its new deadline.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 07881d60-e783-48f3-b4cd-f03cfe846b28
📒 Files selected for processing (4)
src/aiperf/timing/high_res_timer.pysrc/aiperf/timing/strategies/request_rate.pytests/unit/timing/strategies/test_request_rate.pytests/unit/timing/test_high_res_timer.py
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/unit/timing/test_high_res_timer.py
- src/aiperf/timing/high_res_timer.py
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/unit/timing/test_high_res_timer.py`:
- Around line 63-65: Update the cleanup in the timing test around pacer.close()
and pacer._thread.join() so the blocking worker join runs outside the event-loop
thread via an awaitable async mechanism. After cleanup, assert that the worker
thread has terminated, while preserving the existing finally-based resource
cleanup.
- Around line 49-51: Update the cleanup block around first_sleep.cancel() to
assert that awaiting first_sleep raises asyncio.CancelledError rather than
suppressing it, ensuring the test fails if cancellation does not occur.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 565e6190-8c52-4a55-921f-8ec525cfc32b
📒 Files selected for processing (2)
src/aiperf/timing/high_res_timer.pytests/unit/timing/test_high_res_timer.py
🚧 Files skipped from review as they are similar to previous changes (1)
- src/aiperf/timing/high_res_timer.py
Address review feedback on the ThreadPacer tests: - Require the cancelled sleep to actually raise CancelledError instead of suppressing it, so the test fails if cancellation never happens. - Join the pacer worker via asyncio.to_thread so the blocking join runs off the event-loop thread, and assert the worker actually terminated. - Document the generation-gating protocol in _run/_set_tick_if_current. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>
test_cancelled_sleep_does_not_wake_replacement_early failed on CI (linux/3.12) with "pacer worker did not start waiting". The autouse no_sleep fixture in tests/unit/conftest.py rewrites asyncio.sleep to a bare event-loop yield, so the spin loop waiting for the worker thread never released the GIL and never let that thread be scheduled. All 100 iterations completed in microseconds and the test failed. It passed on other Python versions only by scheduling luck. Wait for the worker, and for the elapsed deadline, in a worker thread via asyncio.to_thread so a real time.sleep hands over the interpreter without blocking the event loop. Deadlines widened to leave headroom on loaded runners. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>
ajcasagrande
changed the title
debermudez
left a comment
debermudez
left a comment
There was a problem hiding this comment.
Overall assessment: High-quality, thoroughly-benchmarked performance work. The root cause (event-loop timer quantization + unconditional catch-up reset) is correctly identified and cleanly fixed, and the benchmark evidence — on-box A/B, hardware profiling, Poisson arrival fidelity, and the c4-144 session arc (3,569 → 4,330 → 5,000 req/s) — is exceptional. One correctness concern in the cleanup path and one coverage gap are worth addressing before merge.
Fix order:
- (Recommended) Shield the cleanup gather in
_wait_for_pacer_or_rate_update— see inline comment. - (Optional) Add a test for
execute_phasecancellation during active pacing.
What's working well: ThreadPacer generation tracking for cancellation safety is elegant; test_cancelled_sleep_does_not_wake_replacement_early is a solid negative test; the short-sleep gate (sleep_duration <= RATE_RAMP_UPDATE_INTERVAL) correctly routes long sleeps to the existing asyncio.wait_for path, preserving mid-sleep rate-update behavior for ramps; the fallback chain and IS_LINUX usage are correct; the bounded catch-up window alone recovering ~10% under-delivery is a great isolated proof of the two independent mechanisms.
| if not task.done(): | ||
| task.cancel() | ||
| await asyncio.gather(pacer_task, rate_update_task, return_exceptions=True) | ||
|
|
There was a problem hiding this comment.
Finally-block gather not shielded from re-cancellation. When execute_phase is cancelled (benchmark abort, timeout) while inside _wait_for_pacer_or_rate_update, CancelledError is raised at asyncio.wait. In the common single-cancel case (Python 3.11), the cancel counter is consumed before the finally block's first await, so the gather runs safely. But with multiple concurrent Task.cancel() calls (Python 3.12 eager cancellation, or an external shutdown racing a first cancel), the gather itself can be interrupted before both tasks are awaited. This leaves _tick.wait() inside the pacer's sleep_until pending, producing "Task was destroyed but it is pending!" warnings and potentially leaving the tick event stale for a future sleep_until caller.
Minimal fix:
await asyncio.shield(
asyncio.gather(pacer_task, rate_update_task, return_exceptions=True)
)asyncio.shield lets the gather complete even if the outer task has a pending cancellation, then re-raises CancelledError after cleanup. A follow-up test cancelling execute_phase mid-sleep and asserting the pacer is closed without warnings would cement the guarantee.
There was a problem hiding this comment.
Addressed in eda2b86 — the cleanup gather in _wait_for_pacer_or_rate_update is now wrapped in asyncio.shield, so a re-cancellation cannot interrupt the drain and orphan the pacer's in-flight sleep_until (stale tick event / "Task was destroyed but it is pending!").
Also added the follow-up test you suggested — test_execute_phase_cancelled_mid_pace_drains_pacer_and_closes cancels execute_phase mid-pace and asserts CancelledError propagates, the pacer sleep is drained rather than abandoned, and the pacer is closed. One caveat worth recording: the test cancels twice, but it does not by itself discriminate the shielded gather from the unshielded one. CPython's Task.__step clears _must_cancel when the pending exception is already CancelledError, so a synchronous double-cancel gets absorbed before the finally await; I verified this with a standalone probe across both variants. The test covers the cancellation path (your item 2); the shield is the invariant it protects.
tests/unit/timing is green: 1309 passed.
A second Task.cancel() on execute_phase (eager cancellation on 3.12, or an external shutdown racing a first cancel) sets _must_cancel and interrupts the finally-block gather in _wait_for_pacer_or_rate_update before both child tasks are drained. That leaves the pacer's _tick.wait() pending -- 'Task was destroyed but it is pending!' warnings, and a stale tick event for the next sleep_until caller. Shield the gather so the drain completes regardless. Adds a regression test that cancels execute_phase mid-pace (twice) and asserts the pacer sleep is drained and the pacer closed.
The 1ms interval let setup overhead push the first deadline into the past, so _wait_until_next_target took the sleep_duration <= 0 branch, the pacer was never used, and the test deadlocked on pacer.started.wait(). Use a 50ms interval (well inside RATE_RAMP_UPDATE_INTERVAL) and bound the start wait with wait_for. Also trims the docstring: the repeat cancel documents intent, but CPython clears _must_cancel when the pending exception is already CancelledError, so the test does not by itself discriminate the shielded gather.
Review addressedAddressed 1 of 6 review threads. Fixed:
Skipped (5 threads):
Verification: Note on the new test: it cancels 🤖 Generated with Claude Code |
| return False | ||
|
|
||
| pacer_task = asyncio.create_task(self._pacer.sleep_until(deadline_perf_s)) | ||
| rate_update_task = asyncio.create_task(self._rate_update_event.wait()) |
There was a problem hiding this comment.
Two create_task allocations on every tick
rate_update_task is created unconditionally on every sleep, even though execute_phase already filters the is_set() case before entering here. At 5,000 req/s this is ~10,000 task allocations per second. A fast-path guard at the top of this method (or before creating rate_update_task) would drop that cost in the common case:
if self._rate_update_event.is_set():
return TrueThere was a problem hiding this comment.
Addressed in 966d5e6: added a fast-path is_set() check at the top of _wait_for_pacer_or_rate_update that returns True immediately when a rate update is already pending, avoiding both create_task calls and the asyncio.wait machinery in that case.
| os.read(self._fd, 8) # drain the expiration count | ||
| self._tick.set() | ||
|
|
||
| async def sleep_until(self, deadline_perf_s: float) -> None: |
There was a problem hiding this comment.
Single-waiter contract is advisory, not enforced
The class docstring says "supports one waiter at a time" but nothing prevents a second coroutine from calling sleep_until concurrently. Both would share _tick and both would wake at the first deadline. Since TimerFdPacer is only ever used by the rate loop (a single coroutine), this is safe today, but an assert or a lock would make the invariant explicit and catch future misuse at the call site rather than silently at the wrong wakeup time.
There was a problem hiding this comment.
Addressed in 1eb5c69: added an _active_waiter flag to TimerFdPacer (and ThreadPacer, which has the same contract) with an assert not self._active_waiter at the top of sleep_until and a try/finally to clear it. Concurrent mis-use now raises AssertionError instead of silently firing both waiters at the first deadline.
| Waits on the condition rather than ``time.sleep`` so a cancellation or | ||
| a replacement deadline interrupts the current wait immediately. | ||
| """ | ||
| while True: |
There was a problem hiding this comment.
while/else semantics aren't obvious to most Python readers
The outer while True with an inner while not self._closed: and else: return is correct but the Python while/else idiom (the else runs only when the condition becomes false, not on break) is obscure enough that it trips reviewers. A short comment on the else line — something like # _closed became True: exit the thread — would save the next reader from having to verify it by hand.
There was a problem hiding this comment.
Addressed in 1eb5c69: replaced else: return with if self._closed: return. The semantics are identical but the while/else idiom (else fires when the condition goes false, not on break) is non-obvious enough to trip reviewers, and the explicit guard makes the exit condition clear without needing to know the idiom.
|
Documentation gap: The PR adds env var entries in |
At 5,000 req/s the old code allocated ~10,000 asyncio.Task objects per second unconditionally. A fast-path `is_set()` check at the top of `_wait_for_pacer_or_rate_update` returns immediately when a rate update is already pending, avoiding both task allocations and the asyncio.wait machinery in the common case. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add an `_active_waiter` flag to both TimerFdPacer and ThreadPacer with an assertion in `sleep_until`, surfacing mis-use (two concurrent waiters sharing _tick / _generation) instead of silently misfiring. Replace the `while/else` idiom in ThreadPacer._run with an explicit `if self._closed: return` check. The semantics are identical but the while/else pattern (else runs only when the condition becomes false, not on break) is obscure enough that it trips code reviewers. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
timing-modes-reference.md had no mention of the pacer, the catch-up window, or the AIPERF_TIMING_HIGH_RES_TIMER / MAX_CATCHUP_SECONDS env vars. Add a "High-Resolution Rate Pacing" section covering: - platform selection (timerfd on Linux, ThreadPacer elsewhere) - the two pacing env vars with their defaults and tuning guidance - a step-by-step checklist for diagnosing under-delivery at high QPS Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
for more information, see https://pre-commit.ci
|
Docs addressed in f257731: added a 'High-Resolution Rate Pacing' section to timing-modes-reference.md covering timerfd/ThreadPacer platform selection, both env vars (AIPERF_TIMING_HIGH_RES_TIMER and AIPERF_TIMING_MAX_CATCHUP_SECONDS) with defaults and tuning guidance, and a step-by-step checklist for diagnosing under-delivery at high QPS. |
Review addressedAddressed 4 of 4 review threads. Fixed:
Skipped (0 threads): none 🤖 Generated with Claude Code |
3 participants
What changed
The request-rate loop slept via the event loop's timer wheel, which quantizes sub-millisecond waits to ~1ms under uvloop/libuv. At high rates (e.g. 200us intervals at 5,000 req/s) every sleep overshoots, and the unconditional reset-to-now policy forfeited the overshot schedule on every tick — silent under-delivery.
high_res_timer.py—TimerFdPacer(Linuxtimerfd, kernel hrtimer, observed vialoop.add_reader— the same fd path ZMQ uses) andThreadPacer(cross-platform dedicatedtime.sleepthread waking the loop viacall_soon_threadsafe). Both exposeasync sleep_until(deadline)with ~50us precision. Selected viaAIPERF_TIMING_HIGH_RES_TIMER(default on): timerfd on Linux, else the sleep thread, else event-loop timers.AIPERF_TIMING_MAX_CATCHUP_SECONDS, default0.01) — re-anchor tonowonly once the backlog exceeds the window, so sub-ms oversleeps stay on the original schedule; a genuine multi-second stall still re-anchors instead of firing a burst storm.<= RATE_RAMP_UPDATE_INTERVAL); longer sleeps keep the event-wait so mid-sleep rate updates (ramping,set_request_rate) still wake promptly — where ~1ms precision is irrelevant anyway. This preserves existing dynamic-rate behavior exactly.Why (mechanism)
The open-loop rate pacer was implicitly riding per-message socket wakeups for sub-ms ticks. Under an efficient batched transport the loop instead parks in
epoll_waitand only wakes on the ~1ms OS timer floor; the issue-gap histogram shifts right and reset-to-now forfeits every slightly-late issue. The pacer gives the rate loop its own kernel-hrtimer clock (~50us wakeups) independent of transport activity, and the bounded catch-up window stops sub-ms oversleeps from permanently forfeiting schedule.Evidence
On-box A/B, this change vs
origin/main— zero-latency Rustaiperf-mock-server, constant arrival, OSL=1, 30k requests, 5,000 req/s target:origin/main(reset-to-now + event timers)Reproducible ~10% under-delivery on
main; the bounded catch-up window alone recovers it, and the timerfd pacer holds target with no regression. All runs clean, 0 errors, sub-ms TTFT.c4-standard-144 (the 144-vCPU instance type used in the arXiv inference-benchmarking paper), 5,000 req/s open-loop against
llm-d-inference-sim, 144 workers:0.11.0: 3,776.8 req/s — barely scales with added cores (timing-bound, not CPU-bound).Supporting artifacts (py-spy
TimingManagerprofiles for the good/regressed/fixed builds, and per-arrival-pattern wiregap inter-arrival captures) are archived underbench-bias-repro/results-c4/(pyspy_{good,bad,wt}_tm.speedscope,wiregap_*.json).Validation
tests/unit/timing/test_high_res_timer.py(both pacers honor an absolute deadline within tolerance, past/early deadlines return promptly,close()idempotent) and pacer-factory tests intest_request_rate.py. Existingtest_request_rate_update_wakes_pending_sleepstill passes — mid-sleep rate updates wake promptly with the pacer on.AIPERF_TIMING_HIGH_RES_TIMER/AIPERF_TIMING_MAX_CATCHUP_SECONDSdocumented via regenerateddocs/environment-variables.md.Notes
Re-implemented against main's request-rate loop (the source branch's timing subsystem had diverged wholesale, so this is not a cherry-pick). The
high_res_timer.pymodule is ported verbatim.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes