[None][chore] Add NVTX ranges to per-iteration ADP sync points in PyExecutor#16422
[None][chore] Add NVTX ranges to per-iteration ADP sync points in PyExecutor#16422lancelly wants to merge 1 commit into
Conversation
…xecutor _handle_disagg_cache_errors_synced (tp_allgather error vote at the top of every executor iteration) and _can_queue (post-schedule batch-size tp_allgather under attention DP) are blocking host-side collectives, but unlike the sibling disagg status checks they carry no NVTX range, so they show up as anonymous gaps in nsys traces. Annotate both for profiling visibility. No functional change. Signed-off-by: Liao Lanyu <108499334+lancelly@users.noreply.github.com>
|
/bot run --disable-fail-fast |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds NVTX range decorators to ChangesPyExecutor instrumentation
Estimated code review effort: 1 (Trivial) | ~2 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
PR_Github #59441 [ run ] triggered by Bot. Commit: |
|
PR_Github #59441 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #59899 [ run ] triggered by Bot. Commit: |
|
PR_Github #59899 [ run ] completed with state
|
Description
_handle_disagg_cache_errors_synced(atp_allgathererror vote at the top of every executor iteration when disagg + attention DP are enabled) and_can_queue(a post-schedule batch-sizetp_allgatherunder attention DP) are blocking host-side collectives on the executor loop's critical path.Unlike the sibling disagg status checks (
_check_disagg_gen_transfer_status,_check_disagg_ctx_schedulable_status,_check_disagg_*_cache_transfer_status,_pad_attention_dp_dummy_request,_fetch_new_requests), they carry no NVTX annotation, so in nsys traces they show up as anonymous gaps — one at the top of each iteration and one between scheduling and forward — and are easy to misread as scheduler overhead or idle host time.This PR adds
@nvtx_rangeto both, matching the existing style in the file. No functional change.Summary by CodeRabbit