Repository navigation
fix(perf): generate HTML report for open-loop runs with no fixed rate (rate=-1.0) - #1503
Merged
Yunnglin merged 2 commits intoJul 22, 2026
Merged
Conversation
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
…enerated Open-loop runs that don't pin a fixed request rate write the run directory as ``rate_-1.0_number_<M>`` (``-1.0`` is the Request Rate sentinel when rate is unset, e.g. workload_trace replay driven by trace arrival offsets). The loader regex had no sign in its character class, so the directory never matched and ``RunLoader.load_all`` dropped it — ``gen_perf_html_report`` then saw zero runs and logged "No benchmark runs found ... Skipping HTML report", silently not producing ``perf_report.html``. Allow an optional leading sign on the rate capture group. Verified by re-running a workload_trace open-loop (rate=-1.0) run against a vLLM endpoint: perf_report.html is now generated (previously skipped). Co-Authored-By: Claude <noreply@anthropic.com>
qiumuyang
force-pushed
the
fix/open-loop-rate-html-report
branch
from
July 22, 2026 06:05
7d7291d to
cb6c0d8
Compare
Follow-up to the rate=-1.0 loader fix. RunData.name and perf_charts._x_axis re-parsed the run directory name with their own sign-less regexes, so an open-loop run with no fixed rate (rate_-1.0_number_M) was still mislabeled as "Parallel 0" and its sweep charts used a "Concurrency" x-axis. - perf_charts._x_axis: detect open-loop via RunData.is_open_loop; drop _RATE_DIR_RE - RunData.name: derive label from is_open_loop/rate; render the rate<0 sentinel as "Open-loop / Number M" instead of a meaningless "Rate -1.0 rps" - add network-free unit test tests/perf/test_perf_report_loader.py
Collaborator
|
Reproduced and verified. Pushed a small follow-up (70ce1c5): derive open-loop from |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Open-loop perf runs that don't pin a fixed request rate produce no
perf_report.html— it's silently skipped, even thoughperformance_summary.txt,benchmark_summary.json,benchmark_percentile.json, andbenchmark_data.dbare all written correctly.For example,
workload_tracereplay (driven by trace arrival offsets, no--rate) ends the run with:Root cause
When rate is unset, the benchmark records
Request Rate = -1.0(sentinel) and writes the run sub-directory asrate_-1.0_number_<M>. The loader regex has no sign in its character class, so the directory never matches andRunLoader.load_all()drops it:With zero runs returned,
gen_perf_html_report()logs "No benchmark runs found ... Skipping HTML report" and returns''.Fix
Allow an optional leading sign on the rate capture group:
Positive-rate run dirs (
rate_1.5_number_4) still match;parallel_*is unaffected.Verification
Re-ran a
workload_traceopen-loop run (rate=-1.0, 30 reqs) against a vLLM endpoint (--enable-prompt-tokens-details):WARNING: No benchmark runs found ... Skipping HTML report.— noperf_report.html.HTML report generated: .../perf_report.html(165 KB interactive report, 26/30 success; 4 failures are unrelatedServerDisconnectedError).Run dir confirmed named
rate_-1.0_number_30— i.e. the negative-rate path that previously failed discovery.🤖 Generated with Claude Code