docs(speed-bench): add generated benchmark summary#413
Open
dutifulbob wants to merge 11 commits into
Open
Conversation
|
Feel free to refactor/reimplement I thought it would be cool to have a single place to see all the perf results |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opened on behalf of Onur Solmaz (
osolmaz).Summary
Speed benchmark CSVs are easy to add but hard to compare at a glance.
This change adds a generated summary table to
speed-bench/README.md.It also adds a small Python updater so the table can be rebuilt whenever CSV files change.
What Changed
The speed bench folder now has one command that rebuilds the summary from the checked-in CSV files.
The generated README block has explicit begin and end markers so future updates replace only that section.
speed-bench/update_summary.pyusing only the Python standard library.speed-bench/README.md.utf-8-sig.Testing
I tested the updater directly and checked that the generated Markdown is clean.
This is a documentation/script-only change; I did not run model inference benchmarks.
python3 speed-bench/update_summary.pyPYTHONDONTWRITEBYTECODE=1 python3 -m py_compile speed-bench/update_summary.pygit diff --check -- speed-bench/README.md speed-bench/update_summary.pyMachine/backend/model quant: not applicable for this docs-only change.
Notable failures: none.
Risks
The risk is low because this does not touch inference code or benchmark CSV data.
The main limitation is that the summary only reflects CSV files in
speed-bench/, not one-off numbers in the top-level README.