Skip to content

Commit 0bcaafd

Browse files
committed
add compile time benchmarks with two modes and add a chart to it
1 parent 9511cb7 commit 0bcaafd

24 files changed

Lines changed: 876 additions & 228 deletions

.github/workflows/docs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@ jobs:
106106
benchmarks/results/x2py-build.json
107107
docs/user/performance.md
108108
docs/user/assets/performance-comparison.svg
109+
docs/user/assets/build-time-comparison.svg
109110
retention-days: 90
110111

111112
build:

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,7 @@ wrapper flags reached the build:
264264
python3 -m x2py points.f90 \
265265
--out geometry_debug \
266266
--out-dir build/geometry_debug \
267+
--jobs 4 \
267268
--verbose \
268269
--compiler gfortran \
269270
--wrapper-fortran-flags=-O2 \
@@ -272,6 +273,8 @@ python3 -m x2py points.f90 \
272273

273274
The verbose output includes native source compilation, generated bridge
274275
compilation, generated Python binding compilation, and the final link command.
276+
Dependency-ready source files and the generated binding may compile
277+
concurrently; `--jobs 1` selects a serial diagnostic build.
275278
The custom wrapper flags appear in the relevant command lines:
276279

277280
```text

benchmarks/README.md

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@ interfaces.
1111
Build timings treat each tool as a black box. A timed sample starts with an
1212
empty output directory and includes source processing, wrapper generation,
1313
native and generated-source compilation, and linking. Import verification runs
14-
after the timer stops. The workloads are:
14+
after the timer stops. Both tools use their normal dependency-aware compiler
15+
concurrency. The workloads are:
1516

1617
- the one-source, five-procedure module used by the runtime suite; and
1718
- the repository's 155-source reference BLAS fixture, with all 155 routines
@@ -22,6 +23,13 @@ default. Tool order alternates between rounds. Set
2223
`X2PY_BUILD_BENCHMARK_RUNS` or `X2PY_BUILD_BENCHMARK_WARMUPS` to change those
2324
counts for local investigation.
2425

26+
Every workload is measured with two compiler profiles:
27+
28+
- development: `-O0`; and
29+
- optimized: `-O3 -march=native -mtune=native`.
30+
31+
Runtime-call measurements continue to use only the optimized profile.
32+
2533
`X2PY_BENCHMARK_FIRST=x2py` is the default measurement order. Set it to
2634
`f2py` to reverse the order. The publication workflow alternates this setting
2735
between runs so one tool is not systematically measured first.
@@ -32,10 +40,11 @@ Run the complete correctness check and rigorous benchmark with:
3240
bash run.sh
3341
```
3442

35-
The script rebuilds both runtime extensions, runs the clean build-time suite,
36-
and applies `-O3 -march=native -mtune=native` to the native Fortran source,
37-
generated Fortran wrapper, and generated C binding. It retains f2py's generated
38-
sources under `build/f2py` for local inspection.
43+
The script rebuilds both runtime extensions, runs both clean-build profiles,
44+
and applies each profile consistently to the native Fortran source, generated
45+
Fortran wrapper, and generated C binding. Runtime extensions use the optimized
46+
profile. It retains f2py's generated sources under `build/f2py` for local
47+
inspection.
3948

4049
To compare existing results without rebuilding:
4150

@@ -73,8 +82,8 @@ Run this command from the repository root. It reads the runtime and build-time
7382
`pyperf` pairs, checks that each pair contains the same benchmarks and compatible
7483
platform metadata, records the host operating-system distribution and compiler,
7584
and updates only the marked result sections in `docs/user/performance.md` plus
76-
`docs/user/assets/performance-comparison.svg`. Explanatory prose and the
77-
reproduction instructions remain hand-maintained.
85+
the runtime and clean-build comparison SVGs in `docs/user/assets/`.
86+
Explanatory prose and the reproduction instructions remain hand-maintained.
7887

7988
The Documentation workflow performs the same generation after successful
8089
correctness checks and rigorous measurements on pushes to `main`. It keeps the

0 commit comments

Comments
 (0)