Summary
Add a performance-benchmarking setup so we can measure each module/submodule's throughput, catch regressions, and quantify optimizations. Include a storage/comparison mechanism to track results over time.
Motivation
jsfeatNext is a real-time CV library (WebARKit) where per-frame cost matters, but we currently have no performance signal — only correctness (the 63-test parity suite). We can't tell if a refactor slowed gaussian_blur down, or by how much a future optimization helps. The #41 shared-cache change, for example, was argued on memory grounds with no throughput numbers to back it. A bench suite closes that gap.
Proposed approach
Harness: Vitest's built-in bench() (tinybench under the hood).
Already available — the repo runs Vitest 4.1.10. No new major dependency, reuses the existing config, TS setup, and the vendored jsfeat oracle. Convention: colocate *.bench.ts next to the code (or under bench/), run via a new "bench": "vitest bench" script.
Coverage: per module AND per submodule (as requested). One bench file per algorithm module, with separate cases for hot submodules/paths, e.g.:
imgproc → grayscale, gaussian_blur (+ convol.ts u8 vs f32 paths), resample (+ resample.ts u8 fast path vs float path), sobel/scharr_derivatives, canny, warp_perspective/affine
orb → describe (+ bit_pattern_31.ts, rectify_patch.ts)
yape / yape06 → detect (+ yape_utils.ts)
fast_corners.detect, optical_flow_lk.track, linalg (svd/lu/cholesky/eigenVV), matmath, math (qsort/gaussian kernel), transform, motion_estimator (ransac/lmeds), the cache pool
- Optionally: jsfeatNext vs the vendored jsfeat oracle side-by-side, to track parity of performance, not just output.
Deterministic inputs: reuse the synthetic deterministic image/matrix generators already in tests/ (fixed test patterns, seeded PRNGs) so runs are stable and comparable across machines/commits.
Storage & comparison mechanism:
vitest bench --outputJson bench/results/<name>.json to emit structured results (hz, mean, p99, samples).
- Commit a baseline JSON snapshot; a small script (or Vitest's
--compare) diffs a new run against it and flags regressions beyond a threshold (e.g. >10% slower).
- Later: optionally wire into CI as a non-blocking informational job (bench numbers on shared runners are noisy — report, don't gate), and/or keep a
bench/HISTORY.md or append-only JSON log to chart trends across releases.
Scope / phasing (incremental, one module per PR — same as #47)
- Harness +
bench script + shared deterministic-input helpers + one exemplar module (imgproc) end-to-end, including the JSON output + baseline compare script.
- Fill in remaining modules/submodules, one PR each, parity suite staying green.
- (Optional, later) CI integration + historical storage.
Impact
Low/Medium — additive, no runtime or public-API change; new dev tooling only. High long-term value: makes performance a tracked, first-class property before/through the 1.0 line.
Open questions
- Storage: commit baseline JSON in-repo (simple, versioned, diff-able) vs an external store/dashboard (richer, more setup)? Leaning in-repo to start.
- Include the jsfeat oracle as a comparison baseline in each bench, or bench jsfeatNext alone? (Oracle comparison is more insightful but ~doubles run time.)
- Regression threshold + whether CI should ever gate on it (recommend: never gate; informational only, given runner noise).
Non-goals
- Not micro-optimizing anything in this issue — this is measurement infrastructure only.
- Not a browser/rendering benchmark (that's app-level; this is library-level throughput).
Summary
Add a performance-benchmarking setup so we can measure each module/submodule's throughput, catch regressions, and quantify optimizations. Include a storage/comparison mechanism to track results over time.
Motivation
jsfeatNext is a real-time CV library (WebARKit) where per-frame cost matters, but we currently have no performance signal — only correctness (the 63-test parity suite). We can't tell if a refactor slowed
gaussian_blurdown, or by how much a future optimization helps. The #41 shared-cache change, for example, was argued on memory grounds with no throughput numbers to back it. A bench suite closes that gap.Proposed approach
Harness: Vitest's built-in
bench()(tinybench under the hood).Already available — the repo runs Vitest 4.1.10. No new major dependency, reuses the existing config, TS setup, and the vendored jsfeat oracle. Convention: colocate
*.bench.tsnext to the code (or underbench/), run via a new"bench": "vitest bench"script.Coverage: per module AND per submodule (as requested). One bench file per algorithm module, with separate cases for hot submodules/paths, e.g.:
imgproc→grayscale,gaussian_blur(+convol.tsu8 vs f32 paths),resample(+resample.tsu8 fast path vs float path),sobel/scharr_derivatives,canny,warp_perspective/affineorb→describe(+bit_pattern_31.ts,rectify_patch.ts)yape/yape06→detect(+yape_utils.ts)fast_corners.detect,optical_flow_lk.track,linalg(svd/lu/cholesky/eigenVV),matmath,math(qsort/gaussian kernel),transform,motion_estimator(ransac/lmeds), thecachepoolDeterministic inputs: reuse the synthetic deterministic image/matrix generators already in
tests/(fixed test patterns, seeded PRNGs) so runs are stable and comparable across machines/commits.Storage & comparison mechanism:
vitest bench --outputJson bench/results/<name>.jsonto emit structured results (hz, mean, p99, samples).--compare) diffs a new run against it and flags regressions beyond a threshold (e.g. >10% slower).bench/HISTORY.mdor append-only JSON log to chart trends across releases.Scope / phasing (incremental, one module per PR — same as #47)
benchscript + shared deterministic-input helpers + one exemplar module (imgproc) end-to-end, including the JSON output + baseline compare script.Impact
Low/Medium — additive, no runtime or public-API change; new dev tooling only. High long-term value: makes performance a tracked, first-class property before/through the 1.0 line.
Open questions
Non-goals