Skip to content

ojph 0.10.2: Highway fat-binary fork with runtime dispatch - #41

Merged
hmaarrfk merged 2 commits into
ramonaoptics:mainfrom
hmaarrfk:ojph-0.10.2
Aug 9, 2026
Merged

hmaarrfk merged 2 commits into
ramonaoptics:mainfrom
hmaarrfk:ojph-0.10.2

Conversation

@hmaarrfk

@hmaarrfk hmaarrfk commented Aug 9, 2026

Copy link
Copy Markdown
Member

Prepared by Mark Harfouche's Claude bot; Mark Harfouche (@hmaarrfk) reviews via the GitHub interface.

Bumps the vendored fork to ojph-1.1.0 — the consolidated Highway series. What users get:

  • Runtime-dispatched fat binary: one kernel source compiled for SSE4.1, AVX2, and three AVX-512 tiers; the best tier is selected per CPU at first use. AVX-512 machines gain ~8–10% encode / ~7% decode end-to-end (up to 2× on individual kernels); pre-AVX2 machines regain vectorized kernels. Validated byte-identical on Arrow Lake, Sapphire Rapids, Zen 3, and Zen 2.
  • irv97 substantially faster: a conversion kernel left scalar during the SIMD rework cost up to 75% on decode; fixed (4096² mask decode 22.5 → 12.8 ms).
  • rev53 audited vs upstream OpenJPH: faster on every operation (encode up to −35%).

Build mechanics: the kernels dispatch through the Highway library now, so tools/build_openjph.py builds a static libhwy when downloading (wheels stay fully self-contained — verified ldd-clean with all target namespaces present) and links the environment's shared libhwy in conda/dev setups. Windows wheels deliberately keep the cl.exe/static-AVX2 build (unchanged behavior).

Release plan: merge on green CI, tag 0.10.2.

https://claude.ai/code/session_015rdawfurPCmefzjoeFfuC4

<details><summary>Claude's draft</summary>

Bump the vendored fork to 1.1.0 (the consolidated Highway series:
foreach_target fat binary SSE4.1/AVX2/AVX3/AVX3_DL/AVX3_SPR selected
at run time, width-portable HT encoder, irv97 recovery, rev53 audit
fixes, clang-cl toolchain support on the fork side).

The kernels now dispatch through the Google Highway library at run
time, so headers alone no longer suffice. tools/build_openjph.py uses
an installed libhwy when present (conda; the extension then links the
shared library) and otherwise downloads Highway 1.4.0 and builds a
static archive that setup.py links into the extension alongside
libojph, keeping wheels fully self-contained. setup.py picks whichever
of the two the install prefix provides, and only links -lhwy when a
libhwy actually exists (OJPH_ALLOW_NO_HWY builds keep linking).

Windows wheels intentionally keep the cl.exe toolchain (static-AVX2
hwy, unchanged behavior); the fork's clang-cl support enables the full
dispatch ladder there later if wanted.

Verified locally: wheel-style path (no conda) downloads and builds
static hwy, extension is self-contained (ldd clean) and contains the
N_SSE4/N_AVX2/N_AVX3/N_AVX3_DL/N_AVX3_SPR target namespaces; conda
path links shared libhwy; 186 tests pass on both; mask perf on this
AVX2 machine unchanged (rev12 5.04/4.44 ms enc/dec).

Resume this Claude session:
```
cd /home/mark/git/jp2k
claude --resume 487c1e20-36f3-416f-8be1-437c25ed67fe
```
</details>
<details><summary>Claude's draft</summary>

Fork 1.1.1 fixes the Highway usability probe (AppleClang defaults to
a pre-C++11 standard; the probe now pins C++17) and scopes
OJPH_REQUIRE_HWY to x86, where the hwy kernels exist. ARM targets
(macOS arm64, linux aarch64, windows ARM64) build the generic kernels
as they always have, instead of failing configure; macOS x86_64 gains
the full fat binary.

Resume this Claude session:
```
cd /home/mark/git/jp2k
claude --resume 487c1e20-36f3-416f-8be1-437c25ed67fe
```
</details>
@hmaarrfk
hmaarrfk merged commit acb6c8e into ramonaoptics:main Aug 9, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant