fix(inkling): IDOT opt-in — the fast path is x86-only and not token-equivalent - #1080
Merged
Conversation
…quivalent The int8-activation fast path exists only under __AVX2__ and quantizes activations per 32-block; the scalar route is byte-exact. With the old on-by-default, the same model with the same settings produced different tokens on x86 vs ARM, and no measurement ever blessed that default. Same class and same one-line fix as olmoe (#1044) and qwen36 (#712 review, kreuzzelg's finding). IDOT=1 restores the fast path explicitly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BColsey
pushed a commit
to BColsey/colibri
that referenced
this pull request
Aug 18, 2026
…gg#1081) Every tiny-oracle job ran on x86, so NEON branches were never compiled, let alone executed — which is how the olmoe (JustVugg#1044) / qwen36 (JustVugg#712 review) / inkling (JustVugg#1080) IDOT class shipped undetected. Two additions: 1. tests/test_int_kernel_exact.c — integer kernels have no rounding excuse: against a pure-C reference on the same inputs they must match bit for bit on every ISA. Covers dot_i4i8 (pair int4xint8), planarize_i4 round-trip, dot_i4p_u (K1 plane layout, the -8*sum(x) identity), and matmul_i4p_idot vs matmul_i4_idot bitwise. The test encodes the real activation contract: qrow_i8 clamps to [-127,127], and -128 must never reach the kernels (the abs/sign VNNI trick cannot represent its negation) — the first draft generated full-range int8 and produced 30 false failures against the SHIPPED kernel, which is the contract documenting itself. 2. An ubuntu-24.04-arm job (timeout-bounded per JustVugg#953's direction): builds every engine (NEON compile coverage — K1's dot_i4p_u NEON branch had never been compiled by anyone), runs the exactness gate with the NEON branches live, and replays the glm_tiny teacher-forcing oracle against a fixture generated on the same runner (same-machine torch reference, no cross-ISA float excuses). The x86 efficiency job also runs the exactness gate, so both ISAs hold the same bar. deepseek-v4 tiny-check on ARM is follow-up (its ARCH plumbing needs checking for aarch64 first). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 18, 2026
Merged
This was referenced Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-through on #712's review finding (credit @kreuzzelg): inkling's IDOT fast path exists only under
__AVX2__and quantizes activations per 32-block, while the scalar route is byte-exact — so with the old on-by-default, the same model with the same settings produced different tokens on x86 vs ARM, and no measurement ever blessed that default.Same class and same one-line fix as olmoe (#1044) and qwen36:
IDOT=1restores the fast path explicitly; the default is now the byte-exact route on every ISA.Both call sites flipped (
matmul_q, the dual variant). Build clean. Note this intentionally changes default behavior on x86 (exact-but-slower); users who had measured and accepted the int8 numerics opt back in with one env var.A companion issue will track the structural gap this exposed: our tiny-oracle gates run on x86 only, so ARM-divergent paths are invisible to CI —
ubuntu-24.04-armrunners can close that.🤖 Generated with Claude Code