Feat/qwen38 flash next

[themorlock]

Summary

Qwen3.8-Flash-Next joins the flock 🐦

• add a dependency-free CPU text engine that runs the official FP8 checkpoint directly
• implement Gated DeltaNet, sparse QSA, PLE n-gram embeddings, four-stream residuals, and native FP8/BF16 weights
• integrate the CLI, server, planner, release packaging, Segment runtime, tests, and documentation

The base text engine is implemented and validated, with a few more optimizations still to squeeze out before this leaves draft.

Validation

• token-exact and logit-validated against the upstream tiny oracle
• end-to-end generation with the official checkpoint
• full repository checks, sanitizers, and Segment conformance

CPU-only for now

Tracks #1242.

[jazir555]

Please add GPU support so this works tiered across SSD, RAM, CPU and GPU like the rest of the engines.

[JustVugg]

@mohamedmastouri2000-boop @ZacharyZcR @terrizoaguimor o someone else can help here for gpu support?

[JustVugg]

I reviewed this against the GLM-5.3-Flash work and built it locally. Short version: the methodology is right, the engine is in good shape, and I do not want it rewritten. Longer version below, including one thing that will bite on merge and that you cannot see from your branch.

What I verified rather than took on trust

• Builds clean with -Wall -Wextra, zero warnings. qwen36 currently emits four, so this is the better-behaved of the two.
• The streaming is real: qwen38_core.h has the per-layer LCache, pread, posix_fadvise, prefetch and parallel reads. I checked because qwen38.c alone has none of it, and an engine that quietly resident-loads everything would defeat the point of the project.
• The 51B PLE table genuinely stays on disk. q38_ple_row reads one row at a time with a bounds check and an explicit byte-offset overflow check.
• Tools are refused with a real APIError, not silently dropped.
• The descriptor is right: supports_accelerator=False, and model_types=('qwen4_exp','qwen4_exp_text') does not collide with qwen36's qwen3_5_moe*.

Three things here are more disciplined than what I shipped for GLM-5.3, and I would rather say so than pretend otherwise: the CI gate under ASan and UBSan, the pinned checkpoint revision so an upstream update cannot change the tensor contract underneath you, and release.yml. You added qwen38 to both the build loop and the copy list. I did not do that for glm53, and v1.9.0 shipped with no archives because of it (#1267).

reducer = max if qwen38 else statistics.median in the planner is the right call for heterogeneous FP8/F32 experts, and it is the same lesson as #856: size the slot by the largest thing that can land in it, not the typical one.

One caveat on my own review: I could not run the oracle, because it needs transformers==5.16.1 and this machine has 5.12.1. So token-exactness is your claim plus CI, not something I independently reproduced.

The thing that will bite on merge

I merged dev into your branch locally to measure the cost. Eleven conflicts, all mechanical unions except one. That part is fine.

The problem is what git merged without a conflict:

segment conformance requires 7 families, found 8
tests/test_segment_adapters_registration.c:16: assert(coli_segment_adapter_count() == 7)
tests/test_edge_adapters_registration.c:16:    assert(coli_edge_adapter_count() == 7)

Your branch and dev each raised the same constant from 6 to 7. Because it is the identical edit, git takes it silently and the result is wrong: with both families registered the answer is 8. It is green on your branch, green on dev, and red only after the merge. Three one-character fixes, but you would have found them by watching CI fail rather than by reading the diff.

The conflicts, and who should fix them

For the record, since these exist only because GLM-5.3 landed after you started, so the realignment is our mess and not yours:

• ci.yml, release.yml, c/Makefile, segment_conformance_fixtures.c, and the three adapter tests: plain unions of the engine list. The fixtures file needs renumbering, glm53 through qwen38 as 1 to 6.
• family_registry.py (resident_inventory vs dense_load_ratio) and openai_server.py (self.family vs self.model_dir): both additive, keep both lines.
• README.md: both sides say "Seven families". Together it is eight.
• resource_plan.py is the only one needing judgement. Keep dev's _dense_in_ram and dense_disk_bytes alongside your expert_fixed_bytes, and keep your version of the decisions list, the one gated on supports_accelerator. Announcing a VRAM tier to a CPU-only engine writes a plan line nobody can act on, so your conditional is the correct behaviour and dev's unconditional one is not.

With those resolved plus the three counters, all four engines build and the Python and C suites pass here.

Say the word and I will push the rebase to your branch rather than making you redo it.

Two questions

1. How much RAM is resident? The doc gives 185.5 GB on disk and says resident matrices stay native BF16, but never states the total. That is the number that decides who can run this, and right now a reader cannot tell whether their machine qualifies. A line in docs/qwen38.md and ideally one real datapoint would close it.
2. Is this finished? The description says a few optimizations remain before it leaves draft, but the PR is not marked draft. If you consider the text engine done, I will treat it as ready once rebased.

On the GPU request above: worth doing, not a blocker. DeepSeek V4 also landed CPU-first and grew a CUDA tier afterwards, and I would rather merge a correct CPU engine than hold it for a backend nobody has measured yet.

Good work, and thank you for matching the oracle-first approach instead of writing the architecture from the paper. It is the reason this is reviewable at all.

[themorlock]

Thanks for the feedback. If you can do the rebase that'd be awesome.

For the CUDA stuff, I agree that we can do that in a separate PR.

Yes, the description is stale. I was working on some optimizations with SIMD kernels but I think that's beyond scope of this PR - can always open that separately in the future.

I added the RAM amounts in docs/qwen38.md.

[JustVugg]

Rebased onto dev and pushed to your branch (eabd569). The conflicts were ours, so the cleanup was ours.

Unions, no judgement needed: ci.yml, release.yml, c/Makefile, the three adapter tests, and segment_conformance_fixtures.c with the wrappers renumbered glm53..qwen38 as 1 to 6. family_registry.py (resident_inventory vs dense_load_ratio) and openai_server.py (self.family vs self.model_dir) were additive, so both sides are in. README now says eight families, since both sides said seven.

One needed a decision. In resource_plan.py I kept dev's _dense_in_ram accounting and dense_disk_bytes next to your expert_fixed_bytes, and for the decisions list I kept yours, the one gated on supports_accelerator. Announcing a VRAM tier to a CPU-only engine writes a plan line nobody can act on, so your conditional is right and dev's unconditional version is not. Worth saying explicitly since it is the one place the merge did not simply take both.

And the three counters, which is the part neither branch could see alone:

test_segment_conformance.c              required_families  7 -> 8
test_segment_adapters_registration.c    count == 7 -> 8
test_edge_adapters_registration.c       count == 7 -> 8

Both branches raised the same constant from 6 to 7, so git took the identical edit without flagging anything and the merged result was wrong.

Verified here: all seven engines build, the Python suites pass, and segment-adapters, edge-adapters and segment-conformance are green, the last one printing ok qwen38 four-stream hyper-residual + GDN recurrent/conv + QSA sparse indexer + PLE.

Thanks for the RAM table in docs/qwen38.md. That was the number a reader needed to decide whether to spend 185 GB, and 9.2 GiB resident with cap 32 at 16.5 GiB total makes the answer plainly yes on an ordinary 24 GB machine. Splitting it into resident weights, cache per slot per layer, scale bank and workspace is more useful than a single total would have been.

Next from my side is running qwen38-tiny-check locally, which I could not do at review time for lack of transformers==5.16.1. I have the pinned environment now. Nothing for you to do unless it turns something up.

[JustVugg]

Merging. Seventh family in the engine.

I re-checked this more carefully than a docs-clean PR would need, because the risk here is not qwen38.c — that is your own file and a defect in it cannot reach another engine. The risk is the five shared files: st.h (+106), segment_adapter_internal.h, route_trace.h, and a line each in segment_adapters.h and edge_adapters.h.

st.h I read line by line, since every engine loads through it. It does not weaken anything. It adds two validations on untrusted input that were missing — data_offsets was cast with (int64_t)off->kids[0]->num without checking the value was a number, finite, non-negative and integral, and there was no overflow guard on data_start + b0 or numel * esz. It adds two new readers, both bounds-checked and additive. And it closes two deliberate leaks.

The leaks were the part I wanted to be sure about, because the old comment said the jval tree was left in memory on purpose, and freeing it would dangle every retained pointer into it. Checked both sites: t->name = strdup(name), and the format stamps are strduped before json_free(inner). Nothing stays aliased, so both frees are correct.

The strongest evidence is not the reading. GLM-5.3 stays token-exact from your branch, text and multimodal, logits within 4.77e-07 and 7e-07. It is the engine that leans hardest on st.h; if those changes broke something it would show there.

Everything else: all eight engines build, qwen36 token-exact at caps 1/8/16, qwen38 token-exact across all eight Q38_PREFILL_BATCH/Q38_NATIVE_BF16/cap combinations, the full make test-c suite exits 0, the Python suites pass, CI 25/25 under an ASan that now genuinely runs.

On that last point, worth stating since it started as your PR being red: adding qwen36 to tools/clean.py is what made the sanitizer step build an instrumented binary for the first time. Your branch did not introduce those failures, it exposed two pre-existing defects in qwen36 — a dangling Model pointer held by the detached PILOT worker, and a heap overflow in load_meta on a container whose two config files disagree. Both fixed in #1277, and the second one is the more serious: reachable from an ordinary model directory.

Thanks for the RAM table, for the pinned checkpoint revision, and for the ASan/UBSan gate in CI — that last one is now the reason this engine is better covered than some of the older ones.

Two follow-ups I would like to open next, neither blocking: tool calling (the gateway already has the shared parser, so it is the template plus a pinned template test), and vision, since the checkpoint is multimodal and the engine already accepts the model.language_model prefix. Both can be built against a synthetic tiny fixture rather than 185 GB.