v1.12.0 launcher and install issues

[florin65]

Commit or version

1.12.0

Environment

Colibrì 1.12.0 — Sandy Bridge / ThinkPad W520 field report

Date: 2026-09-22

Executive summary

Colibrì 1.12.0 was built from source and exercised on a Lenovo ThinkPad W520
(Sandy Bridge, CPU-only) against a real GLM-5.2/5.3 744B MoE checkpoint of
about 429 GB. The source build, package install, native engine startup, model
recognition, RAM admission, real model load, prefill, and multi-token inference
all passed. The desktop remained fully responsive under load and the observed
thermal behaviour stayed controlled.

A token-matched A/B against the same locally built Colibrì 1.6.2 showed:

• sustained decode performance effectively at parity (~0.7% slower in this
  bounded 16-token run);
• prefill about 3.0% slower on 1.12.0;
• about 0.86 GB / 5.0% lower RSS on 1.12.0;
• about 0.8 GB more projected RAM headroom on 1.12.0;
• a materially different residency planner (more pinned state, smaller LRU
  reserve, lower cap per layer).

Two narrow install/integration defects were found and reproduced independently
of the inference result:

1. the installed launcher fails when invoked through /bin/coli or
   /sbin/coli on a merged-/usr system because it uses abspath(__file__)
   rather than resolving the symlink before deriving the libexec path;
2. make install omits v41_dsml.py although openai_server.py imports it
   unconditionally, causing installed coli serve to fail immediately.

Two minimal patches are included with this report.

Exact source/package occurrence

• Upstream source: Colibrì 1.12.0
• Exact source archive SHA-256:
  5681cfea8e4e630ded16e141efccaf57ab140cb92df7893ff60ff7ba5a13cd20
• Final local package: colibri-1.12.0-1-x86_64.pkg.tar.zst
• Package SHA-256:
  4e903d81821e1d5ba269e5ba0f492a8bab5502c948391868fa8636b69b8c1559
• Build target: ARCH=sandybridge
• Runtime dependencies: glibc, libgomp, python

The W520 CPU supports AVX but not AVX2. The x86-64-v3 prebuilt release path is
therefore not applicable to this machine, while the source build with the
Sandy Bridge target is applicable and passed real inference.

The final 1.12.0-1 package payload was verified byte-for-byte identical to
the earlier locally labelled build used for the long inference trials; only
package metadata changed. The final package occurrence is the canonical one
listed above.

Test machine

• Lenovo ThinkPad W520 (4284F18)
• Intel Core i7-2960XM, Sandy Bridge, 4 cores / 8 threads
• 32 GB RAM
• NVIDIA Quadro 2000M, nouveau; inference kept CPU-only
• Artix Linux, dinit user services
• no swap during the trials

Build comparability with 1.6.2

The 1.6.2 and 1.12.0 binaries were built from the same local PKGBUILD lineage.
The build(), check(), and package() logic is unchanged; the 1.12.0 recipe
updates the source version and removes local patches whose changes are already
present upstream. Both versions use the same ARCH=sandybridge build path.

Therefore the A/B comparison materially controls packaging/build-method drift:

same build methodology + same hardware + same model + same runtime knobs

It is not claimed to be a bit-identical build environment across dates.

Package and native-engine gates

The 1.12.0 package passed:

• package install;
• package database/file integrity (124 total files, 0 missing files);
• shared-library resolution for all nine installed native engines;
• Sandy Bridge native-engine startup without SIGILL;
• launcher version/help;
• doctor precondition handling.

Native engines checked:

• colibri
• deepseek_v4
• deepseek_v41
• glm53
• inkling
• kimi_k3
• olmoe
• qwen36
• qwen38

Real model gate

Model: existing GLM-5.2/5.3 checkpoint mounted read-only.

Doctor command:

coli doctor \
  --model /models/glm52_i4_gs64 \
  --gpu none \
  --ram 24 \
  --policy quality \
  --auto-tier

Doctor result after freeing competing inference services:

• model path readable: PASS
• config valid: PASS
• family GLM-5.2/5.3 registered: PASS
• tokenizer present: PASS
• engine binary ready: PASS
• safetensors headers valid: PASS
• backing store available: PASS
• 24 GB RAM admission: PASS
• CPU path selected; GPU explicitly disabled

Reported geometry/plan:

• 142 shards
• 429.3 GB model
• 412.6 GB cold experts
• 11.9 GB dense
• 7.3 GB runtime
• 4.8 GB warm-expert budget in doctor projection
• projected expert residency: 1%

The read-only mount generated the expected persistence warning; it was not a
model/load failure.

Real inference

A first bounded run loaded the model successfully in 24.90 s, completed all 78
prefill layers, and generated OK.

A later token-matched run used:

time coli run \
  --model /models/glm52_i4_gs64 \
  --gpu none \
  --ram 24 \
  --policy quality \
  --auto-tier \
  --ngen 16 \
  "Write the numbers from 1 to 100, separated by commas, and nothing else."

1.12.0 result

• prompt: 24 tokens
• prefill: 1245.96 s
• decode: 16 tokens in 1013.66 s
• decode rate: ~0.01578 tok/s
• forwards: 15 for 16 emitted tokens
• expert hit rate: 4.2% (0.7% pin + 3.5% LRU)
• experts loaded/token: 562.5
• RSS: 16.18 GB
• total wall: 38m13.648s
• planner: cap 2/layer, LRU reserve 3.3 GB, ~0.8 GB pinned,
  projected peak 22.8 GB

1.6.2 controls

Two independent token-matched 1.6.2 controls were run with the same model,
prompt, RAM budget, policy, and generation limit.

Metric	1.6.2 control #1	1.6.2 control #2
Prefill	1208.10 s	1212.28 s
Decode, 16 tokens	1008.74 s	1003.58 s
RSS	17.04 GB	17.04 GB
Expert hit	5.3%	5.3%
Total wall	37m24.080s	37m23.751s

The two controls differed by only ~0.35% in prefill and ~0.51% in decode,
which materially reduces concern that run order/page-cache state explains the
aggregate result.

Using the two-control mean:

• 1.6.2 mean prefill: 1210.19 s
• 1.12.0 prefill: 1245.96 s -> ~2.96% slower
• 1.6.2 mean decode: 1006.16 s
• 1.12.0 decode: 1013.66 s -> ~0.75% slower, effectively parity here
• RSS: 17.04 GB -> 16.18 GB -> ~5.05% lower on 1.12.0
• projected peak: 23.6 GB -> 22.8 GB -> ~0.8 GB more headroom on 1.12.0

Bounded conclusion for this W520 / GLM-5.2 / CPU-only / 24 GB workload:

• real model load: PASS
• sustained prefill: PASS
• sustained multi-token decode: PASS
• no large decode regression observed
• decode: approximately parity with 1.6.2
• prefill: modestly slower (~3%)
• memory footprint/headroom: improved

Host behaviour under load

During real inference the desktop remained responsive, including Firefox and
interactive consoles.

Observed load snapshots included approximately:

• Colibrì engine around 3-4 CPU cores worth of load
• CPU package roughly 64-70 C
• CPU cores roughly 57-70 C
• Quadro/nouveau sensor roughly 58 C
• ThinkPad fan roughly 2899-3230 RPM (54-72% manual PWM in observed snapshots)
• no swap
• no OOM
• no SIGILL
• no observed thermal distress

This is a field observation for this machine, not a general QoS/thermal claim.

---

Finding 1: installed launcher fails through merged-/usr aliases

Suggested issue title:

Installed coli fails via /bin and /sbin on merged-/usr systems

Reproduction

With an otherwise complete installed package:

$ type -a coli
coli is /bin/coli
coli is /sbin/coli
coli is /usr/bin/coli
coli is /usr/sbin/coli

$ /bin/coli --version
ModuleNotFoundError: No module named 'family_registry'

$ /sbin/coli --version
ModuleNotFoundError: No module named 'family_registry'

$ /usr/bin/coli --version
colibri 1.12.0

$ /usr/sbin/coli --version
colibri 1.12.0

readlink -f "$(command -v coli)" resolves to /usr/bin/coli.

Package integrity was clean (124 total files, 0 missing files), so this is not
an omitted-file failure.

Root cause

c/coli currently computes:

HERE = os.path.dirname(os.path.abspath(__file__))

When invoked as /bin/coli, abspath() preserves /bin/coli; it does not
resolve the merged-/usr symlink. The installed-layout fallback therefore
inserts /libexec/colibri rather than /usr/libexec/colibri before the early
family_registry import.

Canonical /usr/bin/coli works because the derived path is
/usr/libexec/colibri.

Minimal fix

Use the symlink-resolved launcher path:

HERE = os.path.dirname(os.path.realpath(__file__))

Included patch:

0001-coli-resolve-launcher-symlinks.patch

Patch validation

The patch was applied cleanly to the exact 1.12.0 source occurrence. A
simulated installed merged-/usr tree was created and all four invocation paths
passed:

/bin/coli       -> colibri 1.12.0
/sbin/coli      -> colibri 1.12.0
/usr/bin/coli   -> colibri 1.12.0
/usr/sbin/coli  -> colibri 1.12.0

---

Finding 2: make install omits v41_dsml.py

Suggested issue title:

make install omits v41_dsml.py, causing installed coli serve to fail

Reproduction

The installed package was complete according to its own manifest, but
starting the persistent service failed with exit status 1:

File "/usr/libexec/colibri/openai_server.py", line 24, in <module>
    import v41_dsml
ModuleNotFoundError: No module named 'v41_dsml'

The exact 1.12.0 source contains:

c/v41_dsml.py
SHA-256 448b46d2e54793da0cef21aadf10557acd9021e36ae8a8a3ab00110c8e7f4f2f

openai_server.py imports both v4_dsml and v41_dsml, but the Makefile
install list contains only v4_dsml.py:

$(INSTALL) -m 644 family_registry.py resource_plan.py doctor.py autotune.py \
        openai_server.py cluster.py v4_dsml.py version.py $(DESTDIR)$(LIBEXECDIR)/

The release closure calculator already reaches v41_dsml.py:

pack_python.needed(c):
openai_server.py  present
v4_dsml.py        present
v41_dsml.py       present
family_registry.py present

The Dockerfile also explicitly includes v41_dsml.py. The omission is
therefore specific to the handwritten make install Python list.

Minimal fix

Install v41_dsml.py next to v4_dsml.py:

openai_server.py cluster.py v4_dsml.py v41_dsml.py version.py

Included patch:

0002-make-install-v41-dsml.patch

Recovery validation

Copying the exact source v41_dsml.py into the installed libexec directory
allowed the existing dinit-managed container service to remain STARTED. The
HTTP endpoint then returned the expected authentication guard response:

{"error":{"message":"Invalid or missing API key.","type":"authentication_error","param":null,"code":"invalid_api_key"}}

This establishes server/auth readiness after restoring the omitted module.

The local hotfix was only a recovery step. The package-level fix should install
the file as mode 0644, consistent with the other support modules.

Patch validation

The patch was applied cleanly to the exact 1.12.0 source occurrence. A staged
make install smoke placed:

mode 0644  /usr/libexec/colibri/v41_dsml.py
SHA-256    448b46d2e54793da0cef21aadf10557acd9021e36ae8a8a3ab00110c8e7f4f2f

Longer-term hardening suggestion

The project already has c/tools/pack_python.py specifically to calculate the
Python runtime closure and avoid handwritten release lists drifting behind the
source. make install still maintains a separate handwritten root-module list.
The immediate one-line fix above is sufficient, but a future invariant/test
could require every root-level Python module reached by pack_python.needed()
to be present in the make install closure as well.

That would encode the stronger rule:

release Python closure == installed Python closure

rather than fixing one missing filename at a time.

---

Included local packaging recipe

PKGBUILD-1.12.0-2 applies both bounded patches downstream while keeping the
same Sandy Bridge build/check/package logic used for the validated 1.12.0
build.

The two patches are intentionally separate because they fix independent
problems:

1. launcher symlink/path resolution;
2. installed Python runtime closure.

No long inference rerun should be required solely for these two patches: they
do not modify the native inference engines or the model planner. The useful
post-package gates are instead:

/bin/coli --version
/sbin/coli --version
/usr/bin/coli --version
/usr/sbin/coli --version
pacman -Qk colibri
coli serve ...
curl http://127.0.0.1:PORT/v1/models

with the expected authenticated-server response for an unauthenticated curl.

0002-make-install-v41-dsml.patch
0001-coli-resolve-launcher-symlinks.patch

Reproduction steps

I was in vacation and my last colibri version used was 1.6.2. When I have returned the version is 1.12.0. I have attached here a report with some compared runs for both those versions on my "vintage" test system Lenovo Thinkpad W520. Thank you for your work.

Expected behavior

All the needed information and proposed patches were included here.

Actual behavior and logs

Please review the information. I hope to be useful for your project.

[JustVugg]

Thank you for the report, the two patches and the A/B against 1.6.2 on the same build recipe. That comparison is the kind we cannot run ourselves (nobody here has a Sandy Bridge laptop with the 744B checkpoint), and decode at parity with 5% less RSS after six minor versions is good to know. The 3% on prefill is noted; on a 24-token prompt over 1246 s it is inside what two runs of the same binary showed elsewhere, but it stays on the list.

Both findings are right, and both ship in 1.12.1:

• Finding 2 (v41_dsml.py missing from make install) was already fixed on dev by fix: stage v41_dsml.py during installation #1610, exactly as your patch does.
• Finding 1 (the launcher through /bin/coli on a merged-/usr system) is coli: resolve the launcher symlink before deriving libexec; install closure test #1691, your patch as is: realpath before deriving the libexec directory.

Your hardening suggestion is in the same PR: tests/test_install_closure.py runs the launcher through a symlinked bin directory (it fails on 1.12.0 with the ModuleNotFoundError you saw, and passes with the fix), and checks that every root-level module tools/pack_python.py reaches is in the handwritten make install list. That is the rule you wrote, release Python closure == installed Python closure, as a test rather than a filename.

On the W520: 1.12.1 carries an SSE4.1 tier for the qwen36 and olmoe int8 GEMV (#1239, from another Sandy Bridge in #1652), not for the GLM engine you ran, so do not expect it to move your numbers; the ARCH=sandybridge build stays the right one.

[JustVugg]

#1691 is merged on dev (your realpath patch plus the install-closure test), and #1610 covered v41_dsml.py earlier. Both ship in 1.12.1. Closing; thank you again for the report and the A/B.