Accelerate household inference with multicore macOS and concurrent Hybrid

[JustVugg]

Implemented

• Bundled native macOS OpenMP, reviewed all-selected resident placement and per-stage timings.
• Concurrent OLMoE local/remote expert execution with router-order accumulation and atomic local fallback.
• TUI-approved household Hybrid: full resident coordinator plus selected range accelerators, with duplicate weights and scratch charged to approved memory budgets.
• Versioned immutable allocation/checkpoint/peer/range permissions, pinned authenticated RPCs, numeric-class readiness probes, no public discovery or relay adoption.
• Accelerators prepare first, then coordinator; retained Hybrid plans reopen without reloading weights.

Verification — current head 51cf80e

CI SUCCESS, all 8 jobs: https://github.com/JustVugg/lumabri/actions/runs/34718165193
Includes Linux and native macOS Intel/Apple Silicon with OpenMP enabled and disabled, installed-package tests, model-family regressions, memory contracts and token oracles.

Local installed Linux candidate passed resident TUI Hybrid on both the small fixture and the existing 651 MB OLMoE fixture: actual remote expert calls plus concurrent coordinator work, second conversation with source offline, no reload or weight mirror writes. Donor-loss cleanup and paused-TUI/requester-crash retention passed. Production handler scope tests, offer/duplicate-memory/restart tests, ASan/UBSan and existing accumulator/fallback/NAT tests passed. Standalone real OLMoE oracle passed exact tokens across local, concurrent Hybrid and bounded fallback with a stopped donor.

Limits and next test

Household Hybrid currently targets OLMoE top-k >= 2 and requires full resident local fallback on the coordinator. Other models keep Segment. No changes to upstream Colibri. The physical PC + Intel Mac A/B is next, using the exact tested packages; no per-chat speedup is claimed before that measurement. PR ready for review/test, not merged. No squash merge.

Mac Intel OpenMP candidate: https://github.com/JustVugg/lumabri/actions/runs/34718165193/artifacts/10305807984
Linux candidate: https://github.com/JustVugg/lumabri/actions/runs/34718165193/artifacts/10305768163

[JustVugg]

Follow-up fix in fee3283 for the physical PC/Intel Mac run: the accelerator reached READY, but the requester cancelled the progressing coordinator at the hard-coded 900-second preparation limit. The source had served 29.14 GB across local and remote readers before rollback. Preparation now uses a progress-renewed 15-minute idle deadline plus a separate 24-hour safety ceiling, with an explicit expiry diagnostic. Heartbeats alone cannot renew progress. Verified resident input now keeps up to four RAM blocks under a 64-MiB per-engine cap, explicitly reserved for both Segment and Edge and released at seal; no disk inference fallback was added. Local gates passed: deadline regression, memory budgets, authenticated interleaved/cross-shard reads and eviction, sealed input, native shim, UBSan input, ASan/UBSan memory budget, household monitor, full 651-MB fixture Hybrid TUI flow including paused UI, requester loss, retained RAM/source-offline second conversation, calibration invalidation, and UI/packaging tests. A 7.4-GB OLMoE loopback run is in progress. Native CI and the updated physical PC/Mac speed test remain required; the earlier physical run produced no tok/s measurement.