Add llama.cpp as a PAIR-managed engine

.cursor/rules/model-registry.mdc

@@ -1,5 +1,5 @@
 ---
-description: Electron-main engine model hub (Ollama committed list + LM Studio catalog)
+description: Electron-main engine model hub (locked Ollama + live Hugging Face catalogs)
 alwaysApply: true
 ---
 <!--
@@ -12,9 +12,10 @@ SPDX-License-Identifier: Apache-2.0
 - The backend has **no model-search RPC**, so the "Add Model" browse is
   served by a standalone Electron-main module, **`src/electron/model-hub/`**
   (main process, because the renderer cannot call `huggingface.co` under
-  CSP+CORS). There is **no generic Hugging Face browse and no shared GGUF
-  registry** — each enabled engine has exactly one curated source declared via
-  `EngineCaps.engineHub`.
+  CSP+CORS). There is no shared GGUF registry: each enabled engine has one
+  source declared via `EngineCaps.engineHub`. llama.cpp additionally supports
+  an explicit, bounded public Hugging Face search through that same
+  engine-specific source.
 - **Ollama** (`ollama-library.ts`): a **locked, committed list**, not a live
   scrape. `src/electron/model-hub/ollama-models.json` (wire shape
   `{ scrapedAt, source, count, models: OllamaTagsModel[] }`) is bundled into
@@ -38,20 +39,29 @@ SPDX-License-Identifier: Apache-2.0
   `https://huggingface.co/api/models?author=lmstudio-community&sort=downloads&direction=-1&limit=500`
   and normalizes each repo to a pull-ready id (e.g.
   `lmstudio-community/Qwen3-8B-GGUF`, accepted by `lms get`). This is the
-  **only** remaining network call in the model hub, and it is main-process only
-  (6-hour TTL + in-flight guard so PAIR never fetches continuously).
-- Both sources normalize to the shared `EngineHubModel`
+  LM Studio live source and is main-process only (6-hour TTL + in-flight guard
+  so PAIR never fetches continuously).
+- **llama.cpp** (`llamacpp-catalog.ts`): a live Hugging Face source with a
+  six-hour in-memory cache. Population queries the top 50 GGUF repositories for
+  each approved publisher (`ggml-org`, `bartowski`, and `unsloth`). An explicit
+  query searches up to 50 matches across all public publishers and uses a
+  bounded query cache. Both paths reject unsafe, private, gated,
+  non-generative, and non-pull-ready metadata. Every result is an exact
+  `owner/repository:Q4_K_M` router pull ID; alternate quantizations are not
+  listed. Never log the user's search text.
+- All sources normalize to the shared `EngineHubModel`
   (`src/shared/types/engine-api.ts`).
-- `index.ts` exposes `getEngineHubModels(engineType)` (Ollama returns the
-  committed list synchronously; LM Studio awaits a cold cache's initial load;
-  returns `{ models }`) and `warmEngineHubs()` (warms **only** LM Studio —
-  Ollama needs none). The `engine:search-hub` handler in `empty-handlers.ts`
-  calls the former; the `overview:ready` handler in `window.ipc.ts` calls the
-  latter. Warm on renderer-ready, **not** on service connect: a catalog fetch
-  started before the window has painted competes with the renderer's own load,
-  and a hanging one leaves an unpainted window behind.
+- `index.ts` exposes `getEngineHubModels(engineType, query?)` (Ollama returns
+  its committed list; LM Studio and llama.cpp await cold caches; a non-empty
+  llama.cpp query uses its bounded search) and `warmEngineHubs()` (warms both
+  live catalogs). The `engine:search-hub` handler in `empty-handlers.ts` calls
+  the former; the `overview:ready` handler in `window.ipc.ts` calls the latter.
+  Warm on renderer-ready, **not** on service connect: a catalog fetch started
+  before the window has painted competes with the renderer's own load, and a
+  hanging one leaves an unpainted window behind.
 - The renderer (`src/ui/components/ModelHub/`) fetches an engine's full
   catalog once (cached per-engine), filters by the search box **locally** on
-  each keystroke, and sorts client-side by Updated / Name / Size. Pulling goes
-  through the backend (`engine:pull-model` → `pull_model`); the hub only
-  produces the pull-ready id.
+  each keystroke, and sorts client-side by Updated / Name / Size. For llama.cpp,
+  Enter/Search sends an explicit query and merges the response with matching
+  populated rows. Pulling goes through the backend
+  (`engine:pull-model` → `pull_model`); the hub only produces the pull-ready id.

.cursor/rules/system-architecture.mdc

@@ -28,8 +28,9 @@ Broker-owned workers:
 - `nvpair-proxy`, one process hosting a facade per enabled engine. It starts
  with no engine and no listener; the broker sends a `facade/enable` per engine
  carrying that engine's port. Clients still address each facade as
- `ollama-proxy:` / `lmstudio-proxy:`, and one supervisor covers them all, so a
- crash is reported against `nvpair-proxy` and restarts every facade together;
+ `ollama-proxy:`, `lmstudio-proxy:`, or `llamacpp-proxy:`, and one supervisor
+ covers them all, so a crash is reported against `nvpair-proxy` and restarts
+ every facade together;
 - `nvpair-node-scanner`;
 - `nvpair-node-info`;
 - `nvpair-workload-manager`;
@@ -104,9 +105,9 @@ The broker uses both for scheduling.
 
 ### Inference HTTP
 
-Ollama and LM Studio proxies expose local client-compatible HTTP endpoints.
-Chat behaves as a local third-party client. Prompts, messages, chunks, and
-response bodies must never be logged.
+Ollama, LM Studio, and llama.cpp proxies expose local client-compatible HTTP
+endpoints. Chat behaves as a local third-party client. Prompts, messages,
+chunks, and response bodies must never be logged.
 
 ## Runtime defaults
 
@@ -115,8 +116,8 @@ Backend-coupled constants belong in
 
 - Prefer live broker or discovery values.
 - Do not assume a proxy port before the broker reports it.
-- Native chat fallbacks are Ollama `127.0.0.1:11434` and LM Studio
-  `127.0.0.1:1234`.
+- Native chat fallbacks are Ollama `127.0.0.1:11434`, LM Studio
+  `127.0.0.1:1234`, and llama.cpp `127.0.0.1:8080`.
 - Cluster pairing currently uses port `14321`.
 - Node telemetry path is `/v1/node-info`.
 
@@ -207,8 +208,16 @@ operations.
   contract.
 
 The model hub is Electron-main functionality under `src/electron/model-hub/`.
-It provides curated Ollama and LM Studio catalogs; model operations still go
-through the engine manager.
+It provides curated Ollama, LM Studio, and llama.cpp catalogs; model operations
+still go through the engine manager. llama.cpp populates a cached catalog from
+approved Hugging Face publishers and supports explicit bounded searches across
+public publishers. Every result remains an exact
+`owner/repository:Q4_K_M` pull ID.
+
+Managed llama.cpp installation uses CUDA archives on Windows and Linux and
+standard Metal-capable archives on macOS. Package choice is based on platform
+and architecture, not the installed driver; CPU fallback remains available.
+Checksums and required runtime artifacts are verified before installation.
 
 ## Pairing and security
 

.gitignore

@@ -16,6 +16,9 @@ token-map.json
 /scripts/inference-dispatcher/inference-dispatcher
 /scripts/inference-dispatcher/inference-dispatcher.exe
 
+# Downloaded, verified installer prerequisites.
+/.build/
+
 # Python bytecode from local runs of ci/release-intent scripts.
 __pycache__/
 *.py[cod]

README.md

@@ -38,7 +38,7 @@ one, and both report live GPU and memory use throughout.
 | **Architectures** | x64 and arm64 on all three. Windows on ARM is experimental. |
 | **Installers** | Windows `.exe`; Linux `.deb`; macOS `.dmg`. On other Linux distributions, [build from source](docs/building.mdx). |
 | **Mixing nodes** | Windows, Linux, and macOS nodes can all be paired with each other |
-| **Inference engines** | Ollama and LM Studio |
+| **Inference engines** | Ollama, LM Studio, and llama.cpp |
 
 **PAIR running on a machine does not mean an engine will.** PAIR itself runs on
 any supported Windows, Linux, or macOS machine. Each engine sets its own requirements
@@ -100,15 +100,19 @@ you want by its full filename instead.
   status there.
 
 - **Get an engine running.** On the node's card, open **Engine settings** and
-  select **Install** next to Ollama or LM Studio. PAIR downloads and sets the
-  engine up for you, so nothing needs to be in place beforehand. If PAIR already
-  found an engine you installed yourself, start that one instead.
+  select **Install** next to Ollama, LM Studio, or llama.cpp. PAIR downloads and
+  sets the engine up for you, so nothing needs to be in place beforehand. The
+  llama.cpp install is a separate, on-demand download and is not selected during
+  first-run setup by default.
 
   ![The Install engines dialog with Ollama downloading, reporting progress as it installs.](docs/assets/onboarding/engine-lifecycle/01-engine-installing.png)
 
 - **Add a model.** Select **Add model** on the same card and download one.
   `qwen4:12b` is used for this example; it can be replaced with a model of your
-  choice.
+  choice. llama.cpp populates this list from popular GGUF repositories on
+  Hugging Face. Type to filter the list, or press Enter to search the wider
+  public catalog. Its exact IDs include the repository and quantization, such
+  as `ggml-org/gemma-3-1b-it-GGUF:Q4_K_M`.
 
   ![A node card with its engine expanded, one model pulling and the Add model button beside the list.](docs/assets/onboarding/getting-started/07-add-model.png)
 
@@ -149,8 +153,9 @@ The reply is ordinary OpenAI-shaped JSON, abbreviated here:
 }
 ```
 
-If you changed a port, or you are using LM Studio rather than Ollama, copy the
-URL from **Endpoints → API endpoints** instead of assuming the one above.
+If you changed a port, or you are using LM Studio or llama.cpp rather than
+Ollama, copy the URL from **Endpoints → API endpoints** instead of assuming the
+one above.
 
 That is a single machine working. To route across machines, pair a second one
 from **Settings → Cluster** and repeat the engine and model steps there. The
@@ -270,7 +275,7 @@ feedback and contributions will help shape priorities.
 
 ### Engines and integrations
 
-- [ ] llama.cpp support.
+- [x] llama.cpp support.
 - [ ] vLLM support.
 - [ ] EXO support.
 - [ ] ComfyUI integration.

desktop/docs/architecture.md

@@ -109,7 +109,7 @@ subscribes to broker relays after `app:ready`, and converts backend responses
 into stable UI contracts.
 
 Electron reports the service connected after broker `app:ready`. The
-broker-owned Ollama and LM Studio proxies remain asynchronous capabilities; a
+broker-owned engine proxies remain asynchronous capabilities; a
 late or failed proxy does not misreport the broker startup as failed. If
 `app:ready` does not arrive within the startup deadline, Overview opens Settings
 
@@ -227,7 +227,7 @@ ordinary environment assignments can be edited locally or by a pinned peer.
 authoritative settings operation rather than forwarding to the engine manager,
 so both entry points validate, restart, and persist identically.
 
-The Ollama and LM Studio proxies are cluster-aware. For model-bearing inference,
+All engine proxies are cluster-aware. For model-bearing inference,
 each proxy first keeps only nodes whose per-engine discovery inventory advertises
 the requested model. Empty and non-matching inventories are excluded; an empty
 owner set returns a local `502`. Routing precedence within the eligible set is:
@@ -236,7 +236,7 @@ owner set returns a local `502`. Routing precedence within the eligible set is:
 2. the priority list emitted by `nvpair-job-scheduler`;
 3. the proxy's deterministic default ordering.
 
-The scheduler combines total pending (queued and running) workload across both
+The scheduler combines total pending (queued and running) workload across all
 engines with a smoothed 0–3 pressure derived from the busiest GPU. Missing,
 invalid, or older-than-10-second telemetry has neutral pressure. It emits the
 order, pending count, and pressure, reranking on meaningful workload, discovery,
@@ -250,7 +250,8 @@ not select or pin proxy routes.
 An NVPAIR-launched engine binds to loopback and is never directly LAN-reachable.
 Peers reach it only through the node's proxy over a cluster-mTLS ingress, so
 discovery advertises the promoted proxy port for `ol`/`lm` rather than the
-engine's private port. This transport security is backend-owned; Electron only
+engine's private port; llama.cpp uses the additional `lc` key. Transport
+security is backend-owned; Electron only
 reflects the advertised proxy port and reads a remote engine's real port from
 `engine:remote-get-installed` facts.
 
@@ -265,6 +266,12 @@ The model hub is Electron-main functionality in `src/electron/model-hub/`:
   fetched live from Hugging Face and cached for six hours. The cache is warmed
   when the Overview renderer reports ready, not when the service connects, so a
   slow or hanging catalog fetch cannot compete with the window's first paint;
+- llama.cpp's six-hour in-memory cache queries the 50 most-downloaded GGUF
+  repositories from each approved publisher (`ggml-org`, `bartowski`, and
+  `unsloth`). Explicit Enter/Search requests query up to 50 public Hugging Face
+  matches across publishers. Both paths keep only public, generative
+  repositories with a primary `Q4_K_M` artifact and emit exact
+  `owner/repository:Q4_K_M` pull IDs;
 - model pulls still run through `nvpair-engine-manager`.
 
 ## Inference Demo

desktop/docs/frontend-api.md

@@ -99,6 +99,14 @@ They do not imply a WebSocket connection. Browser clients are not supported.
 - `onSettingsDisconnected(callback)` reports that a node's settings authority
   became unreachable, so its cached snapshot is stale.
 
+`EngineType` currently includes `ollama`, `lm-studio`, and `llama-cpp`.
+`searchHub(engineType, query?)` serves Ollama's locked local catalog and cached
+live Hugging Face catalogs for LM Studio and llama.cpp. A non-empty llama.cpp
+query performs an explicit bounded search across public Hugging Face GGUF
+repositories; other calls return the populated engine catalog. llama.cpp pull
+IDs are exact `owner/repository:quantization` values; load, unload, and delete
+all use that same exact id.
+
 Port and launch changes work on a clustered peer as well as the local device,
 with one exception: **managed CORS origin settings** are local-only. The owning
 node rejects a change to browser access policy relayed from a peer. Other
@@ -196,8 +204,9 @@ Commands return no state. Renderer stores update from
 - update check, download, install, and status;
 - the node-local Inference Demo (`inferenceDemo.getState()` / `start()` / `stop()`
   and the `demo:state` push). `start()` rejects if a demo is already running on
-  this node or if no local engine exposed a text-generation model; callers show
-  that as an ordinary error banner. Demo state is not synchronized across nodes.
+  this node or if no Ollama, LM Studio, or llama.cpp proxy exposed a
+  text-generation model; callers show that as an ordinary error banner. Demo
+  state is not synchronized across nodes.
 
 `demo:state` is a main-to-renderer broadcast rather than a service push, so it is
 typed in `IpcPushChannelMap` (`shared/types/ipc-channels.ts`) instead of the

desktop/docs/service-contract-exceptions.json

@@ -9,6 +9,7 @@
         "engine:restore-enabled": "Broker-internal startup restoration. nvpair-ui-broker emits engine:restore-enabled directly to its supervised engine-manager after the managed Ollama port gate and on manager respawn; it is not a renderer/UI notification.",
         "ollama-proxy:ready": "Consumed, not missing: the broker relays it and normalizeBrokerProxy (modular-supervisor.ts) strips the `ollama-proxy:` prefix, so the bridge handles the de-prefixed `ready` (sets proxyPort). The literal `ollama-proxy:ready` is intentionally absent from our TS — extractor limitation, not a gap.",
         "lmstudio-proxy:ready": "Consumed, not missing: the LM Studio counterpart of ollama-proxy:ready, de-prefixed by the same normalizeBrokerProxy loop. It only became visible to the checker when METHOD_RE started accepting hyphens in a namespace segment; before that the whole lmstudio-proxy:* surface was silently unmatched.",
+        "llamacpp-proxy:ready": "Consumed, not missing: the llama.cpp counterpart of ollama-proxy:ready, de-prefixed by the same normalizeBrokerProxy loop and mapped to the desktop llama-cpp engine identity.",
         "node/selection-changed": "Automatic routing has no selected-node UI, so PAIR deliberately does not consume proxy selection changes.",
         "proxy/request": "Per-request proxy telemetry is not rendered; workload lifecycle uses the broker workloads stream.",
         "proxy/request-started": "Per-request proxy telemetry is not rendered; see proxy/request.",

desktop/docs/services-api.md

@@ -47,12 +47,6 @@
 - ⚠️ nvpair-ui-broker → engine:set-reserved-port
 - ⚠️ nvpair-ui-broker → engine:unsubscribe
 - ⚠️ nvpair-ui-broker → internal:set-reserved-port
-- ⚠️ nvpair-ui-broker → lmstudio-proxy:get-status
-- ⚠️ nvpair-ui-broker → lmstudio-proxy:set-port
-- ⚠️ nvpair-ui-broker → lmstudio-proxy:unsubscribe
-- ⚠️ nvpair-ui-broker → ollama-proxy:get-status
-- ⚠️ nvpair-ui-broker → ollama-proxy:set-port
-- ⚠️ nvpair-ui-broker → ollama-proxy:unsubscribe
 - ⚠️ nvpair-ui-broker → workloads:unsubscribe
 
 ### Backend binaries not listed in `modular-binaries.ts`
@@ -253,8 +247,6 @@
 | `errors:clear` | notification (we consume) | ✅ yes |
 | `errors:report` | notification (we consume) | ✅ yes |
 | `errors:update` | notification (we consume) | ✅ yes |
-| `lmstudio-proxy:ready` | notification (we consume) | ➖ ignored |
-| `ollama-proxy:ready` | notification (we consume) | ➖ ignored |
 | `workloads:upsert` | notification (we consume) | ✅ yes |
 | `connection/cluster-auto-sync` | request (we call) | ➖ ignored |
 | `connection/cluster-identity` | request (we call) | ✅ yes |
@@ -273,20 +265,12 @@
 | `engine:unsubscribe` | request (we call) | ⚠️ not called |
 | `errors:get-initial` | request (we call) | ✅ yes |
 | `internal:set-reserved-port` | request (we call) | ⚠️ not called |
-| `lmstudio-proxy:get-status` | request (we call) | ⚠️ not called |
-| `lmstudio-proxy:set-port` | request (we call) | ⚠️ not called |
-| `lmstudio-proxy:subscribe` | request (we call) | ✅ yes |
-| `lmstudio-proxy:unsubscribe` | request (we call) | ⚠️ not called |
 | `node/add` | request (we call) | ✅ yes |
 | `node/discovered` | request (we call) | ✅ yes |
 | `node/remove` | request (we call) | ✅ yes |
 | `node/removed` | request (we call) | ✅ yes |
 | `node/updated` | request (we call) | ✅ yes |
 | `nodes/list` | request (we call) | ✅ yes |
-| `ollama-proxy:get-status` | request (we call) | ⚠️ not called |
-| `ollama-proxy:set-port` | request (we call) | ⚠️ not called |
-| `ollama-proxy:subscribe` | request (we call) | ✅ yes |
-| `ollama-proxy:unsubscribe` | request (we call) | ⚠️ not called |
 | `ready` | request (we call) | ✅ yes |
 | `workloads:get-initial` | request (we call) | ✅ yes |
 | `workloads:remove` | request (we call) | ✅ yes |