Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions desktop/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,23 @@ closed `EngineType` union and narrows external strings with `isEngineType()`.
`engineManagerName()` / `engineTypeFromManagerName()` are the one place that
translation to and from the engine manager's own spelling lives.

### Model downloads

A download is started with `engine:action{pull_model}`, or
`engine:remote-pull-model` for a pinned peer, and stopped with
`engine:cancel-pull` or `engine:remote-cancel-pull`. All of them name their

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LMStudio has "model", but IIUC, Ollama has {name: model}

target in a `model` field: an engine can be downloading several models at once,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"several models at once"? I thought it was one at a time in a queue.

and `engine:pull-progress` carries the same field so every frame lands on the
row it belongs to.

Cancellation is a request, not a result. The engine manager stops the transfer
and settles the partial files before answering, and the row clears when the
pull itself settles. A peer is given a much longer budget to answer than the
desktop waits, on purpose — cutting a peer off part-way through a cancel is
worse than waiting for it — so the desktop giving up first means the cancel is
still running, not that it failed. The row stays in Canceling and the cancel
can be issued again.

### Engine settings

Server port, proxy port, and the engine arguments are one authoritative
Expand Down
14 changes: 13 additions & 1 deletion desktop/docs/frontend-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ They do not imply a WebSocket connection. Browser clients are not supported.
- `getInitialState()` returns engine statuses, models, active progress, and available
updates.
- `toggle`, `install`, `uninstall`, and `update` manage engine lifecycle.
- `pullModel`, `loadModel`, `unloadModel`, `deleteModel`, and
- `pullModel`, `cancelModelPull`, `loadModel`, `unloadModel`, `deleteModel`, and
`setModelExpiry` manage models.
- `searchHub(engineType)` returns the curated model catalog for an engine.
- `onStateChanged`, `onProgress`, and `onProgressRemove` expose backend truth.
Expand Down Expand Up @@ -112,6 +112,7 @@ environment assignments pass through without an engine-option catalog.
- `uninstall`;
- `update`;
- `pullModel`;
- `cancelModelPull`;
- `loadModel`;
- `unloadModel`;
- `deleteModel`;
Expand All @@ -121,6 +122,17 @@ Commands return no state. Renderer stores update from
`engines:state-changed`, `engines:progress-changed`, and
`engines:progress-cleared`.

The model-bearing commands — `pullModel`, `cancelModelPull`, `loadModel`,
`unloadModel`, `deleteModel`, and `setModelExpiry` — carry the target in
`model`. It is what lets a node run several downloads at once and have each

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"several downloads at once" - same concern as architecture.md

one cancelled, and progress-tracked, on its own.

`cancelModelPull` reaches `engine:cancel-pull`, or `engine:remote-cancel-pull`
when `nodeId` names a peer. The download is not cancelled when the command
returns: the row moves to Canceling and clears when the pull itself settles. A
cancel whose request outlives its budget leaves the row in Canceling, because
the backend is still working on it, and stays available to issue again.

### `pairApi.workloads`

- `getInitial()` returns active workloads.
Expand Down
2 changes: 2 additions & 0 deletions desktop/docs/services-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@
| `errors:clear` | notification (we consume) | ✅ yes |
| `errors:report` | notification (we consume) | ✅ yes |
| `engine:action` | request (we call) | ✅ yes |
| `engine:cancel-pull` | request (we call) | ✅ yes |
| `engine:configure-launch` | request (we call) | ⚠️ not called |
| `engine:configured-ports` | request (we call) | ⚠️ not called |
| `engine:describe` | request (we call) | ⚠️ not called |
Expand All @@ -109,6 +110,7 @@
| `engine:prepare-shutdown` | request (we call) | ✅ yes |
| `engine:preview-launch` | request (we call) | ⚠️ not called |
| `engine:remote-apply-settings` | request (we call) | ⚠️ not called |
| `engine:remote-cancel-pull` | request (we call) | ✅ yes |
| `engine:remote-delete-model` | request (we call) | ✅ yes |
| `engine:remote-get-installed` | request (we call) | ✅ yes |
| `engine:remote-get-settings` | request (we call) | ⚠️ not called |
Expand Down
21 changes: 21 additions & 0 deletions desktop/docs/services-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ reserved for inference clients.
| `engine:ready` / `engine:state-changed` | Update engine facts and models | `engines:state-changed` |
| `engine:settings-changed` | Validate and republish the owning node's settings snapshot | `engines:settings-changed` |
| `engine:install-progress` / `engine:remote-progress` | Update operation progress | engine progress pushes |
| `engine:pull-progress` | Advance the optimistic pull entry the `model` field names | engine progress pushes |
| `errors:update` | Replace the error snapshot | `errors:update` |
| `cluster:invite-received` | Parse inbound invite | `cluster:invite-received` |
| `cluster:invite-canceled` / `cluster:invite-expired` | Prune the canceled or timed-out inbound invite from the authoritative set | `cluster:pending-invites-changed` |
Expand Down Expand Up @@ -153,6 +154,26 @@ Local engine operations include install, start, stop, uninstall, update, port
changes, and model actions. Remote cluster operations use the engine manager's
remote control surface where supported.

### Cancelling a download

`engine:cancel-pull` stops a download on this node and
`engine:remote-cancel-pull` stops one on a pinned peer. Both name their target
with a `model` field, which is what distinguishes them from the engine-wide
commands: an engine may have several downloads in flight, and the frames on

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"several downloads in flight" - same concern as architecture.md although I think this is closer to reality.

`engine:pull-progress` carry the same field so each one lands on its own row.

Neither returns state. The backend answers only once the transfer has stopped
and its partial files are settled, so the reply is an acknowledgement and the
row clears from the pull's own settling, not from the cancel.

That answer can outlive the desktop's budget. A peer is served by the engine
manager's readiness client, whose response-header budget is far longer than
`MODULAR_CANCEL_PULL_TIMEOUT_MS`, because cutting a peer off mid-cancel is
worse than waiting for it. A timeout therefore means "still cancelling", not
"failed": the row stays in Canceling rather than dropping back to Downloading,
and the cancel can be issued again. Only an explicit rejection restores the
previous status and reports an error.

### Engine settings

Ports and the engine arguments are one authoritative, revisioned record owned by
Expand Down
123 changes: 117 additions & 6 deletions desktop/docs/services-parity.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,15 +267,126 @@ the engine simply shows as stopped and then running.

A local `pull_model` streams live download progress: the engine-manager routes
`engine:action{pull_model}` through its streaming pull path and emits
`engine:pull-progress` (`{ engine, op, stage, percent, message }`) — the local
`engine:pull-progress` (`{ engine, model, op, stage, percent, message }`) — the local
counterpart of `engine:remote-progress`. Personal AI Router consumes it in
`applyLocalEngineProgress` (`modular-supervisor.ts` → `modular-state.ts`),
backfilling the dispatched model (the frame carries none) and advancing the
optimistic pull entry's percent in place, so a local pull shows "Pulling · N%"
keying the frame to its optimistic pull entry by model and advancing that
entry's percent in place, so a local pull shows "Pulling · N%"
to completion just like a remote pull. The awaited action response owns
completion (clearing the entry and refreshing the model list); a CLI-driven pull
(LM Studio) emits a single `pulling` marker and degrades to the indeterminate
spinner.
completion (clearing the entry and refreshing the model list). LM Studio's CLI
progress updates the same percentage display. A pull for an engine that already
has one running is reported as `stage:"queued"` until its turn.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify that it joins onto an existing download? We don't download it twice, right?


The pull request itself stays pending for the whole download, so it is the
request's own response that reports the outcome — a completion or a
cancellation — while progress arrives out of band on the notification:

```mermaid
sequenceDiagram
participant UI as Model Manager
participant Bridge as Electron bridge
participant Manager as Engine manager
participant Engine as Ollama or LM Studio

UI->>Bridge: Download model
Bridge->>UI: Show optimistic "Pulling…" row
Bridge->>Manager: Start pull (request remains pending)
Manager->>Manager: Record the partial files already on disk
Manager->>Engine: Begin download

loop While downloading
Engine-->>Manager: Download status
Manager-->>Bridge: Progress {engine, model, stage, percent}
Bridge-->>UI: Update the matching model row
end

alt Download completes
Engine-->>Manager: Success
Manager-->>Bridge: Original pull request completes
else User cancels
UI->>Bridge: Cancel download
Bridge->>UI: Show "Canceling…"
Bridge->>Manager: Cancel {engine, model}
Manager->>Engine: Stop transfer
Manager->>Manager: Remove the partial files this pull created
Manager-->>Bridge: Cancellation completes
Manager-->>Bridge: Original pull settles as canceled
end

Bridge->>Manager: Refresh model list
Manager-->>Bridge: Authoritative models
Bridge->>UI: Replace temporary download state
```

`engine:cancel-pull` and `engine:remote-cancel-pull` cancel the selected model
download. The UI keeps "Canceling…" visible until the pull settles, including
when the cancel RPC's own budget elapses first — the backend is still working,
so the row is not dropped back to "Downloading". Ollama closes the pull request
and removes partial blobs named by its progress digests; LM Studio receives
Ctrl+C and a negative answer to its background-download prompt, and after
acknowledgement the partial files of the model file it was fetching are
removed. That file is identified by the requested quantization when the request
pins one, and otherwise by the file names the CLI printed; an unpinned pull
whose output named no file removes nothing. Completed model files, other
quantizations, and unrelated downloads are retained, and a cancellation the CLI
never confirmed deletes nothing. Only a cancellation someone requested removes
files: a pull interrupted by app shutdown, a dropped remote connection, or the
action timeout leaves its partial data resumable.

Cleanup looks for partial files only in the default model directories:
`~/.lmstudio/models` for LM Studio, and for Ollama the `OLLAMA_MODELS` the
engine-manager launches it with (`~/.ollama/models` by default). A cancellation
on an install whose models live elsewhere — a relocated LM Studio models
directory, or an adopted Ollama daemon started with a different
`OLLAMA_MODELS` — stops the download but leaves its partial files for the
engine to resume or remove.

Both engines funnel every client on the machine into one cache, so naming a file
is not the same as owning it — Ollama blobs are content-addressed, and the LM
Studio app downloads into the same repository directory. Each pull therefore
records the partial files already present before it starts, and cancelling it
considers only the ones that appeared afterwards. A file that is still growing
once the transfer has stopped is shared with another client even by that
measure, so it survives as well:

```mermaid

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diagram might want to mention if the existing request is already in progress. From trackedPull "A second request for a download already in flight joins it"

flowchart TD
Request["Model pull requested"] --> Tracked["Track by engine + model"]
Tracked --> Busy{"Another PAIR pull active<br/>for this engine?"}

Busy -- No --> Snapshot["Record the partial files<br/>already on disk"]
Busy -- Yes --> Queued["Show as queued"]

Queued --> QueueCancel{"Canceled while queued?"}
QueueCancel -- Yes --> NeverStarted["Remove operation<br/>without starting download"]
QueueCancel -- No --> Wait["Wait for active pull to settle"]
Wait --> Snapshot

Snapshot --> Active["Start download"]
Active --> Cancel{"Cancellation requested?"}
Cancel -- No --> Complete["Download completes normally"]
Cancel -- Yes --> Engine{"Which engine?"}

Engine -- Ollama --> OllamaStop["Close HTTP pull request"]
OllamaStop --> OllamaCandidates["Select partial blobs named<br/>by this pull's digests"]

Engine -- LM Studio --> LMSStop["Send Ctrl+C"]
LMSStop --> LMSPrompt["Answer no to background download"]
LMSPrompt --> LMSCandidates[".part files of the pinned<br/>quantization or a file the CLI named"]

OllamaCandidates --> Owned{"Absent from<br/>the snapshot?"}
LMSCandidates --> Owned
Owned -- No --> Preserve["Preserve completed,<br/>shared, and unrelated files"]
Owned -- Yes --> Quiet{"Held still across<br/>repeated checks?"}
Quiet -- No --> Preserve
Quiet -- Yes --> Delete["Remove the partial file"]

Delete --> Settle["Settle pull and refresh models"]
Preserve --> Settle
Complete --> Settle
NeverStarted --> Settle
Settle --> Next["Allow next queued pull to start"]
```

`engine:models` (and the `em` `GET /v1/models` surface) returns the flat model
union, the per-engine breakdown (`modelsByEngine`), and the per-engine set of
Expand Down
10 changes: 10 additions & 0 deletions desktop/src/electron/service-bridge/empty-handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,11 @@ function routeEngineManagerCommand(payload: WsInvokeRequest<'engine:command'>):
void supervisor.pullModel(engine, payload.engineType, payload.model)
}
break
case 'cancelModelPull':
if (payload.model) {
void supervisor.cancelModelPull(engine, payload.engineType, payload.model)
}
break
case 'deleteModel':
if (payload.model) {
void supervisor.deleteModel(engine, payload.engineType, payload.model)
Expand Down Expand Up @@ -365,6 +370,11 @@ function routeRemoteEngineCommand(payload: WsInvokeRequest<'engine:command'>): v
void supervisor.pullModelRemote(nodeId, engine, payload.engineType, payload.model)
}
break
case 'cancelModelPull':
if (payload.model) {
void supervisor.cancelModelPull(engine, payload.engineType, payload.model, nodeId)
}
break
case 'uninstall':
case 'update':
refuseRemote(
Expand Down
9 changes: 8 additions & 1 deletion desktop/src/electron/service-bridge/json-rpc-subprocess.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ interface JsonRpcError {

export class JsonRpcResponseError extends Error {}

/**
* The request's own budget elapsed. The backend has not answered and has not
* failed either — it is still working — so a caller that showed optimistic
* state must decide whether to keep it rather than assume the operation lost.
*/
export class JsonRpcTimeoutError extends Error {}

type JsonRpcId = number | string

interface JsonRpcMessage {
Expand Down Expand Up @@ -148,7 +155,7 @@ export class JsonRpcSubprocess extends EventEmitter<JsonRpcSubprocessEvents> {
? undefined
: setTimeout(() => {
this.pending.delete(id)
reject(new Error(`${this.name} ${method} timed out`))
reject(new JsonRpcTimeoutError(`${this.name} ${method} timed out`))
}, timeoutMs)
this.pending.set(id, { resolve, reject, timeout })
this.write(message).catch(err => {
Expand Down
Loading
Loading