Skip to content

Probe manual nodes for llmman on 17434 as an Ollama-API engine - #15

Open
ericcurtin wants to merge 1 commit into
NVIDIA:developfrom
ericcurtin:llmman
Open

ericcurtin wants to merge 1 commit into
NVIDIA:developfrom
ericcurtin:llmman

Conversation

@ericcurtin

@ericcurtin ericcurtin commented Sep 4, 2026 •

Copy link
Copy Markdown

Description

Changelog title

Probe llmman on port 17434 for manual nodes

Changelog body

  • Manual nodes running llmman are now detected as Ollama-API engines.

Bumps

  • services: none
  • nvpair-cluster-manager: none
  • nvpair-engine-manager: none
  • nvpair-errors: none
  • nvpair-job-scheduler: none
  • nvpair-manual-nodes: minor
  • nvpair-node-info: none
  • nvpair-node-scanner: none
  • nvpair-node-settings: none
  • nvpair-proxy: none
  • nvpair-tui: none
  • nvpair-ui-broker: none
  • nvpair-workload-manager: none

llmman is a local model runner that serves the Ollama API on port 17434. Manual nodes were probed for Ollama on 11434 only, so a device running llmman was reported as having no inference engine even though ollama-proxy can route to it unchanged.

  • manager.go: probeOllamaAPI tries 11434 then 17434 and reports the answering port in ollama_port; Ollama wins when both answer. 11434 literals become an ollamaPort constant.
  • No broker/proxy/desktop changes: ollama_port is already forwarded to ollama-proxy, which already dials it. llmman is not added as a new engine type, per CONTRIBUTING.
  • manager_test.go: TestProbeOllamaAPIFallsBackToLLMMan covers Ollama-wins, llmman-only and neither.
  • README documents the probe order.

Scope

In: probing llmman on its default port, docs, test, version bump. Out: llmman as a managed local engine, mDNS-discovered nodes, non-default ports.

Validation

Testing: gofmt -l ., go vet ./..., go test ./... -count=1 in services/nvpair-manual-nodes and services/nvpair-ui-broker - clean (macOS arm64, Go 1.26). Not run against a live llmman/Ollama through the full broker/proxy path.

Risk

IPC shapes unchanged; ollama_port may now be 17434. One extra plain-HTTP probe per node per cycle when 11434 does not answer.

Checklist

  • I have read the Contributing Guidelines.
  • Every commit is signed off (git commit -s), certifying the Developer Certificate of Origin.
  • New or existing tests cover the change.
  • Relevant documentation is updated.
  • I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
  • I recorded the validation commands and results above.
  • I declared version bumps in the release-intent block above. services/versions.json is written by automation, not edited by hand.

AI-assisted, reviewed before submitting.

@Noah-Tervalon-Nvidia

Copy link
Copy Markdown
Collaborator

Thanks for opening this up and for your interest in PAIR! We're going to look into this and decide whether or not we want to bring this in.

@ericcurtin

Copy link
Copy Markdown
Author

Thanks for opening this up and for your interest in PAIR! We're going to look into this and decide whether or not we want to bring this in.

@Noah-Tervalon-Nvidia cool, any feedback let me know

@Noah-Tervalon-Nvidia
Noah-Tervalon-Nvidia changed the base branch from main to develop September 21, 2026 21:54
llmman (https://github.com/llmmanorg/llmman) serves the Ollama API on
17434, but manual nodes were only probed for Ollama on 11434, so a node
running llmman showed no inference engine. Retry the probe on 17434 and
report the answering port in ollama_port; the broker and ollama-proxy
already honour that field, so nothing else changes. Ollama wins when
both answer.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@ericcurtin

Copy link
Copy Markdown
Author

Rebased onto develop.

@ericcurtin

Copy link
Copy Markdown
Author

@Noah-Tervalon-Nvidia @ckelseynv Rebased and added the release intent block, PTAL. Thank you!

@ericcurtin

Copy link
Copy Markdown
Author

@nv-pgoode @kjlubick PTAL when you get a chance. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants