Skip to content

Commit 1c3105e

Browse files
committed
docs(examples): document pre-release install + slow local-runtime timeout
Cover recurring install/usage questions from the issue tracker: - #130 / #24: openkb pins a pre-release dependency (pageindex==0.3.0.dev1), which uv/pip skip by default. Show `uv tool install --prerelease=allow` / `pip install --pre`, plus a PATH note for the "command not found" case. - #140: local runtimes (LM Studio on Mac, Ollama, llama.cpp) abort on the default request timeout; document raising litellm.timeout in config. Claude-Session: https://claude.ai/code/session_018WiFnTo1YW9mtw47Fzir9K
1 parent 9530615 commit 1c3105e

1 file changed

Lines changed: 28 additions & 5 deletions

File tree

examples/configuration/README.md

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,27 @@ Everything that controls how OpenKB talks to your LLM lives in two places:
55

66
---
77

8+
## Install
9+
10+
```bash
11+
pip install openkb
12+
```
13+
14+
OpenKB pins a **pre-release** of its PageIndex dependency
15+
(`pageindex==0.3.0.dev1`), which some installers skip by default. If an install
16+
can't resolve `pageindex`, allow pre-releases:
17+
18+
```bash
19+
uv tool install openkb --prerelease=allow # uv
20+
pip install --pre openkb # pip
21+
```
22+
23+
If `openkb` isn't found *after* a successful install, the console-script directory
24+
isn't on your `PATH` (e.g. `pip --user` installs to `~/.local/bin`) — add it to
25+
`PATH`.
26+
27+
---
28+
829
## 1. Initialize a knowledge base
930

1031
```bash
@@ -85,17 +106,19 @@ you set it, LiteLLM uses it. Two keys are special:
85106
- Every other key (`drop_params`, `num_retries`, `ssl_verify`, …) is set on the
86107
`litellm` module as a process-wide global.
87108

88-
#### A local Ollama setup
109+
#### Slow local runtimes (Ollama, LM Studio, llama.cpp)
89110

90-
Ollama rejects some OpenAI-style params; `drop_params` lets LiteLLM strip them
91-
instead of erroring, and a generous `timeout` covers slow local inference:
111+
Local inference can be slow — on a Mac running **LM Studio**, a single compile
112+
call can take minutes, and the **default request timeout will abort it** (this is
113+
the usual cause of failures with local runtimes). Raise `timeout` (in seconds).
114+
Add `drop_params` for backends that reject OpenAI-only params (e.g. Ollama):
92115

93116
```yaml
94-
model: ollama/llama3.1
117+
model: ollama/llama3.1 # or your LM Studio / llama.cpp model id
95118
language: en
96119
litellm:
97120
drop_params: true
98-
timeout: 1200
121+
timeout: 1200 # raise further (e.g. 3600) for large local models
99122
```
100123
101124
#### GitHub Copilot / ChatGPT-subscription providers

0 commit comments

Comments
 (0)