Skip to content

Commit fd1756f

Browse files
docs: OpenRouter response caching guide + rendered architecture diagram (#144)
* docs(examples): document OpenRouter response caching via extra_headers OpenRouter Response Caching (sending X-OpenRouter-Cache / -TTL headers on compiler LLM calls) is achievable through the general extra_headers passthrough — no dedicated config needed. Document it in the configuration guide alongside the other litellm/extra_headers recipes. Refs #39 * docs: replace ASCII architecture diagram with a rendered image Swap the text-based diagram under "Architecture" for a rendered illustration (assets/openkb-architecture.png) covering the same pipeline: raw/ intake -> markitdown (short docs) / PageIndex (long PDFs) -> LLM wiki compilation -> the wiki/ foundation -> query/chat, the Skill Factory, and future generators.
1 parent 8d40458 commit fd1756f

3 files changed

Lines changed: 24 additions & 28 deletions

File tree

README.md

Lines changed: 3 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -119,34 +119,9 @@ LLM_API_KEY=your_llm_api_key
119119

120120
### Architecture
121121

122-
```
123-
raw/ You drop files here
124-
125-
├─ Short docs ──→ markitdown ──→ LLM reads full text
126-
│ │
127-
├─ Long PDFs ──→ PageIndex ────→ LLM reads document trees
128-
│ │
129-
│ ▼
130-
│ Wiki Compilation (using LLM)
131-
│ │
132-
▼ ▼
133-
wiki/ │ ← the foundation
134-
├── index.md Knowledge base overview
135-
├── log.md Operations timeline
136-
├── AGENTS.md Wiki schema (LLM instructions)
137-
├── sources/ Full-text conversions
138-
├── summaries/ Per-document summaries
139-
├── concepts/ Cross-document synthesis
140-
├── entities/ Specific named things (people, orgs, places, products)
141-
├── explorations/ Saved query results
142-
└── reports/ Lint reports
143-
144-
┌──────────────────────┼──────────────────────┐
145-
▼ ▼ ▼
146-
query / chat Skill Factory (future)
147-
(LLM answers from (redistributable ppt / podcast /
148-
the wiki) agent skills) report / …
149-
```
122+
<div align="center">
123+
<img src="assets/openkb-architecture.png" alt="OpenKB Architecture: from raw documents (markitdown / PageIndex) through LLM wiki compilation to the wiki/ foundation, powering query/chat, the Skill Factory, and future generators" width="900" />
124+
</div>
150125

151126
### Short vs Long Document Handling
152127

assets/openkb-architecture.png

1.7 MB
Loading

examples/configuration/README.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,27 @@ litellm:
136136
Copilot-Integration-Id: vscode-chat
137137
```
138138
139+
#### OpenRouter response caching
140+
141+
When your `model` is an `openrouter/*` model, you can opt into OpenRouter's
142+
[Response Caching](https://openrouter.ai/docs/guides/features/response-caching):
143+
identical-payload requests come back in ~80–300 ms with **zero token billing**.
144+
That's a direct win on the compile-retry path (a failed `add` re-runs every
145+
summary/plan/concept call with the same prompts) and on repeated `lint` / dev
146+
iteration. Send the cache headers via `extra_headers`:
147+
148+
```yaml
149+
model: openrouter/anthropic/claude-sonnet-4.5
150+
language: en
151+
extra_headers: # top-level, or nested under `litellm:` — both work
152+
X-OpenRouter-Cache: "true"
153+
X-OpenRouter-Cache-TTL: "600" # optional, 1–86400s (OpenRouter default 300)
154+
```
155+
156+
It's opt-in by design: responses are stored on OpenRouter, so leave it off for
157+
zero-data-retention / regulated content. Only `openrouter/*` models read these
158+
headers; other providers ignore them.
159+
139160
---
140161

141162
## 3. API keys & providers

0 commit comments

Comments
 (0)