⚔️ Local LLM Arena

Compare your local LLMs side by side — stream, score, and pick the best, 100% on your machine.

_{One prompt → every model answers in parallel, with live metrics and a 👑 on the fastest.}

---

Features · Quick start · How it works · Tech stack · Development

---

What it is

Local LLM Arena sends one prompt to up to six local models at once and shows their answers side by side, streaming in real time. Compare speed and quality, run the same model at different settings, and vote for the best answer blind — without ever sending a byte to the cloud.

It runs entirely against a local Ollama instance. No API keys, no accounts, no telemetry. Built for prompt engineers, researchers, and anyone evaluating models on private or sensitive data.

Comparison is the point. A single-model chat is just an arena with one model in it.

Why not just chat?

Other local UIs let you talk to models. Local LLM Arena is built to evaluate them — blind voting, an automated LLM judge, a private Elo leaderboard, and reproducible batch benchmarks. It's a Chatbot Arena you run on your own machine, on your own prompts.

Capability	Local LLM Arena	Open WebUI	LM Studio	Chatbot Arena (cloud)
Runs 100% locally, data stays private	✅	✅	✅	❌ cloud
Side-by-side, up to 6 models in parallel	✅	✅	❌	✅ (2, random)
Blind evaluation + 👍/👎 vote	✅	❌	❌	✅
Automated LLM-as-judge (local or cloud key)	✅	❌	❌	❌
Private Elo leaderboard across runs	✅	❌	❌	✅ (public)
Batch benchmark → reproducible Elo report	✅	❌	❌	❌
Per-model hyperparameter sweeps	✅	partial	✅	❌
Open source (MIT)	✅	✅	❌	✅

Features

• 🆚 Side-by-side comparison — one prompt → up to 6 models answering in parallel, each in its own streaming card.
• 📊 Compare metrics — tokens/sec, time-to-first-token, and token count per model, with a 👑 crown on the fastest.
• 🎭 Blind evaluation — hide model names (“Model A/B/C”), randomize order, vote 👍/👎, then reveal the mapping and vote tally. Voting locks on reveal to keep results honest.
• 🧑‍⚖️ Auto-judge (LLM-as-judge) — score the answers automatically and pick a winner, using a local model or a cloud model with your own API key (Anthropic, OpenRouter, or any OpenAI-compatible endpoint). Answers are anonymized to the judge to avoid bias; cloud judging is opt-in with a clear privacy notice (keys stay in memory only — never written to disk or logs). Candidate order is randomized per call and judge sampling is pinned (temperature 0), so verdicts are position-bias-mitigated and reproducible.
• 🏆 Elo leaderboard — a running, cross-comparison ranking of your models (an offline, private Chatbot Arena), built from judge scores and 👍/👎 votes via pairwise Elo, with 95% bootstrap confidence intervals so you can tell signal from small-sample noise.
• 🧪 Batch benchmark — run a whole prompt set (paste or load .txt/.jsonl/.csv) across your models, auto-judge every prompt, and aggregate a reproducible Elo report — with confidence intervals and a win matrix — exportable as Markdown or JSON for sharing or publication.
• ⚙️ Per-model hyperparameters — temperature, top-p, top-k, repeat-penalty, max-tokens, seed. Run the same model at different settings as separate entries.
• 📦 Model manager — list, pull, and delete Ollama models from the UI.
• 📎 File attach — drop in a text/code file; it’s read locally into your prompt.
• 💾 Export — download a comparison as JSON (masked while blind, full after reveal).
• 🗂️ Sessions — multiple saved comparisons, persisted locally.
• 🌗 Polished UI — distinctive dark/light theme, sanitized markdown + code highlighting, tooltips on every control, keyboard-accessible.

🎭 Blind evaluation in action

_{Hide the names, judge on quality alone, vote, then reveal who was who.}

More screenshots

Quick start (one command)

Prerequisites: Python 3.11+, Node 20+, and Ollama running with at least one model:

ollama pull gemma3:1b

Then, from the project root:

# Windows
./start.ps1

# macOS / Linux
./start.sh

This installs the backend (virtual env + deps) and frontend (npm + production build), then serves the whole app as a single local process at http://127.0.0.1:7860.

Or with Docker

Ollama runs on the host (keeping your GPU); the app runs in a container and reaches it:

docker compose up --build      # -> http://localhost:7860

How it works

Browser (React 19 + Vite + Tailwind v4 + shadcn/ui)
   │  fetch + NDJSON stream  (same-origin /api/*)
   ▼
FastAPI (async, Pydantic-validated)
   │  ollama-python AsyncClient
   ▼
Ollama  ·  localhost:11434  ·  your models, your hardware

• One async backend is the single source of truth. Each model streams one generation (no double calls); all six hyperparameters flow through one code path; out-of-range values are rejected with 422, never a 500.
• The frontend streams each model independently, so cards fill in live and you can regenerate or stop any one of them.
• In production the SPA is built and served by FastAPI itself — one process, one URL, no CORS. In development, Vite hot-reloads and proxies /api to the backend.

See docs/adr/0001-fastapi-react.md for the architecture decision record, and the live OpenAPI docs at http://127.0.0.1:7860/docs.

📄 Technical report — the evaluation methodology in detail: blind-vote bias controls, anonymized LLM-as-judge, the exact pairwise-Elo math, a reproducibility protocol, and an honest limitations section.

Tech stack

Layer	Choice
Frontend	React 19, Vite 8, TypeScript, Tailwind v4, shadcn/ui + Radix, Zustand
Backend	FastAPI, Pydantic, ollama-python (async)
Inference	Ollama (local)
Fonts	Bricolage Grotesque · Hanken Grotesk · JetBrains Mono (bundled offline)

Development

# hot-reload dev (Vite on :5173 proxying to uvicorn on :7860)
./scripts/dev.ps1            # Windows

# or manually
cd backend  && python -m venv .venv && ./.venv/Scripts/python -m pip install -e ".[dev]"
            && uvicorn app.main:app --reload --port 7860
cd frontend && npm install && npm run dev

# tests
cd backend  && pytest          # API + hyperparameter passthrough
cd frontend && npm test        # store/instance helpers + NDJSON stream parser

Project structure

backend/    FastAPI app — routers, services/ollama.py, schemas, tests
frontend/   React app — components/arena, store/arena.ts, lib (api, sse), styles
docs/       ADR, assets (screenshots)
start.ps1 · start.sh   one-command install & run

Privacy

Everything runs locally. There are no external network calls — models run through Ollama on your hardware, fonts are bundled, and conversation history lives only in your browser’s local storage. Nothing is uploaded, logged remotely, or sent for training.

Credits & License

Built on Ollama, FastAPI, React, Tailwind, shadcn/ui, Radix, and Uiverse-style components — see CREDITS.md. Licensed under the MIT License.

Uses Ollama (© Ollama, Inc.), a separate product with its own license.

_{Made for privacy-conscious AI practitioners.}