Your experts. Your GPU. Your data never leaves.
NeuralForge gives every GPU owner a team of domain experts on their desk, aligned to their research, tracking their field, always available, never forgetting.
Status: active development / reference architecture (not a turnkey deploy yet). The architecture, GPU graph algorithms, layered retrieval engine, and a 1,006-test suite are built, but NeuralForge does not yet run cleanly end-to-end from a fresh clone. Known gaps in the current quickstart: some examples reference a prior
/api/v1/*contract being migrated, Qdrant collections need manual init, and Triton requires model-artifact setup. Treat this as a serious work-in-progress reference, not a finished product. See Status and Known Limitations for specifics.
Most knowledge platforms choose between cloud convenience and data control. NeuralForge refuses that trade-off.
| Capability | LangChain + OpenAI | Cloud RAG (Pinecone) | NeuralForge |
|---|---|---|---|
| Data stays on your hardware | Yes | ||
| GPU-accelerated graph algorithms | cuGraph (Louvain, PageRank) | ||
| Expert contradiction detection | Automatic | ||
| Hallucination guardrails | Partial | NeMo Guardrails | |
| Temporal knowledge tracking | Valid-from/to on every edge | ||
| Multi-source ingestion (blogs, YouTube, arXiv, docs) | Plugin | Plugin | Built-in |
| Single-command deployment | docker compose up |
||
| PII scrubbing before storage | Built-in |
NeuralForge Stack
┌──────────────────────────────────────────────────────┐
│ │
│ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ NeuralForge │────▶│ NIM (TensorRT-LLM) │ │
│ │ API :8090 │ │ Llama 3.1 8B :8000 │ │
│ │ │ └──────────────────────────┘ │
│ │ FastAPI │ │
│ │ cuGraph │ ┌──────────────────────────┐ │
│ │ NeMo Rails │────▶│ Triton Inference Server │ │
│ │ Layered CTX │ │ Embed + Rerank :8001 │ │
│ │ │ └──────────────────────────┘ │
│ │ │ │
│ │ │ ┌──────────────────────────┐ │
│ │ │────▶│ Qdrant │ │
│ │ │ │ Vector DB :6333 │ │
│ └──────────────┘ └──────────────────────────┘ │
│ │
│ ┌────────────────┐ │
│ │ NVIDIA GPU(s) │ │
│ │ CUDA 12.5 │ │
│ └────────────────┘ │
└──────────────────────────────────────────────────────┘
Four containers, one GPU, zero cloud dependencies.
| Container | Role | NVIDIA Tech | Port |
|---|---|---|---|
neuralforge-api |
Knowledge platform API | RAPIDS cuGraph, NeMo Guardrails | 8090 |
neuralforge-nim |
LLM inference (TensorRT-LLM optimized) | NIM, TensorRT-LLM | 8000 |
neuralforge-triton |
Embedding + reranking | Triton Inference Server | 8001 |
neuralforge-qdrant |
Vector storage | -- | 6333 |
# 1. Clone
git clone https://github.com/NathanMaine/neuralforge.git
cd neuralforge
# 2. Configure
cp .env.example .env
# Edit .env and add your NGC_API_KEY
# 3. Preflight check
bash check_env.sh
# 4. Launch
docker compose up -dThis is the intended deployment. Note: a fresh clone currently has setup gaps (Qdrant collection init, Triton model artifacts, and some examples referencing a prior API contract). See Status and Known Limitations below before expecting a clean first run.
import httpx
# Add Tim Dettmers' blog as a knowledge source (current contract: POST /api/ingest/blog)
resp = httpx.post("http://localhost:8090/api/ingest/blog", json={
"url": "https://timdettmers.com",
"name": "Tim Dettmers",
"creator": "Tim Dettmers",
"strategy": "auto",
"max_articles": 50,
})
print(resp.json())Or use the CLI examples:
# YouTube channel
python examples/ingest_youtube.py --channel "3Blue1Brown" --name "Grant Sanderson"
# arXiv papers
python examples/ingest_arxiv.py --author "Yann LeCun" --max-papers 20
# Local documents
python examples/ingest_documents.py --folder ./papers --expert "Research Team"# Retrieve layered context (returns search context, not a generated answer)
curl "http://localhost:8090/api/search?q=What+is+LoRA+and+why+does+it+work&layers=3"
# Or get a generated answer via the OpenAI-compatible endpoint
curl -X POST http://localhost:8090/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages": [{"role": "user", "content": "What is LoRA and why does it work?"}]}'Every response flows through 4 context layers:
| Layer | Source | Purpose |
|---|---|---|
| 0 | System identity | Expert persona + attribution rules |
| 1 | cuGraph | PageRank rankings + contradiction warnings |
| 2 | Qdrant + Triton | AAAK-compressed vector search chunks |
| 3 | Deep search | Full uncompressed high-relevance passages |
NeuralForge doesn't just store knowledge -- it discovers structure.
# Where do your experts disagree?
python examples/find_contradictions.py --topic "quantization"When Expert A says "4-bit quantization preserves 99% of performance" and Expert B says "anything below 8-bit significantly degrades reasoning" -- NeuralForge surfaces that disagreement, with citations to both sources.
# Who clusters together?
python examples/expert_communities.pyLouvain community detection (GPU-accelerated via cuGraph) finds natural clusters: researchers who cite each other, authors covering the same topics, institutions with shared methodologies.
# Who carries the most weight on a topic?
python examples/influence_ranking.py --topic "transformer architectures"PageRank computed across the knowledge graph, weighted by topic-specific edge density. Not popularity -- structural authority.
NeuralForge runs graph algorithms directly on the GPU via RAPIDS cuGraph, with automatic networkx fallback for CPU-only environments.
| Algorithm | What It Does | Use Case |
|---|---|---|
| PageRank | Scores node importance by link structure | Expert authority ranking |
| Louvain | Detects communities in the graph | Expert clustering by domain |
| BFS Traversal | Walks the graph from any node | "Show me everything connected to LoRA" |
| Shortest Path | Finds the connection between two nodes | "How is Expert A related to Concept B?" |
| Temporal Filtering | Valid-from/to on every edge | "What was the consensus in 2023?" |
Every query passes through NeMo Guardrails -- three rail types protecting the pipeline:
- PII Scrubbing -- personal information is stripped before it enters the system
- Jailbreak Detection -- prompt injection attempts are caught and blocked
- Topic Relevance -- off-topic queries are redirected
- Hallucination Check -- expert citations are verified against the knowledge graph
- Attribution Verification -- every named expert must exist as a graph node
- Provenance Tracking -- source metadata is appended to every response
- Relevance Filtering -- retrieved chunks are validated for query relevance before reaching the LLM
A reproducible benchmark harness lives in benchmarks/ and measures ingestion (blog with 50 articles), search p95 (single query with graph context), and GPU PageRank over a 10K-node graph, writing results to benchmarks/results/. Measured numbers are in progress and will be published here rather than estimated; target hardware is the DGX Spark (GB10, 128GB) and RTX 4090 (24GB).
NeuralForge is in active development. The architecture, GPU graph algorithms, layered retrieval, NeMo Guardrails integration, and a 1,006-test suite (34 modules, all passing locally) are in place; several operational edges are still being hardened, tracked against docs/CURRENT_API_CONTRACT.md:
- API contract: the current API is served under
/api/*and/v1/*. Some example scripts and older snippets reference a prior/api/v1/*contract and are being migrated; the contract doc lists current routes and a migration map. - Qdrant: a fresh volume currently expects collection initialization; automatic init is in progress.
- Triton: embedding and reranking require the model artifacts described in the docs; a preflight check is being added.
- /health: top-level status can report healthy while a dependency (for example Qdrant) is degraded; health semantics are being tightened.
- Guardrails: NeMo Guardrails are optional at runtime; when disabled,
/healthreflects it.
This is a single-author reference platform, not a turnkey product. Issues and PRs welcome.
neuralforge/
forge/
api/ # FastAPI application + middleware
core/ # NIM client, Triton client, Qdrant client, embeddings
graph/ # cuGraph engine, Parquet store, auto-discovery
guardrails/ # NeMo Guardrails config + custom actions
ingest/ # Blog scraper, document loader, chunker, PII scrubber
layers/ # Layered context engine + AAAK compression
workers/ # Background jobs (discovery, scraping)
mcp/ # Model Context Protocol integration
triton/models/ # Triton model configs (embedding, reranker)
examples/ # 8 standalone quickstart scripts
tests/ # 1,006 passing tests across 34 modules
benchmarks/ # Performance measurement harness
docs/ # Architecture, guides, NVIDIA stack docs
| Guide | Description |
|---|---|
| Architecture | Deep dive on system design and data flow |
| Quickstart | 5-minute deployment guide |
| Ingestion Guide | How to feed your data into NeuralForge |
| NVIDIA Stack | Why each NVIDIA technology was chosen |
| Graph Algorithms | cuGraph operations explained |
| Guardrails Guide | Configuring NeMo Guardrails |
- NVIDIA GPU with 16GB+ VRAM (24GB+ recommended)
- NVIDIA Driver 535+, CUDA 12.5+
- Docker with NVIDIA Container Toolkit
- NGC API key (get one here)
- 50GB+ free disk space (NIM model cache)
NVIDIA Inception Member. Building tools that put GPU-accelerated AI in the hands of individual researchers and small teams.
- GitHub: NathanMaine
- HuggingFace: Nathan-Maine
- LinkedIn: Nathan Maine
Apache 2.0. See LICENSE.