Lyric

AI-powered Telegram chatbot with knowledge graph memory and extensible MCP tool framework.

Architecture Overview

                          ┌──────────────────────────┐
                          │     External Services     │
                          │  ┌────────┐ ┌──────────┐ │
                          │  │ Neo4j  │ │  Ollama  │ │
                          │  │(Graph) │ │(Embedding)│ │
                          │  └────────┘ └──────────┘ │
                          │  ┌────────────────────┐  │
                          │  │ AI Model Server(s) │  │
                          │  │ (OpenAI-compatible)│  │
                          │  └────────────────────┘  │
                          │  ┌──────────┐            │
                          │  │Home Asst.│ (optional) │
                          │  └──────────┘            │
                          └──────────────────────────┘
                                     ▲
                                     │ HTTP/bolt
                                     │
 Telegram User ──► Telegram API ──► Your Server
       │              (or proxy)      │
       │                              ▼
       │    long-polling      ┌───────────────┐    HTTP POST    ┌──────────────┐
       └─────────────────────►│  telebot.py   │ ───────────────►│ main.py:8000 │
                              │  (bot loop)   │                 │  (FastAPI)   │
                              └───────────────┘                └──────┬───────┘
                                                                     │ /chat
                                                                     ▼
                                                          ┌─────────────────────┐
                                                          │  MCPClientAPI        │
                                                          │  (tool dispatch)     │
                                                          └─────────┬───────────┘
                                                                    │
                                              ┌─────────────────────┼─────────────────────┐
                                              ▼                     ▼                     ▼
                                    ┌──────────────┐      ┌──────────────┐      ┌──────────────┐
                                    │ DECIDER      │      │  Neo4j       │      │  MCP Servers │
                                    │ Model        │      │  GraphRAG    │      │  (subprocess)│
                                    └──────────────┘      └──────────────┘      └──────────────┘

Server Requirements

Services You Need to Deploy

Service	Role	Connection	Required
AI Model API	Chat, memory extraction, memory search	HTTP (OpenAI-compatible `/v1/chat/completions`)	Yes — 3 models: DECIDER, MEM, SEARCH
Neo4j 5.x	Knowledge graph storage + vector index	Bolt (`neo4j://` or `bolt://`)	Yes — requires APOC plugin
Ollama	Text embedding for vector search	HTTP (default `http://localhost:11434`)	Yes — embedding model only
Home Assistant	Smart home device control	HTTP REST API	Optional — requires long-lived token
Telegram API Proxy	GFW bypass for Telegram API	HTTP reverse proxy	Optional — see proxy setup below
OpenCode CLI	Shell command execution via AI	Local CLI (`opencode run`)	Yes — installed on the bot server
Amap (高德) API	Weather queries	HTTP REST API	Optional — for weather tool
Node.js + npm	Web search MCP server	Local npx	Optional — for web search tool

Detailed Server Setup

1. AI Model Server (OpenAI-compatible API)

Must serve three distinct model roles via a single endpoint. Each role can use the same or different models:

DECIDER: Main conversational model with tool-calling support. Receives system prompt, user messages, and tool definitions.
MEM: Memory extraction model. Receives raw conversation text, outputs knowledge graph entities and relationships (JSON mode required).
SEARCH: Memory search model. Generates Cypher queries and re-ranks retrieval results.

Configuration:

# Base URL (can be the same for all three)
DECIDER_BASE_URL=http://your-model-server:8080/v1
MEM_BASE_URL=http://your-model-server:8080/v1
SEARCH_BASE_URL=http://your-model-server:8080/v1

# API keys
DECIDER_API_KEY=sk-xxx
MEM_API_KEY=sk-xxx
SEARCH_API_KEY=sk-xxx

# Model names
DECIDER_MODEL_NAME=qwen3-235b-a22b
MEM_MODEL_NAME=qwen3-235b-a22b
SEARCH_MODEL_NAME=qwen3-235b-a22b

The DECIDER model must support:

Tool calling (tools parameter in chat completions)
Chat template kwarg enable_thinking: false

The MEM model must support:

JSON response format (response_format: {type: "json_object"})
Large context windows (20K+ tokens recommended)

2. Neo4j Database

Installation:

Install Neo4j 5.x (Community or Enterprise)

Install the APOC plugin:

# Download APOC jar matching your Neo4j version
wget https://github.com/neo4j/apoc/releases/download/5.14.0/apoc-5.14.0-all.jar
cp apoc-5.14.0-all.jar /path/to/neo4j/plugins/

Restart Neo4j

After Neo4j is running, create the vector index:

CREATE VECTOR INDEX chunks IF NOT EXISTS
FOR (n:Chunk) ON (n.embedding)
OPTIONS {indexConfig: {
  `vector.dimensions`: 1024,
  `vector.similarity_function`: 'cosine'
}}

Verify embedding dimensions (run after Ollama is set up):

python src/back/embedding_dimensions.py

Configuration:

NEO4J_URI=neo4j://your-neo4j-host:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=your-password

3. Ollama (Embedding Server)

ollama pull Qwen3-Embedding:0.6B   # or your preferred embedding model

Ollama must be accessible from the bot server. Default connection is http://localhost:11434 (configured via OLLAMA_HOST env var if remote).

Configuration:

EMBEDDER=Qwen3-Embedding:0.6B

4. Home Assistant (Optional)

Generate a long-lived access token in Home Assistant: Profile → Security → Long-Lived Access Tokens
Note your HA instance URL

Configuration:

HA_URL=http://your-ha-instance:8123
HA_TOKEN=eyJhbGciOi...  # Long-lived access token

5. Telegram API Proxy (For GFW Users)

If your bot server is behind the GFW, Telegram's API (api.telegram.org) may be unreachable. Deploy an nginx reverse proxy on a server outside the GFW.

Example nginx configuration (on your proxy server):

server {
    listen 80;
    listen [::]:80;
    server_name your-proxy-domain.com;
    return 301 https://$host$request_uri;
}

server {
    server_name your-proxy-domain.com;

    listen 443 ssl;
    ssl_certificate     /etc/letsencrypt/live/your-proxy-domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/your-proxy-domain.com/privkey.pem;

    location /bot {
        proxy_pass https://api.telegram.org;
        proxy_set_header Host api.telegram.org;
        proxy_ssl_server_name on;
        proxy_connect_timeout 60s;
        proxy_send_timeout 120s;
        proxy_read_timeout 120s;
        client_max_body_size 50M;
    }
}

Then configure the bot to use the proxy:

TELEGRAM_BASE_URL=https://your-proxy-domain.com

6. OpenCode CLI

Must be installed on the bot server:

npm install -g @anthropic-ai/opencode   # or your installation method

Configuration:

OPENCODE_MODEL_NAME=q36   # model name for opencode to use

7. QQ Bot (Official API — No Ban Risk)

For mainland China users who cannot access Telegram. Uses the official QQ Bot Platform API through the botpy SDK.

Setup:

Create a bot at https://q.qq.com (QQ开放平台)
Go to Developer Settings → get AppID (机器人ID) and AppSecret (凭证)
In sandbox mode (QQ_SANDBOX=true), add yourself as a tester. Only testers can interact.
For public use, submit the bot for review and set QQ_SANDBOX=false

Interaction modes:

Mode	How users interact	Chat ID prefix
C2C (private)	Send messages directly to the bot	`qq_c2c_{openid}`
Group	Add bot to group, @mention it	`qq_group_{group_id}`
Guild channel	@mention in guild channel	`qq_guild_{channel}`
Guild DM	Direct message in guild	`qq_dm_{guild}`

Rate limits (QQ Official Bot restrictions):

Passive replies (replying to a user message): 3 per user per minute
Active messages (bot-initiated): requires user opt-in

Configuration:

QQ_BOT_APP_ID=102xxxxxx
QQ_BOT_SECRET=your-app-secret
QQ_SANDBOX=true

8. OneBot v11 (Personal QQ Account — Maximum Privacy)

⚠️ Risk disclaimer: Uses a personal QQ account for automation, which violates QQ's Terms of Service. Account restriction or ban is possible, especially with high message frequency. Mitigations are built in: only replies to @mentions in groups by default, rate-limited replies, no unsolicited messages. Use at your own risk.

Why OneBot? Privacy comparison:

Aspect	Official QQ Bot API	OneBot (Personal Account)
Message routing	Through QQ's bot servers	QQ's standard encrypted protocol
Who can read messages	Tencent (bot platform)	Only you (local decryption)
Ban risk	None (official)	Yes (ToS violation)
Setup complexity	Simple (create bot online)	Medium (install plugin)
Group message access	Only @mentions	All messages (configurable)

Architecture:

QQ Server ←→ QQ Client (NapCatQQ plugin) ←→ onebot.py → Lyric Chat API
   ↑              ↑                               ↑
 encrypted    local decrypts              local processing
 QQ protocol  via WS localhost            via HTTP localhost

Setup:

Install a OneBot v11-compatible QQ client plugin:
- NapCatQQ (recommended, actively maintained)
- LLOneBot
Configure the plugin:
- Enable reverse WebSocket server
- Host: 127.0.0.1, Port: 3001
- Set an access token
Log in to QQ through the client
Configure .env:

ONEBOT_WS_URL=ws://127.0.0.1:3001
ONEBOT_ACCESS_TOKEN=your-token
ONEBOT_BOT_QQ=          # auto-detected
ONEBOT_GROUP_ONLY_AT=true   # safety: only reply to @mentions in groups
ONEBOT_RATE_LIMIT=0.5       # seconds between replies

Safety features built into the bot:

ONEBOT_GROUP_ONLY_AT=true — only responds when explicitly @mentioned in groups
Per-chat rate limiting (ONEBOT_RATE_LIMIT) — prevents rapid-fire messages
Never sends unsolicited messages to users or groups
All chat content stays local, processed on your machine only

Start:

uv run python src/back/onebot.py
# or via run.sh:
./run.sh lyric --onebot

7. Prompt Injection Detection Model (Highly Recommended)

Uses ProtectAI's DeBERTa-v3 prompt injection detector — 99.93% F1, industry standard with 527k+ downloads. Falls back to regex patterns if model not installed.

Download:

# From HuggingFace (if accessible):
pip install huggingface_hub
python3 -c "
from huggingface_hub import snapshot_download
snapshot_download(
    'protectai/deberta-v3-base-prompt-injection-v2',
    local_dir='src/back/models/blocker/model/',
    ignore_patterns=['*.msgpack','*.h5','*.ot','onnx/*'],
)
"

# From mainland China (hf-mirror):
mkdir -p src/back/models/blocker/model
cd src/back/models/blocker/model
curl -o config.json          https://hf-mirror.com/protectai/deberta-v3-base-prompt-injection-v2/resolve/main/config.json
curl -o model.safetensors    https://hf-mirror.com/protectai/deberta-v3-base-prompt-injection-v2/resolve/main/model.safetensors
curl -o tokenizer.json       https://hf-mirror.com/protectai/deberta-v3-base-prompt-injection-v2/resolve/main/tokenizer.json
curl -o tokenizer_config.json https://hf-mirror.com/protectai/deberta-v3-base-prompt-injection-v2/resolve/main/tokenizer_config.json
curl -o special_tokens_map.json https://hf-mirror.com/protectai/deberta-v3-base-prompt-injection-v2/resolve/main/special_tokens_map.json
curl -o added_tokens.json    https://hf-mirror.com/protectai/deberta-v3-base-prompt-injection-v2/resolve/main/added_tokens.json

Placement:

src/back/models/blocker/model/
├── added_tokens.json
├── config.json
├── model.safetensors       (~704 MB)
├── special_tokens_map.json
├── tokenizer.json
└── tokenizer_config.json

Auto-detected on startup. If files are missing, falls back to regex-based detection automatically.

8. Vision Model (Optional — for image understanding)

Uses Qwen3-VL-4B-Instruct — 4B params, bf16. 4-bit quantization by default, uses ~2GB GPU memory. Works on Pascal+ GPUs.

Memory (8GB GPU):

Component	Memory
Qwen3-VL-4B (4-bit)	~2 GB
DeBERTa blocker (CPU)	0 GB
Qwen3-Embedding (Ollama)	~1.2 GB
Total	~3.2 GB ✓

Download (choose one):

# Method 1: Modelscope (recommended for mainland China)
pip install modelscope
modelscope download Qwen/Qwen3-VL-4B-Instruct \
  --local_dir ~/.cache/huggingface/hub/models--Qwen--Qwen3-VL-4B-Instruct

# Method 2: HuggingFace
huggingface-cli download Qwen/Qwen3-VL-4B-Instruct

# Method 3: Manual download (if above fail)
mkdir -p ~/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct
cd ~/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct
# Download these files from hf-mirror.com/Qwen/Qwen2.5-VL-3B-Instruct/resolve/main/ :
# config.json, model.safetensors, preprocessor_config.json,
# tokenizer.json, tokenizer_config.json, vocab.json, merges.txt

Memory (8GB GPU):

Component	Memory
Qwen3-VL-4B (FP8)	~4 GB
DeBERTa blocker	~0.8 GB
Qwen3-Embedding (Ollama)	~1.2 GB
Total	~6 GB ✓

Configuration:

MODEL_VISION=          # empty = local Qwen2-VL, true = use API vision mode

When MODEL_VISION=true, images are passed as base64 to the DECIDER model directly (requires vision-capable model).

Forwarding & Routing

The system has a three-layer forwarding architecture:

Layer 1: Telegram → Chat API

The Telegram bot (telebot.py) long-polls the Telegram API for new messages. Each incoming text or file is forwarded via HTTP POST to the FastAPI server at CHAT_API_URL (default http://127.0.0.1:8000/chat).

telebot.py:371-397 — text messages forwarded via chat_with_api()
telebot.py:400-425 — images use vision_with_api() with base64-encoded multi-modal content
telebot.py:206-230 — files (PDF, DOCX, audio) are pre-processed locally:
- Documents → MarkItDown (text extraction)
- Audio → FunASR (speech-to-text via paraformer-zh)
- Photos → base64 encoded for vision models

Layer 2: Chat API → AI Model + Tool Dispatch

The /chat endpoint (api/chat.py) delegates to MCPClientAPI.process_chat_completion() in mcp_client/client.py:474-700.

Request flow:

Messages are converted to OpenAI-compatible format, including an auto-generated system prompt listing available tools (client.py:298-471)
The DECIDER model (via AsyncOpenAI) receives the conversation + tool definitions (client.py:488-498)
If the model emits tool_calls, each tool name is parsed by namespace prefix:

"weather__get_weather_by_city"  →  server="weather"  tool="get_weather_by_city"

The tool is routed to the matching MCP server via stdio subprocess call (client.py:259-281)
Tool results are fed back into the conversation; the loop continues up to 8 iterations (client.py:484)
Final response is processed: <emo>, <inf> tags are extracted; content is saved to short-term memory; response is returned to the bot

Key routing logic:

Step	Location	Mechanism
Tool name parsing	`client.py:252-257`	`_parse_prefixed_tool_name()` splits on `__`
Server lookup	`client.py:259-281`	`call_tool()` looks up `self.servers[server_name]`
Tool invocation	`client.py:280`	`server.session.call_tool(tool_name, arguments)` via MCP stdio
Fallback search	`client.py:263-274`	If no prefix, scans all servers for matching tool name
Error propagation	`client.py:549-648`	Tool errors serialized as `role: "tool"` messages in conversation

Layer 3: MCP Server → Subprocess Execution

Each MCP server runs as a stdio subprocess (client.py:181-190), communicating via the Model Context Protocol over stdin/stdout:

mcp_client/config.json
  ├── weather       → uv run mcp_server/weather/main.py
  ├── home-assistant → uv run mcp_server/HA/main.py
  ├── query         → uv run mcp_server/query/main.py     (Neo4j memory query)
  ├── opencode      → python3 mcp_server/opencode/main.py (shell/cmd executor)
  ├── web-search    → npx -y mcp-search-server
  ├── skills        → uv run mcp_server/skills/main.py    (specialized skill loader)
  └── terminal      → python3 mcp_server/terminal/terminal_server.py (sandboxed terminal)

Tools are namespace-prefixed on registration (client.py:208-217) to avoid collisions when multiple servers export identically named tools.

Project Structure

lyric/
├── src/back/
│   ├── main.py                  # FastAPI entry point, route definitions, uvicorn server
│   ├── telebot.py               # Telegram bot: long-polling, file download, message dispatch
│   ├── qqbot.py                 # QQ bot: WebSocket via botpy SDK, C2C + group + guild
│   ├── onebot.py                # OneBot v11: personal QQ account, local privacy
│   ├── init.py                  # App lifecycle: startup init, shutdown cleanup, persistence loop
│   ├── classes.py               # Pydantic models: ChatCompletionRequest, MemRequest, etc.
│   ├── global_var.py            # Global state singleton (conversation history, MCP client)
│   ├── extract.py               # XML tag extractor for <emo>, <inf>, <mem> tags
│   ├── memorize.py              # Neo4j knowledge graph insertion (SimpleKGPipeline)
│   ├── searchmem.py             # Neo4j vector + cypher retrieval (GraphRAG)
│   ├── sl.py                    # Short-term memory: load/save JSON conversation files
│   ├── async_rwa.py             # Async file read/write/append utility
│   ├── embedding_dimensions.py  # Ollama embedding dimension detection
│   ├── tmp.py                   # One-off utility: merge Lyric/lyric nodes in Neo4j
│   ├── .env.example             # Configuration template
│   ├── api/
│   │   ├── chat.py              # POST /chat — main chat completion handler
│   │   ├── root.py              # GET / — health check
│   │   ├── tool.py              # GET /v1/tools — list available tools
│   │   ├── get_servers.py       # GET /v1/servers — list connected servers
│   │   └── createmem.py         # POST /createmem — manually create memory entry
│   ├── mcp_client/
│   │   ├── client.py            # MCP client: server connection, tool dispatch, chat loop
│   │   └── config.json          # MCP server definitions
│   ├── mcp_server/
│   │   ├── weather/main.py      # Weather, time, email sending tools
│   │   ├── HA/main.py           # Home Assistant device control
│   │   ├── query/main.py        # Neo4j knowledge graph query tool
│   │   ├── opencode/main.py     # Shell command executor (opencode run)
│   │   ├── skills/main.py       # Skill loader (list/load specialized instructions)
│   │   └── terminal/terminal_server.py  # Sandboxed terminal executor
│   └── skills/
│       ├── code-reviewer.md     # Code review guidelines
│       ├── debugging.md         # Systematic debugging approach
│       ├── brainstorming.md     # Structured feature design
│       └── shell-expert.md      # Shell command expertise
├── tests/
│   ├── test_telebot.py          # Telegram bot unit tests
│   ├── test_skills.py           # Skills MCP server tests
│   └── test_memory_persistence.py # Memory persistence logic tests
├── pyproject.toml               # Python project configuration (uv/pip)
├── run.sh                       # Convenience launcher script
└── .gitignore

Setup

Prerequisites

Python ≥ 3.12
uv (recommended) or pip
Node.js (for web-search MCP server)
The external services listed in Server Requirements above

Installation

git clone <repo-url>
cd lyric
uv sync
npm install

Configuration

Copy the example env file and fill in your values:

cp src/back/.env.example src/back/.env

Key environment variables:

Variable	Description
`DECIDER_BASE_URL`	Base URL for the decision-making model API
`DECIDER_API_KEY`	API key for the decider model
`DECIDER_MODEL_NAME`	Model name for chat/decision (e.g., `qwen3-235b-a22b`)
`MEM_BASE_URL` / `MEM_MODEL_NAME`	Model for memory extraction
`MEM_API_KEY`	API key for memory model
`SEARCH_BASE_URL` / `SEARCH_MODEL_NAME`	Model for memory search queries
`SEARCH_API_KEY`	API key for search model
`EMBEDDER`	Embedding model name (e.g., `Qwen3-Embedding:0.6B`)
`NEO4J_URI`	Neo4j connection URI (e.g., `neo4j://localhost:7687`)
`NEO4J_USERNAME` / `NEO4J_PASSWORD`	Neo4j credentials
`TELEGRAM_BOT_TOKEN`	Telegram Bot API token
`TELEGRAM_BASE_URL`	Telegram API URL (default: `https://api.telegram.org`; use proxy domain for GFW)
`CHAT_API_URL`	Where the bot sends chat requests (default: `http://127.0.0.1:8000/chat`)
`MAIN_PORT`	FastAPI server port (default: `8000`)
`MEM_RETENTION_TIME`	Days to retain short-term memory files (default: `3`)
`USERNAME`	User's display name in conversation
`ASSISTANT_NAME`	Bot's display name (default: `lyric`)
`amap_api`	Amap (高德) weather API key
`HA_URL` / `HA_TOKEN`	Home Assistant URL and long-lived token
`mail_account` / `mail_api`	QQ email account and SMTP auth code
`OPENCODE_MODEL_NAME`	Model name passed to opencode CLI
`TERMINAL_ALLOWED_DIR`	Filesystem path sandbox for terminal MCP server

MCP Server Configuration

Edit src/back/mcp_client/config.json to add/remove MCP servers:

{
  "mcpServers": {
    "weather": {
      "command": "uv",
      "args": ["--directory", "./mcp_server/weather", "run", "main.py"]
    }
  }
}

Each server's tools are automatically namespaced as <server_name>__<tool_name>.

Running

Quick Start

# Telegram + OneBot (recommended)
./start.sh

# Or use the simpler launcher (Telegram only starts if TELEGRAM_BOT_TOKEN is set):
./run.sh [username] [mcp_config.json] [--onebot|--qq]

Example:

./run.sh lyric src/back/mcp_client/config.json --onebot
``` The username argument becomes the bot's recognized owner in conversation.

### Manual Start

Start the API server:

```bash
uv run python src/back/main.py <username> src/back/mcp_client/config.json

In separate terminals, start your bot frontend(s):

# Telegram bot (auto-skipped if TELEGRAM_BOT_TOKEN not set)
uv run python src/back/telebot.py

# QQ bot (official API)
uv run python src/back/qqbot.py

# OneBot (personal QQ account)
uv run python src/back/onebot.py

In separate terminals, start your bot frontend(s):

# Telegram bot
uv run python src/back/telebot.py

# QQ bot (mainland China users, official API)
uv run python src/back/qqbot.py

# OneBot (personal QQ account, maximum privacy)
uv run python src/back/onebot.py

API Endpoints

Method	Path	Description
`GET`	`/`	Health check
`POST`	`/chat`	Main chat completion (bot calls this internally)
`GET`	`/v1/servers`	List connected MCP servers
`GET`	`/v1/tools`	List all available tools across all servers
`POST`	`/createmem`	Manually create a memory entry
`GET`	`/doc`	API documentation stub

Memory System

Short-Term Memory

Conversation history is stored as JSON files under memory/<username>/memory_plus/<date>.json. Retained for MEM_RETENTION_TIME days (default 3). Managed by sl.py.

Each message is stored as {"role": "...", "content": "..."} — the same format used for OpenAI API calls, enabling direct concatenation into chat context.

Long-Term Memory (Knowledge Graph)

Periodically, short-term conversations are extracted into a Neo4j knowledge graph via memorize.py. The extraction uses a dedicated MEM model that identifies entities, traits, events, and emotions from conversations.

Persistence trigger: Runs when the number of distinct conversation days accumulated since the last persistence reaches MEM_RETENTION_TIME (default 3). A background loop checks every 30 minutes at or after 5 AM (configurable timezone). This works for both:

Daily restart users: on-startup check catches accumulated days
24/7 server deployments: background loop detects when enough days accumulate without restarts

Knowledge graph schema (when using schema mode, via SimpleKGPipeline):

Entities: Person, CoreTrait, BehaviorPattern, Drive, TensionPoint, CopingMechanism, ValueAnchor, StateMarker, VisionProjection, Event, Preference, MyEmotion, Year, Month_Year, Day-Month-Year

Relationships: HAS_TRAIT, EXHIBITS, DRIVEN_BY, HAS_TENSION, USES, HOLDS, IN_STATE, ENVISIONS, EXPERIENCES, LIKES, DISLIKES, WHEN, FRIEND_OF, RELATED_TO, HAPPENS_ON, BELONGS_TO, IS

The create_from_str method (used by the persistence loop) uses schema="FREE" mode which allows the LLM to auto-detect entities. After extraction, APOC mergeNodes is called to deduplicate Lyric/lyric Person nodes.

Memory retrieval uses two strategies (searchmem.py):

Vector similarity — Searchmem (searchmem.py:11-66): cosine similarity via VectorCypherRetriever against a Chunk text embedding index
GraphRAG — Searchmemstrict (searchmem.py:67-134): VectorCypherRetriever + LLM re-ranking via GraphRAG pipeline; the LLM synthesizes a natural language answer from retrieved subgraph

The query MCP server (mcp_server/query/main.py) exposes memory search as a tool callable by the decider model.

MCP Tool Framework

Available Tool Servers

Server	Tools	Purpose
`weather`	`get_weather_by_city`, `get_current_time`, `send_qq_email`	Weather lookup, time queries, email sending
`home-assistant`	`ha_get_device_states`, `ha_control_device`	Smart home automation (lights, switches, climate, etc.)
`query`	`query_graph`	Neo4j knowledge graph search via natural language
`opencode`	`opencode_run`	Shell command and file operation execution via opencode CLI
`terminal`	`execute_command`	Sandboxed terminal with safety filters and background execution
`web-search`	`web_search`	Internet search
`skills`	`list_skills`, `load_skill`	List and load specialized agent skill instructions

Adding a New MCP Server

Create a new directory under src/back/mcp_server/
Implement an MCP server using the mcp Python library (stdio transport)
Add the server definition to src/back/mcp_client/config.json
Set any required environment variables in .env
Restart the API server

The framework auto-discovers tools and registers them with namespace prefixes. The decider model learns tool capabilities from the system prompt.

Skills System

The skills system allows the agent to load specialized instructions for different task domains:

list_skills — lists all available skills with names and descriptions
load_skill — loads a specific skill's full instruction content into the conversation context

Available Skills

Skill	Description
`code-reviewer`	Code review for bugs, style, performance, and security
`debugging`	Systematic debugging approach
`brainstorming`	Structured feature design and creative problem-solving
`shell-expert`	Shell commands and system administration best practices

Adding a New Skill

Create a .md file in src/back/skills/ with frontmatter-style description: and the instruction body
The skill is automatically discovered by list_skills
Custom skills directory can be set via SKILLS_DIR environment variable

Data Flow: Complete Request Lifecycle

1. User sends message in Telegram
2. telebot.py receives via long-polling (getUpdates)
3. telebot.py POSTs to main.py:8000/chat
4. chat.py appends user message to conversation history
5. chat.py calls MCPClientAPI.process_chat_completion()
6. MCP client sends conversation + tool defs to DECIDER model
7. If model returns tool_calls:
   a. Parse tool name prefix → route to correct MCP server
   b. Call tool via stdio subprocess
   c. Append tool result to conversation
   d. Loop back to step 6 (max 8 iterations)
8. Model returns final text response
9. chat.py extracts <emo> and <inf> XML tags from response
10. chat.py saves conversation to today's short-term memory file
11. chat.py returns cleaned response to telebot.py
12. telebot.py sends response to Telegram user
13. Background loop: every 30 min, if enough days accumulated, persist to Neo4j

GFW Considerations

When running the bot from mainland China:

Telegram API is blocked — use TELEGRAM_BASE_URL to point to a proxy server (see Server Requirements §5)
Long-polling timeout — POLLING_TIMEOUT (default 30s) prevents long TCP connections from being detected
SSL verification — TELEGRAM_SSL_VERIFY can be disabled for proxy servers with self-signed certs (not recommended for production)

Testing

uv run pytest tests/

License

MIT

Name		Name	Last commit message	Last commit date
src/back		src/back
tests		tests
.gitignore		.gitignore
README.md		README.md
README_zh.md		README_zh.md
package-lock.json		package-lock.json
package.json		package.json
requirements.txt		requirements.txt
run.sh		run.sh
start.sh		start.sh

Latest commit

History

Folders and files

Repository files navigation

Lyric

Architecture Overview

Server Requirements

Services You Need to Deploy

Detailed Server Setup

1. AI Model Server (OpenAI-compatible API)

2. Neo4j Database

3. Ollama (Embedding Server)

4. Home Assistant (Optional)

5. Telegram API Proxy (For GFW Users)

6. OpenCode CLI

7. QQ Bot (Official API — No Ban Risk)

8. OneBot v11 (Personal QQ Account — Maximum Privacy)

7. Prompt Injection Detection Model (Highly Recommended)

8. Vision Model (Optional — for image understanding)

Forwarding & Routing

Layer 1: Telegram → Chat API

Layer 2: Chat API → AI Model + Tool Dispatch

Layer 3: MCP Server → Subprocess Execution

Project Structure

Setup

Prerequisites

Installation

Configuration

MCP Server Configuration

Running

Quick Start

API Endpoints

Memory System

Short-Term Memory

Long-Term Memory (Knowledge Graph)

MCP Tool Framework

Available Tool Servers

Adding a New MCP Server

Skills System

Available Skills

Adding a New Skill

Data Flow: Complete Request Lifecycle

GFW Considerations

Testing

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages