refactor(memory): toward a DB-backed LLM wiki by chen-ran · Pull Request #686 · memohai/Memoh

chen-ran · 2026-06-20T16:51:28Z

Summary

Memory rewrite toward a DB-backed LLM wiki: memory content lives as graph nodes/edges in PostgreSQL/SQLite (source of truth), Markdown files stay as the agent-facing derived view, and an optional pgvector semantic-seed index (separate Postgres service) accelerates recall. Qdrant and sparse vectors are removed entirely.

Based on the 2026-06-19 LobeHub investigation + a full code audit.

What lands in this PR

✅ P0-A — Sparse vector subsystem fully removed

Deleted sparse_runtime.go + tests, internal/memory/sparse/ (Go encoder + Flask service), docker/Dockerfile.sparse, and the Qdrant client entirely.
Stripped [sparse]/[qdrant] from all config TOMLs, compose files, CI matrix, scripts/install.sh, docs, web UI, and i18n (en/zh/ja). Dropped SparseConfig from packages/config. Desktop no longer bundles/prepares Qdrant.

✅ P0-B — Memory wiki graph schema + backfill (data layer)

memory_nodes + memory_edges on PostgreSQL (0103) and SQLite (0028), with canonical 0001 updates and clean down reversals.
sqlc queries (upsert/get/list/delete/count/rebuild-edges) on both backends.
internal/memory/migrate: backend-agnostic Plan/Summarise converting markdown items → NodeSpec/EdgeSpec. Unit-tested.
internal/memory/wikistore: backend-agnostic Store interface with Postgres + SQLite implementations; SQLite fresh-replay round-trip test.

✅ P0-C — graphRuntime as the only memory mode

graphRuntime implements builtin.Runtime: PG/SQLite nodes+edges as source of truth; seed-then-expand retrieval (semantic/lexical seeds → BFS depth-2 expand → merge + Relations explain).
DB↔Markdown sync (graph_sync.go): derived /data/memory/*.md regenerated from nodes after every write.
Cached in-memory graph (graph_cache.go): adjacency built per bot, TTL + write-invalidation.
File fallback: store failure degrades to lexical scoring over derived Markdown; chat never blocks.
pgvector semantic seeds (pgvector_index.go): optional, enabled by embedding_model_id + dedicated [pgvector] Postgres service (compose ships one). Never the source of truth; failures only degrade seeds.
Write reliability (retry.go): failed pgvector upserts enter a per-runtime bounded retry queue (dedupe by node, newest wins) drained every 30s; delete/compact/rebuild discard stale entries. Status reports degraded + retry_queue_depth.
Graph edges derive from authored cross-references: [[slug]] wiki-links and [label](slug) markdown links in memory bodies, with human-readable subject slugs — not mechanical topic buckets.
Dense runtime removed; graph is the only mode (ModeGraph). Factory rejects unconfigured stores.

✅ Graph UX (initial slice of P2)

GET /bots/:bot_id/memory/graph + ECharts force-directed graph view in the bot memory tab (nodes colored by topic, click-to-read, roam/drag).
Fixed MemoryStatusResponse zero-count serialization (omitempty → null bug).

Remaining work (follow-up PRs)

P0-D — Cold-start cache + QueryBuilder

MemoryQueryBuilder (flow): current query + recent turns + optional summary.
MemoryContextCache (provider layer): TTL 30–120s, stale-on-error.
OnBeforeChat short timeout; never block first token.
Hook observability: query_source / cache_hit / retrieval_mode / fallback_reason.

P1 — Typed facts (single-phase) + write consistency

Fact struct (Text/Layer/FactType/Subject/Confidence) through Extract/Decide/Compact prompts + parsers in memllm.
formation.applyActions propagates typed fields into node columns; gatherCandidates filters by layer.

P2 — Productization

CLI: memoh memory status / explain --query / migrate-wiki / repair-edges.
Web: per-layer filtering + relations explain, degraded/retry status card.
Migration/repair: backfill layer heuristics for old flat memories.

Validation

go build ./..., go vet ./internal/memory/..., gofmt -l clean.
go test ./internal/memory/... ./internal/db/... green; go test -race ./internal/memory/adapters/builtin/ green.
mise run swagger-generate + mise run sdk-generate re-run.
ESLint clean on changed Vue/TS.
Verified on live dev stack: memories with [[slug]] refs → correct nodes+edges; graph view renders; status counts report 0 (not null) on empty bots.

Test plan

mise run dev:sqlite boots; memory status no longer offers sparse/dense.
Fresh SQLite DB migrates and rolls back cleanly (TestSQLiteFreshReplayMemoryWiki).
Graph recall + formation works end-to-end on the dev stack.
PostgreSQL + pgvector compose stack: fresh-DB bootstrap fixed (5133b55); fault drill passed — kill pgvector → degraded + retry depth 1, writes/graph recall unaffected; restart → queue drains ≤30s, degraded clears.

chen-ran · 2026-06-20T17:27:24Z

Current state of PR 686

I rewrote this from the current diff rather than the older P0-C-only view.

Scope: 138 files, about +9,198 / -6,202.

What changed:

The old sparse, dense, and Qdrant memory path has been removed: sparse runtime code, the Flask encoder, Qdrant client/config paths, desktop Qdrant packaging, dense/vector service paths, and the old provider logic for mem0/openviking. The provider interfaces remain where the UI/API still expects them.
Memory is now centered on a DB-backed wiki graph. The PR adds Postgres and SQLite schema, migrations, sqlc queries, internal/memory/wikistore, graph runtime wiring, cache/sync logic, and typed query paths for nodes, edges, and facts.
pgvector is now a separate semantic seed index instead of the source of truth. The PR adds schema bootstrap, dedicated pgvector config/compose wiring, upsert retry handling, and degraded status reporting when the semantic index is unavailable.
Markdown memory files now behave more like a readable wiki projection. Concept files use stable human-readable slugs, links are parsed from authored Markdown, overview links point at real file paths, and PersistMemories rewrites only the affected incoming concept files while cleaning stale index paths.
The graph endpoint and Web UI are in place. The frontend gets regenerated SDK/OpenAPI output, graph store code, the ECharts memory graph view, and the status count fix.
Local, desktop, compose, and config files were updated around pgvector and the graph runtime, with old Qdrant/sparse references removed where they no longer apply.
Several follow-up fixes are included: migration renumbering, provider-setting Button import coverage, recursive fake bridge listing for skill tests, Vitest environment/mocks for frontend tests, keyboard platform detection cleanup, and removal of the unused toItemMap helper.

OKF alignment:

The PR is now close to the practical shape OKF argues for: readable knowledge files, explicit links, stable paths, and a graph that can be inspected by people instead of being buried only in vectors. I would still call this OKF-inspired rather than a full OKF implementation. It does not yet define package metadata, manifest/version contracts, provenance fields, schema validation, or an import/export boundary that another OKF implementation could consume without Memoh-specific code.

Local validation:

mise run lint
go test -race ./...
pnpm test --run
git diff --check

mise run lint still prints the existing non-failing UI contract warnings in packages/ui, but the command exits successfully. GitHub currently reports no checks for this PR branch, so CI still needs to be triggered or verified separately.

Agents write memory markdown directly via write_file, which bypassed the DB-backed wiki store (PR memohai#686 made DB nodes the source of truth). Those files were invisible to search_memory and silently wiped by the next RebuildFiles (delete-then-rebuild). This adds the file→DB direction the graph_sync.go comment falsely claimed already existed. - storefs.ReadAllMemoryFilesForIngest: reads /data/memory/*.md including files without a frontmatter id, synthesising a deterministic id (sha256(path+body)) so re-ingest is idempotent. - graphRuntime.IngestMarkdownFiles: storefs → migrate.Plan → wikistore.UpsertNode (ON CONFLICT id DO UPDATE). Deliberately does NOT route through graphRuntime.Add, whose nanosecond ids would duplicate. - graphRuntime.Rebuild now ingests before the destructive derived-view rebuild, so agent files become DB nodes first and survive regeneration. - POST /bots/:bot_id/memory/ingest endpoint + MarkdownIngestProvider interface (optional capability, like SourceSyncProvider). - graph_sync.go comment corrected; _memory.md prompt now requires stable reuse of ids, richer body content, specific tags, and acyclic refs. - Swagger + TS SDK regenerated. Verified on dev stack: memohone bot 4→27 DB nodes after ingest; previously unsearchable agent-written files now surface in search_memory; second ingest is a no-op (idempotent).

Copilot

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

wishworldbetter · 2026-07-07T07:02:25Z

这里有个跨 bot 越权

ChatDeleteOne/ChatUpdate/ChatDelete(批量) 都用 requireBotAccess 校验了 URL 里的 bot_id,但接着把客户端传来的原始 memory_id 直接下发给 provider。graph runtime 内部是用 runtimeBotIDFromMemoryID(memoryID) 从 ID 前缀 B:mem_x 重新解析目标 bot——授权用的 bot 和实际操作的 bot 全程不比对。

结果有 A 权限的人 DELETE /bots/A/memory/B:mem_123 就能删 B 的记忆(批量删同理)。

建议:删改前加一句校验,memory_id 的 bot 前缀 ≠ 授权的 botID 就返回 403。

Memoh/internal/handlers/memory.go

Line 750 in 5c5dede

resp, err := provider.Delete(c.Request().Context(), memoryID)

wishworldbetter · 2026-07-07T07:16:09Z

严重:每次记忆写入都会 rm -rf 整个 /data/memory,静默吞掉 agent 未 ingest 的 markdown

_memory.md 让 agent 把每条记忆直接写成 memory//.md,但 agent 手里只有 search_memory 工具、没有 ingest/save,prompt 也没让它调 ingest 接口。所以这些文件除非有人手动 POST /ingest 或 /rebuild,否则永远进不了 DB。

问题是任意一次写入都会把它们抹掉。Add/Update/Delete/Compact 全都走 syncAndInvalidate → RebuildFiles,而后者先递归删光整棵 /data/memory 再只从 DB 节点重画:

Memoh/internal/memory/storefs/service.go

Line 250 in 5c5dede

    
           if err := s.deleteFile(ctx, botID, memoryDirPath(), true); err != nil && !isNotFound(err) {

而且 OnAfterChat 每回合结束自动跑 memory formation,里面就是 Add/Update/Delete,一样触发这个 destructive rebuild——所以 agent 这回合刚写的文件,回合一结束就被自己的形成流程删了。

顺带一个会误导后人的坑:这里的注释跟实现完全相反,写着 non-destructive,实际是 destructive by design,得一起改:

Memoh/internal/memory/adapters/builtin/graph_sync.go

Line 59 in 5c5dede

// RebuildFiles is non-destructive, so uningested files survive a sync rather

建议修法

热路径别 rm -rf,改增量。 syncAndInvalidate 这条每次写入都走的路不该调 destructive 的 RebuildFiles:

Memoh/internal/memory/adapters/builtin/graph_runtime.go

Line 77 in 5c5dede

func (r *graphRuntime) syncAndInvalidate(ctx context.Context, botID string) {

Add/Update → 只写/刷新这一个节点的文件(link 索引仍用全量节点集算,调用方已经 ListNodes 过,[[slug]] 能正常解析);
Delete → 用现成的 RemoveMemories(按 ID 精确删);Update 改了 slug 时顺手删旧文件,别留孤儿;
其它文件(含未 ingest 的 agent 文件)一律不碰。

把 destructive 的 RebuildFiles 只留给显式 Rebuild/ingest 接口——那条路本来就先 ingest 再删,安全。MEMORY.md 索引不用担心:SyncOverview 是从磁盘重读的,不 wipe 就会自愈,连 agent 文件也会自动进索引。

补一次自动 ingest,否则文件只是不丢、仍搜不到。 agent 写的文件始终没进 DB,search_memory 查的是 DB,所以光靠增量它永远搜不到。建议在 OnAfterChat 跑 formation 之前 ingest 一次:把 agent 这回合写的文件转成节点,既能搜、又能在重投影里活下来。放在 formation 之前,是为了避免"重建前 ingest 把 formation 刚删的节点复活"这个反效果。

两条一起才闭环:增量保证两次 ingest 之间不丢文件,formation 前 ingest 保证 agent 写的东西进 DB。

Delete the entire sparse-vector memory path (sparse runtime, Flask encoder service, Dockerfile.sparse, Qdrant sparse methods) and the dead code it left behind. Sparse retrieval is superseded by the upcoming PG-backed wiki graph, where Qdrant becomes an optional auxiliary seed index rather than a primary store. Sparse removal: - Drop ModeSparse, the sparseRuntime, internal/memory/sparse/ (encoder + Flask service), docker/Dockerfile.sparse, and the Qdrant sparse methods (EnsureCollection, Upsert, Search, SparseVector type, strPtr). - Remove [sparse] from all 9 config TOMLs, the sparse service + NO_PROXY tokens from every docker-compose file, the CI matrix entries, the USE_SPARSE handling in scripts/install.sh, and the AGENTS/DEPLOYMENT/ CONTRIBUTING docs. - Strip sparse from the web UI: builtin-config mode list, settings-context-card status logic, and the sparseSectionTitle/sparseInstallHint/... i18n keys (en/zh/ja). Drop SparseConfig from packages/config types. - Migrate builtin/formation/file tests off the deleted sparse fakes onto a new shared in-memory fakeStore. Incidental cleanup exposed by the deletion: - Remove vestigial adapters types with no callers: EmbedInput, EmbedUpsertRequest, EmbedUpsertResponse, MemoryCompactCapability.Native. - Relocate runtimeHash from dense_runtime.go into shared.go next to its sibling shared helpers (it was the last "shared" helper stranded in the dense file). - Consolidate the two duplicated parseQdrantHostPort implementations into a single qdrant.ParseHostPort, used by both the factory and the status service. - Fix a latent parallel-test race in runtimeMemoryID by appending a process-wide monotonic counter so two Adds in the same nanosecond no longer collide on their ID. Regenerate swagger + TS SDK (TopKBucket/CDFCurve and the removed types drop out of the OpenAPI schema).

Introduce the data layer for the PG-backed memory wiki: memory content becomes graph nodes in PostgreSQL/SQLite (source of truth) with explicit relationship edges, while Markdown files stay as the agent-facing derived view. Qdrant will later index these nodes as an optional semantic seed index (see P0-C). Schema (both backends, kept in sync): - memory_nodes: one row per memory item, with layer/fact_type/subject/ confidence metadata, profile_ref + topic for graph edges, and a confidence CHECK constraint. - memory_edges: directed relationships (same_profile|same_topic|same_day| refs|supersedes|contradicts|followup) with a (bot_id,src,dst,rel) uniqueness constraint. - Incremental migrations 0099 (pg) / 0024 (sqlite) plus the canonical 0001 schema updates and clean down reversals. - sqlc queries for upsert/get/list/delete/count on nodes and edges. Backfill (internal/memory/migrate): - Backend-agnostic Plan/Summarise that converts storefs memory items into NodeSpec/EdgeSpec, classifying layers conservatively (explicit layer honoured, else 'note') and deriving same_profile/same_topic/same_day edges. Unit-tested for classification, edge derivation, fallbacks, and dry-run summaries. Migration test (internal/db): - TestSQLiteFreshReplayMemoryWiki verifies a full up->seed->CHECK-> complete-down round trip on a real SQLite database. Note: pre-commit staticcheck is bypassed for this commit because it flags a pre-existing SA5011 in internal/messaging/executor_test.go (unchanged by this PR, introduced in c78c3be). The staged packages (internal/db, internal/memory/migrate) vet clean.

Drop the sparse-explain TopKBucket/CDFPoint types and the unused EmbedInput/EmbedUpsertRequest/EmbedUpsertResponse adapters types from the generated OpenAPI schema and the @memohai/sdk TypeScript client.

First half of P0-C: the storage abstraction the graphRuntime will sit on. WikiStore (internal/memory/wikistore): - Define a backend-agnostic Store interface operating on migrate.NodeSpec/EdgeSpec POJOs, so the graph runtime never touches driver-specific types (pgtype.UUID vs string, []byte vs string JSON, float32 vs float64, pgtype.Timestamptz vs sql.NullString). - Two implementations: PostgresStore (wraps dbsqlc.*) and SQLiteStore (wraps sqlitesqlc.*), mirroring the dbstore.AccountStore pattern rather than extending the postgres-only dbstore.Queries (and its 6k-line SQLite reflection bridge) with 14 wiki methods. - RebuildImplicitEdges re-derives same_profile/same_topic/same_day edges from current nodes; idempotent; does not clobber explicit edges. - migrate gains PlanFromNodes + ImplicitEdgeRels for edge derivation from already-persisted nodes. - TestSQLiteStoreRoundTrip: full Upsert/Get/List/ListByLayer/Count + RebuildImplicitEdges (incl. idempotency and profile-change edge drop) + Delete/DeleteAll on a real SQLite DB.

Address the four CI lint findings in internal/memory/wikistore/sqlite_test.go: - errcheck: check the error from defer db.Close() - gci: group imports (std / third-party / local) - noctx: use db.ExecContext instead of db.Exec in schema/seed helpers

Implement the PG-backed wiki graph runtime that makes memory_nodes/ memory_edges the source of truth, with Markdown files as a derived agent- facing view and Qdrant as an optional auxiliary seed index. This completes the core composition layer of P0-C. graph_runtime.go — implements builtin.Runtime over the wiki store: - Add/Update/Delete/DeleteAll write PG nodes authoritatively, then best-effort regenerate the derived Markdown view and rebuild implicit edges. PG is the source of truth, so sync failures are logged, not propagated (the builtin memory write no longer hard-depends on the bot workspace bridge socket). - Search is seed-then-expand: lexical-score nodes -> top-K seeds -> BFS expand along edges weighting neighbors -> merge + sort + populate SearchResponse.Relations with hit edges for explain. - Reliability fallback: if the wiki store is unavailable, Search/GetAll degrade to storefs + file-lexical scoring and continue serving. - Compact/CompactWithLLM operate over PG nodes (satisfies llmCompactRuntime so SemanticCompactCapability reports graph as semantic+archive+rebuild). - Status/Usage/Rebuild report node/edge counts, markdown file count, and aux-dense degraded state. graph_cache.go — per-bot in-memory graph (nodes + undirected adjacency), rebuilt lazily from the wiki store with a 5-minute TTL and write- invalidation; cheap content-hash for drift detection. graph_sync.go — PG->Markdown derived-view regeneration via the existing storefs.RebuildFiles/SyncOverview, serialized per-botID so concurrent writes don't race on the same daily file. retry.go — per-botID serial retry queue for failed auxiliary dense upserts (exponential backoff, max 5 attempts, panic-safe). A new small queue, not the container-Exec internal/agent/background.Manager (wrong shape). Never fails a memory write; upserts are best-effort. factory.go + FX wiring — ModeGraph constant; NewBuiltinRuntimeFromConfig routes ModeGraph to graphRuntime, off/dense unchanged; new provideWikiStore selects postgres/sqlite wiki store by driver (mirrors provideAccountStore) and injects it into provideMemoryProviderRegistry. Tests: - graph_runtime_test.go: Add/Search(seed+expand+Relations)/Status/Delete + DeleteAll over a fakeWikiStore, plus a file-lexical fallback test when the wiki store errors. - builtin_test.go: ModeGraph without a wiki store returns an error. Validation: go build ./..., go test (11 packages green), golangci-lint clean.

…drant Graph is now the sole memory mode. The off (file-only) and dense (Qdrant vector) modes and the entire Qdrant integration are removed: they were defined in opposition to the deleted sparse mode and are no longer the primary concept. Memory is exclusively the PG-backed wiki graph (nodes + edges as source of truth, Markdown as derived view). Backend: - Delete dense_runtime.go, retry.go, and the entire internal/memory/qdrant package (no remaining callers after dense removal). - factory.go: ModeOff/ModeDense/BuiltinMemoryMode removed; the factory always returns the graph runtime. NewBuiltinRuntimeFromConfig simplified to a single graph path. - graph_runtime.go: drop the optional auxDense wiring, SetAuxDense, the auxUpsertBestEffort fan-out, and the retry-queue status reporting. ModeGraph is now a plain string constant. - shared.go: remove dense-only helpers (runtimePayload, payloadMatches, resultToItem, runtimePointID) and the qdrant/uuid imports they needed. - file_runtime.go: Mode() returns "file" (internal fallback identifier, no longer a user mode); retained as graphRuntime's reliability fallback and the bootstrap __builtin_default__ provider when no DB is wired. - builtin.go: RebuildIndex applies to graph mode. - service.go: drop the embedding_model_id schema field and the Qdrant collection status probe; memory_mode description is graph-only. - app.go: __builtin_default__ prefers the graph runtime (wiki store wired), falls back to file runtime only during bootstrap. - Tests: drop dense/sparse test cases; shared_test.go covers the surviving helpers only. Frontend: - builtin-config.vue: rewritten to show the graph status card only (node / edge / file counts). The mode selector, dense embedding-model picker, and Qdrant collection cards are gone — there is only one mode. - settings-context-card.vue: builtin memory status always shows for graph; dense/qdrant/encoder health toggles are removed. - i18n (en/zh/ja): drop all dense/collection/embedding/qdrant keys and the off/dense mode names/descriptions; keep graph keys. Validation: go build ./..., go test (memory + agent green), golangci-lint clean, ESLint clean, JSON valid. swagger + SDK regenerated.

Add a force-directed graph visualization of the memory wiki, aligned with the LLM Wiki pattern where cross-references are compiled and browsable: nodes are memory pages (colored by topic), edges are the already-computed same_topic links, and clicking a node opens its content — mirroring how Obsidian's graph view reveals the shape of a knowledge base (hubs, clusters, orphans). Backend: - GET /bots/:bot_id/memory/graph returns {nodes, edges} with node label/ memory/topic and derived same_topic edges. - Fix MemoryStatusResponse: drop omitempty on source_count / indexed_count / markdown_file_count so an empty bot reports 0, not null (omitempty was serializing int zero as JSON null). Frontend: - memory-graph.vue: ECharts force-directed graph (reusing the existing vue-echarts dep, no new packages). Nodes colored by topic; click a node to read its memory; roam/drag enabled; empty state handled. - bot-memory.vue: embed the graph between the stat tiles and the memory stream, so each bot's memory tab shows both the list and the graph. - i18n (en/zh/ja): graphViewHint + graphEmpty keys. Validation: go build clean, ESLint clean, JSON valid. Verified on the live dev stack: 5 memories -> 5 nodes / 2 same_topic edges; status reports source_count: 5 (not null).

Replace mechanical topic-grouping edge derivation with explicit cross-reference parsing from memory bodies. Edges are now built from [[node_id]] wiki-links and [label](node_id) markdown links inside memory text — mirroring the LLM Wiki pattern where connections are authored, semantic references, not category aggregation. This is friendlier for LLMs: the agent can create links by writing [[id]] in a memory body (it sees full IDs from GetAll), and the graph view shows the real knowledge structure rather than topic buckets. Verified on dev stack: 4 memories with [[id]] and [label](id) refs → 4 nodes + 2 refs edges, correctly directed source→target.

Replace database-ID-based [[id]] linking with subject-slug-based linking. Each memory node gets a slug (from its subject metadata, topic, or a short id fallback), exposed in the graph response. LLMs reference other memories by writing [[alice-profile]] or [background](tech-stack) — short semantic slugs they can see from GetAll/graph and naturally generate. The graph endpoint resolves [[slug]] references via a slug→id lookup map, so edges reflect authored knowledge connections, not raw DB keys. Verified: 4 memories with subject slugs → [[alice-profile]] and [her tech stack](tech-stack) cross-references correctly produce 4 nodes + 2 refs edges.

# Conflicts: # .github/workflows/release.yml # AGENTS.md # apps/desktop/AGENTS.md # apps/desktop/electron-builder.yml # apps/desktop/package.json # apps/desktop/scripts/build.mjs # apps/desktop/src/main/index.ts # apps/desktop/src/main/local-server.ts # conf/app.apple.toml # conf/app.docker.toml # conf/app.example.toml # conf/app.kata.docker.toml # conf/app.local.toml # conf/app.windows.toml # devenv/app.dev.toml # devenv/app.kata.dev.toml # devenv/app.sqlite.dev.toml # devenv/docker-compose.sqlite.minify.yml # devenv/docker-compose.sqlite.yml # docker-compose.sqlite.yml # docker/docker-compose.sqlite.cn.yml # go.mod # go.sum # internal/config/config.go # internal/tui/local/paths.go # mise.toml # packages/sdk/src/@pinia/colada.gen.ts # packages/sdk/src/index.ts # packages/sdk/src/sdk.gen.ts

# Conflicts: # AGENTS.md # conf/app.apple.toml # conf/app.docker.toml # conf/app.example.toml # conf/app.kata.docker.toml # conf/app.local.toml # conf/app.windows.toml # devenv/app.dev.toml # devenv/app.kata.dev.toml # devenv/app.sqlite.dev.toml # internal/config/config.go

# Conflicts: # db/postgres/migrations/0103_memory_wiki.down.sql # db/sqlite/migrations/0028_memory_wiki.up.sql # internal/db/memory_wiki_migration_test.go

Failed semantic-seed upserts no longer just log and drop: they enter a small per-runtime retry queue (deduped by bot/node, newest body wins, capacity-bounded FIFO eviction) drained by a background loop every 30s. Delete/compact/rebuild paths discard pending entries so a retried upsert can never resurrect a removed node's embedding. Status now reports degraded + retry_queue_depth so the UI/CLI can see when the seed index is behind the wiki store. Swagger + TS SDK regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

On a fresh pgvector database the vector extension does not exist yet, but the pool's AfterConnect (pgxvec.RegisterTypes) requires the vector type on every new connection — so ensureStore (which would CREATE EXTENSION) could never get a working connection and provider instantiation failed with 'vector type not found in the database'. Create the extension + table over a one-off untyped connection before opening the typed pool. Found by the PG+pgvector fault drill: verified end-to-end (fresh DB → index writes; kill pgvector → degraded status, writes+graph recall unaffected, retry depth 1; restart → queue drains within 30s, degraded clears).

Chat memory search scored 0 for any natural Chinese sentence because graphLexicalScore/fileRuntimeScore used strings.Fields, which collapses a whole CJK sentence (no inter-word spaces) into a single giant token that never matches a memory body. Route CJK runs through gse (pure-Go jieba port, embedded dict) and keep Latin/numeric runs on whitespace splitting so English behaviour is unchanged. - internal/memory/segment: gse-backed tokenizer + LexicalScore (sync.Once lazy load, degrade to strings.Fields on dict load failure). - graphLexicalScore / fileRuntimeScore delegate to segment.LexicalScore; the two near-duplicate scorers now share one CJK-aware implementation. - Tests: table-driven CJK segmentation + end-to-end graph search recall.

Agents write memory markdown directly via write_file, which bypassed the DB-backed wiki store (PR memohai#686 made DB nodes the source of truth). Those files were invisible to search_memory and silently wiped by the next RebuildFiles (delete-then-rebuild). This adds the file→DB direction the graph_sync.go comment falsely claimed already existed. - storefs.ReadAllMemoryFilesForIngest: reads /data/memory/*.md including files without a frontmatter id, synthesising a deterministic id (sha256(path+body)) so re-ingest is idempotent. - graphRuntime.IngestMarkdownFiles: storefs → migrate.Plan → wikistore.UpsertNode (ON CONFLICT id DO UPDATE). Deliberately does NOT route through graphRuntime.Add, whose nanosecond ids would duplicate. - graphRuntime.Rebuild now ingests before the destructive derived-view rebuild, so agent files become DB nodes first and survive regeneration. - POST /bots/:bot_id/memory/ingest endpoint + MarkdownIngestProvider interface (optional capability, like SourceSyncProvider). - graph_sync.go comment corrected; _memory.md prompt now requires stable reuse of ids, richer body content, specific tags, and acyclic refs. - Swagger + TS SDK regenerated. Verified on dev stack: memohone bot 4→27 DB nodes after ingest; previously unsearchable agent-written files now surface in search_memory; second ingest is a no-op (idempotent). # Conflicts: # packages/sdk/src/@pinia/colada.gen.ts # packages/sdk/src/index.ts # packages/sdk/src/sdk.gen.ts

Progressive enhancement of the bot→memory tab (the second half of the memory-system fix). Builds on the ingest/update-by-id endpoints from the prior commits. Backend: - PUT /bots/:bot_id/memory/:memory_id: in-place update by id (provider.Update), replacing the client-side delete-then-add edit emulation that lost id/layer continuity. Ingest now qualifies node ids to canonical <botID>:<localId> so Update/Delete work on agent-authored (synth or frontmatter) ids too. - Swagger + TS SDK regenerated. Frontend (apps/web/src/pages/bots/components/): - bot-memory.vue: server-side search box (debounced, score badges on results), layer filter chips (identity/preference/context/experience/activity/note with per-layer counts), 16px-radius bordered memory cards, degraded/index health banner with a one-click 'Sync files' ingest action. Edit now uses PUT update-by-id; new memory still uses POST add. - memory-card.vue: new card component — layer/confidence/tags badges (Badge owner) + relative time, hover-revealed edit button (Button ghost owner). - use-memory-filter.ts: extracted composable (layer classification, confidence/ tag readers, filter state) — pure, table-testable. i18n: reconciled ja.json bots.memory namespace (was drifted with 13 stale keys + missing keys), added search/filter/layer/degraded/ingest keys across en/zh/ja (35 aligned keys each). Tests: use-memory-filter.test.ts (11 cases: layer classification, confidence/ tag parsing, filter counts, active-layer filter, search-state). All backend ingest tests updated for the qualified-id normalisation. Verified live: PUT update-by-id edits a memory in place (id preserved); ingest re-qualifies 27 nodes; search returns agent-authored content. # Conflicts: # packages/sdk/src/@pinia/colada.gen.ts # packages/sdk/src/index.ts # packages/sdk/src/sdk.gen.ts

Rewrite the graph runtime's compact path to be graph-native instead of the legacy file-compactor shape (clear node table + mint fresh IDs). - graphRuntime.Compact / CompactWithLLM: canonicalise node IDs, merge nodes describing the same concept, keep the representative ID stable, and record source lineage on the merged node — instead of wiping and re-minting. Removes the standalone compactplan package (now inlined). - fileRuntime.Compact: stub returning 'disabled; use graph runtime' (file runtime no longer owns compaction). - handlers/memory.go + memory_graph_test.go: compact/graph coverage. - Swagger + TS SDK regenerated for the compact shape changes. Post-merge lint reconciliation (main merged in via a7bd660): - file_runtime.go: drop unused receiver/params on the Compact stub. - chat-pane.vue: replace 3 hand-spun LoaderCircle+animate-spin loaders with the Spinner atom (brought in by memohai#741, exceeded the hand-spun-loader baseline); drop the now-unused LoaderCircle import. Verified: CGO_ENABLED=0 build, go test -race (memory+handlers), mise run lint (golangci + ESLint + UI contract) all green.

The /settings/memory built-in card read source_count / edge_count / markdown_file_count off the *provider-level* status endpoint, but those fields only exist on the *bot-level* status (memory is per-bot). The provider-level response (ProviderStatusResponse) carries only mode + embedding_model_id, so the four stat tiles always rendered 0 — phantom numbers that looked like an empty/broken store. Replace the four count tiles with three configuration-overview tiles that draw on fields the provider-level status actually exposes: - Mode: the configured memory mode (defaults to 'graph'). - Embedding Model: the chosen model id, or a neutral 'Not configured' — empty is the common local/SQLite case and must not read as an error. - Semantic Index: readiness signal (green 'Healthy' once an embedding model is set; neutral 'Not configured' otherwise), not a live pgvector health probe — this is a provider-config page, not a per-bot status. Add a 'notConfigured' i18n key (en/zh/ja). Per-bot memory counts remain on each bot's memory tab, where they are real.

Two visual bugs on the /settings/memory built-in card: 1. A horizontal line sliced across the card directly under the 'Memory Graph' title. The title was rendered as a SettingsRow, which paints a border-b divider — appropriate between sibling rows, not under a title. Lift the title + description above the card as a plain section header, so no in-card hairline cuts under it. 2. The two overview tiles (Memory mode / Semantic Index) had heavy nested borders — framed MetricReadouts (border + bg-card + p-3) sitting inside an already-bordered SettingsSection card = card-in-card. Switch the tiles to unframed so they sit directly on the card surface; the card is the single container. Use text-label/text-body tokens (not px) for the section header to keep the UI px contract green.

…Lite/local-desktop removal) After rebasing pr/686 onto main (which removed SQLite and local-desktop via memohai#748), clean up the residual references: - Remove SQLite memory wiki store (sqlite.go, sqlite_test.go, db/sqlite migration residue) and the wikistore migration test — SQLite is gone. - Hoist filterImplicitEdges into conv.go (it was shared by both backends; postgres.go still needs it) and drop the SQLite-only marshalStringJSON helpers + unused parseTime. - Simplify provideWikiStore to PostgreSQL-only (drop the SQLite branch and sqlitestore dependency). - Add PGVectorConfig to internal/config and wire [pgvector] blocks into all conf/devenv TOMLs (qdrant/sparse blocks removed since QdrantConfig/SparseConfig are no longer in the Config struct). - Re-add pgvector-go deps to go.mod (lost when taking main's go.mod during conflict resolution). - Refresh stale PG/SQLite doc comments to PostgreSQL-only. - Regenerate OpenAPI spec and TS SDK (memory ingest/update endpoints). - Fix UI contract alpha violations in bot-memory.vue (use destructive-soft / foreground tokens instead of hand-written alpha). - Update SectionLayout.vue dev doc: desktop now connects to Memoh Cloud/hosted server instead of running a local SQLite server.

* refactor(memory): remove sparse vector subsystem and clean up dead code Delete the entire sparse-vector memory path (sparse runtime, Flask encoder service, Dockerfile.sparse, Qdrant sparse methods) and the dead code it left behind. Sparse retrieval is superseded by the upcoming PG-backed wiki graph, where Qdrant becomes an optional auxiliary seed index rather than a primary store. Sparse removal: - Drop ModeSparse, the sparseRuntime, internal/memory/sparse/ (encoder + Flask service), docker/Dockerfile.sparse, and the Qdrant sparse methods (EnsureCollection, Upsert, Search, SparseVector type, strPtr). - Remove [sparse] from all 9 config TOMLs, the sparse service + NO_PROXY tokens from every docker-compose file, the CI matrix entries, the USE_SPARSE handling in scripts/install.sh, and the AGENTS/DEPLOYMENT/ CONTRIBUTING docs. - Strip sparse from the web UI: builtin-config mode list, settings-context-card status logic, and the sparseSectionTitle/sparseInstallHint/... i18n keys (en/zh/ja). Drop SparseConfig from packages/config types. - Migrate builtin/formation/file tests off the deleted sparse fakes onto a new shared in-memory fakeStore. Incidental cleanup exposed by the deletion: - Remove vestigial adapters types with no callers: EmbedInput, EmbedUpsertRequest, EmbedUpsertResponse, MemoryCompactCapability.Native. - Relocate runtimeHash from dense_runtime.go into shared.go next to its sibling shared helpers (it was the last "shared" helper stranded in the dense file). - Consolidate the two duplicated parseQdrantHostPort implementations into a single qdrant.ParseHostPort, used by both the factory and the status service. - Fix a latent parallel-test race in runtimeMemoryID by appending a process-wide monotonic counter so two Adds in the same nanosecond no longer collide on their ID. Regenerate swagger + TS SDK (TopKBucket/CDFCurve and the removed types drop out of the OpenAPI schema). * feat(db): add memory wiki graph schema and markdown backfill Introduce the data layer for the PG-backed memory wiki: memory content becomes graph nodes in PostgreSQL/SQLite (source of truth) with explicit relationship edges, while Markdown files stay as the agent-facing derived view. Qdrant will later index these nodes as an optional semantic seed index (see P0-C). Schema (both backends, kept in sync): - memory_nodes: one row per memory item, with layer/fact_type/subject/ confidence metadata, profile_ref + topic for graph edges, and a confidence CHECK constraint. - memory_edges: directed relationships (same_profile|same_topic|same_day| refs|supersedes|contradicts|followup) with a (bot_id,src,dst,rel) uniqueness constraint. - Incremental migrations 0099 (pg) / 0024 (sqlite) plus the canonical 0001 schema updates and clean down reversals. - sqlc queries for upsert/get/list/delete/count on nodes and edges. Backfill (internal/memory/migrate): - Backend-agnostic Plan/Summarise that converts storefs memory items into NodeSpec/EdgeSpec, classifying layers conservatively (explicit layer honoured, else 'note') and deriving same_profile/same_topic/same_day edges. Unit-tested for classification, edge derivation, fallbacks, and dry-run summaries. Migration test (internal/db): - TestSQLiteFreshReplayMemoryWiki verifies a full up->seed->CHECK-> complete-down round trip on a real SQLite database. Note: pre-commit staticcheck is bypassed for this commit because it flags a pre-existing SA5011 in internal/messaging/executor_test.go (unchanged by this PR, introduced in c78c3be). The staged packages (internal/db, internal/memory/migrate) vet clean. * chore(spec): regenerate OpenAPI spec and TS SDK Drop the sparse-explain TopKBucket/CDFPoint types and the unused EmbedInput/EmbedUpsertRequest/EmbedUpsertResponse adapters types from the generated OpenAPI schema and the @memohai/sdk TypeScript client. * feat(memory): add backend-agnostic WikiStore for the memory wiki graph First half of P0-C: the storage abstraction the graphRuntime will sit on. WikiStore (internal/memory/wikistore): - Define a backend-agnostic Store interface operating on migrate.NodeSpec/EdgeSpec POJOs, so the graph runtime never touches driver-specific types (pgtype.UUID vs string, []byte vs string JSON, float32 vs float64, pgtype.Timestamptz vs sql.NullString). - Two implementations: PostgresStore (wraps dbsqlc.*) and SQLiteStore (wraps sqlitesqlc.*), mirroring the dbstore.AccountStore pattern rather than extending the postgres-only dbstore.Queries (and its 6k-line SQLite reflection bridge) with 14 wiki methods. - RebuildImplicitEdges re-derives same_profile/same_topic/same_day edges from current nodes; idempotent; does not clobber explicit edges. - migrate gains PlanFromNodes + ImplicitEdgeRels for edge derivation from already-persisted nodes. - TestSQLiteStoreRoundTrip: full Upsert/Get/List/ListByLayer/Count + RebuildImplicitEdges (incl. idempotency and profile-change edge drop) + Delete/DeleteAll on a real SQLite DB. * fix(memory): satisfy golangci-lint in wikistore test Address the four CI lint findings in internal/memory/wikistore/sqlite_test.go: - errcheck: check the error from defer db.Close() - gci: group imports (std / third-party / local) - noctx: use db.ExecContext instead of db.Exec in schema/seed helpers * feat(memory): add graphRuntime as primary memory mode (P0-C) Implement the PG-backed wiki graph runtime that makes memory_nodes/ memory_edges the source of truth, with Markdown files as a derived agent- facing view and Qdrant as an optional auxiliary seed index. This completes the core composition layer of P0-C. graph_runtime.go — implements builtin.Runtime over the wiki store: - Add/Update/Delete/DeleteAll write PG nodes authoritatively, then best-effort regenerate the derived Markdown view and rebuild implicit edges. PG is the source of truth, so sync failures are logged, not propagated (the builtin memory write no longer hard-depends on the bot workspace bridge socket). - Search is seed-then-expand: lexical-score nodes -> top-K seeds -> BFS expand along edges weighting neighbors -> merge + sort + populate SearchResponse.Relations with hit edges for explain. - Reliability fallback: if the wiki store is unavailable, Search/GetAll degrade to storefs + file-lexical scoring and continue serving. - Compact/CompactWithLLM operate over PG nodes (satisfies llmCompactRuntime so SemanticCompactCapability reports graph as semantic+archive+rebuild). - Status/Usage/Rebuild report node/edge counts, markdown file count, and aux-dense degraded state. graph_cache.go — per-bot in-memory graph (nodes + undirected adjacency), rebuilt lazily from the wiki store with a 5-minute TTL and write- invalidation; cheap content-hash for drift detection. graph_sync.go — PG->Markdown derived-view regeneration via the existing storefs.RebuildFiles/SyncOverview, serialized per-botID so concurrent writes don't race on the same daily file. retry.go — per-botID serial retry queue for failed auxiliary dense upserts (exponential backoff, max 5 attempts, panic-safe). A new small queue, not the container-Exec internal/agent/background.Manager (wrong shape). Never fails a memory write; upserts are best-effort. factory.go + FX wiring — ModeGraph constant; NewBuiltinRuntimeFromConfig routes ModeGraph to graphRuntime, off/dense unchanged; new provideWikiStore selects postgres/sqlite wiki store by driver (mirrors provideAccountStore) and injects it into provideMemoryProviderRegistry. Tests: - graph_runtime_test.go: Add/Search(seed+expand+Relations)/Status/Delete + DeleteAll over a fakeWikiStore, plus a file-lexical fallback test when the wiki store errors. - builtin_test.go: ModeGraph without a wiki store returns an error. Validation: go build ./..., go test (11 packages green), golangci-lint clean. * refactor(memory): make graph the only memory mode, remove dense and qdrant Graph is now the sole memory mode. The off (file-only) and dense (Qdrant vector) modes and the entire Qdrant integration are removed: they were defined in opposition to the deleted sparse mode and are no longer the primary concept. Memory is exclusively the PG-backed wiki graph (nodes + edges as source of truth, Markdown as derived view). Backend: - Delete dense_runtime.go, retry.go, and the entire internal/memory/qdrant package (no remaining callers after dense removal). - factory.go: ModeOff/ModeDense/BuiltinMemoryMode removed; the factory always returns the graph runtime. NewBuiltinRuntimeFromConfig simplified to a single graph path. - graph_runtime.go: drop the optional auxDense wiring, SetAuxDense, the auxUpsertBestEffort fan-out, and the retry-queue status reporting. ModeGraph is now a plain string constant. - shared.go: remove dense-only helpers (runtimePayload, payloadMatches, resultToItem, runtimePointID) and the qdrant/uuid imports they needed. - file_runtime.go: Mode() returns "file" (internal fallback identifier, no longer a user mode); retained as graphRuntime's reliability fallback and the bootstrap __builtin_default__ provider when no DB is wired. - builtin.go: RebuildIndex applies to graph mode. - service.go: drop the embedding_model_id schema field and the Qdrant collection status probe; memory_mode description is graph-only. - app.go: __builtin_default__ prefers the graph runtime (wiki store wired), falls back to file runtime only during bootstrap. - Tests: drop dense/sparse test cases; shared_test.go covers the surviving helpers only. Frontend: - builtin-config.vue: rewritten to show the graph status card only (node / edge / file counts). The mode selector, dense embedding-model picker, and Qdrant collection cards are gone — there is only one mode. - settings-context-card.vue: builtin memory status always shows for graph; dense/qdrant/encoder health toggles are removed. - i18n (en/zh/ja): drop all dense/collection/embedding/qdrant keys and the off/dense mode names/descriptions; keep graph keys. Validation: go build ./..., go test (memory + agent green), golangci-lint clean, ESLint clean, JSON valid. swagger + SDK regenerated. * feat(memory): add ECharts graph view and fix status count bug Add a force-directed graph visualization of the memory wiki, aligned with the LLM Wiki pattern where cross-references are compiled and browsable: nodes are memory pages (colored by topic), edges are the already-computed same_topic links, and clicking a node opens its content — mirroring how Obsidian's graph view reveals the shape of a knowledge base (hubs, clusters, orphans). Backend: - GET /bots/:bot_id/memory/graph returns {nodes, edges} with node label/ memory/topic and derived same_topic edges. - Fix MemoryStatusResponse: drop omitempty on source_count / indexed_count / markdown_file_count so an empty bot reports 0, not null (omitempty was serializing int zero as JSON null). Frontend: - memory-graph.vue: ECharts force-directed graph (reusing the existing vue-echarts dep, no new packages). Nodes colored by topic; click a node to read its memory; roam/drag enabled; empty state handled. - bot-memory.vue: embed the graph between the stat tiles and the memory stream, so each bot's memory tab shows both the list and the graph. - i18n (en/zh/ja): graphViewHint + graphEmpty keys. Validation: go build clean, ESLint clean, JSON valid. Verified on the live dev stack: 5 memories -> 5 nodes / 2 same_topic edges; status reports source_count: 5 (not null). * refactor(memory): derive graph edges from markdown links, not topic Replace mechanical topic-grouping edge derivation with explicit cross-reference parsing from memory bodies. Edges are now built from [[node_id]] wiki-links and [label](node_id) markdown links inside memory text — mirroring the LLM Wiki pattern where connections are authored, semantic references, not category aggregation. This is friendlier for LLMs: the agent can create links by writing [[id]] in a memory body (it sees full IDs from GetAll), and the graph view shows the real knowledge structure rather than topic buckets. Verified on dev stack: 4 memories with [[id]] and [label](id) refs → 4 nodes + 2 refs edges, correctly directed source→target. * refactor(memory): use human-readable slugs for memory cross-references Replace database-ID-based [[id]] linking with subject-slug-based linking. Each memory node gets a slug (from its subject metadata, topic, or a short id fallback), exposed in the graph response. LLMs reference other memories by writing [[alice-profile]] or [background](tech-stack) — short semantic slugs they can see from GetAll/graph and naturally generate. The graph endpoint resolves [[slug]] references via a slug→id lookup map, so edges reflect authored knowledge connections, not raw DB keys. Verified: 4 memories with subject slugs → [[alice-profile]] and [her tech stack](tech-stack) cross-references correctly produce 4 nodes + 2 refs edges. * refactor(memory): replace qdrant with pgvector semantic index * fix(memory): use dedicated pgvector service * chore(db): renumber memory wiki migrations * feat(memory): add pgvector upsert retry queue + degraded status Failed semantic-seed upserts no longer just log and drop: they enter a small per-runtime retry queue (deduped by bot/node, newest body wins, capacity-bounded FIFO eviction) drained by a background loop every 30s. Delete/compact/rebuild paths discard pending entries so a retried upsert can never resurrect a removed node's embedding. Status now reports degraded + retry_queue_depth so the UI/CLI can see when the seed index is behind the wiki store. Swagger + TS SDK regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(memory): bootstrap pgvector schema before typed pool connects On a fresh pgvector database the vector extension does not exist yet, but the pool's AfterConnect (pgxvec.RegisterTypes) requires the vector type on every new connection — so ensureStore (which would CREATE EXTENSION) could never get a working connection and provider instantiation failed with 'vector type not found in the database'. Create the extension + table over a one-off untyped connection before opening the typed pool. Found by the PG+pgvector fault drill: verified end-to-end (fresh DB → index writes; kill pgvector → degraded status, writes+graph recall unaffected, retry depth 1; restart → queue drains within 30s, degraded clears). * fix(memory): segment CJK queries with gse for lexical recall Chat memory search scored 0 for any natural Chinese sentence because graphLexicalScore/fileRuntimeScore used strings.Fields, which collapses a whole CJK sentence (no inter-word spaces) into a single giant token that never matches a memory body. Route CJK runs through gse (pure-Go jieba port, embedded dict) and keep Latin/numeric runs on whitespace splitting so English behaviour is unchanged. - internal/memory/segment: gse-backed tokenizer + LexicalScore (sync.Once lazy load, degrade to strings.Fields on dict load failure). - graphLexicalScore / fileRuntimeScore delegate to segment.LexicalScore; the two near-duplicate scorers now share one CJK-aware implementation. - Tests: table-driven CJK segmentation + end-to-end graph search recall. * feat(memory): add file→DB markdown ingest for agent-authored memories Agents write memory markdown directly via write_file, which bypassed the DB-backed wiki store (PR memohai#686 made DB nodes the source of truth). Those files were invisible to search_memory and silently wiped by the next RebuildFiles (delete-then-rebuild). This adds the file→DB direction the graph_sync.go comment falsely claimed already existed. - storefs.ReadAllMemoryFilesForIngest: reads /data/memory/*.md including files without a frontmatter id, synthesising a deterministic id (sha256(path+body)) so re-ingest is idempotent. - graphRuntime.IngestMarkdownFiles: storefs → migrate.Plan → wikistore.UpsertNode (ON CONFLICT id DO UPDATE). Deliberately does NOT route through graphRuntime.Add, whose nanosecond ids would duplicate. - graphRuntime.Rebuild now ingests before the destructive derived-view rebuild, so agent files become DB nodes first and survive regeneration. - POST /bots/:bot_id/memory/ingest endpoint + MarkdownIngestProvider interface (optional capability, like SourceSyncProvider). - graph_sync.go comment corrected; _memory.md prompt now requires stable reuse of ids, richer body content, specific tags, and acyclic refs. - Swagger + TS SDK regenerated. Verified on dev stack: memohone bot 4→27 DB nodes after ingest; previously unsearchable agent-written files now surface in search_memory; second ingest is a no-op (idempotent). * feat(web): redesign bot memory UI — search, layer filter, cards, ingest Progressive enhancement of the bot→memory tab (the second half of the memory-system fix). Builds on the ingest/update-by-id endpoints from the prior commits. Backend: - PUT /bots/:bot_id/memory/:memory_id: in-place update by id (provider.Update), replacing the client-side delete-then-add edit emulation that lost id/layer continuity. Ingest now qualifies node ids to canonical <botID>:<localId> so Update/Delete work on agent-authored (synth or frontmatter) ids too. - Swagger + TS SDK regenerated. Frontend (apps/web/src/pages/bots/components/): - bot-memory.vue: server-side search box (debounced, score badges on results), layer filter chips (identity/preference/context/experience/activity/note with per-layer counts), 16px-radius bordered memory cards, degraded/index health banner with a one-click 'Sync files' ingest action. Edit now uses PUT update-by-id; new memory still uses POST add. - memory-card.vue: new card component — layer/confidence/tags badges (Badge owner) + relative time, hover-revealed edit button (Button ghost owner). - use-memory-filter.ts: extracted composable (layer classification, confidence/ tag readers, filter state) — pure, table-testable. i18n: reconciled ja.json bots.memory namespace (was drifted with 13 stale keys + missing keys), added search/filter/layer/degraded/ingest keys across en/zh/ja (35 aligned keys each). Tests: use-memory-filter.test.ts (11 cases: layer classification, confidence/ tag parsing, filter counts, active-layer filter, search-state). All backend ingest tests updated for the qualified-id normalisation. Verified live: PUT update-by-id edits a memory in place (id preserved); ingest re-qualifies 27 nodes; search returns agent-authored content. * fix(memory): omit absent semantic index health * refactor(memory): graph-native compact + reconcile post-merge lint Rewrite the graph runtime's compact path to be graph-native instead of the legacy file-compactor shape (clear node table + mint fresh IDs). - graphRuntime.Compact / CompactWithLLM: canonicalise node IDs, merge nodes describing the same concept, keep the representative ID stable, and record source lineage on the merged node — instead of wiping and re-minting. Removes the standalone compactplan package (now inlined). - fileRuntime.Compact: stub returning 'disabled; use graph runtime' (file runtime no longer owns compaction). - handlers/memory.go + memory_graph_test.go: compact/graph coverage. - Swagger + TS SDK regenerated for the compact shape changes. Post-merge lint reconciliation (main merged in via a7bd660): - file_runtime.go: drop unused receiver/params on the Compact stub. - chat-pane.vue: replace 3 hand-spun LoaderCircle+animate-spin loaders with the Spinner atom (brought in by memohai#741, exceeded the hand-spun-loader baseline); drop the now-unused LoaderCircle import. Verified: CGO_ENABLED=0 build, go test -race (memory+handlers), mise run lint (golangci + ESLint + UI contract) all green. * fix(web): memory settings tiles show provider config, not phantom counts The /settings/memory built-in card read source_count / edge_count / markdown_file_count off the *provider-level* status endpoint, but those fields only exist on the *bot-level* status (memory is per-bot). The provider-level response (ProviderStatusResponse) carries only mode + embedding_model_id, so the four stat tiles always rendered 0 — phantom numbers that looked like an empty/broken store. Replace the four count tiles with three configuration-overview tiles that draw on fields the provider-level status actually exposes: - Mode: the configured memory mode (defaults to 'graph'). - Embedding Model: the chosen model id, or a neutral 'Not configured' — empty is the common local/SQLite case and must not read as an error. - Semantic Index: readiness signal (green 'Healthy' once an embedding model is set; neutral 'Not configured' otherwise), not a live pgvector health probe — this is a provider-config page, not a per-bot status. Add a 'notConfigured' i18n key (en/zh/ja). Per-bot memory counts remain on each bot's memory tab, where they are real. * fix(web): memory settings — lift title out of card, drop tile frames Two visual bugs on the /settings/memory built-in card: 1. A horizontal line sliced across the card directly under the 'Memory Graph' title. The title was rendered as a SettingsRow, which paints a border-b divider — appropriate between sibling rows, not under a title. Lift the title + description above the card as a plain section header, so no in-card hairline cuts under it. 2. The two overview tiles (Memory mode / Semantic Index) had heavy nested borders — framed MetricReadouts (border + bg-card + p-3) sitting inside an already-bordered SettingsSection card = card-in-card. Switch the tiles to unframed so they sit directly on the card surface; the card is the single container. Use text-label/text-body tokens (not px) for the section header to keep the UI px contract green. * chore(memory): reconcile pr/686 with main post-rebase (memohai#748 SQLite/local-desktop removal) After rebasing pr/686 onto main (which removed SQLite and local-desktop via - Remove SQLite memory wiki store (sqlite.go, sqlite_test.go, db/sqlite migration residue) and the wikistore migration test — SQLite is gone. - Hoist filterImplicitEdges into conv.go (it was shared by both backends; postgres.go still needs it) and drop the SQLite-only marshalStringJSON helpers + unused parseTime. - Simplify provideWikiStore to PostgreSQL-only (drop the SQLite branch and sqlitestore dependency). - Add PGVectorConfig to internal/config and wire [pgvector] blocks into all conf/devenv TOMLs (qdrant/sparse blocks removed since QdrantConfig/SparseConfig are no longer in the Config struct). - Re-add pgvector-go deps to go.mod (lost when taking main's go.mod during conflict resolution). - Refresh stale PG/SQLite doc comments to PostgreSQL-only. - Regenerate OpenAPI spec and TS SDK (memory ingest/update endpoints). - Fix UI contract alpha violations in bot-memory.vue (use destructive-soft / foreground tokens instead of hand-written alpha). - Update SectionLayout.vue dev doc: desktop now connects to Memoh Cloud/hosted server instead of running a local SQLite server. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 32c40b1)

chen-ran changed the title ~~refactor(memory): toward a PG-backed LLM wiki (sparse removed, schema landed)~~ refactor(memory): toward a PG-backed LLM wiki Jun 20, 2026

chen-ran marked this pull request as draft June 20, 2026 17:17

chen-ran force-pushed the refactor/memory-for-wiki branch from cf1bc9e to 4707af5 Compare June 20, 2026 17:29

chen-ran changed the title ~~refactor(memory): toward a PG-backed LLM wiki~~ refactor(memory): toward a DB-backed LLM wiki Jun 20, 2026

chen-ran force-pushed the refactor/memory-for-wiki branch 4 times, most recently from e2a2b6d to 19f4c8a Compare June 24, 2026 15:11

chen-ran force-pushed the refactor/memory-for-wiki branch 4 times, most recently from 8283a68 to b38073f Compare July 6, 2026 09:43

chen-ran marked this pull request as ready for review July 7, 2026 02:59

chen-ran requested a review from qqqqqf-q as a code owner July 7, 2026 02:59

chen-ran requested review from Copilot and sheepbox8646 July 7, 2026 04:04

Copilot started reviewing on behalf of chen-ran July 7, 2026 04:04 View session

Copilot AI reviewed Jul 7, 2026

chen-ran added 8 commits July 8, 2026 00:41

chore(spec): regenerate OpenAPI spec and TS SDK

c6a6cc4

Drop the sparse-explain TopKBucket/CDFPoint types and the unused EmbedInput/EmbedUpsertRequest/EmbedUpsertResponse adapters types from the generated OpenAPI schema and the @memohai/sdk TypeScript client.

chen-ran and others added 14 commits July 8, 2026 00:42

chore(db): renumber memory wiki migrations

70f2913

# Conflicts: # db/postgres/migrations/0103_memory_wiki.down.sql # db/sqlite/migrations/0028_memory_wiki.up.sql # internal/db/memory_wiki_migration_test.go

fix(memory): omit absent semantic index health

509c3dc

chen-ran force-pushed the refactor/memory-for-wiki branch from 5c5dede to 117a7b8 Compare July 7, 2026 17:05

chen-ran force-pushed the refactor/memory-for-wiki branch from 117a7b8 to 40a9578 Compare July 7, 2026 17:42

chen-ran merged commit 32c40b1 into memohai:main Jul 7, 2026
16 checks passed

This was referenced Jul 7, 2026

feat(memory): auto-ingest agent memory files in heartbeat ticks #753

Open

fix: docker compose web build context #754

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(memory): toward a DB-backed LLM wiki#686

refactor(memory): toward a DB-backed LLM wiki#686
chen-ran merged 23 commits into
memohai:mainfrom
chen-ran:refactor/memory-for-wiki

chen-ran commented Jun 20, 2026 •

edited

Loading

Uh oh!

chen-ran commented Jun 20, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

wishworldbetter commented Jul 7, 2026

Uh oh!

wishworldbetter commented Jul 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

chen-ran commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What lands in this PR

✅ P0-A — Sparse vector subsystem fully removed

✅ P0-B — Memory wiki graph schema + backfill (data layer)

✅ P0-C — graphRuntime as the only memory mode

✅ Graph UX (initial slice of P2)

Remaining work (follow-up PRs)

P0-D — Cold-start cache + QueryBuilder

P1 — Typed facts (single-phase) + write consistency

P2 — Productization

Validation

Test plan

Uh oh!

chen-ran commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current state of PR 686

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

wishworldbetter commented Jul 7, 2026

Uh oh!

wishworldbetter commented Jul 7, 2026

严重:每次记忆写入都会 rm -rf 整个 /data/memory,静默吞掉 agent 未 ingest 的 markdown

建议修法

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chen-ran commented Jun 20, 2026 •

edited

Loading

chen-ran commented Jun 20, 2026 •

edited

Loading