Summary
msgvault sends raw text to the embeddings endpoint for both documents and queries. Several
widely-used embedding models — including nomic-embed-text, the model used in the
documentation's own Ollama example — specify a required task-instruction prefix and a
different prefix for queries than for documents. Neither prefix is applied anywhere in the
embed or search path, and Ollama does not add them on the model's behalf.
The result is that the documented default configuration runs nomic-embed-text outside its
specified input format. Embeddings still compute and search still returns results, so nothing
fails loudly — but the published retrieval quality for that model assumes correctly instructed
inputs, and asymmetric query/document roles are exactly what the prefixes encode.
I have not measured the quality delta on email data. The claim here is a spec violation with
a plausible retrieval cost, not a measured number. See "How to measure" below — the tooling to
settle it is already in flight.
Why this is worth fixing rather than documenting away
The vector-search guide's primary configuration example is Ollama +
nomic-embed-text,
and the same pairing appears in the PostgreSQL example
and the configuration reference.
So this is not an exotic combination a user has to go out of their way to reach — it is the
path of least resistance through the docs.
Evidence
The model requires the prefixes. Nomic's v1.5 model card states the prompt "must include"
a task-instruction prefix and prescribes the asymmetric RAG mapping — search_document: for
documents, search_query: for queries
(model card,
training repo). Ollama's library page identifies
nomic-embed-text:latest with v1.5.
Ollama does not supply them. Ollama's embedding handler passes each submitted string to the
runner (modulo truncation); it has no way to infer document-vs-query intent and does not try.
msgvault does not supply them either. All references below are pinned to main @
7cf18c370176fa39f006bc964a7079f931c8b6a2:
Worth noting that the seam already exists: EmbedQuery and EmbedDocuments are separate methods
on the client, so query and document text are already distinguishable at the call site. They
currently funnel into the same Embed() with identical treatment.
Secondary observation. Preprocess runs before chunking —
worker.go#L714
then worker.go#L752
— so on a multi-chunk message the Subject: line lands only on chunk 0. That is defensible as-is,
but it matters for the fix: a task prefix must be applied per chunk, after chunking, not folded
into the preprocessed string.
Which models this affects
Prefix conventions are model-specific, which is the argument for a model-agnostic knob rather than
special-casing nomic:
| Model |
Convention |
nomic-embed-text v1/v1.5 |
Required. search_document: / search_query: |
E5 family (multilingual-e5-*) |
Required. passage: / query: |
| EmbeddingGemma |
Required. Task-specific prompt templates for query vs document |
mxbai-embed-large |
Query-side only (Represent this sentence for searching relevant passages: ) |
snowflake-arctic-embed |
Query-side only |
| BGE v1.5 (English) |
Query-side instruction recommended for retrieval |
bge-m3 |
None. Explicitly dropped the instruction requirement |
gte / gte-modernbert |
None. |
| Qwen3-Embedding (0.6B/4B/8B) |
Optional, query-side only; documents take none |
| Voyage |
Handled by its own API contract, already modeled via api_format |
Apple NL via afm |
None |
So today's prefix-free behavior is correct for bge-m3, gte, and Qwen3 (modulo the optional
query instruction), and incorrect for the model the docs lead with.
Proposed fix
Add two optional, model-agnostic fields to [vector.embeddings]:
[vector.embeddings]
document_prefix = "search_document: "
query_prefix = "search_query: "
Semantics:
document_prefix is prepended to each chunk after chunking (see the secondary observation
above), not to the preprocessed message.query_prefix is prepended on every query embed, covering both EmbedQuery call sites in
hybrid/engine.go and any future one.- Both default to empty, so existing installs and prefix-free models see no behavior change.
- Both must enter the generation fingerprint alongside
model, dimension, preprocessing, and
max_input_chars (config.go#L384,
#L419).
Without that, adding a prefix would silently mix prefixed queries against an unprefixed index —
strictly worse than the status quo. With it, the existing index_stale path already does the
right thing and demands a rebuild.
- Prefix characters should not count against
max_input_chars when chunking, or a long prefix
quietly shrinks the usable chunk.
An alternative shape would be presets keyed off the existing api_format enum
(config.go#L25-L27,
added in #589). That is friendlier for the common case but needs a new entry per model family and
still wants an escape hatch, so explicit prefix strings seem like the better primitive — possibly
with presets layered on later.
Whichever shape wins, the docs should also state the convention for the recommended model, since a
user copying the example config today has no way to know a prefix is expected.
How to measure it
This is exactly what #649 (msgvault eval, following #367) is built for: it scores P@10,
nDCG@10, R@100, MAP, and MRR against a qrels file and records the embedding model, dimension, and
generation fingerprint on every run, so a prefixed and an unprefixed generation are directly
comparable.
One caveat for whoever runs it: measure --mode vector in isolation as well as hybrid. Hybrid's
BM25 leg can mask degradation in the vector leg, so a hybrid-only comparison may understate the
effect.
Related
#367 — retrieval-quality eval proposal. This is the closest thing to a public measurement of
msgvault's retrieval quality, and it is the reason I think this issue is worth filing rather than
shrugging at. @fmasi ran msgvault's fts, vector, and hybrid modes against the public TREC 2010
Legal Track (Enron) qrels — real human relevance judgments — and reported that adding a reranking
stage moved P@10 from roughly 0.23 to 0.40 on verbose queries. On their own ~20k-message mailbox the
full stack took coverage@3 from 45% to 84% and recall@1 from 36% to 70% (with the stated caveat that
those queries were LLM-generated and LLM-judged, which is precisely why they re-ran against a public
human-judged set).
The sentence that prompted me to dig: "Two observations, and I might just be holding it wrong."
If the baseline those numbers improve on was measured with an unprefixed nomic-embed-text, then
part of the headroom a reranker is recovering may be quality the embedder never got to express in
the first place. That does not diminish the reranking result — a reranker would likely still help —
but it does mean the vector leg's baseline in that experiment may be understated, and it changes how
you'd read any before/after that uses it as a reference point. Worth re-running the vector leg with
prefixes before concluding how much of the gap is architectural.
#649 — the msgvault eval PR. This is the instrument that would settle the question, and it is
already written. It takes TREC-style qrels and a topics TSV, keeps the metric functions in
internal/eval pure and independently unit-tested, runs any of fts/vector/hybrid, and reports
P@10, nDCG@10, R@100, MAP, and MRR macro-averaged over topics, plus per-query median and p95 latency.
The part that matters here is the provenance block: every run records the embedding model,
dimension, endpoint, vector backend, generation fingerprint, fusion parameters, vector count,
index size on disk, and corpus size. That is exactly what makes a prefixed-vs-unprefixed A/B
legible rather than anecdotal — and if the fix lands as proposed, the prefixes enter the fingerprint,
so msgvault eval would record which variant produced each number without any extra plumbing. The
two changes fit together well enough that it may be worth landing #649 first and using it to justify
this one empirically. It is currently open, mergeable, and reported clean after 15 roborev rounds,
waiting on a maintainer.
#589 — api_format, and the fingerprint precedent. Beyond establishing that per-provider request
shaping belongs in [vector.embeddings], this PR set the precedent for the exact mechanism the fix
needs: contextual generations extend the fingerprint with :avoyage-contextual:v<context_policy_version>,
so switching formats invalidates existing generations instead of silently reusing them. Prefixes
need the same treatment for the same reason, and the pattern to copy already exists in
GenerationFingerprint. #589 also demonstrates the design question worth deciding here: whether a
model's input contract is expressed as a named format (its choice) or as explicit fields (my
proposal above). I lean toward fields because prefix conventions vary per model rather than per
provider, but a maintainer may reasonably prefer consistency with what #589 established.
#497 — embed throughput is CPU-bound in msgvault. Not a cause of this bug, but directly relevant
to anyone who wants to A/B it: builds there sustained 3–4 msg/s while the Ollama endpoint sat idle
at roughly 20× that capacity, so a comparison rebuild costs far more wall-clock than the model or
prefix choice implies. Anyone testing this on a large archive should scope the comparison (e.g.
[vector.embed.scope]) rather than rebuilding the full corpus twice.
#595 — hosted endpoints work today via api_key_env. Relevant because hosted providers have
their own input conventions too — some require task or input-type parameters, some prescribe
instruction text — so a generic prefix knob is useful beyond the local-Ollama case, and reinforces
the argument for model-agnostic fields over a nomic special case.
Environment
Observed by reading main @ 7cf18c37; behavior confirmed present in the shipped v0.19.3
Homebrew build (msgvault embeddings path, sqlite-vec backend).
Summary
msgvault sends raw text to the embeddings endpoint for both documents and queries. Several
widely-used embedding models — including
nomic-embed-text, the model used in thedocumentation's own Ollama example — specify a required task-instruction prefix and a
different prefix for queries than for documents. Neither prefix is applied anywhere in the
embed or search path, and Ollama does not add them on the model's behalf.
The result is that the documented default configuration runs
nomic-embed-textoutside itsspecified input format. Embeddings still compute and search still returns results, so nothing
fails loudly — but the published retrieval quality for that model assumes correctly instructed
inputs, and asymmetric query/document roles are exactly what the prefixes encode.
I have not measured the quality delta on email data. The claim here is a spec violation with
a plausible retrieval cost, not a measured number. See "How to measure" below — the tooling to
settle it is already in flight.
Why this is worth fixing rather than documenting away
The vector-search guide's primary configuration example is Ollama +
nomic-embed-text,and the same pairing appears in the PostgreSQL example
and the configuration reference.
So this is not an exotic combination a user has to go out of their way to reach — it is the
path of least resistance through the docs.
Evidence
The model requires the prefixes. Nomic's v1.5 model card states the prompt "must include"
a task-instruction prefix and prescribes the asymmetric RAG mapping —
search_document:fordocuments,
search_query:for queries(model card,
training repo). Ollama's library page identifies
nomic-embed-text:latestwith v1.5.Ollama does not supply them. Ollama's embedding handler passes each submitted string to the
runner (modulo truncation); it has no way to infer document-vs-query intent and does not try.
msgvault does not supply them either. All references below are pinned to
main@7cf18c370176fa39f006bc964a7079f931c8b6a2:Subject: <subject>\n\n<body>and nothing else —internal/vector/preprocess/preprocess.go#L244internal/vector/hybrid/engine.go#L118and
#L179inputandmodel—internal/vector/embed/client.go#L69-L70EmbeddingsConfighas no field that could express either prefix —internal/vector/config.go#L184Worth noting that the seam already exists:
EmbedQueryandEmbedDocumentsare separate methodson the client, so query and document text are already distinguishable at the call site. They
currently funnel into the same
Embed()with identical treatment.Secondary observation.
Preprocessruns before chunking —worker.go#L714then
worker.go#L752— so on a multi-chunk message the
Subject:line lands only on chunk 0. That is defensible as-is,but it matters for the fix: a task prefix must be applied per chunk, after chunking, not folded
into the preprocessed string.
Which models this affects
Prefix conventions are model-specific, which is the argument for a model-agnostic knob rather than
special-casing nomic:
nomic-embed-textv1/v1.5search_document:/search_query:multilingual-e5-*)passage:/query:mxbai-embed-largeRepresent this sentence for searching relevant passages:)snowflake-arctic-embedbge-m3gte/gte-modernbertapi_formatafmSo today's prefix-free behavior is correct for
bge-m3,gte, and Qwen3 (modulo the optionalquery instruction), and incorrect for the model the docs lead with.
Proposed fix
Add two optional, model-agnostic fields to
[vector.embeddings]:Semantics:
document_prefixis prepended to each chunk after chunking (see the secondary observationabove), not to the preprocessed message.
query_prefixis prepended on every query embed, covering bothEmbedQuerycall sites inhybrid/engine.goand any future one.model,dimension, preprocessing, andmax_input_chars(config.go#L384,#L419).Without that, adding a prefix would silently mix prefixed queries against an unprefixed index —
strictly worse than the status quo. With it, the existing
index_stalepath already does theright thing and demands a rebuild.
max_input_charswhen chunking, or a long prefixquietly shrinks the usable chunk.
An alternative shape would be presets keyed off the existing
api_formatenum(
config.go#L25-L27,added in #589). That is friendlier for the common case but needs a new entry per model family and
still wants an escape hatch, so explicit prefix strings seem like the better primitive — possibly
with presets layered on later.
Whichever shape wins, the docs should also state the convention for the recommended model, since a
user copying the example config today has no way to know a prefix is expected.
How to measure it
This is exactly what #649 (
msgvault eval, following #367) is built for: it scores P@10,nDCG@10, R@100, MAP, and MRR against a qrels file and records the embedding model, dimension, and
generation fingerprint on every run, so a prefixed and an unprefixed generation are directly
comparable.
One caveat for whoever runs it: measure
--mode vectorin isolation as well ashybrid. Hybrid'sBM25 leg can mask degradation in the vector leg, so a hybrid-only comparison may understate the
effect.
Related
#367 — retrieval-quality eval proposal. This is the closest thing to a public measurement of
msgvault's retrieval quality, and it is the reason I think this issue is worth filing rather than
shrugging at. @fmasi ran msgvault's
fts,vector, andhybridmodes against the public TREC 2010Legal Track (Enron) qrels — real human relevance judgments — and reported that adding a reranking
stage moved P@10 from roughly 0.23 to 0.40 on verbose queries. On their own ~20k-message mailbox the
full stack took coverage@3 from 45% to 84% and recall@1 from 36% to 70% (with the stated caveat that
those queries were LLM-generated and LLM-judged, which is precisely why they re-ran against a public
human-judged set).
The sentence that prompted me to dig: "Two observations, and I might just be holding it wrong."
If the baseline those numbers improve on was measured with an unprefixed
nomic-embed-text, thenpart of the headroom a reranker is recovering may be quality the embedder never got to express in
the first place. That does not diminish the reranking result — a reranker would likely still help —
but it does mean the vector leg's baseline in that experiment may be understated, and it changes how
you'd read any before/after that uses it as a reference point. Worth re-running the vector leg with
prefixes before concluding how much of the gap is architectural.
#649 — the
msgvault evalPR. This is the instrument that would settle the question, and it isalready written. It takes TREC-style qrels and a topics TSV, keeps the metric functions in
internal/evalpure and independently unit-tested, runs any offts/vector/hybrid, and reportsP@10, nDCG@10, R@100, MAP, and MRR macro-averaged over topics, plus per-query median and p95 latency.
The part that matters here is the provenance block: every run records the embedding model,
dimension, endpoint, vector backend, generation fingerprint, fusion parameters, vector count,
index size on disk, and corpus size. That is exactly what makes a prefixed-vs-unprefixed A/B
legible rather than anecdotal — and if the fix lands as proposed, the prefixes enter the fingerprint,
so
msgvault evalwould record which variant produced each number without any extra plumbing. Thetwo changes fit together well enough that it may be worth landing #649 first and using it to justify
this one empirically. It is currently open, mergeable, and reported clean after 15 roborev rounds,
waiting on a maintainer.
#589 —
api_format, and the fingerprint precedent. Beyond establishing that per-provider requestshaping belongs in
[vector.embeddings], this PR set the precedent for the exact mechanism the fixneeds: contextual generations extend the fingerprint with
:avoyage-contextual:v<context_policy_version>,so switching formats invalidates existing generations instead of silently reusing them. Prefixes
need the same treatment for the same reason, and the pattern to copy already exists in
GenerationFingerprint. #589 also demonstrates the design question worth deciding here: whether amodel's input contract is expressed as a named format (its choice) or as explicit fields (my
proposal above). I lean toward fields because prefix conventions vary per model rather than per
provider, but a maintainer may reasonably prefer consistency with what #589 established.
#497 — embed throughput is CPU-bound in msgvault. Not a cause of this bug, but directly relevant
to anyone who wants to A/B it: builds there sustained 3–4 msg/s while the Ollama endpoint sat idle
at roughly 20× that capacity, so a comparison rebuild costs far more wall-clock than the model or
prefix choice implies. Anyone testing this on a large archive should scope the comparison (e.g.
[vector.embed.scope]) rather than rebuilding the full corpus twice.#595 — hosted endpoints work today via
api_key_env. Relevant because hosted providers havetheir own input conventions too — some require task or input-type parameters, some prescribe
instruction text — so a generic prefix knob is useful beyond the local-Ollama case, and reinforces
the argument for model-agnostic fields over a nomic special case.
Environment
Observed by reading
main@7cf18c37; behavior confirmed present in the shipped v0.19.3Homebrew build (
msgvault embeddingspath, sqlite-vec backend).