Newly-created VikingDB collections: path-field (uri) filtering returns 0 for a long window after Status=READY — path scalar index build lag, self-heals; READY is not a reliable readiness signal

[gaomind]

Summary

The server-side fix for #1119 (PathScope / path-field hierarchical filtering on Volcengine VikingDB) appears to be not effective for newly-created collections. On a collection created on 2026-07-02 via CreateVikingdbCollection (Version=2025-06-09), any filter on the path-typed uri field matches zero rows — including exact full-path matches — while Eq filters on string fields in the same request work correctly (verified with a negative control).

The same fix WAS verified working by us in May 2026 against an older collection (created before the fix landed), directly on our production VikingDB instance — which is why we removed our downstream workaround at the time. The regression surfaced only after we recreated the collection (to pick up the new content/FullText schema for vikingdb-based grep).

Impact

Production-blocking for any OpenViking deployment on the volcengine backend with a recently created collection. Every /api/v1/search/find compiles the tenant isolation filter as And(Eq(account_id), Or(PathScope(uri, ...))); since the PathScope leg matches nothing, all semantic recall returns empty (agent memories are written and vectorized successfully, but can never be recalled). Writes / fs reads / fs grep are unaffected.

Environment

• OpenViking v0.4.6, storage.vectordb.backend=volcengine (ak/sk console + data plane)
• Region: cn-beijing
• Collection: context, ResourceId vdb-d952f89776f14b5c8d3fe5f8c2af3f13, created 2026-07-02T06:36:00Z via Action=CreateVikingdbCollection Version=2025-06-09
• Index: default, Status=READY, ReloadStatus=done; GetVikingdbIndex confirms uri IS present in ScalarIndex with FieldType=path
• Stored rows confirmed present via unfiltered search; sample stored value: "uri": "/user/6d3406500f3e865c/peers/openclaw-remote_pm/memories/cases/final-probe.md"

Reproduction

Direct POST /api/vikingdb/data/search/vector with a real 4096-dim query embedding, same body shape OpenViking sends:

#	filter	rows	request_id
1	(none)	5 (top hit is the expected doc, score 0.667)	02178297826179900000000000000000000ffff0a0057bba8b008
2	{"op":"must","field":"account_id","conds":["gaomingda"]}	5	02178297826198800000000000000000000ffff0a0055099f93bf
3	negative control {"op":"must","field":"account_id","conds":["nonexistent-acct"]}	0 ✓ (proves filtering is active)	02178297826210100000000000000000000ffff0a0049c6a263bc
4	{"op":"must","field":"uri","conds":["/user/6d3406500f3e865c"],"para":"-d=-1"}	0 ✗	02178297826220000000000000000000000ffff0a004621347d13
5	full OpenViking tenant filter (And of #2 + #4)	0 ✗	02178297826232300000000000000000000ffff0a0049c617ecb6
6	{"op":"must","field":"uri","conds":["/"],"para":"-d=-1"}	5 (matches everything)	02178297826252100000000000000000000ffff0a005509c6a0df

Additional matrix via search/random on the same collection: exact full path with -d=0 → 0; exact full path without para → 0; -d=1..7 from "/" → 0 at every depth; no-leading-slash / trailing-slash variants → 0; {"op":"prefix",...} → 400 InvalidParameter: unsupported 'op' value: prefix.

So the observed semantics on this collection are: only "/" + -d=-1 matches (trivially, everything); every non-root ancestor path — including exact full paths — matches nothing. Note the para parameter is not simply ignored (if it were, "/" would exact-match zero rows), which suggests the path index / ancestor computation is partially active but broken for non-root paths.

Timeline vs #1119

• 2026-04/05: fix(search): PathScope filter returns zero results on Volcengine VikingDB backend #1119 reported (identical symptom); maintainers confirmed a server-side fix; "已修复上线".
• 2026-05-07: we verified "para":"-d=-1" working directly against our production instance (old collection) and removed our downstream workaround accordingly.
• 2026-07-02: recreated the collection via the 2025-06-09 API (to enable the new content FullText schema) → path filtering on the new collection exhibits the original fix(search): PathScope filter returns zero results on Volcengine VikingDB backend #1119 symptom again.

Expected vs Actual

• Expected: {"op":"must","field":"uri","conds":["/user/x"],"para":"-d=-1"} matches all rows whose uri is under /user/x (as it did on the old collection after the fix(search): PathScope filter returns zero results on Volcengine VikingDB backend #1119 fix).
• Actual: matches zero rows on the newly-created collection.

Ask

1. Does the fix(search): PathScope filter returns zero results on Volcengine VikingDB backend #1119 server-side fix cover collections/indexes created via the 2025-06-09 CreateVikingdbCollection pipeline? If not, can it be extended?
2. If a collection-level setting/rebuild is required to enable hierarchical path filtering on new collections, please document it.
3. Happy to provide more request_ids / run additional probes on our instance.

[gaomind]

Addendum: controlled A/B experiment rules out schema/FullText as the variable

To isolate whether the new v0.4.6 collection schema (added content FullText / search_tags) is what breaks path filtering, we created two fresh collections side by side (same account, same region cn-beijing, same 2025-06-09 API, same index creation as OpenViking does — hnsw/cosine/int8, date_time fields excluded from ScalarIndex), then upserted identical rows (real 4096-dim vectors, uri = /user/testu/peers/agent1/memories/cases/{a,b}.md):

• ovdiaga — v0.3-era schema (NO content/FullText/search_tags; i.e. the same schema shape as our old collection on which the fix(search): PathScope filter returns zero results on Volcengine VikingDB backend #1119 fix was verified working in May)
• ovdiagb — v0.4.6 schema (WITH content + FullText + search_tags)

Result — both collections exhibit the identical failure:

filter	ovdiaga	ovdiagb
{"op":"must","field":"account_id","conds":["diagacct"]}	2 ✓	2 ✓
{"op":"must","field":"uri","conds":["/user/testu"],"para":"-d=-1"}	0	0
{"op":"must","field":"uri","conds":["/user/testu/peers/agent1/memories/cases/a.md"],"para":"-d=0"} (exact full path)	0	0
{"op":"must","field":"uri","conds":["/"],"para":"-d=-1"}	2 (matches all)	2 (matches all)

Upsert request_ids: 02178298127844100000000000000000000f… (ovdiaga), 02178298127857800000000000000000000f… (ovdiagb); index create request_ids: 20260702162715A9A06CE6619A04427F2D, 202607021627162016268DA45F04C9A820.

Conclusion: the failure is independent of collection schema / FullText config and independent of the client request shape. Any collection created now (2026-07-02, cn-beijing) has non-functional path-field filtering (only "/" + -d=-1 matches, trivially), while the same filter verified working in May on a collection created earlier. This points to a service-side change between 2026-05-07 and 2026-07-02 affecting the path index for newly-created collections.

We are keeping ovdiaga / ovdiagb (with their 2 rows each) in place as a ready-made reproduction for your investigation.

[gaomind]

Consolidated conclusion from our full test matrix (TL;DR for maintainers)

After the original report and the A/B addendum above, here is the complete picture in one place.

What our testing has RULED OUT

Hypothesis	How it was ruled out
Client request shape wrong	Requests replayed byte-identical to what OpenViking sends (search/vector body incl. real 4096-dim query vector); unfiltered search returns the expected doc at score 0.667
Filtering not active at all	Negative control: Eq(account_id="nonexistent") returns 0 while Eq(account_id=<real>) returns all — scalar filtering demonstrably works in the same requests
Data/vectors not present	Unfiltered + Eq-filtered searches return the rows; agg count correct; stored uri values confirmed via search/random ("/user/.../cases/final-probe.md", FieldType path)
Index misconfigured	GetVikingdbIndex: Status=READY, uri present in ScalarIndex as path; index created exactly as OpenViking creates it (hnsw/cosine/int8, date_time excluded)
Collection schema / FullText (content, search_tags)	A/B experiment: two fresh collections, one with the v0.3-era schema (identical shape to our previously-working collection), one with the v0.4.6 schema — both fail identically
para silently ignored (must = exact match, the original #1119 analysis)	Not the case this time: "/" + -d=-1 matches ALL rows (an exact-match interpretation would match zero), while exact full-path conds with -d=0 matches zero — i.e. path semantics are partially active but broken for every non-root path

The only remaining variable

Server-side path-index behavior for newly-created collections. The same filter ({"op":"must","field":"uri","conds":["/user/x"],"para":"-d=-1"}) was verified working by us on 2026-05-07 against a collection created earlier; every collection created on 2026-07-02 (three of them: our production context, plus diagnostic ovdiaga/ovdiagb) exhibits: only "/" with -d=-1 matches (trivially, everything); every non-root ancestor — including exact full paths — matches zero rows.

Impact

On the volcengine backend, OpenViking compiles its tenant-isolation filter as And(Eq(account_id), Or(PathScope(uri,...))) for every /api/v1/search/find. With PathScope matching nothing, all semantic recall returns empty — memories are written and vectorized successfully but can never be recalled. This makes the volcengine backend unusable for any newly-provisioned OpenViking deployment.

Reproduction assets kept live for you

Account/region as in the Environment section; collections ovdiaga and ovdiagb (2 rows each, real vectors, uri under /user/testu/peers/agent1/memories/cases/) are retained untouched. Replay: POST /api/vikingdb/data/search/random with the filters from the table in the addendum. All request_ids provided above.

Two concrete questions

1. Can the path-index/filtering behavior for newly-created collections be fixed (regression window: 2026-05-07 → 2026-07-02, cn-beijing)?
2. After the fix ships: will collections created during the broken window (e.g. our production context, ResourceId vdb-d952f89776f14b5c8d3fe5f8c2af3f13) start working automatically, or do they need to be recreated / re-indexed? This determines our production remediation steps, so explicit guidance would be appreciated.

[gaomind]

Correction: this is a build delay of the path scalar index, not a permanent bug

I need to correct my earlier reports on this issue. After testing across a longer time span, the "path filtering permanently broken on newly-created collections" conclusion was wrong. The real behavior is a build delay of the path scalar index: GetVikingdbIndex returns Status=READY while the path (uri) scalar index is still building in the background; PathScope / must-on-uri filters return 0 during that window, then self-heal.

New evidence

1. All three collections I reported as "broken" now return correct results. The prod context collection and the two diagnostic collections I left for repro (ovdiaga / ovdiagb) — hours later, the exact same PathScope queries that returned 0 now return the full set. The only variable was time. (So those two are no longer a repro; they're now evidence of self-healing.)

2. Quantified with a fresh collection. New collection, wait for Status=READY + data visible (agg count = N), then poll the bare data-plane search/random with {"op":"must","field":"uri","conds":["/user/testu"],"para":"-d=-1"} every 3 minutes:

   • at READY + >54min: still 0 rows
   • self-heals afterwards (a long, hours-scale window)
   • throughout the window, the unfiltered baseline and Eq(account_id) return rows — only the path filter is affected.

3. Rules out "already fixed server-side". If path filtering had been fixed globally, a freshly-created collection would filter correctly immediately. It doesn't — a newly-created collection reproduces the 0-rows window every time. So it is a per-collection build delay, not a global on/off.

What went wrong in my earlier reports (so the retraction is credible)

Every "proof" I posted — the A/B across schemas, the negative control, the 6 request_ids, the raw-API reproduction — was collected inside the same time window: right after the index reported READY, while the path scalar index was still building. They all shared that one hidden variable, so a cross-checked evidence matrix pointed confidently at the wrong conclusion. In particular, the negative control (Eq(fake_account) → 0) only proved the scalar filter engine was alive; it did not prove the path scalar index had finished building. That's on me.

The actual, actionable issue

GetVikingdbIndex returning Status=READY (and ReloadStatus=done) is not a reliable signal that path-field (uri) filtering is queryable. The path scalar index keeps building for a long time (hours-scale in our measurements) after READY, during which every PathScope filter — i.e. every tenant-scoped /search/find — silently returns 0. For an operator this looks exactly like "recall is broken", which is how I got fooled.

Questions for the maintainers:

1. Is this path scalar index build delay expected? Roughly how long should it take?
2. Is there an index status field (beyond Status=READY / ReloadStatus=done) that reflects when path / scalar filtering is actually queryable? If so, please document it — it's the signal operators need after a rebuild.
3. If READY is intended to imply queryable, this looks like a readiness-reporting gap worth closing for the path index.

I'm retracting the "permanent regression" framing — apologies for the noise. On our side we've dropped the client-side workaround we'd built, returned to upstream, and now gate collection rebuilds on a live-probe (write a row → assert it's recallable via a PathScope find) before cutting traffic over, instead of trusting READY. Happy to re-run any probe you'd like.