Skip to content

RAG Builder: Weaviate as a vector store - #84

Merged
Criptic merged 2 commits into
mainfrom
feature/rag-builder-weaviate
Nov 10, 2026
Merged

Criptic merged 2 commits into
mainfrom
feature/rag-builder-weaviate

Conversation

@Criptic

@Criptic Criptic commented Nov 10, 2026

Copy link
Copy Markdown
Member

Summary

The RAG runtime, the steps and the ingestion job have had Weaviate since 2.5.0, but the RAG Builder offered only pgvector and SingleStore. This adds Weaviate to the Builder.

  • Backend list. RAG_BACKENDS gets a Weaviate row. Each row now carries its default port (Weaviate: 8080) and whether the store has databases.
  • No database for Weaviate. A Weaviate setup takes WEAVIATE_RAG_USER/_PW and WEAVIATE_HOST/_PORT from the credential domain and needs no WEAVIATE_DB.
    • The setup records the backend's name where a database goes, never the RAGSTORE_DB a lookup would fall back to, which names another store.
    • The Load step requires that field, and SAS Studio does not run a flow with a required field left empty, so the generated flow carries the same value.
  • Options pane. A new option, RAG: offer Weaviate (enable_weaviate). mdb's MERGED_REPORT_LABELS has it, and the two tests that count the labels now say 33 and 9.
  • Help text. The store's help says the backends carry the same chunk features and that Weaviate keeps no run history (English and German).
  • Docs.
    • The RAG guide lists the Weaviate entries of the credential domain and warns about the RAGSTORE_* fallback.
    • The Options table has the new option.
    • The guide records what was measured live on 2026-11-10, listed below.

Measured live (2026-11-10, Weaviate 1.39.12 on the DGX Spark)

The custom steps ran from x86 SAS Viya (viya4-s2) through an ingress route to Weaviate's /v1. The Builder itself was tested with its mock suite only, not on a live SAS Viya.

  • Ingestion steps: List Documents through Load Vector Store ingested the documents, skipped the unchanged ones and replaced a changed one.
  • Enrich Chunks: stored a prompt output as a Weaviate property and the context header.
  • Retrieve Context: returned the expected document first.
  • Purge Documents: removed one document's chunks.
  • Register Setup: registered the retrieval model and generated the scheduled job from a SAS Studio flow. That job ingested a change.
  • Retrieval model: answered in a compute session in 0.2 to 0.5 s, and in SAS Micro Analytic Service in 0.06 to 0.12 s.

Release note

Until the next release, mdb package-check shows enable_weaviate as drift: the shipped report lacks the label. At the release, the Prompt Builder report needs one save in Visual Analytics with the new page, as in 2.4.0.

Test plan

  • Full CI as the workflow runs it, on Python 3.11 and 3.14: mdb 2,700, RAG 1,562, logging 1,410, setup 465, po 14. package-check --release fails as expected before the release's package PR.
  • npm run build. The committed dist/index.html is that build, byte for byte; a rebuild of unchanged main also reproduces its committed page exactly.
  • All 13 verify suites of the Prompt Builder: 2,180 checks, 0 failures. They ran both on Windows with Edge and on Linux arm64 with Chromium.
  • New verify section weaviate in verify-rag. It saves and manifests a setup on Weaviate, then checks the host, port 8080, the backend's name as the database, the tag, the job parameters, the Load step of the flow and the MANIFEST block of the retrieval model.

🤖 Generated with Claude Code

Criptic and others added 2 commits November 10, 2026 14:22
The RAG runtime, the steps and the ingestion job have had Weaviate since
2.5.0; the Builder offered pgvector and SingleStore only.

- RAG_BACKENDS gets a Weaviate row, and each row now carries its default
  port (8080 for Weaviate) and whether the store has databases. Weaviate
  has none: a setup needs WEAVIATE_RAG_USER/_PW and WEAVIATE_HOST/_PORT,
  no WEAVIATE_DB, and records the backend's name where a database goes -
  never the RAGSTORE_DB a lookup would fall back to, which names another
  store. The Load step requires the field, and a flow with a required
  field left empty does not run in SAS Studio, so the flow gets it too.
- The Options pane offers "RAG: offer Weaviate" (enable_weaviate), from
  the same list; mdb's MERGED_REPORT_LABELS has it, and the two tests
  that count the labels say 33 and 9.
- The help text of the store says the backends carry the same chunk
  features and that Weaviate keeps no run history (en, de).
- verify: all-keys holds every store, a RAGSTORE_DB included, and a new
  section saves and manifests a setup on Weaviate: host, port 8080, its
  name as the database, the tag, the job parameters, the Load step of the
  flow and the MANIFEST block of the retrieval model.
- Docs: the Weaviate entries of the credential domain, the option, and
  what was measured live on 2026-11-10 (the custom steps from x86 SAS
  Viya through an ingress route, Enrich, Purge, Register Setup with a
  flow and its generated job, the retrieval model).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: David Weik <geekupyourlife@gmail.com>
…vice

Published to maslocal and scored with the credentials in its options:
status ok for three questions, the expected document first, 0.06 to 0.12
seconds in the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: David Weik <geekupyourlife@gmail.com>
@Criptic
Criptic merged commit 72c96dd into main Nov 10, 2026
4 checks passed
@Criptic
Criptic deleted the feature/rag-builder-weaviate branch November 10, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant