Skip to content

Migrate persistence to embedded Turso vectors#179

Open
EyJunge1 wants to merge 4 commits into
tickernelz:mainfrom
EyJunge1:feature/turso-only
Open

Migrate persistence to embedded Turso vectors#179
EyJunge1 wants to merge 4 commits into
tickernelz:mainfrom
EyJunge1:feature/turso-only

Conversation

@EyJunge1

Copy link
Copy Markdown

Summary

  • replace the SQLite/USearch backend stack with embedded Turso/libSQL and native vector indexes
  • migrate existing memory shards safely with backups, locking, verification, resume support, and dimension-aware re-embedding
  • harden lifecycle, shard scoping, web API authentication, CI platform checks, and upgrade coverage

Test plan

  • bun run typecheck
  • bun test (222 tests)
  • bun run build
  • bun run format:check
  • verify the packaged libSQL vector extension in the platform smoke workflow

Made with Cursor

Replace the SQLite and USearch stack with native libSQL vector storage, including safe legacy migration, lifecycle hardening, API authentication, and upgrade coverage.

Co-authored-by: Cursor <cursoragent@cursor.com>
@tickernelz

Copy link
Copy Markdown
Owner

I tried to merge this PR after integrating the newer PRs in this batch, but it now has merge conflicts in:

  • src/config.ts
  • src/index.ts
  • src/services/web-server.ts

Please rebase or merge the latest main and resolve the conflicts. This is a large persistence migration, so please also rerun the full build and test suite after resolving them; we can merge it afterward.

Preserve the embedded libSQL migration while incorporating the latest authentication, provider, and web-server updates from main.

Co-authored-by: Cursor <cursoragent@cursor.com>
@EyJunge1

Copy link
Copy Markdown
Author

Resolved the merge conflicts against the latest main and pushed the update to feature/turso-only.

Validation completed:

  • bun run typecheck
  • bun test (249 passing)
  • bun run build
  • bun run format:check

The resolution preserves the Turso/libSQL migration while integrating the current web authentication and provider updates.

EyJunge1 and others added 2 commits July 22, 2026 10:32
Align direct dependencies with current releases while retaining the latest supported TypeScript 6 compiler and its stricter type checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
Adopt the current compiler release after a clean typecheck, build, and full test run.

Co-authored-by: Cursor <cursoragent@cursor.com>
@EyJunge1

EyJunge1 commented Jul 22, 2026

Copy link
Copy Markdown
Author

Release note

This is planned as a major 3.0.0 release because it replaces the persistence format and vector backend. Existing data is migrated with backups and verification, but downgrading to v2 is not supported after migration.

All automated validation is green:

  • bun run typecheck
  • bun test (249 passing)
  • bun run build
  • bun run format:check

Please independently test the migration from existing SQLite/USearch data to embedded Turso/libSQL, including backup creation, verification, recovery, and real-world data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants