Skip to content

plkumar/simple-vectordb-cpp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

27 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

SimpleVectorDB - C++ Vector Database with WebAssembly

A fast HNSW (Hierarchical Navigable Small World) vector database implementation in C++ with WebAssembly bindings for JavaScript, React, and Angular.

Search Implementation

πŸ“¦ Packages

This project includes three ready-to-use packages for web applications:

Core WebAssembly module with JavaScript/TypeScript bindings.

npm install @simple-vectordb/wasm

React hooks and components for easy integration.

npm install @simple-vectordb/react @simple-vectordb/wasm

Angular service and module for seamless integration.

npm install @simple-vectordb/angular @simple-vectordb/wasm

πŸš€ Quick Start

Vanilla JavaScript

import { initializeWasm, SimpleVectorDB } from '@simple-vectordb/wasm';

// Initialize WASM (do this once)
await initializeWasm();

// Create database
const db = new SimpleVectorDB(5, 0.62, 10);

// Insert vectors
db.insert([1.0, 2.0, 3.0]);

// Search for nearest neighbors
const results = db.search([1.1, 2.1, 3.1], 5);

// Save to JSON
const json = db.toJSON();

// Load from JSON
const db2 = await SimpleVectorDB.fromJSON(json);

// Cleanup
db.delete();

React

import { VectorDBProvider, useVectorDB } from '@simple-vectordb/react';

function App() {
  return (
    <VectorDBProvider>
      <VectorSearch />
    </VectorDBProvider>
  );
}

function VectorSearch() {
  const { insert, search, save, load, isReady } = useVectorDB();

  const handleInsert = () => {
    insert([1.0, 2.0, 3.0]);
  };

  const handleSearch = () => {
    const results = search([1.1, 2.1, 3.1], 5);
    console.log(results);
  };

  const handleSave = () => {
    const json = save();
    localStorage.setItem('db', json);
  };

  const handleLoad = async () => {
    const json = localStorage.getItem('db');
    await load(json);
  };

  if (!isReady) return <div>Loading...</div>;

  return (
    <div>
      <button onClick={handleInsert}>Insert</button>
      <button onClick={handleSearch}>Search</button>
      <button onClick={handleSave}>Save</button>
      <button onClick={handleLoad}>Load</button>
    </div>
  );
}

Angular

import { Component, OnInit, OnDestroy } from '@angular/core';
import { VectorDBService } from '@simple-vectordb/angular';

@Component({
  selector: 'app-search',
  template: `
    <button (click)="insert()">Insert</button>
    <button (click)="search()">Search</button>
    <button (click)="saveDB()">Save</button>
    <button (click)="loadDB()">Load</button>
    <div *ngFor="let result of results">
      Distance: {{ result.distance }}, Index: {{ result.nodeIndex }}
    </div>
  `
})
export class SearchComponent implements OnInit, OnDestroy {
  results = [];

  constructor(private vectorDB: VectorDBService) {}

  async ngOnInit() {
    await this.vectorDB.createDatabase(5, 0.62, 10);
  }

  insert() {
    this.vectorDB.insert([1.0, 2.0, 3.0]).subscribe(
      () => console.log('Inserted')
    );
  }

  search() {
    this.vectorDB.search([1.1, 2.1, 3.1], 5).subscribe(
      results => this.results = results
    );
  }

  saveDB() {
    this.vectorDB.save().subscribe(
      json => localStorage.setItem('db', json)
    );
  }

  loadDB() {
    const json = localStorage.getItem('db');
    this.vectorDB.load(json).subscribe(
      () => console.log('Loaded')
    );
  }

  ngOnDestroy() {
    this.vectorDB.destroy();
  }
}

🎯 Features

  • Fast HNSW Algorithm: Efficient approximate nearest neighbor search
  • WebAssembly: Near-native C++ performance in the browser
  • TypeScript Support: Full type definitions included
  • Framework Integration: Purpose-built hooks and services for React and Angular
  • Persistence: Save and load indexes as JSON
  • Memory Management: Proper cleanup and resource management
  • RxJS Integration: Observable-based API for Angular
  • React Hooks: Modern React patterns with hooks and context

πŸ“Š Use Cases

  • Semantic Search: Find similar text embeddings
  • Image Search: Search similar image vectors
  • Recommendation Systems: Find similar items
  • Clustering: Group similar vectors
  • Anomaly Detection: Find outliers in vector space

βš™οΈ Configuration

All packages support HNSW configuration parameters:

new SimpleVectorDB(L, mL, efc)

Parameters:

  • L (default: 5): Number of layers in HNSW graph

    • Higher = Better recall, more memory
    • Typical range: 3-10
  • mL (default: 0.62): Layer assignment multiplier

    • Controls layer distribution
    • Usually keep at 0.62
  • efc (default: 10): Construction search breadth

    • Higher = Better quality, slower build
    • Typical range: 5-50

πŸ—οΈ Building from Source

Prerequisites

  • C++ compiler with C++17 support
  • CMake 3.10+
  • Emscripten SDK (for WebAssembly)
  • Node.js 14+ (for packages)

Build C++ Native

# Create build directory
mkdir build
cd build

# Configure with CMake
cmake ..

# Compile
make

# Run
./SimpleHNSW

Build WebAssembly (Automated)

# Set up Emscripten environment
source /path/to/emsdk/emsdk_env.sh

# Run build script (builds and copies to packages)
./build-packages.sh

Build WebAssembly (Manual)

# Set up Emscripten environment
source /path/to/emsdk/emsdk_env.sh

# Create build directory
mkdir build-wasm
cd build-wasm

# Configure with CMake
emcmake cmake -DEMSCRIPTEN=1 ..

# Build
emmake make

# Copy outputs to package directory
cp simple-vectordb.js simple-vectordb.wasm ../packages/simple-vectordb-wasm/

Publishing Packages

# Build WASM first
./build-packages.sh

# Publish core WASM package
cd packages/simple-vectordb-wasm
npm publish --access public

# Publish React package
cd ../react-simple-vectordb
npm publish --access public

# Publish Angular package
cd ../angular-simple-vectordb
npm publish --access public

πŸ“ Project Structure

simple-vectordb-cpp/
β”œβ”€β”€ src/                        # C++ source code
β”‚   β”œβ”€β”€ simple_hnsw.h          # Core HNSW implementation
β”‚   β”œβ”€β”€ wasm_bindings.cpp      # Emscripten bindings
β”‚   β”œβ”€β”€ priority_queue.h       # Data structures
β”‚   β”œβ”€β”€ lru_cache.h           # Caching utilities
β”‚   └── util.cpp              # Helper functions
β”œβ”€β”€ packages/
β”‚   β”œβ”€β”€ simple-vectordb-wasm/  # Core WASM package
β”‚   β”‚   β”œβ”€β”€ index.js          # JavaScript wrapper
β”‚   β”‚   β”œβ”€β”€ simple-vectordb.d.ts  # TypeScript definitions
β”‚   β”‚   └── simple-vectordb.js/.wasm  # Built files
β”‚   β”œβ”€β”€ react-simple-vectordb/ # React package
β”‚   β”‚   └── src/
β”‚   β”‚       β”œβ”€β”€ hooks.js      # React hooks
β”‚   β”‚       └── context.js    # React context
β”‚   └── angular-simple-vectordb/  # Angular package
β”‚       └── src/lib/
β”‚           β”œβ”€β”€ vector-db.service.ts
β”‚           └── vector-db.module.ts
β”œβ”€β”€ examples/
β”‚   β”œβ”€β”€ react-example/        # React demo app
β”‚   └── angular-example/      # Angular demo app
β”œβ”€β”€ CMakeLists.txt            # CMake configuration
└── build-packages.sh         # Build automation script

πŸ›οΈ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     Application Layer                   β”‚
β”‚  (React/Angular Applications)           β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     Framework Layer                     β”‚
β”‚  (@simple-vectordb/react|angular)       β”‚
β”‚  β€’ Hooks/Services                       β”‚
β”‚  β€’ Lifecycle Management                 β”‚
β”‚  β€’ State Management                     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     Core WASM Layer                     β”‚
β”‚  (@simple-vectordb/wasm)                β”‚
β”‚  β€’ JavaScript API                       β”‚
β”‚  β€’ Type Conversions                     β”‚
β”‚  β€’ Memory Management                    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     WebAssembly Module                  β”‚
β”‚  (simple-vectordb.wasm)                 β”‚
β”‚  β€’ HNSW Algorithm                       β”‚
β”‚  β€’ Vector Operations                    β”‚
β”‚  β€’ Distance Calculations                β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     C++ Implementation                  β”‚
β”‚  β€’ SimpleHNSWIndex                      β”‚
β”‚  β€’ Core algorithms                      β”‚
β”‚  β€’ Data structures                      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ§ͺ Examples

Complete example applications are available:

πŸ“– API Reference

TypeScript Types

interface SearchResult {
  distance: number;
  nodeIndex: number;
}

// React
interface UseVectorDBResult {
  db: SimpleVectorDB | null;
  insert: (vector: number[]) => void;
  search: (query: number[], k: number) => SearchResult[];
  save: () => string;
  load: (json: string) => Promise<void>;
  isReady: boolean;
}

// Angular - all methods return Observables
class VectorDBService {
  initialize(): Observable<void>;
  createDatabase(L?: number, mL?: number, efc?: number): Promise<void>;
  insert(vector: number[]): Observable<void>;
  search(query: number[], k: number): Observable<SearchResult[]>;
  save(): Observable<string>;
  load(json: string): Observable<void>;
  destroy(): void;
}

🚨 Error Handling

React

try {
  insert(vector);
  const results = search(query, k);
} catch (error) {
  console.error('Operation failed:', error);
}

Angular

this.vectorDB.search(query, k).subscribe({
  next: results => this.results = results,
  error: error => console.error('Search failed:', error)
});

πŸ”§ Performance Tips

  1. Batch inserts: Insert all vectors before searching
  2. Appropriate k: Don't request more results than needed
  3. Memory management: Call delete() on WASM instances when done
  4. Persistence: Use binary formats for large databases (future enhancement)
  5. Dimensions: Keep vector dimensions consistent across all operations

🌐 Browser Compatibility

  • Chrome 57+
  • Firefox 52+
  • Safari 11+
  • Edge 16+

🀝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

πŸ“„ License

MIT

πŸ”— Resources