diff --git a/README.md b/README.md index d96e560..c620674 100644 --- a/README.md +++ b/README.md @@ -1,2223 +1,312 @@ -# MidStream - -**Real-Time LLM Streaming with Lean Agentic Learning & Temporal Analysis** - -[![License](https://img.shields.io/badge/License-MIT%20OR%20Apache--2.0-blue.svg)](#-license) -[![Rust](https://img.shields.io/badge/Rust-1.71+-orange.svg)](https://www.rust-lang.org/) -[![TypeScript](https://img.shields.io/badge/TypeScript-5.3+-blue.svg)](https://www.typescriptlang.org/) -[![Node.js](https://img.shields.io/badge/Node.js-18+-green.svg)](https://nodejs.org/) -[![WASM](https://img.shields.io/badge/WASM-Ready-purple.svg)](wasm/) -[![Crates.io](https://img.shields.io/badge/crates.io-5%20published-orange.svg)](https://crates.io/search?q=temporal) -[![Security](https://img.shields.io/badge/Security-A+-brightgreen.svg)](security-report.json) -[![Tests](https://img.shields.io/badge/Tests-139%20passing-brightgreen.svg)](npm/src/__tests__) -[![CI/CD](https://img.shields.io/badge/CI%2FCD-Active-blue.svg)](.github/workflows/) -[![Docs](https://img.shields.io/badge/docs-complete-success.svg)](docs/) - -**πŸŽ‰ All 5 Core Crates Published on crates.io!** - -- [temporal-compare](https://crates.io/crates/temporal-compare) β€’ [nanosecond-scheduler](https://crates.io/crates/nanosecond-scheduler) β€’ [temporal-attractor-studio](https://crates.io/crates/temporal-attractor-studio) β€’ [temporal-neural-solver](https://crates.io/crates/temporal-neural-solver) β€’ [strange-loop](https://crates.io/crates/strange-loop) - -> **Created by rUv** - Advanced real-time LLM streaming platform with autonomous agents, temporal pattern detection, and multi-modal introspection. - ---- - -## πŸ“‘ Table of Contents - -- [What is MidStream?](#-what-is-midstream) -- [Features](#-features) -- [Quick Start](#-quick-start) -- [Architecture](#-architecture) -- [Rust Workspace Crates](#-rust-workspace-crates) -- [Installation](#-installation) -- [WASM/Browser Support](#-wasmbrowser-support) -- [Performance Benchmarks](#-performance-benchmarks) -- [API Reference](#-api-reference) -- [Examples](#-examples) -- [Development](#-development) -- [CI/CD](#-cicd) -- [Testing](#-testing) -- [Security](#-security) -- [Contributing](#-contributing) -- [License](#-license) - ---- - -## πŸ’‘ What is MidStream? - -MidStream is a powerful platform that makes AI conversations smarter and more responsive. Instead of waiting for an AI to finish speaking before understanding what it's saying, MidStream analyzes responses **as they stream in real-time**β€”enabling instant insights, pattern detection, and intelligent decision-making. - -### The Problem It Solves - -Traditional AI systems process responses only after completion, missing opportunities to: -- **Detect patterns early** in conversations -- **React instantly** to user needs -- **Analyze behavior** as it unfolds -- **Understand context** in real-time -- **Make predictions** before conversations end - -### How MidStream Helps - -MidStream combines cutting-edge technologies to deliver: - -**🎯 Real-Time Intelligence**: Analyze AI responses as they're generated, not after. Detect intents, patterns, and behaviors instantlyβ€”enabling proactive responses and smarter interactions. - -**πŸ€– Autonomous Learning**: Built-in agents that learn from every conversation, automatically adapting and improving over time without manual intervention. The system gets smarter with each interaction. - -**πŸ“Š Deep Pattern Analysis**: Advanced temporal analysis reveals hidden patterns in conversations, predicting user needs and detecting system behaviors that traditional analytics miss. - -**πŸŽ₯ Multi-Modal Understanding**: Process text, audio, and video streams simultaneously. Perfect for voice assistants, video calls, live streaming platforms, and real-time customer support. - -**πŸ” Production-Ready**: Enterprise-grade security, comprehensive testing, and performance optimization ensure reliability for mission-critical applications. - -### Who It's For - -- **Developers** building real-time AI applications -- **Businesses** needing intelligent customer support -- **Researchers** studying conversation dynamics -- **Product Teams** creating voice/video AI experiences -- **Anyone** who wants smarter, faster AI interactions - -Built with Rust for performance and TypeScript for flexibility, MidStream combines the best of both worldsβ€”blazing speed with developer-friendly tools. - ---- - -## πŸš€ Features - -### 🎯 Core Capabilities -- **πŸ”„ Real-Time LLM Streaming** - Low-latency streaming with OpenAI Realtime API & custom providers -- **πŸ€– Lean Agentic Learning** - Autonomous agents with formal reasoning and meta-learning -- **πŸ“Š Temporal Analysis** - Pattern detection, attractor analysis, and Lyapunov exponents -- **πŸŽ₯ Multi-Modal Streaming** - Text, audio, and video stream introspection (RTMP/WebRTC/HLS) -- **πŸ“ˆ Real-Time Dashboard** - Minimal console UI with live metrics and visualizations -- **🧠 Meta-Learning** - Adaptive learning from conversation patterns and behaviors -- **πŸ” Production Ready** - Comprehensive security, error handling, and performance optimization - -### πŸŽ›οΈ Dashboard & Monitoring -- Real-time metrics (FPS, latency, uptime, tokens) -- Temporal analysis visualization (attractors, stability, chaos detection) -- Pattern detection and classification -- Multi-stream monitoring (text/audio/video) -- Configurable refresh rates (100-1000ms) -- Event-driven updates with memory management - -### πŸŽ₯ Streaming Integration -- **QUIC/HTTP/3** - Multiplexed transport with 0-RTT and stream prioritization -- **RTMP/RTMPS** - Real-Time Messaging Protocol support -- **WebRTC** - Peer-to-peer audio/video streaming -- **HLS** - HTTP Live Streaming support -- **WebSocket/SSE** - Bidirectional and server-sent events -- Audio transcription framework (Whisper-ready) -- Video object detection framework (TensorFlow-ready) - -### πŸ¦€ Rust Workspace Crates -- **temporal-compare** - Pattern matching with DTW, LCS, edit distance -- **nanosecond-scheduler** - Ultra-low-latency real-time task scheduling -- **temporal-attractor-studio** - Dynamical systems & Lyapunov analysis -- **temporal-neural-solver** - LTL verification with neural reasoning -- **strange-loop** - Meta-learning & self-referential systems - -### πŸ”¬ Advanced Analysis -- **Pattern Detection** - Dynamic Time Warping (DTW), LCS, edit distance -- **Attractor Analysis** - Fixed point, periodic, chaotic behavior detection -- **Lyapunov Exponents** - System stability measurement -- **Meta-Learning** - Policy adaptation and reward optimization -- **Knowledge Graphs** - Dynamic, evolving knowledge structures -- **Temporal Logic** - Sequence analysis and prediction - -### πŸ›‘οΈ Security & Quality -- 10/10 security checks passed -- No hardcoded credentials -- HTTPS/WSS enforcement -- Input validation & sanitization -- Rate limiting & error handling -- Comprehensive test coverage (100% new code) - ---- - -## πŸ“¦ Quick Start - -### Prerequisites -```bash -# Required -- Rust 1.71+ (for core engine) -- Node.js 18+ (for CLI/Dashboard) -- npm or yarn - -# Optional -- Docker (for containerized deployment) -- OpenAI API key (for Realtime API) -``` - -### Installation - -```bash -# Clone the repository -git clone https://github.com/ruvnet/midstream.git -cd midstream - -# Install Node.js dependencies -cd npm -npm install - -# Build TypeScript -npm run build:ts -``` - -### Run the Dashboard Demo - -```bash -# Full demo with all features -npm run demo - -# Specific demos -npm run demo:text # Text streaming only -npm run demo:audio # Audio streaming only -npm run demo:video # Video streaming only -npm run demo:openai # OpenAI Realtime API - -# QUIC demos -npm run quic-demo # Interactive QUIC demo -npm run quic-demo:server # QUIC server -npm run quic-demo:client # QUIC client -npm run quic-demo:multistream # Multi-stream demo -npm run quic-demo:benchmark # Performance benchmark -``` - -### Basic Usage - -#### Real-Time Dashboard -```typescript -import { MidStreamDashboard } from 'midstream-cli'; - -const dashboard = new MidStreamDashboard(); -dashboard.start(100); // Refresh every 100ms - -// Process messages -dashboard.processMessage('Hello, world!', 5); - -// Process streams -const audioData = Buffer.alloc(1024); -dashboard.processStream('audio-1', audioData, 'audio'); -``` - -#### OpenAI Realtime Integration -```typescript -import { OpenAIRealtimeClient } from 'midstream-cli'; - -const client = new OpenAIRealtimeClient({ - apiKey: process.env.OPENAI_API_KEY, - model: 'gpt-4o-realtime-preview-2024-10-01', - voice: 'alloy' -}); - -client.on('response.text.delta', (delta) => { - console.log(delta); -}); - -await client.connect(); -client.sendText('Analyze this conversation...'); -``` - -#### Restream Integration -```typescript -import { RestreamClient } from 'midstream-cli'; - -const client = new RestreamClient({ - webrtcSignaling: 'wss://signaling.example.com', - enableTranscription: true, - enableObjectDetection: true -}); - -client.on('frame', (frame) => { - console.log(`Frame ${frame.frameNumber}`); -}); - -await client.connectWebRTC(); -``` - -#### QUIC Integration -```typescript -import { createQuicServer, connectQuic } from 'midstream-cli'; - -// Server -const server = createQuicServer({ port: 4433, maxStreams: 1000 }); -server.on('connection', (connection) => { - connection.on('stream', (stream) => { - stream.on('data', (data) => { - console.log('Received:', data.toString()); - }); - }); -}); -await server.listen(); - -// Client -const connection = await connectQuic('localhost', 4433); -const stream = await connection.openBiStream({ priority: 10 }); -stream.write('Hello QUIC!'); -``` - ---- - -## πŸ—οΈ Architecture - -MidStream is built as a modern, modular workspace combining high-performance Rust crates with flexible TypeScript/Node.js tooling. - -### System Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ MidStream Platform β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ TypeScript/Node.js Layer β”‚ β”‚ -β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ -β”‚ β”‚ β”‚ Dashboard β”‚ β”‚ OpenAI RT β”‚ β”‚ QUIC β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ (Console) β”‚ β”‚ Client β”‚ β”‚ Client β”‚ β”‚ β”‚ -β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ β”‚ WASM Bindings Layer β”‚ β”‚ β”‚ -β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β” β”‚ β”‚ -β”‚ β”‚ β”‚ Lean Agentic β”‚ β”‚ Temporal β”‚ β”‚ QUIC β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ WASM β”‚ β”‚ Analysis β”‚ β”‚ Multi β”‚ β”‚ β”‚ -β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Rust Core Workspace β”‚ β”‚ -β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ -β”‚ β”‚ β”‚ temporal- β”‚ β”‚ nanosecond- β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ compare β”‚ β”‚ scheduler β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ (Pattern Match) β”‚ β”‚ (Real-time) β”‚ β”‚ β”‚ -β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ -β”‚ β”‚ β”‚ temporal- β”‚ β”‚ temporal-neural-β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ attractor- β”‚ β”‚ solver β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ studio β”‚ β”‚ (LTL Logic) β”‚ β”‚ β”‚ -β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ -β”‚ β”‚ β”‚ strange-loop β”‚ β”‚ quic- β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ (Meta-Learn) β”‚ β”‚ multistream β”‚ β”‚ β”‚ -β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ - β”‚ β”‚ β”‚ - β–Ό β–Ό β–Ό - β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” - β”‚ OpenAI β”‚ β”‚ Restream β”‚ β”‚ Custom β”‚ - β”‚ Realtime β”‚ β”‚ (RTMP/WebRTC)β”‚ β”‚ Providers β”‚ - β”‚ API β”‚ β”‚ β”‚ β”‚ β”‚ - β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -### Workspace Structure - -``` -midstream/ -β”œβ”€β”€ crates/ # Rust workspace (6 crates, 3,171 LOC) -β”‚ β”œβ”€β”€ temporal-compare/ # Pattern matching & sequence analysis -β”‚ β”œβ”€β”€ nanosecond-scheduler/ # Ultra-low-latency scheduling -β”‚ β”œβ”€β”€ temporal-attractor-studio/ # Dynamical systems analysis -β”‚ β”œβ”€β”€ temporal-neural-solver/ # Temporal logic verification -β”‚ β”œβ”€β”€ strange-loop/ # Meta-learning & self-reference -β”‚ └── quic-multistream/ # QUIC/HTTP3 transport (native + WASM) -β”œβ”€β”€ npm/ # TypeScript/Node.js packages -β”‚ β”œβ”€β”€ src/ # Source code -β”‚ β”‚ β”œβ”€β”€ agent.ts # Lean Agentic learning -β”‚ β”‚ β”œβ”€β”€ dashboard.ts # Real-time dashboard -β”‚ β”‚ β”œβ”€β”€ openai-realtime.ts # OpenAI Realtime API -β”‚ β”‚ β”œβ”€β”€ restream-integration.ts # Video streaming -β”‚ β”‚ β”œβ”€β”€ streaming.ts # WebSocket/SSE -β”‚ β”‚ └── mcp-server.ts # MCP protocol -β”‚ β”œβ”€β”€ examples/ # Demo applications -β”‚ └── __tests__/ # 104 tests (100% passing) -β”œβ”€β”€ wasm-bindings/ # WASM compilation target -β”œβ”€β”€ hyprstream-main/ # Core streaming engine -β”œβ”€β”€ examples/ # Rust examples -└── docs/ # Documentation - -Total: 6 Rust crates, 139 tests passing, 3,171+ LOC -``` - -### Component Overview - -| Component | Purpose | Technology | Status | Tests | -|-----------|---------|-----------|--------|-------| -| **temporal-compare** | Pattern matching, DTW, LCS | Rust | βœ… Production | 8/8 | -| **nanosecond-scheduler** | Real-time task scheduling | Rust + Tokio | βœ… Production | 6/6 | -| **temporal-attractor-studio** | Dynamical systems analysis | Rust + nalgebra | βœ… Production | 6/6 | -| **temporal-neural-solver** | LTL verification & logic | Rust + ndarray | βœ… Production | 7/7 | -| **strange-loop** | Meta-learning framework | Rust | βœ… Production | 8/8 | -| **quic-multistream** | QUIC/HTTP3 transport | Rust (native + WASM) | βœ… Production | 37/37 | -| **Dashboard** | Real-time monitoring UI | TypeScript | βœ… Functional | 26/26 | -| **OpenAI Realtime** | Text/audio streaming | TypeScript | βœ… Functional | 26/26 | -| **Restream** | Multi-protocol video | TypeScript | βœ… Framework | 15/15 | - -### Integration Patterns - -1. **Native Rust β†’ WASM**: High-performance crates compile to WebAssembly -2. **TypeScript β†’ WASM**: Node.js interfaces with WASM modules -3. **Streaming Protocols**: QUIC, WebSocket, SSE, RTMP, WebRTC -4. **Multi-Modal**: Text, audio, video processing in parallel -5. **Event-Driven**: Reactive architecture with async/await - ---- - -## πŸ¦€ Rust Workspace Crates - -MidStream provides **five published Rust crates** available on [crates.io](https://crates.io/), plus one local workspace crate. All core crates are production-ready and actively maintained. - -### Published Crates on crates.io - -All five core crates are published and ready to use in your projects: - -- **[temporal-compare](https://crates.io/crates/temporal-compare)** v0.1.x -- **[nanosecond-scheduler](https://crates.io/crates/nanosecond-scheduler)** v0.1.x -- **[temporal-attractor-studio](https://crates.io/crates/temporal-attractor-studio)** v0.1.x -- **[temporal-neural-solver](https://crates.io/crates/temporal-neural-solver)** v0.1.x -- **[strange-loop](https://crates.io/crates/strange-loop)** v0.1.x - -Simply add them to your `Cargo.toml`: - -```toml -[dependencies] -temporal-compare = "0.1" -nanosecond-scheduler = "0.1" -temporal-attractor-studio = "0.1" -temporal-neural-solver = "0.1" -strange-loop = "0.1" -``` - -### 1. temporal-compare - -[![Crates.io](https://img.shields.io/crates/v/temporal-compare.svg)](https://crates.io/crates/temporal-compare) -[![Documentation](https://docs.rs/temporal-compare/badge.svg)](https://docs.rs/temporal-compare) - -**Advanced temporal sequence comparison and pattern matching** - -```toml -[dependencies] -temporal-compare = "0.1" -``` - -**Features:** -- Dynamic Time Warping (DTW) for sequence alignment -- Longest Common Subsequence (LCS) detection -- Edit Distance (Levenshtein) computation -- Pattern matching with caching -- Efficient LRU cache for repeated comparisons - -**Quick Start:** -```rust -use temporal_compare::{Sequence, TemporalElement, SequenceComparator}; - -// Create sequences -let seq1 = Sequence { - elements: vec![ - TemporalElement { value: "hello", timestamp: 0 }, - TemporalElement { value: "world", timestamp: 100 }, - ] -}; - -// Compare sequences -let comparator = SequenceComparator::new(); -let distance = comparator.dtw_distance(&seq1, &seq2)?; -let lcs = comparator.lcs(&seq1, &seq2)?; -``` - -**Performance:** -- DTW: O(nΓ—m) with optimized dynamic programming -- LCS: O(nΓ—m) with space optimization -- Edit Distance: O(nΓ—m) with configurable weights -- Cache hit rate: >85% for typical workloads - -**Platform Support:** Native (Linux, macOS, Windows), WASM - ---- - -### 2. nanosecond-scheduler - -[![Crates.io](https://img.shields.io/crates/v/nanosecond-scheduler.svg)](https://crates.io/crates/nanosecond-scheduler) -[![Documentation](https://docs.rs/nanosecond-scheduler/badge.svg)](https://docs.rs/nanosecond-scheduler) - -**Ultra-low-latency real-time task scheduler** - -```toml -[dependencies] -nanosecond-scheduler = "0.1" -``` - -**Features:** -- Nanosecond-precision scheduling -- Priority-based task queues -- Lock-free concurrent execution -- Deadline-aware scheduling -- Zero-allocation hot paths - -**Quick Start:** -```rust -use nanosecond_scheduler::{Scheduler, Task, Priority}; -use std::time::Duration; - -let scheduler = Scheduler::new(4); // 4 worker threads - -// Schedule high-priority task -scheduler.schedule(Task { - priority: Priority::High, - deadline: Duration::from_millis(10), - work: Box::new(|| { - // Ultra-low-latency work - }), -})?; - -scheduler.run().await?; -``` - -**Performance:** -- Scheduling latency: <50 nanoseconds (p50) -- Throughput: >1M tasks/second -- Jitter: <100 nanoseconds (p99) -- Zero allocations in hot path - -**Platform Support:** Native (Linux, macOS, Windows) - ---- - -### 3. temporal-attractor-studio - -[![Crates.io](https://img.shields.io/crates/v/temporal-attractor-studio.svg)](https://crates.io/crates/temporal-attractor-studio) -[![Documentation](https://docs.rs/temporal-attractor-studio/badge.svg)](https://docs.rs/temporal-attractor-studio) - -**Dynamical systems and strange attractors analysis** - -```toml -[dependencies] -temporal-attractor-studio = "0.1" -``` - -**Features:** -- Fixed-point attractor detection -- Periodic orbit analysis -- Chaotic behavior detection -- Lyapunov exponent calculation -- Phase space reconstruction - -**Quick Start:** -```rust -use temporal_attractor_studio::{AttractorAnalyzer, SystemState}; - -let analyzer = AttractorAnalyzer::new(); - -// Analyze time series -let states: Vec = vec![/* ... */]; -let attractor = analyzer.detect_attractor(&states)?; -let lyapunov = analyzer.compute_lyapunov_exponent(&states)?; - -match attractor { - AttractorType::FixedPoint(point) => println!("Stable at {:?}", point), - AttractorType::Periodic(period) => println!("Period: {}", period), - AttractorType::Chaotic => println!("Chaotic behavior detected"), -} -``` - -**Performance:** -- Attractor detection: <5ms for 1000-point series -- Lyapunov computation: <10ms for 1000 points -- Phase space reconstruction: O(n log n) - -**Platform Support:** Native (Linux, macOS, Windows), WASM - ---- - -### 4. temporal-neural-solver - -[![Crates.io](https://img.shields.io/crates/v/temporal-neural-solver.svg)](https://crates.io/crates/temporal-neural-solver) -[![Documentation](https://docs.rs/temporal-neural-solver/badge.svg)](https://docs.rs/temporal-neural-solver) - -**Temporal logic verification with neural reasoning** - -```toml -[dependencies] -temporal-neural-solver = "0.1" -``` - -**Features:** -- Linear Temporal Logic (LTL) verification -- Neural network integration for pattern learning -- Sequence prediction -- Temporal constraint solving -- Proof generation - -**Quick Start:** -```rust -use temporal_neural_solver::{LTLSolver, Formula, Trace}; - -let solver = LTLSolver::new(); - -// Define LTL formula: "always (request β†’ eventually response)" -let formula = Formula::always( - Formula::implies( - Formula::atomic("request"), - Formula::eventually(Formula::atomic("response")) - ) -); - -// Verify trace -let trace: Trace = vec![/* state sequence */]; -let result = solver.verify(&formula, &trace)?; -``` - -**Performance:** -- Formula verification: <1ms for simple formulas -- Neural prediction: <2ms per prediction -- Proof generation: <5ms for typical proofs - -**Platform Support:** Native (Linux, macOS, Windows) - ---- - -### 5. strange-loop - -[![Crates.io](https://img.shields.io/crates/v/strange-loop.svg)](https://crates.io/crates/strange-loop) -[![Documentation](https://docs.rs/strange-loop/badge.svg)](https://docs.rs/strange-loop) +
-**Self-referential systems and meta-learning** +# MidStream -```toml -[dependencies] -strange-loop = "0.1" -``` +**Real-time LLM streaming with inflight analysis** -**Features:** -- Meta-learning framework -- Self-referential system modeling -- Policy adaptation -- Reward optimization -- Knowledge graph integration -- Experience replay - -**Quick Start:** -```rust -use strange_loop::{MetaLearner, Policy, Experience}; - -let mut learner = MetaLearner::new(); - -// Learn from experience -let experience = Experience { - state: vec![1.0, 2.0, 3.0], - action: "move_forward", - reward: 1.5, - next_state: vec![1.1, 2.1, 3.1], -}; - -learner.update(&experience)?; - -// Adapt policy -let new_policy = learner.adapt_policy()?; -let action = new_policy.select_action(&state)?; -``` +[![crates.io β€” 6 libs](https://img.shields.io/badge/crates.io-6_published_libs-orange?style=for-the-badge&logo=rust&logoColor=white)](https://crates.io/users/ruvnet) +[![License: MIT OR Apache-2.0](https://img.shields.io/badge/License-MIT_OR_Apache--2.0-yellow?style=for-the-badge)](#license) +[![MSRV: 1.81](https://img.shields.io/badge/MSRV-1.81-blue?style=for-the-badge&logo=rust&logoColor=white)](docs/adr/0023-msrv-policy.md) +[![WASM Ready](https://img.shields.io/badge/WASM-Ready-654FF0?style=for-the-badge&logo=webassembly&logoColor=white)](#wasm--browser) +[![QUIC Transport](https://img.shields.io/badge/QUIC-Transport-10b981?style=for-the-badge&logo=lightning&logoColor=white)](docs/adr/0021-quic-implementation-quinn.md) -**Performance:** -- Policy update: <3ms per experience -- Meta-learning iteration: <10ms -- Knowledge graph query: <1ms -- Experience replay: >10K samples/second +[![Star on GitHub](https://img.shields.io/github/stars/ruvnet/midstream?style=for-the-badge&logo=github&color=gold)](https://github.com/ruvnet/midstream) +[![CI](https://img.shields.io/github/actions/workflow/status/ruvnet/midstream/rust-ci.yml?branch=main&style=for-the-badge&label=CI&logo=github-actions&logoColor=white)](.github/workflows/rust-ci.yml) +[![Supply-chain audit](https://img.shields.io/github/actions/workflow/status/ruvnet/midstream/audit.yml?branch=main&style=for-the-badge&label=audit&logo=rustsec&logoColor=white)](.github/workflows/audit.yml) +[![ADRs: 41](https://img.shields.io/badge/ADRs-41_decisions-6366f1?style=for-the-badge&logo=adblock&logoColor=white)](docs/adr/README.md) -**Platform Support:** Native (Linux, macOS, Windows), WASM +
---- +Treat an LLM token stream as a first-class signal β€” pattern-match it, score it, intervene on it β€” *while the tokens are still arriving*. MidStream is the Rust workspace + WASM bindings + npm shims that make that practical: a nanosecond scheduler, a multi-stream QUIC transport, dynamical-systems analysis, and a self-referential meta-learning loop, all under one consistent feature flag policy and a hard supply-chain gate. -### 6. quic-multistream +### Why MidStream? -**QUIC/HTTP3 multiplexed streaming (native + WASM)** - *Local workspace crate* +> Most "LLM tooling" treats the response as a black box that opens at the end. MidStream treats it as a stream of evidence that you can analyze, gate, and steer per chunk. Built in Rust by [`rUv`](https://ruv.io) so the analysis layer adds microseconds, not seconds β€” and ships the same code path to native, browser, and edge via WASM. -> **Note**: This crate is currently a local workspace crate and not yet published to crates.io. The five crates above are all published and available for use. +### What MidStream Does -```toml -[dependencies] -quic-multistream = { path = "crates/quic-multistream" } # Local only -``` +One workspace, six published libraries, one binary. The libraries are independently useful (pattern matching, real-time scheduling, attractor analysis, QUIC multi-stream) and compose into a single inflight-analysis pipeline. The binary wires them together with the OpenAI Realtime API and a console dashboard. -**Features:** -- QUIC protocol support (0-RTT, multiplexing) -- WebTransport for WASM targets -- Stream prioritization -- Bidirectional and unidirectional streams -- Congestion control -- Native and browser support - -**Quick Start (Native):** -```rust -use quic_multistream::native::{QuicServer, QuicClient}; - -// Server -let server = QuicServer::bind("0.0.0.0:4433").await?; -while let Some(conn) = server.accept().await { - let stream = conn.accept_bi().await?; - // Handle stream -} - -// Client -let client = QuicClient::connect("localhost:4433").await?; -let stream = client.open_bi().await?; -stream.write_all(b"Hello QUIC!").await?; ``` - -**Quick Start (WASM/Browser):** -```rust -use quic_multistream::wasm::WebTransport; - -let transport = WebTransport::connect("https://example.com:4433").await?; -let stream = transport.create_bidirectional_stream().await?; -// Use stream in browser +Provider stream ──▢ zero-copy Bytes ──▢ inflight pipeline ──▢ decisions + (OpenAI RT, (ADR-0006) β”‚ + Anthropic, β”œβ”€ temporal-compare (DTW, LCS, edit-distance) + custom) β”œβ”€ scheduler (ns-scale priority queue) + β”œβ”€ attractor-studio (Lyapunov, phase-space) + β”œβ”€ neural-solver (LTL + neural reasoning) + β”œβ”€ strange-loop (meta-learning) + └─ quic-multistream (transport, 0-RTT) + β”‚ + AIMDS safety gate (optional) + β”‚ + β–Ό + Dashboard Β· MCP Β· WASM Β· npm ``` -**Performance:** -- 0-RTT connection establishment -- Multiplexing: 1000+ concurrent streams -- Throughput: Line-rate on modern hardware -- Latency: <1ms overhead vs raw TCP - -**Platform Support:** Native (Linux, macOS, Windows), WASM (browser via WebTransport) +> **New to MidStream?** Start with one crate. `temporal-compare` and `scheduler` ship clean tests, run on WASM, and have zero unsafe code β€” they're the easiest entry points. Move up to `strange-loop` once you want the full meta-learning loop. --- -## πŸ“¦ Installation - -### Prerequisites - -**Required:** -- **Rust 1.71+** - For using published crates - ```bash - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh - ``` -- **Node.js 18+** - For TypeScript/CLI tools (optional) - ```bash - # Using nvm (recommended) - nvm install 18 - nvm use 18 - ``` +## Quick Start -**Optional:** -- **wasm-pack** - For WASM compilation - ```bash - cargo install wasm-pack - ``` -- **Docker** - For containerized deployments -- **OpenAI API Key** - For Realtime API integration +Three different entry points depending on what you need. Pick one: -### Quick Install +| | **Single library** | **Full workspace** | **WASM / browser** | +|---|---|---|---| +| What it gives you | One crate (e.g. just pattern matching) | All 6 libs + binary + dashboard | Same Rust code, compiled to WASM, in npm | +| What lives in your tree | A line in your `Cargo.toml` | Cloned repo, full build | `npm install @midstream/wasm` | +| Best for | Embedding analysis in an existing app | Running the full inflight pipeline | Browser apps, edge workers | -#### Option 1: Use Published Crates (Recommended) - -All five core crates are published on [crates.io](https://crates.io/) and ready to use: +### Path A β€” One crate ```bash -# Create a new Rust project -cargo new my-midstream-app -cd my-midstream-app -``` - -Add to your `Cargo.toml`: +# Pattern matching only +cargo add midstreamer-temporal-compare -```toml -[dependencies] -# Published MidStream crates from crates.io -temporal-compare = "0.1" -nanosecond-scheduler = "0.1" -temporal-attractor-studio = "0.1" -temporal-neural-solver = "0.1" -strange-loop = "0.1" +# Real-time scheduler only +cargo add midstreamer-scheduler -# For QUIC support (local workspace crate, not yet published) -# quic-multistream = { git = "https://github.com/ruvnet/midstream", branch = "main" } +# Or the full library stack +cargo add midstreamer-temporal-compare midstreamer-scheduler \ + midstreamer-attractor midstreamer-neural-solver \ + midstreamer-strange-loop midstreamer-quic ``` -Then build your project: +### Path B β€” Full workspace ```bash -cargo build --release -``` - -**That's it!** All dependencies will be downloaded from crates.io automatically. - -#### Option 2: From npm (Coming Soon) - -```bash -# Install CLI globally -npm install -g midstream-cli - -# Or use in project -npm install midstream-cli -``` - -#### Option 3: From Source (Development) - -For development or to use the latest features: - -```bash -# Clone repository git clone https://github.com/ruvnet/midstream.git cd midstream -# Install Node.js dependencies -cd npm -npm install - -# Build TypeScript -npm run build:ts - -# Build Rust workspace -cd .. -cargo build --release --workspace - -# Build WASM (optional) -cd wasm-bindings -wasm-pack build --target nodejs --out-dir ../npm/wasm -``` - -#### Option 4: Individual Published Crates - -Install specific crates as needed: +# Build everything +cargo build --workspace --release -```toml -[dependencies] -# Use only the crates you need from crates.io -temporal-compare = "0.1" # Pattern matching and DTW -nanosecond-scheduler = "0.1" # Real-time scheduling -temporal-attractor-studio = "0.1" # Dynamical systems analysis -temporal-neural-solver = "0.1" # LTL verification -strange-loop = "0.1" # Meta-learning - -# Additional dependencies -tokio = { version = "1.42", features = ["full"] } -serde = { version = "1.0", features = ["derive"] } -``` +# Run the binary +cargo run --release --bin midstream -- --help -Browse crates on crates.io: -- πŸ“¦ [temporal-compare](https://crates.io/crates/temporal-compare) -- πŸ“¦ [nanosecond-scheduler](https://crates.io/crates/nanosecond-scheduler) -- πŸ“¦ [temporal-attractor-studio](https://crates.io/crates/temporal-attractor-studio) -- πŸ“¦ [temporal-neural-solver](https://crates.io/crates/temporal-neural-solver) -- πŸ“¦ [strange-loop](https://crates.io/crates/strange-loop) - -### Verify Installation - -```bash -# Check Rust installation -cargo --version -rustc --version - -# Check Node.js installation -node --version -npm --version - -# Run tests -cd npm && npm test # TypeScript tests -cd .. && cargo test # Rust tests - -# Run demos -cd npm && npm run demo # Interactive dashboard -``` - ---- - -## 🌐 WASM/Browser Support - -MidStream crates compile to WebAssembly for browser and edge deployment. - -### Browser Integration - -#### Install via npm - -```bash -npm install midstream-wasm -``` - -#### Use in Browser - -```html - - - - - - -

MidStream WASM Demo

- - -``` - -### WASM Performance - -| Metric | Target | Achieved | -|--------|--------|----------| -| Binary Size (compressed) | <100KB | 65KB (Brotli) | -| Load Time (3G) | <500ms | 320ms | -| Message Processing | <1ms | 0.15ms (p50) | -| WebSocket Send | <0.1ms | 0.05ms (p50) | -| Throughput | >25K msg/s | 50K+ msg/s | - -### Supported Platforms - -| Platform | Native | WASM | Status | -|----------|--------|------|--------| -| **Linux (x86_64)** | βœ… | βœ… | Full support | -| **Linux (ARM64)** | βœ… | βœ… | Full support | -| **macOS (Intel)** | βœ… | βœ… | Full support | -| **macOS (Apple Silicon)** | βœ… | βœ… | Full support | -| **Windows (x64)** | βœ… | βœ… | Full support | -| **Chrome/Edge** | N/A | βœ… | WebTransport | -| **Firefox** | N/A | ⚠️ | Partial (no QUIC) | -| **Safari** | N/A | ⚠️ | Partial (no QUIC) | - -### WASM Features - -1. **Zero-Copy Processing**: Direct buffer access when possible -2. **WebTransport Support**: QUIC in the browser -3. **WebSocket Fallback**: For browsers without WebTransport -4. **Optimized Binary**: Tree-shaking and LTO enabled -5. **Async/Await**: Native Promise integration - ---- - -## ⚑ Performance Benchmarks - -Comprehensive performance testing across all components. - -### Rust Crate Benchmarks - -Run benchmarks with: -```bash +# Run the bench suite cargo bench --workspace ``` -#### temporal-compare - -``` -DTW Distance (100 elements): time: [245.67 Β΅s 248.92 Β΅s 252.48 Β΅s] -LCS (100 elements): time: [189.23 Β΅s 191.45 Β΅s 193.89 Β΅s] -Edit Distance (100 elements): time: [156.78 Β΅s 158.92 Β΅s 161.34 Β΅s] -Pattern Match (cached): time: [12.45 Β΅s 12.78 Β΅s 13.12 Β΅s] -``` - -#### nanosecond-scheduler - -``` -Schedule Task (single): time: [45.23 ns 46.89 ns 48.67 ns] -Schedule Task (batch of 100): time: [3.89 Β΅s 4.12 Β΅s 4.38 Β΅s] -Execute Task (low priority): time: [1.23 Β΅s 1.28 Β΅s 1.34 Β΅s] -Execute Task (high priority): time: [0.89 Β΅s 0.94 Β΅s 0.99 Β΅s] -Throughput: 1.12M tasks/second -``` - -#### temporal-attractor-studio +See [`docs/GETTING_STARTED.md`](docs/GETTING_STARTED.md) for the full zero-to-working-build walkthrough. -``` -Fixed Point Detection (1K pts): time: [3.45 ms 3.52 ms 3.59 ms] -Lyapunov Exponent (1K pts): time: [8.92 ms 9.15 ms 9.38 ms] -Periodic Orbit (1K pts): time: [4.23 ms 4.35 ms 4.47 ms] -Chaos Detection: time: [2.78 ms 2.85 ms 2.92 ms] -``` - -#### temporal-neural-solver - -``` -LTL Verification (simple): time: [0.89 ms 0.92 ms 0.95 ms] -LTL Verification (complex): time: [3.45 ms 3.52 ms 3.59 ms] -Neural Prediction: time: [1.67 ms 1.72 ms 1.77 ms] -Proof Generation: time: [4.23 ms 4.35 ms 4.47 ms] -``` - -#### strange-loop - -``` -Policy Update (single exp): time: [2.34 ms 2.41 ms 2.48 ms] -Meta-Learning Iteration: time: [8.92 ms 9.15 ms 9.38 ms] -Knowledge Graph Query: time: [0.67 Β΅s 0.72 Β΅s 0.77 Β΅s] -Experience Replay (100 samples): time: [8.45 ms 8.67 ms 8.89 ms] -``` - -#### quic-multistream - -``` -Connection Establishment (0-RTT): time: [0.12 ms 0.15 ms 0.18 ms] -Stream Creation: time: [0.05 ms 0.06 ms 0.07 ms] -Send 1KB: time: [0.23 Β΅s 0.25 Β΅s 0.27 Β΅s] -Throughput (single stream): 4.2 Gbps -Concurrent Streams (1000): time: [15.3 ms 15.8 ms 16.3 ms] -``` - -### End-to-End Benchmarks - -#### Lean Agentic System +### Path C β€” WASM / browser ```bash -cargo bench --bench lean_agentic_bench -``` - -``` -Action Verification: 2.34 ms (p50), 5.67 ms (p99) -Theorem Proving: 1.89 ms (p50), 3.45 ms (p99) -Planning: 4.56 ms (p50), 7.89 ms (p99) -Knowledge Graph Update: 0.67 ms (p50), 1.23 ms (p99) - -Full Pipeline (10 messages): 78.3 ms (p50), 145 ms (p99) -Full Pipeline (100 messages): 589 ms (p50), 756 ms (p99) -Full Pipeline (500 messages): 2.8 sec (p50), 3.7 sec (p99) - -Concurrent Sessions (100): 1.45 sec (p50), 2.8 sec (p99) -``` - -### TypeScript/WASM Benchmarks - -```bash -cd npm && npm run benchmark -``` +# In a node project +npm install @midstream/wasm +# Or build from source +cd npm-wasm +npm install +npm run build:wasm ``` -Dashboard Message Processing: <10ms average -Stream Processing (1MB chunks): <5ms per chunk -WebSocket Send: 0.05ms (p50), 0.18ms (p99) -SSE Receive: 0.20ms (p50), 0.70ms (p99) -Memory Usage (baseline): 45MB -Memory Usage (1000 messages): 62MB -Memory Usage (10K messages): 128MB +```js +import { TemporalCompare, Scheduler } from '@midstream/wasm'; -Throughput (single client): 50K+ msg/s -Throughput (100 concurrent): 25K+ msg/s +const cmp = new TemporalCompare(); +const distance = cmp.dtw(seriesA, seriesB); ``` -### Performance Targets vs Achieved - -| Component | Target | Achieved | Status | -|-----------|--------|----------|--------| -| **Message Processing** | <20ms | 10ms (avg) | βœ… Exceeded | -| **Scheduling Latency** | <100ns | 46ns (p50) | βœ… Exceeded | -| **Throughput** | >50 chunks/s | >1000/s | βœ… Exceeded | -| **Concurrent Sessions** | 100+ | 100+ | βœ… Met | -| **WASM Binary Size** | <100KB | 65KB | βœ… Exceeded | -| **Memory Efficiency** | <100MB | <128MB | βœ… Met | - --- -## πŸ“š Documentation - -### Core Documentation -- **[Dashboard Guide](plans/DASHBOARD_README.md)** - Complete dashboard usage and API reference -- **[Implementation Summary](plans/IMPLEMENTATION_SUMMARY.md)** - Architecture and technical details -- **[Verification Report](plans/VERIFICATION_REPORT.md)** - Complete functionality verification -- **[Lean Agentic Guide](plans/LEAN_AGENTIC_GUIDE.md)** - Autonomous learning system guide -- **[WASM Performance Guide](plans/WASM_PERFORMANCE_GUIDE.md)** - WebAssembly optimization guide -- **[Benchmarks & Optimizations](plans/BENCHMARKS_AND_OPTIMIZATIONS.md)** - Performance analysis - -### API Reference - -#### Dashboard API -```typescript -class MidStreamDashboard { - start(refreshRate: number): void - stop(): void - processMessage(message: string, tokens?: number): void - processStream(streamId: string, data: Buffer, type: 'audio'|'video'|'text'): void - getState(): DashboardState - getAgent(): MidStreamAgent -} -``` - -#### OpenAI Realtime API -```typescript -class OpenAIRealtimeClient { - connect(): Promise - disconnect(): void - sendText(text: string): void - sendAudio(audio: string): void - updateSession(config: SessionConfig): void - on(event: string, callback: Function): void -} -``` - -#### Restream API -```typescript -class RestreamClient { - connectRTMP(): Promise - connectWebRTC(): Promise - connectHLS(url: string): Promise - disconnect(): void - getAnalysis(): StreamAnalysis - on(event: string, callback: Function): void -} -``` - -#### QUIC API -```typescript -class QuicConnection { - connect(): Promise - openBiStream(config?: QuicStreamConfig): Promise - openUniStream(config?: QuicStreamConfig): Promise - close(): void - getStats(): QuicConnectionStats - getAgent(): MidStreamAgent -} - -class QuicServer { - listen(): Promise - close(): void - getConnectionCount(): number - on(event: string, callback: Function): void -} - -class QuicStream { - write(data: Buffer | string): boolean - close(): void - setPriority(priority: number): void - on(event: string, callback: Function): void -} -``` +## What You Get + +| | Capability | What it means | +|---|------------|---------------| +| ⚑ | **Inflight analysis** | Pattern-match, score, and gate LLM tokens as they arrive β€” not after the response completes | +| πŸ¦€ | **6 published Rust libraries** | Each crate ships independently on crates.io with its own changelog and MSRV gate | +| 🌐 | **WASM-first design** | Every library that doesn't touch the OS compiles to `wasm32-unknown-unknown` with no source forks | +| πŸ“ | **Honest benchmarks** | `cargo bench --workspace` against real workloads, not mocks β€” see [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) | +| πŸ”’ | **Hard supply-chain gate** | `cargo audit` + `cargo deny` block every PR; ADR-0014 | +| πŸ§ͺ | **proptest + fuzz baseline** | Every published library has a property-test suite; `midstream-fuzz` has libfuzzer targets β€” ADR-0038 | +| 🚦 | **Bounded backpressure** | Every async channel has a bound; no unbounded `mpsc`, ever β€” ADR-0007 | +| πŸ›‘οΈ | **Secure-by-default TLS** | QUIC defaults to the platform verifier; `SkipServerVerification` is opt-in behind an `insecure-` feature flag β€” ADR-0011 | +| πŸ“Š | **Console dashboard** | Live FPS, latency, attractor type, chaos detection, stream health β€” `ratatui`-based, no JS toolchain needed | +| πŸ”Œ | **MCP tool surface** | Namespaced, versioned tools for swarm/agent integrations β€” ADR-0032 | +| πŸ“š | **41 ADRs** | Every architectural decision is documented; superseding is via new ADRs, not edits β€” [`docs/adr/`](docs/adr/README.md) | +| 🧾 | **Dual MIT OR Apache-2.0** | Whole workspace; ADR-0036 | --- -## πŸ“– Examples - -MidStream includes comprehensive examples for all major use cases. - -### Example 1: Real-Time Customer Support Dashboard - -```typescript -import { MidStreamDashboard } from 'midstream-cli'; -import { OpenAIRealtimeClient } from 'midstream-cli'; - -const dashboard = new MidStreamDashboard(); -const openai = new OpenAIRealtimeClient({ - apiKey: process.env.OPENAI_API_KEY, - model: 'gpt-4o-realtime-preview-2024-10-01' -}); - -// Start real-time monitoring -dashboard.start(100); // 100ms refresh - -// Connect to OpenAI Realtime -await openai.connect(); - -// Handle responses -openai.on('response.text.delta', (delta) => { - dashboard.processMessage(delta, 5); - - // Get agent analysis - const agent = dashboard.getAgent(); - const patterns = agent.detectPattern(history, ['greeting', 'issue', 'resolution']); - - if (patterns.confidence > 0.85) { - console.log(`Detected pattern: ${patterns.pattern} with ${patterns.confidence} confidence`); - } -}); - -// Send user message -openai.sendText('I need help with my account'); -``` - -### Example 2: Video Stream Analysis with Pattern Detection - -```typescript -import { RestreamClient } from 'midstream-cli'; -import { MidStreamDashboard } from 'midstream-cli'; - -const dashboard = new MidStreamDashboard(); -const restream = new RestreamClient({ - enableObjectDetection: true, - enableTranscription: true -}); - -// Monitor video stream -restream.on('frame', (frame) => { - dashboard.processStream(frame.streamId, frame.data, 'video'); -}); - -// Detect objects in video -restream.on('objects_detected', (data) => { - console.log(`Frame ${data.frameNumber}: ${data.objects.length} objects detected`); - - // Analyze patterns over time - const agent = dashboard.getAgent(); - const temporalPattern = agent.detectTemporalPattern(data.objects); - - if (temporalPattern.type === 'recurring') { - console.log('Recurring object pattern detected'); - } -}); - -await restream.connectWebRTC(); -``` - -### Example 3: Low-Latency Multiplexed Streaming with QUIC - -```typescript -import { createQuicServer, connectQuic } from 'midstream-cli'; - -// Server -const server = createQuicServer({ - port: 4433, - maxStreams: 1000, - cert: './cert.pem', - key: './key.pem' -}); - -server.on('connection', (connection) => { - console.log('New QUIC connection'); - - connection.on('stream', async (stream) => { - // Multiplexed streams with priorities - stream.setPriority(stream.metadata.priority || 5); - - stream.on('data', (data) => { - console.log(`Received on stream ${stream.id}: ${data.toString()}`); - stream.write(`Echo: ${data}`); - }); - }); -}); - -await server.listen(); - -// Client -const conn = await connectQuic('localhost', 4433); - -// Create multiple streams with different priorities -const videoStream = await conn.openBiStream({ priority: 10 }); -const audioStream = await conn.openBiStream({ priority: 9 }); -const telemetryStream = await conn.openUniStream({ priority: 1 }); - -// Send data -videoStream.write(videoFrame); -audioStream.write(audioChunk); -telemetryStream.write(JSON.stringify({ cpu: 45, mem: 62 })); -``` - -### Example 4: Meta-Learning Agent with Strange Loop - -Using the published `strange-loop` crate from crates.io: - -```toml -[dependencies] -strange-loop = "0.1" # Published on crates.io -``` - -```rust -use strange_loop::{MetaLearner, Policy, Experience}; - -#[tokio::main] -async fn main() -> Result<(), Box> { - let mut learner = MetaLearner::new(); - - // Simulate conversation learning - for i in 0..1000 { - // Collect experience from environment - let experience = Experience { - state: get_conversation_state(), - action: select_response(), - reward: get_user_feedback(), - next_state: get_next_state(), - }; - - // Update meta-learner - learner.update(&experience)?; - - // Every 100 iterations, adapt policy - if i % 100 == 0 { - let new_policy = learner.adapt_policy()?; - println!("Policy adapted. New strategy: {:?}", new_policy.strategy); - } - } - - // Get learned knowledge - let knowledge = learner.get_knowledge_graph()?; - println!("Learned {} concepts", knowledge.num_entities()); - - Ok(()) -} -``` - -### Example 5: Temporal Pattern Analysis - -Using published crates from crates.io: - -```toml -[dependencies] -temporal-attractor-studio = "0.1" # Published on crates.io -temporal-compare = "0.1" # Published on crates.io -``` - -```rust -use temporal_attractor_studio::{AttractorAnalyzer, SystemState}; -use temporal_compare::{Sequence, SequenceComparator}; - -fn analyze_conversation_dynamics(messages: Vec) -> Result { - let analyzer = AttractorAnalyzer::new(); - - // Convert messages to system states - let states: Vec = messages.iter() - .map(|m| SystemState::from_message(m)) - .collect(); - - // Detect conversation attractor - let attractor = analyzer.detect_attractor(&states)?; - let lyapunov = analyzer.compute_lyapunov_exponent(&states)?; - - match attractor { - AttractorType::FixedPoint(point) => { - println!("Conversation converging to stable state: {:?}", point); - } - AttractorType::Periodic(period) => { - println!("Periodic conversation pattern (period: {})", period); - } - AttractorType::Chaotic if lyapunov > 0.0 => { - println!("Chaotic conversation dynamics detected"); - } - _ => println!("Complex dynamics"), - } - - Ok(Analysis { attractor, lyapunov }) -} -``` - -### More Examples - -Browse the full example collection: - -- **[Dashboard Demo](npm/examples/dashboard-demo.ts)** - Full-featured dashboard demo -- **[QUIC Demo](npm/examples/quic-demo.ts)** - Interactive QUIC client/server -- **[OpenAI Streaming](npm/examples/openai-streaming.ts)** - Real-time OpenAI integration -- **[Lean Agentic Streaming](examples/lean_agentic_streaming.rs)** - Rust agentic system -- **[OpenRouter Integration](examples/openrouter.rs)** - Alternative LLM provider -- **[QUIC Server](examples/quic_server.rs)** - Production QUIC server - ---- - -## πŸ› οΈ Development - -### Building from Source - -```bash -# Clone and setup -git clone https://github.com/ruvnet/midstream.git -cd midstream - -# Install dependencies -cd npm && npm install - -# Build all components -npm run build # Builds TypeScript + WASM -npm run build:ts # TypeScript only -npm run build:wasm # WASM only - -# Build Rust workspace -cd .. -cargo build --workspace - -# Build for release (optimized) -cargo build --release --workspace - -# Build specific crate -cargo build -p temporal-compare --release -``` - -### Running Tests - -```bash -# TypeScript tests -cd npm -npm test # Run all tests -npm test:watch # Watch mode -npm test:coverage # With coverage - -# Rust tests -cd .. -cargo test --workspace # All crates -cargo test -p temporal-compare # Specific crate -cargo test -- --nocapture # Show output - -# Integration tests -cargo test --test '*' - -# Doc tests -cargo test --doc -``` - -### Running Benchmarks - -```bash -# Rust benchmarks -cargo bench --workspace # All benchmarks -cargo bench -p nanosecond-scheduler # Specific crate -cargo bench -- --save-baseline main # Save baseline +## Rust Workspace -# TypeScript benchmarks (if available) -cd npm && npm run benchmark -``` +Six published library crates at the same workspace version (currently `0.2.1`). Each one is independently useful and can be added without pulling the others. -### Code Quality +
+πŸ“¦ The six libraries (click to expand) -```bash -# Rust -cargo fmt --all --check # Format check -cargo clippy --all-targets # Linting -cargo audit # Security audit - -# TypeScript -npm run lint # ESLint -npm run format # Prettier -``` +| Crate | What it does | crates.io | +|---|---|---| +| [**midstreamer-temporal-compare**](crates/temporal-compare/) | Sequence comparison: Dynamic Time Warping (DTW), Longest Common Subsequence (LCS), edit distance, with LTL extensions | [![v](https://img.shields.io/crates/v/midstreamer-temporal-compare?label=&color=orange&logo=rust)](https://crates.io/crates/midstreamer-temporal-compare) | +| [**midstreamer-scheduler**](crates/nanosecond-scheduler/) | Ultra-low-latency priority queue with strict `Ord` semantics (priority-major, deadline-minor). Designed for ns-scale arrival rates | [![v](https://img.shields.io/crates/v/midstreamer-scheduler?label=&color=orange&logo=rust)](https://crates.io/crates/midstreamer-scheduler) | +| [**midstreamer-attractor**](crates/temporal-attractor-studio/) | Dynamical-systems analysis: Lyapunov exponents, phase-space reconstruction, attractor classification (fixed-point / periodic / chaotic) | [![v](https://img.shields.io/crates/v/midstreamer-attractor?label=&color=orange&logo=rust)](https://crates.io/crates/midstreamer-attractor) | +| [**midstreamer-neural-solver**](crates/temporal-neural-solver/) | Linear Temporal Logic (LTL) verification fused with a small neural reasoner. Used for inflight safety/intent checks | [![v](https://img.shields.io/crates/v/midstreamer-neural-solver?label=&color=orange&logo=rust)](https://crates.io/crates/midstreamer-neural-solver) | +| [**midstreamer-strange-loop**](crates/strange-loop/) | Self-referential meta-learning: the system can observe and adjust its own analysis policy mid-stream | [![v](https://img.shields.io/crates/v/midstreamer-strange-loop?label=&color=orange&logo=rust)](https://crates.io/crates/midstreamer-strange-loop) | +| [**midstreamer-quic**](crates/quic-multistream/) | QUIC multi-stream transport (quinn-backed) for native; thin shim on the browser side. 0-RTT, stream prioritization, secure-by-default TLS | [![v](https://img.shields.io/crates/v/midstreamer-quic?label=&color=orange&logo=rust)](https://crates.io/crates/midstreamer-quic) | -### Project Structure Details +The `midstream` binary at the workspace root wires these into the inflight pipeline. -``` -midstream/ -β”œβ”€β”€ .github/ -β”‚ └── workflows/ # CI/CD pipelines -β”‚ β”œβ”€β”€ rust-ci.yml # Rust testing & builds -β”‚ └── release.yml # Release automation -β”œβ”€β”€ crates/ # Rust workspace -β”‚ β”œβ”€β”€ temporal-compare/ -β”‚ β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”‚ └── lib.rs # Main library code -β”‚ β”‚ β”œβ”€β”€ tests/ # Integration tests -β”‚ β”‚ β”œβ”€β”€ benches/ # Benchmarks -β”‚ β”‚ └── Cargo.toml # Crate manifest -β”‚ β”œβ”€β”€ nanosecond-scheduler/ -β”‚ β”œβ”€β”€ temporal-attractor-studio/ -β”‚ β”œβ”€β”€ temporal-neural-solver/ -β”‚ β”œβ”€β”€ strange-loop/ -β”‚ └── quic-multistream/ -β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”œβ”€β”€ lib.rs # Common code -β”‚ β”‚ β”œβ”€β”€ native.rs # Native implementation -β”‚ β”‚ └── wasm.rs # WASM implementation -β”‚ └── Cargo.toml -β”œβ”€β”€ npm/ -β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”œβ”€β”€ agent.ts # Lean agentic learning -β”‚ β”‚ β”œβ”€β”€ dashboard.ts # Real-time dashboard -β”‚ β”‚ β”œβ”€β”€ openai-realtime.ts # OpenAI integration -β”‚ β”‚ β”œβ”€β”€ restream-integration.ts -β”‚ β”‚ β”œβ”€β”€ streaming.ts # WebSocket/SSE -β”‚ β”‚ └── mcp-server.ts # MCP protocol -β”‚ β”œβ”€β”€ __tests__/ # Jest tests -β”‚ β”œβ”€β”€ examples/ # Demo applications -β”‚ β”œβ”€β”€ scripts/ # Utility scripts -β”‚ └── package.json -β”œβ”€β”€ wasm-bindings/ # WASM compilation target -β”œβ”€β”€ examples/ # Rust examples -β”œβ”€β”€ plans/ # Documentation -β”œβ”€β”€ Cargo.toml # Workspace manifest -└── README.md # This file -``` +
---- +
+🧰 The three npm packages (click to expand) -## πŸ”„ CI/CD - -MidStream uses GitHub Actions for comprehensive CI/CD. - -### Workflows - -#### 1. Rust CI/CD (`.github/workflows/rust-ci.yml`) - -**Triggers:** -- Push to `main`, `develop` -- Pull requests to `main` -- Manual dispatch - -**Jobs:** -- **Format Check**: `cargo fmt --check` -- **Clippy Lints**: `cargo clippy -- -D warnings` -- **Test Matrix**: - - OS: Ubuntu, macOS, Windows - - Rust: stable, nightly - - 3Γ—2 = 6 combinations -- **Build Crates**: Individual crate builds -- **WASM Build**: WebAssembly compilation -- **Benchmarks**: Performance regression detection -- **Documentation**: `cargo doc` with deployment -- **Security Audit**: `cargo audit` -- **Code Coverage**: Codecov integration - -**Build Matrix:** -```yaml -strategy: - matrix: - os: [ubuntu-latest, macos-latest, windows-latest] - rust: [stable, nightly] -``` +| Package | What it does | +|---|---| +| [**@midstream/wasm**](npm-wasm/) | The Rust workspace compiled to WASM β€” same `temporal-compare`, `scheduler`, `attractor`, `strange-loop` available from JS/TS | +| [**midstream**](npm/) | Node CLI + dashboard. Calls the Rust binary or `@midstream/wasm` depending on environment | +| [**lean-agentic-js**](lean-agentic-js/) | The agentic-loop tooling (action / observation / plan / learning) as standalone JS/TS, with its own test suite | -#### 2. Release Workflow (`.github/workflows/release.yml`) +
-**Triggers:** -- Tags matching `v*.*.*` -- Manual dispatch with version input +
+🌐 WASM / browser support -**Jobs:** -- **Create Release**: GitHub release with changelog -- **Build Release Binaries**: - - Linux (x86_64, ARM64) - - macOS (Intel, Apple Silicon) - - Windows (x64) -- **Publish Crates**: Automated crates.io publishing -- **Update Documentation**: Versioned docs deployment +Every Rust library that doesn't touch the OS compiles to `wasm32-unknown-unknown` from the same source tree: -**Release Process:** ```bash -# Automatic on tag push -git tag -a v0.2.0 -m "Release v0.2.0" -git push origin v0.2.0 - -# Or manual trigger via GitHub Actions UI -``` - -### CI Performance - -| Job | Average Duration | Success Rate | -|-----|-----------------|--------------| -| Format Check | ~30s | 100% | -| Clippy | ~3min | 98% | -| Tests (Ubuntu/stable) | ~8min | 99% | -| Tests (macOS/stable) | ~10min | 97% | -| Tests (Windows/stable) | ~12min | 95% | -| WASM Build | ~5min | 99% | -| Benchmarks | ~15min | 98% | -| Documentation | ~6min | 100% | - -### Quality Gates - -Pull requests must pass: -- βœ… All format checks -- βœ… All clippy lints (zero warnings) -- βœ… All tests on all platforms -- βœ… Security audit (no vulnerabilities) -- βœ… Documentation builds successfully -- βœ… WASM compilation succeeds - ---- - -## πŸ§ͺ Testing - -Comprehensive test coverage across all components. - -### Test Statistics - -``` -Total Tests: 139 passing - -TypeScript/npm: - Test Suites: 5 suites - Tests: 104 total - βœ… Dashboard: 26/26 (100%) - βœ… OpenAI Realtime: 26/26 (100%) - βœ… QUIC Integration: 37/37 (100%) - βœ… Restream: 15/15 (100%) - βœ… Agent: Pass - -Rust Workspace: - Crates: 6 crates - Tests: 35+ total - βœ… temporal-compare: 8/8 (100%) - βœ… nanosecond-scheduler: 6/6 (100%) - βœ… temporal-attractor-studio: 6/6 (100%) - βœ… temporal-neural-solver: 7/7 (100%) - βœ… strange-loop: 8/8 (100%) - βœ… quic-multistream: (native + WASM tests) - -Lines of Code: 3,171+ production Rust code -Test Coverage: >85% (Rust), >90% (TypeScript new code) +cargo build -p midstreamer-temporal-compare --target wasm32-unknown-unknown --no-default-features +cargo build -p midstreamer-scheduler --target wasm32-unknown-unknown --no-default-features +cargo build -p midstreamer-strange-loop --target wasm32-unknown-unknown --no-default-features ``` -### Running Tests - -```bash -# All TypeScript tests -cd npm -npm test - -# With coverage report -npm run test:coverage - -# Watch mode for development -npm run test:watch - -# Specific test file -npm test -- openai-realtime.test.ts - -# All Rust tests -cargo test --workspace --all-features - -# Specific crate -cargo test -p temporal-compare - -# With output -cargo test -- --nocapture - -# Integration tests only -cargo test --test '*' +The `npm-wasm/` package builds with `wasm-pack`, ships TypeScript declarations, and works in Node β‰₯ 18, modern browsers, and edge runtimes (Cloudflare Workers, Deno, Bun). Network egress from the WASM sandbox is gated by an allowlist β€” ADR-0015. -# Doc tests -cargo test --doc -``` +
-### Test Types - -#### 1. Unit Tests -```rust -// Example from temporal-compare -#[test] -fn test_dtw_distance() { - let seq1 = create_test_sequence(&[1, 2, 3]); - let seq2 = create_test_sequence(&[1, 2, 4]); - let comparator = SequenceComparator::new(); - let distance = comparator.dtw_distance(&seq1, &seq2).unwrap(); - assert!(distance > 0.0); -} -``` +
+πŸ›οΈ Architecture overview -#### 2. Integration Tests -```typescript -// Example from OpenAI Realtime -describe('OpenAIRealtimeClient', () => { - it('should connect and handle responses', async () => { - const client = new OpenAIRealtimeClient({ apiKey: 'test' }); - await client.connect(); - expect(client.isConnected()).toBe(true); - }); -}); ``` - -#### 3. Simulation Tests -```rust -// Example from lean agentic benchmarks -#[test] -fn test_high_frequency_streaming() { - let agent = create_test_agent(); - let messages: Vec<_> = (0..1000).map(|i| format!("Message {}", i)).collect(); - - for msg in messages { - agent.process_message(&msg, 5).unwrap(); - } - - let metrics = agent.get_metrics(); - assert!(metrics.throughput > 50.0); // >50 msg/s -} + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β”‚ LLM provider stream β”‚ + β”‚ (OpenAI RT Β· Anthropic Β· ...)β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β–Ό + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β”‚ midstream binary β”‚ + β”‚ (zero-copy Bytes) β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ + β–Ό + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β”‚ Inflight pipeline β”‚ + β”‚ β”‚ + β”‚ temporal-compare ─► scheduler ─► attractor β”‚ + β”‚ β”‚ β”‚ β”‚ β”‚ + β”‚ β–Ό β–Ό β–Ό β”‚ + β”‚ neural-solver strange-loop dashboard β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β–Ό β–Ό β–Ό + MCP server WASM bundle OTLP traces + (ADR-0032) (ADR-0003) (ADR-0010) ``` -#### 4. Property-Based Tests -```rust -use proptest::prelude::*; - -proptest! { - #[test] - fn dtw_distance_symmetric(a in any::>(), b in any::>()) { - let d1 = dtw_distance(&a, &b); - let d2 = dtw_distance(&b, &a); - assert!((d1 - d2).abs() < 1e-10); - } -} -``` +See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the long form, and [`docs/adr/`](docs/adr/README.md) for the decisions that shaped each layer. -### Security Testing - -```bash -# Run security audit -npx ts-node scripts/security-check.ts - -# Results: -# βœ… No hardcoded credentials -# βœ… HTTPS/WSS enforcement -# βœ… Input validation present -# βœ… Rate limiting configured -# βœ… Secure error handling -# βœ… No sensitive data logging -# βœ… CORS properly configured -# βœ… Environment variable usage -# βœ… No eval() or unsafe code -# βœ… Dependencies up to date - -# Overall Score: A+ (10/10 checks passed) -``` +
--- -## 🎯 Use Cases - -### Real-Time Customer Support -```typescript -const dashboard = new MidStreamDashboard(); -const agent = dashboard.getAgent(); - -// Analyze conversation patterns -agent.processMessage('I need help with my order'); -const patterns = agent.detectPattern(history, ['greeting', 'problem', 'solution']); -``` - -### Video Stream Analysis -```typescript -const client = new RestreamClient({ - enableObjectDetection: true, - enableTranscription: true -}); - -client.on('objects_detected', (data) => { - console.log(`Detected: ${data.objects.length} objects`); -}); -``` - -### Voice Agent with OpenAI -```typescript -const openai = new OpenAIRealtimeClient({ apiKey }); -const dashboard = new MidStreamDashboard(); - -openai.on('response.audio.delta', (audio) => { - dashboard.processStream('openai', Buffer.from(audio, 'base64'), 'audio'); -}); -``` +## Performance -### Low-Latency Multiplexed Streaming with QUIC -```typescript -const connection = await connectQuic('localhost', 4433); +`cargo bench --workspace` runs the full criterion suite. Honest benchmarks against real workloads, not mocks β€” ADR-0009. -// High-priority video stream -const videoStream = await connection.openBiStream({ priority: 10 }); -videoStream.write(videoFrame); +Highlights from the current baseline (Ubuntu 24.04, Ryzen 9 7950X): -// Medium-priority audio stream -const audioStream = await connection.openBiStream({ priority: 9 }); -audioStream.write(audioChunk); +| Operation | p50 | Notes | +|---|---|---| +| `temporal-compare::dtw` (128 Γ— 128 floats) | ~38 Β΅s | SIMD-friendly; no allocations on the hot path | +| `scheduler::push` + `pop` (priority-major) | ~85 ns / ~120 ns | Lock-free queue, ADR-0008 | +| `attractor::lyapunov` (1 K-point trajectory) | ~2.4 ms | Single-threaded | +| `strange-loop` self-update cycle | ~340 Β΅s | Per inflight observation | +| QUIC `connect` (0-RTT, loopback) | ~180 Β΅s | Platform verifier, ADR-0011 | -// Low-priority telemetry -const telemetryStream = await connection.openUniStream({ priority: 1 }); -telemetryStream.write(stats); - -// Get connection statistics -const stats = connection.getStats(); -console.log(`RTT: ${stats.rtt}ms, Throughput: ${stats.bytesSent} bytes`); -``` +Full numbers (CSV + plots + methodology) in [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md). --- -## πŸ” Security +## Documentation -### Security Features -- βœ… Environment variable management -- βœ… No hardcoded credentials -- βœ… HTTPS/WSS enforcement -- βœ… Input validation -- βœ… Rate limiting -- βœ… Error handling -- βœ… Secure logging -- βœ… CORS configuration +Four canonical docs. Everything else is either an ADR or archived. -### Security Audit Results -``` -Critical: 0 -High: 0 -Medium: 0 -Low: 0 +| Doc | When to read it | +|---|---| +| **[Architecture](docs/ARCHITECTURE.md)** | What midstream is, how the pieces fit together, where each component lives. The *how-it-works* doc. | +| **[Getting Started](docs/GETTING_STARTED.md)** | Zero to a working local build. Prerequisites, install, first run. The *how-do-I-start* doc. | +| **[Benchmarks](docs/BENCHMARKS.md)** | Methodology + current numbers, with the historical drift documented. The *is-it-actually-fast* doc. | +| **[Security](docs/SECURITY.md)** | Threat model, posture, supply-chain gates, how to report issues. The *should-I-trust-it* doc. | -Overall Score: A+ (100%) -Status: Production Ready -``` +Plus the **41 [ADRs](docs/adr/README.md)** β€” every architectural decision, immutable, with status tracked (Proposed β†’ Accepted β†’ Superseded). Browse the [index](docs/adr/README.md) by topic: foundational Β· perf/SOTA Β· security Β· API Β· transport Β· TS surface Β· dashboard Β· governance. --- -## πŸ“Š Performance - -### Benchmarks -``` -Dashboard Refresh: 100ms (configurable) -Message Processing: <10ms average -Stream Processing: <5ms per chunk -Memory Usage: <50MB baseline -CPU Usage: <5% idle, <15% active -Throughput: 1000+ messages/sec -``` - -### Optimization Features -- Configurable buffer sizes -- Automatic memory management -- Event-driven architecture -- Non-blocking I/O -- Connection pooling -- Intelligent caching - ---- - -## πŸ› οΈ Development - -### Project Structure -``` -midstream/ -β”œβ”€β”€ npm/ # Node.js/TypeScript packages -β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”œβ”€β”€ agent.ts # Lean Agentic learning -β”‚ β”‚ β”œβ”€β”€ dashboard.ts # Real-time dashboard -β”‚ β”‚ β”œβ”€β”€ restream-integration.ts # Video streaming -β”‚ β”‚ β”œβ”€β”€ openai-realtime.ts # OpenAI integration -β”‚ β”‚ β”œβ”€β”€ streaming.ts # WebSocket/SSE -β”‚ β”‚ └── mcp-server.ts # MCP protocol -β”‚ β”œβ”€β”€ examples/ # Demo applications -β”‚ β”œβ”€β”€ scripts/ # Utility scripts -β”‚ └── __tests__/ # Test suites -β”œβ”€β”€ src/ # Rust core engine -β”‚ β”œβ”€β”€ lean_agentic/ # Lean agentic system -β”‚ β”œβ”€β”€ bin/ # Binaries -β”‚ └── tests/ # Rust tests -β”œβ”€β”€ wasm-bindings/ # WASM bindings -β”œβ”€β”€ hyprstream-main/ # Streaming engine -└── docs/ # Documentation -``` - -### Building from Source +## Development ```bash -# Build TypeScript -cd npm -npm run build:ts +# Run the workspace tests (skip midstream root + hyprstream; broken under --all-features) +cargo test --workspace --exclude midstream --exclude hyprstream --all-features -# Build Rust (when network available) -cd .. -cargo build --release +# Lint +cargo clippy --workspace --exclude midstream --exclude hyprstream --all-targets --all-features -- -D warnings -# Build WASM -cd wasm-bindings -wasm-pack build --target nodejs -``` +# Format check +cargo fmt --all -- --check ---- +# Supply-chain gate (matches CI) +cargo deny --workspace check --hide-inclusion-graph advisories +cargo deny --workspace check --hide-inclusion-graph bans +cargo deny --workspace check --hide-inclusion-graph licenses +cargo deny --workspace check --hide-inclusion-graph sources +cargo audit -## 🀝 Contributing - -We welcome contributions from the community! MidStream is an open-source project that thrives on collaboration. - -### How to Contribute - -1. **Fork the Repository** - ```bash - gh repo fork ruvnet/midstream - cd midstream - ``` - -2. **Create a Feature Branch** - ```bash - git checkout -b feature/amazing-feature - ``` - -3. **Make Your Changes** - - Write clean, documented code - - Follow existing code style - - Add tests for new features - - Update documentation - -4. **Test Your Changes** - ```bash - # Run all tests - cargo test --workspace - cd npm && npm test - - # Check formatting - cargo fmt --check - npm run lint - - # Run security audit - cargo audit - npx ts-node scripts/security-check.ts - ``` - -5. **Commit Your Changes** - ```bash - git add . - git commit -m "Add amazing feature" - ``` - -6. **Push and Create PR** - ```bash - git push origin feature/amazing-feature - gh pr create --title "Add amazing feature" --body "Description of changes" - ``` - -### Contribution Guidelines - -**Code Style:** -- Rust: Follow `rustfmt` defaults -- TypeScript: ESLint + Prettier configuration -- Maximum line length: 100 characters -- Use meaningful variable names -- Add inline comments for complex logic - -**Testing:** -- Write tests for all new features -- Maintain >85% test coverage -- Include both unit and integration tests -- Add benchmarks for performance-critical code - -**Documentation:** -- Update README if adding major features -- Add doc comments to public APIs -- Include usage examples -- Update CHANGELOG.md - -**Commit Messages:** -``` -(): - - +# Property tests +cargo test --workspace --exclude midstream --exclude hyprstream --test 'proptest_*' -