Feature/chat mode phase2 - #1104
Closed
ShuhaoZhangTony wants to merge 141 commits into
Closed
ShuhaoZhangTony wants to merge 141 commits into
ShuhaoZhangTony wants to merge 141 commits into
Conversation
- Add pull_request trigger for PR merge to main/main-dev - Update job conditions to handle PR merge events - Fix branch detection for PR merge scenarios - Update version bump logic to use 'micro' for main branch (not 'minor') - This ensures version bump is triggered when PR is merged Resolves issue where PR #1054 merge didn't auto-trigger version bump. With this fix, future PR merges will automatically bump version and publish.
- Move 'import re' to file top imports - Remove duplicate 'import re' statements inside functions - Fixes UnboundLocalError: cannot access local variable 're' This error occurred when sage-dev package version set was called in GitHub Actions workflow during version bump.
- Add skip conditions to prevent unnecessary CI runs on version bump commits - Modified workflows: - build-test.yml: Skip build/test for version-only changes - code-quality.yml: Skip quality checks for version-only changes - examples-test.yml: Skip example tests for version-only changes - deployment-check.yml: Skip deployment checks for version bumps - pip-installation-test.yml: Skip installation tests for version bumps Version bump commits only change _version.py files and don't require: - Full build and test suite - Code quality checks - Example tests - Deployment validation This prevents the 'X' (skipped/failed) status on main branch after version bumps. Only publish-pypi.yml needs to skip version bump commits to avoid infinite loops.
✨ Major Changes:
1. 📁 Data Reorganization (按数据特性分类)
- Created shared data/ directory at sage-benchmark/src/sage/data/
- Organized by data characteristics instead of benchmark names:
* data/qa/ - QA knowledge bases and queries (shared across RAG benchmarks)
* data/locomo/ - LoCoMo datasets (for memory benchmarks)
- Moved data_loaders utilities to benchmark_memory/data_loaders/
- Benefits: Better data sharing, avoid duplication, clearer naming
2. 🔧 Path Updates
- Updated 11 YAML config files: data_path from benchmark_rag/data → data/qa
- Updated Python imports in benchmark_memory experiments
- Updated test_config_loading.py with new data paths
3. 🎯 Type Safety Improvements
- build_chroma_index.py: Complete type overhaul
* Added Embedder Protocol for embedding model interface
* Added Document/ChunkDocument TypedDict for structured types
* Added @overload decorators for _to_2dlist type inference
* Zero Pylance errors, no type: ignore used
- document_loaders.py: Added complete type annotations
* All loader classes return dict[str, Any]
* LoaderFactory properly typed with precise Union types
* Type information propagates correctly to consumers
4. 📚 Documentation
- Created data/README.md with design philosophy and usage
- Created data/__init__.py with directory overview
- Updated benchmark_rag/__init__.py documentation
Files Changed: 33 files
- Renamed/Moved: 12 files (data reorganization)
- Modified: 19 files (paths, types, imports)
- Added: 2 files (documentation)
✨ Changes: 1. 📦 Moved Data Scripts to data/locomo/ - Moved locomo_download.py → data/locomo/download.py - Moved locomo_dataloader.py → data/locomo/dataloader.py - Removed benchmark_memory/data_loaders/ directory - Better centralized data management 2. 📚 Added Documentation - Created data/locomo/README.md with download instructions - Created data/locomo/__init__.py for package structure - Updated data/README.md with download command info - Updated data/__init__.py with usage instructions 3. 🔧 Updated Imports - Updated 4 files in benchmark_memory/experiment/locomo/ - Changed from: sage.benchmark.benchmark_memory.data_loaders.locomo_dataloader - Changed to: sage.data.locomo.dataloader Benefits: ✅ Data scripts live with data, not with benchmarks ✅ Easier to discover and use: `python -m sage.data.locomo.download` ✅ Clear separation: benchmarks consume data, don't manage it ✅ Better scalability for future datasets
- Update test to reference correct data location: qa/ instead of benchmark_rag/ - Aligns with project's data-centric organization structure - All 5 tests now passing
- Move benchmark data to separate repository: https://github.com/intellistream/sageData - Remove data directory from SAGE repository - Add sageData as Git submodule at packages/sage-benchmark/src/sage/data - All tests still passing with submodule setup - Benefits: Better data management, smaller main repo, easier data updates Data structure maintained: - qa/: Question-Answering datasets (queries.jsonl, knowledge bases) - locomo/: Long-Context Memory datasets and loaders
- Create and push main-dev branch in sageData repository - Update .gitmodules to use main-dev branch for sageData - sageData now follows same branching strategy as other submodules - All 7 submodules now properly managed with main/main-dev branches - Enables development and testing of new datasets before merging to main
问题分析： - 原实现在每次put/get操作时使用ray.get()同步等待 - 导致网络延迟累积，10-100x性能下降 - 批量操作中单次延迟被放大N倍 核心优化： 1. 批量Put操作 - 缓冲100个items后批量发送 - 异步提交，避免同步阻塞 - 智能刷新机制（自动/手动） 2. 批量Get操作 - 新增get_batch()方法 - 一次网络调用获取多个items - 减少网络往返次数 3. 性能统计 - 跟踪批量操作次数 - 监控平均批量大小 - 便于性能调优 性能测试结果（本地）： - Put操作提升: 477.7 → 841.1 items/sec (1.8x) - Get操作提升: 350.2 → 927.6 items/sec (2.65x) - 预期分布式环境提升: 10-50x 代码质量改进： - 添加完整类型注解（typing.Any, Optional, list, dict等） - Pylance错误从87个降至2个（Ray内部API的不可避免错误） - 向后兼容，无破坏性变更 相关文件： - ray_queue_descriptor.py: 核心优化实现 - test_ray_queue_optimization.py: 性能测试套件 - PERFORMANCE_OPTIMIZATION_RAY_QUEUE.md: 用户指南 - OPTIMIZATION_REPORT.md: 技术报告 - TEST_RESULTS.md: 测试结果文档 Closes #1074
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ue_descriptor/ray_queue_descriptor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- 新增 docs/CHECKPOINT_SYSTEM.md：详细说明检查点系统的功能、用法和 API - 新增 docs/DEPENDENCY_VERIFICATION.md：详细说明依赖验证系统的安全机制 - 修复 logging.sh：添加缺失的 log_success() 函数 - 完善文档：包含使用示例、API 参考、故障排查等内容 修复了本地 CI/CD 测试中发现的两个问题： 1. 缺失的核心文档（CHECKPOINT_SYSTEM.md 和 DEPENDENCY_VERIFICATION.md） 2. 日志系统缺失 log_success 函数 测试验证： - ✅ 所有日志函数正常工作 - ✅ 文档完整性达到 100% (5/5) - ✅ 测试通过率提升至 100% (13/13)
Update sageFlow to commit f67fb93 which fixes the stream_runtime library linkage issue. The previous commit introduced PRIVATE dependencies that were not properly propagated to the sageflow shared library, causing undefined symbol errors during linking.
…builds Update all three middleware submodules (sage_db, sage_tsdb, sage_flow) to use Python3 Development.Module instead of Development component. This fixes the cibuildwheel build failure in manylinux environments. - sage_db: fac84c1 - sage_tsdb: a37763c - sage_flow: f358d18 The Development component requires Python3_LIBRARIES which is not available in wheel build environments, while Development.Module only needs headers.
Update sageFlow to commit df0ac7d which converts stream_runtime to an OBJECT library to properly export StreamEnvironment symbols in the sageflow shared library. This fixes undefined reference errors during test linking.
- Update docs-public submodule reference to 75d541a - Previous commit 1e39ffd was no longer available in remote - Fixes submodule fetch errors in CI/CD pipeline
Problem: - CI test workflow was failing with 'unknown parameter: --minimal' error - The --minimal flag was renamed to --core in quickstart.sh refactoring - Documentation still referenced the old --minimal flag Solution: - Add --minimal as a legacy alias for --core in argument_parser.sh (maintains backward compatibility for existing scripts/docs) - Update CI workflow to use --core instead of --minimal - Update all documentation to use --core as the primary flag - Update help messages and examples across all docs Changes: - tools/install/download_tools/argument_parser.sh: Add --minimal alias - .github/workflows/test-env-cleanup.yml: Use --core in tests - README.md, CONTRIBUTING.md: Update examples to use --core - docs/: Update all --minimal references to --core - docs-public/: Update installation guides and examples - tools/install/: Update help text and error messages Both --minimal and --core now work correctly, with --core being the recommended flag going forward. Fixes: Test Environment Isolation and Cleanup workflow failure
… npm CLI - Added sage-gateway package with OpenAI-compatible API, session manager, and JSON storage - Implemented ChatMode component with session list, streaming messages, and pipeline conversion - Added chatStore for state management with Zustand - Enhanced studio backend API with chat endpoints and pipeline recommender service - Integrated sage studio npm command for unified frontend dependency management - Updated documentation and verification scripts - All tests passing: TypeScript compiles cleanly, unit tests green, verify_chat_ui passes Co-authored-by: GitHub Copilot <noreply@github.com>
- Implement NeuroMemSessionStorage using SAGE's native memory system - Use TextStorage for session data and MetadataStorage for stats - Add environment-based backend selection (file/neuromem) - Full test coverage (11 tests) for NeuroMem storage - Update documentation with NeuroMem configuration - Add seed parameter to OpenAI client for reproducibility - Fix test fixtures to properly mock environment variables
- Add 7 end-to-end integration tests covering full chat workflows - Test dev mode, multi-turn conversations, streaming responses - Test session persistence across adapter instances - Test error handling and session stats - Test NeuroMem backend configuration - Clean up outdated TODO comments - All 37 gateway tests passing
- Add comprehensive CHANGELOG.md for gateway - Update TEST_CHAT_UI.md with phase completion status - Document all features, tests, and configuration - Mark Phases 1-5 as complete (37 tests passing)
- Install react-markdown, remark-gfm, react-syntax-highlighter - Create MessageContent component with rich formatting - Support code blocks with syntax highlighting (VS Code Dark+ theme) - Add copy button for code blocks - Style tables, lists, blockquotes, headings - Add @tailwindcss/typography plugin for prose styling - Maintain simple rendering for user messages - Enhanced AI responses with full Markdown support
- Document Markdown rendering implementation - Document syntax highlighting (VS Code Dark+ theme) - Document rich text formatting features - Update TEST_CHAT_UI.md with completed Phase 6 items - Mark code highlighting and Markdown as complete - Note remaining items: file upload, advanced context
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR introduces benchmark scripts and configuration files for the AMME2E (Approximate Matrix Multiplication Evaluation - End-to-End) testing framework, implementing phase 2 of chat mode functionality. The changes add comprehensive benchmarking infrastructure for testing various matrix multiplication algorithms across different streaming modes (eager/lazy) and dataset configurations.
Key Changes:
- Added Python scripts for running AMME2E benchmarks across multiple execution modes (static eager/lazy, 1stream1static, 2stream)
- Implemented visualization utilities for generating performance graphs and charts
- Added configuration templates for different benchmark scenarios
Reviewed Changes
Copilot reviewed 123 out of 1121 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| AMME2E_run.bash | Orchestrates benchmark execution across different test scenarios |
| drawSVI.py (multiple locations) | Generates SVI (latency/error) comparison visualizations |
| drawTogether.py (multiple locations) | Creates combined performance metric charts |
| config_e2e_*.csv (multiple locations) | Configuration templates for various benchmark modes |
| groupLine.py, groupBar2.py, accuBar.py (multiple locations) | Matplotlib-based plotting utilities |
| OoOCommon.py (multiple locations) | Shared configuration and plotting helper functions |
| autoParase.py (multiple locations) | CSV parsing utilities for benchmark results |
| result CSV files | Default/empty result templates |
| pythonTest.py | Basic PyTorch integration test for LibAMM library |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.