[Tracker] Graphite 1.0

[mkh-user]

Topic

This issue tracks progress in the Graphite 1.0 roadmap, including tasks, priorities, and decisions.

Context

Graphite needs some structural code and non-code redesigns and refactors. This need includes almost everywhere in the project, same as documentation, API, maintenance flow, etc.

Task List

Project Direction & Scope 🎯

• Fully and precisely define the project goal
• Limit scope to the specified goal
• Define what Graphite is not
• Finalize the current roadmap

General 🧩

• Improve error handling and error messages
• Review and refactor error types (Errors refactor)
• Use caching where possible (beyond current node type field cache)

Schema & Data Manipulation (DSL) 📐

• Improve block-detection logic
• Support in-block comments
• Support inline comments
• Replace load_dsl() with parse()
• Support removing definitions (batched removes)
• Validate node types in relation creation
• Support single quotes in DSL
• Validate value types in all cases
• Add any field type
• Add time field type
• Complete data types support
• Make skipping DSL easier
• Add SQL-like features to fields (default, validation, etc.)
• Evaluate the complete robustness of Graphite's DSL

Storage & Persistence 💾

• Replace Pickle with JSON (security improvement)
• Migration utilities
• Schema validation
• Stabilize save file order
• Spec version check
• Redesign save/load API
• Replace JSON with a secure binary format (performance improvement)
• Ensure backward compatibility for legacy JSON files
• Remove deprecated Pickle migration utilities
• Remove unsafe load support
• Add optional compression to reduce storage file size

Query Engine ⚡

• Add aggregation queries
• query.all()
• Untyped traverse
• Distinct results always
• Atomicity
• Redesign query module
• Improve IDE support
• Add basic indexes on frequently used property fields (user-defined)
• Redesign query engine to optimize execution order (replace step-by-step execution)
• (Idea) Full-featured Query Optimizer (like large databases)
• Document query performance with timings for various scenarios (up to 500K nodes)

Algorithms 📊

• Create algorithms module
• BFS
• Distance-based shortest path
• DFS
• Weight-based shortest path with custom weight calculation function

Design & Graph Features 🕸️

• Real support for undirected graphs (via both flag, with proper indexing and calculations)
• Remove and redesign named relationship reverse support (reverse capability)
• Allow users to define custom weight calculation functions (instead of static field-based weights)
• (Idea) Temporal Graphs support (based on user demand)

Performance Benchmarks 📈

• Scalable to 1M nodes, 1M relations with <2.5 GB (actual: 1.4 GB) — link
• Update scale notices and benchmarks archive

Maintenance & Tooling 🛠️

• Pytest, Pylint, Type check CI
• 100% Test coverage
• Complete types
• Multiple files splitting
• Temporarily lock coverage at 100%
• Migrate to uv
• Migrate to ruff
• Add pre-commit
• Improve library modularity
• Refactor automated checks and rules
• Refactor issue labels and templates
• Refactor projects and milestones
• Review developer docs and guides
• Decide to soften test coverage criteria
• Review tests and remove duplicates
• Documentation for tests

Documentation 📚

• Basic proof
• Update documentation template
• Automate API reference generation
• Basic proof rewrite
• Refactor docstrings for automated API reference
• Stable documentation (complete proof) with at least 3 complete examples from knowledge graph domain
• Improve existing examples
• Update guide content
• Complete missing guides
• Rewrite contribution documents
• Remove update announcements
• Create sample repository with simple knowledge graph (e.g., movies, scientific papers, or real-world data)
• Add computer network modeling usage example
• Finalize feature list for version 1.0

Tooling & Integration 🔧

• (Idea) CLI for statistical reporting (with potential for more features)
• Integration with relational databases for data ingestion
• (Idea) Sample plugin for Jupyter Notebook visualization

API Extensibility & Interoperability 🌐

• (Idea) Lightweight HTTP layer for external interaction
  • Server Mode for concurrent requests
    • Transaction support
  • Document with curl examples
• Move HTTP subsections to a separate issue

This list will be updated.