Description
Cleora is a groundbreaking Rust-powered graph embedding engine designed to provide deterministic and highly accurate graph embeddings. Unlike traditional methods that rely on random walks, negative sampling, and GPU clusters, Cleora computes the exact distribution of all possible walks in a single sparse matrix power. This approach eliminates noise and stochastic approximation, ensuring perfect reproducibility and significantly faster computation on a single CPU core.
The core of Cleora's innovation lies in its algorithm. It constructs a sparse transition matrix from the input graph, handling heterogeneous hypergraphs with typed, multi-relational edges natively. Each iteration of matrix multiplication with this transition matrix captures the full distribution of walks of a specific length. This is followed by L2-normalized propagation, where each node's embedding is updated to the normalized average of its neighbors' embeddings. This process requires only a few iterations for co-occurrence similarity and slightly more for contextual similarity, making it orders of magnitude faster than sampling-based methods.
Key advantages of Cleora include its speed, accuracy, and efficiency. It is up to 240 times faster than GraphSAGE and uses significantly less memory than other libraries like NetMF. The engine is deterministic by default, crucial for reproducible research and production ML pipelines. It natively supports heterogeneous hypergraphs, eliminating the need for complex graph preprocessing. Furthermore, Cleora boasts a minimal install size of around 5 MB with only numpy and scipy as dependencies, avoiding the headaches of GPU driver installations and heavy frameworks.
Cleora is production-ready, offering stable and inductive embeddings that support the embedding of new nodes without retraining the entire graph. Its capabilities extend to various applications, including recommendation systems, knowledge graphs, entity resolution, fraud detection, and drug discovery. The engine's efficiency is highlighted by case studies like Zomato, which replaced GraphSAGE with Cleora, reducing embedding generation time from 20 hours to under 5 minutes for millions of users.
Cleora provides a comprehensive suite of tools, including built-in alternative algorithms for comparison, a rich evaluation suite for assessing embedding quality, graph sampling capabilities, and hyperparameter tuning. It also includes a CLI tool for easy integration into scripting and CI/CD pipelines. The entire library is open-source, free to use, modify, and deploy, offering a cost-effective solution compared to GPU-based alternatives.
Cleora Graph Embedding Engine's Core Features
Deterministic graph embedding computation via sparse matrix powers
CPU-only execution, no GPU required
Handles heterogeneous hypergraphs with typed, multi-relational edges natively
Eliminates random walk sampling and skip-gram training
Achieves high accuracy on real-world graphs
Significantly faster than GPU-based and sampling-based methods
Minimal dependencies and small install size (~5 MB)
Supports inductive learning for new nodes without retraining
Includes built-in alternative algorithms for comparison
Provides a rich evaluation suite for embedding quality
Offers a CLI tool for scripting and CI/CD integration
Open-source and free to use, modify, and deploy
How to use Cleora Graph Embedding Engine?
Install: pip install pycleora
Input Data: Feed edge lists, interaction logs, or knowledge triples in TSV format.
Graph Construction: Cleora builds a heterogeneous hypergraph from the input data.
Matrix Power: Computes sparse transition matrix and its powers to capture walk distributions.
Embedding Generation: L2-normalized propagation generates deterministic embedding vectors.
Downstream ML: Utilize generated embeddings for recommendations, classification, search, etc.
Cleora Graph Embedding Engine's Use Cases
- Recommendation Systems
- Knowledge Graphs
- Entity Resolution
- Fraud Detection
- Social Network Analysis
- Drug Discovery
- Supply Chain Optimization
- Customer Segmentation
