Cleora Graph Embedding Engine

Cleora is a Rust-powered graph embedding engine that deterministically computes graph embeddings using sparse Markov propagation. It offers a CPU-only, parameter-free, and fast alternative to GPU-intensive methods, achieving high accuracy on real-world graphs without sampling or noise, making it ideal for production ML pipelines.

访问URL

Cleora Graph Embedding Engine screenshot

描述

Cleora is a groundbreaking Rust-powered graph embedding engine designed to provide deterministic and highly accurate graph embeddings. Unlike traditional methods that rely on random walks, negative sampling, and GPU clusters, Cleora computes the exact distribution of all possible walks in a single sparse matrix power. This approach eliminates noise and stochastic approximation, ensuring perfect reproducibility and significantly faster computation on a single CPU core.

The core of Cleora's innovation lies in its algorithm. It constructs a sparse transition matrix from the input graph, handling heterogeneous hypergraphs with typed, multi-relational edges natively. Each iteration of matrix multiplication with this transition matrix captures the full distribution of walks of a specific length. This is followed by L2-normalized propagation, where each node's embedding is updated to the normalized average of its neighbors' embeddings. This process requires only a few iterations for co-occurrence similarity and slightly more for contextual similarity, making it orders of magnitude faster than sampling-based methods.

Key advantages of Cleora include its speed, accuracy, and efficiency. It is up to 240 times faster than GraphSAGE and uses significantly less memory than other libraries like NetMF. The engine is deterministic by default, crucial for reproducible research and production ML pipelines. It natively supports heterogeneous hypergraphs, eliminating the need for complex graph preprocessing. Furthermore, Cleora boasts a minimal install size of around 5 MB with only numpy and scipy as dependencies, avoiding the headaches of GPU driver installations and heavy frameworks.

Cleora is production-ready, offering stable and inductive embeddings that support the embedding of new nodes without retraining the entire graph. Its capabilities extend to various applications, including recommendation systems, knowledge graphs, entity resolution, fraud detection, and drug discovery. The engine's efficiency is highlighted by case studies like Zomato, which replaced GraphSAGE with Cleora, reducing embedding generation time from 20 hours to under 5 minutes for millions of users.

Cleora provides a comprehensive suite of tools, including built-in alternative algorithms for comparison, a rich evaluation suite for assessing embedding quality, graph sampling capabilities, and hyperparameter tuning. It also includes a CLI tool for easy integration into scripting and CI/CD pipelines. The entire library is open-source, free to use, modify, and deploy, offering a cost-effective solution compared to GPU-based alternatives.