Description
lakeFS is an open-source data version control system designed to transform your object storage into Git-like repositories. This allows users to manage their data lakes with the same precision and control as they manage code, using branches, commits, and merges. The platform is particularly beneficial for AI and data engineering teams, providing a control plane that ensures data governance, reproducibility, and reduced access friction.
Read More
With lakeFS, users can perform Git-like operations on data, such as branching and merging, which facilitates the creation of reproducible data pipelines and the enforcement of data quality gates. This system is built on a highly scalable architecture that manages the data lifecycle, provenance, and unified access, making it ideal for AI and data engineering teams.
lakeFS supports a wide range of integrations with popular data engineering tools and technologies, including Spark, Delta Lake, AWS CLI, Airflow, and more. It is compatible with all major cloud providers like AWS, Azure, and GCP, ensuring that data and metadata remain securely stored within the user's VPC.
The platform offers a free open-source version, as well as a managed cloud service. It is designed to simplify regulatory audits by providing built-in evidence and preventive data controls, reducing compliance risks. lakeFS also enhances productivity by allowing teams to test pipeline and model changes in isolation without data duplication, and to instantly roll back from data incidents.
Overall, lakeFS empowers organizations to unlock data management at scale, accelerating AI and ML initiatives by applying proven engineering best practices to data management.
- CategoryMLOps & Model Deployment
- Pricing ModelFreemium
- DeploymentCloud
- PlatformsAWS, Azure, GCP
lakeFS's Core Features
Open-source data version control
Git-like operations for data
Branch and merge data
Reproducible data pipelines
Data quality gates
Integrates with popular data tools
Supports AWS, Azure, GCP
Zero-copy branching
Role-based access control
Audit logs
How to use lakeFS?
Configure: Set up lakeFS with your object storage
Use: Perform Git-like operations on your data
Integrate: Connect with your existing data tools
Optimize: Enhance data governance and reproducibility
lakeFS's Use Cases
- Data Governance
- AI Training
- Data Versioning
- Pipeline Testing
- Audit Readiness
