Description
ARBR is a self-hosted AI gateway and control plane designed to streamline the management of AI requests across various model providers. It acts as a central hub for routing, observability, budget management, and governance, ensuring that each AI request is handled efficiently and cost-effectively. By providing a single OpenAI-compatible endpoint, ARBR allows applications to seamlessly integrate with over 100 providers and 3,000 models, offering a comprehensive view of AI usage and performance.
Read More
The platform enables users to route tasks to the most suitable model, monitor token usage, and enforce budget limits before execution. It also provides detailed analytics and reporting, allowing users to measure spend, performance, and reliability by provider, model, application, workflow, team, and user. ARBR's smart routing capabilities ensure that requests are automatically or manually directed to the appropriate model, while caching and token limits help prevent unnecessary costs.
ARBR's governance features include hard limits and organization-wide switches to prevent overspending, as well as identity and access controls to manage user permissions. The platform also supports canary and shadow deployments for new models, enabling safe rollouts with instant rollback and regression gates.
With its open-source nature and compatibility with existing infrastructure, ARBR is a versatile solution for teams running AI in production. It integrates with popular monitoring tools like OpenTelemetry, Datadog, Grafana, and Prometheus, providing a comprehensive view of AI operations. ARBR is designed to be reliable and easy to deploy, with options for Docker and local setups.
- CategoryMLOps & Model Deployment
- Pricing ModelOpen Source
- DeploymentDocker, Local
- PlatformsOpenAI-compatible
ARBR's Core Features
OpenAI-compatible endpoint
Routing and observability
Budget management and governance
Smart model selection
Detailed analytics and reporting
Identity and access controls
Canary and shadow deployments
Open-source under MIT license
How to use ARBR?
Deploy: Set up ARBR in your environment
Configure: Point applications to the ARBR gateway
Monitor: Use dashboards for insights
Optimize: Adjust routing and budgets
ARBR's Use Cases
- Cost Management
- Model Governance
- Performance Monitoring
- Smart Routing
- Safe Model Deployment
