Description
Databricks introduces DBRX, a new, state-of-the-art open large language model (LLM) designed to redefine efficiency and quality in AI. DBRX sets new benchmarks for open LLMs, offering capabilities previously exclusive to closed model APIs. It surpasses GPT-3.5 and is competitive with Gemini 1.0 Pro, demonstrating particular strength in code generation, outperforming specialized models like CodeLLaMA-70B.
Read More
DBRX achieves its advanced performance through a fine-grained mixture-of-experts (MoE) architecture, enabling up to 2x faster inference than LLaMA2-70B and a significantly smaller active parameter count compared to models like Grok-1. When hosted on Databricks Model Serving, DBRX can achieve up to 150 tokens per second per user. Training MoE models with DBRX's approach is also approximately 2x more FLOP-efficient than training dense models for equivalent quality.
The model was pre-trained on 12 trillion tokens of text and code data, utilizing a carefully curated dataset developed with Databricks tools like Apache Spark and Unity Catalog. DBRX employs rotary position encodings (RoPE), gated linear units (GLU), and grouped query attention (GQA), with a maximum context length of 32,000 tokens. Both the base model (DBRX Base) and the instruction-tuned model (DBRX Instruct) are available on Hugging Face under an open license.
DBRX Instruct leads in various benchmarks, including the Hugging Face Open LLM Leaderboard and the Databricks Model Gauntlet. It shows exceptional performance in programming and mathematics, outscoring models like Grok-1 and CodeLLaMA-70B on HumanEval and GSM8k. DBRX Instruct also demonstrates strong performance on long-context tasks and retrieval augmented generation (RAG), outperforming GPT-3.5 Turbo in many scenarios.
Databricks offers DBRX through its Foundation Model APIs with pay-as-you-go pricing and provisioned throughput options for production applications. Customers can also privately host DBRX by downloading it from the Databricks Marketplace and deploying it on Model Serving. This release empowers enterprises to control their data and destiny in the GenAI landscape, providing access to powerful and efficient LLM tools.
- CategoryAI Models & LLMs
- Pricing ModelUsage-based
- DeploymentCloud
- PlatformsWeb
DBRX LLM Highlights
State-of-the-art open LLM
Mixture-of-experts (MoE) architecture
132B total parameters, 36B active parameters
Trained on 12T tokens of text and code
32K token context window
Surpasses GPT-3.5 on benchmarks
Competitive with Gemini 1.0 Pro
Strong performance in programming and math
Up to 2x faster inference than LLaMA2-70B
Available on Hugging Face
Accessible via Databricks Foundation Model APIs
Supports retrieval augmented generation (RAG)
Getting Started with DBRX LLM
Access Model: Utilize Databricks Foundation Model APIs or download from Hugging Face.
Authenticate: Set up necessary API keys or access credentials.
Set Up Environment: Configure your development environment with required libraries.
Integrate via API: Implement API calls for DBRX in your applications.
Optimize: Fine-tune model parameters or leverage Databricks Model Serving for performance.
DBRX LLM's Use Cases
- Code Generation
- Content Creation
- Data Analysis
- Question Answering
- Summarization
- Chatbots
- Research Assistance
