DBRX LLM

Description

Databricks introduces DBRX, a new, state-of-the-art open large language model (LLM) designed to redefine efficiency and quality in AI. DBRX sets new benchmarks for open LLMs, offering capabilities previously exclusive to closed model APIs. It surpasses GPT-3.5 and is competitive with Gemini 1.0 Pro, demonstrating particular strength in code generation, outperforming specialized models like CodeLLaMA-70B.

DBRX achieves its advanced performance through a fine-grained mixture-of-experts (MoE) architecture, enabling up to 2x faster inference than LLaMA2-70B and a significantly smaller active parameter count compared to models like Grok-1. When hosted on Databricks Model Serving, DBRX can achieve up to 150 tokens per second per user. Training MoE models with DBRX's approach is also approximately 2x more FLOP-efficient than training dense models for equivalent quality.

The model was pre-trained on 12 trillion tokens of text and code data, utilizing a carefully curated dataset developed with Databricks tools like Apache Spark and Unity Catalog. DBRX employs rotary position encodings (RoPE), gated linear units (GLU), and grouped query attention (GQA), with a maximum context length of 32,000 tokens. Both the base model (DBRX Base) and the instruction-tuned model (DBRX Instruct) are available on Hugging Face under an open license.

DBRX Instruct leads in various benchmarks, including the Hugging Face Open LLM Leaderboard and the Databricks Model Gauntlet. It shows exceptional performance in programming and mathematics, outscoring models like Grok-1 and CodeLLaMA-70B on HumanEval and GSM8k. DBRX Instruct also demonstrates strong performance on long-context tasks and retrieval augmented generation (RAG), outperforming GPT-3.5 Turbo in many scenarios.

Databricks offers DBRX through its Foundation Model APIs with pay-as-you-go pricing and provisioned throughput options for production applications. Customers can also privately host DBRX by downloading it from the Databricks Marketplace and deploying it on Model Serving. This release empowers enterprises to control their data and destiny in the GenAI landscape, providing access to powerful and efficient LLM tools.

DBRX LLM Highlights

• State-of-the-art open LLM

• Mixture-of-experts (MoE) architecture

• 132B total parameters, 36B active parameters

• Trained on 12T tokens of text and code

• 32K token context window

• Surpasses GPT-3.5 on benchmarks

• Competitive with Gemini 1.0 Pro

• Strong performance in programming and math

• Up to 2x faster inference than LLaMA2-70B

• Available on Hugging Face

• Accessible via Databricks Foundation Model APIs

• Supports retrieval augmented generation (RAG)

Getting Started with DBRX LLM

1. 1

   Access Model: Utilize Databricks Foundation Model APIs or download from Hugging Face.

2. 2

   Authenticate: Set up necessary API keys or access credentials.

3. 3

   Set Up Environment: Configure your development environment with required libraries.

4. 4

   Integrate via API: Implement API calls for DBRX in your applications.

5. 5

   Optimize: Fine-tune model parameters or leverage Databricks Model Serving for performance.

DBRX LLM's Use Cases

• Code Generation
• Content Creation
• Data Analysis
• Question Answering
• Summarization
• Chatbots
• Research Assistance

FAQ from DBRX LLM

• What is DBRX?

  DBRX is an open, general-purpose large language model (LLM) developed by Databricks, setting a new state-of-the-art for open LLMs in various benchmarks.

• How does DBRX compare to other LLMs?

  DBRX surpasses GPT-3.5 and is competitive with Gemini 1.0 Pro. It also excels in programming tasks, outperforming specialized models like CodeLLaMA-70B.

• What makes DBRX efficient?

  DBRX utilizes a fine-grained mixture-of-experts (MoE) architecture, which improves training and inference performance, making it up to 2x faster than LLaMA2-70B.

• What kind of data was DBRX trained on?

  DBRX was pre-trained on 12 trillion tokens of carefully curated text and code data.

• What is the context window size of DBRX?

  DBRX has a maximum context length of 32,000 tokens.

• Is DBRX available for commercial use?

  Yes, DBRX is available for Databricks customers via APIs and can be privately hosted. The base and instruct models are also available on Hugging Face under an open license.

• What are the key strengths of DBRX?

  DBRX's key strengths include its state-of-the-art performance on language understanding, programming, and mathematics benchmarks, alongside its efficiency in training and inference.

• Can DBRX be used for code generation?

  Yes, DBRX is an especially capable code model, surpassing specialized models like CodeLLaMA-70B on programming tasks.

• Where can I access DBRX?

  DBRX is available through Databricks Foundation Model APIs, and its base and instruct models can be downloaded from Hugging Face.

• What is the parameter count of DBRX?

  DBRX has 132 billion total parameters, with 36 billion active parameters per input.