Description
MiniCPM5-2B is the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer model built for on-device, local deployment, and resource-constrained scenarios. This model scales up the same training recipe and achieves state-of-the-art (SOTA) performance within the 2B-class open-source models. It remains competitive with larger 4B-class models, particularly excelling in areas such as coding, mathematics, long-context understanding, tool use, and agentic tasks.
Read More
The model is trained using high-quality datasets from the UltraData family, including UltraX for web pre-training, UltraData-Code for coding capabilities, UltraData-SFT-Agent-2609 for agent training, and UltraData-RL-2609 for reinforcement learning. These datasets contribute to the model's comprehensive capabilities, making it suitable for various applications such as local assistants, coding agents, and reasoning scenarios.
MiniCPM5-2B employs a standard LlamaForCausalLM architecture with 2,516,756,480 parameters, 42 layers, and a context length of 131,072. It is designed to provide native long-context support while maintaining a small deployment footprint. The model's training involves a full-stack practice of UltraData Tiered Data Management, covering base training, mid-training, and post-training stages.
During post-training, the model undergoes Supervised Fine-Tuning (SFT), Reinforcement Learning (RL), and On-Policy Distillation (OPD) to enhance its capabilities further. RL + OPD significantly improves the model's reasoning and general capabilities, merging the expertise of 16 specialized models into one release model.
MiniCPM5-2B is available in various formats suitable for different deployment scenarios, including BF16, GGUF, and MLX. It supports multiple backends such as Transformers, vLLM, SGLang, and llama.cpp, allowing for flexible deployment across different platforms and devices. The model is released under the Apache-2.0 License, ensuring open access and collaboration.
- CategoryAI Models & LLMs
MiniCPM5-2B Highlights
Dense 2B Transformer model
On-device deployment
Resource-constrained scenarios
State-of-the-art performance
High-quality training datasets
Standard LlamaForCausalLM architecture
Native long-context support
UltraData Tiered Data Management
Getting Started with MiniCPM5-2B
Access model page: Visit the Hugging Face model page
Authenticate: Log in to your Hugging Face account
Configure: Set up the model parameters and environment
Send prompts: Use the model for your specific tasks
Fine-tune: Adjust the model for specialized applications
MiniCPM5-2B's Use Cases
- Local Assistants
- Coding Agents
- Tool Use Workflows
- Reasoning Scenarios
- Agentic Tasks
