Description
SmolLM3 is a 3 billion parameter language model developed to advance the capabilities of small models in artificial intelligence. It is designed to support dual mode reasoning and is multilingual, natively supporting six languages: English, French, Spanish, German, Italian, and Portuguese. The model is fully open, with open weights and comprehensive training details, including public data mixtures and training configurations.
Read More
The architecture of SmolLM3 is a decoder-only transformer using GQA and NoPE with a 3:1 ratio. It was pretrained on 11.2 trillion tokens using a staged curriculum that included web, code, math, and reasoning data. Post-training involved midtraining on 140 billion reasoning tokens, followed by supervised fine-tuning and alignment through Anchored Preference Optimization (APO).
SmolLM3 is optimized for hybrid reasoning and supports long context processing, trained on a 64k context and capable of handling up to 128k tokens using YARN extrapolation. The model can be deployed using vLLM and SGLang, compatible with OpenAI format APIs. It also supports tool calling, allowing integration with various tools through XML or Python function calls.
For local inference, SmolLM3 can be used with llama.cpp, ONNX, MLX, MLC, and ExecuTorch. Quantized checkpoints are available for efficient deployment. The model's performance has been evaluated across various benchmarks, demonstrating strong results in multilingual Q&A, competitive programming, and instruction following tasks.
SmolLM3 is a valuable tool for developers and researchers looking to leverage AI for multilingual and complex reasoning tasks. However, users should be aware that the generated content may not always be factually accurate or free from biases present in the training data.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud
- PlatformsWeb
SmolLM3 Language Model Highlights
3B parameter language model
Supports dual mode reasoning
Multilingual: 6 languages
Long context processing up to 128k tokens
Open model with open weights
Instruct model optimized for hybrid reasoning
Tool calling support
Compatible with vLLM and SGLang
Getting Started with SmolLM3 Language Model
Access page: Visit the Hugging Face model page
Load model: Use transformers v4.53.0 or latest vllm
Configure environment: Set sampling parameters and context length
Integrate: Use tool calling with XML or Python functions
Fine-tune: Apply supervised fine-tuning and alignment
SmolLM3 Language Model's Use Cases
- Multilingual Q&A
- Hybrid Reasoning
- Tool Integration
- Long Context Processing
- Instruction Following
