Description
Inkling 2 Mini is a versatile multimodal model engineered to process both text and image inputs, subsequently generating text outputs. Developed by Thinking Machines Lab, Inc., this model is designed for broad applicability, supporting English and other languages, as well as multiple coding languages. It is particularly suited for developers engaged in creating AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation (RAG) systems. Furthermore, it is capable of general-purpose conversational use, instruction-following, and various natural language and visual tasks.
Read More
The model is released with open weights under an Apache 2.0 license, encouraging research, fine-tuning, and integration into third-party products by downstream developers. It is distributed as a single checkpoint of approximately 20GB, utilizing mixed NVFP4 and MXFP8 precisions. This compact size allows for local inference on compatible hardware, including 1x NVIDIA DGX Spark, Apple silicon devices with 32GB or more memory (MacBooks, Mac minis, Studios), and other GPUs with a recommended 32GB or more of VRAM.
Inkling 2 Mini supports local inference through several frameworks, including Hugging Face, SGLang, vLLM, TokenSpeed, NVIDIA Dynamo, Unsloth (including quantized GGUF versions), llama.cpp (for quantized GGUF versions), Llama App, Ollama, and Exo Labs (on Apple silicon). It can also be post-trained with Miles and SkyRL. The model features a 24-layer decoder-only transformer architecture with sparse Mixture-of-Experts (MoE) feed-forward layers. Its middle 12 layers run three times with shared weights, effectively providing 48 layer applications per token from 24 layers of weights. Each layer routes tokens to 8 of 128 experts, and attention is a hybrid of local and global layers. Images are encoded via a vision transformer, with image and text inputs projected into a shared hidden space for joint processing by the decoder. The model has 26.4 billion parameters, with 3.4 billion active parameters, and supports a context window of up to 1 million tokens.
Training data for Inkling 2 Mini encompasses a wide array of content types, including text, images, and video, sourced from publicly available repositories, third-party acquisitions, internal generation, and synthetic augmentation. Data curation involves cleaning, processing, and modifying datasets, including deduplication and filtering for quality and safety. Safety evaluations were conducted, covering everyday human-AI interaction and dangerous-capability testing, with mitigations applied through alignment training. The model's limitations include potential hallucination, occasional instruction-following failures, degraded performance in long conversations, and biases from training data. It is less capable than larger models in factual recall and knowledge-heavy questions, and may exhibit repetitive reasoning or agentic unreliability, especially in non-English languages. Developers are advised to apply human oversight, run agentic workloads in sandboxes, and conduct independent evaluations before deployment.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentRuns locally
- PlatformsWeb
Inkling 2 Mini Highlights
Multimodal Input: Accepts both text and image inputs for processing.
Text Output Generation: Produces text outputs based on multimodal inputs.
General-Purpose Model: Designed for a wide range of applications and tasks.
Multilingual Support: Functions in English and other languages.
Multiple Coding Language Support: Compatible across various coding environments.
Open Weights Release: Available for download and use with open weights.
Apache 2.0 License: Provides flexibility for research, fine-tuning, and integration.
Developer-Focused: Intended for building AI-powered applications.
Agentic System Support: Suitable for developing agentic and tool-use systems.
Coding Assistant Capability: Can be used to create coding assistants.
Chatbot Integration: Facilitates the development of chatbots.
Retrieval-Augmented Generation (RAG) Systems: Supports RAG system creation.
Local Inference Capability: Can run on local AI compute with specified hardware.
Extensive Framework Support: Compatible with Hugging Face, SGLang, vLLM, and more.
Post-Training Options: Can be post-trained with Miles and SkyRL.
Large Context Window: Supports up to 1 million tokens.
Getting Started with Inkling 2 Mini
Download Model Weights: Access and download the Inkling 2 Mini weights from Hugging Face.
Access via Tinker Platform: Utilize the Tinker fine-tuning platform for API access and model interaction.
Install Tinker Renderers: Install the tml-renderers package for Tinker platform integration.
Configure Local Hardware: Ensure required hardware (e.g., NVIDIA DGX Spark, Apple silicon with 32GB+ memory) is available.
Install Software Frameworks: Set up supported local inference frameworks like Hugging Face, SGLang, or llama.cpp.
Run Inference: Execute the model locally using chosen frameworks for text generation from image and text inputs.
Inkling 2 Mini's Use Cases
- AI Application Development
- Agentic Systems Creation
- Coding Assistant Tools
- Chatbot Implementation
- Retrieval-Augmented Generation
- General Conversational AI
- Natural Language Processing
- Visual Task Processing
- Model Fine-tuning
- Third-Party Product Integration







