Description
Granite Switch 4.1 30B is a sophisticated modular large language model (LLM) developed by IBM, available on Hugging Face. This model is engineered to provide a versatile solution for various AI tasks by composing a 30-billion parameter base model with specialized adapters and an embedded speech-to-text encoder. This architecture allows a single hosted endpoint to seamlessly transition between different functionalities, including generating natural language, making general and specialized decisions, and transcribing speech into text.
Read More
The core of Granite Switch 4.1 30B is the ibm-granite/granite-4.1-30b base model, which boasts a 128K context window. It is augmented by two primary adapters: a 'decision' aLoRA for general-purpose decision-making and a 'guardian-core' aLoRA for detecting safety risks such as harm, jailbreaking, profanity, violence, sexual content, social bias, and unethical behavior. These adapters are activated on demand using control tokens within the chat template, providing dynamic functionality.
A key feature of this model is its embedded speech-to-text capability, powered by ibm-granite/granite-speech-5.0-470m-turboctc. This encoder supports transcription in seven languages: English, French, German, Spanish, Portuguese, Italian, and Japanese, outputting punctuated and capitalized transcripts. This means that audio input can be processed directly on the same endpoint as chat and decision tasks, eliminating the need for a separate speech server.
Granite Switch 4.1 30B is licensed under Apache 2.0 and supports backends like HuggingFace Transformers and vLLM. It is designed for developers and organizations looking to integrate advanced language, decision, and speech processing into their applications. The model is also available in other configurations, including an 8B base version and complete Granite Libraries bundles. Ethical considerations emphasize the need for application-level safety testing and monitoring, as the embedded guardian-core adapter provides an additional layer of safety but is not a substitute for comprehensive deployment safeguards.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud
- PlatformsWeb
Granite Switch 4.1 30B Highlights
Modular LLM architecture with base model and adapters
30 billion parameters for the base model
128K token context window
Decision-making adapter (aLoRA) for general-purpose decisions
Guardian-core adapter (aLoRA) for safety risk detection
Embedded multilingual speech-to-text (ASR) encoder
Supports English, French, German, Spanish, Portuguese, Italian, Japanese for ASR
Automatic language detection for audio input
Single hosted endpoint for chat, decisions, and speech-to-text
Adapters activated on demand via control tokens
Compatible with HuggingFace Transformers and vLLM backends
Apache 2.0 License
KV cache normalization for adapters
No joint training of adapters, allowing independent development
Single checkpoint for HuggingFace and vLLM without conversion
Getting Started with Granite Switch 4.1 30B
Access Model: Navigate to the ibm-granite/granite-switch-4.1-30b page on Hugging Face.
Install Dependencies: Ensure Python (>=3.11, <3.14), granite-switch (>=0.1.1), PyTorch (>=2.11.0), and Transformers (>=5.16, <=5.17.0) are installed.
Load Model: Use the HuggingFace Transformers library to load the model, specifying the desired revision if not the latest.
Activate Adapters: Pass the adapter name (e.g., 'guardian-core', 'decision') to the chat template via `chat_template_kwargs` in your request.
Process Audio Input: Set `asr_enabled: true` in `config.json` to enable speech-to-text transcription for audio inputs.
Send Prompts: Construct prompts for text generation, decision-making, or speech transcription, utilizing the activated adapters.
Interpret Outputs: For decision adapters, read calibrated probabilities over option labels; for guardian-core, parse the JSON verdict.
Granite Switch 4.1 30B's Use Cases
- Multilingual Chatbots
- Automated Decision Systems
- Content Moderation
- Speech Transcription Services
- Intelligent Virtual Assistants
- Language Understanding
- AI-Powered Customer Support














