Description
Sarvam-Prism is a specialized decision model developed by Sarvam, leveraging the powerful Sarvam-105B backbone. Its primary function is text classification, where it processes a given context and one or more questions, each accompanied by a fixed set of possible answers. Unlike generative AI models, Sarvam-Prism does not create new text; instead, it assigns a probability to each predefined answer option in a single forward pass, ensuring that the output is always one of the provided choices.
Read More
This model is particularly suited for applications requiring precise, constrained decision-making rather than open-ended text generation. It operates by transforming each question into a sequence that includes the state, instructions, and every option, followed by an answer marker. A small decision head then reads the hidden state at the end of each option and at the answer marker to produce probabilities for each option. The model's architecture includes specific attention mask adjustments and FP8 kernels for efficient inference.
Sarvam-Prism is available for download and can be run on Hugging Face, with specific hardware requirements including an NVIDIA GPU with FP8 support and approximately 120 GB of free memory (e.g., B200 or H200). It also requires Python 3.10+ and a C compiler. For those without the necessary hardware, Sarvam-Prism is also offered as a hosted API from Sarvam, eliminating the need for local GPU resources.
The model's performance is optimized for speed, achieving a median of 44 ms per request on an NVIDIA B200, as measured on the public Decision Index benchmark. It supports various request formats, including 'choice', 'yes_no', and 'score' types, allowing for flexible question design. All questions within a request are scored together in batches, with each question being scored independently. The context limit is set at 131,072 tokens per question, accommodating extensive input.
The weights are stored in a custom FP8 layout, making them incompatible with general-purpose inference servers or standard models like AutoModelForCausalLM. The model's inference code, including input encoding, attention mask, decision head, and FP8 kernels, is provided within the `prism.py` file. Sarvam-Prism is released under the Apache 2.0 License, making it accessible for a wide range of applications.
- CategoryAI Models & LLMs
- Pricing ModelFreemium
- DeploymentCloud, Self-hosted
- PlatformsWeb
Sarvam-Prism Highlights
Text classification decision model
Built on Sarvam-105B backbone
Returns probability for predefined answers
Does not generate text, only selects from options
Available for download on Hugging Face
Hosted API available from Sarvam (no GPU needed)
Requires NVIDIA GPU with FP8 support (e.g., B200/H200)
Requires Python 3.10+ and C compiler
Optimized for speed with median 44 ms per request
Supports 'choice', 'yes_no', and 'score' question types
Processes questions in batches
Context limit of 131,072 tokens per question
Weights stored in custom FP8 layout
Inference code provided in `prism.py`
Licensed under Apache 2.0
Getting Started with Sarvam-Prism
Download Model and Code: Obtain the Sarvam-Prism model and associated code from its Hugging Face repository.
Install Dependencies: Set up the required Python 3.10+ environment and install all necessary dependencies.
Prepare Hardware: Ensure you have an NVIDIA GPU with FP8 support and at least 120 GB of free memory, such as a B200 or H200.
Load Model: Use `Prism.from_pretrained("sarvamai/prism")` or load the model with `prism.py` as instructed.
Format Request: Structure your input with a `state` (context) and `questions` (map of question names to question objects), specifying `type`, `instructions`, and `criteria`.
Run Inference: Execute the model with your formatted request to receive probabilities for each answer option.
Access API (Optional): Utilize the hosted API from Sarvam if local GPU resources are unavailable, following the API documentation.
Sarvam-Prism's Use Cases
- Structured Decision Making
- Text Classification
- Survey Analysis
- Question Answering
- Sentiment Scoring
- Compliance Checking
- Content Moderation
