Description
MetaEncoder v1 is a powerful feature extraction model developed by Facebook, boasting 28 billion parameters. Hosted on Hugging Face, it provides a robust solution for developers and researchers working with complex data types. The model is designed for ease of use, requiring `transformers>=5.15` and an 80 GB GPU for optimal performance in bf16 precision.
Read More
At its core, MetaEncoder v1 offers three primary functionalities: `encode`, `retrieve`, and `decide`. The `encode` function allows users to embed various inputs—such as strings, PIL images, or dictionaries containing text, images, or video frames—into a (n, 6656) float32 tensor. These embeddings are L2-normalized and can be used for storage, clustering, or comparison tasks. The model is sensitive to instructions, meaning the same input can be embedded differently based on the provided context, enhancing its adaptability for specific use cases.
The `retrieve` function is designed for ranking a corpus against a given query. Users can pass a query and a corpus (either a list of items or a pre-encoded tensor) along with optional instructions for both. It returns a list of `Hit` objects, ordered from best to first, making it ideal for search and information retrieval applications. This capability is particularly efficient for large corpora, as the corpus can be encoded once and then searched repeatedly.
Finally, the `decide` function enables the model to answer Jev-style questions about a given state. This state can be a string or any JSON value, optionally accompanied by images or video. Questions are structured with types like 'noul' (yes/no), 'choice' (multiple options), or 'score' (ordered levels), each with specific instructions and criteria. The model provides answers with probabilities, confidence scores, and raw cosine similarities. A `decide_batch` function is also available for processing multiple states simultaneously, and a calibration feature allows users to fine-tune the model's temperature to their specific data for improved accuracy.
A key advantage of MetaEncoder v1 is its internal prompt formatting, which eliminates the need for callers to manually write prompts. The wrapper ensures inputs are formatted exactly as they were during training, maintaining consistency and simplifying integration. This includes handling system and user turns, image and video encoding, and pooling. For advanced users, direct access to `modeling_metaencoder.py` is possible to bypass the wrapper for custom prompt formatting and pooling, offering flexibility while still leveraging the model's core capabilities.
- CategoryAI Models & LLMs
- DeploymentCloud
- PlatformsHugging Face
MetaEncoder v1 Highlights
Feature Extraction Model
28 Billion Parameters
Embed Inputs (encode)
Rank Corpus Against Query (retrieve)
Answer Jev-style Questions (decide)
Supports Text, Image, and Video Inputs
Internal Prompt Formatting
Instruction-Based Embedding
Batch Decision Processing
Model Calibration for Temperature Fitting
L2-Normalized Embeddings
No Manual Prompt Writing Required
Compatible with transformers>=5.15
Requires 80 GB GPU (bf16)
Direct Access to Modeling for Customization
Getting Started with MetaEncoder v1
Access Model: Navigate to the facebook/meta-encoder-v1 page on Hugging Face.
Set Up Environment: Ensure `transformers>=5.15` is installed and an 80 GB GPU is available for bf16 precision.
Embed Inputs: Use the `encode(inputs, instruction=None)` call to generate embeddings for text, images, or video.
Retrieve Information: Utilize `retrieve(query, corpus, top_k=None, query_instruction=None, corpus_instruction=None)` to rank a corpus against a query.
Make Decisions: Employ `decide(state, questions, images=None, video=None)` to answer structured questions about a given state.
Calibrate Model: For improved accuracy, use `model.calibrate(answers, labels)` with a labelled dataset to fit the temperature.
MetaEncoder v1's Use Cases
- Information Retrieval
- Data Clustering
- Semantic Search
- Question Answering Systems
- Content Understanding
- Multimodal Data Processing
- Decision Support Systems







