Description
nomic-embed-text-v2-moe is a cutting-edge multilingual Mixture of Experts (MoE) text embedding model designed to excel in multilingual retrieval tasks. It supports around 100 languages and is trained on over 1.6 billion pairs, making it highly effective for diverse linguistic applications. The model offers a flexible embedding dimension, utilizing Matryoshka Embeddings to achieve up to three times reduction in storage costs with minimal performance degradation.
The architecture of nomic-embed-text-v2-moe includes 475 million total parameters, with 305 million active during inference. It features an MoE configuration with eight experts and top-2 routing, allowing for efficient processing and high performance. The model's maximum sequence length is 512 tokens, and it supports embedding dimensions ranging from 768 to 256.
nomic-embed-text-v2-moe is fully open-source, with model weights, code, and training data available for public use. This transparency ensures reproducibility and facilitates further research and development. The model has demonstrated superior performance on benchmarks like BEIR and MIRACL, outperforming models in the same parameter class and maintaining competitiveness with larger models.
Despite its strengths, users should be aware of certain limitations. Performance may vary across different languages, and the resource requirements can be higher than traditional dense models due to the MoE architecture. Additionally, users must enable trust_remote_code=True when loading the model to utilize the custom architecture implementation.
Overall, nomic-embed-text-v2-moe is a powerful tool for multilingual text embedding tasks, offering flexibility, efficiency, and high performance for a wide range of applications.
nomic-embed-text-v2-moe Highlights
State-of-the-art multilingual performance
Supports approximately 100 languages
Trained on over 1.6 billion pairs
Flexible embedding dimensions
Open-source model weights and code
Mixture of Experts architecture
Efficient storage with Matryoshka Embeddings
High performance on BEIR and MIRACL benchmarks
Getting Started with nomic-embed-text-v2-moe
Access page: Visit the Hugging Face model page
Load model: Use SentenceTransformers or Transformers
Configure environment: Set up GPU for best performance
Integrate: Use task instruction prefixes for queries and documents
Fine-tune: Adjust embedding dimensions for specific needs
nomic-embed-text-v2-moe's Use Cases
- Multilingual Retrieval
- Text Embedding
- Data Analytics
- Language Processing
- Research and Development
