Jina Embeddings v3

Description

Jina Embeddings v3 is a sophisticated multilingual text embedding model developed by Jina AI. It is designed to cater to a wide range of natural language processing (NLP) applications. The model is based on the Jina-XLM-RoBERTa architecture and incorporates Rotary Position Embeddings (RoPE), allowing it to handle long input sequences up to 8192 tokens. This capability is particularly beneficial for applications requiring extensive text processing.

The model features five LoRA adapters, enabling efficient generation of task-specific embeddings. Users can customize embeddings for various tasks, including retrieval queries, passage embeddings, clustering, classification, and text-matching. This flexibility makes Jina Embeddings v3 suitable for asymmetric retrieval tasks, clustering and re-ranking applications, classification tasks, and tasks that quantify text similarity.

Jina Embeddings v3 supports Matryoshka Embeddings, offering flexible embedding sizes ranging from 32 to 1024. This feature allows users to truncate embeddings to fit specific application needs. The model is tuned for 30 languages, including Arabic, Chinese, English, French, German, Hindi, Japanese, Korean, and Spanish, among others.

A notable update in the model is the resolution of a bug in the encode function, ensuring consistent normalization of truncated embeddings. Users are advised to apply mean pooling when integrating the model, as this approach yields high-quality sentence embeddings. The model can be used via the Jina Embedding API or directly through the Transformers package.

Jina Embeddings v3 is compatible with GPUs supporting FlashAttention-2, such as Ampere, Ada, or Hopper GPUs. It is licensed under CC BY-NC 4.0, with commercial usage inquiries directed to Jina AI. The model has seen significant usage, with over 2 million downloads last month, and is listed on AWS and Azure platforms.

Jina Embeddings v3 Highlights

• Multilingual support for 30 languages

• Extended sequence length up to 8192 tokens

• Task-specific embeddings with LoRA adapters

• Matryoshka Embeddings for flexible sizes

• Mean pooling for high-quality sentence embeddings

• Compatibility with FlashAttention-2 GPUs

• Fine-tuning support with SentenceTransformerTrainer

• ONNX inference for efficient processing

• Bug fix for encode function normalization

• CC BY-NC 4.0 license

Getting Started with Jina Embeddings v3

1. 1

   Access page: Visit huggingface.co/jinaai/jina-embeddings-v3

2. 2

   Load model: Use AutoModel.from_pretrained

3. 3

   Configure environment: Ensure GPU compatibility

4. 4

   Integrate: Apply mean pooling for embeddings

5. 5

   Fine-tune: Use SentenceTransformerTrainer for tasks

Jina Embeddings v3's Use Cases

• Text Retrieval
• Text Classification
• Clustering
• Text Matching
• Multilingual Processing

FAQ from Jina Embeddings v3

• What is Jina Embeddings v3 used for?

  Jina Embeddings v3 is used for multilingual text embedding across various NLP tasks, including retrieval, classification, and text-matching.

• How does Jina Embeddings v3 handle long input sequences?

  It uses Rotary Position Embeddings to process input sequences up to 8192 tokens.

• Can Jina Embeddings v3 be fine-tuned?

  Yes, it supports fine-tuning with SentenceTransformerTrainer for specific tasks.

• What languages does Jina Embeddings v3 support?

  It supports 30 languages, including English, Chinese, Arabic, and Spanish.

• What is the Matryoshka Embeddings feature?

  It allows flexible embedding sizes, enabling truncation to fit application needs.

• Is Jina Embeddings v3 compatible with all GPUs?

  It is compatible with GPUs supporting FlashAttention-2, such as Ampere, Ada, or Hopper GPUs.

• What license is Jina Embeddings v3 under?

  It is licensed under CC BY-NC 4.0.

• How can I integrate Jina Embeddings v3?

  You can integrate it using mean pooling for high-quality sentence embeddings.