jina-colbert-v2

jina-colbert-v2 is a 560M-parameter multilingual late-interaction retrieval model supporting 89 languages. It is the first multilingual ColBERT-like model to generate compact token-level embeddings (64–128 dimensions per token), enabling scalable, cost-effective multilingual search. Matryoshka representation learning allows dimension reduction from 128 to 64 with only 1.5% performance decrease, halving storage requirements.

The model builds upon the ColBERT late-interaction architecture with a modified XLM-RoBERTa backbone (560M parameters), enhanced with rotary position embeddings and optimized with Flash Attention. Training involves two key stages: (1) initial pretraining with diverse weakly-supervised data from various languages, (2) fine-tuning with labeled triplet data and supervised distillation from a larger teacher model. The key innovation is the implementation of Matryoshka representation learning for multi-vector embeddings: the model produces token-level vectors in 128, 96, or 64 dimensions from a single training process, enabling dynamic storage optimization without retraining. The 8,192-token document context (extendable to 12,288) and 32-token query limit are managed through ALiBi positional encodings.

The model achieves a 6.5% improvement over the original ColBERT-v2 on English tasks, with an average score of 0.521 across 14 BEIR benchmarks. It outperforms traditional BM25-based retrieval methods across all tested languages on MIRACL benchmarks, showing particular strength in cross-lingual scenarios. Dropping from 128 to 64 dimensions results in only a 1.5% performance decrease while halving storage requirements — for example, storing 100 million documents with 64-dimension vectors costs $659.62/month on AWS, compared to $1,319.24 for 128 dimensions. The model's multilingual coverage (89 languages) and compact token embeddings make it uniquely suited for global search applications.

The model requires CUDA-capable hardware for optimal performance. It supports document lengths up to 8,192 tokens (extendable to 12,288) while limiting queries to 32 tokens. For production deployment, it is available through the Jina Search Foundation API, AWS marketplace, and Azure, with a non-commercial version on Hugging Face. When implementing, specify whether you are embedding queries or documents — the model uses asymmetric encoding. The model is not designed for real-time processing of extremely large document collections without proper indexing; use FAISS, Qdrant, or Weaviate for ANN search. For domain-specific tasks, consider fine-tuning on your corpus. For single-vector retrieval with higher dimensional output, consider jina-embeddings-v4 (multi-vector mode) or jina-embeddings-v5-text-small.