jina-reranker-v3.5

jina-reranker-v3.5 is a 0.6B-parameter listwise reranker that matches 4B-parameter models at roughly 7× fewer parameters. It introduces a hybrid attention schedule (three sliding-window layers + two global layers) paired with a three-stage self-distillation recipe that transfers quality from a full-attention teacher to a sparse-attention student. It is a drop-in replacement for jina-reranker-v3, maintaining the same API schema while improving efficiency and domain coverage.

The architecture retains the LBNL listwise design of jina-reranker-v3: the query and all candidate documents share a single context window, and embeddings are read from each document's final token. The key innovation is the hybrid attention schedule. Instead of uniform global attention at every layer, the model uses three sliding-window attention layers (restricting each position's attention to a local window) followed by two global attention layers. The terminal layer is pinned to global attention because the LBNL readout mechanism requires the final token to attend to the full context. Training proceeds in three stages: (1) a full-attention teacher model is trained on a curated multi-domain mixture spanning legal, medical, financial, multilingual, and structured retrieval; (2) a sparse-attention student is initialized from the teacher and fine-tuned with staged adaptation, gradually increasing the proportion of training steps using the hybrid schedule; (3) a final distillation pass where the student learns to match the teacher's soft ranking distributions, recovering over 97% of the teacher's ranking quality while running the cheaper sparse-attention inference path.

Under a unified top-100 protocol with jina-embeddings-v5-text-small as the first-stage retriever, the model scores 63.20 NDCG@10 on BEIR, 74.11 on MIRACL, 70.95 on RTEB, and 48.3 on Struct-IR, improving over jina-reranker-v3 (62.10 / 72.20 / 68.01 / 38.7) on every axis. Its BEIR score edges out the 4B Qwen3-Reranker (62.28) at roughly 7× fewer parameters. Relative gains over v3 include +1.10 BEIR, +2.6% MIRACL, +4.3% RTEB (with large lifts on legal retrieval), and +24.8% on structured-data ranking (Struct-IR). Listwise inference runs 1.22×–1.56× faster than v3 from short to long contexts (A100, FlashAttention-2). The multi-domain training mixture yields particularly strong improvements on semi-structured data (tables, JSON, XML) and domain-specific corpora.

jina-reranker-v3.5 is a drop-in replacement for jina-reranker-v3: the request schema is unchanged, so switching the model string requires no code changes. It ranks a query against the full candidate list in a single call — pass documents through the standard rerank endpoint and use top_n to limit returned results, or set return_embeddings to also retrieve document embeddings. The model is especially strong on structured-data ranking, legal and other domain-specific retrieval, and multilingual collections. Its hybrid-attention design keeps latency low on long candidate lists. The 131K token context window supports very long documents without chunking. Available under CC-BY-NC-4.0 on Hugging Face; contact Jina AI for commercial use. For air-gapped or on-premises deployments, quantized weights are available. Pair with jina-embeddings-v5-text-small or jina-embeddings-v4 as the first-stage retriever.