jina-code-embeddings-1.5b

jina-code-embeddings-1.5b is a 1.5B-parameter code embedding model that maps programming code and natural language queries into a shared 1536-dimensional semantic space. It supports 32K token context windows, enabling retrieval over whole files and multi-file code contexts. It achieves state-of-the-art code retrieval performance, significantly outperforming its 0.5B sibling and substantially larger dedicated code retrieval models.

The model is built on an autoregressive transformer backbone pretrained on both natural language and code corpora, leveraging the code-generation pretraining objective to produce rich semantic representations. It extracts embeddings via Last-Token-Pooling from the final position of the sequence — the autoregressive model's causal attention naturally attends to the full input context, and the code-generation objective produces representations that capture both syntactic structure and functional semantics. Training uses a two-stage recipe: (1) contrastive pretraining on large-scale code-text pairs, (2) supervised fine-tuning on curated code retrieval datasets with hard-negative mining. The 32K context length is enabled through rotary position embeddings with a tuned base frequency. Matryoshka representation learning allows dimension truncation from 1536 to 128.

The model achieves a 79.04% overall average and 78.94% MTEB Code average, establishing new benchmarks for its parameter class. Notable scores: 98.41% on HumanEval, 90.13% on MBPP, 98.02% on WikiSQL, 99.44% on CodeChefXLang, 92.54% on CodeTransOceanContest (code-to-code), 86.45% on COIR-CodeSearchNet (NL2Code), 96.34% on Doc2Code, 92.37% on StackOverflowQA, and 86.33% on SWE-Bench (vs. 83.00% for the 0.5B variant). It surpasses larger alternatives and shows consistent improvements over the 0.5B variant, particularly on complex multi-file and cross-repository tasks.

Strategically employ instruction prefixes based on retrieval requirements: nl2code, code2code, code2nl, techqa, code2completion. Maintain consistency across your pipeline — use the same prefix type for queries and documents in a given retrieval scenario. The enhanced 1.5B capacity is ideal for complex scenarios involving multiple programming paradigms and extensive codebases. Profile use cases to determine optimal Matryoshka dimension balancing quality and resources; 256–512 dimensions is a good starting point for indexing, 1536 for re-ranking. Use batch size 256 for production alignment with training. The model is excellent for cross-repository and cross-language searches given 99.44% CodeChefXLang performance. Implement as the primary retrieval component in RAG systems, paired with jina-reranker-v3.5 for precision on top-50 candidates. Use cosine similarity for embedding comparison. Optimal for enterprise deployments requiring both performance and efficiency with sub-second latency.