Sign in Subscribe
2 min read

Jina Embeddings V4: A Universal Multimodal Model for Text, Image, and Code Retrieval

Jina Embeddings V4: A Universal Multimodal Model for Text, Image, and Code Retrieval

Jina AI’s Latest Breakthrough: A Single Model for Everything

Jina AI has just dropped a bombshell in the world of AI embeddings with jina-embeddings-v4, a 3.8 billion-parameter multimodal model that unifies text, image, and even code retrieval into a single architecture. The model, detailed in a new arXiv paper, promises state-of-the-art performance across a dizzying array of tasks—from semantic search to parsing complex visual documents like charts, tables, and diagrams.

Why This Matters

Embedding models are the unsung heroes of AI, transforming raw data (text, images, etc.) into numerical vectors that machines can understand. Most models specialize—CLIP for images, BERT for text, ColBERT for late-interaction retrieval—but Jina’s latest offering throws specialization out the window. Instead, jina-embeddings-v4 is a Swiss Army knife, capable of handling:

  • Text retrieval (asymmetric queries, symmetric similarity)
  • Image retrieval (including visually rich documents)
  • Cross-modal search (text-to-image, image-to-text)
  • Code retrieval (natural language to code snippets)

And it does all this while supporting both single-vector (dense) and multi-vector (late interaction) embeddings, a flexibility that could save businesses from deploying multiple specialized models.

Key Innovations

  1. Unified Multimodal Architecture
  • Built on Qwen2.5-VL-3B-Instruct, the model processes text and images through a shared pathway, minimizing the notorious modality gap that plagues dual-encoder models like CLIP. Images are tokenized into "image tokens" and fed into the same LLM backbone as text, ensuring cohesive semantic alignment.
  1. Dual Output Modes
  • Single-vector embeddings (2048D, truncatable to 128D) for efficient similarity search.
  • Multi-vector embeddings (128D per token) for late-interaction retrieval, offering higher precision at the cost of compute.
  1. Task-Specific LoRA Adapters
  • Three lightweight adapters (60M params each) fine-tune the model for:
    • Asymmetric retrieval (short queries vs. long documents)
    • Semantic similarity (symmetric matching)
    • Code search (natural language to programming languages)
  1. Jina-VDR: A New Benchmark for Visual Documents
  • The team introduced Jina-VDR, a multilingual benchmark spanning 30+ datasets (charts, maps, manuals, etc.) to evaluate performance on visually rich content. Early results show jina-embeddings-v4 outperforms rivals like ColPali and CLIP-style models by a wide margin.

Performance Highlights

  • Text Retrieval: Competitive with voyage-3 and gemini-embedding-001 on MTEB/MMTEB benchmarks.
  • Visual Documents: Dominates ViDoRe and Jina-VDR, especially in late-interaction mode (nDCG@10: 90.17 vs. 65.5 for BM25+OCR).
  • Code Search: Matches general-purpose models but lags behind specialized tools like voyage-code.
  • Cross-Modal Alignment: Smashes the modality gap, with image-text cosine similarities ~2x closer than CLIP’s.

The Bottom Line

Jina’s new model is a game-changer for businesses that need a single, scalable solution for multimodal retrieval. Whether you’re searching through PDFs, matching product images to descriptions, or digging up code snippets, jina-embeddings-v4 offers a unified—and surprisingly efficient—alternative to juggling multiple AI tools.

For the full technical deep dive, check out the arXiv paper. And if you’re ready to test it, Jina AI has open-sourced the benchmark datasets on Hugging Face.