UEmbed: Unified Sparse and Dense Multimodal Embeddings
UEmbed is a decoder-only multimodal embedding model that produces both sparse lexical and dense representations in a single causal forward pass. It appends N learnable special tokens to the input and partitions the vocabulary into N disjoint subsets, with each token's hidden state predicting sparse weights over its assigned subset. The model is released at 2B, 4B, and 9B scales, trained on public data. UEmbed-9B achieves 71.8 (dense) and 71.0 (sparse) on MMEB-v2, outperforming publicly trained multimodal embedding models like RzenEmbed, and remains competitive on BEIR against strong dense and sparse baselines.
UEmbed introduces a unified decoder-only architecture for multimodal embeddings, generating both sparse and dense representations in one forward pass. By appending learnable special tokens and partitioning the vocabulary, it efficiently produces sparse lexical vectors alongside dense embeddings. The model is available in 2B, 4B, and 9B parameter versions, all trained on public data. The 9B variant sets new performance marks on MMEB-v2 for publicly trained models and shows competitive results on the BEIR benchmark.
The key innovation is the use of a causal decoder-only transformer to simultaneously produce sparse and dense embeddings without auxiliary cross-modal modules. The sparse representation is constructed by assigning disjoint vocabulary subsets to N special tokens, whose hidden states predict token-level weights, enabling efficient and scalable sparse retrieval within a unified model.
This approach could simplify multimodal search pipelines by replacing separate dense and sparse retrieval models with a single model, reducing infrastructure complexity and cost. The release of multiple model sizes (2B, 4B, 9B) suggests a focus on practical deployment across different resource constraints.
UEmbed offers a more efficient and unified solution for multimodal retrieval, potentially lowering operational costs and improving retrieval quality for enterprises building search, recommendation, or RAG systems. Its competitive performance with publicly available data makes it an attractive option for organizations seeking to avoid proprietary data dependencies.
Future signals to watch include adoption in commercial search and RAG systems, performance on additional multimodal benchmarks, and potential extensions to other modalities or languages. The model's public data training may also spur further open-source development and benchmarking.