ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval¶

Conference: ECCV 2026
arXiv: 2606.20280
Code: None
Area: Multimodal VLM / Reinforcement Learning / Information Retrieval
Keywords: Universal Multimodal Retrieval, Reinforcement Learning, Grain Blindness, Ranking-Driven, Verifiable Rewards

TL;DR¶

ELVA proposes a rule-based verifiable reward RL framework that co-optimizes the retrieval-ranking capability of MLLMs through ranking rewards and margin rewards. This addresses the "grain blindness" issue that arises when adapting contrastive learning to retrieval tasks. ELVA achieves SOTA performance on both M-BEIR and a self-built multi-grained benchmark MRBench, with a 13.1% improvement on MRBench.

Background & Motivation¶

Universal Multimodal Retrieval (UMR) aims to handle diverse cross-modal retrieval tasks (text-to-image, image-to-text, interleaved text-image retrieval, etc.) under a unified framework. Recent work has made significant progress by adapting Multimodal Large Language Models (MLLMs) to retrieval tasks via contrastive learning, leveraging their powerful cross-modal representation capabilities. However, this study reveals that existing methods commonly suffer from "grain blindness" when adapting the contrastive learning paradigm to retrieval tasks—models tend to ignore fine-grained semantic information contained within a query (e.g., a query like "Charmander breathing fire" contains both the action "breathing fire" and the entity "Charmander"), leading to poor performance on complex queries that require capturing multiple levels of granularity simultaneously.

The root cause of grain blindness lies in the binary classification nature of contrastive learning: it only distinguishes between positive and negative samples, treating all negative samples equally and ignoring the varying grain information carried by different negative samples. For example, for the query "standing dog", a negative sample of "lying dog" and another of "standing cat" provide discriminative clues from the "pose" grain and the "entity" grain, respectively, yet contrastive learning cannot exploit this difference. This paper suggests that models should rank negative samples based on their similarity to the positive sample, thereby learning different granular information from each negative sample. Key Insight: Upgrade retrieval training from "distinguishing positive/negative" to "learning to rank", replace manual ranking labels with the exploration capabilities of reinforcement learning, and let the model autonomously discover the hierarchical structure among negative samples to capture all granular information within the query.

Method¶

Overall Architecture¶

ELVA employs a three-stage training framework. The first two stages follow the pre-training and instruction fine-tuning pipelines of LamRA, while the third stage introduces RL fine-tuning based on rule-based verifiable rewards. Given a multimodal query \(q\) and a candidate set \(\Omega = \{c_n\}_{n=1}^{N}\) (comprising 1 positive sample and N negative samples), the goal is to retrieve the top-k most relevant candidates. While the first two stages achieve preliminary retrieval capabilities through contrastive learning, the model performs poorly on complex queries due to grain blindness. The core of ELVA's third stage is to allow the policy model to generate \(G\) independent groups of embeddings (rollouts) for the same query, each containing the query embedding, the positive sample embedding, and all negative sample embeddings. It then evaluates the quality of each embedding group through two verifiable reward functions, and finally optimizes the policy model using the GRPO algorithm while constraining it with KL divergence to prevent the model from drifting too far from the reference policy.

The framework utilizes a three-stage sequential training, where each rollout in the RL stage contains a complete embedding generation and reward calculation loop.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Query q + Candidate Set Ω<br/>Positive + N Negative Samples"] --> B["Stage 1: Pre-training<br/>NLI Dataset + InfoNCE Loss"]
    B --> C["Stage 2: Instruction Tuning<br/>M-BEIR Dataset + InfoNCE Loss"]
    C --> D["Preliminary Retrieval Model<br/>Suffers from Grain Blindness"]
    D --> E["Stage 3: ELVA RL Fine-Tuning<br/>Generative Embedding Extraction + G Rollouts"]
    E --> F["G Groups of Embeddings<br/>(Query / Positive / N Negatives)"]
    F --> G["Verifiable Reward Calculation<br/>Ranking Reward + Margin Reward"]
    G --> H["GRPO Optimization<br/>Policy Model + KL Constraint"]
    H --> I["Final Retrieval Model<br/>Eliminates Grain Blindness"]

Key Designs¶

1. Generative Embedding Extraction: Enabling RL Exploration

Traditional MLLM retrieval methods extract the hidden states of fixed prompt tokens (such as EOS) as embeddings. In an RL framework, this approach produces deterministic outputs with zero variance, preventing the policy gradient from exploring effectively. ELVA redesigns embedding extraction into a generative paradigm: the model first autoregressively generates a textual description of the input, and then outputs a special token [RET], taking the hidden state at this token's position as the retrieval embedding. This change introduces sufficient representation variance, allowing the multi-group rollout mechanism of GRPO to produce meaningful embedding variations, which provides a foundation for RL exploration. In implementation, the [RET] token is registered in the LLM's vocabulary, and the model generates embeddings using the template: "{image}{query}... [RET]", where the image and query modalities can be combined flexibly.

2. Verifiable Reward Function Design: Ranking Reward + Margin Reward

This is ELVA's core innovation. Traditional RL relies on human preference models or discrete ranking metrics (such as NDCG) as rewards, where the former is costly and the latter provides discontinuous signals with high variance. ELVA extends Reinforcement Learning with Verifiable Rewards (RLVR) to the multimodal retrieval domain, designing two complementary rule-based reward functions:

Margin Reward is used to ensure a sufficient similarity margin between positive and negative samples, inspired by the triplet loss. Given a query \(q\), a positive sample \(c_p\), and a negative sample set \(\{c_n\}\), the hardest negative sample that is most similar to the query is first selected. The model is then required to maintain a similarity difference between the positive sample and this hardest negative sample that exceeds a threshold \(\delta\):

\[R_{\text{Margin}} = \max(0, \cos(q, c_p) - \max_n \cos(q, c_n) - \delta)\]

Ranking Reward is a continuous ranking optimization objective which encourages the model to rank positive samples at the top of the candidate list while positioning highly similar negative samples near the top. For a ranking result obtained from a rollout (sorted by similarity from high to low), let the positive sample be at rank \(r\) with similarity \(s_{(r)}\):

\[R_{\text{Rank}} = s_{(r)} \cdot \frac{1}{1 + \log r} - \gamma \sum_{k \neq r} s_{(k)} \cdot (\log k - 1)\]

The first term encourages ranking the positive sample highly using a logarithmically decaying positional weight. The second term penalizes highly similar negative samples, with the penalty strength logarithmically decreasing as the rank \(k\) increases—this means that negative samples closer to the query's semantics (ranked higher) receive a smaller penalty, thereby forming an internal ranking structure. This continuous design provides a smooth reward signal, avoiding the jumping and instability issues associated with discrete metric rewards. The final total reward is \(R_{\text{total}} = \alpha R_{\text{Margin}} + \varepsilon R_{\text{Ranking}}\), with default weights of \(\alpha=0.4\) and \(\varepsilon=0.6\), where the ranking reward weight is slightly higher to align more directly with the Recall@K evaluation metric.

3. Balanced Negative Sampling Strategy

The construction of data during the RL training phase is crucial for optimization stability. Prior work directly used top-100 candidates as negative samples. However, this causes negative samples to be excessively similar to the query, resulting in narrow reward distributions and vanishing gradients. ELVA designs a hybrid negative sampling strategy: for each query's candidate set, 50 samples are difficult negative samples selected from the top-50 after filtering out excessively high-similarity candidates (to control difficulty and enhance ranking capability), while the other 50 are randomly sampled from the entire candidate pool (to increase distribution diversity and prevent overfitting). The entire RL training set is constructed by sampling approximately 11k training instances at a 1% ratio from M-BEIR's 1.1 million samples. This hybrid strategy ensures sufficiently large reward variance and a rich, stable learning signal.

A Complete Example¶

Taking the query "standing dog" as an example to run through the process of the ELVA RL stage. This query contains two grains: the entity "dog" and the pose "standing". The candidate set contains 1 positive sample (an image of a standing dog) and 100 negative samples (50 of which are filtered hard negative samples—such as "lying dog" and "standing cat", and 50 are random negative samples). The policy model generates \(G=8\) groups of rollout embeddings. In a typical rollout, the model might rank the positive sample at the 3rd position (\(r=3\)), preceded by "lying dog" (ranked 1st, same entity but different pose) and "standing cat" (ranked 2nd, same pose but different entity). The first term of the ranking reward \(s_{(3)} \cdot 1/(1+\log3)\) encourages the model to improve the positive sample's rank, while the second term penalizes these two highly-ranked negative samples—but since they rank high, the logarithmic penalty \(\log k\) is small, limiting the penalty strength and allowing the model to perceive the different granular information carried by these negative samples. Meanwhile, the margin reward ensures a minimum gap between the positive sample and the hardest negative sample. The relative rewards of the 8 rollouts are compared by GRPO to produce optimization signals. After training, t-SNE visualization shows that while the baseline model entangles the positive sample ("standing dog") with "lying dog" and "standing cat" (embedding distance of 0.07), ELVA significantly widens the distance (0.15), validating the preservation of granular information.

Loss & Training¶

The first two stages use the InfoNCE loss for contrastive learning:

\[\mathcal{L}_{\text{InfoNCE}} = -\frac{1}{N}\sum_{n=1}^{N}\log\left[\frac{\exp(\cos(e_q, e_c^+)/\tau)}{\sum_{m=1}^{N}\exp(\cos(e_q, e_{c_m})/\tau)}\right]\]

In the third stage, ELVA uses GRPO (Group Relative Policy Optimization) for policy optimization. For each query \(q\), \(G=8\) groups of embeddings are generated as actions, and the total reward \(R_{\text{total}}\) for each group is calculated. The advantage function is obtained by normalizing group rewards with their mean and standard deviation. The policy is updated using a PPO-style clipped surrogate objective, alongside a KL divergence regularization term (coefficient \(\beta=0.2\)) to prevent the policy from drifting too far from the reference model (i.e., the Stage 2 instruction-tuned model). The learning rate in the RL stage is \(1 \times 10^{-6}\) for 1 epoch, taking about 16 hours on 8 H20 GPUs. Throughout all stages, the vision encoder remains frozen, and only the language model part is fine-tuned using LoRA.

Key Experimental Results¶

Main Results¶

ELVA is evaluated on the 16 subtasks of the M-BEIR benchmark, which covers 8 different query-candidate modality combinations.

Method	VN	COCO	F200K	WebQA	EDIS	NIGHTS	OVEN	InfoS	FIQ	CIRR	Avg.
CLIP-L (zero-shot)	43.3	61.1	6.6	36.2	43.3	26.1	24.2	20.5	7.0	13.2	32.5
Qwen2.5-VL-7B (zero-shot)	40.2	71.9	20.3	71.9	49.4	25.5	42.4	32.1	25.0	55.1	46.7
UniIR-CLIP (supervised)	42.6	81.1	18.0	84.7	59.4	32.0	45.5	27.9	24.4	44.6	50.6
MM-Embed-7B	41.0	71.3	17.1	95.9	68.8	32.4	42.1	42.3	25.7	50.0	52.7
PUMA-3B	35.7	79.5	25.8	86.2	58.2	31.4	52.7	48.3	30.6	49.9	54.4
LamRA-Ret-7B	41.6	81.5	28.7	86.0	62.6	32.1	54.1	52.1	33.2	53.1	56.6
ELVA-7B	43.5	83.0	29.2	91.0	63.5	32.8	56.0	55.5	34.6	55.4	58.7

Note: The table above is simplified to show representative metrics (R@5 or R@10) for each task; Avg. is the average of 16 subtasks. Please refer to the original paper for the full 16-column results. ELVA-7B improves by an average of 2.1 percentage points over the LamRA-Ret-7B baseline, and up to 6.0% on the challenging InfoS configuration of \((q^i, q^t) \rightarrow (c^i, c^t)\).

On the multi-grained benchmark MRBench, ELVA achieves a relative improvement of 13.1% compared to the SOTA method LamRA-Ret-7B:

Method	COCO* (t2i)	COCO* (i2t)	NIGHTS*	CIRR*	Avg.
Qwen2-VL-7B	39.2	32.0	15.6	10.8	34.3
LamRA-Ret-7B	50.4	54.0	22.4	25.7	38.1
ELVA-7B	55.5	60.6	24.1	32.5	43.2

Ablation Study¶

#	Configuration	VN (R@5)	COCO (R@5)	F200K (R@10)	Avg.	Description
1	w/o Ranking Reward	41.7	82.0	28.4	57.2	Removes ranking reward, decreases by 1.5 percentage points
2	w/o Margin Reward	42.3	82.2	28.5	58.1	Removes margin reward, decreases by 0.6 percentage points
3	w/o Negative Ranking	42.8	82.0	28.5	58.1	Removes negative ranking term (second term) in ranking reward
4	w/o Negative Sampling	43.1	82.2	28.5	58.2	Removes balanced negative sampling, directly uses full candidates
5	w/o Random Negatives	43.3	82.6	28.9	58.4	Uses only hard negatives, without random negatives
6	ELVA-7B (Full)	43.5	83.0	29.2	58.7	Full model

Note: Avg. is the average of all 16 M-BEIR subtasks. Data is sourced from Table 8 of the original paper.

Key Findings¶

Ranking Reward has the greatest contribution: Removing the ranking reward leads to an average decrease of 1.5 percentage points, showing that global ranking optimization aligns better with the final retrieval metric (Recall@K) than local margin constraints. Removing the margin reward alone only reduces performance by 0.6 percentage points, but the best synergy is achieved when both are used, validating the complementary design of "local margin + global ranking".
Negative ranking term is indispensable: Removing the second term of the ranking reward (negative ranking penalty) results in a 0.6 percentage point drop, proving that explicitly optimizing the hierarchical structure among negative samples helps capture multi-grained information, rather than relying solely on high-ranking positive samples.
Balanced negative sampling is key to RL stability: Performance drops when the sampling strategy is removed. The addition of random negative samples (Row 4 vs. Row 5) provides distribution diversity and prevents overfitting. Reward weight ablation shows that \(\alpha=0.4, \varepsilon=0.6\) (with slightly higher weight on ranking reward) is the optimal configuration.
Framework generalizability (plug-and-play): Directly applying ELVA's RL stage as a post-training step to PUMA and MM-Embed yields improvements of +1.9 and +2.2 percentage points, respectively, demonstrating that ELVA's RL framework can serve as a universal enhancement module.
Zero-shot video retrieval generalization: Although never fine-tuned on video data, ELVA still achieves strong zero-shot text-to-video retrieval results on MSR-VTT and MSVD, preserving Qwen2-VL's inherent video understanding capabilities and indicating that RL fine-tuning avoids catastrophic forgetting.

Highlights & Insights¶

Formalizing the concept of "grain blindness" has methodological value: The paper does not just observe the phenomenon but provides a rigorous mathematical definition—\(d(f_\theta(q), f_\theta(q\setminus\{g_k\})) < \delta\) indicates that removing a specific grain causes the embedding distance to fall below the discriminative threshold. It points out that the root cause is Gradient Starvation in contrastive learning: once dominant grains provide a sufficient similarity margin to distinguish positive and negative samples, the gradient signals for helper grains are suppressed, leading to premature convergence. This analytical framework can be extended to diagnose representation collapse in other contrastive learning settings.
The continuous design of the ranking reward is highly ingenious: Traditional ranking metrics like NDCG produce discrete jumps in reward signals within RL, leading to unstable training. ELVA directly replaces discrete positional rewards with a continuous function of similarity scores (\(s_{(r)} \cdot 1/(1+\log r)\)), which preserves the semantics of ranking optimization while providing smooth gradients. This idea is transferable to any scenario requiring RL to optimize ranking tasks (such as recommendation systems and document ranking).
Generative embedding extraction unlocking RL exploration is an underappreciated design: While seemingly just changing the token extraction method (from a fixed EOS to an autoregressively generated [RET]), this modification is an enabling condition for RL to run—deterministic embeddings have no rollout variance, rendering GRPO's relative advantage comparison meaningless. This insight can be applied to other tasks where RL exploration needs to be introduced to deterministic models.
The modular design of the three-stage pipeline is highly practical: The first two stages (pre-training + instruction tuning) completely follow existing workflows. The third stage (RL) is superimposed as a "plug-in". Pre-trained retrieval models can directly integrate ELVA to perform post-training optimization without redesigning the entire pipeline.

Limitations & Future Work¶

High inference cost of MLLMs: This is a common bottleneck for all MLLM-based retrieval methods. Although the authors suggest mitigating this via feature pre-computation, layer pruning, or using a lightweight ELVA-2B version, the inference latency remains a hurdle for real-world deployment.
Performance gap in video retrieval: Although ELVA outperforms most methods in zero-shot video retrieval, it still falls behind models specialized in video training like InternVideo2. Synthesizing video data fine-tuning is required in the future to bridge this gap.
Scaling behavior of larger models remains unexplored: Currently based on Qwen2-VL-7B, the efficacy on larger MLLMs (e.g., 72B scale) has not been validated. Whether the KL constraint in RL training needs adjustment for larger models remains an open question.
Dataset dependency: MRBench is constructed by automatic filtering and manual verification from M-BEIR, covering a limited variety of grain types (entity + action, attribute + entity, etc.). More comprehensive multi-granularity scenarios (such as multi-entity interactions and spatio-temporal relations) have not been evaluated.

vs. LamRA / PUMA: These approaches only adapt MLLMs to retrieval tasks via contrastive learning. ELVA adds RL ranking fine-tuning on top of them. The fundamental difference lies in upgrading the training objective from "binary classification of positive/negative" to "ranking optimization that learns granularity hierarchy". ELVA can serve as a direct enhancement module for these methods (as validated in the experiments).
vs. Search-R3 / ReasonRank: These also use RL for learning to rank, but rely on continuous similarity scores (which easily saturate after top-rank convergence) and discrete metric rewards (discontinuous signals with high variance), respectively. ELVA's ranking reward is both continuous and non-saturating (with the margin reward ensuring continuous signals) and additionally models the ranking structure among negative samples.
vs. VLM-R1 / Vision-R1: These RL-based MLLMs target vision reasoning tasks rather than retrieval. ELVA is the first to employ RLVR in the retrieval domain, with the core differences being the design of reward functions (ranking-aware vs. reasoning correctness) and representation extraction methods (generative embedding vs. token generation).

Rating¶

Novelty: ⭐⭐⭐⭐ First to formalize the concept of "grain blindness" in retrieval tasks and solve it using RLVR. The continuous design of the ranking reward is creative, though the overall framework pipeline (contrastive pre-training + RL fine-tuning) is not an entirely new paradigm.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive and rigorous experimental design covering 16 M-BEIR subtasks, unseen datasets, held-out tasks, the MRBench multi-grained benchmark, zero-shot video retrieval, ablation studies, reward weight analysis, and framework generalizability validation.
Writing Quality: ⭐⭐⭐⭐ Clear conceptual definitions (mathematical formulation of granularity, causal chain of gradient starvation) and complete methodological motivation. However, there are minor inconsistencies in Table numbering (e.g., referencing Table 5 in text while the actual data aligns with Table 6—verification against the original paper is recommended).
Value: ⭐⭐⭐⭐ The proposed RLVR ranking framework can serve as a universal retrieval enhancement module (validated plug-and-play). The grain blindness analysis and continuous ranking reward design hold transfer value for other retrieval/ranking tasks, though the inference overhead of MLLMs limits the scope of practical deployment.