Dual-View Training for Instruction-Following Information Retrieval¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: Information Retrieval & RAG
Keywords: Instruction-Following Retrieval, Polarity Reversal, Dual-View Contrastive Learning, Hard Negative Mining, Dense Retrieval
TL;DR¶
Dual-View Training introduces a data synthesis strategy based on polarity reversal: given a query and a document pair where one is relevant and the other is an instruction-excluded negative, an LLM generates a complementary instruction that completely swaps their relevance labels, boosting FollowIR instruction sensitivity (p-MRR) by +57.5% across both 305M and 560M encoders without degrading general retrieval effectiveness.
Background & Motivation¶
Instruction-following information retrieval (IF-IR) marks a crucial evolution from traditional keyword and semantic matching toward controllable, constraint-aware neural retrieval systems. In complex question answering, enterprise search, and retrieval-augmented generation (RAG) pipelines, queries rarely appear in isolation; instead, users routinely provide explicit instance-specific directives specifying required timeframes, geographic boundaries, target audience levels, required evidence formats, or strict exclusions. Despite significant progress in training dense bi-encoders on instruction-rich corpora, state-of-the-art retrievers remain notoriously insensitive to fine-grained instructional nuances. Systematic evaluations on benchmark suites like FollowIR reveal that existing models predominantly rely on broad query-document topical similarity, failing to adjust ranking when instructional constraints demand that a previously preferred document be discarded.
The primary limitation behind this behavioral failure stems from the static nature of standard contrastive supervision. Pioneering frameworks like Promptriever demonstrated that incorporating instruction negativesโdocuments that match the core topical query but violate the user instructionโsignificantly outperforms generic random or BM25 hard negatives. However, within any given training instance, each document's relevance label remains strictly unidirectional and fixed. Because the target passage is always positive and the instruction negative is always negative, dual encoders can easily discover superficial shortcuts: they minimize contrastive loss by projecting broad topic tokens into nearby vector spaces, without ever learning the rigorous conditional logic dictated by the instruction's constraints.
This paper tackles the challenge by recognizing an untapped structural symmetry: every instruction negative implicitly defines a complementary facet under which it ought to become the uniquely relevant document. Core idea: prompt a large language model to synthesize complementary instructions that invert the relevance polarity of existing document pairs, pairing the original view with the reversed view in contrastive training so that no instruction-agnostic query representation can satisfy both objectives simultaneously.
Method¶
Overall Architecture¶
Dual-View Training does not alter the underlying dense dual-encoder architecture or inference mechanics; rather, it fundamentally enriches the supervision signal via symmetric data synthesis. Given a query \(q\), an original instruction \(I_{\text{orig}}\), a positive document \(D^+\), and an instruction negative \(D^-\), the system prompts an LLM (Qwen3-Next-80B-A3B-Instruct) to synthesize a complementary instruction \(I_{\text{new}}\). This generated instruction must maintain topical alignment with \(q\) while explicitly preferring the distinctive attributes of \(D^-\) and enforcing objective exclusion criteria that demote \(D^+\) into an instruction negative. During bi-encoder fine-tuning with an InfoNCE contrastive objective, training batches incorporate both \((q \oplus I_{\text{orig}}, D^+)\) and \((q \oplus I_{\text{new}}, D^-)\) as simultaneous positive targets. Because identical query and passage texts demand diametrically opposed relevance judgments depending strictly on the attached instruction, the model is compelled to internalize fine-grained instructional constraints.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Tuple<br/>Query q + Instruction I_orig + Positive D+ + Negative D-"] --> B["Passage Profiling & Lever Selection<br/>Extract D- distinct attributes and establish hard exclusion rules"]
B --> C["Complementary Instruction Synthesis<br/>LLM generation with 4-criterion audit and explicit abstention"]
C --> D["Dual-View Contrastive Fine-Tuning<br/>Simultaneously optimize (qโI_orig, D+) and (qโI_new, D-)"]
D --> E["Output<br/>Instruction-Sensitive Dense Retriever"]
Key Designs¶
1. Passage Profiling & Lever Selection: Grounding Semantic Contrast in Explicit Reversal Levers
Prior data augmentation techniques for retrieval typically apply generic paraphrasing, which fails to induce rigorous boundary discrimination. To overcome this limitation, the synthesis prompt formalizes a multi-step profiling and lever selection pipeline. The generator first extracts fine-grained attributes of \(D^+\) across multiple axes, including domain, geography, timeframe, audience level, format, methodology, and evidential constraints. It then isolates the specific attributes of \(D^-\) that differentiate it from \(D^+\) while cataloging all other in-batch instruction negatives \(N_1, \dots, N_k\) that must remain excluded. Using this profile, the model selects concrete reversal leversโsuch as narrowing domain scope, inverting temporal recency, specifying format constraints (e.g., runnable code, bulleted checklists), or enforcing explicit negative filters (e.g., "strictly exclude unwitnessed eruptions"). This ensures the new instruction actively selects \(D^-\) while erecting an impermeable, objective barrier against \(D^+\).
2. Complementary Instruction Synthesis: Enforcing a Four-Criterion Audit with Abstention
Unconstrained synthetic generation easily introduces task artifacts, hallucinations, or logically unsatisfiable constraints. Dual-view synthesis prevents label noise through a strict operational definition requiring four simultaneous conditions: (1) \(D^-\) must become unambiguously relevant under \(I_{\text{new}}\); (2) \(D^+\) must remain topically related to \(q\) but be strictly excluded by \(I_{\text{new}}\)'s constraints; (3) all remaining background instruction negatives must stay excluded; and (4) the generated text must be concise (\(\le 2\) sentences), fluent, imperative, and completely devoid of meta-task language or passage identifiers. When an unambiguous reversal is structurally impossible, the generator must trigger an explicit abstention (<answer>None</answer>), which is cleanly discarded from training. Independent human audits verified pass rates of 100% and 98% on 100 inspected samples, while an automated cross-family audit using Gemma-4-31B verified polarity consistency on 97.9% of 1,000 samples.
3. Dual-View Contrastive Fine-Tuning: Directly Penalizing Instruction-Agnostic Shortcuts
Standard instruction fine-tuning trains retrievers on isolated queries, allowing models to cheat by latching onto dominant content keywords. Dual-view fine-tuning counteracts this representation collapse by pairing original and reversed views within the same contrastive training batch. Given pooled query-document representations, the encoder optimizes an InfoNCE objective with temperature \(\tau = 0.02\) across 30 hard negatives per query (including 1โ3 instruction negatives): $$ \mathcal{L} = -\log \frac{\exp(\text{sim}(q \oplus I, D^+) / \tau)}{\exp(\text{sim}(q \oplus I, D^+) / \tau) + \sum_{j=1}^{K} \exp(\text{sim}(q \oplus I, D^-_j) / \tau)} $$ Because the retriever must rank \(D^+\) above \(D^-\) under \(q \oplus I_{\text{orig}}\), but must reverse their relative positions under \(q \oplus I_{\text{new}}\), any static query embedding that ignores the instruction prefix will suffer heavy loss penalties across both viewpoints, directly forcing the dual-encoder geometry to condition on the instruction's semantics.
A Worked Example¶
Consider an example from the volcanology domain: - Query \(q\): "Which type of volcano eruption has not been seen?" - Original Instruction \(I_{\text{orig}}\): "A document is relevant if it describes a specific type of volcano that has not been directly observed erupting, and provides information about its formation or characteristics." - Document \(D^+\) (Original Positive): Subglacial Volcanoes: Unseen Eruptions (explains eruptions beneath ice sheets that have never been witnessed by human eyes). - Document \(D^-\) (Original Instruction Negative): Impact of Volcanic Eruptions on Climate (discusses broad environmental consequences of historic eruptions like Mount Tambora; topically relevant to volcanoes but fails to discuss unobserved eruption types). - Synthesized Instruction \(I_{\text{new}}\): "Select documents that discuss the broader environmental impacts or research insights associated with unwitnessed eruption types, strictly excluding descriptions of the volcano's physical formation or structural classification." - Reversed Labels: Under \(I_{\text{new}}\), \(D^-\) becomes the true positive because it focuses squarely on environmental and climate impacts, while \(D^+\) is demoted to an instruction negative because it focuses on physical formation mechanisms.
Loss & Training¶
The framework is evaluated across two bi-encoder backbones: gte-multilingual-mlm-base (305M parameters, initialized from contrastive pretraining on 1.41 billion query-document pairs across C4, mC4, CC News, and Wikipedia) and bge-m3-retromae (560M parameters). Training runs on a single 8 \(\times\) NVIDIA H200 GPU node with a per-GPU batch size of 32 (total batch size 256), using AdamW with a learning rate of \(3 \times 10^{-5}\), linear warmup over 10% of steps, and linear decay for 1 epoch (~12 hours per run). Queries and instructions are concatenated up to a maximum length of 512 tokens; documents are independently encoded up to 512 tokens. Under the primary 880k-example full-budget regime, the proposed All-syn recipe pairs ~440k original instruction instances with ~440k polarity-reversed synthetic instances, directly replacing the non-instruction half of a standard Promptriever joint-data mixture without expanding total training compute.
Key Experimental Results¶
Main Results¶
The primary evaluation assesses whether dual-view training improves instruction sensitivity without sacrificing core ranking effectiveness or zero-shot retrieval capacity. Performance is benchmarked on FollowIR (p-MRR and Score), InfoSearch (Length and Keyword p-MRR), MAIR (IFEval and InstructIR nDCG@10), and standard BEIR.
| Training Data | Budget | FollowIR p-MRR โ | FollowIR Score โ | InfoSearch Length p-MRR โ | InfoSearch Keyword p-MRR โ | MAIR IFEval nDCG@10 โ | MAIR InstructIR nDCG@10 โ |
|---|---|---|---|---|---|---|---|
| Ins-orig (gte-305M) | 480k | 5.21 | 21.33 | 4.06 | 2.06 | 32.14 | 89.16 |
| Ins-para (Paraphrase Control) | 480k | 5.04 | 20.58 | 3.05 | 3.28 | 31.11 | 88.72 |
| Ins-syn (Diagnostic) | 480k | 7.57 | 19.73 | 9.02 | 5.61 | 36.13 | 87.97 |
| All-orig (Promptriever Joint) | 880k | 5.27 | 20.85 | -23.22 | -49.65 | 24.33 | 85.54 |
| All-syn (Proposed) | 880k | 8.30 | 21.38 | 31.91 | 12.13 | 34.08 | 90.74 |
Cross-backbone generalization on the 560M-parameter bge-m3-retromae model confirms that sensitivity improvements scale across architectures:
| Training Data | Budget | FollowIR p-MRR โ | FollowIR Score โ | InfoSearch Length p-MRR โ | InfoSearch Keyword p-MRR โ | MAIR IFEval nDCG@10 โ | MAIR InstructIR nDCG@10 โ |
|---|---|---|---|---|---|---|---|
| Ins-orig (bge-560M) | 480k | 9.40 | 22.26 | 19.00 | 4.18 | 27.12 | 90.40 |
| Ins-syn (Diagnostic) | 480k | 11.47 | 19.76 | 28.64 | 48.42 | 29.37 | 89.53 |
| All-orig (Joint Control) | 880k | 8.84 | 20.69 | -27.08 | -62.04 | 25.68 | 90.67 |
| All-syn (Proposed) | 880k | 13.92 | 20.99 | 40.15 | 49.62 | 27.07 | 90.80 |
Evaluating plain queries without instructions on standard BEIR datasets shows that All-syn preserves general semantic matching: FEVER (91.52 vs 91.33), MS MARCO (43.85 vs 43.65), Touche2020 (26.88 vs 25.42), and TREC-COVID (77.12 vs 77.27), resulting in an average nDCG@10 of 59.84 vs 59.42 for All-orig.
Ablation Study¶
To confirm that label reversalโrather than additional LLM-generated phrasingโdrives the performance gains, the authors evaluated a size-matched paraphrase control (Ins-para) using the identical generator and seed instances. Furthermore, per-subset breakdowns on FollowIR isolate behavior across distinct document corpora:
| Configuration / Subset | Robust04 p-MRR / Score | News21 p-MRR / Score | Core17 p-MRR / Score | Aggregate FollowIR p-MRR |
|---|---|---|---|---|
| Ins-orig (480k Original) | 2.19 / 19.56 | 3.90 / 26.52 | 9.55 / 17.91 | 5.21 |
| Ins-para (480k Paraphrase Control) | โ / โ | โ / โ | โ / โ | 5.04 |
| Ins-syn (480k Polarity Reversal) | 5.34 / 18.34 | 5.31 / 24.56 | 12.05 / 16.28 | 7.57 |
| All-orig (880k Joint Baseline) | 3.58 / 16.61 | 5.54 / 26.26 | 6.68 / 19.69 | 5.27 |
| All-syn (880k Full Dual-View) | 4.93 / 16.96 | 7.25 / 28.04 | 12.72 / 19.14 | 8.30 |
Key Findings¶
- Label reversal provides the critical supervision signal: The Ins-para control yields a FollowIR p-MRR of only 5.04 (comparable to 5.21 for Ins-orig), proving that exposing the retriever to extra LLM-generated text variations without altering relevance boundaries yields zero sensitivity gains. In contrast, inverting relevance polarity (Ins-syn) increases p-MRR to 7.57 (+45.3%).
- Dual-view prevents constraint inversion collapse: In standard joint training (All-orig), mixing in non-instruction data causes InfoSearch length and keyword p-MRR to collapse to -23.22 and -49.65, meaning the model systematically ranks documents opposite to the user's explicit directive. All-syn completely rescues this failure, driving p-MRR up to +31.91 and +12.13.
- Substitution budget tradeoffs: In the 480k diagnostic regime, replacing original instruction data with synthetic views causes a mild drop in FollowIR Score (21.33 to 19.73). However, in the 880k full-budget recipe (All-syn), where synthetic views replace non-instruction data, FollowIR Score increases from 20.85 to 21.38 alongside massive sensitivity gains.
Highlights & Insights¶
- Symmetric data recycling for hard negatives: Rather than discarding instruction negatives as passive non-matches, polarity reversal re-purposes them into positive targets, generating high-utility contrastive supervision at near-zero incremental data curation cost.
- Strict four-criterion quality gate with abstention: Implementing structured profiling, explicit constraint levers, and an active
Nonerefusal pathway effectively insulates synthetic training sets from hallucinated or unachievable instruction directives. - Purely data-centric, zero-parameter footprint: The technique requires no auxiliary ranking heads, adapter modules, or inference modifications, delivering consistent +57.5% sensitivity improvements across diverse bi-encoder backbones via data composition alone.
Limitations & Future Work¶
- Reliance on pre-existing instruction negatives: The synthesis pipeline requires seed instances that already contain at least one valid instruction negative; adapting the approach to raw web corpora requires an initial negative mining or annotation phase.
- Single generator and prompt template: All experiments relied on Qwen3-Next-80B-A3B-Instruct with a single structured prompt; broader generator swaps and prompt perturbation analyses remain to be explored.
- Minor trade-offs on specific subsets: While aggregate metrics and sensitivity consistently surge, individual collection scores exhibit mild variation (e.g., Core17 Score drops slightly from 19.69 to 19.14 in All-syn), pointing to subtle trade-offs in long-tail ranking calibration.
Related Work & Insights¶
- vs Promptriever (Weller et al., 2025b): While Promptriever established the utility of instruction negatives in joint training, it treats them strictly as negative distractors. Dual-View Training unlocks their dual role by actively inverting their relevance polarity under synthetic complementary facets.
- vs InF-IR (Zhuang et al., 2025): InF-IR mines diverse instruction-tied negatives under a unidirectional setting. Dual-view training establishes bidirectional symmetric constraints over fixed document pairs, preventing representation shortcuts without bloating negative pool memory.
Rating¶
- Novelty: โญโญโญโญโ An elegant polarity reversal framing that transforms hard negative mining into symmetric dual-view supervision.
- Experimental Thoroughness: โญโญโญโญโญ Rigorous controls (including paraphrase baselines, bootstrap significance testing, and cross-backbone verification).
- Writing Quality: โญโญโญโญโญ Exceptionally clear distinction between instruction sensitivity (p-MRR) and ranking effectiveness (Score/nDCG).
- Value: โญโญโญโญโญ A practical, high-impact recipe for training instruction-following dense retrievers in RAG and enterprise search.