🔍 Information Retrieval & RAG¶

🧪 ICML2026 · 26 paper notes

📌 Same area in other venues: 🔬 ICLR2026 (81) · 💬 ACL2026 (73) · 🤖 AAAI2026 (21) · 🧠 NeurIPS2025 (25) · 📹 ICCV2025 (5) · 🧪 ICML2025 (6)

🔥 Top topics: RAG ×9 · Adversarial Robustness ×2 · Alignment/RLHF ×2

BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs: Ours proposes BlitzRank, a zero-shot reranking framework based on tournament graphs. By accumulating \(\binom{k}{2}\) preference pairs generated by each \(k\)-wise comparison into a global preference graph and utilizing transitive closure to infer additional ranking relations, it achieves Pareto optimality across 14 benchmarks and 5 LLM oracles—reducing token consumption by 25–40% while matching or exceeding the accuracy of existing methods.
CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels: The CARE framework is proposed, which leverages three-way complementary experts—VLM text embeddings, image features, and original labels—to achieve reliable label correction in long-tailed noisy label scenarios through a class-adaptive Top-\(K\) consensus mechanism, consistently surpassing SOTA by up to 3.0% on synthetic and real-world benchmarks.
Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning: Graph-R1 reformulates GraphRAG as an end-to-end RL framework featuring a "knowledge hypergraph environment + multi-turn think–query–retrieve–answer agent + outcome-oriented GRPO." By utilizing lightweight n-ary hypergraph construction and dual-path hyperedge retrieval with RRF fusion, it improves the F1 score of 7B models from Search-R1's 46.19 to 57.82 across six standard RAG datasets.
HGMem: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling: This paper reconstructs the working memory in multi-step RAG from a "flat list of facts" into a hypergraph. Each hyperedge serves as a memory point that can be updated, inserted, or merged. By leveraging the inherent ability of hyperedges to connect \(n \geq 2\) entities, the system allows memory to continuously consolidate low-order facts into high-order concepts during interactions, significantly improving performance in long-context QA tasks that require "global sense-making."
Hierarchical Abstract Tree for Cross-Document Retrieval-Augmented Generation: Ψ-RAG replaces RAPTOR's k-means with a "merge-collapse" hierarchical clustering to construct cross-document abstraction trees. It incorporates a retrieval-response Agent with multi-turn rewriting capabilities and a hybrid BM25 index, enabling Tree-RAG to match or exceed Graph-RAG in corpus-level, multi-hop QA for the first time. The average F1 score is 25.9% higher than RAPTOR and 7.4% higher than HippoRAG 2.
How can embedding models bind concepts?: This paper formalizes the question of "why embedding models fail to bind concepts" as a "complexity problem of the binding function." Through geometric analysis, it demonstrates that CLIP's scene embeddings decompose additively into objects and concepts (explaining why they are probeable in unimodal settings but fail cross-modally). Furthermore, it proves on controlled Transformers that with sufficient data coverage, models learn low-complexity binding dominated by multiplicative interactions between concepts, achieving systematic generalization to unseen object combinations.
LARE: Low-Attention Region Encoding for Text–Image Retrieval: LARE is a training-free text–image retrieval framework: it extracts "low-attention" regions from a frozen vision encoder, re-encodes them, and integrates them into global similarity scores via confidence gating. This significantly improves recall for CLIP/SigLIP-style dual-encoders in crowded scenes with small or rare objects while maintaining performance on standard datasets.
LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding: LazyAttention defers RoPE positional encoding from the KV cache write stage to on-the-fly execution within the attention kernel. This allows a single physical KV copy to be reused by any logical position. On skewed RAG workloads, it reduces TTFT by 1.37× and improves throughput by 1.40× compared to SOTA Block-Attention, with negligible loss in generation quality.
LEMUR: Learned Multi-Vector Retrieval: Lemur transforms multi-vector similarity search into a supervised learning problem. By using a two-layer MLP to map token-level embeddings to a low-dimensional latent space and leveraging existing single-vector ANNS indices for retrieval, it achieves speeds an order of magnitude faster than methods like PLAID and MUVERA.
Less Is More: Elevating RAG via Performance-Driven Context Compression: CORE-RAG trains a 1.5B small compressor using GRPO reinforcement learning with "performance-as-reward," compressing retrieved top-k documents into summaries of ~3% original length. It not only avoids performance degradation but also achieves an average improvement of 3.3 EM over full-context RAG across four QA benchmarks.
Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG: This paper designs a controllable method to measure "language preference" in multilingual RAG using internal signals (next-token citation prediction probability). It finds that six open-source LLMs systematically prefer citing English documents during long-form generation, even when English documents are irrelevant—suggesting language itself influences citation selection more than document relevance.
ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World: ML-Embed extends the Matryoshka concept from one dimension (representation dimension) to three dimensions—implementing full-stack nested training across embedding parameters (MEL), model depth (MLL), and representation dimension (MRL). Simultaneously, it constructs a multilingual training set of 50 million samples covering 282 natural languages and 40 programming languages. A family of open-source models (140M-8B) is released, achieving first place on 9 out of 17 MTEB benchmarks, with improvements of \(+22.89\) in Polish and \(+6.88\) in Vietnamese.
ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs: ParisKV reduces Top-\(k\) KV retrieval decoding latency by 17–44× compared to MagicPIG/PQCache on million-token contexts by normalizing and randomly rotating keys/queries onto a unit hypersphere and replacing prefill-learned centroids with "data-independent analytical centroids." Combined with a two-stage GPU-native "collision voting + 4-bit quantized reranking" pipeline and UVA-based on-demand KV fetching, it achieves or exceeds full attention accuracy in 7 out of 9 long-generation tasks.
Predictive Prefetching for Retrieval-Augmented Generation: By learning "semantic precursors appearing 8–16 tokens before uncertainty" from transformer hidden states and attention patterns, this paper introduces a trio consisting of RetrievalPredictor + ContextMonitor + QueryGenerator. This transforms RAG retrieval from a synchronous blocking process into predictive asynchronous prefetching. On benchmarks such as HotpotQA, it reduces end-to-end latency by 43.5% and Time to First Token (TTFT) by 62.4%, while maintaining answer quality within 1% of synchronous RAG.
Ranking-Free RAG: Replacing Re-Ranking with Selection in RAG for Sensitive Domains: This paper introduces METEORA, a trio consisting of a DPO-trained rationale generator, statistical elbow detection, and a shared-framework Verifier. It replaces the uninterpretable, top-\(k\)-dependent re-ranker in RAG, achieving higher recall, an 80% reduction in evidence volume, and a 4.4× improvement in adversarial robustness across six sensitive domain datasets.
REAL: Resolving Knowledge Conflicts in Knowledge-Intensive Visual Question Answering via Reasoning-Pivot Alignment: This paper proposes the REAL framework, which redefines knowledge conflicts in KI-VQA using "Reasoning-Pivots" (atomic nodes/edges in a reasoning chain that must rely on external evidence for completion). By training a pivot-aware conflict discriminator via RPA-SFT and a training-free contrastive decoding strategy via RPGD, it achieves improvements of +3.8%, +1.6%, and +3.6% on E-VQA, InfoSeek, and A-OKVQA, respectively.
Position: Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspective: This ICML position paper argues that current AI reliability methods (RAG / Self-consistency / RLHF / Agent Memory) only verify explicit knowledge, whereas AI's true power stems from "implicit knowledge"—the \(80-95\%\) of information in training data not formally recorded by humans. The authors propose Knowledge Objects (KOs) as an infrastructure to externalize AI's implicit reasoning into structured products that are human-checkable, verifiable, and endorsable, allowing the cost of a single human verification to yield long-term compounding benefits for a community.
ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards: ReSeek adds a JUDGE action to RL-trained search agents and utilizes BGE-reranker to calculate "ideal judgments" as process rewards. This enables agents to "soft-mask" invalid information and re-query after each retrieval. It also proposes FictionalHot, an anti-contamination benchmark based on fictional entities, achieving an average EM of 0.377 on Qwen2.5-7B, outperforming ZeroSearch by +3.1.
Retriever Portfolios: A Principled Approach to Adaptive RAG: This paper reformulates the "which retriever to choose" problem in RAG as a best-of-\(k\) combinatorial optimization problem. By greedily selecting a complementary size-\(k\) portfolio from 360 candidates offline and training a lightweight contrastive router to dispatch queries to the top-\(\ell\) members online, this approach outperforms both single-retriever baselines and inference-time tuning methods (like Vendi-RAG) across four QA benchmarks, while significantly reducing token and latency costs.
Seeing to Generalize: How Visual Data Corrects Binding Shortcuts: This paper replicates the anomalous phenomenon of "VLMs outperforming their base LLMs on pure text tasks" using a controlled synthetic "color-shape-item" retrieval task. Mechanistic interpretability proves that image training shifts the model's variable binding strategy from "positional shortcuts" to "symbolic matching." This shift is retained upon returning to text-only inputs, increasing OOD retrieval accuracy from 37.2% to 69.5%. A consistent "symbolic/positional ratio increase" is observed across the real Qwen2/2.5/3 families.
Self-Augmenting Retrieval for Diffusion Language Models: By leveraging "tentative predictions" provided simultaneously for all positions during the denoising process of Diffusion Language Models as look-ahead signals, the authors propose SARDI: a training-free and retriever-agnostic dynamic RAG framework. SARDI re-retrieves evidence using uncommitted tokens at each denoising step, outperforming both diffusion and autoregressive training-free retrieval baselines on 5 multi-hop QA benchmarks while achieving up to 8x higher throughput.
Temporal Preference Optimization for Unsupervised Retrieval: This paper proposes TPOUR, which transplants DPO-style preference learning to the temporal dimension of retrieval. This enables unsupervised dense retrievers to prioritize "temporally aligned" documents among semantically similar but misaligned versions, while achieving zero-shot generalization to unseen years using temporal vector interpolation.
Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAG: This paper points out that while existing RAG poisoning attacks can manipulate LLM outputs using a small number of malicious passages, they are not truly stealthy. Successful low-budget attacks inevitably cause the model to focus excessive attention on malicious passages. Consequently, the authors filter out anomalous passages using a Normalized Passage Attention Score (NPAS) and a variance-based AV Filter. Across a setup of 4 datasets × 5 LLMs × 5 attacks, it improves RACC by up to 20% compared to Certified Robust RAG.
Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference: This paper unifies optimizations for modern long-context LLM inference—such as sparse attention, RAG, and compressed context memory—into a four-stage "Prepare Memory → Compute Relevancy → Retrieval → Apply to Inference" memory processing pipeline. It quantitatively demonstrates that this pipeline accounts for 22%-97% of total latency and exhibits highly heterogeneous computational characteristics across stages. Based on these insights, a GPU-FPGA heterogeneous system is proposed: compute-intensive/regular operations remain on the GPU, while memory-intensive/sparse/irregular operations are offloaded to the FPGA. Evaluated on MI210 + Alveo U55C, the system achieves up to 2.2× end-to-end acceleration and a 4.7× reduction in energy consumption.
Understanding LoRA as Knowledge Memory: An Empirical Analysis: The authors perform a systematic empirical audit using PhoneBook and a newly constructed PaperQA benchmark, treating LoRA as a knowledge memory unit that can be independently trained, loaded, and combined. They quantitatively provide full-link design guidelines covering "Rank \(\rightarrow\) Capacity \(\rightarrow\) Efficiency \(\rightarrow\) Multi-module Combination \(\rightarrow\) Complementarity with RAG/ICL."
Vector Linking based on Cross-Model Local Isometric Consistency: This paper introduces the problem of vector linking—discovering object correspondences between embedding clouds produced by two different encoders under black-box constraints. The core observation is that independently trained contrastive learning encoders maintain local isometric consistency (similarity preserved up to a scaling factor) over short distances. Based on this, a multi-view geometric hashing bootstrap framework is proposed, requiring only 15-30 seed pairs to recover 79-90% of overlapping objects.