Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models

Conference: ACL 2025
arXiv: 2408.13533
Code: https://github.com/jinyangwu/NoiserBench
Area: LLM Agent / RAG
Keywords: RAG, noise, retrieval-augmented generation, benchmark, beneficial noise

TL;DR

This paper defines 7 noise types in RAG systems from a linguistic perspective and builds the NoiserBench comprehensive evaluation framework. Through large-scale experiments on 8 LLMs, it discovers that noise can be categorized into harmful noise (counterfactual, supportive, orthographic) and beneficial noise (semantic, datatype, illegal sentence). Remarkably, beneficial noise can improve model accuracy by $1\text}3\%$.

Background & Motivation

Background: RAG is a mainstream method for mitigating LLM hallucinations, enhancing generation by retrieving relevant documents from external knowledge sources. However, in reality, retrieved documents inevitably contain various types of noise.

Limitations of Prior Work: - Existing research on RAG noise only defines 2-3 noise types, which is far from sufficient to cover the complexity of real-world retrieval scenarios. - Prior work defaultly assumes "all noise is harmful", ignoring the potential positive effects of noise. - There is a lack of a systematic noise taxonomy and standardized evaluation benchmarks.

Key Challenge: The types of noise in real retrieval environments are diverse, but researchers' understanding of noise is oversimplified ("noise = harmful"), failing to guide the robustness optimization of actual RAG systems.

Goal: Establish a comprehensive RAG noise taxonomy, quantify the impact of various noise types, and reveal the existence and mechanism of beneficial noise.

Key Insight: Define noise types from a linguistic perspective and employ large-scale experimental validation rather than a priori assumptions to determine the positive and negative effects of noise.

Core Idea: RAG noise is not entirely a "Pandora's box" (harmful); some noise acts as an "Aladdin's lamp" (beneficial) — semantic noise, datatype noise, and illegal sentence noise can actually improve performance by promoting standardized answers and enhancing the model's discriminative ability.

Method

Overall Architecture

Define 7 noise types \(\rightarrow\) Construct NoiserBench (8 datasets \(\times\) 7 noise types) \(\rightarrow\) Evaluate 8 LLMs \(\rightarrow\) Analyze the mechanisms of beneficial/harmful noise.

Key Designs

1. 7-Type Noise Taxonomy (Linguistic Perspective):

   • Beneficial Noise:
     • Semantic Noise (SeN): Off-topic documents with low semantic relevance to the query.
     • Datatype Noise (DN): Mixed data types (e.g., URLs, code mixed into text).
     • Illegal Sentence Noise (ISN): Grammatically incorrect, fragmented sentences.
   • Harmful Noise:
     • Counterfactual Noise (CN): Counterfactual or false information — the most destructive.
     • Supportive Noise (SuN): Highly semantically relevant but containing no answer information.
     • Orthographic Noise (ON): Writing errors such as spelling mistakes, word stretching, etc.
     • Prior Noise (PN): Questions based on incorrect premises.
   • Design Motivation: Categorized into passive (harmful) vs. active (beneficial) practical dimensions to guide the noise processing strategies of actual RAG systems.

2. NoiserBench Construction Pipeline:

   • Step 1: QA Instance Generation — Sourced from existing datasets or generated using ChatGPT.
   • Step 2: Entailment Verification — Using BART-large-MNLI to ensure the evidence supports the answer (\(p \geq 0.8\)).
   • Step 3: Noise Injection — Constructing noisy documents using search engines, Wikipedia dumps, the textnoisr tool, etc.
   • Step 4: Formatting as Multiple-Choice Questions — 4 options (correct answer + 2 counterfactuals + "Uncertain") to facilitate automatic evaluation.
   • Design Motivation: A standardized process ensures controlled introduction and fair evaluation of different noise types.

3. Analysis of Beneficial Noise Mechanisms:

   • Function: Explaining why certain noises are beneficial from the perspective of internal mechanisms.
   • Finding 1: Beneficial noise promotes more standardized answer formats — resulting in more standardized model outputs.
   • Finding 2: Beneficial noise provides clearer reasoning paths — noise acts as a "contrastive signal" helping the model focus on the correct context.
   • Finding 3: Beneficial noise increases the model's confidence in the correct context — similar to the effect of contrastive learning.

Key Experimental Results

Main Results (Llama3-8B-Instruct)

Noise Type	Category	Weighted Average Accuracy	Change vs. Golden Only
Golden Only	-	86.57%	-
+ Counterfactual	Harmful	45.58%	-40.99%
+ Supportive	Harmful	85.37%	-1.20%
+ Orthographic	Harmful	83.99%	-2.58%
+ Semantic	Beneficial	88.73%	+2.16%
+ Datatype	Beneficial	86.91%	+0.34%
+ Illegal Sentence	Beneficial	89.89%	+3.32%

Cross-Model Consistency (Effect of ISN)

Model	Golden Only \(\rightarrow\) +ISN Change
Llama3-8B	+3.32%
Qwen2-7B	+1.65%
Llama3-70B	+0.87%
Mixtral-8x7B	+2.10%
Vicuna-13B	+1.45%

Key Findings

• Counterfactual noise is the most destructive: Dropping accuracy on average by $40\text{52\%$, far exceeding other harmful noises, because models find it challenging to distinguish between correct and incorrect facts.
• Illegal Sentence Noise (ISN) provides the largest and most stable improvement: Consistently improving performance by \(1\text{--}3\%\) across 8 models and 7 datasets, making it the strongest beneficial noise.
• The effect of beneficial noise is more pronounced in multi-hop reasoning: On 2WikiMQA and Bamboogle, the improvement from ISN is as high as \(7.6\%\).
• There is an optimal noise ratio: ISN performs best at a \(50\%\) ratio, while excess noise degrades performance.
• Beneficial noise can even counteract harmful noise: When ISN and CN are introduced simultaneously, the hybrid effect is better than introducing CN alone.

Highlights & Insights

• The finding that "noise can be beneficial" overturns the default assumption in the RAG field: This implies that RAG systems should not simply filter out all noise, but should instead differentiate between noise types. This can be transferred to data augmentation strategies — intentionally injecting an appropriate amount of "harmless noise" may improve model robustness.
• The explanation of illegal sentence noise acting as an "attention calibrator" is highly insightful: Meaningless sentences force the model to pay closer attention to meaningful content, similar to the phenomenon of "white noise improving concentration" in audio. This is transferable to prompt engineering — adding a small amount of irrelevant noise in the context may enhance the model's judgment.
• The linguistic taxonomy of 7 noise types provides a standardized framework for RAG robustness research: Filling a critical gap in the field.

Limitations & Future Work

• Evaluation is limited to multiple-choice formats: The impact of noise on open-ended generation tasks may differ.
• Insufficient study on the interaction effects between different noise types: In reality, various noises coexist, but the experiments primarily test single noise types.
• The mechanism explanation of beneficial noise is not yet profound enough: Attention analysis or probing experiments are required to further validate the causal relationship.
• The impact of retriever quality is not considered: Different retrievers return documents with differing noise distributions.

Related Work & Insights

• vs. Cuconasu et al. (2024): They only defined 3 types of noise, whereas this work expands them to 7 and discovers beneficial noise.
• vs. RobustRAG (Xiang et al., 2024): RobustRAG assumes all noise is harmful and designs defense mechanisms, while the findings of this paper suggest that certain noises should be preserved.
• vs. Self-RAG (Asai et al., 2024): Self-RAG uses special tokens to filter irrelevant retrieval, whereas this paper discovers that "irrelevant" retrieval might actually be beneficial.

Rating

• Novelty: ⭐⭐⭐⭐ The concept of "beneficial noise" is novel, and the 7-type taxonomy is comprehensive.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ 8 models \(\times\) 8 datasets \(\times\) 7 noise types, yielding a massive volume of experiments.
• Writing Quality: ⭐⭐⭐⭐ The analogy of "Pandora's Box vs. Aladdin's Lamp" is vivid, and the paper is clearly structured.
• Value: ⭐⭐⭐⭐ Offers direct guidance for the noise treatment strategies of RAG systems.

Related Papers

• [ICLR 2026] When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation
• [ACL 2025] Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Models
• [ACL 2025] Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
• [ACL 2025] Enhancing Lexicon-Based Text Embeddings with Large Language Models
• [ACL 2025] RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models