Skip to content

eTracer: Polarity-Aware Evidence Grounding

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/chubohao/eTracer
Area: Information Retrieval & RAG
Keywords: Evidence Grounding, Polarity Awareness, Retrieval-Augmented Generation, Fact Verification, Sentence-Level Alignment

TL;DR

This paper formalizes the task of Polarity-Aware Evidence Grounding and introduces ETRACER-BENCH, the first human-annotated benchmark spanning 7,905 sentence-level evidence pairs with three-way polarity labels, alongside ETRACER, an encoder-based post-hoc framework that jointly recovers fine-grained evidence localization and signed polarity in a single forward pass under full self-attention, matching a 235B LLM at 1/600 the parameter scale.

Background & Motivation

Retrieval-augmented generation (RAG) grounds generated outputs in retrieved documents, substantially curbing factual hallucination in large language models. However, in high-stakes domains such as evidence-based medicine or multi-source intelligence analysis, verifying whether generated responses are genuinely truthful remains challenging. When retrieved documents contain conflicting, outdated, or disputed assertions, human auditors must invest heavy cognitive effort into verifying whether each individual sentence is backed by authoritative evidence, unsupported, or directly contradicted by the retrieved context. Current attribution and fact-checking methods consistently fail to satisfy two essential properties simultaneously: verification granularity and verification completeness.

In terms of granularity, existing systems typically operate at passage or document levels, forcing users to scan lengthy source text to verify a single statement; conversely, decomposing responses into atomic claims frequently introduces segmentation errors and produces evaluation units misaligned with natural reading workflows. In terms of completeness, dominant paradigms remain strictly support-only or collapse pairwise scores into an aggregate consistency metric, improperly conflating unsupported claims with actively contradicted assertions. While natural language inference (NLI) models can categorize polarity on isolated sentence pairs, applying pairwise NLI to RAG presupposes that relevant evidence has already been localized, and inherently discards two critical cross-sentence signals: the contrastive cue between co-occurring supporting and contradicting sentences of similar surface form, and cross-sentence evidence aggregation where partial information across multiple context sentences jointly entails a claim or implies a contradiction.

To overcome these dual limitations, this paper establishes the task of polarity-aware evidence grounding at sentence-to-sentence granularity. Core idea: formalize polarity-aware grounding as signed evidence matrix recovery over response-context sentence pairs, exploiting transformer encoder self-attention to jointly localize evidence and classify three-way polarity via boundary-token representations in a single forward pass.

Method

Overall Architecture

Given a generated response \(R = \{r_i\}_{i=1}^n\) segmented into \(n\) sentences and its retrieved context \(S = \{s_j\}_{j=1}^m\) segmented into \(m\) sentences, polarity-aware evidence grounding learns a mapping function \(\phi: R \times S \to \{+1, -1, 0\}\). For every pair \((r_i, s_j)\), \(\phi\) assigns a signed evidential label: \(+1\) if \(s_j\) supports \(r_i\), \(-1\) if \(s_j\) contradicts \(r_i\), and \(0\) if \(s_j\) is irrelevant to \(r_i\). Applying \(\phi\) over all \(n \times m\) pairs yields a signed evidence map \(\Phi \in \{+1, -1, 0\}^{n \times m}\).

The central insight of ETRACER is the structural correspondence between the \(n \times m\) signed output matrix and the bidirectional self-attention computation already performed by a transformer encoder over the concatenated sequence. Rather than performing \(n \times m\) independent forward passes, ETRACER concatenates the entire response and all retrieved context sentences into a single input sequence under ModernBERT-large's native 8,192-token context window. Learned boundary tokens [R_SEP] and [S_SEP] demarcate individual sentences, and a lightweight MLP readout gathers boundary representations to predict the full signed evidence map in a single pass.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Concatenated Input Sequence<br/>[CLS] Response Sentences [SEP] Context Sentences"] --> B["Context-Aware Signed Scorer<br/>Full Bidirectional Attention + Boundary Token Readout"]
    B --> C["Majority-Class Loss Reweighting<br/>Heavily Penalizing Irrelevant Class to Sharpen Boundary"]
    C --> D["Document-Order Augmentation & Dual Views<br/>8 Shuffled Permutations + Source-Type Filtering"]
    D --> E["Output Signed Evidence Map nร—m<br/>Single Forward Pass Joint Localization & Polarity"]

Key Designs

1. Context-Aware Signed Scorer: recovering full evidence localization and polarity in a single forward pass

To overcome the fundamental blindness of pairwise scoring to cross-sentence contrast and evidence synthesis, ETRACER feeds the full response and complete retrieved context into the encoder simultaneously. Sentences are delineated by learned special boundary tokens [R_SEP] and [S_SEP] (fine-tuned from [SEP]), with the native [SEP] separating the response and context blocks:

\[[CLS] \; r_1 \; [R\_SEP] \dots r_n \; [R\_SEP] \; [SEP] \; s_1 \; [S\_SEP] \dots s_m \; [S\_SEP]\]

Under full bidirectional self-attention, every boundary token's hidden state (\(h_{r_i}\) and \(h_{s_j}\)) contextualizes its sentence against all other sentences in the input. For any response-context sentence pair \((r_i, s_j)\), its polarity probability vector is computed directly by concatenating their boundary representations through a shared classification MLP:

\[p(\cdot \mid r_i, s_j) = \text{softmax}\left(\text{MLP}\left([h_{r_i}; h_{s_j}]\right)\right) \in \mathbb{R}^3\]

Taking \(\arg\max\) during inference recovers the signed label. Stacking these predictions over all \(n \times m\) pairs yields the full signed evidence map \(\Phi\) in a single forward pass, collapsing inference cost from approximately 73 passes to 1 while enabling the encoder to detect contrastive surface discrepancies between conflicting documents.

2. Majority-Class Loss Reweighting: sharpening non-evidence rejection boundaries

In realistic RAG verification, the pair-level label distribution is heavily skewed toward irrelevant sentences: irrelevant pairs account for 84.6% of all pairs, whereas supporting pairs constitute 8.9% and contradicting pairs only 6.5%. Conventional wisdom for imbalanced classification recommends up-weighting minority classes; however, in fine-grained evidence localization, doing so aggressively inflates false positives by classifying distractor sentences with topical overlap as supporting or contradicting.

ETRACER adopts a counterintuitive yet empirically optimal weighting strategy. The model is optimized over all \(n \times m\) pairs via class-weighted cross-entropy loss:

\[\mathcal{L} = \sum_{i=1}^n \sum_{j=1}^m w_{y_{ij}} \text{CE}(\hat{y}_{ij}, y_{ij})\]

where class weights are set to \([w_{\text{sup}}, w_{\text{irr}}, w_{\text{con}}] = [1, 10, 1]\). By heavily up-weighting the dominant irrelevant class tenfold, the model prioritizes reliable non-evidence rejection. This sharpens the localization boundary against distractor sentences while preserving strong gradient updates on the small population of true evidence pairs.

3. Document-Order Augmentation & Dual Views: eliminating positional shortcuts and enforcing cross-sentence reasoning

Presenting documents in static concatenation order risks inducing superficial positional shortcuts, such as assuming that adversarial contradictions always appear at the end of the context. ETRACER introduces a dual-strategy offline data augmentation pipeline during training: document-order shuffling and document-type filtering. Shuffling applies 8 random permutations to the order of retrieved documents for each response while strictly preserving within-document sentence order and response text; type filtering generates two separate views containing natural-only retrieved documents and synthetic-only adversarial documents.

This expands the 490 training responses to 5,386 instances (\(\sim 11\times\)). While pairwise models cannot benefit from document-order permutations because they process isolated sentence pairs independently, ETRACER's global attention mechanism is compelled by shuffling to ignore superficial concatenation order and focus on true semantic and evidential dependency across documents.

A Worked Example

Consider a biomedical QA instance: response sentence \(r_1\) states "Drinking water before bed improves airway stability and reduces sleep apnea." The retrieved context includes Document A (lifestyle recommendations), Document B (alcohol impacts), and an adversarial clinical excerpt Document C (sleep medicine trial). 1. Concatenation & Encoding: The response and all documents are concatenated into a single sequence within the 8,192-token window and encoded by ModernBERT under full bidirectional self-attention; 2. Representation Gathering: The model gathers boundary representations \(h_{r_1}\), \(h_{s_{A2}}\) ("Drinking some water before bed helps keep airway moist"), and \(h_{s_{C4}}\) ("Excessive fluid intake immediately prior to sleep exacerbates pharyngeal edema in obstructive apnea"); 3. Classification Readout: The MLP heads score \([h_{r_1}; h_{s_{A2}}]\) as \(+1\) (Support) and \([h_{r_1}; h_{s_{C4}}]\) as \(-1\) (Contradict), while all distractor context sentences are assigned \(0\) (Irrelevant) driven by the tenfold rejection penalty; 4. Output Matrix: The model recovers the complete row for \(r_1\) in the signed matrix in one shot, capturing both positive support and critical clinical contradiction.

Loss & Training

Trained with AdamW optimizer at peak learning rate \(2 \times 10^{-5}\), with linear warmup over the first 3% of steps followed by linear decay to zero, weight decay 0.01, classification head dropout 0.15, gradient clipping at 1.0, and bf16 mixed precision. Gradient checkpointing is enabled to fit 8,192-token sequences on a single 48GB NVIDIA A6000 GPU with per-device batch size 2 and 8-step gradient accumulation (effective batch size 16). Training runs up to 35 epochs with early stopping (patience 5) based on dev split Citation F1.

Key Experimental Results

Main Results

Evaluated on the held-out test split of ETRACER-BENCH (140 responses, 348 response sentences, 3,365 context sentences, 10,233 evaluated pairs). Metrics include Support F1 (S.F1), Contradict F1 (C.F1), and their macro average Citation F1 (Cit.F1).

Category Method Params Forward Passes Support P Support R Support F1 Contradict P Contradict R Contradict F1 Citation F1
Prompt GPT-4o (0-shot) โ€” 1* 88.60 68.96 77.56 91.02 48.64 63.40 70.48
Prompt GPT-4o (7-shot) โ€” 1* 85.39 71.95 78.10 87.31 64.96 74.50 76.30
Prompt Qwen3-235B (0-shot) 235B 1* 79.18 72.95 75.94 92.42 70.24 79.82 77.88
Prompt Qwen3-235B (7-shot) 235B 1* 78.80 77.49 78.14 89.51 81.92 85.55 81.85
Fine-tuned DeBERTa-v3 (MNLI) 435M โ‰ˆ73 66.03 34.48 45.30 52.15 69.92 59.74 52.52
Fine-tuned BART-large (MNLI) 406M โ‰ˆ73 69.84 34.15 45.87 40.47 68.32 50.83 48.35
Fine-tuned DeBERTa-v3 (ETB) 435M โ‰ˆ73 71.51 73.73 72.60 74.03 76.64 75.31 73.96
Fine-tuned BART-large (ETB) 406M โ‰ˆ73 74.27 70.40 72.28 71.75 77.60 74.56 73.42
Fine-tuned \(M_{\text{pair}}\) (ETB) 395M โ‰ˆ73 74.70 70.07 72.31 74.11 66.40 70.04 71.18
Fine-tuned ETRACER (\(M_{\text{con}}\)) 395M 1 84.10 68.63 75.58 90.59 87.84 89.20 82.39

Note: 1 denotes a single LLM prompt call; \(M_{\text{pair}}\) shares ETRACER's backbone (ModernBERT-large), optimization parameters, and loss objective, serving as the direct controlled ablation for context-aware scoring.*

Ablation Study

Test-time selective attention masking on the identical trained ETRACER checkpoint isolates the exact contribution of each cross-sentence information flow (model parameters and decoding remain unchanged; only the attention mask varies):

Test-Time Attention Masking Config Support F1 Contradict F1 Citation F1 \(\Delta\) (vs. Full) Note
ETRACER (Full model) 75.58 89.20 82.39 โ€” Full bidirectional attention
Mask context cross-attn 57.47 75.58 66.53 -15.86 Context sentences cannot attend across sentences
Mask response cross-attn 73.78 85.22 79.50 -2.89 Response sentences cannot attend across sentences
Mask both 60.04 67.56 63.80 -18.59 Context and response cross-sentence attention masked
Context Shuffling (\(M_{\text{con}}\) shuffled) 70.42 70.35 70.39 -12.00 Context sentences shuffled at test time (converges to \(M_{\text{pair}}\))

Breaking down contradiction difficulty, ETRACER attains 85.41 F1 on implicit contradictions, dropping only 9.77 points from explicit contradictions (95.18). This gap is narrower than that of Qwen3-235B (\(\Delta = -9.83\)) and substantially more robust than the pairwise baseline \(M_{\text{pair}}\) (\(\Delta = -22.91\)).

Key Findings

  • Context-awareness drives contradiction detection: Holding backbone, data, and loss constant, ETRACER outperforms the matched pairwise baseline \(M_{\text{pair}}\) by +19.16 Contradict F1 (89.20 vs. 70.04; non-overlapping 95% bootstrap CIs). Ablation reveals that masking context-side cross-attention drops Support F1 by 18.11 and Contradict F1 by 13.62, showing that contradiction relies heavily on comparing co-occurring sentences of similar surface form, while support detection relies on multi-sentence evidence aggregation.
  • Specialized compact models match massive LLMs: With 395M parameters, ETRACER achieves a Citation F1 of 82.39, statistically on par with Qwen3-235B 7-shot (81.85) and outperforming GPT-4o 7-shot (76.30), demonstrating that polarity-aware grounding can be solved efficiently with targeted small encoder architectures rather than general-purpose LLM scaling.
  • User study confirms a false-completeness bias in support-only grounding: A pre-registered within-subjects user study (\(n=30\)) demonstrates that polarity-aware grounding (E) doubles contradiction detection accuracy over the document baseline (D) (51.9% vs. 27.9%, \(d=0.82, p=0.0003\)). Crucially, support-only grounding (S) yields numerically lower contradiction identification (23.4%) than no grounding at all (27.9%), revealing that highlighting only supporting passages induces a false sense of completeness that misleads users into overlooking conflicts.

Highlights & Insights

  • Single-pass full-matrix inference: Mapping the \(n \times m\) output structure directly onto transformer encoder boundary-token self-attention eliminates redundant forward passes and enables global cross-sentence comparison in a unified mechanism.
  • Majority-class up-weighting trick: Counterintuitively penalizing the 84.6% irrelevant majority class tenfold sharpens the distractor rejection boundary, demonstrating that reliable evidence grounding depends primarily on filtering non-evidence rather than inflating minority-class recall.
  • Broad applicability: The boundary-token Cartesian readout paradigm is readily transferable to cross-document coreference resolution, multi-hop reasoning chain tracing, and clinical/legal conflict auditing.

Limitations & Future Work

  • Long-context limits beyond native window: While ModernBERT's 8,192-token capacity accommodates all benchmark instances, scaling to dozens of retrieved documents requires sliding-window extensions. Although sliding windows at \(T=1,024\) preserve Citation F1 within 0.8 points of the native result, cross-window dependencies remain partially degraded.
  • Scarcity of naturally occurring contradictions: Natural conflicts in the benchmark comprise 94 pairs (24 in the test split), with the remainder constructed adversarially. While ETRACER achieves 75% accuracy (18/24) on natural test conflicts (vs. 1/24 for \(M_{\text{pair}}\)), expanding genuine cross-document conflict corpora in domains like evidence-based medicine remains vital.
  • Pragmatic contradiction granularity: The current formulation focuses on three-way polarity (support, contradict, irrelevant), without explicitly modeling more nuanced pragmatic phenomena such as temporal deprecation, regulatory supersession, or divergent evidence strength.
  • vs ALCE / HAGRID / FActScore / MiniCheck: Prior RAG evaluation methods either operate at coarse passage levels or decompose responses into atomic claims, while remaining strictly support-only. ETRACER adopts the natural sentence unit and introduces polarity awareness.
  • vs LongCite / TracSum: LongCite and TracSum achieve sentence-to-sentence citations but exclusively ground supporting evidence. ETRACER preserves fine granularity while resolving both support and contradiction.
  • vs SummaC / AlignScore: Consistency scoring tools compute internal pairwise NLI matrices but collapse them into a single scalar consistency score, discarding localization. ETRACER recovers both fine-grained localization and signed polarity in a single structure.

Rating

  • Novelty: โญโญโญโญโญ [Pioneers sentence-to-sentence polarity-aware grounding and unifies output matrix recovery with encoder self-attention]
  • Experimental Thoroughness: โญโญโญโญโญ [Extensive 7-domain benchmark, rigorous matched ablations, test-time attention masking, and a pre-registered human user study]
  • Writing Quality: โญโญโญโญโญ [Exemplary clarity, mathematically self-consistent formulation, sharp insight into the false-completeness bias]
  • Value: โญโญโญโญโญ [Resolves a foundational blind spot in RAG attribution, providing an efficient audit mechanism for high-stakes AI applications]