Gated Residual Discrepancy for Efficient Bi-Encoder Factuality Verification¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/lzy248/grd
Area: Information Retrieval / RAG
Keywords: factuality verification, bi-encoder, asymmetric entailment, residual discrepancy, adaptive gating
TL;DR¶
To overcome symmetric similarity saturation and contextual noise vulnerability in bi-encoders for factuality verification, GRD introduces a multi-slot dual-view residual discrepancy operator with adaptive gating amplification, matching cross-encoder accuracy (79.99% Macro-F1) while delivering a 23.55ร inference speedup.
Background & Motivation¶
Large language models (LLMs) have achieved remarkable success across diverse generative benchmarks, but their persistent propensity for specious factual hallucinations continues to undermine their deployment in mission-critical applications. Consequently, post-hoc factuality verification against reliable grounding evidence has become an indispensable safeguard. Existing high-accuracy verification pipelinesโwhether powered by LLM-as-a-judge or Natural Language Inference (NLI) cross-encodersโfundamentally rely on joint encoding that concatenates the evidence document and the claim. In fine-grained decompose-then-verify scenarios where generated passages are split into multiple atomic units, lengthy evidence documents must be redundantly encoded for every claim, creating severe throughput bottlenecks and prohibitive computational overhead.
Bi-encoder architectures provide an attractive alternative by decoupling evidence and claim representation, enabling offline pre-computation and caching of evidence embeddings alongside lightweight online interaction. However, standard bi-encoders face two fatal limitations in factuality verification: first, fact verification is inherently an asymmetric entailment problem (evidence entailing a claim does not imply the reverse), causing conventional symmetric metrics (cosine similarity or dot product) to produce similarity saturation where unsupported claims with high lexical overlap receive falsely high scores; second, bi-encoders are severely vulnerable to contextual noise in long documents, where benign syntactic variations and extensive background text drown out subtle factual discrepancies. While prior methods like E-Verify attempt to concatenate symmetric features with differences or rely on dual atomic decomposition of evidence, they inflate preprocessing costs and degrade bi-encoder efficiency advantages.
This paper's angle is: completely discard symmetric similarity features in favor of a pure difference-dominant paradigm, coupled with dynamic in-embedding denoising. Core idea: encode texts into multi-granularity semantic slots via pooling-based multi-head attention, extract dual-view vector differences combining directional and magnitude residuals to model asymmetric entailment, and employ an adaptive gating mechanism to dynamically filter background context noise, enabling highly sensitive and robust verification directly over broad unfragmented evidence.
Method¶
Overall Architecture¶
The GRD framework decouples verification into multi-view encoding and embedding-space residual interaction. Given an unfragmented evidence document \(e\) and a claim \(c\), a shared text encoder with a Pooling-based Multi-Head Attention (PMA) module compresses token representations into \(K\) semantic slot vectors. The evidence slot vectors are pre-computed and cached offline, while incoming claims are encoded on the fly. The GRD operator then processes corresponding slot pairs, extracting complementary directional and magnitude residuals, passes them through an adaptive gating module to suppress irrelevant background noise, applies element-wise max-pooling across slots to capture the strongest factual conflict, and computes the final entailment probability via a lightweight MLP classifier.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Evidence e & Claim c"] --> B["Multi-Granularity Semantic Encoding<br/>Shared encoder extracts K slot vectors"]
B --> C["Dual-View Residual Extraction<br/>Concatenate directional and magnitude residuals"]
C --> D["Adaptive Gating Amplification<br/>Parallel projection and dynamic gating mask"]
D --> E["Slot Max-Pooling & Scoring<br/>Max-pooling isolates strongest local conflict"]
E --> F["Output Entailment Score y_hat<br/>Support / Unsupport verdict"]
Key Designs¶
1. Multi-Granularity Semantic Encoding: eliminating global vector information bottlenecks
Standard bi-encoders compress an entire passage into a single [CLS] token embedding, causing fine-grained local facts to be overshadowed by dominant global semantics in long documents. To expand representational resolution, GRD incorporates a Pooling-based Multi-Head Attention (PMA) module. Given token-level contextual representations \(T_x \in \mathbb{R}^{L \times d}\) extracted from an encoder, PMA maintains \(K\) learnable seed vectors \(S = \{s_1, \dots, s_K\} \in \mathbb{R}^{K \times d}\) as queries to aggregate \(T_x\) across multi-head cross-attention, followed by layer normalization and a feed-forward network to yield \(K\) slot vectors \(H_x \in \mathbb{R}^{K \times d}\), which are individually \(\ell_2\)-normalized into \(\tilde{H}_x = \{\tilde{h}^{(1)}_x, \dots, \tilde{h}^{(K)}_x\}\). Since evidence and claims share the same PMA seeds, corresponding slots are implicitly aligned in embedding sub-spaces, distributing entity, temporal, and action semantics across separate vectors.
2. Dual-View Residual Extraction: explicitly modeling asymmetric entailment and deviation salience Rather than fusing symmetric similarities that suffer from saturation under lexical overlap, GRD relies solely on vector discrepancies. For each slot \(k \in \{1, \dots, K\}\), it extracts two complementary views: $\(r_k = [\delta_k; \mu_k] \in \mathbb{R}^{2d}\)$ where the directional residual \(\delta_k = \tilde{h}^{(k)}_e - \tilde{h}^{(k)}_c\) preserves the geometric orientation of semantic displacement, distinguishing semantic surplus (evidence entailment) from information deficit (unsupported hallucination); the magnitude residual \(\mu_k = |\tilde{h}^{(k)}_e - \tilde{h}^{(k)}_c|\) measures the element-wise absolute deviation intensity, providing a sensitive gauge to differentiate benign syntactic paraphrasing from sharp, high-magnitude factual contradictions.
3. Adaptive Gating Amplification: dynamic in-embedding filtering of contextual noise When matching claims against lengthy evidence, extraneous contextual content and benign syntactic variation naturally introduce non-zero residual shifts. To prevent noise accumulation, GRD projects \(r_k\) into hidden dimension \(d_h\), computing candidate features and a gating mask in parallel: $\(\phi_k = \text{ReLU}(W_f r_k + b_f), \quad g_k = \sigma(W_g r_k + b_g)\)$ The gated residual is obtained via element-wise multiplication followed by layer normalization: \(\tilde{r}_k = \text{LN}(\phi_k \odot g_k)\). For dimensions reflecting benign paraphrasing or extraneous context, \(g_k\) approaches zero to suppress noise; for dimensions indicating severe factual clashes, \(g_k\) approaches one, allowing the discrepancy to pass through and undergo non-linear amplification. This eliminates the need for evidence-side atomic decomposition.
4. Slot Max-Pooling & Scoring: preserving decisive local contradiction signals In factuality verification, a single decisive factual error renders an otherwise correct claim unsupported. Mean pooling risks averaging away subtle conflicts; therefore, GRD performs element-wise max pooling across all \(K\) slots: $\(\bar{r}_j = \max_{k=1}^K \tilde{r}_{k,j}\)$ The aggregated vector \(\bar{r} \in \mathbb{R}^{d_h}\) preserves the most acute local conflict signal, and is passed to a two-layer MLP with sigmoid activation to output the final verification probability: $\(\hat{y} = \sigma(W_2 \text{ReLU}(W_1 \bar{r} + b_1) + b_2)\)$
Loss & Training¶
The framework is optimized end-to-end with binary cross-entropy loss: $\(\mathcal{L} = -\frac{1}{N}\sum_{i=1}^N \left[ y_i \log \hat{y}_i + (1 - y_i) \log(1 - \hat{y}_i) \right]\)$ A two-stage training scheme ensures optimization stability: in Stage 1, the pre-trained language model backbone (BGE-large-en-v1.5) is frozen while only PMA and GRD parameters are trained with Adam at learning rate \(5 \times 10^{-5}\) for 4 epochs; in Stage 2, LoRA adapters (\(r=8\)) are inserted into \(W_Q, W_V\) of the final two Transformer layers, fine-tuning higher-level representations at a reduced learning rate of \(2 \times 10^{-5}\) for an additional 4 epochs with batch size 32.
Key Experimental Results¶
Main Results¶
Macro-F1 (%) across eight factuality verification and NLI benchmarks:
| Model Category | Model | FEVER | SNLI | MNLI | Wiki-EN | Bio-S | Bio-H | WebNLG | Climate | SciFact | Macro-F1 Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| LLM Upper Bound | Qwen2.5-7B | 93.51 | 95.76 | 92.69 | 98.25 | 93.15 | 82.92 | 86.41 | 69.84 | 83.54 | 88.45 |
| Compact Generative | Llama-3.2-3B | 89.59 | 91.73 | 82.44 | 89.07 | 80.82 | 51.91 | 57.97 | 32.79 | 27.14 | 67.05 |
| Compact Generative | Qwen2.5-0.5B | 82.43 | 89.30 | 78.55 | 86.05 | 72.75 | 66.74 | 77.54 | 54.47 | 58.27 | 74.01 |
| Cross-Encoder | MiniCheck-DeBERTa | 89.72 | 90.68 | 84.83 | 91.55 | 87.16 | 76.95 | 59.67 | 48.96 | 52.50 | 75.78 |
| Cross-Encoder | DeBERTa-v3-NLI | 93.46 | 94.23 | 92.93 | 96.57 | 88.30 | 74.68 | 75.65 | 53.68 | 60.73 | 81.14 |
| Cross-Encoder | NLI-RoBERTa | 87.23 | 94.87 | 90.72 | 93.34 | 78.48 | 63.52 | 61.52 | 55.97 | 54.26 | 75.55 |
| Symmetric Bi-Encoder | BERTScore | 33.56 | 33.77 | 34.21 | 36.61 | 37.87 | 25.35 | 23.77 | 32.14 | 39.69 | 33.00 |
| Symmetric Bi-Encoder | ColBERT | 33.60 | 42.39 | 35.52 | 53.08 | 38.00 | 27.20 | 28.24 | 32.14 | 39.69 | 36.65 |
| Prior Bi-Encoder SOTA | E-Verify | 60.87 | 81.23 | 79.22 | 96.97 | 79.15 | 65.08 | 65.29 | 52.81 | 48.97 | 69.95 |
| Proposed Method | GRD (Ours) | 87.42 | 96.17 | 90.43 | 98.15 | 81.59 | 70.85 | 72.05 | 64.54 | 58.69 | 79.99 |
Efficiency and latency comparison on FEVER with BM25 retrieving top-5 evidence candidates:
| Model | Params | Offline FLOPs | Online FLOPs | GPU Latency | Relative Speedup |
|---|---|---|---|---|---|
| Qwen2.5-7B | 7.7B | โ | 12.09 TFLOPs | 50.32 ms | 0.15ร |
| MiniCheck-DeBERTa | 435M | โ | 291.82 GFLOPs | 23.59 ms | 0.33ร |
| DeBERTa-v3-NLI (Baseline) | 184M | โ | 83.52 GFLOPs | 7.77 ms | 1.00ร |
| E-Verify | 114M | 8.13 GFLOPs | 1.11 GFLOPs | 0.13 ms | 59.77ร |
| GRD (Ours) | 342M | 28.69 GFLOPs | 3.96 GFLOPs | 0.33 ms | 23.55ร |
Ablation Study¶
Ablation of directional residual \(\delta_k\), magnitude residual \(\mu_k\), and adaptive gating \(g_k\) (Macro-F1 %):
| Configuration | FEVER | MNLI | Bio-S | WebNLG | Observations |
|---|---|---|---|---|---|
| Full GRD Model | 87.42 | 90.43 | 81.59 | 72.05 | Default configuration |
| w/o Directional Residual \(\delta_k\) | 82.82 | 87.35 | 67.83 | 60.19 | Severe drops on Bio-S (-13.76%) and WebNLG (-11.86%), failing asymmetric entailment |
| w/o Magnitude Residual \(\mu_k\) | 81.13 | 88.52 | 72.63 | 69.04 | Drops 6.29% on FEVER and 8.96% on Bio-S, failing conflict severity scaling |
| w/o Adaptive Gating \(g_k\) | 85.20 | 84.49 | 80.08 | 70.54 | Drops across all datasets (-5.94% on MNLI), failing to filter context noise |
Key Findings¶
- Failure of symmetric metrics: Symmetric baselines (BERTScore 33.00%, ColBERT 36.65%) fail drastically due to similarity saturation under high lexical overlap, confirming that explicit directional discrepancy modeling is mandatory for factuality verification.
- Robustness across evidence context lengths: When scaling Bio-S evidence context from atomic sentences up to 2,000 characters, E-Verify collapses from 79.15% to below 30%, whereas GRD remains stable. The performance gap between GRD and its gating-ablated counterpart widens steadily from 1.51 points to 7.43 points, quantitatively demonstrating the gate's ability to filter accumulating background noise.
- Computational efficiency and scalability: GRD narrows the gap with the strongest cross-encoder (DeBERTa-v3-NLI, 81.14%) to only 1.15 points while reducing online FLOPs from 83.52 G to 3.96 G (23.55ร speedup). Scaling retrieved candidates from 1 to 20 incurs negligible marginal cost due to offline evidence pre-computation.
Highlights & Insights¶
- Difference-dominant paradigm: Proves that fusing symmetric similarity is actively counterproductive in bi-encoder fact checking, and isolating residual vector differences yields sharper discriminative boundaries.
- Complementary dual-view residuals: Directional residuals capture asymmetric entailment polarity (information surplus vs. deficit), while magnitude residuals capture deviation severity, together differentiating benign paraphrasing from real contradictions.
- Practical unfragmented verification: Eliminates the heavy preprocessing burden of evidence-side atomic decomposition, enabling direct verification over broad, natural document passages.
Limitations & Future Work¶
- Weakness on multi-step reasoning: On reasoning-heavy benchmarks like SciFact (58.69% vs. Qwen2.5-7B's 83.54%), compressed vector spaces cannot match the deep cross-attention layers of massive generative models on complex scientific deductions.
- Cascaded verification architectures: GRD is best positioned as a high-throughput, low-latency first-pass filter that resolves the vast majority of direct factual queries, routing only ambiguous, reasoning-intensive claims to heavier LLM judges.
Related Work & Insights¶
- vs Cross-Encoders (MiniCheck / DeBERTa-v3-NLI): Cross-encoders achieve deep joint attention but incur prohibitive \(O(L_e + L_c)^2\) online computation per claim; GRD achieves 98.5% of their accuracy while slashing online computation by over an order of magnitude.
- vs E-Verify: E-Verify degrades severely as evidence context expands due to symmetric similarity contamination; GRD maintains robustness across long evidence via pure residual modeling and adaptive gating.
Rating¶
- Novelty: โญโญโญโญ [Pioneers a difference-dominant paradigm with dual-view residuals and in-embedding adaptive gating for bi-encoder fact checking]
- Experimental Thoroughness: โญโญโญโญโญ [Extensive evaluations across 8 benchmarks, context stress tests up to 2000 chars, efficiency scaling, and matched-capacity fine-tuning]
- Writing Quality: โญโญโญโญโญ [Clear structural narrative, rigorous mathematical definitions, and comprehensive ablation analyses]
- Value: โญโญโญโญโญ [Substantially lowers the computational barrier for fine-grained factuality verification in high-throughput RAG pipelines]