Skip to content

Why and When to Use Statistical Significance in Explanation Evaluation: A Systematic Analysis

Conference: EMNLP 2026
Paper: ACL Anthology
Area: interpretability
Keywords: explanation evaluation, statistical significance testing, proportional rigour, faithfulness, hypothesis testing

TL;DR

Systematically auditing 494 verified explainable NLP papers across the ACL Anthology (2020โ€“2025) reveals a persistent rigour gap where only 25.30% substantiate evaluation claims with statistical significance, motivating a risk-proportionate bipartite mapping framework between claims and testing regimes.

Background & Motivation

With the widespread deployment of large language models and agentic frameworks in high-stakes domains such as clinical diagnosis, legal outcome prediction, financial risk analysis, and automated fact-checking, explainable AI (XAI) has transitioned from an auxiliary visualization tool to a decisive foundation for model justification, human decision support, and regulatory compliance. Over recent years, the NLP community has formulated various evaluation dimensions, most notably faithfulness (whether explanations reflect internal model decision mechanisms) and plausibility (whether explanations align with human cognitive expectations). However, methodological practices for evaluating these properties remain severely fragmented: many studies still rely on qualitative case demonstrations or aggregate metric summaries without measures of uncertainty, rarely justifying their evaluation protocols relative to the claims being asserted.

This methodological heterogeneity incurs acute scientific and ethical risks. When researchers make comparative claims asserting that a proposed explanation method surpasses baselines, they implicitly assert that the observed differences are systematic and generalizable across populations. However, explanation evaluations are inherently susceptible to three primary noise sources: test set sampling variance, stochastic model optimization and initialisation, and severe evaluation instability arising from input perturbations and human rater subjectivity. Without rigorous statistical hypothesis testing and uncertainty quantification, point-estimate differences risk facilitating "explanation-washing" or "fairwashing"โ€”packaging stochastic experimental noise as systematic algorithmic superiority and inducing misplaced trust in vulnerable models.

While general NLP performance evaluation has benefited from established statistical testing guidelines, analogous standards for interpretability research remain absent. This paper addresses why explanation evaluation necessitates statistical significance testing and when such testing must be applied. Core idea: Conduct a systematic empirical audit of 494 explainable NLP papers to uncover a persistent rigour gap, and establish a bipartite mapping framework grounded in proportional rigour that systematically connects research claims, data dependency structures, and appropriate statistical testing regimes from exploratory analysis to high-stakes deployment.

Method

Overall Architecture

The paper designs an end-to-end systematic audit pipeline and a dual-tree bipartite decision framework for explainable NLP. The audit procedure extracts 2,009 candidate papers from the ACL Anthology via rule-based regex filtering, filters them with GPT-4o for XAI focus and explicit explanation evaluation, refines them to main conference and Findings venues (593 candidates), and conducts 100% manual full-text examination to yield a verified corpus of 494 papers. Drawing upon empirical findings, the authors construct a bipartite mapping framework that connects explainability claims to statistical testing regimes through the mediating structural properties of evaluation signals.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Corpus and Evaluation Objectives Input"] --> B["Systematic Literature Audit and Rigour Gap Diagnosis"]
    B --> C["Six-Tier Claim Hierarchy and Proportional Rigour Scaling"]
    C --> D["Signal Dependency Structure and Test Model Bipartite Mapping"]
    D --> E["Formal Protocol Verification and Reporting Output"]

Key Designs

1. Systematic Literature Audit and Rigour Gap Diagnosis: Methodological Heterogeneity -> Five-Stage Screening Pipeline with Gap Attribution -> Establishing Empirical Boundaries Explainable NLP research frequently conflates general task performance testing with explanation-specific testing, obscuring the actual state of scientific reproducibility. To resolve this, the authors establish a rigorous five-stage screening pipeline: Stage-I rule-based filtering captures 2,009 papers from the ACL Anthology; Stage-II utilizes GPT-4o for binary classification of XAI focus and explanation evaluation (achieving 94.1% raw agreement and 88.2% PABAK against manual verification across 593 papers); Stage-III and Stage-IV filter for main conference and Findings tracks (593 papers); Stage-V executes 100% full-text human review to establish whether statistical tests substantiate explanation metrics rather than task performance. For the 80 unvalidated papers analyzed in depth, three substantive rigour gaps are identified: Implicit Generalisation (46.25%), where competitive superiority is claimed without testing whether differences exceed experimental noise; Inferential Reporting Gap (22.50%), where rank correlations (Spearman/Pearson) are reported without corresponding p-values; and Incomplete Rigour (21.25%), where significance tests are run for task performance (Accuracy/F1) but omitted for explanation quality.

2. Six-Tier Claim Hierarchy and Proportional Rigour Scaling: Evidential Mismatch -> Downstream Risk-Proportionate Evidentiary Burden -> Graded Statistical Mandates Prior literature either dismissed exploratory interpretability or tolerated unsubstantiated high-stakes claims. The framework introduces the principle of proportional rigour, asserting that the required statistical burden must scale with downstream societal risk across six primary claim categories: - Illustrative Insight: Qualitative inspection of individual prediction instances for exploratory understanding; statistical testing is Not required; - Models Comparison (M): Benchmarking explanation methods or model variants on aggregate signals; statistical testing is Recommended; - Robustness (R) & Stability/Identity (S): Asserting consistency across perturbations, random seeds, or distributions; statistical testing is Required; - Bias Detection (B): Providing evidence of reliance on demographic or spurious attributes; given direct fairness implications, statistical testing is Strongly required; - Decision Support (H): Demonstrating improved user trust, usability, or human-AI task performance; given high rater variance, statistical testing is Required; - Deployment Readiness (D): Justifying real-world deployment or legal/regulatory compliance in high-stakes settings; statistical testing is Mandatory.

3. Signal Dependency Structure and Test Model Bipartite Mapping: Statistical Assumption Violations -> Signal Decoupling and Data Dependence Alignment -> Principled Hypothesis Testing Regimes Explanation evaluation signals violate standard independent and identically distributed (i.i.d.) assumptions due to instance-level pairing and repeated human judgments. The bipartite mapping pairs each evaluation signal with statistical tests tailored to its specific data structure: - For model comparisons on faithfulness curves (e.g., AOPC or AUC under deletion/insertion), metrics are paired across instances; recommended regimes include paired permutation tests or Wilcoxon signed-rank tests on per-instance AUC, with cluster bootstrap confidence intervals, and the Friedman test followed by Nemenyi post-hoc analysis for \(\ge 3\) methods; - For fairness disparities and demographic attribution comparisons, data structures consist of independent or grouped samples with covariates; recommended regimes include Welch's t-test, Mannโ€“Whitney U test, or ANOVA/Kruskalโ€“Wallis with multiple-testing corrections (FDR); - For human trust, simulatability, and binary task success, repeated measurements across subjects and items induce complex nested variance; recommended regimes center on Generalized Linear Mixed Models (GLMMs) with logistic link functions, incorporating fixed effects for conditions alongside random intercepts and random slopes for subjects and items to disentangle explanation utility from subject variability.

Formal Definition and Evaluation Protocol

Let \(f_\theta: \mathcal{X} \to \mathcal{Y}\) represent a trained model mapping input \(x\) to prediction \(y = f_\theta(x)\). An explanation method is defined as a parameterised function: $\(E_\phi: \mathcal{X} \times \mathcal{Y} \to \mathcal{Z}\)$ where \(\mathcal{Z}\) denotes the explanation domain (e.g., feature attributions, natural language rationales, or discrete tokens). An evaluation protocol is defined as \(\mathcal{M} = (\mathcal{A}, \mathcal{T})\), comprising behavioural assumptions \(\mathcal{A}\) and test procedure \(\mathcal{T}\).

Faithfulness requires satisfying necessary conditions under \(\mathcal{M}\): removing explanation-identified salient features \(x_{-k}\) must induce substantial degradation \(|f_\theta(x) - f_\theta(x_{-k})| \gg 0\), whereas removing non-salient features leaves predictions invariant. Robustness under input perturbation \(\delta\) (where \(f_\theta(x+\delta) \approx f_\theta(x)\)) requires: $\(\mathcal{D}(E_\phi(x, y), E_\phi(x+\delta, y')) \le \epsilon\)$ In formal hypothesis testing, the null hypothesis posits zero true treatment effect: \(H_0: \Delta_{\text{metric}} = 0\). Statistical tests evaluate whether observed metric differentials exceed the stochastic noise envelope under explicit distributional and dependency assumptions, ensuring that claimed model properties reflect systematic algorithmic realities.

Key Experimental Results

Main Results

Longitudinal trends and venue breakdown across 494 verified explainable NLP papers from ACL Anthology (2020โ€“2025) subjected to 100% full-text manual review.

Year Total Evaluated Papers Papers with Significance Testing Testing Ratio (%) Main Conference Verified (w/ SigTest) Findings Verified (w/ SigTest)
2020 31 7 22.58% 27 (6) 4 (1)
2021 53 18 33.96% 48 (14) 5 (4)
2022 65 17 26.15% 45 (14) 20 (3)
2023 91 20 21.98% 55 (9) 36 (11)
2024 106 27 25.47% 65 (22) 41 (5)
2025 148 36 24.32% 89 (27) 59 (9)
Full Corpus Total 494 125 25.30% 329 (92) 165 (33)

Ablation Study

Breakdown of underlying rigour gaps identified through targeted re-analysis of 80 papers that evaluate explanations without statistical significance testing.

Rigour Gap Category Sample Count (N=80) Proportion (%) Primary Manifestation & Methodological Root Cause
Implicit Generalisation 37 46.25% Competitive superiority asserted via point estimates without measuring noise margin
Inferential Reporting Gap 18 22.50% Correlation coefficients reported (Spearman/Pearson) while omitting p-values and FDR corrections
Incomplete Rigour 17 21.25% Significance testing conducted for downstream task accuracy/F1 but omitted for explanations
Illustrative Insights 8 10.00% Purely exploratory case studies with small samples where formal statistical inference is inapplicable

Key Findings

  • Stagnant Rigour amid Explosive Growth: Although annual explainable NLP publication volume expanded nearly fivefold between 2020 and 2025 (from 31 to 148 papers), the proportion of studies substantiating claims with significance testing remained flat between 21% and 34%, averaging 25.30% across the full corpus.
  • Extreme Disparity across Explanation Dimensions: Among the 125 substantiated papers, significance testing is predominantly concentrated in Plausibility evaluations (70.40%, 88/125 papers), followed by Faithfulness (43.20%, 54/125 papers), while downstream Utility accounts for only 26.40% (33/125 papers).
  • Venue Discrepancy: Main conference papers consistently provide a higher and more stable absolute volume of significant testing (n=92) compared to Findings (n=33), though both tracks fail to exceed a 30% overall testing adoption rate.

Highlights & Insights

  • Delivers the first comprehensive multi-year empirical meta-survey (494 full-text verified papers, 2020โ€“2025) rigorously auditing statistical significance practices in explainable NLP, uncovering that fewer than one in four papers provide statistical backing for explanation claims.
  • Formulates the principle of proportional rigour, offering a realistic framework that scales evidential requirements with downstream decision risks rather than demanding universal hypothesis testing.
  • Establishes a concrete bipartite decision map linking explainability claims to data dependence structures, identifying paired permutation tests and GLMMs as essential paradigms for multi-annotator and perturbation-based XAI evaluation.

Limitations & Future Work

  • Lexical regex filtering at the title stage may exclude relevant model analysis and representation probing papers that omit explicit XAI terminology in their titles.
  • The 25.30% testing rate tracks the mere presence of statistical tests without evaluating their statistical appropriateness (e.g., failure to account for multiplicity or improper application of parametric tests to non-normal distributions), serving as an optimistic upper bound.
  • The majority of papers (91.7%, 453/494) were single-annotated, and double-annotated validation (41 papers) exhibited the "Kappa Paradox" (78.0% raw agreement vs. -0.082 Cohen's \(\kappa\), corrected to 0.561 PABAK) due to severe class imbalance.
  • vs Dror et al. (2018): Dror et al. established foundational guidelines for statistical significance in general NLP model comparisons; this work extends testing principles specifically to the complex dependency and perturbation structures of XAI evaluation.
  • vs Jacovi and Goldberg (2020): Jacovi and Goldberg formalized the conceptual separation between faithfulness and plausibility; this paper advances the field by establishing when and how claims within these dimensions require inferential statistical substantiation.

Rating

  • Novelty: โญโญโญโญโ˜† Pioneers large-scale empirical quantification of the statistical rigour gap in NLP interpretability evaluation.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive manual full-text audit across 494 peer-reviewed papers spanning six publication years.
  • Writing Quality: โญโญโญโญโญ Lucid conceptual exposition with clear taxonomy and highly actionable bipartite mapping guidelines.
  • Value: โญโญโญโญโญ Essential methodological compass for combating explanation-washing and grounding trustworthy AI in empirical evidence.