Skip to content

RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Nexround/RACE
Area: LLM Reasoning
Keywords: Mechanistic Interpretability, Neuron Functional Consistency, Residual-Direction Alignment, Bayesian Uncertainty Estimation, Targeted Suppression

TL;DR

Addressing the excessive computational cost and sample-to-sample variance oversimplification in existing mechanistic interpretability, this paper introduces RACE, a forward-pass statistical framework that decomposes module residual updates and leverages Normal-Inverse-Gamma conjugate Bayesian modeling to rank domain-functional neurons with extreme scalability and high intervention specificity.

Background & Motivation

Large Language Models exhibit impressive capabilities across code generation, mathematical reasoning, and multi-turn planning, yet deciphering their internal mechanisms remains a fundamental challenge in mechanistic interpretability. Prevailing techniques—ranging from causal tracing and activation patching to gradient-based attribution and dictionary learning via sparse autoencoders—predominantly focus on instance-level analyses. While highly capable of explaining model decisions on isolated inputs, real-world deployment needs, such as domain-specific model pruning, safety capability auditing, and fine-grained behavioral steering, demand population-level characterizations of neuron functions across an entire target domain.

Existing task-neuron mapping methodologies face a severe dual bottleneck: computational inefficiency and statistical oversimplification. Gradient-based attribution and iterative causal interventions require backpropagation across thousands of tokens, incurring prohibitive computational overhead that severely hinders domain-wide neuron auditing on 10B+ scale models. Concurrently, prior approaches typically aggregate neuron contributions across observations via simple empirical averaging. This scalar averaging discards cross-sample distributional dynamics, inadvertently rewarding bursty "outlier neurons" that fire violently on a few idiosyncratic prompts but remain silent or antagonistic across the broader target domain, thereby polluting domain-specific rankings.

Because Transformer computation is mediated by incremental additive writes to the residual stream, neurons serving consistent domain-specific functions must systematically contribute positive, low-variance alignment along the host module's update trajectory during the forward pass. The core idea is to evaluate neuron functional consistency at forward-pass cost via Residual-Direction Alignment (RDA) onto the host module's update axis, distill noisy per-observation signals into posterior mean and variance distributions through a Normal-Inverse-Gamma (NIG) conjugate model, and apply Conservative Alignment Magnitude (CAM) lower credible bounds alongside Reference-Set Filtering (RSF) to isolate high-fidelity, domain-specific neurons.

Method

Overall Architecture

The RACE framework operates in four cohesive stages: observation collection across target and reference sets, module-local evidence extraction via Residual-Direction Alignment (RDA), closed-form Bayesian evidence aggregation using an NIG conjugate model, and candidate selection governed by Conservative Alignment Magnitude (CAM) combined with Reference-Set Filtering (RSF). Operating strictly during standard forward passes, the pipeline maintains \(O(1)\) streaming sufficient statistics per neuron, computes analytic Student's t marginal posteriors, penalizes noisy or sparse observations through a conservative credible lower bound, and filters out general-purpose linguistic infrastructure before downstream intervention.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Target & Reference Observation Sets<br/>D_c and D_ref with active token positions T_c"] --> B["Residual-Direction Alignment RDA<br/>Module-local residual decomposition and unit projection"]
    B --> C["Bayesian Conjugate Evidence Aggregation<br/>Online sufficient statistics and Student's t marginal posterior"]
    C --> D["Conservative Alignment Magnitude CAM Scoring<br/>One-sided credible lower bound penalizing variance & noise"]
    D --> E["Reference-Set Filtering RSF<br/>Excluding shared general neurons via reference ranking"]
    E --> F["Isolated Domain-Specific Neurons<br/>Suppression intervention & fine-grained behavioral steering"]

Key Designs

1. Residual-Direction Alignment: Forward-Only Geometric Attribution to Local Residual Updates

Aligning intermediate neurons directly against the final token logits conflates local functional contributions with network depth and downstream non-linear transformations, as intermediate representations are frequently rotated, overwritten, or erased by subsequent layers. RDA addresses this by evaluating each neuron against its host module's own residual update \(\Delta r_{l,m}\). The module output is decomposed into an exact sum of rank-1 neuron writes: $\(\Delta r_{\mathrm{ATTN}} = \sum_i h_i W_O[:, i], \qquad \Delta r_{\mathrm{MLP}} = \sum_j a_j W_{\mathrm{down}}[:, j]\)$ Normalizing this update defines the observation-specific unit axis \(\hat{d} = \Delta r / (\|\Delta r\|_2 + \epsilon)\). Projecting each neuron's vector write \(v_j\) onto \(\hat{d}\) produces the activation-modulated signed alignment score: $\(e_{\mathrm{MLP}, j}^{(t)} = a_j^{(t)} \cdot (W_{\mathrm{down}}[:, j]^\top \hat{d}_{\mathrm{MLP}}^{(t)})\)$ Geometrically, \(v_j = e_j \hat{d} + q_j\) with \(\sum_j q_j = 0\) and \(\sum_j e_j = \|\Delta r\|_2\), establishing \(e_j\) as an exact partition of the module update norm and its first-order sensitivity under neuron rescaling. This projection filters out orthogonal components that cancel under superposition, yielding signed, module-local evidence at zero backpropagation cost.

2. Bayesian Conjugate Evidence Aggregation: Closed-Form Modeling of Evidence Dispersion

To distinguish stable domain-wide contributors from spurious outlier spikes, RACE treats per-observation evidence \(e_j^{(t)}\) as noisy draws from a latent Gaussian \(\mathcal{N}(\mu_j, \sigma_j^2)\) governed by a conjugate Normal-Inverse-Gamma (NIG) prior \(\mathrm{NIG}(\mu_0, \lambda_0, \alpha_0, \beta_0)\). Given \(n\) observations across \(T_c\), the model tracks the empirical mean \(\bar{e}_j\) and centered sum of squares \(\mathrm{SS}_j = \sum_t (e_j^{(t)} - \bar{e}_j)^2\), updating hyperparameters via closed-form arithmetic: $\(\lambda_n = \lambda_0 + n, \quad \alpha_n = \alpha_0 + \frac{n}{2}, \quad \mu_{n,j} = \frac{\lambda_0 \mu_0 + n \bar{e}_j}{\lambda_n}, \quad \beta_{n,j} = \beta_0 + \frac{\mathrm{SS}_j}{2} + \frac{\lambda_0 n (\bar{e}_j - \mu_0)^2}{2 \lambda_n}\)$ Integrating out \(\sigma_j^2\) yields the marginal posterior over mean alignment as a Student's t-distribution with \(2\alpha_n\) degrees of freedom and scale parameter \(\sigma_{\mu, j} = \sqrt{\beta_{n,j} / (\alpha_n \lambda_n)}\). The scale parameter directly reflects epistemic and observation uncertainty, widening intervals whenever evidence is scarce, highly dispersed, or fluctuating in sign.

3. Conservative Alignment Magnitude: Quantile-Calibrated Lower Credible Bound Ranking

RACE scores neurons using the Conservative Alignment Magnitude (CAM), defined as the one-sided \((1 - \gamma)\) lower credible bound on the posterior mean of positive module alignment: $\(\rho_j(\gamma) = \mu_{n,j} - t_{1-\gamma, 2\alpha_n} \cdot \sigma_{\mu,j}, \qquad \mathrm{CAM}_j(\gamma) = \mathbb{I}[\mu_{n,j} > 0] \max(0, \rho_j(\gamma))\)$ where \(t_{1-\gamma, 2\alpha_n}\) is the \((1 - \gamma)\)-quantile of the Student's t-distribution. Neurons that achieve high average activation through a handful of massive outliers suffer an inflated \(\mathrm{SS}_j\), which widens \(\sigma_{\mu, j}\) and drags the conservative bound down to zero. Conversely, neurons with consistently positive, low-variance alignment across the observation set maintain tight confidence radii, earning top rankings while noisy, sign-fluctuating candidates are rigorously penalized.

4. Reference-Set Filtering: Disentangling Domain Specialization from Shared Linguistic Infrastructure

Because foundational language models represent features in superposition, unconstrained domain rankings can be dominated by polysemantic or general-purpose neurons supporting fundamental language generation. RSF resolves this by scoring neurons over a general reference corpus \(D_{\mathrm{ref}}\) (e.g., WikiText-2 or FineWeb) under the identical scoring rule, identifying an exclusion set \(B_{\ell}^{\mathrm{ref}} = \mathrm{Top}_{K_{\mathrm{ref}}}^{\mathrm{sref}}(U_\ell)\) for each layer and module. When selecting candidate neurons from the target ranking \(\Pi_\ell^{\mathrm{tar}}\), RSF traverses the list, skips any neuron present in \(B_\ell^{\mathrm{ref}}\), and extracts the first \(K_{\mathrm{sel}}\) domain-exclusive neurons. This ensures that targeted suppression selectively degrades the target domain while preserving general linguistic competence.

Key Experimental Results

Main Results

RACE was evaluated across Qwen3-4B-it, OLMo-3.1-32B-it, and Llama-3.1-8B-it spanning Code Generation (MBPP+, HumanEval+), Mathematical Reasoning (MATH-500, AMC), and general retention benchmarks (MMLU-Redux, GPQA, ARC). Evaluation uses the Intervention Specificity Index (ISI): $\(\mathrm{ISI} = \max\left(0, \log \frac{\Delta_T}{\Delta_G + \delta}\right)\)$ measuring target degradation \(\Delta_T\) against general degradation \(\Delta_G\) with smoothing \(\delta = 0.01\).

The table below reports post-suppression benchmark accuracy (%) and ISI on Qwen3-4B-it in the code domain under \(R_{\mathrm{MBPP+} \setminus \mathrm{WikiText-2}}\) with top-1% neuron suppression per layer:

Module Method MBPP+ (♠) ↓ HumanEval+ (♡) ↓ MATH-500 AMC MMLU-Redux GPQA ISI ↑
ATTN GxAct 8.73 7.93 17.20 2.24 47.67 21.72 0.26
ATTN AttnLRP 81.75 86.59 90.03 75.37 80.67 46.97 0.05
ATTN Act. Mean 0.00 0.00 2.60 0.75 6.07 5.05 0.00
ATTN Emp. Mean 66.40 78.66 88.34 63.43 78.54 40.40 0.00
ATTN Emp. SNR 68.52 84.15 93.44 76.86 79.49 40.40 0.09
ATTN Neg. CAM 81.75 82.93 92.61 83.58 81.53 45.45 0.00
ATTN RACE (CAM) 65.87 80.49 90.30 68.66 79.07 44.41 0.29
MLP GxAct 23.81 19.51 79.23 46.27 78.49 40.40 1.29
MLP AttnLRP 0.00 3.05 61.37 35.07 76.39 31.82 1.07
MLP Act. Mean 11.38 12.21 88.73 72.09 82.61 40.98 2.22
MLP Emp. Mean 5.82 2.44 86.44 63.43 82.30 55.05 2.26
MLP Emp. SNR 83.33 84.76 93.83 69.40 78.98 38.89 0.00
MLP Neg. CAM 68.25 84.76 93.23 81.34 81.72 48.99 1.04
MLP RACE (CAM) 3.17 1.22 85.41 75.11 82.35 52.53 2.65
Base Qwen3-4B-it 82.28 83.54 94.40 87.31 81.37 45.45 —

Computational Profiling & Internal Ablations

Profiling was performed on Qwen3-4B-it using PyTorch profiler on a 64-input/16-output window (baseline forward pass = 130.1 GFLOPs), alongside intervention evaluation in the math domain (\(R_{\mathrm{MATH-500} \setminus \mathrm{WikiText-2}}\)):

Method Category Evaluated Protocol / Ablation Extra FLOPs Relative Forward Overhead MATH-500 (♠) AMC (♡) GPQA (Retained) ISI ↑
Forward Base Unmodified Model Inference — 1.00× 94.40 87.31 45.45 —
Gradient Base GxAct (Gradient × Activation) 18.77 TFLOPs 144.29× 30.80 9.70 41.41 1.46
Gradient Base AttnLRP (Relevance Propagation) 18.82 TFLOPs 144.62× 46.00 31.34 43.43 0.78
Ablation Act. Mean (Unsigned Magnitude) — — 58.40 47.76 40.91 0.75
Ablation Emp. Mean (No Prior, No Penalty) — — 27.20 12.69 39.39 1.36
Ablation Emp. SNR (Frequentist Ratio) — — 95.00 86.57 39.90 0.00
Ablation Neg. CAM (Opposite Direction) — — 95.60 85.08 44.44 0.00
Proposed RACE (RDA + CAM, MLP) 81.55 GFLOPs 0.63× 29.00 11.94 40.40 1.76

Key Findings

  • Drastic Computational Efficiency Advantage: RDA introduces only 81.55 GFLOPs of extra computation (0.63× forward equivalent), representing a \(>220\times\) reduction in computational overhead compared to gradient baselines GxAct (144.29×) and AttnLRP (144.62×), enabling domain-wide auditing on 32B models in minutes.
  • Asymmetric Modularity (MLP vs. Attention): MLP neurons demonstrate high domain exclusivity (all-domain Jaccard overlap across Math, Code, and Language is only 0.085), where suppressing 1% of target MLP neurons drops code accuracy from 82.28% to 3.17% with an ISI of 2.65. Conversely, attention output channels are heavily shared across domains (Jaccard of 0.264, a 3.1× increase), functioning as general-purpose routing hubs that lose domain signal after reference filtering.
  • Failure Mode of Frequentist SNR: Emp. SNR completely collapses on MLP interventions (ISI = 0.00) because sparse activations yield near-zero empirical standard deviation in the denominator, causing tiny random noise to be ranked highest. In contrast, the NIG prior regularizes posterior uncertainty (\(\sigma_{\mu, j} > 0\)), ensuring robust conservative screening even in low-sample regimes.

Highlights & Insights

  • Module-Local Decomposition Bypasses Depth Confounding: By projecting directly onto the immediate module update rather than global logits, RACE eliminates depth bias and accounts for later layers overwriting earlier features.
  • Closed-Form Conjugate Streaming with Negligible Memory: Maintaining running sufficient statistics requires only 0.95 MB of additional state for all 497,664 neurons in Qwen3-4B-it, adding no measurable peak GPU memory overhead over standard inference.
  • Precise Stylistic Steering Without Catastrophic Forgetting: On PyComp-1K, suppressing just 10 neurons per layer reduces Python comprehension generation by 70.9% on MBPP+ while maintaining 97.73% functional pass@1 accuracy on solutions using comprehensions.

Limitations & Future Work

  • Linear Residual Projections: RDA projects rank-1 updates linearly onto module axes, which cannot capture complex non-linear feature interactions or distributed polysemantic features requiring non-linear decoding.
  • Diminished Selectivity in Attention Modules: Attention output channels exhibit high cross-domain reuse, rendering RSF less effective at isolating pure domain-specific attention neurons without degrading broad linguistic routing.
  • Sensitivity to Observation Prompt Diversity: Although CAM is robust to finite sample sizes, highly biased prompt distributions can skew the active token set \(T_c\), necessitating structured prompt diversification in practice.
  • vs Gradient Attribution (GxAct, AttnLRP): Gradient methods require full backward passes with target token backpropagation, suffering from gradient saturation and high compute; RACE is forward-only, \(>220\times\) faster, and yields higher intervention specificity.
  • vs Knowledge Neurons: Knowledge neuron localization relies on heuristic activation thresholds over single prompts; RACE operates on rigorous population-level statistical inference with formal credible lower bounds.
  • vs Sparse Autoencoders (SAEs): SAEs demand training separate massive autoencoders at each layer; RACE works natively on the model's raw neuron basis, providing instant zero-training interpretability.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates neuron functional consistency as Bayesian posterior inference over forward residual stream projections.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Spans 4B to 32B models, code/math domains, distributional perplexity shifts, fine-grained syntax steering, and profiler-verified FLOP benchmarks.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulations, clean geometric derivations, and well-structured comparative tables.
  • Value: ⭐⭐⭐⭐⭐ Provides an ultra-scalable, mathematically sound tool for LLM capability auditing, targeted pruning, and behavioral alignment.